mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-19 19:01:42 +00:00
206 lines
9.9 KiB
Markdown
206 lines
9.9 KiB
Markdown
---
|
|
title: "EKS HTTPS Ingress (LBC + ACM)"
|
|
description: "Serve a Spinifex-themed demo app over HTTPS on EKS using the AWS Load Balancer Controller addon and an ACM certificate — an internet-facing ALB provisioned from a Kubernetes Ingress, using Terraform on Spinifex."
|
|
category: "Terraform Workbooks"
|
|
tags:
|
|
- terraform
|
|
- eks
|
|
- kubernetes
|
|
- ingress
|
|
- alb
|
|
- acm
|
|
- https
|
|
- workbook
|
|
resources:
|
|
- title: "AWS Load Balancer Controller"
|
|
url: "https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/"
|
|
- title: "Terraform AWS Provider"
|
|
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
|
|
- title: "Terraform Kubernetes Provider"
|
|
url: "https://registry.terraform.io/providers/hashicorp/kubernetes/latest"
|
|
- title: "Spinifex Repository"
|
|
url: "https://github.com/mulgadc/spinifex"
|
|
---
|
|
|
|
# Terraform: EKS HTTPS Ingress (LBC + ACM)
|
|
|
|
> Builds on the quickstart. Instead of a raw NodePort, it serves the Spinifex-themed demo app over **HTTPS** through an ALB that the **AWS Load Balancer Controller** provisions from a Kubernetes **Ingress**, terminating TLS with a certificate held in **ACM** — the same pattern you'd use on AWS EKS.
|
|
|
|
## Table of Contents
|
|
|
|
- [Overview](#overview)
|
|
- [Instructions](#instructions)
|
|
- [Troubleshooting](#troubleshooting)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This is the second rung of the EKS ladder. It takes the cluster + demo app from [EKS Quickstart](../eks-quickstart) and publishes the app the way you would in production: a Kubernetes `Ingress` reconciled by the **AWS Load Balancer Controller (LBC)** into an internet-facing **Application Load Balancer** that terminates TLS with an **ACM** certificate.
|
|
|
|
To make room for an internet-facing ALB with private workers, the network grows from the quickstart's two public subnets to a public/private split: the ALB and a **NAT gateway** sit in the public subnets, the workers sit in the private subnets and reach ECR through the NAT. The cluster is tagged `spinifex.io/managed-ingress = "false"`, which disables K3s' built-in traefik/servicelb so the LBC owns ingress.
|
|
|
|
The LBC is installed as a managed **addon** (`aws_eks_addon`). Spinifex wires the controller's AWS credentials and the cluster's ELB-eligible subnets at the node level and injects those subnets into the `alb` `IngressClassParams`, so the Ingress needs **no IRSA role and no subnet tags** — just `ingressClassName: alb` and a handful of annotations.
|
|
|
|
**What you'll learn:**
|
|
|
|
- Installing the `aws-load-balancer-controller` addon and letting it own ingress
|
|
- Disabling the built-in K3s ingress with the `spinifex.io/managed-ingress` tag
|
|
- Importing a self-signed certificate into ACM and attaching it to an ALB listener
|
|
- Driving an ALB entirely from a Kubernetes `Ingress` (`ingressClassName: alb`)
|
|
- A public/private VPC with a NAT gateway for private workers
|
|
|
|
**What gets created**
|
|
|
|
| Resource | Name | Purpose |
|
|
|---|---|---|
|
|
| VPC | `eks-https-vpc` | Isolated network (10.31.0.0/16) |
|
|
| Subnets | `eks-https-public-a/-b`, `eks-https-private-a/-b` | Public (ALB + NAT) and private (workers) |
|
|
| Internet + NAT Gateway | `eks-https-igw`, `eks-https-nat` | Public egress; private-worker egress to ECR |
|
|
| IAM Roles | `eks-https-cluster-role`, `eks-https-node-role` | Control-plane and worker roles |
|
|
| ECR Repository | `spinifex-demo` | Holds the demo image the workers pull |
|
|
| EKS Cluster | `eks-https` | Public + private endpoints; `managed-ingress=false` |
|
|
| Node Group | `workers` | `node_desired_size` `t3.large` worker(s) — 1 or 3 |
|
|
| Addon | `aws-load-balancer-controller` | Provisions the ALB from the Ingress |
|
|
| ACM Certificate | `eks-https-ingress` | Self-signed, imported; attached to the HTTPS listener |
|
|
| SG Ingress Rule | `eks-https-alb-nodeport` | Admits the VPC CIDR to the workers' NodePort |
|
|
| K8s Deployment + Service | `spinifex-demo` | Themed demo image, 2 replicas, NodePort |
|
|
| K8s Ingress | `spinifex-demo` | `ingressClassName: alb`, HTTPS via the ACM cert |
|
|
|
|
**Spinifex specifics**
|
|
|
|
- **The LBC needs no IRSA or subnet tagging here.** Credentials and ELB-eligible subnets are injected by Spinifex at the node level and into the `alb` `IngressClassParams`. The Ingress just sets `ingressClassName: alb` plus the `alb.ingress.kubernetes.io/*` annotations.
|
|
- **`spinifex.io/managed-ingress = "false"`** disables the K3s built-in traefik/servicelb so the LBC owns ingress, matching AWS parity. Omit the tag (or set it `true`) and the built-in ingress stays on.
|
|
- The ALB targets the workers' **NodePort** (`target-type: instance`). The one SG rule admits the VPC CIDR on that port so the in-VPC ALB can reach the workers; the worker SG is otherwise auto-managed.
|
|
- Spinifex ACM supports **ImportCertificate only** — the cert is generated by the `tls` provider and imported, not requested.
|
|
- **Leave `addon_version` unset** (see the [quickstart/addons notes](../eks-quickstart)) — the AWS provider and the Spinifex catalog disagree on the version string format, so let the server choose.
|
|
|
|
**Prerequisites:**
|
|
|
|
- Spinifex installed and running, with the `eks-node` image (carrying the LBC bundle) available
|
|
- OpenTofu or Terraform, plus `kubectl`, the AWS CLI, and Docker
|
|
- The demo image built and pushed to ECR (see [`../demo-app/README.md`](../demo-app/README.md))
|
|
|
|
## Instructions
|
|
|
|
### Step 1. Get the Template
|
|
|
|
```bash
|
|
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
|
|
cd spinifex-tf
|
|
git sparse-checkout set docs/terraform-workbooks
|
|
cd docs/terraform-workbooks/eks-https-ingress
|
|
```
|
|
|
|
Or create a `main.tf` file and paste the full configuration below.
|
|
|
|
<!-- INCLUDE: main.tf lang:hcl -->
|
|
|
|
### Step 2. Deploy the Cluster
|
|
|
|
The cluster and the demo app are **two root modules with separate state**. Apply the cluster first (set `node_desired_size=3` for an HA-shaped cluster):
|
|
|
|
```bash
|
|
export AWS_PROFILE=spinifex
|
|
tofu init
|
|
tofu apply
|
|
```
|
|
|
|
This brings up the cluster, the workers, the NAT gateway, the LBC addon, the ACM cert, and the `spinifex-demo` ECR repository. Give it a few minutes to reach `ACTIVE` and for the LBC addon to report healthy.
|
|
|
|
### Step 3. Build and Push the Demo Image
|
|
|
|
```bash
|
|
cd ../demo-app
|
|
REGISTRY=$(cd ../eks-https-ingress && tofu output -raw ecr_repository_url)
|
|
REGISTRY_HOST=${REGISTRY%%/*}
|
|
aws ecr get-login-password | docker login --username AWS --password-stdin "$REGISTRY_HOST"
|
|
docker build -t "${REGISTRY}:latest" .
|
|
docker push "${REGISTRY}:latest"
|
|
cd ../eks-https-ingress
|
|
```
|
|
|
|
### Step 4. Deploy the App and Ingress
|
|
|
|
```bash
|
|
cd workloads
|
|
tofu init
|
|
tofu apply
|
|
```
|
|
|
|
> **Why two modules?** The Kubernetes provider in `workloads/` reads the cluster endpoint from a live `data "aws_eks_cluster"` source, so it's only configured while the cluster exists. Always destroy `workloads/` before the cluster, or the provider falls back to `http://localhost:80` and reports `connection refused`. Keep `AWS_PROFILE=spinifex` exported for both applies.
|
|
|
|
<!-- INCLUDE: workloads/main.tf lang:hcl -->
|
|
|
|
### Step 5. Open the Demo over HTTPS
|
|
|
|
The LBC takes a minute or two to provision the ALB after the Ingress is created. The Ingress publishes the ALB's **DNS name**, which has no resolver yet (northstar will add one) — resolve it to the ALB's public IP with `describe-load-balancers` so you can reach it without DNS or `/etc/hosts`:
|
|
|
|
```bash
|
|
# Point kubectl at the cluster first (writes ~/.kube/config) — without this every
|
|
# kubectl call fails to connect:
|
|
aws eks update-kubeconfig --name eks-https --region ap-southeast-2
|
|
|
|
kubectl get ingress spinifex-demo -o wide
|
|
DNSNAME=$(kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
|
|
IP=$(aws elbv2 describe-load-balancers \
|
|
--query "LoadBalancers[?DNSName=='${DNSNAME}'].AvailabilityZones[].LoadBalancerAddresses[].IpAddress | [0]" \
|
|
--output text)
|
|
curl -k "https://$IP" # self-signed cert: -k skips verification
|
|
```
|
|
|
|
Open `https://$IP` in a browser (accept the self-signed certificate). The Spinifex-themed page reports the pod, node, cluster, and region that answered — refresh to watch requests land on different replicas, now over HTTPS through the ALB.
|
|
|
|
### Cleanup
|
|
|
|
Destroy in reverse — the app/Ingress first (so the LBC tears the ALB down while the cluster is up), then the cluster:
|
|
|
|
```bash
|
|
cd workloads
|
|
tofu destroy
|
|
cd ..
|
|
tofu destroy
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### The Ingress Has No Address
|
|
|
|
The LBC populates `status.loadBalancer` once it has provisioned the ALB. If it stays empty:
|
|
|
|
```bash
|
|
kubectl describe ingress spinifex-demo # events show LBC reconcile errors
|
|
kubectl -n kube-system logs deploy/aws-load-balancer-controller --tail=50
|
|
aws eks describe-addon --cluster-name eks-https --addon-name aws-load-balancer-controller --query 'addon.status'
|
|
```
|
|
|
|
Confirm the addon is `ACTIVE` and the cluster carries `spinifex.io/managed-ingress = "false"` (otherwise the built-in ingress competes).
|
|
|
|
### HTTPS Loads but Returns 502 / 504
|
|
|
|
The ALB reached the workers but the backend didn't answer. Check the demo pods and the NodePort SG rule:
|
|
|
|
```bash
|
|
kubectl get pods -o wide
|
|
kubectl get svc spinifex-demo
|
|
aws ec2 describe-security-groups --filters "Name=group-name,Values=eks-cluster-eks-https-nodegroup-sg" \
|
|
--query 'SecurityGroups[0].IpPermissions'
|
|
```
|
|
|
|
The `eks-https-alb-nodeport` rule must admit the VPC CIDR on the NodePort, and the target group health check (`/healthz`) must pass.
|
|
|
|
### Addon Fails With "no baked bundle"
|
|
|
|
Spinifex only installs addons whose manifests are baked into the `eks-node` AMI. If `describe-addon` reports `CREATE_FAILED` with `no baked bundle`, the running worker image predates the LBC bundle — rebuild and republish the `eks-node` image, then recreate the addon.
|
|
|
|
### Pods Stuck in ImagePullBackOff
|
|
|
|
The private workers pull from ECR through the NAT gateway. Confirm the image was pushed (`tofu output ecr_repository_url`), the NAT route is in place, and the node role carries `AmazonEC2ContainerRegistryReadOnly`.
|
|
|
|
### Provider Connection Refused
|
|
|
|
```bash
|
|
sudo systemctl status spinifex.target
|
|
curl -k https://localhost:9999/
|
|
```
|