mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-19 19:01:42 +00:00
e84753f9b9
* docs: bring doc titles and descriptions into SEO length limits Rewrite the frontmatter title and description of every published doc so the rendered <title> lands in 50-60 characters (including the site suffix) and the meta description lands in 150-160 characters. * docs: move SEO titles to a seoTitle field and restore display titles The frontmatter title renders as the page H1 and the sidebar label, so tuning it for the title tag shortened nothing and lengthened the visible navigation. Put the search-facing string in seoTitle and give title back its original value.
207 lines
9.9 KiB
Markdown
207 lines
9.9 KiB
Markdown
---
|
|
title: "EKS HTTPS Ingress (LBC + ACM)"
|
|
seoTitle: "EKS HTTPS Ingress with LBC and ACM — Spinifex Docs"
|
|
description: "Serve a demo app over HTTPS on EKS using the AWS Load Balancer Controller addon and an ACM certificate, with an internet-facing ALB built from an Ingress."
|
|
category: "Terraform Workbooks"
|
|
tags:
|
|
- terraform
|
|
- eks
|
|
- kubernetes
|
|
- ingress
|
|
- alb
|
|
- acm
|
|
- https
|
|
- workbook
|
|
resources:
|
|
- title: "AWS Load Balancer Controller"
|
|
url: "https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/"
|
|
- title: "Terraform AWS Provider"
|
|
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
|
|
- title: "Terraform Kubernetes Provider"
|
|
url: "https://registry.terraform.io/providers/hashicorp/kubernetes/latest"
|
|
- title: "Spinifex Repository"
|
|
url: "https://github.com/mulgadc/spinifex"
|
|
---
|
|
|
|
# Terraform: EKS HTTPS Ingress (LBC + ACM)
|
|
|
|
> Builds on the quickstart. Instead of a raw NodePort, it serves the Spinifex-themed demo app over **HTTPS** through an ALB that the **AWS Load Balancer Controller** provisions from a Kubernetes **Ingress**, terminating TLS with a certificate held in **ACM** — the same pattern you'd use on AWS EKS.
|
|
|
|
## Table of Contents
|
|
|
|
- [Overview](#overview)
|
|
- [Instructions](#instructions)
|
|
- [Troubleshooting](#troubleshooting)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This is the second rung of the EKS ladder. It takes the cluster + demo app from [EKS Quickstart](../eks-quickstart) and publishes the app the way you would in production: a Kubernetes `Ingress` reconciled by the **AWS Load Balancer Controller (LBC)** into an internet-facing **Application Load Balancer** that terminates TLS with an **ACM** certificate.
|
|
|
|
To make room for an internet-facing ALB with private workers, the network grows from the quickstart's two public subnets to a public/private split: the ALB and a **NAT gateway** sit in the public subnets, the workers sit in the private subnets and reach ECR through the NAT. The cluster is tagged `spinifex.io/managed-ingress = "false"`, which disables K3s' built-in traefik/servicelb so the LBC owns ingress.
|
|
|
|
The LBC is installed as a managed **addon** (`aws_eks_addon`). Spinifex wires the controller's AWS credentials and the cluster's ELB-eligible subnets at the node level and injects those subnets into the `alb` `IngressClassParams`, so the Ingress needs **no IRSA role and no subnet tags** — just `ingressClassName: alb` and a handful of annotations.
|
|
|
|
**What you'll learn:**
|
|
|
|
- Installing the `aws-load-balancer-controller` addon and letting it own ingress
|
|
- Disabling the built-in K3s ingress with the `spinifex.io/managed-ingress` tag
|
|
- Importing a self-signed certificate into ACM and attaching it to an ALB listener
|
|
- Driving an ALB entirely from a Kubernetes `Ingress` (`ingressClassName: alb`)
|
|
- A public/private VPC with a NAT gateway for private workers
|
|
|
|
**What gets created**
|
|
|
|
| Resource | Name | Purpose |
|
|
|---|---|---|
|
|
| VPC | `eks-https-vpc` | Isolated network (10.31.0.0/16) |
|
|
| Subnets | `eks-https-public-a/-b`, `eks-https-private-a/-b` | Public (ALB + NAT) and private (workers) |
|
|
| Internet + NAT Gateway | `eks-https-igw`, `eks-https-nat` | Public egress; private-worker egress to ECR |
|
|
| IAM Roles | `eks-https-cluster-role`, `eks-https-node-role` | Control-plane and worker roles |
|
|
| ECR Repository | `spinifex-demo` | Holds the demo image the workers pull |
|
|
| EKS Cluster | `eks-https` | Public + private endpoints; `managed-ingress=false` |
|
|
| Node Group | `workers` | `node_desired_size` `t3.large` worker(s) — 1 or 3 |
|
|
| Addon | `aws-load-balancer-controller` | Provisions the ALB from the Ingress |
|
|
| ACM Certificate | `eks-https-ingress` | Self-signed, imported; attached to the HTTPS listener |
|
|
| SG Ingress Rule | `eks-https-alb-nodeport` | Admits the VPC CIDR to the workers' NodePort |
|
|
| K8s Deployment + Service | `spinifex-demo` | Themed demo image, 2 replicas, NodePort |
|
|
| K8s Ingress | `spinifex-demo` | `ingressClassName: alb`, HTTPS via the ACM cert |
|
|
|
|
**Spinifex specifics**
|
|
|
|
- **The LBC needs no IRSA or subnet tagging here.** Credentials and ELB-eligible subnets are injected by Spinifex at the node level and into the `alb` `IngressClassParams`. The Ingress just sets `ingressClassName: alb` plus the `alb.ingress.kubernetes.io/*` annotations.
|
|
- **`spinifex.io/managed-ingress = "false"`** disables the K3s built-in traefik/servicelb so the LBC owns ingress, matching AWS parity. Omit the tag (or set it `true`) and the built-in ingress stays on.
|
|
- The ALB targets the workers' **NodePort** (`target-type: instance`). The one SG rule admits the VPC CIDR on that port so the in-VPC ALB can reach the workers; the worker SG is otherwise auto-managed.
|
|
- Spinifex ACM supports **ImportCertificate only** — the cert is generated by the `tls` provider and imported, not requested.
|
|
- **Leave `addon_version` unset** (see the [quickstart/addons notes](../eks-quickstart)) — the AWS provider and the Spinifex catalog disagree on the version string format, so let the server choose.
|
|
|
|
**Prerequisites:**
|
|
|
|
- Spinifex installed and running, with the `eks-node` image (carrying the LBC bundle) available
|
|
- OpenTofu or Terraform, plus `kubectl`, the AWS CLI, and Docker
|
|
- The demo image built and pushed to ECR (see [`../demo-app/README.md`](../demo-app/README.md))
|
|
|
|
## Instructions
|
|
|
|
### Step 1. Get the Template
|
|
|
|
```bash
|
|
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
|
|
cd spinifex-tf
|
|
git sparse-checkout set docs/terraform-workbooks
|
|
cd docs/terraform-workbooks/eks-https-ingress
|
|
```
|
|
|
|
Or create a `main.tf` file and paste the full configuration below.
|
|
|
|
<!-- INCLUDE: main.tf lang:hcl -->
|
|
|
|
### Step 2. Deploy the Cluster
|
|
|
|
The cluster and the demo app are **two root modules with separate state**. Apply the cluster first (set `node_desired_size=3` for an HA-shaped cluster):
|
|
|
|
```bash
|
|
export AWS_PROFILE=spinifex
|
|
tofu init
|
|
tofu apply
|
|
```
|
|
|
|
This brings up the cluster, the workers, the NAT gateway, the LBC addon, the ACM cert, and the `spinifex-demo` ECR repository. Give it a few minutes to reach `ACTIVE` and for the LBC addon to report healthy.
|
|
|
|
### Step 3. Build and Push the Demo Image
|
|
|
|
```bash
|
|
cd ../demo-app
|
|
REGISTRY=$(cd ../eks-https-ingress && tofu output -raw ecr_repository_url)
|
|
REGISTRY_HOST=${REGISTRY%%/*}
|
|
aws ecr get-login-password | docker login --username AWS --password-stdin "$REGISTRY_HOST"
|
|
docker build -t "${REGISTRY}:latest" .
|
|
docker push "${REGISTRY}:latest"
|
|
cd ../eks-https-ingress
|
|
```
|
|
|
|
### Step 4. Deploy the App and Ingress
|
|
|
|
```bash
|
|
cd workloads
|
|
tofu init
|
|
tofu apply
|
|
```
|
|
|
|
> **Why two modules?** The Kubernetes provider in `workloads/` reads the cluster endpoint from a live `data "aws_eks_cluster"` source, so it's only configured while the cluster exists. Always destroy `workloads/` before the cluster, or the provider falls back to `http://localhost:80` and reports `connection refused`. Keep `AWS_PROFILE=spinifex` exported for both applies.
|
|
|
|
<!-- INCLUDE: workloads/main.tf lang:hcl -->
|
|
|
|
### Step 5. Open the Demo over HTTPS
|
|
|
|
The LBC takes a minute or two to provision the ALB after the Ingress is created. The Ingress publishes the ALB's **DNS name**, which has no resolver yet (northstar will add one) — resolve it to the ALB's public IP with `describe-load-balancers` so you can reach it without DNS or `/etc/hosts`:
|
|
|
|
```bash
|
|
# Point kubectl at the cluster first (writes ~/.kube/config) — without this every
|
|
# kubectl call fails to connect:
|
|
aws eks update-kubeconfig --name eks-https --region ap-southeast-2
|
|
|
|
kubectl get ingress spinifex-demo -o wide
|
|
DNSNAME=$(kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
|
|
IP=$(aws elbv2 describe-load-balancers \
|
|
--query "LoadBalancers[?DNSName=='${DNSNAME}'].AvailabilityZones[].LoadBalancerAddresses[].IpAddress | [0]" \
|
|
--output text)
|
|
curl -k "https://$IP" # self-signed cert: -k skips verification
|
|
```
|
|
|
|
Open `https://$IP` in a browser (accept the self-signed certificate). The Spinifex-themed page reports the pod, node, cluster, and region that answered — refresh to watch requests land on different replicas, now over HTTPS through the ALB.
|
|
|
|
### Cleanup
|
|
|
|
Destroy in reverse — the app/Ingress first (so the LBC tears the ALB down while the cluster is up), then the cluster:
|
|
|
|
```bash
|
|
cd workloads
|
|
tofu destroy
|
|
cd ..
|
|
tofu destroy
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### The Ingress Has No Address
|
|
|
|
The LBC populates `status.loadBalancer` once it has provisioned the ALB. If it stays empty:
|
|
|
|
```bash
|
|
kubectl describe ingress spinifex-demo # events show LBC reconcile errors
|
|
kubectl -n kube-system logs deploy/aws-load-balancer-controller --tail=50
|
|
aws eks describe-addon --cluster-name eks-https --addon-name aws-load-balancer-controller --query 'addon.status'
|
|
```
|
|
|
|
Confirm the addon is `ACTIVE` and the cluster carries `spinifex.io/managed-ingress = "false"` (otherwise the built-in ingress competes).
|
|
|
|
### HTTPS Loads but Returns 502 / 504
|
|
|
|
The ALB reached the workers but the backend didn't answer. Check the demo pods and the NodePort SG rule:
|
|
|
|
```bash
|
|
kubectl get pods -o wide
|
|
kubectl get svc spinifex-demo
|
|
aws ec2 describe-security-groups --filters "Name=group-name,Values=eks-cluster-eks-https-nodegroup-sg" \
|
|
--query 'SecurityGroups[0].IpPermissions'
|
|
```
|
|
|
|
The `eks-https-alb-nodeport` rule must admit the VPC CIDR on the NodePort, and the target group health check (`/healthz`) must pass.
|
|
|
|
### Addon Fails With "no baked bundle"
|
|
|
|
Spinifex only installs addons whose manifests are baked into the `eks-node` AMI. If `describe-addon` reports `CREATE_FAILED` with `no baked bundle`, the running worker image predates the LBC bundle — rebuild and republish the `eks-node` image, then recreate the addon.
|
|
|
|
### Pods Stuck in ImagePullBackOff
|
|
|
|
The private workers pull from ECR through the NAT gateway. Confirm the image was pushed (`tofu output ecr_repository_url`), the NAT route is in place, and the node role carries `AmazonEC2ContainerRegistryReadOnly`.
|
|
|
|
### Provider Connection Refused
|
|
|
|
```bash
|
|
sudo systemctl status spinifex.target
|
|
curl -k https://localhost:9999/
|
|
```
|