Files
spinifex/docs/terraform-workbooks/eks-https-ingress/README.md
T
Ben Duncan 8b3c706ea3 docs: rework the multi-node install guide, fix cross-doc links
Multi-node install
- Lead with the fact that the guide builds a three-node cluster, and say
  how to go beyond three. The callout sits inside ## Overview because the
  docs site discards everything above the first section heading.
- Replace the OVN-only sizing prose with a table covering all three
  distributed layers, and move the OVN control-plane detail into the VPC
  networking guide, linked from here.
- Add minimum and recommended hardware.
- Move the object storage layout to the end as background; it was the
  second thing a reader hit.
- Collapse the ISO conversion notes from a section into one skippable
  box, now that --force is unconditional in Step 4 and the formation
  port needs no manual intervention.
- Expand verification with spx get nodes and spx top nodes, including
  what to look for in each.
- Drop the install-node.sh shortcut (internal tooling) and the
  "adding a node later" section (no good story yet).

Corrections found while writing
- Servers 4+ were described as running no database. They run predastore
  and viperblock like every other node; only the OVN database cluster is
  capped at three. Fixed in four places.
- Step 4 never needed the firewall down: spx admin init opens the
  formation port itself for the formation window. Only Step 3's
  peer-scoped OVN ports are affected.
- `nft list table inet spinifex_filter | grep spinifex_peers` could never
  match — the peer list is an nft define, expanded at load time. Anyone
  following it would have seen empty output and assumed the worst.
- Disambiguate "database": with predastore, NATS and OVN all in play it
  needed qualifying throughout.
- Align the example output region with the AWS_REGION the guide exports.

Cross-doc links
- Rewrite relative and category-prefixed links to the canonical
  /docs/<slug> form. The site publishes docs flat, so ../foo/README.md
  and /docs/category/foo both 404.
- Point the six links to unpublished docs at the source repo instead.
- Fix four fragments written against GitHub's anchor algorithm.
2026-08-21 14:14:28 -05:00

9.9 KiB

title, seoTitle, description, category, tags, resources
title seoTitle description category tags resources
EKS HTTPS Ingress (LBC + ACM) EKS HTTPS Ingress with LBC and ACM — Spinifex Docs Serve a demo app over HTTPS on EKS using the AWS Load Balancer Controller addon and an ACM certificate, with an internet-facing ALB built from an Ingress. Terraform Workbooks
terraform
eks
kubernetes
ingress
alb
acm
https
workbook
title url
AWS Load Balancer Controller https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/
title url
Terraform AWS Provider https://registry.terraform.io/providers/hashicorp/aws/latest
title url
Terraform Kubernetes Provider https://registry.terraform.io/providers/hashicorp/kubernetes/latest
title url
Spinifex Repository https://github.com/mulgadc/spinifex

Terraform: EKS HTTPS Ingress (LBC + ACM)

Builds on the quickstart. Instead of a raw NodePort, it serves the Spinifex-themed demo app over HTTPS through an ALB that the AWS Load Balancer Controller provisions from a Kubernetes Ingress, terminating TLS with a certificate held in ACM — the same pattern you'd use on AWS EKS.

Table of Contents


Overview

This is the second rung of the EKS ladder. It takes the cluster + demo app from EKS Quickstart and publishes the app the way you would in production: a Kubernetes Ingress reconciled by the AWS Load Balancer Controller (LBC) into an internet-facing Application Load Balancer that terminates TLS with an ACM certificate.

To make room for an internet-facing ALB with private workers, the network grows from the quickstart's two public subnets to a public/private split: the ALB and a NAT gateway sit in the public subnets, the workers sit in the private subnets and reach ECR through the NAT. The cluster is tagged spinifex.io/managed-ingress = "false", which disables K3s' built-in traefik/servicelb so the LBC owns ingress.

The LBC is installed as a managed addon (aws_eks_addon). Spinifex wires the controller's AWS credentials and the cluster's ELB-eligible subnets at the node level and injects those subnets into the alb IngressClassParams, so the Ingress needs no IRSA role and no subnet tags — just ingressClassName: alb and a handful of annotations.

What you'll learn:

  • Installing the aws-load-balancer-controller addon and letting it own ingress
  • Disabling the built-in K3s ingress with the spinifex.io/managed-ingress tag
  • Importing a self-signed certificate into ACM and attaching it to an ALB listener
  • Driving an ALB entirely from a Kubernetes Ingress (ingressClassName: alb)
  • A public/private VPC with a NAT gateway for private workers

What gets created

Resource Name Purpose
VPC eks-https-vpc Isolated network (10.31.0.0/16)
Subnets eks-https-public-a/-b, eks-https-private-a/-b Public (ALB + NAT) and private (workers)
Internet + NAT Gateway eks-https-igw, eks-https-nat Public egress; private-worker egress to ECR
IAM Roles eks-https-cluster-role, eks-https-node-role Control-plane and worker roles
ECR Repository spinifex-demo Holds the demo image the workers pull
EKS Cluster eks-https Public + private endpoints; managed-ingress=false
Node Group workers node_desired_size t3.large worker(s) — 1 or 3
Addon aws-load-balancer-controller Provisions the ALB from the Ingress
ACM Certificate eks-https-ingress Self-signed, imported; attached to the HTTPS listener
SG Ingress Rule eks-https-alb-nodeport Admits the VPC CIDR to the workers' NodePort
K8s Deployment + Service spinifex-demo Themed demo image, 2 replicas, NodePort
K8s Ingress spinifex-demo ingressClassName: alb, HTTPS via the ACM cert

Spinifex specifics

  • The LBC needs no IRSA or subnet tagging here. Credentials and ELB-eligible subnets are injected by Spinifex at the node level and into the alb IngressClassParams. The Ingress just sets ingressClassName: alb plus the alb.ingress.kubernetes.io/* annotations.
  • spinifex.io/managed-ingress = "false" disables the K3s built-in traefik/servicelb so the LBC owns ingress, matching AWS parity. Omit the tag (or set it true) and the built-in ingress stays on.
  • The ALB targets the workers' NodePort (target-type: instance). The one SG rule admits the VPC CIDR on that port so the in-VPC ALB can reach the workers; the worker SG is otherwise auto-managed.
  • Spinifex ACM supports ImportCertificate only — the cert is generated by the tls provider and imported, not requested.
  • Leave addon_version unset (see the quickstart/addons notes) — the AWS provider and the Spinifex catalog disagree on the version string format, so let the server choose.

Prerequisites:

  • Spinifex installed and running, with the eks-node image (carrying the LBC bundle) available
  • OpenTofu or Terraform, plus kubectl, the AWS CLI, and Docker
  • The demo image built and pushed to ECR (see docs/terraform-workbooks/demo-app)

Instructions

Step 1. Get the Template

git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform-workbooks
cd docs/terraform-workbooks/eks-https-ingress

Or create a main.tf file and paste the full configuration below.

Step 2. Deploy the Cluster

The cluster and the demo app are two root modules with separate state. Apply the cluster first (set node_desired_size=3 for an HA-shaped cluster):

export AWS_PROFILE=spinifex
tofu init
tofu apply

This brings up the cluster, the workers, the NAT gateway, the LBC addon, the ACM cert, and the spinifex-demo ECR repository. Give it a few minutes to reach ACTIVE and for the LBC addon to report healthy.

Step 3. Build and Push the Demo Image

cd ../demo-app
REGISTRY=$(cd ../eks-https-ingress && tofu output -raw ecr_repository_url)
REGISTRY_HOST=${REGISTRY%%/*}
aws ecr get-login-password | docker login --username AWS --password-stdin "$REGISTRY_HOST"
docker build -t "${REGISTRY}:latest" .
docker push "${REGISTRY}:latest"
cd ../eks-https-ingress

Step 4. Deploy the App and Ingress

cd workloads
tofu init
tofu apply

Why two modules? The Kubernetes provider in workloads/ reads the cluster endpoint from a live data "aws_eks_cluster" source, so it's only configured while the cluster exists. Always destroy workloads/ before the cluster, or the provider falls back to http://localhost:80 and reports connection refused. Keep AWS_PROFILE=spinifex exported for both applies.

Step 5. Open the Demo over HTTPS

The LBC takes a minute or two to provision the ALB after the Ingress is created. The Ingress publishes the ALB's DNS name, which has no resolver yet (northstar will add one) — resolve it to the ALB's public IP with describe-load-balancers so you can reach it without DNS or /etc/hosts:

# Point kubectl at the cluster first (writes ~/.kube/config) — without this every
# kubectl call fails to connect:
aws eks update-kubeconfig --name eks-https --region ap-southeast-2

kubectl get ingress spinifex-demo -o wide
DNSNAME=$(kubectl get ingress spinifex-demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
IP=$(aws elbv2 describe-load-balancers \
  --query "LoadBalancers[?DNSName=='${DNSNAME}'].AvailabilityZones[].LoadBalancerAddresses[].IpAddress | [0]" \
  --output text)
curl -k "https://$IP"          # self-signed cert: -k skips verification

Open https://$IP in a browser (accept the self-signed certificate). The Spinifex-themed page reports the pod, node, cluster, and region that answered — refresh to watch requests land on different replicas, now over HTTPS through the ALB.

Cleanup

Destroy in reverse — the app/Ingress first (so the LBC tears the ALB down while the cluster is up), then the cluster:

cd workloads
tofu destroy
cd ..
tofu destroy

Troubleshooting

The Ingress Has No Address

The LBC populates status.loadBalancer once it has provisioned the ALB. If it stays empty:

kubectl describe ingress spinifex-demo            # events show LBC reconcile errors
kubectl -n kube-system logs deploy/aws-load-balancer-controller --tail=50
aws eks describe-addon --cluster-name eks-https --addon-name aws-load-balancer-controller --query 'addon.status'

Confirm the addon is ACTIVE and the cluster carries spinifex.io/managed-ingress = "false" (otherwise the built-in ingress competes).

HTTPS Loads but Returns 502 / 504

The ALB reached the workers but the backend didn't answer. Check the demo pods and the NodePort SG rule:

kubectl get pods -o wide
kubectl get svc spinifex-demo
aws ec2 describe-security-groups --filters "Name=group-name,Values=eks-cluster-eks-https-nodegroup-sg" \
  --query 'SecurityGroups[0].IpPermissions'

The eks-https-alb-nodeport rule must admit the VPC CIDR on the NodePort, and the target group health check (/healthz) must pass.

Addon Fails With "no baked bundle"

Spinifex only installs addons whose manifests are baked into the eks-node AMI. If describe-addon reports CREATE_FAILED with no baked bundle, the running worker image predates the LBC bundle — rebuild and republish the eks-node image, then recreate the addon.

Pods Stuck in ImagePullBackOff

The private workers pull from ECR through the NAT gateway. Confirm the image was pushed (tofu output ecr_repository_url), the NAT route is in place, and the node role carries AmazonEC2ContainerRegistryReadOnly.

Provider Connection Refused

sudo systemctl status spinifex.target
curl -k https://localhost:9999/