* feat: join nodegroup workers via cluster NLB endpoint Under the Set-B managed-CP-VPC topology the control plane runs in the system-account CP VPC while nodegroup workers launch in the customer VPC. The agent user-data set K3S_URL to the in-VPC control-plane ENI IP (:6443), which sits in a different, unpeered VPC and is unreachable, so the join path was dead. Point K3S_URL at the published cluster NLB endpoint (meta.Endpoint, https://<frontend-ip>:443, the :443->:6443 listener kubectl already uses). Guard CreateNodegroup on a non-empty endpoint. mulga-siv-298 * feat: gate nodegroup ACTIVE on workers registering Ready launchNodegroupInfra flipped the nodegroup to ACTIVE the moment RunInstances returned, so a worker that booted but never joined left the nodegroup falsely ACTIVE with no usable capacity. Gate ACTIVE on the cluster's Ready-node count (meta.NodeCount, refreshed by the cluster reconciler from the CP NATS state report — Ready nodes only) rising by the desired worker count over the create-time baseline. On timeout the nodegroup is marked CREATE_FAILED with its instance IDs retained for the reclaim path. Mirrors the CP reconciler's healthy-observe gating. Bounded by injectable timeout/poll knobs. mulga-siv-299 * feat(eks): private endpoint access via cross-account Set A ENI Make EndpointPrivateAccess real: in-VPC workers and kubectl reach the control plane through a customer-VPC (Set A) resident ENI on the cluster NLB's LB VM, removing the public-NLB hairpin and the NAT-GW egress path. Teardown: the Set A ENI is an extra cross-account NIC on the LB VM, not its primary, so the instance-terminate cascade never reclaims it. Detach (store-clear) the stale attachment before deleting, so private clusters delete cleanly instead of looping on InvalidNetworkInterface.InUse. Refs: mulga-siv-301 * fix(vpc): honor SecurityGroupRuleIds in RevokeSecurityGroup ingress/egress Modern Terraform (aws_vpc_security_group_ingress_rule) revokes per-rule resources by SecurityGroupRuleId with empty IpPermissions. The handlers only read IpPermissions, so they returned success while removing nothing; the surviving SG-reference rule then pinned both SGs undeletable (DependencyViolation), deadlocking destroy of mutually-referencing SGs. Resolve SecurityGroupRuleIds to their stored rules and remove them (malformed -> InvalidSecurityGroupRuleId.Malformed, absent -> NotFound). Refs: mulga-siv-321 * fix(vpc): reap the main route table on DeleteVpc CreateVpc auto-creates a main route table and DeleteRouteTable refuses to delete a main RT (AWS-faithful), so nothing reclaimed it — every deleted VPC leaked one orphaned main RT in the rtb KV bucket. DeleteVpc now reaps the VPC's main route table before deleting the VPC record. Refs: mulga-siv-322 * fix: make AttachRolePolicy and AssociateRouteTable atomic via KV CAS Concurrent writes to a single role / route table record used a blind read-modify-Put with no optimistic-concurrency guard, so parallel writers racing from the same revision clobbered each other and lost updates. Verified on env19: three parallel policy attachments to one node role kept only one; two parallel subnet associations to one route table kept only one (the provider then timed out waiting for state 'associated'). Route the mutating paths (AttachRolePolicy/DetachRolePolicy, AssociateRouteTable/DisassociateRouteTable) through a revision-guarded Update + bounded retry, matching the existing accountCounterBucket CAS idiom. Add concurrent regression tests covering both lost updates. * docs: add EKS Terraform workbooks (quickstart, addons-access, https-ingress) Three OpenTofu workbooks demonstrating EKS on Spinifex, each ending in a browsable load-balanced demo app: - eks-quickstart: minimal single-node cluster + NodePort demo - eks-addons-access: CoreDNS managed addon + access-entry RBAC - eks-https-ingress: 3-node cluster behind an HTTPS ALB terminating TLS with an ACM-imported self-signed certificate Each covers the Spinifex-specific endpoints, the auto-managed worker SG NodePort pattern, and the omitted addon_version constraint.
title, description, category, tags, resources
| title | description | category | tags | resources | |||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 3-Node EKS behind an HTTPS ALB (ACM TLS) | Provision an HA-shaped EKS cluster with public + private API endpoints and three workers, deploy a demo app, and publish it through an internet-facing ALB that terminates TLS with an ACM-imported certificate, using Terraform on Spinifex. | Terraform Workbooks |
|
|
Terraform: 3-Node EKS behind an HTTPS ALB
The largest EKS workbook, and the clearest demo. An HA-shaped cluster with public + private API endpoints and three workers, running a demo app published over HTTPS through an internet-facing ALB that terminates TLS with an ACM-imported certificate.
Table of Contents
Overview
This workbook is sized for a 3-node Spinifex deployment and is built to show off. It brings together the pieces from the smaller EKS examples, deploys a demo app across three workers, and fronts it with a TLS-terminating load balancer. When apply finishes, fetch the ALB's public IP and open https://<ip> — the nginxdemos/hello page reports which pod answered, and refreshing moves between the three replicas spread across the cluster. That single page demonstrates a multi-node Kubernetes cluster load-balancing behind HTTPS.
- Public + private API endpoints.
endpoint_public_accessserveskubectland the Kubernetes provider from your host (CIDR-restricted);endpoint_private_accessexposes an in-VPC private endpoint the workers use to join. - Private workers + NAT. The three workers sit in private subnets. They join the cluster over the in-VPC private endpoint, and a NAT gateway gives them outbound access to pull the demo container image.
- CoreDNS installed as a managed addon.
- TLS via ACM. A self-signed certificate is generated with the
tlsprovider and imported into ACM (Spinifex ACM supportsImportCertificate, notRequestCertificate). The ALB's HTTPS listener references the ACM ARN and forwards to the workers' NodePort. - The app, deployed by Terraform. The Kubernetes provider deploys the Deployment + NodePort Service, and one security-group rule lets the ALB reach the NodePort.
What you'll learn:
- Running an EKS cluster with both public and private API endpoints
- Placing workers in private subnets that join over the private endpoint and egress via NAT
- Importing a certificate into ACM and terminating TLS on an ALB HTTPS listener
- Wiring an ALB target group to managed node-group workers via a NodePort, including the SG rule that allows it
- Deploying a load-balanced workload with the Kubernetes provider
What gets created
| Resource | Name | Purpose |
|---|---|---|
| VPC | eks-https-vpc |
Isolated network (10.32.0.0/16) |
| Public Subnets | eks-https-public-a/-b |
Host the ALB and NAT gateway |
| Private Subnets | eks-https-private-a/-b |
Host the workers |
| Internet Gateway | eks-https-igw |
Internet for the public subnets |
| NAT Gateway (+ EIP) | eks-https-nat |
Outbound for the private workers (image pulls) |
| Route Tables | eks-https-public-rt, eks-https-private-rt |
Public via IGW; private via NAT |
| IAM Roles | eks-https-cluster-role, eks-https-node-role |
Control-plane and worker roles |
| EKS Cluster | eks-https |
Public (CIDR-restricted) + private endpoints |
| Node Group | workers |
Three t3.large workers in the private subnets |
| Addon | coredns |
Cluster DNS |
| ACM Cert | eks-https-ingress |
Self-signed cert imported into ACM |
| ALB + HTTPS Listener | eks-https-alb |
Internet-facing, terminates TLS on :443 |
| Target Group | eks-https-tg |
Instance targets on the NodePort |
| SG Ingress Rule | eks-https-alb-nodeport |
Lets the ALB SG reach the worker NodePort |
| K8s Deployment + Service | hello |
nginxdemos/hello, 3 replicas, NodePort |
Spinifex specifics
- ACM supports
ImportCertificate,DescribeCertificate,ListCertificates,DeleteCertificate— there is noRequestCertificate/DNS-validation flow, hence the self-signed import. - The ALB HTTPS listener resolves the cert from ACM. Valid SSL policies are
ELBSecurityPolicy-2016-08(default) andELBSecurityPolicy-TLS13-1-2-2021-06. - The worker SG is auto-managed (
vpc_config.security_group_idsis ignored) and admits only intra-cluster traffic. The ALB→NodePort rule is added by looking the SG up by its deterministic name and admitting the ALB SG. - Worker discovery uses the
spinifex:eks-clustertag Spinifex stamps on node-group instances; the target-group attachment count is the knownnode_desired_size. - Leave
addon_versionunset (see the addons workbook for why pinning it can't satisfy both the provider and the catalog).
Prerequisites:
- A 3-node Spinifex deployment, installed and running
- The Spinifex
eks-nodeimage available on the cluster - OpenTofu or Terraform, plus
kubectland the AWS CLI
Instructions
Step 1. Get the Template
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform-workbooks
cd docs/terraform-workbooks/eks-https-ingress
Or create a main.tf file and paste the full configuration below.
Step 2. Deploy
export AWS_PROFILE=spinifex
tofu init
tofu apply
Expect several minutes: the control plane bootstraps, three workers launch and join over the private endpoint, the NAT gateway comes up, CoreDNS installs, the ALB is provisioned, and the Kubernetes provider rolls out the demo app.
Keep
AWS_PROFILE=spinifexexported for the wholeapply— the Kubernetes provider authenticates withaws eks get-tokenagainst the Spinifex STS endpoint.
Tighten access in production.
api_public_access_cidrandalb_ingress_cidrboth default to0.0.0.0/0. Set them to your own CIDR:export TF_VAR_api_public_access_cidr="203.0.113.10/32" export TF_VAR_alb_ingress_cidr="203.0.113.10/32"
Step 3. Open the Demo over HTTPS
The ALB DNS name (*.elb.spinifex.local) won't resolve from your host, so fetch its public IP:
ALB_IP=$(aws elbv2 describe-load-balancers --names eks-https-alb \
--query 'LoadBalancers[0].AvailabilityZones[].LoadBalancerAddresses[].IpAddress' \
--output text)
The certificate is self-signed, so a browser will warn (proceed anyway) and curl needs -k:
curl -k https://$ALB_IP
curl -k https://$ALB_IP # refresh: the "Server name" changes between pods
Open https://$ALB_IP in a browser and refresh to watch requests land on different pods across the three nodes.
Step 4. Inspect (optional)
aws eks update-kubeconfig --name eks-https --region ap-southeast-2
kubectl get nodes
kubectl get pods -o wide # three replicas, one per node
TG_ARN=$(aws elbv2 describe-target-groups --names eks-https-tg \
--query 'TargetGroups[0].TargetGroupArn' --output text)
aws elbv2 describe-target-health --target-group-arn "$TG_ARN"
Cleanup
tofu destroy
Troubleshooting
Targets Unhealthy / ALB Returns 5xx
The ALB health-checks / on the NodePort. Targets turn healthy once the demo pods are running and the SG rule admits the ALB. Check, in order:
kubectl get pods -o wide # pods Running?
aws ec2 describe-security-groups \
--filters "Name=group-name,Values=eks-cluster-eks-https-nodegroup-sg" \
--query 'SecurityGroups[0].IpPermissions' # NodePort rule present?
aws elbv2 describe-target-health --target-group-arn "$TG_ARN"
If pods are ImagePullBackOff, the workers can't reach Docker Hub — confirm the NAT gateway is available:
aws ec2 describe-nat-gateways --query 'NatGateways[].[NatGatewayId,State]'
Workers Never Join
The workers join over the private endpoint. Confirm it's enabled and the node group is progressing:
aws eks describe-cluster --name eks-https --query 'cluster.resourcesVpcConfig.endpointPrivateAccess'
aws eks describe-nodegroup --cluster-name eks-https --nodegroup-name workers --query 'nodegroup.status'
Target-Group Attachment Errors at Apply
The attachment indexes the discovered worker IDs by node_desired_size. If apply errors with an index out of range, the workers weren't all tagged/running when the aws_instances lookup ran. Re-run tofu apply — the node group will be ACTIVE by then:
aws ec2 describe-instances --filters "Name=tag:spinifex:eks-cluster,Values=eks-https" \
--query 'Reservations[].Instances[].[InstanceId,State.Name]' --output text
kubernetes provider: connection refused / Unauthorized
The provider runs aws eks get-token against the Spinifex STS endpoint. If the cluster was still CREATING when the provider first connected, re-run tofu apply. Otherwise confirm:
aws sts get-caller-identity
aws eks describe-cluster --name eks-https --query 'cluster.status'
ACM Import / HTTPS Listener Errors
aws acm list-certificates
aws acm describe-certificate --certificate-arn "$(tofu output -raw certificate_arn)" \
--query 'Certificate.Status'
A CertificateNotFound on the listener means the ACM ARN didn't resolve — check the cert exists and belongs to the same account/profile.
Provider Connection Refused
sudo systemctl status spinifex.target
curl -k https://localhost:9999/