1 Commits

Author	SHA1	Message	Date
Josh Siviour	2acb37342d	feat(eks): aws-load-balancer-controller addon — real ALB Ingress (L7, IRSA) (#376) ... * fix(ec2): stamp subnet AvailabilityZoneId for single-AZ ALB resolution DescribeSubnets omitted AvailabilityZoneId, so the AWS Load Balancer Controller's DescribeAvailabilityZones-by-ZoneId lookup resolved no AZ info ("cannot resolve AZ info for AZs: []") and never built the ALB model. Stamp the single zone-id on every subnet and share it with DescribeAvailabilityZones through one SingleZoneID constant so the two responses can never drift. * feat(eks): add aws-load-balancer-controller v2.8.1 addon bundle Baked manifests for the LBC managed addon (CRDs, controller Deployment/ RBAC/webhooks/Service, IngressClass alb, IRSA render markers). Two wirings make it work against the spinifex gateway: - --aws-api-endpoints carries sts= alongside elasticloadbalancing/ec2/ acm. The aws-sdk-go v1 credential bootstrap resolves the STS client from the session endpoint resolver, not AWS_ENDPOINT_URL_STS, so the web-identity AssumeRoleWithWebIdentity must be pointed at the gateway here or it falls back to real AWS STS. - --feature-gates=ALBSingleSubnet=true drops the ALB minimum subnet count from two to one so an ALB provisions in a single-AZ deployment. * feat(eks): render IRSA into managed addons + node-minted webhook cert Extend the addon-sync renderer to inject the projected-token / role-arn / gateway STS-endpoint + CA block into managed-addon pod specs via opt-in markers, and substitute cluster-name/region/vpc-id/gateway-endpoint/CA placeholders. Add cmd/eks-webhook-cert: the eks-node AMI has no openssl, so this mints (then reuses) a self-signed ECDSA-P256 serving cert for the admission webhook, cached on disk so the render is byte-stable across ticks. Wire both into the image build (manifest.conf) plus INSTALL_FILES for the LBC bundle. * feat(eks): surface cluster VPC id as EKS_VPC_ID for the LB controller Thread the control-plane VPC id through K3sServerInput.VpcID into the first-boot EKS_VPC_ID env so the in-cluster aws-load-balancer-controller can pass --aws-vpc-id to the gateway elbv2/ec2 handlers. * fix(eks): point the LB controller at the customer VPC + customer-facing gateway The aws-load-balancer-controller ran with two control-plane-scoped values that customer worker pods cannot use: - --aws-vpc-id was set from the managed control-plane VPC (cpRefs.VpcID), which the gateway's DescribeVpcs does not return for customer calls, so backend-SG auto-create failed with InvalidVpcID.NotFound. Source it from meta.ResourcesVpcConfig.VpcId, the customer cluster VPC where the workers and their subnets live and where the ALB is provisioned. - The addon render baked the mgmt gateway endpoint (EKS_GATEWAY_URL / SystemGatewayURL) into pod specs, unreachable from worker pods. Add a distinct EKS_ADDON_GATEWAY_URL (the customer-facing GatewayBaseURL, same host as the OIDC issuer) and substitute it for {{GATEWAY_ENDPOINT}} in managed addon manifests. EKS_GATEWAY_URL stays mgmt for the CP's own bootstrap, state-report and token-webhook paths. * fix(elbv2): resolve forward target group from ForwardConfig listenerActionFromSDK read only the legacy flat Action.TargetGroupArn, so a forward action carrying its target group via ForwardConfig.TargetGroups (the shape the aws-load-balancer-controller emits) left the action target empty and CreateRule/CreateListener rejected it with MissingParameter. Flatten the first target group from ForwardConfig when the flat field is absent. * fix(ec2): mirror CreateTags/DeleteTags into subnet/vpc records The generic S3 tag store and the per-resource KV records are disjoint: DescribeSubnets/DescribeVpcs read tags from their KV record, while CreateTags only wrote the generic store. Subnet tags applied after create were therefore invisible to tag-filtered describes, breaking aws-load-balancer-controller subnet auto-discovery (kubernetes.io/role/elb). Write through CreateTags/DeleteTags to the owning subnet/vpc record so tag-filtered describes observe them. Resource types the VPC service does not own are skipped; the generic store stays their record of truth. * test(daemon): pin synthetic capacity in host-dependent allocate tests TestCanAllocate_CountEdgeCases/Mixed_instance_types and TestHandleEC2DescribeInstanceTypes/NoFilterReturnsSupportedRegardlessOfAllocation called rm.allocate against live-detected host capacity. allocate gates on readMemAvailableGB (live /proc MemAvailable), so a box already running VMs fails the allocation spuriously, breaking the preflight gate locally. Pin hostVCPU/hostMemGB/reserved and nil readMemAvailableGB to a synthetic host, matching the sibling Capacity_decreases_after_allocation subtest. * fix(eks): route apiserver->pod traffic via agent tunnel (egress-selector cluster) In the managed-CP topology the apiserver VM sits in the system CP VPC with no route to the worker pod network in the unpeered customer VPC. k3s default egress-selector-mode=agent tunnels only apiserver->kubelet, so kubectl logs/exec work but apiserver->pod traffic (admission webhooks, aggregated APIs) is attempted directly and times out -- the aws-load-balancer-controller webhooks fail with context deadline exceeded, blocking Service/Ingress creation cluster-wide. Set egress-selector-mode=cluster so apiserver->pod/service traffic routes through the agent's outbound tunnel, selected by the node hosting the destination, without requiring inbound CP->worker reachability. * feat: bump aws-load-balancer-controller addon bundle to v2.11.0 v2.8.1 predates the ALBSingleSubnet feature gate, so single-AZ spinifex clusters fail Ingress reconcile (subnet resolver dedups to 1 < required 2) and never provision an ALB. v2.11.0 honors --feature-gates=ALBSingleSubnet=true. Replace the baked bundle: new CRDs (ingressclassparams + targetgroupbindings), controller manifest, and image tag v2.11.0, re-applying the spinifex deltas (IRSA SA annotation, node-minted webhook serving-cert secret, gateway-CA secret, gateway aws-api-endpoints, caBundle on every webhook clientConfig, alb IngressClass). Pick up the new ClusterRole configmaps rule. Drop v2.8.1. * fix: point LBC at the gateway via AWS_ENDPOINT_URL (sdk-v2) LBC v2.11 uses aws-sdk-go-v2, which keys --aws-api-endpoints on SDK service IDs (e.g. 'Elastic Load Balancing v2'), not the legacy lowercase names the bundle passed. Those overrides were silently ignored, so every non-STS call (ec2/elbv2/acm) went to real AWS and failed TLS against the gateway CA -- no ALB could provision. Inject AWS_ENDPOINT_URL in the shared IRSA env block (sdk-v2 global base-endpoint override; AWS_ENDPOINT_URL_STS still takes precedence for STS) and drop the dead --aws-api-endpoints flag from the controller args. * fix: synthesize taggable ARN for default listener rule Gateway DescribeRules returned the synthetic IsDefault rule with a null RuleArn. LBC read the arn-less default rule after creating a listener and issued DescribeTags with an empty resource ARN, which the gateway rejected with 400 MissingParameter, aborting reconcile. Set a deterministic RuleArn on the default rule, resolve it back to its parent listener in DescribeTags (empty tags) and DescribeRules by-arn. Also disable shield/waf/wafv2 in the LBC bundle to drop unsupported calls. * feat: inject ELB subnets into alb IngressClassParams for single-AZ ALB LBC's tag auto-discovery never threads the ALBSingleSubnet feature gate, so a single-AZ cluster (subnets sharing one AZ) dedups to 1<2 subnets and aborts Ingress reconcile. Only the explicit-subnet path honors the gate. Resolve the cluster's ELB-eligible subnets, dedup to one per AZ, and surface them as EKS_ELB_SUBNET_IDS. The addon sync renders them into the alb IngressClassParams .spec.subnets.ids so every Ingress takes the explicit path. An empty set falls back to auto-discovery, preserving the native UX. * fix: route LBC service clients to gateway via --aws-api-endpoints, link IngressClassParams Live verification showed two gaps in the LBC addon bundle: LBC installs a per-client BaseEndpoint resolver, so the global AWS_ENDPOINT_URL is ignored for the ec2/elbv2/acm clients (calls hit real AWS, failing TLS). Redirect them with --aws-api-endpoints keyed on the aws-sdk-go-v2 service IDs (EC2, "Elastic Load Balancing v2", ACM, STS) rather than the legacy lowercase names the flag never matched. Drop the misleading AWS_ENDPOINT_URL env; the web-identity STS client still resolves via AWS_ENDPOINT_URL_STS. The injected IngressClassParams was orphaned: the alb IngressClass had no .spec.parameters, so LBC fell back to subnet auto-discovery (which fails on a single-AZ cluster). Link the IngressClass to the IngressClassParams so the explicit-subnet path is taken. * fix: stamp EKS-parity providerID on worker nodes for LBC ip targets k3s registers workers with providerID k3s://<name>, which the AWS Load Balancer Controller rejects (providerID invalid for EC2 instances) when resolving nodes for ip-mode target registration. Resolve the instance-id and AZ from IMDSv2 at agent boot and set --kubelet-arg=provider-id=aws:///<az>/<id> so LBC can map the node to its EC2 instance. On any IMDS miss the agent keeps the default providerID and still starts. * fix(elbv2): honor SubnetMappings in CreateLoadBalancer LBC supplies ALB subnets via SubnetMappings, not the plain Subnets list. createLoadBalancer only read input.Subnets, so ENI allocation was skipped, VpcId/AZ stayed empty, and the HAProxy data-plane VM never launched — the ALB went active but could not serve traffic. Add flattenSubnetIDs to read both sources and reuse it in createLoadBalancer and desiredSubnetSet. * fix(vm): capture GCInterval before spawning GC goroutine TestStart_SweepsImmediatelyThenStopsOnCtxCancel mutates the package global GCInterval and restores it in a defer, racing the Start goroutine's read at time.NewTicker(GCInterval) under -race. Read GCInterval on the caller's goroutine into a local before spawning so the global access is ordered with the test's defer; the goroutine touches only the captured local.	2026-06-18 10:53:05 +10:00