8 Commits

Author	SHA1	Message	Date
Josh Siviour	b23fa3bf60	feat(eks): managed control-plane VPC (Set B) + control-plane lifecycle hardening (#356) ... * feat(eks): VM-side managed-addon delivery transport Stage CreateAddon manifests in KV host-side; the control-plane VM pulls the staged set through the AWS gateway (signed GET /clusters/{name}/internal-addons/ {accountId}), renders the baked bundles into the K3s auto-deploy dir, and reports per-addon delivery status on a dedicated eks.addon.*.status channel that the cluster reconciler CASes onto AWS-visible AddonStatus (CREATING->ACTIVE). - gateway: ListInternalAddons route + ListStagedAddonManifests NATS action - handlers: addon status reconciler (nextAddonStatus), staging installer - eks-node AMI: eks-gateway-fetch binary + mulga-eks-addon-sync agent - agent renders idempotently (cmp before replace) so K3s auto-deploy does not re-apply every tick and clear the Addon CR .status.gvks readiness signal - agent mirrors per-tick delivery outcomes to the serial console for host-side observability * fix(eks): provision cluster NLB synchronously so the endpoint carries the real front-end IP The cluster NLB front-end IP is assigned by the async LB-VM launch (launchLBVMAsync), but EnsureClusterNLB read it back synchronously right after CreateLoadBalancer returned — losing the race and falling back to the synthetic <lb>.elb.spinifex.local DNS name. That name is unresolvable (no Route53) and is not in the apiserver serving-cert SAN, so kubectl against the cluster never connects. Split the elbv2 create: the public CreateLoadBalancer stays async (267.4 — an inline multi-minute VM boot head-of-line-blocks the gateway responder), and a new CreateLoadBalancerSync drives the data-plane launch inline and returns the LB carrying its front-end address. The EKS cluster-create worker runs off the gateway responder, so it uses the sync entry and reads the IP from the return value instead of racing persisted state. No polling. - elbv2: factor provisionLBDataPlane out of launchLBVMAsync; add CreateLoadBalancerSync - eks: ensureClusterLB uses the sync entry, sets FrontendIP from the result, drops the read-back; EnsureClusterNLB fails loud (ErrClusterNLBFrontendIPUnavailable) when no front-end IP is allocated rather than shipping the DNS-name fallback - eks: drop the DNS-name endpoint fallback; the endpoint is always the front-end IP * feat: auto-wire IGW for internet-facing EKS cluster endpoint An internet-facing cluster NLB allocates an external front-end IP and the OVN dnat_and_snat for it, but the external switch + localnet + gateway LRP that make that IP answerable on the wire are built only on IGW attach. The control-plane NLB is a system instance, so ensure an IGW on cluster build and reclaim it on teardown. EnsureClusterIGW (idempotent, reuses a customer IGW untouched) runs before EnsureClusterNLB when the endpoint is public; DeleteClusterIGW (ownership- scoped) removes only the cluster-owned IGW in purgeClusterInfra, before the VPC delete. New IGW dep wired from the daemon. * fix(eks): tear down infra on failed cluster launch to stop LB EIP leak A CreateCluster that fails during its background launch flipped the meta to FAILED but performed no teardown, relying on a later same-name create to reclaim and purge the infra. EKS cluster names are unique per create, so that reclaim never fires: the internet-facing load-balancer VM keeps running with its allocated+associated dnat_and_snat EIP indefinitely. Over repeated create/delete cycles the EIP pool fills with allocations bound to terminated instances, so a later allocation's advertised endpoint and its live OVN NAT diverge onto different pool addresses. - failClusterLaunch now runs a best-effort purgeClusterInfra on the in-memory meta (freshest refs even when the failing step was the meta write itself) before MarkClusterFailed. purgeClusterInfra is idempotent, so a later reclaim or DeleteCluster re-purge is a no-op; the FAILED meta is retained for observability. - Daemon releaseSystemInstanceEIP extracted and called from both TerminateSystemInstance and cleanupFailedSystemInstance, so a launch that fails after AssociateAddress releases the eipService allocation instead of leaking it (MarkFailed's ReleasePublicIP only knows the externalIPAM path). * feat: EKS managed control-plane VPC (AWS Set B) composed from EC2 APIs CreateCluster now auto-builds the managed control-plane VPC under the system account, the spinifex analogue of AWS EKS's hidden managed-account VPC: a public subnet for the internet-facing NLB, private subnet(s) for the k3s control-plane VM(s), and a NAT gateway giving the private CP egress for image pulls. It is composed from the real EC2 VPC-family APIs (CreateVpc/Subnet, IGW, RouteTable/Route, NatGateway), so per-subnet egress gating is wired by the existing topology subscribers off the route-table events those APIs publish — no direct OVN mutation. This resolves the mulga-siv-292 root cause: the internet-facing NLB and the private CP were crammed into one customer subnet with no IGW route, so both were treated as private and dropped by the lr_in_policy 1100 rule, killing the LB-VM return leg and lb-agent heartbeat. The CP only survived via a per-instance 1200 reroute bandaid, now removed — the topology expresses egress instead. Placement is uniform single/multi-node (CP always in the private subnet). The system LB VM heartbeats over the mgmt bridge (SystemGatewayURL) rather than the WAN. Managed-VPC teardown reclaims NAT GW + EIP, route tables, subnets, IGW, and the VPC in dependency order. Customer subnetIds (Set A / worker side) remain consumed-not-created and are deferred to the worker round. * fix(eks): bake cluster-owner account into CP-VM EKS_ACCOUNT_ID The managed control-plane VPC (Set B) launches the K3s control-plane VMs and their ENIs under the system (infra) account, but the on-VM agents — bootstrap publish, state report, token-review, and add-on fetch — must key on the cluster-OWNER (customer) account that owns the cluster meta, namespaces, and AccessEntry KV. Add ClusterAccountID to K3sServerInput and bake it into the VM's EKS_ACCOUNT_ID, decoupling it from the infra AccountID the VM launches under. Without this the VM reported state and fetched add-ons against the system account, so the cluster never observed a healthy control plane (stuck CREATING, null CA) and kubectl/add-on delivery timed out. Also refresh the e2e suite doc comment to describe the Set B topology and the data-plane subtests (kubectl + managed-addon delivery) that now run. * fix(eks): explicitly delete addon objects on unstage K3s does not delete the applied resources when an auto-deploy manifest file is removed from the server manifests dir (documented behaviour). The addon-sync GC pass relied on that, so after DeleteAddon the rendered manifest was dropped but its Namespace/ConfigMap stayed standing — the e2e AddonDelivery GC wait timed out. Copy the manifest aside, drop the original so K3s stops tracking it, then kubectl delete the objects (--wait=false to avoid finalizer stalls).	2026-06-12 22:29:24 +10:00
Josh Siviour	74b91b5cc5	feat(eks): HA control plane — 3-node etcd quorum spread across distinct hosts (#335) ... * feat(eks): node-targetable control-plane VM (HA spread transport prereq) Groundwork for HA control-plane placement (mulga-siv-231.7.1): make the k3s control-plane VM placeable on a specific Spinifex host. - Add a sys.medium instance type (2 vCPU / 4 GB) and boot the control-plane VM as it instead of t3.medium. A sys.* type keeps the CP VM out of customer DescribeInstanceTypes and makes the daemon register the node-targeted system.LaunchInstance.{type}.{nodeID} subject (only sys.* types get it). - Add K3sServerInput.TargetNodeID and a LaunchSystemInstanceOnNode method on the launcher: empty/local node launches in-process (unchanged single-CP path); any other node dispatches over the node-targeted subject, where the receiving daemon runs the launch and owns the VM. The spread orchestrator (reserve N hosts, launch per host, verify, rollback, <3-host fallback) lands separately; CreateCluster still launches locally (TargetNodeID empty), so behaviour is unchanged. * feat(eks): HA control-plane spread orchestrator Place N=3 k3s server VMs across distinct Spinifex hosts via a per-cluster spread placement group, with all-or-nothing rollback and single-CP fallback. - placeControlPlane: capacity fan-out (HostScheduler) -> >=3-host gate -> ensure spread group (system account) -> ReserveSpreadNodes(Min=Max=3) -> parallel per-node LaunchK3sServerVM(TargetNodeID) -> node-vms placement verify -> FinalizeSpreadInstances. Any launch/verify/finalize failure terminates launched VMs, releases the reservation, drops the group. - <3 schedulable hosts or a lost reservation race falls back to a single local control-plane VM (prior behaviour). - ClusterMeta gains ControlPlaneNodes[] + ControlPlaneSpreadGroup; [0] is mirrored into the scalar ControlPlane* fields for back-compat. DeleteCluster tears down every CP VM and the spread group. - Wire PlacementGroup + Scheduler into EKSServiceDeps + orchestration-deps gate. * fix(eks): size HA control-plane spread on raw host headroom SchedulableHosts matched the CP type by name in node.status's per-type capacity list, but that list omits sys.* types (system types are hidden from customers). The CP VM is sys.medium, so the match never hit and SchedulableHosts returned 0 on every node — placeControlPlane always fell back to a single CP, making the >=3-host spread unreachable in production. System VMs consume host vCPU/memory like any guest, so size the fit against each node's raw schedulable headroom (Total - Reserved - Alloc) and the type's footprint instead. Adds instancetypes.SpecForSystemType for the sys.* vCPU/memory spec and a pure nodeFitsSystemInstance helper. * feat(eks): k3s server-join role for HA control-plane quorum Servers 2..N now boot in join mode: server: https://<first-server>:6443 + a shared cluster token, without cluster-init, so they join the first server's embedded-etcd quorum instead of each cluster-initing an independent datastore. - K3sServerInput gains JoinToken (shared cluster secret, set on every server) and ServerURL (set => join server, no cluster-init, server-join role). - buildK3sUserData branches cluster-init vs server:, emits token:, and selects SPINIFEX_K3S_ROLE server vs server-join. - launchControlPlaneSpread launches nodes[0] first (cluster-init), derives the join URL from its ENI IP, then launches the rest in parallel as join servers; a first-server failure skips the joins and rolls back. - CreateCluster mints a per-cluster crypto-random token (GenerateK3sClusterToken) threaded into the placement template. - eks-node-role.sh adds a server-join case: token-webhook + k3s + state-report, but not k3s-first-boot (the first server already publishes the bootstrap artifacts; a join re-publish only races the bootstrap bus). * fix: open etcd-peer/kubelet ingress on EKS control-plane SG for HA join Join servers (231.7.2) register over the supervisor port 6443 but their embedded etcd could not peer with the quorum: the control-plane SG only admitted 6443 from the VPC CIDR, omitting etcd client+peer 2379-2380 and kubelet 10250 between control-plane nodes. A single cluster-init server never exercises those ports, so the gap stayed hidden until multi-server HA — the join registered but its etcd never replicated the raft log, leaving the node perpetually NotReady. Add EnsureControlPlaneHAIngress with self-referencing (cpSG->cpSG) rules for etcd 2379-2380 and kubelet 10250, called from CreateCluster after EnsureControlPlaneIngress. Verified live on a 3-node cluster: join servers reach Ready and form a 3-member etcd quorum that survives loss of the original first server. * feat: register all control-plane ENI IPs as EKS cluster NLB targets Register every HA control-plane server ENI IP in the cluster NLB target group, not just the primary. The TG already health-checks :6443, so a dead member drains and kubectl/apiserver traffic fails over to surviving etcd-quorum members. Add RegisterClusterTargets batch helper; the single RegisterClusterTarget now delegates to it.	2026-06-10 12:25:47 +10:00
Josh Siviour	4ef19fc488	feat(ui): EKS console (#310)	2026-06-09 10:57:50 +10:00
Josh Siviour	496424c41c	test(eks): EKS control-plane E2E suite (Sprint 6g) (#304) ... * test(eks): EKS control-plane e2e suite (Phase 1, API-level) Add tests/e2e/eks/eks_test.go driving the EKS lifecycle against awsgw: CreateCluster -> ACTIVE (endpoint + CA asserted), kubeconfig artifact assembly, AccessEntry CRUD, get-token presign decode, DeleteCluster -> gone. Harness: EKS SDK client on AWSClient + WaitForEKSCluster{Active,Deleted} pollers. Wire e2e-eks into the Makefile and the single-node CI suite list. * test(eks): Phase 2 kubectl get-nodes data-plane gate Add the reachability shim (harness/eks_mgmt.go) that reads the control-plane VM's br-mgmt IP from the cluster KV meta over NATS, plus a kubectl wrapper (harness/kubectl.go) that skips when kubectl is absent. KubectlGetNodes points kubectl at https://<mgmtIP>:6443 (insecure-skip-tls-verify; cert SANs the NLB DNS, not the mgmt IP) with an aws eks get-token exec block and polls until the control-plane node is Ready — the get-token webhook auth regression gate. IRSA pod + the real NLB/Route53 passthrough are deferred to a follow-up. * test(eks): tolerate doubled NotFound code + dial loopback NATS DeleteCluster detection matched the SDK constant ResourceNotFoundException, but awsgw emits ResourceNotFoundExceptionException on the wire — match the NotFound code by substring so deletion is detected immediately instead of timing out. The mgmt-IP shim dialed NATS.Host verbatim (0.0.0.0:4222), which the serving cert does not SAN; rewrite the wildcard bind to loopback. Validated live: all 6 subtests green, kubectl get nodes shows the control-plane node Ready via the get-token webhook auth path. * feat: build system images with libguestfs (no host block devices) Replace the qemu-nbd + sudo mount + chroot + /dev-bind flow in build-system-image.sh with libguestfs (virt-customize / virt-resize / guestfish). Every operation runs in an isolated userspace appliance VM with its own kernel, so the build touches no host block device, no host mount, and needs no sudo — removing the failure class that wiped the host's /dev on shared CI runners (a leaked /dev bind traversed by rm -rf). Applies to all distros: alpine resizes via guestfish e2fsck+resize2fs, ubuntu via virt-resize; customization (packages, binaries, files, setup script, service enable, cache cleanup) is one ordered virt-customize invocation. LIBGUESTFS_BACKEND=direct (no libvirt on runners). Wire the eks-node AMI into the single-node e2e job: the runner builds the device-free raw, ships it to the VM, and the VM imports it (predastore/NATS are VM-only) so the eks suite's CreateCluster has an AMI to boot. * feat(eks): bootstrap + state-report via AWS gateway broker Converge the EKS control-plane → host channel onto the ELBv2 lb-agent model: the K3s server VM publishes its bootstrap envelopes and periodic state reports via SigV4-signed HTTPS POST to the AWS gateway, instead of dialing core NATS directly. A new gateway action (PublishInternal, POST /clusters/{name}/internal-publish) relays the payload verbatim onto the existing eks.bus.* / eks.state.* subjects, so the daemon-side subscriber, KV persistence/encryption, and reconciler are unchanged. This removes the VM's need to reach NATS at all, so NATS can stay cluster-internal behind its bind/firewall/mTLS controls — the prior direct path broke when NATS was not VM-reachable (e2e CreateCluster timeout) and was DDIL-fragile (fire-and-forget nats pub, no ack/retry). - gateway/eks: PublishInternal broker + route; closed-set channel/kind validation - cmd/eks-gateway-publish: one-shot SigV4 publisher (retry-until-ack) baked into the eks-node AMI; replaces the nats-cli publisher in k3s-first-boot.sh and mulga-eks-state-report.sh - handlers/eks + daemon: hand the K3s server VM the mgmt-reachable gateway URL + system (Predastore) SigV4 creds + gateway CA instead of NATS URL/token/CA; new resolveSystemGatewayBaseURL (mgmt-bridge-first) - eks-node image: build/install eks-gateway-publish, drop nats-cli download * feat: migrate eks-token-webhook to the AWS gateway broker The TokenReview webhook dialed core NATS directly (STS verify + AccessEntry KV read). On a mgmt-only control-plane VM with NATS bound off-subnet that fails, so the webhook exited at startup, the apiserver token-webhook kubeconfig was never written, and the apiserver crash-looped — the cluster never reached ACTIVE. Converge it onto the same gateway-broker model as bootstrap/state-report and ELBv2's lb-agent: the webhook SigV4-signs an HTTPS POST {accountId,token} to /clusters/{name}/token-review, and the gateway runs the verify + lookup host-side and returns the resolved identity. The VM now speaks no core NATS. - internal/eksgw: shared on-VM SigV4 client (also adopted by eks-gateway-publish). - handlers/eks/token_review.go: Authenticate decision core (moved from the webhook binary) + ResolveTokenReview host-side wiring. - gateway/eks token-review route + WebhookTokenReview handler. - webhook: drop NATS, relay via eksgw.Client; broker fault -> 503 (retryable) not a silent deny. - initd: export EKS_GATEWAY_* / EKS_ACCESS_* creds, not SPINIFEX_NATS_*. * fix: build eks-node helpers with GOFIPS140; cover token-review broker eks-gateway-publish blank-imports internal/fipsboot, whose init() panics unless the binary is built with GOFIPS140=v1.0.0. The eks-node AMI build omitted the flag, so the binary paniced at first boot right after k3s came up — k3s-first-boot (set -eu) died before publishing the bootstrap token and the cluster never reached ACTIVE. Build both eks-node helpers with GOFIPS140=v1.0.0, matching cmd/lb-agent. Add NATS-backed unit tests for ResolveTokenReview and the WebhookTokenReview gateway handler (verify responder + account KV) to cover the new broker path. * fix: export first-boot env so eks-gateway-publish child sees it k3s-first-boot.sh and mulga-eks-state-report.sh sourced /etc/spinifex-eks/ first-boot.env but never exported the vars, so the eks-gateway-publish child process inherited an empty environment and exited '--account-id is required'. The control-plane VM came up healthy but never published its bootstrap envelopes, so the cluster never reached ACTIVE. Wrap the source in set -a / set +a so the KEY=value lines export to child processes. * fix: make awsgw start --host override actually bind The 'awsgw start --host' override printed 'Overwriting awsgw host' but the assignment was commented out, so AWSGW always bound the spinifex.toml host (a specific WAN IP, e.g. 192.168.0.10). A system instance (EKS K3s control plane, internal LB agent) has a mgmt-only NIC on-link to the br-mgmt IP with no route to the WAN IP, so its SigV4 POSTs to the gateway were refused and the cluster never bootstrapped. Honour --host (e2e passes 0.0.0.0:9999) so AWSGW listens on all interfaces incl. br-mgmt; the serving cert already SANs every local interface IP, and WAN clients still reach it. * test(eks): install kubectl on e2e VM so KubectlGetNodes runs The eks data-plane subtest (TestEKS/KubectlGetNodes) drives the full aws eks get-token -> kubectl -> apiserver -> token-webhook -> gateway broker -> STS verify path — the only end-to-end exercise of the token-review broker. The e2e binaries run on the VM, where kubectl was absent, so harness.NewKubectl t.Skip'd it and the path went unvalidated. Install a checksum-verified kubectl (pinned to the k3s server version) on the VM, gated on e2e-eks.	2026-06-07 20:42:25 +10:00
Josh Siviour	5aaf8c23d2	fix(eks): make aws eks get-token → kubectl auth work end-to-end (Sprint 6f) (#303) ... * feat(eks): webhook observability — console log sink + per-request TokenReview decision * fix(eks): wire token-webhook authenticator into generated k3s apiserver config The generated /etc/rancher/k3s/config.yaml overrides the AMI-baked config.yaml.skel, but its kube-apiserver-arg list omitted the authentication-token-webhook-config-file arg. The running apiserver therefore never loaded the token webhook, so bearer tokens minted by 'aws eks get-token' 401'd without the webhook ever being invoked. * fix(eks): echo TokenReview audiences so api-audiences apiserver accepts the principal With --api-audiences set, kube-apiserver sends spec.audiences in the TokenReview and discards an authenticated=true response that does not confirm a matching audience. The webhook returned an empty status.audiences, so a valid IAM principal surfaced as a 401. Echo the request's audiences on success. * fix(eks): pin token-webhook version v1 so apiserver accepts the response kube-apiserver --authentication-token-webhook-version defaults to v1beta1, but the eks-token-webhook responds with an authentication.k8s.io/v1 TokenReview. The apiserver decodes the response against the expected GVK and rejects the mismatch, surfacing a valid principal as a 401. Pin v1 to match the webhook.	2026-06-06 22:11:50 +10:00
Josh Siviour	a22bc2f716	feat(eks): 6d nodegroup provisioning + NATS-gated cluster health (#300) ... * feat(eks): nodegroup provisioning + K3s agent join (6d) CreateNodegroup launches N customer-owned worker EC2 from the unified eks-node AMI in agent role; each joins the cluster via K3S_URL + K3S_TOKEN baked into cloud-init (first-boot.env SPINIFEX_K3S_ROLE=agent + agent.env). Workers are customer-owned and DescribeInstances-visible (no ManagedBy, no mgmt NIC) via the normal RunInstances path, matching real EKS. K3S_URL targets the control-plane ENI's in-VPC private IP on :6443 — the cluster NLB DNS endpoint is not resolvable in-VPC yet; switch to the NLB endpoint on :443 once authoritative in-VPC DNS lands. K3S_TOKEN is the decrypted cluster node-token persisted at server bootstrap. Describe/List/Update(scaling)/Delete round out CRUD. Cluster<->nodegroup SG ingress rules (apiserver 6443, flannel VXLAN 8472/udp, kubelet 10250, intra-nodegroup) authorized idempotently. UpdateNodegroupVersion deferred. * feat: gate EKS cluster health via NATS state reports The K3s apiserver binds the VPC node-ip, unreachable from the host-side spinifex daemon, so the reconciler cannot probe /healthz directly. Add a control-plane state reporter that publishes {healthz, node_count} over the management NATS bus the daemon already shares. - eks-node server role runs mulga-eks-state-report as an OpenRC service, publishing eks.state.{account}.{cluster}.server every 30s. - ClusterReconciler subscribes to that subject, gates CREATING -> ACTIVE on a healthy report, and records Ready node_count into ClusterMeta. - node_count counts Ready nodes only; k3s retains terminated workers as NotReady until pruned, so a raw count never drops on scale-down.	2026-06-06 00:52:18 +10:00
Josh Siviour	f9c08130ba	fix(eks): embedded etcd datastore + Alpine IMDS-route fix + CP-VM visibility (#299) ... * fix: switch EKS datastore to embedded etcd + fix Alpine cloud-init IMDS route Flip the K3s control-plane datastore from SQLite/kine back to embedded etcd (cluster-init: true, etcd-expose-metrics: true). The earlier abandonment premise — that etcd's per-commit fsync stalls past the apiserver's 5s deadline on the viperblock-backed root volume — was wrong: etcd inits fine on an unchanged viperblock (apiserver ready 25-30s, clusters reach ACTIVE). The real blocker was a network-config regression, fixed here. Alpine's cloud-init eni renderer crashes (KeyError: 'gateway') on the gateway-less IMDS on-link route, which failed the whole network-config render so the static mgmt NIC never came up (no NATS bootstrap, no /healthz). Fixes: - generateNetworkConfig: add emitIMDSRoute bool; selectNetworkConfigForFamily passes false for Alpine so the eni renderer never sees the route. - Deliver the IMDS route out-of-band via a persistent /etc/local.d script + OpenRC local service. Resolve the VPC NIC via its default route at runtime (ip route show default), not a hardcoded name: the persistent-net rename to vpc0 doesn't apply on the Alpine AMI, so the NIC is eth0 at first boot. - EKS path bakes the same route into buildK3sUserData's own write_files/runcmd; the generic template Alpine block fires only when no #cloud-config user-data, avoiding a duplicate write_files key (yaml.safe_load keeps last, silently dropping a block). - eks-node image: config.yaml.skel adds cluster-init; comments updated. * fix: surface CreateCluster step failures at ERROR + map subnet faults; add auth-mode/delete test cases CreateCluster wrapped its step failures in bare fmt.Errorf that the request surface collapses to an opaque ServerInternal ("contact AWS re:Post"), so the real cause never reached the operator. Add logCreateErr to log each step's precise cause at ERROR before returning, and map a subnet-resolve failure (caller-supplied parameter) to InvalidParameterValue instead of a retryable ServerInternal the SDK pointlessly retries. (mulga-siv-165.4) Tests (mulga-siv-165.15): reject authenticationMode=API_AND_CONFIG_MAP (only the CONFIG_MAP value was covered); DeleteCluster on a nonexistent cluster via the full-deps path returns ResourceNotFound (the shim path short-circuits to ServiceUnavailable before the meta lookup). * fix: hide EKS control-plane VMs from customer DescribeInstances EKS K3s control-plane VMs are owned by the customer account (their ENI lives in the customer VPC for the NLB target), so unlike LB microVMs (owned by the system account) they leaked into customer ec2:DescribeInstances. The spinifex:managed-by tag only filtered the UI Nodes page, not the EC2 API. Add a caller-aware exclusion: skip any ManagedBy-tagged VM unless the caller is root. Preserves operator visibility, matches LB behaviour, and leaves future customer-owned worker nodes (no ManagedBy) visible. * docs: correct stale SQLite datastore comments in eks-node image (now embedded etcd)	2026-06-05 17:55:31 +10:00
Josh Siviour	f14fb64562	feat(eks): unify eks-server + eks-agent into one eks-node AMI (#288) ... * feat(eks): unify eks-server + eks-agent into one eks-node AMI Collapse the two K3s AMIs into a single spinifex-eks-node image whose role (server vs agent) is chosen per-instance at first boot, halving build time, storage, and CVE-rotation surface. - scripts/images/eks-node/: superset manifest + merged setup.sh; all role services baked but only crond + the new eks-node-role selector enabled. - eks-node-role first-boot selector reads SPINIFEX_K3S_ROLE, rc-update adds + starts the matching services (server: k3s + eks-token-webhook + k3s-first-boot publisher; agent: k3s-agent), records /etc/spinifex-eks/role, self-disables. Paths are env-overridable; eks-node-role_test.sh covers all five cases. - mulga-eks-etcd-snapshot.sh early-exits on agent nodes. - k3s_server_vm.go seeds SPINIFEX_K3S_ROLE=server into first-boot.env. - Makefile/dev-install: build-eks-node-image / import-eks-node-image replace the server+agent targets. - Delete scripts/images/eks-server + eks-agent. The launcher resolves the AMI by the spinifex:managed-by=eks tag, so running clusters (which hold the resolved AMI ID) are unaffected. AMI bake + cluster boot are operational integration steps, not yet run. * fix(eks): order eks-node-role after cloud-config/cloud-final The first-boot role selector reads /etc/spinifex-eks/first-boot.env, which cloud-init delivers via write_files in the cloud-config/cloud-final stages. The selector only depended on net+cgroups, so on a fast boot it could start before write_files landed and read a missing role env. Add an after-ordering on cloud-config + cloud-final so the env file is always present first.	2026-06-04 12:28:56 +10:00