mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
74b91b5cc5
* feat(eks): node-targetable control-plane VM (HA spread transport prereq)
Groundwork for HA control-plane placement (mulga-siv-231.7.1): make the k3s
control-plane VM placeable on a specific Spinifex host.
- Add a sys.medium instance type (2 vCPU / 4 GB) and boot the control-plane VM
as it instead of t3.medium. A sys.* type keeps the CP VM out of customer
DescribeInstanceTypes and makes the daemon register the node-targeted
system.LaunchInstance.{type}.{nodeID} subject (only sys.* types get it).
- Add K3sServerInput.TargetNodeID and a LaunchSystemInstanceOnNode method on
the launcher: empty/local node launches in-process (unchanged single-CP
path); any other node dispatches over the node-targeted subject, where the
receiving daemon runs the launch and owns the VM.
The spread orchestrator (reserve N hosts, launch per host, verify, rollback,
<3-host fallback) lands separately; CreateCluster still launches locally
(TargetNodeID empty), so behaviour is unchanged.
* feat(eks): HA control-plane spread orchestrator
Place N=3 k3s server VMs across distinct Spinifex hosts via a per-cluster
spread placement group, with all-or-nothing rollback and single-CP fallback.
- placeControlPlane: capacity fan-out (HostScheduler) -> >=3-host gate ->
ensure spread group (system account) -> ReserveSpreadNodes(Min=Max=3) ->
parallel per-node LaunchK3sServerVM(TargetNodeID) -> node-vms placement
verify -> FinalizeSpreadInstances. Any launch/verify/finalize failure
terminates launched VMs, releases the reservation, drops the group.
- <3 schedulable hosts or a lost reservation race falls back to a single
local control-plane VM (prior behaviour).
- ClusterMeta gains ControlPlaneNodes[] + ControlPlaneSpreadGroup; [0] is
mirrored into the scalar ControlPlane* fields for back-compat. DeleteCluster
tears down every CP VM and the spread group.
- Wire PlacementGroup + Scheduler into EKSServiceDeps + orchestration-deps gate.
* fix(eks): size HA control-plane spread on raw host headroom
SchedulableHosts matched the CP type by name in node.status's per-type
capacity list, but that list omits sys.* types (system types are hidden
from customers). The CP VM is sys.medium, so the match never hit and
SchedulableHosts returned 0 on every node — placeControlPlane always fell
back to a single CP, making the >=3-host spread unreachable in production.
System VMs consume host vCPU/memory like any guest, so size the fit against
each node's raw schedulable headroom (Total - Reserved - Alloc) and the
type's footprint instead. Adds instancetypes.SpecForSystemType for the
sys.* vCPU/memory spec and a pure nodeFitsSystemInstance helper.
* feat(eks): k3s server-join role for HA control-plane quorum
Servers 2..N now boot in join mode: server: https://<first-server>:6443 +
a shared cluster token, without cluster-init, so they join the first server's
embedded-etcd quorum instead of each cluster-initing an independent datastore.
- K3sServerInput gains JoinToken (shared cluster secret, set on every server)
and ServerURL (set => join server, no cluster-init, server-join role).
- buildK3sUserData branches cluster-init vs server:, emits token:, and selects
SPINIFEX_K3S_ROLE server vs server-join.
- launchControlPlaneSpread launches nodes[0] first (cluster-init), derives the
join URL from its ENI IP, then launches the rest in parallel as join servers;
a first-server failure skips the joins and rolls back.
- CreateCluster mints a per-cluster crypto-random token (GenerateK3sClusterToken)
threaded into the placement template.
- eks-node-role.sh adds a server-join case: token-webhook + k3s + state-report,
but not k3s-first-boot (the first server already publishes the bootstrap
artifacts; a join re-publish only races the bootstrap bus).
* fix: open etcd-peer/kubelet ingress on EKS control-plane SG for HA join
Join servers (231.7.2) register over the supervisor port 6443 but their
embedded etcd could not peer with the quorum: the control-plane SG only
admitted 6443 from the VPC CIDR, omitting etcd client+peer 2379-2380 and
kubelet 10250 between control-plane nodes. A single cluster-init server never
exercises those ports, so the gap stayed hidden until multi-server HA — the
join registered but its etcd never replicated the raft log, leaving the node
perpetually NotReady.
Add EnsureControlPlaneHAIngress with self-referencing (cpSG->cpSG) rules for
etcd 2379-2380 and kubelet 10250, called from CreateCluster after
EnsureControlPlaneIngress. Verified live on a 3-node cluster: join servers
reach Ready and form a 3-member etcd quorum that survives loss of the original
first server.
* feat: register all control-plane ENI IPs as EKS cluster NLB targets
Register every HA control-plane server ENI IP in the cluster NLB target
group, not just the primary. The TG already health-checks :6443, so a
dead member drains and kubectl/apiserver traffic fails over to surviving
etcd-quorum members. Add RegisterClusterTargets batch helper; the single
RegisterClusterTarget now delegates to it.