mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-19 19:01:42 +00:00
Josh Siviour
a22bc2f716
* feat(eks): nodegroup provisioning + K3s agent join (6d)
CreateNodegroup launches N customer-owned worker EC2 from the unified
eks-node AMI in agent role; each joins the cluster via K3S_URL + K3S_TOKEN
baked into cloud-init (first-boot.env SPINIFEX_K3S_ROLE=agent + agent.env).
Workers are customer-owned and DescribeInstances-visible (no ManagedBy,
no mgmt NIC) via the normal RunInstances path, matching real EKS.
K3S_URL targets the control-plane ENI's in-VPC private IP on :6443 — the
cluster NLB DNS endpoint is not resolvable in-VPC yet; switch to the NLB
endpoint on :443 once authoritative in-VPC DNS lands. K3S_TOKEN is the
decrypted cluster node-token persisted at server bootstrap.
Describe/List/Update(scaling)/Delete round out CRUD. Cluster<->nodegroup
SG ingress rules (apiserver 6443, flannel VXLAN 8472/udp, kubelet 10250,
intra-nodegroup) authorized idempotently. UpdateNodegroupVersion deferred.
* feat: gate EKS cluster health via NATS state reports
The K3s apiserver binds the VPC node-ip, unreachable from the host-side
spinifex daemon, so the reconciler cannot probe /healthz directly. Add a
control-plane state reporter that publishes {healthz, node_count} over the
management NATS bus the daemon already shares.
- eks-node server role runs mulga-eks-state-report as an OpenRC service,
publishing eks.state.{account}.{cluster}.server every 30s.
- ClusterReconciler subscribes to that subject, gates CREATING -> ACTIVE on
a healthy report, and records Ready node_count into ClusterMeta.
- node_count counts Ready nodes only; k3s retains terminated workers as
NotReady until pruned, so a raw count never drops on scale-down.
30 lines
1.1 KiB
Plaintext
30 lines
1.1 KiB
Plaintext
#!/sbin/openrc-run
|
|
# mulga-eks-state-report — server-role service that publishes K3s health +
|
|
# node_count to the management NATS bus every STATE_REPORT_INTERVAL seconds. The
|
|
# spinifex cluster reconciler consumes this to gate CREATING → ACTIVE and to
|
|
# track nodegroup join/scale, since the apiserver (bound to the VPC node-ip) is
|
|
# unreachable from the host. Enabled on the server role by eks-node-role.sh.
|
|
|
|
description="Mulga EKS control-plane state reporter"
|
|
command="/usr/local/sbin/mulga-eks-state-report"
|
|
command_background="yes"
|
|
pidfile="/run/mulga-eks-state-report.pid"
|
|
output_log="/var/log/mulga-eks-state-report.log"
|
|
error_log="/var/log/mulga-eks-state-report.log"
|
|
|
|
# Cadence the report loop sleeps between publishes. 30s is well under the
|
|
# reconciler's 90s stale window, so a missed tick does not flap health.
|
|
export STATE_REPORT_INTERVAL="${STATE_REPORT_INTERVAL:-30}"
|
|
|
|
depend() {
|
|
need net
|
|
after k3s
|
|
}
|
|
|
|
start_pre() {
|
|
if [ ! -f /etc/spinifex-eks/first-boot.env ]; then
|
|
eerror "/etc/spinifex-eks/first-boot.env not present — cloud-init did not seed NATS creds"
|
|
return 1
|
|
fi
|
|
}
|