mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
f9c08130ba
* fix: switch EKS datastore to embedded etcd + fix Alpine cloud-init IMDS route
Flip the K3s control-plane datastore from SQLite/kine back to embedded etcd
(cluster-init: true, etcd-expose-metrics: true). The earlier abandonment
premise — that etcd's per-commit fsync stalls past the apiserver's 5s deadline
on the viperblock-backed root volume — was wrong: etcd inits fine on an
unchanged viperblock (apiserver ready 25-30s, clusters reach ACTIVE). The real
blocker was a network-config regression, fixed here.
Alpine's cloud-init eni renderer crashes (KeyError: 'gateway') on the
gateway-less IMDS on-link route, which failed the whole network-config render
so the static mgmt NIC never came up (no NATS bootstrap, no /healthz). Fixes:
- generateNetworkConfig: add emitIMDSRoute bool; selectNetworkConfigForFamily
passes false for Alpine so the eni renderer never sees the route.
- Deliver the IMDS route out-of-band via a persistent /etc/local.d script +
OpenRC local service. Resolve the VPC NIC via its default route at runtime
(ip route show default), not a hardcoded name: the persistent-net rename to
vpc0 doesn't apply on the Alpine AMI, so the NIC is eth0 at first boot.
- EKS path bakes the same route into buildK3sUserData's own write_files/runcmd;
the generic template Alpine block fires only when no #cloud-config user-data,
avoiding a duplicate write_files key (yaml.safe_load keeps last, silently
dropping a block).
- eks-node image: config.yaml.skel adds cluster-init; comments updated.
* fix: surface CreateCluster step failures at ERROR + map subnet faults; add auth-mode/delete test cases
CreateCluster wrapped its step failures in bare fmt.Errorf that the request
surface collapses to an opaque ServerInternal ("contact AWS re:Post"), so the
real cause never reached the operator. Add logCreateErr to log each step's
precise cause at ERROR before returning, and map a subnet-resolve failure
(caller-supplied parameter) to InvalidParameterValue instead of a retryable
ServerInternal the SDK pointlessly retries. (mulga-siv-165.4)
Tests (mulga-siv-165.15): reject authenticationMode=API_AND_CONFIG_MAP (only
the CONFIG_MAP value was covered); DeleteCluster on a nonexistent cluster via
the full-deps path returns ResourceNotFound (the shim path short-circuits to
ServiceUnavailable before the meta lookup).
* fix: hide EKS control-plane VMs from customer DescribeInstances
EKS K3s control-plane VMs are owned by the customer account (their ENI
lives in the customer VPC for the NLB target), so unlike LB microVMs
(owned by the system account) they leaked into customer
ec2:DescribeInstances. The spinifex:managed-by tag only filtered the UI
Nodes page, not the EC2 API.
Add a caller-aware exclusion: skip any ManagedBy-tagged VM unless the
caller is root. Preserves operator visibility, matches LB behaviour, and
leaves future customer-owned worker nodes (no ManagedBy) visible.
* docs: correct stale SQLite datastore comments in eks-node image (now embedded etcd)
48 lines
1.8 KiB
Plaintext
Executable File
48 lines
1.8 KiB
Plaintext
Executable File
#!/sbin/openrc-run
|
|
# K3s server control plane. The per-cluster config.yaml sets cluster-init, so
|
|
# k3s uses the embedded etcd datastore (required for multi-server HA). Data
|
|
# lives at /var/lib/rancher/k3s; subsequent starts reuse it.
|
|
|
|
description="K3s Kubernetes server (Mulga EKS control plane)"
|
|
command="/usr/local/bin/k3s"
|
|
command_args="server --config /etc/rancher/k3s/config.yaml"
|
|
command_background="yes"
|
|
pidfile="/run/k3s.pid"
|
|
output_log="/var/log/k3s.log"
|
|
error_log="/var/log/k3s.log"
|
|
|
|
# Generous start timeout — K3s bootstraps containerd, initialises the embedded
|
|
# etcd datastore, pulls system images. First boot can take 2-3 minutes on cold cache.
|
|
start_stop_daemon_args="--wait 5000"
|
|
|
|
depend() {
|
|
need net cgroups
|
|
after firewall
|
|
}
|
|
|
|
start_pre() {
|
|
# config.yaml is written by k3s-first-boot from the skeleton + cloud-init
|
|
# data. Refuse to start if neither exists.
|
|
if [ ! -f /etc/rancher/k3s/config.yaml ]; then
|
|
if [ -f /etc/rancher/k3s/config.yaml.skel ]; then
|
|
cp /etc/rancher/k3s/config.yaml.skel /etc/rancher/k3s/config.yaml
|
|
else
|
|
eerror "no /etc/rancher/k3s/config.yaml and no .skel to derive from"
|
|
return 1
|
|
fi
|
|
fi
|
|
|
|
# The apiserver is configured with --authentication-token-webhook-config-file
|
|
# pointing at the kubeconfig eks-token-webhook writes at startup. `before k3s`
|
|
# orders the webhook's start but it backgrounds, so wait (bounded) for the
|
|
# kubeconfig to land before letting the apiserver load a missing file.
|
|
i=0
|
|
while [ ! -f /etc/spinifex-eks/token-webhook.kubeconfig ] && [ "${i}" -lt 30 ]; do
|
|
sleep 1
|
|
i=$((i + 1))
|
|
done
|
|
if [ ! -f /etc/spinifex-eks/token-webhook.kubeconfig ]; then
|
|
ewarn "eks-token-webhook kubeconfig absent after ${i}s; IAM token auth will be unavailable until the webhook starts"
|
|
fi
|
|
}
|