Merge remote-tracking branch 'origin/feat/reboot-relaunch' into e2e/integration

fix(vm): reboot by relaunching QEMU after a clean guest shutdown
RebootInstances powered the guest down under -action shutdown=pause and then ran system_reset + cont in the same QEMU process. On some hosts the reset left the guest silent in firmware with QMP still answering, while a fresh QEMU on the same volumes always booted. A reboot now shuts the guest down over ACPI (forcing it off after 60s, as before), ends that QEMU, and starts a fresh one on the same mounted volumes, taps, IMDS datapath and identity, the way nova's soft reboot and Proxmox's qm reboot do. The instance stays running in Describe, no hooks fire, the console log is appended rather than truncated, and the existing QMP client is reconnected in place so the heartbeat carries on. A relaunched guest that writes nothing to its console within 60s has its run state, vCPUs, vCPU thread states and registers (sampled twice) logged, and if it has printed before, its QEMU is replaced once more. A relaunch that cannot start goes to crash recovery. Stop, terminate and hot-plug wait for a relaunch in flight so volumes are never sealed under a process being started. Guest-initiated reboots stay an in-process reset. The daemon now logs the host kernel, QEMU version and the OVMF package and checksum at start, and every reboot relaunch carries them.
2026-11-01 08:49:44 +00:00 · 2026-11-01 14:11:32 +10:00 · 2026-11-01 14:08:55 +10:00 · 2026-11-01 13:47:38 +10:00 · 2026-11-01 13:46:54 +10:00 · 2026-11-01 13:41:13 +10:00
185 changed files with 6863 additions and 2222 deletions
@@ -8,7 +8,8 @@
    "arch": "x86_64",
    "runner": "ci-single",
    "extra": "",
-    "suite": ""
+    "suite": "",
+    "timeout_minutes": 60
  },
  {
    "cell": 2,
@@ -19,7 +20,8 @@
    "arch": "x86_64",
    "runner": "ci-single",
    "extra": "",
-    "suite": ""
+    "suite": "",
+    "timeout_minutes": 60
  },
  {
    "cell": 3,
@@ -30,7 +32,8 @@
    "arch": "x86_64",
    "runner": "ci-single",
    "extra": "",
-    "suite": ""
+    "suite": "",
+    "timeout_minutes": 60
  },
  {
    "cell": 8,
@@ -200,7 +203,8 @@
    "arch": "x86_64",
    "runner": "ci-single",
    "extra": "",
-    "suite": ""
+    "suite": "",
+    "timeout_minutes": 60
  },
  {
    "cell": 35,
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Usage: hypervisor-disk-preflight.sh <env.tfvars>
+#
+# Fails fast when a hypervisor named in the tofu env is short of disk, before
+# `tofu apply` tries to define a domain there. libvirt writes each domain's XML
+# under /etc/libvirt/qemu on the root filesystem and its disks into the pool, and
+# a full one fails the apply minutes in as "Domain Creation Failed".
+#
+# Hosts are reached the way the libvirt provider reaches them (ssh <user>@<name>,
+# resolved through ~/.ssh/config). A host that cannot be reached is reported and
+# skipped: tofu's own error for it is clearer than anything said here.
+#
+# Env: HV_SSH_USER (tf-user), HV_MIN_ROOT_FREE_MB (1024), HV_MIN_POOL_FREE_GB (20).
+set -uo pipefail
+
+tfvars="${1:?usage: $0 <env.tfvars>}"
+user="${HV_SSH_USER:-tf-user}"
+min_root_kb=$(( ${HV_MIN_ROOT_FREE_MB:-1024} * 1024 ))
+min_pool_kb=$(( ${HV_MIN_POOL_FREE_GB:-20} * 1024 * 1024 ))
+
+if [ ! -r "$tfvars" ]; then
+  echo "::error::hypervisor disk preflight: cannot read $tfvars" >&2
+  exit 1
+fi
+
+# One node per line in every env file: { name = "...", ..., storage_pool = "..." }.
+pairs=$(grep -v '^[[:space:]]*#' "$tfvars" |
+  sed -nE 's/.*name[[:space:]]*=[[:space:]]*"([^"]+)".*storage_pool[[:space:]]*=[[:space:]]*"([^"]+)".*/\1 \2/p')
+if [ -z "$pairs" ]; then
+  echo "::warning::hypervisor disk preflight: no nodes with a storage_pool in $tfvars; nothing checked"
+  exit 0
+fi
+
+# Prints "<root-avail-kb> <pool-avail-kb> <pool-path>" for the pool named in $1.
+# shellcheck disable=SC2016 # expanded on the hypervisor, not here
+remote='set -u
+root=$(df -Pk /etc/libvirt/qemu | awk "NR==2{print \$4}")
+path=$(virsh -c qemu:///system pool-dumpxml "$1" 2>/dev/null | sed -n "s:.*<path>\(.*\)</path>.*:\1:p" | head -n1)
+pool=-
+[ -n "$path" ] && pool=$(df -Pk "$path" | awk "NR==2{print \$4}")
+echo "$root $pool ${path:--}"'
+
+fail=0
+while read -r host pool; do
+  [ -n "$host" ] || continue
+  if ! out=$(ssh -o BatchMode=yes -o ConnectTimeout=10 "${user}@${host}" bash -s -- "$pool" <<<"$remote" 2>&1); then
+    echo "::warning::hypervisor disk preflight: cannot reach $host ($out); its disk is unchecked"
+    continue
+  fi
+  read -r root_kb pool_kb pool_path <<<"$(tail -n1 <<<"$out")"
+  if ! [[ "$root_kb" =~ ^[0-9]+$ ]]; then
+    echo "::warning::hypervisor disk preflight: unexpected answer from $host: $out"
+    continue
+  fi
+  echo "$host: root fs $((root_kb / 1024)) MiB free; pool $pool ($pool_path) $([[ "$pool_kb" =~ ^[0-9]+$ ]] && echo "$((pool_kb / 1024 / 1024)) GiB free" || echo "not found")"
+  if [ "$root_kb" -lt "$min_root_kb" ]; then
+    echo "::error::hypervisor $host has $((root_kb / 1024)) MiB free on the filesystem holding /etc/libvirt/qemu (need ${HV_MIN_ROOT_FREE_MB:-1024} MiB): libvirt cannot write a domain definition there. Free space on $host before re-running." >&2
+    fail=1
+  fi
+  if [[ "$pool_kb" =~ ^[0-9]+$ ]] && [ "$pool_kb" -lt "$min_pool_kb" ]; then
+    echo "::error::hypervisor $host has $((pool_kb / 1024 / 1024)) GiB free in storage pool $pool at $pool_path (need ${HV_MIN_POOL_FREE_GB:-20} GiB): the node's disk cannot grow. Free space on $host before re-running." >&2
+    fail=1
+  fi
+done <<<"$pairs"
+
+exit "$fail"
@@ -0,0 +1,87 @@
+#!/bin/sh
+# hypervisor-disk-preflight_test.sh — drives hypervisor-disk-preflight.sh against
+# a stub ssh that answers for each hypervisor from a canned file, so every
+# verdict is checked without a real host.
+set -eu
+
+SCRIPT_DIR=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd)
+PREFLIGHT="$SCRIPT_DIR/hypervisor-disk-preflight.sh"
+TMP=$(mktemp -d)
+trap 'rm -rf "$TMP"' EXIT
+
+fails=0
+pass() { echo "  ok   - $1"; }
+fail() { echo "  FAIL - $1"; fails=$((fails + 1)); }
+
+# The stub answers "<root-kb> <pool-kb> <path>" from $FAKE/<host>, or exits 255
+# when that host has no file, the way ssh does for a host it cannot reach.
+mkdir -p "$TMP/bin" "$TMP/hosts"
+cat > "$TMP/bin/ssh" <<'STUB'
+#!/bin/sh
+for a in "$@"; do case $a in *@*) host=${a#*@} ;; esac; done
+cat >/dev/null
+if [ -f "$FAKE/$host" ]; then cat "$FAKE/$host"; exit 0; fi
+echo "ssh: Could not resolve hostname $host" >&2
+exit 255
+STUB
+chmod +x "$TMP/bin/ssh"
+PATH="$TMP/bin:$PATH"
+FAKE="$TMP/hosts"
+export PATH FAKE
+
+cat > "$TMP/env.tfvars" <<'TFVARS'
+nodes = [
+  { name = "wattle", wan_ip = "192.0.2.1", storage_pool = "drive1-pool", base_image = "/x.qcow2" },
+  # { name = "retired", wan_ip = "192.0.2.9", storage_pool = "drive1-pool", base_image = "/x.qcow2" },
+  { name = "banksia", wan_ip = "192.0.2.2", storage_pool = "nvme1-pool", base_image = "/x.qcow2" },
+]
+TFVARS
+
+GiB=1048576
+plenty="$((50 * GiB)) $((200 * GiB)) /mnt/drive1/pool"
+
+# run <label> <want-rc> <want-text>
+run() {
+    rc=0
+    out=$(bash "$PREFLIGHT" "$TMP/env.tfvars" 2>&1) || rc=$?
+    if [ "$rc" -ne "$2" ]; then
+        fail "$1: exit $rc, want $2"
+        echo "$out" | sed 's/^/        /'
+        return
+    fi
+    case $out in
+        *"$3"*) pass "$1" ;;
+        *) fail "$1: output lacks '$3'"; echo "$out" | sed 's/^/        /' ;;
+    esac
+}
+
+echo "-- hypervisor-disk-preflight.sh"
+
+echo "$plenty" > "$FAKE/wattle"
+echo "$plenty" > "$FAKE/banksia"
+run "every host has room" 0 "banksia: root fs 51200 MiB free"
+
+echo "0 $((200 * GiB)) /mnt/drive1/pool" > "$FAKE/wattle"
+run "a full root filesystem fails fast" 1 "::error::hypervisor wattle has 0 MiB free on the filesystem holding /etc/libvirt/qemu"
+
+echo "$plenty" > "$FAKE/wattle"
+echo "$((50 * GiB)) $((3 * GiB)) /mnt/nvme1/pool" > "$FAKE/banksia"
+run "a nearly full pool fails fast" 1 "::error::hypervisor banksia has 3 GiB free in storage pool nvme1-pool"
+
+echo "$plenty" > "$FAKE/banksia"
+rm -f "$FAKE/wattle"
+run "an unreachable host is reported, not failed" 0 "::warning::hypervisor disk preflight: cannot reach wattle"
+
+echo "$plenty" > "$FAKE/wattle"
+echo "$((50 * GiB)) - -" > "$FAKE/banksia"
+run "a pool libvirt does not know is reported, not failed" 0 "pool nvme1-pool (-) not found"
+
+echo "0 0 /x" > "$FAKE/retired"
+echo "$plenty" > "$FAKE/banksia"
+run "a commented-out node is never checked" 0 "wattle: root fs"
+
+if [ "$fails" -ne 0 ]; then
+    echo "$fails failure(s)"
+    exit 1
+fi
+echo "  hypervisor-disk-preflight ok"
@@ -130,6 +130,10 @@ e2e_suite_timeout() {
    # budget on a node stopping and starting: seven units down, a raft election,
    # and the whole service stack back up with every stream caught up.
    kvquorum) echo "40m" ;;
+    # single is the broadest suite and finishes, it does not hang: passing runs
+    # took 24-29m, and an Oracle host ran out at 30m inside the last test's
+    # cleanup. The default left it no headroom at all.
+    single) echo "40m" ;;
    *) echo "30m" ;;
  esac
 }
@@ -184,6 +184,14 @@ jobs:
            echo "running qemu: $(pgrep -c qemu-system-x86 || echo 0)"
          } >> "${ARTIFACT_DIR}/hypervisor-load.txt"

+      # A hypervisor out of disk fails the apply minutes in, as "Domain Creation
+      # Failed" from libvirt writing the domain XML. Say so before starting.
+      - name: Preflight hypervisor disk
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          "${GITHUB_WORKSPACE}/spinifex/.github/scripts/hypervisor-disk-preflight.sh" \
+            "envs/${SPINIFEX_CI_ENV}.tfvars"
+
      - name: Provision VMs
        working-directory: mulga/scripts/tofu-cluster
        env:
@@ -241,6 +249,11 @@ jobs:
          . "${GITHUB_WORKSPACE}/spinifex/.github/scripts/suite-sets.sh"
          e2e_suites_assert_subset "e2e-nightly single" "${E2E_SUITES_NIGHTLY_SINGLE}" "${E2E_SUITES_SINGLE}"

+          # The select job sized this cell's budget from these same timeouts, so
+          # resolve them here and carry the pairs into the remote loop.
+          SUITE_TMOS=""
+          for s in ${E2E_SUITES_NIGHTLY_SINGLE}; do SUITE_TMOS="${SUITE_TMOS}${s}:$(e2e_suite_timeout "${s}") "; done
+
          INVENTORY=$(tofu output -json inventory)
          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
@@ -250,9 +263,10 @@ jobs:
             for SUITE in ${E2E_SUITES_NIGHTLY_SINGLE}; do \
               echo \"::group::suite \$SUITE\"; \
               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               TMO=30m; for kv in ${SUITE_TMOS}; do case \$kv in \$SUITE:*) TMO=\${kv#*:} ;; esac; done; \
               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=single \
                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
-                 ./\$SUITE.test -test.v -test.timeout=30m \
+                 ./\$SUITE.test -test.v -test.timeout=\$TMO \
                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log'; \
               rc=\${PIPESTATUS[0]}; echo \"::endgroup::\"; \
               if [ \$rc -ne 0 ]; then echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; FAIL=1; fi; \
@@ -324,16 +338,20 @@ jobs:
            NODES_CSV="${NODES_CSV:+$NODES_CSV,}$ip"
          done

+          SUITE_TMOS=""
+          for s in ${E2E_SUITES_NIGHTLY_MULTI}; do SUITE_TMOS="${SUITE_TMOS}${s}:$(e2e_suite_timeout "${s}") "; done
+
          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${NODE1_IP} \
            "set -u; mkdir -p '${ARTIFACT_DIR}'; cd \$HOME/e2e-tests/_bin; FAIL=0; \
             for SUITE in ${E2E_SUITES_NIGHTLY_MULTI}; do \
               echo \"::group::suite \$SUITE\"; \
               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               TMO=30m; for kv in ${SUITE_TMOS}; do case \$kv in \$SUITE:*) TMO=\${kv#*:} ;; esac; done; \
               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=multinode \
                 SPINIFEX_NODES='${NODES_CSV}' SPINIFEX_NODE_IPS='${NODES_CSV}' \
                 SPINIFEX_SSH_USER=tf-user SPINIFEX_SSH_KEY=\$HOME/.ssh/tf-user-ap-southeast-2 \
                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
-                 ./\$SUITE.test -test.v -test.timeout=30m \
+                 ./\$SUITE.test -test.v -test.timeout=\$TMO \
                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log'; \
               rc=\${PIPESTATUS[0]}; echo \"::endgroup::\"; \
               if [ \$rc -ne 0 ]; then echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; FAIL=1; fi; \
@@ -247,6 +247,12 @@ test-build-scripts:
 		echo "-- $$t"; \
 		sh "$$t" || exit 1; \
 	done
+	@echo -e "\n....Running .github/scripts/*_test.sh...."
+	@for t in $$(find .github/scripts -maxdepth 1 -name '*_test.sh' | sort); do \
+		echo "-- $$t"; \
+		sh "$$t" || exit 1; \
+	done
+	shellcheck -S warning .github/scripts/hypervisor-disk-preflight.sh .github/scripts/hypervisor-disk-preflight_test.sh
 	@echo -e "\n....Running shellcheck over build/scripts/**/*.sh + the node scripts...."
 	shellcheck -S warning $$(find build/scripts -name '*.sh' | sort) $(NODE_SCRIPTS)
 	shellcheck -S warning -x scripts/build-oracle-appliance.sh $$(find scripts/appliance -name '*.sh' | sort)
@@ -104,7 +104,6 @@ var allowlist = []allowance{

 	{From: "domains/sts/awsapi", To: "domains/iam", Debt: "IAM implementation consumers in STS, Ochre and ELBv2", Reason: "GetCallerIdentity takes IAMService; needs an IAM capability"},

-	{From: "gateway", To: "domains/sts/awsapi", Debt: "gateway (service registration)", Reason: "IAM, STS, EKS and RDS dispatch stays in gateway until the ingress registration seam"},
 	{From: "domains/ecs", To: "domains/ec2/ebs/metadata", Debt: "close-out: handlers/ecs (ADR-0004 blocked-dependency list; inventory-ecs boundary imports)", Reason: "legacy ECS calls another domain's implementation as an internal API"},
 	{From: "domains/ecs", To: "domains/ec2/eip", Debt: "close-out: handlers/ecs (ADR-0004 blocked-dependency list; inventory-ecs boundary imports)", Reason: "legacy ECS calls another domain's implementation as an internal API"},
 	{From: "domains/ecs", To: "domains/ec2/instancetypes", Debt: "close-out: handlers/ecs (ADR-0004 blocked-dependency list; inventory-ecs boundary imports)", Reason: "legacy ECS calls another domain's implementation as an internal API"},
@@ -1,11 +1,10 @@
-# spinifex-unit-version: 1
+# spinifex-unit-version: 2
 [Unit]
 Description=Spinifex host firewall (nftables input policy)
 Documentation=file:/etc/spinifex/firewall/spinifex.nft
 Wants=network-pre.target
 Before=network-pre.target shutdown.target
 DefaultDependencies=no
-ConditionPathExists=/etc/spinifex/firewall/peers.nft

 [Service]
 Type=oneshot
@@ -13,8 +12,11 @@ RemainAfterExit=yes
 StandardInput=null
 ProtectSystem=full
 ProtectHome=true
-ExecStart=/usr/local/lib/spinifex/spinifex-firewall-apply
-ExecReload=/usr/local/lib/spinifex/spinifex-firewall-apply
+# Unconditional: connection tracking loads on every node, so a flow that opens
+# while the policy is not armed survives the arm. The policy itself still loads
+# only where the daemon has written a peer file.
+ExecStart=/usr/local/lib/spinifex/spinifex-firewall-apply boot
+ExecReload=/usr/local/lib/spinifex/spinifex-firewall-apply boot
 TimeoutStopSec=10s

 # No ExecStop: flushing on stop would take vpcd's MASQUERADE and per-EIP FORWARD
@@ -0,0 +1,50 @@
+// eks-etcd-member removes a dead peer from the embedded etcd cluster of an EKS
+// K3s control-plane VM. A member that is already gone exits 0; any other
+// failure exits non-zero.
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"io"
+	"os"
+	"time"
+
+	_ "github.com/mulgadc/bluebottle/pkg/fipsboot"
+
+	"github.com/mulgadc/spinifex/spinifex/agents/eks/etcdmember"
+)
+
+func main() {
+	os.Exit(run(os.Args[1:], os.Stdout, os.Stderr))
+}
+
+func run(args []string, stdout, stderr io.Writer) int {
+	if len(args) < 1 || args[0] != "remove" {
+		fmt.Fprintln(stderr, "usage: eks-etcd-member remove --peer-ip 10.0.0.12 [--self-ip 10.0.0.10]")
+		return 2
+	}
+	fs := flag.NewFlagSet("remove", flag.ContinueOnError)
+	fs.SetOutput(stderr)
+	var cfg etcdmember.Config
+	endpoint := fs.String("endpoint", etcdmember.DefaultEndpoint, "etcd client endpoint")
+	certDir := fs.String("cert-dir", etcdmember.DefaultCertDir, "directory holding the K3s etcd client.crt, client.key and server-ca.crt")
+	timeout := fs.Duration("timeout", 30*time.Second, "overall deadline")
+	fs.StringVar(&cfg.PeerIP, "peer-ip", "", "peer URL host of the member to remove")
+	fs.StringVar(&cfg.SelfIP, "self-ip", "", "this host's own address; removal of it is refused")
+	if err := fs.Parse(args[1:]); err != nil {
+		return 2
+	}
+
+	ctx, cancel := context.WithTimeout(context.Background(), *timeout)
+	defer cancel()
+
+	cli, err := etcdmember.Dial(*endpoint, *certDir, *timeout)
+	if err != nil {
+		fmt.Fprintln(stderr, "etcd member remove failed:", err)
+		return 1
+	}
+	defer func() { _ = cli.Close() }()
+	return etcdmember.Run(ctx, cli, cfg, stdout)
+}
@@ -0,0 +1,63 @@
+package main
+
+import (
+	"bytes"
+	"testing"
+
+	"github.com/mulgadc/spinifex/internal/testkit/fakeetcd"
+	"github.com/stretchr/testify/assert"
+)
+
+var fakeMembers = []fakeetcd.Member{
+	{ID: 0xa1, Name: "cp-a", PeerURLs: []string{"https://10.0.0.10:2380"}},
+	{ID: 0xb2, Name: "cp-b", PeerURLs: []string{"https://10.0.0.12:2380"}},
+}
+
+func TestRunRequiresTheRemoveSubcommand(t *testing.T) {
+	t.Parallel()
+	for _, args := range [][]string{nil, {"list"}} {
+		var out, errOut bytes.Buffer
+		assert.Equal(t, 2, run(args, &out, &errOut))
+		assert.Contains(t, errOut.String(), "usage: eks-etcd-member remove")
+	}
+}
+
+func TestRunRejectsUnknownFlags(t *testing.T) {
+	t.Parallel()
+	var out, errOut bytes.Buffer
+	assert.Equal(t, 2, run([]string{"remove", "--bogus"}, &out, &errOut))
+}
+
+func TestRunFailsWhenEtcdCertificatesAreMissing(t *testing.T) {
+	t.Parallel()
+	var out, errOut bytes.Buffer
+	code := run([]string{"remove", "--peer-ip", "10.0.0.12", "--cert-dir", t.TempDir()}, &out, &errOut)
+	assert.Equal(t, 1, code)
+	assert.Contains(t, errOut.String(), "etcd member remove failed")
+}
+
+func TestRunRemovesTheNamedPeer(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, fakeMembers)
+	var out, errOut bytes.Buffer
+
+	code := run([]string{"remove", "--peer-ip", "10.0.0.12", "--self-ip", "10.0.0.10",
+		"--endpoint", srv.Endpoint, "--cert-dir", srv.CertDir}, &out, &errOut)
+
+	assert.Zero(t, code, errOut.String()+out.String())
+	assert.Contains(t, out.String(), "removed etcd member 10.0.0.12")
+	assert.Equal(t, []uint64{0xb2}, srv.Removed())
+}
+
+func TestRunTreatsAnAbsentPeerAsSuccess(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, fakeMembers)
+	var out, errOut bytes.Buffer
+
+	code := run([]string{"remove", "--peer-ip", "10.0.0.99",
+		"--endpoint", srv.Endpoint, "--cert-dir", srv.CertDir}, &out, &errOut)
+
+	assert.Zero(t, code)
+	assert.Contains(t, out.String(), "member already absent")
+	assert.Empty(t, srv.Removed())
+}
@@ -60,6 +60,10 @@ moved to another node; the local daemon subscribes to stop the guest that
 was using it. Node-addressed, like mount and unmount: the guest is local, and
 no other node's daemon has anything to stop.

+The event is sent as a request and repeated until the daemon replies with `VolumeFencedAck`, because the fence often runs while the node's NATS server is restarting and a single publish can reach no subscriber.
+The daemon replies only when the event carries a reply subject, so a plain publish is still accepted.
+Repeats are harmless, since fencing a guest that is already fenced changes nothing.
+
 ## Abandon route

 `VolumeAbandonSubject(node)` is `ebs.<node>.abandon`. The daemon uses
@@ -19,3 +19,11 @@ type VolumeFencedEvent struct {
 	Winner string `json:"winner"`
 	Reason string `json:"reason"`
 }
+
+// VolumeFencedAck is the daemon's answer to a VolumeFencedEvent. Any answer
+// ends the sender's retries; Instance names the guest stopped, if one was, and
+// Error says why the daemon could not act on the event.
+type VolumeFencedAck struct {
+	Instance string `json:"instance,omitempty"`
+	Error    string `json:"error,omitempty"`
+}
@@ -37,3 +37,18 @@ func TestVolumeFencedEvent_JSON(t *testing.T) {
 	require.NoError(t, json.Unmarshal([]byte(golden), &decoded))
 	assert.Equal(t, event, decoded)
 }
+
+// TestVolumeFencedAck_JSON pins the daemon's answer, which is what ends
+// viperblockd's retries of a fence notice.
+func TestVolumeFencedAck_JSON(t *testing.T) {
+	golden := `{"instance":"i-1","error":"bad event"}`
+	ack := viperblocklegacyv1.VolumeFencedAck{Instance: "i-1", Error: "bad event"}
+
+	marshaled, err := json.Marshal(ack)
+	require.NoError(t, err)
+	assert.JSONEq(t, golden, string(marshaled))
+
+	empty, err := json.Marshal(viperblocklegacyv1.VolumeFencedAck{})
+	require.NoError(t, err)
+	assert.JSONEq(t, `{}`, string(empty), "an ack for a volume with no local guest carries nothing")
+}
@@ -26,7 +26,7 @@ Every service is `PartOf=spinifex.target` and ordered by its own `After=`/`Wants
 | `spinifex-qmp-collector.service` | `spx service qmp-collector start` | Reads the `contracts/telemetry/v1` per-VM sidecar, polls telemetry QMP sockets and tap counters, publishes to NATS `metrics.ec2.*` |
 | `spinifex-shutdown.service` | `/bin/true`; the work is its `ExecStop` | Runs `spx admin node drain --local --unless-restarting`. `After=` every service, so it stops first and drains guests while the stack is still up |
 | `spinifex-nats-watchdog.service` / `.timer` | `nats-js-watchdog.sh` | JetStream liveness probe every 30s. `try-restart`s NATS when JetStream stops accepting writes while the process stays alive — what `Restart=on-failure` cannot catch |
-| `spinifex-firewall.service` | `spinifex-firewall-apply` | Oneshot nftables input policy, `WantedBy=sysinit.target` rather than the target. No `ExecStop`: remove it with `nft delete table inet spinifex_filter` |
+| `spinifex-firewall.service` | `spinifex-firewall-apply boot` | Oneshot nftables input policy, `WantedBy=sysinit.target` rather than the target. Loads the `inet spinifex_conntrack` tracking table on every node, and the policy only once the daemon has written a peer file. No `ExecStop`: remove the policy with `spinifex-firewall-apply disable` |
 | `regenerate-ssh-host-keys.service` | `ssh-keygen -A` | Oneshot for AMI-built nodes, conditional on the host key being absent |

 #### Operations
@@ -368,6 +368,7 @@ while the branch is in flight; they do not describe the umbrella repository's
 | `spinifex#1354` | `spinifex/gateway/ec2.go` (the EC2 action table, `ec2Handler`, `ec2Actions`, the dispatch pipeline), `ec2_idempotency.go`, `ec2_validation.go`, and the EC2 token-bucket fields of `GatewayConfig` | `spinifex/domains/ec2/awsapi` (`Handler`, `NewHandler`, `Deps`, `Quota`, `Dispatch`, `ActionNames`, `InputHasField`), composed by `spinifex/gateway/ec2.go` | Slice 9a of the EC2 move: the capabilities first, with no behaviour change. The EC2 action table, idempotency and validation move to `domains/ec2/awsapi` behind `Deps{NATS, IAM, Quota, ActiveNodes, ExpectedNodes, NodeIDs, AZ, Endpoint, InstanceStatus}`; the gateway keeps the composition and `EC2_Request`, still on the legacy switch until slice 9b registers EC2 through the ADR-0002 seam. `Quota` is declared in `awsapi` because `domains/admission/quota` imports the EC2 action packages and cannot be imported back, and the gateway wraps `quota.Service` to supply the NATS instance-type resolver its retype gate takes. The instance and spot-instance adapters take the two-method `InstanceProfiles` interface (`ResolveInstanceProfile`, `CanonicalResourceARN`) instead of `iamdomain.IAMService`, which pays off the `spotinstance` to `domains/admission/quota` and `domains/iam` allowlist rows. Allowlist changes are 20 deletions and no additions: the 18 `gateway` to `domains/ec2/awsapi/*` rows and those 2. The pipeline characterisation tests committed first (error envelopes, check order, gateway-local actions, idempotent replay, PassRole, ImportKeyPair padding, launch-template expansion before authorization, RunInstances instance-profile refusal order) pass unchanged. Package-local tests drive every routed action through `Dispatch` against a stub daemon. No change to `dispatch.Actor` or `dispatch.Invocation`. Live on env14: DescribeRegions, DescribeAvailabilityZones and DescribeAccountAttributes, an idempotent CreateVolume replay and its parameter mismatch, the DryRun and MaxResults errors, a no-policy user refused with UnauthorizedOperation (including on the gateway-local DescribeRegions), an assumed-role session allowed and refused per its policy, and one RunInstances to running then terminated. |
 | `spinifex#1355` | `spinifex/gateway/ec2.go` (`EC2_Request` in production, the legacy service switch, its `supportedServices` and JSON tables) and the `Quota` field of `GatewayConfig` | `spinifex/domains/ec2/awsapi` (`NewRegistration`, `OperationInventory`, `ServiceName`), composed by `spinifex/runtime/roles/awsgw` (`ec2.go`: `newEC2Registration`, `advertisedEndpoint`; `ec2quota`) | Slice 9b of the EC2 move: EC2 registers through the ADR-0002 seam as service `ec2` and the legacy gateway switch is empty. The EC2 query-protocol error envelope is kept through `ErrorEnvelopeXML`, and the policy denial match is now `*dispatch.PolicyDenialError`. The characterisation tests committed first also pin the Authorize service name and action, the trace span and audit action, the throttle key (account plus action) and the cluster-unavailable gate, which runs before policy and action lookup for gateway-local and NATS-backed actions alike. They and the 9a tests pass unchanged apart from setup: STS and EC2 test setups now register the services they sign as, and the two tests that claimed a legacy service name are removed because no legacy name remains. `EC2_Request` survives only as a test shim. Allowlist changes are two deletions (the `gateway` to `domains/ec2/awsapi` and `domains/ec2/awsapi/instance` rows) and one addition, the per-service model-coverage row `cmd/aws-model-coverage` to `domains/ec2/awsapi`. `startInstanceStatus` stays in `awsgw.go` because ADR-0007:S6 records the raw instance-state uses by file. No change to `dispatch.Actor` or `dispatch.Invocation`. Live on env14: DescribeRegions, DescribeAvailabilityZones and DescribeAccountAttributes, an idempotent CreateVolume replay and its parameter mismatch, the DryRun and InvalidVolume.NotFound errors, a no-policy user refused with UnauthorizedOperation (including on the gateway-local DescribeRegions), an assumed-role session allowed and refused per its policy, and one RunInstances to running then terminated. The gateway request logs carry `service=ec2` and the per-action label. Not exercised live: a throttled request (the default limits are not reachable from one client; the throttle key is pinned by the characterisation tests) and an agent-signed EC2 call. |
 | `spinifex#1363` | `spinifex/gateway` context keys (`ctxService`, `ctxRegion`, `ctxAction`, the principal and access key id keys, and the unread `ctxQueryArgs`), the duplicate `callerPrincipal` and `principalContext` types, `gateway/caller_arn.go` (`buildCallerARN`), and the direct `policyauth`, `accountadmission` and NATS-connection reads in the auth pipeline | `spinifex/ingress/aws/reqctx` (the request record), `credverify.CallerARN`, and the gateway-consumed capabilities `PolicyEvaluator`, `AccountAdmission` and `Readiness`, composed by `spinifex/runtime/roles/awsgw` (`clusterready`) | Slice 11a of the gateway removal: the request-context record and the ADR-0002:S8 capabilities, with no behaviour change. The record is a copy-on-write value under one context key; the principal is sealed once and a second `WithPrincipal` keeps the first caller. The layering rule `S7_RequestRecordPrivateToIngress` lets only `gateway` and `ingress/aws/*` import `reqctx`, with a fault-injection test, and slice 11c moves the package to `ingress/aws/internal/reqctx` and drops the rule. The caller ARN is built once in `credverify` (root cases kept) and filled on `Principal.CallerARN`. The policy evaluator, account admission (`credential.ErrAccountNotActive`, with `accountadmission.ErrNotActive` an alias) and cluster readiness (`Available() bool`) are interfaces the gateway consumes and fails closed on when nil. The step 1 characterisation tests (throttle against both readiness gates and the policy gate for ECS and IAM, `/admin` skipping `GetService` and the second gate, the nil-connection difference between the two gates, five error writers each minting a request id that differs from the audit record, and the credential-scope region reaching only the span) pass unchanged apart from setup. No allowlist rows change, and there is no change to `dispatch.Actor` or `dispatch.Invocation`. Debt for slice 11c: the test-only `defaultCapabilities` seam (a package variable set by a test `init` so about 60 gateway test files can build a config from `IAMService` alone), the nil-connection case of the early readiness gate, the remaining `NATSConn` readers (`DiscoverActiveNodes` and the field that `awsgw` and the integration harness set), and the `IAMService`, `STSService`, `Config`, `RootCAs` and `InternalSuffix` fields of `GatewayConfig`, which no production code reads. Live on env14: DescribeRegions, DescribeAvailabilityZones and DescribeAccountAttributes, an idempotent CreateVolume replay and its parameter mismatch, the DryRun and InvalidVolume.NotFound errors, a bad access key refused with InvalidClientTokenId on a Query service and InvalidClientTokenIdException on a JSON service, a no-policy user refused with UnauthorizedOperation, and an assumed-role session allowed and refused per its policy. Not exercised live: a suspended-account refusal, a throttled request and the NATS-down 503, which the step 1 tests pin. |
+| `spinifex#1378` | `spinifex/gateway` `oidc_discovery.go` and `registry.go` (the `/oidc/eks` routes and the `/v2` mount), the `DiscoverActiveNodes` method and its cache, and the `GatewayConfig` fields `Config`, `RootCAs`, `InternalSuffix`, `Version`, `Commit`, `IAMService`, `STSService`, `NATSConn` and `ExpectedNodes` | `spinifex/runtime/roles/awsgw` (`PublicRoutes`, `NodeDiscovery.ActiveNodes`, `composeGateway`); the gateway keeps one `PublicRoutes` mount hook and the `Admin` registration name supplied by the role |
 | `spinifex#1334` | `spinifex/gateway/oidc_discovery.go` (`eksAccountKV`, the discovery-document and JWKS read and construction) | `spinifex/domains/eks/cluster` (`OIDCPublisher`, `OIDCDiscoveryDocument`), composed by `spinifex/runtime/roles/awsgw` | The EKS domain now reads its own account bucket and builds the per-cluster discovery document and JWKS responses (ADR-0005:S4), and the gateway only maps the unauthenticated `/oidc/eks/...` routes onto an `OIDCPublisher` interface it is handed. Responses are byte-identical, pinned by `oidc_discovery_current_behaviour_test.go`, whose assertions are unchanged. The gateway no longer imports `domains/eks/cluster` in production code. `STS` keeps its own `ClusterJWKSResolver` (`OIDCJWKSReader`), untouched. Recorded debt, not repaired here: a stored JWKS is served verbatim with 200 even when it is not valid JSON. |
 The EC2 contract row is intentionally different from the directory moves: it creates
 a compatibility boundary. `ec2.cmd.*` retains its deployed one-token NATS
@@ -155,6 +155,8 @@ Restarting the daemon is what re-arms it: the node rebuilds its peer list from t

 `disable` removes Spinifex's own table and the peer file and leaves every other nftables table alone. It is idempotent, and it is the recovery path — it runs even on a node with no policy installed, so it cannot leave you with a node you can neither arm nor disarm.

+`disable` leaves `table inet spinifex_conntrack` loaded, and the boot-time unit loads it on every node whether or not the policy is armed. It accepts everything and exists only to keep connection tracking running. Without it, a connection opened while the policy is off, such as an OVN RAFT link formed during that window, is first seen mid-stream when the policy comes back, and its replies are dropped as new connections. Do not delete it.
+
 > [!WARNING]
 > Do not flush the whole nftables ruleset to clear the policy. The VPC daemon writes SNAT and per-elastic-IP forward rules into the `ip nat` and `ip filter` tables and reinstalls them only when it starts, so `nft flush ruleset` breaks elastic IPs silently until the next `systemctl restart spinifex-vpcd`. Use `disable`, which only ever touches `inet spinifex_filter`.
 > On Oracle Linux with IPsec on, a flush also removes `inet spinifex_ipsec_guard`, so Geneve to a peer without an SA goes out in clear until the daemon puts the table back, within 60 seconds.
@@ -171,6 +173,7 @@ Everything lives under `/etc/spinifex/firewall/`. Which files are safe to edit f
 | `open-ports.nft` | The apply helper — the formation window only | No. Reset closed on install and upgrade |
 | `custom.nft` | **You** — `trusted_ssh_peers` and anything else you add | **Yes.** Created once, never rewritten |
 | `mode` | `setup.sh` — whether the install path armed the policy | No |
+| `conntrack.nft` | `setup.sh` — the tracking table, loaded armed or not | No. Edits are overwritten |

 The daemon cannot write `/etc` itself, and a blanket `nft` grant would be root-equivalent, so changes go through `/usr/local/lib/spinifex/spinifex-firewall-apply` — a root helper that accepts a fixed set of verbs and re-validates every address handed to it.

@@ -190,10 +193,10 @@ Dropped packets are logged, rate-limited to five a minute, so this tells you whe
 sudo journalctl -k | grep 'spinifex-fw drop'
 ```

-An empty log with a connection still failing means the policy is not the cause. A node with no table loaded at all is not armed:
+An empty log with a connection still failing means the policy is not the cause. A node with no peer file is not armed, and its boot-time unit loads only the tracking table:

 ```bash
-systemctl is-enabled spinifex-firewall.service
+sudo test -e /etc/spinifex/firewall/peers.nft && echo armed || echo not armed
 ```

 ## Troubleshooting
@@ -106,7 +106,7 @@ done
 | `spinifex-rds-postgres` | RDS, `--engine postgres`         | Alpine 3.24.1 + PostgreSQL 18 + `rds-init`      |
 | `spinifex-rds-mariadb`  | RDS, `--engine mariadb`          | Alpine 3.24.1 + MariaDB 11.8 + `rds-init`       |
 | `spinifex-ecs-node`     | ECS container instances          | Alpine 3.21.7 + containerd + ecs-agent          |
-| `spinifex-eks-node`     | EKS control plane and nodegroups | Alpine 3.21.7 + K3s v1.32.5 + eks-token-webhook |
+| `spinifex-eks-node`     | EKS control plane and nodegroups | Alpine 3.21.7 + K3s v1.32.5 + eks-token-webhook + eks-etcd-member |

 All four boot **BIOS**, unlike the distro images, which boot UEFI. That is a property of how they are built and nothing you need to pass.

@@ -35,6 +35,8 @@ resources:

 An EKS cluster on Spinifex has two parts: a **control plane** (the managed Kubernetes API server, provisioned on a Spinifex-managed VM and fronted by a network load balancer) and one or more **managed node groups** of worker VMs that run your pods. You provide the VPC, subnets, and IAM roles; Spinifex creates and reconciles the cluster, its load balancer, and its security groups.

+Control-plane hosts are not Kubernetes Nodes, as on AWS: `kubectl get nodes` lists only your node group workers, so a cluster with no node group lists none, and no pod runs on a control-plane host. Add-ons that used to tolerate the control-plane `CriticalAddonsOnly` taint, such as CoreDNS, metrics-server and konnectivity-agent, run on workers, so a Ready worker is required before they become available.
+
 Authentication uses **EKS access entries** (the API authentication mode) — IAM principals are mapped to Kubernetes RBAC groups, and `kubectl` authenticates with `aws eks get-token` exactly as it does on AWS.

 <p align="center">
@@ -172,6 +174,8 @@ aws eks update-kubeconfig --name demo
 kubectl get nodes
 ```

+The list shows the node group workers only. The control-plane VMs never appear as Nodes.
+
 ### 4. Expose a workload

 Workers join an auto-managed security group that only allows intra-cluster traffic. To reach a NodePort service, add one ingress rule to that SG (look it up by its deterministic name):
@@ -284,6 +288,12 @@ The workbooks form a ladder. **eks-quickstart** is a NodePort demo; **eks-https-

 :::

+### Upgrade notes
+
+**Control-plane hosts no longer register as Nodes.** `kubectl get nodes` shows workers only, so node counts drop by the number of control planes, for example 6 to 3 on a three-control-plane cluster with three workers. Add-ons that tolerated `CriticalAddonsOnly` now run on workers.
+
+**Existing clusters keep the old behaviour.** A cluster created before this change keeps registering its control planes as tainted Nodes until it is recreated. Recreate the cluster to adopt the new behaviour: automatic control-plane replacement does not currently work when the replacement reuses the failed member's IP, so it is not a supported way to upgrade.
+
 ## Troubleshooting

 **Cluster stuck in `CREATING`.** The control plane is still booting. Check `aws eks describe-cluster --name <name> --query 'cluster.health'` for reported issues. The control plane needs egress to bootstrap, so confirm its subnet has an Internet Gateway route.
@@ -237,6 +237,7 @@ Notes that make the difference between this working and locking you out:
 - **Use a dedicated table and never flush the others.** `vpcd` writes MASQUERADE and per-EIP FORWARD rules into the `ip filter` and `ip nat` tables, and only reinstalls them when the service starts. `iptables -F`, `nft flush ruleset`, `ufw enable` and `firewalld` all destroy them silently, and instance networking stays broken until the next restart.
 - **Do not add a `forward` hook.** nftables evaluates every table registered on a hook and any `drop` is final, so a default-deny forward chain here would override `vpcd`'s accepts in the other table and break every routed-NAT instance and every EIP. Filtering forwarded guest traffic is the security group's job, not the host firewall's.
 - **`vpcd` owns a second table named `spinifex_mss`, separate from `inet spinifex_filter`.** It is a `bridge` table on the veth uplink and an `inet` table on the routed uplink, it has a `forward` hook with policy `accept`, and it only rewrites the MSS option of TCP SYN packets on the host hop to the gateway. It never drops or accepts traffic and does not replace the forward-hook advice above. `vpcd` re-installs it every 30 seconds, so a flush is repaired within that interval, and there is none on a physical uplink.
+- **Keep connection tracking running while the policy is unloaded.** The kernel tracks connections only while some rule uses conntrack, so a node with no ruleset tracks nothing. A connection that opens then, such as an OVN RAFT link, is first seen mid-stream when the policy loads, and if the server's end speaks first its packet looks like a new connection to an ephemeral port and is dropped. Spinifex keeps `table inet spinifex_conntrack`, which accepts everything, loaded on every node for this reason.
 - **`output` is deliberately untouched.** The metadata service's reply path egresses through per-instance policy routing; filtering `output` breaks it in ways that are hard to attribute.
 - **On a single-NIC node the cluster and encap sets are the node's public addresses**, because the planes collapse. The rules are still correct, but they are no longer a boundary — an upstream ACL is the only real control there.

@@ -153,6 +153,8 @@ kubectl get nodes
 kubectl get pods -o wide
 ```

+`kubectl get nodes` lists the worker nodes only, because control-plane hosts are not Kubernetes Nodes.
+
 `kubectl get pods -o wide` shows the demo pods and which node each landed on.

 ### Cleanup
@@ -29,6 +29,7 @@ The largest suite, 27 top-level tests. Notable entry points:
 - `TestInstanceMetadata` — IMDS from inside a guest.
 - **`TestIMDSDatapathSurvivesVswitchdRestart`** restarts `ovs-vswitchd` under a guest with an Elastic IP, once through systemd and once with SIGKILL. After each, the guest must answer over its EIP and fetch its instance ID through IMDSv2 within 60 seconds, and `br-imds` must hold all 7 of the guest's own flows. It is sequential because it takes the whole node's datapath down.
 - `TestStopStart`, `TestEarlyRebootLiveness`, `TestGuestChurnDurability`, `TestSnapshotLifecycle`, `TestSnapshotBackedLaunch`, `TestCreateImage`, `TestVolumeLifecycle`, `TestSpotInstanceLifecycle`, `TestLaunchTemplateBoot`.
+- **`TestRebootLoop`** is opt-in and skipped in a whole-suite run. Name it in `-test.run`, or set `SPINIFEX_REBOOT_LOOP_ROUNDS` (default 10), and it reboots one fresh guest through the API that many times and then once from inside, requiring a new boot ID every time.
 - `TestNegativeErrorPaths` — the `8a`–`8t` AWS error-shape cases (malformed AMI ID, invalid instance type, volume in use, detach-root forbidden, unknown or malformed instance IDs on the lifecycle verbs, and so on).
 - **`TestKVReplication`** — the single-node half of the KV replication policy, and it is not the multi-node assertion with a smaller number in it. On one node one replica per bucket is the correct answer rather than the defect, so what it proves is the opposite: that nothing declared a replica count this deployment cannot place, that `spx admin kv replicas` exits 0 and reports `WANT 1`, and that `--repair` run twice against a cluster with nothing to repair changes no bucket's replica count and leaves every bucket serving. The multi-node suites cannot cover any of that, because on three nodes the unplaceable count is the right one. It lives here rather than in its own suite so it runs in every single-node nightly cell, across every host and network permutation.

@@ -124,7 +125,7 @@ An ALB whose targets all fail their health check keeps serving from them instead

 ### `eks`, `ecs`, `ecr` — container services

-`eks`: cluster create/delete, access entries, addon delivery, `GetToken`, kubeconfig artifacts, the EBS CSI volume path, an IRSA web-identity exchange under the stock AWS trust policy (`StringEquals` on the issuer host-path `:sub`/`:aud`, a mismatched `:sub` denied with `AccessDenied`, a `StringLike` `:sub` granted), and an IRSA pod wired with a projected SA token. Worker IMDS is covered too: a managed nodegroup worker reports IMDSv2 required with hop limit 2, a pod with no IRSA resolves to the node role through IMDS, and a nodegroup built from a launch template takes the template's metadata options. `ecs`: clusters, container instances, the three network modes (`awsvpc`, `bridge`, `host`), task-role credentials fetched from the container credentials endpoint, a service behind an ELB, and ENI release on terminate. `ecr`: control-plane CRUD plus a real OCI data plane — docker login/build/push/pull round trips, multi-layer images, and raw blob HTTP.
+`eks`: cluster create/delete, access entries, addon delivery, `GetToken`, kubeconfig artifacts, the EBS CSI volume path, an IRSA web-identity exchange under the stock AWS trust policy (`StringEquals` on the issuer host-path `:sub`/`:aud`, a mismatched `:sub` denied with `AccessDenied`, a `StringLike` `:sub` granted), and an IRSA pod wired with a projected SA token. Worker IMDS is covered too: a managed nodegroup worker reports IMDSv2 required with hop limit 2, a pod with no IRSA resolves to the node role through IMDS, and a nodegroup built from a launch template takes the template's metadata options. Control-plane isolation is covered as well: no control-plane host registers as a Node, a DaemonSet tolerating every taint runs on workers only, metrics-server is Ready, and logs work for every kube-system pod. `ecs`: clusters, container instances, the three network modes (`awsvpc`, `bridge`, `host`), task-role credentials fetched from the container credentials endpoint, a service behind an ELB, and ENI release on terminate. `ecr`: control-plane CRUD plus a real OCI data plane — docker login/build/push/pull round trips, multi-layer images, and raw blob HTTP.

 ### `rds` — managed databases

@@ -135,7 +136,7 @@ The create/describe/connect path against a live cluster: a DB VM that actually b
 - **`storagefault`** — what a guest does when its storage backend stops answering, and what stop/start does when the seal failed. **The claim is about corruption, not availability.** SIGSTOPs a cluster-wide service, so it always runs last and in its own cell. Three-node, because the RS(2,1) shard floor and cross-node takeover after a failed seal do not exist on one node.
 - **`partialblock`** — concurrent sub-block writes to the same block surviving a cold reread through NBD from a real guest.
 - **`storagegrowth`** — volume growth and image pull. Requires `external_mode=pool`; skips on a nat env, which has no EIP and therefore no guest SSH.
- **`reboot`** — a slow guest waited for and keeping what it wrote, and a wedged guest reset once its budget expires. Proves a real reboot by requiring a *new* boot ID.
+- **`reboot`** — a slow guest waited for and keeping what it wrote, and a wedged guest forced off and relaunched once its budget expires. Proves a real reboot by requiring a *new* boot ID.
 - **`diskperf`** — guest disk performance gate, pinned fio profile on a dedicated blank volume. Also requires `external_mode=pool` for guest SSH.

 ### Other
@@ -43,6 +43,9 @@ require (
 	github.com/spf13/pflag v1.0.10
 	github.com/spf13/viper v1.21.0
 	github.com/stretchr/testify v1.12.1
+	go.etcd.io/etcd/api/v3 v3.5.35
+	go.etcd.io/etcd/client/pkg/v3 v3.5.35
+	go.etcd.io/etcd/client/v3 v3.5.35
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.72.0
 	go.opentelemetry.io/otel v1.47.0
 	go.opentelemetry.io/otel/metric v1.47.0
@@ -57,6 +60,7 @@ require (
 	golang.org/x/term v0.47.0
 	golang.org/x/time v0.16.0
 	golang.org/x/tools v0.52.0
+	google.golang.org/grpc v1.83.2
 	gopkg.in/ini.v1 v1.67.3
 )

@@ -107,6 +111,8 @@ require (
 	github.com/containerd/plugin v1.1.0 // indirect
 	github.com/containerd/ttrpc v1.2.9 // indirect
 	github.com/containerd/typeurl/v2 v2.3.0 // indirect
+	github.com/coreos/go-semver v0.3.0 // indirect
+	github.com/coreos/go-systemd/v22 v22.7.0 // indirect
 	github.com/dgraph-io/badger/v4 v4.9.6 // indirect
 	github.com/dgraph-io/ristretto/v2 v2.4.0 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
@@ -122,6 +128,8 @@ require (
 	github.com/go-playground/validator/v10 v10.30.5 // indirect
 	github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
 	github.com/gofrs/flock v0.10.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/golang/protobuf v1.5.4 // indirect
 	github.com/google/flatbuffers v25.12.19+incompatible // indirect
 	github.com/google/go-tpm v0.9.8 // indirect
 	github.com/google/uuid v1.6.0 // indirect
@@ -195,7 +203,10 @@ require (
 	go.opentelemetry.io/otel/log v1.47.0 // indirect
 	go.opentelemetry.io/otel/sdk/log v1.47.0 // indirect
 	go.opentelemetry.io/proto/otlp v1.11.0 // indirect
+	go.uber.org/atomic v1.7.0 // indirect
+	go.uber.org/multierr v1.6.0 // indirect
 	go.uber.org/nilaway v0.0.0-20260721205819-4a0f7653488a // indirect
+	go.uber.org/zap v1.17.0 // indirect
 	golang.org/x/exp/typeparams v0.0.0-20260611194520-c48552f49976 // indirect
 	golang.org/x/mod v0.42.0 // indirect
 	golang.org/x/net v0.61.0 // indirect
@@ -204,7 +215,6 @@ require (
 	golang.org/x/vuln v1.6.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20260928230214-8a89bd6388cc // indirect
-	google.golang.org/grpc v1.83.2 // indirect
 	google.golang.org/protobuf v1.36.12 // indirect
 	tags.cncf.io/container-device-interface v1.1.1 // indirect
 	tags.cncf.io/container-device-interface/specs-go v1.1.1 // indirect
@@ -128,6 +128,10 @@ github.com/containerd/ttrpc v1.2.9 h1:ha0ak962T0s3CA/RoZ6S6xiWZQF24GrBaEpiGX1uih
 github.com/containerd/ttrpc v1.2.9/go.mod h1:jjtQRwXm4DL3KsHKW8vDiUOV6wO0hi6IPhmJhxU7aEs=
 github.com/containerd/typeurl/v2 v2.3.0 h1:HZHPhRWo5XMy3QGQoPrUzbW/2ckwjfweHmOwlkIrPAQ=
 github.com/containerd/typeurl/v2 v2.3.0/go.mod h1:Qk+PAdUYArVj41TnGi6rJ+48RF0PkcTc4i/taoBcK0w=
+github.com/coreos/go-semver v0.3.0 h1:wkHLiw0WNATZnSG7epLsujiMCgPAc9xhjJ4tgnAxmfM=
+github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
+github.com/coreos/go-systemd/v22 v22.7.0 h1:LAEzFkke61DFROc7zNLX/WA2i5J8gYqe0rSj9KI28KA=
+github.com/coreos/go-systemd/v22 v22.7.0/go.mod h1:xNUYtjHu2EDXbsxz1i41wouACIwT7Ybq9o0BQhMwD0w=
 github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -177,6 +181,8 @@ github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPE
 github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
 github.com/gofrs/flock v0.10.0 h1:SHMXenfaB03KbroETaCMtbBg3Yn29v4w1r+tgy4ff4k=
 github.com/gofrs/flock v0.10.0/go.mod h1:FirDy1Ing0mI2+kB6wk+vyyAH+e6xiE+EYA0jnzV9jc=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
 github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
 github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
 github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
@@ -243,6 +249,8 @@ github.com/josharian/native v1.1.0 h1:uuaP0hAbW7Y4l0ZRQ6C9zfb7Mg1mbFKry/xzDAfmtL
 github.com/josharian/native v1.1.0/go.mod h1:7X/raswPFr05uY3HiLlYeyQntB6OO7E/d2Cu7qoaN2w=
 github.com/jstemmer/go-junit-report/v2 v2.1.0 h1:X3+hPYlSczH9IMIpSC9CQSZA0L+BipYafciZUWHEmsc=
 github.com/jstemmer/go-junit-report/v2 v2.1.0/go.mod h1:mgHVr7VUo5Tn8OLVr1cKnLuEy0M92wdRntM99h7RkgQ=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/klauspost/compress v1.20.3 h1:zWNoFqq/lSuYQRim3dUXlXeA0IqwuZGicdG5ZTfY5j4=
 github.com/klauspost/compress v1.20.3/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
 github.com/klauspost/cpuid/v2 v2.4.1 h1:I9KxadAkvo+QjtcEE2f2tyfEvmIla/T0m+14PrZNl0U=
@@ -332,6 +340,7 @@ github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8
 github.com/pierrec/lz4/v4 v4.1.14/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
 github.com/pierrec/lz4/v4 v4.1.18 h1:xaKrnTkyoqfh1YItXl56+6KJNVYWlEEPuAQW9xsplYQ=
 github.com/pierrec/lz4/v4 v4.1.18/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
+github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -400,9 +409,17 @@ github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavM
 github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
 github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78 h1:ilQV1hzziu+LLM3zUTJ0trRztfwgjqKnBWNtSRkbmwM=
 github.com/youmark/pkcs8 v0.0.0-20240726163527-a2c0da244d78/go.mod h1:aL8wCCfTfSfmXjznFBSZNN13rSJjlIOI1fUNAtF7rmI=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
 go.etcd.io/bbolt v1.5.0 h1:S7GAl7Fxv12yohbwFfIbQCGDWbQbtDGPET4P/bD4lxU=
 go.etcd.io/bbolt v1.5.0/go.mod h1:mkltfYE5aUHQxUct9N9V+Kp7aSjFqjgrhcXIS70Lrdk=
+go.etcd.io/etcd/api/v3 v3.5.35 h1:lB//Av9mFrkljv94XDNAY+qiyQA9W6S/TTA2Uwx2jsI=
+go.etcd.io/etcd/api/v3 v3.5.35/go.mod h1:z1FGQolu/tpmQ2MPFwQNt7G3cXBiSkTevZBhfOtEArQ=
+go.etcd.io/etcd/client/pkg/v3 v3.5.35 h1:nUZutZyJVvsKddK6zddXMBxIhmVi0SzPMBRS0FnNM3o=
+go.etcd.io/etcd/client/pkg/v3 v3.5.35/go.mod h1:0MNjMMqajrOI0/l8OLFkJBhE5Vm6BAi7cerKlrO38hw=
+go.etcd.io/etcd/client/v3 v3.5.35 h1:dk0aSLjY5D1eOR0SNnPXWOWCVA9yIzJoB8a912qxmoI=
+go.etcd.io/etcd/client/v3 v3.5.35/go.mod h1:LjtCl6G5li7PgsGjoQIpYXz/OvSbI1pCxpSp1kqo9P4=
 go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
 go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
 go.opentelemetry.io/contrib/bridges/otelslog v0.21.0 h1:/CdgE52i1OEPHhsGUqrggYNRaZy4v4YgVenGd52dRBI=
@@ -439,18 +456,26 @@ go.opentelemetry.io/otel/trace v1.47.0 h1:JOjX/Oci8K94QHddo+bbfya/Ai/nf6/dt9ZfrF
 go.opentelemetry.io/otel/trace v1.47.0/go.mod h1:jNaSLa2PZEYFG6fRjJABAu+bw4FS08uDmPg28lTghu0=
 go.opentelemetry.io/proto/otlp v1.11.0 h1:5rrYs0Ykyj50sdU/JU0x8etU+LubXWb+gED6TbEdMIk=
 go.opentelemetry.io/proto/otlp v1.11.0/go.mod h1:SmVizdCOAm3XBtG1g1NnOdhW6jtddT72hLMhv8VwA8E=
+go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
+go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
 go.uber.org/goleak v1.3.1-0.20260602155401-5f791e3ef83b h1:FHYZT12VzXnb6P399kd1i9Ai7aLF+CFSUsOFylDt7Lk=
 go.uber.org/goleak v1.3.1-0.20260602155401-5f791e3ef83b/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
 go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
 go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
+go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
+go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU=
 go.uber.org/nilaway v0.0.0-20260721205819-4a0f7653488a h1:CsRd824LAzMmlpu9nYv2PHOqPQYv4Zjd4kGM+g0ahS0=
 go.uber.org/nilaway v0.0.0-20260721205819-4a0f7653488a/go.mod h1:i9Hi8GeZFLBtUCXP73ZNSHKZ4K2JtxT0pLmo3sbJzlo=
+go.uber.org/zap v1.17.0 h1:MTjgFu6ZLKvY6Pvaqk97GlxNBuMpV4Hy/3P6tRGlI2U=
+go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo=
 go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
 go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
 go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
 go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54=
 golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w=
@@ -458,24 +483,33 @@ golang.org/x/exp v0.0.0-20241108190413-2d47ceb2692f h1:XdNn9LlyWAhLVp6P/i8QYBW+h
 golang.org/x/exp v0.0.0-20241108190413-2d47ceb2692f/go.mod h1:D5SMRVC3C2/4+F/DB1wZsLRnSNimn2Sp/NPsCrsv8ak=
 golang.org/x/exp/typeparams v0.0.0-20260611194520-c48552f49976 h1:GTD/WuaexTazIG/SxLOz4rEKZPDVilmVVC2nz4xhwfE=
 golang.org/x/exp/typeparams v0.0.0-20260611194520-c48552f49976/go.mod h1:PqrXSW65cXDZH0k4IeUbhmg/bcAZDbzNz3byBpKCsXo=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
 golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
 golang.org/x/mod v0.42.0 h1:ICeL3t2L30ljSPHHxDnR4fqVcBt/zR2zJAb9juavmQg=
 golang.org/x/mod v0.42.0/go.mod h1:u5fzIrRInkL8MaO1cbRkUHT64v303YIT5KEqpNr7jRs=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
 golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw=
 golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA=
 golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -508,6 +542,8 @@ golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
 golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
 golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
 golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
 golang.org/x/tools v0.52.0 h1:ryxg4U63oaUYk7uOLTF37n43f4qgtGvqV4Ioit1BaEs=
@@ -519,6 +555,9 @@ golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated/go.mod h1:RVAQXBGN
 golang.org/x/vuln v1.6.0 h1:FeMO9Rm/HwyduOztbvKcOw+zvDEPr4I4aQNSfevFcKY=
 golang.org/x/vuln v1.6.0/go.mod h1:bWlG2493/sjR7ksvicBgMrznH3eYQEyK8ifUYBrqUbg=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
 gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
 google.golang.org/genproto/googleapis/api v0.0.0-20260928230214-8a89bd6388cc h1:rZHRz0ogq4Pid1IKtA/ivHi8XONVF5tqUQXA2TtgpkE=
@@ -536,6 +575,7 @@ gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
 gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
 gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 pgregory.net/rapid v1.3.0 h1:vBvO0VSqti75J1jjYqpgPNBLKMd1+gxa9fYo7vk/Exc=
 pgregory.net/rapid v1.3.0/go.mod h1:dPlE4OBBxgXPqkP79flB6sJL1dx5azpI7HQ9MY9Z7uk=
@@ -0,0 +1,217 @@
+// Package fakeetcd serves the etcd Cluster and Maintenance gRPC APIs over mutual
+// TLS with the file layout K3s uses, so a real etcd client can be driven in tests.
+package fakeetcd
+
+import (
+	"context"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"crypto/tls"
+	"crypto/x509"
+	"crypto/x509/pkix"
+	"encoding/pem"
+	"errors"
+	"math/big"
+	"net"
+	"os"
+	"path/filepath"
+	"sync"
+	"testing"
+	"time"
+
+	pb "go.etcd.io/etcd/api/v3/etcdserverpb"
+	"google.golang.org/grpc"
+	"google.golang.org/grpc/credentials"
+)
+
+// Member is one entry of the fake cluster's member list.
+type Member struct {
+	ID       uint64
+	Name     string
+	PeerURLs []string
+}
+
+// Server is a running fake etcd.
+type Server struct {
+	// Endpoint is the https URL clients dial.
+	Endpoint string
+	// CertDir holds client.crt, client.key and server-ca.crt.
+	CertDir string
+
+	mu      sync.Mutex
+	members []Member
+	selfID  uint64
+	removed []uint64
+	failAll error
+}
+
+// Start serves members with selfID as the answering member. It stops with the test.
+func Start(t *testing.T, selfID uint64, members []Member) *Server {
+	t.Helper()
+	dir := t.TempDir()
+	caCert, caKey := newCA(t)
+	writePEM(t, filepath.Join(dir, "server-ca.crt"), "CERTIFICATE", caCert.Raw)
+	serverPair := issue(t, caCert, caKey, x509.ExtKeyUsageServerAuth)
+	clientPair := issue(t, caCert, caKey, x509.ExtKeyUsageClientAuth)
+	writeKeyPair(t, dir, "client", clientPair)
+
+	pool := x509.NewCertPool()
+	pool.AddCert(caCert)
+	lis, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
+	}
+	creds := credentials.NewTLS(&tls.Config{
+		Certificates: []tls.Certificate{serverPair.tls},
+		ClientCAs:    pool,
+		ClientAuth:   tls.RequireAndVerifyClientCert,
+		MinVersion:   tls.VersionTLS12,
+	})
+	s := &Server{Endpoint: "https://" + lis.Addr().String(), CertDir: dir, members: members, selfID: selfID}
+	gs := grpc.NewServer(grpc.Creds(creds))
+	pb.RegisterClusterServer(gs, &cluster{s})
+	pb.RegisterMaintenanceServer(gs, &maintenance{s: s})
+	go func() { _ = gs.Serve(lis) }()
+	t.Cleanup(gs.Stop)
+	return s
+}
+
+// Removed returns the member IDs removed so far.
+func (s *Server) Removed() []uint64 {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	return append([]uint64(nil), s.removed...)
+}
+
+// FailWith makes every later call return err.
+func (s *Server) FailWith(err error) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.failAll = err
+}
+
+type cluster struct{ s *Server }
+
+func (c *cluster) MemberList(context.Context, *pb.MemberListRequest) (*pb.MemberListResponse, error) {
+	c.s.mu.Lock()
+	defer c.s.mu.Unlock()
+	if c.s.failAll != nil {
+		return nil, c.s.failAll
+	}
+	resp := &pb.MemberListResponse{Header: &pb.ResponseHeader{MemberId: c.s.selfID}}
+	for _, m := range c.s.members {
+		resp.Members = append(resp.Members, &pb.Member{ID: m.ID, Name: m.Name, PeerURLs: m.PeerURLs})
+	}
+	return resp, nil
+}
+
+func (c *cluster) MemberRemove(_ context.Context, req *pb.MemberRemoveRequest) (*pb.MemberRemoveResponse, error) {
+	c.s.mu.Lock()
+	defer c.s.mu.Unlock()
+	if c.s.failAll != nil {
+		return nil, c.s.failAll
+	}
+	c.s.removed = append(c.s.removed, req.ID)
+	return &pb.MemberRemoveResponse{Header: &pb.ResponseHeader{MemberId: c.s.selfID}}, nil
+}
+
+func (c *cluster) MemberAdd(context.Context, *pb.MemberAddRequest) (*pb.MemberAddResponse, error) {
+	return nil, errors.New("not implemented")
+}
+
+func (c *cluster) MemberUpdate(context.Context, *pb.MemberUpdateRequest) (*pb.MemberUpdateResponse, error) {
+	return nil, errors.New("not implemented")
+}
+
+func (c *cluster) MemberPromote(context.Context, *pb.MemberPromoteRequest) (*pb.MemberPromoteResponse, error) {
+	return nil, errors.New("not implemented")
+}
+
+type maintenance struct {
+	pb.UnimplementedMaintenanceServer
+
+	s *Server
+}
+
+func (m *maintenance) Status(context.Context, *pb.StatusRequest) (*pb.StatusResponse, error) {
+	m.s.mu.Lock()
+	defer m.s.mu.Unlock()
+	if m.s.failAll != nil {
+		return nil, m.s.failAll
+	}
+	return &pb.StatusResponse{Header: &pb.ResponseHeader{MemberId: m.s.selfID}}, nil
+}
+
+type pair struct {
+	tls tls.Certificate
+	der []byte
+	key *ecdsa.PrivateKey
+}
+
+func newCA(t *testing.T) (*x509.Certificate, *ecdsa.PrivateKey) {
+	t.Helper()
+	key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Fatalf("ca key: %v", err)
+	}
+	tmpl := &x509.Certificate{
+		SerialNumber:          big.NewInt(1),
+		Subject:               pkix.Name{CommonName: "fake-etcd-ca"},
+		NotBefore:             time.Now().Add(-time.Hour),
+		NotAfter:              time.Now().Add(time.Hour),
+		IsCA:                  true,
+		BasicConstraintsValid: true,
+		KeyUsage:              x509.KeyUsageCertSign,
+	}
+	der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
+	if err != nil {
+		t.Fatalf("ca cert: %v", err)
+	}
+	cert, err := x509.ParseCertificate(der)
+	if err != nil {
+		t.Fatalf("parse ca: %v", err)
+	}
+	return cert, key
+}
+
+func issue(t *testing.T, ca *x509.Certificate, caKey *ecdsa.PrivateKey, usage x509.ExtKeyUsage) pair {
+	t.Helper()
+	key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Fatalf("key: %v", err)
+	}
+	tmpl := &x509.Certificate{
+		SerialNumber: big.NewInt(time.Now().UnixNano()),
+		Subject:      pkix.Name{CommonName: "fake-etcd"},
+		NotBefore:    time.Now().Add(-time.Hour),
+		NotAfter:     time.Now().Add(time.Hour),
+		KeyUsage:     x509.KeyUsageDigitalSignature,
+		ExtKeyUsage:  []x509.ExtKeyUsage{usage},
+		IPAddresses:  []net.IP{net.ParseIP("127.0.0.1")},
+		DNSNames:     []string{"localhost"},
+	}
+	der, err := x509.CreateCertificate(rand.Reader, tmpl, ca, &key.PublicKey, caKey)
+	if err != nil {
+		t.Fatalf("cert: %v", err)
+	}
+	return pair{tls: tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}, der: der, key: key}
+}
+
+func writeKeyPair(t *testing.T, dir, name string, p pair) {
+	t.Helper()
+	keyDER, err := x509.MarshalECPrivateKey(p.key)
+	if err != nil {
+		t.Fatalf("marshal key: %v", err)
+	}
+	writePEM(t, filepath.Join(dir, name+".crt"), "CERTIFICATE", p.der)
+	writePEM(t, filepath.Join(dir, name+".key"), "EC PRIVATE KEY", keyDER)
+}
+
+func writePEM(t *testing.T, path, blockType string, der []byte) {
+	t.Helper()
+	data := pem.EncodeToMemory(&pem.Block{Type: blockType, Bytes: der})
+	if err := os.WriteFile(path, data, 0o600); err != nil {
+		t.Fatalf("write %s: %v", path, err)
+	}
+}
@@ -0,0 +1,43 @@
+package gwfix
+
+import (
+	"sync"
+
+	iamdomain "github.com/mulgadc/spinifex/spinifex/domains/iam"
+	stsdomain "github.com/mulgadc/spinifex/spinifex/domains/sts"
+	"github.com/nats-io/nats.go"
+)
+
+// Deps are the collaborators a test composes a gateway from. The gateway
+// config no longer carries them, so test setup keeps them beside the config.
+type Deps struct {
+	IAM   iamdomain.IAMService
+	STS   stsdomain.STSService
+	NATS  *nats.Conn
+	Nodes int
+}
+
+var (
+	depsMu sync.Mutex
+	depsOf = map[any]*Deps{}
+)
+
+// WithDeps records d for gw and returns gw.
+func WithDeps[P comparable](gw P, d Deps) P {
+	depsMu.Lock()
+	defer depsMu.Unlock()
+	depsOf[gw] = &d
+	return gw
+}
+
+// DepsOf returns the deps recorded for gw, creating an empty set when none.
+func DepsOf[P comparable](gw P) *Deps {
+	depsMu.Lock()
+	defer depsMu.Unlock()
+	d, ok := depsOf[gw]
+	if !ok {
+		d = &Deps{}
+		depsOf[gw] = d
+	}
+	return d
+}
@@ -130,6 +130,7 @@ Installed into the EKS node AMI and run at boot or on a timer. All host communic
 | `k3s-first-boot.sh` | Runs once after the server is healthy: reads the node-token and admin kubeconfig, rewrites the kubeconfig server address to the cluster NLB endpoint, and publishes both to the host. |
 | `konnectivity-server-prestart.sh` | Waits for the K3s CA and admin kubeconfig, mints the konnectivity serving cert, and writes the apiserver replica count for the caller to source. |
 | `mulga-eks-addon-sync.sh` | Pulls staged managed-addon descriptors through the gateway, renders each bundle into the K3s auto-deploy dir, and reports per-addon delivery status back. |
+| `eks-etcd-member` | Go helper built from `cmd/eks-etcd-member` and installed on control-plane (server) images. `remove --peer-ip <ip> [--self-ip <ip>]` evicts a terminated peer's etcd member over the local client endpoint, because control planes run no agent and k3s never prunes them. First boot calls it for a replacement control plane and treats an absent member as success. |
 | `mulga-eks-etcd-snapshot.sh` | Snapshots K3s embedded etcd to predastore. Installed into two crond periodic dirs and picks its tier — `frequent` or `daily` — from the dir it was invoked from. |
 | `mulga-eks-k3s-recovery.sh` | Applies a control-plane etcd-quorum recovery directive before k3s starts. Pull-based and a no-op unless the reconciler escalated a wedged control plane. |
 | `mulga-eks-state-report.sh` | Publishes server health (`healthz`, node count) to the cluster reconciler. The apiserver binds the VPC node-ip, which the host daemon cannot probe directly. |
@@ -90,6 +90,12 @@ else
    log "WARNING: clock not synced, continuing with $(date -uIs)"
 fi

+# ── Connection tracking ─────────────────────────────────────────────────────
+# Before OVN opens any link, so arming the firewall later cannot drop one the
+# kernel first saw mid-stream. Non-fatal; the unit reloads it on later boots.
+/usr/local/lib/spinifex/spinifex-firewall-apply track \
+    || log "WARNING: connection tracking not loaded; arming the firewall may drop OVN links"
+
 # ── OVS and OVN ─────────────────────────────────────────────────────────────
 # Start them before setup-ovn.sh so their databases exist when it runs: on
 # physical hardware first-boot DB initialisation outlasts its own timeout.
@@ -24939,11 +24939,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-applicationset-controller
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -25191,11 +25189,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-dex-server
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -25326,11 +25322,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-notifications-controller
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -25436,11 +25430,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-redis
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -25530,11 +25522,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-repo-server
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -25895,11 +25885,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-server
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -26317,11 +26305,9 @@ spec:
      labels:
        app.kubernetes.io/name: argocd-application-controller
    spec:
-      # Local delta over the vendored upstream manifest: a cluster with a
-      # GPU-only nodegroup has no untainted node — the CP carries
-      # CriticalAddonsOnly and every worker carries nvidia.com/gpu, so without
-      # these the pod stays Pending forever. Upstream never needs it because
-      # real EKS control planes are not schedulable nodes.
+      # Local delta over upstream: kept for clusters created before control-plane
+      # hosts stopped registering as Nodes (CriticalAddonsOnly), and for GPU-only
+      # nodegroups whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -44,12 +44,9 @@ spec:
        kubernetes.io/os: linux
      serviceAccountName: ebs-csi-controller-sa
      priorityClassName: system-cluster-critical
-      # A cluster with a GPU-only nodegroup has no untainted node: the CP carries
-      # CriticalAddonsOnly (EKS parity keeps user workloads off it) and every
-      # worker carries nvidia.com/gpu (nodegroup_userdata.go). Without these
-      # tolerations the controller has nowhere to schedule and stays Pending
-      # forever. Real EKS never needs this: its control plane isn't a schedulable
-      # node, so the stock upstream manifest carries no such toleration.
+      # Kept for clusters created before control-plane hosts stopped registering
+      # as Nodes (they carried CriticalAddonsOnly), and for GPU-only nodegroups
+      # whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -348,12 +348,9 @@ spec:
          readOnly: true
 {{IRSA_VOLUME_MOUNT}}
      priorityClassName: system-cluster-critical
-      # A cluster with a GPU-only nodegroup has no untainted node: the CP carries
-      # CriticalAddonsOnly (EKS parity keeps user workloads off it) and every
-      # worker carries nvidia.com/gpu (nodegroup_userdata.go). Without these
-      # tolerations the controller has nowhere to schedule and stays Pending
-      # forever. Real EKS never needs this: its control plane isn't a schedulable
-      # node, so the stock upstream manifest carries no such toleration.
+      # Kept for clusters created before control-plane hosts stopped registering
+      # as Nodes (they carried CriticalAddonsOnly), and for GPU-only nodegroups
+      # whose workers all carry nvidia.com/gpu.
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
@@ -162,12 +162,9 @@ jq -n --arg k "${KUBECONFIG_REWRITTEN}" '{adminKubeconfig: $k}'      | publish_e
 jq -n --arg j "${JWKS}"                 '{jwks: $j}'                  | publish_envelope k3s-oidc-jwks
 jq -n --arg c "${CA_B64}"               '{certificateAuthority: $c}' | publish_envelope k3s-ca

-# 3.5 Prune a terminated control-plane peer this VM replaces. The spinifex
-# reconciler sets EKS_ETCD_PRUNE_PEER_IP to the dead member's node IP when this
-# VM is provisioned as its replacement (member-count reconcile). Deleting the
-# dead Node makes k3s embedded-etcd remove the stale member, so quorum width
-# returns to N rather than N+1-with-a-dead-peer. Best-effort: a failure leaves
-# the member for an operator and never blocks first boot.
+# 3.5 Prune the terminated CP peer this VM replaces: delete its Node if it still
+# has one, else remove its etcd member directly (CPs run no agent). Best-effort:
+# a failure is logged and never blocks first boot.
 if [ -n "${EKS_ETCD_PRUNE_PEER_IP:-}" ]; then
    log "pruning terminated CP peer at ${EKS_ETCD_PRUNE_PEER_IP} from etcd"
    dead_node=$(kubectl --kubeconfig "${KUBECONFIG_FILE}" get nodes \
@@ -181,7 +178,15 @@ if [ -n "${EKS_ETCD_PRUNE_PEER_IP:-}" ]; then
            log "WARN: delete of dead node ${dead_node} failed; etcd member left for operator"
        fi
    else
-        log "no node has internal IP ${EKS_ETCD_PRUNE_PEER_IP}; nothing to prune"
+        # The default-route source address is the address k3s advertises to etcd peers.
+        self_ip=$(ip -4 route get 1.1.1.1 2>/dev/null \
+            | awk '{for (i = 1; i < NF; i++) if ($i == "src") {print $(i + 1); exit}}') || self_ip=""
+        if member_out=$(eks-etcd-member remove --peer-ip "${EKS_ETCD_PRUNE_PEER_IP}" \
+            ${self_ip:+--self-ip "${self_ip}"} 2>&1); then
+            log "${member_out}"
+        else
+            log "WARN: etcd prune failed for peer ${EKS_ETCD_PRUNE_PEER_IP}: ${member_out}; member left for operator"
+        fi
    fi
 fi

@@ -0,0 +1,123 @@
+#!/bin/sh
+# Test for the etcd peer-prune step (3.5) of k3s-first-boot.sh: kubectl and the helpers
+# are stubbed on PATH (jq required) and state paths rewritten into a temp dir.
+# Run: sh scripts/images/eks-node/k3s-first-boot_test.sh
+set -eu
+
+SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
+SRC="${SCRIPT_DIR}/k3s-first-boot.sh"
+WORK=$(mktemp -d)
+trap 'rm -rf "${WORK}"' EXIT
+
+PASS=0
+FAIL=0
+pass() { PASS=$((PASS + 1)); echo "ok   - $1"; }
+fail() { FAIL=$((FAIL + 1)); echo "FAIL - $1"; }
+
+STUBBIN="${WORK}/bin"
+mkdir -p "${STUBBIN}"
+
+# kubectl stub: readiness and JWKS succeed; "get nodes" returns the fixture;
+# "delete node" is recorded so the test can see which Node the script removed.
+cat > "${STUBBIN}/kubectl" <<'EOF'
+#!/bin/sh
+case "$*" in
+    *"--raw=/readyz"*)         echo ok ;;
+    *"--raw=/openid/v1/jwks"*) echo '{"keys":[]}' ;;
+    *"get nodes"*)             printf '%s' "${NODES_BODY:-}" ;;
+    *"delete node"*)           echo "$*" >> "${DELETE_LOG}" ;;
+    *) : ;;
+esac
+EOF
+
+# eks-gateway-publish stub: swallow the envelope.
+cat > "${STUBBIN}/eks-gateway-publish" <<'EOF'
+#!/bin/sh
+cat > /dev/null
+EOF
+
+# eks-etcd-member stub: record argv; exit status is configurable so the
+# best-effort contract (a helper failure never blocks first boot) can be tested.
+cat > "${STUBBIN}/eks-etcd-member" <<'EOF'
+#!/bin/sh
+echo "$*" >> "${MEMBER_LOG}"
+exit "${MEMBER_RC:-0}"
+EOF
+
+chmod +x "${STUBBIN}"/*
+PATH="${STUBBIN}:${PATH}"
+export PATH
+
+command -v jq >/dev/null 2>&1 || { echo "SKIP - jq not installed"; exit 0; }
+
+# Rewrite the script's absolute state paths into the sandbox.
+mkdir -p "${WORK}/state" "${WORK}/etc" "${WORK}/k3s/server" "${WORK}/rancher"
+sed -e "s|/var/lib/spinifex-eks|${WORK}/state|g" \
+    -e "s|/etc/spinifex-eks|${WORK}/etc|g" \
+    -e "s|/var/lib/rancher/k3s/server|${WORK}/k3s/server|g" \
+    -e "s|/etc/rancher/k3s|${WORK}/rancher|g" \
+    "${SRC}" > "${WORK}/first-boot.sh"
+
+cat > "${WORK}/rancher/k3s.yaml" <<'EOF'
+apiVersion: v1
+clusters:
+- cluster:
+    certificate-authority-data: Q0FEQVRB
+    server: https://127.0.0.1:6443
+EOF
+echo node-token > "${WORK}/k3s/server/node-token"
+
+# run_case <prune-ip-or-empty>: fresh sentinel + env, then run the script copy.
+run_case() {
+    prune_ip=$1
+    : > "${WORK}/state/first-boot.pending"
+    {
+        echo "EKS_GATEWAY_URL=https://gw.invalid:9999"
+        echo "EKS_ACCOUNT_ID=000000000001"
+        echo "EKS_CLUSTER_NAME=demo"
+        echo "EKS_NLB_ENDPOINT=https://demo.invalid"
+        [ -z "${prune_ip}" ] || echo "EKS_ETCD_PRUNE_PEER_IP=${prune_ip}"
+    } > "${WORK}/etc/first-boot.env"
+    : > "${WORK}/delete.log"
+    : > "${WORK}/member.log"
+    DELETE_LOG="${WORK}/delete.log" MEMBER_LOG="${WORK}/member.log" \
+        EKS_NODE_INIT=none sh "${WORK}/first-boot.sh" > "${WORK}/out.log" 2>&1
+}
+
+# --- Case 1: dead peer has no Node (all-CPs-without-agent steady state) -> the
+# --- explicit member-remove helper is called with the peer IP.
+NODES_BODY="" run_case 10.0.0.12 && rc=0 || rc=$?
+[ "${rc}" -eq 0 ] && pass "no-node: first boot exits 0" || fail "no-node: exit ${rc}"
+[ ! -s "${WORK}/delete.log" ] && pass "no-node: no Node delete attempted" || fail "no-node: unexpected delete: $(cat "${WORK}/delete.log")"
+grep -q -- 'remove --peer-ip 10.0.0.12' "${WORK}/member.log" \
+    && pass "no-node: eks-etcd-member remove --peer-ip 10.0.0.12 called" \
+    || fail "no-node: member-remove helper not called with the peer IP (member.log: $(cat "${WORK}/member.log"))"
+
+# --- Case 2: dead peer still has a Node (mixed fleet) -> the Node is deleted so
+# --- k3s drops the member itself.
+NODES_BODY="ip-10-0-0-12 10.0.0.12" run_case 10.0.0.12 && rc=0 || rc=$?
+[ "${rc}" -eq 0 ] && pass "node-present: first boot exits 0" || fail "node-present: exit ${rc}"
+grep -q 'delete node ip-10-0-0-12' "${WORK}/delete.log" \
+    && pass "node-present: dead Node deleted" \
+    || fail "node-present: dead Node not deleted (delete.log: $(cat "${WORK}/delete.log"))"
+
+# --- Case 3: not a replacement (no prune IP) -> neither path runs.
+NODES_BODY="" run_case "" && rc=0 || rc=$?
+[ "${rc}" -eq 0 ] && pass "no-prune: first boot exits 0" || fail "no-prune: exit ${rc}"
+{ [ ! -s "${WORK}/delete.log" ] && [ ! -s "${WORK}/member.log" ]; } \
+    && pass "no-prune: neither delete nor member-remove ran" \
+    || fail "no-prune: unexpected prune activity"
+
+# --- Case 4: the helper fails -> best effort, first boot still completes (the
+# --- sentinel is consumed) and the helper was actually attempted.
+NODES_BODY="" MEMBER_RC=1 run_case 10.0.0.12 && rc=0 || rc=$?
+[ "${rc}" -eq 0 ] && pass "helper-fails: first boot still exits 0" || fail "helper-fails: exit ${rc}"
+[ ! -e "${WORK}/state/first-boot.pending" ] && pass "helper-fails: sentinel consumed" || fail "helper-fails: sentinel left behind"
+[ -s "${WORK}/member.log" ] && pass "helper-fails: helper was attempted" || fail "helper-fails: helper never attempted"
+grep -Eq 'prune.*fail.*10\.0\.0\.12' "${WORK}/out.log" \
+    && pass "helper-fails: prune failure logged with the peer IP" \
+    || fail "helper-fails: no prune-failure line naming 10.0.0.12 (out.log: $(cat "${WORK}/out.log"))"
+
+echo
+echo "passed=${PASS} failed=${FAIL}"
+[ "${FAIL}" -eq 0 ]
@@ -7,14 +7,9 @@
 # the K3s auto-deploy dir by k3s.initd, which fills the __KONN_HOST__ /
 # __KONN_AGENT_IMAGE__ tokens at boot.
 #
-# Also tolerates the control-plane CriticalAddonsOnly=true:NoExecute taint and
-# the GPU nvidia.com/gpu=present:NoSchedule taint, matching every other
-# CP-critical addon in this repo (aws-load-balancer-controller, ebs-csi-driver,
-# argocd): a zero-worker or GPU-only cluster has no untainted node, and other
-# addons already schedule onto the CP, so it needs its own tunnel too — without
-# an agent there, kubectl logs/exec against CP-hosted pods and apiserver
-# aggregated-API egress (metrics-server discovery) both fail. Real EKS never
-# needs this: its control plane isn't a schedulable node.
+# Also tolerates CriticalAddonsOnly, for clusters created before control-plane
+# hosts stopped registering as Nodes, and the GPU nvidia.com/gpu taint so a
+# GPU-only nodegroup still gets a tunnel agent.
 apiVersion: v1
 kind: ServiceAccount
 metadata:
@@ -44,11 +39,9 @@ spec:
    spec:
      serviceAccountName: konnectivity-agent
      priorityClassName: system-node-critical
-      # Without these, a zero-worker or GPU-only cluster (CP tainted
-      # CriticalAddonsOnly, workers tainted nvidia.com/gpu) shows DESIRED=0 for
-      # this DaemonSet: no tunnel exists, so kubectl logs/exec against any pod
-      # fails "No agent available" and apiserver aggregated-API egress
-      # (e.g. metrics-server discovery) fails, which stalls namespace GC.
+      # CriticalAddonsOnly is kept for clusters created before control-plane hosts
+      # stopped registering as Nodes. nvidia.com/gpu keeps the agent on GPU-only
+      # nodegroups, where logs/exec and aggregated-API egress need a tunnel.
      tolerations:
        - key: CriticalAddonsOnly
          operator: Exists
@@ -22,9 +22,9 @@ APK_PACKAGES="ca-certificates curl jq iptables ip6tables conntrack-tools cgroup-
 # at a pinned tag (its GitHub releases ship only container images, and its go.mod
 # carries replace directives that block `go install pkg@version`). The agent half
 # is the matching proxy-agent container image, pinned in k3s.initd.
-BUILD_COMMANDS="CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-token-webhook ./cmd/eks-token-webhook && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-gateway-publish ./cmd/eks-gateway-publish && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-gateway-fetch ./cmd/eks-gateway-fetch && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-webhook-cert ./cmd/eks-webhook-cert && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-konnectivity-cert ./cmd/eks-konnectivity-cert && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/ecr-credential-provider ./cmd/ecr-credential-provider && rm -rf /tmp/anp-build && git clone --depth 1 --branch v0.30.3 https://github.com/kubernetes-sigs/apiserver-network-proxy /tmp/anp-build && ( cd /tmp/anp-build && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o /tmp/anp-build/konnectivity-server ./cmd/server ) && cp /tmp/anp-build/konnectivity-server bin/konnectivity-server && rm -rf /tmp/anp-build"
+BUILD_COMMANDS="CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-token-webhook ./cmd/eks-token-webhook && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-gateway-publish ./cmd/eks-gateway-publish && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-gateway-fetch ./cmd/eks-gateway-fetch && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-webhook-cert ./cmd/eks-webhook-cert && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-konnectivity-cert ./cmd/eks-konnectivity-cert && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/eks-etcd-member ./cmd/eks-etcd-member && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o bin/ecr-credential-provider ./cmd/ecr-credential-provider && rm -rf /tmp/anp-build && git clone --depth 1 --branch v0.30.3 https://github.com/kubernetes-sigs/apiserver-network-proxy /tmp/anp-build && ( cd /tmp/anp-build && CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -o /tmp/anp-build/konnectivity-server ./cmd/server ) && cp /tmp/anp-build/konnectivity-server bin/konnectivity-server && rm -rf /tmp/anp-build"

-INSTALL_BINARIES="bin/eks-token-webhook:/usr/local/bin/eks-token-webhook bin/eks-gateway-publish:/usr/local/bin/eks-gateway-publish bin/eks-gateway-fetch:/usr/local/bin/eks-gateway-fetch bin/eks-webhook-cert:/usr/local/bin/eks-webhook-cert bin/eks-konnectivity-cert:/usr/local/bin/eks-konnectivity-cert bin/konnectivity-server:/usr/local/bin/konnectivity-server bin/ecr-credential-provider:/usr/local/bin/ecr-credential-provider"
+INSTALL_BINARIES="bin/eks-token-webhook:/usr/local/bin/eks-token-webhook bin/eks-gateway-publish:/usr/local/bin/eks-gateway-publish bin/eks-gateway-fetch:/usr/local/bin/eks-gateway-fetch bin/eks-webhook-cert:/usr/local/bin/eks-webhook-cert bin/eks-konnectivity-cert:/usr/local/bin/eks-konnectivity-cert bin/eks-etcd-member:/usr/local/bin/eks-etcd-member bin/konnectivity-server:/usr/local/bin/konnectivity-server bin/ecr-credential-provider:/usr/local/bin/ecr-credential-provider"

 # All role services ship; none but the selector are enabled at bake time.
 # eks-node-role picks server vs agent at first boot and enables the rest.
@@ -99,7 +99,8 @@ profile_binaries() {
 ./cmd/eks-gateway-publish:usr/local/bin/eks-gateway-publish \
 ./cmd/eks-gateway-fetch:usr/local/bin/eks-gateway-fetch \
 ./cmd/eks-webhook-cert:usr/local/bin/eks-webhook-cert \
-./cmd/eks-konnectivity-cert:usr/local/bin/eks-konnectivity-cert"
+./cmd/eks-konnectivity-cert:usr/local/bin/eks-konnectivity-cert \
+./cmd/eks-etcd-member:usr/local/bin/eks-etcd-member"
            ;;
        ecs) echo "./cmd/ecs-agent:usr/local/bin/ecs-agent" ;;
        *)   echo "" ;;
@@ -39,6 +39,7 @@ FIREWALL_PEERS="${FIREWALL_DIR}/peers.nft"
 FIREWALL_LOCAL="${FIREWALL_DIR}/local.nft"
 FIREWALL_OPEN="${FIREWALL_DIR}/open-ports.nft"
 FIREWALL_CUSTOM="${FIREWALL_DIR}/custom.nft"
+FIREWALL_TRACK="${FIREWALL_DIR}/conntrack.nft"
 FIREWALL_MODE_FILE="${SPINIFEX_FIREWALL_MODE_FILE:-${FIREWALL_DIR}/mode}"
 FIREWALL_APPLY="/usr/local/lib/spinifex/spinifex-firewall-apply"

@@ -749,6 +750,28 @@ RULES
    $SUDO chmod 0644 "$FIREWALL_RULES"
    info "  $FIREWALL_RULES"

+    # The kernel only tracks connections while some rule uses conntrack. Without
+    # this table a disarmed node tracks nothing, and a flow opened then (OVN's
+    # RAFT links) is first seen mid-stream at arm time, from whichever end speaks
+    # first. Seen from the server, that is a new flow to an ephemeral port, and
+    # the policy drops it. Kept loaded whether or not the policy is armed.
+    $SUDO tee "$FIREWALL_TRACK" > /dev/null << 'TRACK'
+# Spinifex connection tracking. Managed by scripts/setup.sh — edits are overwritten.
+# Accepts everything and filters nothing; it exists so conntrack keeps running
+# while inet spinifex_filter is not loaded. Never delete it on a running node.
+table inet spinifex_conntrack
+delete table inet spinifex_conntrack
+
+table inet spinifex_conntrack {
+    chain input {
+        type filter hook input priority filter; policy accept;
+        ct state invalid counter
+    }
+}
+TRACK
+    $SUDO chmod 0644 "$FIREWALL_TRACK"
+    info "  $FIREWALL_TRACK"
+
    # Host-local facts the policy needs that the daemon has no view of. Separate
    # from peers.nft because the daemon rewrites that file and would drop this.
    # Refreshed on every setup.sh run, so an sshd port change is picked up by an
@@ -816,6 +839,7 @@ set -eu
 RULES="/etc/spinifex/firewall/spinifex.nft"
 PEERS="/etc/spinifex/firewall/peers.nft"
 OPEN="/etc/spinifex/firewall/open-ports.nft"
+TRACK="/etc/spinifex/firewall/conntrack.nft"

 # 0 is a sentinel that keeps the set non-empty, because nft rejects an empty
 # one. No packet can match it: the kernel has no destination port 0.
@@ -837,13 +861,33 @@ write_open_ports() {
 # Deliberately ahead of the policy-file check below: turning the firewall off is
 # the recovery path, and refusing to run it because the policy is missing leaves
 # an operator with a node they can neither arm nor disarm.
+#
+# The tracking table goes in in the same transaction as the delete, so conntrack
+# never stops: a flow that opens while disarmed is then tracked from its SYN,
+# and re-arming does not drop it. The boot unit stays enabled to reload it.
 if [ "${1:-}" = "disable" ]; then
-    nft delete table inet spinifex_filter 2>/dev/null || true
+    if [ -r "$TRACK" ]; then
+        printf 'include "%s"\ntable inet spinifex_filter\ndelete table inet spinifex_filter\n' "$TRACK" | nft -f - \
+            || nft delete table inet spinifex_filter 2>/dev/null || true
+    else
+        nft delete table inet spinifex_filter 2>/dev/null || true
+    fi
    rm -f "$PEERS"
-    systemctl disable --now spinifex-firewall.service >/dev/null 2>&1 || true
    exit 0
 fi

+# track loads only the tracking table. boot is the unit's start: tracking on
+# every node, and the policy on top only where the daemon has written peers.
+if [ "${1:-}" = "track" ] || [ "${1:-}" = "boot" ]; then
+    if [ -r "$TRACK" ]; then
+        nft -f "$TRACK"
+    elif [ "$1" = "track" ]; then
+        echo "no tracking table at $TRACK" >&2
+        exit 1
+    fi
+    if [ "$1" = "track" ] || [ ! -r "$PEERS" ]; then exit 0; fi
+fi
+
 # Every remaining verb writes the policy, so it has to exist. Failing here
 # changes nothing, which leaves a node whose daemon has not yet resolved cluster
 # membership reachable rather than cut off from a cluster it cannot name.
@@ -917,10 +961,9 @@ if [ "${1:-}" = "set-peers" ]; then
    fi
    rm -f "$tmp" "$bak"

-    # The policy is loaded, so make it survive a reboot too. `disable` turns this
-    # unit off, and forming a cluster means disabling the firewall on every node
-    # first; without this, re-arming would look complete while silently leaving
-    # the node bare after its next boot. Idempotent and quiet in the normal case.
+    # The policy is loaded, so make it survive a reboot too. An older helper's
+    # `disable` turned this unit off; without this, re-arming such a node would
+    # look complete while leaving it bare after its next boot. Idempotent.
    systemctl enable spinifex-firewall.service >/dev/null 2>&1 || true
    exit 0
 fi
@@ -937,6 +980,13 @@ APPLY
    $SUDO chmod 0755 "$FIREWALL_APPLY"
    info "  $FIREWALL_APPLY"

+    # Now, not at the next boot: a fresh install forms its cluster before then,
+    # and an upgrade must start tracking before the daemon next arms. A chroot
+    # would load it into the build host's kernel.
+    if [ "${ISO_BUILD:-0}" != "1" ]; then
+        $SUDO "$FIREWALL_APPLY" track || warn "  Could not load $FIREWALL_TRACK: arming later may drop live cluster connections"
+    fi
+
    # Say which state ssh is in either way. A node whose allowlist was never
    # narrowed reads as configured otherwise, and that is the whole risk here.
    _ssh_srcs="$(grep -o '{[^}]*}' "$FIREWALL_CUSTOM" 2>/dev/null | tr -d '{}' | tr -s ' ')"
@@ -1926,9 +1976,9 @@ EOF
    # takes effect once enabled) — without this the JetStream ENOSPC-latch
    # watchdog is inert and a full disk requires a manual restart forever.
    $SUDO systemctl enable --now spinifex-nats-watchdog.timer
-    # Enabled, not started: its ConditionPathExists holds it inert until the
-    # daemon writes a peer file, so this only decides that a node which has one
-    # gets the policy back at boot, before any service opens a socket.
+    # Enabled on every node: at boot it loads connection tracking everywhere,
+    # and the policy only on a node whose daemon has written a peer file, so it
+    # is back before any service opens a socket.
    $SUDO systemctl enable spinifex-firewall.service
    info "Systemd units installed and enabled (per-service users)"
 }
@@ -0,0 +1,214 @@
+#!/bin/sh
+# setup_firewall_conntrack_test.sh — arms the real host firewall under a TCP
+# flow that opened while it was disarmed, in two network namespaces.
+#
+# The policy, the tracking table and spinifex-firewall-apply are cut out of
+# setup.sh with their paths moved under a temp dir, and the helper's verbs are
+# run inside the client namespace. Needs root or passwordless sudo, ip, nft and
+# python3; skips without them. A stub systemctl keeps the helper off the host.
+set -eu
+
+SCRIPT_DIR=$(CDPATH='' cd -- "$(dirname -- "$0")" && pwd)
+SETUP="$SCRIPT_DIR/setup.sh"
+
+echo "-- setup.sh firewall: flows that outlive a disarm"
+
+if [ "$(id -u)" = 0 ]; then SUDO=""
+elif sudo -n true 2>/dev/null; then SUDO="sudo -n"
+else echo "  skip - needs root or passwordless sudo"; exit 0
+fi
+for bin in ip nft python3; do
+    if ! PATH="$PATH:/usr/sbin:/sbin" command -v "$bin" > /dev/null 2>&1; then
+        echo "  skip - $bin not installed"; exit 0
+    fi
+done
+
+TMP=$(mktemp -d)
+S="fwct-s-$$"
+C="fwct-c-$$"
+cleanup() {
+    $SUDO ip netns del "$S" 2> /dev/null || true
+    $SUDO ip netns del "$C" 2> /dev/null || true
+    rm -rf "$TMP"
+}
+trap cleanup EXIT
+
+fails=0
+pass() { echo "  ok   - $1"; }
+fail() { echo "  FAIL - $1"; fails=$((fails + 1)); }
+
+# heredoc <tag> — the body of setup.sh's quoted heredoc named <tag>, with the
+# firewall directory moved under $TMP.
+heredoc() {
+    sed -n "/<< '$1'\$/,/^$1\$/p" "$SETUP" | sed '1d;$d' | sed "s#/etc/spinifex/firewall#$TMP/fw#g"
+}
+
+mkdir -p "$TMP/fw" "$TMP/bin"
+heredoc RULES > "$TMP/fw/spinifex.nft"
+heredoc APPLY > "$TMP/apply"
+heredoc TRACK > "$TMP/fw/conntrack.nft"
+printf 'define spinifex_ssh_ports = { 22 }\n' > "$TMP/fw/local.nft"
+printf 'define spinifex_open_ports = { 0 }\n' > "$TMP/fw/open-ports.nft"
+printf 'define trusted_ssh_peers = { 0.0.0.0/0 }\n' > "$TMP/fw/custom.nft"
+printf '#!/bin/sh\nexit 0\n' > "$TMP/bin/systemctl"
+chmod +x "$TMP/bin/systemctl"
+if [ ! -s "$TMP/fw/spinifex.nft" ] || [ ! -s "$TMP/apply" ]; then
+    echo "  FAIL - could not cut the policy or the helper out of setup.sh"; exit 1
+fi
+
+# One process owns both ends, so the order packets are sent in is exact. The
+# server is 10.99.0.1 (a peer) and also holds 10.99.0.9 (not a peer).
+cat > "$TMP/flow.py" << 'PY'
+import os, socket, subprocess, sys
+
+srv_ns, cli_ns, script = sys.argv[1], sys.argv[2], sys.argv[3]
+
+def enter(ns):
+    fd = os.open("/run/netns/" + ns, os.O_RDONLY)
+    os.setns(fd, os.CLONE_NEWNET)
+    os.close(fd)
+
+enter(srv_ns)
+listener = socket.socket()
+listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+listener.bind(("10.99.0.1", 6644))
+listener.listen(1)
+enter(cli_ns)
+
+cli = srv = None
+for step in script.split("@@"):
+    if step == "connect":
+        cli = socket.create_connection(("10.99.0.1", 6644), timeout=3)
+        srv, _ = listener.accept()
+        cli.settimeout(2)
+        srv.settimeout(2)
+    elif step in ("s2c", "c2s"):
+        tx, rx = (srv, cli) if step == "s2c" else (cli, srv)
+        tx.sendall(b"x" * 300)
+        got = b""
+        try:
+            while len(got) < 300:
+                got += rx.recv(300 - len(got))
+        except socket.timeout:
+            print("lost " + step)
+            sys.exit(1)
+    else:
+        subprocess.run(step, shell=True, check=True)
+PY
+
+# dial <client source> <server addr> <port> — a new connection from the server
+# namespace to the client, which listens on the port. 0 when it connects.
+cat > "$TMP/dial.py" << 'PY'
+import os, socket, sys
+
+srv_ns, cli_ns, src, port = sys.argv[1], sys.argv[2], sys.argv[3], int(sys.argv[4])
+fd = os.open("/run/netns/" + cli_ns, os.O_RDONLY)
+os.setns(fd, os.CLONE_NEWNET)
+listener = socket.socket()
+listener.bind(("10.99.0.3", port))
+listener.listen(1)
+fd = os.open("/run/netns/" + srv_ns, os.O_RDONLY)
+os.setns(fd, os.CLONE_NEWNET)
+dialer = socket.socket()
+dialer.bind((src, 0))
+dialer.settimeout(2)
+try:
+    dialer.connect(("10.99.0.3", port))
+except OSError:
+    sys.exit(1)
+PY
+
+# A new pair of namespaces, and no peer file: every case starts disarmed.
+fresh() {
+    $SUDO rm -f "$TMP/fw/peers.nft"
+    $SUDO ip netns del "$S" 2> /dev/null || true
+    $SUDO ip netns del "$C" 2> /dev/null || true
+    $SUDO ip netns add "$S"
+    $SUDO ip netns add "$C"
+    $SUDO ip link add vs netns "$S" type veth peer name vc netns "$C"
+    $SUDO ip -n "$S" addr add 10.99.0.1/24 dev vs
+    $SUDO ip -n "$S" addr add 10.99.0.9/24 dev vs
+    $SUDO ip -n "$C" addr add 10.99.0.3/24 dev vc
+    for l in "$S vs" "$C vc"; do
+        # shellcheck disable=SC2086  # "<netns> <link>"
+        set -- $l
+        $SUDO ip -n "$1" link set lo up
+        $SUDO ip -n "$1" link set "$2" up
+    done
+}
+
+# Helper verbs and raw nft, run as root inside the client namespace.
+in_c="$SUDO env PATH=$TMP/bin:$PATH:/usr/sbin:/sbin ip netns exec $C"
+verb() { echo "$in_c sh $TMP/apply $*"; }
+arm="$(verb set-peers) < $TMP/peers"
+printf '10.99.0.1\n10.99.0.1\n' > "$TMP/peers"
+
+# flow <desc> <steps> — passes when every transfer is delivered.
+flow() {
+    if out=$($SUDO python3 "$TMP/flow.py" "$S" "$C" "$2" 2>&1); then pass "$1"; else fail "$1: $out"; fi
+}
+
+# Control first: with nothing tracking, arming under an open flow must drop the
+# server's next segment, or this kernel cannot show the failure and the cases
+# below prove nothing.
+fresh
+if $SUDO python3 "$TMP/flow.py" "$S" "$C" "connect@@s2c@@c2s@@$arm@@s2c" > /dev/null 2>&1; then
+    fail "control: a flow opened with no conntrack user survived the arm, so this kernel cannot reproduce the drop"
+else
+    pass "control: a flow opened with no conntrack user loses the server's next segment once armed"
+fi
+
+fresh
+flow "a flow opened after disable survives set-peers, server speaking first" \
+    "$(verb disable)@@connect@@s2c@@c2s@@$arm@@s2c@@c2s"
+
+fresh
+flow "a flow opened after boot with no peer file survives set-peers, server speaking first" \
+    "$(verb boot)@@connect@@s2c@@c2s@@$arm@@s2c@@c2s"
+
+fresh
+flow "a flow opened while armed survives disable and a re-arm, server speaking first" \
+    "$arm@@connect@@s2c@@c2s@@$(verb disable)@@s2c@@c2s@@s2c@@$arm@@s2c@@c2s"
+
+fresh
+flow "a flow opened before track survives set-peers once it has carried traffic" \
+    "connect@@s2c@@c2s@@$(verb track)@@s2c@@c2s@@$arm@@s2c@@c2s"
+
+# The tracking table must not open anything the policy drops.
+fresh
+sh -c "$(verb boot) && $arm"
+if $SUDO python3 "$TMP/dial.py" "$S" "$C" 10.99.0.1 6644; then
+    pass "armed: a peer still reaches a cluster port"
+else
+    fail "armed: a peer cannot reach a cluster port"
+fi
+if $SUDO python3 "$TMP/dial.py" "$S" "$C" 10.99.0.9 6644; then
+    fail "armed: a non-peer reaches a cluster port"
+else
+    pass "armed: a non-peer is dropped on a cluster port"
+fi
+if $SUDO python3 "$TMP/dial.py" "$S" "$C" 10.99.0.1 7777; then
+    fail "armed: a peer reaches a port the policy does not open"
+else
+    pass "armed: a peer is dropped on a port the policy does not open"
+fi
+
+# disable leaves only the tracking table, which accepts everything.
+sh -c "$(verb disable)"
+tables=$($in_c nft list tables | tr '\n' ' ')
+case "$tables" in
+    *spinifex_filter*) fail "disable left inet spinifex_filter loaded: $tables" ;;
+    *spinifex_conntrack*) pass "disable removes the policy and keeps the tracking table" ;;
+    *) fail "disable removed the tracking table: '$tables'" ;;
+esac
+if $SUDO python3 "$TMP/dial.py" "$S" "$C" 10.99.0.9 7777; then
+    pass "disarmed: the tracking table drops nothing"
+else
+    fail "disarmed: the tracking table dropped a connection"
+fi
+
+if [ "$fails" -gt 0 ]; then
+    echo "  $fails check(s) failed"
+    exit 1
+fi
+echo "  setup_firewall_conntrack ok"
@@ -293,18 +293,20 @@ fi
 # file behind it is unreachable after the next reboot and unexplainable to
 # whoever inherits it.
 #
-# Only our own table and our own marked rules, never a global flush: setup.sh
-# creates exactly `table inet spinifex_filter`, and vpcd writes iptables rules
-# carrying the comments below. Anything else on this host belongs to someone
-# else.
+# Only our own tables and our own marked rules, never a global flush: setup.sh
+# creates exactly `table inet spinifex_filter` and `table inet
+# spinifex_conntrack`, and vpcd writes iptables rules carrying the comments
+# below. Anything else on this host belongs to someone else.
 # ---------------------------------------------------------------------------
 log "unloading the host firewall policy"
 if command -v nft >/dev/null 2>&1; then
-    if sudo nft list table inet spinifex_filter >/dev/null 2>&1; then
-        run sudo nft delete table inet spinifex_filter
-    else
-        log "  no inet spinifex_filter table loaded"
-    fi
+    for nft_table in spinifex_filter spinifex_conntrack; do
+        if sudo nft list table inet "$nft_table" >/dev/null 2>&1; then
+            run sudo nft delete table inet "$nft_table"
+        else
+            log "  no inet $nft_table table loaded"
+        fi
+    done
 fi
 if command -v iptables-save >/dev/null 2>&1 && ! $DRY_RUN; then
    for marker in spinifex-nat-egress spinifex-eip-ingress spinifex-imds spinifex-imds-remap; do
@@ -641,9 +643,11 @@ if ! $DRY_RUN; then
    for svc in nats gw daemon storage northstar viperblock vpcd ui; do
        id -u "spinifex-${svc}" >/dev/null 2>&1 && LEFT+=("user spinifex-${svc}")
    done
-    if command -v nft >/dev/null 2>&1 && sudo nft list table inet spinifex_filter >/dev/null 2>&1; then
-        LEFT+=("nft table inet spinifex_filter")
-    fi
+    for nft_table in spinifex_filter spinifex_conntrack; do
+        if command -v nft >/dev/null 2>&1 && sudo nft list table inet "$nft_table" >/dev/null 2>&1; then
+            LEFT+=("nft table inet $nft_table")
+        fi
+    done
    $PURGE_DATA && [ -e "$DATA_DIR" ] && ! mountpoint -q "$DATA_DIR" 2>/dev/null && LEFT+=("$DATA_DIR")

    if [ "${#LEFT[@]}" -gt 0 ]; then
@@ -158,9 +158,10 @@ fi

 # The nft teardown must name our own table. A global flush would take the
 # host's other rules with it.
-grep -q 'nft delete table inet spinifex_filter' "$UNINSTALL" \
-    && pass "nft teardown deletes only inet spinifex_filter" \
-    || fail "nft teardown does not delete inet spinifex_filter by name"
+grep -q 'nft delete table inet "$nft_table"' "$UNINSTALL" \
+    && grep -q 'for nft_table in spinifex_filter spinifex_conntrack; do' "$UNINSTALL" \
+    && pass "nft teardown deletes only inet spinifex_filter and spinifex_conntrack" \
+    || fail "nft teardown does not delete inet spinifex_filter and spinifex_conntrack by name"
 grep -qE 'nft[[:space:]]+(-[a-z][[:space:]]+)*flush ruleset' "$UNINSTALL" \
    && fail "nft teardown flushes the whole ruleset" \
    || pass "nft teardown never flushes the whole ruleset"
@@ -0,0 +1,79 @@
+package etcdmember
+
+import (
+	"context"
+	"fmt"
+	"path/filepath"
+	"time"
+
+	"go.etcd.io/etcd/client/pkg/v3/transport"
+	clientv3 "go.etcd.io/etcd/client/v3"
+)
+
+// DefaultEndpoint and DefaultCertDir match where K3s serves embedded etcd and
+// writes its client certificate.
+const (
+	DefaultEndpoint = "https://127.0.0.1:2379"
+	DefaultCertDir  = "/var/lib/rancher/k3s/server/tls/etcd"
+)
+
+// EtcdClient is a Client backed by a real etcd connection.
+type EtcdClient struct {
+	cli      *clientv3.Client
+	endpoint string
+}
+
+var _ Client = (*EtcdClient)(nil)
+
+// Dial connects to etcd with the K3s etcd client certificate in certDir.
+func Dial(endpoint, certDir string, timeout time.Duration) (*EtcdClient, error) {
+	tlsInfo := transport.TLSInfo{
+		CertFile:      filepath.Join(certDir, "client.crt"),
+		KeyFile:       filepath.Join(certDir, "client.key"),
+		TrustedCAFile: filepath.Join(certDir, "server-ca.crt"),
+	}
+	tlsCfg, err := tlsInfo.ClientConfig()
+	if err != nil {
+		return nil, fmt.Errorf("etcd client TLS: %w", err)
+	}
+	cli, err := clientv3.New(clientv3.Config{
+		Endpoints:   []string{endpoint},
+		DialTimeout: timeout,
+		TLS:         tlsCfg,
+	})
+	if err != nil {
+		return nil, fmt.Errorf("dial etcd %s: %w", endpoint, err)
+	}
+	return &EtcdClient{cli: cli, endpoint: endpoint}, nil
+}
+
+// Close releases the connection.
+func (e *EtcdClient) Close() error { return e.cli.Close() }
+
+// MemberList implements Client.
+func (e *EtcdClient) MemberList(ctx context.Context) ([]Member, error) {
+	resp, err := e.cli.MemberList(ctx)
+	if err != nil {
+		return nil, err
+	}
+	members := make([]Member, 0, len(resp.Members))
+	for _, m := range resp.Members {
+		members = append(members, Member{ID: m.ID, Name: m.Name, PeerURLs: m.PeerURLs})
+	}
+	return members, nil
+}
+
+// MemberRemove implements Client.
+func (e *EtcdClient) MemberRemove(ctx context.Context, id uint64) error {
+	_, err := e.cli.MemberRemove(ctx, id)
+	return err
+}
+
+// SelfID implements Client using the local endpoint's own member ID.
+func (e *EtcdClient) SelfID(ctx context.Context) (uint64, error) {
+	resp, err := e.cli.Status(ctx, e.endpoint)
+	if err != nil {
+		return 0, err
+	}
+	return resp.Header.MemberId, nil
+}
@@ -0,0 +1,99 @@
+package etcdmember_test
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"testing"
+	"time"
+
+	"github.com/mulgadc/spinifex/internal/testkit/fakeetcd"
+	"github.com/mulgadc/spinifex/spinifex/agents/eks/etcdmember"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+var members = []fakeetcd.Member{
+	{ID: 0xa1, Name: "cp-a", PeerURLs: []string{"https://10.0.0.10:2380"}},
+	{ID: 0xb2, Name: "cp-b", PeerURLs: []string{"https://10.0.0.12:2380"}},
+}
+
+func dial(t *testing.T, srv *fakeetcd.Server) *etcdmember.EtcdClient {
+	t.Helper()
+	cli, err := etcdmember.Dial(srv.Endpoint, srv.CertDir, 5*time.Second)
+	require.NoError(t, err)
+	t.Cleanup(func() { _ = cli.Close() })
+	return cli
+}
+
+func TestEtcdClientListsMembersOverMutualTLS(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+	cli := dial(t, srv)
+
+	got, err := cli.MemberList(t.Context())
+	require.NoError(t, err)
+	require.Len(t, got, 2)
+	assert.Equal(t, etcdmember.Member{ID: 0xb2, Name: "cp-b", PeerURLs: []string{"https://10.0.0.12:2380"}}, got[1])
+}
+
+func TestEtcdClientSelfIDIsTheAnsweringMember(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+
+	id, err := dial(t, srv).SelfID(t.Context())
+	require.NoError(t, err)
+	assert.Equal(t, uint64(0xa1), id)
+}
+
+func TestEtcdClientRemovesTheRequestedMember(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+
+	require.NoError(t, dial(t, srv).MemberRemove(t.Context(), 0xb2))
+	assert.Equal(t, []uint64{0xb2}, srv.Removed())
+}
+
+func TestEtcdClientSurfacesServerErrors(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+	cli := dial(t, srv)
+	srv.FailWith(errors.New("etcdserver: unhealthy cluster"))
+
+	_, err := cli.MemberList(t.Context())
+	assert.ErrorContains(t, err, "unhealthy cluster")
+	assert.ErrorContains(t, cli.MemberRemove(t.Context(), 0xb2), "unhealthy cluster")
+	_, err = cli.SelfID(t.Context())
+	assert.ErrorContains(t, err, "unhealthy cluster")
+	assert.Empty(t, srv.Removed())
+}
+
+func TestDialFailsWithoutClientCertificates(t *testing.T) {
+	t.Parallel()
+	_, err := etcdmember.Dial("https://127.0.0.1:2379", t.TempDir(), time.Second)
+	require.Error(t, err)
+	assert.ErrorContains(t, err, "etcd client TLS")
+}
+
+func TestRunRemovesTheDeadPeerThroughARealClient(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+	var out bytes.Buffer
+
+	code := etcdmember.Run(context.Background(), dial(t, srv),
+		etcdmember.Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"}, &out)
+
+	assert.Zero(t, code, out.String())
+	assert.Equal(t, []uint64{0xb2}, srv.Removed())
+}
+
+func TestRunRefusesToRemoveTheLocalMemberThroughARealClient(t *testing.T) {
+	t.Parallel()
+	srv := fakeetcd.Start(t, 0xa1, members)
+	var out bytes.Buffer
+
+	code := etcdmember.Run(context.Background(), dial(t, srv), etcdmember.Config{PeerIP: "10.0.0.10"}, &out)
+
+	assert.NotZero(t, code)
+	assert.Empty(t, srv.Removed())
+}
@@ -0,0 +1,108 @@
+// Package etcdmember implements eks-etcd-member, which runs inside an EKS K3s
+// control-plane VM and removes a dead peer from the embedded etcd cluster.
+package etcdmember
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"io"
+	"net/url"
+)
+
+// Member is one etcd cluster member.
+type Member struct {
+	ID       uint64
+	Name     string
+	PeerURLs []string
+}
+
+// Client is the slice of the etcd cluster API the removal needs.
+type Client interface {
+	MemberList(ctx context.Context) ([]Member, error)
+	MemberRemove(ctx context.Context, id uint64) error
+	// SelfID is the ID of the member this client is connected to.
+	SelfID(ctx context.Context) (uint64, error)
+}
+
+// Config selects the member to remove.
+type Config struct {
+	PeerIP string
+	SelfIP string
+}
+
+var (
+	// ErrNoMatch means no member advertises the peer IP.
+	ErrNoMatch = errors.New("no etcd member matches the peer IP")
+	// ErrAmbiguous means more than one member advertises the peer IP.
+	ErrAmbiguous = errors.New("more than one etcd member matches the peer IP")
+	// ErrSelf means the matched member is the local one.
+	ErrSelf = errors.New("refusing to remove the local etcd member")
+)
+
+// Remove removes the single member whose peer URL host equals cfg.PeerIP.
+func Remove(ctx context.Context, c Client, cfg Config) error {
+	if cfg.PeerIP == "" {
+		return errors.New("peer IP is required")
+	}
+	if cfg.PeerIP == cfg.SelfIP {
+		return fmt.Errorf("%w: %s", ErrSelf, cfg.PeerIP)
+	}
+	members, err := c.MemberList(ctx)
+	if err != nil {
+		return fmt.Errorf("list etcd members: %w", err)
+	}
+	var matches []Member
+	for _, m := range members {
+		if advertises(m, cfg.PeerIP) {
+			matches = append(matches, m)
+		}
+	}
+	switch len(matches) {
+	case 0:
+		return fmt.Errorf("%w: %s", ErrNoMatch, cfg.PeerIP)
+	case 1:
+	default:
+		return fmt.Errorf("%w: %s (%d members)", ErrAmbiguous, cfg.PeerIP, len(matches))
+	}
+	selfID, err := c.SelfID(ctx)
+	if err != nil {
+		return fmt.Errorf("identify the local etcd member: %w", err)
+	}
+	if matches[0].ID == selfID {
+		return fmt.Errorf("%w: %s", ErrSelf, matches[0].Name)
+	}
+	if err := c.MemberRemove(ctx, matches[0].ID); err != nil {
+		return fmt.Errorf("remove etcd member %s: %w", matches[0].Name, err)
+	}
+	return nil
+}
+
+// advertises reports whether any peer URL of m has host ip. Matching the parsed
+// host keeps 10.0.0.12 from matching 10.0.0.123.
+func advertises(m Member, ip string) bool {
+	for _, raw := range m.PeerURLs {
+		u, err := url.Parse(raw)
+		if err == nil && u.Hostname() == ip {
+			return true
+		}
+	}
+	return false
+}
+
+// Run removes the peer and returns the process exit code. A member that is
+// already gone is success so a first-boot rerun stays idempotent.
+func Run(ctx context.Context, c Client, cfg Config, out io.Writer) int {
+	err := Remove(ctx, c, cfg)
+	switch {
+	case err == nil:
+		_, _ = fmt.Fprintf(out, "removed etcd member %s\n", cfg.PeerIP)
+		return 0
+	case errors.Is(err, ErrNoMatch):
+		_, _ = fmt.Fprintf(out, "member already absent: %s\n", cfg.PeerIP)
+		return 0
+	default:
+		_, _ = fmt.Fprintf(out, "etcd member remove failed: %v\n", err)
+		return 1
+	}
+}
@@ -0,0 +1,150 @@
+package etcdmember
+
+import (
+	"bytes"
+	"context"
+	"errors"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+type fakeClient struct {
+	members   []Member
+	listErr   error
+	removeErr error
+	removed   []uint64
+	selfID    uint64
+	selfErr   error
+}
+
+func (f *fakeClient) MemberList(context.Context) ([]Member, error) {
+	return f.members, f.listErr
+}
+
+func (f *fakeClient) MemberRemove(_ context.Context, id uint64) error {
+	f.removed = append(f.removed, id)
+	return f.removeErr
+}
+
+func (f *fakeClient) SelfID(context.Context) (uint64, error) {
+	return f.selfID, f.selfErr
+}
+
+var _ Client = (*fakeClient)(nil)
+
+func threeMembers() []Member {
+	return []Member{
+		{ID: 1, Name: "cp-1", PeerURLs: []string{"https://10.0.0.10:2380"}},
+		{ID: 2, Name: "cp-2", PeerURLs: []string{"https://10.0.0.12:2380"}},
+		{ID: 3, Name: "cp-3", PeerURLs: []string{"https://10.0.0.11:2380"}},
+	}
+}
+
+func TestRemove_RemovesOnlyTheMemberWithThatPeerIP(t *testing.T) {
+	c := &fakeClient{members: threeMembers()}
+	require.NoError(t, Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"}))
+	assert.Equal(t, []uint64{2}, c.removed)
+}
+
+func TestRemove_MatchesTheHostNotAPrefix(t *testing.T) {
+	c := &fakeClient{members: []Member{
+		{ID: 1, Name: "cp-1", PeerURLs: []string{"https://10.0.0.10:2380"}},
+		{ID: 2, Name: "cp-2", PeerURLs: []string{"https://10.0.0.123:2380"}},
+	}}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, ErrNoMatch)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_AnyAdvertisedPeerURLCounts(t *testing.T) {
+	c := &fakeClient{members: []Member{
+		{ID: 1, Name: "cp-1", PeerURLs: []string{"https://10.0.0.10:2380"}},
+		{ID: 2, Name: "cp-2", PeerURLs: []string{"https://127.0.0.1:2380", "https://10.0.0.12:2380"}},
+	}}
+	require.NoError(t, Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"}))
+	assert.Equal(t, []uint64{2}, c.removed)
+}
+
+func TestRemove_RefusesTheLocalMemberByIDWithoutASelfIP(t *testing.T) {
+	c := &fakeClient{members: threeMembers(), selfID: 2}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12"})
+	require.ErrorIs(t, err, ErrSelf)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_RemovesNothingWhenTheLocalMemberCannotBeIdentified(t *testing.T) {
+	boom := errors.New("status unavailable")
+	c := &fakeClient{members: threeMembers(), selfErr: boom}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12"})
+	require.ErrorIs(t, err, boom)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_RefusesWhenNothingMatches(t *testing.T) {
+	c := &fakeClient{members: threeMembers()}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.99", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, ErrNoMatch)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_RefusesWhenMoreThanOneMatches(t *testing.T) {
+	c := &fakeClient{members: []Member{
+		{ID: 1, Name: "cp-1", PeerURLs: []string{"https://10.0.0.10:2380"}},
+		{ID: 2, Name: "cp-2", PeerURLs: []string{"https://10.0.0.12:2380"}},
+		{ID: 3, Name: "cp-2-old", PeerURLs: []string{"https://10.0.0.12:2380"}},
+	}}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, ErrAmbiguous)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_RefusesToRemoveItself(t *testing.T) {
+	c := &fakeClient{members: threeMembers()}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.10", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, ErrSelf)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_ListFailureRemovesNothing(t *testing.T) {
+	boom := errors.New("etcd unreachable")
+	c := &fakeClient{members: threeMembers(), listErr: boom}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, boom)
+	assert.Empty(t, c.removed)
+}
+
+func TestRemove_RemoveFailureIsReturned(t *testing.T) {
+	boom := errors.New("member not found")
+	c := &fakeClient{members: threeMembers(), removeErr: boom}
+	err := Remove(context.Background(), c, Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"})
+	require.ErrorIs(t, err, boom)
+	assert.Equal(t, []uint64{2}, c.removed)
+}
+
+func TestRun_ExitCodeMapping(t *testing.T) {
+	boom := errors.New("etcd unreachable")
+	cfg := Config{PeerIP: "10.0.0.12", SelfIP: "10.0.0.10"}
+	cases := []struct {
+		name     string
+		client   *fakeClient
+		cfg      Config
+		wantCode int
+		wantOut  string
+	}{
+		{"removed", &fakeClient{members: threeMembers()}, cfg, 0, "removed etcd member 10.0.0.12"},
+		{"already absent is success", &fakeClient{members: threeMembers()}, Config{PeerIP: "10.0.0.99", SelfIP: "10.0.0.10"}, 0, "member already absent: 10.0.0.99"},
+		{"self refused", &fakeClient{members: threeMembers()}, Config{PeerIP: "10.0.0.10", SelfIP: "10.0.0.10"}, 1, "refusing to remove the local etcd member"},
+		{"ambiguous refused", &fakeClient{members: append(threeMembers(), Member{ID: 4, Name: "dup", PeerURLs: []string{"https://10.0.0.12:2380"}})}, cfg, 1, "more than one etcd member"},
+		{"list error", &fakeClient{listErr: boom}, cfg, 1, "etcd unreachable"},
+		{"remove error", &fakeClient{members: threeMembers(), removeErr: boom}, cfg, 1, "etcd unreachable"},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			var out bytes.Buffer
+			assert.Equal(t, tc.wantCode, Run(context.Background(), tc.client, tc.cfg, &out))
+			assert.Contains(t, out.String(), tc.wantOut)
+		})
+	}
+}
@@ -34,7 +34,8 @@ func ValidateRebootInstancesInput(input *ec2.RebootInstancesInput) error {
 }

 // RebootInstances sends reboot commands to specified instances via NATS.
-// Unlike stop+start, reboot keeps the instance running and sends a QMP system_reset.
+// Unlike stop+start, reboot keeps the instance running, on its node and volumes,
+// while the guest is shut down and started on a fresh QEMU.
 // Returns an empty response on success (AWS returns no state-change data).
 func RebootInstances(ctx context.Context, input *ec2.RebootInstancesInput, natsConn *nats.Conn, accountID string) (*ec2.RebootInstancesOutput, error) {
 	if err := ValidateRebootInstancesInput(input); err != nil {
@@ -7,14 +7,14 @@ import (
 )

 // NewRegistration returns the EC2 AWS Query control-plane registration. Errors
-// render in the EC2 <Response><Errors> body, which the gateway keys on the
-// service name.
+// render in the EC2 <Response><Errors> body.
 func NewRegistration(deps Deps) dispatch.Registration {
 	return dispatch.Registration{
 		Service:       ServiceName,
 		ResolveAction: resolveAction,
 		Dispatch:      NewHandler(deps).Dispatch,
 		Errors:        dispatch.ErrorEnvelopeXML,
+		Traits:        dispatch.ServiceTraits{XMLErrorShape: dispatch.XMLErrorShapeEC2},
 		Inventory:     OperationInventory(),
 	}
 }
@@ -298,6 +298,31 @@ func TestProvisionReplacementCP_LaunchesJoinOnFreeHost(t *testing.T) {
 		"replacement is told to prune the dead peer from etcd once joined")
 }

+func TestProvisionReplacementCP_RendersDisableAgent(t *testing.T) {
+	t.Parallel()
+	sched := &fakeHostScheduler{hosts: []string{"node-a", "node-b", "node-c"}}
+	inst := &seqK3sInst{}
+	svc := newPlacerService(sched, &fakePlacer{}, &seqK3sVPC{}, inst)
+	svc.deps.SystemAccessKey = "AKIATEST"
+	svc.deps.SystemSecretKey = "secret"
+	tmpl := validK3sInput()
+
+	_, err := svc.ProvisionReplacementCP(context.Background(), ReplacementCPRequest{
+		ClusterName:  "alpha",
+		Template:     &tmpl,
+		JoinURL:      k3sServerJoinURL("10.0.0.10"),
+		ExcludeHosts: []string{"node-a", "node-b"},
+		MemberCount:  haControlPlaneCount,
+		DeadPeerIP:   "10.0.0.12",
+	})
+
+	require.NoError(t, err)
+	assert.Contains(t, inst.userData["node-c"], "disable-agent: true",
+		"a replacement renders the current config, so it joins without an agent")
+	assert.Contains(t, inst.userData["node-c"], "EKS_ETCD_PRUNE_PEER_IP=10.0.0.12",
+		"the prune hint still reaches first boot")
+}
+
 func TestProvisionReplacementCP_NoFreeHostErrors(t *testing.T) {
 	t.Parallel()
 	sched := &fakeHostScheduler{hosts: []string{"node-a", "node-b"}}
@@ -598,11 +598,9 @@ func buildK3sUserData(in K3sServerInput) string {
 	// out and hold a tunnel to every apiserver replica, so apiserver→pod/kubelet
 	// egress is HA-correct (no single-VIP fan-out 502). EKS-parity datapath.
 	configLines = append(configLines, "egress-selector-mode: disabled")
-	configLines = append(configLines,
-		// Prevent user workloads on the CP (EKS parity). k3s packaged addons tolerate CriticalAddonsOnly.
-		"node-taint:",
-		"  - CriticalAddonsOnly=true:NoExecute",
-	)
+	// No agent: a control-plane host never registers as a Node (EKS parity), so
+	// no pod runs there and none needs a taint to stay away.
+	configLines = append(configLines, "disable-agent: true")
 	configLines = append(configLines,
 		"disable:",
 		"  - traefik",
@@ -461,6 +461,44 @@ func TestBuildK3sUserData_EgressSelectorDisabledWithKonnConfig(t *testing.T) {
 		"the EgressSelectorConfiguration file routes the cluster egress to the konn socket")
 }

+func TestBuildK3sUserData_DisablesAgentOnClusterInit(t *testing.T) {
+	t.Parallel()
+	in := validK3sInput()
+	in.ServerURL = ""
+
+	ud := buildK3sUserData(in)
+	assert.Contains(t, ud, "cluster-init: true")
+	assert.Contains(t, ud, "disable-agent: true",
+		"a control-plane host must not register as a Node (EKS parity)")
+}
+
+func TestBuildK3sUserData_DisablesAgentOnJoin(t *testing.T) {
+	t.Parallel()
+	in := validK3sInput()
+	in.ServerURL = k3sServerJoinURL("10.0.0.10")
+	in.JoinToken = "tok"
+
+	ud := buildK3sUserData(in)
+	assert.Contains(t, ud, "server: "+in.ServerURL)
+	assert.Contains(t, ud, "disable-agent: true",
+		"join servers run without an agent too")
+}
+
+func TestBuildK3sUserData_DisableAgentKeepsControlPlaneConfig(t *testing.T) {
+	t.Parallel()
+	in := validK3sInput()
+	in.PrivateEndpointIP = "10.20.0.5"
+
+	ud := buildK3sUserData(in)
+	require.Contains(t, ud, "disable-agent: true")
+	assert.Contains(t, ud, "egress-selector-mode: disabled")
+	assert.Contains(t, ud, "egress-selector-config-file="+k3sEgressSelectorConfigPath)
+	assert.Contains(t, ud, "advertise-address: 10.20.0.5")
+	for _, addon := range []string{"traefik", "servicelb", "local-storage"} {
+		assert.Contains(t, ud, "  - "+addon, "packaged addon %s stays disabled", addon)
+	}
+}
+
 func TestEgressSelectorConfigYAML_TunnelsClusterOnly(t *testing.T) {
 	t.Parallel()
 	cfg := egressSelectorConfigYAML()
@@ -587,10 +625,9 @@ func TestLaunchK3sServerVM_UserDataContainsAllArtifacts(t *testing.T) {
 	assert.Contains(t, udata, "FAKEPUB")

 	assert.Contains(t, udata, "path: "+k3sConfigPath)
-	// The control plane is tainted so user workloads never schedule on it (EKS
-	// parity, and it keeps image pulls off the etcd disk).
-	assert.Contains(t, udata, "node-taint:")
-	assert.Contains(t, udata, "  - CriticalAddonsOnly=true:NoExecute")
+	// The control plane runs no agent, so it is never a Node and needs no taint.
+	assert.Contains(t, udata, "disable-agent: true")
+	assert.NotContains(t, udata, "node-taint:")
 	// AWS parity: K3s' bundled traefik + servicelb are always disabled; Service
 	// type=LoadBalancer / Ingress are the AWS LB Controller's job. local-storage is
 	// disabled too so its local-path provisioner does not add a second default
@@ -376,6 +376,7 @@ func NewRegistration(deps Deps) dispatch.Registration {
 		ResolveAction: resolveAction,
 		Dispatch:      deps.dispatch,
 		Errors:        dispatch.ErrorEnvelopeXML,
+		Traits:        dispatch.ServiceTraits{XMLThrottleCode: awserrors.ErrorThrottling},
 		Inventory:     OperationInventory(),
 	}
 }
@@ -56,10 +56,9 @@ type IGWManager interface {
 	RebindGatewayIP(ctx context.Context, vpcID, newIP string, prefixLen int) error
 }

-// RoutedIngressHooks deliver host-side ingress plumbing in routed-NAT mode:
-// Ensure installs the host route sending VPC-CIDR traffic into OVN via the
-// gateway LRP transit IP; Remove tears it down on detach. Both idempotent.
-// Nil hooks (all other modes) are never called.
+// RoutedIngressHooks report routed-NAT ingress changes to the host: Ensure when
+// a VPC's gateway transit IP is set or re-asserted, Remove on detach. Both must
+// be idempotent. Nil hooks (all other modes) are never called.
 type RoutedIngressHooks struct {
 	Ensure func(ctx context.Context, vpcID, vpcCIDR, gwLrpIP string) error
 	Remove func(ctx context.Context, vpcID, vpcCIDR, gwLrpIP string) error
@@ -295,7 +294,7 @@ func (m *igwManager) AttachIGW(ctx context.Context, spec IGWSpec) error {
 // ensureRoutedLeg installs the routed-mode datapath legs for a VPC: the OVN
 // SNAT (VPC CIDR → gateway LRP transit IP) and the host ingress route into
 // it. No-op outside routed mode or without a gateway IP. Idempotent — the
-// reconcile loop replays AttachIGW, re-ensuring host state lost on reboot.
+// reconcile loop replays AttachIGW to re-ensure the SNAT.
 func (m *igwManager) ensureRoutedLeg(ctx context.Context, vpcID, gwLrpIP string) error {
 	if m.natMode != policy.NATModeRouted || gwLrpIP == "" {
 		return nil
@@ -4,9 +4,15 @@ import (
 	"context"
 	"fmt"
 	"log/slog"
+	"net/netip"
 	"strings"
 )

+// VPCIngressRouteProto tags every VPC ingress route this package installs, so
+// the routes it may prune are told apart from the transit /24 and the EIP /32s
+// that share the spx-nat veth. Unassigned in iproute2's rt_protos.
+const VPCIngressRouteProto = "201"
+
 // EnsureVPCIngressRoute installs the host route that makes a VPC's private IPs
 // reachable in routed-NAT mode: traffic to the VPC CIDR enters OVN via the
 // gateway LRP's transit IP on the spx-nat veth. Idempotent (route replace).
@@ -18,44 +24,72 @@ func EnsureVPCIngressRoute(ctx context.Context, r Runner, vpcCIDR, gwLrpIP strin
 		return fmt.Errorf("EnsureVPCIngressRoute: gwLrpIP required")
 	}
 	if out, err := r.Run(ctx, "ip", "route", "replace", vpcCIDR,
-		"via", gwLrpIP, "dev", NATTransitHostEnd); err != nil {
+		"via", gwLrpIP, "dev", NATTransitHostEnd, "proto", VPCIngressRouteProto); err != nil {
 		return fmt.Errorf("install VPC ingress route %s via %s: %s: %w", vpcCIDR, gwLrpIP, string(out), err)
 	}
 	slog.Info("VPC ingress route installed", "vpc_cidr", vpcCIDR, "gw_lrp_ip", gwLrpIP)
 	return nil
 }

-// VPCIngressRouteVia returns the gateway IP of the existing transit-veth host
-// route for a VPC CIDR, or "" when none exists. Lets callers detect a
-// duplicate-CIDR collision (two VPCs sharing e.g. 172.31.0.0/16) before
-// replacing or deleting a route another VPC installed.
-func VPCIngressRouteVia(ctx context.Context, r Runner, vpcCIDR string) (string, error) {
-	if vpcCIDR == "" {
-		return "", fmt.Errorf("VPCIngressRouteVia: vpcCIDR required")
-	}
-	out, err := r.Run(ctx, "ip", "route", "show", vpcCIDR, "dev", NATTransitHostEnd)
+// ListVPCIngressRoutes returns the VPC ingress routes on this host, VPC CIDR to
+// gateway LRP transit IP. Only routes carrying VPCIngressRouteProto are listed.
+func ListVPCIngressRoutes(ctx context.Context, r Runner) (map[string]string, error) {
+	out, err := r.Run(ctx, "ip", "-o", "route", "show", "dev", NATTransitHostEnd, "proto", VPCIngressRouteProto)
 	if err != nil {
-		return "", fmt.Errorf("show VPC ingress route %s: %s: %w", vpcCIDR, string(out), err)
+		return nil, fmt.Errorf("list VPC ingress routes: %s: %w", string(out), err)
 	}
-	fields := strings.Fields(string(out))
-	for i, f := range fields {
-		if f == "via" && i+1 < len(fields) {
-			return fields[i+1], nil
+	routes := map[string]string{}
+	for line := range strings.SplitSeq(string(out), "\n") {
+		fields := strings.Fields(line)
+		if len(fields) == 0 {
+			continue
 		}
+		dst, err := netip.ParsePrefix(fields[0])
+		if err != nil {
+			slog.Warn("VPC ingress route list: unparseable destination", "line", strings.TrimSpace(line))
+			continue
+		}
+		via := ""
+		for i, f := range fields {
+			if f == "via" && i+1 < len(fields) {
+				via = fields[i+1]
+			}
+		}
+		routes[dst.Masked().String()] = via
 	}
-	return "", nil
+	return routes, nil
 }

-// RemoveVPCIngressRoute deletes the host route for a VPC CIDR on IGW detach.
-// A missing route is not an error (idempotent teardown).
+// RemoveVPCIngressRoute deletes the host route for a VPC CIDR. A missing route
+// is not an error (idempotent teardown).
 func RemoveVPCIngressRoute(ctx context.Context, r Runner, vpcCIDR string) error {
 	if vpcCIDR == "" {
 		return fmt.Errorf("RemoveVPCIngressRoute: vpcCIDR required")
 	}
-	if _, err := r.Run(ctx, "ip", "route", "del", vpcCIDR, "dev", NATTransitHostEnd); err != nil {
+	if _, err := r.Run(ctx, "ip", "route", "del", vpcCIDR, "dev", NATTransitHostEnd, "proto", VPCIngressRouteProto); err != nil {
 		slog.Debug("VPC ingress route already absent", "vpc_cidr", vpcCIDR, "err", err)
 		return nil
 	}
 	slog.Info("VPC ingress route removed", "vpc_cidr", vpcCIDR)
 	return nil
 }
+
+// VPCIngressRoutes is this host's VPC ingress route table behind a Runner.
+type VPCIngressRoutes struct {
+	Runner Runner
+}
+
+// List is ListVPCIngressRoutes.
+func (v VPCIngressRoutes) List(ctx context.Context) (map[string]string, error) {
+	return ListVPCIngressRoutes(ctx, v.Runner)
+}
+
+// Ensure is EnsureVPCIngressRoute.
+func (v VPCIngressRoutes) Ensure(ctx context.Context, vpcCIDR, gwLrpIP string) error {
+	return EnsureVPCIngressRoute(ctx, v.Runner, vpcCIDR, gwLrpIP)
+}
+
+// Remove is RemoveVPCIngressRoute.
+func (v VPCIngressRoutes) Remove(ctx context.Context, vpcCIDR string) error {
+	return RemoveVPCIngressRoute(ctx, v.Runner, vpcCIDR)
+}
@@ -14,7 +14,7 @@ func TestEnsureVPCIngressRoute_InstallsRoute(t *testing.T) {
 	if err := EnsureVPCIngressRoute(context.Background(), r, "10.0.0.0/16", "100.127.0.10"); err != nil {
 		t.Fatalf("EnsureVPCIngressRoute: %v", err)
 	}
-	want := "ip route replace 10.0.0.0/16 via 100.127.0.10 dev " + NATTransitHostEnd
+	want := "ip route replace 10.0.0.0/16 via 100.127.0.10 dev " + NATTransitHostEnd + " proto " + VPCIngressRouteProto
 	if !r.called(want) {
 		t.Errorf("missing route call:\n  want %q\n  got  %v", want, r.calls)
 	}
@@ -43,43 +43,52 @@ func TestEnsureVPCIngressRoute_RouteFailure(t *testing.T) {
 	}
 }

-func TestVPCIngressRouteVia_ParsesHolder(t *testing.T) {
+// The list is what the per-node pass prunes from, so it must carry only the
+// tagged routes and key them exactly as intent renders a VPC CIDR.
+func TestListVPCIngressRoutes_ParsesTaggedRoutes(t *testing.T) {
 	r := newStubRunner()
-	r.expect("ip route show", []byte("172.31.0.0/16 via 100.127.0.239 dev spx-nat-host \n"), nil)
+	r.expect("ip -o route show", []byte("172.31.0.0/16 via 100.127.0.16 \n10.1.0.0/16 via 100.127.0.17 \n"), nil)

-	via, err := VPCIngressRouteVia(context.Background(), r, "172.31.0.0/16")
+	routes, err := ListVPCIngressRoutes(context.Background(), r)
 	if err != nil {
-		t.Fatalf("VPCIngressRouteVia: %v", err)
+		t.Fatalf("ListVPCIngressRoutes: %v", err)
 	}
-	if via != "100.127.0.239" {
-		t.Errorf("via = %q, want 100.127.0.239", via)
+	want := map[string]string{"172.31.0.0/16": "100.127.0.16", "10.1.0.0/16": "100.127.0.17"}
+	if len(routes) != len(want) {
+		t.Fatalf("routes = %v, want %v", routes, want)
 	}
-	want := "ip route show 172.31.0.0/16 dev " + NATTransitHostEnd
-	if !r.called(want) {
-		t.Errorf("missing route show call:\n  want %q\n  got  %v", want, r.calls)
+	for cidr, via := range want {
+		if routes[cidr] != via {
+			t.Errorf("routes[%s] = %q, want %q", cidr, routes[cidr], via)
+		}
+	}
+	call := "ip -o route show dev " + NATTransitHostEnd + " proto " + VPCIngressRouteProto
+	if !r.called(call) {
+		t.Errorf("missing list call:\n  want %q\n  got  %v", call, r.calls)
 	}
 }

-func TestVPCIngressRouteVia_NoRoute(t *testing.T) {
+func TestListVPCIngressRoutes_Empty(t *testing.T) {
 	r := newStubRunner()
-	r.expect("ip route show", nil, nil)
+	r.expect("ip -o route show", nil, nil)

-	via, err := VPCIngressRouteVia(context.Background(), r, "172.31.0.0/16")
+	routes, err := ListVPCIngressRoutes(context.Background(), r)
 	if err != nil {
-		t.Fatalf("VPCIngressRouteVia: %v", err)
+		t.Fatalf("ListVPCIngressRoutes: %v", err)
 	}
-	if via != "" {
-		t.Errorf("via = %q, want empty for absent route", via)
+	if len(routes) != 0 {
+		t.Errorf("routes = %v, want none", routes)
 	}
 }

-func TestVPCIngressRouteVia_ValidatesArgs(t *testing.T) {
+// A failed list must surface: read as empty it would prune nothing, but read as
+// a partial table the caller could not tell what it was missing.
+func TestListVPCIngressRoutes_Failure(t *testing.T) {
 	r := newStubRunner()
-	if _, err := VPCIngressRouteVia(context.Background(), r, ""); err == nil {
-		t.Error("empty vpcCIDR: expected error")
-	}
-	if len(r.calls) != 0 {
-		t.Errorf("validation failures must not run commands: %v", r.calls)
+	r.expect("ip -o route show", []byte("Cannot find device"), fmt.Errorf("exit 1"))
+
+	if _, err := ListVPCIngressRoutes(context.Background(), r); err == nil {
+		t.Fatal("expected list failure to be returned")
 	}
 }

@@ -90,7 +99,7 @@ func TestRemoveVPCIngressRoute_DeletesRoute(t *testing.T) {
 	if err := RemoveVPCIngressRoute(context.Background(), r, "10.0.0.0/16"); err != nil {
 		t.Fatalf("RemoveVPCIngressRoute: %v", err)
 	}
-	want := "ip route del 10.0.0.0/16 dev " + NATTransitHostEnd
+	want := "ip route del 10.0.0.0/16 dev " + NATTransitHostEnd + " proto " + VPCIngressRouteProto
 	if !r.called(want) {
 		t.Errorf("missing route del call:\n  want %q\n  got  %v", want, r.calls)
 	}
@@ -0,0 +1,168 @@
+package reconcile
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"log/slog"
+	"time"
+
+	"github.com/mulgadc/spinifex/spinifex/domains/network/ovn"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/policy"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/topology"
+	loop "github.com/mulgadc/spinifex/spinifex/foundation/lifecycle/reconciler"
+	"github.com/nats-io/nats.go/jetstream"
+)
+
+// Host ingress cadence. The resync is short because a gateway IP rebind after a
+// DHCP lease move writes NB, not KV, so no watch announces it. The retry covers
+// an IGW whose gateway LRP the attaching node has not created yet.
+const (
+	defaultHostIngressResync = 30 * time.Second
+	hostIngressPendingRetry  = 5 * time.Second
+)
+
+// HostIngressRoutes is one host's table of routed-NAT VPC ingress routes, VPC
+// CIDR to the gateway LRP transit IP the traffic enters OVN through.
+type HostIngressRoutes interface {
+	// List returns only the routes Ensure installed, keyed by CIDR.
+	List(ctx context.Context) (map[string]string, error)
+	Ensure(ctx context.Context, cidr, via string) error
+	Remove(ctx context.Context, cidr string) error
+}
+
+// HostIngress converges this host's VPC ingress routes from intent. Every node
+// runs one, without the drift lease: the routes are the host's own, and the
+// attach event that used to install them reaches one vpcd of the queue group.
+type HostIngress struct {
+	// OVN is read only, for each attached VPC's gateway LRP transit IP.
+	OVN    ovn.Client
+	Routes HostIngressRoutes
+	// LoadIntent reads intent fresh each pass. An error abandons the pass, so
+	// an unreadable store never prunes a route.
+	LoadIntent func(ctx context.Context) (IntentState, error)
+	// PreferredVPC wins a CIDR shared by several attached VPCs; see ingressOutranks.
+	PreferredVPC string
+	// Resync is the backstop between passes. Zero means 30s.
+	Resync time.Duration
+}
+
+// hostIngressPlan is the route table a pass converges to. held names CIDRs whose
+// winning VPC has no resolvable gateway IP yet: whatever route they have is kept
+// as it is, since the winner is known but where it enters OVN is not.
+type hostIngressPlan struct {
+	want    map[string]string
+	held    map[string]struct{}
+	pending bool
+}
+
+// ingressOutranks reports whether VPC a takes a shared CIDR from VPC b: the
+// preferred VPC first, then the lower VPC ID. Every node ranks the same intent
+// the same way, so the holder never depends on which node saw which event.
+func ingressOutranks(a, b, preferred string) bool {
+	if a == preferred || b == preferred {
+		return a == preferred && b != preferred
+	}
+	return a < b
+}
+
+// planHostIngress picks one gateway per CIDR across the attached VPCs in intent.
+// gatewayIP is asked only about each CIDR's winner; "" marks it pending.
+func planHostIngress(intent IntentState, preferred string, gatewayIP func(vpcID string) string) hostIngressPlan {
+	winners := map[string]string{}
+	for vpcID := range intent.IGWs {
+		vpc, ok := intent.VPCs[vpcID]
+		if !ok || !vpc.CIDR.IsValid() {
+			continue
+		}
+		cidr := vpc.CIDR.Masked().String()
+		if cur, ok := winners[cidr]; !ok || ingressOutranks(vpcID, cur, preferred) {
+			winners[cidr] = vpcID
+		}
+	}
+	plan := hostIngressPlan{want: map[string]string{}, held: map[string]struct{}{}}
+	for cidr, vpcID := range winners {
+		if gw := gatewayIP(vpcID); gw != "" {
+			plan.want[cidr] = gw
+			continue
+		}
+		plan.held[cidr] = struct{}{}
+		plan.pending = true
+	}
+	return plan
+}
+
+// gatewayTransitIP is the stamped gateway IP of vpcID's gateway LRP, falling
+// back to its network address. "" when the port is absent or unreadable.
+func (h *HostIngress) gatewayTransitIP(ctx context.Context, vpcID string) string {
+	lrp, err := h.OVN.GetLogicalRouterPort(ctx, topology.GatewayRouterPort(vpcID))
+	if err != nil || lrp == nil {
+		return ""
+	}
+	if ip := lrp.ExternalIDs[policy.GatewayIPExtIDKey]; ip != "" {
+		return ip
+	}
+	return gatewayLRPIP(lrp)
+}
+
+// Pass converges the routes once. It asks to be revisited while a winning VPC's
+// gateway is still being created; the loop's resync covers everything else.
+func (h *HostIngress) Pass(ctx context.Context) (time.Duration, error) {
+	intent, err := h.LoadIntent(ctx)
+	if err != nil {
+		return 0, fmt.Errorf("load intent for host ingress routes: %w", err)
+	}
+	have, err := h.Routes.List(ctx)
+	if err != nil {
+		return 0, err
+	}
+	plan := planHostIngress(intent, h.PreferredVPC, func(vpcID string) string {
+		return h.gatewayTransitIP(ctx, vpcID)
+	})
+
+	var errs []error
+	for cidr, via := range plan.want {
+		if have[cidr] == via {
+			continue
+		}
+		if err := h.Routes.Ensure(ctx, cidr, via); err != nil {
+			errs = append(errs, err)
+		}
+	}
+	for cidr, via := range have {
+		_, wanted := plan.want[cidr]
+		_, held := plan.held[cidr]
+		if wanted || held {
+			continue
+		}
+		slog.InfoContext(ctx, "reconcile/host-ingress: pruning route no attached VPC holds",
+			"vpc_cidr", cidr, "gw_lrp_ip", via)
+		if err := h.Routes.Remove(ctx, cidr); err != nil {
+			errs = append(errs, err)
+		}
+	}
+
+	var revisit time.Duration
+	if plan.pending {
+		revisit = hostIngressPendingRetry
+	}
+	return revisit, errors.Join(errs...)
+}
+
+// RunHostIngress runs h on every intent change, every nudge and every
+// resync until ctx ends. nudge carries the attach and detach events
+// this node handled, which can land before the gateway they announce.
+func RunHostIngress(ctx context.Context, js jetstream.JetStream, h *HostIngress, nudge <-chan struct{}) {
+	resync := h.Resync
+	if resync <= 0 {
+		resync = defaultHostIngressResync
+	}
+	loop.Run(ctx, loop.Config{
+		Name:      "vpcd/host-ingress",
+		Sources:   []loop.Source{intentSource(js)},
+		Reconcile: h.Pass,
+		Trigger:   nudge,
+		Resync:    resync,
+		Debounce:  driftDebounce,
+	})
+}
@@ -0,0 +1,439 @@
+package reconcile_test
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"maps"
+	"net/netip"
+	"sync"
+	"testing"
+	"time"
+
+	testutil "github.com/mulgadc/spinifex/internal/testkit"
+	ec2igw "github.com/mulgadc/spinifex/spinifex/domains/ec2/igw"
+	ec2vpc "github.com/mulgadc/spinifex/spinifex/domains/ec2/vpc"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/external"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/ovn/mock"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/ovn/nbdb"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/policy"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/reconcile"
+	"github.com/mulgadc/spinifex/spinifex/domains/network/topology"
+	"github.com/mulgadc/spinifex/spinifex/foundation/state/kvutil"
+	"github.com/nats-io/nats.go/jetstream"
+)
+
+// hostRoutes is one node's host route table.
+type hostRoutes struct {
+	mu     sync.Mutex
+	routes map[string]string
+	lists  int
+}
+
+func newHostRoutes(seed map[string]string) *hostRoutes {
+	r := &hostRoutes{routes: map[string]string{}}
+	maps.Copy(r.routes, seed)
+	return r
+}
+
+func (h *hostRoutes) List(context.Context) (map[string]string, error) {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	h.lists++
+	return maps.Clone(h.routes), nil
+}
+
+func (h *hostRoutes) Ensure(_ context.Context, cidr, via string) error {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	h.routes[cidr] = via
+	return nil
+}
+
+func (h *hostRoutes) Remove(_ context.Context, cidr string) error {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	delete(h.routes, cidr)
+	return nil
+}
+
+func (h *hostRoutes) listCount() int {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	return h.lists
+}
+
+func (h *hostRoutes) snapshot() map[string]string {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	return maps.Clone(h.routes)
+}
+
+// clusterIntent is the control-plane store every node reads.
+type clusterIntent struct {
+	mu   sync.Mutex
+	vpcs map[string]topology.VPCSpec
+	igws map[string]external.IGWSpec
+	err  error
+}
+
+func newClusterIntent() *clusterIntent {
+	return &clusterIntent{vpcs: map[string]topology.VPCSpec{}, igws: map[string]external.IGWSpec{}}
+}
+
+func (c *clusterIntent) addVPC(id, cidr string) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	c.vpcs[id] = topology.VPCSpec{VPCID: id, CIDR: netip.MustParsePrefix(cidr)}
+}
+
+func (c *clusterIntent) attach(vpcID, igwID string) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	c.igws[vpcID] = external.IGWSpec{VPCID: vpcID, InternetGatewayID: igwID}
+}
+
+func (c *clusterIntent) detach(vpcID string) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	delete(c.igws, vpcID)
+}
+
+func (c *clusterIntent) load(context.Context) (reconcile.IntentState, error) {
+	c.mu.Lock()
+	defer c.mu.Unlock()
+	if c.err != nil {
+		return reconcile.IntentState{}, c.err
+	}
+	return reconcile.IntentState{VPCs: maps.Clone(c.vpcs), IGWs: maps.Clone(c.igws)}, nil
+}
+
+// node is one hypervisor's vpcd as far as host ingress goes.
+type node struct {
+	routes *hostRoutes
+	ingest *reconcile.HostIngress
+}
+
+func newNode(m *mock.Client, intent *clusterIntent, preferred string, seed map[string]string) *node {
+	routes := newHostRoutes(seed)
+	return &node{routes: routes, ingest: &reconcile.HostIngress{
+		OVN: m, Routes: routes, LoadIntent: intent.load, PreferredVPC: preferred,
+	}}
+}
+
+func (n *node) pass(t *testing.T) time.Duration {
+	t.Helper()
+	revisit, err := n.ingest.Pass(t.Context())
+	if err != nil {
+		t.Fatalf("host ingress pass: %v", err)
+	}
+	return revisit
+}
+
+func seedRouter(t *testing.T, m *mock.Client, vpcID, cidr string) {
+	t.Helper()
+	if err := m.CreateLogicalRouter(t.Context(), &nbdb.LogicalRouter{
+		Name:        topology.VPCRouter(vpcID),
+		ExternalIDs: map[string]string{"spinifex:vpc_id": vpcID, "spinifex:cidr": cidr},
+	}); err != nil {
+		t.Fatalf("seed router %s: %v", vpcID, err)
+	}
+}
+
+// seedGateway stands in for the gateway LRP an attach creates in NB.
+func seedGateway(t *testing.T, m *mock.Client, vpcID, gwIP string) {
+	t.Helper()
+	if err := m.CreateLogicalRouterPort(t.Context(), topology.VPCRouter(vpcID), &nbdb.LogicalRouterPort{
+		Name:        topology.GatewayRouterPort(vpcID),
+		Networks:    []string{gwIP + "/24"},
+		ExternalIDs: map[string]string{policy.GatewayIPExtIDKey: gwIP},
+	}); err != nil {
+		t.Fatalf("seed gateway %s: %v", vpcID, err)
+	}
+}
+
+// routedIGW is the IGW manager on the node the queue group hands an event to.
+// Its hooks only nudge, as in vpcd.
+func routedIGW(t *testing.T, m *mock.Client, nudges *int) external.IGWManager {
+	t.Helper()
+	nat, err := policy.NewNATManager(m, policy.NATModeRouted,
+		policy.WithSNATExemptSet(policy.NATExemptSetName, []string{"100.127.0.0/24"}))
+	if err != nil {
+		t.Fatalf("NewNATManager: %v", err)
+	}
+	nudge := func(context.Context, string, string, string) error { *nudges++; return nil }
+	mgr, err := external.NewIGWManager(external.IGWManagerConfig{
+		OVN: m, Routes: policy.NewRouteManager(m), NAT: nat,
+		Pool: &external.ExternalPoolConfig{
+			Name: "nat-transit", Gateway: "100.127.0.1", PrefixLen: 24,
+			GwLrpRangeStart: "100.127.0.240", GwLrpRangeEnd: "100.127.0.243",
+		},
+		Allocator: external.NewStaticRangeAllocator(m),
+		Chassis:   []string{"chassis-a"}, NATMode: policy.NATModeRouted,
+		RoutedIngress: &external.RoutedIngressHooks{Ensure: nudge, Remove: nudge},
+	})
+	if err != nil {
+		t.Fatalf("NewIGWManager: %v", err)
+	}
+	return mgr
+}
+
+func assertRoutes(t *testing.T, name string, got, want map[string]string) {
+	t.Helper()
+	if !maps.Equal(got, want) {
+		t.Errorf("%s routes = %v, want %v", name, got, want)
+	}
+}
+
+// The attach event reaches one vpcd of the queue group. Every other node must
+// still get the route from intent, or a host that neither handled the event
+// nor won a drift lease has no path into the VPC until a full resync.
+func TestHostIngress_AttachHandledOnOneNodeConvergesEveryNode(t *testing.T) {
+	m := mock.New()
+	intent := newClusterIntent()
+	seedRouter(t, m, "vpc-1", "10.1.0.0/16")
+	intent.addVPC("vpc-1", "10.1.0.0/16")
+	a := newNode(m, intent, "", nil)
+	b := newNode(m, intent, "", nil)
+
+	intent.attach("vpc-1", "igw-1")
+	nudges := 0
+	if err := routedIGW(t, m, &nudges).AttachIGW(t.Context(), external.IGWSpec{VPCID: "vpc-1", InternetGatewayID: "igw-1"}); err != nil {
+		t.Fatalf("AttachIGW on node A: %v", err)
+	}
+	if nudges == 0 {
+		t.Fatal("attach on node A never nudged its host ingress loop")
+	}
+
+	a.pass(t)
+	// Node B saw only the intent change, never the event.
+	b.pass(t)
+	want := map[string]string{"10.1.0.0/16": "100.127.0.240"}
+	assertRoutes(t, "node A", a.routes.snapshot(), want)
+	assertRoutes(t, "node B", b.routes.snapshot(), want)
+}
+
+// Detach has the same reach as attach, so the route must leave every node, not
+// only the one that ran DetachIGW.
+func TestHostIngress_DetachPrunesOnEveryNode(t *testing.T) {
+	m := mock.New()
+	intent := newClusterIntent()
+	seedRouter(t, m, "vpc-1", "10.1.0.0/16")
+	intent.addVPC("vpc-1", "10.1.0.0/16")
+	intent.attach("vpc-1", "igw-1")
+	nudges := 0
+	mgr := routedIGW(t, m, &nudges)
+	if err := mgr.AttachIGW(t.Context(), external.IGWSpec{VPCID: "vpc-1", InternetGatewayID: "igw-1"}); err != nil {
+		t.Fatalf("AttachIGW: %v", err)
+	}
+	nodes := []*node{newNode(m, intent, "", nil), newNode(m, intent, "", nil), newNode(m, intent, "", nil)}
+	for _, n := range nodes {
+		n.pass(t)
+		assertRoutes(t, "attached", n.routes.snapshot(), map[string]string{"10.1.0.0/16": "100.127.0.240"})
+	}
+
+	intent.detach("vpc-1")
+	if err := mgr.DetachIGW(t.Context(), "vpc-1"); err != nil {
+		t.Fatalf("DetachIGW on node A: %v", err)
+	}
+	for i, n := range nodes {
+		n.pass(t)
+		if got := n.routes.snapshot(); len(got) != 0 {
+			t.Errorf("node %d kept %v after detach, want no ingress route", i, got)
+		}
+	}
+}
+
+// Every account's default VPC is 172.31.0.0/16, so two attached VPCs contend
+// for one host route. Each node must pick the same holder whatever route it
+// had before and whichever order it met the VPCs in.
+func TestHostIngress_SharedCIDRPicksTheSameHolderOnEveryNode(t *testing.T) {
+	cases := []struct {
+		name      string
+		preferred string
+		wantVia   string
+	}{
+		{"preferred VPC wins over a lower ID", "vpc-b", "100.127.0.17"},
+		{"lower VPC ID wins without a preferred VPC", "", "100.127.0.16"},
+		{"a preferred VPC outside the contest changes nothing", "vpc-other", "100.127.0.16"},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			m := mock.New()
+			intent := newClusterIntent()
+			for vpcID, gw := range map[string]string{"vpc-a": "100.127.0.16", "vpc-b": "100.127.0.17"} {
+				seedRouter(t, m, vpcID, "172.31.0.0/16")
+				seedGateway(t, m, vpcID, gw)
+				intent.addVPC(vpcID, "172.31.0.0/16")
+				intent.attach(vpcID, "igw-"+vpcID)
+			}
+			want := map[string]string{"172.31.0.0/16": tc.wantVia}
+			// Seeded with each holder, as if the event for either VPC had
+			// landed on that node first; repeated so map order cannot hide a
+			// choice that depends on it.
+			for _, seed := range []string{"", "100.127.0.16", "100.127.0.17"} {
+				for range 20 {
+					var initial map[string]string
+					if seed != "" {
+						initial = map[string]string{"172.31.0.0/16": seed}
+					}
+					n := newNode(m, intent, tc.preferred, initial)
+					n.pass(t)
+					assertRoutes(t, "seed "+seed, n.routes.snapshot(), want)
+				}
+			}
+		})
+	}
+}
+
+// Until the winning VPC's gateway exists the route is left alone: switching it
+// to the loser and back would cut the CIDR over twice for nothing.
+func TestHostIngress_PendingWinnerHoldsTheRouteAndRevisits(t *testing.T) {
+	m := mock.New()
+	intent := newClusterIntent()
+	seedRouter(t, m, "vpc-a", "172.31.0.0/16")
+	seedRouter(t, m, "vpc-b", "172.31.0.0/16")
+	seedGateway(t, m, "vpc-b", "100.127.0.17")
+	intent.addVPC("vpc-a", "172.31.0.0/16")
+	intent.addVPC("vpc-b", "172.31.0.0/16")
+	intent.attach("vpc-a", "igw-a")
+	intent.attach("vpc-b", "igw-b")
+	n := newNode(m, intent, "vpc-a", map[string]string{"172.31.0.0/16": "100.127.0.9"})
+
+	if revisit := n.pass(t); revisit <= 0 {
+		t.Errorf("revisit = %v, want a short retry while vpc-a's gateway is missing", revisit)
+	}
+	assertRoutes(t, "pending", n.routes.snapshot(), map[string]string{"172.31.0.0/16": "100.127.0.9"})
+
+	seedGateway(t, m, "vpc-a", "100.127.0.16")
+	if revisit := n.pass(t); revisit != 0 {
+		t.Errorf("revisit = %v after convergence, want 0", revisit)
+	}
+	assertRoutes(t, "converged", n.routes.snapshot(), map[string]string{"172.31.0.0/16": "100.127.0.16"})
+}
+
+// A restarted vpcd has no memory of the events it handled. Its first pass must
+// rebuild the table from intent: repoint a route left on a stale gateway, add
+// one it never had, and prune one whose VPC detached while it was down.
+func TestHostIngress_RestartConvergesFromIntent(t *testing.T) {
+	m := mock.New()
+	intent := newClusterIntent()
+	for vpcID, v := range map[string][2]string{
+		"vpc-1": {"10.1.0.0/16", "100.127.0.20"},
+		"vpc-2": {"10.2.0.0/16", "100.127.0.21"},
+	} {
+		seedRouter(t, m, vpcID, v[0])
+		seedGateway(t, m, vpcID, v[1])
+		intent.addVPC(vpcID, v[0])
+		intent.attach(vpcID, "igw-"+vpcID)
+	}
+	n := newNode(m, intent, "", map[string]string{
+		"10.1.0.0/16": "100.127.0.99",
+		"10.9.0.0/16": "100.127.0.30",
+	})
+
+	n.pass(t)
+	assertRoutes(t, "after restart", n.routes.snapshot(), map[string]string{
+		"10.1.0.0/16": "100.127.0.20",
+		"10.2.0.0/16": "100.127.0.21",
+	})
+}
+
+// An unreadable store is not an empty one: reading it as empty would strip
+// every VPC's ingress route from every node at once.
+func TestHostIngress_IntentReadFailureNeverPrunes(t *testing.T) {
+	m := mock.New()
+	intent := newClusterIntent()
+	intent.err = errors.New("kv unavailable")
+	seed := map[string]string{"10.1.0.0/16": "100.127.0.20"}
+	n := newNode(m, intent, "", seed)
+
+	if _, err := n.ingest.Pass(t.Context()); err == nil {
+		t.Fatal("expected the intent read failure to be returned")
+	}
+	assertRoutes(t, "after failed read", n.routes.snapshot(), seed)
+}
+
+// The loop has to wake on an intent write by itself, on every node at once,
+// while another node holds the drift lease and no event arrives, well inside
+// its resync.
+func TestRunHostIngress_IntentWriteConvergesEveryNodeWithoutLeaseOrEvent(t *testing.T) {
+	_, nc, js := testutil.StartTestJetStream(t)
+	ctx := t.Context()
+	vpcs, err := kvutil.GetOrCreateBucket(ctx, js, ec2vpc.KVBucketVPCs, 1)
+	if err != nil {
+		t.Fatalf("create VPC bucket: %v", err)
+	}
+	igws, err := kvutil.GetOrCreateBucket(ctx, js, ec2igw.KVBucketIGW, 1)
+	if err != nil {
+		t.Fatalf("create IGW bucket: %v", err)
+	}
+	put(t, vpcs, "acct/vpc-1", ec2vpc.VPCRecord{VpcId: "vpc-1", CidrBlock: "10.1.0.0/16"})
+	release, elected := reconcile.AcquireLeader(ctx, nc, reconcile.KVBucketVPCDReconcile, "node-elsewhere")
+	if !elected {
+		t.Fatal("could not take the drift lease for another node")
+	}
+	t.Cleanup(release)
+
+	m := mock.New()
+	seedRouter(t, m, "vpc-1", "10.1.0.0/16")
+	seedGateway(t, m, "vpc-1", "100.127.0.20")
+	loopCtx, cancel := context.WithCancel(ctx)
+	var wg sync.WaitGroup
+	t.Cleanup(func() {
+		cancel()
+		wg.Wait()
+	})
+	nodes := make([]*hostRoutes, 3)
+	for i := range nodes {
+		nodes[i] = newHostRoutes(nil)
+		h := &reconcile.HostIngress{
+			OVN: m, Routes: nodes[i], Resync: time.Minute,
+			LoadIntent: func(ctx context.Context) (reconcile.IntentState, error) {
+				return reconcile.LoadIntentFromKV(ctx, js, "us-east-1a")
+			},
+		}
+		wg.Go(func() { reconcile.RunHostIngress(loopCtx, js, h, nil) })
+	}
+	// The startup pass has to be over before the attach is written, or it could
+	// be what converged a node rather than the watch.
+	for _, n := range nodes {
+		for n.listCount() == 0 {
+			time.Sleep(10 * time.Millisecond)
+		}
+	}
+
+	want := map[string]string{"10.1.0.0/16": "100.127.0.20"}
+	converged := func() bool {
+		for _, n := range nodes {
+			if !maps.Equal(n.snapshot(), want) {
+				return false
+			}
+		}
+		return true
+	}
+	// Watchers attach asynchronously and see updates only, so the write is
+	// repeated until one lands rather than racing the loop's startup.
+	deadline := time.Now().Add(10 * time.Second)
+	for !converged() {
+		if time.Now().After(deadline) {
+			for i, n := range nodes {
+				t.Errorf("node %d routes = %v 10s after the attach was written, want %v", i, n.snapshot(), want)
+			}
+			t.FailNow()
+		}
+		put(t, igws, "acct/igw-1", ec2igw.IGWRecord{InternetGatewayId: "igw-1", VpcId: "vpc-1", State: "available"})
+		time.Sleep(500 * time.Millisecond)
+	}
+}
+
+func put(t *testing.T, kv jetstream.KeyValue, key string, rec any) {
+	t.Helper()
+	b, err := json.Marshal(rec)
+	if err != nil {
+		t.Fatalf("marshal %s: %v", key, err)
+	}
+	if _, err := kv.Put(t.Context(), key, b); err != nil {
+		t.Fatalf("put %s: %v", key, err)
+	}
+}
@@ -89,6 +89,7 @@ func NewRegistration(deps Deps) dispatch.Registration {
 		ResolveAction: resolveAction,
 		Dispatch:      deps.dispatch,
 		Errors:        dispatch.ErrorEnvelopeXML,
+		Traits:        dispatch.ServiceTraits{XMLThrottleCode: awserrors.ErrorThrottling, GlobalSigningRegion: "us-east-1"},
 		Inventory:     OperationInventory(),
 	}
 }
@@ -0,0 +1,32 @@
+package awsapi
+
+import (
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+)
+
+// TestAnonymousArgs_Classification covers the body-peek helper for anonymous action detection.
+func TestAnonymousArgs_Classification(t *testing.T) {
+	t.Run("anonymous action recognised, body restored", func(t *testing.T) {
+		const body = "Action=AssumeRoleWithWebIdentity&WebIdentityToken=tok"
+		req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
+		args, ok := anonymousArgs(req)
+		require.True(t, ok)
+		assert.Equal(t, "AssumeRoleWithWebIdentity", args["Action"])
+
+		restored, _ := io.ReadAll(req.Body)
+		assert.Equal(t, body, string(restored), "body must be restored for downstream read")
+	})
+
+	t.Run("non-anonymous action rejected", func(t *testing.T) {
+		req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=GetCallerIdentity"))
+		_, ok := anonymousArgs(req)
+		assert.False(t, ok)
+	})
+}
@@ -107,6 +107,7 @@ func NewRegistration(deps Deps) dispatch.Registration {
 		ResolveAction: resolveAction,
 		Dispatch:      deps.dispatch,
 		Errors:        dispatch.ErrorEnvelopeXML,
+		Traits:        dispatch.ServiceTraits{XMLThrottleCode: awserrors.ErrorThrottling},
 		Inventory:     OperationInventory(),
 	}
 }
@@ -138,6 +139,63 @@ func NewAnonymousAssumeRoleWithWebIdentity(deps Deps) AnonymousFunc {
 	}
 }

+// anonymousActions lists the STS actions that carry no SigV4; a web-identity
+// token in the body authenticates them.
+var anonymousActions = map[string]bool{
+	"AssumeRoleWithWebIdentity": true,
+}
+
+// NewPreAuth serves the unsigned web-identity exchange ahead of SigV4. A
+// request with an Authorization header, a body over the payload cap or any
+// other action is left to the signed chain.
+func NewPreAuth(deps Deps) dispatch.PreAuth {
+	run := NewAnonymousAssumeRoleWithWebIdentity(deps)
+	return dispatch.PreAuth{
+		Service: ServiceName,
+		Serve: func(w http.ResponseWriter, r *http.Request) (bool, error) {
+			if r.Method != http.MethodPost || r.Header.Get("Authorization") != "" {
+				return false, nil
+			}
+			args, ok := anonymousArgs(r)
+			if !ok {
+				return false, nil
+			}
+			xmlOutput, err := run(args)
+			if err != nil {
+				return true, err
+			}
+			w.Header().Set("Content-Type", "text/xml")
+			w.WriteHeader(http.StatusOK)
+			if _, err := w.Write(xmlOutput); err != nil {
+				slog.Error("Failed to write STS response", "err", err)
+			}
+			return true, nil
+		},
+	}
+}
+
+// anonymousArgs peeks the request body and reports whether its Action is one
+// served without SigV4. This runs ahead of authentication, so an unauthenticated
+// client controls the allocation: one byte past the cap detects an oversize body,
+// which is handed back unread rather than dispatched.
+func anonymousArgs(r *http.Request) (map[string]string, bool) {
+	body, err := io.ReadAll(io.LimitReader(r.Body, sigv4.MaxPayloadLen+1))
+	if err != nil {
+		return nil, false
+	}
+	if int64(len(body)) > sigv4.MaxPayloadLen {
+		r.Body = io.NopCloser(io.MultiReader(bytes.NewReader(body), r.Body))
+		return nil, false
+	}
+	r.Body = io.NopCloser(bytes.NewReader(body))
+
+	args, err := query.ParseArgs(string(body))
+	if err != nil || !anonymousActions[args["Action"]] {
+		return nil, false
+	}
+	return args, true
+}
+
 // resolveAction is the one Query Action parser telemetry and dispatch share.
 // It leaves the body readable for the dispatcher and answers "" on any failure.
 func resolveAction(r *http.Request) string {
@@ -2,6 +2,7 @@ package gateway

 import (
 	"github.com/go-chi/chi/v5"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	acmawsapi "github.com/mulgadc/spinifex/spinifex/domains/acm/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/telemetry"
 	"github.com/stretchr/testify/assert"
@@ -53,7 +54,7 @@ func TestACMPipeline_TraceActionEnricher_SpanNamedServiceAction(t *testing.T) {
 	defer otel.SetTracerProvider(prev)

 	gw, _ := acmPipelineApp(t, allowAllACMDocs())
-	subscribeACMCounter(t, gw.NATSConn, "acm.ListCertificates", []byte(`{}`))
+	subscribeACMCounter(t, gwfix.DepsOf(gw).NATS, "acm.ListCertificates", []byte(`{}`))

 	r := chi.NewRouter()
 	r.Use(otelsetup.HTTPMiddleware("awsgw"))
@@ -95,16 +95,12 @@ func actorProbeGateway(t *testing.T, e *cbEnv) (http.Handler, *[]actorObservatio
 			return nil
 		},
 	}))
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		NATSConn:       e.nc,
 		Region:         gwfix.Region,
-		ExpectedNodes:  1,
-		IAMService:     e.counter,
-		STSService:     e.sts,
-	}
+	}, gwfix.Deps{NATS: e.nc, Nodes: 1, IAM: e.counter, STS: e.sts})
 	withRealAuth(gw)
-	gw.AnonymousSTS = stsawsapi.NewAnonymousAssumeRoleWithWebIdentity(stsawsapi.Deps{STS: gw.STSService, IAM: gw.IAMService})
+	gw.PreAuth = append(gw.PreAuth, stsawsapi.NewPreAuth(stsawsapi.Deps{STS: gwfix.DepsOf(gw).STS, IAM: gwfix.DepsOf(gw).IAM}))
 	require.NoError(t, b.Register(stsRegistration(gw)))
 	gw.Services = b.Build()
 	return gw.SetupRoutes(), &seen
@@ -362,3 +358,9 @@ func TestActorProjection_AgentRolesNotAssumableOverHTTPS(t *testing.T) {
 	assert.Equal(t, e.accountID, o.RDSAccountID)
 	assert.NotEqual(t, global, o.RDSAccountID)
 }
+
+const (
+	testDiscAccount = "000000000001"
+	testDiscCluster = "h1"
+	testDiscIssuer  = "https://10.0.0.1:9999/oidc/eks/ap-southeast-2/000000000001/h1"
+)
@@ -1,6 +1,7 @@
 package gateway

 import (
+	"context"
 	"encoding/json"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
 	"log/slog"
@@ -8,9 +9,16 @@ import (
 	"uuid"

 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
-	operatoradmin "github.com/mulgadc/spinifex/spinifex/operator/admin"
 )

+// AdminSurface names the registration the private /admin surface dispatches
+// to and how the request ID reaches it. The composing role supplies both, so
+// the gateway names no service.
+type AdminSurface struct {
+	Service       string
+	WithRequestID func(ctx context.Context, requestID string) context.Context
+}
+
 // Admin_Request serves the private /admin/<Method> surface. The route stays
 // here so it keeps its place ahead of the service selector; the work is the
 // spinifex registration's, dispatched exactly as a Query request is. Only the
@@ -18,9 +26,11 @@ import (
 func (gw *GatewayConfig) Admin_Request(w http.ResponseWriter, r *http.Request) {
 	requestID := uuid.NewV4().String()
 	w.Header().Set("X-Amzn-Requestid", requestID)
-	r = r.WithContext(operatoradmin.WithRequestID(r.Context(), requestID))
+	if gw.Admin.WithRequestID != nil {
+		r = r.WithContext(gw.Admin.WithRequestID(r.Context(), requestID))
+	}

-	entry, ok := gw.registered(operatoradmin.ServiceName)
+	entry, ok := gw.registered(gw.Admin.Service)
 	if !ok {
 		slog.Error("Admin: spinifex service is not registered", "requestId", requestID)
 		gw.writeAdminError(w, requestID, awserrors.ErrorInternalError, "")
@@ -29,12 +39,12 @@ func (gw *GatewayConfig) Admin_Request(w http.ResponseWriter, r *http.Request) {

 	action := reqctx.Action(r.Context())
 	if err := gw.dispatchRegistered(entry, w, r); err != nil {
-		slog.Log(r.Context(), serviceErrorLevel(operatoradmin.ServiceName, err), "Service request error",
-			"service", operatoradmin.ServiceName, "action", action, "requestId", requestID, "error", err)
+		slog.Log(r.Context(), serviceErrorLevel(gw.Admin.Service, err), "Service request error",
+			"service", gw.Admin.Service, "action", action, "requestId", requestID, "error", err)
 		gw.writeAdminErrorFor(w, requestID, err)
 		return
 	}
-	slog.Info("Service request completed", "service", operatoradmin.ServiceName, "action", action, "requestId", requestID)
+	slog.Info("Service request completed", "service", gw.Admin.Service, "action", action, "requestId", requestID)
 }

 // adminErrorBody is the JSON error envelope for /admin. This surface is not an
@@ -3,6 +3,7 @@ package gateway
 import (
 	"bytes"
 	"encoding/json"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	"log/slog"
 	"net/http"
 	"strings"
@@ -54,7 +55,7 @@ func TestAdminOutcome_LogFollowsTheRealResult(t *testing.T) {
 	}{
 		{"success", awsidentifiers.DefaultAccountID(), nil, http.StatusOK, ""},
 		{"denied caller", authzAccountID, nil, http.StatusForbidden, "INFO"},
-		{"cluster fault", awsidentifiers.DefaultAccountID(), func(gw *GatewayConfig) { gw.NATSConn = nil }, http.StatusServiceUnavailable, "ERROR"},
+		{"cluster fault", awsidentifiers.DefaultAccountID(), func(gw *GatewayConfig) { gwfix.DepsOf(gw).NATS = nil }, http.StatusServiceUnavailable, "ERROR"},
 	}
 	for _, tc := range cases {
 		t.Run(tc.name, func(t *testing.T) {
@@ -62,7 +62,7 @@ func decodeAdminError(t *testing.T, rec *httptest.ResponseRecorder) adminErrorBo
 // An unknown method is rejected before authorization, so it never discloses
 // whether the caller would have been permitted.
 func TestAdminRequestUnknownMethod(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	rec := adminRequest(gw, "PurgeEverything", `{}`)

@@ -71,7 +71,7 @@ func TestAdminRequestUnknownMethod(t *testing.T) {
 }

 func TestAdminRequestDeniesUnauthorizedCaller(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	rec := adminRequest(gw, "CreateAccount", `{}`)

@@ -83,7 +83,7 @@ func TestAdminRequestDeniesUnauthorizedCaller(t *testing.T) {
 // The request ID ties a client failure to the gateway log line, so it must be
 // present on the header and in the body even for a denial.
 func TestAdminRequestAlwaysCarriesRequestID(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	rec := adminRequest(gw, "CreateAccount", `{}`)

@@ -106,7 +106,7 @@ func TestWriteAdminErrorFallsBackOnUnknownCode(t *testing.T) {
 // The router serves every verb from one pattern, so the handler is what
 // refuses a GET — a second route would shadow the POST one.
 func TestAdminRequestRejectsNonPost(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))
 	req := httptest.NewRequest(http.MethodGet, "/admin/CreateAccount", nil)
 	rctx := chi.NewRouteContext()
 	rctx.URLParams.Add("method", "CreateAccount")
@@ -146,7 +146,7 @@ func authorizedAdminRequestForMethod(gw *GatewayConfig, method, body string) *ht
 // An authorized caller must still get a clear answer when the cluster is not
 // reachable, rather than a panic or a silent success.
 func TestAdminRequestReportsClusterUnavailable(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: createAccountPolicy()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: createAccountPolicy()})))

 	rec := authorizedAdminRequest(gw, `{}`)

@@ -156,7 +156,7 @@ func TestAdminRequestReportsClusterUnavailable(t *testing.T) {
 // Each admin method is a separate grant. The signup Worker holds
 // spinifex:CreateAccount and must not reach deletion with it.
 func TestCreateAccountGrantDoesNotAllowDeleteAccount(t *testing.T) {
-	gw := withOperator(withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: createAccountPolicy()}))
+	gw := withOperator(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: createAccountPolicy()})))

 	for _, method := range []string{"DeleteAccount", "DescribeAccountDeletion", "ListAccounts"} {
 		t.Run(method, func(t *testing.T) {
@@ -148,7 +148,7 @@ func (gw *GatewayConfig) SigV4AuthMiddleware() func(http.Handler) http.Handler {
 					slog.Warn("Auth failure: verification failed",
 						"accessKeyID", sig.Credential.AccessKeyID, "sourceIP", src.logIP,
 						"service", sig.Credential.Service,
-						"action", mismatchAction(r, sig.Credential.Service),
+						"action", gw.mismatchAction(r, sig.Credential.Service),
 						"canonicalRequest", sig.RedactedCanonicalRequest(),
 						"err", sigErr)
 					// Fingerprinted by the signature as well as the key id: each guess at a
@@ -248,9 +248,9 @@ func signingTime(r *http.Request) string {

 // mismatchAction recovers the Action for a rejected query-protocol request so the log names
 // the operation. Auth fails before the dispatcher parses it, which otherwise leaves the
-// trace as a bare "POST /". S3 is skipped: its body is object data, not query args.
-func mismatchAction(r *http.Request, service string) string {
-	if service == "s3" {
+// trace as a bare "POST /". A scope whose body is object data is skipped.
+func (gw *GatewayConfig) mismatchAction(r *http.Request, service string) string {
+	if traits, ok := gw.selector().Traits(service); ok && traits.PayloadBody {
 		return ""
 	}

@@ -328,7 +328,7 @@ func (gw *GatewayConfig) writeSigV4Error(w http.ResponseWriter, r *http.Request,
 		return
 	}

-	xmlError := xmlErrorBody(svc, errorCode, errorMsg.Message, requestID, r.URL.Path)
+	xmlError := gw.xmlErrorBody(svc, errorCode, errorMsg.Message, requestID, r.URL.Path)

 	w.Header().Set("Content-Type", envelope.XMLContentType)
 	w.WriteHeader(errorMsg.HTTPCode)
@@ -357,19 +357,13 @@ func signedService(r *http.Request) string {
 	return reqctx.Service(r.Context())
 }

-// globalSigningRegions maps a global AWS service to the region its clients sign
-// with when they use the default endpoint. A client given an endpoint override
-// signs with its own configured region instead, so both are accepted.
-var globalSigningRegions = map[string]string{
-	"route53": "us-east-1",
-}
-
 // expectedSigningRegion returns the credential-scope region a request for
 // service must carry: the global service's signing region when the client
-// claimed it, else the gateway's own region.
+// claimed it, else the gateway's own region. A client given an endpoint
+// override signs with its own configured region, so both are accepted.
 func (gw *GatewayConfig) expectedSigningRegion(service, claimed string) string {
-	if global, ok := globalSigningRegions[service]; ok && claimed == global {
-		return global
+	if traits, ok := gw.selector().Traits(service); ok && traits.GlobalSigningRegion != "" && claimed == traits.GlobalSigningRegion {
+		return traits.GlobalSigningRegion
 	}
 	return gw.Region
 }
@@ -19,7 +19,7 @@ import (
 func statusGateRouter(t *testing.T, svc iamdomain.IAMService) (chi.Router, *http.Request) {
 	t.Helper()

-	gw := withEC2(withRealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: svc}))
+	gw := withEC2(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: svc})))
 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
 	r.HandleFunc("/*", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) })
@@ -125,14 +125,10 @@ func newCBEnv(t *testing.T, opts cbEnvOpts) *cbEnv {

 // gateway builds a router over the env's services with its own (cold) caches.
 func (e *cbEnv) gateway() http.Handler {
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		NATSConn:       e.nc,
 		Region:         gwfix.Region,
-		ExpectedNodes:  1,
-		IAMService:     e.counter,
-		STSService:     e.sts,
-	}
+	}, gwfix.Deps{NATS: e.nc, Nodes: 1, IAM: e.counter, STS: e.sts})
 	withIAMAndSTS(withRealAuth(gw))
 	return gw.SetupRoutes()
 }
@@ -99,14 +99,12 @@ func TestSigV4Auth_FailureLogsLoopbackGatedClientIP(t *testing.T) {
 		t.Run(tc.name, func(t *testing.T) {
 			rl := authlimit.NewAuthRateLimiter()
 			defer rl.Stop()
-			gw := &GatewayConfig{
+			gw := gwfix.WithDeps(&GatewayConfig{
 				DisableLogging: true,
 				Region:         gwfix.Region,
-				IAMService:     &gwfix.IAM{MasterKey: gwfix.MasterKey},
 				Credentials:    credverify.New(&gwfix.IAM{MasterKey: gwfix.MasterKey}, &gwfix.STS{}),
-				STSService:     &gwfix.STS{},
 				RateLimiter:    rl,
-			}
+			}, gwfix.Deps{IAM: &gwfix.IAM{MasterKey: gwfix.MasterKey}, STS: &gwfix.STS{}})
 			withEC2(gw)
 			handler := gw.SigV4AuthMiddleware()(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
 				t.Fatal("an unknown session credential must not reach the handler")
@@ -11,6 +11,7 @@ import (
 	ecsawsapi "github.com/mulgadc/spinifex/spinifex/domains/ecs/awsapi"
 	eksawsapi "github.com/mulgadc/spinifex/spinifex/domains/eks/awsapi"
 	ochreawsapi "github.com/mulgadc/spinifex/spinifex/domains/ochre/awsapi"
+	route53awsapi "github.com/mulgadc/spinifex/spinifex/domains/route53/awsapi"
 	taggingawsapi "github.com/mulgadc/spinifex/spinifex/domains/tagging/awsapi"
 	awsidentifiers "github.com/mulgadc/spinifex/spinifex/foundation/aws/identifiers"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/credential"
@@ -57,11 +58,10 @@ func setupTestApp(accessKey, secretKey string) http.Handler {
 		},
 	}

-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))
 	withServices(gw, awsapi.NewRegistration(awsapi.Deps{}), ec2Registration(gw))

 	r := chi.NewRouter()
@@ -90,7 +90,7 @@ func userIDApp(t *testing.T, svc *gwfix.IAM) (http.Handler, *string) {
 		},
 	}

-	gw := withEC2(withRealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: svc}))
+	gw := withEC2(withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: svc})))
 	seen := new(string)
 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -185,11 +185,10 @@ func TestSigV4Auth_ValidSignatureWithQueryString(t *testing.T) {
 }

 func TestSigV4Auth_NilIAMService(t *testing.T) {
-	gw := withEC2(withRealAuth(&GatewayConfig{
+	gw := withEC2(withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     nil,
-	}))
+	}, gwfix.Deps{IAM: nil})))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -417,11 +416,10 @@ func TestSigV4Auth_ContextPropagation(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -499,11 +497,10 @@ func TestSigV4Auth_ContextDoesNotLeakBetweenRequests(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -539,11 +536,10 @@ func TestSigV4Auth_ContextDoesNotLeakBetweenRequests(t *testing.T) {

 func TestSigV4Auth_LookupAccessKeyUnexpectedError(t *testing.T) {
 	// A non-NoSuchEntity error from LookupAccessKey should yield 500 InternalError.
-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     &gwfix.ErrorLookupIAM{Err: errors.New("nats: connection closed")},
-	})
+	}, gwfix.Deps{IAM: &gwfix.ErrorLookupIAM{Err: errors.New("nats: connection closed")}}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -616,11 +612,10 @@ func TestSigV4Auth_SignedContentType(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -743,11 +738,10 @@ func TestCheckPolicy_NonRootNoPolicies_Denied(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -803,11 +797,10 @@ func TestCheckPolicy_NonRootWithAllow_Passes(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -863,11 +856,10 @@ func TestCheckPolicy_NonRootWithExplicitDeny_Denied(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -924,11 +916,10 @@ func TestCheckPolicy_RootGlobalAccount_Bypasses(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -966,11 +957,10 @@ func TestCheckPolicy_MissingAccountID_InternalError(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -1010,11 +1000,10 @@ func TestCheckPolicy_GetUserPoliciesError_InternalError(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -1040,12 +1029,10 @@ func TestSigV4Auth_NATSDisconnectedShortCircuit(t *testing.T) {
 	}
 	nc.Close() // disconnected — IsConnected() returns false

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		IAMService:     nil,
-	})
+	}, gwfix.Deps{NATS: nc, IAM: nil}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -1126,12 +1113,10 @@ func mountSessionGateway(t *testing.T, cred *stsdomain.SessionCredential) (http.
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     iamMock,
-		STSService:     stsMock,
-	})
+	}, gwfix.Deps{IAM: iamMock, STS: stsMock}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -1294,7 +1279,7 @@ func TestSigV4Auth_AKIA_PrincipalTypeUser(t *testing.T) {
 			},
 		},
 	}
-	gw := withEC2RealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: iamMock})
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: iamMock}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -1324,12 +1309,10 @@ func TestSigV4Auth_Session_STSServiceNil(t *testing.T) {
 		AccessKeys: map[string]*iamdomain.AccessKey{},
 	}
 	_ = encryptedSecret
-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     iamMock,
-		STSService:     nil,
-	})
+	}, gwfix.Deps{IAM: iamMock, STS: nil}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -1398,12 +1381,10 @@ func TestCheckPolicy_SessionNameCollision(t *testing.T) {
 		Tokens:   map[string]string{gwfix.SessionAKID: gwfix.SessionToken},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     iamMock,
-		STSService:     stsMock,
-	})
+	}, gwfix.Deps{IAM: iamMock, STS: stsMock}))

 	handler := setupPolicyTestHandler(gw)

@@ -1442,12 +1423,10 @@ func newAssumedRoleEnforcementGateway(t *testing.T, cred *stsdomain.SessionCrede
 		Sessions: map[string]*stsdomain.SessionCredential{cred.AccessKeyID: cred},
 		Tokens:   map[string]string{cred.AccessKeyID: gwfix.SessionToken},
 	}
-	return withEC2RealAuth(&GatewayConfig{
+	return withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     iamMock,
-		STSService:     stsMock,
-	})
+	}, gwfix.Deps{IAM: iamMock, STS: stsMock}))
 }

 // setupPolicyResourceTestHandler is setupPolicyTestHandler for resource-scoped
@@ -1673,7 +1652,8 @@ func TestMismatchAction_RecoversActionAndRewindsBody(t *testing.T) {
 	req := httptest.NewRequest(http.MethodPost, "/", bytes.NewReader(body))
 	req.Header.Set("Content-Type", "application/x-www-form-urlencoded")

-	assert.Equal(t, "ImportKeyPair", mismatchAction(req, "ec2"))
+	gw := withServices(&GatewayConfig{})
+	assert.Equal(t, "ImportKeyPair", gw.mismatchAction(req, "ec2"))

 	// The error path still writes a response, so the body must survive the peek.
 	rest, err := io.ReadAll(req.Body)
@@ -1684,10 +1664,11 @@ func TestMismatchAction_RecoversActionAndRewindsBody(t *testing.T) {
 func TestMismatchAction_SkipsS3AndPrefersQueryString(t *testing.T) {
 	// An S3 body is object data, so it must never be parsed as query args.
 	s3req := httptest.NewRequest(http.MethodPut, "/bucket/key", bytes.NewReader([]byte("Action=NotReally")))
-	assert.Empty(t, mismatchAction(s3req, "s3"))
+	gw := withServices(&GatewayConfig{})
+	assert.Empty(t, gw.mismatchAction(s3req, "s3"))

 	qreq := httptest.NewRequest(http.MethodGet, "/?Action=DescribeInstances", nil)
-	assert.Equal(t, "DescribeInstances", mismatchAction(qreq, "ec2"))
+	assert.Equal(t, "DescribeInstances", gw.mismatchAction(qreq, "ec2"))
 }

 func TestRedactedCanonicalRequest_MasksSessionToken(t *testing.T) {
@@ -1829,7 +1810,7 @@ func TestSigV4Auth_ResolvesCtxActionForNonQueryServices(t *testing.T) {
 			},
 		},
 	}
-	gw := withEC2RealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: mockSvc})
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: mockSvc}))
 	b := dispatch.NewBuilder()
 	require.NoError(t, b.Register(awsapi.NewRegistration(awsapi.Deps{})))
 	require.NoError(t, b.Register(acmawsapi.NewRegistration(acmawsapi.Deps{})))
@@ -1888,7 +1869,7 @@ func TestSigV4Auth_ResolvesCtxActionForNonQueryServices(t *testing.T) {
 // default endpoint and for the configured region against an override. Every
 // other service stays pinned to the gateway's region.
 func TestExpectedSigningRegion(t *testing.T) {
-	gw := &GatewayConfig{Region: "ap-southeast-2"}
+	gw := withServices(&GatewayConfig{Region: "ap-southeast-2"}, route53awsapi.NewRegistration(route53awsapi.Deps{}))
 	for _, tc := range []struct {
 		service, claimed, want string
 	}{
@@ -2022,11 +2003,10 @@ func TestCheckPolicy_RootNonGlobalAccount_StillEvaluated(t *testing.T) {
 		},
 	}

-	gw := withEC2RealAuth(&GatewayConfig{
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
-	})
+	}, gwfix.Deps{IAM: mockSvc}))

 	handler := setupPolicyTestHandler(gw)

@@ -2168,7 +2148,7 @@ func TestSigV4Auth_UnknownAKIDPrefix_NoLookup(t *testing.T) {
 }

 func TestWriteSigV4Error_ResponseFormat(t *testing.T) {
-	gw := withEC2RealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: allowAllIAMService()})
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: allowAllIAMService()}))

 	testCases := []struct {
 		errorCode      string
@@ -2219,7 +2199,7 @@ func TestWriteSigV4Error_ResponseFormat(t *testing.T) {
 }

 func TestWriteSigV4Error_IgnoresClientRequestID(t *testing.T) {
-	gw := withEC2RealAuth(&GatewayConfig{DisableLogging: true, Region: gwfix.Region, IAMService: allowAllIAMService()})
+	gw := withEC2RealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: gwfix.Region}, gwfix.Deps{IAM: allowAllIAMService()}))

 	w := httptest.NewRecorder()
 	req := httptest.NewRequest(http.MethodGet, "/", nil)
@@ -1,6 +1,8 @@
 package gateway

 import (
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
+	operatoradmin "github.com/mulgadc/spinifex/spinifex/operator/admin"
 	"net/http"
 	"time"

@@ -29,18 +31,29 @@ func (allowAdmission) Admit(string, string) error { return nil }

 type natsReadiness struct{ gw *GatewayConfig }

-func (n natsReadiness) Available() bool { return n.gw.NATSConn != nil && n.gw.NATSConn.IsConnected() }
+func (n natsReadiness) Available() bool {
+	return gwfix.DepsOf(n.gw).NATS != nil && gwfix.DepsOf(n.gw).NATS.IsConnected()
+}

 // withFakeCapabilities fills whichever capability a test left unset.
 func withFakeCapabilities(gw *GatewayConfig) *GatewayConfig {
+	withTestAdmin(gw)
 	if gw.Policy == nil {
 		gw.Policy = allowPolicy{}
 	}
 	if gw.Admission == nil {
 		gw.Admission = allowAdmission{}
 	}
-	if gw.Readiness == nil && gw.NATSConn != nil {
+	if gw.Readiness == nil && gwfix.DepsOf(gw).NATS != nil {
 		gw.Readiness = natsReadiness{gw}
 	}
 	return gw
 }
+
+// withTestAdmin points the /admin surface at the spinifex registration, as the
+// composing role does.
+func withTestAdmin(gw *GatewayConfig) {
+	if gw.Admin.Service == "" {
+		gw.Admin = AdminSurface{Service: operatoradmin.ServiceName, WithRequestID: operatoradmin.WithRequestID}
+	}
+}
@@ -122,7 +122,7 @@ func emittedKeys(t *testing.T) map[string]string {
 		},
 	}

-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: &gwfix.IAM{}})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: &gwfix.IAM{}}))
 	union := make(map[string]string)
 	for name, principal := range principals {
 		r := httptest.NewRequest(http.MethodPost, "/?prefix=home/&max-keys=100", nil)
@@ -131,7 +131,7 @@ func emittedKeys(t *testing.T) map[string]string {
 		// Resolved the way the middleware resolves it, so the gate cannot pass on
 		// a value only the test supplies.
 		userID, _, err := stsdomain.ResolveCallerUser(principal.AccountID, principal.PrincipalType,
-			principal.Identity, principal.AssumedRoleID, gw.IAMService)
+			principal.Identity, principal.AssumedRoleID, gwfix.DepsOf(gw).IAM)
 		require.NoError(t, err)
 		principal.UserID = userID
 		for key := range policyauth.RequestConditionKeys(r, principal) {
@@ -194,12 +194,11 @@ func conditionedPolicyIAMService(effect string, cond map[string]map[string]iamdo
 // nil keys from checkPolicyResources would leave every other test green while
 // silently killing every conditional Allow on this path.
 func TestCheckPolicyResources_ConditionalAllowHonoursSourceIP(t *testing.T) {
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService: conditionedPolicyIAMService("Allow", map[string]map[string]iamdomain.ConditionValue{
-			iampolicy.OpIPAddress: {iampolicy.KeySourceIP: {"10.0.0.0/8"}},
-		}),
-	})
+	}, gwfix.Deps{IAM: conditionedPolicyIAMService("Allow", map[string]map[string]iamdomain.ConditionValue{
+		iampolicy.OpIPAddress: {iampolicy.KeySourceIP: {"10.0.0.0/8"}},
+	})}))

 	tests := []struct {
 		name    string
@@ -233,12 +232,11 @@ func TestCheckPolicyResources_ConditionalAllowHonoursSourceIP(t *testing.T) {
 // the one direction where an absent key widens access. Pin it so the omission
 // is a decision rather than an accident.
 func TestCheckPolicyResources_ConditionalDenyHonoursSourceIP(t *testing.T) {
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService: conditionedPolicyIAMService("Deny", map[string]map[string]iamdomain.ConditionValue{
-			iampolicy.OpIPAddress: {iampolicy.KeySourceIP: {"203.0.113.0/24"}},
-		}),
-	})
+	}, gwfix.Deps{IAM: conditionedPolicyIAMService("Deny", map[string]map[string]iamdomain.ConditionValue{
+		iampolicy.OpIPAddress: {iampolicy.KeySourceIP: {"203.0.113.0/24"}},
+	})}))

 	for _, tt := range []struct {
 		name    string
@@ -1,117 +0,0 @@
-package gateway_test
-
-import (
-	"context"
-	"encoding/json"
-	"sync/atomic"
-	"testing"
-	"time"
-
-	clusterv1 "github.com/mulgadc/spinifex/contracts/cluster/v1"
-	"github.com/mulgadc/spinifex/internal/testkit"
-	"github.com/mulgadc/spinifex/spinifex/gateway"
-	"github.com/nats-io/nats.go"
-)
-
-// respondAsNodes answers the discovery subject as n distinct nodes and reports
-// how many discovery rounds were served.
-func respondAsNodes(t *testing.T, nc *nats.Conn, n int) *atomic.Int64 {
-	t.Helper()
-
-	var rounds atomic.Int64
-	for i := range n {
-		node := string(rune('a' + i))
-		sub, err := nc.Subscribe(clusterv1.NodesDiscoverSubject, func(msg *nats.Msg) {
-			if node == "a" {
-				rounds.Add(1)
-			}
-			payload, err := json.Marshal(clusterv1.NodeDiscoverResponse{Node: node})
-			if err != nil {
-				return
-			}
-			_ = msg.Respond(payload)
-		})
-		if err != nil {
-			t.Fatalf("subscribe: %v", err)
-		}
-		t.Cleanup(func() { _ = sub.Unsubscribe() })
-	}
-	if err := nc.Flush(); err != nil {
-		t.Fatalf("flush: %v", err)
-	}
-	return &rounds
-}
-
-// Discovery fronts eight EC2 handlers, so whatever it costs is added to every
-// one of their calls. The gather counts whoever answers and therefore cannot
-// know when to stop, so it always sits out its whole timeout — that cost
-// has to be paid once in a while, not once per request.
-func TestDiscoverActiveNodesKeepsItsCostOffTheRequestPath(t *testing.T) {
-	const timeout = 50 * time.Millisecond
-	gateway.SetDiscoverActiveNodesTimeoutForTest(t, timeout)
-	_, nc := testutil.StartTestNATS(t)
-	respondAsNodes(t, nc, 4)
-
-	gw := &gateway.GatewayConfig{NATSConn: nc, ExpectedNodes: 4}
-
-	if count := gw.DiscoverActiveNodes(context.Background()); count != 4 {
-		t.Fatalf("DiscoverActiveNodes = %d, want 4", count)
-	}
-
-	start := time.Now()
-	for range 5 {
-		gw.DiscoverActiveNodes(context.Background())
-	}
-	elapsed := time.Since(start)
-
-	// Five uncached calls would cost five timeouts, so two bounds the cached path well clear of that.
-	if elapsed > 2*timeout {
-		t.Errorf("five further discoveries took %v; each was paying the fan-out timeout again", elapsed)
-	}
-}
-
-// Membership does not change per request. Re-deriving it on every call put a
-// fan-out in front of every API call that needs it, which is most of them.
-func TestDiscoverActiveNodesReusesARecentAnswer(t *testing.T) {
-	gateway.SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	_, nc := testutil.StartTestNATS(t)
-	rounds := respondAsNodes(t, nc, 3)
-
-	gw := &gateway.GatewayConfig{NATSConn: nc, ExpectedNodes: 3}
-
-	if count := gw.DiscoverActiveNodes(context.Background()); count != 3 {
-		t.Fatalf("first discovery = %d, want 3", count)
-	}
-	first := rounds.Load()
-
-	for range 5 {
-		if count := gw.DiscoverActiveNodes(context.Background()); count != 3 {
-			t.Fatalf("cached discovery = %d, want 3", count)
-		}
-	}
-
-	if got := rounds.Load(); got != first {
-		t.Errorf("discovery ran %d more time(s) within the TTL, want the cached answer reused", got-first)
-	}
-}
-
-// A fallback is not evidence of anything: caching one would make a momentary
-// NATS problem outlive itself for the whole TTL.
-func TestDiscoverActiveNodesDoesNotCacheAFallback(t *testing.T) {
-	gateway.SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	_, nc := testutil.StartTestNATS(t)
-
-	gw := &gateway.GatewayConfig{NATSConn: nc, ExpectedNodes: 2}
-
-	if count := gw.DiscoverActiveNodes(context.Background()); count != 2 {
-		t.Fatalf("fallback = %d, want the configured node count", count)
-	}
-
-	rounds := respondAsNodes(t, nc, 2)
-	if count := gw.DiscoverActiveNodes(context.Background()); count != 2 {
-		t.Fatalf("second discovery = %d, want 2", count)
-	}
-	if rounds.Load() == 0 {
-		t.Error("no discovery was published after a fallback, so the fallback had been cached")
-	}
-}
@@ -148,27 +148,25 @@ func dispatchSeamApp(t *testing.T, docs []iamdomain.PolicyDocument, regs ...disp
 		require.NoError(t, b.Register(reg))
 	}

-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
 		Services:       b.Build(),
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "alice",
-						AccountID:       ecrTestAccount,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "alice",
+					AccountID:       ecrTestAccount,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	})
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}}))

 	r := chi.NewRouter()
 	r.Use(audit.Middleware)
@@ -342,7 +340,7 @@ func TestDispatchSeam_NATSDownGetsSame503AsLegacy(t *testing.T) {
 	b := dispatch.NewBuilder()
 	require.NoError(t, b.Register(fakeJSONRegistration(&dispatchCapture{})))

-	gw := withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService(), Services: b.Build()})
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Services: b.Build()}, gwfix.Deps{IAM: allowAllIAMService()}))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), fakeJSONService)
@@ -369,17 +367,16 @@ func TestDispatchSeam_UnregisteredUnknownServiceStillInvalidAction(t *testing.T)

 	encryptedSecret, err := iamdomain.EncryptSecret(gwfix.SecretKey, gwfix.MasterKey)
 	require.NoError(t, err)
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
 		Services:       b.Build(),
-		IAMService: &gwfix.IAM{
-			MasterKey: gwfix.MasterKey,
-			AccessKeys: map[string]*iamdomain.AccessKey{
-				gwfix.AccessKey: {AccessKeyID: gwfix.AccessKey, SecretAccessKey: encryptedSecret, UserName: "root", Status: "Active"},
-			},
+	}, gwfix.Deps{IAM: &gwfix.IAM{
+		MasterKey: gwfix.MasterKey,
+		AccessKeys: map[string]*iamdomain.AccessKey{
+			gwfix.AccessKey: {AccessKeyID: gwfix.AccessKey, SecretAccessKey: encryptedSecret, UserName: "root", Status: "Active"},
 		},
-	})
+	}}))

 	r := chi.NewRouter()
 	r.Use(gw.SigV4AuthMiddleware())
@@ -3,6 +3,7 @@ package gateway
 import (
 	"context"
 	"github.com/mulgadc/spinifex/internal/testkit"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	ec2awsapi "github.com/mulgadc/spinifex/spinifex/domains/ec2/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
 	"github.com/mulgadc/spinifex/spinifex/foundation/messaging/nats"
@@ -48,12 +49,10 @@ func TestDeleteVolumeAction_MissingConfiguredNodeIsRetryableAndDeletesNothing(t
 	}
 	require.NoError(t, nc.Flush())

-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		NATSConn:       nc,
-		IAMService:     allowAllIAMService(),
 		Credentials:    credverify.New(allowAllIAMService(), nil),
-	}
+	}, gwfix.Deps{NATS: nc, IAM: allowAllIAMService()})
 	withEC2(gw, func(d *ec2awsapi.Deps) { d.NodeIDs = []string{"node-1", "node-2"} })
 	req := setupEC2Request("Action=DeleteVolume&VolumeId=vol-0123456789abcdef0", "123456789012")
 	ctx, cancel := context.WithTimeout(req.Context(), 30*time.Second)
@@ -1,6 +1,7 @@
 package gateway

 import (
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	ec2awsapi "github.com/mulgadc/spinifex/spinifex/domains/ec2/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
 	"github.com/stretchr/testify/assert"
@@ -11,7 +12,7 @@ import (
 // The gateway has no NATS connection, so any request that reached its handler
 // would fail ServerInternal; DryRunOperation proves nothing was dispatched.
 func TestEC2Request_DryRunAnswersEveryAction(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	actions := ec2awsapi.ActionNames()

@@ -1,6 +1,7 @@
 package gateway

 import (
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	"github.com/stretchr/testify/assert"
 	"net/http"
 	"strconv"
@@ -10,7 +11,7 @@ import (
 // Captured from AWS: the combination is refused even when MaxResults is also
 // out of range.
 func TestEC2Request_DescribeInstancesMaxResultsWithInstanceIds(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	for _, n := range []string{"2", "5"} {
 		w := serveEC2(gw, "Action=DescribeInstances&InstanceId.1=i-0123456789abcdef0&MaxResults="+n)
@@ -22,12 +23,12 @@ func TestEC2Request_DescribeInstancesMaxResultsWithInstanceIds(t *testing.T) {

 // AWS answers DryRun ahead of a MaxResults out of range.
 func TestEC2Request_MaxResultsAfterDryRun(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))
 	assertDryRun(t, dispatchEC2(t, gw, "Action=DescribeLaunchTemplates&MaxResults=0&DryRun=true"))
 }

 func TestEC2Request_MaxResultsInRange(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	for _, body := range []string{
 		"Action=DescribeCapacityReservations&MaxResults=1",
@@ -48,7 +49,7 @@ func TestEC2Request_MaxResultsInRange(t *testing.T) {

 // The refusals are AWS's own, captured by scripts/aws-compliance/aws-response-diff/scen_e.py.
 func TestEC2Request_MaxResultsOutOfRange(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	cases := []struct {
 		action string
@@ -85,7 +86,7 @@ func TestEC2Request_MaxResultsOutOfRange(t *testing.T) {
 // Each of an action's own types reaches the handler, which fails only on the
 // absent NATS connection.
 func TestEC2Request_TagSpecificationAllowedTypes(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	for _, rt := range []string{"instance", "volume", "network-interface", "spot-instances-request"} {
 		assertPermitted(t, dispatchEC2(t, gw, tagSpecBody("RunInstances", rt)+"&ImageId=ami-1&MinCount=1&MaxCount=1"))
@@ -94,7 +95,7 @@ func TestEC2Request_TagSpecificationAllowedTypes(t *testing.T) {
 }

 func TestEC2Request_TagSpecificationResourceType(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService()}))

 	cases := []struct {
 		name, body, code, resourceType string
@@ -59,7 +59,7 @@ func TestECRInventory_ReachableStubsAnswer501(t *testing.T) {
 	}
 	assert.ElementsMatch(t, want, reachableStubs)

-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	for _, action := range reachableStubs {
 		t.Run(action, func(t *testing.T) {
 			err := gw.serveECR(httptest.NewRecorder(),
@@ -74,7 +74,7 @@ func TestECRInventory_ReachableStubsAnswer501(t *testing.T) {
 // unsupported scanning action answers OperationNotSupportedException (400),
 // not NotImplemented (501).
 func TestECRInventory_ScanningActionsReturnOperationNotSupported(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	for _, action := range awsapi.UnsupportedActionNames() {
 		t.Run(action, func(t *testing.T) {
 			err := gw.serveECR(httptest.NewRecorder(),
@@ -122,7 +122,7 @@ func TestECRPipeline_TraceActionEnricher_SpanNamedServiceAction(t *testing.T) {
 // carrying another service's prefix but naming an action outside awsapi.Actions
 // fails as an unrecognised ECR action, not as a cross-service error.
 func TestECRRequest_ECSPrefixedNonECRActionIsInvalidAction(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECR(httptest.NewRecorder(),
 		setupECRRequest("AmazonEC2ContainerServiceV20141113.ListClusters", "{}"))
 	require.Error(t, err)
@@ -132,7 +132,7 @@ func TestECRRequest_ECSPrefixedNonECRActionIsInvalidAction(t *testing.T) {
 // TestECRRequest_TrailingDotIsMissingAction pins that a target ending in "."
 // resolves to an empty action, the same as an absent target.
 func TestECRRequest_TrailingDotIsMissingAction(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECR(httptest.NewRecorder(),
 		setupECRRequest(awsapi.TargetPrefix+".", "{}"))
 	require.Error(t, err)
@@ -143,7 +143,7 @@ func TestECRRequest_TrailingDotIsMissingAction(t *testing.T) {
 // any "<prefix>.<Action>" shape (not just the ECR prefix) dispatches on the
 // bare action name today.
 func TestECRRequest_UnrelatedPrefixDispatchesByBareAction(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECR(httptest.NewRecorder(), setupECRRequest("a.b.ListRepositories", "{}"))
 	require.Error(t, err)
 	assert.Equal(t, awserrors.ErrorNotImplemented, err.Error())
@@ -154,11 +154,10 @@ func TestECRRequest_UnrelatedPrefixDispatchesByBareAction(t *testing.T) {
 func TestGetAuthorizationToken_IgnoresRequestSuppliedIdentity(t *testing.T) {
 	iss, verify := newECRAuth(t)
 	endpoint := awsapi.RepositoryEndpoint{Region: ecrTestRegion, ServicesDomain: ecrTestSuffix}
-	gw := withECR(withVerifier(&GatewayConfig{
-		Region: ecrTestRegion, InternalSuffix: ecrTestSuffix,
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		Region:         ecrTestRegion,
 		DisableLogging: true,
-		IAMService:     allowAllIAMService(),
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})
+	}, gwfix.Deps{IAM: allowAllIAMService()})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})

 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(`{"registryIds":["999999999999"]}`))
 	ctx := withPrincipalField(req.Context(), fieldAccountID, ecrTestAccount)
@@ -17,11 +17,10 @@ import (

 func TestECRRequest_GetAuthorizationTokenDispatched(t *testing.T) {
 	iss, _ := newECRAuth(t)
-	gw := withECR(withVerifier(&GatewayConfig{
-		Region: ecrTestRegion, InternalSuffix: ecrTestSuffix,
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		Region:         ecrTestRegion,
 		DisableLogging: true,
-		IAMService:     allowAllIAMService(),
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, awsapi.RepositoryEndpoint{
+	}, gwfix.Deps{IAM: allowAllIAMService()})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, awsapi.RepositoryEndpoint{
 		Region: ecrTestRegion, ServicesDomain: ecrTestSuffix,
 	})})
 	w := httptest.NewRecorder()
@@ -33,11 +32,10 @@ func TestECRRequest_GetAuthorizationTokenDispatched(t *testing.T) {
 func TestGetAuthorizationToken_MintsUsableToken(t *testing.T) {
 	iss, verify := newECRAuth(t)
 	endpoint := awsapi.RepositoryEndpoint{Region: ecrTestRegion, ServicesDomain: ecrTestSuffix}
-	gw := withECR(withVerifier(&GatewayConfig{
-		Region: ecrTestRegion, InternalSuffix: ecrTestSuffix,
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		Region:         ecrTestRegion,
 		DisableLogging: true,
-		IAMService:     allowAllIAMService(),
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})
+	}, gwfix.Deps{IAM: allowAllIAMService()})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})

 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("{}"))
 	ctx := withPrincipalField(req.Context(), fieldAccountID, ecrTestAccount)
@@ -78,7 +76,7 @@ func TestGetAuthorizationToken_MintsUsableToken(t *testing.T) {
 }

 func TestGetAuthorizationToken_MissingComposition(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	req := setupECRRequest(awsapi.TargetPrefix+".GetAuthorizationToken", "{}")
 	err := gw.serveECR(httptest.NewRecorder(), req)
 	require.Error(t, err)
@@ -86,10 +84,9 @@ func TestGetAuthorizationToken_MissingComposition(t *testing.T) {
 }

 func TestGetAuthorizationToken_NoIssuerNotImplemented(t *testing.T) {
-	gw := withECR(withVerifier(&GatewayConfig{
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService:     allowAllIAMService(),
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(nil, awsapi.RepositoryEndpoint{})})
+	}, gwfix.Deps{IAM: allowAllIAMService()})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(nil, awsapi.RepositoryEndpoint{})})
 	req := setupECRRequest(awsapi.TargetPrefix+".GetAuthorizationToken", "{}")
 	ctx := withPrincipalField(req.Context(), fieldAccountID, ecrTestAccount)
 	err := gw.serveECR(httptest.NewRecorder(), req.WithContext(ctx))
@@ -107,11 +104,10 @@ func TestGetAuthorizationToken_PathedUserRoundTrips(t *testing.T) {
 	iamSvc.userPolicies[ecrPrincipalTestAccount+"|dev"] = allow
 	iss, verify := newECRAuth(t)
 	endpoint := awsapi.RepositoryEndpoint{Region: ecrTestRegion, ServicesDomain: ecrTestSuffix}
-	gw := withECR(withVerifier(&GatewayConfig{
-		Region: ecrTestRegion, InternalSuffix: ecrTestSuffix,
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		Region:         ecrTestRegion,
 		DisableLogging: true,
-		IAMService:     iamSvc,
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})
+	}, gwfix.Deps{IAM: iamSvc})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})

 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("{}"))
 	ctx := withPrincipalField(req.Context(), fieldAccountID, ecrPrincipalTestAccount)
@@ -138,7 +134,7 @@ func TestGetAuthorizationToken_PathedUserRoundTrips(t *testing.T) {
 	claims, err := verify.Verify(jwtStr)
 	require.NoError(t, err)
 	assert.Equal(t, ecrPathedUserARN, claims.Subject)
-	got, failure := credverify.New(gw.IAMService, gw.STSService).Rehydrate(claims.AccessKeyID, claims.AccountID)
+	got, failure := credverify.New(gwfix.DepsOf(gw).IAM, gwfix.DepsOf(gw).STS).Rehydrate(claims.AccessKeyID, claims.AccountID)
 	require.Nil(t, failure)
 	assert.Equal(t, ecrPathedUserARN, got.CallerARN)
 }
@@ -146,11 +142,10 @@ func TestGetAuthorizationToken_PathedUserRoundTrips(t *testing.T) {
 func TestGetAuthorizationToken_ProxyEndpointCarriesPort(t *testing.T) {
 	iss, _ := newECRAuth(t)
 	endpoint := awsapi.RepositoryEndpoint{Region: ecrTestRegion, ServicesDomain: ecrTestSuffix, RegistryPort: "9999"}
-	gw := withECR(withVerifier(&GatewayConfig{
-		Region: ecrTestRegion, InternalSuffix: ecrTestSuffix,
+	gw := withECR(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		Region:         ecrTestRegion,
 		DisableLogging: true,
-		IAMService:     allowAllIAMService(),
-	}), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})
+	}, gwfix.Deps{IAM: allowAllIAMService()})), awsapi.Deps{AuthorizationToken: awsapi.NewAuthorizationTokenActionService(iss, endpoint)})

 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("{}"))
 	ctx := withPrincipalField(req.Context(), fieldAccountID, ecrTestAccount)
@@ -1,17 +1,11 @@
 package gateway

 import (
-	"encoding/json"
 	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
 	"net/http"
 	"net/http/httptest"
 	"strings"
-	"testing"
-
-	"github.com/go-chi/chi/v5"
-	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
 )

 // setupECRRequest builds an ECR control-plane request context. It carries a
@@ -31,23 +25,3 @@ func setupECRRequest(target, body string) *http.Request {
 	ctx = withPrincipalField(ctx, fieldAccessKey, "AKIAECRTESTCONTROLPL1")
 	return req.WithContext(ctx)
 }
-
-func TestOCIRegistry_StubReturns501(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
-	r := chi.NewRouter()
-	gw.mountRegistry(r)
-
-	w := httptest.NewRecorder()
-	r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/v2/team/app/tags/list", nil))
-
-	assert.Equal(t, http.StatusNotImplemented, w.Code)
-
-	var body struct {
-		Errors []struct {
-			Code string `json:"code"`
-		} `json:"errors"`
-	}
-	require.NoError(t, json.Unmarshal(w.Body.Bytes(), &body))
-	require.Len(t, body.Errors, 1)
-	assert.Equal(t, "UNSUPPORTED", body.Errors[0].Code)
-}
@@ -19,7 +19,7 @@ func TestECSRegisterTaskDefinition_DenialRendersAccessDenied403(t *testing.T) {
 			return gwfix.AllowPolicyResource("ecs:RegisterTaskDefinition", "*"), nil
 		},
 	}
-	gw := withRealAuth(&GatewayConfig{DisableLogging: true, IAMService: mock})
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: mock}))
 	req := setupECSUserRequest("AmazonEC2ContainerServiceV20141113.RegisterTaskDefinition",
 		registerTaskDefinitionBody(t, testECSPassRoleTargetARN))

@@ -1,6 +1,7 @@
 package gateway

 import (
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
 	"github.com/stretchr/testify/assert"
@@ -13,7 +14,7 @@ import (

 // An action whose real handler has not yet landed still resolves to the 501 stub.
 func TestECSRequest_KnownActionNotImplemented(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECS(httptest.NewRecorder(),
 		setupECSRequest("AmazonEC2ContainerServiceV20141113.UpdateCluster", "{}"))
 	require.Error(t, err)
@@ -27,7 +28,7 @@ func TestECSRequest_MissingAccountID(t *testing.T) {
 	req.Header.Set("X-Amz-Target", "AmazonEC2ContainerServiceV20141113.ListClusters")
 	req = withTestIdentity(req.WithContext(reqctx.WithService(req.Context(), "ecs")))

-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	// Account-less requests are rejected by the policy gate, which runs ahead of
 	// the handler's own account guard.
 	err := gw.serveECS(httptest.NewRecorder(), req)
@@ -36,14 +37,14 @@ func TestECSRequest_MissingAccountID(t *testing.T) {
 }

 func TestECSRequest_MissingTarget(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECS(httptest.NewRecorder(), setupECSRequest("", ""))
 	require.Error(t, err)
 	assert.Equal(t, awserrors.ErrorMissingAction, err.Error())
 }

 func TestECSRequest_UnknownAction(t *testing.T) {
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	err := gw.serveECS(httptest.NewRecorder(),
 		setupECSRequest("AmazonEC2ContainerServiceV20141113.MadeUpAction", "{}"))
 	require.Error(t, err)
@@ -237,7 +237,7 @@ func TestErrorEnvelope_ClusterUnavailable(t *testing.T) {
 	b := dispatch.NewBuilder()
 	require.NoError(t, b.Register(envelopeRegistration(restJSONService, dispatch.ErrorEnvelopeRESTJSON, c, nil)))
 	require.NoError(t, b.Register(envelopeRegistration(json10Service, dispatch.ErrorEnvelopeJSON10Query, c, nil)))
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService(), Services: b.Build()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Services: b.Build()}, gwfix.Deps{IAM: allowAllIAMService()}))

 	call := func(svc string) *http.Response {
 		req := httptest.NewRequest(http.MethodPost, "/", nil)
@@ -3,29 +3,20 @@ package gateway

 import (
 	"context"
-	"crypto/x509"
-	"encoding/json"
 	"errors"
 	"fmt"
-	"github.com/mulgadc/spinifex/spinifex/foundation/messaging/nats"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/pipeline"
 	authlimit "github.com/mulgadc/spinifex/spinifex/ingress/aws/ratelimit"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
 	"log/slog"
 	"net/http"
 	"strconv"
-	"sync"
-	"time"
 	"uuid"

 	"github.com/go-chi/chi/v5"
 	"github.com/mulgadc/bluebottle/pkg/iampolicy"
 	bbotel "github.com/mulgadc/bluebottle/pkg/otelsetup"
 	"github.com/mulgadc/bluebottle/pkg/ratelimit"
-	clusterv1 "github.com/mulgadc/spinifex/contracts/cluster/v1"
-	iamdomain "github.com/mulgadc/spinifex/spinifex/domains/iam"
-	stsdomain "github.com/mulgadc/spinifex/spinifex/domains/sts"
-	stsawsapi "github.com/mulgadc/spinifex/spinifex/domains/sts/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
 	"github.com/mulgadc/spinifex/spinifex/foundation/telemetry"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/audit"
@@ -34,7 +25,6 @@ import (
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/dispatch"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/envelope"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/query"
-	"github.com/nats-io/nats.go"
 	"go.opentelemetry.io/otel/attribute"
 	"go.opentelemetry.io/otel/trace"
 )
@@ -43,29 +33,12 @@ import (
 // facts, auth and throttling collaborators, and lazily built caches. Its methods are the
 // per-service request handlers.
 type GatewayConfig struct {
-	Debug          bool       `json:"debug"`
-	DisableLogging bool       `json:"disable_logging"`
-	NATSConn       *nats.Conn // Shared NATS connection for service communication
-	Config         string     // Shared AWS Gateway config for S3 auth
-	// RootCAs is the cluster CA pool, used to verify predastore meta nodes
-	// dialed directly for GetStorageStatus. Nil disables that verification,
-	// so admin storage status reports every meta node unreachable rather than
-	// dialing with no trust root.
-	RootCAs       *x509.CertPool
-	ExpectedNodes int    // Number of expected spinifex nodes for multi-node operations
-	Region        string // Region this gateway is running in
-	// The last discovered node count and when it was discovered, so the
-	// discovery fan-out runs once per activeNodesTTL rather than once per
-	// request that needs to know how many nodes to wait for.
-	activeNodesMu    sync.RWMutex
-	activeNodesCount int
-	activeNodesAt    time.Time
-	InternalSuffix   string // Internal DNS suffix for AWS-parity endpoints (e.g. spinifex.internal)
-	IAMService       iamdomain.IAMService
-	STSService       stsdomain.STSService
-	// AnonymousSTS serves the unsigned STS actions that authenticate by a token
-	// in the body. Nil answers them with InternalError.
-	AnonymousSTS stsawsapi.AnonymousFunc
+	Debug          bool   `json:"debug"`
+	DisableLogging bool   `json:"disable_logging"`
+	Region         string // Region this gateway is running in
+	// PreAuth serves requests that authenticate by their own payload, ahead of
+	// SigV4 and throttling, in order.
+	PreAuth []dispatch.PreAuth

 	// Credentials verifies SigV4 credentials. Nil fails every request closed.
 	Credentials credential.Verifier
@@ -81,15 +54,13 @@ type GatewayConfig struct {
 	// early refusal in the auth middleware and makes the service gate answer
 	// cluster-unavailable.
 	Readiness pipeline.Readiness
-	Version   string // Build-time version string (set from cmd.Version)
-	Commit    string // Build-time commit hash (set from cmd.Commit)
-	// OIDC serves the unauthenticated EKS OIDC discovery and JWKS endpoints. Nil
-	// answers them with a 500 (e.g. in unit tests of unrelated routes).
-	OIDC OIDCPublisher
-	// RegistryHandler serves the registry host's /v2 subtree. It sits outside the
-	// SigV4 group and the AWS seam, which ADR-0002:S5 keeps off the registry. Nil
-	// answers 501.
-	RegistryHandler http.Handler
+	// PublicRoutes mounts the routes served ahead of SigV4 and the throttle.
+	// They authenticate themselves or need no authentication. Nil mounts none.
+	PublicRoutes func(chi.Router)
+
+	// Admin configures the private /admin surface. Without a Service, every
+	// /admin request is answered InternalError.
+	Admin AdminSurface

 	// Services holds the signed AWS services dispatched through the
 	// registration seam. Nil means nothing is registered, so every signed
@@ -159,19 +130,15 @@ func (gw *GatewayConfig) SetupRoutes() http.Handler {
 		r.Use(audit.RequestLogger)
 	}

-	// Anonymous STS (AssumeRoleWithWebIdentity) is dispatched ahead of SigV4 —
-	// these calls carry a web-identity JWT, not AWS credentials.
-	r.Use(gw.anonymousSTSInterceptor)
+	// Pre-authentication handlers run ahead of SigV4: their requests carry a
+	// payload token, not AWS credentials.
+	r.Use(gw.preAuthInterceptor)

-	// Unauthenticated OIDC discovery endpoints (IRSA) bypass auth and throttle.
-	r.Group(func(pub chi.Router) {
-		pub.Get("/oidc/eks/{region}/{accountID}/{clusterName}/.well-known/openid-configuration", gw.OIDCDiscoveryDocument)
-		pub.Get("/oidc/eks/{region}/{accountID}/{clusterName}/keys", gw.OIDCJWKS)
-	})
-
-	// OCI Distribution registry (/v2/*). Token/host-authenticated rather than
-	// SigV4-credential-scoped, so it mounts outside the SigV4 group.
-	gw.mountRegistry(r)
+	// Public routes: unauthenticated or self-authenticating, so outside the
+	// SigV4 group.
+	if gw.PublicRoutes != nil {
+		gw.PublicRoutes(r)
+	}

 	// Authenticated AWS API surface.
 	r.Group(func(auth chi.Router) {
@@ -202,6 +169,25 @@ func (gw *GatewayConfig) SetupRoutes() http.Handler {
 	return r
 }

+// preAuthInterceptor offers each request to the pre-authentication handlers in
+// order; one that takes it ends the chain, and its errors render under the
+// service it declares.
+func (gw *GatewayConfig) preAuthInterceptor(next http.Handler) http.Handler {
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		for _, pre := range gw.PreAuth {
+			handled, err := pre.Serve(w, r)
+			if !handled {
+				continue
+			}
+			if err != nil {
+				gw.ErrorHandler(w, r.WithContext(reqctx.WithService(r.Context(), pre.Service)), err)
+			}
+			return
+		}
+		next.ServeHTTP(w, r)
+	})
+}
+
 // throttleKeyFuncs returns the KeyFunc slice for the API throttle middleware,
 // keyed by account-id and action from the SigV4 auth context.
 func (gw *GatewayConfig) throttleKeyFuncs() []ratelimit.KeyFunc {
@@ -261,7 +247,7 @@ func (gw *GatewayConfig) writeClusterUnavailable(w http.ResponseWriter, r *http.
 		return
 	}

-	xmlBody := xmlErrorBody(svc, awserrors.ErrorServiceUnavailable, clusterUnavailableMsg, requestID, r.URL.Path)
+	xmlBody := gw.xmlErrorBody(svc, awserrors.ErrorServiceUnavailable, clusterUnavailableMsg, requestID, r.URL.Path)

 	w.Header().Set("Content-Type", envelope.XMLContentType)
 	w.WriteHeader(http.StatusServiceUnavailable)
@@ -276,8 +262,8 @@ func (gw *GatewayConfig) writeThrottleError(w http.ResponseWriter, r *http.Reque
 	svc := reqctx.Service(r.Context())

 	errorCode := awserrors.ErrorRequestLimitExceeded
-	if svc == "iam" || svc == "sts" || svc == "eks" || svc == "route53" {
-		errorCode = awserrors.ErrorThrottling
+	if traits, ok := gw.selector().Traits(svc); ok && traits.XMLThrottleCode != "" {
+		errorCode = traits.XMLThrottleCode
 	}
 	errorMsg := awserrors.LookupErrorMessage(svc, errorCode)

@@ -296,7 +282,7 @@ func (gw *GatewayConfig) writeThrottleError(w http.ResponseWriter, r *http.Reque
 		return
 	}

-	xmlErr := xmlErrorBody(svc, errorCode, errorMsg.Message, requestID, r.URL.Path)
+	xmlErr := gw.xmlErrorBody(svc, errorCode, errorMsg.Message, requestID, r.URL.Path)

 	w.Header().Set("Content-Type", envelope.XMLContentType)
 	w.WriteHeader(errorMsg.HTTPCode)
@@ -454,7 +440,7 @@ func (gw *GatewayConfig) ErrorHandler(w http.ResponseWriter, r *http.Request, er
 		return
 	}

-	xmlError := xmlErrorBody(svc, code, errorMsg.Message, requestId, r.URL.Path)
+	xmlError := gw.xmlErrorBody(svc, code, errorMsg.Message, requestId, r.URL.Path)

 	slog.Debug("Generated error response", "error", err, "code", code, "xml", string(xmlError), "requestId", requestId)

@@ -471,24 +457,26 @@ func ParseAWSQueryArgs(body string) (map[string]string, error) {
 	return query.ParseArgs(body)
 }

-// xmlErrorBody renders an error in the XML envelope svc's clients expect. The
-// one place the service-to-envelope mapping lives, so a service cannot be
-// added to some emitters and missed by others. JSON services never reach
-// here — every caller checks jsonErrorService first.
-func xmlErrorBody(svc, code, message, requestID, resource string) []byte {
-	switch svc {
-	case "s3":
+// xmlErrorBody renders an error in the XML envelope svc's clients expect, from
+// the shape its registration (or the composition, for a scope it does not
+// serve) declares. JSON services never reach here — every caller checks
+// errorProtocol first. No scope yet means the signature never parsed; that
+// keeps EC2's shape rather than changing a path this is not about.
+func (gw *GatewayConfig) xmlErrorBody(svc, code, message, requestID, resource string) []byte {
+	if svc == "" {
+		return envelope.EC2Body(code, message, requestID)
+	}
+	traits, _ := gw.selector().Traits(svc)
+	switch traits.XMLErrorShape {
+	case dispatch.XMLErrorShapeS3:
 		return envelope.S3Body(code, message, requestID, resource)
-	case "ec2", "spinifex", "":
-		// Both speak the Action-parameter query protocol, and their clients
-		// parse the EC2 <Response><Errors> shape, not the generic ErrorResponse.
-		// Empty means the signature never parsed, so no service is known yet;
-		// keep EC2's shape there rather than changing a path this is not about.
+	case dispatch.XMLErrorShapeEC2:
+		// The Action-parameter query clients parse the EC2 <Response><Errors>
+		// shape, not the generic ErrorResponse.
 		return envelope.EC2Body(code, message, requestID)
 	default:
-		// iam, sts, elasticloadbalancing, rds, and any unenumerated or
-		// unserved scope: the generic REST-XML ErrorResponse envelope, never
-		// EC2's shape, which a REST-XML client cannot deserialize.
+		// Any other or unserved scope: the generic REST-XML ErrorResponse,
+		// never EC2's shape, which a REST-XML client cannot deserialize.
 		return envelope.IAMBody(code, message, requestID)
 	}
 }
@@ -496,78 +484,6 @@ func xmlErrorBody(svc, code, message, requestID, resource string) []byte {
 // How long a discovered node count is reused before it is gathered again.
 // Membership does not change per request, and re-deriving it on every call put
 // the discovery fan-out's whole timeout in front of every API call it fronts.
-const activeNodesTTL = 5 * time.Second
-
-// discoverActiveNodesTimeout is how long the discovery fan-out collects replies.
-// A var only so tests can shorten it.
-var discoverActiveNodesTimeout = 500 * time.Millisecond
-
-// DiscoverActiveNodes discovers the number of active spinifex daemon nodes in the
-// cluster by publishing a discovery request and counting unique responses. It
-// carries the request context so the discovery fan-out joins the caller's trace.
-// Returns the number of active nodes (minimum 1 if fallback is needed).
-//
-// The result is cached for activeNodesTTL. The gather itself stays unbounded:
-// it counts whoever answers, which may legitimately exceed the configured node
-// count during a grow, and bounding it by that count would stop early and drop
-// a live node's resources from every fan-out that follows.
-func (gw *GatewayConfig) DiscoverActiveNodes(ctx context.Context) int {
-	if gw.NATSConn == nil {
-		slog.WarnContext(ctx, "DiscoverActiveNodes: NATS connection not available, using ExpectedNodes fallback", "fallback", gw.ExpectedNodes)
-		return gw.ExpectedNodes
-	}
-
-	if count, ok := gw.cachedActiveNodes(); ok {
-		return count
-	}
-
-	frames, _, err := natsmsg.Gather(ctx, gw.NATSConn, clusterv1.NodesDiscoverSubject, []byte("{}"),
-		natsmsg.GatherOpts{Timeout: discoverActiveNodesTimeout})
-	if err != nil {
-		slog.ErrorContext(ctx, "DiscoverActiveNodes: fan-out failed, using ExpectedNodes fallback", "err", err, "fallback", gw.ExpectedNodes)
-		return gw.ExpectedNodes
-	}
-
-	nodesSeen := make(map[string]bool)
-	for _, frame := range frames {
-		var response clusterv1.NodeDiscoverResponse
-		if err := json.Unmarshal(frame.Data, &response); err != nil {
-			slog.DebugContext(ctx, "DiscoverActiveNodes: Failed to unmarshal response", "err", err)
-			continue
-		}
-		nodesSeen[response.Node] = true
-	}
-
-	activeNodes := len(nodesSeen)
-	if activeNodes == 0 {
-		slog.WarnContext(ctx, "DiscoverActiveNodes: No nodes responded, using ExpectedNodes fallback", "fallback", gw.ExpectedNodes)
-		return gw.ExpectedNodes
-	}
-
-	gw.rememberActiveNodes(activeNodes)
-	slog.DebugContext(ctx, "DiscoverActiveNodes: Discovered active nodes", "count", activeNodes)
-	return activeNodes
-}
-
-// cachedActiveNodes returns a count discovered within the TTL. Only a real
-// discovery is cached: a fallback is not evidence of anything and caching one
-// would make a momentary NATS problem outlive itself.
-func (gw *GatewayConfig) cachedActiveNodes() (int, bool) {
-	gw.activeNodesMu.RLock()
-	defer gw.activeNodesMu.RUnlock()
-	if gw.activeNodesCount == 0 || time.Since(gw.activeNodesAt) > activeNodesTTL {
-		return 0, false
-	}
-	return gw.activeNodesCount, true
-}
-
-func (gw *GatewayConfig) rememberActiveNodes(count int) {
-	gw.activeNodesMu.Lock()
-	defer gw.activeNodesMu.Unlock()
-	gw.activeNodesCount = count
-	gw.activeNodesAt = time.Now()
-}
-
 // recordResolvedAction renames the current span to service.action and tags
 // aws.service/aws.action. Called from both SigV4 auth and checkPolicyResource,
 // so it must be idempotent — the last call before the response wins.
@@ -1,7 +1,6 @@
 package gateway

 import (
-	"context"
 	"encoding/json"
 	"encoding/xml"
 	"errors"
@@ -10,10 +9,13 @@ import (
 	acmawsapi "github.com/mulgadc/spinifex/spinifex/domains/acm/awsapi"
 	ec2awsapi "github.com/mulgadc/spinifex/spinifex/domains/ec2/awsapi"
 	awsapi "github.com/mulgadc/spinifex/spinifex/domains/ecr/awsapi"
+	elbv2awsapi "github.com/mulgadc/spinifex/spinifex/domains/elbv2/awsapi"
+	rdsawsapi "github.com/mulgadc/spinifex/spinifex/domains/rds/awsapi"
 	route53awsapi "github.com/mulgadc/spinifex/spinifex/domains/route53/awsapi"
 	taggingawsapi "github.com/mulgadc/spinifex/spinifex/domains/tagging/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/messaging/nats"
 	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
+	operatoradmin "github.com/mulgadc/spinifex/spinifex/operator/admin"
 	"io"
 	"net/http"
 	"net/http/httptest"
@@ -24,7 +26,6 @@ import (
 	"github.com/aws/aws-sdk-go/aws"
 	awsec2 "github.com/aws/aws-sdk-go/service/ec2"
 	"github.com/mulgadc/bluebottle/pkg/ratelimit"
-	clusterv1 "github.com/mulgadc/spinifex/contracts/cluster/v1"
 	"github.com/mulgadc/spinifex/internal/testkit"
 	ochreawsapi "github.com/mulgadc/spinifex/spinifex/domains/ochre/awsapi"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
@@ -74,9 +75,15 @@ func TestXMLErrorBody_EnvelopePerService(t *testing.T) {
 		{"route53", "ErrorResponse"},
 	}

+	gw := withServices(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}),
+		ec2Registration(&GatewayConfig{}), operatoradmin.NewRegistration(operatoradmin.Deps{}),
+		iamRegistration(&GatewayConfig{}), stsRegistration(&GatewayConfig{}),
+		elbv2awsapi.NewRegistration(elbv2awsapi.Deps{}), rdsawsapi.NewRegistration(rdsawsapi.Deps{}),
+		route53awsapi.NewRegistration(route53awsapi.Deps{}))
+
 	for _, tc := range tests {
 		t.Run(tc.svc, func(t *testing.T) {
-			body := xmlErrorBody(tc.svc, "AccessDenied", "denied", "req-1", "/path")
+			body := gw.xmlErrorBody(tc.svc, "AccessDenied", "denied", "req-1", "/path")

 			assert.Equal(t, tc.root, xmlRootName(t, body))
 			assert.Contains(t, string(body), "<Code>AccessDenied</Code>")
@@ -85,7 +92,7 @@ func TestXMLErrorBody_EnvelopePerService(t *testing.T) {
 }

 func TestErrorHandler_IAMService(t *testing.T) {
-	gw := withIAM(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withIAM(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "iam")
@@ -107,7 +114,7 @@ func TestErrorHandler_IAMService(t *testing.T) {
 }

 func TestErrorHandler_UnknownError(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -128,7 +135,7 @@ func TestErrorHandler_UnknownError(t *testing.T) {
 }

 func TestErrorHandler_WrappedErrorCode(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -152,7 +159,7 @@ func TestErrorHandler_WrappedErrorCode(t *testing.T) {
 // fidelity gap: a call site that names the blocking resource via awserrors.Errorf
 // must have that wording reach the client instead of the generic ErrorLookup text.
 func TestErrorHandler_PrefersCallSiteMessage(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -177,7 +184,7 @@ func TestErrorHandler_PrefersCallSiteMessage(t *testing.T) {
 // ResourceInUseException collision check: ACM's DeleteCertificate must not
 // surface EKS's "cluster already exists" wording for the shared wire code.
 func TestErrorHandler_ACMResourceInUse_UsesACMWording(t *testing.T) {
-	gw := withACM(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}), acmawsapi.Deps{})
+	gw := withACM(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})), acmawsapi.Deps{})
 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "acm")
 	req = req.WithContext(ctx)
@@ -198,7 +205,7 @@ func TestErrorHandler_ACMResourceInUse_UsesACMWording(t *testing.T) {
 // own ResourceInUseException must keep its existing wording, unaffected by the
 // ACM-specific override.
 func TestErrorHandler_EKSResourceInUse_UsesEKSWording(t *testing.T) {
-	gw := withContainers(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withContainers(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))
 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "eks")
 	req = req.WithContext(ctx)
@@ -218,7 +225,7 @@ func TestErrorHandler_EKSResourceInUse_UsesEKSWording(t *testing.T) {
 // case: an error path that supplies no message (the vast majority of call
 // sites) must keep rendering exactly today's ErrorLookup text, unchanged.
 func TestErrorHandler_NoMessageSupplied_MatchesErrorLookup(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -240,7 +247,7 @@ func TestErrorHandler_NoMessageSupplied_MatchesErrorLookup(t *testing.T) {
 // validation code outside EC2: the default must not be EC2's Reserved Instance
 // or resource-ID wording, whichever envelope the service uses.
 func TestErrorHandler_NonEC2ValidationDefaults_DropEC2Wording(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	b := dispatch.NewBuilder()
 	require.NoError(t, b.Register(awsapi.NewRegistration(awsapi.Deps{})))
 	require.NoError(t, b.Register(acmawsapi.NewRegistration(acmawsapi.Deps{})))
@@ -267,7 +274,7 @@ func TestErrorHandler_NonEC2ValidationDefaults_DropEC2Wording(t *testing.T) {
 }

 func TestErrorHandler_ELBv2Service(t *testing.T) {
-	gw := withELBv2(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withELBv2(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "elasticloadbalancing")
@@ -289,7 +296,7 @@ func TestErrorHandler_ELBv2Service(t *testing.T) {
 }

 func TestErrorHandler_EC2Service(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -309,7 +316,7 @@ func TestErrorHandler_EC2Service(t *testing.T) {
 }

 func TestErrorHandler_IgnoresClientRequestID(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -327,7 +334,7 @@ func TestErrorHandler_IgnoresClientRequestID(t *testing.T) {
 }

 func TestErrorHandler_ContentTypeXML(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "ec2")
@@ -346,95 +353,6 @@ func startTestNATS(t *testing.T) *nats.Conn {
 	return nc
 }

-func TestDiscoverActiveNodes_NilNATS(t *testing.T) {
-	gw := &GatewayConfig{
-		ExpectedNodes: 3,
-		NATSConn:      nil,
-	}
-
-	result := gw.DiscoverActiveNodes(context.Background())
-	assert.Equal(t, 3, result)
-}
-
-func TestDiscoverActiveNodes_NoResponders(t *testing.T) {
-	SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	nc := startTestNATS(t)
-
-	gw := &GatewayConfig{
-		ExpectedNodes: 5,
-		NATSConn:      nc,
-	}
-
-	result := gw.DiscoverActiveNodes(context.Background())
-	assert.Equal(t, 5, result)
-}
-
-func TestDiscoverActiveNodes_WithResponders(t *testing.T) {
-	SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	nc := startTestNATS(t)
-
-	for _, nodeName := range []string{"node-1", "node-2"} {
-		name := nodeName
-		_, err := nc.Subscribe(clusterv1.NodesDiscoverSubject, func(msg *nats.Msg) {
-			resp := clusterv1.NodeDiscoverResponse{Node: name}
-			data, _ := json.Marshal(resp)
-			msg.Respond(data)
-		})
-		require.NoError(t, err)
-	}
-	require.NoError(t, nc.Flush())
-
-	gw := &GatewayConfig{
-		ExpectedNodes: 1,
-		NATSConn:      nc,
-	}
-
-	result := gw.DiscoverActiveNodes(context.Background())
-	assert.Equal(t, 2, result)
-}
-
-func TestDiscoverActiveNodes_InvalidJSON(t *testing.T) {
-	SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	nc := startTestNATS(t)
-
-	_, err := nc.Subscribe(clusterv1.NodesDiscoverSubject, func(msg *nats.Msg) {
-		msg.Respond([]byte("not json"))
-	})
-	require.NoError(t, err)
-	require.NoError(t, nc.Flush())
-
-	gw := &GatewayConfig{
-		ExpectedNodes: 4,
-		NATSConn:      nc,
-	}
-
-	result := gw.DiscoverActiveNodes(context.Background())
-	assert.Equal(t, 4, result)
-}
-
-func TestDiscoverActiveNodes_DuplicateNodes(t *testing.T) {
-	SetDiscoverActiveNodesTimeoutForTest(t, 50*time.Millisecond)
-	nc := startTestNATS(t)
-
-	for range 2 {
-		_, err := nc.Subscribe(clusterv1.NodesDiscoverSubject, func(msg *nats.Msg) {
-			resp := clusterv1.NodeDiscoverResponse{Node: "same-node"}
-			data, _ := json.Marshal(resp)
-			msg.Respond(data)
-		})
-		require.NoError(t, err)
-	}
-	require.NoError(t, nc.Flush())
-
-	gw := &GatewayConfig{
-		ExpectedNodes: 5,
-		NATSConn:      nc,
-	}
-
-	result := gw.DiscoverActiveNodes(context.Background())
-	assert.Equal(t, 1, result)
-}
-
 func TestParseAWSQueryArgs(t *testing.T) {
 	tests := []struct {
 		name     string
@@ -656,7 +574,7 @@ func TestParseAWSQueryArgs_CapturedClientBodies(t *testing.T) {
 }

 func TestRequest_UnsupportedService(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "s3")
 	req = req.WithContext(ctx)
@@ -684,7 +602,7 @@ func TestRequest_MalformedQueryString_EndToEnd(t *testing.T) {
 	for _, tc := range tests {
 		t.Run(tc.name, func(t *testing.T) {
 			// A live NATS connection bypasses the cluster-unavailable gate.
-			gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: connectedNATS(t), IAMService: allowAllIAMService()})
+			gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: connectedNATS(t), IAM: allowAllIAMService()}))
 			if tc.service == "elasticloadbalancing" {
 				withELBv2(gw)
 			}
@@ -711,7 +629,7 @@ func TestRequest_MalformedQueryString_EndToEnd(t *testing.T) {
 }

 func TestRequest_EC2MissingAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: connectedNATS(t), IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: connectedNATS(t), IAM: allowAllIAMService()}))
 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(""))
 	ctx := reqctx.WithService(req.Context(), "ec2")
 	ctx = withPrincipalField(ctx, fieldAccountID, "123456789012")
@@ -728,7 +646,7 @@ func TestRequest_EC2MissingAction(t *testing.T) {
 }

 func TestRequest_IAMNilService(t *testing.T) {
-	gw := withIAM(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: nil, NATSConn: connectedNATS(t)}))
+	gw := withIAM(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: nil, NATS: connectedNATS(t)})))
 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=test"))
 	ctx := reqctx.WithService(req.Context(), "iam")
 	ctx = withPrincipalField(ctx, fieldAccountID, "123456789012")
@@ -766,7 +684,7 @@ func setupEC2Request(body string, accountID string) *http.Request {
 }

 func TestEC2Request_MissingAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	req := setupEC2Request("", "123456789012")
 	w := httptest.NewRecorder()

@@ -776,7 +694,7 @@ func TestEC2Request_MissingAction(t *testing.T) {
 }

 func TestEC2Request_UnknownAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	req := setupEC2Request("Action=FakeAction", "123456789012")
 	w := httptest.NewRecorder()

@@ -786,7 +704,7 @@ func TestEC2Request_UnknownAction(t *testing.T) {
 }

 func TestEC2Request_NilNATSNonLocalAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: nil, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nil, IAM: allowAllIAMService()}))
 	req := setupEC2Request("Action=DescribeInstances", "123456789012")
 	w := httptest.NewRecorder()

@@ -796,12 +714,10 @@ func TestEC2Request_NilNATSNonLocalAction(t *testing.T) {
 }

 func TestEC2Request_NilNATSLocalAction(t *testing.T) {
-	gw := withEC2(withEC2Verifier(&GatewayConfig{
+	gw := withEC2(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		NATSConn:       nil,
 		Region:         "us-east-1",
-		IAMService:     allowAllIAMService(),
-	}), func(d *ec2awsapi.Deps) { d.AZ = "us-east-1a" })
+	}, gwfix.Deps{NATS: nil, IAM: allowAllIAMService()})), func(d *ec2awsapi.Deps) { d.AZ = "us-east-1a" })
 	req := setupEC2Request("Action=DescribeRegions", "123456789012")
 	w := httptest.NewRecorder()

@@ -817,7 +733,7 @@ func TestEC2Request_NilNATSLocalAction(t *testing.T) {
 }

 func TestEC2Request_MissingAccountID(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: nil, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nil, IAM: allowAllIAMService()}))
 	// Use a local action so we don't fail on nil NATS first
 	req := setupEC2Request("Action=DescribeRegions", "")
 	w := httptest.NewRecorder()
@@ -830,7 +746,7 @@ func TestEC2Request_MissingAccountID(t *testing.T) {

 // Static attributes are served without NATS; only default-vpc needs a lookup.
 func TestEC2Request_DescribeAccountAttributes(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: nil, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nil, IAM: allowAllIAMService()}))
 	req := setupEC2Request("Action=DescribeAccountAttributes&AttributeName.1=max-instances", "123456789012")
 	w := httptest.NewRecorder()

@@ -862,7 +778,7 @@ func TestEC2Request_DescribeAccountAttributes_DefaultVPC(t *testing.T) {
 	require.NoError(t, err)
 	t.Cleanup(func() { _ = sub.Unsubscribe() })

-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: nc, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nc, IAM: allowAllIAMService()}))
 	req := setupEC2Request("Action=DescribeAccountAttributes&AttributeName.1=default-vpc", "123456789012")
 	w := httptest.NewRecorder()

@@ -872,12 +788,10 @@ func TestEC2Request_DescribeAccountAttributes_DefaultVPC(t *testing.T) {
 }

 func TestEC2Request_DescribeAvailabilityZones(t *testing.T) {
-	gw := withEC2(withEC2Verifier(&GatewayConfig{
+	gw := withEC2(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		NATSConn:       nil,
 		Region:         "us-east-1",
-		IAMService:     allowAllIAMService(),
-	}), func(d *ec2awsapi.Deps) { d.AZ = "us-east-1a" })
+	}, gwfix.Deps{NATS: nil, IAM: allowAllIAMService()})), func(d *ec2awsapi.Deps) { d.AZ = "us-east-1a" })
 	req := setupEC2Request("Action=DescribeAvailabilityZones", "123456789012")
 	w := httptest.NewRecorder()

@@ -904,7 +818,7 @@ func shortenPolicyResolveRetry(t *testing.T) {
 // --- Throttle middleware integration tests ---

 func TestThrottleKeyFuncs_ExtractsAccountAndAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	keyFuncs := gw.throttleKeyFuncs()
 	require.Len(t, keyFuncs, 2)

@@ -923,7 +837,7 @@ func TestThrottleKeyFuncs_ExtractsAccountAndAction(t *testing.T) {
 }

 func TestThrottleKeyFuncs_UnknownAction(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	keyFuncs := gw.throttleKeyFuncs()

 	// No ctxAction in context — should return "unknown".
@@ -937,7 +851,7 @@ func TestThrottleKeyFuncs_UnknownAction(t *testing.T) {
 }

 func TestThrottleKeyFuncs_MissingAccountID(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))
 	keyFuncs := gw.throttleKeyFuncs()

 	// No ctxAccountID in context — should return an error.
@@ -949,7 +863,7 @@ func TestThrottleKeyFuncs_MissingAccountID(t *testing.T) {
 }

 func TestWriteThrottleError_EC2(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "ec2")
@@ -967,7 +881,7 @@ func TestWriteThrottleError_EC2(t *testing.T) {
 }

 func TestWriteThrottleError_IAM(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withIAM(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "iam")
@@ -987,7 +901,7 @@ func TestWriteThrottleError_IAM(t *testing.T) {
 // Route 53 throttles with a 400 carrying the Throttling code, which SDK
 // retryers classify by code rather than status.
 func TestWriteThrottleError_Route53(t *testing.T) {
-	gw := &GatewayConfig{DisableLogging: true}
+	gw := withServices(&GatewayConfig{DisableLogging: true}, route53awsapi.NewRegistration(route53awsapi.Deps{}))
 	req := httptest.NewRequest(http.MethodGet, "/2013-04-01/hostedzone", nil)
 	req = req.WithContext(reqctx.WithService(req.Context(), "route53"))
 	w := httptest.NewRecorder()
@@ -1028,11 +942,10 @@ func TestThrottleMiddleware_Integration(t *testing.T) {
 	throttler := ratelimit.New(cfg)
 	defer throttler.Stop()

-	gw := withEC2Verifier(&GatewayConfig{
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Throttler:      throttler,
-		IAMService:     allowAllIAMService(),
-	})
+	}, gwfix.Deps{IAM: allowAllIAMService()}))

 	inner := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
 		w.WriteHeader(http.StatusOK)
@@ -1069,7 +982,7 @@ func TestThrottleMiddleware_Integration(t *testing.T) {

 func TestThrottleMiddleware_DisabledConfig(t *testing.T) {
 	// When Throttler is nil, SetupRoutes skips middleware — no panic.
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, Throttler: nil, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Throttler: nil}, gwfix.Deps{IAM: allowAllIAMService()}))
 	handler := gw.SetupRoutes()

 	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=DescribeInstances"))
@@ -1093,7 +1006,7 @@ func TestThrottleMiddleware_PerActionIsolation(t *testing.T) {
 	throttler := ratelimit.New(cfg)
 	defer throttler.Stop()

-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	inner := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
 		w.WriteHeader(http.StatusOK)
@@ -1124,7 +1037,7 @@ func TestThrottleMiddleware_PerActionIsolation(t *testing.T) {
 }

 func TestRequest_ClusterUnavailableNilConn_EC2(t *testing.T) {
-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()}))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "ec2")
@@ -1144,7 +1057,7 @@ func TestRequest_ClusterUnavailableNilConn_EC2(t *testing.T) {
 }

 func TestRequest_ClusterUnavailableNilConn_IAM(t *testing.T) {
-	gw := withIAM(withEC2Verifier(&GatewayConfig{DisableLogging: true, IAMService: allowAllIAMService()}))
+	gw := withIAM(withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{IAM: allowAllIAMService()})))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "iam")
@@ -1167,7 +1080,7 @@ func TestRequest_ClusterUnavailableClosedConn(t *testing.T) {
 	require.NoError(t, err)
 	nc.Close()

-	gw := withEC2Verifier(&GatewayConfig{DisableLogging: true, NATSConn: nc, IAMService: allowAllIAMService()})
+	gw := withEC2Verifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nc, IAM: allowAllIAMService()}))

 	req := httptest.NewRequest(http.MethodPost, "/", nil)
 	ctx := reqctx.WithService(req.Context(), "ec2")
@@ -68,7 +68,7 @@ import (
 // role composes it. An unset deps.NATS takes gw.NATSConn.
 func withACM(gw *GatewayConfig, deps acmawsapi.Deps) *GatewayConfig {
 	if deps.NATS == nil {
-		deps.NATS = gw.NATSConn
+		deps.NATS = gwfix.DepsOf(gw).NATS
 	}
 	b := dispatch.NewBuilder()
 	if err := b.Register(acmawsapi.NewRegistration(deps)); err != nil {
@@ -87,26 +87,24 @@ func acmPipelineApp(t *testing.T, docs []iamdomain.PolicyDocument) (*GatewayConf
 	require.NoError(t, err)
 	_, nc := testutil.StartTestNATS(t)

-	gw := withVerifier(&GatewayConfig{
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "alice",
-						AccountID:       authzAccountID,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "alice",
+					AccountID:       authzAccountID,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	})
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}}))

 	withACM(gw, acmawsapi.Deps{})

@@ -215,20 +213,19 @@ func newRDSGatewayWithResolver(
 	}
 	// Both branches are wired: an assumed-role principal resolves by role, and a
 	// test that only stubbed users would miss it entirely.
-	return withRealAuth(&GatewayConfig{
+	return withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         "ap-southeast-2",
-		IAMService: &gwfix.PolicyIAM{
-			GetUserPoliciesFn: counted,
-			GetRolePoliciesFn: counted,
-		},
-	}), probe
+	}, gwfix.Deps{IAM: &gwfix.PolicyIAM{
+		GetUserPoliciesFn: counted,
+		GetRolePoliciesFn: counted,
+	}})), probe
 }

 // withRDS registers RDS on gw with the NATS connection and node count it holds,
 // as the awsgw role composes it. Set both on gw before calling.
 func withRDS(gw *GatewayConfig) *GatewayConfig {
-	deps := rdsawsapi.Deps{NATS: gw.NATSConn, ExpectedNodes: gw.ExpectedNodes}
+	deps := rdsawsapi.Deps{NATS: gwfix.DepsOf(gw).NATS, ExpectedNodes: gwfix.DepsOf(gw).Nodes}
 	if quotaFor(gw) != nil {
 		deps.Quota = quotaFor(gw)
 	}
@@ -244,7 +241,7 @@ func withRDS(gw *GatewayConfig) *GatewayConfig {
 // as the awsgw role composes it. Set both on gw before calling.
 func withELBv2(gw *GatewayConfig) *GatewayConfig {
 	b := dispatch.NewBuilder()
-	reg := elbv2awsapi.NewRegistration(elbv2awsapi.Deps{NATS: gw.NATSConn, Quota: quotaFor(gw)})
+	reg := elbv2awsapi.NewRegistration(elbv2awsapi.Deps{NATS: gwfix.DepsOf(gw).NATS, Quota: quotaFor(gw)})
 	if err := b.Register(reg); err != nil {
 		panic(err)
 	}
@@ -257,8 +254,8 @@ func withELBv2(gw *GatewayConfig) *GatewayConfig {
 func serveAccountLimits(t *testing.T, gw *GatewayConfig, accountID, body string) ([]byte, error) {
 	t.Helper()
 	_, nc := testutil.StartTestNATS(t)
-	gw.NATSConn = nc
-	gw.IAMService = allowAllIAMService()
+	gwfix.DepsOf(gw).NATS = nc
+	gwfix.DepsOf(gw).IAM = allowAllIAMService()
 	withELBv2(gw)

 	w := httptest.NewRecorder()
@@ -305,26 +302,24 @@ func elbv2PipelineApp(t *testing.T, docs []iamdomain.PolicyDocument, quota *admi
 	require.NoError(t, err)
 	_, nc := testutil.StartTestNATS(t)

-	gw := withVerifier(&GatewayConfig{
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "alice",
-						AccountID:       authzAccountID,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "alice",
+					AccountID:       authzAccountID,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	})
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}}))
 	withELBv2(withQuota(gw, quota))

 	r := chi.NewRouter()
@@ -380,6 +375,10 @@ func withServices(gw *GatewayConfig, regs ...dispatch.Registration) *GatewayConf
 			panic(err)
 		}
 	}
+	// The composition declares the scopes it knows but does not serve.
+	if err := b.DeclareScope("s3", dispatch.ServiceTraits{XMLErrorShape: dispatch.XMLErrorShapeS3, PayloadBody: true}); err != nil {
+		panic(err)
+	}
 	gw.Services = b.Build()
 	withFakeCapabilities(gw)
 	return gw
@@ -394,9 +393,9 @@ func withIAM(gw *GatewayConfig) *GatewayConfig {
 // and node count gw holds, as the awsgw role composes it. Set them on gw first.
 func iamRegistration(gw *GatewayConfig) dispatch.Registration {
 	return iamawsapi.NewRegistration(iamawsapi.Deps{
-		IAM: gw.IAMService,
+		IAM: gwfix.DepsOf(gw).IAM,
 		CountInstanceProfileAssociations: func(ctx context.Context, accountID, profileARN string) (int, error) {
-			return ec2instanceapi.CountInstanceProfileAssociations(ctx, gw.NATSConn, gw.DiscoverActiveNodes(ctx), accountID, profileARN)
+			return ec2instanceapi.CountInstanceProfileAssociations(ctx, gwfix.DepsOf(gw).NATS, activeNodes(gw)(ctx), accountID, profileARN)
 		},
 	})
 }
@@ -788,11 +787,10 @@ func (svc *echoIAMService) UpdateAssumeRolePolicy(accountID string, input *iam.U

 // setupIAMRequestHandler wires an http.Handler for IAM_Request with injected SigV4 context values.
 func setupIAMRequestHandler(svc iamdomain.IAMService) http.Handler {
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService:     svc,
 		Credentials:    credverify.New(svc, nil),
-	}
+	}, gwfix.Deps{IAM: svc})
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := r.Context()
 		ctx = reqctx.WithService(ctx, "iam")
@@ -885,20 +883,20 @@ type iamExample struct {
 // stsRegistration is the STS registration over the IAMService and STSService
 // gw holds, as the awsgw role composes it. Set them on gw first.
 func stsRegistration(gw *GatewayConfig) dispatch.Registration {
-	return stsawsapi.NewRegistration(stsawsapi.Deps{STS: gw.STSService, IAM: gw.IAMService})
+	return stsawsapi.NewRegistration(stsawsapi.Deps{STS: gwfix.DepsOf(gw).STS, IAM: gwfix.DepsOf(gw).IAM})
 }

 // withIAMAndSTS registers IAM and STS together on gw, with the unsigned
 // web-identity capability, as awsgw composes them.
 func withIAMAndSTS(gw *GatewayConfig) *GatewayConfig {
-	gw.AnonymousSTS = stsawsapi.NewAnonymousAssumeRoleWithWebIdentity(stsawsapi.Deps{STS: gw.STSService, IAM: gw.IAMService})
+	gw.PreAuth = append(gw.PreAuth, stsawsapi.NewPreAuth(stsawsapi.Deps{STS: gwfix.DepsOf(gw).STS, IAM: gwfix.DepsOf(gw).IAM}))
 	return withServices(gw, iamRegistration(gw), stsRegistration(gw))
 }

 // withSTS registers STS alone on gw and composes its unsigned web-identity
 // capability, as awsgw does.
 func withSTS(gw *GatewayConfig) *GatewayConfig {
-	gw.AnonymousSTS = stsawsapi.NewAnonymousAssumeRoleWithWebIdentity(stsawsapi.Deps{STS: gw.STSService, IAM: gw.IAMService})
+	gw.PreAuth = append(gw.PreAuth, stsawsapi.NewPreAuth(stsawsapi.Deps{STS: gwfix.DepsOf(gw).STS, IAM: gwfix.DepsOf(gw).IAM}))
 	return withServices(gw, stsRegistration(gw))
 }

@@ -1071,11 +1069,9 @@ func setupSTSRequestHandler(p stsRequestParams) http.Handler {
 	if p.iamSvc == nil {
 		p.iamSvc = &flexMockIAMService{}
 	}
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		STSService:     p.stsSvc,
-		IAMService:     p.iamSvc,
-	})
+	}, gwfix.Deps{STS: p.stsSvc, IAM: p.iamSvc}))
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := r.Context()
 		ctx = reqctx.WithService(ctx, "sts")
@@ -1162,26 +1158,24 @@ func taggingPipelineApp(t *testing.T, docs []iamdomain.PolicyDocument, agg *tagg
 	require.NoError(t, err)
 	_, nc := testutil.StartTestNATS(t)

-	gw := withVerifier(&GatewayConfig{
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "alice",
-						AccountID:       authzAccountID,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "alice",
+					AccountID:       authzAccountID,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	})
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}}))
 	withTagging(gw, agg)

 	r := chi.NewRouter()
@@ -1223,7 +1217,7 @@ func newTaggingCBGateway(t *testing.T) (*GatewayConfig, *ec2tags.TagsServiceImpl
 		})
 	})

-	gw := withVerifier(&GatewayConfig{DisableLogging: true, Region: authzRegion, IAMService: allowAllIAMService(), NATSConn: nc})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true, Region: authzRegion}, gwfix.Deps{IAM: allowAllIAMService(), NATS: nc}))
 	return gw, svc
 }

@@ -1267,29 +1261,25 @@ func operatorPipelineApp(t *testing.T, accountID string, docs []iamdomain.Policy
 	require.NoError(t, err)
 	_, nc := testutil.StartTestNATS(t)

-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		Version:        "v9.9.9-test",
-		Commit:         "abc1234",
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "admin",
-						AccountID:       accountID,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "admin",
+					AccountID:       accountID,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	}
-	gw.IAMService = operatorIAM{gw.IAMService.(*gwfix.PolicyIAM)}
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}})
+	gwfix.DepsOf(gw).IAM = operatorIAM{gwfix.DepsOf(gw).IAM.(*gwfix.PolicyIAM)}
 	if mutate != nil {
 		mutate(gw)
 	}
@@ -1374,15 +1364,14 @@ var spinifexAdminActions = func() map[string]bool {
 func withOperator(gw *GatewayConfig) *GatewayConfig {
 	b := dispatch.NewBuilder()
 	deps := operatoradmin.Deps{
-		Version:     gw.Version,
-		Commit:      gw.Commit,
-		NATS:        gw.NATSConn,
-		RootCAs:     gw.RootCAs,
-		ActiveNodes: gw.DiscoverActiveNodes,
+		Version:     "v9.9.9-test",
+		Commit:      "abc1234",
+		NATS:        gwfix.DepsOf(gw).NATS,
+		ActiveNodes: activeNodes(gw),

-		IAM:           gw.IAMService,
+		IAM:           gwfix.DepsOf(gw).IAM,
 		Quota:         quotaFor(gw),
-		ExpectedNodes: gw.ExpectedNodes,
+		ExpectedNodes: gwfix.DepsOf(gw).Nodes,
 	}
 	reg := operatoradmin.NewRegistration(deps)
 	if err := b.Register(reg); err != nil {
@@ -1531,7 +1520,7 @@ type jsonErrorEnvelope struct {
 // role composes it. An unset deps.NATS takes gw.NATSConn.
 func withECR(gw *GatewayConfig, deps awsapi.Deps) *GatewayConfig {
 	if deps.NATS == nil {
-		deps.NATS = gw.NATSConn
+		deps.NATS = gwfix.DepsOf(gw).NATS
 	}
 	b := dispatch.NewBuilder()
 	if err := b.Register(awsapi.NewRegistration(deps)); err != nil {
@@ -1550,27 +1539,24 @@ func ecrPipelineApp(t *testing.T, docs []iamdomain.PolicyDocument) (*GatewayConf
 	require.NoError(t, err)
 	_, nc := testutil.StartTestNATS(t)

-	gw := withVerifier(&GatewayConfig{
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		InternalSuffix: ecrTestSuffix,
-		NATSConn:       nc,
-		IAMService: &gwfix.PolicyIAM{
-			IAM: gwfix.IAM{
-				MasterKey: gwfix.MasterKey,
-				AccessKeys: map[string]*iamdomain.AccessKey{
-					gwfix.AccessKey: {
-						AccessKeyID:     gwfix.AccessKey,
-						SecretAccessKey: encryptedSecret,
-						UserName:        "alice",
-						AccountID:       ecrTestAccount,
-						Status:          "Active",
-					},
+	}, gwfix.Deps{NATS: nc, IAM: &gwfix.PolicyIAM{
+		IAM: gwfix.IAM{
+			MasterKey: gwfix.MasterKey,
+			AccessKeys: map[string]*iamdomain.AccessKey{
+				gwfix.AccessKey: {
+					AccessKeyID:     gwfix.AccessKey,
+					SecretAccessKey: encryptedSecret,
+					UserName:        "alice",
+					AccountID:       ecrTestAccount,
+					Status:          "Active",
 				},
 			},
-			GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
 		},
-	})
+		GetUserPoliciesFn: func(_, _ string) ([]iamdomain.PolicyDocument, error) { return docs, nil },
+	}}))

 	withECR(gw, awsapi.Deps{})

@@ -1749,11 +1735,11 @@ func (gw *GatewayConfig) serveECS(w http.ResponseWriter, r *http.Request) error
 // mock IAM service it holds, as the awsgw role composes them.
 func withContainers(gw *GatewayConfig) *GatewayConfig {
 	b := dispatch.NewBuilder()
-	if err := b.Register(ecsawsapi.NewRegistration(ecsawsapi.Deps{NATS: gw.NATSConn})); err != nil {
+	if err := b.Register(ecsawsapi.NewRegistration(ecsawsapi.Deps{NATS: gwfix.DepsOf(gw).NATS})); err != nil {
 		panic(err)
 	}
-	roles, _ := gw.IAMService.(IAMServiceLike)
-	if err := b.Register(eksawsapi.NewRegistration(eksawsapi.Deps{NATS: gw.NATSConn, Roles: mockRoleResolver{iam: roles}})); err != nil {
+	roles, _ := gwfix.DepsOf(gw).IAM.(IAMServiceLike)
+	if err := b.Register(eksawsapi.NewRegistration(eksawsapi.Deps{NATS: gwfix.DepsOf(gw).NATS, Roles: mockRoleResolver{iam: roles}})); err != nil {
 		panic(err)
 	}
 	gw.Services = b.Build()
@@ -1860,11 +1846,11 @@ func serveEC2(gw *GatewayConfig, body string) *httptest.ResponseRecorder {
 // rest.
 func ec2Registration(gw *GatewayConfig, mutate ...func(*ec2awsapi.Deps)) dispatch.Registration {
 	deps := ec2awsapi.Deps{
-		NATS:          gw.NATSConn,
-		IAM:           gw.IAMService,
+		NATS:          gwfix.DepsOf(gw).NATS,
+		IAM:           gwfix.DepsOf(gw).IAM,
 		Quota:         ec2quota.Adapter{Service: quotaFor(gw)},
-		ActiveNodes:   gw.DiscoverActiveNodes,
-		ExpectedNodes: gw.ExpectedNodes,
+		ActiveNodes:   activeNodes(gw),
+		ExpectedNodes: gwfix.DepsOf(gw).Nodes,
 	}
 	for _, m := range mutate {
 		m(&deps)
@@ -1992,15 +1978,12 @@ func ec2Pipeline(t *testing.T, mutate func(*ec2PipelineEnv), statements ...iamdo
 	_, nc, _ := testutil.StartTestJetStream(t)
 	daemon := newEC2Daemon(t, nc)

-	cfg := &GatewayConfig{
+	cfg := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		NATSConn:       nc,
-		IAMService:     svc,
-		ExpectedNodes:  1,
-	}
+	}, gwfix.Deps{NATS: nc, IAM: svc, Nodes: 1})
 	if env.NoNATS {
-		cfg.NATSConn = nil
+		gwfix.DepsOf(cfg).NATS = nil
 	}
 	gw := withEC2(withRealAuth(cfg), func(d *ec2awsapi.Deps) {
 		d.AZ = gwfix.Region + "a"
@@ -2159,3 +2142,9 @@ func (d *ec2Daemon) reply(subject, body string) {
 	defer d.mu.Unlock()
 	d.replies[subject] = body
 }
+
+// activeNodes answers the configured node count, which is what discovery falls
+// back to when no daemon answers; no gateway test runs a responder.
+func activeNodes(gw *GatewayConfig) func(context.Context) int {
+	return func(context.Context) int { return gwfix.DepsOf(gw).Nodes }
+}
@@ -164,11 +164,10 @@ func TestIAMRequest_ListUsers_Success(t *testing.T) {
 }

 func TestIAMRequest_MissingAccountID(t *testing.T) {
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService:     &flexMockIAMService{},
 		Credentials:    credverify.New(&flexMockIAMService{}, nil),
-	}
+	}, gwfix.Deps{IAM: &flexMockIAMService{}})
 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "iam")
 		r = withTestIdentity(r.WithContext(ctx))
@@ -188,11 +187,10 @@ func TestIAMRequest_MissingAccountID(t *testing.T) {
 }

 func TestIAMRequest_NilService(t *testing.T) {
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService:     nil,
 		Credentials:    credverify.New(nil, nil),
-	}
+	}, gwfix.Deps{IAM: nil})
 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := r.Context()
 		ctx = reqctx.WithService(ctx, "iam")
@@ -0,0 +1,70 @@
+package gateway
+
+//test:in-package — pins the route order of the unauthenticated mounts against
+// the SigV4 group and the throttle, through the real router.
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"testing"
+
+	"github.com/go-chi/chi/v5"
+	awserrors "github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
+	"github.com/stretchr/testify/assert"
+)
+
+func mountsApp(t *testing.T) http.Handler {
+	t.Helper()
+	gw := reqctxGateway(t, reqctxGatewayEnv{NATS: natsUp, Throttler: newThrottler(t)})
+	gw.PublicRoutes = func(r chi.Router) {
+		ok := func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) }
+		r.Get("/oidc/eks/{region}/{accountID}/{clusterName}/.well-known/openid-configuration", ok)
+		r.Get("/oidc/eks/{region}/{accountID}/{clusterName}/keys", ok)
+		r.Mount("/v2", http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
+			w.WriteHeader(http.StatusTeapot)
+		}))
+	}
+	return gw.SetupRoutes()
+}
+
+// OIDC discovery and the registry sit ahead of SigV4 and the throttle: an
+// unsigned caller is served, however many times, with a bucket of one.
+func TestMounts_PublicRoutesPrecedeSigV4AndThrottle(t *testing.T) {
+	app := mountsApp(t)
+	for _, tc := range []struct {
+		path string
+		want int
+	}{
+		{"/oidc/eks/ap-southeast-2/123456789012/c/.well-known/openid-configuration", http.StatusOK},
+		{"/oidc/eks/ap-southeast-2/123456789012/c/keys", http.StatusOK},
+		{"/v2/", http.StatusTeapot},
+		{"/v2/team/app/tags/list", http.StatusTeapot},
+	} {
+		for range 4 {
+			resp := doRequest(app, httptest.NewRequest(http.MethodGet, tc.path, nil))
+			assert.Equal(t, tc.want, resp.StatusCode, tc.path)
+		}
+	}
+}
+
+// Only the exact /v2 subtree is the registry; a sibling path stays behind SigV4.
+func TestMounts_RegistryPrefixIsExact(t *testing.T) {
+	resp := doRequest(mountsApp(t), httptest.NewRequest(http.MethodGet, "/v20/x", nil))
+	assert.NotEqual(t, http.StatusTeapot, resp.StatusCode)
+	assert.GreaterOrEqual(t, resp.StatusCode, http.StatusBadRequest)
+}
+
+// Unsigned /admin is refused by SigV4, so the mount cannot be reached without it.
+func TestMounts_AdminSitsBehindSigV4(t *testing.T) {
+	resp := doRequest(mountsApp(t), httptest.NewRequest(http.MethodPost, "/admin/CreateAccount", nil))
+	assert.GreaterOrEqual(t, resp.StatusCode, http.StatusBadRequest)
+}
+
+// Without an admin registration name the surface refuses, rather than
+// dispatching to whatever service the empty name selects.
+func TestAdmin_UnconfiguredSurfaceAnswersInternalError(t *testing.T) {
+	gw := withServices(&GatewayConfig{DisableLogging: true})
+	rec := adminRequest(gw, "CreateAccount", `{}`)
+	assert.Equal(t, http.StatusInternalServerError, rec.Code)
+	assert.Equal(t, awserrors.ErrorInternalError, decodeAdminError(t, rec).Error.Code)
+}
@@ -1,37 +0,0 @@
-package gateway
-
-import (
-	"log/slog"
-	"net/http"
-
-	"github.com/go-chi/chi/v5"
-)
-
-// OIDCPublisher serves the unauthenticated per-cluster OIDC discovery document
-// and JWKS (IRSA). The EKS domain owns the records they are built from; the
-// gateway only maps the route onto the account and cluster.
-type OIDCPublisher interface {
-	ServeDiscoveryDocument(w http.ResponseWriter, r *http.Request, accountID, clusterName string)
-	ServeJWKS(w http.ResponseWriter, r *http.Request, accountID, clusterName string)
-}
-
-// OIDCDiscoveryDocument serves the cluster's OpenID discovery document.
-func (gw *GatewayConfig) OIDCDiscoveryDocument(w http.ResponseWriter, r *http.Request) {
-	if gw.OIDC == nil {
-		slog.Error("OIDC discovery: publisher not configured")
-		http.Error(w, "internal error", http.StatusInternalServerError)
-		return
-	}
-	gw.OIDC.ServeDiscoveryDocument(w, r, chi.URLParam(r, "accountID"), chi.URLParam(r, "clusterName"))
-}
-
-// OIDCJWKS serves the cluster's JWKS document as referenced by the discovery
-// document's jwks_uri.
-func (gw *GatewayConfig) OIDCJWKS(w http.ResponseWriter, r *http.Request) {
-	if gw.OIDC == nil {
-		slog.Error("OIDC JWKS: publisher not configured")
-		http.Error(w, "internal error", http.StatusInternalServerError)
-		return
-	}
-	gw.OIDC.ServeJWKS(w, r, chi.URLParam(r, "accountID"), chi.URLParam(r, "clusterName"))
-}
@@ -3,6 +3,7 @@ package gateway
 import (
 	"encoding/json"
 	operatorv1 "github.com/mulgadc/spinifex/contracts/operator/v1"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	admissionquota "github.com/mulgadc/spinifex/spinifex/domains/admission/quota"
 	iamdomain "github.com/mulgadc/spinifex/spinifex/domains/iam"
 	awsidentifiers "github.com/mulgadc/spinifex/spinifex/foundation/aws/identifiers"
@@ -101,7 +102,7 @@ func TestOperatorAdmin_ListAccounts_JSONSuccess(t *testing.T) {
 // Unlike the Query surface, /admin authorizes first and only then checks the
 // cluster, so NATS down is visible only to an authorized caller.
 func TestOperatorAdmin_NATSDownAnswers503OnlyAfterAuthorization(t *testing.T) {
-	down := func(gw *GatewayConfig) { gw.NATSConn = nil }
+	down := func(gw *GatewayConfig) { gwfix.DepsOf(gw).NATS = nil }

 	_, app := operatorPipelineApp(t, awsidentifiers.DefaultAccountID(), allowAllOperatorDocs(), down)
 	resp := adminCall(t, app, http.MethodPost, "ListAccounts", "spinifex", `{}`)
@@ -227,7 +228,7 @@ func TestOperatorQuery_ModelAccessWithoutStoreIsServerInternal(t *testing.T) {
 // The cluster-unavailable check is in the shared Request path, ahead of
 // authorization, so even a caller with no grant sees 503 when NATS is down.
 func TestOperatorQuery_NATSDownAnswers503BeforeAuthorization(t *testing.T) {
-	_, app := operatorPipelineApp(t, authzAccountID, nil, func(gw *GatewayConfig) { gw.NATSConn = nil })
+	_, app := operatorPipelineApp(t, authzAccountID, nil, func(gw *GatewayConfig) { gwfix.DepsOf(gw).NATS = nil })
 	resp := doRequest(app, spinifexQuery(t, "GetVersion", nil))

 	require.Equal(t, http.StatusServiceUnavailable, resp.StatusCode)
@@ -18,7 +18,7 @@ import (
 func TestOperatorQuery_GetInstanceMetrics_PolicyOnAnotherInstanceIsDenied(t *testing.T) {
 	otherARN := "arn:aws:ec2:" + gwfix.Region + ":" + streamAccount + ":instance/i-0fffffffffffffff0"
 	gw, app := operatorPipelineApp(t, streamAccount, instanceMetricsDocs(otherARN), nil)
-	answerWatches(t, gw.NATSConn)
+	answerWatches(t, gwfix.DepsOf(gw).NATS)
 	srv := httptest.NewServer(app)
 	t.Cleanup(srv.Close)

@@ -32,7 +32,7 @@ func TestOperatorQuery_GetInstanceMetrics_PolicyOnAnotherInstanceIsDenied(t *tes
 func TestOperatorQuery_GetInstanceMetrics_StreamsThroughThePipeline(t *testing.T) {
 	instanceARN := "arn:aws:ec2:" + gwfix.Region + ":" + streamAccount + ":instance/" + streamInstance
 	gw, app := operatorPipelineApp(t, streamAccount, instanceMetricsDocs(instanceARN), nil)
-	answerWatches(t, gw.NATSConn)
+	answerWatches(t, gwfix.DepsOf(gw).NATS)
 	srv := httptest.NewServer(app)
 	t.Cleanup(srv.Close)

@@ -65,10 +65,9 @@ func quotaTestGateway(t *testing.T) (*GatewayConfig, string) {
 	quota.SetOverrides(kv)

 	svc := &quotaIAMService{known: account}
-	return withQuota(withVerifier(&GatewayConfig{
+	return withQuota(withVerifier(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		IAMService:     svc,
-	}), quota), account
+	}, gwfix.Deps{IAM: svc})), quota), account
 }

 // quotaStubChecker answers AccountTerminating from a fixed set, so a test can
@@ -49,14 +49,12 @@ func setupTestAppWithRateLimiter(t *testing.T, accessKey, secretKey string, rl *
 	require.NoError(t, err)
 	t.Cleanup(nc.Close)

-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     mockSvc,
 		Credentials:    credverify.New(mockSvc, nil),
 		RateLimiter:    rl,
-		NATSConn:       nc,
-	}
+	}, gwfix.Deps{IAM: mockSvc, NATS: nc})
 	withEC2(gw)

 	r := chi.NewRouter()
@@ -4,6 +4,7 @@ import (
 	"context"
 	"github.com/aws/aws-sdk-go/service/rds"
 	"github.com/mulgadc/spinifex/internal/testkit"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	rdsinstance "github.com/mulgadc/spinifex/spinifex/domains/rds/instance"
 	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
 	natsmsg "github.com/mulgadc/spinifex/spinifex/foundation/messaging/nats"
@@ -30,7 +31,7 @@ func TestRDSRequest_CreateDBInstance_EmptyMasterUserPasswordReturns400(t *testin
 	require.NoError(t, err)
 	t.Cleanup(func() { _ = sub.Unsubscribe() })

-	gw := withVerifier(&GatewayConfig{DisableLogging: true, NATSConn: nc, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nc, IAM: allowAllIAMService()}))
 	req := setupRDSRequest("Action=CreateDBInstance&DBInstanceIdentifier=orders-db&Engine=postgres&" +
 		"DBInstanceClass=db.t3.medium&AllocatedStorage=20&MasterUsername=appuser&DBName=orders")

@@ -45,7 +46,7 @@ func TestRDSRequest_CreateDBInstance_EmptyMasterUserPasswordReturns400(t *testin

 func TestRDSRequest_MalformedScalarReturnsHTTP400(t *testing.T) {
 	_, nc, _ := testutil.StartTestJetStream(t)
-	gw := withVerifier(&GatewayConfig{DisableLogging: true, NATSConn: nc, IAMService: allowAllIAMService()})
+	gw := withVerifier(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{NATS: nc, IAM: allowAllIAMService()}))
 	for _, body := range []string{
 		"Action=CreateDBInstance&AllocatedStorage=abc",
 		"Action=CreateDBInstance&MultiAZ=yes",
@@ -1,6 +1,7 @@
 package gateway

 import (
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
 	"github.com/mulgadc/spinifex/spinifex/runtime/roles/awsgw/accountadmission"
 	"github.com/mulgadc/spinifex/spinifex/runtime/roles/awsgw/clusterready"
 	"github.com/mulgadc/spinifex/spinifex/runtime/roles/awsgw/credverify"
@@ -12,10 +13,11 @@ import (
 // internals (the middleware stash, the admin method gate, the account gate)
 // use it; every other gateway test runs on the fake verifier.
 func withRealAuth(gw *GatewayConfig) *GatewayConfig {
-	gw.Credentials = credverify.New(gw.IAMService, gw.STSService)
-	gw.Policy = policyauth.New(gw.IAMService, policyauth.WithRetryBase(policyRetryBase))
-	gw.Admission = accountadmission.New(gw.IAMService)
-	if gw.NATSConn != nil {
+	withTestAdmin(gw)
+	gw.Credentials = credverify.New(gwfix.DepsOf(gw).IAM, gwfix.DepsOf(gw).STS)
+	gw.Policy = policyauth.New(gwfix.DepsOf(gw).IAM, policyauth.WithRetryBase(policyRetryBase))
+	gw.Admission = accountadmission.New(gwfix.DepsOf(gw).IAM)
+	if gwfix.DepsOf(gw).NATS != nil {
 		gw.Readiness = realReadiness{gw}
 	}
 	return gw
@@ -31,5 +33,5 @@ func withEC2RealAuth(gw *GatewayConfig) *GatewayConfig {
 type realReadiness struct{ gw *GatewayConfig }

 func (r realReadiness) Available() bool {
-	return r.gw.NATSConn != nil && clusterready.New(r.gw.NATSConn).Available()
+	return gwfix.DepsOf(r.gw).NATS != nil && clusterready.New(gwfix.DepsOf(r.gw).NATS).Available()
 }
@@ -1,24 +0,0 @@
-package gateway
-
-import (
-	"net/http"
-
-	"github.com/go-chi/chi/v5"
-)
-
-// mountRegistry mounts the registry handler at /v2, ahead of the SigV4 group. The
-// registry authenticates by its own token rather than SigV4, so it is not part of the
-// AWS seam. Only the exact /v2 subtree is the registry; /v20 and the like stay SigV4.
-func (gw *GatewayConfig) mountRegistry(r chi.Router) {
-	h := gw.RegistryHandler
-	if h == nil {
-		h = http.HandlerFunc(registryNotConfigured)
-	}
-	r.Mount("/v2", h)
-}
-
-func registryNotConfigured(w http.ResponseWriter, _ *http.Request) {
-	w.Header().Set("Content-Type", "application/json")
-	w.WriteHeader(http.StatusNotImplemented)
-	_, _ = w.Write([]byte(`{"errors":[{"code":"UNSUPPORTED","message":"registry not configured"}]}`))
-}
@@ -97,13 +97,11 @@ func reqctxGateway(t *testing.T, env reqctxGatewayEnv, statements ...iamdomain.S
 			nc.Close()
 		}
 	}
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         reqctxRegion,
-		NATSConn:       nc,
-		IAMService:     svc,
 		Throttler:      env.Throttler,
-	})
+	}, gwfix.Deps{NATS: nc, IAM: svc}))
 	regs := []dispatch.Registration{
 		ecsawsapi.NewRegistration(ecsawsapi.Deps{NATS: nc}),
 		iamRegistration(gw),
@@ -115,6 +113,7 @@ func reqctxGateway(t *testing.T, env reqctxGatewayEnv, statements ...iamdomain.S
 			ResolveAction: func(*http.Request) string { return "ProbeAction" },
 			Dispatch:      env.Probe,
 			Errors:        dispatch.ErrorEnvelopeXML,
+			Traits:        dispatch.ServiceTraits{GlobalSigningRegion: "us-east-1"},
 			Inventory:     dispatch.Inventory{Registered: []string{"ProbeAction"}},
 		})
 	}
@@ -29,13 +29,12 @@ func auditRouter(t *testing.T, keys map[string]*iamdomain.AccessKey) (http.Handl
 	rl := authlimit.NewAuthRateLimiter()
 	t.Cleanup(rl.Stop)

-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		IAMService:     &gwfix.IAM{MasterKey: gwfix.MasterKey, AccessKeys: keys},
 		Credentials:    credverify.New(&gwfix.IAM{MasterKey: gwfix.MasterKey, AccessKeys: keys}, nil),
 		RateLimiter:    rl,
-	}
+	}, gwfix.Deps{IAM: &gwfix.IAM{MasterKey: gwfix.MasterKey, AccessKeys: keys}})
 	withEC2(gw)

 	var got *audit.Record
@@ -0,0 +1,152 @@
+package gateway
+
+import (
+	"bytes"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+
+	"github.com/mulgadc/bluebottle/pkg/sigv4"
+	stsawsapi "github.com/mulgadc/spinifex/spinifex/domains/sts/awsapi"
+	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
+	"github.com/mulgadc/spinifex/spinifex/ingress/aws/dispatch"
+	"github.com/mulgadc/spinifex/spinifex/ingress/aws/envelope"
+	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
+)
+
+// The throttle rejection per service, through the registrations production
+// composes where there is one. EKS is a JSON registration, so it never takes
+// the XML arm.
+func TestWriteThrottleError_ByService(t *testing.T) {
+	for _, tc := range []struct {
+		svc, code, contains string
+		status              int
+		contentType         string
+		with                func(*GatewayConfig) *GatewayConfig
+	}{
+		{"iam", awserrors.ErrorThrottling, "<ErrorResponse>", http.StatusServiceUnavailable, "application/xml", withIAM},
+		{"sts", awserrors.ErrorThrottling, "<ErrorResponse>", http.StatusServiceUnavailable, "application/xml", withSTS},
+		{"eks", "", `"__type"`, 0, envelope.JSONContentType, withContainers},
+		{"s3", awserrors.ErrorRequestLimitExceeded, "<Error>", http.StatusServiceUnavailable, "application/xml", func(gw *GatewayConfig) *GatewayConfig { return withServices(gw) }},
+		{"unserved-service", awserrors.ErrorRequestLimitExceeded, "<ErrorResponse>", http.StatusServiceUnavailable, "application/xml", nil},
+		{"", awserrors.ErrorRequestLimitExceeded, "<Response>", http.StatusServiceUnavailable, "application/xml", nil},
+	} {
+		t.Run(tc.svc, func(t *testing.T) {
+			gw := &GatewayConfig{DisableLogging: true}
+			if tc.with != nil {
+				gw = tc.with(gw)
+			}
+			req := httptest.NewRequest(http.MethodPost, "/", nil)
+			req = req.WithContext(reqctx.WithService(req.Context(), tc.svc))
+			w := httptest.NewRecorder()
+
+			gw.writeThrottleError(w, req)
+
+			body, _ := io.ReadAll(w.Result().Body)
+			if tc.status != 0 {
+				assert.Equal(t, tc.status, w.Result().StatusCode)
+			}
+			assert.Equal(t, tc.contentType, w.Result().Header.Get("Content-Type"))
+			if tc.code != "" {
+				assert.Contains(t, string(body), "<Code>"+tc.code+"</Code>")
+			}
+			assert.Contains(t, string(body), tc.contains)
+		})
+	}
+}
+
+// An unparsed signature leaves no service, and the rejection keeps EC2's shape.
+func TestXMLErrorBody_EmptyServiceKeepsEC2Shape(t *testing.T) {
+	body := (&GatewayConfig{}).xmlErrorBody("", "AccessDenied", "denied", "req-1", "/")
+
+	assert.Equal(t, "Response", xmlRootName(t, body))
+	assert.Contains(t, string(body), "<Errors>")
+}
+
+// A global service accepts its own signing region and the gateway's, and
+// nothing else; a regional service accepts only the gateway's.
+func TestReqctxPipeline_GlobalSigningRegionRejectsOthers(t *testing.T) {
+	probe := func(w http.ResponseWriter, _ dispatch.Invocation) error {
+		w.WriteHeader(http.StatusOK)
+		return nil
+	}
+	app := reqctxGateway(t, reqctxGatewayEnv{NATS: natsUp, Probe: probe}, statement("Allow", "*", "*")).SetupRoutes()
+
+	body := []byte("Action=ProbeAction")
+	for _, tc := range []struct {
+		region string
+		want   int
+	}{
+		{"us-east-1", http.StatusOK},
+		{reqctxRegion, http.StatusOK},
+		{"eu-west-1", http.StatusForbidden},
+	} {
+		t.Run(tc.region, func(t *testing.T) {
+			req := httptest.NewRequest(http.MethodPost, "/", bytes.NewReader(body))
+			req.Header.Set("Content-Type", "application/x-www-form-urlencoded; charset=utf-8")
+			got := doRequest(app, signReqctx(t, req, body, "route53", tc.region))
+			assert.Equal(t, tc.want, got.StatusCode)
+		})
+	}
+}
+
+// The anonymous exchange runs ahead of authentication, so an oversize body is
+// handed back unread to the signed chain instead of being dispatched.
+func TestAnonymousSTS_OversizeBodyPassesThroughUnread(t *testing.T) {
+	gw := &GatewayConfig{DisableLogging: true, PreAuth: []dispatch.PreAuth{stsawsapi.NewPreAuth(stsawsapi.Deps{})}}
+	big := "Action=AssumeRoleWithWebIdentity&WebIdentityToken=" + strings.Repeat("a", int(sigv4.MaxPayloadLen)+1)
+
+	var seen []byte
+	next := http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) { seen, _ = io.ReadAll(r.Body) })
+	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(big))
+	rec := httptest.NewRecorder()
+
+	gw.preAuthInterceptor(next).ServeHTTP(rec, req)
+
+	assert.Equal(t, big, string(seen), "the whole body must reach the next handler")
+}
+
+// With no anonymous capability composed, the action fails closed.
+func TestAnonymousSTS_NilCapabilityIsAnInternalError(t *testing.T) {
+	gw := &GatewayConfig{DisableLogging: true, PreAuth: []dispatch.PreAuth{stsawsapi.NewPreAuth(stsawsapi.Deps{})}}
+	req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=AssumeRoleWithWebIdentity&WebIdentityToken=x"))
+	rec := httptest.NewRecorder()
+
+	gw.preAuthInterceptor(http.HandlerFunc(func(http.ResponseWriter, *http.Request) {
+		t.Fatal("must not fall through to the signed chain")
+	})).ServeHTTP(rec, req)
+
+	require.Equal(t, http.StatusInternalServerError, rec.Code)
+	assert.Contains(t, rec.Body.String(), "<Code>"+awserrors.ErrorInternalError+"</Code>")
+}
+
+// A pre-authentication request is served before SigV4 and before the
+// throttle, so an exhausted bucket never reaches it.
+func TestPreAuth_RunsAheadOfSigV4AndThrottle(t *testing.T) {
+	served := 0
+	gw := withIAM(withVerifier(gwfix.WithDeps(&GatewayConfig{
+		DisableLogging: true,
+		Throttler:      newThrottler(t),
+		PreAuth: []dispatch.PreAuth{{
+			Service: "svc",
+			Serve: func(w http.ResponseWriter, _ *http.Request) (bool, error) {
+				served++
+				w.WriteHeader(http.StatusOK)
+				return true, nil
+			},
+		}},
+	}, gwfix.Deps{IAM: allowAllIAMService()})))
+	app := gw.SetupRoutes()
+
+	for range 5 {
+		req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=Anything"))
+		assert.Equal(t, http.StatusOK, doRequest(app, req).StatusCode)
+	}
+	assert.Equal(t, 5, served)
+}
@@ -1,80 +0,0 @@
-package gateway
-
-import (
-	"bytes"
-	"errors"
-	"github.com/mulgadc/spinifex/spinifex/ingress/aws/reqctx"
-	"io"
-	"log/slog"
-	"net/http"
-
-	"github.com/mulgadc/bluebottle/pkg/sigv4"
-	"github.com/mulgadc/spinifex/spinifex/foundation/aws/errors"
-)
-
-// anonymousSTSInterceptor routes unsigned IRSA bootstrap actions
-// (AssumeRoleWithWebIdentity) to the STS dispatcher before the SigV4 middleware.
-// Callers carry no AWS credentials — authentication is via the ServiceAccount JWT
-// in the body. Signed requests (Authorization header present) pass through unchanged.
-func (gw *GatewayConfig) anonymousSTSInterceptor(next http.Handler) http.Handler {
-	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		if r.Method == http.MethodPost && r.Header.Get("Authorization") == "" {
-			if args, ok := gw.anonymousSTSArgs(r); ok {
-				r = r.WithContext(reqctx.WithService(r.Context(), "sts"))
-				if err := gw.serveAnonymousSTS(w, args); err != nil {
-					gw.ErrorHandler(w, r, err)
-				}
-				return
-			}
-		}
-		next.ServeHTTP(w, r)
-	})
-}
-
-// anonymousSTSArgs peeks the request body, parses query args, and reports
-// whether the Action is permitted without SigV4. Non-anonymous requests pass through unchanged.
-func (gw *GatewayConfig) anonymousSTSArgs(r *http.Request) (map[string]string, bool) {
-	// This runs ahead of SigV4, so an unauthenticated client controls the
-	// allocation. Read one byte past the cap to detect an oversized body and
-	// hand it back to the normal chain unread rather than dispatching it.
-	body, err := io.ReadAll(io.LimitReader(r.Body, sigv4.MaxPayloadLen+1))
-	if err != nil {
-		return nil, false
-	}
-	if int64(len(body)) > sigv4.MaxPayloadLen {
-		r.Body = io.NopCloser(io.MultiReader(bytes.NewReader(body), r.Body))
-		return nil, false
-	}
-	r.Body = io.NopCloser(bytes.NewReader(body))
-
-	args, err := ParseAWSQueryArgs(string(body))
-	if err != nil || !anonymousSTSActions[args["Action"]] {
-		return nil, false
-	}
-	return args, true
-}
-
-// anonymousSTSActions lists STS actions that carry no SigV4; authenticated by
-// a web-identity JWT instead. anonymousSTSInterceptor routes these before auth.
-var anonymousSTSActions = map[string]bool{
-	"AssumeRoleWithWebIdentity": true,
-}
-
-// serveAnonymousSTS runs an unsigned STS action through the composed
-// capability and writes its XML response.
-func (gw *GatewayConfig) serveAnonymousSTS(w http.ResponseWriter, args map[string]string) error {
-	if gw.AnonymousSTS == nil {
-		slog.Error("STS: anonymous service not initialized")
-		return errors.New(awserrors.ErrorInternalError)
-	}
-	xmlOutput, err := gw.AnonymousSTS(args)
-	if err != nil {
-		return err
-	}
-	w.Header().Set("Content-Type", "text/xml")
-	w.WriteHeader(http.StatusOK)
-	if _, err := w.Write(xmlOutput); err != nil {
-		slog.Error("Failed to write STS response", "err", err)
-	}
-	return nil
-}
@@ -2,50 +2,29 @@ package gateway

 import (
 	"github.com/aws/aws-sdk-go/service/sts"
+	"github.com/mulgadc/spinifex/internal/testkit/gwfix"
+	stsawsapi "github.com/mulgadc/spinifex/spinifex/domains/sts/awsapi"
+	"github.com/mulgadc/spinifex/spinifex/ingress/aws/dispatch"
 	"github.com/stretchr/testify/assert"
-	"github.com/stretchr/testify/require"
-	"io"
 	"net/http"
 	"net/http/httptest"
 	"strings"
 	"testing"
 )

-// TestAnonymousSTSArgs_Classification covers the body-peek helper for anonymous action detection.
-func TestAnonymousSTSArgs_Classification(t *testing.T) {
-	gw := &GatewayConfig{DisableLogging: true}
-
-	t.Run("anonymous action recognised, body restored", func(t *testing.T) {
-		const body = "Action=AssumeRoleWithWebIdentity&WebIdentityToken=tok"
-		req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
-		args, ok := gw.anonymousSTSArgs(req)
-		require.True(t, ok)
-		assert.Equal(t, "AssumeRoleWithWebIdentity", args["Action"])
-
-		restored, _ := io.ReadAll(req.Body)
-		assert.Equal(t, body, string(restored), "body must be restored for downstream read")
-	})
-
-	t.Run("non-anonymous action rejected", func(t *testing.T) {
-		req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=GetCallerIdentity"))
-		_, ok := gw.anonymousSTSArgs(req)
-		assert.False(t, ok)
-	})
-}
-
 // TestAnonymousSTS_SignedRequestFallsThrough confirms a request with an Authorization
 // header is not intercepted — it passes through to the signed surface.
 func TestAnonymousSTS_SignedRequestFallsThrough(t *testing.T) {
-	gw := &GatewayConfig{
+	gw := gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
-		STSService: &flexMockSTSService{
-			assumeWebIdentityFn: func(*sts.AssumeRoleWithWebIdentityInput) (*sts.AssumeRoleWithWebIdentityOutput, error) {
-				t.Fatal("signed request must not reach the anonymous STS dispatcher")
-				return nil, nil
-			},
+	}, gwfix.Deps{STS: &flexMockSTSService{
+		assumeWebIdentityFn: func(*sts.AssumeRoleWithWebIdentityInput) (*sts.AssumeRoleWithWebIdentityOutput, error) {
+			t.Fatal("signed request must not reach the anonymous STS dispatcher")
+			return nil, nil
 		},
-	}
+	}})

+	gw.PreAuth = []dispatch.PreAuth{stsawsapi.NewPreAuth(stsawsapi.Deps{STS: gwfix.DepsOf(gw).STS})}
 	nextCalled := false
 	next := http.HandlerFunc(func(http.ResponseWriter, *http.Request) { nextCalled = true })

@@ -54,6 +33,6 @@ func TestAnonymousSTS_SignedRequestFallsThrough(t *testing.T) {
 	req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential=...")

 	rec := httptest.NewRecorder()
-	gw.anonymousSTSInterceptor(next).ServeHTTP(rec, req)
+	gw.preAuthInterceptor(next).ServeHTTP(rec, req)
 	assert.True(t, nextCalled, "signed request must pass through to next handler")
 }
@@ -58,7 +58,7 @@ func TestSTSRequest_AssumeRole_NilIAMService_InternalError(t *testing.T) {
 			return nil, nil
 		},
 	}
-	gw := withRealAuth(&GatewayConfig{DisableLogging: true, STSService: svc, IAMService: nil})
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{STS: svc, IAM: nil}))
 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "sts")
 		ctx = withPrincipalField(ctx, fieldAccountID, awsidentifiers.GlobalAccountID)
@@ -186,23 +186,21 @@ func TestSTSRequest_GetCallerIdentity_PathedUserThroughMiddleware(t *testing.T)
 	const pathed = "arn:aws:iam::123456789012:user/eng/alice"
 	encryptedSecret, err := iamdomain.EncryptSecret(gwfix.SecretKey, gwfix.MasterKey)
 	require.NoError(t, err)
-	gw := withRealAuth(&GatewayConfig{
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{
 		DisableLogging: true,
 		Region:         gwfix.Region,
-		STSService:     &flexMockSTSService{},
-		IAMService: &gwfix.IAM{
-			MasterKey: gwfix.MasterKey,
-			AccessKeys: map[string]*iamdomain.AccessKey{gwfix.AccessKey: {
-				AccessKeyID: gwfix.AccessKey, SecretAccessKey: encryptedSecret,
-				UserName: "alice", AccountID: "123456789012", Status: "Active",
-			}},
-			GetUserFn: func(string, *iam.GetUserInput) (*iam.GetUserOutput, error) {
-				return &iam.GetUserOutput{User: &iam.User{
-					UserName: aws.String("alice"), UserId: aws.String("AIDAALICE"), Arn: aws.String(pathed),
-				}}, nil
-			},
+	}, gwfix.Deps{STS: &flexMockSTSService{}, IAM: &gwfix.IAM{
+		MasterKey: gwfix.MasterKey,
+		AccessKeys: map[string]*iamdomain.AccessKey{gwfix.AccessKey: {
+			AccessKeyID: gwfix.AccessKey, SecretAccessKey: encryptedSecret,
+			UserName: "alice", AccountID: "123456789012", Status: "Active",
+		}},
+		GetUserFn: func(string, *iam.GetUserInput) (*iam.GetUserOutput, error) {
+			return &iam.GetUserOutput{User: &iam.User{
+				UserName: aws.String("alice"), UserId: aws.String("AIDAALICE"), Arn: aws.String(pathed),
+			}}, nil
 		},
-	})
+	}}))
 	withServices(gw, stsRegistration(gw), ec2Registration(gw))
 	handler := gw.SigV4AuthMiddleware()(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		if err := gw.STS_Request(w, r); err != nil {
@@ -343,7 +341,7 @@ func TestSTSRequest_MissingAction(t *testing.T) {
 }

 func TestSTSRequest_NilService_InternalError(t *testing.T) {
-	gw := withRealAuth(&GatewayConfig{DisableLogging: true, STSService: nil, IAMService: &flexMockIAMService{}})
+	gw := withRealAuth(gwfix.WithDeps(&GatewayConfig{DisableLogging: true}, gwfix.Deps{STS: nil, IAM: &flexMockIAMService{}}))
 	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		ctx := reqctx.WithService(r.Context(), "sts")
 		ctx = withPrincipalField(ctx, fieldAccountID, awsidentifiers.GlobalAccountID)
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Ben Duncan	1c0ebba139	Merge remote-tracking branch 'origin/feat/reboot-relaunch' into e2e/integration	2026-11-01 14:11:32 +10:00
Ben Duncan	c6b8a3ae8f	fix(vm): reboot by relaunching QEMU after a clean guest shutdown RebootInstances powered the guest down under -action shutdown=pause and then ran system_reset + cont in the same QEMU process. On some hosts the reset left the guest silent in firmware with QMP still answering, while a fresh QEMU on the same volumes always booted. A reboot now shuts the guest down over ACPI (forcing it off after 60s, as before), ends that QEMU, and starts a fresh one on the same mounted volumes, taps, IMDS datapath and identity, the way nova's soft reboot and Proxmox's qm reboot do. The instance stays running in Describe, no hooks fire, the console log is appended rather than truncated, and the existing QMP client is reconnected in place so the heartbeat carries on. A relaunched guest that writes nothing to its console within 60s has its run state, vCPUs, vCPU thread states and registers (sampled twice) logged, and if it has printed before, its QEMU is replaced once more. A relaunch that cannot start goes to crash recovery. Stop, terminate and hot-plug wait for a relaunch in flight so volumes are never sealed under a process being started. Guest-initiated reboots stay an in-process reset. The daemon now logs the host kernel, QEMU version and the OVMF package and checksum at start, and every reboot relaunch carries them.	2026-11-01 14:08:55 +10:00
Ben Duncan	3963509ee5	Merge remote-tracking branch 'origin/feat/fence-hardening' into e2e/integration	2026-11-01 13:47:38 +10:00
Ben Duncan	aa9787b7b0	fix(fence): stop the writer first, deliver the fence notice, kill the fenced guest Three defects in the fence path made partition recovery fail: - onVolumeLeaseLost read the lease holder before killing nbdkit, with no deadline, spending the 5s margin between local validity and the server TTL. The kill now comes first and the read is bounded at 2s. - The fence notice was one NATS publish, lost when the node's NATS restarts mid-fence. It is now a request repeated until the daemon acknowledges, dropped once the volume is exported here again. - A fenced guest got a 20s ACPI powerdown, a 20s pidfile wait and a SIGTERM, keeping QEMU alive while a survivor relaunched it. It is now SIGKILLed, and resumeAfterIOError never resumes a fenced guest.	2026-11-01 13:46:54 +10:00
Ben Duncan	cf497be6bd	Merge remote-tracking branch 'origin/feat/e2e-test-hardening' into e2e/integration	2026-11-01 13:41:13 +10:00
Ben Duncan	835849e32b	Merge remote-tracking branches 'origin/feat/fw-conntrack' and 'origin/feat/vpc-node-routes' into e2e/integration	2026-11-01 13:37:00 +10:00
Ben Duncan	ee50998792	fix(firewall): keep conntrack running while the host policy is disarmed The kernel tracks connections only while some rule uses conntrack. With inet spinifex_filter unloaded nothing tracked, so OVN RAFT links formed while disarmed (or before the daemon's first arm) were first seen mid-stream at arm time. With tcp_loose a packet from the server end became a new flow from peer:6643/6644 to an ephemeral port and hit the drop rule, and RAFT lost its leader. Add table inet spinifex_conntrack (accept-only) and keep it loaded on every node: setup.sh loads it at install, the boot unit loads it unconditionally, firstboot loads it before OVN, and disable swaps the policy for it in one transaction instead of turning conntrack off.	2026-11-01 13:34:35 +10:00
Ben Duncan	9733e6b4e4	fix(vpcd): converge every node's VPC ingress routes from intent The routed-NAT host route to a VPC CIDR was installed only by the vpcd that received the IGW attach event, which goes through a queue group, and by whichever node won a leased drift pass. Every other node waited for the five-minute resync, and with two VPCs on one CIDR each node kept whichever route it met first. Each node now runs a host ingress loop outside the drift lease. It reads intent and the gateway LRPs on every intent write, on a nudge from the local IGW hooks and every 30s, installs or repoints the routes, and prunes tagged routes no attached VPC holds. A shared CIDR goes to the bootstrap VPC, then to the lowest VPC ID.	2026-11-01 13:33:11 +10:00
Ben Duncan	e911084e24	fix(ci): fail a nightly cell fast when a hypervisor it provisions on is out of disk	2026-11-01 13:31:46 +10:00
Ben Duncan	e39d75edf6	fix(ci): give the single suite a 40m budget and the nightly cells that run it 60m	2026-11-01 13:27:55 +10:00
Ben Duncan	61be1cad5c	fix(e2e): make the storagefault root-load check observe the load it started	2026-11-01 13:26:49 +10:00
Ben Duncan	7820f900f2	fix(e2e): retry the diskperf fio install with backoff before calling it an infra fault	2026-11-01 13:24:35 +10:00
Ben Duncan	e8226423b0	fix(e2e): skip a catalog entry whose third-party mirror is unreachable, still fail on a non-2xx	2026-11-01 13:23:58 +10:00
Ben Duncan	42a61e5229	fix(e2e): skip a TryReady caller once the suite's SSH datapath is marked broken	2026-11-01 13:22:20 +10:00
Ben Duncan	b8e37e441e	fix(vm): release an auto-assigned public IP before a stop publishes stopped	2026-11-01 13:21:05 +10:00
Josh Siviour	990f32dad4	refactor: move the gateway's public routes, admin wiring and unread config out (11d-1) (#1378 ) * test: pin the order of the unauthenticated mounts against SigV4 and the throttle * refactor: mount the OIDC and registry routes through a public-routes hook * refactor: take the admin registration name and request-id context as gateway config * refactor: move node discovery out of the gateway into the awsgw role * refactor: drop the gateway config fields production never reads (Config, RootCAs, InternalSuffix) * refactor: drop the gateway Version and Commit config fields * refactor: compose the production gateway config in one tested function * refactor: drop the gateway IAMService, STSService, NATSConn and ExpectedNodes config fields Production never reads them: composeGateway builds Policy and Readiness from its own inputs. Tests that used the config as a carrier now keep those collaborators in gwfix.Deps, recorded per config with gwfix.WithDeps and read back with gwfix.DepsOf. Gateway non-test code now imports nothing under domains/, runtime/ or operator/. Test edits are setup only: 153 GatewayConfig literals in 45 files had the four keys moved into gwfix.WithDeps(..., gwfix.Deps{...}), and 113 field reads became gwfix.DepsOf(x).{IAM,STS,NATS,Nodes}. No assertion changed. Script 1, literal rewrite (balanced-brace scanner, keys moved out of the literal): import re,sys,subprocess,os KEYS={'IAMService':'IAM','STSService':'STS','NATSConn':'NATS','ExpectedNodes':'Nodes'} def scan_close(s,i): # s[i]=='{' ; return index of matching '}' ; skips strings, runes, comments d=0;n=len(s);k=i while k<n: c=s[k] if c=='"': k+=1 while s[k]!='"': k+= 2 if s[k]=='\\' else 1 elif c=='`': k=s.index('`',k+1) elif c=="'": k+=1 while s[k]!="'": k+= 2 if s[k]=='\\' else 1 elif s.startswith('//',k): k=s.index('\n',k) continue elif s.startswith('/',k): k=s.index('/',k)+1 elif c in '{([': d+=1 elif c in '})]': d-=1 if d==0: return k k+=1 raise Exception('unbalanced') def split_top(body): parts=[];d=0;start=0;k=0;n=len(body) while k<n: c=body[k] if c=='"': k+=1 while body[k]!='"': k+= 2 if body[k]=='\\' else 1 elif c=='`': k=body.index('`',k+1) elif c=="'": k+=1 while body[k]!="'": k+= 2 if body[k]=='\\' else 1 elif body.startswith('//',k): k=body.index('\n',k); continue elif body.startswith('/',k): k=body.index('/',k)+1 elif c in '{([': d+=1 elif c in '})]': d-=1 elif c==',' and d==0: parts.append(body[start:k]); start=k+1 k+=1 parts.append(body[start:]) return parts changed={} files=[l for l in subprocess.check_output(['git','ls-files','.go']).decode().split() if os.path.exists(l)] SKIP={'spinifex/gateway/gateway.go','spinifex/runtime/roles/awsgw/compose.go','spinifex/runtime/roles/awsgw/awsgw.go'} lit=re.compile(r'(&?)((?:gateway\.)?GatewayConfig)\{') total=0 for f in files: if f in SKIP: continue s=open(f).read() if 'GatewayConfig{' not in s: continue out=[];pos=0;cnt=0 for m in lit.finditer(s): if m.start()<pos: continue o=m.end()-1 c=scan_close(s,o) body=s[o+1:c] parts=split_top(body) keep=[];moved=[] for p in parts: km=re.match(r'\s(IAMService\|STSService\|NATSConn\|ExpectedNodes)\s:\s(.)$',p,re.S) if km: moved.append((KEYS[km.group(1)],km.group(2).strip())) else: keep.append(p) if not moved: continue # rebuild kb=','.join(keep) if kb.strip()=='' : kb='' elif not kb.rstrip().endswith('\n') and '\n' in body: pass d=', '.join(f'{k}: {v}' for k,v in moved) new=f'gwfix.WithDeps({m.group(1)}{m.group(2)}{{{kb}}}, gwfix.Deps{{{d}}})' if not m.group(1): new=f'gwfix.WithDeps(&{m.group(2)}{{{kb}}}, gwfix.Deps{{{d}}})' # value literals flagged below out.append((m.start(),c+1,new)); pos=c+1; cnt+=1 if not m.group(1): print('VALUE LITERAL',f,s.count('\n',0,m.start())+1) if not out: continue r=[];last=0 for a,b,n in out: r.append(s[last:a]);r.append(n);last=b r.append(s[last:]) s2=''.join(r) if 'testkit/gwfix"' not in s2: s2=re.sub(r'import \(\n','import (\n\t"github.com/mulgadc/spinifex/internal/testkit/gwfix"\n',s2,1) open(f,'w').write(s2); total+=cnt; changed[f]=cnt print('literals rewritten',total,'files',len(changed)) Script 2, read rewrite (driven by the compiler's "has no field or method" errors for the type, so only GatewayConfig receivers are touched): import re,subprocess,sys,collections M={'IAMService':'IAM','STSService':'STS','NATSConn':'NATS','ExpectedNodes':'Nodes'} def run(): env=dict(__import__('os').environ,GOWORK='off') out='' for tags,pk in [('',['./spinifex/gateway','./spinifex/runtime/roles/awsgw/...']),('integration',['./tests/integration/...'])]: cmd=['go','test','-count=1','-run','NONE_X','-gcflags=-e']+(['-tags',tags] if tags else [])+pk r=subprocess.run(cmd,capture_output=True,text=True,env=env) out+=r.stdout+r.stderr return out fixed=0 for it in range(8): out=run() errs=re.findall(r'^(\S+\.go):(\d+):(\d+): (.?) undefined \(type (\?(?:gateway\.)?GatewayConfig) has no field or method (\w+)\)',out,re.M) if not errs: print(out[-3000:]); break byfile=collections.defaultdict(list) for f,l,c,expr,t,fld in errs: byfile[f].append((int(l),int(c),expr,fld)) for f,es in byfile.items(): lines=open(f).read().split('\n') for l,c,expr,fld in sorted(set(es),reverse=True): line=lines[l-1] # expr like 'gw.IAMService' or 'x.y.IAMService' new=f'gwfix.DepsOf({expr[:-len(fld)-1]}).{M[fld]}' if expr not in line: print('MISS',f,l,expr); continue lines[l-1]=line.replace(expr,new) fixed+=1 s='\n'.join(lines) if 'testkit/gwfix"' not in s: s=re.sub(r'import \(\n','import (\n\t"github.com/mulgadc/spinifex/internal/testkit/gwfix"\n',s,count=1) open(f,'w').write(s) print('read sites fixed',fixed) docs: record the gateway mount hooks and config-field removal in the package boundary migration * docs: fill in the 11d-1 PR number in the package boundary migration	2026-11-01 13:04:31 +10:00
Josh Siviour	06d51b7e45	refactor: move service-named pipeline behaviour into registration metadata (ADR-0002:S9) (#1374 ) * test: pin service-keyed gateway behaviour before it moves to registration metadata * refactor: declare the XML throttle code on the registration The pipeline picked Throttling for iam, sts, eks and route53 by name. EKS is a JSON registration and never reached that arm, so it is not carried over. * refactor: declare the XML error shape on the registration EC2 and the operator surface declare the EC2 shape. S3 is not served here, so the composition declares its scope traits and a rejected S3 request keeps S3's body. * refactor: declare the global signing region on the registration * refactor: supply the unsigned web-identity exchange as a pre-auth handler STS composition now hands the pipeline a dispatch.PreAuth. The gateway no longer recognises STS or the action by name, and the layering allowlist row for gateway -> sts/awsapi is paid off. * refactor: declare payload-body scopes as traits The action recovery for a rejected request skipped S3 by name. The composition now declares the S3 scope as carrying object data, and the gateway reads the trait. * test: wire the integration harness through PreAuth	2026-11-01 11:34:05 +10:00
Josh Siviour	593dd58831	fix: EKS control-plane VMs are not Kubernetes Nodes (#1373 ) * test: pin control-plane no-agent config, etcd member removal and CP isolation e2e * test: tighten CP isolation e2e and assert a logged prune failure * fix: run EKS control planes without a k3s agent and prune etcd members directly Control-plane VMs no longer register as Nodes, so pods cannot schedule onto a network that cannot reach the workers or the advertised endpoint. With no Node to delete, a replaced peer's etcd member is removed by a new eks-etcd-member helper that matches exactly one member by peer-URL host. * fix: refuse to remove the local etcd member by ID, not only by IP Self-protection no longer depends on first boot deriving its own address: the helper asks the connected endpoint for its member ID and refuses a match on it. * test(e2e): align EKS suite with control planes that are not Nodes Assert zero Nodes and delivered addon objects before any nodegroup. Check Ready worker, metrics APIService, addon deletion and pod logs once a worker exists, scoping logs to Ready nodes and waiting 10m for metrics-server to reschedule. * docs: EKS control-plane hosts are not Kubernetes Nodes Document that kubectl get nodes lists workers only, add upgrade notes for existing clusters, and list the eks-etcd-member helper. * docs: explain why addon manifests keep the CriticalAddonsOnly toleration Comment-only. The toleration stays for clusters created before control-plane hosts stopped registering as Nodes. * test: cover the etcd member client and CLI against a TLS fake etcd The real etcd client and the command had no tests, leaving the PR below the diff coverage threshold. A fake etcd serving the Cluster and Maintenance APIs over mutual TLS drives the client with the K3s certificate layout, and main becomes a testable run function.	2026-11-01 10:55:57 +10:00