release: update go.mod for v1.8.0

chore: rebuild frontend with latest deps
chore: bump the npm-dependencies group (#360 )
2026-10-24 05:04:03 +00:00 · 2026-06-16 09:32:18 +10:00 · 2026-06-16 09:06:50 +10:00 · 2026-06-16 08:58:38 +10:00 · 2026-06-16 08:57:51 +10:00 · 2026-06-16 00:59:36 +10:00
1855 changed files with 321430 additions and 35536 deletions
@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+set -e
+
+branch=$(git rev-parse --abbrev-ref HEAD)
+
+# Only enforce preflight on dev and main
+if [ "$branch" != "dev" ] && [ "$branch" != "main" ]; then
+  exit 0
+fi
+
+# Only run preflight if Go files were staged
+if git diff --cached --name-only | grep -q '\.go$'; then
+  echo "Go files staged on $branch — running make preflight..."
+  make preflight
+else
+  echo "No Go files staged, skipping preflight."
+fi
@@ -0,0 +1,15 @@
+# CODEOWNERS — GitHub PR review enforcement
+#
+# Format: <path pattern>  <owner> [<owner> …]
+# Order matters — last matching pattern wins.
+#
+# Set the matching branch protection rule on `dev` and `main` to
+# "Require review from Code Owners" or this file is advisory only.
+
+# Network layer is governed by ADR-0006 invariant tests.
+# Changes here must respect those clauses
+# see spinifex/network/invariants/
+# docs/development/proposals/az-local-architecture/0006-spinifex-network-layer-contract.md
+/spinifex/network/          @joshsiv-mulga
+/spinifex/vpcd/             @joshsiv-mulga
+/spinifex/network/invariants/ @joshsiv-mulga
@@ -0,0 +1,104 @@
+# e2e-analyze
+
+Post-test failure-clustering action for the Go E2E suites.
+
+## What it does
+
+Runs after `go-junit-report` has converted each suite's stdout into
+`junit-<suite>.xml`. For every JUnit file matched by `junit-glob`:
+
+1. Parses each `<testcase>` with a `<failure>` / `<error>` child.
+2. Strips ANSI escapes, then normalises dynamic noise (UUIDs, EC2 IDs,
+   IPs, timestamps, request-ids, source line numbers) to produce a stable
+   _signature_ per failure.
+3. Picks the **earliest non-cascade** failure per suite as the likely
+   root cause. "Cascade" = failure message matches a fixture-dependency
+   marker (`must populate fix.`, `Should NOT be empty`,
+   `Expected value not to be nil`). When every failure is a cascade
+   (whole suite tripped on one upstream gap), the earliest cascade is
+   the fallback root.
+4. Buckets the remaining failures by signature and renders a markdown
+   report. The report is appended to `$GITHUB_STEP_SUMMARY` and also
+   written to `<log-dir>/analysis.md` so it travels with the uploaded
+   artifact bundle.
+
+The action **never fails the job** — exit-code semantics stay with the
+existing "Fail job if any suite failed" gate.
+
+## Inputs
+
+| Input        | Required | Description |
+|--------------|----------|-------------|
+| `junit-glob` | yes      | Shell glob for the JUnit XML files (e.g. `${ARTIFACT_DIR}/junit-*.xml`). |
+| `log-dir`    | yes      | Directory the rendered report is written into (`analysis.md`). |
+| `title`      | no       | Heading at the top of the summary section. Default `E2E failure analysis`. |
+
+## Outputs
+
+None. The summary is appended to `$GITHUB_STEP_SUMMARY`; the same
+content lands at `<log-dir>/analysis.md`.
+
+## Local invocation
+
+Outside Actions, the analyser prints to stdout instead of appending to a
+summary file:
+
+```sh
+cd spinifex/.github/actions/e2e-analyze
+go run . -junit-glob "/tmp/artifacts/junit-*.xml" -log-dir /tmp/artifacts
+```
+
+## Tests
+
+Golden-file tests live in `testdata/`. Each subdir holds a scenario:
+
+| Scenario           | Source                                    |
+|--------------------|-------------------------------------------|
+| `cascade-vpcs`     | Synthetic: one Phase 5 root + 15 cascades |
+| `single-failure`   | Real artifact from run `26037892507`      |
+| `no-failures`      | Synthetic: clean run                      |
+
+Run normally:
+
+```sh
+go test ./.github/actions/e2e-analyze/
+```
+
+When the report format changes intentionally, regenerate the snapshots:
+
+```sh
+go test ./.github/actions/e2e-analyze/ -update
+```
+
+## Per-failure bundles (Stage 2)
+
+After parsing, the action materialises a bundle directory per failure
+under `<log-dir>/analysis/NN-<TestName>/`. The report links to it from
+each entry so a reader on the Actions page can click straight through
+to the relevant log slice.
+
+Each bundle contains:
+
+| File              | Source                                                                            |
+|-------------------|-----------------------------------------------------------------------------------|
+| `failure.txt`     | Synthesised summary: test, start, duration, file-site, error line.                |
+| `journal.log`     | `spinifex-journal.log` sliced to `[StartAt − 5s, StartAt + Duration + 5s]`.       |
+| `test.log`        | `test-<suite>.log` block between `=== RUN <name>` and the matching `--- FAIL`.    |
+| `<other files>`   | Anything the harness already dropped into `<log-dir>/<TestName>/` at run-time.    |
+
+The journal slice relies on the workflow running
+`journalctl --output=short-iso` so timestamps are parseable. Lines
+without a parseable timestamp inherit the previous line's
+in-window/out-of-window classification (so multi-line records and stack
+traces stay contiguous).
+
+## Roadmap
+
+- **Stage 1 (shipped):** first-failure surfacing + cascade clustering.
+- **Stage 2 (this update):** time-window correlation — per-failure bundle
+  with sliced journal + test-log + any per-test artifacts.
+- **Stage 3:** rule-based root-cause classifier driven by
+  `spinifex/tests/e2e/.failure-rules.yaml`.
+
+See `docs/development/improvements/e2e-go-failure-analysis.md` in the
+parent mulga repo.
@@ -0,0 +1,38 @@
+name: E2E Failure Analyzer
+description: >-
+  Post-test analyzer for the Go E2E suites. Parses junit-*.xml emitted by
+  go-junit-report, clusters failures by signature, picks the earliest
+  non-cascade failure per suite as the likely root cause, and appends a
+  human-readable report to $GITHUB_STEP_SUMMARY. Always exits 0 — the
+  existing "Fail job if any suite failed" gate keeps owning exit-code
+  semantics.
+
+inputs:
+  junit-glob:
+    description: Shell glob matching JUnit XML files to analyze.
+    required: true
+  log-dir:
+    description: >-
+      Artifact directory. The rendered report is also written to
+      $log-dir/analysis.md so it travels with the uploaded artifact.
+    required: true
+  title:
+    description: Heading rendered at the top of the summary section.
+    required: false
+    default: "E2E failure analysis"
+
+runs:
+  using: composite
+  steps:
+    - id: analyze
+      shell: bash
+      working-directory: ${{ github.action_path }}
+      env:
+        ANALYZE_JUNIT_GLOB: ${{ inputs.junit-glob }}
+        ANALYZE_LOG_DIR: ${{ inputs.log-dir }}
+        ANALYZE_TITLE: ${{ inputs.title }}
+      run: |
+        go run . \
+          -junit-glob "$ANALYZE_JUNIT_GLOB" \
+          -log-dir "$ANALYZE_LOG_DIR" \
+          -title "$ANALYZE_TITLE"
@@ -0,0 +1,552 @@
+// Package main implements the e2e-analyze GitHub Action.
+//
+// Stage 1 of docs/development/improvements/e2e-go-failure-analysis.md:
+// cluster failures from go-junit-report's junit-*.xml by error signature,
+// pick the earliest non-cascade failure per suite as the likely root cause,
+// and render a human-readable report.
+//
+// This file contains the pure logic (parsing, signature extraction,
+// clustering, rendering). Side-effect handling lives in main.go.
+package main
+
+import (
+	"encoding/xml"
+	"fmt"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+	"time"
+)
+
+// JUnit XML schema produced by go-junit-report v2. Stable across versions
+// we care about; only fields used downstream are decoded.
+type junitSuites struct {
+	XMLName xml.Name     `xml:"testsuites"`
+	Suites  []junitSuite `xml:"testsuite"`
+}
+
+type junitSuite struct {
+	Name      string    `xml:"name,attr"`
+	Tests     int       `xml:"tests,attr"`
+	Failures  int       `xml:"failures,attr"`
+	Errors    int       `xml:"errors,attr"`
+	Skipped   int       `xml:"skipped,attr"`
+	Timestamp string    `xml:"timestamp,attr"`
+	Time      float64   `xml:"time,attr"`
+	Cases     []junitTC `xml:"testcase"`
+}
+
+type junitTC struct {
+	Name      string        `xml:"name,attr"`
+	Classname string        `xml:"classname,attr"`
+	Time      float64       `xml:"time,attr"`
+	Failure   *junitFailure `xml:"failure"`
+	Error     *junitFailure `xml:"error"`
+	Skipped   *junitSkipped `xml:"skipped"`
+	SystemOut string        `xml:"system-out"`
+}
+
+type junitFailure struct {
+	Message string `xml:"message,attr"`
+	Type    string `xml:"type,attr"`
+	Body    string `xml:",chardata"`
+}
+
+type junitSkipped struct {
+	Message string `xml:"message,attr"`
+}
+
+// Failure is the analyzer's normalized view of one failed testcase.
+type Failure struct {
+	SuiteFile string    // e.g. "single" (derived from junit-single.xml)
+	Name      string    // e.g. "TestSingleNode/Phase5_LaunchInstance"
+	Duration  float64   // seconds
+	Order     int       // position within the suite (0-based)
+	StartAt   time.Time // estimated wall-clock start: suite.timestamp + sum(prior durations)
+	Error     string    // last meaningful error line, ANSI-stripped
+	FileHint  string    // e.g. "vpc_test.go:227", parsed from failure body
+	Cascade   bool      // signature matched a known downstream pattern
+	Signature string    // normalized error key for bucketing
+	// OffsetFromSuiteStart is sum(prior testcase durations) at the time
+	// this test started. Persisted separately from StartAt so a caller
+	// that has a more authoritative suite wall-clock origin (e.g. a
+	// `test-<suite>.start` file written by the workflow before the suite
+	// runs) can recompute StartAt without re-parsing the XML.
+	OffsetFromSuiteStart time.Duration
+	// BundlePath is the relative path (from log-dir) to the per-failure
+	// bundle written by Stage 2's WriteBundles. Empty when bundles were
+	// not generated (e.g. unit tests that call Render directly).
+	BundlePath string
+}
+
+// SuiteReport summarises one junit-*.xml file.
+type SuiteReport struct {
+	File      string // basename, e.g. "junit-single.xml"
+	Label     string // short name, e.g. "single"
+	Total     int
+	FailCount int
+	Root      *Failure    // nil if no failures
+	Cascades  []Failure   // everything else that failed in this suite
+	Buckets   [][]Failure // failures grouped by signature (root's bucket first)
+}
+
+// Report is the full analysis across every junit file.
+type Report struct {
+	Title  string
+	Suites []SuiteReport
+}
+
+// ansiSeq matches ANSI CSI escapes the harness sprinkles through log
+// output (colours, bold). Three forms in the wild:
+//   - real ESC: `\x1b[36m…`
+//   - U+FFFD replacement (XML serialiser dropped the ESC byte): `�[36m…`
+//   - bare CSI with the leading byte gone entirely: `[36m…`
+//
+// All three are stripped so colour state doesn't fork the signature bucket.
+var ansiSeq = regexp.MustCompile(
+	"\x1b\\[[0-9;]*[a-zA-Z]" + // real ESC CSI
+		"|�\\[[0-9;]*[a-zA-Z]" + // replacement-char CSI
+		"|\\[[0-9]{1,2}(?:;[0-9]{1,2})*m", // bare colour code, "mode" form only
+)
+
+// cascadeMarkers are message fragments that only appear in tests that
+// depend on a fixture field populated by an earlier phase. Their presence
+// means the failure is downstream of some other failure that happened
+// first; the report collapses them under that root cause.
+var cascadeMarkers = []string{
+	"must populate fix.",
+	"Should NOT be empty",
+	"Expected value not to be nil",
+}
+
+// noiseReplacers normalise dynamic identifiers so that "ten failures, same
+// bug" bucket together. Pattern ordering matters: longer/more-specific
+// patterns must precede the catch-alls.
+var noiseReplacers = []struct {
+	re   *regexp.Regexp
+	repl string
+}{
+	{regexp.MustCompile(`\b(vpc|subnet|sg|rtb|igw|i|ami|vol|snap|rtbassoc|key)-[0-9a-f]{8,}\b`), `${1}-<id>`},
+	{regexp.MustCompile(`\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b`), `<uuid>`},
+	{regexp.MustCompile(`\b[0-9a-f]{32,}\b`), `<hex>`},
+	{regexp.MustCompile(`\b\d{1,3}(\.\d{1,3}){3}(:\d+)?\b`), `<ip>`},
+	{regexp.MustCompile(`\b\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?(?:Z|[+-]\d{2}:?\d{2})?\b`), `<ts>`},
+	{regexp.MustCompile(`request-id:\s*\S+`), `request-id: <id>`},
+	{regexp.MustCompile(`(_test\.go):\d+`), `${1}:<line>`},
+	{regexp.MustCompile(`\s+`), ` `},
+}
+
+// fileHintRe pulls the last "foo_test.go:NNN" we see in a failure body so
+// the report can point a reader at the source line. Test files dominate the
+// hint set; harness/helper file matches go through goFileLineRe below.
+var fileHintRe = regexp.MustCompile(`([A-Za-z0-9_]+_test\.go:\d+)`)
+
+// goFileLineRe matches any "foo.go:NNN:" prefix (test or non-test). Used to
+// surface harness/helper-level error text like
+// `ec2helpers.go:164: describe-vpcs: InvalidParameterValue …` which Go test
+// emits when a non-test file calls t.Errorf.
+var goFileLineRe = regexp.MustCompile(`^[A-Za-z0-9_]+\.go:\d+:\s*(.+)$`)
+
+// testifyErrorPlaceholder is testify's "Error:" boilerplate when the
+// assertion failed because of a wrapped error. The next non-blank line in
+// the trace holds the actual error text.
+const testifyErrorPlaceholder = "Received unexpected error:"
+
+// stripANSI removes CSI escapes; leaves everything else intact.
+func stripANSI(s string) string {
+	return ansiSeq.ReplaceAllString(s, "")
+}
+
+// extractErrorLine finds the most useful single-line summary in a failure
+// body. Priority order, highest first:
+//  1. Testify "Messages:" line — user-supplied context, the most signal-rich.
+//  2. Testify "Error:" content (with one-line lookahead when the placeholder
+//     "Received unexpected error:" defers to the next line).
+//  3. Leading "foo.go:NNN: text" content line — t.Errorf from non-test code.
+//  4. Last `_test.go:NNN: text` line — bare t.Fatalf/Errorf from a test file.
+//  5. First non-empty line of the body.
+func extractErrorLine(body string) string {
+	body = stripANSI(body)
+	lines := strings.Split(body, "\n")
+
+	var (
+		msgLine      string
+		errLine      string
+		goFileLine   string
+		lastTestLine string
+	)
+	for i, raw := range lines {
+		l := strings.TrimSpace(raw)
+		if l == "" {
+			continue
+		}
+		if strings.HasPrefix(l, "Messages:") {
+			msgLine = strings.TrimSpace(strings.TrimPrefix(l, "Messages:"))
+			continue
+		}
+		if strings.HasPrefix(l, "Error:") && errLine == "" {
+			content := strings.TrimSpace(strings.TrimPrefix(l, "Error:"))
+			if content == testifyErrorPlaceholder {
+				// Look ahead for the wrapped error text on the next non-blank line.
+				for j := i + 1; j < len(lines); j++ {
+					nxt := strings.TrimSpace(lines[j])
+					if nxt == "" {
+						continue
+					}
+					if strings.HasPrefix(nxt, "Test:") || strings.HasPrefix(nxt, "Messages:") {
+						break
+					}
+					content = content + " " + nxt
+					break
+				}
+			}
+			errLine = content
+			continue
+		}
+		if m := goFileLineRe.FindStringSubmatch(l); m != nil {
+			// Keep the LAST occurrence: t.Log / harness traces emit
+			// progress lines from the same file:line pattern before the
+			// failing assertion finally fires.
+			goFileLine = strings.TrimSpace(m[1])
+			continue
+		}
+		if fileHintRe.MatchString(l) {
+			lastTestLine = l
+		}
+	}
+	switch {
+	case msgLine != "":
+		return msgLine
+	case errLine != "":
+		return errLine
+	case goFileLine != "":
+		return goFileLine
+	case lastTestLine != "":
+		// "vpc_test.go:227: Eventually: condition not met …" → trim the file prefix.
+		if i := strings.Index(lastTestLine, ": "); i > 0 {
+			return strings.TrimSpace(lastTestLine[i+2:])
+		}
+		return lastTestLine
+	}
+	// Last resort: first non-empty line of the body.
+	for _, raw := range lines {
+		l := strings.TrimSpace(raw)
+		if l != "" {
+			return l
+		}
+	}
+	return ""
+}
+
+// errorTraceRe extracts the path from testify's "Error Trace:" line — e.g.
+//
+//	Error Trace:	tests/e2e/harness/ec2helpers.go:164
+//
+// — which points at the assertion site even when it lives in helper code
+// outside *_test.go.
+var errorTraceRe = regexp.MustCompile(`Error Trace:\s*(\S+\.go:\d+)`)
+
+// extractFileHint returns the most precise pointer at the assertion site
+// we can pull from the failure body: testify Error Trace wins, otherwise
+// the last `*_test.go:NNN` line in the body.
+func extractFileHint(body string) string {
+	body = stripANSI(body)
+	if m := errorTraceRe.FindStringSubmatch(body); m != nil {
+		return m[1]
+	}
+	all := fileHintRe.FindAllString(body, -1)
+	if len(all) == 0 {
+		return ""
+	}
+	return all[len(all)-1]
+}
+
+// signature normalises an error line so different runs of the same root
+// cause bucket together. Empty signatures (e.g. parent-test placeholders
+// where the failure body is empty) bucket together under "".
+func signature(errLine string) string {
+	s := strings.ToLower(errLine)
+	for _, r := range noiseReplacers {
+		s = r.re.ReplaceAllString(s, r.repl)
+	}
+	return strings.TrimSpace(s)
+}
+
+// isCascade reports whether the error line contains any of the well-known
+// fixture-propagation markers — i.e. "this test failed because something
+// upstream of it failed first".
+func isCascade(errLine string) bool {
+	for _, m := range cascadeMarkers {
+		if strings.Contains(errLine, m) {
+			return true
+		}
+	}
+	return false
+}
+
+// computeLeafSet returns the testcase names that are leaves — i.e. no
+// other testcase name in `cases` starts with `<name>/`. go-junit-report
+// emits one <testcase> for the parent Go test AND each subtest, with the
+// parent's time = sum of its children, so summing every entry into the
+// suite's cumulative offset double-counts. Restrict accumulation to
+// leaves only.
+func computeLeafSet(cases []junitTC) map[string]bool {
+	leaf := make(map[string]bool, len(cases))
+	for _, tc := range cases {
+		leaf[tc.Name] = true
+	}
+	for _, tc := range cases {
+		// Strip the deepest path segment; any ancestor of this test is
+		// not a leaf.
+		name := tc.Name
+		for {
+			i := strings.LastIndex(name, "/")
+			if i <= 0 {
+				break
+			}
+			name = name[:i]
+			if _, ok := leaf[name]; ok {
+				leaf[name] = false
+			}
+		}
+	}
+	return leaf
+}
+
+// parseStartTime turns a junit timestamp attr (RFC3339) into a time.Time;
+// zero value if unparseable.
+func parseStartTime(s string) time.Time {
+	if s == "" {
+		return time.Time{}
+	}
+	t, err := time.Parse(time.RFC3339, s)
+	if err != nil {
+		return time.Time{}
+	}
+	return t
+}
+
+// suiteLabel turns "junit-single.xml" into "single".
+func suiteLabel(path string) string {
+	base := filepath.Base(path)
+	base = strings.TrimSuffix(base, ".xml")
+	base = strings.TrimPrefix(base, "junit-")
+	if base == "" {
+		return filepath.Base(path)
+	}
+	return base
+}
+
+// ParseFile decodes one junit-*.xml file into our normalized failure list.
+// Returns the suite label, total testcase count, and the (ordered) failures.
+func ParseFile(path string, data []byte) (SuiteReport, error) {
+	var doc junitSuites
+	if err := xml.Unmarshal(data, &doc); err != nil {
+		return SuiteReport{}, fmt.Errorf("parse %s: %w", path, err)
+	}
+
+	rep := SuiteReport{
+		File:  filepath.Base(path),
+		Label: suiteLabel(path),
+	}
+
+	order := 0
+	for _, suite := range doc.Suites {
+		start := parseStartTime(suite.Timestamp)
+		// go-junit-report emits a testcase for both the parent Go test
+		// (e.g. `TestSingleNode`) AND each subtest (`TestSingleNode/PhaseX`).
+		// The parent's `time` attribute is the aggregate of its children, so
+		// summing every testcase double-counts. Pre-compute the leaf set so
+		// only leaves contribute to the suite's running offset.
+		isLeaf := computeLeafSet(suite.Cases)
+		cumul := 0.0
+		for _, tc := range suite.Cases {
+			rep.Total++
+			leaf := isLeaf[tc.Name]
+			var body string
+			switch {
+			case tc.Failure != nil:
+				body = tc.Failure.Body
+			case tc.Error != nil:
+				body = tc.Error.Body
+			default:
+				if leaf {
+					cumul += tc.Time
+				}
+				continue
+			}
+
+			// Skip the synthetic parent-test "Failed" stub that go-junit-report
+			// emits for any Go test whose subtests failed. It has an empty body
+			// and would otherwise outrank real failures by XML order.
+			trimmed := strings.TrimSpace(stripANSI(body))
+			if trimmed == "" {
+				if leaf {
+					cumul += tc.Time
+				}
+				continue
+			}
+
+			errLine := extractErrorLine(body)
+			offset := time.Duration(cumul * float64(time.Second))
+			f := Failure{
+				SuiteFile:            rep.Label,
+				Name:                 tc.Name,
+				Duration:             tc.Time,
+				Order:                order,
+				Error:                errLine,
+				FileHint:             extractFileHint(body),
+				Cascade:              isCascade(errLine),
+				Signature:            signature(errLine),
+				OffsetFromSuiteStart: offset,
+			}
+			if !start.IsZero() {
+				f.StartAt = start.Add(offset)
+			}
+			rep.FailCount++
+			rep.Cascades = append(rep.Cascades, f) // temp staging; split below
+			order++
+			if leaf {
+				cumul += tc.Time
+			}
+		}
+	}
+
+	if rep.FailCount == 0 {
+		rep.Cascades = nil
+		return rep, nil
+	}
+
+	all := rep.Cascades
+	rep.Cascades = nil
+
+	// Root = earliest non-cascade. If every failure is a cascade (the whole
+	// suite tripped on the same upstream gap), fall back to earliest overall.
+	rootIdx := -1
+	for i, f := range all {
+		if !f.Cascade {
+			rootIdx = i
+			break
+		}
+	}
+	if rootIdx == -1 {
+		rootIdx = 0
+	}
+	root := all[rootIdx]
+	rep.Root = &root
+
+	// Bucket the remainder by signature so the report can collapse the
+	// "ten failures, same bug" pattern.
+	bucketKeys := []string{}
+	buckets := map[string][]Failure{}
+	for i, f := range all {
+		if i == rootIdx {
+			continue
+		}
+		if _, seen := buckets[f.Signature]; !seen {
+			bucketKeys = append(bucketKeys, f.Signature)
+		}
+		buckets[f.Signature] = append(buckets[f.Signature], f)
+		rep.Cascades = append(rep.Cascades, f)
+	}
+
+	// Largest bucket first; ties broken by first-seen order so output is
+	// deterministic across runs.
+	sort.SliceStable(bucketKeys, func(i, j int) bool {
+		return len(buckets[bucketKeys[i]]) > len(buckets[bucketKeys[j]])
+	})
+	for _, k := range bucketKeys {
+		rep.Buckets = append(rep.Buckets, buckets[k])
+	}
+
+	return rep, nil
+}
+
+// Render writes the markdown report. Format is documented in
+// docs/development/improvements/e2e-go-failure-analysis.md (Stage 1).
+func Render(r Report) string {
+	var b strings.Builder
+	fmt.Fprintf(&b, "## %s\n\n", r.Title)
+
+	totalFail := 0
+	for _, s := range r.Suites {
+		totalFail += s.FailCount
+	}
+	if totalFail == 0 {
+		b.WriteString("✅ No failures across any suite.\n")
+		return b.String()
+	}
+
+	for _, s := range r.Suites {
+		if s.FailCount == 0 {
+			fmt.Fprintf(&b, "### Suite `%s`: ✅ pass (%d tests)\n\n", s.Label, s.Total)
+			continue
+		}
+		fmt.Fprintf(&b, "### Suite `%s`: %d failed, 1 root cause likely\n\n", s.Label, s.FailCount)
+
+		if s.Root != nil {
+			b.WriteString("**Root cause (earliest non-cascade)**\n\n")
+			fmt.Fprintf(&b, "- Test: `%s`\n", s.Root.Name)
+			if !s.Root.StartAt.IsZero() {
+				fmt.Fprintf(&b, "- Start: %s (duration %.1fs)\n",
+					s.Root.StartAt.Format("15:04:05"), s.Root.Duration)
+			} else {
+				fmt.Fprintf(&b, "- Duration: %.1fs\n", s.Root.Duration)
+			}
+			if s.Root.Error != "" {
+				fmt.Fprintf(&b, "- Error: %s\n", oneLine(s.Root.Error))
+			}
+			if s.Root.FileHint != "" {
+				fmt.Fprintf(&b, "- File:  `%s`\n", s.Root.FileHint)
+			}
+			if s.Root.BundlePath != "" {
+				fmt.Fprintf(&b, "- Bundle: `%s/`\n", s.Root.BundlePath)
+			}
+			b.WriteString("\n")
+		}
+
+		if len(s.Buckets) > 0 {
+			fmt.Fprintf(&b, "**Cascaded failures (%d) — grouped by signature**\n\n",
+				len(s.Cascades))
+			for _, bucket := range s.Buckets {
+				sample := bucket[0]
+				label := oneLine(sample.Error)
+				if label == "" {
+					label = "(no error message)"
+				}
+				fmt.Fprintf(&b, "- %d× %s\n", len(bucket), label)
+				// List up to 5 test names per bucket; collapse the rest.
+				const cap = 5
+				for i, f := range bucket {
+					if i == cap && len(bucket) > cap {
+						fmt.Fprintf(&b, "  - … and %d more\n", len(bucket)-cap)
+						break
+					}
+					if f.BundlePath != "" {
+						fmt.Fprintf(&b, "  - `%s` → `%s/`\n", f.Name, f.BundlePath)
+					} else {
+						fmt.Fprintf(&b, "  - `%s`\n", f.Name)
+					}
+				}
+			}
+			b.WriteString("\n")
+		}
+	}
+
+	return b.String()
+}
+
+// oneLine collapses whitespace and caps length so a runaway assertion
+// message doesn't blow out the summary card width.
+func oneLine(s string) string {
+	s = strings.TrimSpace(s)
+	s = strings.ReplaceAll(s, "\n", " ")
+	s = strings.ReplaceAll(s, "\r", " ")
+	s = regexp.MustCompile(`\s+`).ReplaceAllString(s, " ")
+	const max = 160
+	if len(s) > max {
+		s = s[:max-1] + "…"
+	}
+	return s
+}
@@ -0,0 +1,191 @@
+package main
+
+import (
+	"flag"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	"testing"
+)
+
+// updateGolden lets `go test ./.github/actions/e2e-analyze/ -update` rewrite
+// the expected analysis.md files when the rendering format changes
+// intentionally. Day-to-day, leave it unset: tests diff against the on-disk
+// snapshot to catch unintended drift in signature extraction or layout.
+var updateGolden = flag.Bool("update", false, "rewrite testdata/*/analysis.md from current output")
+
+// TestGoldenFixtures runs the full ParseFile + Render pipeline against each
+// fixture directory under testdata/ and asserts the output matches the
+// adjacent analysis.md snapshot. Each subdir is one named scenario; the
+// fixture XMLs match the same junit-*.xml glob the production action uses
+// so behaviour stays representative of CI.
+func TestGoldenFixtures(t *testing.T) {
+	entries, err := os.ReadDir("testdata")
+	if err != nil {
+		t.Fatalf("read testdata: %v", err)
+	}
+	for _, e := range entries {
+		if !e.IsDir() {
+			continue
+		}
+		name := e.Name()
+		t.Run(name, func(t *testing.T) {
+			dir := filepath.Join("testdata", name)
+			matches, err := filepath.Glob(filepath.Join(dir, "junit-*.xml"))
+			if err != nil {
+				t.Fatalf("glob: %v", err)
+			}
+			sort.Strings(matches)
+			if len(matches) == 0 {
+				t.Fatalf("no junit-*.xml fixtures in %s", dir)
+			}
+
+			rep := Report{Title: "E2E failure analysis"}
+			for _, p := range matches {
+				data, err := os.ReadFile(p)
+				if err != nil {
+					t.Fatalf("read %s: %v", p, err)
+				}
+				sr, err := ParseFile(p, data)
+				if err != nil {
+					t.Fatalf("parse %s: %v", p, err)
+				}
+				rep.Suites = append(rep.Suites, sr)
+			}
+
+			got := Render(rep)
+			goldenPath := filepath.Join(dir, "analysis.md")
+			if *updateGolden {
+				if err := os.WriteFile(goldenPath, []byte(got), 0o644); err != nil {
+					t.Fatalf("write golden: %v", err)
+				}
+				return
+			}
+			want, err := os.ReadFile(goldenPath)
+			if err != nil {
+				t.Fatalf("read golden (run with -update to seed): %v", err)
+			}
+			if string(want) != got {
+				t.Errorf("rendered output differs from %s\n--- want ---\n%s\n--- got ---\n%s",
+					goldenPath, string(want), got)
+			}
+		})
+	}
+}
+
+func TestSignatureCollapsesNoise(t *testing.T) {
+	a := signature("describe-vpcs: InvalidParameterValue request-id: abc-123 vpc-deadbeefcafe1234")
+	b := signature("describe-vpcs: InvalidParameterValue request-id: xyz-999 vpc-1111222233334444")
+	if a != b {
+		t.Errorf("signatures should collapse to same bucket:\n  a=%q\n  b=%q", a, b)
+	}
+	if !strings.Contains(a, "<id>") || !strings.Contains(a, "vpc-<id>") {
+		t.Errorf("expected normalised placeholders in signature, got %q", a)
+	}
+}
+
+func TestCascadeDetection(t *testing.T) {
+	cases := []struct {
+		in   string
+		want bool
+	}{
+		{"Phase 5 must populate fix.InstanceID", true},
+		{"Should NOT be empty", true},
+		{"Expected value not to be nil", true},
+		{"describe-vpcs: InvalidParameterValue", false},
+		{"Eventually: condition not met within 3m0s", false},
+	}
+	for _, c := range cases {
+		if got := isCascade(c.in); got != c.want {
+			t.Errorf("isCascade(%q) = %v, want %v", c.in, got, c.want)
+		}
+	}
+}
+
+func TestExtractErrorLine_MessagesWinsOverError(t *testing.T) {
+	body := `    lifecycle_test.go:28:
+        Error Trace:	tests/e2e/single/lifecycle_test.go:28
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase5a_pre_ClusterStats`
+	got := extractErrorLine(body)
+	want := "Phase 5 must populate fix.InstanceID"
+	if got != want {
+		t.Errorf("extractErrorLine = %q, want %q", got, want)
+	}
+}
+
+func TestExtractErrorLine_FallsBackToLastTestLine(t *testing.T) {
+	body := "    vpc_test.go:227: Eventually: condition not met within 3m0s: [SSH handshake never completed]"
+	got := extractErrorLine(body)
+	if !strings.HasPrefix(got, "Eventually:") {
+		t.Errorf("expected fallback to test-line content, got %q", got)
+	}
+}
+
+func TestExtractFileHint_PicksLastMatch(t *testing.T) {
+	body := "ec2helpers.go:50: setup\n    vpc_test.go:227: Eventually: condition not met"
+	got := extractFileHint(body)
+	want := "vpc_test.go:227"
+	if got != want {
+		t.Errorf("extractFileHint = %q, want %q", got, want)
+	}
+}
+
+func TestParseFile_SkipsEmptyParentFailure(t *testing.T) {
+	// The synthetic "TestSingleNode" parent gets a <failure message="Failed"/>
+	// with no body whenever any subtest fails. The analyzer must skip it so
+	// it doesn't outrank the real first failure on XML order.
+	xml := []byte(`<?xml version="1.0"?>
+<testsuites><testsuite name="" tests="2" failures="2">
+  <testcase name="TestX" time="1.0"><failure message="Failed"></failure></testcase>
+  <testcase name="TestX/RealFailure" time="0.5"><failure message="Failed"><![CDATA[
+    foo_test.go:10:
+        Messages: real assertion message
+]]></failure></testcase>
+</testsuite></testsuites>`)
+	sr, err := ParseFile("junit-x.xml", xml)
+	if err != nil {
+		t.Fatalf("ParseFile: %v", err)
+	}
+	if sr.FailCount != 1 {
+		t.Fatalf("FailCount = %d, want 1 (empty parent should be skipped)", sr.FailCount)
+	}
+	if sr.Root == nil || sr.Root.Name != "TestX/RealFailure" {
+		t.Fatalf("expected real failure as root, got %+v", sr.Root)
+	}
+}
+
+func TestParseFile_CascadeFallbackWhenAllCascades(t *testing.T) {
+	// If every failure is a cascade marker, the earliest one becomes root
+	// so the report isn't blank.
+	xml := []byte(`<?xml version="1.0"?>
+<testsuites><testsuite name="" tests="2" failures="2">
+  <testcase name="TestX/A" time="1.0"><failure message="Failed"><![CDATA[
+    Messages: Phase 5 must populate fix.InstanceID
+]]></failure></testcase>
+  <testcase name="TestX/B" time="1.0"><failure message="Failed"><![CDATA[
+    Messages: Phase 5 must populate fix.InstanceID
+]]></failure></testcase>
+</testsuite></testsuites>`)
+	sr, err := ParseFile("junit-x.xml", xml)
+	if err != nil {
+		t.Fatalf("ParseFile: %v", err)
+	}
+	if sr.Root == nil || sr.Root.Name != "TestX/A" {
+		t.Fatalf("expected TestX/A as fallback root, got %+v", sr.Root)
+	}
+}
+
+func TestRender_NoFailuresIsClean(t *testing.T) {
+	out := Render(Report{
+		Title: "E2E failure analysis",
+		Suites: []SuiteReport{
+			{File: "junit-single.xml", Label: "single", Total: 3, FailCount: 0},
+		},
+	})
+	if !strings.Contains(out, "No failures") {
+		t.Errorf("expected zero-failure banner, got:\n%s", out)
+	}
+}
@@ -0,0 +1,208 @@
+package main
+
+// Stage 2 of docs/development/improvements/e2e-go-failure-analysis.md:
+// for each failure surfaced by Stage 1, materialise a per-failure bundle
+// directory containing the failure summary, a time-window slice of the
+// daemon journal, a testcase-window slice of the suite's stdout log, and
+// any per-test artifacts the harness already dumped at run-time.
+
+import (
+	"errors"
+	"fmt"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"time"
+)
+
+// bundlePad is the per-failure window padding either side of the
+// [StartAt, StartAt+Duration] testcase span. Five seconds matches the
+// plan doc: long enough to catch the daemon's pre-test setup and
+// post-test cleanup chatter, short enough to keep the slice focused.
+const bundlePad = 5 * time.Second
+
+// nameSanitiseRe replaces characters unsafe in directory names with `_`.
+// Tracks tests/e2e/harness/artifacts.go which uses the same convention
+// when the test's own dump dir is created, so we can pick its contents
+// up by path without a second translation table.
+var nameSanitiseRe = regexp.MustCompile(`[^A-Za-z0-9._-]+`)
+
+func sanitiseTestName(s string) string {
+	return nameSanitiseRe.ReplaceAllString(s, "_")
+}
+
+// ApplySuiteStartFiles overrides each failure's StartAt by reading
+// <logDir>/test-<suiteLabel>.start (RFC3339 timestamp written by the
+// workflow before the suite runs). Required because go-junit-report's
+// `timestamp` attribute on <testsuite> records the time the XML was
+// produced — i.e. after the suite has finished — so it can't be used
+// as the suite wall-clock origin for journal slicing.
+//
+// Missing or unparseable files are silently ignored: the analyzer
+// falls back to the junit timestamp and the bundle still gets created,
+// it just may have an empty (or off-window) journal slice.
+func ApplySuiteStartFiles(rep *Report, logDir string) {
+	if logDir == "" {
+		return
+	}
+	for si := range rep.Suites {
+		s := &rep.Suites[si]
+		startPath := filepath.Join(logDir, fmt.Sprintf("test-%s.start", s.Label))
+		data, err := os.ReadFile(startPath)
+		if err != nil {
+			continue
+		}
+		ts, err := time.Parse(time.RFC3339, strings.TrimSpace(string(data)))
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: parse %s: %v\n", startPath, err)
+			continue
+		}
+		if s.Root != nil {
+			s.Root.StartAt = ts.Add(s.Root.OffsetFromSuiteStart)
+		}
+		for ci := range s.Cascades {
+			s.Cascades[ci].StartAt = ts.Add(s.Cascades[ci].OffsetFromSuiteStart)
+		}
+		for bi := range s.Buckets {
+			for fi := range s.Buckets[bi] {
+				f := &s.Buckets[bi][fi]
+				f.StartAt = ts.Add(f.OffsetFromSuiteStart)
+			}
+		}
+	}
+}
+
+// WriteBundles materialises per-failure bundles under <logDir>/analysis/.
+// Mutates rep in-place so each Failure.BundlePath holds the relative
+// path the renderer will link from the summary.
+//
+// Failures with no parseable wall-clock start still get a bundle dir
+// (and the testcase-window test.log slice), but no journal slice — the
+// time window is unknown.
+//
+// Errors are non-fatal: a bundle that partially fails still gets linked
+// from the report so a reader sees what's available, and the analyzer
+// keeps printing warnings so CI logs surface the gap.
+func WriteBundles(rep *Report, logDir string) {
+	if logDir == "" {
+		return
+	}
+	base := filepath.Join(logDir, "analysis")
+	if err := os.MkdirAll(base, 0o755); err != nil {
+		fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: mkdir %s: %v\n", base, err)
+		return
+	}
+
+	for si := range rep.Suites {
+		s := &rep.Suites[si]
+		if s.Root != nil {
+			writeOneBundle(logDir, base, s.Label, s.Root)
+		}
+		for ci := range s.Cascades {
+			writeOneBundle(logDir, base, s.Label, &s.Cascades[ci])
+		}
+		// Bucket entries share storage with Cascades but live in a
+		// separate slice in SuiteReport; mirror the bundle path so the
+		// renderer's bucket loop sees it too.
+		for bi := range s.Buckets {
+			for fi := range s.Buckets[bi] {
+				f := &s.Buckets[bi][fi]
+				if bp := findBundlePath(s, f.Name); bp != "" {
+					f.BundlePath = bp
+				}
+			}
+		}
+	}
+}
+
+// findBundlePath looks up a Failure's already-written BundlePath by test
+// name within a suite. Buckets reference the same logical failure as
+// Cascades but are independent slice values, so we copy the path across
+// rather than re-deriving it.
+func findBundlePath(s *SuiteReport, testName string) string {
+	if s.Root != nil && s.Root.Name == testName {
+		return s.Root.BundlePath
+	}
+	for _, c := range s.Cascades {
+		if c.Name == testName {
+			return c.BundlePath
+		}
+	}
+	return ""
+}
+
+func writeOneBundle(logDir, base, suiteLabel string, f *Failure) {
+	name := fmt.Sprintf("%02d-%s", f.Order+1, sanitiseTestName(f.Name))
+	dir := filepath.Join(base, name)
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: mkdir %s: %v\n", dir, err)
+		return
+	}
+	f.BundlePath = filepath.Join("analysis", name)
+
+	writeFailureSummary(dir, suiteLabel, f)
+
+	if !f.StartAt.IsZero() {
+		start := f.StartAt.Add(-bundlePad)
+		end := f.StartAt.Add(time.Duration(f.Duration * float64(time.Second))).Add(bundlePad)
+		journalSrc := filepath.Join(logDir, "spinifex-journal.log")
+		if _, err := SliceJournal(journalSrc, start, end, filepath.Join(dir, "journal.log")); err != nil {
+			fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: slice journal for %s: %v\n", f.Name, err)
+		}
+	}
+
+	testLogSrc := filepath.Join(logDir, fmt.Sprintf("test-%s.log", suiteLabel))
+	if _, err := SliceTestLog(testLogSrc, f.Name, filepath.Join(dir, "test.log")); err != nil {
+		fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: slice test log for %s: %v\n", f.Name, err)
+	}
+
+	// Per-test artifact subdir written by harness.ArtifactDir lives at
+	// <logDir>/<TestName with / replaced by _>/. Copy any files into
+	// the bundle so qemu/ovs/etc. dumps already collected by the test
+	// travel alongside the analysis.
+	perTestSrc := filepath.Join(logDir, strings.ReplaceAll(f.Name, "/", "_"))
+	if entries, err := os.ReadDir(perTestSrc); err == nil {
+		for _, e := range entries {
+			if e.IsDir() {
+				continue
+			}
+			_ = copyFile(filepath.Join(perTestSrc, e.Name()), filepath.Join(dir, e.Name()))
+		}
+	} else if !os.IsNotExist(err) && !errorsIsNotDir(err) {
+		fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: read per-test dir %s: %v\n", perTestSrc, err)
+	}
+}
+
+// errorsIsNotDir handles the case where the harness wrote a file at the
+// per-test name instead of a directory — best-effort skip.
+func errorsIsNotDir(err error) bool {
+	var pe *fs.PathError
+	if !errors.As(err, &pe) {
+		return false
+	}
+	return pe.Err != nil && strings.Contains(pe.Err.Error(), "not a directory")
+}
+
+func writeFailureSummary(dir, suiteLabel string, f *Failure) {
+	var b strings.Builder
+	fmt.Fprintf(&b, "Suite:    %s\n", suiteLabel)
+	fmt.Fprintf(&b, "Test:     %s\n", f.Name)
+	if !f.StartAt.IsZero() {
+		fmt.Fprintf(&b, "Start:    %s\n", f.StartAt.Format(time.RFC3339))
+	}
+	fmt.Fprintf(&b, "Duration: %.3fs\n", f.Duration)
+	if f.FileHint != "" {
+		fmt.Fprintf(&b, "Site:     %s\n", f.FileHint)
+	}
+	if f.Cascade {
+		b.WriteString("Cascade:  yes (downstream of another failure)\n")
+	}
+	if f.Error != "" {
+		b.WriteString("\nError:\n")
+		b.WriteString(oneLine(f.Error))
+		b.WriteString("\n")
+	}
+	_ = os.WriteFile(filepath.Join(dir, "failure.txt"), []byte(b.String()), 0o644)
+}
@@ -0,0 +1,229 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestSliceJournal_KeepsOnlyWindow(t *testing.T) {
+	src := filepath.Join(t.TempDir(), "journal.log")
+	dst := filepath.Join(t.TempDir(), "out.log")
+	body := strings.Join([]string{
+		"2026-05-19T12:32:30+1000 host spinifex-daemon[1]: pre",
+		"  continuation of pre",
+		"2026-05-19T12:32:35+1000 host spinifex-daemon[1]: IN window",
+		"  multiline inside window",
+		"2026-05-19T12:32:45+1000 host spinifex-daemon[1]: post",
+		"",
+	}, "\n")
+	if err := os.WriteFile(src, []byte(body), 0o644); err != nil {
+		t.Fatalf("write src: %v", err)
+	}
+	start, _ := time.Parse(time.RFC3339, "2026-05-19T12:32:33+10:00")
+	end, _ := time.Parse(time.RFC3339, "2026-05-19T12:32:40+10:00")
+	n, err := SliceJournal(src, start, end, dst)
+	if err != nil {
+		t.Fatalf("SliceJournal: %v", err)
+	}
+	if n != 2 {
+		t.Fatalf("written = %d, want 2", n)
+	}
+	got, _ := os.ReadFile(dst)
+	if !strings.Contains(string(got), "IN window") || !strings.Contains(string(got), "multiline inside window") {
+		t.Errorf("missing in-window content:\n%s", got)
+	}
+	if strings.Contains(string(got), "pre") || strings.Contains(string(got), "post") {
+		t.Errorf("kept out-of-window content:\n%s", got)
+	}
+}
+
+func TestSliceJournal_AcceptsRFC3339ColonOffset(t *testing.T) {
+	// systemd's `journalctl --output=short-iso` ships RFC3339 timestamps
+	// (offset includes a colon, e.g. "+10:00") on most versions. Earlier
+	// regex only matched "+1000" — make sure both forms work.
+	src := filepath.Join(t.TempDir(), "journal.log")
+	dst := filepath.Join(t.TempDir(), "out.log")
+	body := strings.Join([]string{
+		"2026-05-19T12:32:30+10:00 host spinifex-daemon[1]: pre",
+		"2026-05-19T12:32:35+10:00 host spinifex-daemon[1]: IN window",
+		"2026-05-19T12:32:50+10:00 host spinifex-daemon[1]: post",
+		"",
+	}, "\n")
+	if err := os.WriteFile(src, []byte(body), 0o644); err != nil {
+		t.Fatalf("write src: %v", err)
+	}
+	start, _ := time.Parse(time.RFC3339, "2026-05-19T12:32:33+10:00")
+	end, _ := time.Parse(time.RFC3339, "2026-05-19T12:32:40+10:00")
+	n, err := SliceJournal(src, start, end, dst)
+	if err != nil {
+		t.Fatalf("SliceJournal: %v", err)
+	}
+	if n != 1 {
+		t.Fatalf("written = %d, want 1", n)
+	}
+	got, _ := os.ReadFile(dst)
+	if !strings.Contains(string(got), "IN window") {
+		t.Errorf("missing in-window content:\n%s", got)
+	}
+}
+
+func TestSliceJournal_MissingSrcIsNotError(t *testing.T) {
+	dst := filepath.Join(t.TempDir(), "out.log")
+	n, err := SliceJournal("/no/such/file", time.Now(), time.Now(), dst)
+	if err != nil {
+		t.Fatalf("unexpected err: %v", err)
+	}
+	if n != 0 {
+		t.Errorf("written = %d, want 0", n)
+	}
+}
+
+func TestSliceTestLog_PicksOneTestcase(t *testing.T) {
+	src := filepath.Join(t.TempDir(), "test.log")
+	dst := filepath.Join(t.TempDir(), "out.log")
+	body := strings.Join([]string{
+		"=== RUN   TestSingleNode",
+		"  before subtests",
+		"=== RUN   TestSingleNode/Phase5_LaunchInstance",
+		"    runinstances",
+		"    ec2helpers.go:164: describe-vpcs: InvalidParameterValue",
+		"--- FAIL: TestSingleNode/Phase5_LaunchInstance (2.50s)",
+		"=== RUN   TestSingleNode/Phase6_Volumes",
+		"    other content",
+		"--- FAIL: TestSingleNode/Phase6_Volumes (0.10s)",
+		"--- FAIL: TestSingleNode (10.0s)",
+		"",
+	}, "\n")
+	if err := os.WriteFile(src, []byte(body), 0o644); err != nil {
+		t.Fatalf("write src: %v", err)
+	}
+	n, err := SliceTestLog(src, "TestSingleNode/Phase5_LaunchInstance", dst)
+	if err != nil {
+		t.Fatalf("SliceTestLog: %v", err)
+	}
+	if n != 4 {
+		t.Fatalf("written = %d, want 4 (RUN + 2 inner + FAIL)", n)
+	}
+	got, _ := os.ReadFile(dst)
+	s := string(got)
+	if !strings.Contains(s, "describe-vpcs: InvalidParameterValue") {
+		t.Errorf("missing failure line:\n%s", s)
+	}
+	if strings.Contains(s, "Phase6_Volumes") || strings.Contains(s, "before subtests") {
+		t.Errorf("leaked sibling test content:\n%s", s)
+	}
+}
+
+func TestSliceTestLog_UnknownTestcaseIsEmpty(t *testing.T) {
+	src := filepath.Join(t.TempDir(), "test.log")
+	dst := filepath.Join(t.TempDir(), "out.log")
+	if err := os.WriteFile(src, []byte("=== RUN   TestX\n--- PASS: TestX (0.0s)\n"), 0o644); err != nil {
+		t.Fatalf("write src: %v", err)
+	}
+	n, err := SliceTestLog(src, "TestY", dst)
+	if err != nil {
+		t.Fatalf("SliceTestLog: %v", err)
+	}
+	if n != 0 {
+		t.Errorf("written = %d, want 0", n)
+	}
+	if _, err := os.Stat(dst); err != nil {
+		t.Errorf("dst not created: %v", err)
+	}
+}
+
+// TestWriteBundles_EndToEnd builds a fake artifact dir with a junit XML,
+// a journal log, a test log, and a per-test artifact subdir, then runs
+// the full ParseFile → WriteBundles pipeline and asserts the bundle dir
+// contains everything the report links to.
+func TestWriteBundles_EndToEnd(t *testing.T) {
+	dir := t.TempDir()
+
+	junit := `<?xml version="1.0"?>
+<testsuites><testsuite name="" tests="2" failures="1" timestamp="2026-05-19T12:32:30+10:00" time="3.0">
+  <testcase name="TestSingleNode/Phase5_LaunchInstance" time="2.500">
+    <failure message="Failed"><![CDATA[
+    ec2helpers.go:164: describe-vpcs: InvalidParameterValue (status 400)
+        Error Trace: tests/e2e/harness/ec2helpers.go:164
+        Test: TestSingleNode/Phase5_LaunchInstance]]></failure>
+  </testcase>
+</testsuite></testsuites>`
+	if err := os.WriteFile(filepath.Join(dir, "junit-single.xml"), []byte(junit), 0o644); err != nil {
+		t.Fatalf("write junit: %v", err)
+	}
+
+	journal := strings.Join([]string{
+		"2026-05-19T12:32:20+1000 host spinifex-daemon[1]: too-early",
+		"2026-05-19T12:32:30+1000 host spinifex-daemon[1]: in-window-start",
+		"2026-05-19T12:32:32+1000 host spinifex-daemon[1]: describe-vpcs filter=is_default",
+		"2026-05-19T12:32:50+1000 host spinifex-daemon[1]: too-late",
+		"",
+	}, "\n")
+	if err := os.WriteFile(filepath.Join(dir, "spinifex-journal.log"), []byte(journal), 0o644); err != nil {
+		t.Fatalf("write journal: %v", err)
+	}
+
+	testlog := strings.Join([]string{
+		"=== RUN   TestSingleNode/Phase5_LaunchInstance",
+		"    ec2helpers.go:164: describe-vpcs: InvalidParameterValue",
+		"--- FAIL: TestSingleNode/Phase5_LaunchInstance (2.50s)",
+		"",
+	}, "\n")
+	if err := os.WriteFile(filepath.Join(dir, "test-single.log"), []byte(testlog), 0o644); err != nil {
+		t.Fatalf("write test log: %v", err)
+	}
+
+	// Per-test artifact subdir (matches harness.ArtifactDir convention).
+	perTest := filepath.Join(dir, "TestSingleNode_Phase5_LaunchInstance")
+	if err := os.MkdirAll(perTest, 0o755); err != nil {
+		t.Fatalf("mkdir per-test: %v", err)
+	}
+	if err := os.WriteFile(filepath.Join(perTest, "qemu-console.log"), []byte("[qemu] boot\n"), 0o644); err != nil {
+		t.Fatalf("write per-test file: %v", err)
+	}
+
+	data, _ := os.ReadFile(filepath.Join(dir, "junit-single.xml"))
+	sr, err := ParseFile("junit-single.xml", data)
+	if err != nil {
+		t.Fatalf("ParseFile: %v", err)
+	}
+	rep := Report{Title: "E2E failure analysis", Suites: []SuiteReport{sr}}
+	WriteBundles(&rep, dir)
+
+	if rep.Suites[0].Root == nil {
+		t.Fatal("no root")
+	}
+	bp := rep.Suites[0].Root.BundlePath
+	if bp == "" {
+		t.Fatal("BundlePath not set")
+	}
+	bundle := filepath.Join(dir, bp)
+	for _, want := range []string{"failure.txt", "journal.log", "test.log", "qemu-console.log"} {
+		if _, err := os.Stat(filepath.Join(bundle, want)); err != nil {
+			t.Errorf("missing %s in bundle %s: %v", want, bp, err)
+		}
+	}
+
+	jl, _ := os.ReadFile(filepath.Join(bundle, "journal.log"))
+	if !strings.Contains(string(jl), "describe-vpcs filter=is_default") {
+		t.Errorf("journal.log lacks in-window line:\n%s", jl)
+	}
+	if strings.Contains(string(jl), "too-early") || strings.Contains(string(jl), "too-late") {
+		t.Errorf("journal.log includes out-of-window content:\n%s", jl)
+	}
+
+	tl, _ := os.ReadFile(filepath.Join(bundle, "test.log"))
+	if !strings.Contains(string(tl), "--- FAIL: TestSingleNode/Phase5_LaunchInstance") {
+		t.Errorf("test.log missing FAIL marker:\n%s", tl)
+	}
+
+	out := Render(rep)
+	wantLink := fmt.Sprintf("- Bundle: `%s/`", bp)
+	if !strings.Contains(out, wantLink) {
+		t.Errorf("render missing bundle link %q:\n%s", wantLink, out)
+	}
+}
@@ -0,0 +1,78 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+)
+
+func main() {
+	var junitGlob, logDir, title string
+	flag.StringVar(&junitGlob, "junit-glob", os.Getenv("ANALYZE_JUNIT_GLOB"), "glob matching junit-*.xml files")
+	flag.StringVar(&logDir, "log-dir", os.Getenv("ANALYZE_LOG_DIR"), "artifact dir to write analysis.md into")
+	flag.StringVar(&title, "title", os.Getenv("ANALYZE_TITLE"), "report heading")
+	flag.Parse()
+
+	if title == "" {
+		title = "E2E failure analysis"
+	}
+	if junitGlob == "" {
+		fmt.Fprintln(os.Stderr, "::error::e2e-analyze: -junit-glob is required")
+		os.Exit(0) // analyzer is advisory only; existing gate owns exit code
+	}
+
+	paths, err := filepath.Glob(junitGlob)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: bad glob %q: %v\n", junitGlob, err)
+		os.Exit(0)
+	}
+	sort.Strings(paths)
+
+	report := Report{Title: title}
+	for _, p := range paths {
+		data, err := os.ReadFile(p)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: read %s: %v\n", p, err)
+			continue
+		}
+		sr, err := ParseFile(p, data)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: %v\n", err)
+			continue
+		}
+		report.Suites = append(report.Suites, sr)
+	}
+
+	// Stage 2: prefer authoritative per-suite start times written by the
+	// workflow over junit-report's conversion-time `timestamp` attribute,
+	// then materialise per-failure bundles so Render can link them.
+	if logDir != "" {
+		ApplySuiteStartFiles(&report, logDir)
+		WriteBundles(&report, logDir)
+	}
+
+	out := Render(report)
+
+	if logDir != "" {
+		if err := os.MkdirAll(logDir, 0o755); err == nil {
+			if err := os.WriteFile(filepath.Join(logDir, "analysis.md"), []byte(out), 0o644); err != nil {
+				fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: write analysis.md: %v\n", err)
+			}
+		}
+	}
+
+	if summaryPath := os.Getenv("GITHUB_STEP_SUMMARY"); summaryPath != "" {
+		f, err := os.OpenFile(summaryPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
+		if err == nil {
+			defer f.Close()
+			if _, werr := f.WriteString(out); werr != nil {
+				fmt.Fprintf(os.Stderr, "::warning::e2e-analyze: append summary: %v\n", werr)
+			}
+		}
+	} else {
+		// Local invocation (developer running the tool by hand). Mirror to stdout.
+		fmt.Print(out)
+	}
+}
@@ -0,0 +1,176 @@
+package main
+
+// Stage 2 of docs/development/improvements/e2e-go-failure-analysis.md:
+// per-failure log slicers used by bundle.go to materialise time-window /
+// testcase-window log excerpts next to each named failure.
+
+import (
+	"bufio"
+	"io"
+	"os"
+	"regexp"
+	"strings"
+	"time"
+)
+
+// shortIsoTSRe matches the timestamp prefix produced by
+// `journalctl --output=short-iso`. The offset format varies across
+// systemd versions: most ship "+10:00" (RFC3339), some ship "+1000"
+// (no colon). Match either — the parser below picks the right layout.
+var shortIsoTSRe = regexp.MustCompile(`^(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:[+-]\d{2}:?\d{2}|Z))\s`)
+
+// shortIsoLayouts are tried in order; the first parse that succeeds wins.
+var shortIsoLayouts = []string{
+	time.RFC3339,               // 2026-05-19T12:32:30+10:00
+	"2006-01-02T15:04:05-0700", // 2026-05-19T12:32:30+1000
+}
+
+func parseShortIso(s string) (time.Time, error) {
+	var lastErr error
+	for _, layout := range shortIsoLayouts {
+		t, err := time.Parse(layout, s)
+		if err == nil {
+			return t, nil
+		}
+		lastErr = err
+	}
+	return time.Time{}, lastErr
+}
+
+// SliceJournal copies lines from src whose `short-iso` timestamp falls in
+// [start, end] to dst. Lines without a parseable timestamp inherit the
+// previous line's classification — that keeps multi-line journal records
+// (stack traces, panics) intact rather than orphaning their continuation
+// lines. Lines that precede the first parseable timestamp are dropped.
+//
+// Returns the number of lines written; missing src is not an error (the
+// journal may be empty if collection failed).
+func SliceJournal(src string, start, end time.Time, dst string) (int, error) {
+	f, err := os.Open(src)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+	defer f.Close()
+
+	out, err := os.Create(dst)
+	if err != nil {
+		return 0, err
+	}
+	defer out.Close()
+
+	bw := bufio.NewWriter(out)
+	defer bw.Flush()
+
+	sc := bufio.NewScanner(f)
+	// Journal lines can be long (panic dumps); bump scanner buffer.
+	sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
+
+	keep := false
+	written := 0
+	for sc.Scan() {
+		line := sc.Text()
+		if m := shortIsoTSRe.FindStringSubmatch(line); m != nil {
+			ts, err := parseShortIso(m[1])
+			if err != nil {
+				continue
+			}
+			keep = !ts.Before(start) && !ts.After(end)
+		}
+		if keep {
+			if _, err := bw.WriteString(line); err != nil {
+				return written, err
+			}
+			if err := bw.WriteByte('\n'); err != nil {
+				return written, err
+			}
+			written++
+		}
+	}
+	return written, sc.Err()
+}
+
+// runRePrefix and finishRePrefix bracket a single testcase's output in
+// `go test -v` stdout. Subtests have their own `=== RUN` / `--- FAIL`
+// markers, so slicing on the exact testcase name yields just that test's
+// lines (including any nested phase logs).
+var (
+	runRe    = regexp.MustCompile(`^=== RUN\s+(\S+)$`)
+	finishRe = regexp.MustCompile(`^\s*---\s+(PASS|FAIL|SKIP):\s+(\S+)\s+\(`)
+)
+
+// SliceTestLog copies lines belonging to the named testcase from src to
+// dst. The block starts at `=== RUN <name>` and ends at the matching
+// `--- PASS|FAIL|SKIP: <name> (…)` line (inclusive). If the testcase is
+// never seen the destination file is created empty so the bundle has a
+// stable shape.
+//
+// Returns the number of lines written; missing src is not an error.
+func SliceTestLog(src, testName, dst string) (int, error) {
+	f, err := os.Open(src)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+	defer f.Close()
+
+	out, err := os.Create(dst)
+	if err != nil {
+		return 0, err
+	}
+	defer out.Close()
+
+	bw := bufio.NewWriter(out)
+	defer bw.Flush()
+
+	sc := bufio.NewScanner(f)
+	sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
+
+	in := false
+	written := 0
+	for sc.Scan() {
+		line := sc.Text()
+		if !in {
+			if m := runRe.FindStringSubmatch(strings.TrimRight(line, "\r")); m != nil && m[1] == testName {
+				in = true
+			} else {
+				continue
+			}
+		}
+		if _, err := bw.WriteString(line); err != nil {
+			return written, err
+		}
+		if err := bw.WriteByte('\n'); err != nil {
+			return written, err
+		}
+		written++
+		if m := finishRe.FindStringSubmatch(line); m != nil && m[2] == testName {
+			return written, nil
+		}
+	}
+	return written, sc.Err()
+}
+
+// copyFile is a small helper used when bundling pre-collected per-test
+// artifacts. Best-effort: missing src is silently skipped.
+func copyFile(src, dst string) error {
+	in, err := os.Open(src)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil
+		}
+		return err
+	}
+	defer in.Close()
+	out, err := os.Create(dst)
+	if err != nil {
+		return err
+	}
+	defer out.Close()
+	_, err = io.Copy(out, in)
+	return err
+}
@@ -0,0 +1,23 @@
+## E2E failure analysis
+
+### Suite `single`: 16 failed, 1 root cause likely
+
+**Root cause (earliest non-cascade)**
+
+- Test: `TestSingleNode/Phase5_LaunchInstance`
+- Start: 12:30:00 (duration 2.5s)
+- Error: Received unexpected error: InvalidParameterValue (HTTP 400)
+- File:  `tests/e2e/harness/ec2helpers.go:164`
+
+**Cascaded failures (15) — grouped by signature**
+
+- 14× Phase 5 must populate fix.InstanceID
+  - `TestSingleNode/Phase5a_pre_ClusterStats`
+  - `TestSingleNode/Phase5a_Metadata`
+  - `TestSingleNode/Phase5b_DescribeInstances`
+  - `TestSingleNode/Phase5c_StopStart`
+  - `TestSingleNode/Phase6_Volumes`
+  - … and 9 more
+- 1× Phase 2 must populate fix.AZName
+  - `TestSingleNode/Phase8_NatGW`
+
@@ -0,0 +1,143 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<testsuites tests="20" failures="17" skipped="0">
+	<testsuite name="" tests="20" failures="17" errors="0" id="0" hostname="banksia" skipped="0" time="600.000" timestamp="2026-04-12T12:30:00+10:00">
+		<testcase name="TestSingleNode" classname="" time="600.000">
+			<failure message="Failed"></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase1_Environment" classname="" time="0.300">
+			<system-out><![CDATA[
+&#x2501;&#x2501; Phase 1 &#x2501;&#x2501;]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase2_Discovery" classname="" time="0.010"/>
+		<testcase name="TestSingleNode/Phase3_KeyPairs" classname="" time="0.540"/>
+		<testcase name="TestSingleNode/Phase4_Image" classname="" time="0.060"/>
+		<testcase name="TestSingleNode/Phase5_LaunchInstance" classname="" time="2.500">
+			<failure message="Failed"><![CDATA[
+    ec2helpers.go:164: describe-vpcs: InvalidParameterValue: Filter 'is_default' is invalid (status 400)
+    	Error Trace:	tests/e2e/harness/ec2helpers.go:164
+    	Error:      	Received unexpected error:
+    	            	InvalidParameterValue (HTTP 400)
+    	Test:       	TestSingleNode/Phase5_LaunchInstance]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_pre_ClusterStats" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    lifecycle_test.go:28:
+        Error Trace:	tests/e2e/single/lifecycle_test.go:28
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase5a_pre_ClusterStats]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_Metadata" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    lifecycle_test.go:57:
+        Error Trace:	tests/e2e/single/lifecycle_test.go:57
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase5a_Metadata]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5b_DescribeInstances" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    lifecycle_test.go:109:
+        Error Trace:	tests/e2e/single/lifecycle_test.go:109
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase5b_DescribeInstances]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5c_StopStart" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    lifecycle_test.go:222:
+        Error Trace:	tests/e2e/single/lifecycle_test.go:222
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase5c_StopStart]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase6_Volumes" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    volume_test.go:23:
+        Error Trace:	tests/e2e/single/volume_test.go:23
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase6_Volumes]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase6b_AttachVolume" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    volume_test.go:24:
+        Error Trace:	tests/e2e/single/volume_test.go:24
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase6b_AttachVolume]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7_Tags" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    tags_test.go:23:
+        Error Trace:	tests/e2e/single/tags_test.go:23
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7_Tags]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7a_Snapshots" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    snapshot_test.go:23:
+        Error Trace:	tests/e2e/single/snapshot_test.go:23
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7a_Snapshots]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7b_DescribeInstanceAttribute" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    instance_test.go:81:
+        Error Trace:	tests/e2e/single/instance_test.go:81
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7b_DescribeInstanceAttribute]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7c_ModifyInstanceAttribute" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    instance_test.go:92:
+        Error Trace:	tests/e2e/single/instance_test.go:92
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7c_ModifyInstanceAttribute]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7d_Console" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    instance_test.go:121:
+        Error Trace:	tests/e2e/single/instance_test.go:121
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7d_Console]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7e_Negative" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    negative_test.go:43:
+        Error Trace:	tests/e2e/single/negative_test.go:43
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase7e_Negative]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NatGW" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    natgw_test.go:39:
+        Error Trace:	tests/e2e/single/natgw_test.go:39
+        Error:      	Should NOT be empty
+        Messages:   	Phase 2 must populate fix.AZName
+        Test:       	TestSingleNode/Phase8_NatGW]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8a_DataPath" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    datapath_test.go:48:
+        Error Trace:	tests/e2e/single/datapath_test.go:48
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase8a_DataPath]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase9_SecurityGroups" classname="" time="0.020">
+			<failure message="Failed"><![CDATA[
+    securitygroup_test.go:35:
+        Error Trace:	tests/e2e/single/securitygroup_test.go:35
+        Error:      	Should NOT be empty
+        Messages:   	Phase 5 must populate fix.InstanceID
+        Test:       	TestSingleNode/Phase9_SecurityGroups]]></failure>
+		</testcase>
+	</testsuite>
+</testsuites>
@@ -0,0 +1,3 @@
+## E2E failure analysis
+
+✅ No failures across any suite.
@@ -0,0 +1,8 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<testsuites tests="3" failures="0" skipped="0">
+	<testsuite name="" tests="3" failures="0" errors="0" id="0" hostname="banksia" skipped="0" time="1.500" timestamp="2026-04-12T12:30:00+10:00">
+		<testcase name="TestSingleNode" classname="" time="1.500"/>
+		<testcase name="TestSingleNode/Phase1_Environment" classname="" time="0.300"/>
+		<testcase name="TestSingleNode/Phase2_Discovery" classname="" time="1.200"/>
+	</testsuite>
+</testsuites>
@@ -0,0 +1,11 @@
+## E2E failure analysis
+
+### Suite `single`: 1 failed, 1 root cause likely
+
+**Root cause (earliest non-cascade)**
+
+- Test: `TestSingleNode/Phase8b_VPCSubnetE2E`
+- Start: 00:26:18 (duration 200.5s)
+- Error: Eventually: condition not met within 3m0s: [SSH handshake 192.168.0.202:22 never completed]
+- File:  `vpc_test.go:227`
+
@@ -0,0 +1,993 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<testsuites tests="79" failures="2" skipped="6">
+	<testsuite name="" tests="79" failures="2" errors="0" id="0" hostname="banksia" skipped="6" time="1708.140" timestamp="2026-05-19T00:19:45+10:00">
+		<testcase name="TestSingleNode" classname="" time="800.320">
+			<failure message="Failed"></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase1_Environment" classname="" time="0.300">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 1 — Environment ━━�[0m
+�[2m· checking /dev/kvm�[0m
+    discovery_test.go:29: �[2m· checking /dev/kvm�[0m
+�[2mkvm�[0m=writable
+    discovery_test.go:38: �[2mkvm�[0m=writable
+�[2m· waiting for AWS gateway�[0m
+    discovery_test.go:45: �[2m· waiting for AWS gateway�[0m
+�[2mgateway�[0m=https://192.168.0.10:9999/
+    discovery_test.go:55: �[2mgateway�[0m=https://192.168.0.10:9999/
+�[2m· waiting for daemon NATS subscriptions�[0m
+    discovery_test.go:60: �[2m· waiting for daemon NATS subscriptions�[0m
+�[2m· describe-regions / describe-availability-zones�[0m
+    discovery_test.go:73: �[2m· describe-regions / describe-availability-zones�[0m
+�[2maz�[0m=ap-southeast-2a �[2mregion�[0m=ap-southeast-2
+    discovery_test.go:84: �[2maz�[0m=ap-southeast-2a �[2mregion�[0m=ap-southeast-2]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase1b_ClusterStats" classname="" time="9.200">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 1b — Cluster Stats CLI ━━�[0m
+�[2m· spx get nodes�[0m
+    discovery_test.go:101: �[2m· spx get nodes�[0m
+�[2m· spx top nodes�[0m
+    discovery_test.go:105: �[2m· spx top nodes�[0m
+�[2m· spx get vms (empty before any launches)�[0m
+    discovery_test.go:112: �[2m· spx get vms (empty before any launches)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase2_Discovery" classname="" time="0.010">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 2 — Discovery & Metadata ━━�[0m
+�[2m· discovering nano instance type�[0m
+    discovery_test.go:133: �[2m· discovering nano instance type�[0m
+�[2minstance_type�[0m=t3.nano �[2march�[0m=x86_64 �[2maz�[0m=ap-southeast-2a
+    discovery_test.go:159: �[2minstance_type�[0m=t3.nano �[2march�[0m=x86_64 �[2maz�[0m=ap-southeast-2a]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase2b_SerialConsole" classname="" time="0.020">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 2b — Serial Console Access ━━�[0m
+�[2m· default state should be disabled�[0m
+    discovery_test.go:168: �[2m· default state should be disabled�[0m
+�[2m· enable�[0m
+    discovery_test.go:173: �[2m· enable�[0m
+�[2mstate�[0m=enabled
+    discovery_test.go:180: �[2mstate�[0m=enabled
+�[2m· disable�[0m
+    discovery_test.go:182: �[2m· disable�[0m
+�[2mstate�[0m=disabled
+    discovery_test.go:189: �[2mstate�[0m=disabled]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase3_KeyPairs" classname="" time="0.540">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 3 — SSH Key Management ━━�[0m
+�[2m· create-key-pair "test-key-1"�[0m
+    keypair_test.go:37: �[2m· create-key-pair "test-key-1"�[0m
+�[2mkey�[0m=test-key-1 �[2mpem�[0m=/tmp/TestSingleNode1289015487/001/test-key-1.pem
+    keypair_test.go:51: �[2mkey�[0m=test-key-1 �[2mpem�[0m=/tmp/TestSingleNode1289015487/001/test-key-1.pem
+�[2m· import-key-pair "test-key-2" (local-generated RSA)�[0m
+    keypair_test.go:58: �[2m· import-key-pair "test-key-2" (local-generated RSA)�[0m
+�[2mimported�[0m=test-key-2
+    keypair_test.go:65: �[2mimported�[0m=test-key-2
+�[2m· describe-key-pairs (both present)�[0m
+    keypair_test.go:67: �[2m· describe-key-pairs (both present)�[0m
+�[2m· delete "test-key-2"�[0m
+    keypair_test.go:72: �[2m· delete "test-key-2"�[0m
+�[2m· describe-key-pairs (only test-key-1 remains)�[0m
+    keypair_test.go:76: �[2m· describe-key-pairs (only test-key-1 remains)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase4_Image" classname="" time="0.060">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 4 — Image Management ━━�[0m
+�[2mimage�[0m=ubuntu-24.04-x86_64 �[2march�[0m=x86_64
+    image_test.go:33: �[2mimage�[0m=ubuntu-24.04-x86_64 �[2march�[0m=x86_64
+�[2m· spx admin images import --name ubuntu-24.04-x86_64�[0m
+    image_test.go:35: �[2m· spx admin images import --name ubuntu-24.04-x86_64�[0m
+�[2m· spx import did not yield AMI ID; falling back to describe-images�[0m
+    image_test.go:48: �[2m· spx import did not yield AMI ID; falling back to describe-images�[0m
+�[2m· describe-images by AMI ID (verify exactly one match)�[0m
+    image_test.go:78: �[2m· describe-images by AMI ID (verify exactly one match)�[0m
+�[2mami�[0m=ami-6f60f2b3bf04b5453 �[2mname�[0m=ami-ubuntu-24.04-x86_64
+    image_test.go:86: �[2mami�[0m=ami-6f60f2b3bf04b5453 �[2mname�[0m=ami-ubuntu-24.04-x86_64]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5_LaunchInstance" classname="" time="10.830">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5 — Instance Lifecycle ━━�[0m
+�[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2msg�[0m=sg-764a7fa2a39577204 �[2msubnet�[0m=subnet-cd62e4c1c3cb50c63
+    instance_test.go:31: �[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2msg�[0m=sg-764a7fa2a39577204 �[2msubnet�[0m=subnet-cd62e4c1c3cb50c63
+�[2m· run-instances ami=ami-6f60f2b3bf04b5453 type=t3.nano key=test-key-1�[0m
+    instance_test.go:35: �[2m· run-instances ami=ami-6f60f2b3bf04b5453 type=t3.nano key=test-key-1�[0m
+�[2minstance�[0m=i-0c615e5dd74050f53
+    instance_test.go:48: �[2minstance�[0m=i-0c615e5dd74050f53
+    instance_test.go:50: instance i-0c615e5dd74050f53 reached state running
+�[2mroot_volume�[0m=vol-2230138124fb51ee2 �[2mroot_device�[0m=
+    instance_test.go:70: �[2mroot_volume�[0m=vol-2230138124fb51ee2 �[2mroot_device�[0m=]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_pre_ClusterStats" classname="" time="3.040">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5a-pre — Cluster Stats CLI (with running VM) ━━�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_Metadata" classname="" time="0.510">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5a — Instance Metadata Validation ━━�[0m
+�[2mtype�[0m=t3.nano �[2mkey�[0m=test-key-1 �[2mimage�[0m=ami-6f60f2b3bf04b5453 �[2mbdms�[0m=1
+    instance_test.go:108: �[2mtype�[0m=t3.nano �[2mkey�[0m=test-key-1 �[2mimage�[0m=ami-6f60f2b3bf04b5453 �[2mbdms�[0m=1]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_ii_SSH" classname="" time="49.450">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5a-ii — SSH Connectivity & Volume Verification ━━�[0m
+�[2mssh_host�[0m=192.168.0.201 �[2mssh_port�[0m=22
+    instance_test.go:127: �[2mssh_host�[0m=192.168.0.201 �[2mssh_port�[0m=22
+�[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+    instance_test.go:130: �[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+�[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+    lifecycle_test.go:384: �[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+�[2m· ssh id�[0m
+    instance_test.go:136: �[2m· ssh id�[0m
+�[2m· lsblk root-volume cross-check vs API�[0m
+    instance_test.go:140: �[2m· lsblk root-volume cross-check vs API�[0m
+�[2mguest_gib�[0m=4 �[2mapi_gib�[0m=4
+    instance_test.go:159: �[2mguest_gib�[0m=4 �[2mapi_gib�[0m=4
+�[2m· ssh hostname�[0m
+    instance_test.go:161: �[2m· ssh hostname�[0m
+�[2mhostname�[0m=spinifex-vm-0c615e5d �[2mmatches_short_id�[0m=0c615e5d
+    instance_test.go:172: �[2mhostname�[0m=spinifex-vm-0c615e5d �[2mmatches_short_id�[0m=0c615e5d]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5a_iii_Console" classname="" time="0.010">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5a-iii — Console Output ━━�[0m
+�[2minstance�[0m=i-0c615e5dd74050f53 �[2mhas_output�[0m=true
+    instance_test.go:191: �[2minstance�[0m=i-0c615e5dd74050f53 �[2mhas_output�[0m=true]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5b_Volume" classname="" time="4.540">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5b — Volume Lifecycle (Attach/Detach) ━━�[0m
+�[2m· create-volume size=10 az=ap-southeast-2a�[0m
+    volume_test.go:26: �[2m· create-volume size=10 az=ap-southeast-2a�[0m
+�[2mvolume�[0m=vol-a885f79173e0a8030
+    volume_test.go:34: �[2mvolume�[0m=vol-a885f79173e0a8030
+    volume_test.go:39: volume vol-a885f79173e0a8030 reached state available
+�[2m· modify-volume size=20�[0m
+    volume_test.go:42: �[2m· modify-volume size=20�[0m
+�[2mresized_gib�[0m=20
+    volume_test.go:66: �[2mresized_gib�[0m=20
+�[2m· attach-volume vol-a885f79173e0a8030 -> i-0c615e5dd74050f53 as /dev/sdf�[0m
+    volume_test.go:68: �[2m· attach-volume vol-a885f79173e0a8030 -> i-0c615e5dd74050f53 as /dev/sdf�[0m
+    volume_test.go:76: volume vol-a885f79173e0a8030 reached state in-use
+�[2m· detach-volume vol-a885f79173e0a8030�[0m
+    volume_test.go:90: �[2m· detach-volume vol-a885f79173e0a8030�[0m
+    volume_test.go:96: volume vol-a885f79173e0a8030 reached state available
+�[2m· delete-volume vol-a885f79173e0a8030�[0m
+    volume_test.go:98: �[2m· delete-volume vol-a885f79173e0a8030�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5b_ii_VolumeStatus" classname="" time="0.000">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5b-ii — DescribeVolumeStatus ━━�[0m
+�[2mvolume�[0m=vol-2230138124fb51ee2 �[2mstatus�[0m=ok
+    volume_test.go:158: �[2mvolume�[0m=vol-2230138124fb51ee2 �[2mstatus�[0m=ok]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5c_Snapshot" classname="" time="0.190">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5c — Snapshot Lifecycle ━━�[0m
+�[2m· create-snapshot volume=vol-2230138124fb51ee2�[0m
+    snapshot_test.go:34: �[2m· create-snapshot volume=vol-2230138124fb51ee2�[0m
+�[2msnapshot�[0m=snap-1188f92980e6470aa �[2msize�[0m=4 �[2mstate�[0m=completed �[2mprogress�[0m=100%
+    snapshot_test.go:50: �[2msnapshot�[0m=snap-1188f92980e6470aa �[2msize�[0m=4 �[2mstate�[0m=completed �[2mprogress�[0m=100%
+    snapshot_test.go:59: snapshot snap-1188f92980e6470aa reached state completed
+�[2m· describe-snapshots snap-1188f92980e6470aa�[0m
+    snapshot_test.go:61: �[2m· describe-snapshots snap-1188f92980e6470aa�[0m
+�[2m· copy-snapshot src=snap-1188f92980e6470aa region=ap-southeast-2�[0m
+    snapshot_test.go:76: �[2m· copy-snapshot src=snap-1188f92980e6470aa region=ap-southeast-2�[0m
+�[2mcopy_snapshot�[0m=snap-cde6b5ddf467e2c0f
+    snapshot_test.go:86: �[2mcopy_snapshot�[0m=snap-cde6b5ddf467e2c0f
+    snapshot_test.go:88: snapshot snap-cde6b5ddf467e2c0f reached state completed
+�[2m· describe-snapshots snap-1188f92980e6470aa,snap-cde6b5ddf467e2c0f (both visible)�[0m
+    snapshot_test.go:90: �[2m· describe-snapshots snap-1188f92980e6470aa,snap-cde6b5ddf467e2c0f (both visible)�[0m
+�[2m· delete-snapshot snap-1188f92980e6470aa (original)�[0m
+    snapshot_test.go:107: �[2m· delete-snapshot snap-1188f92980e6470aa (original)�[0m
+�[2m· verify copy snap-cde6b5ddf467e2c0f still completed�[0m
+    snapshot_test.go:135: �[2m· verify copy snap-cde6b5ddf467e2c0f still completed�[0m
+�[2m· delete-snapshot snap-cde6b5ddf467e2c0f (copy)�[0m
+    snapshot_test.go:144: �[2m· delete-snapshot snap-cde6b5ddf467e2c0f (copy)�[0m
+�[2mdeleted_original�[0m=snap-1188f92980e6470aa
+    snapshot_test.go:151: �[2mdeleted_original�[0m=snap-1188f92980e6470aa]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5d_SnapshotBackedLaunch" classname="" time="0.010">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5d — Snapshot-Backed Instance Launch ━━�[0m
+�[2mami�[0m=ami-6f60f2b3bf04b5453 �[2msnapshot�[0m=snap-ami-6f60f2b3bf04b5453
+    snapshot_test.go:182: �[2mami�[0m=ami-6f60f2b3bf04b5453 �[2msnapshot�[0m=snap-ami-6f60f2b3bf04b5453]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5e_CreateImage" classname="" time="0.660">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 5e — CreateImage Lifecycle ━━�[0m
+�[2m· create-image instance=i-0c615e5dd74050f53 name=e2e-custom-ami (no-reboot)�[0m
+    createimage_test.go:26: �[2m· create-image instance=i-0c615e5dd74050f53 name=e2e-custom-ami (no-reboot)�[0m
+�[2mcustom_ami�[0m=ami-7dcf263a412dd14c6
+    createimage_test.go:36: �[2mcustom_ami�[0m=ami-7dcf263a412dd14c6
+    createimage_test.go:38: image ami-7dcf263a412dd14c6 reached state available
+�[2m· describe-images ami-7dcf263a412dd14c6�[0m
+    createimage_test.go:40: �[2m· describe-images ami-7dcf263a412dd14c6�[0m
+�[2mcustom_ami_snapshot�[0m=snap-dde89b9ea2ef2e05a
+    createimage_test.go:67: �[2mcustom_ami_snapshot�[0m=snap-dde89b9ea2ef2e05a]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase5f_SecurityGroupEgress" classname="" time="4.880">
+			<skipped message="Skipped"><![CDATA[
+�[1m�[36m━━ Phase 5f — Security Group Enforcement (egress ACL) ━━�[0m
+�[2m· discover default gateway inside VM�[0m
+    securitygroup_test.go:53: �[2m· discover default gateway inside VM�[0m
+�[2mprobe_gateway�[0m=172.31.0.1
+    securitygroup_test.go:66: �[2mprobe_gateway�[0m=172.31.0.1
+�[2m· 5f-1 baseline egress (allow-all)�[0m
+    securitygroup_test.go:77: �[2m· 5f-1 baseline egress (allow-all)�[0m
+    securitygroup_test.go:80: Phase 5f: baseline ICMP did not succeed; env may block ICMP regardless of SG
+        Output:
+        PING 172.31.0.1 (172.31.0.1) 56(84) bytes of data.
+        
+        --- 172.31.0.1 ping statistics ---
+        3 packets transmitted, 0 received, 100% packet loss, time 2044ms
+        ]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/Phase6_TagManagement" classname="" time="0.380">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 6 — Tag Management ━━�[0m
+�[2m· 6a create-tags instance=i-0c615e5dd74050f53 (Name, Environment, DeleteMe)�[0m
+    tags_test.go:41: �[2m· 6a create-tags instance=i-0c615e5dd74050f53 (Name, Environment, DeleteMe)�[0m
+�[2m· 6b describe-tags resource-id=i-0c615e5dd74050f53 want=3�[0m
+    tags_test.go:53: �[2m· 6b describe-tags resource-id=i-0c615e5dd74050f53 want=3�[0m
+�[2m· 6c create-tags volume=vol-2230138124fb51ee2 (Name, Environment)�[0m
+    tags_test.go:65: �[2m· 6c create-tags volume=vol-2230138124fb51ee2 (Name, Environment)�[0m
+�[2m· 6d describe-tags key=Environment want=2�[0m
+    tags_test.go:76: �[2m· 6d describe-tags key=Environment want=2�[0m
+�[2m· 6e describe-tags resource-type=instance want=3�[0m
+    tags_test.go:85: �[2m· 6e describe-tags resource-type=instance want=3�[0m
+�[2m· 6f overwrite instance Name -> e2e-test-updated�[0m
+    tags_test.go:95: �[2m· 6f overwrite instance Name -> e2e-test-updated�[0m
+�[2m· 6g delete-tags Key=DeleteMe (no value)�[0m
+    tags_test.go:114: �[2m· 6g delete-tags Key=DeleteMe (no value)�[0m
+�[2m· 6h delete-tags Key=Environment,Value=production (no-op)�[0m
+    tags_test.go:130: �[2m· 6h delete-tags Key=Environment,Value=production (no-op)�[0m
+�[2m· 6i delete-tags Key=Environment,Value=testing (instance only)�[0m
+    tags_test.go:148: �[2m· 6i delete-tags Key=Environment,Value=testing (instance only)�[0m
+�[2m· 6j final describe-tags resource-id=i-0c615e5dd74050f53 want=1�[0m
+    tags_test.go:171: �[2m· 6j final describe-tags resource-id=i-0c615e5dd74050f53 want=1�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7_StopStart" classname="" time="8.090">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 7 — Instance State Transitions (Stop) ━━�[0m
+�[2m· stop-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:30: �[2m· stop-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:36: instance i-0c615e5dd74050f53 reached state stopped
+�[2m· reboot-instances on stopped instance (should fail)�[0m
+    lifecycle_test.go:40: �[2m· reboot-instances on stopped instance (should fail)�[0m
+�[2mstopped_reject�[0m=IncorrectInstanceState
+    lifecycle_test.go:47: �[2mstopped_reject�[0m=IncorrectInstanceState]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7a_AttachToStoppedError" classname="" time="0.300">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 7a — Attach Volume to Stopped Instance (Error Path) ━━�[0m
+�[2m· create-volume size=10 az=ap-southeast-2a�[0m
+    lifecycle_test.go:60: �[2m· create-volume size=10 az=ap-southeast-2a�[0m
+�[2mtest_volume�[0m=vol-8d3e1335938a378cd
+    lifecycle_test.go:68: �[2mtest_volume�[0m=vol-8d3e1335938a378cd
+    lifecycle_test.go:82: volume vol-8d3e1335938a378cd reached state available
+�[2m· attach-volume vol-8d3e1335938a378cd -> i-0c615e5dd74050f53 (should fail)�[0m
+    lifecycle_test.go:84: �[2m· attach-volume vol-8d3e1335938a378cd -> i-0c615e5dd74050f53 (should fail)�[0m
+�[2m· delete-volume vol-8d3e1335938a378cd�[0m
+    lifecycle_test.go:94: �[2m· delete-volume vol-8d3e1335938a378cd�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7b_ModifyInstanceAttribute" classname="" time="55.620">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 7b — ModifyInstanceAttribute ━━�[0m
+�[2mfrom_type�[0m=t3.nano �[2mto_type�[0m=t3.small
+    lifecycle_test.go:120: �[2mfrom_type�[0m=t3.nano �[2mto_type�[0m=t3.small
+�[2mexpected_vcpus�[0m=2 �[2mexpected_mem_mib�[0m=2048
+    lifecycle_test.go:142: �[2mexpected_vcpus�[0m=2 �[2mexpected_mem_mib�[0m=2048
+�[2m· modify-instance-attribute i-0c615e5dd74050f53 type=t3.small�[0m
+    lifecycle_test.go:144: �[2m· modify-instance-attribute i-0c615e5dd74050f53 type=t3.small�[0m
+�[2m· start-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:165: �[2m· start-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:171: instance i-0c615e5dd74050f53 reached state running
+�[2mssh_host�[0m=192.168.0.201 �[2mssh_port�[0m=22
+    lifecycle_test.go:177: �[2mssh_host�[0m=192.168.0.201 �[2mssh_port�[0m=22
+�[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+    lifecycle_test.go:179: �[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+�[2m· ssh nproc�[0m
+    lifecycle_test.go:182: �[2m· ssh nproc�[0m
+�[2mvm_vcpus�[0m=2
+    lifecycle_test.go:188: �[2mvm_vcpus�[0m=2
+�[2m· ssh MemTotal�[0m
+    lifecycle_test.go:190: �[2m· ssh MemTotal�[0m
+�[2mvm_mem_mib�[0m=1967 �[2mthreshold_mib�[0m=1740
+    lifecycle_test.go:200: �[2mvm_mem_mib�[0m=1967 �[2mthreshold_mib�[0m=1740
+�[2m· stop-instances i-0c615e5dd74050f53 (end of 7b)�[0m
+    lifecycle_test.go:206: �[2m· stop-instances i-0c615e5dd74050f53 (end of 7b)�[0m
+    lifecycle_test.go:211: instance i-0c615e5dd74050f53 reached state stopped]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7c_pre_Reboot" classname="" time="83.480">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 7c-pre — Reboot Running Instance ━━�[0m
+�[2m· start-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:226: �[2m· start-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:231: instance i-0c615e5dd74050f53 reached state running
+�[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+    lifecycle_test.go:236: �[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+�[2mpre_reboot_private_ip�[0m=172.31.0.4
+    lifecycle_test.go:246: �[2mpre_reboot_private_ip�[0m=172.31.0.4
+�[2m· reboot-instances i-0c615e5dd74050f53�[0m
+    lifecycle_test.go:248: �[2m· reboot-instances i-0c615e5dd74050f53�[0m
+�[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+    lifecycle_test.go:279: �[2m· waiting for SSH handshake 192.168.0.201:22�[0m
+�[2m· ssh uptime�[0m
+    lifecycle_test.go:282: �[2m· ssh uptime�[0m
+�[2muptime_secs�[0m=25
+    lifecycle_test.go:288: �[2muptime_secs�[0m=25]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase7c_RunInstancesMultiCount" classname="" time="47.560">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 7c — RunInstances with MinCount/MaxCount > 1 ━━�[0m
+�[2m· run-instances ami=ami-6f60f2b3bf04b5453 type=t3.small count=2�[0m
+    lifecycle_test.go:309: �[2m· run-instances ami=ami-6f60f2b3bf04b5453 type=t3.small count=2�[0m
+�[2msibling_1�[0m=i-9bfadd8ebd378f846 �[2msibling_2�[0m=i-448a9e2bf45c58658
+    lifecycle_test.go:327: �[2msibling_1�[0m=i-9bfadd8ebd378f846 �[2msibling_2�[0m=i-448a9e2bf45c58658
+    lifecycle_test.go:342: instance i-9bfadd8ebd378f846 reached state running
+    lifecycle_test.go:342: instance i-448a9e2bf45c58658 reached state running
+�[2m· terminate-instances i-9bfadd8ebd378f846 i-448a9e2bf45c58658�[0m
+    lifecycle_test.go:345: �[2m· terminate-instances i-9bfadd8ebd378f846 i-448a9e2bf45c58658�[0m
+    lifecycle_test.go:352: instance i-9bfadd8ebd378f846 reached state terminated
+    lifecycle_test.go:352: instance i-448a9e2bf45c58658 reached state terminated]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths" classname="" time="0.000">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 8 — Negative / Error Path Tests ━━�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8a_InvalidAMIIDMalformed" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· run-instances image-id=notanami�[0m
+    negative_test.go:55: �[2m· run-instances image-id=notanami�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8b_InvalidInstanceType" classname="" time="0.210">
+			<system-out><![CDATA[�[2m· run-instances instance-type=x99.superlarge�[0m
+    negative_test.go:70: �[2m· run-instances instance-type=x99.superlarge�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8c_VolumeInUse" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· attach-volume vol-2230138124fb51ee2 (root, in-use)�[0m
+    negative_test.go:85: �[2m· attach-volume vol-2230138124fb51ee2 (root, in-use)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8d_DetachRootForbidden" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· detach-volume vol-2230138124fb51ee2 (root)�[0m
+    negative_test.go:97: �[2m· detach-volume vol-2230138124fb51ee2 (root)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8e_DeleteSnapshotNotFound" classname="" time="0.030">
+			<system-out><![CDATA[�[2m· delete-snapshot snap-nonexistent000000�[0m
+    negative_test.go:108: �[2m· delete-snapshot snap-nonexistent000000�[0m
+�[2m· describe-volumes vol-0000000000000dead�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8f_InvalidActionRawHTTP" classname="" time="0.040">
+			<system-out><![CDATA[�[2m· POST Action=DescribeFakeThings�[0m
+    negative_test.go:122: �[2m· POST Action=DescribeFakeThings�[0m
+    negative_test.go:125: �[2mstatus�[0m=400 �[2mcode�[0m=InvalidAction]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8g_InvalidAMIIDNotFound" classname="" time="0.240">
+			<system-out><![CDATA[�[2m· reboot-instances i-nonexistent�[0m
+�[2m· run-instances image-id=ami-0000000000000dead�[0m
+    negative_test.go:138: �[2m· run-instances image-id=ami-0000000000000dead�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8h_InvalidKeyPairNotFound" classname="" time="0.260">
+			<system-out><![CDATA[�[2m· run-instances key-name=nonexistent-key-xyz�[0m
+    negative_test.go:153: �[2m· run-instances key-name=nonexistent-key-xyz�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8i_DeleteVolumeNotFound" classname="" time="0.040">
+			<system-out><![CDATA[�[2m· delete-volume vol-0000000000000dead�[0m
+    negative_test.go:168: �[2m· delete-volume vol-0000000000000dead�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8j_CreateKeyPairDuplicate" classname="" time="0.040">
+			<system-out><![CDATA[    negative_test.go:178: �[2m· create-key-pair test-key-1 (duplicate)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8k_ImportKeyPairDuplicate" classname="" time="0.040">
+			<system-out><![CDATA[�[2m· import-key-pair test-key-1 (duplicate)�[0m
+    negative_test.go:198: �[2m· import-key-pair test-key-1 (duplicate)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8l_InvalidKeyFormat" classname="" time="0.040">
+			<system-out><![CDATA[    negative_test.go:210: �[2m· import-key-pair bad-format-key (bad material)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8m_DescribeVolumesNotFound" classname="" time="0.040">
+			<system-out><![CDATA[    negative_test.go:230: �[2m· describe-volumes vol-0000000000000dead�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8n_DescribeImagesNotFound" classname="" time="0.040">
+			<system-out><![CDATA[�[2m· describe-images ami-0000000000000dead�[0m
+    negative_test.go:240: �[2m· describe-images ami-0000000000000dead�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8o_CreateImageDuplicateName" classname="" time="0.560">
+			<system-out><![CDATA[�[2m· create-key-pair test-key-1 (duplicate)�[0m
+�[2m· create-image name=e2e-custom-ami (duplicate)�[0m
+�[2m· import-key-pair bad-format-key (bad material)�[0m
+    negative_test.go:252: �[2m· create-image name=e2e-custom-ami (duplicate)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8p_DeleteKeyPairIdempotent" classname="" time="0.020">
+			<system-out><![CDATA[�[2m· delete-key-pair nonexistent-key-99999 (idempotent)�[0m
+    negative_test.go:276: �[2m· delete-key-pair nonexistent-key-99999 (idempotent)�[0m
+�[2mstatus�[0m=400 �[2mcode�[0m=InvalidAction]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8q_ModifyAttributeOnRunning" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· modify-instance-attribute i-0c615e5dd74050f53 (running)�[0m
+    negative_test.go:289: �[2m· modify-instance-attribute i-0c615e5dd74050f53 (running)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8_NegativeErrorPaths/8r_RebootInstanceNotFound" classname="" time="0.040">
+			<system-out><![CDATA[    negative_test.go:302: �[2m· reboot-instances i-nonexistent�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM1_UserCRUD" classname="" time="0.070">
+			<system-out><![CDATA[
+�[1m�[36m━━ IAM Phase 1 — User CRUD ━━�[0m
+�[2m· list-users (root auth sanity)�[0m
+    iam_test.go:109: �[2m· list-users (root auth sanity)�[0m
+�[2m· create-user "alice"�[0m
+    iam_test.go:114: �[2m· create-user "alice"�[0m
+�[2muser�[0m=alice �[2marn�[0m=arn:aws:iam::000000000001:user/alice
+    iam_test.go:120: �[2muser�[0m=alice �[2marn�[0m=arn:aws:iam::000000000001:user/alice
+�[2m· create-user "bob" path="/engineering/"�[0m
+    iam_test.go:122: �[2m· create-user "bob" path="/engineering/"�[0m
+�[2m· create-user "alice" again (expect EntityAlreadyExists)�[0m
+    iam_test.go:131: �[2m· create-user "alice" again (expect EntityAlreadyExists)�[0m
+�[2m· get-user "alice"�[0m
+    iam_test.go:137: �[2m· get-user "alice"�[0m
+�[2m· get-user nonexistent (expect NoSuchEntity)�[0m
+    iam_test.go:142: �[2m· get-user nonexistent (expect NoSuchEntity)�[0m
+�[2m· list-users (>=3 root+alice+bob)�[0m
+    iam_test.go:148: �[2m· list-users (>=3 root+alice+bob)�[0m
+�[2m· list-users --path-prefix "/engineering/" (expect 1)�[0m
+    iam_test.go:154: �[2m· list-users --path-prefix "/engineering/" (expect 1)�[0m
+    iam_test.go:193: �[2m· create-user "iam-phase1-ephemeral" (delete idempotency probe)�[0m
+�[2m· delete-user "iam-phase1-ephemeral"�[0m
+    iam_test.go:197: �[2m· delete-user "iam-phase1-ephemeral"�[0m
+�[2m· delete-user "iam-phase1-ephemeral" again (expect NoSuchEntity)�[0m
+    iam_test.go:201: �[2m· delete-user "iam-phase1-ephemeral" again (expect NoSuchEntity)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM1_UserCRUD/UpdateUser" classname="" time="0.000">
+			<skipped message="Skipped"><![CDATA[    iam_test.go:165: daemon UpdateUser handler not implemented — InvalidAction
+�[2m· create-user "iam-phase1-ephemeral" (delete idempotency probe)�[0m]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/IAM2_AccessKeyLifecycle" classname="" time="0.040">
+			<system-out><![CDATA[
+�[1m�[36m━━ IAM Phase 2 — Access Key Lifecycle ━━�[0m
+�[2m· create-access-key user=alice (key 1)�[0m
+    iam_test.go:218: �[2m· create-access-key user=alice (key 1)�[0m
+�[2mkey1�[0m=AKIA4984AE57CF8D721A879A
+    iam_test.go:227: �[2mkey1�[0m=AKIA4984AE57CF8D721A879A
+�[2m· create-access-key user=alice (key 2)�[0m
+    iam_test.go:229: �[2m· create-access-key user=alice (key 2)�[0m
+�[2mkey2�[0m=AKIAA1CD38D4F0E573216E11
+    iam_test.go:236: �[2mkey2�[0m=AKIAA1CD38D4F0E573216E11
+�[2m· create-access-key user=alice key 3 (expect LimitExceeded)�[0m
+    iam_test.go:238: �[2m· create-access-key user=alice key 3 (expect LimitExceeded)�[0m
+�[2m· create-access-key user=ghost (expect NoSuchEntity)�[0m
+    iam_test.go:246: �[2m· create-access-key user=ghost (expect NoSuchEntity)�[0m
+�[2m· list-access-keys user=alice (expect 2)�[0m
+    iam_test.go:254: �[2m· list-access-keys user=alice (expect 2)�[0m
+�[2m· list-access-keys user=bob (expect 0)�[0m
+    iam_test.go:261: �[2m· list-access-keys user=bob (expect 0)�[0m
+�[2m· update-access-key AKIA4984AE57CF8D721A879A -> Inactive�[0m
+    iam_test.go:268: �[2m· update-access-key AKIA4984AE57CF8D721A879A -> Inactive�[0m
+�[2m· update-access-key AKIA4984AE57CF8D721A879A -> Active�[0m
+    iam_test.go:279: �[2m· update-access-key AKIA4984AE57CF8D721A879A -> Active�[0m
+�[2m· delete-access-key AKIAA1CD38D4F0E573216E11 (key 2)�[0m
+    iam_test.go:287: �[2m· delete-access-key AKIAA1CD38D4F0E573216E11 (key 2)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM3_UserAuthentication" classname="" time="0.000">
+			<skipped message="Skipped"><![CDATA[
+�[1m�[36m━━ IAM Phase 3 — User Authentication ━━�[0m
+    iam_test.go:311: daemon IAM signature/principal lookup not implemented — mulga-siv-100]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD" classname="" time="0.040">
+			<system-out><![CDATA[
+�[1m�[36m━━ IAM Phase 4 — Policy CRUD ━━�[0m
+�[2m· create-policy EC2ReadOnly�[0m
+    iam_test.go:384: �[2m· create-policy EC2ReadOnly�[0m
+�[2mpolicy�[0m=EC2ReadOnly �[2marn�[0m=arn:aws:iam::000000000001:policy/EC2ReadOnly �[2maccount�[0m=000000000001
+    iam_test.go:393: �[2mpolicy�[0m=EC2ReadOnly �[2marn�[0m=arn:aws:iam::000000000001:policy/EC2ReadOnly �[2maccount�[0m=000000000001
+    iam_test.go:440: �[2m· create-policy duplicate (expect EntityAlreadyExists)�[0m
+�[2m· create-policy malformed (expect MalformedPolicyDocument)�[0m
+    iam_test.go:449: �[2m· create-policy malformed (expect MalformedPolicyDocument)�[0m
+�[2m· get-policy EC2ReadOnly�[0m
+    iam_test.go:458: �[2m· get-policy EC2ReadOnly�[0m
+�[2m· get-policy nonexistent (expect NoSuchEntity)�[0m
+    iam_test.go:463: �[2m· get-policy nonexistent (expect NoSuchEntity)�[0m
+�[2m· get-policy-version EC2ReadOnly v1�[0m
+    iam_test.go:470: �[2m· get-policy-version EC2ReadOnly v1�[0m
+    iam_test.go:519: �[2m· list-policies (>=5)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/create_policies_parallel" classname="" time="0.000"></testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/create_policies_parallel/FullAdmin" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· create-policy FullAdmin path=/admin/�[0m
+    iam_test.go:402: �[2m· create-policy FullAdmin path=/admin/�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/create_policies_parallel/DenyTerminate" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· create-policy DenyTerminate (mixed Allow+Deny)�[0m
+    iam_test.go:413: �[2m· create-policy DenyTerminate (mixed Allow+Deny)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/create_policies_parallel/IAMReadOnly" classname="" time="0.020">
+			<system-out><![CDATA[�[2m· create-policy IAMReadOnly�[0m
+    iam_test.go:422: �[2m· create-policy IAMReadOnly�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/create_policies_parallel/EC2DescribeAll" classname="" time="0.020">
+			<system-out><![CDATA[�[2m· create-policy EC2DescribeAll (wildcard)�[0m
+    iam_test.go:431: �[2m· create-policy EC2DescribeAll (wildcard)�[0m
+�[2m· create-policy duplicate (expect EntityAlreadyExists)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/IAM4_PolicyCRUD/PolicyVersionLifecycle" classname="" time="0.000">
+			<skipped message="Skipped"><![CDATA[    iam_test.go:484: daemon CreatePolicyVersion handler not implemented — InvalidAction
+�[2m· list-policies (>=5)�[0m]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/IAM5_PolicyAttachmentEnforcement" classname="" time="0.000">
+			<skipped message="Skipped"><![CDATA[
+�[1m�[36m━━ IAM Phase 5 — Policy Attachment & Enforcement ━━�[0m
+    iam_test.go:535: daemon IAM scoped-credential enforcement not implemented — mulga-siv-100]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/IAM6_PolicyLifecycle" classname="" time="0.000">
+			<skipped message="Skipped"><![CDATA[
+�[1m�[36m━━ IAM Phase 6 — Policy Lifecycle (Detach & Delete) ━━�[0m
+    iam_test.go:718: daemon DetachUserPolicy not implemented — mulga-siv-100]]></skipped>
+		</testcase>
+		<testcase name="TestSingleNode/IAM7_Cleanup" classname="" time="0.050">
+			<system-out><![CDATA[
+�[1m�[36m━━ IAM Phase 7 — Cleanup ━━�[0m
+�[2m· cleanup user alice�[0m
+    iam_test.go:770: �[2m· cleanup user alice�[0m
+�[2m· cleanup user bob�[0m
+    iam_test.go:770: �[2m· cleanup user bob�[0m
+�[2m· cleanup user charlie�[0m
+    iam_test.go:770: �[2m· cleanup user charlie�[0m
+�[2mremaining_users�[0m=1 �[2mnames�[0m=[admin]
+    iam_test.go:779: �[2mremaining_users�[0m=1 �[2mnames�[0m=[admin]
+�[2m· cleanup policy arn:aws:iam::000000000001:policy/EC2ReadOnly�[0m
+    iam_test.go:806: �[2m· cleanup policy arn:aws:iam::000000000001:policy/EC2ReadOnly�[0m
+�[2m· cleanup policy arn:aws:iam::000000000001:policy/admin/FullAdmin�[0m
+    iam_test.go:806: �[2m· cleanup policy arn:aws:iam::000000000001:policy/admin/FullAdmin�[0m
+�[2m· cleanup policy arn:aws:iam::000000000001:policy/IAMReadOnly�[0m
+    iam_test.go:806: �[2m· cleanup policy arn:aws:iam::000000000001:policy/IAMReadOnly�[0m
+�[2m· cleanup policy arn:aws:iam::000000000001:policy/EC2DescribeAll�[0m
+    iam_test.go:806: �[2m· cleanup policy arn:aws:iam::000000000001:policy/EC2DescribeAll�[0m
+�[2m· cleanup policy arn:aws:iam::000000000001:policy/DenyTerminate�[0m
+    iam_test.go:806: �[2m· cleanup policy arn:aws:iam::000000000001:policy/DenyTerminate�[0m
+�[2mremaining_policies�[0m=1 �[2mnames�[0m=[AdministratorAccess]
+    iam_test.go:812: �[2mremaining_policies�[0m=1 �[2mnames�[0m=[AdministratorAccess]]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping" classname="" time="114.890">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 8 (acct) — EC2 Account Scoping ━━�[0m
+    account_test.go:75: �[2m· Phase 8 acct cleanup (deferred)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step1_AccountSetup" classname="" time="1.220">
+			<system-out><![CDATA[�[2m· create account 'Team Alpha'�[0m
+    account_test.go:206: �[2m· create account 'Team Alpha'�[0m
+�[2malpha_account�[0m=000000000002 �[2malpha_key�[0m=AKIA374CBBB2A15D3E2EE822
+    account_test.go:211: �[2malpha_account�[0m=000000000002 �[2malpha_key�[0m=AKIA374CBBB2A15D3E2EE822
+�[2m· create account 'Team Beta'�[0m
+    account_test.go:213: �[2m· create account 'Team Beta'�[0m
+�[2mbeta_account�[0m=000000000003 �[2mbeta_key�[0m=AKIA0AA0B2845E3BD5A26D01
+    account_test.go:218: �[2mbeta_account�[0m=000000000003 �[2mbeta_key�[0m=AKIA0AA0B2845E3BD5A26D01]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step2_InstanceScoping" classname="" time="52.380">
+			<system-out><![CDATA[�[2m· create per-account key pairs�[0m
+    account_test.go:242: �[2m· create per-account key pairs�[0m
+�[2m· alpha run-instances�[0m
+    account_test.go:250: �[2m· alpha run-instances�[0m
+�[2malpha_instance�[0m=i-a3072c2387630b8ee
+    account_test.go:262: �[2malpha_instance�[0m=i-a3072c2387630b8ee
+�[2m· beta run-instances�[0m
+    account_test.go:264: �[2m· beta run-instances�[0m
+�[2mbeta_instance�[0m=i-ad32cbb0afaee5eee
+    account_test.go:276: �[2mbeta_instance�[0m=i-ad32cbb0afaee5eee
+    account_test.go:278: instance i-a3072c2387630b8ee reached state running
+    account_test.go:279: instance i-ad32cbb0afaee5eee reached state running
+�[2m· alpha sees only own instances�[0m
+    account_test.go:281: �[2m· alpha sees only own instances�[0m
+�[2m· beta sees only own instances�[0m
+    account_test.go:285: �[2m· beta sees only own instances�[0m
+�[2m· alpha OwnerId matches alpha account�[0m
+    account_test.go:289: �[2m· alpha OwnerId matches alpha account�[0m
+�[2m· cross-account stop alpha->beta blocked�[0m
+    account_test.go:296: �[2m· cross-account stop alpha->beta blocked�[0m
+�[2m· cross-account terminate beta->alpha blocked�[0m
+    account_test.go:304: �[2m· cross-account terminate beta->alpha blocked�[0m
+�[2m· cross-account reboot alpha->beta blocked�[0m
+    account_test.go:312: �[2m· cross-account reboot alpha->beta blocked�[0m
+�[2m· stop alpha instance for cross-account start/modify/console tests�[0m
+    account_test.go:321: �[2m· stop alpha instance for cross-account start/modify/console tests�[0m
+    account_test.go:326: instance i-a3072c2387630b8ee reached state stopped
+�[2m· cross-account start beta->alpha blocked�[0m
+    account_test.go:328: �[2m· cross-account start beta->alpha blocked�[0m
+�[2m· cross-account modify-instance-attribute beta->alpha blocked�[0m
+    account_test.go:336: �[2m· cross-account modify-instance-attribute beta->alpha blocked�[0m
+�[2m· cross-account get-console-output beta->alpha blocked�[0m
+    account_test.go:345: �[2m· cross-account get-console-output beta->alpha blocked�[0m
+�[2m· restart alpha instance�[0m
+    account_test.go:354: �[2m· restart alpha instance�[0m
+    account_test.go:359: instance i-a3072c2387630b8ee reached state running]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step3_VolumeScoping" classname="" time="24.140">
+			<system-out><![CDATA[�[2m· alpha create-volume�[0m
+    account_test.go:366: �[2m· alpha create-volume�[0m
+�[2malpha_vol�[0m=vol-ceef82652369780af
+    account_test.go:375: �[2malpha_vol�[0m=vol-ceef82652369780af
+�[2m· beta create-volume�[0m
+    account_test.go:377: �[2m· beta create-volume�[0m
+�[2mbeta_vol�[0m=vol-08024f32d23d9e18c
+    account_test.go:386: �[2mbeta_vol�[0m=vol-08024f32d23d9e18c
+    account_test.go:388: volume vol-ceef82652369780af reached state available
+    account_test.go:389: volume vol-08024f32d23d9e18c reached state available
+�[2m· alpha sees only own volumes�[0m
+    account_test.go:391: �[2m· alpha sees only own volumes�[0m
+�[2m· cross-account describe-volume by id blocked�[0m
+    account_test.go:395: �[2m· cross-account describe-volume by id blocked�[0m
+�[2m· cross-account delete-volume blocked�[0m
+    account_test.go:403: �[2m· cross-account delete-volume blocked�[0m
+�[2m· cross-account attach-volume (beta attaches alpha's vol) blocked�[0m
+    account_test.go:411: �[2m· cross-account attach-volume (beta attaches alpha's vol) blocked�[0m
+�[2m· alpha attach-volume to own instance�[0m
+    account_test.go:422: �[2m· alpha attach-volume to own instance�[0m
+    account_test.go:429: volume vol-ceef82652369780af reached state in-use
+�[2m· cross-account detach-volume blocked�[0m
+    account_test.go:431: �[2m· cross-account detach-volume blocked�[0m
+�[2m· cross-account modify-volume blocked�[0m
+    account_test.go:439: �[2m· cross-account modify-volume blocked�[0m
+�[2m· alpha detach own volume�[0m
+    account_test.go:449: �[2m· alpha detach own volume�[0m
+    account_test.go:454: volume vol-ceef82652369780af reached state available]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step4_KeyPairScoping" classname="" time="0.370">
+			<system-out><![CDATA[�[2m· alpha create-key-pair alpha-key�[0m
+    account_test.go:466: �[2m· alpha create-key-pair alpha-key�[0m
+�[2m· beta create-key-pair beta-key�[0m
+    account_test.go:473: �[2m· beta create-key-pair beta-key�[0m
+�[2m· alpha sees only own keys�[0m
+    account_test.go:478: �[2m· alpha sees only own keys�[0m
+�[2m· namespace isolation: shared-name in both accounts�[0m
+    account_test.go:483: �[2m· namespace isolation: shared-name in both accounts�[0m
+�[2m· beta deletes alpha-key (idempotent, no cross-account effect)�[0m
+    account_test.go:497: �[2m· beta deletes alpha-key (idempotent, no cross-account effect)�[0m
+�[2m· alpha import-key-pair imported-key�[0m
+    account_test.go:503: �[2m· alpha import-key-pair imported-key�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step5_SnapshotScoping" classname="" time="0.060">
+			<system-out><![CDATA[�[2m· alpha create-snapshot�[0m
+    account_test.go:522: �[2m· alpha create-snapshot�[0m
+�[2malpha_snap�[0m=snap-2a47896f4304d805c
+    account_test.go:530: �[2malpha_snap�[0m=snap-2a47896f4304d805c
+�[2m· beta create-snapshot�[0m
+    account_test.go:532: �[2m· beta create-snapshot�[0m
+�[2mbeta_snap�[0m=snap-bf5744e78bd7c6908
+    account_test.go:540: �[2mbeta_snap�[0m=snap-bf5744e78bd7c6908
+�[2m· alpha sees only own snapshots (owner-ids=self)�[0m
+    account_test.go:542: �[2m· alpha sees only own snapshots (owner-ids=self)�[0m
+�[2m· cross-account delete-snapshot blocked (UnauthorizedOperation)�[0m
+    account_test.go:560: �[2m· cross-account delete-snapshot blocked (UnauthorizedOperation)�[0m
+�[2m· cross-account create-snapshot of alpha's volume blocked�[0m
+    account_test.go:568: �[2m· cross-account create-snapshot of alpha's volume blocked�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step6_VPCSubnetScoping" classname="" time="0.030">
+			<system-out><![CDATA[�[2m· alpha create-vpc 10.0.0.0/16�[0m
+    account_test.go:582: �[2m· alpha create-vpc 10.0.0.0/16�[0m
+�[2malpha_vpc�[0m=vpc-7f60df5e87183fb59
+    account_test.go:589: �[2malpha_vpc�[0m=vpc-7f60df5e87183fb59
+�[2m· beta create-vpc 10.0.0.0/16 (same CIDR, no conflict)�[0m
+    account_test.go:591: �[2m· beta create-vpc 10.0.0.0/16 (same CIDR, no conflict)�[0m
+�[2mbeta_vpc�[0m=vpc-7e8c859d3985c8218
+    account_test.go:598: �[2mbeta_vpc�[0m=vpc-7e8c859d3985c8218
+�[2m· alpha describe-vpcs isolation�[0m
+    account_test.go:600: �[2m· alpha describe-vpcs isolation�[0m
+�[2m· cross-account describe-vpc by id blocked�[0m
+    account_test.go:605: �[2m· cross-account describe-vpc by id blocked�[0m
+�[2m· cross-account delete-vpc blocked�[0m
+    account_test.go:613: �[2m· cross-account delete-vpc blocked�[0m
+�[2m· alpha create-subnet 10.0.1.0/24�[0m
+    account_test.go:621: �[2m· alpha create-subnet 10.0.1.0/24�[0m
+�[2m· beta create-subnet 10.0.1.0/24�[0m
+    account_test.go:630: �[2m· beta create-subnet 10.0.1.0/24�[0m
+�[2m· alpha describe-subnets isolation�[0m
+    account_test.go:639: �[2m· alpha describe-subnets isolation�[0m
+�[2m· cross-account create-subnet in other VPC blocked�[0m
+    account_test.go:648: �[2m· cross-account create-subnet in other VPC blocked�[0m
+�[2m· cross-account delete-subnet blocked�[0m
+    account_test.go:657: �[2m· cross-account delete-subnet blocked�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step7_IGWEIGWScoping" classname="" time="0.030">
+			<system-out><![CDATA[�[2m· alpha create-internet-gateway�[0m
+    account_test.go:673: �[2m· alpha create-internet-gateway�[0m
+�[2malpha_igw�[0m=igw-ee966e3e923caf630
+    account_test.go:678: �[2malpha_igw�[0m=igw-ee966e3e923caf630
+�[2m· beta create-internet-gateway�[0m
+    account_test.go:680: �[2m· beta create-internet-gateway�[0m
+�[2mbeta_igw�[0m=igw-b46e66f2065889bb9
+    account_test.go:685: �[2mbeta_igw�[0m=igw-b46e66f2065889bb9
+�[2m· alpha describe-internet-gateways isolation�[0m
+    account_test.go:687: �[2m· alpha describe-internet-gateways isolation�[0m
+�[2m· cross-account describe IGW by id blocked�[0m
+    account_test.go:696: �[2m· cross-account describe IGW by id blocked�[0m
+�[2m· cross-account delete IGW blocked�[0m
+    account_test.go:704: �[2m· cross-account delete IGW blocked�[0m
+�[2m· cross-account attach IGW (alpha attaches beta's IGW) blocked�[0m
+    account_test.go:712: �[2m· cross-account attach IGW (alpha attaches beta's IGW) blocked�[0m
+�[2m· alpha attach own IGW to own VPC�[0m
+    account_test.go:722: �[2m· alpha attach own IGW to own VPC�[0m
+�[2m· cross-account detach IGW blocked�[0m
+    account_test.go:729: �[2m· cross-account detach IGW blocked�[0m
+�[2m· alpha create-egress-only-internet-gateway�[0m
+    account_test.go:739: �[2m· alpha create-egress-only-internet-gateway�[0m
+�[2malpha_eigw�[0m=eigw-51ba82da927e49393
+    account_test.go:746: �[2malpha_eigw�[0m=eigw-51ba82da927e49393
+�[2m· beta create-egress-only-internet-gateway�[0m
+    account_test.go:748: �[2m· beta create-egress-only-internet-gateway�[0m
+�[2mbeta_eigw�[0m=eigw-e88d642e78ab4fe5a
+    account_test.go:755: �[2mbeta_eigw�[0m=eigw-e88d642e78ab4fe5a
+�[2m· alpha describe-eigws isolation�[0m
+    account_test.go:757: �[2m· alpha describe-eigws isolation�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step8_AccountSettings" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· alpha enable-ebs-encryption-by-default�[0m
+    account_test.go:794: �[2m· alpha enable-ebs-encryption-by-default�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step9_GlobalResources" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· describe-regions identical across accounts�[0m
+    account_test.go:829: �[2m· describe-regions identical across accounts�[0m
+�[2m· describe-availability-zones identical across accounts�[0m
+    account_test.go:834: �[2m· describe-availability-zones identical across accounts�[0m
+�[2m· describe-instance-types identical across accounts�[0m
+    account_test.go:839: �[2m· describe-instance-types identical across accounts�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step10_EdgeCases" classname="" time="1.200">
+			<system-out><![CDATA[�[2m· create empty Gamma account�[0m
+    account_test.go:849: �[2m· create empty Gamma account�[0m
+�[2mgamma_account�[0m=000000000004
+    account_test.go:852: �[2mgamma_account�[0m=000000000004
+�[2m· root create-key-pair root-scoping-key�[0m
+    account_test.go:879: �[2m· root create-key-pair root-scoping-key�[0m
+�[2m· non-existent volume id -> InvalidVolume.NotFound�[0m
+    account_test.go:895: �[2m· non-existent volume id -> InvalidVolume.NotFound�[0m
+�[2m· non-existent snapshot id -> InvalidSnapshot.NotFound�[0m
+    account_test.go:903: �[2m· non-existent snapshot id -> InvalidSnapshot.NotFound�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8Acct_AccountScoping/Step11_Cleanup" classname="" time="32.440">
+			<system-out><![CDATA[�[2m· terminate alpha instance�[0m
+    account_test.go:920: �[2m· terminate alpha instance�[0m
+�[2m· terminate beta instance�[0m
+    account_test.go:926: �[2m· terminate beta instance�[0m
+�[2m· delete snapshots�[0m
+    account_test.go:939: �[2m· delete snapshots�[0m
+�[2m· delete volumes�[0m
+    account_test.go:955: �[2m· delete volumes�[0m
+�[2m· delete key pairs�[0m
+    account_test.go:969: �[2m· delete key pairs�[0m
+�[2m· delete EIGWs�[0m
+    account_test.go:979: �[2m· delete EIGWs�[0m
+�[2m· detach + delete IGWs�[0m
+    account_test.go:993: �[2m· detach + delete IGWs�[0m
+�[2m· delete subnets�[0m
+    account_test.go:1013: �[2m· delete subnets�[0m
+�[2m· delete VPCs�[0m
+    account_test.go:1027: �[2m· delete VPCs�[0m
+�[2m· Phase 8 acct cleanup (deferred)�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8b_VPCSubnetE2E" classname="" time="200.480">
+			<failure message="Failed"><![CDATA[
+�[1m�[36m━━ Phase 8b — VPC Public/Private Subnet E2E ━━�[0m
+�[2m· create-vpc 10.99.0.0/16�[0m
+    vpc_test.go:56: �[2m· create-vpc 10.99.0.0/16�[0m
+�[2mvpc�[0m=vpc-39eff985799c43069 �[2mcidr�[0m=10.99.0.0/16
+    vpc_test.go:67: �[2mvpc�[0m=vpc-39eff985799c43069 �[2mcidr�[0m=10.99.0.0/16
+�[2m· create-subnet 10.99.1.0/24 (public)�[0m
+    vpc_test.go:70: �[2m· create-subnet 10.99.1.0/24 (public)�[0m
+�[2m· modify-subnet-attribute MapPublicIpOnLaunch=true subnet-22aba054e10588254�[0m
+    vpc_test.go:87: �[2m· modify-subnet-attribute MapPublicIpOnLaunch=true subnet-22aba054e10588254�[0m
+�[2mpub_subnet�[0m=subnet-22aba054e10588254 �[2mmap_public_ip�[0m=true
+    vpc_test.go:93: �[2mpub_subnet�[0m=subnet-22aba054e10588254 �[2mmap_public_ip�[0m=true
+�[2m· create-subnet 10.99.2.0/24 (private)�[0m
+    vpc_test.go:96: �[2m· create-subnet 10.99.2.0/24 (private)�[0m
+�[2mpriv_subnet�[0m=subnet-f53aeddb73c134696 �[2mmap_public_ip�[0m=false
+    vpc_test.go:111: �[2mpriv_subnet�[0m=subnet-f53aeddb73c134696 �[2mmap_public_ip�[0m=false
+�[2m· create-internet-gateway�[0m
+    vpc_test.go:114: �[2m· create-internet-gateway�[0m
+�[2m· attach-internet-gateway igw-1520b5a34480da982 -> vpc-39eff985799c43069�[0m
+    vpc_test.go:133: �[2m· attach-internet-gateway igw-1520b5a34480da982 -> vpc-39eff985799c43069�[0m
+�[2migw�[0m=igw-1520b5a34480da982
+    vpc_test.go:139: �[2migw�[0m=igw-1520b5a34480da982
+�[2m· create-route-table (vpc=vpc-39eff985799c43069)�[0m
+    vpc_test.go:142: �[2m· create-route-table (vpc=vpc-39eff985799c43069)�[0m
+�[2m· associate-route-table rtb-662df5e424cbdb60c <- subnet-22aba054e10588254�[0m
+    vpc_test.go:159: �[2m· associate-route-table rtb-662df5e424cbdb60c <- subnet-22aba054e10588254�[0m
+�[2m· create-route 0.0.0.0/0 -> igw-1520b5a34480da982�[0m
+    vpc_test.go:176: �[2m· create-route 0.0.0.0/0 -> igw-1520b5a34480da982�[0m
+�[2mrtb�[0m=rtb-662df5e424cbdb60c �[2massoc�[0m=rtbassoc-5510bf1dc6924eb0b �[2mroute�[0m=0.0.0.0/0->igw-1520b5a34480da982
+    vpc_test.go:192: �[2mrtb�[0m=rtb-662df5e424cbdb60c �[2massoc�[0m=rtbassoc-5510bf1dc6924eb0b �[2mroute�[0m=0.0.0.0/0->igw-1520b5a34480da982
+�[2m· run-instances ami=ami-6f60f2b3bf04b5453 subnet=subnet-22aba054e10588254�[0m
+    vpc_test.go:195: �[2m· run-instances ami=ami-6f60f2b3bf04b5453 subnet=subnet-22aba054e10588254�[0m
+�[2minstance�[0m=i-3177888cba3796d42
+    vpc_test.go:216: �[2minstance�[0m=i-3177888cba3796d42
+    vpc_test.go:218: instance i-3177888cba3796d42 reached state running
+�[2mpublic_ip�[0m=192.168.0.202 �[2mprivate_ip�[0m=10.99.1.4
+    vpc_test.go:224: �[2mpublic_ip�[0m=192.168.0.202 �[2mprivate_ip�[0m=10.99.1.4
+�[2m· waiting for SSH handshake 192.168.0.202:22�[0m
+    vpc_test.go:227: �[2m· waiting for SSH handshake 192.168.0.202:22�[0m
+    vpc_test.go:227: Eventually: condition not met within 3m0s: [SSH handshake 192.168.0.202:22 never completed]]]></failure>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8c_RouteTableValidation" classname="" time="0.030">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 8c — Route Table Validation ━━�[0m
+�[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2msubnet�[0m=subnet-cd62e4c1c3cb50c63
+    vpc_test.go:292: �[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2msubnet�[0m=subnet-cd62e4c1c3cb50c63]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8c_RouteTableValidation/Step1_DefaultMainRouteTable" classname="" time="0.000">
+			<system-out><![CDATA[�[2m· describe-route-tables filter=vpc-id,association.main�[0m
+    vpc_test.go:297: �[2m· describe-route-tables filter=vpc-id,association.main�[0m
+�[2mmain_rtb�[0m=rtb-e42012569f92f7184
+    vpc_test.go:309: �[2mmain_rtb�[0m=rtb-e42012569f92f7184
+�[2mlocal_route�[0m=172.31.0.0/16
+    vpc_test.go:332: �[2mlocal_route�[0m=172.31.0.0/16
+�[2mdefault_route�[0m=0.0.0.0/0->igw-79c2e643660b4b7e3
+    vpc_test.go:350: �[2mdefault_route�[0m=0.0.0.0/0->igw-79c2e643660b4b7e3]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8c_RouteTableValidation/Step2_CustomRouteTableLifecycle" classname="" time="0.020">
+			<system-out><![CDATA[�[2m· create-route-table (vpc=vpc-531252db3b4e1fc08)�[0m
+    vpc_test.go:363: �[2m· create-route-table (vpc=vpc-531252db3b4e1fc08)�[0m
+�[2mcustom_rtb�[0m=rtb-5e6c46936f3e44167
+    vpc_test.go:379: �[2mcustom_rtb�[0m=rtb-5e6c46936f3e44167
+�[2mcustom_local_route�[0m=172.31.0.0/16
+    vpc_test.go:396: �[2mcustom_local_route�[0m=172.31.0.0/16
+�[2mvpc_igw�[0m=igw-79c2e643660b4b7e3
+    vpc_test.go:410: �[2mvpc_igw�[0m=igw-79c2e643660b4b7e3
+�[2m· create-route 0.0.0.0/0 -> igw-79c2e643660b4b7e3�[0m
+    vpc_test.go:415: �[2m· create-route 0.0.0.0/0 -> igw-79c2e643660b4b7e3�[0m
+�[2m· associate-route-table rtb-5e6c46936f3e44167 <- subnet-cd62e4c1c3cb50c63�[0m
+    vpc_test.go:434: �[2m· associate-route-table rtb-5e6c46936f3e44167 <- subnet-cd62e4c1c3cb50c63�[0m
+�[2mcustom_assoc�[0m=rtbassoc-da6377d37847d3d4a
+    vpc_test.go:450: �[2mcustom_assoc�[0m=rtbassoc-da6377d37847d3d4a
+�[2m· disassociate-route-table rtbassoc-da6377d37847d3d4a�[0m
+    vpc_test.go:452: �[2m· disassociate-route-table rtbassoc-da6377d37847d3d4a�[0m
+�[2m· delete-route 0.0.0.0/0�[0m
+    vpc_test.go:460: �[2m· delete-route 0.0.0.0/0�[0m
+�[2m· delete-route-table rtb-5e6c46936f3e44167�[0m
+    vpc_test.go:468: �[2m· delete-route-table rtb-5e6c46936f3e44167�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8c_RouteTableValidation/Step3_ErrorPaths" classname="" time="0.010">
+			<system-out><![CDATA[�[2m· create-route-table (scratch for error paths)�[0m
+    vpc_test.go:481: �[2m· create-route-table (scratch for error paths)�[0m
+�[2m· delete-route local=172.31.0.0/16 (must fail)�[0m
+    vpc_test.go:517: �[2m· delete-route local=172.31.0.0/16 (must fail)�[0m
+�[2mdelete_local_err�[0m=InvalidParameterValue
+    vpc_test.go:529: �[2mdelete_local_err�[0m=InvalidParameterValue
+�[2m· delete-route-table rtb-e42012569f92f7184 (main, must fail)�[0m
+    vpc_test.go:533: �[2m· delete-route-table rtb-e42012569f92f7184 (main, must fail)�[0m
+�[2mdelete_main_err�[0m=DependencyViolation
+    vpc_test.go:540: �[2mdelete_main_err�[0m=DependencyViolation]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8d_NATGateway" classname="" time="106.010">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 8d — NAT Gateway E2E ━━�[0m
+�[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2mpub_subnet�[0m=subnet-cd62e4c1c3cb50c63 �[2msg�[0m=sg-764a7fa2a39577204
+    natgw_test.go:54: �[2mvpc�[0m=vpc-531252db3b4e1fc08 �[2mpub_subnet�[0m=subnet-cd62e4c1c3cb50c63 �[2msg�[0m=sg-764a7fa2a39577204
+�[2m· create-subnet 172.31.16.0/20 (private)�[0m
+    natgw_test.go:58: �[2m· create-subnet 172.31.16.0/20 (private)�[0m
+�[2mpriv_subnet�[0m=subnet-2360a40e28371e214
+    natgw_test.go:71: �[2mpriv_subnet�[0m=subnet-2360a40e28371e214
+�[2m· run-instances bastion (default subnet)�[0m
+    natgw_test.go:74: �[2m· run-instances bastion (default subnet)�[0m
+�[2m· run-instances private (private subnet)�[0m
+    natgw_test.go:98: �[2m· run-instances private (private subnet)�[0m
+    natgw_test.go:118: instance i-268402f0e28b5e477 reached state running
+    natgw_test.go:119: instance i-db7f3ed6e4cdaa5aa reached state running
+�[2mbastion�[0m=i-268402f0e28b5e477 �[2mbastion_ip�[0m=192.168.0.202 �[2mprivate�[0m=i-db7f3ed6e4cdaa5aa �[2mprivate_ip�[0m=172.31.16.4
+    natgw_test.go:128: �[2mbastion�[0m=i-268402f0e28b5e477 �[2mbastion_ip�[0m=192.168.0.202 �[2mprivate�[0m=i-db7f3ed6e4cdaa5aa �[2mprivate_ip�[0m=172.31.16.4
+�[2m· waiting for SSH handshake 192.168.0.202:22�[0m
+    natgw_test.go:133: �[2m· waiting for SSH handshake 192.168.0.202:22�[0m
+�[2m· scp keypair -> bastion:/tmp/key.pem�[0m
+    natgw_test.go:139: �[2m· scp keypair -> bastion:/tmp/key.pem�[0m
+�[2m· wait for private SSH via bastion�[0m
+    natgw_test.go:151: �[2m· wait for private SSH via bastion�[0m
+�[2m· baseline: private VM has no internet�[0m
+    natgw_test.go:172: �[2m· baseline: private VM has no internet�[0m
+�[2m· allocate-address (NAT EIP)�[0m
+    natgw_test.go:178: �[2m· allocate-address (NAT EIP)�[0m
+�[2meip�[0m=192.168.0.203 �[2malloc�[0m=eipalloc-2d81753157cd62a50
+    natgw_test.go:195: �[2meip�[0m=192.168.0.203 �[2malloc�[0m=eipalloc-2d81753157cd62a50
+�[2m· create-nat-gateway in subnet-cd62e4c1c3cb50c63�[0m
+    natgw_test.go:197: �[2m· create-nat-gateway in subnet-cd62e4c1c3cb50c63�[0m
+�[2mnat_gw�[0m=nat-812216dd5105180c5
+    natgw_test.go:216: �[2mnat_gw�[0m=nat-812216dd5105180c5
+    natgw_test.go:218: nat gateway nat-812216dd5105180c5 reached state available
+�[2m· create-route-table�[0m
+    natgw_test.go:223: �[2m· create-route-table�[0m
+�[2m· associate-route-table rtb-8d6a4abd5f666ec13 <- subnet-2360a40e28371e214�[0m
+    natgw_test.go:241: �[2m· associate-route-table rtb-8d6a4abd5f666ec13 <- subnet-2360a40e28371e214�[0m
+�[2m· create-route 0.0.0.0/0 -> nat-812216dd5105180c5�[0m
+    natgw_test.go:259: �[2m· create-route 0.0.0.0/0 -> nat-812216dd5105180c5�[0m
+�[2m· verify private VM reaches 8.8.8.8 via NAT GW�[0m
+    natgw_test.go:279: �[2m· verify private VM reaches 8.8.8.8 via NAT GW�[0m
+�[2m· delete-nat-gateway nat-812216dd5105180c5�[0m
+    natgw_test.go:293: �[2m· delete-nat-gateway nat-812216dd5105180c5�[0m
+    natgw_test.go:318: nat gateway nat-812216dd5105180c5 reached state deleted
+�[2m· verify private VM lost internet after NAT GW teardown�[0m
+    natgw_test.go:329: �[2m· verify private VM lost internet after NAT GW teardown�[0m]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase8e_SGToSGDatapath" classname="" time="80.940">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 8e — Security Group SG-to-SG Datapath (OVN) ━━�[0m
+�[2m· 8e-1 create sge-client + sge-target security groups�[0m
+    datapath_test.go:61: �[2m· 8e-1 create sge-client + sge-target security groups�[0m
+�[2mclient_sg�[0m=sg-56e73c764a649f7ff �[2mtarget_sg�[0m=sg-67ad5efaa8be83b79
+    datapath_test.go:83: �[2mclient_sg�[0m=sg-56e73c764a649f7ff �[2mtarget_sg�[0m=sg-67ad5efaa8be83b79
+�[2m· 8e-1 authorize target-sg ingress tcp/8080 from sg-56e73c764a649f7ff�[0m
+    datapath_test.go:91: �[2m· 8e-1 authorize target-sg ingress tcp/8080 from sg-56e73c764a649f7ff�[0m
+�[2m· 8e-2 launch client-vm + target-vm�[0m
+    datapath_test.go:109: �[2m· 8e-2 launch client-vm + target-vm�[0m
+    datapath_test.go:132: instance i-eae6263d9b9fda069 reached state running
+    datapath_test.go:133: instance i-9d9411a83549d34e3 reached state running
+�[2mclient_priv�[0m=172.31.0.5 �[2mclient_eni�[0m=eni-0f878da0143028915 �[2mtarget_priv�[0m=172.31.0.6 �[2mtarget_eni�[0m=eni-3bd355e19c7fdd45c
+    datapath_test.go:142: �[2mclient_priv�[0m=172.31.0.5 �[2mclient_eni�[0m=eni-0f878da0143028915 �[2mtarget_priv�[0m=172.31.0.6 �[2mtarget_eni�[0m=eni-3bd355e19c7fdd45c
+�[2m· 8e-3 ovn-nbctl port_group membership�[0m
+    datapath_test.go:151: �[2m· 8e-3 ovn-nbctl port_group membership�[0m
+�[2mclient_pg�[0m=sg_56e73c764a649f7ff �[2mtarget_pg�[0m=sg_67ad5efaa8be83b79
+    datapath_test.go:163: �[2mclient_pg�[0m=sg_56e73c764a649f7ff �[2mtarget_pg�[0m=sg_67ad5efaa8be83b79
+�[2m· 8e-3 ovn-sbctl address_set sg_56e73c764a649f7ff_ip4 contains 172.31.0.5�[0m
+    datapath_test.go:170: �[2m· 8e-3 ovn-sbctl address_set sg_56e73c764a649f7ff_ip4 contains 172.31.0.5�[0m
+�[2m· 8e-3 wait for client-vm SSH at 192.168.0.202:22�[0m
+    datapath_test.go:188: �[2m· 8e-3 wait for client-vm SSH at 192.168.0.202:22�[0m
+�[2m· 8e-4 allowed traffic client -> target:172.31.0.6:8080�[0m
+    datapath_test.go:199: �[2m· 8e-4 allowed traffic client -> target:172.31.0.6:8080�[0m
+�[2mstep4�[0m=allowed_traffic_ok
+    datapath_test.go:208: �[2mstep4�[0m=allowed_traffic_ok
+�[2m· 8e-5 denied traffic client -> target:22 (no SSH ingress)�[0m
+    datapath_test.go:213: �[2m· 8e-5 denied traffic client -> target:22 (no SSH ingress)�[0m
+�[2mstep5�[0m=denied_traffic_ok
+    datapath_test.go:217: �[2mstep5�[0m=denied_traffic_ok
+�[2m· 8e-6 revoke target-sg ingress, retest (sync RequestEvent contract)�[0m
+    datapath_test.go:222: �[2m· 8e-6 revoke target-sg ingress, retest (sync RequestEvent contract)�[0m
+�[2m· 8e-6 verify client -> target:8080 now blocked�[0m
+    datapath_test.go:240: �[2m· 8e-6 verify client -> target:8080 now blocked�[0m
+�[2mstep6�[0m=revoke_blocked_ok
+    datapath_test.go:244: �[2mstep6�[0m=revoke_blocked_ok
+�[2m· 8e-7 re-authorize target-sg ingress, verify traffic restored�[0m
+    datapath_test.go:250: �[2m· 8e-7 re-authorize target-sg ingress, verify traffic restored�[0m
+�[2mstep7�[0m=restore_ok
+    datapath_test.go:271: �[2mstep7�[0m=restore_ok
+    datapath_test.go:127: instance i-eae6263d9b9fda069 reached state terminated
+    datapath_test.go:127: instance i-9d9411a83549d34e3 reached state terminated]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase9_Teardown" classname="" time="8.300">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 9 — Teardown ━━�[0m
+�[2m· delete-snapshot snap-dde89b9ea2ef2e05a (custom AMI backing)�[0m
+    teardown_test.go:33: �[2m· delete-snapshot snap-dde89b9ea2ef2e05a (custom AMI backing)�[0m
+�[2mdeleted_snapshot�[0m=snap-dde89b9ea2ef2e05a
+    teardown_test.go:39: �[2mdeleted_snapshot�[0m=snap-dde89b9ea2ef2e05a
+�[2m· deregister-image ami-7dcf263a412dd14c6�[0m
+    teardown_test.go:51: �[2m· deregister-image ami-7dcf263a412dd14c6�[0m
+�[2mderegistered_ami�[0m=ami-7dcf263a412dd14c6
+    teardown_test.go:57: �[2mderegistered_ami�[0m=ami-7dcf263a412dd14c6
+�[2m· terminate-instances i-0c615e5dd74050f53�[0m
+    teardown_test.go:74: �[2m· terminate-instances i-0c615e5dd74050f53�[0m
+�[2mterminated_instance�[0m=i-0c615e5dd74050f53
+    teardown_test.go:81: �[2mterminated_instance�[0m=i-0c615e5dd74050f53
+�[2m· delete-key-pair test-key-1�[0m
+    teardown_test.go:94: �[2m· delete-key-pair test-key-1�[0m
+�[2mdeleted_keypair�[0m=test-key-1
+    teardown_test.go:100: �[2mdeleted_keypair�[0m=test-key-1]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase9a_VerifyTeardown" classname="" time="0.560">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 9a — Verify Teardown ━━�[0m
+�[2m· describe-instances i-0c615e5dd74050f53�[0m
+    teardown_test.go:123: �[2m· describe-instances i-0c615e5dd74050f53�[0m
+�[2minstance_state�[0m=terminated
+    teardown_test.go:135: �[2minstance_state�[0m=terminated
+�[2m· describe-images ami-7dcf263a412dd14c6�[0m
+    teardown_test.go:147: �[2m· describe-images ami-7dcf263a412dd14c6�[0m
+�[2mami_reaped�[0m=ami-7dcf263a412dd14c6
+    teardown_test.go:159: �[2mami_reaped�[0m=ami-7dcf263a412dd14c6
+�[2m· describe-key-pairs (must not contain test-key-1)�[0m
+    teardown_test.go:167: �[2m· describe-key-pairs (must not contain test-key-1)�[0m
+�[2mkeypairs_listed�[0m=1
+    teardown_test.go:176: �[2mkeypairs_listed�[0m=1]]></system-out>
+		</testcase>
+		<testcase name="TestSingleNode/Phase9b_FinalClusterStats" classname="" time="3.070">
+			<system-out><![CDATA[
+�[1m�[36m━━ Phase 9b — Final Cluster Stats ━━�[0m
+�[2mspx_get_vms_bytes�[0m=118
+    teardown_test.go:208: �[2mspx_get_vms_bytes�[0m=118]]></system-out>
+		</testcase>
+	</testsuite>
+</testsuites>
@@ -0,0 +1 @@
+__pycache__/
@@ -0,0 +1,46 @@
+# junit-report
+
+Composite action that renders a glob of JUnit XML files into the GitHub
+Actions job summary and emits one `::error` annotation per failed test case.
+
+## Usage
+
+```yaml
+- name: Publish test summary
+  if: always()
+  uses: ./spinifex/.github/actions/junit-report   # or your own checkout path
+  with:
+    junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+    title: "Go E2E results"
+    # fail-on-empty: "true"   # opt-in: fail the step if the glob matches nothing
+```
+
+Outputs:
+
+- `failures` — total `<failure>` + `<error>` counts across all matched files.
+- `total`    — total `<testcase>` count across all matched files.
+
+The action expects standard JUnit XML (e.g. emitted by
+`gotestsum --junitfile`, `go-junit-report`, or pytest's `--junitxml`). Both
+single `<testsuite>` and wrapping `<testsuites>` roots are accepted.
+
+## Companion convention: `::group::` for noisy steps
+
+Long-running shell phases (tofu apply, package installs, tar-over-ssh) should
+wrap their output in `::group::name` / `::endgroup::` markers so the Actions
+UI collapses them by default. Pattern used in the e2e-go / e2e-ddil workflows:
+
+```bash
+run() {
+  local label="$1"; shift
+  echo "::group::${label}"
+  "$@" 2>&1 | tee -a "$TOFU_LOG"
+  local rc=${PIPESTATUS[0]}
+  echo "::endgroup::"
+  return "$rc"
+}
+run "tofu apply" tofu apply -var-file=... -auto-approve -input=false
+```
+
+Combined: groups make the live UI readable; this action makes the run header
+land on the failures without scrolling the log.
@@ -0,0 +1,41 @@
+name: JUnit Report
+description: >-
+  Render JUnit XML files into the GitHub Actions job summary and emit
+  inline ::error annotations for each failed test case. Designed to be
+  called once per job after the test step (with `if: always()`) so summary
+  + annotations appear on both passing and failing runs.
+
+inputs:
+  junit-glob:
+    description: Shell glob matching JUnit XML files to render.
+    required: true
+  title:
+    description: Heading rendered at the top of the job summary section.
+    required: false
+    default: "Test results"
+  fail-on-empty:
+    description: >-
+      When "true", the action fails if the glob matches no files. Useful
+      for catching silently-skipped test phases.
+    required: false
+    default: "false"
+
+outputs:
+  failures:
+    description: Total failed + errored test cases across all matched files.
+    value: ${{ steps.render.outputs.failures }}
+  total:
+    description: Total test cases across all matched files.
+    value: ${{ steps.render.outputs.total }}
+
+runs:
+  using: composite
+  steps:
+    - id: render
+      shell: bash
+      env:
+        JUNIT_GLOB: ${{ inputs.junit-glob }}
+        JUNIT_TITLE: ${{ inputs.title }}
+        JUNIT_FAIL_ON_EMPTY: ${{ inputs.fail-on-empty }}
+      run: |
+        python3 "${{ github.action_path }}/render.py"
@@ -0,0 +1,281 @@
+#!/usr/bin/env python3
+"""Render JUnit XML files into GitHub Actions job summary + annotations.
+
+Inputs (env):
+    JUNIT_GLOB           Glob matching JUnit XML files.
+    JUNIT_TITLE          Heading text for the summary section.
+    JUNIT_FAIL_ON_EMPTY  "true" to exit non-zero when glob matches nothing.
+
+GitHub-Actions outputs (via $GITHUB_OUTPUT):
+    failures             Sum of <failures> + <errors> across all suites.
+    total                Sum of <tests> across all suites.
+
+Side effects:
+    Appends a markdown section to $GITHUB_STEP_SUMMARY with a per-file
+    totals table and a collapsible details block per failed test.
+    Emits one `::error` workflow command per failed/errored test case so
+    they surface as annotations in the run UI.
+"""
+
+from __future__ import annotations
+
+import glob
+import os
+import re
+import sys
+import xml.etree.ElementTree as ET
+from dataclasses import dataclass
+
+
+# Strip ANSI SGR escapes from skipped/failure CDATA so the rendered reason is
+# plain text (CDATA preserves the colour codes Go emits via harness.Phase).
+_ANSI_RE = re.compile(r"\x1b\[[0-9;]*m")
+
+
+def _strip_ansi(s: str) -> str:
+    return _ANSI_RE.sub("", s)
+
+
+# Last `<file>.go:<line>: <reason>` line in a CDATA block is the t.Skipf /
+# t.Fatal call site — that's the actionable signal. Phase banners and probe
+# logs above it are noise for the summary view.
+_SKIP_REASON_RE = re.compile(r"([A-Za-z0-9_.\-/]+\.go:\d+:\s*.+)$")
+
+
+def _extract_skip_reason(cdata: str) -> str:
+    cdata = _strip_ansi(cdata).strip()
+    if not cdata:
+        return ""
+    for line in reversed(cdata.splitlines()):
+        line = line.strip()
+        m = _SKIP_REASON_RE.search(line)
+        if m:
+            return m.group(1).strip()
+    return cdata.splitlines()[-1].strip()
+
+
+# Workflow commands carry their payload on one line; literal newlines must
+# be percent-escaped per GitHub's workflow-command spec.
+def _esc(s: str) -> str:
+    return (
+        s.replace("%", "%25")
+        .replace("\r", "%0D")
+        .replace("\n", "%0A")
+    )
+
+
+@dataclass
+class Case:
+    suite: str
+    classname: str
+    name: str
+    time: float
+    status: str  # "pass" | "fail" | "error" | "skip"
+    message: str
+    detail: str
+
+
+@dataclass
+class FileTotals:
+    path: str
+    tests: int
+    failures: int
+    errors: int
+    skipped: int
+    time: float
+    cases: list[Case]
+
+
+def _parse_file(path: str) -> FileTotals:
+    root = ET.parse(path).getroot()
+    # JUnit XML has two shapes: <testsuite> as root, or <testsuites> wrapping
+    # multiple <testsuite> children. Iterating with .iter() handles both.
+    suites = list(root.iter("testsuite"))
+    tests = sum(int(s.get("tests", 0) or 0) for s in suites)
+    failures = sum(int(s.get("failures", 0) or 0) for s in suites)
+    errors = sum(int(s.get("errors", 0) or 0) for s in suites)
+    skipped = sum(int(s.get("skipped", 0) or 0) for s in suites)
+    time = sum(float(s.get("time", 0) or 0) for s in suites)
+
+    cases: list[Case] = []
+    for s in suites:
+        suite_name = s.get("name", "")
+        for tc in s.findall("testcase"):
+            status = "pass"
+            message = ""
+            detail = ""
+            for tag in ("failure", "error"):
+                node = tc.find(tag)
+                if node is not None:
+                    status = "fail" if tag == "failure" else "error"
+                    message = (node.get("message") or "").strip()
+                    detail = (node.text or "").strip()
+                    break
+            skip_node = tc.find("skipped")
+            if skip_node is not None and status == "pass":
+                status = "skip"
+                # Skipped reason: prefer the CDATA call-site line, fall back
+                # to the @message attribute if CDATA is empty.
+                cdata = (skip_node.text or "").strip()
+                message = _extract_skip_reason(cdata) or (
+                    skip_node.get("message") or ""
+                ).strip()
+            cases.append(
+                Case(
+                    suite=suite_name,
+                    classname=tc.get("classname", ""),
+                    name=tc.get("name", ""),
+                    time=float(tc.get("time", 0) or 0),
+                    status=status,
+                    message=message,
+                    detail=detail,
+                )
+            )
+
+    # Drop container test cases: any case whose name is a strict prefix of
+    # another case's name (with "/" separator) is a parent t.Run wrapper.
+    # Go reports the parent's status as a rollup of its children, so counting
+    # both double-tallies the same outcome (e.g. TestSingleNode failing
+    # because Phase8b_VPCSubnetE2E failed would show as 2 failures, not 1).
+    names = {c.name for c in cases}
+    prefixes = {n.rsplit("/", 1)[0] for n in names if "/" in n}
+    # A name is a container if some other name starts with "<name>/".
+    # Use the prefixes set rather than O(n^2) scanning.
+    def _is_container(name: str) -> bool:
+        return name in prefixes
+
+    leaf_cases = [c for c in cases if not _is_container(c.name)]
+
+    # Recompute totals from leaf cases so the per-file counts in the summary
+    # table match the de-duplicated failure list below it.
+    leaf_tests = len(leaf_cases)
+    leaf_failures = sum(1 for c in leaf_cases if c.status == "fail")
+    leaf_errors = sum(1 for c in leaf_cases if c.status == "error")
+    leaf_skipped = sum(1 for c in leaf_cases if c.status == "skip")
+
+    return FileTotals(
+        path=path,
+        tests=leaf_tests,
+        failures=leaf_failures,
+        errors=leaf_errors,
+        skipped=leaf_skipped,
+        time=time,
+        cases=leaf_cases,
+    )
+
+
+def _file_label(path: str) -> str:
+    base = os.path.basename(path)
+    if base.endswith(".xml"):
+        base = base[:-4]
+    if base.startswith("junit-"):
+        base = base[len("junit-"):]
+    return base or path
+
+
+def main() -> int:
+    pattern = os.environ.get("JUNIT_GLOB", "").strip()
+    title = os.environ.get("JUNIT_TITLE", "Test results").strip() or "Test results"
+    fail_on_empty = os.environ.get("JUNIT_FAIL_ON_EMPTY", "").strip().lower() == "true"
+
+    if not pattern:
+        print("::error::junit-report: junit-glob input is empty", file=sys.stderr)
+        return 1
+
+    paths = sorted(glob.glob(pattern))
+    summary_path = os.environ.get("GITHUB_STEP_SUMMARY", "/dev/null")
+    output_path = os.environ.get("GITHUB_OUTPUT", "/dev/null")
+
+    with open(summary_path, "a", encoding="utf-8") as summary, \
+            open(output_path, "a", encoding="utf-8") as outputs:
+
+        summary.write(f"## {title}\n\n")
+
+        if not paths:
+            summary.write(f"_No JUnit files matched `{pattern}`._\n\n")
+            outputs.write("failures=0\n")
+            outputs.write("total=0\n")
+            if fail_on_empty:
+                print(f"::error::junit-report: no files matched {pattern}", file=sys.stderr)
+                return 1
+            return 0
+
+        totals_by_file = [_parse_file(p) for p in paths]
+        grand_total = sum(f.tests for f in totals_by_file)
+        grand_fail = sum(f.failures + f.errors for f in totals_by_file)
+        grand_skip = sum(f.skipped for f in totals_by_file)
+        grand_pass = grand_total - grand_fail - grand_skip
+        grand_time = sum(f.time for f in totals_by_file)
+
+        if grand_fail == 0:
+            verdict = f"✅ **pass** — {grand_pass} / {grand_total}"
+        else:
+            verdict = f"❌ **{grand_fail} failed** — {grand_pass} / {grand_total} pass"
+        skip_part = f", {grand_skip} skipped" if grand_skip else ""
+        summary.write(f"{verdict}{skip_part} &nbsp; · &nbsp; {grand_time:.1f}s\n\n")
+
+        # Compact table — single status column, shorter headers. Drops
+        # separate err column (errors fold into fail count) so narrow
+        # screens don't wrap.
+        summary.write("| | Suite | Pass | Fail | Skip | Time |\n")
+        summary.write("|:-:|---|---:|---:|---:|---:|\n")
+        for f in totals_by_file:
+            fails = f.failures + f.errors
+            passed = f.tests - fails - f.skipped
+            icon = "✅" if fails == 0 else "❌"
+            summary.write(
+                f"| {icon} | `{_file_label(f.path)}` | {passed} | {fails} | "
+                f"{f.skipped} | {f.time:.1f}s |\n"
+            )
+        summary.write("\n")
+
+        failed_cases = [
+            c for f in totals_by_file for c in f.cases if c.status in ("fail", "error")
+        ]
+        if failed_cases:
+            summary.write("---\n\n**Failures**\n\n")
+            for c in failed_cases:
+                header = f"{c.classname}.{c.name}" if c.classname else c.name
+                # One line per failure: name + truncated message. Full
+                # detail lives in the test artifact log; deep nesting and
+                # long messages wrap badly on narrow screens.
+                msg_short = (c.message or "").splitlines()[0] if c.message else ""
+                if len(msg_short) > 120:
+                    msg_short = msg_short[:117] + "…"
+                line = f"- ❌ `{header}`"
+                if msg_short:
+                    line += f" — {msg_short}"
+                summary.write(line + "\n")
+
+                # Workflow annotation — surfaces in run header + Files-changed view.
+                # classname for Go tests is the package path; map to a file when we can.
+                file_hint = c.classname.replace(".", "/")
+                msg = c.message or f"{c.status}: {header}"
+                print(f"::error file={file_hint},title={header}::{_esc(msg)}")
+            summary.write("\n")
+
+        skipped_cases = [
+            c for f in totals_by_file for c in f.cases if c.status == "skip"
+        ]
+        if skipped_cases:
+            summary.write("---\n\n<details><summary><b>Skipped</b> ")
+            summary.write(f"({len(skipped_cases)})</summary>\n\n")
+            for c in skipped_cases:
+                header = f"{c.classname}.{c.name}" if c.classname else c.name
+                reason = (c.message or "").splitlines()[0] if c.message else ""
+                if len(reason) > 160:
+                    reason = reason[:157] + "…"
+                line = f"- ⏭️ `{header}`"
+                if reason:
+                    line += f" — {reason}"
+                summary.write(line + "\n")
+            summary.write("\n</details>\n\n")
+
+        outputs.write(f"failures={grand_fail}\n")
+        outputs.write(f"total={grand_total}\n")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,225 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="620" viewBox="0 0 900 620">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="bar-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#161b22"/>
+      <stop offset="45%" style="stop-color:#1f2028"/>
+      <stop offset="100%" style="stop-color:#161b22"/>
+    </linearGradient>
+    <linearGradient id="client-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="gw-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#DC262633"/>
+      <stop offset="100%" style="stop-color:#7F1D1D14"/>
+    </linearGradient>
+    <linearGradient id="nats-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#F87171"/>
+      <stop offset="50%" style="stop-color:#DC2626"/>
+      <stop offset="100%" style="stop-color:#7F1D1D"/>
+    </linearGradient>
+  </defs>
+
+  <rect width="900" height="620" fill="url(#bg)" rx="12"/>
+
+  <!-- Title bar -->
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">SPINIFEX ARCHITECTURE</text>
+  <text x="24" y="52" font-family="'Segoe UI', system-ui, sans-serif" font-size="16" fill="#f5f5f5" font-weight="500">AWS SDK → Gateway → NATS → Service daemons</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">stateless · message-driven · horizontally scalable</text>
+
+  <!-- ═══ CLIENT LAYER ════════════════════════════════════ -->
+  <rect x="16" y="72" width="868" height="78" rx="8" fill="url(#client-grad)" stroke="#3fb950" stroke-width="1" stroke-opacity="0.5"/>
+  <text x="32" y="93" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#3fb950" font-weight="700" letter-spacing="1">CLIENTS</text>
+  <text x="32" y="108" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">Standard AWS tooling — unmodified, no shims, no custom SDK</text>
+
+  <g transform="translate(32, 120)" font-family="'SF Mono', ui-monospace, monospace" font-size="10">
+    <rect x="0" y="0" width="86" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="43" y="15" fill="#7ee787" text-anchor="middle">aws-cli</text>
+
+    <rect x="94" y="0" width="96" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="142" y="15" fill="#7ee787" text-anchor="middle">Terraform</text>
+
+    <rect x="198" y="0" width="76" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="236" y="15" fill="#7ee787" text-anchor="middle">boto3</text>
+
+    <rect x="282" y="0" width="98" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="331" y="15" fill="#7ee787" text-anchor="middle">aws-sdk-go</text>
+
+    <rect x="388" y="0" width="86" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="431" y="15" fill="#7ee787" text-anchor="middle">kubectl</text>
+
+    <rect x="482" y="0" width="86" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="525" y="15" fill="#7ee787" text-anchor="middle">Packer</text>
+
+    <rect x="576" y="0" width="120" height="22" rx="4" fill="#3fb950" fill-opacity="0.12" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.8"/>
+    <text x="636" y="15" fill="#7ee787" text-anchor="middle">your application</text>
+  </g>
+
+  <!-- ═══ HTTPS + SigV4 connector ═══════════════════════════ -->
+  <line x1="450" y1="150" x2="450" y2="178" stroke="#DC2626" stroke-width="1.2" stroke-dasharray="3,3" opacity="0.7"/>
+  <rect x="390" y="154" width="120" height="18" rx="9" fill="#0a0a0a" stroke="#DC2626" stroke-width="0.8" stroke-opacity="0.5"/>
+  <text x="450" y="166" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#F87171" text-anchor="middle">HTTPS · SigV4 · :9999</text>
+
+  <!-- ═══ AWS GATEWAY ════════════════════════════════════ -->
+  <rect x="16" y="178" width="868" height="68" rx="8" fill="url(#gw-grad)" stroke="#DC2626" stroke-width="1.3" stroke-opacity="0.75"/>
+  <text x="32" y="200" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" font-weight="700" letter-spacing="1">AWS GATEWAY</text>
+  <text x="32" y="215" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">spinifex/services/awsgw · TLS termination · SigV4 verification · IAM policy eval · NATS routing</text>
+
+  <g transform="translate(32, 224)" font-family="'SF Mono', ui-monospace, monospace" font-size="9">
+    <rect x="0" y="0" width="78" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="39" y="11" fill="#FCA5A5" text-anchor="middle">TLS 1.3</text>
+
+    <rect x="86" y="0" width="78" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="125" y="11" fill="#FCA5A5" text-anchor="middle">SigV4 auth</text>
+
+    <rect x="172" y="0" width="98" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="221" y="11" fill="#FCA5A5" text-anchor="middle">policy eval</text>
+
+    <rect x="278" y="0" width="98" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="327" y="11" fill="#FCA5A5" text-anchor="middle">req routing</text>
+
+    <rect x="384" y="0" width="88" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="428" y="11" fill="#FCA5A5" text-anchor="middle">rate-limit</text>
+
+    <rect x="480" y="0" width="98" height="16" rx="3" fill="#7F1D1D" fill-opacity="0.3" stroke="#DC2626" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="529" y="11" fill="#FCA5A5" text-anchor="middle">audit log</text>
+  </g>
+
+  <!-- ═══ NATS BUS ═══════════════════════════════════════ -->
+  <line x1="450" y1="246" x2="450" y2="272" stroke="#DC2626" stroke-width="1.2" stroke-dasharray="3,3" opacity="0.7"/>
+  <rect x="16" y="272" width="868" height="34" rx="6" fill="url(#nats-grad)" opacity="0.92"/>
+  <text x="32" y="293" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#f5f5f5" font-weight="700" letter-spacing="1.5">NATS BUS</text>
+  <text x="140" y="293" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#fee2e2" opacity="0.95">ec2.*   ebs.*   s3.*   vpc.*   iam.*</text>
+  <text x="876" y="293" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#fee2e2" text-anchor="end" opacity="0.9">queue groups · fan-out · request/reply</text>
+
+  <!-- ═══ DAEMON FANOUT CONNECTORS ═══════════════════════ -->
+  <g stroke="#DC2626" stroke-width="0.9" stroke-dasharray="3,3" opacity="0.55" fill="none">
+    <line x1="100" y1="306" x2="100" y2="328"/>
+    <line x1="275" y1="306" x2="275" y2="328"/>
+    <line x1="450" y1="306" x2="450" y2="328"/>
+    <line x1="625" y1="306" x2="625" y2="328"/>
+    <line x1="800" y1="306" x2="800" y2="328"/>
+  </g>
+
+  <!-- ═══ SERVICE DAEMONS ROW ═══════════════════════════ -->
+  <!-- EC2 -->
+  <g transform="translate(28, 328)">
+    <rect width="144" height="82" rx="7" fill="#7F1D1D" fill-opacity="0.14" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">ec2.* daemon</text>
+    <line x1="16" y1="30" x2="128" y2="30" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <text x="72" y="48" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">RunInstances</text>
+    <text x="72" y="62" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">TerminateInstances</text>
+    <text x="72" y="76" font-family="'Segoe UI', sans-serif" font-size="9" fill="#a1a1aa" text-anchor="middle">QMP · cloud-init</text>
+  </g>
+
+  <!-- EBS -->
+  <g transform="translate(204, 328)">
+    <rect width="144" height="82" rx="7" fill="#7F1D1D" fill-opacity="0.14" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">ebs.* daemon</text>
+    <line x1="16" y1="30" x2="128" y2="30" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <text x="72" y="48" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">CreateVolume</text>
+    <text x="72" y="62" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">AttachVolume</text>
+    <text x="72" y="76" font-family="'Segoe UI', sans-serif" font-size="9" fill="#a1a1aa" text-anchor="middle">snapshots · WAL</text>
+  </g>
+
+  <!-- S3 -->
+  <g transform="translate(380, 328)">
+    <rect width="144" height="82" rx="7" fill="#7F1D1D" fill-opacity="0.14" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">s3.* daemon</text>
+    <line x1="16" y1="30" x2="128" y2="30" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <text x="72" y="48" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">PutObject · GetObject</text>
+    <text x="72" y="62" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">Multipart · presigned</text>
+    <text x="72" y="76" font-family="'Segoe UI', sans-serif" font-size="9" fill="#a1a1aa" text-anchor="middle">erasure-coded</text>
+  </g>
+
+  <!-- VPC -->
+  <g transform="translate(556, 328)">
+    <rect width="144" height="82" rx="7" fill="#7F1D1D" fill-opacity="0.14" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">vpc.* daemon</text>
+    <line x1="16" y1="30" x2="128" y2="30" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <text x="72" y="48" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">Subnets · SGs · RTs</text>
+    <text x="72" y="62" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">ALB · NAT · IGW</text>
+    <text x="72" y="76" font-family="'Segoe UI', sans-serif" font-size="9" fill="#a1a1aa" text-anchor="middle">OVN · OVS</text>
+  </g>
+
+  <!-- IAM -->
+  <g transform="translate(732, 328)">
+    <rect width="144" height="82" rx="7" fill="#7F1D1D" fill-opacity="0.14" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">iam.* daemon</text>
+    <line x1="16" y1="30" x2="128" y2="30" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <text x="72" y="48" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">Users · Roles · Keys</text>
+    <text x="72" y="62" font-family="'Segoe UI', sans-serif" font-size="10" fill="#f5f5f5" text-anchor="middle">Policy evaluation</text>
+    <text x="72" y="76" font-family="'Segoe UI', sans-serif" font-size="9" fill="#a1a1aa" text-anchor="middle">audit trail</text>
+  </g>
+
+  <!-- ═══ DAEMON → BACKEND CONNECTORS ═══════════════════ -->
+  <g stroke="#71717a" stroke-width="0.8" stroke-dasharray="2,3" opacity="0.55" fill="none">
+    <line x1="100" y1="410" x2="100" y2="440"/>
+    <line x1="276" y1="410" x2="276" y2="440"/>
+    <line x1="452" y1="410" x2="452" y2="440"/>
+    <line x1="628" y1="410" x2="628" y2="440"/>
+    <line x1="804" y1="410" x2="804" y2="440"/>
+  </g>
+
+  <!-- ═══ BACKEND RESOURCES BAR ═════════════════════════ -->
+  <rect x="16" y="440" width="868" height="26" rx="5" fill="url(#bar-grad)"/>
+  <text x="450" y="458" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle" letter-spacing="1.5">BACKEND RESOURCES · local hardware · no cloud dependency</text>
+
+  <!-- ═══ BACKEND ROW ═══════════════════════════════════ -->
+  <!-- QEMU/KVM -->
+  <g transform="translate(28, 474)">
+    <rect width="144" height="62" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" text-anchor="middle" font-weight="600">QEMU / KVM</text>
+    <text x="72" y="40" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">hypervisor</text>
+    <text x="72" y="54" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a" text-anchor="middle">host kernel · virtio</text>
+  </g>
+
+  <!-- Viperblock -->
+  <g transform="translate(204, 474)">
+    <rect width="144" height="62" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" text-anchor="middle" font-weight="600">Viperblock</text>
+    <text x="72" y="40" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">block storage</text>
+    <text x="72" y="54" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a" text-anchor="middle">NVMe WAL · NBD</text>
+  </g>
+
+  <!-- Predastore -->
+  <g transform="translate(380, 474)">
+    <rect width="144" height="62" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" text-anchor="middle" font-weight="600">Predastore</text>
+    <text x="72" y="40" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">object storage</text>
+    <text x="72" y="54" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a" text-anchor="middle">Reed–Solomon</text>
+  </g>
+
+  <!-- OVN/OVS -->
+  <g transform="translate(556, 474)">
+    <rect width="144" height="62" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" text-anchor="middle" font-weight="600">OVN / OVS</text>
+    <text x="72" y="40" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">SDN fabric</text>
+    <text x="72" y="54" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a" text-anchor="middle">overlays · flow rules</text>
+  </g>
+
+  <!-- IAM store -->
+  <g transform="translate(732, 474)">
+    <rect width="144" height="62" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="72" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" text-anchor="middle" font-weight="600">Identity store</text>
+    <text x="72" y="40" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">users · keys · rules</text>
+    <text x="72" y="54" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a" text-anchor="middle">local · audit-logged</text>
+  </g>
+
+  <!-- ═══ FOOTER NOTES ══════════════════════════════════ -->
+  <g transform="translate(16, 558)">
+    <rect width="868" height="48" rx="6" fill="#0a0a0a" stroke="#27272a" stroke-width="0.8"/>
+    <text x="24" y="20" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">
+      <tspan fill="#F87171" font-weight="600">Queue groups</tspan> load-balance ops across daemon replicas.
+      <tspan fill="#F87171" font-weight="600"> Fan-out</tspan> (no queue group) dispatches describe calls to every node.
+    </text>
+    <text x="24" y="38" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">
+      Daemons are <tspan fill="#F87171" font-weight="600">stateless</tspan> — scale horizontally by starting more.
+      No etcd, no Kubernetes, no external control plane. Systemd units, a NATS cluster, and your hardware.
+    </text>
+  </g>
+</svg>
@@ -0,0 +1,154 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="300" viewBox="0 0 900 300">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a;stop-opacity:1" />
+      <stop offset="50%" style="stop-color:#0d1117;stop-opacity:1" />
+      <stop offset="100%" style="stop-color:#0a0a0a;stop-opacity:1" />
+    </linearGradient>
+    <radialGradient id="vignette" cx="50%" cy="50%" r="70%">
+      <stop offset="0%" style="stop-color:#7F1D1D;stop-opacity:0.08"/>
+      <stop offset="100%" style="stop-color:#7F1D1D;stop-opacity:0"/>
+    </radialGradient>
+    <linearGradient id="mulga-grad" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#DC2626;stop-opacity:0.95" />
+      <stop offset="100%" style="stop-color:#7F1D1D;stop-opacity:0.95" />
+    </linearGradient>
+    <linearGradient id="rule" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#DC2626;stop-opacity:0" />
+      <stop offset="50%" style="stop-color:#F87171;stop-opacity:0.45" />
+      <stop offset="100%" style="stop-color:#7F1D1D;stop-opacity:0" />
+    </linearGradient>
+    <filter id="node-glow">
+      <feGaussianBlur in="SourceGraphic" stdDeviation="3" />
+    </filter>
+  </defs>
+
+  <!-- Background -->
+  <rect width="900" height="300" fill="url(#bg)" rx="12"/>
+  <rect width="900" height="300" fill="url(#vignette)" rx="12"/>
+
+  <!-- Mesh network background (mulgadc style) -->
+  <g opacity="0.35">
+    <!-- Connections (drawn first, under nodes) -->
+    <g stroke="#7F1D1D" stroke-width="0.6" fill="none">
+      <line x1="80" y1="60" x2="200" y2="40"/>
+      <line x1="80" y1="60" x2="150" y2="140"/>
+      <line x1="200" y1="40" x2="330" y2="90"/>
+      <line x1="150" y1="140" x2="260" y2="180"/>
+      <line x1="150" y1="140" x2="60" y2="220"/>
+      <line x1="330" y1="90" x2="260" y2="180"/>
+      <line x1="260" y1="180" x2="120" y2="260"/>
+      <line x1="60" y1="220" x2="120" y2="260"/>
+
+      <line x1="620" y1="50" x2="760" y2="90"/>
+      <line x1="620" y1="50" x2="700" y2="160"/>
+      <line x1="760" y1="90" x2="850" y2="160"/>
+      <line x1="700" y1="160" x2="820" y2="220"/>
+      <line x1="850" y1="160" x2="820" y2="220"/>
+      <line x1="700" y1="160" x2="600" y2="240"/>
+      <line x1="600" y1="240" x2="820" y2="220"/>
+      <line x1="760" y1="90" x2="700" y2="160"/>
+    </g>
+    <!-- Nodes -->
+    <g fill="#DC2626">
+      <circle cx="80" cy="60" r="2.5"/>
+      <circle cx="200" cy="40" r="2"/>
+      <circle cx="330" cy="90" r="2.5"/>
+      <circle cx="150" cy="140" r="3"/>
+      <circle cx="260" cy="180" r="2"/>
+      <circle cx="60" cy="220" r="2"/>
+      <circle cx="120" cy="260" r="2.5"/>
+
+      <circle cx="620" cy="50" r="2"/>
+      <circle cx="760" cy="90" r="3"/>
+      <circle cx="700" cy="160" r="2.5"/>
+      <circle cx="850" cy="160" r="2"/>
+      <circle cx="820" cy="220" r="2.5"/>
+      <circle cx="600" cy="240" r="2"/>
+    </g>
+    <!-- Pulse rings (static "about-to-pulse" look) -->
+    <g fill="none" stroke="#F87171" stroke-width="0.8" opacity="0.5">
+      <circle cx="150" cy="140" r="6"/>
+      <circle cx="760" cy="90" r="6"/>
+      <circle cx="700" cy="160" r="8"/>
+    </g>
+  </g>
+
+  <!-- Main emblem: cluster-node stack with NATS bus -->
+  <g transform="translate(200, 150)">
+    <!-- Outer glow ring -->
+    <circle cx="0" cy="0" r="64" fill="none" stroke="url(#mulga-grad)" stroke-width="1.5" opacity="0.35" filter="url(#node-glow)"/>
+
+    <!-- Top node -->
+    <rect x="-24" y="-52" width="48" height="22" rx="3" fill="none" stroke="#DC2626" stroke-width="1.8"/>
+    <circle cx="-16" cy="-41" r="1.6" fill="#F87171"/>
+    <circle cx="-8" cy="-41" r="1.6" fill="#F87171"/>
+    <line x1="0" y1="-41" x2="16" y2="-41" stroke="#F87171" stroke-width="0.9" opacity="0.6"/>
+
+    <!-- Middle node (active) -->
+    <rect x="-30" y="-14" width="60" height="28" rx="3" fill="url(#mulga-grad)" opacity="0.22" stroke="#DC2626" stroke-width="2"/>
+    <circle cx="-20" cy="0" r="2" fill="#F87171"/>
+    <circle cx="-10" cy="0" r="2" fill="#F87171"/>
+    <line x1="-2" y1="0" x2="22" y2="0" stroke="#F87171" stroke-width="1.2" opacity="0.8"/>
+
+    <!-- Bottom node -->
+    <rect x="-24" y="30" width="48" height="22" rx="3" fill="none" stroke="#7F1D1D" stroke-width="1.8"/>
+    <circle cx="-16" cy="41" r="1.6" fill="#DC2626"/>
+    <circle cx="-8" cy="41" r="1.6" fill="#DC2626"/>
+    <line x1="0" y1="41" x2="16" y2="41" stroke="#DC2626" stroke-width="0.9" opacity="0.6"/>
+
+    <!-- Radiating mesh ports -->
+    <g stroke-width="0.9" opacity="0.55">
+      <line x1="-30" y1="0" x2="-58" y2="-26" stroke="#DC2626"/>
+      <line x1="-30" y1="0" x2="-64" y2="0" stroke="#DC2626"/>
+      <line x1="-30" y1="0" x2="-58" y2="26" stroke="#DC2626"/>
+      <line x1="30" y1="0" x2="58" y2="-26" stroke="#7F1D1D"/>
+      <line x1="30" y1="0" x2="64" y2="0" stroke="#7F1D1D"/>
+      <line x1="30" y1="0" x2="58" y2="26" stroke="#7F1D1D"/>
+    </g>
+    <g fill="#F87171">
+      <circle cx="-58" cy="-26" r="2.5"/>
+      <circle cx="-64" cy="0" r="2.5"/>
+      <circle cx="-58" cy="26" r="2.5"/>
+    </g>
+    <g fill="#DC2626">
+      <circle cx="58" cy="-26" r="2.5"/>
+      <circle cx="64" cy="0" r="2.5"/>
+      <circle cx="58" cy="26" r="2.5"/>
+    </g>
+  </g>
+
+  <!-- Title -->
+  <text x="340" y="120" font-family="'Segoe UI', system-ui, -apple-system, sans-serif" font-size="56" font-weight="700" fill="#f5f5f5" letter-spacing="-1.5">Spinifex</text>
+
+  <!-- Subtitle -->
+  <text x="340" y="155" font-family="'Segoe UI', system-ui, -apple-system, sans-serif" font-size="17" fill="#a1a1aa" letter-spacing="0.3">Open-source AWS-compatible platform</text>
+  <text x="340" y="180" font-family="'Segoe UI', system-ui, -apple-system, sans-serif" font-size="17" fill="#a1a1aa" letter-spacing="0.3">for secure edge deployments</text>
+
+  <!-- Rule -->
+  <rect x="340" y="200" width="440" height="1" fill="url(#rule)" rx="1"/>
+
+  <!-- Service pills -->
+  <g font-family="'Segoe UI', system-ui, -apple-system, sans-serif" font-size="11" font-weight="500">
+    <rect x="340" y="215" width="50" height="24" rx="12" fill="#7F1D1D" fill-opacity="0.25" stroke="#DC2626" stroke-width="0.5" stroke-opacity="0.55"/>
+    <text x="365" y="231" fill="#F87171" text-anchor="middle">EC2</text>
+
+    <rect x="398" y="215" width="50" height="24" rx="12" fill="#7F1D1D" fill-opacity="0.25" stroke="#DC2626" stroke-width="0.5" stroke-opacity="0.55"/>
+    <text x="423" y="231" fill="#F87171" text-anchor="middle">EBS</text>
+
+    <rect x="456" y="215" width="45" height="24" rx="12" fill="#7F1D1D" fill-opacity="0.25" stroke="#DC2626" stroke-width="0.5" stroke-opacity="0.55"/>
+    <text x="478" y="231" fill="#F87171" text-anchor="middle">S3</text>
+
+    <rect x="509" y="215" width="55" height="24" rx="12" fill="#7F1D1D" fill-opacity="0.25" stroke="#DC2626" stroke-width="0.5" stroke-opacity="0.55"/>
+    <text x="536" y="231" fill="#F87171" text-anchor="middle">VPC</text>
+
+    <rect x="572" y="215" width="50" height="24" rx="12" fill="#7F1D1D" fill-opacity="0.25" stroke="#DC2626" stroke-width="0.5" stroke-opacity="0.55"/>
+    <text x="597" y="231" fill="#F87171" text-anchor="middle">IAM</text>
+
+    <rect x="630" y="215" width="78" height="24" rx="12" fill="#18181b" stroke="#52525b" stroke-width="0.5" stroke-opacity="0.8"/>
+    <text x="669" y="231" fill="#a1a1aa" text-anchor="middle">Air-Gapped</text>
+  </g>
+
+  <!-- Tagline -->
+  <text x="340" y="265" font-family="'Segoe UI', system-ui, -apple-system, sans-serif" font-size="11" fill="#71717a" letter-spacing="1.8">YOUR INFRASTRUCTURE · YOUR HARDWARE · YOUR CLOUD</text>
+</svg>
@@ -0,0 +1,175 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="720" viewBox="0 0 900 720">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="bar-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#161b22"/>
+      <stop offset="45%" style="stop-color:#1f2028"/>
+      <stop offset="100%" style="stop-color:#161b22"/>
+    </linearGradient>
+    <linearGradient id="client-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="gw-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#DC262633"/>
+      <stop offset="100%" style="stop-color:#7F1D1D14"/>
+    </linearGradient>
+    <linearGradient id="nats-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#F87171"/>
+      <stop offset="50%" style="stop-color:#DC2626"/>
+      <stop offset="100%" style="stop-color:#7F1D1D"/>
+    </linearGradient>
+    <linearGradient id="daemon-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#DC262622"/>
+      <stop offset="100%" style="stop-color:#7F1D1D0a"/>
+    </linearGradient>
+  </defs>
+
+  <rect width="900" height="720" fill="url(#bg)" rx="12"/>
+
+  <!-- Title -->
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">HIGH-LEVEL ARCHITECTURE</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">SDK → Gateway → NATS → Daemons → VMs</text>
+
+  <!-- ═══ AWS SDK / CLI ══════════════════════════════════ -->
+  <rect x="16" y="50" width="868" height="76" rx="8" fill="url(#client-grad)" stroke="#3fb950" stroke-width="1" stroke-opacity="0.5"/>
+  <text x="32" y="72" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700" letter-spacing="1">AWS SDK / CLI</text>
+  <text x="32" y="90" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">Users interact with Spinifex using standard AWS SDKs and CLIs against a custom endpoint URL</text>
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10.5">
+    <rect x="32" y="98" width="480" height="20" rx="4" fill="#0a0a0a" stroke="#3fb950" stroke-width="0.6" stroke-opacity="0.4"/>
+    <text x="42" y="112" fill="#7ee787">$ aws ec2 run-instances --endpoint-url https://localhost:9999 ...</text>
+  </g>
+
+  <!-- Connector: HTTPS + SigV4 -->
+  <line x1="450" y1="126" x2="450" y2="154" stroke="#DC2626" stroke-width="1.2" stroke-dasharray="3,3" opacity="0.7"/>
+  <rect x="380" y="128" width="140" height="20" rx="10" fill="#0a0a0a" stroke="#DC2626" stroke-width="0.8" stroke-opacity="0.5"/>
+  <text x="450" y="142" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#F87171" text-anchor="middle">HTTPS · SigV4 Auth</text>
+
+  <!-- ═══ AWS GATEWAY ════════════════════════════════════ -->
+  <rect x="16" y="154" width="868" height="104" rx="8" fill="url(#gw-grad)" stroke="#DC2626" stroke-width="1.3" stroke-opacity="0.75"/>
+  <text x="32" y="176" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" font-weight="700" letter-spacing="1">AWS GATEWAY · port 9999</text>
+  <text x="876" y="176" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#71717a" text-anchor="end">spinifex/services/awsgw/awsgw.go</text>
+
+  <!-- Two-column feature bullets -->
+  <g font-family="'Segoe UI', sans-serif" font-size="10.5" fill="#f5f5f5">
+    <text x="40" y="200">● TLS termination</text>
+    <text x="40" y="220">● EC2 Query Protocol parsing</text>
+    <text x="40" y="240">● XML response generation</text>
+
+    <text x="460" y="200">● AWS SigV4 authentication</text>
+    <text x="460" y="220">● Routes to service handlers</text>
+    <text x="460" y="240">● NATS client connection</text>
+  </g>
+
+  <!-- Connector: JSON over NATS -->
+  <line x1="450" y1="258" x2="450" y2="286" stroke="#DC2626" stroke-width="1.2" stroke-dasharray="3,3" opacity="0.7"/>
+  <rect x="380" y="260" width="140" height="20" rx="10" fill="#0a0a0a" stroke="#DC2626" stroke-width="0.8" stroke-opacity="0.5"/>
+  <text x="450" y="274" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#F87171" text-anchor="middle">JSON over NATS</text>
+
+  <!-- ═══ NATS BROKER ═══════════════════════════════════ -->
+  <rect x="16" y="286" width="868" height="124" rx="8" fill="#1a0a0a" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.65"/>
+  <rect x="16" y="286" width="868" height="28" rx="8" fill="url(#nats-grad)" opacity="0.9"/>
+  <text x="32" y="305" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#f5f5f5" font-weight="700" letter-spacing="1.5">NATS MESSAGE BROKER</text>
+  <text x="876" y="305" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#fee2e2" text-anchor="end" opacity="0.9">request/reply · queue groups · fan-out</text>
+
+  <!-- Two-column topics/queue groups -->
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10">
+    <text x="40" y="332" fill="#F87171" font-weight="600">Topics</text>
+    <text x="40" y="350" fill="#f5f5f5">ec2.RunInstances</text>
+    <text x="40" y="368" fill="#f5f5f5">ec2.DescribeInstances</text>
+    <text x="40" y="386" fill="#f5f5f5">ec2.cmd.{instanceID}</text>
+    <text x="40" y="404" fill="#f5f5f5">ebs.mount  ·  ebs.unmount</text>
+
+    <text x="460" y="332" fill="#F87171" font-weight="600">Delivery mode</text>
+    <text x="460" y="350" fill="#a1a1aa">spinifex-workers <tspan fill="#71717a">(load-balanced)</tspan></text>
+    <text x="460" y="368" fill="#a1a1aa">no queue group <tspan fill="#71717a">(fan-out to all nodes)</tspan></text>
+    <text x="460" y="386" fill="#a1a1aa">per-instance <tspan fill="#71717a">(start / stop / terminate)</tspan></text>
+    <text x="460" y="404" fill="#a1a1aa">Viperblock request/reply</text>
+  </g>
+
+  <!-- Fan-out connectors to 3 daemons -->
+  <g stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6" fill="none">
+    <line x1="450" y1="410" x2="450" y2="430"/>
+    <line x1="450" y1="430" x2="155" y2="452"/>
+    <line x1="450" y1="430" x2="450" y2="452"/>
+    <line x1="450" y1="430" x2="745" y2="452"/>
+  </g>
+
+  <!-- ═══ DAEMON ROW ════════════════════════════════════ -->
+  <!-- Daemon 1 -->
+  <g transform="translate(32, 452)">
+    <rect width="246" height="124" rx="8" fill="url(#daemon-grad)" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" font-weight="700">Spinifex Daemon</text>
+    <text x="226" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="end">Node 1</text>
+    <line x1="16" y1="32" x2="230" y2="32" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <g font-family="'Segoe UI', sans-serif" font-size="10.5" fill="#f5f5f5">
+      <text x="20" y="52">● NATS subscriber</text>
+      <text x="20" y="70">● Resource manager</text>
+      <text x="20" y="88">● QEMU/KVM launcher</text>
+      <text x="20" y="106">● QMP monitoring</text>
+    </g>
+    <text x="20" y="119" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#71717a">daemon.go</text>
+  </g>
+
+  <!-- Daemon 2 -->
+  <g transform="translate(327, 452)">
+    <rect width="246" height="124" rx="8" fill="url(#daemon-grad)" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" font-weight="700">Spinifex Daemon</text>
+    <text x="226" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="end">Node 2</text>
+    <line x1="16" y1="32" x2="230" y2="32" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <g font-family="'Segoe UI', sans-serif" font-size="10.5" fill="#f5f5f5">
+      <text x="20" y="52">● NATS subscriber</text>
+      <text x="20" y="70">● Resource manager</text>
+      <text x="20" y="88">● QEMU/KVM launcher</text>
+      <text x="20" y="106">● QMP monitoring</text>
+    </g>
+    <text x="20" y="119" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#71717a">daemon.go</text>
+  </g>
+
+  <!-- Daemon N -->
+  <g transform="translate(622, 452)">
+    <rect width="246" height="124" rx="8" fill="url(#daemon-grad)" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.7"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" font-weight="700">Spinifex Daemon</text>
+    <text x="226" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="end">Node N</text>
+    <line x1="16" y1="32" x2="230" y2="32" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+    <g font-family="'Segoe UI', sans-serif" font-size="10.5" fill="#f5f5f5">
+      <text x="20" y="52">● NATS subscriber</text>
+      <text x="20" y="70">● Resource manager</text>
+      <text x="20" y="88">● QEMU/KVM launcher</text>
+      <text x="20" y="106">● QMP monitoring</text>
+    </g>
+    <text x="20" y="119" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#71717a">daemon.go</text>
+  </g>
+
+  <!-- Daemon → VM connectors -->
+  <g stroke="#71717a" stroke-width="0.8" stroke-dasharray="2,3" opacity="0.55" fill="none">
+    <line x1="155" y1="576" x2="155" y2="610"/>
+    <line x1="450" y1="576" x2="450" y2="610"/>
+    <line x1="745" y1="576" x2="745" y2="610"/>
+  </g>
+
+  <!-- ═══ VM BAR + ROW ═══════════════════════════════════ -->
+  <rect x="16" y="610" width="868" height="22" rx="5" fill="url(#bar-grad)"/>
+  <text x="450" y="625" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle" letter-spacing="1.5">WORKLOADS · QEMU/KVM virtual machines per node</text>
+
+  <!-- VM boxes -->
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+    <rect x="32" y="642" width="246" height="60" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="155" y="665" fill="#e4e4e7" text-anchor="middle" font-weight="600">QEMU / KVM VMs</text>
+    <text x="155" y="683" fill="#a1a1aa" text-anchor="middle" font-size="9.5">virtio-blk · virtio-net · cloud-init</text>
+    <text x="155" y="695" fill="#71717a" text-anchor="middle" font-size="9">Node 1 workloads</text>
+
+    <rect x="327" y="642" width="246" height="60" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="450" y="665" fill="#e4e4e7" text-anchor="middle" font-weight="600">QEMU / KVM VMs</text>
+    <text x="450" y="683" fill="#a1a1aa" text-anchor="middle" font-size="9.5">virtio-blk · virtio-net · cloud-init</text>
+    <text x="450" y="695" fill="#71717a" text-anchor="middle" font-size="9">Node 2 workloads</text>
+
+    <rect x="622" y="642" width="246" height="60" rx="7" fill="#27272a" fill-opacity="0.7" stroke="#71717a" stroke-width="1" stroke-opacity="0.6"/>
+    <text x="745" y="665" fill="#e4e4e7" text-anchor="middle" font-weight="600">QEMU / KVM VMs</text>
+    <text x="745" y="683" fill="#a1a1aa" text-anchor="middle" font-size="9.5">virtio-blk · virtio-net · cloud-init</text>
+    <text x="745" y="695" fill="#71717a" text-anchor="middle" font-size="9">Node N workloads</text>
+  </g>
+</svg>
@@ -0,0 +1,46 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="180" viewBox="0 0 900 180">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="180" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="28" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">BASTION · only entry point into a private subnet</text>
+
+  <!-- WAN -->
+  <g transform="translate(40, 56)">
+    <rect width="200" height="60" rx="8" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="100" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">WAN</text>
+    <text x="100" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">operator / laptop</text>
+  </g>
+
+  <!-- Arrow: SSH -->
+  <line x1="240" y1="86" x2="320" y2="86" stroke="#58a6ff" stroke-width="1.4" stroke-dasharray="4,3" opacity="0.75"/>
+  <polygon points="316,82 328,86 316,90" fill="#58a6ff" opacity="0.8"/>
+  <rect x="244" y="62" width="80" height="18" rx="9" fill="#0a0a0a" stroke="#58a6ff" stroke-width="0.6" stroke-opacity="0.55"/>
+  <text x="284" y="75" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#8ab4f8" text-anchor="middle">SSH</text>
+
+  <!-- Bastion -->
+  <g transform="translate(338, 56)">
+    <rect width="220" height="60" rx="8" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.3" stroke-opacity="0.6"/>
+    <text x="110" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">Bastion</text>
+    <text x="110" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#7ee787" text-anchor="middle">public subnet</text>
+  </g>
+
+  <!-- Arrow: SSH (private IP) -->
+  <line x1="558" y1="86" x2="638" y2="86" stroke="#a855f7" stroke-width="1.4" stroke-dasharray="4,3" opacity="0.75"/>
+  <polygon points="634,82 646,86 634,90" fill="#a855f7" opacity="0.8"/>
+  <rect x="552" y="62" width="140" height="18" rx="9" fill="#0a0a0a" stroke="#a855f7" stroke-width="0.6" stroke-opacity="0.55"/>
+  <text x="622" y="75" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#c084fc" text-anchor="middle">SSH (private IP)</text>
+
+  <!-- App server -->
+  <g transform="translate(656, 56)">
+    <rect width="220" height="60" rx="8" fill="#a855f71a" stroke="#a855f7" stroke-width="1.3" stroke-opacity="0.6"/>
+    <text x="110" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#c084fc" text-anchor="middle" font-weight="700">App Server</text>
+    <text x="110" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#d8b4fe" text-anchor="middle">private subnet · no internet</text>
+  </g>
+
+  <text x="450" y="156" font-family="'Segoe UI', sans-serif" font-size="10" fill="#71717a" text-anchor="middle" font-style="italic">App server has no route to the internet — all access hops through the bastion.</text>
+</svg>
@@ -0,0 +1,98 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="480" viewBox="0 0 900 480">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="inet-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="pub-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#3fb9501f"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="priv-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#a855f71f"/>
+      <stop offset="100%" style="stop-color:#a855f708"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="480" fill="url(#bg)" rx="12"/>
+
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">NGINX + ALB · VPC WITH PUBLIC AND PRIVATE SUBNETS</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">10.20.0.0/16 · multi-AZ</text>
+
+  <!-- Internet -->
+  <rect x="350" y="54" width="200" height="40" rx="8" fill="url(#inet-grad)" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.55"/>
+  <text x="450" y="78" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">Internet</text>
+
+  <!-- connector to IGW -->
+  <line x1="450" y1="94" x2="450" y2="116" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <polygon points="446,112 454,112 450,120" fill="#71717a" opacity="0.7"/>
+
+  <!-- IGW -->
+  <rect x="380" y="122" width="140" height="44" rx="7" fill="#DC26261a" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.65"/>
+  <text x="450" y="142" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">IGW</text>
+  <text x="450" y="158" font-family="'Segoe UI', sans-serif" font-size="10" fill="#FCA5A5" text-anchor="middle">nginx-alb-igw</text>
+
+  <!-- Fan-out to two public subnets -->
+  <line x1="450" y1="166" x2="450" y2="184" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="230" y1="184" x2="670" y2="184" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="230" y1="184" x2="230" y2="202" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="670" y1="184" x2="670" y2="202" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+
+  <!-- public_a -->
+  <g transform="translate(108, 202)">
+    <rect width="244" height="104" rx="8" fill="url(#pub-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700">public_a</text>
+    <text x="224" y="24" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#7ee787" text-anchor="end">az-a</text>
+    <line x1="16" y1="32" x2="228" y2="32" stroke="#3fb950" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <rect x="20" y="42" width="200" height="22" rx="4" fill="#3fb9501a" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="120" y="57" font-family="'SF Mono', ui-monospace, monospace" font-size="10.5" fill="#7ee787" text-anchor="middle" font-weight="600">ALB ENI</text>
+
+    <rect x="20" y="70" width="200" height="22" rx="4" fill="#3fb9501a" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="120" y="85" font-family="'SF Mono', ui-monospace, monospace" font-size="10.5" fill="#7ee787" text-anchor="middle" font-weight="600">NAT GW</text>
+  </g>
+
+  <!-- public_b -->
+  <g transform="translate(548, 202)">
+    <rect width="244" height="104" rx="8" fill="url(#pub-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700">public_b</text>
+    <text x="224" y="24" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#7ee787" text-anchor="end">az-b</text>
+    <line x1="16" y1="32" x2="228" y2="32" stroke="#3fb950" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <rect x="20" y="56" width="200" height="22" rx="4" fill="#3fb9501a" stroke="#3fb950" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="120" y="71" font-family="'SF Mono', ui-monospace, monospace" font-size="10.5" fill="#7ee787" text-anchor="middle" font-weight="600">ALB ENI</text>
+  </g>
+
+  <!-- SNAT arrow from public_a NAT → private row -->
+  <line x1="230" y1="306" x2="230" y2="340" stroke="#F87171" stroke-width="1.2" stroke-dasharray="3,3" opacity="0.65"/>
+  <polygon points="226,336 234,336 230,344" fill="#F87171" opacity="0.8"/>
+  <rect x="256" y="314" width="240" height="20" rx="10" fill="#0a0a0a" stroke="#F87171" stroke-width="0.6" stroke-opacity="0.55"/>
+  <text x="376" y="328" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#FCA5A5" text-anchor="middle">SNAT for private subnets (egress)</text>
+
+  <!-- private_a -->
+  <g transform="translate(108, 350)">
+    <rect width="244" height="104" rx="8" fill="url(#priv-grad)" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#c084fc" font-weight="700">private_a</text>
+    <text x="224" y="24" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#d8b4fe" text-anchor="end">az-a</text>
+    <line x1="16" y1="32" x2="228" y2="32" stroke="#a855f7" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <rect x="20" y="44" width="200" height="48" rx="5" fill="#a855f71a" stroke="#a855f7" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="120" y="64" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#d8b4fe" text-anchor="middle" font-weight="700">nginx_1</text>
+    <text x="120" y="80" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">Debian 12 · cloud-init</text>
+  </g>
+
+  <!-- private_b -->
+  <g transform="translate(548, 350)">
+    <rect width="244" height="104" rx="8" fill="url(#priv-grad)" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#c084fc" font-weight="700">private_b</text>
+    <text x="224" y="24" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#d8b4fe" text-anchor="end">az-b</text>
+    <line x1="16" y1="32" x2="228" y2="32" stroke="#a855f7" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <rect x="20" y="44" width="200" height="48" rx="5" fill="#a855f71a" stroke="#a855f7" stroke-opacity="0.5" stroke-width="0.6"/>
+    <text x="120" y="64" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#d8b4fe" text-anchor="middle" font-weight="700">nginx_2</text>
+    <text x="120" y="80" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">Debian 12 · cloud-init</text>
+  </g>
+</svg>
@@ -0,0 +1,44 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="180" viewBox="0 0 900 180">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="180" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="28" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">S3 WEBAPP · Flask on EC2 with Predastore backend</text>
+
+  <!-- Browser -->
+  <g transform="translate(40, 56)">
+    <rect width="200" height="60" rx="8" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="100" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">Browser</text>
+    <text x="100" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">end user</text>
+  </g>
+
+  <!-- Arrow: HTTP -->
+  <line x1="240" y1="86" x2="320" y2="86" stroke="#58a6ff" stroke-width="1.4" stroke-dasharray="4,3" opacity="0.75"/>
+  <polygon points="316,82 328,86 316,90" fill="#58a6ff" opacity="0.8"/>
+  <rect x="244" y="62" width="80" height="18" rx="9" fill="#0a0a0a" stroke="#58a6ff" stroke-width="0.6" stroke-opacity="0.55"/>
+  <text x="284" y="75" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#8ab4f8" text-anchor="middle">HTTP</text>
+
+  <!-- EC2 Instance -->
+  <g transform="translate(338, 56)">
+    <rect width="240" height="60" rx="8" fill="#DC26261a" stroke="#DC2626" stroke-width="1.3" stroke-opacity="0.65"/>
+    <text x="120" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" text-anchor="middle" font-weight="700">EC2 Instance</text>
+    <text x="120" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#FCA5A5" text-anchor="middle">Flask webapp · port 80</text>
+  </g>
+
+  <!-- Arrow: S3 API (boto3) -->
+  <line x1="578" y1="86" x2="658" y2="86" stroke="#a855f7" stroke-width="1.4" stroke-dasharray="4,3" opacity="0.75"/>
+  <polygon points="654,82 666,86 654,90" fill="#a855f7" opacity="0.8"/>
+  <rect x="570" y="62" width="144" height="18" rx="9" fill="#0a0a0a" stroke="#a855f7" stroke-width="0.6" stroke-opacity="0.55"/>
+  <text x="642" y="75" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#c084fc" text-anchor="middle">S3 API (boto3)</text>
+
+  <!-- Predastore -->
+  <g transform="translate(676, 56)">
+    <rect width="200" height="60" rx="8" fill="#a855f71a" stroke="#a855f7" stroke-width="1.3" stroke-opacity="0.6"/>
+    <text x="100" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#c084fc" text-anchor="middle" font-weight="700">Predastore</text>
+    <text x="100" y="46" font-family="'Segoe UI', sans-serif" font-size="10" fill="#d8b4fe" text-anchor="middle">S3 store · port 8443</text>
+  </g>
+</svg>
@@ -0,0 +1,78 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="330" viewBox="0 0 900 330">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="cluster-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="pool-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#DC262622"/>
+      <stop offset="100%" style="stop-color:#7F1D1D08"/>
+    </linearGradient>
+    <linearGradient id="node-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#58a6ff22"/>
+      <stop offset="100%" style="stop-color:#58a6ff08"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="330" fill="url(#bg)" rx="12"/>
+
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">CONFIGURATION LEVELS · spinifex.toml</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">three scopes: cluster · pool · node</text>
+
+  <!-- File badge -->
+  <rect x="24" y="50" width="140" height="24" rx="6" fill="#161b22" stroke="#52525b" stroke-width="0.6"/>
+  <text x="94" y="67" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#e4e4e7" text-anchor="middle">spinifex.toml</text>
+
+  <!-- spine -->
+  <line x1="52" y1="74" x2="52" y2="298" stroke="#52525b" stroke-width="1" opacity="0.6"/>
+
+  <!-- ═══ Level 1: [network] cluster-wide ═══════════════════ -->
+  <line x1="52" y1="108" x2="92" y2="108" stroke="#52525b" stroke-width="1" opacity="0.6"/>
+  <rect x="92" y="86" width="764" height="56" rx="8" fill="url(#cluster-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+  <text x="108" y="108" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700">[network]</text>
+  <text x="220" y="108" font-family="'Segoe UI', sans-serif" font-size="11" fill="#f5f5f5">Cluster-wide · which mode?</text>
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10">
+    <text x="108" y="128" fill="#a1a1aa">└─</text>
+    <text x="128" y="128" fill="#7ee787">external_mode</text>
+    <text x="224" y="128" fill="#a1a1aa">=</text>
+    <text x="238" y="128" fill="#f5f5f5">"pool"</text>
+  </g>
+
+  <!-- ═══ Level 2: [[network.external_pools]] ═══════════════ -->
+  <line x1="52" y1="176" x2="92" y2="176" stroke="#52525b" stroke-width="1" opacity="0.6"/>
+  <rect x="92" y="154" width="764" height="78" rx="8" fill="url(#pool-grad)" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.65"/>
+  <text x="108" y="176" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" font-weight="700">[[network.external_pools]]</text>
+  <text x="328" y="176" font-family="'Segoe UI', sans-serif" font-size="11" fill="#f5f5f5">IP ranges · what IPs to use?</text>
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10">
+    <text x="108" y="196" fill="#a1a1aa">├─</text>
+    <text x="128" y="196" fill="#FCA5A5">name</text>
+    <text x="158" y="196" fill="#71717a">·</text>
+    <text x="166" y="196" fill="#FCA5A5">range_start</text>
+    <text x="246" y="196" fill="#71717a">·</text>
+    <text x="254" y="196" fill="#FCA5A5">range_end</text>
+
+    <text x="108" y="212" fill="#a1a1aa">├─</text>
+    <text x="128" y="212" fill="#FCA5A5">gateway</text>
+    <text x="182" y="212" fill="#71717a">·</text>
+    <text x="190" y="212" fill="#FCA5A5">prefix_len</text>
+
+    <text x="108" y="228" fill="#a1a1aa">└─</text>
+    <text x="128" y="228" fill="#FCA5A5">region</text>
+    <text x="174" y="228" fill="#71717a">·</text>
+    <text x="182" y="228" fill="#FCA5A5">az</text>
+    <text x="202" y="228" fill="#71717a">(optional)</text>
+  </g>
+
+  <!-- ═══ Level 3: [nodes.NAME.vpcd] ════════════════════════ -->
+  <line x1="52" y1="268" x2="92" y2="268" stroke="#52525b" stroke-width="1" opacity="0.6"/>
+  <rect x="92" y="246" width="764" height="56" rx="8" fill="url(#node-grad)" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6"/>
+  <text x="108" y="268" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#58a6ff" font-weight="700">[nodes.NAME.vpcd]</text>
+  <text x="266" y="268" font-family="'Segoe UI', sans-serif" font-size="11" fill="#f5f5f5">Per-node · which NIC?</text>
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10">
+    <text x="108" y="288" fill="#a1a1aa">└─</text>
+    <text x="128" y="288" fill="#8ab4f8">external_interface</text>
+  </g>
+</svg>
@@ -0,0 +1,63 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="110" viewBox="0 0 900 110">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="110" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="22" font-family="'Segoe UI', system-ui, sans-serif" font-size="10" fill="#a1a1aa" letter-spacing="2" font-weight="600">DATA PATH · VM to WAN via OVN localnet</text>
+
+  <!-- VM TAP -->
+  <g transform="translate(24, 40)">
+    <rect width="118" height="48" rx="7" fill="#a855f71a" stroke="#a855f7" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="59" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#c084fc" text-anchor="middle" font-weight="700">VM TAP</text>
+    <text x="59" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">guest interface</text>
+  </g>
+  <polygon points="148,60 160,64 148,68" fill="#71717a" opacity="0.7"/>
+  <line x1="142" y1="64" x2="148" y2="64" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- br-int -->
+  <g transform="translate(166, 40)">
+    <rect width="102" height="48" rx="7" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="51" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#58a6ff" text-anchor="middle" font-weight="700">br-int</text>
+    <text x="51" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">OVS bridge</text>
+  </g>
+  <polygon points="274,60 286,64 274,68" fill="#71717a" opacity="0.7"/>
+  <line x1="268" y1="64" x2="274" y2="64" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- OVN logical pipeline -->
+  <g transform="translate(292, 40)">
+    <rect width="192" height="48" rx="7" fill="#DC26261a" stroke="#DC2626" stroke-width="1.1" stroke-opacity="0.65"/>
+    <text x="96" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" text-anchor="middle" font-weight="700">OVN logical pipeline</text>
+    <text x="96" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#FCA5A5" text-anchor="middle">switch → router → SNAT/DNAT</text>
+  </g>
+  <polygon points="490,60 502,64 490,68" fill="#71717a" opacity="0.7"/>
+  <line x1="484" y1="64" x2="490" y2="64" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- localnet -->
+  <g transform="translate(508, 40)">
+    <rect width="120" height="48" rx="7" fill="#DC26260d" stroke="#DC2626" stroke-width="1" stroke-opacity="0.5" stroke-dasharray="4,3"/>
+    <text x="60" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" text-anchor="middle" font-weight="700">localnet</text>
+    <text x="60" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#FCA5A5" text-anchor="middle">OVN port type</text>
+  </g>
+  <polygon points="634,60 646,64 634,68" fill="#71717a" opacity="0.7"/>
+  <line x1="628" y1="64" x2="634" y2="64" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- br-wan -->
+  <g transform="translate(652, 40)">
+    <rect width="110" height="48" rx="7" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="55" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#58a6ff" text-anchor="middle" font-weight="700">br-wan</text>
+    <text x="55" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">OVS bridge</text>
+  </g>
+  <polygon points="768,60 780,64 768,68" fill="#71717a" opacity="0.7"/>
+  <line x1="762" y1="64" x2="768" y2="64" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- WAN -->
+  <g transform="translate(786, 40)">
+    <rect width="90" height="48" rx="7" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="45" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#3fb950" text-anchor="middle" font-weight="700">WAN</text>
+    <text x="45" y="38" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">uplink</text>
+  </g>
+</svg>
@@ -0,0 +1,74 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="360" viewBox="0 0 900 360">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="host-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#58a6ff22"/>
+      <stop offset="100%" style="stop-color:#58a6ff08"/>
+    </linearGradient>
+    <linearGradient id="vm-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="360" fill="url(#bg)" rx="12"/>
+
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">TWO DHCP CONVERSATIONS · completely independent</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">pool mode · static or dhcp source</text>
+
+  <!-- ═══ HOST ↔ ROUTER DHCP ══════════════════════════════ -->
+  <rect x="24" y="54" width="852" height="138" rx="10" fill="url(#host-grad)" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6"/>
+  <text x="44" y="78" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#58a6ff" font-weight="700" letter-spacing="0.8">HOST ↔ ROUTER DHCP</text>
+  <text x="876" y="78" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#8ab4f8" text-anchor="end">on br-ext · veth pair → br-wan</text>
+  <line x1="24" y1="88" x2="876" y2="88" stroke="#58a6ff" stroke-opacity="0.25" stroke-width="0.5"/>
+
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+    <!-- step 1 -->
+    <circle cx="54" cy="108" r="9" fill="#58a6ff" fill-opacity="0.15" stroke="#58a6ff" stroke-width="1"/>
+    <text x="54" y="112" fill="#58a6ff" text-anchor="middle" font-weight="700" font-size="10">1</text>
+    <text x="74" y="112" fill="#f5f5f5">Spinifex asks router:</text>
+    <text x="228" y="112" fill="#a1a1aa">"give me a public IP for this VM"</text>
+
+    <!-- step 2 -->
+    <circle cx="54" cy="132" r="9" fill="#58a6ff" fill-opacity="0.15" stroke="#58a6ff" stroke-width="1"/>
+    <text x="54" y="136" fill="#58a6ff" text-anchor="middle" font-weight="700" font-size="10">2</text>
+    <text x="74" y="136" fill="#f5f5f5">Router responds:</text>
+    <text x="196" y="136" fill="#8ab4f8" font-weight="600">192.168.1.75</text>
+
+    <!-- step 3 -->
+    <circle cx="54" cy="156" r="9" fill="#58a6ff" fill-opacity="0.15" stroke="#58a6ff" stroke-width="1"/>
+    <text x="54" y="160" fill="#58a6ff" text-anchor="middle" font-weight="700" font-size="10">3</text>
+    <text x="74" y="160" fill="#f5f5f5">Spinifex stores IP, tells OVN:</text>
+    <text x="274" y="160" fill="#F87171" font-weight="600">dnat_and_snat(192.168.1.75 ↔ 172.31.0.4)</text>
+  </g>
+
+  <text x="44" y="181" font-family="'Segoe UI', sans-serif" font-size="10" fill="#71717a" font-style="italic">Only used in DHCP source mode. Static mode picks from range.</text>
+
+  <!-- separator -->
+  <line x1="24" y1="206" x2="876" y2="206" stroke="#27272a" stroke-width="0.8"/>
+  <text x="450" y="220" font-family="'SF Mono', ui-monospace, monospace" font-size="9" fill="#52525b" text-anchor="middle" letter-spacing="2">INDEPENDENT · no packets cross between</text>
+
+  <!-- ═══ VM ↔ OVN DHCP ═══════════════════════════════════ -->
+  <rect x="24" y="234" width="852" height="110" rx="10" fill="url(#vm-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+  <text x="44" y="258" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700" letter-spacing="0.8">VM ↔ OVN DHCP</text>
+  <text x="876" y="258" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#7ee787" text-anchor="end">on tap → br-int → OVN logical switch</text>
+  <line x1="24" y1="268" x2="876" y2="268" stroke="#3fb950" stroke-opacity="0.25" stroke-width="0.5"/>
+
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+    <circle cx="54" cy="288" r="9" fill="#3fb950" fill-opacity="0.15" stroke="#3fb950" stroke-width="1"/>
+    <text x="54" y="292" fill="#3fb950" text-anchor="middle" font-weight="700" font-size="10">1</text>
+    <text x="74" y="292" fill="#f5f5f5">cloud-init asks:</text>
+    <text x="190" y="292" fill="#a1a1aa">"what's my IP?"</text>
+
+    <circle cx="54" cy="312" r="9" fill="#3fb950" fill-opacity="0.15" stroke="#3fb950" stroke-width="1"/>
+    <text x="54" y="316" fill="#3fb950" text-anchor="middle" font-weight="700" font-size="10">2</text>
+    <text x="74" y="316" fill="#f5f5f5">OVN DHCP responds:</text>
+    <text x="226" y="316" fill="#a1a1aa">"you are</text>
+    <text x="284" y="316" fill="#7ee787" font-weight="600">172.31.0.4</text>
+    <text x="356" y="316" fill="#a1a1aa">"</text>
+  </g>
+
+  <text x="44" y="336" font-family="'Segoe UI', sans-serif" font-size="10" fill="#71717a" font-style="italic">VM never sees 192.168.1.x or the router.</text>
+</svg>
@@ -0,0 +1,44 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="170" viewBox="0 0 900 170">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="170" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="24" font-family="'Segoe UI', system-ui, sans-serif" font-size="10" fill="#a1a1aa" letter-spacing="2" font-weight="600">DIRECT BRIDGE MODE · dedicated WAN NIC</text>
+
+  <!-- Row 1: Management NIC (independent, no WAN bridge path) -->
+  <g transform="translate(24, 42)">
+    <rect width="852" height="44" rx="7" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.55"/>
+    <text x="20" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" font-weight="700">Management NIC</text>
+    <text x="20" y="36" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa">host IP · SSH · spinifex services · overlay (Geneve)</text>
+    <text x="840" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#7ee787" text-anchor="end" font-weight="600">independent path</text>
+  </g>
+
+  <!-- Row 2: WAN NIC → br-wan → localnet -->
+  <g transform="translate(24, 100)">
+    <rect width="160" height="44" rx="7" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="80" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#58a6ff" text-anchor="middle" font-weight="700">WAN NIC</text>
+    <text x="80" y="36" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">dedicated physical</text>
+  </g>
+  <polygon points="198,118 210,122 198,126" fill="#71717a" opacity="0.7"/>
+  <line x1="186" y1="122" x2="198" y2="122" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <g transform="translate(214, 100)">
+    <rect width="220" height="44" rx="7" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="110" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#58a6ff" text-anchor="middle" font-weight="700">br-wan</text>
+    <text x="110" y="36" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#a1a1aa" text-anchor="middle">OVS port (NIC added directly)</text>
+  </g>
+  <polygon points="448,118 460,122 448,126" fill="#71717a" opacity="0.7"/>
+  <line x1="434" y1="122" x2="448" y2="122" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <g transform="translate(464, 100)">
+    <rect width="180" height="44" rx="7" fill="#DC26261a" stroke="#DC2626" stroke-width="1.1" stroke-opacity="0.65"/>
+    <text x="90" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" text-anchor="middle" font-weight="700">OVN localnet</text>
+    <text x="90" y="36" font-family="'Segoe UI', sans-serif" font-size="9.5" fill="#FCA5A5" text-anchor="middle">external switch port</text>
+  </g>
+
+  <text x="670" y="130" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#F87171" font-weight="600">distributed NAT · any MAC · all frames</text>
+</svg>
@@ -0,0 +1,91 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="440" viewBox="0 0 900 440">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="host-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#27272a88"/>
+      <stop offset="100%" style="stop-color:#27272a22"/>
+    </linearGradient>
+    <linearGradient id="br-int-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#a855f71f"/>
+      <stop offset="100%" style="stop-color:#a855f708"/>
+    </linearGradient>
+    <linearGradient id="br-wan-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#58a6ff1f"/>
+      <stop offset="100%" style="stop-color:#58a6ff08"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="440" fill="url(#bg)" rx="12"/>
+
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">NETWORK FLOW · bare-metal host</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">br-int overlay · br-wan WAN uplink</text>
+
+  <!-- ═══ Bare-Metal Host container ═══════════════════════ -->
+  <rect x="40" y="60" width="820" height="284" rx="12" fill="url(#host-grad)" stroke="#71717a" stroke-width="1.2" stroke-opacity="0.7"/>
+  <text x="58" y="84" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#e4e4e7" font-weight="700" letter-spacing="1">BARE-METAL HOST</text>
+  <line x1="40" y1="96" x2="860" y2="96" stroke="#52525b" stroke-opacity="0.5" stroke-width="0.5"/>
+
+  <!-- br-int bridge (left) -->
+  <g transform="translate(80, 122)">
+    <rect width="260" height="176" rx="10" fill="url(#br-int-grad)" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#c084fc" font-weight="700">br-int</text>
+    <text x="240" y="24" font-family="'Segoe UI', sans-serif" font-size="10" fill="#d8b4fe" text-anchor="end">(overlay)</text>
+    <line x1="20" y1="34" x2="240" y2="34" stroke="#a855f7" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+      <rect x="30" y="52" width="200" height="28" rx="5" fill="#a855f71a" stroke="#a855f7" stroke-opacity="0.4" stroke-width="0.6"/>
+      <text x="130" y="70" fill="#d8b4fe" text-anchor="middle">tap-eni1</text>
+
+      <rect x="30" y="88" width="200" height="28" rx="5" fill="#a855f71a" stroke="#a855f7" stroke-opacity="0.4" stroke-width="0.6"/>
+      <text x="130" y="106" fill="#d8b4fe" text-anchor="middle">tap-eni2</text>
+
+      <rect x="30" y="124" width="200" height="28" rx="5" fill="#a855f71a" stroke="#a855f7" stroke-opacity="0.4" stroke-width="0.6"/>
+      <text x="130" y="142" fill="#d8b4fe" text-anchor="middle">tap-eni3</text>
+    </g>
+  </g>
+
+  <!-- OVN NAT pipeline arrow in middle -->
+  <g transform="translate(352, 200)">
+    <line x1="0" y1="10" x2="148" y2="10" stroke="#DC2626" stroke-width="1.4" stroke-dasharray="4,3" opacity="0.75"/>
+    <polygon points="148,5 160,10 148,15" fill="#DC2626" opacity="0.8"/>
+    <text x="78" y="0" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" text-anchor="middle" font-weight="700">OVN</text>
+    <text x="78" y="28" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#FCA5A5" text-anchor="middle">NAT pipeline</text>
+  </g>
+
+  <!-- br-wan bridge (right) -->
+  <g transform="translate(520, 122)">
+    <rect width="260" height="176" rx="10" fill="url(#br-wan-grad)" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="20" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#58a6ff" font-weight="700">br-wan</text>
+    <text x="240" y="24" font-family="'Segoe UI', sans-serif" font-size="10" fill="#8ab4f8" text-anchor="end">(WAN)</text>
+    <line x1="20" y1="34" x2="240" y2="34" stroke="#58a6ff" stroke-opacity="0.3" stroke-width="0.5"/>
+
+    <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+      <rect x="30" y="52" width="200" height="60" rx="5" fill="#58a6ff1a" stroke="#58a6ff" stroke-opacity="0.45" stroke-width="0.8"/>
+      <text x="130" y="74" fill="#8ab4f8" text-anchor="middle" font-weight="600">spx-ext-{nic}</text>
+      <text x="130" y="92" fill="#a1a1aa" text-anchor="middle" font-size="10">(macvlan)</text>
+    </g>
+  </g>
+
+  <!-- macvlan bridge mode label -->
+  <text x="650" y="314" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="middle" font-style="italic">macvlan bridge mode</text>
+
+  <!-- WAN NIC inside host, bottom -->
+  <line x1="650" y1="298" x2="650" y2="320" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <g transform="translate(500, 320)">
+    <rect width="300" height="20" rx="5" fill="#161b22" stroke="#52525b" stroke-width="0.6"/>
+    <text x="150" y="14" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle">WAN NIC · host IP unchanged</text>
+  </g>
+
+  <!-- connector out -->
+  <line x1="650" y1="344" x2="650" y2="380" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <polygon points="646,376 654,376 650,384" fill="#71717a" opacity="0.7"/>
+
+  <!-- Physical WAN -->
+  <g transform="translate(500, 388)">
+    <rect width="300" height="40" rx="7" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.55"/>
+    <text x="150" y="25" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" text-anchor="middle" font-weight="700">Physical WAN</text>
+  </g>
+</svg>
@@ -0,0 +1,57 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="320" viewBox="0 0 900 320">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="320" fill="url(#bg)" rx="12"/>
+
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">MACVLAN FALLBACK · single-NIC host</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">WAN NIC == SSH NIC · host IP preserved</text>
+
+  <!-- WAN NIC (parent) -->
+  <g transform="translate(280, 58)">
+    <rect width="340" height="54" rx="8" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.3" stroke-opacity="0.65"/>
+    <text x="170" y="24" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#58a6ff" text-anchor="middle" font-weight="700">WAN NIC</text>
+    <text x="170" y="42" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">host IP unchanged · parent interface</text>
+  </g>
+
+  <!-- Split point -->
+  <line x1="450" y1="112" x2="450" y2="132" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <line x1="260" y1="132" x2="640" y2="132" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <line x1="260" y1="132" x2="260" y2="148" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <line x1="640" y1="132" x2="640" y2="148" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- Child 1: host traffic -->
+  <g transform="translate(120, 148)">
+    <rect width="280" height="48" rx="7" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="140" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#3fb950" text-anchor="middle" font-weight="700">host traffic</text>
+    <text x="140" y="38" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">SSH · management · overlay</text>
+  </g>
+
+  <!-- Child 2: macvlan -->
+  <g transform="translate(500, 148)">
+    <rect width="280" height="48" rx="7" fill="#DC26261a" stroke="#DC2626" stroke-width="1.1" stroke-opacity="0.65"/>
+    <text x="140" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" text-anchor="middle" font-weight="700">spx-ext-{nic}</text>
+    <text x="140" y="38" font-family="'Segoe UI', sans-serif" font-size="10" fill="#FCA5A5" text-anchor="middle">macvlan · bridge mode · no IP</text>
+  </g>
+
+  <!-- br-wan -->
+  <line x1="640" y1="196" x2="640" y2="216" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <polygon points="636,212 644,212 640,220" fill="#71717a" opacity="0.7"/>
+
+  <g transform="translate(500, 220)">
+    <rect width="280" height="40" rx="7" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="140" y="25" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#58a6ff" text-anchor="middle" font-weight="700">br-wan</text>
+  </g>
+
+  <!-- localnet -->
+  <line x1="640" y1="260" x2="640" y2="276" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <polygon points="636,272 644,272 640,280" fill="#71717a" opacity="0.7"/>
+
+  <g transform="translate(500, 280)">
+    <rect width="280" height="30" rx="6" fill="#DC26260d" stroke="#DC2626" stroke-width="1" stroke-opacity="0.5" stroke-dasharray="4,3"/>
+    <text x="140" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" text-anchor="middle" font-weight="600">localnet → OVN ext switch</text>
+  </g>
+</svg>
@@ -0,0 +1,131 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="560" viewBox="0 0 900 560">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="wan-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="ovs-grad" x1="0%" y1="0%" x2="100%" y2="0%">
+      <stop offset="0%" style="stop-color:#58a6ff22"/>
+      <stop offset="100%" style="stop-color:#58a6ff0a"/>
+    </linearGradient>
+    <linearGradient id="router-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#DC262622"/>
+      <stop offset="100%" style="stop-color:#7F1D1D0a"/>
+    </linearGradient>
+    <linearGradient id="pub-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#3fb95022"/>
+      <stop offset="100%" style="stop-color:#3fb95008"/>
+    </linearGradient>
+    <linearGradient id="priv-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#a855f722"/>
+      <stop offset="100%" style="stop-color:#a855f708"/>
+    </linearGradient>
+  </defs>
+
+  <rect width="900" height="560" fill="url(#bg)" rx="12"/>
+
+  <!-- Title -->
+  <text x="24" y="30" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" letter-spacing="2.5" font-weight="600">NETWORK PATH</text>
+  <text x="876" y="30" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="end">OVN logical topology · bridge mapping · NAT</text>
+
+  <!-- WAN -->
+  <rect x="300" y="54" width="300" height="40" rx="8" fill="url(#wan-grad)" stroke="#3fb950" stroke-width="1" stroke-opacity="0.55"/>
+  <text x="450" y="78" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">Physical Network (WAN)</text>
+
+  <!-- connector -->
+  <line x1="450" y1="94" x2="450" y2="112" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+
+  <!-- br-wan bridge -->
+  <rect x="340" y="112" width="220" height="52" rx="8" fill="url(#ovs-grad)" stroke="#58a6ff" stroke-width="1.2" stroke-opacity="0.65"/>
+  <text x="450" y="134" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#58a6ff" text-anchor="middle" font-weight="700">br-wan</text>
+  <text x="450" y="152" font-family="'Segoe UI', sans-serif" font-size="10" fill="#a1a1aa" text-anchor="middle">OVS bridge · physical uplink</text>
+
+  <!-- connector with bridge-mappings label -->
+  <line x1="450" y1="164" x2="450" y2="186" stroke="#71717a" stroke-width="1" opacity="0.5"/>
+  <rect x="316" y="170" width="268" height="18" rx="9" fill="#0a0a0a" stroke="#58a6ff" stroke-width="0.6" stroke-opacity="0.5"/>
+  <text x="450" y="183" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#8ab4f8" text-anchor="middle">ovn-bridge-mappings: external:br-wan</text>
+
+  <!-- VPC Logical Router (the big box) -->
+  <rect x="60" y="208" width="780" height="196" rx="10" fill="url(#router-grad)" stroke="#DC2626" stroke-width="1.4" stroke-opacity="0.75"/>
+  <text x="78" y="232" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#F87171" font-weight="700" letter-spacing="1">VPC LOGICAL ROUTER</text>
+  <line x1="60" y1="242" x2="840" y2="242" stroke="#DC2626" stroke-opacity="0.25" stroke-width="0.5"/>
+
+  <!-- Router ports row -->
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="11">
+    <rect x="82" y="254" width="120" height="42" rx="5" fill="#0d1117" stroke="#F87171" stroke-opacity="0.6" stroke-width="0.8"/>
+    <text x="142" y="270" fill="#F87171" text-anchor="middle" font-weight="600">Gateway port</text>
+    <text x="142" y="286" fill="#a1a1aa" text-anchor="middle" font-size="9.5">public IP</text>
+
+    <text x="215" y="282" fill="#71717a">─────</text>
+
+    <rect x="260" y="254" width="120" height="42" rx="5" fill="#0d1117" stroke="#F87171" stroke-opacity="0.6" stroke-width="0.8"/>
+    <text x="320" y="270" fill="#F87171" text-anchor="middle" font-weight="600">ext switch</text>
+    <text x="320" y="286" fill="#a1a1aa" text-anchor="middle" font-size="9.5">OVN logical</text>
+
+    <text x="393" y="282" fill="#71717a">─────</text>
+
+    <rect x="438" y="254" width="120" height="42" rx="5" fill="#0d1117" stroke="#F87171" stroke-opacity="0.6" stroke-width="0.8"/>
+    <text x="498" y="270" fill="#F87171" text-anchor="middle" font-weight="600">localnet port</text>
+    <text x="498" y="286" fill="#a1a1aa" text-anchor="middle" font-size="9.5">→ br-wan</text>
+  </g>
+
+  <!-- NAT rules + Routes, two columns on right -->
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10.5">
+    <text x="598" y="260" fill="#F87171" font-weight="600">NAT rules</text>
+    <text x="598" y="276" fill="#f5f5f5">SNAT:</text>
+    <text x="644" y="276" fill="#a1a1aa">VPC CIDR → gw IP</text>
+    <text x="598" y="292" fill="#f5f5f5">dnat_and_snat:</text>
+    <text x="704" y="292" fill="#a1a1aa">pub ↔ priv</text>
+  </g>
+
+  <g font-family="'SF Mono', ui-monospace, monospace" font-size="10.5">
+    <text x="82" y="330" fill="#F87171" font-weight="600">Routes</text>
+    <text x="82" y="346" fill="#f5f5f5">0.0.0.0/0</text>
+    <text x="154" y="346" fill="#a1a1aa">→ WAN gateway</text>
+  </g>
+
+  <!-- Subnet branches inside router -->
+  <line x1="150" y1="362" x2="150" y2="386" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="150" y1="386" x2="305" y2="386" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="305" y1="386" x2="305" y2="404" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+
+  <line x1="150" y1="386" x2="515" y2="386" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+  <line x1="515" y1="386" x2="515" y2="404" stroke="#DC2626" stroke-width="1" stroke-dasharray="3,3" opacity="0.6"/>
+
+  <!-- Subnet A (public) -->
+  <g transform="translate(215, 404)">
+    <rect width="180" height="124" rx="8" fill="url(#pub-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="90" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#3fb950" text-anchor="middle" font-weight="700">subnet-a</text>
+    <text x="90" y="38" font-family="'Segoe UI', sans-serif" font-size="10" fill="#7ee787" text-anchor="middle">(public)</text>
+    <line x1="16" y1="46" x2="164" y2="46" stroke="#3fb950" stroke-opacity="0.3" stroke-width="0.5"/>
+    <text x="90" y="64" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#f5f5f5" text-anchor="middle">10.0.1.0/24</text>
+    <g font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa">
+      <circle cx="40" cy="88" r="2.5" fill="#3fb950"/>
+      <text x="52" y="92">eni-1</text>
+      <circle cx="40" cy="108" r="2.5" fill="#3fb950"/>
+      <text x="52" y="112">eni-2</text>
+    </g>
+  </g>
+
+  <!-- Subnet B (private) -->
+  <g transform="translate(425, 404)">
+    <rect width="180" height="124" rx="8" fill="url(#priv-grad)" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="90" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="13" fill="#c084fc" text-anchor="middle" font-weight="700">subnet-b</text>
+    <text x="90" y="38" font-family="'Segoe UI', sans-serif" font-size="10" fill="#d8b4fe" text-anchor="middle">(private)</text>
+    <line x1="16" y1="46" x2="164" y2="46" stroke="#a855f7" stroke-opacity="0.3" stroke-width="0.5"/>
+    <text x="90" y="64" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#f5f5f5" text-anchor="middle">10.0.2.0/24</text>
+    <g font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa">
+      <circle cx="40" cy="88" r="2.5" fill="#a855f7"/>
+      <text x="52" y="92">eni-3</text>
+      <circle cx="40" cy="108" r="2.5" fill="#a855f7"/>
+      <text x="52" y="112">eni-4</text>
+    </g>
+  </g>
+
+  <!-- Footnote -->
+  <text x="24" y="548" font-family="'Segoe UI', sans-serif" font-size="10" fill="#71717a">Cross-host traffic uses Geneve tunnels (UDP 6081) over the overlay NIC · ovn-controller programs OpenFlow rules on br-int</text>
+</svg>
@@ -0,0 +1,56 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="110" viewBox="0 0 900 110">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="110" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="24" font-family="'Segoe UI', system-ui, sans-serif" font-size="10" fill="#a1a1aa" letter-spacing="2" font-weight="600">PRIVATE SUBNET · no default route</text>
+
+  <!-- Instance -->
+  <g transform="translate(40, 42)">
+    <rect width="180" height="44" rx="7" fill="#a855f71a" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="90" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#c084fc" text-anchor="middle" font-weight="700">Instance</text>
+    <text x="90" y="34" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#d8b4fe" text-anchor="middle">10.0.2.5</text>
+  </g>
+
+  <!-- arrow -->
+  <g stroke="#DC2626" stroke-width="1.2" opacity="0.6" fill="none">
+    <line x1="220" y1="64" x2="260" y2="64" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="260,60 268,64 260,68" fill="#DC2626" opacity="0.6"/>
+
+  <!-- Router -->
+  <g transform="translate(275, 42)">
+    <rect width="180" height="44" rx="7" fill="#DC26261a" stroke="#DC2626" stroke-width="1.2" stroke-opacity="0.65"/>
+    <text x="90" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" text-anchor="middle" font-weight="700">VPC Router</text>
+    <text x="90" y="34" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#FCA5A5" text-anchor="middle">lookup 0.0.0.0/0</text>
+  </g>
+
+  <!-- arrow -->
+  <g stroke="#DC2626" stroke-width="1.2" opacity="0.6" fill="none">
+    <line x1="455" y1="64" x2="495" y2="64" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="495,60 503,64 495,68" fill="#DC2626" opacity="0.6"/>
+
+  <!-- [no default route] -->
+  <g transform="translate(510, 42)">
+    <rect width="196" height="44" rx="7" fill="#0a0a0a" stroke="#71717a" stroke-width="1" stroke-opacity="0.7" stroke-dasharray="4,3"/>
+    <text x="98" y="20" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#a1a1aa" text-anchor="middle" font-weight="600">[no default route]</text>
+    <text x="98" y="34" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#71717a" text-anchor="middle">no IGW attached</text>
+  </g>
+
+  <!-- arrow to dropped -->
+  <g stroke="#71717a" stroke-width="1.2" opacity="0.6" fill="none">
+    <line x1="706" y1="64" x2="746" y2="64" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="746,60 754,64 746,68" fill="#71717a" opacity="0.6"/>
+
+  <!-- dropped -->
+  <g transform="translate(760, 42)">
+    <rect width="116" height="44" rx="7" fill="#27272a" stroke="#52525b" stroke-width="1" stroke-opacity="0.7"/>
+    <text x="58" y="27" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#a1a1aa" text-anchor="middle" font-weight="600">dropped</text>
+  </g>
+</svg>
@@ -0,0 +1,74 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="170" viewBox="0 0 900 170">
+  <defs>
+    <linearGradient id="bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+  </defs>
+  <rect width="900" height="170" fill="url(#bg)" rx="10"/>
+
+  <text x="24" y="24" font-family="'Segoe UI', system-ui, sans-serif" font-size="10" fill="#a1a1aa" letter-spacing="2" font-weight="600">PUBLIC SUBNET · 1:1 NAT managed by OVN</text>
+
+  <!-- OUTBOUND row -->
+  <text x="24" y="52" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#3fb950" font-weight="700">Outbound</text>
+
+  <g transform="translate(112, 38)">
+    <rect width="180" height="40" rx="6" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="90" y="18" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#7ee787" text-anchor="middle" font-weight="700">Instance</text>
+    <text x="90" y="32" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle">10.0.1.5</text>
+  </g>
+
+  <g stroke="#3fb950" stroke-width="1.1" opacity="0.6" fill="none">
+    <line x1="292" y1="58" x2="352" y2="58" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="352,54 360,58 352,62" fill="#3fb950" opacity="0.7"/>
+
+  <g transform="translate(368, 38)">
+    <rect width="130" height="40" rx="6" fill="#DC26261a" stroke="#DC2626" stroke-width="1.1" stroke-opacity="0.65"/>
+    <text x="65" y="25" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" text-anchor="middle" font-weight="700">SNAT</text>
+  </g>
+
+  <g stroke="#3fb950" stroke-width="1.1" opacity="0.6" fill="none">
+    <line x1="498" y1="58" x2="558" y2="58" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="558,54 566,58 558,62" fill="#3fb950" opacity="0.7"/>
+
+  <g transform="translate(574, 38)">
+    <rect width="272" height="40" rx="6" fill="#3fb9501a" stroke="#3fb950" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="136" y="18" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#7ee787" text-anchor="middle" font-weight="700">exits as 203.0.113.10</text>
+    <text x="136" y="32" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle">source IP rewritten</text>
+  </g>
+
+  <!-- INBOUND row -->
+  <text x="24" y="110" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#58a6ff" font-weight="700">Inbound</text>
+
+  <g transform="translate(112, 96)">
+    <rect width="180" height="40" rx="6" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="90" y="18" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#8ab4f8" text-anchor="middle" font-weight="700">WAN client</text>
+    <text x="90" y="32" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle">→ 203.0.113.10</text>
+  </g>
+
+  <g stroke="#58a6ff" stroke-width="1.1" opacity="0.6" fill="none">
+    <line x1="292" y1="116" x2="352" y2="116" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="352,112 360,116 352,120" fill="#58a6ff" opacity="0.7"/>
+
+  <g transform="translate(368, 96)">
+    <rect width="130" height="40" rx="6" fill="#DC26261a" stroke="#DC2626" stroke-width="1.1" stroke-opacity="0.65"/>
+    <text x="65" y="25" font-family="'SF Mono', ui-monospace, monospace" font-size="12" fill="#F87171" text-anchor="middle" font-weight="700">DNAT</text>
+  </g>
+
+  <g stroke="#58a6ff" stroke-width="1.1" opacity="0.6" fill="none">
+    <line x1="498" y1="116" x2="558" y2="116" stroke-dasharray="3,3"/>
+  </g>
+  <polygon points="558,112 566,116 558,120" fill="#58a6ff" opacity="0.7"/>
+
+  <g transform="translate(574, 96)">
+    <rect width="272" height="40" rx="6" fill="#58a6ff1a" stroke="#58a6ff" stroke-width="1.1" stroke-opacity="0.6"/>
+    <text x="136" y="18" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#8ab4f8" text-anchor="middle" font-weight="700">arrives at 10.0.1.5</text>
+    <text x="136" y="32" font-family="'SF Mono', ui-monospace, monospace" font-size="10" fill="#a1a1aa" text-anchor="middle">dest IP rewritten</text>
+  </g>
+
+  <!-- Footnote -->
+  <text x="24" y="158" font-family="'Segoe UI', sans-serif" font-size="9" fill="#71717a">The instance OS only sees its private IP (10.0.1.5) — the public address is managed entirely by OVN's dnat_and_snat rule.</text>
+</svg>
@@ -0,0 +1,144 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="900" height="560" viewBox="0 0 900 560">
+  <defs>
+    <linearGradient id="pf-bg" x1="0%" y1="0%" x2="100%" y2="100%">
+      <stop offset="0%" style="stop-color:#0a0a0a"/>
+      <stop offset="100%" style="stop-color:#0d1117"/>
+    </linearGradient>
+    <linearGradient id="hw-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#52525b;stop-opacity:0.18"/>
+      <stop offset="100%" style="stop-color:#52525b;stop-opacity:0.06"/>
+    </linearGradient>
+    <linearGradient id="os-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#a855f7;stop-opacity:0.18"/>
+      <stop offset="100%" style="stop-color:#a855f7;stop-opacity:0.05"/>
+    </linearGradient>
+    <linearGradient id="spx-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#DC2626;stop-opacity:0.28"/>
+      <stop offset="100%" style="stop-color:#7F1D1D;stop-opacity:0.1"/>
+    </linearGradient>
+    <linearGradient id="app-grad" x1="0%" y1="0%" x2="0%" y2="100%">
+      <stop offset="0%" style="stop-color:#3fb950;stop-opacity:0.18"/>
+      <stop offset="100%" style="stop-color:#3fb950;stop-opacity:0.05"/>
+    </linearGradient>
+  </defs>
+
+  <rect width="900" height="560" fill="url(#pf-bg)" rx="12"/>
+
+  <!-- Subtle mesh dots background -->
+  <g opacity="0.12" fill="#7F1D1D">
+    <circle cx="40" cy="40" r="1"/><circle cx="120" cy="30" r="1"/><circle cx="220" cy="50" r="1"/>
+    <circle cx="320" cy="25" r="1"/><circle cx="440" cy="45" r="1"/><circle cx="560" cy="30" r="1"/>
+    <circle cx="680" cy="50" r="1"/><circle cx="780" cy="30" r="1"/><circle cx="860" cy="45" r="1"/>
+    <circle cx="60" cy="530" r="1"/><circle cx="160" cy="520" r="1"/><circle cx="260" cy="540" r="1"/>
+    <circle cx="380" cy="525" r="1"/><circle cx="500" cy="540" r="1"/><circle cx="620" cy="520" r="1"/>
+    <circle cx="740" cy="540" r="1"/><circle cx="840" cy="525" r="1"/>
+  </g>
+
+  <!-- Title -->
+  <text x="450" y="32" font-family="'Segoe UI', system-ui, sans-serif" font-size="11" fill="#a1a1aa" text-anchor="middle" letter-spacing="2.8">THE MULGA PLATFORM</text>
+  <text x="450" y="58" font-family="'Segoe UI', system-ui, sans-serif" font-size="20" font-weight="600" fill="#f5f5f5" text-anchor="middle">AWS-style workflows, on your own stack</text>
+
+  <!-- Layer 4: Applications (top) -->
+  <g transform="translate(60, 95)">
+    <rect width="780" height="88" rx="10" fill="url(#app-grad)" stroke="#3fb950" stroke-width="1.2" stroke-opacity="0.55"/>
+    <text x="20" y="26" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#3fb950" letter-spacing="1.5" font-weight="600">LAYER 4 · APPLICATIONS &amp; TOOLING</text>
+    <text x="20" y="48" font-family="'Segoe UI', sans-serif" font-size="15" fill="#f5f5f5" font-weight="500">Your workloads — unmodified AWS tooling</text>
+
+    <g transform="translate(20, 60)" font-family="'Segoe UI', sans-serif" font-size="10.5">
+      <rect x="0" y="0" width="70" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="35" y="14" fill="#7ee787" text-anchor="middle">aws-cli</text>
+
+      <rect x="78" y="0" width="78" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="117" y="14" fill="#7ee787" text-anchor="middle">Terraform</text>
+
+      <rect x="164" y="0" width="88" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="208" y="14" fill="#7ee787" text-anchor="middle">AWS SDKs</text>
+
+      <rect x="260" y="0" width="88" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="304" y="14" fill="#7ee787" text-anchor="middle">Kubernetes</text>
+
+      <rect x="356" y="0" width="70" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="391" y="14" fill="#7ee787" text-anchor="middle">Packer</text>
+
+      <rect x="434" y="0" width="98" height="20" rx="10" fill="#3fb950" fill-opacity="0.14" stroke="#3fb950" stroke-opacity="0.45" stroke-width="0.5"/>
+      <text x="483" y="14" fill="#7ee787" text-anchor="middle">Your Services</text>
+    </g>
+  </g>
+
+  <!-- Down arrow -->
+  <g transform="translate(450, 188)" opacity="0.45">
+    <line x1="0" y1="0" x2="0" y2="14" stroke="#a1a1aa" stroke-width="1"/>
+    <polygon points="-4,12 4,12 0,18" fill="#a1a1aa"/>
+  </g>
+
+  <!-- Layer 3: Spinifex (highlighted) -->
+  <g transform="translate(60, 215)">
+    <rect width="780" height="140" rx="10" fill="url(#spx-grad)" stroke="#DC2626" stroke-width="1.5" stroke-opacity="0.75"/>
+    <text x="20" y="26" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#F87171" letter-spacing="1.5" font-weight="600">LAYER 3 · SPINIFEX · AWS ORCHESTRATION</text>
+    <text x="20" y="50" font-family="'Segoe UI', sans-serif" font-size="15" fill="#f5f5f5" font-weight="500">AWS Gateway → NATS → Service Daemons · SigV4, IAM, VPC-aware</text>
+
+    <g transform="translate(20, 66)" font-family="'Segoe UI', sans-serif">
+      <!-- EC2 -->
+      <rect x="0" y="0" width="118" height="58" rx="6" fill="#DC2626" fill-opacity="0.1" stroke="#DC2626" stroke-opacity="0.6" stroke-width="1"/>
+      <text x="59" y="18" font-size="13" fill="#F87171" text-anchor="middle" font-weight="600">EC2</text>
+      <text x="59" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">QEMU/KVM VMs</text>
+      <text x="59" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">cloud-init · AMIs</text>
+
+      <!-- EBS -->
+      <rect x="126" y="0" width="118" height="58" rx="6" fill="#DC2626" fill-opacity="0.1" stroke="#DC2626" stroke-opacity="0.6" stroke-width="1"/>
+      <text x="185" y="18" font-size="13" fill="#F87171" text-anchor="middle" font-weight="600">EBS</text>
+      <text x="185" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">Viperblock</text>
+      <text x="185" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">WAL · NBD · snapshots</text>
+
+      <!-- S3 -->
+      <rect x="252" y="0" width="118" height="58" rx="6" fill="#DC2626" fill-opacity="0.1" stroke="#DC2626" stroke-opacity="0.6" stroke-width="1"/>
+      <text x="311" y="18" font-size="13" fill="#F87171" text-anchor="middle" font-weight="600">S3</text>
+      <text x="311" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">Predastore</text>
+      <text x="311" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">Reed–Solomon EC</text>
+
+      <!-- VPC -->
+      <rect x="378" y="0" width="118" height="58" rx="6" fill="#7F1D1D" fill-opacity="0.22" stroke="#F87171" stroke-opacity="0.55" stroke-width="1"/>
+      <text x="437" y="18" font-size="13" fill="#FCA5A5" text-anchor="middle" font-weight="600">VPC</text>
+      <text x="437" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">OVN / OVS</text>
+      <text x="437" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">subnets · SG · ALB</text>
+
+      <!-- IAM -->
+      <rect x="504" y="0" width="118" height="58" rx="6" fill="#7F1D1D" fill-opacity="0.22" stroke="#F87171" stroke-opacity="0.55" stroke-width="1"/>
+      <text x="563" y="18" font-size="13" fill="#FCA5A5" text-anchor="middle" font-weight="600">IAM</text>
+      <text x="563" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">SigV4 · policies</text>
+      <text x="563" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">users · roles · keys</text>
+
+      <!-- Control -->
+      <rect x="630" y="0" width="110" height="58" rx="6" fill="#27272a" fill-opacity="0.8" stroke="#71717a" stroke-opacity="0.6" stroke-width="1"/>
+      <text x="685" y="18" font-size="13" fill="#e4e4e7" text-anchor="middle" font-weight="600">Control</text>
+      <text x="685" y="34" font-size="9.5" fill="#f5f5f5" text-anchor="middle">NATS bus</text>
+      <text x="685" y="48" font-size="9.5" fill="#a1a1aa" text-anchor="middle">spx · daemons</text>
+    </g>
+  </g>
+
+  <!-- Down arrow -->
+  <g transform="translate(450, 360)" opacity="0.45">
+    <line x1="0" y1="0" x2="0" y2="14" stroke="#a1a1aa" stroke-width="1"/>
+    <polygon points="-4,12 4,12 0,18" fill="#a1a1aa"/>
+  </g>
+
+  <!-- Layer 2: OS -->
+  <g transform="translate(60, 387)">
+    <rect width="780" height="62" rx="10" fill="url(#os-grad)" stroke="#a855f7" stroke-width="1.2" stroke-opacity="0.5"/>
+    <text x="20" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#c084fc" letter-spacing="1.5" font-weight="600">LAYER 2 · OPERATING SYSTEM</text>
+    <text x="20" y="44" font-family="'Segoe UI', sans-serif" font-size="14" fill="#f5f5f5">Debian / Ubuntu LTS · systemd · KVM · Linux bridge · headless autoinstall</text>
+  </g>
+
+  <!-- Down arrow -->
+  <g transform="translate(450, 454)" opacity="0.45">
+    <line x1="0" y1="0" x2="0" y2="14" stroke="#a1a1aa" stroke-width="1"/>
+    <polygon points="-4,12 4,12 0,18" fill="#a1a1aa"/>
+  </g>
+
+  <!-- Layer 1: Hardware -->
+  <g transform="translate(60, 481)">
+    <rect width="780" height="62" rx="10" fill="url(#hw-grad)" stroke="#71717a" stroke-width="1.2" stroke-opacity="0.6"/>
+    <text x="20" y="22" font-family="'SF Mono', ui-monospace, monospace" font-size="11" fill="#a1a1aa" letter-spacing="1.5" font-weight="600">LAYER 1 · HARDWARE</text>
+    <text x="20" y="44" font-family="'Segoe UI', sans-serif" font-size="14" fill="#f5f5f5">Single node · multi-node cluster · edge appliance · air-gapped rack · commodity x86 / ARM</text>
+  </g>
+</svg>
@@ -0,0 +1,52 @@
+version: 2
+updates:
+  - package-ecosystem: gomod
+    target-branch: dev
+    directory: /
+    schedule:
+      interval: weekly
+      day: monday
+      timezone: Australia/Sydney
+    cooldown:
+      default-days: 3
+    groups:
+      go-dependencies:
+        patterns:
+          - "*"
+
+  - package-ecosystem: github-actions
+    target-branch: dev
+    directory: /
+    schedule:
+      interval: weekly
+      day: monday
+      timezone: Australia/Sydney
+    cooldown:
+      default-days: 3
+    ignore:
+      - dependency-name: "*"
+        update-types:
+          - "version-update:semver-minor"
+          - "version-update:semver-patch"
+    groups:
+      github-actions:
+        patterns:
+          - "*"
+
+  - package-ecosystem: npm
+    target-branch: dev
+    directory: /spinifex/services/spinifexui/frontend
+    schedule:
+      interval: weekly
+      day: monday
+      timezone: Australia/Sydney
+    cooldown:
+      default-days: 3
+    ignore:
+      - dependency-name: "@types/node"
+        update-types:
+          - "version-update:semver-major"
+    groups:
+      npm-dependencies:
+        patterns:
+          - "*"
@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+# capture-net-diag.sh — write a snapshot of network / OVN / OVS / IPsec
+# state into $1 (a directory the caller already created on the local host).
+# Designed to run on a spinifex node after an E2E suite, before tofu destroy
+# pulls the rug out. Every command is independent — `|| true` so a missing
+# binary on one path doesn't abort the rest of the capture.
+#
+# Output layout (under $1/net-diag/):
+#   openvswitch-ipsec.status         systemctl status (output-only)
+#   openvswitch-ipsec.journal        last 200 journal lines
+#   ipsec.openvswitch_conf           ovs-vsctl get Open_vSwitch . other_config
+#   ipsec.xfrm_state                 ip xfrm state list
+#   ipsec.xfrm_policy                ip xfrm policy list
+#   ovn.sb_show                      ovn-sbctl show
+#   ovn.nb_show                      ovn-nbctl show
+#   ovs.show                         ovs-vsctl show
+#   ovs.flows_br_int                 ovs-ofctl dump-flows br-int
+#   ovs.tunnels                      ovs-appctl ofproto/list-tunnels br-int
+#   net.links                        ip -s link show
+#   net.routes                       ip route show table all
+#   net.addrs                        ip -br addr show
+#   systemd.failed_units             systemctl list-units --state=failed
+set +e
+OUT="${1:?usage: capture-net-diag.sh <output-dir>}"
+mkdir -p "$OUT"
+
+run() {
+    local label="$1"; shift
+    {
+        echo "# $*"
+        "$@" 2>&1
+    } > "$OUT/$label" || true
+}
+
+run openvswitch-ipsec.status            sudo systemctl status openvswitch-ipsec.service --no-pager --full --lines=50
+run openvswitch-ipsec.journal           sudo journalctl -u openvswitch-ipsec.service --no-pager --output=short-iso -n 200
+run ipsec.openvswitch_conf              sudo ovs-vsctl get Open_vSwitch . other_config
+run ipsec.xfrm_state                    sudo ip xfrm state list
+run ipsec.xfrm_policy                   sudo ip xfrm policy list
+
+run ovn.sb_show                         sudo ovn-sbctl show
+run ovn.nb_show                         sudo ovn-nbctl show
+run ovn.sb_chassis                      sudo ovn-sbctl list chassis
+
+run_per_lr() {
+    local label="$1"; shift
+    local subcmd="$1"; shift
+    {
+        echo "# ovn-nbctl $subcmd <each-logical-router>"
+        local lrs
+        lrs="$(sudo ovn-nbctl --bare --columns=name list Logical_Router 2>/dev/null)"
+        for lr in $lrs; do
+            echo
+            echo "=== $lr ==="
+            sudo ovn-nbctl "$subcmd" "$lr" 2>&1
+        done
+    } > "$OUT/$label" || true
+}
+
+run_per_lr ovn.lr_routes                lr-route-list
+run_per_lr ovn.lr_policies              lr-policy-list
+run_per_lr ovn.lr_nat                   lr-nat-list
+
+run ovs.show                            sudo ovs-vsctl show
+run ovs.flows_br_int                    sudo ovs-ofctl dump-flows br-int
+run ovs.tunnels                         sudo ovs-appctl ofproto/list-tunnels br-int
+
+run net.links                           ip -s link show
+run net.routes                          ip route show table all
+run net.addrs                           ip -br addr show
+run net.neigh                           ip neigh show
+run net.bridge_fdb                      bridge fdb show
+run net.conntrack                       sudo conntrack -L
+
+run systemd.failed_units                sudo systemctl list-units --state=failed --no-pager
+
+# Background pcap (started by start-net-capture.sh, if present).
+if [ -f /tmp/spinifex-e2e-tcpdump.pid ]; then
+    PID="$(cat /tmp/spinifex-e2e-tcpdump.pid 2>/dev/null)"
+    sudo kill -INT "$PID" 2>/dev/null || true
+    sleep 1
+    if [ -f /tmp/spinifex-e2e-wan.pcap ]; then
+        sudo cp /tmp/spinifex-e2e-wan.pcap "$OUT/wan.pcap" 2>/dev/null || true
+        sudo chmod 0644 "$OUT/wan.pcap" 2>/dev/null || true
+    fi
+fi
+
+exit 0
@@ -0,0 +1,63 @@
+# shellcheck shell=bash
+# Source from a workflow `run:` block:
+#   source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+#
+# Provides colored banners + ::group::/::endgroup:: wrappers. ANSI escapes
+# render in the GitHub Actions live log; the same lines also land in any
+# artifact log (tee'd output) without breaking parsers because they only
+# affect terminal display.
+#
+# Functions kept short and unindented on purpose — wide nested output wraps
+# badly on narrow terminals.
+
+if [[ -n "${GITHUB_ACTIONS:-}" ]] || [[ -t 1 ]]; then
+  _R=$'\033[0m'
+  _B=$'\033[1m'    # bold
+  _D=$'\033[2m'    # dim
+  _RED=$'\033[31m'
+  _GRN=$'\033[32m'
+  _YEL=$'\033[33m'
+  _BLU=$'\033[34m'
+  _CYA=$'\033[36m'
+else
+  _R="" _B="" _D="" _RED="" _GRN="" _YEL="" _BLU="" _CYA=""
+fi
+
+# Single-line rule using box-drawing chars — narrow enough that GitHub's
+# timestamp prefix + this rule still fit on an 80-col terminal.
+# banner LABEL — bold cyan title with surrounding rule. Used at the start
+# of an inherently multi-line phase (test run, artifact pull) so the live
+# log has an obvious visual marker.
+banner() {
+  printf '\n%s%s━━ %s ━━%s\n' "$_B" "$_CYA" "$*" "$_R"
+}
+
+# Status indicators. Single char prefix only — keeps width on narrow logs.
+ok()   { printf '%s✓%s %s\n' "$_GRN" "$_R" "$*"; }
+bad()  { printf '%s✗%s %s\n' "$_RED" "$_R" "$*"; }
+warn() { printf '%s!%s %s\n' "$_YEL" "$_R" "$*"; }
+note() { printf '%s·%s %s\n' "$_D"   "$_R" "$*"; }
+
+# group_open LABEL / group_close — thin wrapper around GitHub's collapse
+# markers. No extra body printed; banner() is the visible inside-group
+# heading when callers want one.
+group_open()  { echo "::group::$*"; }
+group_close() { echo "::endgroup::"; }
+
+# quiet_run LABEL LOG_FILE -- CMD ARGS …
+# Run CMD redirected to LOG_FILE (append). Print one colored ok/✗ line.
+# On failure, dump the last 80 lines of LOG_FILE inside a collapsed group
+# so the failing tail lands in the UI without surfacing the whole apply.
+quiet_run() {
+  local label="$1" log="$2"; shift 2
+  if "$@" >> "$log" 2>&1; then
+    ok "$label"
+  else
+    local rc=$?
+    bad "$label (rc=$rc) — last 80 lines of $(basename "$log"):"
+    group_open "tail $(basename "$log")"
+    tail -n 80 "$log"
+    group_close
+    return "$rc"
+  fi
+}
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# start-net-capture.sh — kick off background tcpdump on br-wan.
+# Pair: capture-net-diag.sh stops it and copies the pcap into net-diag/.
+#
+# Filter: anything on br-wan involving the EIP pool (192.168.0.0/23) plus
+# ARP for the same range. Single ring buffer (-W 1 -G 0) → grows until stop;
+# capped at 64 MiB via -C 64. Snaplen 200 bytes (headers + a bit) to keep
+# size sane while preserving L2/L3/L4 + first payload bytes.
+set +e
+
+PCAP=/tmp/spinifex-e2e-wan.pcap
+PIDFILE=/tmp/spinifex-e2e-tcpdump.pid
+
+if [ -f "$PIDFILE" ] && kill -0 "$(cat "$PIDFILE" 2>/dev/null)" 2>/dev/null; then
+    echo "[start-net-capture] tcpdump already running pid=$(cat "$PIDFILE")"
+    exit 0
+fi
+
+sudo rm -f "$PCAP" "$PIDFILE"
+
+if ! command -v tcpdump >/dev/null 2>&1; then
+    echo "[start-net-capture] tcpdump not installed — skipping" >&2
+    exit 0
+fi
+if ! ip link show br-wan >/dev/null 2>&1; then
+    echo "[start-net-capture] br-wan absent — skipping" >&2
+    exit 0
+fi
+
+sudo nohup tcpdump -i br-wan -s 200 -C 64 -W 1 -U \
+    -w "$PCAP" \
+    '(net 192.168.0.0/23) or (arp and (arp[24:4]&0xfffffe00=0xc0a80000))' \
+    >/tmp/spinifex-e2e-tcpdump.log 2>&1 &
+PID=$!
+echo "$PID" | sudo tee "$PIDFILE" >/dev/null
+disown
+
+sleep 1
+if ! kill -0 "$PID" 2>/dev/null; then
+    echo "[start-net-capture] tcpdump died — see /tmp/spinifex-e2e-tcpdump.log" >&2
+    cat /tmp/spinifex-e2e-tcpdump.log >&2 || true
+    sudo rm -f "$PIDFILE"
+    exit 1
+fi
+
+echo "[start-net-capture] tcpdump pid=$PID pcap=$PCAP"
+exit 0
@@ -0,0 +1,148 @@
+name: E2E Bare-Metal
+
+on:
+  workflow_dispatch:
+  workflow_call:
+    inputs:
+      ref:
+        description: "spinifex ref to checkout/test (default: triggering ref)"
+        required: false
+        type: string
+        default: ""
+
+permissions:
+  contents: read
+  statuses: write
+
+env:
+  GIT_BRANCH: ${{ github.ref_name }}
+
+jobs:
+  e2e_baremetal:
+    name: Bare-Metal Single-Node E2E
+    runs-on: [ self-hosted, ci-baremetal ]
+    timeout-minutes: 120
+    concurrency:
+      group: e2e-baremetal-cluster
+      cancel-in-progress: false
+    env:
+      BM_ISO_CACHE: /var/cache/spinifex-iso-builder
+      ARTIFACT_DIR: /tmp/spinifex-e2e-nightly-cell-20-${{ github.run_id }}
+    steps:
+      - name: Clean Workspace
+        run: |
+          sudo umount -R -l "${BM_ISO_CACHE}/rootfs" 2>/dev/null || true
+          if [ ! -c /dev/null ]; then sudo rm -f /dev/null; sudo mknod -m 666 /dev/null c 1 3; fi
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/checkout@v6
+        with:
+          ref: ${{ inputs.ref || github.ref }}
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: |
+            scripts/iso-builder
+            scripts/baremetal
+          path: mulga
+
+      - name: Install SSH Key
+        run: |
+          echo "${{ secrets.BM_SSH_PRIVATE_KEY }}" > "${RUNNER_TEMP}/bm-ci"
+          chmod 0600 "${RUNNER_TEMP}/bm-ci"
+          ssh-keygen -y -f "${RUNNER_TEMP}/bm-ci" > "${RUNNER_TEMP}/bm-ci.pub"
+
+      - name: Build Artifacts
+        run: |
+          mulga/scripts/baremetal/bm-build-artifacts.sh \
+            "$GITHUB_WORKSPACE" \
+            "${RUNNER_TEMP}/spinifex-tests.tar" \
+            "${RUNNER_TEMP}/bm-bins"
+
+      - name: Build PXE Artifacts
+        run: |
+          sudo make -C mulga/scripts/iso-builder \
+            build-rootfs build-squashfs build-initramfs \
+            CACHE="${BM_ISO_CACHE}" \
+            SPINIFEX_BIN="${RUNNER_TEMP}/bm-bins/spx" \
+            INSTALLER_BIN="${RUNNER_TEMP}/bm-bins/spinifex-installer" \
+            SPINIFEX_SRC="${GITHUB_WORKSPACE}" \
+            VIPERBLOCK_SRC="${RUNNER_TEMP}/bm-bins/viperblock" \
+            VERSION="$(cd "$GITHUB_WORKSPACE" && git describe --tags --always 2>/dev/null || echo dev)"
+
+      - name: Write ePDU credentials
+        run: |
+          install -m 0600 /dev/null "${RUNNER_TEMP}/epdu.env"
+          cat >> "${RUNNER_TEMP}/epdu.env" <<EOF
+          EPDU_HOST=192.168.1.63
+          EPDU_SNMP_USER=ci-bmpower
+          EPDU_SNMP_SEC_LEVEL=authPriv
+          EPDU_SNMP_AUTH_PROTO=MD5
+          EPDU_SNMP_AUTH_PASS=${{ secrets.BM_EPDU_AUTH_PASS }}
+          EPDU_SNMP_PRIV_PROTO=DES
+          EPDU_SNMP_PRIV_PASS=${{ secrets.BM_EPDU_PRIV_PASS }}
+          EOF
+
+      - name: Cache VM images
+        run: |
+          UBUNTU_IMG="${BM_ISO_CACHE}/ubuntu-26.04-cloudimg-amd64.img"
+          if [ ! -f "$UBUNTU_IMG" ]; then
+            echo "Downloading Ubuntu 26.04 cloud image..."
+            sudo curl -fsSL -o "$UBUNTU_IMG" \
+              "https://cloud-images.ubuntu.com/resolute/current/resolute-server-cloudimg-amd64.img"
+            echo "  Downloaded $(du -h "$UBUNTU_IMG" | cut -f1)"
+          else
+            echo "Ubuntu 26.04 cloud image cached ($(du -h "$UBUNTU_IMG" | cut -f1))"
+          fi
+
+      - name: Install go-junit-report on runner
+        run: go install github.com/jstemmer/go-junit-report/v2@latest
+
+      - name: Run Bare-Metal E2E
+        env:
+          BM_CI_PASSWORD: ${{ secrets.BM_CI_PASSWORD }}
+          BM_SSH_KEY: ${{ runner.temp }}/bm-ci
+          BM_EPDU_ENV: ${{ runner.temp }}/epdu.env
+          BM_UBUNTU_IMAGE: ${{ env.BM_ISO_CACHE }}/ubuntu-26.04-cloudimg-amd64.img
+          SPX_EXTERNAL_POOL: ${{ env.SPINIFEX_CI_POOL }}
+        run: |
+          mkdir -p "${ARTIFACT_DIR}"
+          set -o pipefail
+          sudo -E mulga/scripts/baremetal/run-baremetal-e2e.sh \
+            --vmlinuz   "${BM_ISO_CACHE}/vmlinuz" \
+            --initrd    "${BM_ISO_CACHE}/initrd.img" \
+            --squashfs  "${BM_ISO_CACHE}/spinifex.squashfs" \
+            --tests-tar "${RUNNER_TEMP}/spinifex-tests.tar" \
+            2>&1 | tee "${ARTIFACT_DIR}/test-baremetal.log"
+
+      - name: Generate JUnit report
+        if: always()
+        run: |
+          LOG="${ARTIFACT_DIR}/test-baremetal.log"
+          [ -f "$LOG" ] || exit 0
+          go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-baremetal.xml" || true
+
+      - name: Analyze e2e failures
+        if: always()
+        uses: ./.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "E2E failure analysis (cell 20)"
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-artifacts-cell-20-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
@@ -0,0 +1,205 @@
+name: E2E DDIL
+
+# Phase 1 trigger: workflow_dispatch only. Nightly cron + on-main-merge
+# land in mulga-siv-5.8 once ≥4 scenarios are asserting. See
+# docs/development/improvements/ddil-e2e-test-harness.md §4 / §Dependencies.
+on:
+  workflow_dispatch:
+    inputs:
+      quarantined:
+        description: "Comma-separated scenario letters to quarantine (A..F)"
+        required: false
+        default: ""
+
+permissions:
+  contents: read
+
+env:
+  GIT_BRANCH: ${{ github.ref_name }}
+
+jobs:
+  ddil:
+    name: DDIL Real Multi-Node
+    runs-on: [ self-hosted, ci-multi ]
+    timeout-minutes: 60
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-ddil-${{ github.run_id }}.log
+    steps:
+      - name: Clean Workspace
+        run: sudo rm -rf "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        name: Checkout Spinifex
+        with:
+          path: spinifex
+
+      - uses: actions/checkout@v6
+        name: Checkout Mulga (tofu-cluster)
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - name: Install gotestsum
+        env:
+          GOTESTSUM_VERSION: "1.12.0"
+        run: |
+          # Prebuilt tarball, not `go install`: gotestsum v1.12.0 pulls
+          # x/tools v0.19.0 whose tokeninternal static-assert pegs a
+          # narrow Go window (~1.22). The self-hosted runner's Go falls
+          # outside that window, so source-install fails with
+          # `invalid array length -delta * delta`. A prebuilt binary
+          # sidesteps toolchain coupling entirely.
+          BIN_DIR="${RUNNER_TEMP}/gotestsum-bin"
+          mkdir -p "${BIN_DIR}"
+          if [ ! -x "${BIN_DIR}/gotestsum" ]; then
+            curl -fsSL \
+              "https://github.com/gotestyourself/gotestsum/releases/download/v${GOTESTSUM_VERSION}/gotestsum_${GOTESTSUM_VERSION}_linux_amd64.tar.gz" \
+              | tar -xz -C "${BIN_DIR}" gotestsum
+          fi
+          echo "${BIN_DIR}" >> "${GITHUB_PATH}"
+          "${BIN_DIR}/gotestsum" --version
+
+      - name: Build Install Artifacts
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VMs
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+          quiet_run "tofu init"                "$TOFU_LOG" tofu init -input=false
+          quiet_run "tofu workspace select"    "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy (pre-clean)" "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
+          quiet_run "scrub-env (pre-clean)"    "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          quiet_run "tofu apply"               "$TOFU_LOG" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          ok "VMs provisioned"
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}" \
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Run DDIL Scenarios
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -o pipefail
+          INVENTORY=$(tofu output -json inventory)
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          SSH_USER=$(echo "$INVENTORY" | jq -r '.ssh_user')
+          REGION=$(echo "$INVENTORY" | jq -r '.region')
+          # DDIL_NODES must be the cluster-traffic IPs: PartitionNode
+          # installs iptables DROP rules for each peer, so the runner's
+          # SSH source IP must differ from these. On single-NIC
+          # tofu-cluster, wan_ip == cluster IP and the runner's outbound
+          # IP is distinct — see e2e-ddil-test-harness.md §3 Scenario C.
+          NODE_IPS=$(echo "$INVENTORY" | jq -r '[.nodes[].wan_ip] | join(",")')
+          PRIMARY_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          echo "DDIL_NODES=${NODE_IPS}"
+
+          # Witness VMs talk to the cluster's AWS gateway via aws-sdk-go,
+          # which resolves credentials from the standard chain. spx admin
+          # init writes [spinifex] to ${SSH_USER}'s ~/.aws/credentials on
+          # node1 — pull them over SSH and export as the SDK's env-var
+          # entries so the harness can launch instances without depending
+          # on a cluster-bootstrapped profile being present on the runner.
+          CREDS=$(ssh -i "${SSH_KEY}" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
+            "${SSH_USER}@${PRIMARY_IP}" 'cat ~/.aws/credentials')
+          AWS_ACCESS_KEY_ID=$(echo "$CREDS" | awk -F '= *' '/^aws_access_key_id/ {print $2; exit}')
+          AWS_SECRET_ACCESS_KEY=$(echo "$CREDS" | awk -F '= *' '/^aws_secret_access_key/ {print $2; exit}')
+          if [ -z "$AWS_ACCESS_KEY_ID" ] || [ -z "$AWS_SECRET_ACCESS_KEY" ]; then
+            echo "::error::Failed to extract spinifex AWS credentials from ${PRIMARY_IP}"
+            exit 1
+          fi
+
+          GUEST_KEY="${RUNNER_TEMP}/spinifex-key"
+          scp -i "${SSH_KEY}" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
+            "${SSH_USER}@${PRIMARY_IP}:~/.ssh/spinifex-key" "${GUEST_KEY}"
+          chmod 600 "${GUEST_KEY}"
+
+          mkdir -p "${RUNNER_TEMP}/ddil-results"
+          cd "${GITHUB_WORKSPACE}/spinifex"
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          banner "DDIL scenarios"
+          DDIL_NODES="${NODE_IPS}" \
+          DDIL_SSH_USER="${SSH_USER}" \
+          DDIL_SSH_KEY="${SSH_KEY}" \
+          DDIL_GUEST_SSH_KEY="${GUEST_KEY}" \
+          DDIL_QUARANTINED="${{ inputs.quarantined }}" \
+          AWS_REGION="${REGION}" \
+          AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \
+          AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
+            gotestsum --jsonfile "${RUNNER_TEMP}/ddil-results/results.json" \
+                      --junitfile "${RUNNER_TEMP}/ddil-results/junit-ddil.xml" \
+                      --format testname \
+                      -- -tags=e2e -timeout 45m ./tests/e2e/ddil/scenarios/...
+          rc=$?
+          if [[ $rc -eq 0 ]]; then ok "all scenarios passed"; else bad "scenarios failed (rc=$rc)"; fi
+          exit $rc
+
+      - name: Publish test summary
+        if: always()
+        uses: ./spinifex/.github/actions/junit-report
+        with:
+          junit-glob: ${{ runner.temp }}/ddil-results/junit-*.xml
+          title: "DDIL scenario results"
+
+      - name: Collect Cluster Logs
+        if: failure()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          INVENTORY=$(tofu output -json inventory)
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          SSH_USER=$(echo "$INVENTORY" | jq -r '.ssh_user')
+          LOG_DIR="${RUNNER_TEMP}/ddil-results/cluster-logs"
+          mkdir -p "${LOG_DIR}"
+
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          mapfile -t NODE_IPS < <(echo "$INVENTORY" | jq -r '.nodes[].wan_ip')
+          for ip in "${NODE_IPS[@]}"; do
+            SSH="ssh -i ${SSH_KEY} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ${SSH_USER}@${ip}"
+            ${SSH} 'sudo journalctl -u "spinifex-*" --no-pager -n 5000' > "${LOG_DIR}/${ip}-spinifex.log" 2>&1
+            ${SSH} 'ps auxw | grep -E "qemu-system|spinifex" | grep -v grep' > "${LOG_DIR}/${ip}-procs.txt" 2>&1
+            ${SSH} 'ls -la /var/lib/spinifex 2>/dev/null; ls -la /var/log/spinifex 2>/dev/null' > "${LOG_DIR}/${ip}-state.txt" 2>&1
+            ok "$ip: $(wc -l < "${LOG_DIR}/${ip}-spinifex.log") journal lines, $(wc -l < "${LOG_DIR}/${ip}-procs.txt") procs"
+          done
+
+      - name: Upload DDIL Results
+        if: always()
+        uses: actions/upload-artifact@v7
+        with:
+          name: ddil-results-${{ github.run_id }}
+          path: ${{ runner.temp }}/ddil-results
+          retention-days: 30
+
+      - name: Teardown
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          quiet_run "tofu workspace select" "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy"          "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          quiet_run "scrub-env"             "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          ok "teardown complete"
@@ -0,0 +1,334 @@
+name: E2E Nightly
+
+on:
+  schedule:
+    - cron: "0 14 * * 0-4" # 00:00 Mon-Fri AEST = 14:00 Sun-Thu UTC
+  workflow_dispatch:
+    inputs:
+      cells:
+        description: "Comma-separated cell numbers to run (blank = all)"
+        required: false
+
+permissions:
+  contents: read
+
+# Scheduled runs are loaded from the default branch (main), so the schedule
+# trigger must stay in sync on main for cron to fire.
+env:
+  GIT_BRANCH: ${{ github.event_name == 'schedule' && 'dev' || github.ref_name }}
+  UBUNTU_GOLD_IMAGE: /mnt/disk1/gold-images/spinifex-image-builder-ubuntu-24.04.qcow2
+
+jobs:
+  matrix:
+    name: "cell-${{ matrix.cell }} (${{ matrix.topo }}/${{ matrix.net }}/${{
+      matrix.host }}${{ matrix.extra && format('/{0}', matrix.extra) || '' }})"
+    strategy:
+      fail-fast: false
+      max-parallel: 3
+      matrix:
+        include:
+          - { cell: 1,  install: binary, topo: single,     net: dhcp,   host: debian-13,    arch: x86_64, runner: ci-single, extra: "" }
+          - { cell: 2,  install: binary, topo: single,     net: static, host: ubuntu-24.04, arch: x86_64, runner: ci-single, extra: "" }
+          - { cell: 3,  install: binary, topo: single,     net: dhcp,   host: ubuntu-24.04, arch: x86_64, runner: ci-single, extra: "" }
+          - { cell: 8,  install: binary, topo: real-multi, net: dhcp,   host: debian-13,    arch: x86_64, runner: ci-multi,  extra: "" }
+          - { cell: 9,  install: binary, topo: real-multi, net: static, host: ubuntu-24.04, arch: x86_64, runner: ci-multi,  extra: "" }
+          - { cell: 10, install: binary, topo: real-multi, net: dhcp,   host: ubuntu-24.04, arch: x86_64, runner: ci-multi,  extra: "" }
+          - { cell: 17, install: binary, topo: single,     net: static, host: debian-13,    arch: x86_64, runner: ci-pseudo, extra: "tofu-examples" }
+          - { cell: 18, install: binary, topo: single,     net: static, host: debian-13,    arch: x86_64, runner: ci-single, extra: "reboot-single" }
+
+    runs-on: [ self-hosted, "${{ matrix.runner }}" ]
+    timeout-minutes: 35
+    env:
+      TOFU_LOG: /tmp/e2e-nightly-cell-${{ matrix.cell }}-${{ github.run_id }}.log
+      ARTIFACT_DIR: /tmp/spinifex-e2e-nightly-cell-${{ matrix.cell }}-${{ github.run_id }}
+
+    steps:
+      # blank or scheduled run = full matrix.
+      # Input must be comma-separated with no whitespace (e.g. '1,2,17', not '1, 2').
+      - name: Gate on Dispatch Input
+        id: gate
+        env:
+          CELL: ${{ matrix.cell }}
+          CELLS: ${{ github.event_name == 'workflow_dispatch' && inputs.cells || '' }}
+        run: |
+          if [ -z "$CELLS" ] || [[ ",${CELLS}," == *",${CELL},"* ]]; then
+            echo "selected=true" >> "$GITHUB_OUTPUT"
+            echo "cell ${CELL} selected"
+          else
+            echo "selected=false" >> "$GITHUB_OUTPUT"
+            echo "cell ${CELL} not in '${CELLS}' — subsequent steps will skip"
+          fi
+
+      - name: Clean Workspace
+        if: steps.gate.outputs.selected == 'true'
+        run: |
+          sudo umount -R -l /var/cache/spinifex-iso-builder/rootfs 2>/dev/null || true
+          if [ ! -c /dev/null ]; then sudo rm -f /dev/null; sudo mknod -m 666 /dev/null c 1 3; fi
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        if: steps.gate.outputs.selected == 'true'
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        if: steps.gate.outputs.selected == 'true'
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - uses: actions/checkout@v6
+        if: steps.gate.outputs.selected == 'true'
+        with:
+          path: spinifex
+
+      - name: Build Install Artifacts
+        if: steps.gate.outputs.selected == 'true'
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VMs
+        if: steps.gate.outputs.selected == 'true'
+        working-directory: mulga/scripts/tofu-cluster
+        env:
+          MATRIX_HOST: ${{ matrix.host }}
+        run: |
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+
+          TOFU_VAR_ARGS=""
+          if [ "$MATRIX_HOST" = "ubuntu-24.04" ]; then
+            TOFU_VAR_ARGS="-var=host_image_override=${UBUNTU_GOLD_IMAGE}"
+          fi
+
+          tofu init -input=false >> "$TOFU_LOG" 2>&1
+          tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
+          tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars ${TOFU_VAR_ARGS} -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
+          ./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
+          tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars ${TOFU_VAR_ARGS} -auto-approve -input=false >> "$TOFU_LOG" 2>&1
+
+      - name: Bootstrap via Binary Install
+        if: steps.gate.outputs.selected == 'true'
+        working-directory: mulga/scripts/tofu-cluster
+        env:
+          MATRIX_NET: ${{ matrix.net }}
+        run: |
+          if [ "$MATRIX_NET" = "dhcp" ]; then
+            unset SPX_EXTERNAL_POOL
+            export SPX_EXTERNAL_SOURCE=dhcp
+          else
+            export SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}"
+          fi
+
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Run Single Node E2E
+        if: steps.gate.outputs.selected == 'true' && matrix.topo == 'single' && matrix.extra == ''
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${WAN_IP} \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; cd \$HOME/e2e-tests/_bin; FAIL=0; \
+             for SUITE in single cert lb; do \
+               echo \"::group::suite \$SUITE\"; \
+               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=single \
+                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
+                 ./\$SUITE.test -test.v -test.timeout=30m \
+                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log'; \
+               rc=\${PIPESTATUS[0]}; echo \"::endgroup::\"; \
+               if [ \$rc -ne 0 ]; then echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; FAIL=1; fi; \
+             done; exit \$FAIL"
+
+      - name: Run Multi Node E2E
+        if: steps.gate.outputs.selected == 'true' && matrix.topo == 'real-multi'
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          NODE1_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+
+          NODES_CSV=""
+          NODE_COUNT=$(echo "$INVENTORY" | jq '.nodes | length')
+          for i in $(seq 0 $((NODE_COUNT - 1))); do
+            ip=$(echo "$INVENTORY" | jq -r ".nodes[$i].wan_ip")
+            NODES_CSV="${NODES_CSV:+$NODES_CSV,}$ip"
+          done
+
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${NODE1_IP} \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; cd \$HOME/e2e-tests/_bin; FAIL=0; \
+             for SUITE in multinode cert lb; do \
+               echo \"::group::suite \$SUITE\"; \
+               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=multinode \
+                 SPINIFEX_NODES='${NODES_CSV}' SPINIFEX_NODE_IPS='${NODES_CSV}' \
+                 SPINIFEX_SSH_USER=tf-user SPINIFEX_SSH_KEY=\$HOME/.ssh/tf-user-ap-southeast-2 \
+                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
+                 ./\$SUITE.test -test.v -test.timeout=30m \
+                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log'; \
+               rc=\${PIPESTATUS[0]}; echo \"::endgroup::\"; \
+               if [ \$rc -ne 0 ]; then echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; FAIL=1; fi; \
+             done; exit \$FAIL"
+
+      - name: Run Tofu Workbooks
+        if: steps.gate.outputs.selected == 'true' && matrix.extra == 'tofu-examples'
+        working-directory: mulga/scripts/tofu-cluster
+        # The driver emits go test -v markers per workbook; teeing to
+        # test-tofu.log feeds the same go-junit-report -> e2e-analyze -> upload
+        # chain the Go suites use, so cell 17 produces an RCA bundle too.
+        run: |
+          set +e
+          mkdir -p "${ARTIFACT_DIR}"
+          INVENTORY=$(tofu output -json inventory)
+          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          date -u +%Y-%m-%dT%H:%M:%SZ > "${ARTIFACT_DIR}/test-tofu.start"
+          ssh -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${WAN_IP} \
+            "cd ~/e2e-tests && ./run-tofu-examples-e2e.sh" \
+            2>&1 | tee "${ARTIFACT_DIR}/test-tofu.log"
+          rc=${PIPESTATUS[0]}
+          if [ $rc -ne 0 ]; then echo "tofu: FAILED" >> "${ARTIFACT_DIR}/suite-status.txt"; fi
+          exit $rc
+
+      - name: Run Reboot Resilience E2E
+        if: steps.gate.outputs.selected == 'true' && matrix.extra == 'reboot-single'
+        working-directory: mulga/scripts/tofu-cluster
+        # reboot.test runs from the runner (NOT via ssh-and-run) so it survives the
+        # systemctl-reboot-induced ssh disconnect mid-run.
+        run: |
+          set +e
+          mkdir -p "${ARTIFACT_DIR}"
+          INVENTORY=$(tofu output -json inventory)
+          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+
+          REBOOT_DIR="${RUNNER_TEMP}/reboot-e2e"
+          mkdir -p "$REBOOT_DIR"
+          tar xf "${RUNNER_TEMP}/spinifex-tests.tar" -C "$REBOOT_DIR"
+          chmod +x "$REBOOT_DIR/_bin/reboot.test"
+
+          SSH_OPTS=(-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i "$SSH_KEY")
+          CREDS=$(ssh "${SSH_OPTS[@]}" tf-user@${WAN_IP} 'cat ~/.aws/credentials')
+          AWS_AK=$(echo "$CREDS" | awk -F '= *' '/^aws_access_key_id/ {print $2; exit}')
+          AWS_SK=$(echo "$CREDS" | awk -F '= *' '/^aws_secret_access_key/ {print $2; exit}')
+          echo "::add-mask::${AWS_AK}"; echo "::add-mask::${AWS_SK}"
+
+          date -u +%Y-%m-%dT%H:%M:%SZ > "${ARTIFACT_DIR}/test-reboot.start"
+          GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=single \
+            SPINIFEX_WAN_IP="${WAN_IP}" SPINIFEX_SSH_KEY="${SSH_KEY}" \
+            SPINIFEX_AWS_ENDPOINT="https://${WAN_IP}:9999" \
+            SPINIFEX_AWS_ACCESS_KEY_ID="${AWS_AK}" SPINIFEX_AWS_SECRET_ACCESS_KEY="${AWS_SK}" \
+            ARTIFACT_DIR="${ARTIFACT_DIR}" \
+            "$REBOOT_DIR/_bin/reboot.test" -test.v -test.timeout=30m \
+            2>&1 | tee "${ARTIFACT_DIR}/test-reboot.log"
+          rc=${PIPESTATUS[0]}
+          if [ $rc -ne 0 ]; then echo "reboot: FAILED" >> "${ARTIFACT_DIR}/suite-status.txt"; fi
+          exit $rc
+
+      - name: Collect e2e artifacts
+        if: always() && steps.gate.outputs.selected == 'true'
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          mkdir -p "${ARTIFACT_DIR}"
+          INVENTORY=$(tofu output -json inventory 2>/dev/null) || { echo "no inventory; skipping"; exit 0; }
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path'); SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          NODE_COUNT=$(echo "$INVENTORY" | jq '.nodes | length')
+          NODE1_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_OPTS=(-o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "$SSH_KEY")
+
+          ssh "${SSH_OPTS[@]}" tf-user@${NODE1_IP} "tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
+            | tar -C "${ARTIFACT_DIR}" -xf - 2>/dev/null
+
+          for i in $(seq 0 $((NODE_COUNT - 1))); do
+            IP=$(echo "$INVENTORY" | jq -r ".nodes[$i].wan_ip"); idx=$((i + 1))
+            mkdir -p "${ARTIFACT_DIR}/vm-console-node${idx}"
+            ssh "${SSH_OPTS[@]}" tf-user@${IP} 'sudo sh -c "cd /run/spinifex 2>/dev/null && tar -cf - console-*.log 2>/dev/null"' \
+              | tar -C "${ARTIFACT_DIR}/vm-console-node${idx}" -xf - 2>/dev/null
+            ssh "${SSH_OPTS[@]}" tf-user@${IP} 'sudo journalctl -u "spinifex-*" --no-pager --output=short-iso --since "60 min ago"' \
+              > "${ARTIFACT_DIR}/spinifex-journal-node${idx}.log" 2>/dev/null
+          done
+
+          cat "${ARTIFACT_DIR}"/spinifex-journal-node*.log \
+            > "${ARTIFACT_DIR}/spinifex-journal.log" 2>/dev/null || true
+          echo "collected: $(ls "${ARTIFACT_DIR}" 2>/dev/null | wc -l) entries"
+
+      - name: Install go-junit-report
+        if: always() && steps.gate.outputs.selected == 'true'
+        run: |
+          go install github.com/jstemmer/go-junit-report/v2@latest
+          echo "$HOME/go/bin" >> "$GITHUB_PATH"
+
+      - name: Convert test logs to JUnit XML
+        if: always() && steps.gate.outputs.selected == 'true'
+        run: |
+          shopt -s nullglob
+          for LOG in "${ARTIFACT_DIR}"/test-*.log; do
+            BASE=$(basename "$LOG" .log); SUITE=${BASE#test-}
+            go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-${SUITE}.xml" || true
+          done
+
+      - name: Analyze e2e failures
+        if: always() && steps.gate.outputs.selected == 'true'
+        uses: ./spinifex/.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "E2E failure analysis (cell ${{ matrix.cell }})"
+
+      - uses: actions/upload-artifact@v7
+        if: always() && steps.gate.outputs.selected == 'true'
+        with:
+          name: e2e-artifacts-cell-${{ matrix.cell }}-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Teardown
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        env:
+          MATRIX_HOST: ${{ matrix.host }}
+        run: |
+          TOFU_VAR_ARGS=""
+          if [ "$MATRIX_HOST" = "ubuntu-24.04" ]; then
+            TOFU_VAR_ARGS="-var=host_image_override=${UBUNTU_GOLD_IMAGE}"
+          fi
+          tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
+          tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars ${TOFU_VAR_ARGS} -auto-approve -input=false >> "$TOFU_LOG" 2>&1
+          ./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
+
+  # Bare-metal single-node suite. Doesn't fit the tofu matrix shape, so it
+  # runs as a sibling job via the reusable e2e-baremetal workflow. Gated like a
+  # matrix cell ("cell 20"): every weeknight cron, or dispatch with 20 selected
+  # (blank cells = all). Tests `dev` on schedule to match GIT_BRANCH above.
+  baremetal:
+    name: cell-20 (baremetal/pxe/real-hw)
+    if: >-
+      github.event_name == 'schedule' ||
+      inputs.cells == '' ||
+      contains(format(',{0},', inputs.cells), ',20,')
+    permissions:
+      contents: read
+      statuses: write
+    uses: ./.github/workflows/e2e-baremetal.yml
+    with:
+      ref: ${{ github.event_name == 'schedule' && 'dev' || github.ref_name }}
+    secrets: inherit
@@ -0,0 +1,799 @@
+# E2E (per-suite blocks) — EXPERIMENTAL, workflow_dispatch only.
+#
+# Restructures the looped e2e.yml into env-grouped job blocks so each suite
+# surfaces as its own node in the Actions graph (mulga-siv-283):
+#
+#   single_provision ─┬─ single_suite (matrix: single, cert, lb, eks)
+#                     └─ single_teardown (if: always)
+#   multi_provision  ─┬─ multi_suite  (matrix: multinode)
+#                     └─ multi_teardown (if: always)
+#
+# Constraint: tofu state is runner-filesystem-local, so *_provision and
+# *_teardown must execute on the same self-hosted runner (one runner per label
+# today). Teardown reuses the persisted mulga/ tofu-cluster workspace and does
+# NOT re-checkout mulga (that would wipe local state). Suite-matrix legs reach
+# the already-provisioned VM over its WAN IP using the key path the provision
+# job emits; they share the runner home dir, so the key file persists.
+#
+# This file does NOT post the gating `e2e` commit status — it runs alongside
+# e2e.yml for validation. Once green across a few manual runs, fold the
+# triggers from e2e.yml here and retire the loop.
+
+name: E2E (suite blocks)
+
+on:
+  workflow_dispatch:
+    inputs:
+      single_suites:
+        description: "Single-node suites to run as matrix legs"
+        required: false
+        default: "single cert lb eks"
+      run_multinode:
+        description: "Also run the multinode env block"
+        type: boolean
+        required: false
+        default: true
+      test_run:
+        description: "Optional -test.run regex (empty = all tests in suite)"
+        required: false
+        default: ""
+
+permissions:
+  contents: read
+
+env:
+  GIT_BRANCH: ${{ github.ref_name }}
+  TEST_RUN: ${{ inputs.test_run || '' }}
+
+jobs:
+  # ---------------------------------------------------------------------------
+  # Single-node env
+  # ---------------------------------------------------------------------------
+  single_provision:
+    name: Single env · provision
+    runs-on: [ self-hosted, ci-single ]
+    timeout-minutes: 40
+    outputs:
+      wan_ip: ${{ steps.vm.outputs.wan_ip }}
+      ssh_key: ${{ steps.vm.outputs.ssh_key }}
+      suites: ${{ steps.suites.outputs.list }}
+      matrix: ${{ steps.suites.outputs.matrix }}
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-single-${{ github.run_id }}.log
+    steps:
+      - name: Reap Orphan Procs
+        run: |
+          set +e
+          UID_SELF=$(id -u)
+          WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in
+                "$WS"|"$WS"/*) echo "reap $sig pid=$pid cwd=$cwd"; kill "-$sig" "$pid" 2>/dev/null ;;
+              esac
+            done
+          }
+          reap TERM; sleep 3; reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          true
+
+      - name: Clean Workspace
+        run: |
+          awk -v w="$GITHUB_WORKSPACE" '$2 ~ "^"w {print $2}' /proc/mounts \
+            | sort -r | xargs -r -I{} sudo umount -l -R {} || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Re-enable Go toolchain auto-download
+        run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
+
+      - name: Compute suite matrix
+        id: suites
+        run: |
+          LIST="${{ inputs.single_suites || 'single cert lb eks' }}"
+          echo "list=${LIST}" >> "$GITHUB_OUTPUT"
+          JSON=$(printf '%s\n' $LIST | jq -R . | jq -cs .)
+          echo "matrix=${JSON}" >> "$GITHUB_OUTPUT"
+
+      - name: Reclaim docker build cache
+        run: |
+          docker container prune -f || true
+          docker image prune -f || true
+          docker buildx prune -af --filter "until=24h" || true
+          docker system df || true
+
+      - name: Build Install Artifacts
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VM
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+          quiet_run "tofu init"                "$TOFU_LOG" tofu init -input=false
+          quiet_run "tofu workspace select"    "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy (pre-clean)" "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
+          quiet_run "scrub-env (pre-clean)"    "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          quiet_run "tofu apply"               "$TOFU_LOG" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          ok "VM provisioned"
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}" \
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Resolve VM SSH details
+        id: vm
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          echo "wan_ip=${WAN_IP}" >> "$GITHUB_OUTPUT"
+          echo "ssh_key=${SSH_KEY}" >> "$GITHUB_OUTPUT"
+
+      - name: Build eks-node AMI (libguestfs, runner)
+        if: contains(steps.suites.outputs.list, 'eks')
+        working-directory: spinifex
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          banner "build eks-node AMI (libguestfs — no host devices)"
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends libguestfs-tools qemu-utils
+          export LIBGUESTFS_BACKEND=direct
+          sudo chmod 0644 /boot/vmlinuz-* || true
+          group_open "make build-eks-node-image"
+          make build-eks-node-image
+          group_close
+          ok "raw: $(du -h /tmp/eks-node-image-build/eks-node-alpine.raw | cut -f1)"
+
+      - name: Import eks-node AMI on VM
+        if: contains(steps.suites.outputs.list, 'eks')
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          eval "$(grep -E '^(ALPINE_VERSION|AMI_NAME|SYSTEM_TAG)=' \
+            spinifex/scripts/images/eks-node/manifest.conf)"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          banner "import eks-node AMI to VM image store"
+          scp "${SSH_OPTS[@]}" \
+            /tmp/eks-node-image-build/eks-node-alpine.raw \
+            "$VM":/tmp/eks-node-alpine.raw
+          ssh "${SSH_OPTS[@]}" "$VM" \
+            "spx admin images import --file /tmp/eks-node-alpine.raw \
+               --distro alpine --version ${ALPINE_VERSION} --arch x86_64 \
+               --boot-mode bios --ami-name ${AMI_NAME} --tag ${SYSTEM_TAG}" >/dev/null
+          ok "eks-node AMI imported"
+
+      - name: Install kubectl on VM
+        if: contains(steps.suites.outputs.list, 'eks')
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          banner "install kubectl on VM (eks data-plane subtest)"
+          ssh "${SSH_OPTS[@]}" "$VM" 'bash -s' <<'EOF'
+          set -euo pipefail
+          KVER=v1.32.5
+          case "$(uname -m)" in
+            x86_64)  ARCH=amd64 ;;
+            aarch64) ARCH=arm64 ;;
+            *) echo "unsupported arch $(uname -m)" >&2; exit 1 ;;
+          esac
+          cd /tmp
+          curl -fsSLO "https://dl.k8s.io/release/${KVER}/bin/linux/${ARCH}/kubectl"
+          curl -fsSLO "https://dl.k8s.io/release/${KVER}/bin/linux/${ARCH}/kubectl.sha256"
+          echo "$(cat kubectl.sha256)  kubectl" | sha256sum -c -
+          sudo install -m 0755 kubectl /usr/local/bin/kubectl
+          kubectl version --client
+          EOF
+          ok "kubectl installed on VM"
+
+      - name: Build E2E test binaries on runner
+        working-directory: spinifex/tests/e2e
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          # teardown always builds: the run-scoped sweep binary runs in the
+          # teardown job, it is never a selectable suite leg.
+          SUITES_BUILD="${{ steps.suites.outputs.list }} teardown"
+          banner "build ${SUITES_BUILD// /, }"
+          group_open "go test -c"
+          make e2e-build SUITES_BUILD="${SUITES_BUILD}"
+          group_close
+          for f in _bin/*.test; do
+            ok "$(basename "$f") $(stat -c '%s' "$f" | numfmt --to=iec)"
+          done
+
+      - name: Ship test binaries to VM
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "rm -rf \$HOME/spinifex-e2e-bin && mkdir -p \$HOME/spinifex-e2e-bin"
+          tar -C spinifex/tests/e2e/_bin -cf - . | ssh "${SSH_OPTS[@]}" "$VM" "tar -xf - -C \$HOME/spinifex-e2e-bin"
+          ok "shipped binaries to $VM"
+
+      - name: Start packet capture on VM br-wan
+        run: |
+          set +e
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          scp "${SSH_OPTS[@]}" \
+            "${GITHUB_WORKSPACE}/spinifex/.github/scripts/start-net-capture.sh" \
+            "$VM":/tmp/start-net-capture.sh
+          ssh "${SSH_OPTS[@]}" "$VM" "bash /tmp/start-net-capture.sh" || true
+
+  single_suite:
+    name: Single env · suite
+    needs: single_provision
+    runs-on: [ self-hosted, ci-single ]
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        suite: ${{ fromJSON(needs.single_provision.outputs.matrix) }}
+    env:
+      WAN_IP: ${{ needs.single_provision.outputs.wan_ip }}
+      SSH_KEY: ${{ needs.single_provision.outputs.ssh_key }}
+      ARTIFACT_DIR: /tmp/spinifex-e2e-single-${{ github.run_id }}-${{ matrix.suite }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Install go-junit-report
+        run: |
+          go install github.com/jstemmer/go-junit-report/v2@latest
+          echo "$HOME/go/bin" >> "$GITHUB_PATH"
+
+      - name: Run ${{ matrix.suite }}
+        id: e2e
+        run: |
+          set +e
+          SUITE="${{ matrix.suite }}"
+          VM_HOST="tf-user@${WAN_IP}"
+          TEST_RUN_FLAG=""
+          [ -n "${TEST_RUN}" ] && TEST_RUN_FLAG="-test.run=${TEST_RUN}"
+          cleanup_remote() {
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "$SSH_KEY" "$VM_HOST" \
+              "pkill -u tf-user -TERM -f \"/spinifex-e2e-bin/${SUITE}\.test( |\$)\" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f \"/spinifex-e2e-bin/${SUITE}\.test( |\$)\" 2>/dev/null; true" \
+              2>/dev/null || true
+          }
+          trap cleanup_remote EXIT INT TERM
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" "$VM_HOST" \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; cd \$HOME/spinifex-e2e-bin; \
+             R=\$'\033[0m' B=\$'\033[1m' C=\$'\033[36m' G=\$'\033[32m' RED=\$'\033[31m'; \
+             printf '\n%s%s━━ suite %s ━━%s\n' \"\$B\" \"\$C\" \"${SUITE}\" \"\$R\"; \
+             date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-${SUITE}.start'; \
+             GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=single \
+               SPINIFEX_E2E_RUN_ID='${{ github.run_id }}' \
+               ARTIFACT_DIR='${ARTIFACT_DIR}' \
+               ./${SUITE}.test -test.v -test.timeout=30m ${TEST_RUN_FLAG} \
+               2>&1 | tee '${ARTIFACT_DIR}/test-${SUITE}.log' \
+                    | sed -uE 's/^[[:space:]]+([a-zA-Z0-9_]+_test\.go:[0-9]+: )?//'; \
+             exit \${PIPESTATUS[0]}"
+          echo "rc=$?" >> "$GITHUB_OUTPUT"
+          exit 0
+
+      - name: Collect suite artifacts from VM
+        if: always()
+        run: |
+          set +e
+          mkdir -p "${ARTIFACT_DIR}"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${SSH_KEY}")
+          VM=tf-user@${WAN_IP}
+          ssh "${SSH_OPTS[@]}" "$VM" "tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
+            | tar -C "${ARTIFACT_DIR}" -xf - 2>/dev/null
+          true
+
+      - name: Convert test log to JUnit XML
+        if: always()
+        run: |
+          shopt -s nullglob
+          for LOG in "${ARTIFACT_DIR}"/test-*.log; do
+            BASE=$(basename "$LOG" .log); SUITE=${BASE#test-}
+            go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-${SUITE}.xml" || true
+          done
+
+      - name: Publish test summary
+        if: always()
+        uses: ./spinifex/.github/actions/junit-report
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          title: "E2E ${{ matrix.suite }} results"
+
+      - name: Analyze e2e failures
+        if: always()
+        uses: ./spinifex/.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "E2E ${{ matrix.suite }} failure analysis"
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-single-${{ matrix.suite }}-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Fail job if suite failed
+        if: always()
+        run: |
+          if [ "${{ steps.e2e.outputs.rc }}" != "0" ]; then
+            echo "::error::suite ${{ matrix.suite }} failed (rc=${{ steps.e2e.outputs.rc }})"
+            exit 1
+          fi
+
+  single_teardown:
+    name: Single env · teardown
+    needs: [ single_provision, single_suite ]
+    if: always()
+    runs-on: [ self-hosted, ci-single ]
+    timeout-minutes: 20
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-single-td-${{ github.run_id }}.log
+      ARTIFACT_DIR: /tmp/spinifex-e2e-single-diag-${{ github.run_id }}
+      WAN_IP: ${{ needs.single_provision.outputs.wan_ip }}
+      SSH_KEY: ${{ needs.single_provision.outputs.ssh_key }}
+    steps:
+      # Reuse the persisted mulga/ tofu-cluster workspace from single_provision.
+      # Re-checkout only spinifex (for ci-fmt + diag scripts); checking out mulga
+      # again would wipe the local tofu state and orphan the VM.
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Capture network/IPsec diagnostics on VM
+        if: ${{ needs.single_provision.result == 'success' && env.WAN_IP != '' }}
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          mkdir -p "${ARTIFACT_DIR}"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "${SSH_KEY}")
+          VM=tf-user@${WAN_IP}
+          scp "${SSH_OPTS[@]}" \
+            "${GITHUB_WORKSPACE}/spinifex/.github/scripts/capture-net-diag.sh" \
+            "$VM":/tmp/capture-net-diag.sh 2>/dev/null
+          ssh "${SSH_OPTS[@]}" "$VM" "bash /tmp/capture-net-diag.sh '${ARTIFACT_DIR}/net-diag'" 2>/dev/null
+          ssh "${SSH_OPTS[@]}" "$VM" 'sudo journalctl -u "spinifex-*" --no-pager --output=short-iso --since "90 min ago"' \
+            > "${ARTIFACT_DIR}/spinifex-journal.log" 2>/dev/null
+          mkdir -p "${ARTIFACT_DIR}/lb-console"
+          ssh "${SSH_OPTS[@]}" "$VM" 'sudo sh -c "cd /run/spinifex 2>/dev/null && tar -cf - console-*.log 2>/dev/null"' \
+            | tar -C "${ARTIFACT_DIR}/lb-console" -xf - 2>/dev/null
+          true
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-single-diag-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: ignore
+          retention-days: 14
+
+      # Reclaim any resource leaked by a crashed/interrupted suite before the VM
+      # is destroyed. Scoped to this run's tag, best-effort, never fails the job.
+      - name: Sweep leaked run resources on VM
+        if: ${{ needs.single_provision.result == 'success' && env.WAN_IP != '' }}
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "${SSH_KEY}")
+          VM=tf-user@${WAN_IP}
+          banner "sweep run-tagged resources (run ${{ github.run_id }})"
+          ssh "${SSH_OPTS[@]}" "$VM" \
+            "cd \$HOME/spinifex-e2e-bin 2>/dev/null && \
+             SPINIFEX_E2E=1 SPINIFEX_MODE=single \
+             SPINIFEX_E2E_RUN_ID='${{ github.run_id }}' \
+             ./teardown.test -test.v -test.run TestSweepRunResources -test.timeout=10m" \
+            2>&1 || true
+          true
+
+      - name: Teardown VM
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          if [ -n "${WAN_IP}" ] && [ -n "${SSH_KEY}" ]; then
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "${SSH_KEY}" \
+              tf-user@${WAN_IP} \
+              'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+              2>/dev/null || true
+          fi
+          UID_SELF=$(id -u); WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in "$WS"|"$WS"/*) kill "-$sig" "$pid" 2>/dev/null ;; esac
+            done
+          }
+          reap TERM; sleep 2; reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          quiet_run "tofu workspace select" "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy"          "$TOFU_LOG" timeout 600 tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          quiet_run "scrub-env"             "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}"
+          ok "teardown complete"
+
+  # ---------------------------------------------------------------------------
+  # Multinode env
+  # ---------------------------------------------------------------------------
+  multi_provision:
+    name: Multi env · provision
+    if: ${{ inputs.run_multinode }}
+    runs-on: [ self-hosted, ci-multi ]
+    timeout-minutes: 45
+    outputs:
+      node1_ip: ${{ steps.cluster.outputs.node1_ip }}
+      ssh_key: ${{ steps.cluster.outputs.ssh_key }}
+      nodes_csv: ${{ steps.cluster.outputs.nodes_csv }}
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-multi-${{ github.run_id }}.log
+    steps:
+      - name: Reap Orphan Procs
+        run: |
+          set +e
+          UID_SELF=$(id -u); WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in "$WS"|"$WS"/*) echo "reap $sig pid=$pid"; kill "-$sig" "$pid" 2>/dev/null ;; esac
+            done
+          }
+          reap TERM; sleep 3; reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          true
+
+      - name: Clean Workspace
+        run: |
+          awk -v w="$GITHUB_WORKSPACE" '$2 ~ "^"w {print $2}' /proc/mounts \
+            | sort -r | xargs -r -I{} sudo umount -l -R {} || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Re-enable Go toolchain auto-download
+        run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
+
+      - name: Reclaim docker build cache
+        run: |
+          docker container prune -f || true
+          docker image prune -f || true
+          docker buildx prune -af --filter "until=24h" || true
+          docker system df || true
+
+      - name: Build Install Artifacts
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VMs
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+          quiet_run "tofu init"                "$TOFU_LOG" tofu init -input=false
+          quiet_run "tofu workspace select"    "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy (pre-clean)" "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
+          quiet_run "scrub-env (pre-clean)"    "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          quiet_run "tofu apply"               "$TOFU_LOG" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          ok "3-node cluster provisioned"
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}" \
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Resolve cluster SSH details
+        id: cluster
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          NODE1_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path'); SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          NODE_COUNT=$(echo "$INVENTORY" | jq '.nodes | length')
+          NODES_CSV=""
+          for i in $(seq 0 $((NODE_COUNT - 1))); do
+            ip=$(echo "$INVENTORY" | jq -r ".nodes[$i].wan_ip")
+            if [ -z "$NODES_CSV" ]; then NODES_CSV="$ip"; else NODES_CSV="$NODES_CSV,$ip"; fi
+          done
+          echo "node1_ip=${NODE1_IP}" >> "$GITHUB_OUTPUT"
+          echo "ssh_key=${SSH_KEY}" >> "$GITHUB_OUTPUT"
+          echo "nodes_csv=${NODES_CSV}" >> "$GITHUB_OUTPUT"
+
+      - name: Build multinode binary on runner
+        working-directory: spinifex/tests/e2e
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          banner "build multinode, teardown"
+          group_open "go test -c"
+          make e2e-build SUITES_BUILD="multinode teardown"
+          group_close
+          ok "$(basename _bin/multinode.test) $(stat -c '%s' _bin/multinode.test | numfmt --to=iec)"
+
+      - name: Ship test binaries to node1
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.cluster.outputs.node1_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "rm -rf \$HOME/spinifex-e2e-bin && mkdir -p \$HOME/spinifex-e2e-bin"
+          tar -C spinifex/tests/e2e/_bin -cf - . | ssh "${SSH_OPTS[@]}" "$VM" "tar -xf - -C \$HOME/spinifex-e2e-bin"
+          ok "shipped binaries to $VM"
+
+      - name: Start packet capture on all nodes
+        run: |
+          set +e
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          IFS=',' read -ra NODE_LIST <<< "${{ steps.cluster.outputs.nodes_csv }}"
+          for ip in "${NODE_LIST[@]}"; do
+            [ -z "$ip" ] && continue
+            scp "${SSH_OPTS[@]}" \
+              "${GITHUB_WORKSPACE}/spinifex/.github/scripts/start-net-capture.sh" \
+              tf-user@"$ip":/tmp/start-net-capture.sh
+            ssh "${SSH_OPTS[@]}" tf-user@"$ip" "bash /tmp/start-net-capture.sh" || true
+          done
+
+  multi_suite:
+    name: Multi env · suite
+    needs: multi_provision
+    runs-on: [ self-hosted, ci-multi ]
+    timeout-minutes: 60
+    strategy:
+      fail-fast: false
+      matrix:
+        suite: [ multinode ]
+    env:
+      NODE1_IP: ${{ needs.multi_provision.outputs.node1_ip }}
+      SSH_KEY: ${{ needs.multi_provision.outputs.ssh_key }}
+      NODES_CSV: ${{ needs.multi_provision.outputs.nodes_csv }}
+      ARTIFACT_DIR: /tmp/spinifex-e2e-multi-${{ github.run_id }}-${{ matrix.suite }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Install go-junit-report
+        run: |
+          go install github.com/jstemmer/go-junit-report/v2@latest
+          echo "$HOME/go/bin" >> "$GITHUB_PATH"
+
+      - name: Run ${{ matrix.suite }}
+        id: e2e
+        run: |
+          set +e
+          SUITE="${{ matrix.suite }}"
+          VM_HOST="tf-user@${NODE1_IP}"
+          TEST_RUN_FLAG=""
+          [ -n "${TEST_RUN}" ] && TEST_RUN_FLAG="-test.run=${TEST_RUN}"
+          cleanup_remote() {
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "$SSH_KEY" "$VM_HOST" \
+              "pkill -u tf-user -TERM -f \"/spinifex-e2e-bin/${SUITE}\.test( |\$)\" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f \"/spinifex-e2e-bin/${SUITE}\.test( |\$)\" 2>/dev/null; true" \
+              2>/dev/null || true
+          }
+          trap cleanup_remote EXIT INT TERM
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" "$VM_HOST" \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; cd \$HOME/spinifex-e2e-bin; \
+             printf '\n━━ suite %s ━━\n' \"${SUITE}\"; \
+             date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-${SUITE}.start'; \
+             GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=multinode \
+               SPINIFEX_E2E_RUN_ID='${{ github.run_id }}' \
+               SPINIFEX_NODES='${NODES_CSV}' SPINIFEX_NODE_IPS='${NODES_CSV}' \
+               SPINIFEX_SSH_USER=tf-user SPINIFEX_SSH_KEY=\$HOME/.ssh/tf-user-ap-southeast-2 \
+               ARTIFACT_DIR='${ARTIFACT_DIR}' \
+               ./${SUITE}.test -test.v -test.timeout=30m ${TEST_RUN_FLAG} \
+               2>&1 | tee '${ARTIFACT_DIR}/test-${SUITE}.log' \
+                    | sed -uE 's/^[[:space:]]+([a-zA-Z0-9_]+_test\.go:[0-9]+: )?//'; \
+             exit \${PIPESTATUS[0]}"
+          echo "rc=$?" >> "$GITHUB_OUTPUT"
+          exit 0
+
+      - name: Collect suite artifacts from node1
+        if: always()
+        run: |
+          set +e
+          mkdir -p "${ARTIFACT_DIR}"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${SSH_KEY}")
+          ssh "${SSH_OPTS[@]}" tf-user@${NODE1_IP} "tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
+            | tar -C "${ARTIFACT_DIR}" -xf - 2>/dev/null
+          true
+
+      - name: Convert test log to JUnit XML
+        if: always()
+        run: |
+          shopt -s nullglob
+          for LOG in "${ARTIFACT_DIR}"/test-*.log; do
+            BASE=$(basename "$LOG" .log); SUITE=${BASE#test-}
+            go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-${SUITE}.xml" || true
+          done
+
+      - name: Publish test summary
+        if: always()
+        uses: ./spinifex/.github/actions/junit-report
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          title: "Multi-Node E2E ${{ matrix.suite }} results"
+
+      - name: Analyze e2e failures
+        if: always()
+        uses: ./spinifex/.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "Multi-Node E2E ${{ matrix.suite }} failure analysis"
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-multi-${{ matrix.suite }}-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Fail job if suite failed
+        if: always()
+        run: |
+          if [ "${{ steps.e2e.outputs.rc }}" != "0" ]; then
+            echo "::error::suite ${{ matrix.suite }} failed (rc=${{ steps.e2e.outputs.rc }})"
+            exit 1
+          fi
+
+  multi_teardown:
+    name: Multi env · teardown
+    needs: [ multi_provision, multi_suite ]
+    if: always() && inputs.run_multinode
+    runs-on: [ self-hosted, ci-multi ]
+    timeout-minutes: 20
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-multi-td-${{ github.run_id }}.log
+      SSH_KEY: ${{ needs.multi_provision.outputs.ssh_key }}
+      NODES_CSV: ${{ needs.multi_provision.outputs.nodes_csv }}
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      # Reclaim leaked run-tagged resources before the cluster is destroyed.
+      - name: Sweep leaked run resources on node1
+        if: ${{ needs.multi_provision.result == 'success' && needs.multi_provision.outputs.node1_ip != '' }}
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "${SSH_KEY}")
+          VM=tf-user@${{ needs.multi_provision.outputs.node1_ip }}
+          banner "sweep run-tagged resources (run ${{ github.run_id }})"
+          ssh "${SSH_OPTS[@]}" "$VM" \
+            "cd \$HOME/spinifex-e2e-bin 2>/dev/null && \
+             SPINIFEX_E2E=1 SPINIFEX_MODE=multinode \
+             SPINIFEX_E2E_RUN_ID='${{ github.run_id }}' \
+             ./teardown.test -test.v -test.run TestSweepRunResources -test.timeout=10m" \
+            2>&1 || true
+          true
+
+      - name: Teardown cluster
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          if [ -n "${NODES_CSV}" ] && [ -n "${SSH_KEY}" ]; then
+            IFS=',' read -ra NODE_LIST <<< "${NODES_CSV}"
+            for ip in "${NODE_LIST[@]}"; do
+              ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "${SSH_KEY}" \
+                tf-user@"$ip" \
+                'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+                2>/dev/null || true
+            done
+          fi
+          UID_SELF=$(id -u); WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in "$WS"|"$WS"/*) kill "-$sig" "$pid" 2>/dev/null ;; esac
+            done
+          }
+          reap TERM; sleep 2; reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          quiet_run "tofu workspace select" "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy"          "$TOFU_LOG" timeout 600 tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          quiet_run "scrub-env"             "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}"
+          ok "teardown complete"
+
+  e2e_suites_status:
+    name: E2E suite blocks · summary
+    if: always()
+    needs: [ single_suite, multi_suite ]
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    steps:
+      - name: Summarize
+        run: |
+          echo "## E2E suite-block results" >> "$GITHUB_STEP_SUMMARY"
+          echo "single_suite: ${{ needs.single_suite.result }}" >> "$GITHUB_STEP_SUMMARY"
+          echo "multi_suite:  ${{ needs.multi_suite.result }}" >> "$GITHUB_STEP_SUMMARY"
+          case "${{ needs.single_suite.result }}" in success|skipped) ;; *) exit 1 ;; esac
+          case "${{ needs.multi_suite.result }}" in success|skipped) ;; *) exit 1 ;; esac
@@ -1,70 +1,915 @@
-name: Hive E2E Test
+name: E2E

 on:
  push:
-    branches: [main, e2e]
+    branches:
+      - main
+      - dev
+    paths:
+      - "**/*.go"
+      - go.mod
+      - go.sum
+      - Makefile
+      - tests/e2e/**
+      - .github/workflows/e2e.yml
  workflow_dispatch:
+    inputs:
+      sha:
+        description: "PR head SHA to report status on"
+        required: false
+      suites:
+        description: "Space-separated Make targets to run (e.g. 'e2e-single e2e-multinode e2e-cert e2e-lb e2e-eks')"
+        required: false
+        default: "e2e-single e2e-multinode e2e-cert e2e-lb"
+      test_run:
+        description: "Optional regex for -test.run (e.g. '^TestMultinodeIPSec$'). Empty = run all tests in suite."
+        required: false
+        default: ""
+
+permissions:
+  contents: read
+  statuses: write
+
+env:
+  GIT_BRANCH: ${{ github.ref_name }}
+  SUITES: ${{ inputs.suites || 'e2e-single e2e-multinode e2e-cert e2e-lb' }}
+  TEST_RUN: ${{ inputs.test_run || '' }}

 jobs:
-  build_e2e_image:
-    name: Build E2E Image
-    runs-on: self-hosted
-
+  # Shadow-mode suite selector. Computes which suites the diff *would* run and logs it to the job summary.
+  # Does NOT gate or override the real SUITES env yet — it runs alongside the full matrix so we
+  # can compare predicted vs actual before flipping it on.
+  select_suites:
+    name: Select Suites (shadow)
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
    steps:
-      - name: Checkout Code
-        uses: actions/checkout@v6
+      - uses: actions/checkout@v6
        with:
-          path: hive
-
-      - name: Clone Dependencies
-        run: ./hive/scripts/clone-deps.sh
-
-      - name: Build E2E Image
+          fetch-depth: 0
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: go.mod
+          cache: false
+      - name: Compute targeted suite selection
        run: |
-          docker build \
-            -t hive-e2e:latest \
-            -f hive/tests/e2e/Dockerfile.e2e \
-            .
-
-  hive_e2e_test:
-    name: Hive Single-Node E2E Test
-    runs-on: self-hosted
-    needs: build_e2e_image
+          set -euo pipefail
+          BASE="${{ github.event.before }}"
+          HEAD="${{ github.sha }}"
+          # First push to a branch reports an all-zero before-SHA; fall back to
+          # the first parent so the diff is still meaningful.
+          if [ -z "$BASE" ] || [ "$BASE" = "0000000000000000000000000000000000000000" ]; then
+            BASE="${HEAD}~1"
+          fi
+          echo "## Targeted suite selection (shadow)" >> "$GITHUB_STEP_SUMMARY"
+          echo '```' >> "$GITHUB_STEP_SUMMARY"
+          echo "base=$BASE head=$HEAD" >> "$GITHUB_STEP_SUMMARY"
+          go run ./tests/e2e/select-suites/cmd/select-suites \
+            --manifest docs/service-interfaces.yaml \
+            --base-sha "$BASE" --head-sha "$HEAD" --repo-root . \
+            2>>"$GITHUB_STEP_SUMMARY" \
+            | tee -a "$GITHUB_STEP_SUMMARY"
+          echo '```' >> "$GITHUB_STEP_SUMMARY"
+          echo "Shadow only — actual run uses SUITES='${SUITES}'." >> "$GITHUB_STEP_SUMMARY"

+  e2e_single:
+    name: Single-Node E2E
+    # Skip when the dispatch input requests no in-VM suites (e.g.
+    # suites='e2e-multinode' only). Push triggers and the default SUITES list
+    # both include single/cert/lb so the inputs.suites null-check keeps the
+    # job running on plain PR pushes.
+    if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.suites, 'e2e-single') || contains(github.event.inputs.suites, 'e2e-cert') || contains(github.event.inputs.suites, 'e2e-lb') || contains(github.event.inputs.suites, 'e2e-eks')
+    runs-on: [self-hosted, ci-single]
+    timeout-minutes: 45
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-single-${{ github.run_id }}.log
+      ARTIFACT_DIR: /tmp/spinifex-e2e-artifacts-${{ github.run_id }}
    steps:
-      - name: Run E2E Tests
+      - name: Reap Orphan Procs
        run: |
-          docker run --privileged --rm \
-            -v /dev/kvm:/dev/kvm \
-            --name hive-e2e-test \
-            hive-e2e:latest
+          set +e
+          UID_SELF=$(id -u)
+          WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in
+                "$WS"|"$WS"/*) echo "reap $sig pid=$pid cwd=$cwd"; kill "-$sig" "$pid" 2>/dev/null ;;
+              esac
+            done
+          }
+          reap TERM
+          sleep 3
+          reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          true

-  hive_multinode_e2e_test:
-    name: Hive Multi-Node E2E Test
-    runs-on: self-hosted
-    needs: build_e2e_image
+      - name: Clean Workspace
+        run: |
+          awk -v w="$GITHUB_WORKSPACE" '$2 ~ "^"w {print $2}' /proc/mounts \
+            | sort -r | xargs -r -I{} sudo umount -l -R {} || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*

+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Re-enable Go toolchain auto-download
+        run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
+
+      - name: Reclaim docker build cache
+        run: |
+          docker container prune -f || true
+          docker image prune -f || true
+          docker buildx prune -af --filter "until=24h" || true
+          docker system df || true
+
+      - name: Build Install Artifacts
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VM
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+          quiet_run "tofu init"                "$TOFU_LOG" tofu init -input=false
+          quiet_run "tofu workspace select"    "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy (pre-clean)" "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
+          quiet_run "scrub-env (pre-clean)"    "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          quiet_run "tofu apply"               "$TOFU_LOG" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          ok "VM provisioned"
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}" \
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Resolve VM SSH details
+        id: vm
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          echo "wan_ip=${WAN_IP}" >> "$GITHUB_OUTPUT"
+          echo "ssh_key=${SSH_KEY}" >> "$GITHUB_OUTPUT"
+
+      # The eks suite's CreateCluster boots a control-plane VM from the
+      # eks-node AMI, so that AMI must exist in the VM's image store before the
+      # suite runs. build-system-image.sh customizes the image with libguestfs
+      # (an isolated userspace appliance) — it touches no host block device, so
+      # it is safe on the shared runner. The build needs the spinifex source +
+      # network (runner only); the import needs predastore/NATS (VM only), so
+      # build on the runner then ship the raw and import on the VM.
+      - name: Build eks-node AMI (libguestfs, runner)
+        if: contains(env.SUITES, 'e2e-eks')
+        working-directory: spinifex
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          banner "build eks-node AMI (libguestfs — no host devices)"
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends libguestfs-tools qemu-utils
+          # Debian/Ubuntu ship /boot/vmlinuz-* mode 0600; the libguestfs
+          # appliance needs to read a host kernel. direct backend = no libvirt.
+          export LIBGUESTFS_BACKEND=direct
+          sudo chmod 0644 /boot/vmlinuz-* || true
+          group_open "make build-eks-node-image"
+          make build-eks-node-image
+          group_close
+          ok "raw: $(du -h /tmp/eks-node-image-build/eks-node-alpine.raw | cut -f1)"
+
+      - name: Import eks-node AMI on VM
+        if: contains(env.SUITES, 'e2e-eks')
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          eval "$(grep -E '^(ALPINE_VERSION|AMI_NAME|SYSTEM_TAG)=' \
+            spinifex/scripts/images/eks-node/manifest.conf)"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          banner "import eks-node AMI to VM image store"
+          scp "${SSH_OPTS[@]}" \
+            /tmp/eks-node-image-build/eks-node-alpine.raw \
+            "$VM":/tmp/eks-node-alpine.raw
+          # Drop the import progress bar (pterm redraws to stdout — thousands of
+          # lines over the SSH pipe). Errors go to stderr, so they survive; the
+          # exit code still gates the step.
+          ssh "${SSH_OPTS[@]}" "$VM" \
+            "spx admin images import --file /tmp/eks-node-alpine.raw \
+               --distro alpine --version ${ALPINE_VERSION} --arch x86_64 \
+               --boot-mode bios --ami-name ${AMI_NAME} --tag ${SYSTEM_TAG}" >/dev/null
+          ok "eks-node AMI imported"
+
+      # The eks data-plane subtest (TestEKS/KubectlGetNodes) drives the full
+      # `aws eks get-token` → kubectl → apiserver → token-webhook → gateway
+      # broker → STS verify path. The e2e binaries run ON the VM, so kubectl
+      # must be on the VM's PATH or harness.NewKubectl t.Skips the subtest.
+      # Pin to the k3s server version (setup.sh K3S_VERSION) — kubectl is
+      # ±1-minor skew tolerant; matching is simplest. Checksum-verified.
+      - name: Install kubectl on VM
+        if: contains(env.SUITES, 'e2e-eks')
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          banner "install kubectl on VM (eks data-plane subtest)"
+          ssh "${SSH_OPTS[@]}" "$VM" 'bash -s' <<'EOF'
+          set -euo pipefail
+          KVER=v1.32.5
+          case "$(uname -m)" in
+            x86_64)  ARCH=amd64 ;;
+            aarch64) ARCH=arm64 ;;
+            *) echo "unsupported arch $(uname -m)" >&2; exit 1 ;;
+          esac
+          cd /tmp
+          curl -fsSLO "https://dl.k8s.io/release/${KVER}/bin/linux/${ARCH}/kubectl"
+          curl -fsSLO "https://dl.k8s.io/release/${KVER}/bin/linux/${ARCH}/kubectl.sha256"
+          echo "$(cat kubectl.sha256)  kubectl" | sha256sum -c -
+          sudo install -m 0755 kubectl /usr/local/bin/kubectl
+          kubectl version --client
+          EOF
+          ok "kubectl installed on VM"
+
+      - name: Build E2E test binaries on runner
+        working-directory: spinifex/tests/e2e
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SUITES_BUILD=$(echo "${SUITES}" | sed 's/e2e-//g' | tr ' ' '\n' | grep -vE '^(multinode)$' | tr '\n' ' ')
+          if [ -z "$(echo "${SUITES_BUILD}" | tr -d ' ')" ]; then
+            echo "no in-VM suites requested (SUITES=${SUITES}); nothing to build"
+            exit 0
+          fi
+          banner "build ${SUITES_BUILD// /, }"
+          group_open "go test -c"
+          make e2e-build SUITES_BUILD="${SUITES_BUILD}"
+          group_close
+          for f in _bin/*.test; do
+            ok "$(basename "$f") $(stat -c '%s' "$f" | numfmt --to=iec)"
+          done
+
+      - name: Install go-junit-report on runner
+        run: |
+          go install github.com/jstemmer/go-junit-report/v2@latest
+          echo "$HOME/go/bin" >> "$GITHUB_PATH"
+
+      - name: Ship test binaries to VM
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "rm -rf \$HOME/spinifex-e2e-bin && mkdir -p \$HOME/spinifex-e2e-bin"
+          tar -C spinifex/tests/e2e/_bin -cf - . | ssh "${SSH_OPTS[@]}" "$VM" "tar -xf - -C \$HOME/spinifex-e2e-bin"
+          ok "shipped binaries to $VM"
+
+      - name: Start packet capture on VM br-wan
+        run: |
+          set +e
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          scp "${SSH_OPTS[@]}" \
+            "${GITHUB_WORKSPACE}/spinifex/.github/scripts/start-net-capture.sh" \
+            "$VM":/tmp/start-net-capture.sh
+          ssh "${SSH_OPTS[@]}" "$VM" "bash /tmp/start-net-capture.sh" || true
+
+      - name: Run E2E suites
+        id: e2e
+        run: |
+          set +e
+          SUITES_RUN=$(echo "${SUITES}" | sed 's/e2e-//g' | tr ' ' '\n' | grep -vE '^(multinode)$' | tr '\n' ' ')
+          if [ -z "$(echo "${SUITES_RUN}" | tr -d ' ')" ]; then
+            echo "no in-VM suites requested (SUITES=${SUITES}); skipping"
+            echo "rc=0" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          if [ -n "${TEST_RUN}" ]; then
+            TEST_RUN_FLAG="-test.run=${TEST_RUN}"
+          else
+            TEST_RUN_FLAG=""
+          fi
+          VM_HOST="tf-user@${{ steps.vm.outputs.wan_ip }}"
+          SSH_KEY="${{ steps.vm.outputs.ssh_key }}"
+          cleanup_remote() {
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "$SSH_KEY" "$VM_HOST" \
+              'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+              2>/dev/null || true
+          }
+          trap cleanup_remote EXIT INT TERM
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" \
+            "$VM_HOST" \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; \
+             cd \$HOME/spinifex-e2e-bin; \
+             R=\$'\033[0m' B=\$'\033[1m' C=\$'\033[36m' G=\$'\033[32m' RED=\$'\033[31m'; \
+             FAIL=0; \
+             for SUITE in ${SUITES_RUN}; do \
+               echo \"::group::suite \$SUITE\"; \
+               printf '\n%s%s━━ suite %s ━━%s\n' \"\$B\" \"\$C\" \"\$SUITE\" \"\$R\"; \
+               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=single \
+                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
+                 ./\$SUITE.test -test.v -test.timeout=30m ${TEST_RUN_FLAG} \
+                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log' \
+                      | sed -uE 's/^[[:space:]]+([a-zA-Z0-9_]+_test\.go:[0-9]+: )?//'; \
+               rc=\${PIPESTATUS[0]}; \
+               echo \"::endgroup::\"; \
+               if [ \$rc -ne 0 ]; then \
+                 printf '%s✗%s suite %s failed (exit %d)\n' \"\$RED\" \"\$R\" \"\$SUITE\" \"\$rc\"; \
+                 echo \"::error::suite \$SUITE failed (exit \$rc)\"; \
+                 echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; \
+                 FAIL=1; \
+               else \
+                 printf '%s✓%s suite %s passed\n' \"\$G\" \"\$R\" \"\$SUITE\"; \
+               fi; \
+             done; \
+             exit \$FAIL"
+          E2E_RC=$?
+          echo "rc=${E2E_RC}" >> "$GITHUB_OUTPUT"
+          exit 0
+
+      - name: Capture network/IPsec diagnostics on VM
+        if: always()
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          scp "${SSH_OPTS[@]}" \
+            "${GITHUB_WORKSPACE}/spinifex/.github/scripts/capture-net-diag.sh" \
+            "$VM":/tmp/capture-net-diag.sh
+          ssh "${SSH_OPTS[@]}" "$VM" "bash /tmp/capture-net-diag.sh '${ARTIFACT_DIR}/net-diag'"
+          ok "net diagnostics captured"
+
+      - name: Collect artifacts from VM
+        if: always()
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          mkdir -p "${ARTIFACT_DIR}"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.vm.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.vm.outputs.wan_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
+            | tar -C "${ARTIFACT_DIR}" -xf - 2>/dev/null
+          ok "artifacts pulled to ${ARTIFACT_DIR}"
+          ssh "${SSH_OPTS[@]}" "$VM" 'sudo journalctl -u "spinifex-*" --no-pager --output=short-iso --since "60 min ago"' \
+            > "${ARTIFACT_DIR}/spinifex-journal.log" 2>/dev/null
+          ok "journal: $(wc -l < "${ARTIFACT_DIR}/spinifex-journal.log" 2>/dev/null || echo 0) lines"
+          # LB/system microVM serial consoles. The lb-agent logs its nginx (NLB)
+          # engine activation and any `nginx -t`/reload error + active-probe
+          # results to the guest serial console, which the host journal never
+          # captures. These files persist in /run/spinifex past VM teardown, so
+          # collecting them here makes an NLB "0/N healthy" timeout diagnosable.
+          mkdir -p "${ARTIFACT_DIR}/lb-console"
+          ssh "${SSH_OPTS[@]}" "$VM" 'sudo sh -c "cd /run/spinifex 2>/dev/null && tar -cf - console-*.log 2>/dev/null"' \
+            | tar -C "${ARTIFACT_DIR}/lb-console" -xf - 2>/dev/null
+          ok "lb-console: $(ls "${ARTIFACT_DIR}/lb-console" 2>/dev/null | wc -l) files"
+
+      - name: Convert test logs to JUnit XML
+        if: always()
+        run: |
+          shopt -s nullglob
+          for LOG in "${ARTIFACT_DIR}"/test-*.log; do
+            BASE=$(basename "$LOG" .log)
+            SUITE=${BASE#test-}
+            go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-${SUITE}.xml" || true
+          done
+
+      - name: Publish test summary
+        if: always()
+        uses: ./spinifex/.github/actions/junit-report
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          title: "E2E results"
+
+      - name: Analyze e2e failures
+        if: always()
+        uses: ./spinifex/.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "E2E failure analysis"
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-artifacts-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Teardown
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          if [ -n "${{ steps.vm.outputs.wan_ip }}" ] && [ -n "${{ steps.vm.outputs.ssh_key }}" ]; then
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
+              -i "${{ steps.vm.outputs.ssh_key }}" \
+              tf-user@${{ steps.vm.outputs.wan_ip }} \
+              'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+              2>/dev/null || true
+          fi
+          UID_SELF=$(id -u)
+          WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in
+                "$WS"|"$WS"/*) kill "-$sig" "$pid" 2>/dev/null ;;
+              esac
+            done
+          }
+          reap TERM
+          sleep 2
+          reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          quiet_run "tofu workspace select" "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy"          "$TOFU_LOG" timeout 600 tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          quiet_run "scrub-env"             "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}"
+          ok "teardown complete"
+
+      - name: Fail job if any suite failed
+        if: always()
+        run: |
+          if [ "${{ steps.e2e.outputs.rc }}" != "0" ] || [ -s "${ARTIFACT_DIR}/suite-status.txt" ]; then
+            echo "::error::E2E suites failed (rc=${{ steps.e2e.outputs.rc }})"
+            cat "${ARTIFACT_DIR}/suite-status.txt" 2>/dev/null || true
+            exit 1
+          fi
+
+  e2e_real_multi:
+    name: Real Multi-Node E2E
+    if: github.event_name != 'workflow_dispatch' || contains(github.event.inputs.suites, 'e2e-multinode')
+    runs-on: [self-hosted, ci-multi]
+    timeout-minutes: 60
+    env:
+      TOFU_LOG: /tmp/e2e-tofu-multi-${{ github.run_id }}.log
+      ARTIFACT_DIR: /tmp/spinifex-e2e-multi-artifacts-${{ github.run_id }}
    steps:
-      - name: Run Multi-Node E2E Tests
+      - name: Reap Orphan Procs
        run: |
-          docker run --privileged --rm \
-            -v /dev/kvm:/dev/kvm \
-            --cap-add=NET_ADMIN \
-            --name hive-multinode-e2e-test \
-            hive-e2e:latest \
-            ./tests/e2e/run-multinode-e2e.sh
+          set +e
+          UID_SELF=$(id -u)
+          WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in
+                "$WS"|"$WS"/*) echo "reap $sig pid=$pid cwd=$cwd"; kill "-$sig" "$pid" 2>/dev/null ;;
+              esac
+            done
+          }
+          reap TERM
+          sleep 3
+          reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          true

-  cleanup:
-    name: Cleanup
-    runs-on: self-hosted
-    needs: [hive_e2e_test, hive_multinode_e2e_test]
+      - name: Clean Workspace
+        run: |
+          awk -v w="$GITHUB_WORKSPACE" '$2 ~ "^"w {print $2}' /proc/mounts \
+            | sort -r | xargs -r -I{} sudo umount -l -R {} || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Re-enable Go toolchain auto-download
+        run: echo "GOTOOLCHAIN=auto" >> "$GITHUB_ENV"
+
+      - name: Reclaim docker build cache
+        run: |
+          docker container prune -f || true
+          docker image prune -f || true
+          docker buildx prune -af --filter "until=24h" || true
+          docker system df || true
+
+      - name: Build Install Artifacts
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          ./build-install-artifacts.sh "${GIT_BRANCH}" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Provision VMs
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          export TF_VAR_git_branch="${GIT_BRANCH}"
+          quiet_run "tofu init"                "$TOFU_LOG" tofu init -input=false
+          quiet_run "tofu workspace select"    "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy (pre-clean)" "$TOFU_LOG" tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false || true
+          quiet_run "scrub-env (pre-clean)"    "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}" || true
+          quiet_run "tofu apply"               "$TOFU_LOG" tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          ok "3-node cluster provisioned"
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SPX_EXTERNAL_POOL="${SPINIFEX_CI_POOL}" \
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" \
+            "${RUNNER_TEMP}/setup.sh" \
+            "${RUNNER_TEMP}/spinifex-distro.tar.gz" \
+            --test-tar "${RUNNER_TEMP}/spinifex-tests.tar"
+
+      - name: Resolve cluster SSH details
+        id: cluster
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          INVENTORY=$(tofu output -json inventory)
+          NODE1_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
+          SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
+          SSH_KEY="${SSH_KEY/#\~/$HOME}"
+          NODE_COUNT=$(echo "$INVENTORY" | jq '.nodes | length')
+          NODES_CSV=""
+          for i in $(seq 0 $((NODE_COUNT - 1))); do
+            ip=$(echo "$INVENTORY" | jq -r ".nodes[$i].wan_ip")
+            if [ -z "$NODES_CSV" ]; then NODES_CSV="$ip"; else NODES_CSV="$NODES_CSV,$ip"; fi
+          done
+          echo "node1_ip=${NODE1_IP}" >> "$GITHUB_OUTPUT"
+          echo "ssh_key=${SSH_KEY}" >> "$GITHUB_OUTPUT"
+          echo "nodes_csv=${NODES_CSV}" >> "$GITHUB_OUTPUT"
+
+      - name: Build E2E test binaries on runner
+        working-directory: spinifex/tests/e2e
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          # eks is excluded on multinode: EKS KV is a clustered (Raft) JetStream
+          # stream and create-cluster needs an elected leader, but multinode
+          # JetStream clustering isn't solid yet (mulga-omq/z2z/5yg). EKS is
+          # validated single-node only for now; re-enable here once clustered
+          # JetStream + multinode EKS are a deliberate target.
+          SUITES_BUILD=$(echo "${SUITES}" | sed 's/e2e-//g' | tr ' ' '\n' | grep -vE '^(single|eks)$' | tr '\n' ' ')
+          if [ -z "$(echo "${SUITES_BUILD}" | tr -d ' ')" ]; then
+            echo "no multinode-eligible suites requested (SUITES=${SUITES}); nothing to build"
+            exit 0
+          fi
+          banner "build ${SUITES_BUILD// /, }"
+          group_open "go test -c"
+          make e2e-build SUITES_BUILD="${SUITES_BUILD}"
+          group_close
+          for f in _bin/*.test; do
+            ok "$(basename "$f") $(stat -c '%s' "$f" | numfmt --to=iec)"
+          done
+
+      - name: Install go-junit-report on runner
+        run: |
+          go install github.com/jstemmer/go-junit-report/v2@latest
+          echo "$HOME/go/bin" >> "$GITHUB_PATH"
+
+      - name: Ship test binaries to node1
+        run: |
+          set -euo pipefail
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.cluster.outputs.node1_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "rm -rf \$HOME/spinifex-e2e-bin && mkdir -p \$HOME/spinifex-e2e-bin"
+          tar -C spinifex/tests/e2e/_bin -cf - . | ssh "${SSH_OPTS[@]}" "$VM" "tar -xf - -C \$HOME/spinifex-e2e-bin"
+          ok "shipped binaries to $VM"
+
+      - name: Start packet capture on all nodes br-wan
+        run: |
+          set +e
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          IFS=',' read -ra NODE_LIST <<< "${{ steps.cluster.outputs.nodes_csv }}"
+          for ip in "${NODE_LIST[@]}"; do
+              [ -z "$ip" ] && continue
+              scp "${SSH_OPTS[@]}" \
+                  "${GITHUB_WORKSPACE}/spinifex/.github/scripts/start-net-capture.sh" \
+                  tf-user@"$ip":/tmp/start-net-capture.sh
+              ssh "${SSH_OPTS[@]}" tf-user@"$ip" "bash /tmp/start-net-capture.sh" || true
+          done
+
+      - name: Pre-flight diagnostics on node1
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh" 2>/dev/null || true
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.cluster.outputs.node1_ip }}
+          mkdir -p "${ARTIFACT_DIR}"
+          DIAG="${ARTIFACT_DIR}/preflight-diag.log"
+          {
+            echo "=== id (tf-user groups) ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'id; echo; getent group spinifex || true'
+            echo
+            echo "=== ls -la /etc/spinifex/ ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'ls -la /etc/spinifex/ 2>&1; echo; ls -la /etc/spinifex/nats/ 2>&1'
+            echo
+            echo "=== spinifex.toml NATS section ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'grep -A4 "\[nodes\\..*\\.nats" /etc/spinifex/spinifex.toml 2>&1 | head -40'
+            echo
+            echo "=== systemctl is-active spinifex-* ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'for s in spinifex-nats spinifex-predastore spinifex-awsgw spinifex-vpcd spinifex-daemon; do printf "%-22s %s\n" "$s" "$(systemctl is-active $s 2>&1)"; done'
+            echo
+            echo "=== which spx + version ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'which spx; spx --version 2>&1 || true'
+            echo
+            echo "=== spx get nodes (direct ssh, tf-user) ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'spx get nodes --timeout 5s 2>&1; echo "exit=$?"'
+            echo
+            echo "=== spx top nodes (direct ssh, tf-user) ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'spx top nodes 2>&1 | head -30; echo "exit=$?"'
+            echo
+            echo "=== NATS routez (curl 127.0.0.1:8222) ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'curl -fsS http://127.0.0.1:8222/routez 2>&1 | head -40'
+            echo
+            echo "=== aws describe-instance-types via gateway (tf-user) ==="
+            ssh "${SSH_OPTS[@]}" "$VM" 'AWS_PROFILE=spinifex AWS_CA_BUNDLE=/etc/spinifex/ca.pem aws --endpoint-url https://${{ steps.cluster.outputs.node1_ip }}:9999 ec2 describe-instance-types --query "InstanceTypes[*].InstanceType" --output text 2>&1 | head -5'
+          } 2>&1 | tee "$DIAG"
+          true
+
+      - name: Run E2E suites
+        id: e2e
+        run: |
+          set +e
+          SUITES_RUN=$(echo "${SUITES}" | sed 's/e2e-//g' | tr ' ' '\n' | grep -vE '^(single|eks)$' | tr '\n' ' ')
+          if [ -z "$(echo "${SUITES_RUN}" | tr -d ' ')" ]; then
+            echo "no multinode-eligible suites requested (SUITES=${SUITES}); skipping"
+            echo "rc=0" >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          if [ -n "${TEST_RUN}" ]; then
+            TEST_RUN_FLAG="-test.run=${TEST_RUN}"
+          else
+            TEST_RUN_FLAG=""
+          fi
+          VM_HOST="tf-user@${{ steps.cluster.outputs.node1_ip }}"
+          SSH_KEY="${{ steps.cluster.outputs.ssh_key }}"
+          NODES_CSV="${{ steps.cluster.outputs.nodes_csv }}"
+          cleanup_remote() {
+            ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 -i "$SSH_KEY" "$VM_HOST" \
+              'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+              2>/dev/null || true
+          }
+          trap cleanup_remote EXIT INT TERM
+          ssh -tt -o StrictHostKeyChecking=no -i "$SSH_KEY" \
+            "$VM_HOST" \
+            "set -u; mkdir -p '${ARTIFACT_DIR}'; \
+             cd \$HOME/spinifex-e2e-bin; \
+             R=\$'\033[0m' B=\$'\033[1m' C=\$'\033[36m' G=\$'\033[32m' RED=\$'\033[31m'; \
+             FAIL=0; \
+             for SUITE in ${SUITES_RUN}; do \
+               echo \"::group::suite \$SUITE\"; \
+               printf '\n%s%s━━ suite %s ━━%s\n' \"\$B\" \"\$C\" \"\$SUITE\" \"\$R\"; \
+               date -u +%Y-%m-%dT%H:%M:%SZ > '${ARTIFACT_DIR}/test-'\$SUITE'.start'; \
+               GITHUB_ACTIONS=true FORCE_COLOR=1 SPINIFEX_E2E=1 SPINIFEX_MODE=multinode \
+                 SPINIFEX_NODES='${NODES_CSV}' \
+                 SPINIFEX_NODE_IPS='${NODES_CSV}' \
+                 SPINIFEX_SSH_USER=tf-user \
+                 SPINIFEX_SSH_KEY=\$HOME/.ssh/tf-user-ap-southeast-2 \
+                 ARTIFACT_DIR='${ARTIFACT_DIR}' \
+                 ./\$SUITE.test -test.v -test.timeout=30m ${TEST_RUN_FLAG} \
+                 2>&1 | tee '${ARTIFACT_DIR}/test-'\$SUITE'.log' \
+                      | sed -uE 's/^[[:space:]]+([a-zA-Z0-9_]+_test\.go:[0-9]+: )?//'; \
+               rc=\${PIPESTATUS[0]}; \
+               echo \"::endgroup::\"; \
+               if [ \$rc -ne 0 ]; then \
+                 printf '%s✗%s suite %s failed (exit %d)\n' \"\$RED\" \"\$R\" \"\$SUITE\" \"\$rc\"; \
+                 echo \"::error::suite \$SUITE failed (exit \$rc)\"; \
+                 echo \"\$SUITE: FAILED\" >> '${ARTIFACT_DIR}/suite-status.txt'; \
+                 FAIL=1; \
+               else \
+                 printf '%s✓%s suite %s passed\n' \"\$G\" \"\$R\" \"\$SUITE\"; \
+               fi; \
+             done; \
+             exit \$FAIL"
+          E2E_RC=$?
+          echo "rc=${E2E_RC}" >> "$GITHUB_OUTPUT"
+          exit 0
+
+      - name: Capture network/IPsec diagnostics on all nodes
+        if: always()
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          IFS=',' read -ra NODE_LIST <<< "${{ steps.cluster.outputs.nodes_csv }}"
+          NODE1_IP="${NODE_LIST[0]}"
+          scp "${SSH_OPTS[@]}" \
+            "${GITHUB_WORKSPACE}/spinifex/.github/scripts/capture-net-diag.sh" \
+            tf-user@"$NODE1_IP":/tmp/capture-net-diag.sh
+          for i in "${!NODE_LIST[@]}"; do
+            idx=$((i + 1))
+            ip="${NODE_LIST[$i]}"
+            if [ "$idx" -ne 1 ]; then
+              scp "${SSH_OPTS[@]}" \
+                "${GITHUB_WORKSPACE}/spinifex/.github/scripts/capture-net-diag.sh" \
+                tf-user@"$ip":/tmp/capture-net-diag.sh
+            fi
+            ssh "${SSH_OPTS[@]}" tf-user@"$ip" \
+              "bash /tmp/capture-net-diag.sh '${ARTIFACT_DIR}/net-diag-node${idx}'"
+            if [ "$idx" -ne 1 ]; then
+              ssh "${SSH_OPTS[@]}" tf-user@"$ip" \
+                "tar -C '${ARTIFACT_DIR}' -cf - 'net-diag-node${idx}' 2>/dev/null || true" \
+                | ssh "${SSH_OPTS[@]}" tf-user@"$NODE1_IP" \
+                  "tar -C '${ARTIFACT_DIR}' -xf - 2>/dev/null"
+            fi
+            ok "net diagnostics node${idx} captured"
+          done
+
+      - name: Collect artifacts from node1
+        if: always()
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          mkdir -p "${ARTIFACT_DIR}"
+          SSH_OPTS=(-o StrictHostKeyChecking=no -i "${{ steps.cluster.outputs.ssh_key }}")
+          VM=tf-user@${{ steps.cluster.outputs.node1_ip }}
+          ssh "${SSH_OPTS[@]}" "$VM" "tar -C '${ARTIFACT_DIR}' -cf - . 2>/dev/null || true" \
+            | tar -C "${ARTIFACT_DIR}" -xf - 2>/dev/null
+          ok "artifacts pulled to ${ARTIFACT_DIR}"
+          IFS=',' read -ra NODE_LIST <<< "${{ steps.cluster.outputs.nodes_csv }}"
+          for i in "${!NODE_LIST[@]}"; do
+            idx=$((i + 1))
+            ip="${NODE_LIST[$i]}"
+            ssh "${SSH_OPTS[@]}" tf-user@"$ip" 'sudo journalctl -u "spinifex-*" --no-pager --output=short-iso --since "60 min ago"' \
+              > "${ARTIFACT_DIR}/spinifex-journal-node${idx}.log" 2>/dev/null
+            ok "journal node${idx}: $(wc -l < "${ARTIFACT_DIR}/spinifex-journal-node${idx}.log" 2>/dev/null || echo 0) lines"
+          done
+          for i in "${!NODE_LIST[@]}"; do
+            idx=$((i + 1))
+            ip="${NODE_LIST[$i]}"
+            ssh "${SSH_OPTS[@]}" tf-user@"$ip" 'sudo journalctl -u openvswitch-ipsec.service --no-pager --output=short-iso --since "60 min ago"' \
+              > "${ARTIFACT_DIR}/openvswitch-ipsec-node${idx}.log" 2>/dev/null
+          done
+          # LB/system microVM serial consoles, per node. The lb-agent logs its
+          # nginx (NLB) engine activation and any `nginx -t`/reload error +
+          # active-probe results to the guest serial console (never in the host
+          # journal). Files persist in /run/spinifex past VM teardown.
+          for i in "${!NODE_LIST[@]}"; do
+            idx=$((i + 1))
+            ip="${NODE_LIST[$i]}"
+            mkdir -p "${ARTIFACT_DIR}/lb-console/node${idx}"
+            ssh "${SSH_OPTS[@]}" tf-user@"$ip" 'sudo sh -c "cd /run/spinifex 2>/dev/null && tar -cf - console-*.log 2>/dev/null"' \
+              | tar -C "${ARTIFACT_DIR}/lb-console/node${idx}" -xf - 2>/dev/null
+            ok "lb-console node${idx}: $(ls "${ARTIFACT_DIR}/lb-console/node${idx}" 2>/dev/null | wc -l) files"
+          done
+
+      - name: Convert test logs to JUnit XML
+        if: always()
+        run: |
+          shopt -s nullglob
+          for LOG in "${ARTIFACT_DIR}"/test-*.log; do
+            BASE=$(basename "$LOG" .log)
+            SUITE=${BASE#test-}
+            go-junit-report -in "$LOG" -out "${ARTIFACT_DIR}/junit-${SUITE}.xml" || true
+          done
+
+      - name: Publish test summary
+        if: always()
+        uses: ./spinifex/.github/actions/junit-report
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          title: "Multi-Node E2E results"
+
+      - name: Analyze e2e failures
+        if: always()
+        uses: ./spinifex/.github/actions/e2e-analyze
+        with:
+          junit-glob: ${{ env.ARTIFACT_DIR }}/junit-*.xml
+          log-dir: ${{ env.ARTIFACT_DIR }}
+          title: "Multi-Node E2E failure analysis"
+
+      - uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: e2e-multi-artifacts-${{ github.run_id }}
+          path: ${{ env.ARTIFACT_DIR }}
+          if-no-files-found: warn
+          retention-days: 14
+
+      - name: Teardown
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          set +e
+          source "${GITHUB_WORKSPACE}/spinifex/.github/scripts/ci-fmt.sh"
+          if [ -n "${{ steps.cluster.outputs.node1_ip }}" ] && [ -n "${{ steps.cluster.outputs.ssh_key }}" ]; then
+            IFS=',' read -ra NODE_LIST <<< "${{ steps.cluster.outputs.nodes_csv }}"
+            for ip in "${NODE_LIST[@]}"; do
+              ssh -o StrictHostKeyChecking=no -o ConnectTimeout=5 \
+                -i "${{ steps.cluster.outputs.ssh_key }}" \
+                tf-user@"$ip" \
+                'pkill -u tf-user -TERM -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; sleep 1; pkill -u tf-user -KILL -f "/spinifex-e2e-bin/.*\.test( |$)" 2>/dev/null; true' \
+                2>/dev/null || true
+            done
+          fi
+          UID_SELF=$(id -u)
+          WS="${GITHUB_WORKSPACE}"
+          reap() {
+            local sig="$1"
+            for pid in $(pgrep -u "$UID_SELF" -x tofu) $(pgrep -u "$UID_SELF" -x terraform); do
+              cwd=$(readlink "/proc/$pid/cwd" 2>/dev/null) || continue
+              case "$cwd" in
+                "$WS"|"$WS"/*) kill "-$sig" "$pid" 2>/dev/null ;;
+              esac
+            done
+          }
+          reap TERM
+          sleep 2
+          reap KILL
+          find "$WS" -name '.terraform.tfstate.lock.info' -delete 2>/dev/null
+          quiet_run "tofu workspace select" "$TOFU_LOG" tofu workspace select -or-create "${SPINIFEX_CI_ENV}"
+          quiet_run "tofu destroy"          "$TOFU_LOG" timeout 600 tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false
+          quiet_run "scrub-env"             "$TOFU_LOG" ./scrub-env.sh "${SPINIFEX_CI_ENV}"
+          ok "teardown complete"
+
+      - name: Fail job if any suite failed
+        if: always()
+        run: |
+          if [ "${{ steps.e2e.outputs.rc }}" != "0" ] || [ -s "${ARTIFACT_DIR}/suite-status.txt" ]; then
+            echo "::error::Multi-node E2E suites failed (rc=${{ steps.e2e.outputs.rc }})"
+            cat "${ARTIFACT_DIR}/suite-status.txt" 2>/dev/null || true
+            exit 1
+          fi
+
+  e2e_status:
+    name: Report E2E Status
    if: always()
-
+    needs: [e2e_single, e2e_real_multi]
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
    steps:
-      - name: Remove Stale Containers
+      - name: Report Status to PR
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REPORT_SHA: ${{ inputs.sha || github.sha }}
        run: |
-          docker rm -f hive-e2e-test 2>/dev/null || true
-          docker rm -f hive-multinode-e2e-test 2>/dev/null || true
-
-      - name: Prune Docker Images
-        run: docker image prune -f
+          if [[ -z "${REPORT_SHA}" ]]; then
+            echo "::error::no SHA available (inputs.sha and github.sha both empty)"
+            exit 1
+          fi
+          single_ok=true; multi_ok=true
+          case "${{ needs.e2e_single.result }}" in success|skipped) ;; *) single_ok=false ;; esac
+          case "${{ needs.e2e_real_multi.result }}" in success|skipped) ;; *) multi_ok=false ;; esac
+          if $single_ok && $multi_ok; then
+            STATE="success"
+            DESC="E2E suites passed"
+          else
+            STATE="failure"
+            DESC="E2E failed: single=${{ needs.e2e_single.result }} multi=${{ needs.e2e_real_multi.result }}"
+          fi
+          gh api "repos/${{ github.repository }}/statuses/${REPORT_SHA}" \
+            -f state="$STATE" \
+            -f context=e2e \
+            -f description="$DESC" \
+            -f target_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
@@ -1,40 +0,0 @@
-name: Hive Build and Test
-
-on:
-  push:
-    branches: [main]
-  pull_request:
-  workflow_dispatch:
-
-jobs:
-  hive_build_and_test:
-    name: Hive Build and Test
-    runs-on: ubuntu-24.04
-
-    steps:
-      - name: Checkout Code
-        uses: actions/checkout@v6
-
-      - name: Setup Go
-        uses: actions/setup-go@v6
-        with:
-          cache-dependency-path: "go.sum"
-          go-version-file: "go.mod"
-
-      - name: Run Tests
-        run: LOG_IGNORE=1 go test -v -timeout 120s ./...
-
-      - name: Go Vet
-        run: go vet ./...
-
-      - name: Static Check
-        run: go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./...
-
-      - name: Go Vulnerability Check
-        run: go tool govulncheck ./...
-
-      # - name: Gosec Security Scanner
-      #   run: go tool gosec -exclude-generated ./...
-
-      - name: Build
-        run: make go_build
@@ -0,0 +1,96 @@
+name: Publish EKS node AMI
+
+# Builds the eks-node system AMI (libguestfs, on the runner) and publishes the
+# compressed qcow2 + .sha256 sidecar to the Cloudflare R2 bucket that backs
+# iso.mulgadc.com/system-ami/. Operators then install it on any cluster with:
+#   spx admin images import --name spinifex-eks-node
+#
+# Manual-only: publishing overwrites the mirror artifact, so it is a deliberate
+# action, not something every push should trigger.
+
+on:
+  workflow_dispatch:
+    inputs:
+      dry_run:
+        description: "Build + convert only; skip the R2 upload"
+        type: boolean
+        default: false
+
+permissions:
+  contents: read
+
+concurrency:
+  group: publish-eks-ami
+  cancel-in-progress: false
+
+jobs:
+  publish:
+    name: Build + publish eks-node AMI
+    runs-on: [ self-hosted, ci-single ]
+
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          ref: dev
+          path: viperblock
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          ref: dev
+          path: predastore
+
+      - name: Setup Go
+        uses: actions/setup-go@v6
+        with:
+          go-version-file: "spinifex/go.mod"
+          # Self-hosted runner: the Go module + build cache already persists on
+          # local disk between runs. setup-go's cache-save ("Post Setup Go")
+          # otherwise re-uploads hundreds of MB to the Actions cache backend,
+          # taking longer than the build itself.
+          cache: false
+
+      - name: Setup Go Workspace
+        run: go work init ./spinifex ./viperblock ./predastore
+
+      - name: Install build + upload tooling
+        run: |
+          set -euo pipefail
+          sudo apt-get update -qq
+          sudo apt-get install -y --no-install-recommends libguestfs-tools qemu-utils
+          if ! command -v aws &>/dev/null; then
+            curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
+            unzip -q /tmp/awscliv2.zip -d /tmp/aws-install
+            sudo /tmp/aws-install/aws/install
+            rm -rf /tmp/awscliv2.zip /tmp/aws-install
+          fi
+
+      - name: Build + publish to R2
+        working-directory: spinifex
+        env:
+          # Map the repo's R2 secrets onto the names publish-system-image.sh reads.
+          AWS_ACCESS_KEY_ID: ${{ secrets.CF_R2_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.CF_R2_SECRET_ACCESS_KEY }}
+          R2_ENDPOINT: ${{ secrets.CF_R2_ENDPOINT_URL }}
+          R2_BUCKET: spinifex-iso
+          DRY_RUN: ${{ inputs.dry_run }}
+        run: |
+          set -euo pipefail
+          # libguestfs needs to read a host kernel; Ubuntu ships it mode 0600.
+          # direct backend = no libvirt on the runner.
+          export LIBGUESTFS_BACKEND=direct
+          sudo chmod 0644 /boot/vmlinuz-* || true
+          FLAGS=(--build)
+          if [[ "${DRY_RUN}" == "true" ]]; then
+            FLAGS+=(--dry-run)
+          fi
+          ./scripts/publish-system-image.sh \
+            scripts/images/eks-node/manifest.conf "${FLAGS[@]}"
@@ -0,0 +1,375 @@
+name: Release
+
+on:
+  push:
+    tags: [ "v*" ]
+
+permissions:
+  contents: write
+
+jobs:
+  distro-amd64:
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          path: predastore
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          path: viperblock
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: Set up Go
+        uses: actions/setup-go@v6
+        with:
+          go-version: "1.26.4"
+
+      - name: Build microVM artifacts (lb-agent + kernel + initramfs)
+        run: cd spinifex && make build-microvm-image
+
+      - name: Build amd64 Tarball
+        env:
+          VERSION: ${{ github.ref_name }}
+        run: cd spinifex && make distro-amd64 VERSION="$VERSION"
+
+      - name: Upload amd64 Artifacts
+        uses: actions/upload-artifact@v7
+        with:
+          name: distro-amd64
+          path: |
+            spinifex/dist/spinifex-${{ github.ref_name }}-linux-amd64.tar.gz
+            spinifex/dist/spinifex-${{ github.ref_name }}-linux-amd64.tar.gz.sha256
+            spinifex/scripts/setup.sh
+
+  distro-arm64:
+    runs-on: ubuntu-24.04-arm
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          path: predastore
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          path: viperblock
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v4
+
+      - name: Build arm64 Tarball
+        env:
+          VERSION: ${{ github.ref_name }}
+        run: cd spinifex && make distro-arm64 VERSION="$VERSION"
+
+      - name: Upload arm64 Artifacts
+        uses: actions/upload-artifact@v7
+        with:
+          name: distro-arm64
+          path: |
+            spinifex/dist/spinifex-${{ github.ref_name }}-linux-arm64.tar.gz
+            spinifex/dist/spinifex-${{ github.ref_name }}-linux-arm64.tar.gz.sha256
+
+  release:
+    needs: [ distro-amd64, distro-arm64 ]
+    runs-on: ubuntu-24.04
+    steps:
+      - name: Download amd64 Artifacts
+        uses: actions/download-artifact@v8
+        with:
+          name: distro-amd64
+          path: artifacts
+
+      - name: Download arm64 Artifacts
+        uses: actions/download-artifact@v8
+        with:
+          name: distro-arm64
+          path: artifacts
+
+      - name: Create Draft GitHub Release
+        uses: softprops/action-gh-release@v3
+        with:
+          draft: true
+          files: |
+            artifacts/**/*.tar.gz
+            artifacts/**/*.sha256
+            artifacts/**/setup.sh
+          generate_release_notes: true
+
+      - name: Upload Tarball for Verify Job
+        uses: actions/upload-artifact@v7
+        with:
+          name: release-assets
+          path: artifacts/
+
+  verify:
+    needs: release
+    runs-on: [ self-hosted, ci-single ]
+    timeout-minutes: 25
+    env:
+      TOFU_LOG: /tmp/release-verify-${{ github.run_id }}.log
+      TAG: ${{ github.ref_name }}
+      REPO: ${{ github.repository }}
+    steps:
+      - name: Clean Workspace
+        run: |
+          awk -v w="$GITHUB_WORKSPACE" '$2 ~ "^"w {print $2}' /proc/mounts \
+            | sort -r | xargs -r -I{} sudo umount -l -R {} || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/tofu-cluster
+          path: mulga
+
+      - name: Download Release Assets
+        uses: actions/download-artifact@v8
+        with:
+          name: release-assets
+          path: release-assets
+
+      - name: Provision VM
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          tofu init -input=false >> "$TOFU_LOG" 2>&1
+          tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
+          tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
+          ./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
+          tofu apply -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
+
+      - name: Bootstrap via Binary Install
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          SETUP_SH=$(find "${GITHUB_WORKSPACE}/release-assets" -name "setup.sh" | head -1)
+          TARBALL=$(find "${GITHUB_WORKSPACE}/release-assets" -name "*-linux-amd64.tar.gz" | head -1)
+          ./bootstrap-install.sh "${SPINIFEX_CI_ENV}" "$SETUP_SH" "$TARBALL"
+
+      - name: Teardown
+        if: always()
+        working-directory: mulga/scripts/tofu-cluster
+        run: |
+          tofu workspace select -or-create "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1
+          tofu destroy -var-file=envs/${SPINIFEX_CI_ENV}.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
+          ./scrub-env.sh "${SPINIFEX_CI_ENV}" >> "$TOFU_LOG" 2>&1 || true
+
+  baremetal:
+    needs: release
+    permissions:
+      contents: read
+      statuses: write
+    uses: ./.github/workflows/e2e-baremetal.yml
+    secrets: inherit
+
+  promote:
+    needs: [ verify, baremetal ]
+    runs-on: ubuntu-24.04
+    env:
+      TAG: ${{ github.ref_name }}
+      REPO: ${{ github.repository }}
+    steps:
+      - name: Promote Release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          if [[ "$TAG" =~ -(dev|rc|alpha|beta) ]]; then
+            gh release edit "$TAG" --repo "$REPO" --draft=false --prerelease
+            echo "Release ${TAG} promoted as prerelease"
+          else
+            gh release edit "$TAG" --repo "$REPO" --draft=false
+            echo "Release ${TAG} promoted to latest"
+          fi
+
+  # Annotate the (still-draft) release if either gate failed.
+  mark-failed:
+    needs: [ verify, baremetal ]
+    if: failure()
+    runs-on: ubuntu-24.04
+    env:
+      TAG: ${{ github.ref_name }}
+      REPO: ${{ github.repository }}
+    steps:
+      - name: Mark Release Failed
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          EXISTING_BODY=$(gh release view "$TAG" --repo "$REPO" --json body -q .body)
+          NOTES_FILE=$(mktemp)
+          printf 'Verification failed (tofu install or bare-metal boot) — see workflow run for details.\n\n%s' "$EXISTING_BODY" > "$NOTES_FILE"
+          gh release edit "$TAG" --repo "$REPO" --notes-file "$NOTES_FILE"
+          rm -f "$NOTES_FILE"
+
+  iso:
+    needs: release
+    runs-on: [ self-hosted, ci-single ]
+    # ISO build (~60-90m) overlaps the parallel baremetal run (~120m); the
+    # upload gate then waits for baremetal to finish, so the job must outlast it.
+    timeout-minutes: 180
+    steps:
+      - name: Clean Workspace
+        run: |
+          ROOTFS="${GITHUB_WORKSPACE}/mulga/scripts/iso-builder/.cache/rootfs"
+          sudo umount -l -R "${ROOTFS}" 2>/dev/null || true
+          sudo umount -l "${ROOTFS}/dev/pts" 2>/dev/null || true
+          sudo umount -l "${ROOTFS}/dev" 2>/dev/null || true
+          sudo umount -l "${ROOTFS}/sys" 2>/dev/null || true
+          sudo umount -l "${ROOTFS}/proc" 2>/dev/null || true
+          sudo rm -rf --one-file-system "$GITHUB_WORKSPACE"/*
+
+      - uses: actions/create-github-app-token@v3
+        id: app-token
+        with:
+          client-id: ${{ secrets.APP_ID }}
+          private-key: ${{ secrets.APP_PRIVATE_KEY }}
+          repositories: mulga
+
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+          ref: ${{ github.ref }}
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          path: predastore
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          path: viperblock
+
+      - name: Checkout Mulga iso-builder
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/mulga
+          token: ${{ steps.app-token.outputs.token }}
+          sparse-checkout: scripts/iso-builder
+          path: mulga
+
+      - name: Clean ISO builder cache
+        run: sudo make -C mulga/scripts/iso-builder clean
+
+      - name: Set up Go
+        uses: actions/setup-go@v6
+        with:
+          go-version-file: spinifex/go.mod
+          cache: false
+
+      - name: Configure Go workspace
+        run: go work init ./spinifex ./predastore ./viperblock
+
+      - name: Install ISO builder dependencies
+        run: |
+          sudo apt-get update -qq
+          sudo apt-get install -y \
+            debootstrap squashfs-tools xorriso grub-pc-bin grub-efi-amd64-bin \
+            grub-common mtools busybox-static ovmf zstd \
+            nbdkit nbdkit-plugin-dev pkg-config gcc
+
+      - name: Build spinifex binaries
+        run: cd spinifex && make build
+
+      - name: Build viperblock
+        run: cd viperblock && make build
+
+      - name: Build release ISO
+        id: build-iso
+        env:
+          VERSION: ${{ github.ref_name }}
+        run: |
+          sudo make -C mulga/scripts/iso-builder release \
+            VERSION="${VERSION}" \
+            SPINIFEX_SRC="${GITHUB_WORKSPACE}/spinifex" \
+            VIPERBLOCK_SRC="${GITHUB_WORKSPACE}/viperblock" \
+            SPINIFEX_BIN="${GITHUB_WORKSPACE}/spinifex/bin/spx" \
+            INSTALLER_BIN="${GITHUB_WORKSPACE}/spinifex/bin/spinifex-installer"
+
+      - name: Wait for verify + baremetal
+        id: wait-verify
+        if: steps.build-iso.outcome == 'success'
+        continue-on-error: true
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          echo "Waiting for verify + baremetal jobs to complete..."
+          while true; do
+            JOBS=$(gh api \
+              "repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/jobs" --paginate)
+            VERIFY=$(echo "$JOBS" | jq -r '.jobs[] | select(.name == "verify") | .conclusion // empty')
+            BAREMETAL=$(echo "$JOBS" | jq -r '.jobs[] | select(.name | startswith("baremetal")) | .conclusion // empty')
+            if [ -n "$VERIFY" ] && [ -n "$BAREMETAL" ]; then
+              echo "verify=$VERIFY baremetal=$BAREMETAL"
+              { [ "$VERIFY" = "success" ] && [ "$BAREMETAL" = "success" ]; } && exit 0
+              exit 1
+            fi
+            echo "Still waiting (verify=${VERIFY:-pending} baremetal=${BAREMETAL:-pending}), retry 30s..."
+            sleep 30
+          done
+
+      - name: Ensure AWS CLI is installed
+        if: steps.build-iso.outcome == 'success' && steps.wait-verify.outcome == 'success'
+        run: |
+          if ! command -v aws &>/dev/null; then
+            curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip
+            unzip -q /tmp/awscliv2.zip -d /tmp/aws-install
+            sudo /tmp/aws-install/aws/install
+            rm -rf /tmp/awscliv2.zip /tmp/aws-install
+          fi
+
+      - name: Upload ISO to Cloudflare R2
+        id: upload
+        if: steps.build-iso.outcome == 'success' && steps.wait-verify.outcome == 'success'
+        env:
+          AWS_ACCESS_KEY_ID: ${{ secrets.CF_R2_ACCESS_KEY_ID }}
+          AWS_SECRET_ACCESS_KEY: ${{ secrets.CF_R2_SECRET_ACCESS_KEY }}
+          CF_R2_ENDPOINT_URL: ${{ secrets.CF_R2_ENDPOINT_URL }}
+        run: |
+          aws --endpoint-url="${CF_R2_ENDPOINT_URL}" \
+            s3 cp "mulga/scripts/iso-builder/dist/spinifex.iso" \
+                  "s3://spinifex-iso/spinifex.iso"
+          aws --endpoint-url="${CF_R2_ENDPOINT_URL}" \
+            s3 cp "mulga/scripts/iso-builder/dist/spinifex.iso.sha256" \
+                  "s3://spinifex-iso/spinifex.iso.sha256"
+
+      - name: Mark ISO failed
+        if: always() && (steps.build-iso.outcome == 'failure' || steps.upload.outcome == 'failure')
+        env:
+          GH_TOKEN: ${{ github.token }}
+          TAG: ${{ github.ref_name }}
+          REPO: ${{ github.repository }}
+        run: |
+          EXISTING_BODY=$(gh release view "$TAG" --repo "$REPO" --json body -q .body)
+          NOTES_FILE=$(mktemp)
+          printf 'ISO build/upload failed — see workflow run for details.\n\n%s' "$EXISTING_BODY" > "$NOTES_FILE"
+          gh release edit "$TAG" --repo "$REPO" --notes-file "$NOTES_FILE"
+          rm -f "$NOTES_FILE"
@@ -0,0 +1,149 @@
+name: Build and Test
+
+on:
+  push:
+    branches:
+      - main
+      - dev
+    paths:
+      - "**/*.go"
+      - go.mod
+      - go.sum
+      - Makefile
+      - .github/workflows/spinifex.yml
+  pull_request:
+    branches:
+      - main
+      - dev
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  build_and_test:
+    name: Build and Test
+    runs-on: ubuntu-24.04
+
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+          fetch-depth: 2
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          ref: dev
+          path: viperblock
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          ref: dev
+          path: predastore
+
+      - name: Setup Go
+        uses: actions/setup-go@v6
+        with:
+          cache-dependency-path: "spinifex/go.sum"
+          go-version-file: "spinifex/go.mod"
+
+      - name: Setup Go Workspace
+        run: go work init ./spinifex ./viperblock ./predastore
+
+      - name: Build Spinifex
+        run: make -C spinifex build
+
+      - name: Manifest check + drift lint
+        run: make -C spinifex manifest-check manifest-lint
+
+      - name: Run Tests with Coverage
+        run: make -C spinifex test-cover
+
+      - name: Run Action Tests
+        run: make -C spinifex test-actions
+
+      - name: Check Diff Coverage
+        run: cd spinifex && scripts/diff-coverage.sh coverage.out --base HEAD~1
+
+  race_detection:
+    name: Race Detection
+    runs-on: ubuntu-24.04
+
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          ref: dev
+          path: viperblock
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          ref: dev
+          path: predastore
+
+      - name: Setup Go
+        uses: actions/setup-go@v6
+        with:
+          cache-dependency-path: "spinifex/go.sum"
+          go-version-file: "spinifex/go.mod"
+
+      - name: Setup Go Workspace
+        run: go work init ./spinifex ./viperblock ./predastore
+
+      - name: Run Tests with Race Detector
+        run: make -C spinifex test-race
+
+  lint_and_security:
+    name: Lint and Security
+    runs-on: ubuntu-24.04
+
+    steps:
+      - name: Checkout Spinifex
+        uses: actions/checkout@v6
+        with:
+          path: spinifex
+
+      - name: Checkout Viperblock
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/viperblock
+          ref: dev
+          path: viperblock
+
+      - name: Checkout Predastore
+        uses: actions/checkout@v6
+        with:
+          repository: mulgadc/predastore
+          ref: dev
+          path: predastore
+
+      - name: Setup Go
+        uses: actions/setup-go@v6
+        with:
+          cache-dependency-path: "spinifex/go.sum"
+          go-version-file: "spinifex/go.mod"
+
+      - name: Setup Go Workspace
+        run: go work init ./spinifex ./viperblock ./predastore
+
+      - name: Lint
+        uses: golangci/golangci-lint-action@v9
+        with:
+          version: v2.12.2
+          working-directory: spinifex
+
+      - name: Govulncheck
+        run: make -C spinifex govulncheck
@@ -2,72 +2,58 @@ name: UI Build and Test

 on:
  push:
-    branches: [main, ui]
-    paths: ["hive-ui/**", ".github/workflows/ui.yml"]
+    branches:
+      - main
+      - dev
+    paths:
+      - spinifex/services/spinifexui/frontend/**
+      - .github/workflows/ui.yml
  pull_request:
+    branches:
+      - main
+      - dev
+    paths:
+      - spinifex/services/spinifexui/frontend/**
+      - .github/workflows/ui.yml
  workflow_dispatch:

+permissions:
+  contents: read
+
 jobs:
-  frontend:
-    name: Frontend Build and Test
+  ui_build_and_test:
+    name: UI Build and Test
    runs-on: ubuntu-24.04
    defaults:
      run:
-        working-directory: hive-ui/frontend
+        working-directory: spinifex/services/spinifexui/frontend

    steps:
      - name: Checkout Code
        uses: actions/checkout@v6
+        with:
+          sparse-checkout: spinifex/services/spinifexui/frontend

      - name: Setup pnpm
-        uses: pnpm/action-setup@v4
+        uses: pnpm/action-setup@v6
        with:
-          package_json_file: hive-ui/frontend/package.json
+          package_json_file: spinifex/services/spinifexui/frontend/package.json

      - name: Setup Node.js
        uses: actions/setup-node@v6
        with:
          cache: "pnpm"
-          cache-dependency-path: hive-ui/frontend/pnpm-lock.yaml
-          node-version-file: hive-ui/frontend/.nvmrc
+          cache-dependency-path: spinifex/services/spinifexui/frontend/pnpm-lock.yaml
+          node-version-file: spinifex/services/spinifexui/frontend/.nvmrc

-      - name: Install frontend dependencies
+      - name: Install Dependencies
        run: pnpm install --frozen-lockfile

-      - name: Run Biome formatter check
-        run: pnpm check
+      - name: Run Lint
+        run: pnpm lint

-      - name: Build frontend
+      - name: Run Tests with Coverage
+        run: pnpm test:coverage
+
+      - name: Build Frontend
        run: pnpm build
-
-  webserver:
-    name: Webserver Build and Test
-    runs-on: ubuntu-24.04
-    defaults:
-      run:
-        working-directory: hive-ui/webserver
-
-    steps:
-      - name: Checkout Code
-        uses: actions/checkout@v6
-
-      - name: Setup Go
-        uses: actions/setup-go@v6
-        with:
-          cache-dependency-path: hive-ui/webserver/go.sum
-          go-version-file: hive-ui/webserver/go.mod
-
-      - name: Go Vet
-        run: go vet ./...
-
-      - name: Static Check
-        run: go tool staticcheck ./...
-
-      - name: Go Vulnerability Check
-        run: go tool govulncheck ./...
-
-      - name: Gosec Security Scanner
-        run: go tool gosec -exclude-generated ./...
-
-      - name: Build
-        run: go build ./...
@@ -1,9 +1,25 @@
+# Terraform/OpenTofu
+.terraform/
+.terraform.lock.hcl
+*.tfstate
+*.tfstate.*
+*.tfvars
+!example.tfvars
+scripts/iac/proxmox/clusters/
+
+# Generated SSH private keys from terraform examples (e.g. local_file.nginx_pem)
+*.pem
+
 # Go workspace
 go.work
 go.work.sum

 # Build outputs
 bin/
+dist/
+build/microvm/vmlinuz
+build/microvm/initramfs.cpio.gz
+!spinifex/services/spinifexui/frontend/dist/
 tmp/
 *.test
 *.out
@@ -17,9 +33,17 @@ Thumbs.db

 # Environment
 .env
+.venv/

 # Test reports
 *-report.txt

-# hive-ui
-hive-ui/certs/
+# Internal docs (moved to private mulga repo)
+docs/INTERNAL.md
+docs/PRIVATE.md
+docs/DEVNOTES.md
+docs/PLAN.md
+docs/TODO.md
+docs/development/
+docs/internal/
+scripts/iac/aws/spinifex-test.pem
@@ -0,0 +1,136 @@
+version: "2"
+
+formatters:
+  enable:
+    - gofmt
+  settings:
+    gofmt:
+      simplify: false # modernize linter covers gofmt -s simplifications (Go 1.23+)
+
+linters:
+  enable:
+    - asasalint # catches pass []any as any in variadic func(...any)
+    - asciicheck # checks that code does not contain non-ASCII identifiers
+    - bidichk # checks for dangerous unicode bidi character sequences
+    - bodyclose # checks whether HTTP response body is closed successfully
+    - canonicalheader # checks whether net/http.Header uses canonical header keys
+    - decorder # checks declaration order of types, consts, vars, and funcs
+    - durationcheck # checks for two durations multiplied together
+    - embeddedstructfieldcheck # checks embedded sync.Mutex/RWMutex in structs
+    - errcheck # checks for unchecked errors in type assertions and I/O
+    - errchkjson # checks types passed to json encoding functions, flags omittable error checks
+    - errname # checks that sentinel errors are prefixed with Err, error types suffixed with Error
+    - exptostd # detects golang.org/x/exp/ functions replaceable by stdlib
+    - fatcontext # detects nested contexts in loops
+    - forcetypeassert # flags unchecked type assertions that can panic
+    - gocheckcompilerdirectives # validates go compiler directive comments (//go:)
+    - goprintffuncname # checks that printf-like functions are named with f at the end
+    - gosec # inspects source code for security problems
+    - govet # reports suspicious constructs (Printf alignment, shadow, etc.)
+    - ineffassign # detects assignments to existing variables that are not used
+    - makezero # finds slice declarations with non-zero initial length followed by append
+    - mirror # catches wrong mirror patterns in bytes/strings usage
+    - musttag # checks structs passed to marshal/unmarshal have proper tags
+    - modernize # modernizes code to use newer Go idioms (go fix analyzers)
+    - nakedret # finds naked returns in functions
+    - nilerr # catches code that returns nil even after checking err != nil
+    - nilnesserr # catches returning a different nil error variable after err != nil check
+    - nolintlint # reports ill-formed or insufficient nolint directives
+    - nosprintfhostport # flags fmt.Sprintf host:port — use net.JoinHostPort for IPv6 safety
+    - perfsprint # detects fmt.Sprintf calls replaceable with faster alternatives
+    - predeclared # catches shadowing of Go builtins (len, cap, error, etc.)
+    - reassign # checks that package variables are not reassigned
+    - recvcheck # checks for receiver type consistency (pointer vs value)
+    - sloglint # enforces consistent log/slog usage (key-value pairs, no globals)
+    - staticcheck # extensive static analysis checks
+    - unconvert # removes unnecessary type conversions
+    - usestdlibvars # detects hardcoded values that should use stdlib constants
+    - usetesting # catches os.TempDir() etc. in tests — should use t.TempDir()
+    - wastedassign # finds assignments to variables that are never subsequently read
+    - whitespace # detects leading and trailing whitespace in functions
+
+  settings:
+    embeddedstructfieldcheck:
+      forbid-mutex: true
+
+    errcheck:
+      check-type-assertions: true
+      exclude-functions:
+        - (*github.com/nats-io/nats.go.Subscription).Unsubscribe # cleanup-only, errors not actionable
+        - (*github.com/kdomanski/iso9660.ImageWriter).Cleanup # deferred cleanup, errors not actionable
+
+    govet:
+      enable-all: true
+      disable:
+        - fieldalignment
+        - shadow # too noisy to retrofit — all 50 findings are err re-declarations
+        - unusedwrite # false positives on struct field assignments before marshal/send
+
+    staticcheck:
+      checks:
+        - all
+        - -ST1000 # package comment
+        - -ST1003 # naming conventions (Id→ID etc.) — too noisy to retrofit
+        - -ST1012 # error var naming — errname covers this + error type suffixes
+        - -ST1013 # HTTP status constants — usestdlibvars is broader
+        - -ST1016 # receiver name consistency (too noisy to retrofit)
+        - -ST1020 # godoc on exported types — not enforcing yet
+        - -ST1021 # godoc on exported type groups — not enforcing yet
+        - -ST1022 # godoc on exported vars/consts — not enforcing yet
+        - -SA1019 # deprecated usage (handle separately)
+        - -SA4006 # value overwritten — ineffassign + wastedassign cover this better
+        - -SA9005 # ineffective break (staticcheck false positives)
+
+    gosec:
+      excludes:
+        - G104 # unchecked errors — errcheck is stricter
+        - G116 # bidi chars — bidichk is more thorough
+        - G117 # use of net/http serve functions (we handle TLS separately)
+        - G703 # path traversal taint analysis — false positives on config file loading
+        - G705 # XSS taint analysis — false positives on internal error responses
+        - G706 # log injection taint analysis — false positives on structured slog usage
+
+    modernize:
+      disable:
+        - newexpr # don't replace aws.String() with new() — AWS SDK idiom
+        - stringsbuilder # don't replace small string concatenations with strings.Builder
+
+    perfsprint:
+      concat-loop: false
+      err-error: false # don't flag err.Error()
+      errorf: false # don't convert fmt.Errorf to errors.New
+      strconcat: false
+
+    nakedret:
+      max-func-lines: 0
+
+    nolintlint:
+      require-explanation: true
+      require-specific: true
+
+    reassign:
+      patterns:
+        - ".*"
+
+  exclusions:
+    presets:
+      - std-error-handling
+      - common-false-positives
+    rules:
+      - path: '_test\.go'
+        linters:
+          - bodyclose
+          - errcheck
+          - errchkjson
+          - gosec
+
+      # spinifex-specific: gosec exclusions
+      - path: "cmd/"
+        linters:
+          - errcheck # CLI glue code — viper/cobra/pterm errors not actionable
+          - gosec # matches current -exclude-dir=cmd
+
+      - text: "G204"
+        linters: [gosec] # subprocess execution — legitimate in VM orchestration
+      - text: "G304"
+        linters: [gosec] # file path from variable — necessary for config loading
@@ -1,7 +0,0 @@
-{
-  "recommendations": [
-    "biomejs.biome",
-    "bradlc.vscode-tailwindcss",
-    "golang.go"
-  ]
-}
@@ -1,36 +0,0 @@
-{
-  "files.readonlyInclude": {
-    "**/routeTree.gen.ts": true
-  },
-  "files.watcherExclude": {
-    "**/routeTree.gen.ts": true
-  },
-  "search.exclude": {
-    "**/routeTree.gen.ts": true
-  },
-  "[css][json][jsonc]": {
-    "editor.defaultFormatter": "biomejs.biome",
-    "editor.tabSize": 2
-  },
-  "[javascript][javascriptreact][typescript][typescriptreact]": {
-    "editor.codeActionsOnSave": {
-      "source.fixAll.biome": "explicit",
-      "source.organizeImports.biome": "explicit"
-    },
-    "editor.defaultFormatter": "biomejs.biome",
-    "editor.tabSize": 2
-  },
-  "[go]": {
-    "editor.defaultFormatter": "golang.go",
-    "editor.tabSize": 4
-  },
-  "biome.configurationPath": "./hive-ui/frontend/biome.jsonc",
-  "biome.requireConfiguration": true,
-  "biome.lsp.bin": "./hive-ui/frontend/node_modules/.bin/biome",
-  "editor.formatOnSave": true,
-  "editor.formatOnPaste": true,
-  "files.insertFinalNewline": true,
-  "files.trimFinalNewlines": true,
-  "files.trimTrailingWhitespace": true,
-  "typescript.tsdk": "./hive-ui/frontend/node_modules/typescript/lib"
-}
@@ -1,214 +0,0 @@
-# CLAUDE.md
-
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
-
-## Project Overview
-
-This is the Hive platform - an AWS-compatible infrastructure stack for bare-metal, edge, and on-premise deployments. Hive orchestrates multiple independent components that work together to provide AWS-compatible services.
-
-## Component Repositories
-
-The Hive platform consists of these independent repositories:
-
- **[viperblock](../viperblock/)** - High-performance block storage service (EBS alternative) with WAL-backed storage
- **[predastore](../predastore/)** - S3-compatible object storage service with Reed-Solomon erasure coding
- **nbdkit/** - Network Block Device (NBD) server for exporting block devices (upstream project)
- **hive/** (this repo) - VM orchestration service and platform coordinator
-
-Each component can be used independently or as part of the integrated Hive stack. See component-specific `CLAUDE.md` files for detailed development guidance:
- `../viperblock/CLAUDE.md` - Viperblock development guide
- `../predastore/CLAUDE.md` - Predastore development guide
-
-## Cross-Repository Development Setup
-
-For platform development, clone all repositories to the same parent directory:
-```bash
-mkdir mulga
-cd mulga
-git clone <hive-repo-url> hive
-git clone <viperblock-repo-url> viperblock
-git clone <predastore-repo-url> predastore
-```
-
-The `hive/go.mod` uses local replace directives for cross-component development:
-```go
-replace github.com/mulgadc/viperblock => ../viperblock
-replace github.com/mulgadc/predastore => ../predastore
-```
-
-## Build Commands
-
-### Hive (Platform Orchestrator)
-```bash
-make build          # Build the hive binary
-make test           # Run Go tests
-make bench          # Run benchmarks
-make clean          # Clean build artifacts
-```
-
-### **MANDATORY: Unit Testing Requirements**
-
-**⚠️ CRITICAL: Always run unit tests before any commit or push operation.**
-
-```bash
-# REQUIRED before any git commit or push
-make test           # Must pass with zero failures
-
-# Example workflow:
-make test                    # Verify all tests pass
-git add .                    # Stage changes
-git commit -m "..."          # Only after tests pass
-git push origin main         # Only after tests pass
-```
-
-**Testing Policy:**
- **All unit tests MUST pass** before committing changes
- **No exceptions** - failing tests block commits
- Tests must complete without errors or panics
- Use `make test` command to run the full test suite
- If tests fail, fix issues before proceeding with git operations
-
-### Component Build Commands
-For detailed build instructions, see component-specific CLAUDE.md files:
-
-**Viperblock** (`../viperblock/CLAUDE.md`):
-```bash
-cd ../viperblock && make build    # Build sfs, vblock binaries + NBD plugin
-```
-
-**Predastore** (`../predastore/CLAUDE.md`):
-```bash
-cd ../predastore && make build    # Build s3d binary
-```
-
-**NBDkit** (system dependency):
-```bash
-cd nbdkit && autoreconf -i && ./configure && make && make install
-```
-
-## Architecture Notes
-
-### Message-Driven Microservices Architecture
-
-Hive uses a **scalable message-driven architecture** that separates API handling from request processing:
-
-```
-AWS SDK (custom endpoint) → Hive API Gateway → NATS Topics → Specialized Daemons
-                                    ↑                              ↓
-                         AWS-compatible response ←  NATS Topics  ←  Processing Results
-```
-
-#### Core Components:
-
-1. **Hive API Gateway Service** - TLS endpoint that handles AWS SDK requests
-2. **Specialized Daemon Services** - Horizontally scalable workers for each AWS service type
-3. **NATS Message Broker** - Topic-based routing with queue groups for load balancing
-
-#### Current Implementation (daemon.go):
- **Gateway Logic**: `StartAWSGateway()` handles AWS API calls on port 9999
- **NATS Integration**: Uses request-response pattern: `d.natsConn.Request("ec2.runinstances", data, 30s)`
- **Queue Groups**: Load balancing via `"hive-workers"` queue group
- **Service Coordination**: All inter-service communication via NATS topics
-
-### Inter-component Dependencies
- **Service Startup Order**: NATS → Predastore → Viperblock → NBDkit → Gateway → Daemons
- **Message Flow**: Gateway translates AWS calls to NATS messages, daemons process and respond
- **Scaling**: Multiple daemon instances can subscribe to same topics for horizontal scaling
- All Go projects use local module replacements for development
-
-### Storage Backend Integration
- **Viperblock Integration**: EBS volumes mounted via NATS messaging (`ebs.mount` topic)
- **Predastore Integration**: S3 operations proxied through dedicated S3 daemons
- **NBDkit Integration**: Block device access for VM storage
- **Volume Lifecycle**: Coordinated through NATS between EC2 and EBS daemons
-
-### Key Design Patterns
- **Message-Driven**: All service communication via NATS topics (ec2.*, ebs.*, s3.*, vpc.*)
- **Queue Groups**: Load balancing and fault tolerance via NATS queue groups
- **Request-Response**: 30-second timeout pattern for AWS API compatibility
- **Service Specialization**: Dedicated daemon types for each AWS service
- **Horizontal Scaling**: Multiple daemon instances handle same topic types
-
-### NATS Topic Structure
-
-The system uses structured NATS topics for service communication:
-
-```bash
-# EC2 Service Topics:
-ec2.runinstances              # Launch new instances
-ec2.describeinstances         # Query instance status
-ec2.startinstances           # Start stopped instances
-ec2.stopinstances            # Stop running instances
-ec2.terminateinstances       # Terminate instances
-
-# EBS Service Topics:
-ebs.createvolume             # Create new EBS volume
-ebs.attachvolume             # Attach volume to instance
-ebs.detachvolume             # Detach volume
-ebs.describevolumes          # List volumes
-ebs.mount                    # Mount volume (internal)
-ebs.unmount                  # Unmount volume (internal)
-
-# S3 Service Topics:
-s3.createbucket              # Create S3 bucket
-s3.putobject                 # Store object
-s3.getobject                 # Retrieve object
-s3.listbuckets               # List buckets
-
-# VPC Service Topics:
-vpc.createvpc                # Create VPC
-vpc.createsubnet             # Create subnet
-vpc.describevpcs             # List VPCs
-
-# Health and Discovery:
-health.ec2.daemon.{id}       # Daemon health heartbeats
-discovery.services           # Service registration
-```
-
-### Go Module Structure
-All Go projects use:
- Go 1.23+ with module mode
- Local replace directives for cross-component development
- Standard build flags: `-ldflags "-s -w"` for optimized binaries
- Test environment variable: `LOG_IGNORE=1` to suppress logs during testing
-
-### Development Workflow
-
-#### Quick Setup (Recommended)
-```bash
-# 1. Clone dependencies (if not already done)
-./scripts/clone-deps.sh
-
-# 2. Setup development environment
-./scripts/dev-setup.sh
-
-# 3. Start all services
-./scripts/start-dev.sh
-
-# 4. Stop all services when done
-./scripts/stop-dev.sh
-```
-
-#### Manual Development Process
-1. **Cross-repo Setup**: Clone all repositories to same parent directory
-2. **Service Dependencies**: Start services in order: NATS → Predastore → Viperblock → NBDkit → Gateway → Daemons
-3. **Build Order**: Build components: predastore → viperblock → hive
-4. **Testing**: Use `make test` in each component to verify changes
-5. **Message-Driven Development**: Test NATS message flows between gateway and daemons
-6. **Scaling**: Test with multiple daemon instances subscribing to same topics
-
-#### Development Tools
- **Hot Reloading**: Use `air` for automatic restarts during development
- **TLS Certificates**: Auto-generated self-signed certificates for HTTPS endpoints
- **Log Monitoring**: Centralized logs in `data/logs/` directory
- **AWS CLI Testing**: Test endpoints with `aws --endpoint-url https://localhost:9999`
-
-### Development Roadmap
-For major feature development and architectural changes, refer to `HIVE_DEVELOPMENT_PLAN.md` which contains:
- Comprehensive roadmap for transforming Hive from proof-of-concept to beta release
- Detailed implementation phases with specific tasks and file structures
- AWS API compatibility implementation guidelines
- Development environment automation plans
- Testing and validation frameworks
-
-When implementing new AWS services or enhancing existing functionality, follow the structured approach outlined in the development plan.
@@ -1,399 +0,0 @@
-# Hive Architecture
-
-Hive is an AWS-compatible infrastructure platform for bare-metal, edge, and on-premise deployments. It provides EC2-compatible VM orchestration backed by QEMU/KVM, with storage provided by companion projects Viperblock (EBS) and Predastore (S3).
-
-## High-Level Architecture
-
-```
-┌─────────────────────────────────────────────────────────────────────────────┐
-│                              AWS SDK / CLI                                  │
-│                    (aws ec2 run-instances --endpoint-url ...)               │
-└─────────────────────────────────────────────────────────────────────────────┘
-                                      │
-                                      │ HTTPS (SigV4 Auth)
-                                      ▼
-┌─────────────────────────────────────────────────────────────────────────────┐
-│                           AWS Gateway (Port 9999)                           │
-│                                                                             │
-│   • TLS termination                    • AWS SigV4 authentication           │
-│   • EC2 Query Protocol parsing         • Routes to service handlers         │
-│   • XML response generation            • NATS client connection             │
-│                                                                             │
-│   File: hive/services/awsgw/awsgw.go                                        │
-└─────────────────────────────────────────────────────────────────────────────┘
-                                      │
-                                      │ JSON over NATS
-                                      ▼
-┌─────────────────────────────────────────────────────────────────────────────┐
-│                            NATS Message Broker                              │
-│                                                                             │
-│   Topics:                              Queue Groups:                        │
-│   • ec2.RunInstances                   • hive-workers (load balanced)       │
-│   • ec2.DescribeInstances              • No queue (fan-out to all nodes)    │
-│   • ec2.StartInstances                                                      │
-│   • ec2.StopInstances                                                       │
-│   • ec2.TerminateInstances                                                  │
-│   • ebs.mount / ebs.unmount                                                 │
-└─────────────────────────────────────────────────────────────────────────────┘
-                                      │
-                    ┌─────────────────┼─────────────────┐
-                    ▼                 ▼                 ▼
-┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
-│   Hive Daemon        │ │   Hive Daemon        │ │   Hive Daemon        │
-│   (Node 1)           │ │   (Node 2)           │ │   (Node N)           │
-│                      │ │                      │ │                      │
-│ • NATS subscriber    │ │ • NATS subscriber    │ │ • NATS subscriber    │
-│ • Resource manager   │ │ • Resource manager   │ │ • Resource manager   │
-│ • QEMU/KVM launcher  │ │ • QEMU/KVM launcher  │ │ • QEMU/KVM launcher  │
-│ • QMP monitoring     │ │ • QMP monitoring     │ │ • QMP monitoring     │
-│                      │ │                      │ │                      │
-│ File: daemon.go      │ │ File: daemon.go      │ │ File: daemon.go      │
-└──────────────────────┘ └──────────────────────┘ └──────────────────────┘
-          │                       │                       │
-          ▼                       ▼                       ▼
-┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
-│   QEMU/KVM VMs       │ │   QEMU/KVM VMs       │ │   QEMU/KVM VMs       │
-└──────────────────────┘ └──────────────────────┘ └──────────────────────┘
-```
-
-## Request Flow
-
-### 1. AWS SDK Request
-
-Users interact with Hive using standard AWS SDKs or the AWS CLI by pointing to a custom endpoint:
-
-```bash
-aws --endpoint-url https://localhost:9999 --no-verify-ssl ec2 run-instances \
-    --image-id ami-debian12 \
-    --instance-type t3.micro \
-    --key-name my-keypair
-```
-
-The AWS SDK formats this as an HTTP POST with:
- AWS SigV4 authentication headers
- EC2 Query Protocol body (Action=RunInstances&ImageId=ami-debian12...)
-
-### 2. AWS Gateway
-
-The gateway (`hive/services/awsgw/awsgw.go:57-107`) is the entry point:
-
-```go
-// awsgw.go:74 - Connect to NATS
-natsConn, err := nats.Connect(nodeConfig.NATS.Host, opts...)
-
-// awsgw.go:89-94 - Create gateway with NATS connection
-gw := gateway.GatewayConfig{
-    NATSConn: natsConn,
-    Config:   nodeConfig.AWSGW.Config,
-}
-
-// awsgw.go:103 - Start TLS listener
-log.Fatal(app.ListenTLS(nodeConfig.AWSGW.Host, nodeConfig.AWSGW.TLSCert, nodeConfig.AWSGW.TLSKey))
-```
-
-Request routing (`hive/gateway/gateway.go:125-156`):
-
-1. **Authentication**: SigV4 middleware validates AWS credentials (`gateway.go:104`)
-2. **Service Detection**: Extracts service name (ec2, iam, account) from Authorization header (`gateway.go:158-186`)
-3. **Action Dispatch**: Routes to service-specific handler (`gateway.go:136-145`)
-
-```go
-// gateway.go:136-145
-switch svc {
-case "ec2":
-    err = gw.EC2_Request(ctx)
-case "account":
-    err = gw.Account_Request(ctx)
-case "iam":
-    err = gw.IAM_Request(ctx)
-}
-```
-
-### 3. EC2 Handler
-
-The EC2 handler (`hive/gateway/ec2.go`) parses the Action parameter and delegates to specific handlers:
-
-```go
-// ec2.go - Action routing
-switch action {
-case "RunInstances":
-    output, err := gateway_ec2_instance.RunInstances(input, gw.NATSConn)
-case "DescribeInstances":
-    output, err := gateway_ec2_instance.DescribeInstances(input, gw.NATSConn, expectedNodes)
-// ... more actions
-}
-```
-
-### 4. NATS Messaging
-
-The gateway communicates with daemons via NATS request/response (`hive/handlers/ec2/instance/service_nats.go:24-57`):
-
-```go
-// service_nats.go:24-57
-func (s *NATSInstanceService) RunInstances(input *ec2.RunInstancesInput) (*ec2.Reservation, error) {
-    // Marshal input to JSON
-    jsonData, err := json.Marshal(input)
-
-    // Send request to daemon via NATS with 5 minute timeout
-    msg, err := s.natsConn.Request("ec2.RunInstances", jsonData, 5*time.Minute)
-
-    // Unmarshal response
-    var reservation ec2.Reservation
-    err = json.Unmarshal(msg.Data, &reservation)
-
-    return &reservation, nil
-}
-```
-
-### 5. Daemon Processing
-
-Daemons (`hive/daemon/daemon.go`) subscribe to NATS topics and handle requests:
-
-```go
-// daemon.go:319 - Subscribe with queue group for load balancing
-d.natsSubscriptions["ec2.RunInstances"], err = d.natsConn.QueueSubscribe(
-    "ec2.RunInstances",
-    "hive-workers",      // Queue group - only one daemon handles each request
-    d.handleEC2RunInstances,
-)
-
-// daemon.go:373 - Subscribe without queue group for fan-out
-d.natsSubscriptions["ec2.DescribeInstances"], err = d.natsConn.Subscribe(
-    "ec2.DescribeInstances",
-    d.handleEC2DescribeInstances,  // All daemons respond
-)
-```
-
-**Queue Groups**: Topics subscribed with a queue group (`hive-workers`) are load-balanced - only one daemon handles each request. Topics without a queue group fan out to all daemons.
-
-### 6. VM Launch
-
-When a daemon handles `RunInstances`:
-
-1. **Resource Check**: Validates CPU/memory availability (`daemon.go:66-74`)
-2. **Volume Generation**: Creates boot, cloud-init, and EFI volumes via Viperblock
-3. **Volume Mount**: Sends `ebs.mount` request to Viperblock, receives NBD URI
-4. **QEMU Launch**: Builds and executes QEMU command with NBD-backed disks
-5. **QMP Monitoring**: Establishes QEMU Machine Protocol connection for VM management
-
-```go
-// daemon.go - Volume mounting via NATS
-msg, err := d.natsConn.Request("ebs.mount", ebsMountRequest, 10*time.Second)
-
-// daemon.go - QEMU launch with NBD storage
-cmd := exec.Command("qemu-system-x86_64",
-    "-drive", fmt.Sprintf("file=nbd:%s,format=raw", nbdURI),
-    // ... additional QEMU args
-)
-```
-
-## Key Components
-
-### Daemon Structure
-
-```go
-// daemon.go:76-102
-type Daemon struct {
-    node            string                  // Node identifier
-    clusterConfig   *config.ClusterConfig   // Cluster-wide configuration
-    config          *config.Config          // Node-specific configuration
-    natsConn        *nats.Conn              // NATS connection
-    resourceMgr     *ResourceManager        // CPU/Memory tracking
-    instanceService *InstanceServiceImpl    // EC2 instance operations
-    keyService      *KeyServiceImpl         // SSH key operations
-    imageService    *ImageServiceImpl       // AMI operations
-    Instances       vm.Instances            // Local VMs
-    clusterApp      *fiber.App              // HTTP cluster manager
-}
-```
-
-### Resource Manager
-
-Tracks available and allocated CPU/memory to prevent overcommit:
-
-```go
-// daemon.go:66-74
-type ResourceManager struct {
-    mu            sync.RWMutex
-    availableVCPU int
-    availableMem  float64
-    allocatedVCPU int
-    allocatedMem  float64
-    instanceTypes map[string]InstanceType  // t3.micro, t3.small, etc.
-}
-```
-
-### NATS Topics
-
-| Topic | Queue Group | Purpose |
-|-------|-------------|---------|
-| `ec2.RunInstances` | `hive-workers` | Launch new instances |
-| `ec2.DescribeInstances` | None (fan-out) | Query all nodes for instances |
-| `ec2.StartInstances` | `hive-workers` | Start stopped instances |
-| `ec2.StopInstances` | `hive-workers` | Stop running instances |
-| `ec2.TerminateInstances` | `hive-workers` | Terminate instances |
-| `ec2.CreateKeyPair` | `hive-workers` | Generate SSH keypair |
-| `ec2.DescribeKeyPairs` | `hive-workers` | List SSH keypairs |
-| `ec2.DescribeImages` | `hive-workers` | List AMIs |
-| `ebs.mount` | - | Mount volume via Viperblock |
-| `ebs.unmount` | - | Unmount volume |
-| `hive.admin.{node}.health` | None | Node health checks |
-
-### Multi-Node Aggregation
-
-For operations that need data from all nodes (like `DescribeInstances`), the gateway uses inbox-based fan-out (`hive/gateway/ec2/instance/DescribeInstances.go`):
-
-```go
-// DescribeInstances.go:16-111
-func DescribeInstances(...) {
-    // Create unique inbox for collecting responses
-    inbox := nats.NewInbox()
-    sub, err := natsConn.SubscribeSync(inbox)
-
-    // Publish to all nodes (no queue group = all daemons respond)
-    err = natsConn.PublishRequest("ec2.DescribeInstances", inbox, jsonData)
-
-    // Collect responses from all nodes with 3-second timeout
-    for {
-        msg, err := sub.NextMsg(3 * time.Second)
-        allReservations = append(allReservations, nodeOutput.Reservations...)
-    }
-
-    return allReservations
-}
-```
-
-## Storage Integration
-
-### Viperblock (EBS)
-
-Block storage volumes are managed via NATS messages to Viperblock:
-
-```go
-// Volume mount request
-type EBSRequest struct {
-    Name      string  // Volume name (vol-xxxxx)
-    VolType   string  // gp2, io1, etc.
-    Boot      bool    // Boot volume flag
-    EFI       bool    // EFI boot volume
-    CloudInit bool    // Cloud-init volume
-    NBDURI    string  // NBD URI returned from mount
-}
-```
-
-Viperblock responds with an NBD URI that QEMU uses to access the block device.
-
-### Predastore (S3)
-
-Object storage used for:
- AMI image storage
- SSH public key storage (`/bucket/ec2/{key-name}.pub`)
- Volume metadata and configuration
-
-## Configuration
-
-Cluster configuration (`hive/config/config.go`):
-
-```go
-type ClusterConfig struct {
-    Epoch   uint64            // Incremented on config changes
-    Node    string            // This node's identifier
-    Version string            // Hive version
-    Nodes   map[string]Config // Configuration for all nodes
-}
-
-type Config struct {
-    Node, Host, Region, AZ string
-    DataDir    string
-    Daemon     DaemonConfig     // Cluster manager port
-    NATS       NATSConfig       // NATS broker address
-    Predastore PredastoreConfig // S3 endpoint
-    AWSGW      AWSGWConfig      // Gateway port, TLS certs
-    AccessKey, SecretKey string
-}
-```
-
-## File Structure
-
-```
-hive/
-├── hive/
-│   ├── services/
-│   │   └── awsgw/
-│   │       └── awsgw.go        # AWS Gateway service entry point
-│   ├── gateway/
-│   │   ├── gateway.go          # Request routing, auth middleware
-│   │   ├── ec2.go              # EC2 action dispatcher
-│   │   └── ec2/
-│   │       └── instance/
-│   │           ├── RunInstances.go
-│   │           └── DescribeInstances.go
-│   ├── handlers/
-│   │   └── ec2/
-│   │       ├── instance/
-│   │       │   ├── service_nats.go  # NATS request/response
-│   │       │   └── service_impl.go  # Business logic
-│   │       ├── key/
-│   │       └── image/
-│   ├── daemon/
-│   │   └── daemon.go           # Daemon entry point, NATS subscriptions
-│   ├── config/
-│   │   └── config.go           # Configuration structures
-│   ├── vm/
-│   │   └── instance.go         # VM instance representation
-│   └── qmp/
-│       └── qmp.go              # QEMU Machine Protocol client
-└── cmd/
-    └── hive/
-        └── main.go             # CLI entry point
-```
-
-## Development
-
-### Running the Stack
-
-1. Start NATS broker
-2. Start Predastore (S3)
-3. Start Viperblock (EBS)
-4. Start Hive Gateway (`hive awsgw`)
-5. Start Hive Daemon(s) (`hive daemon`)
-
-### Testing with AWS CLI
-
-```bash
-# Set endpoint
-export AWS_ENDPOINT_URL=https://localhost:9999
-
-# Run instance
-aws --no-verify-ssl ec2 run-instances \
-    --image-id ami-debian12 \
-    --instance-type t3.micro
-
-# List instances (queries all nodes)
-aws --no-verify-ssl ec2 describe-instances
-
-# Terminate instance
-aws --no-verify-ssl ec2 terminate-instances --instance-ids i-xxxxx
-```
-
-
-## Improvements
-
-Improve `aws ec2 stop-instances` , currently uses a ~30 sec heartbeat to detect if an instance is terminated. When the stop-instance command is sent, run the heartbeat more frequently to detect closer to realtime when the instance has stopped.
-
-Currently a ~54 second window to detect termination.
-
-```sh
-aws ec2 stop-instances --instance-ids i-1e306bb58b99e03bd
-```
-
-```sh
-2026/01/22 20:01:52 INFO QMP event event=POWERDOWN
-2026/01/22 20:02:17 INFO QMP heartbeat instance=i-1e306bb58b99e03bd
-2026/01/22 20:02:17 INFO QMP status status="{\"running\":true,\"singlestep\":false,\"status\":\"running\"}"
-2026/01/22 20:02:47 INFO QMP heartbeat instance=i-1e306bb58b99e03bd
-2026/01/22 20:02:47 ERROR Failed to send QMP command err="encode error: write unix @->/run/user/1000/qmp-i-1e306bb58b99e03bd.sock: write: broken pipe"
-2026/01/22 20:02:47 INFO QMP Status - Instance stopped, exiting heartbeat id=i-1e306bb58b99e03bd
-2026/01/22 20:02:47 INFO Unsubscribing from NATS subject instance=i-1e306bb58b99e03bd
-2026/01/22 20:02:47 INFO Closing QMP client connection instance=i-1e306bb58b99e03bd
-```
@@ -1,669 +0,0 @@
-# Hive Installation
-
-Notes for development environment installation.
-
-## Dependencies
-
-The current supported OS that is validated and tested includes:
-
-* Ubuntu 22.04
-* Ubuntu 24.04
-* Ubuntu 25.10
-
-For the development preview please use one of the supported versions above.
-
-## Download
-
-Create the base directory for the Hive development environment.
-
-```bash
-mkdir -p ~/Development/mulga/
-cd ~/Development/mulga/
-git clone https://github.com/mulgadc/hive.git
-```
-
-### Quick Install
-
-To bootstrap the dependencies of Hive in one simple step (QEMU, Go, AWS CLI)
-
-```bash
-sudo make -C hive quickinstall
-```
-
-### Manual Install
-
-Alternatively, run the following steps to manually setup the required dependencies.
-
-```bash
-sudo add-apt-repository universe
-sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev make gcc unzip xz-utils file
-```
-
-Ensure the Go toolkit is installed for version 1.25.5 or higher. Recommended to install the latest directly from [https://go.dev/dl/](https://go.dev/dl/).
-
-Confirm go is correctly installed, and set in your $PATH.
-
-```bash
-go version
-```
-
-Hive provides AWS API/SDK layer functionality, which requires the AWS CLI tool to be installed to interface with the system.
-
-```bash
-curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
-unzip awscliv2.zip
-sudo ./aws/install
-```
-
-Confirm awscli version > 2.0 is installed which is required by Hive.
-
-[https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
-
-```bash
-aws --version
-```
-
-## Build
-
-Setup the dev environment and package dependencies on [viperblock](https://github.com/mulgadc/viperblock/) and [predastore](https://github.com/mulgadc/predastore/).
-
-```bash
-cd hive
-./scripts/clone-deps.sh    # Clone viperblock + predastore repositories
-./scripts/dev-setup.sh     # Setup complete development environment
-```
-
-Once complete, confirm `./bin/hive` exists and executable.
-
-# Single Node Installation
-
-For rapid development and testing, `hive` can be installed locally as a single node instance. Follow the instructions below for a complete working environment.
-
-## Init
-
-When running Hive for the first time, run the init function to create the default directories for data, config files and layout required.
-
-Example single node installation to get started, this will create a new region `ap-southeast-2` and availability zone `ap-southeast-2a` on your local instance:
-
-```bash
-./bin/hive admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
-```
-
-## Trust CA Certificate (Required)
-
-Hive generates a local Certificate Authority (CA) during initialization. For AWS CLI and other tools to trust Hive services over HTTPS, you must add the CA certificate to your system's trust store.
-
-```bash
-sudo cp ~/hive/config/ca.pem /usr/local/share/ca-certificates/hive-ca.crt
-sudo update-ca-certificates
-```
-
-Verify the certificate was added:
-
-```bash
-ls -la /usr/local/share/ca-certificates/hive-ca.crt
-```
-
-This step is required for AWS CLI 2.33+ which requires proper CA trust chains. Without this, you would need to use `--no-verify-ssl` for every AWS CLI command.
-
-Next, set the AWS profile to use `hive` which points to the local environment.
-
-```bash
-export AWS_PROFILE=hive
-```
-
-## Launch Services
-
-Start the core services for development.
-
-```bash
-./scripts/start-dev.sh
-```
-
-## Create SSH Key
-
-For first install, create or import an existing key pair which can be used to launch EC2 instances.
-
-### Import Existing Key
-
-Import an existing key pair, replace `~/.ssh/id_rsa.pub` with your specified key.
-
-```bash
-aws ec2 import-key-pair --key-name "hive-key" --public-key-material fileb://~/.ssh/id_rsa.pub
-```
-
-If no key exists, generate one using `ssh-keygen` and repeat the command above.
-
-```bash
-ssh-keygen -t rsa
-```
-
-### Create New Key
-
-Alternatively, create a new key pair using the AWS CLI tool and store the JSON output of the AWS SDK using the `jq` command.
-
-```bash
-aws ec2 create-key-pair \
-  --key-name hive-key \
-| jq -r '.KeyMaterial | rtrimstr("\n")' > ~/.ssh/hive-key
-```
-
-Update permissions for the key for ssh to accept reading the file.
-
-```bash
-chmod 600 ~/.ssh/hive-key
-```
-
-Next, generate a public key from the specified private key pair.
-
-```bash
-ssh-keygen -y -f ~/.ssh/hive-key > ~/.ssh/hive-key.pub
-```
-
-### Validate New Key
-
-```bash
-aws ec2 describe-key-pairs
-```
-
-```json
-{
-  "KeyPairs": [
-    {
-      "KeyPairId": "key-3caa0a34f53d12a3",
-      "KeyType": "ed25519",
-      "Tags": [],
-      "CreateTime": "2025-10-28T13:39:23.458000+00:00",
-      "KeyName": "hive-key",
-      "KeyFingerprint": "SHA256:/g/A5OkeZeSydz9WUErXYVdCt00b0VbfN6RLn2YVFAY"
-    }
-  ]
-}
-```
-
-## Create AMI Template
-
-Use the hive CLI tool to import a selected OS image. Note, the source file can be compressed (e.g image.tar.gz, image.gz, image.tar.xz) and the tool will automatically extract and upload the raw OS image as an AMI after validation the disk image contains a UEFI/BIOS boot capability.
-
-Note, when downloading OS images, use supported platforms that support the `cloud-init` feature to automatically bootstrap when using the Hive EC2 functionality to access SSH and networking services.
-
-### Automatic Image Import
-
-Discover available images to automatically download and install. This will pull the images from the distro official mirror and simplify the process to bootstrap a Hive installation with AMIs that include common operating systems.
-
-```bash
-./bin/hive admin images list
-```
-
-```bash
-NAME                 | DISTRO | VERSION | ARCH   | BOOT
-alpine-3.22.2-x86_64 | alpine | 3.22.2  | x86_64 | bios
-debian-12-arm64      | debian | 12      | arm64  | bios
-debian-12-x86_64     | debian | 12      | x86_64 | bios
-ubuntu-24.04-arm64   | ubuntu | 24.04   | arm64  | bios
-ubuntu-24.04-x86_64  | ubuntu | 24.04   | x86_64 | bios
-```
-
-Next, choose the image you would like to import as an AMI. Choose the appropriate `x86_64` or `arm64` architecture depending on your platform.
-
-```bash
-./bin/hive admin images import --name debian-12-arm64 --force
-```
-
-Make note of the Image-ID `ami-XXX`
-
-```bash
-✅ Image import complete. Image-ID (AMI): ami-e29fcc65734aec9ea
-```
-
-Next, verify available disk images to confirm the import was successful, replace `ami-XXX` with the value from the command output.
-
-```bash
-aws ec2 describe-images --image-ids ami-XXX
-```
-
-### Manual Image Import
-
-Using this method you can import any OS disk image. For example, download the Debian 12 image from the repository [https://cloud.debian.org/images/cloud/bookworm/latest/](https://cloud.debian.org/images/cloud/bookworm/latest/).
-
-Download the image:
-
-```bash
-wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-arm64.tar.xz -O ~/debian-12-genericcloud-arm64.tar.xz
-```
-
-Import as an AMI to the backend store:
-
-```bash
-./bin/hive admin images import --file ~/debian-12-genericcloud-arm64.tar.xz --arch arm64 --distro debian --version 12
-```
-
-Next, verify available disk images to confirm the import was successful.
-
-```bash
-aws ec2 describe-images
-```
-
-## Run Instance
-
-Once Hive is successfully installed and bootstrapped with a system AMI and SSH keys, proceed to run an instance. Replace `ami-XXX` with your imported ImageId above.
-
-```bash
-export HIVE_AMI="ami-XXX"
-```
-
-### Query instance types available
-
-Depending on the host platform Hive is installed for the CPU capabilities and vendor (e.g AMD, Intel, ARM), different compute instances will be available.
-
-Query available instance types and choose an instance type available on your host with 2 vCPUs:
-
-```bash
-aws ec2 describe-instance-types
-```
-
-```json
-{
-  "InstanceTypes": [
-    {
-      "InstanceType": "m8a.small",
-      "CurrentGeneration": true,
-      "SupportedRootDeviceTypes": [
-        "ebs"
-      ],
-      "SupportedVirtualizationTypes": [
-        "hvm"
-      ],
-      "Hypervisor": "kvm",
-      "ProcessorInfo": {
-        "SupportedArchitectures": [
-          "x86_64"
-        ]
-      },
-      "VCpuInfo": {
-        "DefaultVCpus": 2
-      },
-      "MemoryInfo": {
-        "SizeInMiB": 1024
-      },
-      "BurstablePerformanceSupported": false
-    },
-  ]
-}
-```
-
-Export the instance-type:
-
-```sh
-export HIVE_INSTANCE="m8a.small"
-```
-
-Next, launch a new instance, note `hive-key` is the SSH key specified in the previous stage. Be sure to specify the instance-type that is available from your host as above.
-
-```bash
-aws ec2 run-instances \
-  --image-id $HIVE_AMI \
-  --instance-type $HIVE_INSTANCE \
-  --key-name hive-key \
-  --security-group-ids sg-0123456789abcdef0 \
-  --subnet-id subnet-6e7f829e \
-  --count 1
-```
-
-A sample response is below from the `RunInstance` request, note the `InstanceId` attribute:
-
-```json
-{
-  "ReservationId": "r-f101157331e261a68",
-  "OwnerId": "123456789012",
-  "Instances": [
-    {
-      "InstanceId": "i-36765eb0c6609e4d2",
-      "ImageId": "ami-15c3dcfe607460f15",
-      "State": {
-        "Code": 0,
-        "Name": "pending"
-      },
-      "KeyName": "hive-key",
-      "InstanceType": "t3.micro",
-      "LaunchTime": "2025-11-12T13:07:47.548000+00:00"
-    }
-  ]
-}
-```
-
-Export the instance ID for following the rest of the tutorial.
-
-```bash
-export INSTANCE_ID="i-XXX"
-```
-
-Next, validate the running instance is ready.
-
-```bash
-aws ec2 describe-instances --instance-ids $INSTANCE_ID
-```
-
-Confirm the `State.Name` attribute is set as `running`.
-
-```json
-{
-  "Reservations": [
-    {
-      "ReservationId": "r-f101157331e261a68",
-      "OwnerId": "123456789012",
-      "Instances": [
-        {
-          "InstanceId": "i-36765eb0c6609e4d2",
-          "ImageId": "ami-15c3dcfe607460f15",
-          "State": {
-            "Code": 16,
-            "Name": "running"
-          },
-          "KeyName": "hive-key",
-          "InstanceType": "t3.micro",
-          "LaunchTime": "2025-11-12T13:07:47.548000+00:00"
-        }
-      ]
-    }
-  ]
-}
-```
-
-## SSH Connection (development)
-
-For a Hive development environment (toggled off for production), a local SSH port forwaring will be active to connect directly to the instance, regardless of the VPC and network settings.
-
-Determine the SSH port allocated.
-
-```bash
-ps auxw | grep $INSTANCE_ID
-```
-
-```bash
-qemu-system-x86_64 -daemonize -pidfile /run/user/1000/i-36765eb0c6609e4d.pid -qmp unix:/run/user/1000/qmp-i-36765eb0c6609e4d2.sock,server,nowait -enable-kvm -M ubuntu -serial pty -cpu host -smp 2 -m 1024 -drive file=nbd://127.0.0.1:42653,format=raw,if=none,media=disk,id=os -drive file=nbd://127.0.0.1:44499,format=raw,if=virtio,media=cdrom,id=cloudinit -device virtio-blk-pci,drive=os,bootindex=1 -device virtio-rng-pci -device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:33683-:22
-```
-
-Note the `hostfwd=tcp:127.0.0.1:33683-:22`, in this case the local port `33683` will connect to the new instance.
-
-```bash
-ssh -i ~/.ssh/hive-key ec2-user@127.0.0.1 -p 33683
-
-...
-
-Linux hive-vm-36765eb0 6.1.0-40-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.153-1 (2025-09-20) x86_64
-ec2-user@hive-vm-36765eb0:~$
-```
-
-Congratulations, your first AMI image is imported, a new EC2 instance launched, and successfully connected via SSH for the configured SSH key, using the OS `cloud-init` procedure.
-
-## Managing Instances
-
-### Stop Instance
-
-To stop a running instance gracefully:
-
-```bash
-aws ec2 stop-instances --instance-ids $INSTANCE_ID
-```
-
-```json
-{
-    "StoppingInstances": [
-        {
-            "InstanceId": "i-36765eb0c6609e4d2",
-            "CurrentState": {
-                "Code": 64,
-                "Name": "stopping"
-            },
-            "PreviousState": {
-                "Code": 16,
-                "Name": "running"
-            }
-        }
-    ]
-```
-
-Next, confirm the instance has stopped as requested.
-
-```bash
-aws ec2 describe-instances --instance-ids $INSTANCE_ID
-```
-
-```json
-{
-  "Reservations": [
-    {
-      "ReservationId": "r-f101157331e261a68",
-      "OwnerId": "123456789012",
-      "Instances": [
-        {
-          "InstanceId": "i-36765eb0c6609e4d2",
-          "ImageId": "ami-15c3dcfe607460f15",
-          "State": {
-            "Code": 80,
-            "Name": "stopped"
-          },
-          "KeyName": "hive-key",
-          "InstanceType": "t3.micro",
-          "LaunchTime": "2025-11-12T13:07:47.548000+00:00"
-        }
-      ]
-    }
-  ]
-}
-```
-
-### Start Instance
-
-To start a previously stopped instance:
-
-```bash
-aws ec2 start-instances --instance-ids $INSTANCE_ID
-```
-
-```json
-{
-    "StartingInstances": [
-        {
-            "InstanceId": "i-36765eb0c6609e4d2",
-            "CurrentState": {
-                "Code": 0,
-                "Name": "pending"
-            },
-            "PreviousState": {
-                "Code": 80,
-                "Name": "stopped"
-            }
-        }
-    ]
-```
-
-Next, validate the instance is running as expected.
-
-```bash
-aws ec2 describe-instances  --instance-ids "i-36765eb0c6609e4d2"
-```
-
-```json
-{
-  "Reservations": [
-    {
-      "ReservationId": "r-f101157331e261a68",
-      "OwnerId": "123456789012",
-      "Instances": [
-        {
-          "InstanceId": "i-36765eb0c6609e4d2",
-          "ImageId": "ami-15c3dcfe607460f15",
-          "State": {
-            "Code": 16,
-            "Name": "running"
-          },
-          "KeyName": "hive-key",
-          "InstanceType": "t3.micro",
-          "LaunchTime": "2025-11-12T13:07:47.548000+00:00"
-        }
-      ]
-    }
-  ]
-}
-```
-
-### Terminate Instance
-
-To terminate an instance, which will first stop the instance, and on success, remove the EBS volumes and permanately remove the instance data.
-
-```bash
-aws ec2 terminate-instances --instance-ids $INSTANCE_ID
-```
-
-```json
-{
-  "TerminatingInstances": [
-    {
-      "InstanceId": "i-36765eb0c6609e4d2",
-      "CurrentState": {
-        "Code": 16,
-        "Name": "running"
-      },
-      "PreviousState": {
-        "Code": 16,
-        "Name": "running"
-      }
-    }
-  ]
-}
-```
-
-Next, validate the instance is removed, this may take a few minutes depending the instance volume size.
-
-```bash
-aws ec2 describe-instances  --instance-ids $INSTANCE_ID
-```
-
-On success no data will be returned, since the instance is no longer available.
-
-# Multi-node Configuration
-
-To create a simulate multi-node installation on a single server (e.g node1, node2, node3) review the script `./scripts/create-multi-node.sh`.
-
-Specifically this will create 3 IPs on a specified ethernet adapter.
-
-```bash
-ip addr add 10.11.12.1/24 dev eth0
-ip addr add 10.11.12.2/24 dev eth0
-ip addr add 10.11.12.3/24 dev eth0
-```
-
-Next, `hive` will be installed and initiated in `$HOME/node1` for configuration files.
-
-```bash
-# Initialize node1
-./bin/hive admin init \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node1 \
--bind 10.11.12.1 \
--cluster-bind 10.11.12.1 \
--port 4432 \
--hive-dir ~/node1/ \
--config-dir ~/node1/config/
-```
-
-Two other nodes will be created, joining node1 to exchange configuration files and identify as a new node.
-
-```bash
-# Join cluster
-./bin/hive admin join \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node2 \
--bind 10.11.12.2 \
--cluster-bind 10.11.12.2 \
--cluster-routes 10.11.12.1:4248 \
--host 10.11.12.1:4432 \
--data-dir ~/node2/ \
--config-dir ~/node2/config/ \
-
-# Start node2
-./scripts/start-dev.sh ~/node2/
-
-echo "Node3 Setup:"
-
-./bin/hive admin join \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node3 \
--bind 10.11.12.3 \
--cluster-bind 10.11.12.3 \
--cluster-routes 10.11.12.1:4248 \
--host 10.11.12.1:4432 \
--data-dir ~/node3/ \
--config-dir ~/node3/config/
-
-./scripts/start-dev.sh ~/node3/
-```
-
-Once configured, each node will have it's own storage for config, data and state in `~/node[1,2,3]` to simulate a unique node and network.
-
-Once the multi-node is configured, follow the original installation instructions above to:
-
- Import an SSH key
- Clone an AMI
- Launch an instance
-
-Note - when using the AWS CLI tool you must specify the IP address of the node (10.11.12.1, 10.11.12.2, or 10.11.12.3).
-
-**Important**: For multi-node setups, the server certificates need to include the additional IP addresses. You may need to regenerate certificates with the `--force` flag after configuring multi-node, or ensure the CA certificate is trusted system-wide (see "Trust CA Certificate" section above).
-
-```bash
-aws --endpoint-url https://10.11.12.3:9999/ ec2 describe-instances --instance-ids i-9f5f648adc57ea46d
-```
-
-```json
-{
-    "Reservations": [
-        {
-            "ReservationId": "r-9d26866c3b0d53bf4",
-            "OwnerId": "123456789012",
-            "Instances": [
-                {
-                    "InstanceId": "i-9f5f648adc57ea46d",
-                    "ImageId": "ami-d0de73dd18fa33ac9",
-                    "State": {
-                        "Code": 16,
-                        "Name": "running"
-                    },
-                    "KeyName": "hive-key",
-                    "InstanceType": "t3.micro",
-                    "LaunchTime": "2025-11-21T10:36:42.834000+00:00"
-                }
-            ]
-        }
-    ]
-```
-
-For debugging, reference the node unique configuration files, e.g
-
-```bash
-tail -n 100 ~/node3/logs/hive.log
-```
-
-```bash
-...
-2025/11/21 22:59:32 Received message on subject: ec2.DescribeInstances
-2025/11/21 22:59:32 Message data: {"DryRun":null,"Filters":null,"InstanceIds":["i-9f5f648adc57ea46d"],"MaxResults":null,"NextToken":null}
-2025/11/21 22:59:32 INFO Processing DescribeInstances request from this node
-2025/11/21 22:59:32 INFO handleEC2DescribeInstances completed count=1
-...
-```
-
-### Stop multi-node Instance
-
-To stop a simulated multi-node instance:
-
-```bash
-./scripts/stop-multi-node.sh
-```
-
-Note if multiple EC2 instances are running, it make take a few minutes to gracefully terminate the instance, unmount the attached EBS volume (via NBD) and push the write-ahead-log (WAL) to the S3 server (predastore).
@@ -1,201 +1,661 @@
-                                 Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
+                    GNU AFFERO GENERAL PUBLIC LICENSE
+                       Version 3, 19 November 2007

-   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.

-   1. Definitions.
+                            Preamble

-      "License" shall mean the terms and conditions for use, reproduction,
-      and distribution as defined by Sections 1 through 9 of this document.
+  The GNU Affero General Public License is a free, copyleft license for
+software and other kinds of works, specifically designed to ensure
+cooperation with the community in the case of network server software.

-      "Licensor" shall mean the copyright owner or entity authorized by
-      the copyright owner that is granting the License.
+  The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works.  By contrast,
+our General Public Licenses are intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users.

-      "Legal Entity" shall mean the union of the acting entity and all
-      other entities that control, are controlled by, or are under common
-      control with that entity. For the purposes of this definition,
-      "control" means (i) the power, direct or indirect, to cause the
-      direction or management of such entity, whether by contract or
-      otherwise, or (ii) ownership of fifty percent (50%) or more of the
-      outstanding shares, or (iii) beneficial ownership of such entity.
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.

-      "You" (or "Your") shall mean an individual or Legal Entity
-      exercising permissions granted by this License.
+  Developers that use our General Public Licenses protect your rights
+with two steps: (1) assert copyright on the software, and (2) offer
+you this License which gives you legal permission to copy, distribute
+and/or modify the software.

-      "Source" form shall mean the preferred form for making modifications,
-      including but not limited to software source code, documentation
-      source, and configuration files.
+  A secondary benefit of defending all users' freedom is that
+improvements made in alternate versions of the program, if they
+receive widespread use, become available for other developers to
+incorporate.  Many developers of free software are heartened and
+encouraged by the resulting cooperation.  However, in the case of
+software used on network servers, this result may fail to come about.
+The GNU General Public License permits making a modified version and
+letting the public access it on a server without ever releasing its
+source code to the public.

-      "Object" form shall mean any form resulting from mechanical
-      transformation or translation of a Source form, including but
-      not limited to compiled object code, generated documentation,
-      and conversions to other media types.
+  The GNU Affero General Public License is designed specifically to
+ensure that, in such cases, the modified source code becomes available
+to the community.  It requires the operator of a network server to
+provide the source code of the modified version running there to the
+users of that server.  Therefore, public use of a modified version, on
+a publicly accessible server, gives the public access to the source
+code of the modified version.

-      "Work" shall mean the work of authorship, whether in Source or
-      Object form, made available under the License, as indicated by a
-      copyright notice that is included in or attached to the work
-      (an example is provided in the Appendix below).
+  An older license, called the Affero General Public License and
+published by Affero, was designed to accomplish similar goals.  This is
+a different license, not a version of the Affero GPL, but Affero has
+released a new version of the Affero GPL which permits relicensing under
+this license.

-      "Derivative Works" shall mean any work, whether in Source or Object
-      form, that is based on (or derived from) the Work and for which the
-      editorial revisions, annotations, elaborations, or other modifications
-      represent, as a whole, an original work of authorship. For the purposes
-      of this License, Derivative Works shall not include works that remain
-      separable from, or merely link (or bind by name) to the interfaces of,
-      the Work and Derivative Works thereof.
+  The precise terms and conditions for copying, distribution and
+modification follow.

-      "Contribution" shall mean any work of authorship, including
-      the original version of the Work and any modifications or additions
-      to that Work or Derivative Works thereof, that is intentionally
-      submitted to Licensor for inclusion in the Work by the copyright owner
-      or by an individual or Legal Entity authorized to submit on behalf of
-      the copyright owner. For the purposes of this definition, "submitted"
-      means any form of electronic, verbal, or written communication sent
-      to the Licensor or its representatives, including but not limited to
-      communication on electronic mailing lists, source code control systems,
-      and issue tracking systems that are managed by, or on behalf of, the
-      Licensor for the purpose of discussing and improving the Work, but
-      excluding communication that is conspicuously marked or otherwise
-      designated in writing by the copyright owner as "Not a Contribution."
+                       TERMS AND CONDITIONS

-      "Contributor" shall mean Licensor and any individual or Legal Entity
-      on behalf of whom a Contribution has been received by Licensor and
-      subsequently incorporated within the Work.
+  0. Definitions.

-   2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
+  "This License" refers to version 3 of the GNU Affero General Public License.

-   3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
-      by such Contributor that are necessarily infringed by their
-      Contribution(s) alone or by combination of their Contribution(s)
-      with the Work to which such Contribution(s) was submitted. If You
-      institute patent litigation against any entity (including a
-      cross-claim or counterclaim in a lawsuit) alleging that the Work
-      or a Contribution incorporated within the Work constitutes direct
-      or contributory patent infringement, then any patent licenses
-      granted to You under this License for that Work shall terminate
-      as of the date such litigation is filed.
+  "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.

-   4. Redistribution. You may reproduce and distribute copies of the
-      Work or Derivative Works thereof in any medium, with or without
-      modifications, and in Source or Object form, provided that You
-      meet the following conditions:
+  "The Program" refers to any copyrightable work licensed under this
+License.  Each licensee is addressed as "you".  "Licensees" and
+"recipients" may be individuals or organizations.

-      (a) You must give any other recipients of the Work or
-          Derivative Works a copy of this License; and
+  To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy.  The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.

-      (b) You must cause any modified files to carry prominent notices
-          stating that You changed the files; and
+  A "covered work" means either the unmodified Program or a work based
+on the Program.

-      (c) You must retain, in the Source form of any Derivative Works
-          that You distribute, all copyright, patent, trademark, and
-          attribution notices from the Source form of the Work,
-          excluding those notices that do not pertain to any part of
-          the Derivative Works; and
+  To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy.  Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.

-      (d) If the Work includes a "NOTICE" text file as part of its
-          distribution, then any Derivative Works that You distribute must
-          include a readable copy of the attribution notices contained
-          within such NOTICE file, excluding those notices that do not
-          pertain to any part of the Derivative Works, in at least one
-          of the following places: within a NOTICE text file distributed
-          as part of the Derivative Works; within the Source form or
-          documentation, if provided along with the Derivative Works; or,
-          within a display generated by the Derivative Works, if and
-          wherever such third-party notices normally appear. The contents
-          of the NOTICE file are for informational purposes only and
-          do not modify the License. You may add Your own attribution
-          notices within Derivative Works that You distribute, alongside
-          or as an addendum to the NOTICE text from the Work, provided
-          that such additional attribution notices cannot be construed
-          as modifying the License.
+  To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies.  Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.

-      You may add Your own copyright statement to Your modifications and
-      may provide additional or different license terms and conditions
-      for use, reproduction, or distribution of Your modifications, or
-      for any such Derivative Works as a whole, provided Your use,
-      reproduction, and distribution of the Work otherwise complies with
-      the conditions stated in this License.
+  An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License.  If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.

-   5. Submission of Contributions. Unless You explicitly state otherwise,
-      any Contribution intentionally submitted for inclusion in the Work
-      by You to the Licensor shall be under the terms and conditions of
-      this License, without any additional terms or conditions.
-      Notwithstanding the above, nothing herein shall supersede or modify
-      the terms of any separate license agreement you may have executed
-      with Licensor regarding such Contributions.
+  1. Source Code.

-   6. Trademarks. This License does not grant permission to use the trade
-      names, trademarks, service marks, or product names of the Licensor,
-      except as required for reasonable and customary use in describing the
-      origin of the Work and reproducing the content of the NOTICE file.
+  The "source code" for a work means the preferred form of the work
+for making modifications to it.  "Object code" means any non-source
+form of a work.

-   7. Disclaimer of Warranty. Unless required by applicable law or
-      agreed to in writing, Licensor provides the Work (and each
-      Contributor provides its Contributions) on an "AS IS" BASIS,
-      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-      implied, including, without limitation, any warranties or conditions
-      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
-      PARTICULAR PURPOSE. You are solely responsible for determining the
-      appropriateness of using or redistributing the Work and assume any
-      risks associated with Your exercise of permissions under this License.
+  A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.

-   8. Limitation of Liability. In no event and under no legal theory,
-      whether in tort (including negligence), contract, or otherwise,
-      unless required by applicable law (such as deliberate and grossly
-      negligent acts) or agreed to in writing, shall any Contributor be
-      liable to You for damages, including any direct, indirect, special,
-      incidental, or consequential damages of any character arising as a
-      result of this License or out of the use or inability to use the
-      Work (including but not limited to damages for loss of goodwill,
-      work stoppage, computer failure or malfunction, or any and all
-      other commercial damages or losses), even if such Contributor
-      has been advised of the possibility of such damages.
+  The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form.  A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.

-   9. Accepting Warranty or Additional Liability. While redistributing
-      the Work or Derivative Works thereof, You may choose to offer,
-      and charge a fee for, acceptance of support, warranty, indemnity,
-      or other liability obligations and/or rights consistent with this
-      License. However, in accepting such obligations, You may act only
-      on Your own behalf and on Your sole responsibility, not on behalf
-      of any other Contributor, and only if You agree to indemnify,
-      defend, and hold each Contributor harmless for any liability
-      incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
+  The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities.  However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work.  For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.

-   END OF TERMS AND CONDITIONS
+  The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.

-   APPENDIX: How to apply the Apache License to your work.
+  The Corresponding Source for a work in source code form is that
+same work.

-      To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "[]"
-      replaced with your own identifying information. (Don't include
-      the brackets!)  The text should be enclosed in the appropriate
-      comment syntax for the file format. We also recommend that a
-      file or class name and description of purpose be included on the
-      same "printed page" as the copyright notice for easier
-      identification within third-party archives.
+  2. Basic Permissions.

-   Copyright [yyyy] [name of copyright owner]
+  All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met.  This License explicitly affirms your unlimited
+permission to run the unmodified Program.  The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work.  This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.

-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at
+  You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force.  You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright.  Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.

-       http://www.apache.org/licenses/LICENSE-2.0
+  Conveying under any other circumstances is permitted solely under
+the conditions stated below.  Sublicensing is not allowed; section 10
+makes it unnecessary.

-   Unless required by applicable law or agreed to in writing, software
-   distributed under the License is distributed on an "AS IS" BASIS,
-   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-   See the License for the specific language governing permissions and
-   limitations under the License.
+  3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+  No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+  When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+  4. Conveying Verbatim Copies.
+
+  You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+  You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+  5. Conveying Modified Source Versions.
+
+  You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+    a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+
+    b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under section
+    7.  This requirement modifies the requirement in section 4 to
+    "keep intact all notices".
+
+    c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy.  This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged.  This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+
+    d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+  A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit.  Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+  6. Conveying Non-Source Forms.
+
+  You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+    a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+
+    b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the
+    Corresponding Source from a network server at no charge.
+
+    c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source.  This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+
+    d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge.  You need not require recipients to copy the
+    Corresponding Source along with the object code.  If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source.  Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+
+    e) Convey the object code using peer-to-peer transmission, provided
+    you inform other peers where the object code and Corresponding
+    Source of the work are being offered to the general public at no
+    charge under subsection 6d.
+
+  A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+  A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling.  In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage.  For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product.  A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+  "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source.  The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+  If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information.  But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+  The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed.  Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+  Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+  7. Additional Terms.
+
+  "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law.  If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+  When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it.  (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.)  You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+  Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+    a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+
+    b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+
+    c) Prohibiting misrepresentation of the origin of that material, or
+    requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+
+    d) Limiting the use for publicity purposes of names of licensors or
+    authors of the material; or
+
+    e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+
+    f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions of
+    it) with contractual assumptions of liability to the recipient, for
+    any liability that these contractual assumptions directly impose on
+    those licensors and authors.
+
+  All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10.  If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term.  If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+  If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+  Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+  8. Termination.
+
+  You may not propagate or modify a covered work except as expressly
+provided under this License.  Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+  However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+  Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+  Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License.  If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+  9. Acceptance Not Required for Having Copies.
+
+  You are not required to accept this License in order to receive or
+run a copy of the Program.  Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance.  However,
+nothing other than this License grants you permission to propagate or
+modify any covered work.  These actions infringe copyright if you do
+not accept this License.  Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+  10. Automatic Licensing of Downstream Recipients.
+
+  Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License.  You are not responsible
+for enforcing compliance by third parties with this License.
+
+  An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations.  If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+  You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License.  For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+  11. Patents.
+
+  A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based.  The
+work thus licensed is called the contributor's "contributor version".
+
+  A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version.  For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+  Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+  In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement).  To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+  If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients.  "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+  If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+  A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License.  You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+  Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+  12. No Surrender of Others' Freedom.
+
+  If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all.  For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+  13. Remote Network Interaction; Use with the GNU General Public License.
+
+  Notwithstanding any other provision of this License, if you modify the
+Program, your modified version must prominently offer all users
+interacting with it remotely through a computer network (if your version
+supports such interaction) an opportunity to receive the Corresponding
+Source of your version by providing access to the Corresponding Source
+from a network server at no charge, through some standard or customary
+means of facilitating copying of software.  This Corresponding Source
+shall include the Corresponding Source for any work covered by version 3
+of the GNU General Public License that is incorporated pursuant to the
+following paragraph.
+
+  Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU General Public License into a single
+combined work, and to convey the resulting work.  The terms of this
+License will continue to apply to the part which is the covered work,
+but the work with which it is combined will remain governed by version
+3 of the GNU General Public License.
+
+  14. Revised Versions of this License.
+
+  The Free Software Foundation may publish revised and/or new versions of
+the GNU Affero General Public License from time to time.  Such new versions
+will be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+  Each version is given a distinguishing version number.  If the
+Program specifies that a certain numbered version of the GNU Affero General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation.  If the Program does not specify a version number of the
+GNU Affero General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+  If the Program specifies that a proxy can decide which future
+versions of the GNU Affero General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+  Later license versions may give you additional or different
+permissions.  However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+  15. Disclaimer of Warranty.
+
+  THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW.  EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE.  THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU.  SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+  16. Limitation of Liability.
+
+  IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+  17. Interpretation of Sections 15 and 16.
+
+  If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU Affero General Public License as published
+    by the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU Affero General Public License for more details.
+
+    You should have received a copy of the GNU Affero General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+  If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source.  For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code.  There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for the
+specific requirements.
+
+  You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU AGPL, see
+<https://www.gnu.org/licenses/>.
@@ -1,21 +1,17 @@
-GO_PROJECT_NAME := hive
+GO_PROJECT_NAME := spx
+SHELL := /bin/bash

 # Detect architecture for cross-platform support
 ARCH := $(shell uname -m)
 ifeq ($(ARCH),x86_64)
  GO_ARCH := amd64
  AWS_ARCH := x86_64
-  # On x86, we can run ARM VMs via emulation
-  QEMU_PACKAGES := qemu-system-x86 qemu-system-arm
 else ifeq ($(ARCH),aarch64)
  GO_ARCH := arm64
  AWS_ARCH := aarch64
-  # On ARM, we can run x86 VMs via emulation
-  QEMU_PACKAGES := qemu-system-aarch64 qemu-system-x86
 else ifeq ($(ARCH),arm64)
  GO_ARCH := arm64
  AWS_ARCH := aarch64
-  QEMU_PACKAGES := qemu-system-aarch64 qemu-system-x86
 else
  $(error Unsupported architecture: $(ARCH). Only x86_64 and aarch64/arm64 are supported.)
 endif
@@ -32,53 +28,196 @@ else
  GO_BUILD_MOD :=
 endif

-build:
-	$(MAKE) go_build
+# Quiet-mode filters (active when QUIET=1, set by preflight via recursive make)
+# Note: grep pipelines use PIPESTATUS[0] so the exit status of `go test`
+# propagates through the filter — otherwise a test failure is swallowed by
+# grep's own (success) exit code and preflight prints "passed" on red.
+ifdef QUIET
+  _Q     = @
+  _COVQ  = 2>&1 | { grep -Ev '^\s*(ok|PASS|\?|=== RUN|--- PASS:)\s' | grep -v 'coverage: 0\.0%' || true; }; exit $${PIPESTATUS[0]}
+  _RACEQ = 2>&1 | { grep -Ev '^\s*(ok|PASS|\?|=== RUN|--- PASS:)\s' || true; }; exit $${PIPESTATUS[0]}
+  _SECQ  = >
+else
+  _Q     =
+  _COVQ  =
+  _RACEQ =
+  _SECQ  = 2>&1 | tee
+endif
+
+build: go_build build-installer build-lb-agent
+
+# Build spinifex-ui frontend (requires pnpm)
+build-ui:
+	@echo -e "\n....Building spinifex-ui frontend...."
+	cd spinifex/services/spinifexui/frontend && pnpm build

 # GO commands
+VERSION ?= $(shell git describe --tags --always --dirty)
+COMMIT  ?= $(shell git rev-parse --short HEAD)
+LDFLAGS := -s -w -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Version=$(VERSION) -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Commit=$(COMMIT)
+
 go_build:
-	@echo "\n....Building $(GO_PROJECT_NAME)"
-	GOEXPERIMENT=greenteagc go build $(GO_BUILD_MOD) -ldflags "-s -w" -o ./bin/$(GO_PROJECT_NAME) cmd/hive/main.go
+	@echo -e "\n....Building $(GO_PROJECT_NAME)"
+	GOFIPS140=v1.0.0 go build $(GO_BUILD_MOD) -ldflags "$(LDFLAGS)" -o ./bin/$(GO_PROJECT_NAME) cmd/spinifex/main.go

-go_run:
-	@echo "\n....Running $(GO_PROJECT_NAME)...."
-	$(GOPATH)/bin/$(GO_PROJECT_NAME)
+build-installer:
+	@echo -e "\n....Building spinifex-installer"
+	GOFIPS140=v1.0.0 go build -ldflags "-s -w" -o ./bin/spinifex-installer cmd/installer/main.go

+build-lb-agent:
+	@echo -e "\n....Building lb-agent (static)"
+	CGO_ENABLED=0 GOFIPS140=v1.0.0 go build -ldflags "-s -w" -o ./bin/lb-agent cmd/lb-agent/main.go
+
+build-system-image: ## Build a system image from manifest (use IMAGE=lb or IMAGE=eks-node)
+ifndef IMAGE
+	$(error IMAGE is required. Usage: make build-system-image IMAGE=lb)
+endif
+	@if [ -f scripts/images/$(IMAGE).conf ]; then \
+		./scripts/build-system-image.sh scripts/images/$(IMAGE).conf $(if $(IMPORT),--import); \
+	elif [ -f scripts/images/$(IMAGE)/manifest.conf ]; then \
+		./scripts/build-system-image.sh scripts/images/$(IMAGE)/manifest.conf $(if $(IMPORT),--import); \
+	else \
+		echo "ERROR: no manifest at scripts/images/$(IMAGE).conf or scripts/images/$(IMAGE)/manifest.conf"; \
+		exit 1; \
+	fi
+
+build-eks-node-image: ## Build the unified eks-node AMI (K3s server+agent; role at first boot; IMPORT=1 to register)
+	$(MAKE) build-system-image IMAGE=eks-node
+
+import-eks-node-image: ## Build + register the eks-node AMI (requires a running cluster)
+	$(MAKE) build-system-image IMAGE=eks-node IMPORT=1
+
+publish-eks-node-image: ## Build + publish the eks-node AMI to Cloudflare R2 (needs R2_ENDPOINT + AWS_* env)
+	./scripts/publish-system-image.sh scripts/images/eks-node/manifest.conf --build
+
+MICROVM_OUT_DIR := build/microvm
+MICROVM_ARTIFACTS := $(MICROVM_OUT_DIR)/vmlinuz $(MICROVM_OUT_DIR)/initramfs.cpio.gz
+MICROVM_INPUTS := scripts/build-microvm-image.sh $(MICROVM_OUT_DIR)/init.sh $(MICROVM_OUT_DIR)/inittab bin/lb-agent
+
+# Grouped target — script writes both files in one run.
+$(MICROVM_ARTIFACTS) &: $(MICROVM_INPUTS)
+	./scripts/build-microvm-image.sh
+
+# Only triggers when bin/lb-agent is missing; preserves the artifact's mtime so
+# build-microvm-image stays correctly stale-aware.
+bin/lb-agent:
+	$(MAKE) build-lb-agent
+
+build-microvm-image: $(MICROVM_ARTIFACTS) ## Build microVM kernel + initramfs (incremental — skips when up to date)
+
+install-microvm: $(MICROVM_ARTIFACTS) ## Install microVM artifacts to /usr/share/spinifex/microvm/
+	sudo install -d /usr/share/spinifex/microvm
+	sudo install -m 0644 $(MICROVM_OUT_DIR)/vmlinuz /usr/share/spinifex/microvm/vmlinuz
+	sudo install -m 0644 $(MICROVM_OUT_DIR)/initramfs.cpio.gz /usr/share/spinifex/microvm/initramfs.cpio.gz
+
+# Preflight — runs the same checks as GitHub Actions (lint + vuln + tests).
+# Use this before committing to catch CI failures locally.
+preflight:
+	@$(MAKE) --no-print-directory QUIET=1 manifest-check manifest-lint lint govulncheck test-cover diff-coverage test-race test-harness
+	@echo -e "\n ✅ Preflight passed — safe to commit."
+
+# E2E harness unit tests. Build-tagged `e2e` so they're skipped by the
+# default `go test ./spinifex/...`. Runs with mocked AWS clients — no
+# infrastructure required, safe to run in CI without a cluster.
+test-harness:
+	@echo -e "\n....Running e2e harness unit tests...."
+	$(_Q)LOG_IGNORE=1 go test -tags=e2e -timeout 60s ./tests/e2e/harness/... $(_RACEQ)
+
+# Validate docs/service-interfaces.yaml. Schema check + cross-reference
+# of services/suites/fixtures + on-disk path existence. Subject content
+# vs source is enforced separately in Bead 5 drift lint.
+manifest-check:
+	@echo -e "\n....Checking service-interfaces.yaml...."
+	@go run ./tests/e2e/manifest-check/cmd/manifest-check -repo-root . -manifest docs/service-interfaces.yaml
+
+# Drift guards (Bead 5): direct-create fixture lint + NATS subject lint,
+# ratcheted against tests/e2e/manifest-lint/baseline.txt. Fails only on NEW
+# drift beyond the baseline.
+manifest-lint:
+	@echo -e "\n....Linting manifest drift (fixtures + subjects)...."
+	@go run ./tests/e2e/manifest-lint/cmd/manifest-lint -repo-root .
+
+# Accept current drift into the baseline. Run after an intentional change.
+manifest-lint-update:
+	@go run ./tests/e2e/manifest-lint/cmd/manifest-lint -repo-root . -update
+
+# Run unit tests
 test:
-	@echo "\n....Running tests for $(GO_PROJECT_NAME)...."
-	LOG_IGNORE=1 go test -v -timeout 120s ./...
+	@echo -e "\n....Running tests for $(GO_PROJECT_NAME)...."
+	LOG_IGNORE=1 go test -timeout 120s ./spinifex/...
+
+# Run unit tests with coverage profile
+COVERPROFILE ?= coverage.out
+test-cover:
+	@echo -e "\n....Running tests with coverage for $(GO_PROJECT_NAME)...."
+	$(_Q)LOG_IGNORE=1 go test -timeout 120s -coverprofile=$(COVERPROFILE) -covermode=atomic ./spinifex/... $(_COVQ)
+	@scripts/check-coverage.sh $(COVERPROFILE) $(QUIET)
+
+# Run unit tests with race detector
+test-race:
+	@echo -e "\n....Running tests with race detector for $(GO_PROJECT_NAME)...."
+	$(_Q)LOG_IGNORE=1 go test -race -timeout 300s ./spinifex/... $(_RACEQ)
+
+# Unit tests for in-repo GitHub Actions (e.g. .github/actions/e2e-analyze).
+# Kept out of `test-cover` so coverage % isn't diluted by CI-only tooling.
+test-actions:
+	@echo -e "\n....Running action tests...."
+	LOG_IGNORE=1 go test -timeout 60s ./.github/actions/...
+
+# Check that new/changed code meets coverage threshold (runs tests first)
+diff-coverage: test-cover
+	@QUIET=$(QUIET) scripts/diff-coverage.sh $(COVERPROFILE)

 bench:
-	@echo "\n....Running benchmarks for $(GO_PROJECT_NAME)...."
-	$(MAKE) easyjson
+	@echo -e "\n....Running benchmarks for $(GO_PROJECT_NAME)...."
 	LOG_IGNORE=1 go test -benchmem -run=. -bench=. ./...

-run:
-	$(MAKE) go_build
-	$(MAKE) go_run
+# Fast iteration: build + install binary + restart all services.
+# Microvm artifacts are reinstalled when they already exist on disk — the rule's
+# input timestamps drive a rebuild only if anything actually changed. On a fresh
+# checkout (no build/microvm/vmlinuz yet) the install step is skipped; run
+# `make install-microvm` explicitly the first time.
+deploy: build
+	sudo install -m 755 bin/spx /usr/local/bin/spx
+	@if [ -f $(MICROVM_OUT_DIR)/vmlinuz ]; then \
+		$(MAKE) install-microvm; \
+	else \
+		echo "[deploy] microvm artifacts absent — run 'make install-microvm' for first-time setup"; \
+	fi
+	sudo systemctl daemon-reload
+	sudo systemctl restart spinifex.target
+
+# Re-run setup.sh after changing systemd units, helper scripts, or logrotate config.
+# Not needed for code-only changes — use deploy for those.
+reinstall:
+	scripts/dev-install.sh

 clean:
-	rm ./bin/$(GO_PROJECT_NAME)
+	rm -f ./bin/$(GO_PROJECT_NAME)
+	rm -rf spinifex/services/spinifexui/frontend/dist

 install-system:
-	@echo "\n....Installing system dependencies for $(ARCH)...."
-	@echo "QEMU packages: $(QEMU_PACKAGES)"
-	apt-get update && sudo apt-get install -y \
-		nbdkit nbdkit-plugin-dev pkg-config $(QEMU_PACKAGES) qemu-utils qemu-kvm \
+	@echo -e "\n....Installing system dependencies for $(ARCH)...."
+	sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
+		-o Dpkg::Options::="--force-confdef" \
+		-o Dpkg::Options::="--force-confold" \
+		nbdkit nbdkit-plugin-dev pkg-config qemu-system-x86 qemu-system-arm qemu-utils qemu-kvm \
+		ovmf qemu-efi-aarch64 \
 		libvirt-daemon-system libvirt-clients libvirt-dev make gcc jq curl \
-		iproute2 netcat-openbsd openssh-client wget git unzip sudo xz-utils file
+		iproute2 netcat-openbsd openssh-client wget git unzip sudo xz-utils file \
+		ovn-central ovn-host openvswitch-switch dhcpcd-base

 install-go:
-	@echo "\n....Installing Go 1.25.5 for $(ARCH) ($(GO_ARCH))...."
+	@echo -e "\n....Installing Go 1.26.4 for $(ARCH) ($(GO_ARCH))...."
 	@if [ ! -d "/usr/local/go" ]; then \
-		curl -L https://go.dev/dl/go1.25.5.linux-$(GO_ARCH).tar.gz | tar -C /usr/local -xz; \
+		curl -L https://go.dev/dl/go1.26.4.linux-$(GO_ARCH).tar.gz | tar -C /usr/local -xz; \
 	else \
 		echo "Go already installed in /usr/local/go"; \
 	fi
 	@echo "Go version: $$(go version)"

 install-aws:
-	@echo "\n....Installing AWS CLI v2 for $(ARCH) ($(AWS_ARCH))...."
+	@echo -e "\n....Installing AWS CLI v2 for $(ARCH) ($(AWS_ARCH))...."
 	@if ! command -v aws >/dev/null 2>&1; then \
 		curl "https://awscli.amazonaws.com/awscli-exe-linux-$(AWS_ARCH).zip" -o "awscliv2.zip"; \
 		unzip -o awscliv2.zip; \
@@ -89,23 +228,67 @@ install-aws:
 	fi

 quickinstall: install-system install-go install-aws
-	@echo "\n✅ Quickinstall complete for $(ARCH)."
+	@echo -e "\n✅ Quickinstall complete for $(ARCH)."
 	@echo "   Please ensure /usr/local/go/bin is in your PATH."
-	@echo "   Installed: Go ($(GO_ARCH)), AWS CLI ($(AWS_ARCH)), QEMU ($(QEMU_PACKAGES))"

-security:
-	@echo "\n....Running security checks for $(GO_PROJECT_NAME)...."
+lint:
+	@echo "Running golangci-lint..."
+	$(_Q)golangci-lint run ./...
+	@echo "  golangci-lint ok"

-	go tool govulncheck ./... > tests/govulncheck-report.txt || true
-	@echo "Govulncheck report saved to tests/govulncheck-report.txt"
+fix:
+	golangci-lint run --fix ./...

-	go tool gosec ./... > tests/gosec-report.txt || true
-	@echo "Gosec report saved to tests/gosec-report.txt"
+govulncheck:
+	@echo "Running govulncheck..."
+	$(_Q)go tool govulncheck ./...
+	@echo "  govulncheck ok"

-	go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./...  > tests/staticcheck-report.txt || true
-	@echo "Staticcheck report saved to tests/staticcheck-report.txt"
+# Build release tarballs — use distro-ARCH for single arch, distro for both
+distro: distro-amd64 distro-arm64
+	@echo ""
+	@echo "Distribution tarballs:"
+	@ls -lh dist/*.tar.gz
+	@echo ""
+	@cat dist/*.sha256

-	go vet ./... 2>&1 | tee tests/govet-report.txt || true
-	@echo "Go vet report saved to tests/govet-report.txt"
+distro-amd64:
+	@echo "Building spinifex $(VERSION) linux/amd64..."
+	@mkdir -p dist/
+	docker buildx build \
+		--platform linux/amd64 \
+		--build-arg VERSION=$(VERSION) \
+		-f build/Dockerfile.distro \
+		--output type=local,dest=dist/amd64/ \
+		../
+	@if [ -f $(MICROVM_OUT_DIR)/vmlinuz ] && [ -f $(MICROVM_OUT_DIR)/initramfs.cpio.gz ]; then \
+		echo "[distro-amd64] staging microvm artifacts into tarball"; \
+		mkdir -p dist/amd64/microvm; \
+		cp $(MICROVM_OUT_DIR)/vmlinuz $(MICROVM_OUT_DIR)/initramfs.cpio.gz dist/amd64/microvm/; \
+	else \
+		echo "[distro-amd64] WARNING: microvm artifacts missing — tarball will not include them"; \
+		echo "[distro-amd64]          run 'make build-microvm-image' before 'make distro-amd64'"; \
+	fi
+	tar -czf dist/spinifex-$(VERSION)-linux-amd64.tar.gz -C dist/amd64 .
+	sha256sum dist/spinifex-$(VERSION)-linux-amd64.tar.gz > dist/spinifex-$(VERSION)-linux-amd64.tar.gz.sha256

-.PHONY: build go_build go_run test bench run clean install-system install-go install-aws quickinstall security
+distro-arm64:
+	@echo "Building spinifex $(VERSION) linux/arm64..."
+	@mkdir -p dist/
+	docker buildx build \
+		--platform linux/arm64 \
+		--build-arg VERSION=$(VERSION) \
+		-f build/Dockerfile.distro \
+		--output type=local,dest=dist/arm64/ \
+		../
+	tar -czf dist/spinifex-$(VERSION)-linux-arm64.tar.gz -C dist/arm64 .
+	sha256sum dist/spinifex-$(VERSION)-linux-arm64.tar.gz > dist/spinifex-$(VERSION)-linux-arm64.tar.gz.sha256
+
+distro-clean:
+	rm -rf dist/
+
+.PHONY: build build-ui build-installer build-lb-agent build-system-image build-eks-node-image import-eks-node-image publish-eks-node-image build-microvm-image install-microvm go_build preflight test test-cover test-race diff-coverage bench test-actions test-harness manifest-check manifest-lint manifest-lint-update \
+	deploy reinstall clean \
+	install-system install-go install-aws quickinstall \
+	lint fix govulncheck \
+	distro distro-amd64 distro-arm64 distro-clean
@@ -1,25 +1,54 @@
-# Hive: An Open Source AWS-Compatible Stack for Bare-Metal, Edge, and On-Prem Deployments
+<p align="center">
+  <img src=".github/assets/banner.svg" alt="Spinifex — AWS-Compatible Infrastructure Stack" width="900">
+</p>

-(Developer Preview) - Hive developed by [Mulga Defense Corporation](https://mulgadc.com/) is an open source infrastructure platform that brings the core services of AWS—like EC2, VPC, EBS, and S3—to environments where running in the cloud isn't an option. Whether you're deploying to edge sites, private data-centers, or need to operate in low-connectivity or highly contested environments, Hive gives you AWS-style workflows on your own hardware.
+<p align="center">
+  <a href="https://go.dev"><img src="https://img.shields.io/badge/Go-1.26+-00ADD8?style=flat-square&logo=go&logoColor=white" alt="Go"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-AGPL--3.0-3fb950?style=flat-square" alt="License"></a>
+  <a href="https://docs.mulgadc.com"><img src="https://img.shields.io/badge/Docs-docs.mulgadc.com-DC2626?style=flat-square" alt="Docs"></a>
+</p>

-## What is Hive?
+<p align="center">
+  <a href="#what-is-spinifex">What is Spinifex?</a> ·
+  <a href="#the-platform">Platform</a> ·
+  <a href="#core-components">Components</a> ·
+  <a href="#architecture-at-a-glance">Architecture</a> ·
+  <a href="#installation">Installation</a> ·
+  <a href="https://docs.mulgadc.com">Docs</a>
+</p>

-Hive replicates essential AWS primitives—virtual machines, block volumes, and object storage—using lightweight, self-contained components that are easy to deploy and integrate.
+---

-It’s designed for developers and operators who need:
+# Spinifex: An Open Source AWS-Compatible Stack for Bare-Metal, Edge, and On-Prem Deployments
+
+Spinifex developed by [Mulga Defense Corporation](https://mulgadc.com) is an open source infrastructure platform that brings the core services of AWS—like EC2, VPC, EBS, and S3—to environments where running in the cloud isn't an option. Whether you're deploying to edge sites, private data-centers, or need to operate in low-connectivity or highly contested environments, Spinifex gives you AWS-style workflows on your own hardware.
+
+## What is Spinifex?
+
+Spinifex replicates essential AWS primitives—virtual machines, block volumes, and object storage—using lightweight, self-contained components that are easy to deploy and integrate.
+
+It's designed for developers and operators who need:

 - Cloud-like infrastructure without the cloud
 - Minimal dependencies, full control
 - Drop-in compatibility with tools like the AWS CLI, SDKs, and Terraform
 - A secure cloud environment you control and own the entire hardware, network and software stack

-You can run Hive on a few servers in a rack, a field site, or anywhere centralized cloud services aren’t feasible.
+You can run Spinifex on a few servers in a rack, a field site, or anywhere centralized cloud services aren't feasible.
+
+## The Platform
+
+<p align="center">
+  <img src=".github/assets/platform.svg" alt="The Mulga Platform — Hardware to Applications" width="900">
+</p>
+
+From commodity hardware up to unmodified AWS tooling, every layer is replaceable and yours to own.

 ## Core Components

-### Hive (Compute Service – EC2 Alternative)
+### Spinifex (Compute Service – EC2 Alternative)

-Hive is a minimal VM orchestration layer built on top of QEMU, exposing APIs similar to EC2. It manages lifecycle operations like start, stop, and terminate, using QEMU’s QMP interface. Designed to be straightforward and scriptable, Hive lets you launch VMs using the AWS CLI, SDKs, or Terraform—without needing Kubernetes or heavyweight orchestrators. Keep in mind, you can also setup a Kubernetes environment using Hive with underlying instances.
+Spinifex is a minimal VM orchestration layer built on top of QEMU, exposing APIs similar to EC2. It manages lifecycle operations like start, stop, and terminate, using QEMU's QMP interface. Designed to be straightforward and scriptable, Spinifex lets you launch VMs using the AWS CLI, SDKs, or Terraform—without needing Kubernetes or heavyweight orchestrators. Keep in mind, you can also setup a Kubernetes environment using Spinifex with underlying instances.

 - EC2-like VM management on bare metal
 - Launches with cloud-init metadata support
@@ -27,7 +56,7 @@ Hive is a minimal VM orchestration layer built on top of QEMU, exposing APIs sim

 ### Viperblock (Block Storage – EBS Alternative)

-[Viperblock](https://github.com/mulgadc/viperblock) is a high-performance, WAL-backed block storage service that replicates volumes across multiple nodes. It’s built for reliability and speed, with support for snapshots, recovery, and direct connection to QEMU instances using NBD or virtio-blk.
+[Viperblock](https://github.com/mulgadc/viperblock) is a high-performance, WAL-backed block storage service that replicates volumes across multiple nodes. It's built for reliability and speed, with support for snapshots, recovery, and direct connection to QEMU instances using NBD or virtio-blk.

 - Fast, durable virtual disks
 - Replication for resilience
@@ -43,53 +72,98 @@ Hive is a minimal VM orchestration layer built on top of QEMU, exposing APIs sim
 - Multipart uploads, streaming reads/writes
 - Data redundancy with Reed-Solomon encoding

+## Architecture at a Glance
+
+<p align="center">
+  <img src=".github/assets/architecture.svg" alt="Spinifex message-driven architecture — AWS Gateway, NATS bus, service daemons" width="900">
+</p>
+
+Every AWS API call is authenticated at the gateway, published to a NATS subject, and answered by whichever daemon claims it. Daemons are stateless — scale horizontally by starting more. No etcd, no Kubernetes, no external control plane. Deep dive: **[`docs/DESIGN.md`](docs/DESIGN.md)**.
+
 ## Key Features

 AWS-Compatible Interfaces – Provision infrastructure with awscli, Terraform, or SDKs you already use.

- Designed for Control – Run on your own terms, whether that’s in a datacenter, remote site, or sensitive environment.
+- Designed for Control – Run on your own terms, whether that's in a datacenter, remote site, or sensitive environment.
 - Minimal Dependencies – Each service is standalone and avoids complex orchestration layers.
 - Works Offline – No reliance on centralized cloud services or external networks.
- Open Source – Licensed under Apache 2.0. Fork it, extend it, or deploy it as-is.
+- Open Source – Licensed under AGPL 3.0. Fork it, extend it, or deploy it as-is.

-## Development Setup
+## Installation

-### For Developers
+Installation requires an Ubuntu / Debian system. See the detailed documentation at [docs.mulgadc.com](https://docs.mulgadc.com) for maintaining and installing Spinifex.

-Get started with Hive development in minutes:
+### Bare Metal ISO
+
+The recommended installation is a [bootable x86 installer](https://iso.mulgadc.com/spinifex.iso) for bare-metal hardware.

 ```bash
-# Clone the repository
-git clone https://github.com/mulgadc/hive.git
-cd hive
-
-# Setup dependencies and development environment
-./scripts/clone-deps.sh    # Clone viperblock + predastore repositories
-./scripts/dev-setup.sh     # Setup complete development environment
+curl -fLO https://iso.mulgadc.com/spinifex.iso
 ```

-Next, follow the detailed [Installation Guide](./INSTALL.md) to bootstrap a new environment with SSH keys, AMI (OS images) and how to spin up new EC2 instances using Hive.
+Follow the [USB install guide](https://docs.mulgadc.com/docs/install-usb) to write the ISO to USB and install on your hardware. The install guide walks through the full process.
+
+
+### Single Node Install
+
+The installation is straightforward to set up and running on a single node for testing purposes. Debian 13 is currently supported, additional Linux distributions are on the immediate roadmap.
+
+>*Prerequisite:* Linux bridge for networking.
+
+Spinifex requires a Linux bridge configured on the host for VM networking. See the [single-node install guide](https://docs.mulgadc.com/docs/install#prerequisites) prerequisites for setup details.
+
+```bash
+curl -fsSL https://install.mulgadc.com | bash
+
+sudo /usr/local/share/spinifex/setup-ovn.sh --management
+
+sudo spx admin init --node node1 --nodes 1
+
+sudo systemctl start spinifex.target
+
+export AWS_PROFILE=spinifex
+
+aws ec2 describe-instance-types
+```
+
+### Development Setup
+
+For a complete development environment see the [Source Install](https://docs.mulgadc.com/docs/install-source) documentation

 ### Component Repositories

-Hive coordinates these independent components:
+Spinifex coordinates these independent components:

 - **[Predastore](https://github.com/mulgadc/predastore)** - S3-compatible object storage
 - **[Viperblock](https://github.com/mulgadc/viperblock)** - EBS-compatible block storage

 Each component can be developed independently. See component-specific documentation for focused development guides.

-## Detailed Installation method
+## Spinifex UI

-See [Installation Guide](INSTALL.md) on how to setup a Hive node to get started.
+Spinifex ships with a built-in web console — an optional alternative to the AWS CLI, SDKs, and Terraform. If you're familiar with the AWS Management Console, the Spinifex UI fills the same role: a browser-based view of your instances, volumes, buckets, VPCs, and IAM resources, without leaving your own network.

-Then configure your AWS CLI to point to Hive's endpoints and start launching VMs, buckets, and volumes using familiar commands and Terraform scripts.
+<p align="center">
+  <img src=".github/assets/spinifex-ui.jpg" alt="Spinifex web console — dashboard view" width="900">
+</p>
+
+The console is served by each node on port `3000` over TLS, and becomes available as soon as `spinifex.target` is up:
+
+```bash
+open https://YOUR_NODE_IP:3000
+```
+
+- **Same API, different surface.** Every action in the UI is the same AWS SigV4 call the CLI makes — so RBAC, audit trails, and IAM policies apply uniformly.
+- **Single sign-on against your AWS credentials.** Log in with the access keys from `~/.aws/credentials` on the node where Spinifex is installed — no separate user database.
+- **Self-hosted, works offline.** The UI is embedded in the Spinifex binary and served from the node itself. No external CDN, no analytics calls, no cloud dependency.
+
+For the full walkthrough — first-time TLS certificate trust, login, and feature tour — see [**Launching the Web UI**](https://docs.mulgadc.com/docs/setting-up-your-cluster#7-launching-the-web-ui) in the cluster setup guide.

 ## Development Philosophy

 ### Built by Engineers, For Engineers

-Hive is developed by experienced infrastructure engineers with deep AWS expertise, including former AWS team members who understand the intricacies of building production-grade cloud services. Our team brings decades of combined experience from AWS, enterprise infrastructure, and edge computing environments.
+Spinifex is developed by experienced infrastructure engineers with deep AWS expertise, including former AWS team members who understand the intricacies of building production-grade cloud services. Our team brings decades of combined experience from AWS, enterprise infrastructure, and edge computing environments.

 **Real-World Experience:**

@@ -100,7 +174,7 @@ Hive is developed by experienced infrastructure engineers with deep AWS expertis

 ### AI-Assisted Development

-While Hive is architected and implemented by experienced engineers, we leverage **Claude Code** (Anthropic's AI coding assistant) to accelerate certain development tasks. This approach combines human expertise with AI efficiency:
+While Spinifex is architected and implemented by experienced engineers, we leverage **Claude Code** (Anthropic's AI coding assistant) to accelerate certain development tasks. This approach combines human expertise with AI efficiency:

 **How We Use Claude Code:**

@@ -116,14 +190,8 @@ While Hive is architected and implemented by experienced engineers, we leverage
 - **Performance Optimization**: Real-world performance tuning and benchmarking
 - **Production Operations**: Deployment strategies and operational procedures

-**Development Artifacts:**
-
- `CLAUDE.md` - Instructions for Claude Code when working on this codebase
- `HIVE_DEVELOPMENT_PLAN.md` - Comprehensive roadmap and implementation strategy
- Component-specific guidance in `viperblock/CLAUDE.md` and `predastore/CLAUDE.md`
-
-This hybrid approach ensures Hive benefits from both proven engineering expertise and modern development acceleration, while maintaining the quality and reliability standards required for production infrastructure.
+This hybrid approach ensures Spinifex benefits from both proven engineering expertise and modern development acceleration, while maintaining the quality and reliability standards required for production infrastructure.

 ## License

-Hive is open source under the Apache 2.0 License. You're free to use, modify, and deploy it—anywhere you need reliable infrastructure without depending on centralized cloud platforms.
+Spinifex is open source under the [GNU Affero General Public License v3.0](LICENSE). You're free to use, modify, and deploy it anywhere you need reliable infrastructure without depending on centralized cloud platforms.
@@ -1,444 +0,0 @@
-# Hive Development Roadmap
-
-## Update Jan 2026
-
-Big rocks:
-
- [DONE] Add multi-node support for predastore (S3) using reed solomon encoding
-  - [DONE] Implement KV store for object lookup, WAL files, offset. Use hash-ring to determine which nodes an object belongs to.
- Implement VPC support using Open vSwitch across multiple nodes, core VPC functionality included
-  - Add support for NVIDIA Bluefield DPU with Open vSwitch
- Implement basic IAM using NATS Jetstream as KV store, vs IAM/access-keys in local config/TOML files for beta.
-  - Move `daemon.go` instances.json state to Jetstream KV
- Add support using the `hive` CLI tool to provision a new user with AWS access-keys/IAM.
-  - Support multi-tenant operations and isolation
- Add support to include capabilities when adding a new hardware node to MulgaOS (e.g EC2 target, S3, EBS, NATs, etc) - Features can be turned on/off depending on hardware scope.
- [DONE] Add simple Web UI console, using the AWS JS SDK, communicating to local AWS gateway.
-  - Implement ShadCNblocks for UI framework
-  - Simple Go webserver, static files, easy build process.
-
-Implementation gaps:
-
- EC2 - Support extended features for `run-instance`
-  - Volume resize of AMI. Note `nbd` does not support live resize events, instance needs to be stopped, resized, and started.
-  - Confirm cloud-init will resize volume on boot and configured correctly.
-  - Attach additional volumes
-  - Attach to VPC / Security group (required Open vSwitch implementation)
-
-### Issues to investigate
-
- When the host disk volume is full and a new instance is launched, disk corruption will occur within the instance since the host volume is out of space. Consider a pre-flight check, check available storage on the S3 server, local viperblock for WAL cache availability and provide improved guard rails to prevent a new instance running, if available disk space is nearing to be exceeded.
-
-Host:
-
-```json
-{"time":"2026-01-27T21:30:29.368164556+10:00","level":"ERROR","msg":"WAL sync failed","error":"write /home/hive/hive/predastore/distributed/nodes/node-2/0000000000000737.wal: no space left on device"}
-```
-
-Instance `dmesg`:
-
-```
-[ 1445.573495] EXT4-fs error (device vda1): ext4_journal_check_start:83: comm systemd-journal: Detected aborted journal
-[ 1445.583565] EXT4-fs (vda1): Remounting filesystem read-only
-```
-
-
-### EC2
-
- [DONE] - describe-instances
- [DONE] - run-instances (count not implemented)
- [DONE] - start-instances
- [DONE] - stop-instances
- [DONE] - terminate-instances
- [DONE] - describe-instance-types
- [DONE] - create-key-pair
- [DONE] - delete-key-pair
- [DONE] - describe-key-pairs
- [DONE] - import-key-pair
- [DONE] - describe-images
- [DONE] - describe-regions
- [DONE] - describe-volumes
-
-### To implement
-
-Easier methods to implement
-
- attach-volume
- copy-image
- copy-snapshot
- copy-volumes
- create-image
- create-snapshot
- create-snapshots
- create-tags
- create-volume
- delete-snapshot
- describe-snapshots
- describe-subnets
- describe-tags
- describe-volume-attribute
- describe-volume-status
- detach-network-interface
- detach-volume
- get-console-output
- monitor-instances
-
-TODO:
-
- allocate-address
- allocate-hosts
- assign-private-ip-addresses
- assign-private-nat-gateway-address
- associate-address
- associate-nat-gateway-address
- associate-route-server
- associate-route-table
- associate-security-group-vpc
- associate-subnet-cidr-block
- attach-internet-gateway
- attach-network-interface
- authorize-security-group-egress
- authorize-security-group-ingress
- create-customer-gateway
- create-default-subnet
- create-default-vpc
- create-dhcp-options
- create-egress-only-internet-gateway
- create-internet-gateway
- create-launch-template
- create-launch-template-version
- create-local-gateway-route
- create-local-gateway-route-table
- create-local-gateway-route-table-virtual-interface-group-association
- create-local-gateway-virtual-interface
- create-local-gateway-virtual-interface-group
- create-nat-gateway
- create-network-acl
- create-network-acl-entry
- create-public-ipv4-pool
- create-route
- create-route-server
- create-route-server-endpoint
- create-route-server-peer
- create-route-table
- create-security-group
- create-subnet
- create-subnet-cidr-reservation
- delete-coip-cidr
- delete-customer-gateway
- delete-dhcp-options
- delete-egress-only-internet-gateway
- delete-internet-gateway
- delete-local-gateway-route
- delete-local-gateway-route-table
- delete-local-gateway-route-table-virtual-interface-group-association
- delete-local-gateway-virtual-interface
- delete-local-gateway-virtual-interface-group
- delete-nat-gateway
- delete-network-acl
- delete-network-interface
- delete-network-interface-permission
- delete-public-ipv4-pool
- delete-route
- delete-route-server
- delete-route-server-endpoint
- delete-route-server-peer
- delete-route-table
- delete-security-group
- delete-subnet
- delete-subnet-cidr-reservation
- delete-tags
- delete-volume
- describe-account-attributes
- describe-addresses
- describe-customer-gateways
- describe-dhcp-options
- describe-egress-only-internet-gateways
- describe-hosts
- describe-internet-gateways
- describe-managed-prefix-lists
- describe-nat-gateways
- describe-network-acls
- describe-route-server-endpoints
- describe-route-server-peers
- describe-route-servers
- describe-route-tables
- describe-security-group-references
- describe-security-group-rules
- describe-security-groups
- describe-volumes-modifications
- detach-internet-gateway
- detach-vpn-gateway
- disable-image
- disable-route-server-propagation
- disable-serial-console-access
- enable-address-transfer
- enable-capacity-manager
- enable-ebs-encryption-by-default
- enable-image
- enable-image-block-public-access
- export-image
- get-console-screenshot
- get-instance-metadata-defaults
- get-instance-tpm-ek-pub
- get-instance-types-from-instance-requirements
- get-instance-uefi-data
- get-launch-template-data
- get-route-server-associations
- get-route-server-propagations
- get-route-server-routing-database
- get-security-groups-for-vpc
- get-serial-console-access-status
- get-snapshot-block-public-access-state
- get-spot-placement-scores
- get-subnet-cidr-reservations
- import-image
- import-snapshot
- modify-instance-attribute
- modify-instance-capacity-reservation-attributes
- modify-instance-connect-endpoint
- modify-instance-cpu-options
- modify-instance-credit-specification
- modify-instance-placement
- modify-network-interface-attribute
- modify-volume
- modify-volume-attribute
- modify-vpn-connection
- modify-vpn-connection-options
- reboot-instances
- register-image
- release-address
- release-hosts
- replace-network-acl-association
- replace-network-acl-entry
- replace-route
- replace-route-table-association
- report-instance-status
- revoke-security-group-egress
- revoke-security-group-ingress
- run-instances
-
-### EBS
-
- complete-snapshot
- get-snapshot-block
- list-changed-blocks
- list-snapshot-blocks
- put-snapshot-block
- start-snapshot
-
-### VPC
-
- create-vpc
- create-vpc-block-public-access-exclusion
- create-vpc-encryption-control
- create-vpc-endpoint
- create-vpc-endpoint-connection-notification
- create-vpc-endpoint-service-configuration
- create-vpc-peering-connection
- delete-vpc
- delete-vpc-block-public-access-exclusion
- delete-vpc-encryption-control
- delete-vpc-endpoint-connection-notifications
- delete-vpc-endpoint-service-configurations
- delete-vpc-endpoints
- delete-vpc-peering-connection
- associate-vpc-cidr-block
- delete-local-gateway-route-table-vpc-association
- create-local-gateway-route-table-vpc-association
- describe-security-group-vpc-associations
- describe-vpc-attribute
- describe-vpc-block-public-access-exclusions
- describe-vpc-block-public-access-options
- describe-vpc-encryption-controls
- describe-vpc-endpoint-associations
- describe-vpc-endpoint-connection-notifications
- describe-vpc-endpoint-connections
- describe-vpc-endpoint-service-configurations
- describe-vpc-endpoint-service-permissions
- describe-vpc-endpoint-services
- describe-vpc-endpoints
- describe-vpc-peering-connections
- describe-vpcs
- modify-vpc-attribute
- modify-vpc-encryption-control
- modify-vpc-endpoint
- modify-vpc-endpoint-connection-notification
- modify-vpc-endpoint-service-configuration
- modify-vpc-tenancy
-
-### S3
-
- Consider moving S3 control/data plane, from predastore, to Hive format.
-
-## Update Nov 2025
-
-[PARTIAL] _ Implement multi-tenant support
-[PARTIAL] _ Move config settings from ~/hive/\*.toml, to using Nats Jetstream for core config which can be synced between nodes.
-
- Implement a lightweight IAM using Nats Jetstream, vs current config files for auth settings.
-  [ONGOING] _ Implement Reed Solomon Encoding for Predastore (S3) objects and a WAL implementation, for storing multiple objects in a single shard (e.g 4MB), with the WAL referencing the location of each object (e.g 4kb min)
-  _ Implement basic KY lookup, object key sha512 (bucket/key), location to shard on S3 (obj.0000124.bin), read WAL (e.g first 4096 bytes) to determine location of the object. e.g key-1234 => obj.0000124.bin, wal header (4096 bytes) > key-1234 == offset location (seek) 16384, len 32768.
-  _ Read multiple nodes (e.g 5 predastore instances, k = 3 (data shards), n = 5 (total shards), n - k = 2 (parity shards) )
-  [ONGOING] _ Complete core scaffolding AWS SDK/API requirements (ec2 describe-instances, run-instances, etc)
- Implement UEFI support for image downloads and `qemu` exec in `vm.go`
- Confirm Alpine Linux, fails import image AMI > (run-instance) ""Failed to read block from AMI source" err="request out of range" - Block size correct?
- Improve shutdown gracefully, `./scripts/stop-dev.sh` waits 60 seconds, while qemu/nbd could still be shutting down.
- Add delete-volume support via EBS (s3 vol-\*) for terminated instance
- Add default LRU cache support for viperblock, depending on the instance type / volume size and system memory available.
-
-## Multi-node setup
-
-[DONE] - Development release - v0.1.0 (single node)
-[PARTIAL] - Production release - v0.2.0 (multi node)
-
-Design layout for multi-node configuration.
-
-```
-# node1
-hive admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 3 --hive-dir ~/hive/
-
-# node2
-hive admin join --region ap-southeast-2 --az ap-southeast-2a --node node2 --host node1.local:8443 --data-dir ~/hive/
-
-# node3, optionally toggle EBS/EC2/NATs support only
-hive admin join --region ap-southeast-2 --az ap-southeast-2a --node node3 --host node1.local:8443 --data-dir ~/hive/ --cap ebs,ec2,nats
-```
-
-If --host is missing, the join command tries multicast broadcast to find parent (leader) node.
-
-Set region with `--region ap-southeast-2` which will create a new Hive cluster for the specified region.
-
-For production, it is recommended to run Hive on at least three physical nodes. In this minimum setup, Hive S3 and EBS volumes use 2 data shards and 1 parity shard RS(2,1) to distribute each chunk across the cluster. This configuration tolerates a full node failure while adding only 1.5x storage overhead.
-
-### Overview:
-
- init creates a cluster-id, node-id, and a short join token, starts a tiny control server on :8443, and writes DNS hint files if available.
-
- init sets target size to 3 and waits until all 3 nodes join and ack.
-
- join nodes contact node1, present the token or cluster-id, and advertise capabilities.
-
- node1 appends them to the member set and immediately pushes the current settings bundle to them.
-
- When member count reaches 3, node1 broadcasts the final cluster settings to all nodes.
-
- Every node writes the same cluster.json and starts the services that match its capabilities.
-
-Node exchange payload
-
-```json
-{
-  "cluster_id": "c-82d5",
-  "node_id": "n-rpi2",
-  "addr": "rpi2.local:8443",
-  "version": "0.1.0",
-  "caps": ["ec2", "s3"],
-  "ts": 1731388800
-}
-```
-
-Settings bundle (identical on all 3 once committed)
-
-```json
-{
-  "cluster_id": "c-82d5",
-  "target_size": 3,
-  "members": [
-    {
-      "node_id": "n-rpi1",
-      "addr": "rpi1.local:8443",
-      "caps": ["ec2", "s3", "nats", "ebs"]
-    },
-    { "node_id": "n-rpi2", "addr": "rpi2.local:8443", "caps": ["ec2", "s3"] },
-    { "node_id": "n-rpi3", "addr": "rpi3.local:8443", "caps": ["nats", "ebs"] }
-  ],
-  "services": {
-    "ec2": { "api_bind": ":9001" },
-    "s3": { "api_bind": ":9002", "replicas": 2 },
-    "nats": { "cluster": "c-82d5", "quorum": 2 },
-    "ebs": { "replicas": 3, "block_size": 4096 }
-  },
-  "epoch": 1,
-  "sig": "ed25519:..." // signed by rpi1 during init
-}
-```
-
-## Original TODO Items → Development Plan Integration
-
-### ✅ **Completed Integration**
-
-All original TODO items have been incorporated into the structured development phases:
-
-| Original TODO                         | Status     | Integrated Into                                | Phase     |
-| ------------------------------------- | ---------- | ---------------------------------------------- | --------- |
-| #1: Binary compile and install.sh     | ✅ Planned | Phase 7: Task 7.1 (Binary Compilation)         | Phase 7   |
-| #2: Move daemon.go to services/hived/ | ✅ Planned | Phase 3: Task 3.2 (Specialized Services)       | Phase 3   |
-| #3: VPC with openvs-switch as `vpcd`  | ✅ Planned | Phase 5: Task 5.1 (VPC with Open vSwitch)      | Phase 5   |
-| #4: AWS HTTP gateway (`awsd`)         | ✅ Planned | Phase 3: Task 3.1 (AWS Gateway Service)        | Phase 3   |
-| #5: AWS SDK v2 input/output           | ✅ Planned | Phase 2: Task 2.1 & Phase 3: Task 3.1          | Phase 2-3 |
-| #6: `hive.service` for systemd        | ✅ Planned | Phase 7: Task 7.2 (System Service Integration) | Phase 7   |
-| #7: Gossip and RAFT protocols         | ✅ Planned | Phase 0: Task 0.1 (Service Registry)           | Phase 0   |
-| #8: etcd/KV for configuration sync    | ✅ Planned | Phase 0: Task 0.1 (Distributed Config)         | Phase 0   |
-| #9: Smithy model code generation      | ✅ Planned | Phase 2: Task 2.1 (Smithy Code Generation)     | Phase 2   |
-
-## Development Phase Overview
-
-**Phase 0**: Distributed Systems Foundation (1-2 weeks)
-
- Gossip, RAFT, and distributed configuration (#7, #8)
-
-**Phase 1**: Development Environment Automation (2-3 weeks)
-
- Multi-service orchestration and hot reloading
-
-**Phase 2**: AWS API Model Implementation (3-4 weeks)
-
- Smithy-based code generation (#9)
- AWS SDK v2 integration (#5)
-
-**Phase 3**: Scalable Gateway and Daemon Architecture (2-3 weeks)
-
- AWS gateway service `awsd` (#4)
- Service refactoring to `hived` (#2)
- VPC daemon `vpcd` foundation (#3)
-
-**Phase 4**: Service Integration and Orchestration (3-4 weeks)
-
- NATS clustering and cross-service coordination
-
-**Phase 5**: Infrastructure Services (4-5 weeks)
-
- VPC networking with Open vSwitch (#3)
- Advanced VM features
-
-**Phase 6**: Testing and Validation Framework (2-3 weeks)
-
- AWS CLI compatibility testing
-
-**Phase 7**: Production Deployment and Packaging (2-3 weeks)
-
- Binary compilation and distribution (#1)
- Systemd service integration (#6)
- Production configuration management
-
-## Service Architecture
-
-The development plan implements these services:
-
- **`awsd`** - AWS API Gateway (TODO #4)
- **`hived`** - Main Hive compute daemon (TODO #2)
- **`vpcd`** - VPC networking daemon (TODO #3)
- **Predastore** - S3 service (existing)
- **Viperblock** - EBS service (existing)
-
-## Getting Started
-
-For development setup:
-
-```bash
-./scripts/dev-setup.sh     # Setup complete development environment
-./scripts/start-dev.sh     # Start all services
-```
-
-For detailed implementation guidance, see [HIVE_DEVELOPMENT_PLAN.md](HIVE_DEVELOPMENT_PLAN.md).
@@ -0,0 +1,49 @@
+ARG VERSION=dev
+
+FROM golang:1.26-bookworm AS builder
+
+ARG VERSION
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    nbdkit nbdkit-plugin-dev pkg-config gcc make \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /build
+
+# Copy module files for dependency caching
+COPY spinifex/go.mod spinifex/go.sum ./spinifex/
+COPY predastore/go.mod predastore/go.sum ./predastore/
+COPY viperblock/go.mod viperblock/go.sum ./viperblock/
+COPY viperblock/nbd/go.mod viperblock/nbd/go.sum ./viperblock/nbd/
+COPY viperblock/nbd/libguestfs.org/nbdkit/go.mod ./viperblock/nbd/libguestfs.org/nbdkit/
+
+# Generate go.work (gitignored, so not in checkout)
+RUN cd spinifex && go work init . ../predastore ../viperblock ../viperblock/nbd ../viperblock/nbd/libguestfs.org/nbdkit
+
+# Download dependencies (cached layer)
+RUN cd spinifex && go mod download
+
+# Copy full source
+COPY spinifex/ ./spinifex/
+COPY predastore/ ./predastore/
+COPY viperblock/ ./viperblock/
+
+# Build spinifex binary (spx)
+RUN COMMIT=$(cd spinifex && git rev-parse --short HEAD 2>/dev/null || echo "unknown") && \
+    cd spinifex && GOFIPS140=v1.0.0 go build -ldflags "-s -w \
+    -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Version=${VERSION} \
+    -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Commit=${COMMIT}" \
+    -o /out/spx cmd/spinifex/main.go
+
+# Build nbdkit viperblock plugin (.so)
+RUN cd viperblock && GOFIPS140=v1.0.0 go build -o /out/nbdkit-viperblock-plugin.so -buildmode=c-shared nbd/viperblock.go
+
+# Copy setup scripts and systemd units
+RUN cp spinifex/scripts/setup.sh /out/setup.sh 2>/dev/null || true
+RUN cp spinifex/scripts/setup-ovn.sh /out/setup-ovn.sh
+RUN mkdir -p /out/systemd && cp spinifex/build/systemd/* /out/systemd/
+RUN mkdir -p /out/scripts && cp spinifex/build/scripts/* /out/scripts/
+RUN cp spinifex/build/logrotate/spinifex /out/logrotate-spinifex
+
+FROM scratch AS export
+COPY --from=builder /out/ /
@@ -0,0 +1,9 @@
+/var/log/spinifex/*.log {
+    daily
+    rotate 14
+    compress
+    delaycompress
+    missingok
+    notifempty
+    copytruncate
+}
@@ -0,0 +1,94 @@
+#!/bin/sh
+mount -t proc  proc  /proc
+mount -t sysfs sys   /sys
+mount -t devtmpfs dev /dev
+
+# virtio_mmio is built-in (CONFIG_VIRTIO_MMIO=y) so virtio-net devices are
+# already enumerated by the time PID 1 runs — no modprobe needed.
+# qemu_fw_cfg and virtio_net are =m in stock Alpine linux-virt, so load them.
+#
+# modprobe needs a usable modules.dep; when the image is built on a host whose
+# depmod can't produce one, busybox modprobe silently no-ops and the module
+# never loads — fw_cfg then never appears and init panics. insmod by explicit
+# path needs no modules.dep, so fall back to it. Handles .ko and .ko.gz.
+load_mod() {
+    modprobe "$1" 2>/dev/null && return 0
+    for ko in $(find /lib/modules -name "$1.ko" -o -name "$1.ko.gz" 2>/dev/null); do
+        insmod "$ko" 2>/dev/null && return 0
+        case "$ko" in
+            *.gz) gunzip -c "$ko" >/tmp/mod.ko 2>/dev/null \
+                  && insmod /tmp/mod.ko 2>/dev/null && return 0 ;;
+        esac
+    done
+    return 1
+}
+load_mod qemu_fw_cfg || load_mod fw_cfg_sysfs || true
+# virtio_net depends on net_failover, which depends on failover. The load_mod
+# insmod fallback loads a single .ko with no dependency resolution, and
+# modules.dep is absent when the image is built on a kmod-less host (depmod
+# skipped), so modprobe can't resolve the chain either. Load it explicitly
+# bottom-up so the NIC comes up regardless of how the image was built —
+# without this the agent has no network and the LB hangs in provisioning.
+load_mod failover || true
+load_mod net_failover || true
+load_mod virtio_net || true
+
+NETCFG=/sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/netcfg/raw
+i=0
+while [ "$i" -lt 25 ]; do
+    [ -f "$NETCFG" ] && break
+    sleep 0.04
+    i=$((i + 1))
+done
+
+if [ ! -f "$NETCFG" ]; then
+    # Dump host/guest fw_cfg state to the serial console before init exits so a
+    # "netcfg not available" panic is diagnosable from the captured console log
+    # (module-not-loaded vs qemu-not-exposing-by_name vs wrong machine type).
+    echo "[init] FATAL: fw_cfg netcfg not available" >&2
+    echo "[init] dmi: vendor=$(cat /sys/class/dmi/id/sys_vendor 2>/dev/null) product=$(cat /sys/class/dmi/id/product_name 2>/dev/null) bios=$(cat /sys/class/dmi/id/bios_version 2>/dev/null)" >&2
+    echo "[init] qemu_fw_cfg dir present: $([ -d /sys/firmware/qemu_fw_cfg ] && echo yes || echo NO)" >&2
+    echo "[init] modules: $(lsmod 2>/dev/null | awk 'NR>1{print $1}' | tr '\n' ',')" >&2
+    echo "[init] by_name listing:" >&2
+    ls -la /sys/firmware/qemu_fw_cfg/by_name/ 2>&1 | sed 's/^/[init]   /' >&2
+    ls -la /sys/firmware/qemu_fw_cfg/by_name/opt/ 2>&1 | sed 's/^/[init]   /' >&2
+    echo "[init] dmesg fw_cfg:" >&2
+    dmesg 2>/dev/null | grep -i fw_cfg | sed 's/^/[init]   /' >&2
+    exit 1
+fi
+
+. "$NETCFG"
+
+for n in 0 1 2 3 4 5; do
+    eval mac=\$NIC${n}_MAC
+    [ -z "$mac" ] && continue
+
+    iface=""
+    for attempt in 1 2 3 4 5; do
+        iface=$(ip -o link | awk -v m="$mac" 'tolower($0) ~ tolower(m) {print $2}' | tr -d :)
+        [ -n "$iface" ] && break
+        sleep 0.02
+    done
+
+    if [ -z "$iface" ]; then
+        echo "[init] WARNING: no interface found for MAC $mac (NIC$n), skipping" >&2
+        continue
+    fi
+
+    eval cidr=\$NIC${n}_CIDR
+    eval gw=\$NIC${n}_GW
+    eval default=\$NIC${n}_DEFAULT
+    eval rdst=\$NIC${n}_ROUTE_DST
+    eval rvia=\$NIC${n}_ROUTE_VIA
+
+    ip link set "$iface" up
+    [ -n "$cidr" ] && ip addr add "$cidr" dev "$iface"
+    [ "$default" = "1" ] && [ -n "$gw" ] && ip route add default via "$gw" dev "$iface"
+    [ -n "$rdst" ] && [ -n "$rvia" ] && ip route add "$rdst" via "$rvia" dev "$iface"
+done
+
+cp /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/lb-agent-env/raw /etc/conf.d/lb-agent
+mkdir -p /etc/ssl/certs
+cp /sys/firmware/qemu_fw_cfg/by_name/opt/spinifex/ca-cert/raw /etc/ssl/certs/ca-certificates.crt
+
+exec /sbin/init
@@ -0,0 +1,4 @@
+::sysinit:/etc/rc.local
+::respawn:/bin/sh -c 'set -a; . /etc/conf.d/lb-agent; set +a; exec /usr/local/bin/lb-agent'
+::ctrlaltdel:/sbin/reboot
+::shutdown:/bin/umount -a -r
@@ -0,0 +1,30 @@
+#!/bin/sh
+# Helper script for spinifex-predastore.service
+# Detects host, port, backend, and node-id from spinifex.toml
+# then execs spx. Falls back to single-node defaults if no config found.
+
+CONF=/etc/spinifex/spinifex.toml
+BIND="0.0.0.0:8443"
+# -1 = co-located mode (all [[db]] peers in one process). Single-node spinifex
+# templates list 3 DB peers on 127.0.0.1; only -1 can serve that topology.
+# Multi-node spinifex.toml emits node_id = N (N >= 1) which overrides below.
+# predastore rejects NODE_ID=0 since d3a02f0 (strconv.Atoi-of-garbage guard).
+NODE_ID="-1"
+
+if [ -f "$CONF" ]; then
+    H=$(awk -F'"' '/\[nodes\..*\.predastore\]/{f=1} f&&/^host/{print $2;exit}' "$CONF")
+    if [ -n "$H" ]; then
+        BIND="$H"
+    fi
+
+    N=$(awk -F'= *' '/node_id/{gsub(/ /,"",$2);print $2;exit}' "$CONF")
+    if [ -n "$N" ]; then
+        NODE_ID="$N"
+    fi
+fi
+
+export SPINIFEX_PREDASTORE_HOST="${BIND%%:*}"
+export SPINIFEX_PREDASTORE_PORT="${BIND##*:}"
+export SPINIFEX_PREDASTORE_NODE_ID="$NODE_ID"
+
+exec /usr/local/bin/spx service predastore start
@@ -0,0 +1,37 @@
+#!/bin/sh
+# Wait for NATS to be reachable before starting a dependent service.
+# Reads the NATS address from spinifex.toml [nodes.*.nats] section,
+# falls back to nats.conf, then to 127.0.0.1:4222.
+
+CONF=/etc/spinifex/spinifex.toml
+NATS_CONF=/etc/spinifex/nats/nats.conf
+NATS_HOST=""
+NATS_PORT="4222"
+TIMEOUT=30
+
+# Try spinifex.toml first
+if [ -f "$CONF" ]; then
+    NATS_BIND=$(awk -F'"' '/\[nodes\..*\.nats\]/{f=1} f&&/^host/{print $2;exit}' "$CONF")
+    if [ -n "$NATS_BIND" ]; then
+        NATS_HOST="${NATS_BIND%%:*}"
+        NATS_PORT="${NATS_BIND##*:}"
+    fi
+fi
+
+# Fall back to nats.conf
+if [ -z "$NATS_HOST" ] && [ -f "$NATS_CONF" ]; then
+    NATS_HOST=$(grep "^listen:" "$NATS_CONF" | sed 's/listen: *//;s/:.*//')
+fi
+
+# Final fallback
+NATS_HOST="${NATS_HOST:-127.0.0.1}"
+
+for i in $(seq 1 "$TIMEOUT"); do
+    if nc -z "$NATS_HOST" "$NATS_PORT" 2>/dev/null; then
+        exit 0
+    fi
+    sleep 1
+done
+
+echo "Timed out waiting for NATS at $NATS_HOST:$NATS_PORT" >&2
+exit 1
@@ -0,0 +1,12 @@
+[Unit]
+Description=Regenerate SSH host keys
+ConditionPathExists=!/etc/ssh/ssh_host_ed25519_key
+Before=ssh.service
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/ssh-keygen -A
+RemainAfterExit=yes
+
+[Install]
+WantedBy=multi-user.target
@@ -0,0 +1,55 @@
+[Unit]
+Description=Spinifex AWS Gateway
+PartOf=spinifex.target
+After=spinifex-nats.service spinifex-daemon.service
+Wants=spinifex-nats.service
+
+[Service]
+Type=simple
+Slice=spinifex-system.slice
+User=spinifex-gw
+Group=spinifex
+ExecStartPre=/var/lib/spinifex/wait-for-nats.sh
+ExecStart=/usr/local/bin/spx service awsgw start
+Restart=on-failure
+RestartSec=5
+
+OOMScoreAdjust=-500
+Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
+Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/awsgw/
+Environment=SPINIFEX_AWSGW_TLS_CERT=/etc/spinifex/server.pem
+Environment=SPINIFEX_AWSGW_TLS_KEY=/etc/spinifex/server.key
+
+# Filesystem access
+NoNewPrivileges=yes
+ProtectSystem=strict
+PrivateDevices=yes
+ReadOnlyPaths=/etc/spinifex/spinifex.toml /etc/spinifex/master.key /etc/spinifex/server.pem /etc/spinifex/server.key /etc/spinifex/ca.pem
+ReadWritePaths=/var/lib/spinifex/awsgw /var/log/spinifex
+
+# Hardening
+ProtectHome=yes
+PrivateTmp=yes
+RestrictSUIDSGID=yes
+LockPersonality=yes
+RestrictRealtime=yes
+RestrictNamespaces=yes
+ProtectClock=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+ProtectControlGroups=yes
+ProtectProc=invisible
+SystemCallArchitectures=native
+MemoryDenyWriteExecute=yes
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
+CapabilityBoundingSet=
+ProtectHostname=yes
+KeyringMode=private
+RemoveIPC=yes
+LimitCORE=0
+LimitNOFILE=65536
+LimitNPROC=4096
+
+[Install]
+WantedBy=spinifex.target
@@ -0,0 +1,65 @@
+[Unit]
+Description=Spinifex Daemon (VM Orchestration)
+PartOf=spinifex.target
+After=spinifex-nats.service spinifex-viperblock.service openvswitch-switch.service ovn-controller.service openvswitch-ipsec.service
+# Wants (not Requires): stay bootable when openvswitch-ipsec is absent
+Wants=openvswitch-ipsec.service
+
+[Service]
+Type=simple
+Slice=spinifex-system.slice
+User=spinifex-daemon
+Group=spinifex
+ExecStart=/usr/local/bin/spx service spinifex start
+Restart=on-failure
+RestartSec=5
+# KillMode=process: guest QEMU survives daemon restart; the new daemon reattaches (DDIL)
+KillMode=process
+TimeoutStopSec=15
+# RG-4: infra tier — reaped only after every guest
+OOMScoreAdjust=-500
+Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
+Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/spinifex/
+Environment=SPINIFEX_WAL_DIR=/var/lib/spinifex/spinifex/
+# Pin to host tmpfs: QEMU pidfiles/QMP sockets must survive PrivateTmp namespace churn (DDIL)
+Environment=XDG_RUNTIME_DIR=/run/spinifex
+
+# RG-9: daemon tier — privileged by necessity (GPU vfio). NoNewPrivileges omitted (RG-10: sudo ip/ovs-vsctl/dhcpcd)
+AmbientCapabilities=CAP_SYS_ADMIN CAP_DAC_OVERRIDE
+ProtectSystem=full
+PrivateDevices=no
+DevicePolicy=closed
+DeviceAllow=/dev/kvm rw
+DeviceAllow=/dev/net/tun rw
+DeviceAllow=char-vfio rw
+DeviceAllow=/dev/vfio/vfio rw
+ReadOnlyPaths=/etc/spinifex/spinifex.toml /etc/spinifex/server.pem /etc/spinifex/server.key /etc/spinifex/ca.pem
+ReadWritePaths=/var/lib/spinifex/spinifex /var/log/spinifex /run/spinifex /run/spinifex/nbd
+
+# RG-9: hardening baseline
+ProtectHome=yes
+PrivateTmp=yes
+RestrictSUIDSGID=yes
+LockPersonality=yes
+RestrictRealtime=yes
+RestrictNamespaces=yes
+ProtectClock=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+ProtectControlGroups=yes
+ProtectProc=invisible
+SystemCallArchitectures=native
+MemoryDenyWriteExecute=yes
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
+ProtectHostname=yes
+KeyringMode=private
+RemoveIPC=yes
+LimitCORE=0
+LimitNOFILE=65536
+LimitNPROC=4096
+# LimitMEMLOCK=infinity: VFIO pins all guest RAM
+LimitMEMLOCK=infinity
+
+[Install]
+WantedBy=spinifex.target
@@ -0,0 +1,8 @@
+[Unit]
+Description=Spinifex guest VM tier
+Before=slices.target
+
+[Slice]
+# RG-3: child of spinifex.slice (spinifex-guests -> spinifex), sibling of
+# spinifex-system. Parent for the per-instance spinifex-instance-<id>.slice
+# leaves (0002 RG-7). Inherits the spinifex.slice ceiling; no extra limit here.
@@ -0,0 +1,49 @@
+[Unit]
+Description=Spinifex NATS Message Bus
+PartOf=spinifex.target
+Before=spinifex-predastore.service spinifex-daemon.service spinifex-awsgw.service spinifex-vpcd.service
+
+[Service]
+Type=simple
+Slice=spinifex-system.slice
+User=spinifex-nats
+Group=spinifex
+ExecStart=/usr/local/bin/spx service nats start
+Restart=on-failure
+RestartSec=5
+OOMScoreAdjust=-500
+Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/nats/nats.conf
+
+# Filesystem access
+NoNewPrivileges=yes
+ProtectSystem=strict
+PrivateDevices=yes
+ReadOnlyPaths=/etc/spinifex/nats /etc/spinifex/server.pem /etc/spinifex/server.key /etc/spinifex/ca.pem
+ReadWritePaths=/var/lib/spinifex/nats /var/log/spinifex
+
+# Hardening
+ProtectHome=yes
+PrivateTmp=yes
+RestrictSUIDSGID=yes
+LockPersonality=yes
+RestrictRealtime=yes
+RestrictNamespaces=yes
+ProtectClock=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectKernelLogs=yes
+ProtectControlGroups=yes
+ProtectProc=invisible
+SystemCallArchitectures=native
+MemoryDenyWriteExecute=yes
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
+CapabilityBoundingSet=
+ProtectHostname=yes
+KeyringMode=private
+RemoveIPC=yes
+LimitCORE=0
+LimitNOFILE=65536
+LimitNPROC=4096
+
+[Install]
+WantedBy=spinifex.target
--- a/Show More
+++ b/Show More