Commit Graph

20 Commits

Author SHA1 Message Date
Ben Duncan 5426dc1bb0 feat: improved certificate authority with auto-discovery, CA download, and cloud-init injection
Auto-discover all node IPs (WAN, mgmt, VPC) and hostname for server
cert SANs — fixes TLS failures when accessing Spinifex via non-localhost
IPs. Adds CA download endpoint (/api/ca.pem) with install instructions
on the login page. Injects CA cert into guest VM cloud-init so EC2
instances trust Spinifex services without --insecure. Adds 'spx admin
cert renew' command for cert regeneration after IP changes.

Includes certificate validation E2E test suite (run-cert-e2e.sh) with
system trust store, SAN, and in-guest TLS verification across single,
pseudo multi-node, and real multi-node modes.

Closes: mulga-777, mulga-778, mulga-779, mulga-780, mulga-781
Plan: docs/development/feature/cert-manager.md
2026-03-28 14:29:07 +10:00
Julian Sommer de1340a331 fix: let reset-dev-env.sh auto-detect chassis ID from hostname
Previously hardcoded CHASSIS_ID="chassis-node1" which could mismatch
the OVS system-id from prior setup-ovn.sh runs, creating stale chassis
entries in the SBDB. Now defers to setup-ovn.sh's auto-detect
(chassis-$(hostname -s)) so the name is always consistent.
2026-03-27 12:44:01 +10:00
Julian Sommer a92ea13798 fixes 2026-03-27 11:30:39 +10:00
Julian Sommer cbc04a2437 fix: chassis names mismatch 2026-03-27 10:26:16 +10:00
Ben Duncan e5801aa6ba fix: NAT Gateway E2E order + CA cert in reset-dev-env (mulga-763)
E2E: associate subnet with route table BEFORE adding the NAT GW route.
CreateRoute publishes SNAT events for associated subnets — if the
association doesn't exist yet, no SNAT rule is created in OVN.

Validated locally: bastion SSH hop works, ping 8.8.8.8 passes with
NAT GW, fails without. All 5 steps confirmed.

reset-dev-env.sh: trust Spinifex CA cert before starting services,
fixing TLS cert verify failures on fresh resets.
2026-03-26 16:23:38 +10:00
Ben Duncan 4d56565c3f feat: veth pair bridge — link Linux bridge to OVS without conversion
setup-ovn.sh now detects whether the default route is on a Linux bridge
or OVS bridge. When it's a Linux bridge (e.g. br-wan from cloud-init),
instead of destructively converting it, the script creates an OVS bridge
(br-ext) and links them with a veth pair. The Linux bridge keeps its IP,
routes, and ports untouched — no WAN interruption.

Also fixes all pre-existing lint issues (gofmt, ineffassign, errchkjson,
gosec, modernize, whitespace) to bring golangci-lint to 0 issues.

Tested: pool mode with static IP range on single-node (ben-banksia).
VM gets public IP 192.168.1.201, SSH works, outbound internet works.
2026-03-25 10:01:37 +10:00
Ben Duncan c2992ebe5e fix: reset-dev-env.sh — pass ADMIN_INIT_ARGS to second admin init --force
The second admin init --force was called without external networking
flags, overwriting the pool/nat config from the first call with
auto-detected NAT+DHCP defaults.
2026-03-24 17:09:48 +10:00
Ben Duncan 1fca43c9f4 refactor: reset-dev-env.sh + stop-dev.sh — use new setup-ovn.sh flags
reset-dev-env.sh: Replace --external-bridge/--external-iface/--direct
with --wan-bridge/--wan-iface/--macvlan. Auto-detect when WAN interface
is already a bridge (tofu-cluster) vs physical NIC.

stop-dev.sh: Replace hardcoded br-external check with dynamic scan of
all OVS bridges — any non-br-int bridge with an IP means OVS is
infrastructure and should not be stopped.

Part of mulga-740.
2026-03-24 15:58:49 +10:00
Ben Duncan 216ca9e710 feat: Phase 12b + Phase 16 — KV reconcile on startup + NAT mode support
Phase 12b: vpcd now reconciles OVN topology from NATS KV on startup
(Pass 2). After subscribing to topics, it reads all VPCs, subnets,
IGW attachments, and ENIs from KV and ensures matching OVN objects
exist. Handles reboots, OVN DB loss, and missed events. All operations
are idempotent.

Phase 16 prep: Add --gateway-ip flag to admin init for NAT mode
(specifies the OVN gateway router's external IP without DHCP).
Update reset-dev-env.sh to support EXTERNAL_MODE=nat env var for
NAT/DHCP mode testing alongside the default pool mode.
2026-03-24 15:03:04 +10:00
Ben Duncan 88baf5e202 feat: direct bridge mode — prefer over macvlan when WAN NIC != SSH NIC
Auto-detect bridge topology: if a dedicated WAN NIC exists (not used for
SSH), add it directly to br-external instead of creating a macvlan.
Direct bridge enables distributed NAT (DNAT on VM's own chassis) and
eliminates macvlan workarounds (MAC alignment, gratuitous ARPs,
centralized NAT bottleneck).

- topology.go: bridge-mode-aware NAT and localnet port options
- vpcd.go: auto-detect bridge mode, skip MAC alignment for direct bridge
- setup-ovn.sh: --direct flag for physical NIC on br-external
- reset-dev-env.sh: SSH NIC detection via SSH_CONNECTION
2026-03-24 01:57:54 +00:00
Julian Sommer 35d0df5b56 feat: placement groups (#85) 2026-03-24 10:52:00 +11:00
Ben Duncan 93aaf0f991 fix: single-node E2E — OVN SB cleanup, chassis naming, macvlan NAT, DNS
Three blockers fixed for VPC public subnet E2E on single-node:

1. OVN SB stale data: reset-dev-env.sh now deletes both NB+SB DB files
   instead of individual NB cleanup. Eliminates ovn-controller commit
   failure loop from stale chassis/port binding entries.

2. Chassis name mismatch: reset-dev-env.sh passes --chassis-id=chassis-node1
   to setup-ovn.sh, aligning OVS system-id with vpcd's chassis-{nodeName}.

3. Macvlan unicast filtering: switched to centralized NAT (no external_mac
   on DNAT rules), set nat-addresses=router on localnet ports, and vpcd
   aligns macvlan MAC with OVN gateway router MAC on startup. Macvlan only
   delivers unicast matching its own MAC — this ensures inbound traffic
   from the LAN reaches OVS/OVN.

Also fixed: DNS for VMs. admin init auto-detects host DNS servers via
resolvectl, stores them in spinifex.toml dns_servers field, and vpcd
propagates them to OVN DHCP options. Fallback: 8.8.8.8 + 1.1.1.1.

Validated E2E: inbound ping, SSH, outbound ping, DNS, curl, apt update.
2026-03-23 10:59:08 +00:00
Ben Duncan daae6ca6b8 refactor: remove single-nic bridge mode, standardize on macvlan
The single-nic IP migration path was fragile and caused connectivity
issues during setup. macvlan is now the only bridge strategy — always
creates spx-ext-{nic} on the WAN NIC. Removed --single-nic flag,
SingleNIC config/detection fields, and updated all docs.
2026-03-23 01:12:22 +00:00
Julian Sommer b7678ab7c5 feat: fix silent failures across spinifex (P2 audit H-25 through H-43)
- PID file failures now prevent service start instead of logging and continuing
- Status() methods use PID-based health checks instead of returning empty string
- NewResourceManager returns error instead of silently defaulting to 8GB
- Crash recovery re-allocates resources before relaunch
- restoreInstances stops after failed resource allocation
- Cluster manager bind errors propagated to Start()
- NATS disconnect/reconnect logged at Warn level
- VPC CIDR fallback logs warning about overbroad rule
- vpcd respond() handles json.Marshal failure
- Network cleanup failures logged instead of silently discarded
- fmt.Println replaced with slog.Debug in NATS service
- Signature mismatch no longer logs expected signature
- extractDiskImagePath uses errors.Is for EOF
- validateDiskImagePath checks file command error
- os.Stat uncommented in admin image import
- net.SplitHostPort errors checked
- Gateway response write errors logged
- vpcd OVN NB address uses leader IP on non-primary nodes
- Release workflow parallelized with native arm64 runner
2026-03-19 13:37:41 +10:00
Julian Sommer 7d43d0f782 hopefully last rename 2026-03-18 13:24:45 +10:00
Ben Duncan 5d93786995 refactor: rename hive to spinifex in all shell scripts
- Update scripts/: start-dev, stop-dev, dev-setup, setup, setup-ovn, etc.
- Update tests/e2e/: all E2E test scripts
- Update scripts/iac/: rename hive-test.sh → spinifex-test.sh, update TF files
- Update config/: rename hive.service and install script
- Binary refs use spx, env vars use SPINIFEX_, paths use spinifex
2026-03-18 01:04:12 +00:00
Julian Sommer 371193a832 feat: IAM v1 (#59) 2026-03-03 15:23:19 +11:00
Julian Sommer 3bca0e69df feat: block device mappings 2026-01-29 14:10:04 +10:00
Julian Sommer 5878669eb0 fix for predastore multi node (#9) 2026-01-27 13:43:01 +11:00
Ben Duncan 83fd425a60 Latest changes 2026-01-27 08:17:06 +10:00