Three blockers fixed for VPC public subnet E2E on single-node:
1. OVN SB stale data: reset-dev-env.sh now deletes both NB+SB DB files
instead of individual NB cleanup. Eliminates ovn-controller commit
failure loop from stale chassis/port binding entries.
2. Chassis name mismatch: reset-dev-env.sh passes --chassis-id=chassis-node1
to setup-ovn.sh, aligning OVS system-id with vpcd's chassis-{nodeName}.
3. Macvlan unicast filtering: switched to centralized NAT (no external_mac
on DNAT rules), set nat-addresses=router on localnet ports, and vpcd
aligns macvlan MAC with OVN gateway router MAC on startup. Macvlan only
delivers unicast matching its own MAC — this ensures inbound traffic
from the LAN reaches OVS/OVN.
Also fixed: DNS for VMs. admin init auto-detects host DNS servers via
resolvectl, stores them in spinifex.toml dns_servers field, and vpcd
propagates them to OVN DHCP options. Fallback: 8.8.8.8 + 1.1.1.1.
Validated E2E: inbound ping, SSH, outbound ping, DNS, curl, apt update.
- admin init now pre-generates VPC/subnet/IGW IDs and writes [bootstrap]
to spinifex.toml before services ever start. No race condition.
- EnsureDefaultVPC accepts optional BootstrapIDs to use pre-determined
IDs instead of generating random ones.
- CreateInternetGatewayWithID allows daemon to use the bootstrap IGW ID.
- Removed writeBootstrapConfig from daemon (no longer needed).
- Fixed bootstrap account_id: uses admin account (000000000001) not
system account (000000000000).
- reconcileIGW now sets spinifex:igw_id in OVN external IDs.
Phase 11b: RunInstances now auto-resolves the default subnet when no
SubnetId is specified, matching AWS behavior. Previously fell back to
SLIRP networking.
Phase 12: vpcd reconciles OVN topology from [bootstrap] config on startup.
The daemon writes a [bootstrap] section to spinifex.toml with the default
VPC/subnet IDs on first run. vpcd reads this on startup and creates any
missing OVN objects (router, switch, DHCP, IGW topology, SNAT, gateway
chassis). All operations are idempotent — safe on every restart.
This solves the bootstrap ordering problem where admin init creates VPC
records before vpcd is running, so NATS events are missed.
EnsureDefaultVPC now returns DefaultVPCInfo with VPC/subnet IDs.
The single-nic IP migration path was fragile and caused connectivity
issues during setup. macvlan is now the only bridge strategy — always
creates spx-ext-{nic} on the WAN NIC. Removed --single-nic flag,
SingleNIC config/detection fields, and updated all docs.
Phase 8a: Wire all 5 EIP gateway actions (AllocateAddress, ReleaseAddress,
AssociateAddress, DisassociateAddress, DescribeAddresses) end-to-end through
gateway → NATS → daemon → EIP service.
Phase 8b: Wire all 7 SG gateway actions (Create/Delete/Describe SecurityGroup,
Authorize/Revoke Ingress/Egress) through the same pipeline.
Critical fix: handleIGWAttach was not calling SetGatewayChassis on the gateway
router port. Without this, OVN has no chassis binding for external traffic —
SNAT/default route exist logically but packets have nowhere to go physically.
This was the root cause of outbound routing failure (can't ping 8.8.8.8).
Default VPC now matches AWS behavior: auto-creates IGW (attached),
MapPublicIpOnLaunch=true on default subnet, and a default security group
with intra-SG inbound allow + all outbound allow.
Comprehensive VPC networking docs rewrite covering modes, bridge setup,
configuration, security groups, EIPs, OVN reference, and troubleshooting.
Admin init output was duplicating install guide steps. Simplified to
just show completion message. Install guides already have all steps.
Set dev_networking=false when external_mode is configured — the hostfwd
NIC adds a confusing second interface inside VMs and is not needed when
public subnets provide WAN-routable IPs.
spx admin init now works with zero external networking flags:
- Auto-detects WAN interface, gateway, prefix from ip route
- Auto-detects single-NIC vs multi-NIC mode
- Defaults to nat mode with DHCP for gateway IP
- Prints detected topology with interface roles
setup-ovn.sh gains --dhcp flag: obtains a DHCP lease on the
macvlan/bridge interface for the OVN gateway SNAT address, then
writes gateway_ip into spinifex.toml. Tested on VM: macvlan gets
separate IP from router DHCP, host keeps its original IP.
Three tiers of external networking:
- Zero flags: auto nat+DHCP (VMs get outbound internet)
- --external-pool=start-end: pool mode (VMs get public IPs)
- --no-external: overlay only (no internet for VMs)
Add --single-nic flag to setup-ovn.sh. When used with --external-bridge,
creates a macvlan sub-interface (spx-ext-{iface}) in bridge mode off the
host's NIC instead of adding the NIC directly to br-external. The host
keeps its IP — no migration, SSH-safe.
Strategy selection:
- Dedicated NIC: --external-iface=eth1 (add directly)
- Single NIC: --external-iface=eth0 --single-nic (macvlan)
- IP migration: --external-iface=eth0 (legacy, no --single-nic)
Add single_nic bool field to VPCDConfig for per-node config. Different
nodes in a cluster can use different strategies. Unit test verifies
single_nic parsing alongside external_interface.
Add external IPAM module (CAS-based IP allocation from admin-defined
pools in NATS KV). Supports multi-pool with region/AZ scoping, pool
fallback logic, gateway IP reservation, and concurrent allocation via
CAS retries. 12 unit tests covering allocation, release, exhaustion,
CAS conflict, multi-pool, fallback, validation, and idempotent init.
Add --external-bridge flag to setup-ovn.sh: creates OVS bridge
br-external, adds WAN NIC, migrates host IP to bridge, and sets
ovn-bridge-mappings=external:br-external for OVN localnet ports.
Add commented [network] section and external_interface to spinifex.toml
config template for discoverability.
Add cluster-wide [network] config with external_mode and multi-pool
support (ExternalPool with region/AZ scoping). Per-node vpcd gets
external_interface for WAN NIC name.
Fix handleIGWAttach to use real external IPs from the configured pool
instead of link-local 169.254.0.1/30. Gateway router port now gets
the pool's gateway IP with correct prefix length, SNAT uses the real
external IP, and the default route points to the actual WAN gateway.
Includes pool selection logic (AZ-scoped → region-scoped → unscoped
fallback) and unit tests for config parsing, IGW with external pool,
explicit gateway IP, and pool selection.
Move EBS, cluster, and node message types from config/ to types/.
Remove duplicate NodeDiscoverResponse from gateway and daemon.
Delete dead EC2StartInstances types and tests from config.
Update all imports across source and test files.
Sharded WAL causes incomplete write errors in E2E tests. Disable by
default in hive.toml template, service.go fallback, and viperblock
constructor. Can be re-enabled via hive.toml shardwal=true.
Add [viperblock] section to hive.toml with shardwal config (default
true). The setting flows from config through viperblockd to nbdkit
plugin args, allowing operators to disable sharded WAL if needed.
Bumps viperblock dependency from v1.0.10 to v1.0.11 which includes
the sharded WAL implementation (UseShardedWAL/OpenShardedWAL), fixing
CI build failures.
Julian Sommer