mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
f877f5edef
* refactor(network/external): introduce Allocator interface + StaticPoolAllocator
Carve external IP allocation out of handlers/ec2/vpc/external_ipam.go into
network/external/static_pool.go behind a typed Allocator interface. The KV
bucket name, schema, CAS retry count, and gateway-IP reservation semantics
are unchanged so existing static-pool data carries over with no manual
migration.
ExternalIPAM stays in handlers/ec2/vpc (L5 cleanup deferred per plan) but
becomes a thin facade that constructs a StaticPoolAllocator and delegates
Allocate / Release / GetPoolRecord through the new interface. Q4 will plug
DHCPPoolAllocator into the same Allocator slot.
Plan: docs/development/improvements/external-pool-dhcp-source.md (Q1).
* feat(network/external/dhcp): package skeleton — Client + Fake + KV store
New network/external/dhcp/ package, restored from the structure deleted
in Slice C (commit 7ff013a) but rehomed under the ADR-0006 layer tree:
- client.go Client interface + AcquireRequest + Lease (with
RenewAt/RebindAt/ExpiresAt helpers).
- nclient4.go NClient4Client — AF_PACKET DORA / Renew / Release
backed by insomniacslk/dhcp/dhcpv4/nclient4.
- fake.go In-memory test fake with per-step hooks for fault
injection and acquire/renew/release call counters.
- kv.go Store over the per-AZ spinifex-dhcp-leases-<az> KV
bucket: Put / Get / Delete / List. Schema matches the
plan (absolute timestamps for acquired/t1/t2/expiry).
- *_test.go Lease-timer arithmetic, fake lifecycle, nclient4 input
validation, KV roundtrip + version-key isolation.
Not yet wired into vpcd, ExternalIPAM, or any allocator — Q3/Q4 plug
it in. Plan: docs/development/improvements/external-pool-dhcp-source.md (Q2).
* feat(network/external/dhcp): DHCPManager + daemon-side NATSClient
Adds the per-vpcd DHCPManager that owns active lease state: KV scan on
startup (re-issues RENEW for non-expired entries, drops the rest),
per-lease renewal goroutines with jittered T1/T2/expiry timers and
post-expiry re-DORA, and NATS request/reply subscribers on
vpc.dhcp.acquire / vpc.dhcp.release. Acquire is idempotent on client-id,
coalesced via singleflight so concurrent retries do not produce
duplicate wire DORAs.
Adds the daemon-side dhcp.NATSClient wrapper (RequestAcquire /
RequestRelease) that ExternalIPAM will route DHCP-sourced allocations
through in Q4. Still no allocator wiring — vpcd does not yet construct
the manager.
* feat(network/external/dhcp): wire DHCP allocators; re-allow source=dhcp
Adds DHCPGatewayLRPAllocator and DHCPPoolAllocator, the two consumers
of the Q3 DHCPManager. The gateway-LRP allocator runs in-process inside
vpcd and DORAs once per VPC (client-id `dhcp-gw-lrp-<vpcID>`, used in
centralized NAT). The pool allocator runs daemon-side and routes
per-EIP / ENI / instance acquisitions through vpcd over the restored
vpc.dhcp.acquire / vpc.dhcp.release topics.
ExternalIPAM now dispatches per-pool: static pools keep the existing
StaticPoolAllocator + KV bucket; DHCP-sourced pools attach a
DHCPPoolAllocator via EnableDHCP. vpcd constructs the dhcp.Manager
when any pool has Source="dhcp" and selects the DHCPGatewayLRPAllocator
for IGWManager; everything else keeps StaticRangeAllocator /
LinkLocalAllocator.
Config validator re-allows source="dhcp", requires bind_bridge, and
rejects range_* / gw_lrp_range_* keys on DHCP pools. Unknown sources
are rejected with the supported list. Allocator.Release uses a
server-side IP→client-id lookup so the existing (pool, ip) signature
keeps working for both static and DHCP pools.
Plan: docs/development/improvements/external-pool-dhcp-source.md (Q4)
* feat(spx admin init): restore --external-source=dhcp + --external-bind-bridge
Source defaults to dhcp when --external-pool is omitted; static remains the
default when a pool range is supplied. Adds PoolBindBridge propagation through
ConfigSettings + formation.NetworkConfig + spinifex.toml template so vpcd's
DHCPManager finds the bridge to DORA on across single-node and joiner paths.
* fix(network/external/dhcp): derive chaddr from client-id when HWAddr unset
DHCPGatewayLRPAllocator and DHCPPoolAllocator (first acquire) own no NIC
and pass an empty HWAddr to the manager. nclient4 rejected the request
with "dhcp acquire: hw_addr is required", breaking nightly cells 1/3/8/10
at AttachIGW. Derive the chaddr from the client-id via utils.HashMAC so
the upstream DHCP server sees a stable 02:xx:xx:xx:xx:xx MAC per AWS
identifier, matching the OVN-side MAC scheme operators already grep for.
* fix(network/external/dhcp): enable IFF_PROMISC on bind_bridge across DORA
Nightly cell 1 timed out at OFFER step with "no matching response packet
received". Upstream router replied unicast to the derived 02:xx chaddr
which the Linux bridge had never learned, so the kernel dropped the
frame before nclient4's AF_PACKET socket could see it.
Reference-counted IFF_PROMISC toggle around every Acquire/Renew/Release
so overlapping leases on the same bridge share one flag flip and the
last release restores prior state. Soft-fails to a warn log when
CAP_NET_ADMIN is unavailable — operator may already have promisc set.
* fix(network/external/dhcp): RFC 2131 retransmission — retry DISCOVER 3×
NClient4 was configured with WithRetry(1) + WithTimeout(60s), meaning a
single DISCOVER per Acquire. When the first frame is lost (STP convergence
on a freshly-up bridge, upstream packet loss) the call blocks the full
60s and fails. Nightly cell 1 hit this at startup: gw-lrp Acquire for two
VPCs timed out back-to-back before any e2e step ran.
Switch to WithRetry(3) with a 5s per-attempt seed; nclient4 doubles the
deadline on each retransmission (5s, 10s, 20s) and sends a fresh DISCOVER
each round. Total budget ~35s, three frames on the wire — recoverable
without the higher layer noticing.
* fix(network/external): three CI-blocking extdhcp bugs
DHCP-source pools failed e2e-nightly cells 1/3/8/10 (net: dhcp) after
the restore-dhcp-source branch landed. Three independent root causes:
* gateway nexthop dropped: GatewayIPAllocator.Allocate had no return
channel for the upstream router learned from lease.Routers. igw fell
back to link-local 169.254.0.2, unreachable on the WAN subnet, so
instance egress and SSH-in died silently. Widen the interface to
return nexthop; DHCP impl pulls Routers[0]; empty Routers is now a
hard error. Static impl returns pool.Gateway as before.
* DORA budget shrunk 6x: WithTimeout(5s) x WithRetry(3) gave a 15 s
wallclock for the full DORA, no outer backoff. Restore the deleted
dhcp_manager.go schedule [4s, 8s, 16s, 32s] under a 90 s budget with
+/- 1 s per-attempt jitter; drop nclient4Retries to 1 so the leaf
stays single-shot and the budget arithmetic lives in one place.
* every vpcd subscribed to vpc.dhcp.acquire: plain nc.Subscribe meant
every vpcd in real-multi cells DORA'd in parallel with the same
derived chaddr, causing upstream-server NAK / lease leak. Switch to
QueueSubscribe with per-AZ queue "vpcd-dhcp-workers-<az>".
Plan: docs/development/bugs/extdhcp-pool-fixes.md (mulga).
Bead: mulga-siv-150.