mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
75fb15be2b
* fix: provision br-mgmt as OVS bridge + loopback guard for LB gateway Convert br-mgmt from an externally-provisioned Linux bridge to an OVS bridge owned by scripts/setup-ovn.sh, giving every deployment type (tofu-cluster, production single-node, production multi-node) a single provisioning path. setup-ovn.sh grows --mgmt-bridge / --mgmt-cidr / --mgmt-iface / --no-mgmt-bridge flags. Default creates br-mgmt with fail-mode=standalone (excluded from ovn-bridge-mappings so ovn-controller ignores it) and assigns the mgmt IP via a systemd-networkd drop-in so it survives reboot. SetupMgmtTapDevice / CleanupMgmtTapDevice switch from `ip link master` to `ovs-vsctl add-port` / `del-port` to match the new bridge type. Add a loopback guard to wireLBAgentConfig's fallback branch: on a production single-node with awsgw bound to 127.0.0.1 and no mgmt bridge, the fallback previously injected 127.0.0.1 into LB VM cloud-init — silently unreachable because the VM's loopback is its own. The guard leaves GatewayURL empty and the existing "LB agent gateway URL not configured" error fires instead. Regression covered in daemon_wire_lb_test.go. * fix: auto-detect primary IP for admin init --bind default Bare `spx admin init` used a 127.0.0.1 default that kept awsgw on loopback only, so system VMs on br-mgmt could never reach it. Auto-detect the default-route source IPv4 instead, which keeps TLS cert SANs specific (still not a wildcard). Dev-install scripts now pass --bind 127.0.0.1 explicitly to preserve loopback-only behavior on laptops. Closes mulga-siv-7. * test: refresh wire lb loopback-bind comment for siv-7 * fix: dev-install scripts should let admin init auto-detect --bind Pinning --bind to 127.0.0.1 on dev installs left awsgw loopback-only while setup-ovn.sh still provisions br-mgmt, so LB VMs hit 10.15.8.1:9999 with no listener. Drop the explicit --bind so auto-detect picks the primary IP and the siv-6 mgmtRouteVia path routes LB traffic to a real listener. * fix: bind awsgw and predastore on 0.0.0.0 in spinifex.toml template PR #144 (nats TLS) made every service listen on --bind's specific IP. For awsgw and predastore this broke localhost-reachability that the UI proxy (spinifexui/spinifexui.go:193-194) and AWS CLI spinifex profile rely on — latent since #144, surfaced by siv-7's auto-detect default on dev installs (502 from the UI). Hardcode awsgw and predastore host fields to 0.0.0.0 so both listen wildcard; loopback clients (UI proxy, spinifex AWS CLI profile, same-host daemon → predastore) keep working, LAN clients still reach them via the LAN-IP SAN on the cert. daemon.host and nats.host stay as {{.BindIP}} because peer-advertised values must not resolve to 0.0.0.0/loopback. predastore.host normalization at config.go:207 rewrites 0.0.0.0 → 127.0.0.1 for local-node consumers, so existing callers see a usable connect address. * revert: siv-7 --bind auto-detect + siv-6 loopback guard (prep mulga-siv-8) Clears the deck before layering the listen-vs-advertise split (mulga-siv-8). The plan restores --bind=0.0.0.0 as the single-node default and introduces an explicit --advertise flag for off-host dial targets; the reverted approach (auto-detect a specific IP at init, hardcode 0.0.0.0 in the template, guard against loopback in wireLBAgentConfig) becomes redundant once that lands. Reverts: -bb0ad3afix: bind awsgw and predastore on 0.0.0.0 in spinifex.toml template -f20a47bfix: dev-install scripts should let admin init auto-detect --bind -49b0b4etest: refresh wire lb loopback-bind comment for siv-7 -b88dfdffix: auto-detect primary IP for admin init --bind default Also drops the loopback guard from siv-6 (74b6e2d) wireLBAgentConfig fallback and its regression test — subsumed by the GatewayURL precedence rework in mulga-siv-8.5 (br-mgmt → AdvertiseIP → error, no silent empty-string fallthrough). The br-mgmt OVS provisioning from74b6e2dis retained. Plan: docs/development/bugs/spx-admin-init-bind.md * feat: admin.DialTarget + AdvertiseHost helpers (mulga-siv-8.2) Introduce the two small address-resolution helpers the listen-vs- advertise split needs, with full unit coverage. No callers yet — dial-site migration lands in mulga-siv-8.5. - DialTarget(listenAddr): rewrites wildcard listen addresses (0.0.0.0:N, [::]:N, :N) to their loopback equivalents so in-process clients dial a target the daemon's cert already has in its SANs. Specific IPs, loopback, hostnames, and malformed input pass through. - AdvertiseHost(listenAddr, advertiseIP): picks the host off-host clients should dial. Wildcard listen → advertiseIP; specific host → that host. Port is dropped so callers append the protocol-appropriate port. Plan: docs/development/bugs/spx-admin-init-bind.md §Proposed Changes/1 * feat: plumb AdvertiseIP through config/formation (mulga-siv-8.3) Pure data-model change. Add AdvertiseIP to: - admin.ConfigSettings (local node, populated by init). - config.Config (per-node; mapstructure "advertise"). - formation.NodeInfo (json "advertise_ip,omitempty") so the join handshake carries it between peers. buildRemoteNodes now prefers n.AdvertiseIP and falls back to n.BindIP so joined nodes from older clusters / pre-siv-8 joiners keep working. No behaviour change yet — templates, CLI flag, and dial-site migration land in siv-8.4 and siv-8.5. * feat: --advertise flag + resolveAdvertiseIP + templates (mulga-siv-8.4) Introduce the listen-vs-advertise split at the CLI boundary. admin init/join: - --bind default for init is 0.0.0.0 (was 127.0.0.1 under siv-7 revert). - New --advertise <IP> flag on both init and join; optional, auto-detected from DetectNetwork().WAN.IP when --bind is wildcard or loopback. - resolveAdvertiseIP(bindIP, advertiseFlag, detected) implements precedence: flag > specific bindIP > WAN auto-detect > 127.0.0.1 + warning. - Join resolves advertise before building the JoinRequest so peers record a reachable dial target (not 0.0.0.0). - isMultiNode: trigger on --nodes >= 2 regardless of bind literal; operator can now run "init --nodes 3" without typing a specific --bind. - Init success message prints bind / advertise / loopback three-way split. Templates: - spinifex.toml: [nodes.X] gains advertise = "{{.AdvertiseIP}}" rendered only when AdvertiseIP is set. - predastore-multinode.toml: iam.nats_url host wrapped in the same {{if eq .BindIP "0.0.0.0"}}localhost{{...}}{{end}} shim that predastore.toml already uses, so co-resident predastore dials its local NATS via loopback instead of 0.0.0.0. Tests cover the resolveAdvertiseIP matrix, the template advertise field (present + omitted), and the predastore nats_url loopback shim. Dial-site migration and GatewayURL rework land in siv-8.5. * feat: dial-site migration + GatewayURL AdvertiseIP fallback (mulga-siv-8.5) Switch every in-process dial site to admin.DialTarget so the wildcard listen address (0.0.0.0) is rewritten to loopback before Dial(): daemon → NATS and → predastore (boot, reconnect, status probe, S3 store, service manifest write), awsgw → NATS. The TLS cert already SANs 127.0.0.1, so loopback-dial a wildcard-listen service is always valid; the old 0.0.0.0 literal was tripping TLS SAN verification. Rework wireLBAgentConfig GatewayURL precedence (spinifex/daemon/daemon.go): 1. AWSGW on specific WAN IP + br-mgmt present → WAN IP (unchanged) 2. br-mgmt present, AWSGW on 0.0.0.0 → br-mgmt IP (unchanged) 3. d.config.AdvertiseIP (single-node default install) → AdvertiseIP (NEW) 4. DevNetworking shim → 10.0.2.2 (unchanged) 5. AWSGW on specific IP, no bridge → that IP (unchanged) 6. Else → log error, skip assignment Kills the pre-siv-8 silent empty-URL failure mode that made CreateLoadBalancer fail with 'missing system credentials: gatewayURL=' on every fresh single-node install. Supersedes siv-6's loopback guard. Extend SetupAWSCredentials with a wanIP parameter (reserved for a future --operator-endpoint flag) and thread AdvertiseIP through finalizeNodeSetup's three call sites (single-node init, multi-node init, join). Print 'Bind / Advertise / Loopback' on multi-node init and join completion to match the single-node summary. Plan: docs/development/bugs/spx-admin-init-bind.md §2, §3