mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
Josh Siviour
628fbbc0c3
* fix: group-scope the OVS/OVN sockets and stop sudoing read-only probes The OVS DB socket and both sets of ctl sockets were chmod 0666 by setup-ovn.sh, with a systemd override re-applying it on every openvswitch-switch restart. Write access to db.sock is full control of the local datapath, so this handed the datapath to any account on the box. Both service users run with Group=spinifex, so the access is now 0660 root:spinifex, /var/run/ovn is 0750, and a shared helper re-applies it from ExecStartPost on openvswitch-switch and ovn-controller alike — the latter is new, and matters because the ovn-controller ctl socket name embeds the pid and is recreated on every start. That also fixes the health probe reporting ovn-controller as not_running on a healthy node. It shelled out to `sudo ovn-appctl`, which only spinifex-vpcd is granted; as spinifex-daemon sudo demanded a password and the swallowed error read as OVN being down. With the ctl socket group-owned the probe needs no privilege, so health.go moves to a non-escalating Runner rather than the alternative of granting the daemon a root-equivalent ovn-appctl rule to read a status. ListIMDSTaps ran `list-ports` plus one `get` per patch port on vpcd's 15s tick, so 1+N sudo invocations every tick, each one logging a pam_limits warning about the core rlimit the unit deliberately pins to 0. It now issues a single JSON query per pass, and the sudoers drop-in disables the PAM session for both service users, which also leaves sudo children inheriting the unit's rlimits. An absent iface-id on a patch port is now skipped with a warning instead of aborting the pass: it is the only record of the full ENI, so such a port cannot join the live set either way, and failing the pass stalled reconcile for every healthy port behind it. A transient query failure still aborts. * fix: group-own the OVS/OVN pidfiles too, and cover the ipsec ctl socket Group-owning the ctl sockets was not enough for `ovn-appctl -t ovn-controller`: it resolves the pid-named socket through the pidfile, and OVS opens that O_RDWR to test the fcntl liveness lock, so a group-readable pidfile still fails the probe with EACCES. The pidfiles now get the same 0660 root:spinifex, which also tightens them from the 0644 they ship with. openvswitch-ipsec joins the override list because setup-ovn.sh restarts it after the sweep runs, leaving its ctl socket as the one file still at the shipped mode. Verified on the dev node: every socket and pidfile is 0660 root:spinifex and survives a restart of all three units, an unprivileged user is refused, and /health reports ovn-controller ok where the sudo probe read not_running. * refactor: stop escalating the OVS/OVN socket clients Now that setup-ovn.sh group-owns the control sockets, ovs-vsctl, ovs-appctl, ovn-nbctl, ovn-sbctl, ovn-appctl and systemctl reach their daemons as the service user. utils.NeedsPrivilege carries that policy and both command builders consult it, so ip, iptables, dhcpcd, sysctl and arping still escalate while the socket clients no longer do. Their sudoers grants are removed: each took unrestricted args and all of them accept --log-file=PATH, so every one was a root-equivalent grant handed out to run a query. ovs-ofctl keeps its grant. It talks to a per-bridge <bridge>.mgmt socket that ovs-vswitchd creates when the bridge appears, including bridges spinifex creates at runtime, so the provisioning sweep cannot group-own it up front. The NB and SB database sockets join the sweep for ovn-nbctl and ovn-sbctl. vpcd carried its own sudoCommand copy that escalated unconditionally, bypassing the policy entirely. It survived only because the grants existed and broke the OVN flows-ready barrier the moment they were removed, so it now delegates to utils.SudoCommand. TestRG10_SudoOnlyThroughThePolicy fails the build if another package reintroduces a local copy. Verified on the dev node: an instance launches, IMDS is served, the flows-ready barrier completes, and no sudo denial appears anywhere in the journal. * refactor: drop every spinifex-vpcd sudoers grant for ambient capabilities vpcd held NOPASSWD grants for ip, iptables, sysctl and arping. The ip grant alone was root-equivalent: sudoers cannot constrain its arguments, so `sudo ip netns exec <ns> /bin/sh` was a root shell for the service account. The unit already grants CAP_NET_ADMIN and CAP_NET_RAW ambiently, and the kernel passes an ambient set to each child on exec, so those tools work as the service user with no escalation at all. Verified on a live node: all sixteen ip subcommands vpcd issues, plus iptables reads and writes, sysctl -w on the net.ipv4.neigh tree, and arping, all succeed under exactly the unit's ambient set. spinifex-vpcd now has no sudoers rules. NeedsPrivilege had to become capability-aware to do this. spx is one binary shared by both service units, and the daemon holds CAP_SYS_ADMIN and CAP_DAC_OVERRIDE but no CAP_NET_ADMIN, so a static tool list cannot express "vpcd runs ip directly, the daemon escalates it". The decision now reads CapAmb from /proc/self/status once and matches it against the capability each invocation needs. sysctl is resolved per key, since CAP_NET_ADMIN covers the net.* trees and nothing else. The dhcpcd grants go too: neither daemon invokes it. Only setup-ovn.sh, which runs as the provisioning user, and the in-guest ecs-agent. TestRG10_VPCDHoldsNoSudoersGrant pins both halves together, so a future grant cannot be re-added without also explaining why the capability stopped working. * chore: restore routeTree.gen.ts swept in by mistake * fix: probe ovn-controller with ovn-appctl, not ovs-appctl checkOVNController's first probe passed a bare -t ovn-controller to ovs-appctl, which resolves it against OVS_RUNDIR — /var/run/openvswitch. ovn-controller's socket and pidfile are in /var/run/ovn, so the probe could never succeed and logged "ovn-controller.pid: open: No such file or directory" on every check. Not a functional break: the next probe globs the full /var/run/ovn path and succeeds, so the function still returned nil. It cost an exec per check and made a healthy node look like it was failing an OVN liveness probe. ovn-appctl defaults OVS_RUNDIR to /var/run/ovn, so the common case now matches first. The glob and systemctl fallbacks stay for older layouts.