mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-20 03:11:46 +00:00
Josh Siviour
1654f98e18
* feat(daemon): live MemAvailable admission gate + fixed-services reserve scope Admission previously charged only guest -m against a flat reserve, blind to the co-located storage VMs, nbdkit, and daemon stack — so it greenlit launches that overcommit real RAM and OOM-kill guests on memory-tight hosts. Add a second admission gate that clamps the accounting-derived launch count by live /proc/meminfo MemAvailable (which already nets reclaimable cache and sees every untracked consumer): admitted = min(accounting, (MemAvailable-reserve)/ guestMem). Reads live per admission, fails open on a /proc read error so a transient fault can't wedge launches, and is disabled by SPINIFEX_ADMISSION_LIVE_MEM=0. Re-scope the host reserve to the fixed infrastructure tier only; per-instance nbdkit is no longer expected to fit inside it (explicit per-instance charge is a follow-up; the live gate covers it implicitly in the meantime). Pure helpers (liveMemCount, parseMemAvailableKB, liveMemAdmissionEnabled) are unit-tested; liveMemGate covered via an injected reader. * feat(daemon): charge nbdkit per-instance in admission accounting An instance's admission memory charge is now its guest -m plus the nbdkit processes backing its volumes (one per attached volume), not guest -m alone. The static reserve no longer has to absorb per-instance nbdkit, which was unsizable and the root of the host-memory-overcommit OOM. Per-volume charge is measured per class (main 768 MiB: 128 cache + plugin working set + in-flight I/O; aux 96 MiB: process floor, 0 cache), env-tunable via SPINIFEX_NBDKIT_MAIN_MIB / SPINIFEX_NBDKIT_AUX_MIB. Default layout is one main + two aux volumes (root + cloudinit + efi) ~= 0.94 GB/instance. Applied at every accounting site through instanceMemChargeMiB so charge and release never drift; the live MemAvailable gate backstops block-device-mapping volumes the static count misses. * feat(vm): three-tier guest OOM ladder + spinifex-guests.slice Replace the flat oom_score_adj=500 on every guest QEMU with a ManagedBy-derived ladder: user guests (ManagedBy=='') stay +500 and are reaped first under host memory pressure; system instances (ELBv2 LB-VMs, EKS control-plane nodes, ManagedBy!='') score 0, ranking above user guests but below the infra tier (-500). Killing a system instance degrades the data path or cluster control plane for many user VMs, so it must outlive them. Add spinifex-guests.slice as a sibling of spinifex-system under spinifex.slice — the parent tier for the per-instance leaves that land with RG-7. Guest QEMU placement into the slice is deferred to the per-instance work. * refactor(systemd): lean unit comments to RG-n tags + RG-9/RG-11 invariants Strip the multi-paragraph rationale novels from spinifex-daemon.service (48->7 comment lines), spinifex-vpcd.service (29->8) and spinifex.slice (15->5), replacing them with settings + terse # RG-n clause tags. Remove plan-doc, bead and CI-run references from comments (project policy). Directives are byte-for-byte unchanged — comment-only. Add RG-9/RG-11 invariant tests asserting per-tier confinement (locked-down storage/control tier drops all caps + NoNewPrivileges=yes; daemon keeps exactly CAP_SYS_ADMIN+CAP_DAC_OVERRIDE and the four-entry DeviceAllow; vpcd is the documented network-tier exception) and the lean-comment budget + no-plan-ref rule.
66 lines
2.1 KiB
Desktop File
66 lines
2.1 KiB
Desktop File
[Unit]
|
|
Description=Spinifex Daemon (VM Orchestration)
|
|
PartOf=spinifex.target
|
|
After=spinifex-nats.service spinifex-viperblock.service openvswitch-switch.service ovn-controller.service openvswitch-ipsec.service
|
|
# Wants (not Requires): stay bootable when openvswitch-ipsec is absent
|
|
Wants=openvswitch-ipsec.service
|
|
|
|
[Service]
|
|
Type=simple
|
|
Slice=spinifex-system.slice
|
|
User=spinifex-daemon
|
|
Group=spinifex
|
|
ExecStart=/usr/local/bin/spx service spinifex start
|
|
Restart=on-failure
|
|
RestartSec=5
|
|
# KillMode=process: guest QEMU survives daemon restart; the new daemon reattaches (DDIL)
|
|
KillMode=process
|
|
TimeoutStopSec=15
|
|
# RG-4: infra tier — reaped only after every guest
|
|
OOMScoreAdjust=-500
|
|
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
|
|
Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/spinifex/
|
|
Environment=SPINIFEX_WAL_DIR=/var/lib/spinifex/spinifex/
|
|
# Pin to host tmpfs: QEMU pidfiles/QMP sockets must survive PrivateTmp namespace churn (DDIL)
|
|
Environment=XDG_RUNTIME_DIR=/run/spinifex
|
|
|
|
# RG-9: daemon tier — privileged by necessity (GPU vfio). NoNewPrivileges omitted (RG-10: sudo ip/ovs-vsctl/dhcpcd)
|
|
AmbientCapabilities=CAP_SYS_ADMIN CAP_DAC_OVERRIDE
|
|
ProtectSystem=full
|
|
PrivateDevices=no
|
|
DevicePolicy=closed
|
|
DeviceAllow=/dev/kvm rw
|
|
DeviceAllow=/dev/net/tun rw
|
|
DeviceAllow=char-vfio rw
|
|
DeviceAllow=/dev/vfio/vfio rw
|
|
ReadOnlyPaths=/etc/spinifex/spinifex.toml /etc/spinifex/server.pem /etc/spinifex/server.key /etc/spinifex/ca.pem
|
|
ReadWritePaths=/var/lib/spinifex/spinifex /var/log/spinifex /run/spinifex /run/spinifex/nbd
|
|
|
|
# RG-9: hardening baseline
|
|
ProtectHome=yes
|
|
PrivateTmp=yes
|
|
RestrictSUIDSGID=yes
|
|
LockPersonality=yes
|
|
RestrictRealtime=yes
|
|
RestrictNamespaces=yes
|
|
ProtectClock=yes
|
|
ProtectKernelTunables=yes
|
|
ProtectKernelModules=yes
|
|
ProtectKernelLogs=yes
|
|
ProtectControlGroups=yes
|
|
ProtectProc=invisible
|
|
SystemCallArchitectures=native
|
|
MemoryDenyWriteExecute=yes
|
|
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK
|
|
ProtectHostname=yes
|
|
KeyringMode=private
|
|
RemoveIPC=yes
|
|
LimitCORE=0
|
|
LimitNOFILE=65536
|
|
LimitNPROC=4096
|
|
# LimitMEMLOCK=infinity: VFIO pins all guest RAM
|
|
LimitMEMLOCK=infinity
|
|
|
|
[Install]
|
|
WantedBy=spinifex.target
|