mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 09:01:31 +00:00
Josh Siviour
7d5ebabecc
* fix(systemd): give every unit an explicit TimeoutStopSec 9 of 12 spinifex systemd units had no TimeoutStopSec and inherited systemd's 90s default, which is either far more or far less time than their own SIGTERM handling needs. Size each one to what its shutdown path actually does: predastore matches viperblock's 30s WAL-drain budget since viperblock flushes through it (After= already sequences viperblock's stop first; this is the backstop if that's ever bypassed), spinifex-ui covers its 30s internal shutdown context plus margin for long-lived streaming proxy connections, nats covers its own JetStream-shutdown/raft-stepdown/client-drain signal handler, and services with no SIGTERM handler at all (awsgw) get a short bound since they exit immediately regardless. Extends the existing unit-file invariant test with a check that every .service file declares TimeoutStopSec, so a newly added unit that forgets one fails CI instead of silently falling back to 90s. * fix(vm): reconcile QEMU processes with no instance record on restore classifyRestoredInstances only walks Manager.Snapshot(), so a qemu-system process with no instance record at all -- a crash mid-launch, a lost KV write, a stale process surviving a wiped state file -- was never handed to killOrphanedQEMU and stayed permanently invisible to recovery. Add a host-process scan that diffs live qemu-system PIDs (matched via /proc/<pid>/comm) against the PIDs claimed by known instances' own -pidfile files. It only reports what it finds, never kills: this daemon cannot distinguish "record not loaded yet" or "owned by another daemon/tenant on this host" from a genuine orphan, and killing a running customer VM to fix an invisibility gap is the wrong trade. Runs at the end of Restore, once every known instance has been classified and relaunched, so a live process still unaccounted for at that point genuinely has no record anywhere. * fix(vm): distinguish recordless QEMU orphans by pidfile ownership A recordless qemu-system process with a stale spinifex pidfile is positively this daemon's own orphan; one with no pidfile at all could belong to another tenant on a shared host. Log the two cases distinctly instead of one blanket warning, so an operator can tell them apart. Also sandbox XDG_RUNTIME_DIR in the shared daemon test harness so Restore()-driven tests never read this box's real runtime directory. * fix(spinifexui): treat ErrServerClosed as a clean shutdown, not a failure Serve returns http.ErrServerClosed after Shutdown is called deliberately (the SIGTERM handler above does exactly that), but Start propagated it as an error, making systemd log a graceful stop as exit code 1 and Result: exit-code -- indistinguishable from a real crash.