mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-19 19:01:42 +00:00
tomnewton-mulga
28ca689130
* feat: add GPU discovery package (mulga-992) Adds spinifex/gpu package with sysfs-based GPU enumeration, IOMMU group member lookup, and a PCI device ID model table. Vendor-specific enrichment via nvidia-smi is attempted opportunistically and falls back to the built-in table when unavailable. All discovery and IOMMU logic is covered by unit tests using mock sysfs trees. * feat: add VFIO bind/unbind and GPU pool manager (mulga-993) Adds gpu.BindVFIO/UnbindVFIO using sysfs driver_override for per-device driver switching, and gpu.Manager which owns the claim/release lifecycle for the GPU pool including full IOMMU group binding, rollback on partial failure, and marking entries unavailable when rebind fails after VM termination. * feat: add G5 instance type generator with GPU model support (mulga-994) - Add GPUModel type, NVIDIAa10g constant, and GPUModelForVendorDevice lookup - Add generateGPUTypes to emit g5.* types with GpuInfo populated - Add IsGPUType helper for consumers filtering GPU-capable types - Update DetectAndGenerate to accept gpuModels []GPUModel parameter * feat: integrate GPU passthrough into daemon lifecycle (mulga-995) - Add gpu_passthrough config flag; discover GPUs and build manager at startup - Gate g5 instance types on GPU pool availability via canAllocateCount - Claim GPU (VFIO bind) in LaunchInstance; inject vfio-pci QEMU device - Release GPU (VFIO unbind, rebind to host driver) in stopInstance - Re-claim GPU on daemon restart when QEMU process is still alive - Add UseUEFI / OVMF support for x86_64 VMs (optional, falls back to SeaBIOS) - Fix Claim to use stored OriginalDriver when device is already vfio-pci bound * feat: add GPU fields to NodeStatus and spx top nodes (mulga-996) - Add TotalGPUs/AllocGPUs to NodeStatusResponse (zero on non-GPU nodes) - Populate from gpuManager in handleNodeStatus - Show GPU (used/total) column in spx top nodes; "-" when no GPUs present * feat: Ubuntu 24.04 GPU AMI build pipeline (mulga-997) Extends build-system-image.sh to support Ubuntu alongside Alpine. Key differences handled: partition table (parted resizepart p1), bind mounts for DKMS (/proc /sys /dev), apt vs apk, systemctl vs rc-update. DISTRO defaults to "alpine" for backward compatibility. Adds ubuntu-gpu.conf manifest and ubuntu-gpu-setup.sh chroot script that installs nvidia-headless-550-server with DKMS pre-built against a pinned linux-image-generic kernel, blacklists nouveau, and prevents unattended-upgrades from invalidating the pre-built module. * Fixed GPU passthrough and added test script * feat: always-probe GPU at startup, surface capability in NodeStatusResponse - Add gpuProbeResult struct and probeGPU() which runs at daemon startup regardless of gpu_passthrough config. Checks sysfs for GPUs, IOMMU groups, and vfio_pci module presence. - Store gpuProbe on Daemon struct; split NewDaemon into probe (always) and activate (opt-in) phases. - Populate GPUCapable, GPUPassthrough, GPUModels on NodeStatusResponse so operators can see hardware state before enabling passthrough. - Update 'spx top' GPU column: '-' = no hardware, '0/N*' = capable but passthrough not active, 'used/total' = passthrough active. * feat: SIGHUP hot-reload for GPU passthrough config (mulga-1001) - Export GenerateGPUTypes for use by the reload path - Add ResourceManager.reloadGPUTypes: mutates instanceTypes map in-place (preserves references held by instanceService) and updates NATS subscriptions without full daemon restart - Separate SIGHUP from SIGTERM/SIGINT: setupReload() listens for SIGHUP in a context-aware loop; setupShutdown no longer intercepts SIGHUP - Add reloadConfig() + applyGPUConfig(): re-reads spinifex.toml on signal, handles false→true (re-probe + activate) and true→false (refuse if instances running, otherwise deactivate) transitions * feat: spx admin gpu status/enable/disable commands (mulga-1000) - admin.SetGPUPassthrough: Go implementation of the idempotent TOML write (replaces the inline Python heredoc in spx-test-gpu.sh) - spx admin gpu status: shows GPU hardware, IOMMU/vfio state, passthrough activation, and advertised g5 instance types via NATS NodeStatusResponse - spx admin gpu enable: checks prerequisites, writes TOML, sends SIGHUP, polls until daemon confirms GPUPassthrough=true - spx admin gpu disable: refuses while GPU instances are running, otherwise writes TOML, sends SIGHUP, polls until confirmed - spx-test-gpu.sh: replace Python TOML heredoc with spx admin gpu enable * fixed admin * fix: filter Intel iGPU from passthrough pool, use lspci for consumer GPU names Discover() now skips non-NVIDIA/AMD vendors so Intel iGPUs never enter the passthrough pool. enrichFromLspci() uses the system PCI ID database via lspci -vmm to populate the model name for any card not in the hardcoded table, without needing nvidia-smi or driver tools. * fix: grant CAP_SYS_ADMIN and vfio device access to spinifex-daemon GPU passthrough requires writing to /sys/bus/pci/drivers/vfio-pci/unbind on instance stop. The kernel gates this on CAP_SYS_ADMIN regardless of file permissions. QEMU subprocesses also need char-vfio rw access to open /dev/vfio/N for the passed-through device. * fixing gpu bining permissions * working state of gpu passthrough on phyiscal box * installer grub image fixes * made gpu-passthrough more user facing * fixed banner * fixed banner * banner fix * gpu passthrough docs * preflight tweak * vpcd fixes to set br-wan to promisc for public IP DHCP processes * fixed vpcd br-wan promisc and veth pair * ssh keygen fix * fixes after running /review-branch * feat: re-wire GPU claim and device injection after vm.Manager refactor GPU passthrough claim (VFIO bind) and QEMU device injection were lost when dev's vm.Manager refactor (#185/#186) moved StartInstance/ CreateQMPClient into vm/lifecycle.go. Re-add both in their correct new homes: - daemon_handlers_instance.go: claim GPU before vmMgr.Run() in both RunInstances and StartStoppedInstance handlers; release on failure - vm/lifecycle.go startQEMU(): inject vfio-pci device with x-vga flag derived from instance.GPUXVGAEnabled when GPUPCIAddress is set * fix: Create spinifex user ~/.ssh and keys in firstboot --------- Co-authored-by: Ben Duncan <5178243+benduncan@users.noreply.github.com> Co-authored-by: Julian Sommer <115516918+juliansommer@users.noreply.github.com>
76 lines
2.7 KiB
Go
76 lines
2.7 KiB
Go
/*
|
|
Copyright © 2026 Mulga Defense Corporation
|
|
|
|
This program is free software: you can redistribute it and/or modify
|
|
it under the terms of the GNU Affero General Public License as published by
|
|
the Free Software Foundation, either version 3 of the License, or
|
|
(at your option) any later version.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU Affero General Public License for more details.
|
|
|
|
You should have received a copy of the GNU Affero General Public License
|
|
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
*/
|
|
|
|
// Package install performs the disk installation steps after the UI has
|
|
// collected configuration.
|
|
package install
|
|
|
|
// Config holds all values collected by the installer UI.
|
|
type Config struct {
|
|
// Disk is the block device path to install onto (e.g. /dev/sda).
|
|
Disk string
|
|
|
|
// WAN interface — always present. The management IP lives on br-wan (a Linux
|
|
// bridge over the NIC), not on the physical NIC itself. This allows OVN/OVS
|
|
// to safely attach to the bridge without disrupting host connectivity.
|
|
WANInterface string
|
|
WANDHCPMode bool
|
|
WANAddress string // empty if WANDHCPMode
|
|
WANMask string // empty if WANDHCPMode
|
|
WANGateway string // empty if WANDHCPMode
|
|
WANDNS []string // empty if WANDHCPMode; e.g. ["1.1.1.1", "8.8.8.8"]
|
|
WANWiFiSSID string // non-empty if a WiFi NIC was selected
|
|
WANWiFiPass string // non-empty if a WiFi NIC was selected
|
|
|
|
// LAN interface — only set when 2+ NICs are present. The Geneve encap IP
|
|
// for OVN tunnels lives on br-lan. When empty, the WAN address is used.
|
|
LANInterface string
|
|
LANDHCPMode bool
|
|
LANAddress string
|
|
LANMask string
|
|
LANDNS []string
|
|
LANWiFiSSID string
|
|
LANWiFiPass string
|
|
|
|
// Node identity
|
|
Hostname string
|
|
|
|
// ClusterRole is "init" or "join".
|
|
ClusterRole string
|
|
|
|
// JoinAddr is the primary node address (host:port) when ClusterRole is "join".
|
|
JoinAddr string
|
|
|
|
// CA certificate (PEM), optional.
|
|
HasCACert bool
|
|
CACert string
|
|
|
|
// RootPassword is the password to set for the root account on the installed system.
|
|
RootPassword string
|
|
|
|
// Email is the operator's email address, used by the call-home telemetry
|
|
// endpoint to notify of important system updates and security advisories.
|
|
// Required on interactive installs; may be empty on headless/CI installs
|
|
// when SPINIFEX_EMAIL is not supplied on the kernel cmdline.
|
|
Email string
|
|
|
|
// GPUPassthrough enables VFIO GPU passthrough on this node.
|
|
// Set via SPINIFEX_GPU_PASSTHROUGH=1 on headless installs.
|
|
// Passes --gpu-passthrough to `spx admin init` during firstboot.
|
|
GPUPassthrough bool
|
|
}
|