From 624c3180eaaddc8192f2a5048d5e668e4a1042d3 Mon Sep 17 00:00:00 2001 From: tomnewton-mulga Date: Thu, 28 May 2026 11:41:00 +1000 Subject: [PATCH] docs: fix gpu-passthrough doc sections and image tags for docs site --- docs/compute/gpu-passthrough/README.md | 66 +++++++------------------- 1 file changed, 18 insertions(+), 48 deletions(-) diff --git a/docs/compute/gpu-passthrough/README.md b/docs/compute/gpu-passthrough/README.md index 7f55bcdff..219c29666 100644 --- a/docs/compute/gpu-passthrough/README.md +++ b/docs/compute/gpu-passthrough/README.md @@ -7,6 +7,11 @@ tags: - passthrough - vfio - ec2 +sections: + - overview + - prerequisites + - instructions + - troubleshooting resources: - title: "Spinifex Repository" url: "https://github.com/mulgadc/spinifex" @@ -14,26 +19,6 @@ resources: url: "https://docs.aws.amazon.com/ec2/latest/instancetypes/ac.html" --- -# GPU Passthrough - -> Configure VFIO GPU passthrough on a Spinifex node to expose NVIDIA or AMD GPUs to EC2 instances. - -> **Note:** GPU passthrough is supported on x86_64 hosts only. - -## Table of Contents - -- [Overview](#overview) -- [Prerequisites](#prerequisites) -- [Host Setup](#host-setup) -- [Enable Passthrough](#enable-passthrough) -- [Status and Monitoring](#status-and-monitoring) -- [GPU Instance Types](#gpu-instance-types) -- [Launching GPU Instances](#launching-gpu-instances) -- [Disable Passthrough](#disable-passthrough) -- [Troubleshooting](#troubleshooting) - ---- - ## Overview Spinifex supports VFIO-based GPU passthrough, binding NVIDIA and AMD GPUs directly to guest VMs via the `vfio-pci` kernel driver. Once configured, the node exposes GPU-enabled EC2 instance types and the GPU is allocated exclusively to individual instances. @@ -42,9 +27,9 @@ At startup, Spinifex always probes for GPU hardware and surfaces the result in t The passthrough state is reflected in the login banner: -![GPU not yet enabled](../../../.github/assets/images/gpu-banner1.png) +GPU not yet enabled ---- +> **Note:** GPU passthrough is supported on x86_64 hosts only. ## Prerequisites @@ -52,9 +37,9 @@ The passthrough state is reflected in the login banner: - An NVIDIA or AMD GPU must be physically installed. - The Spinifex node must be running on bare metal (not inside a VM). ---- +## Instructions -## Host Setup +### Host Setup Run the one-time host configuration as root. This command is idempotent — safe to re-run after a reboot. @@ -79,9 +64,7 @@ Setup complete — reboot required. Then run: sudo spx admin gpu enable ``` ---- - -## Enable Passthrough +### Enable Passthrough After setup and reboot, enable GPU passthrough: @@ -91,14 +74,11 @@ sudo spx admin gpu enable The command writes the configuration, signals the daemon (`SIGHUP`), and waits up to 30 seconds for the daemon to confirm the new state. On success it prints the current GPU status. +GPU enabled -![GPU enabled](../../../.github/assets/images/gpu-enabled.png) +### Status and Monitoring ---- - -## Status and Monitoring - -### Per-node status +**Per-node status:** ```bash spx admin gpu status @@ -108,7 +88,7 @@ spx admin gpu status --node Output includes hardware detected, IOMMU state, vfio-pci binding, passthrough enabled/disabled, GPU pool allocation (`allocated/total`), and the GPU-capable EC2 instance types available on that node. -### Cluster view +**Cluster view:** `spx top nodes` includes a `GPU (used/total)` column: @@ -122,22 +102,16 @@ Output includes hardware detected, IOMMU state, vfio-pci binding, passthrough en spx top nodes ``` ---- +### GPU Instance Types -## GPU Instance Types - -GPU instance types are derived from detected hardware. - -To list available GPU instance types on the current node: +GPU instance types are derived from detected hardware. To list available GPU instance types on the current node: ```bash aws ec2 describe-instance-types \ --filters Name=instance-type,Values=g* ``` ---- - -## Launching GPU Instances +### Launching GPU Instances Import the GPU-enabled AMI before launching: @@ -165,9 +139,7 @@ nvidia-smi rocm-smi ``` ---- - -## Disable Passthrough +### Disable Passthrough Passthrough cannot be disabled while GPU instances are running. Terminate all GPU instances first, then: @@ -177,8 +149,6 @@ sudo spx admin gpu disable The command signals the daemon and waits for confirmation. The GPU is released back to the host kernel on the next reboot (the vfio-pci binding persists until `setup` is re-run without the blacklists). ---- - ## Troubleshooting ### IOMMU not active after reboot