Ben Duncan
8b3c706ea3
Multi-node install - Lead with the fact that the guide builds a three-node cluster, and say how to go beyond three. The callout sits inside ## Overview because the docs site discards everything above the first section heading. - Replace the OVN-only sizing prose with a table covering all three distributed layers, and move the OVN control-plane detail into the VPC networking guide, linked from here. - Add minimum and recommended hardware. - Move the object storage layout to the end as background; it was the second thing a reader hit. - Collapse the ISO conversion notes from a section into one skippable box, now that --force is unconditional in Step 4 and the formation port needs no manual intervention. - Expand verification with spx get nodes and spx top nodes, including what to look for in each. - Drop the install-node.sh shortcut (internal tooling) and the "adding a node later" section (no good story yet). Corrections found while writing - Servers 4+ were described as running no database. They run predastore and viperblock like every other node; only the OVN database cluster is capped at three. Fixed in four places. - Step 4 never needed the firewall down: spx admin init opens the formation port itself for the formation window. Only Step 3's peer-scoped OVN ports are affected. - `nft list table inet spinifex_filter | grep spinifex_peers` could never match — the peer list is an nft define, expanded at load time. Anyone following it would have seen empty output and assumed the worst. - Disambiguate "database": with predastore, NATS and OVN all in play it needed qualifying throughout. - Align the example output region with the AWS_REGION the guide exports. Cross-doc links - Rewrite relative and category-prefixed links to the canonical /docs/<slug> form. The site publishes docs flat, so ../foo/README.md and /docs/category/foo both 404. - Point the six links to unpublished docs at the source repo instead. - Fix four fragments written against GitHub's anchor algorithm.
title, seoTitle, description, category, tags, resources
| title | seoTitle | description | category | tags | resources | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Air-Gapped Install | Install Spinifex in an Air-Gapped Network — Spinifex Docs | Deploy Spinifex in environments without internet connectivity. Covers using a pre-built release tarball, mirrored APT packages, and locally-staged cloud images. | Install |
|
|
Air-Gapped Installation
Deploy Spinifex in environments without internet connectivity.
Table of Contents
Overview
In air-gapped or disconnected environments, Spinifex can be deployed without internet access. This guide covers preparing offline packages on a connected machine, creating USB deployment media, and installing on the target server with package verification.
Instructions
Step 1. Download the Release (on a connected machine)
Each Spinifex release publishes a self-contained tarball, the matching setup.sh, and a SHA-256 checksum to GitHub Releases. Resolve the latest tag and download the assets for your architecture:
ARCH=amd64 # or arm64
TAG=$(curl -fsSL https://api.github.com/repos/mulgadc/spinifex/releases/latest \
| grep '"tag_name"' | cut -d'"' -f4)
BASE="https://github.com/mulgadc/spinifex/releases/download/${TAG}"
curl -fsSLO "${BASE}/spinifex-${TAG}-linux-${ARCH}.tar.gz"
curl -fsSLO "${BASE}/spinifex-${TAG}-linux-${ARCH}.tar.gz.sha256"
curl -fsSLO "${BASE}/setup.sh"
sha256sum -c "spinifex-${TAG}-linux-${ARCH}.tar.gz.sha256"
Step 2. Stage Dependencies (on the connected machine)
Pre-download the APT packages installed by the production setup. apt install --download-only writes .deb files to /var/cache/apt/archives/ without installing them, so the connected machine isn't modified:
sudo apt update
sudo apt install --download-only -y \
nbdkit nbdkit-plugin-dev pkg-config \
qemu-system-x86 qemu-utils \
ovmf qemu-efi-aarch64 \
libvirt-daemon-system libvirt-clients libvirt-dev \
ovn-central ovn-host openvswitch-switch \
dhcpcd-base make gcc jq curl iproute2 netcat-openbsd \
wget unzip xz-utils file
Download AWS CLI v2:
curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" \
-o awscliv2.zip
Mirror the cloud image you intend to run as guest VMs:
mkdir -p images
curl -fsSL "https://cloud.debian.org/images/cloud/trixie/latest/debian-13-genericcloud-amd64.qcow2" \
-o images/debian-13-amd64.qcow2
Step 3. Assemble Transfer Media
mkdir -p /media/spinifex-deploy/{tarball,apt-packages,aws,images}
cp spinifex-${TAG}-linux-${ARCH}.tar.gz /media/spinifex-deploy/tarball/
cp setup.sh /media/spinifex-deploy/
cp /var/cache/apt/archives/*.deb /media/spinifex-deploy/apt-packages/
cp awscliv2.zip /media/spinifex-deploy/aws/
cp images/*.qcow2 /media/spinifex-deploy/images/
Step 4. Install on the Air-Gapped Target
Mount the media, install APT dependencies, install AWS CLI, then run setup.sh with the local tarball:
sudo mount /dev/sdb1 /mnt/usb
sudo dpkg -i /mnt/usb/apt-packages/*.deb
sudo apt-get install -f --no-download # resolve any leftover deps from local cache
cd /tmp && unzip /mnt/usb/aws/awscliv2.zip && sudo ./aws/install
INSTALL_SPINIFEX_TARBALL=/mnt/usb/tarball/spinifex-*-linux-*.tar.gz \
INSTALL_SPINIFEX_SKIP_APT=1 \
INSTALL_SPINIFEX_SKIP_AWS=1 \
bash /mnt/usb/setup.sh
Run setup.sh without sudo — the script handles privilege escalation internally and ends by exec-ing into a newgrp spinifex subshell so your current shell picks up spinifex group membership. Without that membership, AWS CLI cannot traverse /etc/spinifex/ (mode 0750) to read ca.pem and Step 9 will fail with a TLS error. Type exit to leave the subshell when finished.
Step 5. Setup OVN Networking
spx admin init requires OVN/OVS to be configured before the daemon can manage tenant networks. setup-ovn.sh ships in the tarball and runs purely against local commands (no network access required):
sudo /usr/local/share/spinifex/setup-ovn.sh --management
If your WAN interface is a physical NIC rather than a bridge, pass --wan-bridge=br-wan --wan-iface=<nic>.
Step 6. Initialize Without Telemetry
sudo spx admin init --node node1 --nodes 1 --no-telemetry
This generates the cluster configuration, TLS certificates, installs the CA into the system trust store, and writes AWS CLI credentials. See Single-Node Install for what spx admin init does.
Step 7. Start Services
setup.sh enables spinifex.target but does not start it on a fresh install. Start it now so predastore is online before you import images:
sudo systemctl start spinifex.target
Step 8. Import Cloud Images
Register the pre-staged image with Spinifex:
sudo spx admin images import --file /mnt/usb/images/debian-13-amd64.qcow2 \
--distro debian --version 13 --arch x86_64 --boot-mode uefi
Step 9. Verify
export AWS_PROFILE=spinifex
aws ec2 describe-instance-types
aws ec2 describe-images
If both calls return data, your air-gapped install is working. Continue to Setting Up Your Cluster to launch your first instance.
Troubleshooting
Missing APT Dependencies
dpkg -i does not resolve transitive dependencies. After installing, run:
sudo apt-get install -f --no-download
The --no-download flag forces apt to use only what's in the local cache. If a dependency is genuinely missing, add it to the apt --download-only step on the connected machine and re-stage.
Setup.sh Tries to Download Anyway
Confirm both skip flags are exported and that INSTALL_SPINIFEX_TARBALL points at a readable file:
sudo INSTALL_SPINIFEX_TARBALL=/mnt/usb/tarball/spinifex-...tar.gz \
INSTALL_SPINIFEX_SKIP_APT=1 \
INSTALL_SPINIFEX_SKIP_AWS=1 \
bash -x /mnt/usb/setup.sh
Init Telemetry Attempted
spx admin init posts a one-shot record to https://install.mulgadc.com/install. Pass --no-telemetry on every init and join invocation, or export SPX_NO_TELEMETRY=1 in the operator's shell profile.
Image Import Fails
spx admin images import --file requires the distro/version/arch/boot-mode flags so the image registers in the catalogue. If the import succeeds but aws ec2 describe-images returns empty, check journalctl -u spinifex-daemon -f for predastore upload errors.