Ben Duncan
46c0c44cc2
* feat: run every Terraform workbook on OCI, and split the guide in two The OCI topology driver ran the shared driver's default five workbooks, which left ECS and all three EKS variants untested on every platform even though their assertions already existed. It now passes an explicit nine and imports the three appliance images they need — RDS, ECS and EKS each boot their own, found by a spinifex:managed-by tag and never by name. demo-app stays out because it is not a workbook: it has no .tf at all, being the container image the EKS workbooks' nested workloads modules deploy, so listing it would fail on a missing root module rather than test anything. The guide is now two documents. oci-integration/README.md is the end-user path — six steps, deploy first, and it closes the three prerequisites nothing documented before: the clone, the ref, and the SSH keypair the driver refuses to start without. architecture.md holds the sizing, the IAM policy, every variable, the three filtering layers and the troubleshooting. Its firewall advice was also backwards. node_client_cidr_allow_list is an OCI security-list rule over the whole public subnet, and a guest's public address is a secondary private IP on the node's own VNIC, so narrowing it to the operator's address cuts internet ingress to every guest rather than protecting the node. The hardening section now arms the nft policy and narrows SSH through custom.nft, which is the layer that can tell a node's listener from a guest's. The deploy step names --channel dev rather than a pinned tag. The dev tag string grows with every cut, so a pinned example goes stale within the week. * fix(oci): the guide could not work on a fresh tenancy, because the profile name was ours oci_env.py bridges ~/.oci/config into TF_VAR_* — the provider takes the tenancy, user, fingerprint, key and region as Terraform variables and reads no config file itself — but the profile it looked for was the reference tenancy's, and validate-topology.sh passed no way to change it. A fresh account's config holds [DEFAULT], so every deploy would have stopped at "OCI profile 'apacanzset03child03' was not found" before building anything. The profile is now resolved as --oci-profile, then $OCI_CLI_PROFILE, then the reference tenancy, then DEFAULT. A name passed explicitly and absent is an error listing the profiles present, never a fallback: the failure worth preventing is building a cluster in whichever tenancy happened to be first. The reference tenancy still outranks DEFAULT, so existing runs are unchanged, and CI is untouched because it takes the environment route rather than a profile. has_profile exists because configparser treats DEFAULT as the fallback for every other section rather than as a section, so has_section denies it exists even when it is the only thing in the file — which made the first version of this fallback do nothing for exactly the case it was written for. Three documentation corrections found while checking the guide against the code. The IAM policy is ten operations and needs "use vnics", not "read vnics": registering a secondary private IP is an operation on a VNIC, so OCI checks VNIC_ASSIGN, which lives in the use verb, and with read the node forms and looks healthy and every allocation fails NotAuthorizedOrNotFound. These are now the same three statements instance-principal.tf grants, which is the only version of the policy that has been exercised. The guide also promised a worked example of the credential hook that does not exist and should not: nothing here distributes a credential. It now states the contract instead — the argument shape, the two files, and that a non-zero exit stops the deploy before anything is built. And the dev channel's rate limit is documented with the tag as its remedy, rather than left in the driver's help where a reader will not see it. * fix(ci): a failing workbook's log never reached the artifact, so only the runner had it run-tofu-examples-e2e.sh writes its assertion output to the per-topology workbooks.log in the state directory. "Sweep anything left behind" deletes those directories and runs before "Upload the run", so the upload's three oci-state/.validate-* patterns matched nothing and said so to nobody. The first nine-workbook run lost the one thing it was dispatched to find: ecs-quickstart failed on vm-single and the reason existed only on the runner, which does not resolve from a workstation. A collect step now copies *.log, *.txt and *.tsv into the artifact directory before the sweep, prefixed by topology so two topologies cannot overwrite each other. The workflow already records that "Publish the result tables" sits ahead of the sweep for this reason; the logs were simply never given the same treatment. The dead patterns are removed rather than left. A path that cannot match is worse than no path, because it reads as protection that is present — which applies to the credential-hook exclusion too: it now lives in the collect step, which is the only thing that writes into the artifact tree, so the file cannot reach an artifact rather than being filtered out of one. * fix(ci): a state directory kept the last run's logs, and the artifact published them as this run's Collecting the per-topology logs surfaced the next defect immediately: a vm-single-only run produced an artifact holding bm-workbooks.log and nine vm-multi node logs, because those state directories still held the previous run's output. Anyone reading that artifact would conclude three topologies ran when one did, which is worse than the missing log it replaced. Two causes, one each side. validate-topology.sh never cleared its own logs, so a topology's directory accumulated them; it now removes *.log, nodes.txt and cloudinit-*.txt as it starts, before the first line of the new run, and never the tfstate, which is how the sweep finds hosts to destroy. The globs are named rather than a find, so what goes is readable at a glance, and :? stops the shell instead of expanding to /*.log if STATE_DIR is ever unset. That alone does not cover a topology the run skipped, whose directory is never entered. So the collect step now takes only the topologies this run drove, read back from the per-topology log the harness writes rather than from the dispatch input -- a run that names no topology gets the harness's default, and reading that here would be a second copy of the list to disagree with the first. Also proved by this: ecs-quickstart passes on a freshly built vm-single, 172s, where it failed at 293s as the seventh of nine. The failure is order or load dependent on one 32 GB node, not a property of the workbook. * fix(ci): a failed workbook run left no node journal, so the cause was unreadable The workbook log says which API call failed; only the journal says why, and the nodes are destroyed minutes later. An IAM teardown failure on the nightly came down to one swallowed warning that existed nowhere else, and by the time the artifact was read the host it was logged on had been torn down. Warnings and above first, since that is where a swallowed error surfaces, then a bounded tail for context. Unbounded, a multi-hour suite's journal would dwarf the rest of the artifact. The existing state-directory clear keeps these from going stale between runs, and the collect step already globs *.log.
What is Spinifex? · Platform · Deploy · AWS Compatibility · Live Demo · Components · Architecture · Installation · Docs
Spinifex: the open, AWS-compatible cloud you run yourself
Spinifex, built by Mulga, recreates the AWS services your software already uses (EC2, EBS, S3, VPC, IAM) on infrastructure you control. Move workloads off the hyperscalers, run AI at a fraction of the cost, or take the cloud to the edge, without rewriting a line. The exact build you ship to AWS, Spinifex serves on your own hardware: same APIs, same Terraform, same SDKs.
What is Spinifex?
Spinifex is an open-source infrastructure platform that recreates the AWS service surface (EC2, EBS, S3, VPC, IAM) on bare-metal, edge, and on-premises environments. Run cloud-native software without a hyperscaler.
Built for teams that need:
- The AWS tooling you already use (AWS CLI, SDKs, Terraform), with nothing to rewrite
- Full control of the stack: your hardware, your network, your data, your keys
- A cloud that keeps running offline, through disconnection, with no external control plane
- An open core (AGPL-3.0), auditable and yours to keep, with no lock-in
You change where your software runs, not what it is.
The Platform
From commodity hardware up to unmodified AWS tooling, every layer is replaceable and yours to own.
Three Ways To Deploy
The same AWS-compatible platform runs in three shapes. Pick the one that matches your hardware.
Neocloud
Lift and shift onto a partner Neocloud without rewriting workloads. GPU capacity (H100 / H200 / B200), cheaper, available now. Change an endpoint, keep the software.
On-premise
Host Spinifex on your own hardware in your own data centre. You get a multi-node HA cluster integrated with your storage and networking, so the stack, the data, and the jurisdiction stay yours.
Edge
Cloud where the cloud can't reach. Air-gapped sites, vehicles, vessels, factories, and clinics. Compute next to the data, running through disconnection, on hardware you own. The same AWS APIs your software already uses.
AWS Compatibility
Spinifex speaks the AWS API surface. The AWS SDKs, AWS CLI, and Terraform all work unchanged, whether you deploy at the edge, on-premise, or on a partner Neocloud.
| Service | What it is | Status |
|---|---|---|
| EC2 | Compute | Available |
| EBS | Block Storage | Available |
| S3 | Object Storage | Available |
| VPC | Networking | Available |
| IAM | Identity & Auth | Available |
| ALB / NLB | Load Balancers | Available |
| EKS | Kubernetes | Available |
| ECR | Container Registry | Available |
| ECS | Container Service | Available |
| RDS | Databases | Available |
| Bedrock | AI Deployment | Q4 2026 |
Roadmap items ship under the same AWS API surface. Code written for AWS today keeps working the moment they land. Track what's shipped in the release notes.
Live Demo
Create a live sandbox in under a minute. The fastest way to experience the stack. Try Spinifex against one of your own workloads on infrastructure we run for you.
- Live in under a minute. No card, no install.
- Yours for 72 hours to test and experiment.
- Real AWS-compatible endpoint, access keys and console.
- EC2, EBS, S3, VPC, IAM, EKS and RDS, ready to call.
- Point your existing AWS CLI or Terraform straight at it.
Sign up at https://mulgadc.com/signup
Core Components
Spinifex (Compute and Platform Services – EC2, EKS, ECS, RDS)
Spinifex is a minimal VM orchestration layer built on top of QEMU, exposing an EC2 compatible API. It manages lifecycle operations like start, stop, and terminate through QEMU's QMP interface. Designed to be straightforward and scriptable, Spinifex lets you launch VMs with the AWS CLI, SDKs, or Terraform, without Kubernetes or a heavyweight orchestrator.
The managed services build on that same instance layer. An EKS node group, an ECS container instance, and an RDS database are Spinifex VMs on your own hardware, each driven through its own AWS API.
- EC2 – VM lifecycle on bare metal, cloud-init and IMDS metadata, key pairs, tags, spot instances, capacity reservations, and GPU passthrough
- EKS – Managed Kubernetes on k3s, with HA control planes, managed node groups, cluster add-ons, an OIDC provider, and IAM access entries
- ECS – Task and service scheduling across container instances, with awsvpc ENIs, capacity providers, rolling deployments, and GPU-aware placement
- ECR – Private container registry backed by Predastore, with content-addressed blobs deduplicated per account and lifecycle policies
- RDS – Managed PostgreSQL and MariaDB, with parameter groups, automated backups, and DNS endpoints served by Northstar
- ALB / NLB – Layer 7 load balancing on HAProxy and layer 4 on nginx
stream, with health checks and ACM-issued listener certificates - VPC and IAM – OVN-backed networking and security groups, with roles, policies, and SigV4 auth applied across every service
Viperblock (Block Storage – EBS Alternative)
Viperblock is a high-performance, WAL-backed block storage service that replicates volumes across multiple nodes. It's built for reliability and speed, with support for snapshots, recovery, and direct connection to QEMU instances using NBD or virtio-blk.
- Fast, durable virtual disks
- Replication for resilience
- Exposed over NBD or embedded in VMs
- High performance WAL logs on local NVMe drives, reducing IO traffic to S3
- In-memory read/write block cache
Predastore (Object Storage – S3-Compatible)
Predastore is a fully S3-compatible object storage system. It supports the AWS S3 API, including Signature V4 authentication, multipart uploads, and Terraform provisioning. Predastore chunks data and distributes it across nodes using Reed-Solomon erasure coding, so it survives node loss and suits large-scale or low-bandwidth sites.
- S3-compatible API and auth
- Multipart uploads, streaming reads/writes
- Data redundancy with Reed-Solomon encoding
Northstar (Authoritative DNS)
Northstar is a lightweight authoritative DNS server that handles both internal service discovery and public-facing DNS for a Spinifex cluster. Zones are human-readable TOML files, served from the local filesystem or synced from Predastore, keeping DNS configuration in the same object storage as the rest of your infrastructure.
- Authoritative DNS over UDP, TCP, and DNS-over-TLS
- Service discovery for cluster components via SRV records
- Zone files stored locally or in S3-compatible storage
Architecture at a Glance
Every AWS API call is authenticated at the gateway, published to a NATS subject, and answered by whichever daemon claims it. Daemons are stateless: scale horizontally by starting more. No etcd, no Kubernetes, no external control plane. Just systemd units, a NATS cluster, and your hardware. Deep dive: docs/DESIGN.md.
Self-hosted Installation
Installation requires an Ubuntu 26.04 or Debian 13 system. See the detailed documentation at docs.mulgadc.com for installing Spinifex.
Bare Metal ISO
The recommended installation is a bootable x86 installer for bare-metal hardware.
curl -fLO https://iso.mulgadc.com/spinifex.iso
Follow the USB install guide to write the ISO to USB and install on your hardware.
Single Node Install
Prerequisite: Spinifex requires a Linux bridge configured on the host for VM networking. See the single-node install guide for setup details.
curl -fsSL https://install.mulgadc.com | bash
sudo /usr/local/share/spinifex/setup-ovn.sh --management
sudo spx admin init --node node1 --nodes 1
sudo systemctl start spinifex.target
export AWS_PROFILE=spinifex
aws ec2 describe-instance-types
Development Setup
For a complete development environment see the Source Install documentation.
Spinifex UI
Spinifex ships with a built-in web console, an optional alternative to the AWS CLI, SDKs, and Terraform. If you know the AWS Management Console, the Spinifex UI fills the same role: a browser-based view of your instances, volumes, buckets, VPCs, and IAM resources, without leaving your own network.
Each node serves the console on port 3000 over TLS, and it comes up with spinifex.target:
- Same API, different surface. The UI issues the same AWS SigV4 calls the CLI does, so RBAC, audit trails, and IAM policies apply the same way.
- Single sign-on against your AWS credentials. Log in with the access keys from
~/.aws/credentialson the node where Spinifex is installed. There is no separate user database. - Self-hosted, works offline. The Spinifex binary embeds the UI and each node serves it directly, with no external CDN or analytics calls.
For the full walkthrough, covering first-time TLS certificate trust, login, and a feature tour, see Launching the Web UI in the cluster setup guide.
Development Philosophy
Built by Engineers, For Engineers
Spinifex is developed by experienced infrastructure engineers with deep AWS expertise, including former AWS team members who understand the intricacies of building production-grade cloud services. Our team brings decades of combined experience from AWS, enterprise infrastructure, and edge computing environments.
AI-Assisted Development
Spinifex is architected and implemented by experienced engineers, with Claude Code accelerating certain development tasks. The split keeps the quality and reliability standards that production infrastructure demands:
How We Use Claude Code:
- Code Generation: Boilerplate AWS API structures and handlers
- Documentation: Comprehensive development guides and API documentation
- Testing: Test case generation and validation scenarios
- Refactoring: Large-scale code restructuring and optimization
What Remains Human-Driven:
- Architecture Decisions: Core system design and scalability choices
- Security Implementation: Authentication, encryption, and threat modeling
- Performance Optimization: Real-world performance tuning and benchmarking
- Production Operations: Deployment strategies and operational procedures
Trademarks
"AWS", "Amazon Web Services", and all related service names (EC2, EBS, S3, VPC, IAM, and others) are trademarks of Amazon.com, Inc. or its affiliates. Mulga Defense Corporation and Spinifex are independent and not affiliated with, endorsed by, or sponsored by Amazon.com, Inc. or Amazon Web Services, Inc. References to AWS services describe interoperability and compatibility only.
License
Spinifex is open source under the GNU Affero General Public License v3.0, with a commercial license available. You're free to use, modify, and deploy it anywhere you need reliable infrastructure without depending on centralized cloud platforms.