From ecef5e4d2f793b4cf2c96d0a43d0a1ab37be3eed Mon Sep 17 00:00:00 2001 From: Julian Sommer <115516918+juliansommer@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:27:01 +1000 Subject: [PATCH] feat: CONTRIBUTING.md --- CONTRIBUTING.md | 96 +++++++++++++++++++++++++++++++++++++++++++++++++ docs/DESIGN.md | 42 ++++++++++------------ 2 files changed, 114 insertions(+), 24 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..d0503d892 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,96 @@ +# How to Contribute + +We welcome contributions to Spinifex! + +Report security vulnerabilities privately through [SECURITY.md](SECURITY.md), not as a public issue. + +**AWS parity is the design rule.** When Spinifex's behaviour is in question, the answer is what AWS does. If you believe Spinifex diverges from AWS, state what AWS does and show it with a test or a reproduction against a running cluster. + +## Understanding the Codebase + +Every AWS call enters through a SigV4-authenticated gateway, is published to NATS, and is answered by the daemon that claims it. Start with these: + +| Doc | Covers | +|---|---| +| [`docs/DESIGN.md`](docs/DESIGN.md) | Architecture, request flow, daemons, storage integration, file layout. | +| [`docs/COMMANDS.md`](docs/COMMANDS.md) | The `spx admin` CLI and the systemd units it manages. | +| [`docs/coverage/`](docs/coverage/README.md) | Which AWS API operations each service implements. | +| [`scripts/README.md`](scripts/README.md) | Install, dev-environment, and image-build scripts. | + +User-facing docs are published at [docs.mulgadc.com](https://docs.mulgadc.com). + +## Development Setup + +These steps build Spinifex from source and run it as a single-node cluster, following the [Source Install](https://docs.mulgadc.com/docs/install-source) guide. + +### Prerequisites + +- **Ubuntu 26.04** or **Debian 13**, on x86_64 or aarch64 +- A host with hardware virtualisation (KVM) and `sudo` access +- The host's WAN interface enslaved to a Linux bridge named `br-wan`, with the host IP, default route and DHCP on the bridge rather than the bare NIC (see [VPC Networking → Bridge Setup](https://docs.mulgadc.com/docs/vpc-networking#bridge-setup-physical-network-wiring)) + +Check the bridge before continuing: + +```bash +ip -br link show br-wan # bridge exists and is UP +ip route # default route's dev is br-wan +``` + +### Install and Run + +```bash +mkdir -p ~/Development/mulga && cd ~/Development/mulga +git clone https://github.com/mulgadc/spinifex.git +sudo make -C spinifex quickinstall # system packages, Go 1.27, AWS CLI v2 +export PATH=$PATH:/usr/local/go/bin + +cd spinifex +./scripts/clone-deps.sh # sibling repos: viperblock, predastore, northstar, bluebottle +./scripts/dev-install.sh # build, install, init a one-node cluster, start services + +AWS_PROFILE=spinifex aws ec2 describe-instance-types +``` + +A list of instance types means the cluster is up. + +`make preflight` also needs `shellcheck` and `golangci-lint` (CI pins v2.13.0): + +```bash +sudo apt-get install -y shellcheck +curl -sSfL https://golangci-lint.run/install.sh | sh -s -- -b "$(go env GOPATH)/bin" v2.13.0 +``` + +The web console additionally needs Node.js 24 and `pnpm`. + +## Development Workflow + +| Command | Use it to | +|---|---| +| `make deploy` | Rebuild, install `spx`, and restart `spinifex.target` after a code change. | +| `make reinstall` | Re-run `dev-install.sh` after changing systemd units, helper scripts, or logrotate config. | +| `./scripts/reset-dev-env.sh` | Wipe all cluster state and reinstall from scratch, keeping your network topology. | +| `make test` | Run unit tests (`make test-race` for the race detector). | +| `make test-integration` | Run the real gateway router against embedded NATS JetStream. | +| `make preflight` | Run the pre-commit gate: lint, `govulncheck`, coverage, and manifest and script checks. Must pass. | + +Follow the services with `journalctl -u 'spinifex-*' -f`. + +The web console lives in `spinifex/services/spinifexui/frontend/` (`pnpm install`, `pnpm dev`, `pnpm lint`, `pnpm test`). `spx` embeds its checked-in `dist/`, so run `make build-ui` and commit the rebuilt `dist/` with your change. + +## Creating a Pull Request + +1. Fork [mulgadc/spinifex](https://github.com/mulgadc/spinifex) and clone your fork into the same parent directory as the sibling repositories, branching off `dev`: + + ```bash + cd ~/Development/mulga + git clone git@github.com:YOUR_USERNAME/spinifex.git && cd spinifex + git remote add upstream git@github.com:mulgadc/spinifex.git + git fetch upstream && git checkout -b feat/my-feature upstream/dev + ``` + +2. Make your change, then verify it: + - `make preflight` passes. + - `go mod tidy` leaves only intended changes in `go.mod` and `go.sum`, with none of the `clone-deps.sh` `replace` lines. + - Runtime changes are exercised on your dev cluster with `make deploy`. + +3. Commit using [Conventional Commits](https://www.conventionalcommits.org/) (`feat:`, `fix:`, `refactor:`, `docs:`, `chore:`), push, and open a pull request against `dev`, filling in the template. diff --git a/docs/DESIGN.md b/docs/DESIGN.md index a8df09d30..02e8b2e82 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -289,31 +289,25 @@ Configs are generated by `spx admin init` (leader) or `spx admin join` (follower ``` spinifex/ +├── cmd/ +│ ├── spinifex/ # spx CLI: service entry points and `spx admin` +│ ├── installer/ # Platform installer +│ └── ... # In-guest agents (lb, ecs, rds, eks helpers) and dev tools ├── spinifex/ -│ ├── services/ # Long-running services (awsgw, spinifex, viperblockd, -│ │ # predastore, nats, spinifexui) -│ ├── gateway/ # AWS API surface -│ │ ├── gateway.go # chi router, SigV4 auth, throttling -│ │ ├── auth.go # SigV4 verification + IAM lookup -│ │ ├── ec2.go # EC2 action dispatcher -│ │ ├── ec2/ # Per-resource handlers (instance, volume, vpc, ...) -│ │ ├── elbv2/, iam/, policy/ -│ ├── handlers/ # NATS-side service implementations (ec2, elbv2, iam, ...) -│ ├── daemon/ -│ │ ├── daemon.go # Daemon entry, NATS subscription table -│ │ └── daemon_handlers_*.go # One file per resource family -│ ├── admin/ # Cluster bootstrap (init/join, master.key, CA, tokens) -│ ├── config/ # Configuration structures -│ ├── instancetypes/ # t3.*, m5.*, sys.* definitions -│ ├── lbagent/ # In-VM agent for ELB target health -│ ├── vm/ # VM instance representation -│ ├── vpcd/ # VPC daemon (OVN/OVS network programming) -│ └── qmp/ # QEMU Machine Protocol client -└── cmd/ - ├── spinifex/ # spx CLI - ├── lb-agent/ # ELB target agent binary - ├── installer/ # Platform installer - └── ... # ECS agent, ECR credential provider, EKS helper binaries +│ ├── services/ # Long-running services (awsgw, spinifex, nats, predastore, ...) +│ ├── gateway/ # AWS API surface: routing, SigV4 auth, per-service dispatch +│ ├── handlers/ # NATS-side service implementations (ec2, iam, eks, rds, ...) +│ ├── daemon/ # Per-node daemon and its NATS subscription table +│ ├── vm/, qmp/ # VM lifecycle and the QEMU Machine Protocol client +│ ├── vpcd/, network/ # VPC daemon and the layered OVN/OVS network stack +│ ├── admin/ # Cluster bootstrap (init/join, master.key, CA, tokens) +│ └── config/ # Configuration structures +├── internal/ # In-guest SigV4 clients and the AWS Smithy models +├── build/ # systemd units, node helper scripts, microVM build +├── scripts/ # Install, dev-environment, image-build scripts +└── tests/ + ├── integration/ # Gateway router against embedded NATS JetStream + └── e2e/ # Live-cluster suites (see docs/testing/e2e-coverage.md) ``` ## Development