Compare commits

..

327 Commits

Author SHA1 Message Date
Julian Sommer ea8700df99 fix: increase sleep time 2026-03-19 15:02:50 +10:00
Julian Sommer 857cbd75ac fix: use spx service stop instead of shell PID kill in stop-dev.sh 2026-03-19 14:50:50 +10:00
Julian Sommer 0fa38f7986 fix: write credentials to user even when running as sudo 2026-03-19 14:05:48 +10:00
Julian Sommer 1a4d590bf3 fix: nbd log level 2026-03-19 13:45:27 +10:00
Julian Sommer b7678ab7c5 feat: fix silent failures across spinifex (P2 audit H-25 through H-43)
- PID file failures now prevent service start instead of logging and continuing
- Status() methods use PID-based health checks instead of returning empty string
- NewResourceManager returns error instead of silently defaulting to 8GB
- Crash recovery re-allocates resources before relaunch
- restoreInstances stops after failed resource allocation
- Cluster manager bind errors propagated to Start()
- NATS disconnect/reconnect logged at Warn level
- VPC CIDR fallback logs warning about overbroad rule
- vpcd respond() handles json.Marshal failure
- Network cleanup failures logged instead of silently discarded
- fmt.Println replaced with slog.Debug in NATS service
- Signature mismatch no longer logs expected signature
- extractDiskImagePath uses errors.Is for EOF
- validateDiskImagePath checks file command error
- os.Stat uncommented in admin image import
- net.SplitHostPort errors checked
- Gateway response write errors logged
- vpcd OVN NB address uses leader IP on non-primary nodes
- Release workflow parallelized with native arm64 runner
2026-03-19 13:37:41 +10:00
Julian Sommer 11730be685 fix: locate release assets by filename instead of hardcoded path 2026-03-19 12:13:28 +10:00
Julian Sommer a4bc8bee74 fix: pending instance terminate (#80) 2026-03-19 12:47:00 +11:00
Julian Sommer fafc104c30 feat: add release verification job with draft-then-promote flow 2026-03-19 10:52:39 +10:00
dependabot[bot] efa6257850 build(deps): bump google.golang.org/grpc from 1.79.1 to 1.79.3 (#79) 2026-03-19 11:35:06 +11:00
Julian Sommer 7e46f0012b chore: bump more action versions 2026-03-19 10:23:52 +10:00
Julian Sommer 58e6817efe chore: bump docker build action ver 2026-03-19 10:19:02 +10:00
Julian Sommer 71ec2b7ee5 release: update go.mod for v0.4.0 2026-03-19 08:28:16 +10:00
Julian Sommer 9c430eea07 fix: release workflow 2026-03-19 08:17:30 +10:00
Julian Sommer 1be6c69e51 format 2026-03-19 08:03:23 +10:00
Ben Duncan c523f78403 Merge branch 'dev' of github.com:mulgadc/spinifex into dev 2026-03-18 05:08:30 +00:00
Ben Duncan 2918a02fac correct spx path 2026-03-18 05:08:15 +00:00
Julian Sommer 8e3210f023 fix 2026-03-18 14:36:12 +10:00
Julian Sommer 6ba9d3dbf0 remove trailing slash 2026-03-18 14:24:52 +10:00
Ben Duncan 27cdfc4653 Merge branch 'dev' of github.com:mulgadc/spinifex into dev 2026-03-18 04:14:22 +00:00
Ben Duncan adadb5ef4c Fixed noninteractive error on apt get 2026-03-18 04:13:53 +00:00
Julian Sommer d0fc2a2120 last fix 2026-03-18 13:40:27 +10:00
Julian Sommer a09a8efe3c build frontend with new branding 2026-03-18 13:38:47 +10:00
Julian Sommer 7d43d0f782 hopefully last rename 2026-03-18 13:24:45 +10:00
Julian Sommer ba0d8b67ff feat: e2e pipeline speedup (#78) 2026-03-18 13:50:45 +11:00
Ben Duncan 454f381826 Moved to GNU Affero General Public License v3 2026-03-18 02:04:41 +00:00
Ben Duncan b95ff42a2f feat: rename product from Hive to Spinifex
Trademark conflict — bulk rename across entire codebase:
- Go module: github.com/mulgadc/hive → github.com/mulgadc/spinifex
- Binary: hive → spx
- Directories: hive/ → spinifex/, hiveui → spinifexui
- Env vars: HIVE_* → SPINIFEX_*
- Config: hive.toml → spinifex.toml
- Systemd: hive-* → spinifex-*
- NATS: hive-workers → spinifex-workers
- All paths, docs, scripts, CI workflows updated
2026-03-18 01:42:53 +00:00
Ben Duncan 2064c338b3 chore: gofmt formatting after rename 2026-03-18 01:23:11 +00:00
Ben Duncan 4f00cf9e4c refactor: update Dockerfile and build artifacts for spinifex rename
- Dockerfile.distro: update all paths from hive → spinifex
- Binary output name: spx
- Logrotate: logrotate-spinifex
2026-03-18 01:20:27 +00:00
Ben Duncan 48012a4058 refactor: rename all documentation from hive to spinifex
- Rename .claude/rules/hive.md → spinifex.md
- Update CLAUDE.md component description
- Rename doc directories: hive-admin-cli → spinifex-admin-cli, etc.
- Rename doc files: hive-cluster-stats, hive-code-consolidation, etc.
- Bulk update all markdown files across docs/ tree
- Update internal docs: CLUSTER-INSTALL, TESTING, DEVNOTES, BRANCHES
2026-03-18 01:19:05 +00:00
Ben Duncan 55186d95f2 refactor: rename CI/CD workflows from hive to spinifex
- Rename hive.yml → spinifex.yml
- Update workflow names, path triggers, working directories
- Update release.yml tarball references
- Update e2e.yml test paths
- Update ui.yml frontend paths
2026-03-18 01:09:02 +00:00
Ben Duncan c11ac04892 refactor: rename frontend UI branding from hive to spinifex
- package.json: name → spinifex-ui
- index.html: page title → Spinifex Control Panel
- sidebar-layout: Mulga Hive → Mulga Spinifex
- auth.ts: localStorage key → spinifex:v1:aws-credentials
- main.tsx: theme storage key → spinifex-ui-theme
- spinifexui_test.go: update service name assertion
2026-03-18 01:06:55 +00:00
Ben Duncan 9e744e5be6 refactor: rename NATS queue group and topics from hive to spinifex
- hive-workers → spinifex-workers queue group (~16 files)
- hive.admin.* → spinifex.admin.* topics
- hive.nodes.* → spinifex.nodes.* topics
- hive.node.* → spinifex.node.* topics
- hive.cluster.* → spinifex.cluster.* topics
- spinifex-nats server_name already updated in Phase 5
2026-03-18 01:06:03 +00:00
Ben Duncan 5d93786995 refactor: rename hive to spinifex in all shell scripts
- Update scripts/: start-dev, stop-dev, dev-setup, setup, setup-ovn, etc.
- Update tests/e2e/: all E2E test scripts
- Update scripts/iac/: rename hive-test.sh → spinifex-test.sh, update TF files
- Update config/: rename hive.service and install script
- Binary refs use spx, env vars use SPINIFEX_, paths use spinifex
2026-03-18 01:04:12 +00:00
Ben Duncan b97a98e2b7 refactor: rename systemd units and logrotate from hive to spinifex
- Rename all build/systemd/hive-* → spinifex-*
- Rename build/logrotate/hive → spinifex
- Update unit contents: descriptions, dependencies, user/group, paths
- Binary path set to /usr/local/bin/spx
2026-03-18 00:53:34 +00:00
Ben Duncan ee0b92c192 refactor: rename config files, paths, and env vars from hive to spinifex
- Rename hive.toml template → spinifex.toml
- Update all default paths: /etc/hive → /etc/spinifex, ~/hive → ~/spinifex
- Rename all HIVE_* env vars → SPINIFEX_* (SetEnvPrefix, BindEnv calls)
- Update nats.conf template server_name
- Update admin.go init/join commands for new paths and names
- Update paths.go, paths_test.go, config.go, config_test.go
2026-03-18 00:47:01 +00:00
Ben Duncan 00f5b6590e refactor: rename CLI binary to spx and update all command names
- Makefile: GO_PROJECT_NAME → spx, update build/test/dist paths
- root.go: Use → spx, update branding to Spinifex
- service.go: hive/hive-ui commands → spinifex/spinifex-ui
- Service name constants: hive → spinifex, hive-ui → spinifex-ui
- Update service_test.go switch cases
2026-03-18 00:42:34 +00:00
Ben Duncan 06804f5775 refactor: rename directory structure hive → spinifex
- git mv hive/ → spinifex/
- git mv cmd/hive/ → cmd/spinifex/
- git mv services/hive/ → services/spinifex/
- git mv services/hiveui/ → services/spinifexui/
- Update package declarations and all internal import paths
- Update service.go switch cases for new service names
2026-03-18 00:37:30 +00:00
Ben Duncan 767ad0bf5c refactor: rename Go module path from hive to spinifex
Bulk rename github.com/mulgadc/hive → github.com/mulgadc/spinifex
across go.mod and all 178 Go files (~411 import occurrences).
2026-03-18 00:29:24 +00:00
Julian Sommer 4152e4422f chore: remove redundant x/net/http2 and automaxprocs deps
ForceAttemptHTTP2 + NextProtos ALPN already handle HTTP/2 on Go 1.22+.
Go 1.25+ runtime is container-aware for GOMAXPROCS by default.
2026-03-18 10:14:52 +10:00
Julian Sommer 48bbe423df fix: complete termination lifecycle for failed instances 2026-03-18 09:53:48 +10:00
Julian Sommer e6b5b3b135 chore: update vite react plugin 2026-03-18 08:54:57 +10:00
Julian Sommer f526972607 chore: go back to ultracite 2026-03-18 08:46:49 +10:00
Julian Sommer ddcd7f622f chore: bump deps 2026-03-18 08:20:18 +10:00
Ben Duncan 2459e5f7ac improved docs 2026-03-16 12:52:14 +00:00
Ben Duncan 120bfa8953 fix: generate go.work in Dockerfile instead of copying gitignored file 2026-03-16 12:36:56 +00:00
Ben Duncan d8d5fd7d2c New doc links 2026-03-16 12:32:44 +00:00
Ben Duncan b857c7982e Merge remote-tracking branch 'origin/dev' 2026-03-16 12:17:57 +00:00
Ben Duncan 965e65df3b Added test coverage 2026-03-16 12:14:46 +00:00
Ben Duncan 4c4683ba0c installer tweaks for sudo 2026-03-16 12:04:06 +00:00
Ben Duncan a458a28406 new binary mode and systemd process 2026-03-16 11:57:52 +00:00
Ben Duncan 2351d4ff81 New doc templates 2026-03-16 22:47:08 +11:00
Julian Sommer e7a3b5b712 fix: update test for explicit error handling 2026-03-13 16:07:49 +10:00
Julian Sommer 36f820c859 fix: error handling (#76) 2026-03-13 17:01:00 +11:00
Julian Sommer c60a691ac1 fix: start/stop race condition 2026-03-13 05:27:28 +00:00
Julian Sommer 21fd5a7081 fix: IAM replication 2026-03-13 04:33:19 +00:00
Julian Sommer 2b455e365e fix: dropped state 2026-03-13 03:53:38 +00:00
Julian Sommer e6a09d371c fix: multi node e2e retry 2026-03-13 03:37:41 +00:00
Julian Sommer 6f3f348287 chore: bump vite plugin react 2026-03-13 02:52:08 +00:00
Julian Sommer 8e8b17c599 chore: migrate to vite8 2026-03-13 02:39:28 +00:00
Julian Sommer 72aa3e7563 feat: migrate to oxlint (#75) 2026-03-13 13:24:01 +11:00
Julian Sommer 096b3380fc feat: default vpc account scoping (#74) 2026-03-13 13:18:58 +11:00
Julian Sommer 663805cd23 fix: race conditions on node failure 2026-03-13 01:53:00 +00:00
Julian Sommer 380ab2a870 fix: replicate all js buckets 2026-03-13 10:48:00 +10:00
Julian Sommer 98c75cbb83 fix: stop-dev.sh 2026-03-13 09:17:41 +10:00
Julian Sommer 3ffe67271f feat: multi node e2e (#73) 2026-03-13 10:11:26 +11:00
Julian Sommer 4466de04d3 fix: ovn logrotate 2026-03-12 08:24:52 +10:00
Julian Sommer fd91b80851 fix: hardcoded predastore nats url 2026-03-11 13:57:49 +10:00
Julian Sommer 0853eb5068 fix: return error from GenerateNATSToken instead of panicking
Same pattern as H-5 — crypto/rand failure caused a panic. Now returns
(string, error) so callers can handle it gracefully.
2026-03-11 12:08:44 +10:00
Julian Sommer 3d8c9f3812 fix: hive panics (#71) 2026-03-11 13:02:31 +11:00
Julian Sommer 2cac5e2c37 fix: cors 2026-03-11 11:07:44 +10:00
Julian Sommer 0ec13f33eb fix: lint 2026-03-11 10:58:03 +10:00
Julian Sommer 7af16b75b9 feat: 80% real frontend coverage 2026-03-11 10:56:42 +10:00
Julian Sommer 40e09f0886 feat: enforce frontend code coverage 2026-03-11 10:25:04 +10:00
Julian Sommer e26cd7fd82 feat: ui tests 2026-03-11 10:14:31 +10:00
Julian Sommer 8f223d35d3 feat: setup vitest 2026-03-11 09:47:46 +10:00
Julian Sommer b0dbc76b22 chore: frontend deps 2026-03-11 09:32:46 +10:00
Julian Sommer 506bced4fa fix: e2e single 2026-03-11 08:43:05 +10:00
Ben Duncan 438ec3c973 Merge branch 'dev' of github.com:mulgadc/hive into dev 2026-03-10 22:13:11 +10:00
Ben Duncan 11a1f18cb0 removed restrict on hostfwd, vpc issue to ssh in 2026-03-10 22:13:00 +10:00
Julian Sommer 1ff08ab80a fix: seperate system account (#70) 2026-03-10 16:30:34 +11:00
Ben Duncan d17eef2ba9 Merge branch 'dev' of github.com:mulgadc/hive into dev 2026-03-10 10:58:04 +10:00
Ben Duncan b1059899a2 dev networking, remote dhcp, due to double route bug 2026-03-10 10:57:55 +10:00
Julian Sommer 17da8b2eb2 fix: update INSTALL.md 2026-03-10 10:49:29 +10:00
Julian Sommer ee3df227a7 feat: reboot instances (#68) 2026-03-10 11:48:58 +11:00
Julian Sommer 015dbbc272 feat: frontend performance 2026-03-10 10:31:59 +10:00
Julian Sommer 404b3d6246 release: update go.mod for v0.2.3 2026-03-10 09:36:58 +10:00
Julian Sommer 021b0982f4 fix: bump go 1.26.1 references and fix gosec G104 in hiveui 2026-03-10 08:33:04 +10:00
Julian Sommer 7b590957e1 fix: bump go 1.26.1 2026-03-10 08:19:38 +10:00
Julian Sommer c9c3cda651 fix: redirect HTTP to HTTPS on same port for hive-ui
Uses a TLS-detecting listener that peeks the first byte of each
connection. TLS handshakes (0x16) proceed normally; plain HTTP
requests get a 301 redirect to the HTTPS URL. This handles
http://localhost:3000 gracefully instead of showing a cryptic error.
2026-03-10 08:14:19 +10:00
Julian Sommer 1efb4c4a3b fix: config 2026-03-06 14:51:21 +10:00
Julian Sommer 05cb4ac108 feat: frontend refactors (#66) 2026-03-06 15:42:01 +11:00
Julian Sommer 6eac4d6598 feat: IAM Frontend (#65) 2026-03-06 15:14:50 +11:00
Julian Sommer f03a2314c6 feat: terminated instances expiry (#63) 2026-03-06 12:51:27 +11:00
Julian Sommer 2c444b1eba chore: install tsgo 2026-03-06 11:46:21 +10:00
Julian Sommer aea2e2c05a ci: setup go work after 2026-03-06 10:41:45 +10:00
Julian Sommer d90bf53eaf fix: log rotate ovn 2026-03-06 10:10:34 +10:00
Julian Sommer 14c52366da fix: decouple gateway from qmp 2026-03-06 10:06:59 +10:00
Julian Sommer 6737c42601 refactor: consolidate NATS/API types into types/ package (phase 1)
Move EBS, cluster, and node message types from config/ to types/.
Remove duplicate NodeDiscoverResponse from gateway and daemon.
Delete dead EC2StartInstances types and tests from config.
Update all imports across source and test files.
2026-03-06 09:56:04 +10:00
Julian Sommer 19988af768 fix: gofmt formatting 2026-03-06 09:48:16 +10:00
Julian Sommer d20fcd2c38 fix merge 2026-03-06 09:45:34 +10:00
Julian Sommer 1c6950b89e release: update go.mod for v0.2.2 2026-03-06 09:45:34 +10:00
Julian Sommer 94ec56bb59 fix: extract shared type 2026-03-06 09:45:34 +10:00
Julian Sommer 0b2e613c6c fix: diff-coverage.sh 2026-03-06 08:22:30 +10:00
Julian Sommer 2d304528b8 chore: bump deps 2026-03-06 08:15:48 +10:00
Julian Sommer 35f2104325 feat: more tests 2026-03-05 15:09:24 +10:00
Julian Sommer c4027a0380 test: add gateway core routing, IAM auth, and handler test coverage
Squash of dev commits: gateway core routing/policy/middleware tests,
IAM auth and handler tests, awserrors tests, respondWithJSON helper,
utils refactor into focused files, and daemon test speedups.
2026-03-05 14:20:00 +10:00
Julian Sommer 77c6d05042 refactor: replace NYTimes/gziphandler with stdlib compress/gzip
Drop the external gziphandler dependency in hiveui. The replacement
uses a simple http.ResponseWriter wrapper that checks Content-Type
against an allowlist before enabling gzip compression. Same behavior,
zero external deps.
2026-03-05 12:53:46 +10:00
Julian Sommer 8af3d2e5fa feat: migrate off fiber (#62) 2026-03-05 12:53:46 +10:00
Julian Sommer 49c5807ef1 refactor: accounts 2026-03-05 11:41:51 +10:00
Julian Sommer 5fad038b87 feat: IAM account scoping (#61) 2026-03-05 09:06:35 +10:00
Julian Sommer 48c0565d47 fix: ovn shut down on stop-dev.sh 2026-03-03 15:12:44 +10:00
Julian Sommer 371193a832 feat: IAM v1 (#59) 2026-03-03 15:23:19 +11:00
Julian Sommer 9d66ef8233 fix: sanitize error codes to prevent leaking internal errors to clients
Add awserrors.ValidErrorCode() to validate error strings against
ErrorLookup before sending to clients. Replace hardcoded
"ValidationError" strings in utils.go with awserrors constants.
2026-02-27 08:49:18 +10:00
Julian Sommer 992111e809 fix: race condition + frontend refetching 2026-02-27 08:14:50 +10:00
Julian Sommer 12c2e2789e fix: skip diff coverage when instrumentable lines below minimum
The MIN_LOC=100 threshold checked total diff lines, not instrumentable
lines. A commit with many non-instrumentable lines (structs, comments)
but only 1-2 instrumentable lines would unfairly fail at 0% coverage.
Add a minimum instrumentable lines threshold (5) so the check skips
when the sample size is too small to be meaningful.
2026-02-26 11:35:22 +10:00
Julian Sommer 3720a5ba90 feat: blockdevicemappings fix (#55) 2026-02-26 11:47:37 +11:00
Julian Sommer de0e049b16 fix: e2e console output 2026-02-26 08:26:34 +10:00
Julian Sommer 6e3fe148af feat: frontend features 2026-02-26 08:22:55 +10:00
Julian Sommer 81b990a238 fix: e2e verify errored instance didnt restart on launch 2026-02-25 11:33:44 +10:00
Julian Sommer 52f2498699 fix: proper recovery checks e2e 2026-02-25 11:16:07 +10:00
Julian Sommer 61e8eb9d91 fix: reset LaunchTime on instance relaunch to prevent watchdog termination
During cluster restart, restoreInstances resets running instances (with
dead QEMU) to pending for relaunch, but didn't reset LaunchTime. The
pending watchdog (5min timeout) saw the stale LaunchTime from the
original launch and immediately marked instances as failed/terminated,
preventing any relaunch from succeeding.
2026-02-25 10:48:16 +10:00
Julian Sommer 1453beca6b Merge remote-tracking branch 'origin/main' into dev 2026-02-25 10:45:45 +10:00
Julian Sommer d0056ee925 fix: daemon-level retry with backoff for NATS/JetStream startup (#52) 2026-02-25 11:43:10 +11:00
Julian Sommer 55486490aa fix: kv stream recovery (#54) 2026-02-25 11:41:28 +11:00
Julian Sommer b370f6336e docs 2026-02-25 10:01:12 +10:00
Julian Sommer 1910347cce fix: make building on start-dev.sh opt in 2026-02-25 09:56:13 +10:00
dependabot[bot] e08b8140a6 chore: bump github.com/gofiber/fiber/v2 from 2.52.11 to 2.52.12 (#51) 2026-02-25 10:36:09 +11:00
dependabot[bot] a7e0cb6972 chore: bump github.com/nats-io/nats-server/v2 (#50) 2026-02-25 10:35:49 +11:00
Julian Sommer 6b9fc7b600 refactor: daemon 2026-02-25 08:22:46 +10:00
Julian Sommer c2e66e1ab4 refactor: extract respondWithError; fix nil panic in RunInstance
Extract duplicated respondWithError closures into a single package-level
function. Guard against nil ImageId in handleEC2RunInstances to prevent
panic under race conditions. Fix diff-coverage base ref for main branch.
2026-02-24 15:02:57 +10:00
Julian Sommer deae613a75 refactor: split daemon_handlers.go into per-domain files
Split the monolithic daemon_handlers.go into focused files:
  - daemon_handlers_account.go
  - daemon_handlers_console.go
  - daemon_handlers_igw.go
  - daemon_handlers_image.go
  - daemon_handlers_instance.go
  - daemon_handlers_key.go
  - daemon_handlers_snapshot.go
  - daemon_handlers_tag.go
  - daemon_handlers_volume.go
  - daemon_handlers_vpc.go

Add tests for ModifyVolume success, TerminateStoppedInstance with volumes,
DescribeInstanceTypes capacity filter, StartStoppedInstance type-not-available,
AttachVolume/DetachVolume validation paths, RunInstances error paths, and
DescribeInstances malformed ID.

Fix diff-coverage.sh to use git rename/copy detection (-M -C) so file
splits are not incorrectly flagged as all-new code.
2026-02-24 14:28:24 +10:00
Julian Sommer 8188707c91 feat: VPC networking v1 — OVN-based virtual networking for EC2 instances 2026-02-24 14:13:57 +10:00
Julian Sommer b9c528ce4c fix frontend vuln 2026-02-24 11:56:17 +10:00
Julian Sommer 5992d3f056 test: general test coverage improvements across hive packages
Coverage improvements:
- awsec2query: parser byte slice, invalid input, locationName fallback (73% → 83%)
- config: EC2StartInstancesResponse.Respond NATS round-trip, NodeBaseDir, ShardWAL
- daemon: 14 NATS delegate handlers, device allocation, canAllocate (count=0 boundary),
  resource management, shutdown ACK/progress, JetStream WriteServiceManifest,
  fix smallestAllocType to score by both vCPU and memory
- gateway: GenerateEC2ErrorResponse XML validation (fixed no-op assertion),
  DiscoverActiveNodes (5 scenarios), ParseAWSQueryArgs, ParseArgsToStruct (41.6% → 52.1%)
- utils: ConnectNATS (success, token, wrong token, bad address), NATSRequest
- vm: ResetNodeLocalState, PIDFile/QMP, NoGraphic, Serial+Console partial configs,
  NetDevs, ARM64, KVM, full config (68.1% → 91.7%)
2026-02-20 15:09:33 +10:00
Julian Sommer 6bc36455fc test coverage improvements: instance, hiveui, health, utils
Total hive coverage: 54.3% → 55.8% (+1.5%)

- instance/service_impl.go: new test file (generateHostname, RunInstance,
  cloud-init template, constructor)
- hiveui.go: new test file (security headers, gzip middleware, New(),
  Status(), Reload())
- daemon/health.go: extend existing tests (handleInstanceCrash core flow,
  first crash time, unknown instance type, maybeRestart scheduling)
- utils/utils.go: extend existing tests (pidPath all branches, RuntimeDir,
  extractDiskImagePath, WritePidFile error path)
- fix: use smallestAllocType in health tests to avoid CI resource failures
2026-02-20 13:46:31 +10:00
Julian Sommer db45c028be test: e2e + unit coverage for ModifyInstanceAttribute and key service
- Add Phase 7b to single-node E2E: stop instance, modify type (nano →
  xlarge), start, SSH in, verify vCPU count (nproc) and memory
  (/proc/meminfo) match the new instance type
- Add Phase 8q negative test: modify-instance-attribute on running
  instance expects InvalidInstanceID.NotFound
- Improve key service integration test coverage (mulga-580)
- Pin viperblock/predastore CI checkout to main (fixes feature branch CI)
- Update TEST_COVERAGE.md
2026-02-20 12:01:03 +10:00
Julian Sommer ff6ca8a057 feat: modify-instance-attribute (#48) 2026-02-20 12:19:20 +11:00
Julian Sommer df0e8a6459 ci: add race detection, fix test race conditions 2026-02-20 09:40:02 +10:00
Julian Sommer 3dc3d41d79 ci: enforce code coverage 2026-02-20 08:58:44 +10:00
Julian Sommer eba702b1f2 test: add 10 viperblockd coverage tests, fix daemon test sleep
viperblockd: add snapshot handler, sync/unmount/delete with VB instance,
invalid JSON, dual-publish, generic topic routing, and auxiliary volume
suffix tests. Coverage 43.1% → 53.6%.

daemon: write clean shutdown marker in restore tests to skip the 3s
crash-recovery sleep (no QEMU runs in tests). Suite 36s → 6s.
2026-02-20 08:42:30 +10:00
Julian Sommer e6ffecafca gofix: exclude stringsbuilder analyzer from modernize checks 2026-02-19 15:15:24 +10:00
Julian Sommer 27a27f660a feat: add go fix check in ci 2026-02-19 14:43:59 +10:00
Julian Sommer b4dcd3ead9 gofix: modernize codebase with go fix analyzers
Apply safe modernizations across the codebase:
- minmax: replace if/else clamping with min/max builtins
- fmtappendf: replace []byte(fmt.Sprintf) with fmt.Appendf
- forvar: remove redundant loop variable re-declarations
- inline: rename reflect.Ptr to reflect.Pointer
- mapsloop: replace explicit map loops with maps package
- omitzero: remove ineffective omitempty on time.Time fields
- rangeint: replace 3-clause for loops with range-over-int
- slicescontains: replace manual loops with slices.Contains
2026-02-19 14:40:16 +10:00
Julian Sommer 43acb57b9f feat: add e2e tests for serial console access and error compliance 2026-02-19 14:23:55 +10:00
Julian Sommer 17c011e9db fix 2026-02-19 13:17:20 +10:00
Julian Sommer a330e35d5f feat: account settings console access (#47) 2026-02-19 13:05:11 +10:00
Julian Sommer a5ff1ca5a7 feat: more awserrors (#46) 2026-02-19 12:15:45 +11:00
Julian Sommer 8573800914 fix: move instances with unavailable type to stopped on recovery (#45) 2026-02-19 10:31:13 +11:00
Julian Sommer cefbcea485 fix: README.md 2026-02-19 08:30:25 +10:00
Julian Sommer 1bf4a9cfab fix: ci branches 2026-02-18 15:05:28 +10:00
Julian Sommer 09a1e142eb release: update go.mod for v0.2.1 2026-02-18 15:01:36 +10:00
Julian Sommer ca14ecf2bc fix: port splitting (#44) 2026-02-18 15:20:40 +11:00
Julian Sommer 0c65d75371 feat: add coordinated cluster shutdown to multi-node E2E tests (#42) 2026-02-18 13:22:12 +11:00
Julian Sommer b5d2c22e68 chore: remove legacy security alias from Makefile (#43) 2026-02-18 12:23:22 +11:00
Julian Sommer 135f02c386 feat: make hive test use dev versions 2026-02-18 09:36:25 +10:00
Ben Duncan 4986e3fb87 fix: disable sharded WAL by default, bump viperblock v1.0.14
Sharded WAL causes incomplete write errors in E2E tests. Disable by
default in hive.toml template, service.go fallback, and viperblock
constructor. Can be re-enabled via hive.toml shardwal=true.
2026-02-18 07:08:11 +10:00
Ben Duncan 20e67051b0 feat: EC2 GetConsoleOutput via QEMU socket chardev + logfile capture
Replace -serial pty with socket chardev + logfile for serial output:
- Console log written to {RuntimeDir}/console-{instanceID}.log
- Serial socket at {RuntimeDir}/serial-{instanceID}.sock for admin access
- Eliminates PTS extraction deadlock (mulga-lfo) — no more ptsChan
- GetConsoleOutput API handler reads log, returns base64-encoded output
- Per-instance NATS topic ec2.{instanceID}.GetConsoleOutput (no fan-out)
- Admin debugging via: socat -,raw,echo=0 UNIX-CONNECT:serial-*.sock
2026-02-18 06:25:43 +10:00
Ben Duncan 5f370ab75e fix: bump viperblock v1.0.13, add local Docker E2E test docs
Update viperblock dependency to v1.0.13 which fixes CreateSnapshot
skipping WAL flush/consolidate when the VB instance doesn't own the
WAL files (viperblockd snapshot scenario).

Add local Docker E2E test instructions to TESTING.md.
2026-02-18 06:25:20 +10:00
Ben Duncan a1a801b26a fix: bump viperblock v1.0.12 — shard WAL snapshot corruption fix
Viperblock v1.0.12 fixes WriteShardedWALToChunk corrupting NBD plugin's
active WAL files when called from a VB instance that doesn't own the WAL
(viperblockd snapshot path). This caused CreateImage to fail with
"checksum mismatch in shard 0" on the second snapshot.
2026-02-17 22:57:36 +10:00
Ben Duncan c2265920eb fix: stop-dev.sh incorrectly detects single-node as multi-node in Docker E2E
is_multinode() counted all [nodes.*] TOML subsections (daemon, nats, awsgw,
etc.) instead of just top-level node entries. A single-node config has ~8
such sections, causing the single-node Docker E2E to use coordinated cluster
shutdown and exit with code 254.

Fix grep to match only [nodes.X] (not [nodes.X.Y]) and preserve the original
test exit code in the cleanup trap.
2026-02-17 22:15:49 +10:00
Ben Duncan bb1fe5e485 feat: add configurable shard WAL toggle, bump viperblock v1.0.11
Add [viperblock] section to hive.toml with shardwal config (default
true). The setting flows from config through viperblockd to nbdkit
plugin args, allowing operators to disable sharded WAL if needed.

Bumps viperblock dependency from v1.0.10 to v1.0.11 which includes
the sharded WAL implementation (UseShardedWAL/OpenShardedWAL), fixing
CI build failures.
2026-02-17 21:48:44 +10:00
Ben Duncan 1b431b29f7 Merge branch 'main' of github.com:mulgadc/hive 2026-02-17 16:14:49 +10:00
Ben Duncan c645fc46b1 fix: dispatch chunk WAL opening for EFI and cloud-init volumes
With UseShardedWAL now defaulting to true, the EFI and cloud-init
volume preparation must open sharded WAL files instead of always
opening the legacy WAL. Without this, Flush() dispatches to the
sharded path but shard DB files are nil, causing zero-byte writes.
2026-02-17 16:14:00 +10:00
Julian Sommer 83b1435c99 fix: refactor instance type detection (#41) 2026-02-17 15:54:45 +11:00
Julian Sommer 7128a22fdb feat: detect CPU generation instead of vendor for instance types (#40) 2026-02-17 14:59:29 +11:00
Julian Sommer b7396b97c0 fix: use available instance types in allocation test to prevent flaky CI
The FilterUnavailableTypesAfterAllocation test picked a random 2-vCPU
type from the raw instance type map, which could include memory-hungry
types (e.g. r5a.large at 16 GB) that exceed host RAM on CI runners
(~15.6 GB). Now picks from the already-filtered available types list.
2026-02-17 13:30:44 +10:00
Julian Sommer 1735a46f11 fix: e2e 2026-02-17 12:32:40 +10:00
Julian Sommer 8d9da69abb fix: e2e 2026-02-17 11:52:31 +10:00
Julian Sommer c6b89e873a feat: more testing 2026-02-17 11:47:56 +10:00
Ben Duncan 88ee6e3171 merge: resolve conflicts, accept remote changes 2026-02-17 11:43:38 +10:00
Julian Sommer dad0df0e33 fix: remove .vscode 2026-02-17 10:50:24 +10:00
Julian Sommer aeb802448b feat: more instance types 2026-02-17 10:39:59 +10:00
Julian Sommer b648fe7390 fix: merge 2026-02-17 10:32:35 +10:00
Ben Duncan 0e47a376c7 merge 2026-02-17 10:29:29 +10:00
Ben Duncan 81bf36bfc1 Moved .claude to internal repo for development 2026-02-17 10:28:21 +10:00
Ben Duncan 93c5b09ac5 Merge branch 'main' of github.com:mulgadc/hive 2026-02-17 09:42:48 +10:00
Ben Duncan dd7bce20e6 Move internal dev docs to private repo, trim CLAUDE.md for public use
Internal development plans, roadmap, and detailed dev process docs moved
to private mulga repo. CLAUDE.md trimmed to remove SSH credentials, node
IPs, and internal deployment workflows. Gosec fixes for file permissions
and unhandled error.
2026-02-17 09:38:28 +10:00
Julian Sommer 2e9ede516a fix: lint errors 2026-02-17 09:37:09 +10:00
Ben Duncan 244cb69715 Merge branch 'main' of github.com:mulgadc/hive 2026-02-17 09:11:37 +10:00
Ben Duncan bb533dfbb6 multi-server tweaks, ec2 health, IAC templates, misc cleanup 2026-02-17 09:10:13 +10:00
Julian Sommer dae9d1bb42 fix: add t3a family + fix ci 2026-02-17 08:56:06 +10:00
Julian Sommer e4db948910 feat: O(1) snapshot volume checks (#38) 2026-02-17 09:23:02 +11:00
Ben Duncan 962d85edf2 Improved IAC automation testing 2026-02-14 19:27:13 +10:00
Ben Duncan 24924be941 Cluster heartbeat support, improved cluster start/stop, coordinated shutdown in multi-server mode 2026-02-14 16:59:32 +10:00
Ben Duncan c5e561abbf Improved cluster stats and INSTALL.md doc 2026-02-14 16:19:25 +10:00
Ben Duncan 774655eff0 hive top, hive get cmds for cluster health and stats. improved docs for iac, cluster, multi-server restart 2026-02-14 15:38:03 +10:00
Ben Duncan 5863aeed13 Phase 0/1 multi-server-restart enhancements 2026-02-14 13:23:24 +10:00
Ben Duncan febeae8c47 preflight fixes for gh errors 2026-02-14 09:23:42 +10:00
Ben Duncan 3095a90090 e2e tests, multi server improvements, doc updates for workflow 2026-02-14 08:50:24 +10:00
Ben Duncan d80dd0996a Fixed gosec and added go skills 2026-02-14 06:54:00 +10:00
Ben Duncan 25d98dda7e Consolidating documentation 2026-02-13 21:00:22 +10:00
Ben Duncan 6df221309d Merge branch 'main' of github.com:mulgadc/hive 2026-02-13 20:32:08 +10:00
Ben Duncan db0d46ca5b Improved multi-server node testing and tweaks 2026-02-13 20:31:45 +10:00
Ben Duncan cf12a3272e IAC for Proxmox / AWS terraform examples 2026-02-13 11:42:43 +10:00
Julian Sommer 8774fb1e95 feat: distributed start instances (#37) 2026-02-12 15:15:05 +11:00
Julian Sommer fb361dfe49 fix: update instance status BEFORE calling it 2026-02-12 11:11:59 +10:00
Julian Sommer 9d3747bf26 feat: ssh tests in e2e (#36) 2026-02-12 11:57:31 +11:00
Ben Duncan 5b4bad3cb3 Merge branch 'main' of github.com:mulgadc/hive 2026-02-12 10:50:35 +10:00
Ben Duncan 6c4caec652 instance state 2026-02-12 10:37:45 +10:00
Julian Sommer 22af6c4a46 feat: more e2e (#35) 2026-02-12 11:00:00 +11:00
Julian Sommer 815e9a8c36 fix: duplicate subscription 2026-02-12 09:03:24 +10:00
Julian Sommer c7d832a823 fix: nats topic 2026-02-12 08:30:23 +10:00
Julian Sommer 32e2f7bf78 fix: daemon reboot 2026-02-11 16:05:21 +10:00
Julian Sommer 202fffc8db chore: utils tests 2026-02-11 15:21:29 +10:00
Julian Sommer 72709dda12 linter fix 2026-02-11 15:15:30 +10:00
Julian Sommer 0ff03be0dd fix 2026-02-11 15:10:55 +10:00
Julian Sommer 998649cbab feat: createImage (#34) 2026-02-11 16:07:19 +11:00
Julian Sommer efe07a9ed4 chore: bump go version 2026-02-11 12:59:32 +10:00
Julian Sommer 27530ba270 chore: daemon tests 2026-02-11 12:48:21 +10:00
Julian Sommer 467eb0252d chore: more gateway tests 2026-02-11 12:24:00 +10:00
Julian Sommer 2d30743ba5 chore: gateway tests 2026-02-11 12:12:40 +10:00
Julian Sommer 10e65ab66a fix: update ci triggers 2026-02-11 11:44:36 +10:00
Julian Sommer c514d6b937 format 2026-02-11 11:37:52 +10:00
Julian Sommer df70750f09 chore: more tests 2026-02-11 11:37:17 +10:00
Julian Sommer 210ca92b2c fix: dead code 2026-02-11 11:09:00 +10:00
Julian Sommer 5ce3773c42 fix: cryptographically secure ids 2026-02-11 10:51:41 +10:00
Julian Sommer 39b9e0dcd1 fix: split daemon ec2 paths 2026-02-11 10:32:32 +10:00
Julian Sommer b1762f445c fix: centralise nats connection logic 2026-02-11 10:18:30 +10:00
Julian Sommer 221a1ef258 fix: use typed aws errors 2026-02-11 10:11:37 +10:00
Julian Sommer e95c2536da feat: EIGW (#33) 2026-02-11 11:00:53 +11:00
Julian Sommer 2937c8f425 feat: account functionality (#27) 2026-02-11 10:58:52 +11:00
Julian Sommer 1b424928f4 fix: move matrix out of TODO.md 2026-02-11 09:49:22 +10:00
Julian Sommer 7a0d148e71 fix: update Makefile 2026-02-11 09:44:36 +10:00
Julian Sommer 6b4728e702 fix: update CLAUDE.md 2026-02-11 09:43:55 +10:00
Julian Sommer 63e476323d fix: url decode keys 2026-02-11 08:27:02 +10:00
Julian Sommer 6abae11122 feat: real snapshots (#32) 2026-02-10 16:57:50 +11:00
Julian Sommer 8d3808c364 fix: parallelize ci 2026-02-10 10:47:07 +10:00
Julian Sommer 51ffc4c3ab fix: add missing error handling 2026-02-10 10:38:36 +10:00
Julian Sommer b443bbd3e2 fix: permissions 2026-02-10 10:10:39 +10:00
Julian Sommer e545e81e59 fix: run modernize again 2026-02-10 10:02:06 +10:00
Julian Sommer 2491c6f2c2 feat: snapshot e2e (#31) 2026-02-10 10:54:48 +11:00
Julian Sommer 59c57cc074 chore: run modernize 2026-02-10 09:23:25 +10:00
Julian Sommer c5bfedcd98 chore: fmt 2026-02-10 09:10:42 +10:00
Julian Sommer 05a608855c fix: add compile time interface checks 2026-02-10 08:55:23 +10:00
Julian Sommer 513fd07ced feat: snapshots (#30) 2026-02-10 09:45:53 +11:00
Julian Sommer 2977b03cef feat: tags e2e (#29) 2026-02-09 16:28:58 +11:00
Julian Sommer 560e5cb58c feat: tags (#28) 2026-02-09 15:49:59 +11:00
Julian Sommer 09f78fe29b update TODO.md 2026-02-09 11:53:51 +10:00
Julian Sommer f0323bc081 fix: delete dead code (unused handlers) 2026-02-09 11:52:12 +10:00
Julian Sommer dcb08c0bd1 fix: e2e predastore multi node 2026-02-09 11:17:00 +10:00
Julian Sommer 9ba912b745 fix: duplicated unmarshal 2026-02-09 11:14:38 +10:00
Julian Sommer a554b9b884 fix: state change 2026-02-09 10:51:21 +10:00
Julian Sommer e15c077d3a fix: simplify nats handlers 2026-02-09 10:42:47 +10:00
Julian Sommer 88f07a9f3a simplify gateway more 2026-02-09 10:34:12 +10:00
Julian Sommer 45a466f71d fix: viperblock init 2026-02-09 10:03:06 +10:00
Julian Sommer e5038d4a35 fix: deduplicate nilchecks 2026-02-09 09:56:11 +10:00
Julian Sommer 25dd435bc6 fix: use object store instead of s3 client 2026-02-09 09:52:12 +10:00
Julian Sommer daf5f0c780 fix: refactor simple daemon handlers 2026-02-09 09:43:57 +10:00
Julian Sommer e0fea35b19 fix: refactor gateway 2026-02-09 09:35:07 +10:00
Julian Sommer e31e523380 feat: describe-account-attributes (#26) 2026-02-09 10:19:33 +11:00
Julian Sommer 0efa570cc5 fix: update to go 1.25.7 2026-02-09 08:52:32 +10:00
Julian Sommer ef87b41858 feat: describe-availbility-zones (#25) 2026-02-09 09:47:20 +11:00
Julian Sommer 1c2174a10f fix: refactor handlers to use ObjectStore interface (#24) 2026-02-04 16:32:11 +11:00
Julian Sommer 801f0f8135 cleanup dead code 2026-02-04 14:02:48 +10:00
Julian Sommer d0dfea56e8 cleanup comments 2026-02-04 13:54:43 +10:00
Julian Sommer 2566d9d528 make gateway consistent 2026-02-04 13:35:03 +10:00
Julian Sommer a3d7e20e31 feat: describe-volume-status (#23) 2026-02-04 12:46:06 +11:00
Julian Sommer ba990ffd97 feat: attach and detach volume e2e (#22) 2026-02-04 11:39:24 +11:00
Julian Sommer d81e417410 feat: detach-volumes (#21) 2026-02-04 10:54:02 +11:00
Julian Sommer 45ac7c4357 update docs 2026-02-04 08:48:10 +10:00
Julian Sommer fca82939f5 feat: attach-volume (#20) 2026-02-04 09:44:54 +11:00
Julian Sommer 65ad19e604 bump viperblock 2026-02-03 14:00:10 +10:00
Julian Sommer d14158a619 feat: delete on termination 2026-02-03 13:55:19 +10:00
Julian Sommer 3ab1a839db fix: fallback 2026-02-03 12:22:38 +10:00
Julian Sommer 10ded4608b fix: frontend error handling 2026-02-03 12:19:39 +10:00
Julian Sommer 2de2899724 fix: accessibility 2026-02-03 12:05:23 +10:00
Julian Sommer 130e4b9662 feat: create-bucket frontend 2026-02-03 11:54:14 +10:00
Julian Sommer 941aaa6504 move claude rules and skills to root 2026-02-03 11:31:11 +10:00
Julian Sommer 917b66beb9 feat: predastore multinode (#19) 2026-02-03 12:07:06 +11:00
Julian Sommer fdb5cca77e feat: delete-volumes (#18) 2026-02-03 10:47:00 +11:00
Julian Sommer b772b2e0bf feat: use state machine for daemon (#16) 2026-02-02 16:14:36 +11:00
Julian Sommer 875328cfef feat: create-volume (#17) 2026-02-02 15:47:46 +11:00
Julian Sommer 6baf49885b e2e: add modify-volumes (#15) 2026-02-02 14:22:55 +11:00
Julian Sommer 6067f0ccd4 tweak 2026-02-02 11:30:05 +10:00
Julian Sommer e4a2e2972a fix: split daemon handlers 2026-02-02 11:14:35 +10:00
Julian Sommer 731ba4fbe8 feat: refactor Start() daemon 2026-02-02 10:51:16 +10:00
Julian Sommer 7e7d358135 update viperblock version 2026-02-02 10:39:24 +10:00
Julian Sommer ad763eb41d feat: add attach time to volume 2026-02-02 10:34:10 +10:00
Julian Sommer 1a8e273ac6 fix: use slog instead of log 2026-02-02 10:18:04 +10:00
Julian Sommer 91384856ec fix: delete on termination false 2026-02-02 10:12:09 +10:00
Julian Sommer 0e2b715d60 fix: form error handling 2026-02-02 10:08:55 +10:00
Julian Sommer f45a106dda feat: resize volume frontend 2026-02-02 09:36:49 +10:00
Julian Sommer 7aa688e564 fix: table based NATS subscriptions 2026-02-02 08:59:30 +10:00
Julian Sommer 50a0a2e1f1 fix: dead code 2026-02-02 08:35:19 +10:00
Julian Sommer 7f39da8718 fix: state on launch 2026-02-02 08:14:40 +10:00
Julian Sommer d1d6e6d004 todo.md 2026-01-30 15:07:53 +10:00
Julian Sommer 6d24ec2111 feat: modify-volume (resize) (#14) 2026-01-30 16:06:45 +11:00
Julian Sommer 230335adca add rules for claude 2026-01-30 10:59:07 +10:00
Julian Sommer 0b261743d2 fix: use aws-sdk for uploads 2026-01-30 09:38:11 +10:00
Julian Sommer f04d553a02 fix: multipart upload frontend 2026-01-30 08:59:15 +10:00
Julian Sommer dfd36370b0 tweak 2026-01-30 08:18:57 +10:00
benduncan 2fae29d486 Go version bump 2026-01-29 20:23:58 +10:00
benduncan 98701e5208 Install.md tweaks 2026-01-29 20:18:33 +10:00
benduncan 3232d45ff8 Merge branch 'main' of github.com:mulgadc/hive 2026-01-29 20:13:00 +10:00
benduncan d6bfef02ac Latest todo 2026-01-29 20:12:55 +10:00
Julian Sommer 5b55e1d1e0 chore: cut CLAUDE.md down more 2026-01-29 15:55:25 +10:00
Julian Sommer 2aaed28404 chore: trim CLAUDE.md 2026-01-29 15:52:15 +10:00
Julian Sommer e92d67caf0 fix: add default values to run-instances 2026-01-29 15:18:06 +10:00
Julian Sommer 0b1c44d908 fix: volume naming 2026-01-29 15:03:43 +10:00
Julian Sommer 5e53ab1f7e fix: lint issue 2026-01-29 14:14:20 +10:00
Julian Sommer 3bca0e69df feat: block device mappings 2026-01-29 14:10:04 +10:00
Julian Sommer 20de0a7989 feat: count support for run-instances 2026-01-29 12:19:18 +10:00
Julian Sommer 118fb35a1e add hive-ui to start-dev 2026-01-29 11:00:08 +10:00
Julian Sommer dc77e2547b fix: INSTALL.md 2026-01-29 10:46:55 +10:00
Julian Sommer 936828ed53 fix: upgrade to go 1.25.6 to fix vuln 2026-01-29 09:26:27 +10:00
Julian Sommer 454e8dcace revert daemon.go (#13) 2026-01-29 10:22:56 +11:00
Julian Sommer 785d2d6c7b feat: commit frontend build so node isnt a dep for building hive 2026-01-29 09:15:09 +10:00
Julian 5cd5189b56 chore: add logs 2026-01-28 15:49:50 +10:00
Julian a4e7b71d6c fix: e2e verify ssl 2026-01-28 15:36:11 +10:00
Julian adfc54a7f0 fix: check nats sub 2026-01-28 15:26:11 +10:00
Julian 82fa85f992 fix: build first 2026-01-28 15:07:56 +10:00
Julian Sommer de0e9a915e fix: backoffs for jetstream (#12) 2026-01-28 16:00:54 +11:00
Julian 1d816721e8 feat: move hive-ui into main hive binary 2026-01-28 15:00:30 +10:00
Julian 84c0b3e715 fix: remove ssh tests 2026-01-28 14:36:10 +10:00
Julian a423de3380 fix: actual fix 2026-01-28 13:45:13 +10:00
Julian 21a8e2b004 fix: jetstream 2026-01-28 13:36:47 +10:00
Julian 04197d1adc fix: instance-state 2026-01-28 13:19:47 +10:00
Julian bd0c12d45e fix: add t3 and t4g back 2026-01-28 13:15:59 +10:00
Julian a3a32d37c3 fix: SigV4AuthMiddleware 2026-01-28 13:00:02 +10:00
Julian 07056f42a9 fix: pathnames in hive-ui 2026-01-28 11:06:08 +10:00
Julian 528cbc0acd fix: buckets on hive-ui 2026-01-28 10:36:54 +10:00
Julian Sommer 8f0e8d7ead feat: multi-node certs (#11) 2026-01-28 11:36:09 +11:00
Ben Duncan 772da5a181 Updated todo for volumes 2026-01-28 10:26:34 +10:00
Ben Duncan 3912a2a84d Merge branch 'main' of github.com:mulgadc/hive 2026-01-28 10:20:20 +10:00
Ben Duncan bf99270095 EC2 volume todo 2026-01-28 10:20:02 +10:00
Julian 11c1d74a79 embed frontend into binary 2026-01-28 10:15:34 +10:00
Julian 7dca292e67 fix: web-ui to use hive certs 2026-01-28 09:55:01 +10:00
760 changed files with 99265 additions and 28599 deletions
+193 -47
View File
@@ -1,70 +1,216 @@
name: Hive E2E Test
name: Spinifex E2E Tests
on:
push:
branches: [main, e2e]
branches:
- main
- dev
paths:
- "**/*.go"
- go.mod
- go.sum
- Makefile
- tests/e2e/**
- .github/workflows/e2e.yml
workflow_dispatch:
env:
GIT_BRANCH: ${{ github.ref_name }}
jobs:
build_e2e_image:
name: Build E2E Image
runs-on: self-hosted
e2e_single:
name: Single-Node E2E
runs-on: [self-hosted, ci-single]
timeout-minutes: 25
env:
TOFU_LOG: /tmp/e2e-tofu-single-${{ github.run_id }}.log
steps:
- name: Checkout Code
uses: actions/checkout@v6
- name: Clean workspace
run: rm -rf "$GITHUB_WORKSPACE"/*
- uses: actions/create-github-app-token@v3
id: app-token
with:
path: hive
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
repositories: mulga
- name: Clone Dependencies
run: ./hive/scripts/clone-deps.sh
- uses: actions/checkout@v6
with:
repository: mulgadc/mulga
token: ${{ steps.app-token.outputs.token }}
sparse-checkout: scripts/tofu-cluster
path: mulga
- name: Build E2E Image
- name: Clone and build
working-directory: mulga/scripts/tofu-cluster
run: ./build-artifacts.sh "${GIT_BRANCH}" "${RUNNER_TEMP}/spinifex-artifacts.tar"
- name: Provision VM
working-directory: mulga/scripts/tofu-cluster
run: |
docker build \
-t hive-e2e:latest \
-f hive/tests/e2e/Dockerfile.e2e \
.
export TF_VAR_git_branch="${GIT_BRANCH}"
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create env3 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh env3 >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
hive_e2e_test:
name: Hive Single-Node E2E Test
runs-on: self-hosted
needs: build_e2e_image
- name: Bootstrap Cluster
working-directory: mulga/scripts/tofu-cluster
run: |
SPX_PREBUILT=true \
SPX_ARTIFACT_TAR="${RUNNER_TEMP}/spinifex-artifacts.tar" \
./bootstrap.sh env3 >> "$TOFU_LOG" 2>&1
steps:
- name: Run E2E Tests
working-directory: mulga/scripts/tofu-cluster
run: |
docker run --privileged --rm \
-v /dev/kvm:/dev/kvm \
--name hive-e2e-test \
hive-e2e:latest
INVENTORY=$(tofu output -json inventory)
WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
SSH_KEY="${SSH_KEY/#\~/$HOME}"
ssh -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${WAN_IP} \
'cd ~/Development/mulga/spinifex && ./tests/e2e/run-e2e.sh'
hive_multinode_e2e_test:
name: Hive Multi-Node E2E Test
runs-on: self-hosted
needs: build_e2e_image
- name: Teardown
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create env3 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh env3 >> "$TOFU_LOG" 2>&1 || true
e2e_pseudo_multi:
name: Pseudo Multi-Node E2E
runs-on: [self-hosted, ci-pseudo]
timeout-minutes: 25
env:
TOFU_LOG: /tmp/e2e-tofu-pseudo-${{ github.run_id }}.log
steps:
- name: Run Multi-Node E2E Tests
- name: Clean workspace
run: rm -rf "$GITHUB_WORKSPACE"/*
- uses: actions/create-github-app-token@v3
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
repositories: mulga
- uses: actions/checkout@v6
with:
repository: mulgadc/mulga
token: ${{ steps.app-token.outputs.token }}
sparse-checkout: scripts/tofu-cluster
path: mulga
- name: Clone and build
working-directory: mulga/scripts/tofu-cluster
run: ./build-artifacts.sh "${GIT_BRANCH}" "${RUNNER_TEMP}/spinifex-artifacts.tar"
- name: Provision VM
working-directory: mulga/scripts/tofu-cluster
run: |
docker run --privileged --rm \
-v /dev/kvm:/dev/kvm \
--cap-add=NET_ADMIN \
--name hive-multinode-e2e-test \
hive-e2e:latest \
./tests/e2e/run-multinode-e2e.sh
export TF_VAR_git_branch="${GIT_BRANCH}"
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create env4 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env4.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh env4 >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/env4.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
cleanup:
name: Cleanup
runs-on: self-hosted
needs: [hive_e2e_test, hive_multinode_e2e_test]
if: always()
- name: Bootstrap Cluster
working-directory: mulga/scripts/tofu-cluster
run: |
SPX_PREBUILT=true \
SPX_ARTIFACT_TAR="${RUNNER_TEMP}/spinifex-artifacts.tar" \
./bootstrap.sh env4 >> "$TOFU_LOG" 2>&1
- name: Run E2E Tests
working-directory: mulga/scripts/tofu-cluster
run: |
INVENTORY=$(tofu output -json inventory)
WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
SSH_KEY="${SSH_KEY/#\~/$HOME}"
ssh -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${WAN_IP} \
'cd ~/Development/mulga/spinifex && ./tests/e2e/run-pseudo-multinode-e2e.sh'
- name: Teardown
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create env4 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env4.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh env4 >> "$TOFU_LOG" 2>&1 || true
e2e_real_multi:
name: Real Multi-Node E2E
runs-on: [self-hosted, ci-multi]
timeout-minutes: 25
env:
TOFU_LOG: /tmp/e2e-tofu-multi-${{ github.run_id }}.log
steps:
- name: Remove Stale Containers
run: |
docker rm -f hive-e2e-test 2>/dev/null || true
docker rm -f hive-multinode-e2e-test 2>/dev/null || true
- name: Clean workspace
run: rm -rf "$GITHUB_WORKSPACE"/*
- name: Prune Docker Images
run: docker image prune -f
- uses: actions/create-github-app-token@v3
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
repositories: mulga
- uses: actions/checkout@v6
with:
repository: mulgadc/mulga
token: ${{ steps.app-token.outputs.token }}
sparse-checkout: scripts/tofu-cluster
path: mulga
- name: Clone and build
working-directory: mulga/scripts/tofu-cluster
run: ./build-artifacts.sh "${GIT_BRANCH}" "${RUNNER_TEMP}/spinifex-artifacts.tar"
- name: Provision VMs
working-directory: mulga/scripts/tofu-cluster
run: |
export TF_VAR_git_branch="${GIT_BRANCH}"
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create env2 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env2.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh env2 >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/env2.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
- name: Bootstrap Cluster
working-directory: mulga/scripts/tofu-cluster
run: |
SPX_PREBUILT=true \
SPX_ARTIFACT_TAR="${RUNNER_TEMP}/spinifex-artifacts.tar" \
./bootstrap.sh env2 >> "$TOFU_LOG" 2>&1
- name: Run E2E Tests
working-directory: mulga/scripts/tofu-cluster
run: |
INVENTORY=$(tofu output -json inventory)
NODE1_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
SSH_KEY="${SSH_KEY/#\~/$HOME}"
# Build node IP list
NODE_IPS=""
NODE_COUNT=$(echo "$INVENTORY" | jq '.nodes | length')
for i in $(seq 0 $((NODE_COUNT - 1))); do
NODE_IPS="${NODE_IPS} $(echo "$INVENTORY" | jq -r ".nodes[$i].wan_ip")"
done
ssh -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${NODE1_IP} \
"cd ~/Development/mulga/spinifex && ./tests/e2e/run-multinode-e2e.sh ${NODE_IPS}"
- name: Teardown
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create env2 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env2.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh env2 >> "$TOFU_LOG" 2>&1 || true
-40
View File
@@ -1,40 +0,0 @@
name: Hive Build and Test
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
jobs:
hive_build_and_test:
name: Hive Build and Test
runs-on: ubuntu-24.04
steps:
- name: Checkout Code
uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
cache-dependency-path: "go.sum"
go-version-file: "go.mod"
- name: Run Tests
run: LOG_IGNORE=1 go test -v -timeout 120s ./...
- name: Go Vet
run: go vet ./...
- name: Static Check
run: go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./...
- name: Go Vulnerability Check
run: go tool govulncheck ./...
# - name: Gosec Security Scanner
# run: go tool gosec -exclude-generated ./...
- name: Build
run: make go_build
+204
View File
@@ -0,0 +1,204 @@
name: Release
on:
push:
tags: ["v*"]
permissions:
contents: write
jobs:
distro-amd64:
runs-on: ubuntu-24.04
steps:
- name: Checkout Spinifex
uses: actions/checkout@v6
with:
path: spinifex
- name: Checkout Predastore
uses: actions/checkout@v6
with:
repository: mulgadc/predastore
path: predastore
- name: Checkout Viperblock
uses: actions/checkout@v6
with:
repository: mulgadc/viperblock
path: viperblock
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build amd64 Tarball
env:
VERSION: ${{ github.ref_name }}
run: cd spinifex && make distro-amd64 VERSION="$VERSION"
- name: Upload amd64 Artifacts
uses: actions/upload-artifact@v4
with:
name: distro-amd64
path: |
spinifex/dist/spinifex-${{ github.ref_name }}-linux-amd64.tar.gz
spinifex/dist/spinifex-${{ github.ref_name }}-linux-amd64.tar.gz.sha256
spinifex/scripts/setup.sh
distro-arm64:
runs-on: ubuntu-24.04-arm
steps:
- name: Checkout Spinifex
uses: actions/checkout@v6
with:
path: spinifex
- name: Checkout Predastore
uses: actions/checkout@v6
with:
repository: mulgadc/predastore
path: predastore
- name: Checkout Viperblock
uses: actions/checkout@v6
with:
repository: mulgadc/viperblock
path: viperblock
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Build arm64 Tarball
env:
VERSION: ${{ github.ref_name }}
run: cd spinifex && make distro-arm64 VERSION="$VERSION"
- name: Upload arm64 Artifacts
uses: actions/upload-artifact@v4
with:
name: distro-arm64
path: |
spinifex/dist/spinifex-${{ github.ref_name }}-linux-arm64.tar.gz
spinifex/dist/spinifex-${{ github.ref_name }}-linux-arm64.tar.gz.sha256
release:
needs: [distro-amd64, distro-arm64]
runs-on: ubuntu-24.04
steps:
- name: Download amd64 Artifacts
uses: actions/download-artifact@v4
with:
name: distro-amd64
path: artifacts
- name: Download arm64 Artifacts
uses: actions/download-artifact@v4
with:
name: distro-arm64
path: artifacts
- name: Create Draft GitHub Release
uses: softprops/action-gh-release@v2
with:
draft: true
files: |
artifacts/**/*.tar.gz
artifacts/**/*.sha256
artifacts/**/setup.sh
generate_release_notes: true
- name: Upload Tarball for Verify Job
uses: actions/upload-artifact@v4
with:
name: release-assets
path: artifacts/
verify:
needs: release
runs-on: [self-hosted, ci-single]
timeout-minutes: 25
env:
TOFU_LOG: /tmp/release-verify-${{ github.run_id }}.log
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
steps:
- name: Clean Workspace
run: rm -rf "$GITHUB_WORKSPACE"/*
- uses: actions/create-github-app-token@v3
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
repositories: mulga
- uses: actions/checkout@v6
with:
repository: mulgadc/mulga
token: ${{ steps.app-token.outputs.token }}
sparse-checkout: scripts/tofu-cluster
path: mulga
- name: Download Release Assets
uses: actions/download-artifact@v4
with:
name: release-assets
path: release-assets
- name: Provision VM
working-directory: mulga/scripts/tofu-cluster
run: |
tofu init -input=false >> "$TOFU_LOG" 2>&1
tofu workspace select -or-create env3 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1 || true
./scrub-env.sh env3 >> "$TOFU_LOG" 2>&1 || true
tofu apply -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
- name: Bootstrap via Binary Install
working-directory: mulga/scripts/tofu-cluster
run: |
SETUP_SH=$(find "${GITHUB_WORKSPACE}/release-assets" -name "setup.sh" | head -1)
TARBALL=$(find "${GITHUB_WORKSPACE}/release-assets" -name "*-linux-amd64.tar.gz" | head -1)
./bootstrap-install.sh env3 "$SETUP_SH" "$TARBALL"
- name: Run Smoke Test
working-directory: mulga/scripts/tofu-cluster
run: |
INVENTORY=$(tofu output -json inventory)
WAN_IP=$(echo "$INVENTORY" | jq -r '.nodes[0].wan_ip')
SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
SSH_KEY="${SSH_KEY/#\~/$HOME}"
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \
-i "$SSH_KEY" \
smoke-test-install.sh \
"tf-user@${WAN_IP}:/tmp/smoke-test.sh"
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \
-i "$SSH_KEY" "tf-user@${WAN_IP}" \
'bash /tmp/smoke-test.sh'
- name: Promote Release
if: success()
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release edit "$TAG" --repo "$REPO" --draft=false
echo "Release ${TAG} promoted to latest"
- name: Mark Release Failed
if: failure()
env:
GH_TOKEN: ${{ github.token }}
run: |
EXISTING_BODY=$(gh release view "$TAG" --repo "$REPO" --json body -q .body)
NOTES_FILE=$(mktemp)
printf 'Verification failed — see workflow run for details.\n\n%s' "$EXISTING_BODY" > "$NOTES_FILE"
gh release edit "$TAG" --repo "$REPO" --notes-file "$NOTES_FILE"
rm -f "$NOTES_FILE"
- name: Teardown
if: always()
working-directory: mulga/scripts/tofu-cluster
run: |
tofu workspace select -or-create env3 >> "$TOFU_LOG" 2>&1
tofu destroy -var-file=envs/env3.tfvars -auto-approve -input=false >> "$TOFU_LOG" 2>&1
./scrub-env.sh env3 >> "$TOFU_LOG" 2>&1 || true
+133
View File
@@ -0,0 +1,133 @@
name: Spinifex Build and Test
on:
push:
branches:
- main
- dev
paths:
- "**/*.go"
- go.mod
- go.sum
- Makefile
- .github/workflows/spinifex.yml
workflow_dispatch:
jobs:
build_and_test:
name: Spinifex Build and Test
runs-on: ubuntu-24.04
steps:
- name: Checkout Spinifex
uses: actions/checkout@v6
with:
path: spinifex
fetch-depth: 2
- name: Checkout Viperblock
uses: actions/checkout@v6
with:
repository: mulgadc/viperblock
path: viperblock
- name: Checkout Predastore
uses: actions/checkout@v6
with:
repository: mulgadc/predastore
path: predastore
- name: Setup Go
uses: actions/setup-go@v6
with:
cache-dependency-path: "spinifex/go.sum"
go-version-file: "spinifex/go.mod"
- name: Setup Go Workspace
run: go work init ./spinifex ./viperblock ./predastore
- name: Build Spinifex
run: make -C spinifex build
- name: Run Tests with Coverage
run: make -C spinifex test-cover
- name: Check Diff Coverage
run: cd spinifex && scripts/diff-coverage.sh coverage.out --base HEAD~1
race_detection:
name: Spinifex Race Detection
runs-on: ubuntu-24.04
steps:
- name: Checkout Spinifex
uses: actions/checkout@v6
with:
path: spinifex
- name: Checkout Viperblock
uses: actions/checkout@v6
with:
repository: mulgadc/viperblock
path: viperblock
- name: Checkout Predastore
uses: actions/checkout@v6
with:
repository: mulgadc/predastore
path: predastore
- name: Setup Go
uses: actions/setup-go@v6
with:
cache-dependency-path: "spinifex/go.sum"
go-version-file: "spinifex/go.mod"
- name: Setup Go Workspace
run: go work init ./spinifex ./viperblock ./predastore
- name: Run Tests with Race Detector
run: make -C spinifex test-race
lint_and_security:
name: Spinifex Lint and Security
runs-on: ubuntu-24.04
steps:
- name: Checkout Spinifex
uses: actions/checkout@v6
with:
path: spinifex
- name: Checkout Viperblock
uses: actions/checkout@v6
with:
repository: mulgadc/viperblock
path: viperblock
- name: Checkout Predastore
uses: actions/checkout@v6
with:
repository: mulgadc/predastore
path: predastore
- name: Setup Go
uses: actions/setup-go@v6
with:
cache-dependency-path: "spinifex/go.sum"
go-version-file: "spinifex/go.mod"
- name: Setup Go Workspace
run: go work init ./spinifex ./viperblock ./predastore
- name: Check Format
run: make -C spinifex check-format
- name: Check Modernize
run: make -C spinifex check-modernize
- name: Go Vet
run: make -C spinifex vet
- name: Security Checks
run: make -C spinifex security-check
+21 -45
View File
@@ -2,72 +2,48 @@ name: UI Build and Test
on:
push:
branches: [main, ui]
paths: ["hive-ui/**", ".github/workflows/ui.yml"]
pull_request:
branches:
- main
- dev
paths:
- spinifex/services/spinifexui/frontend/**
- .github/workflows/ui.yml
workflow_dispatch:
jobs:
frontend:
name: Frontend Build and Test
ui_build_and_test:
name: UI Build and Test
runs-on: ubuntu-24.04
defaults:
run:
working-directory: hive-ui/frontend
working-directory: spinifex/services/spinifexui/frontend
steps:
- name: Checkout Code
uses: actions/checkout@v6
with:
sparse-checkout: spinifex/services/spinifexui/frontend
- name: Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v5
with:
package_json_file: hive-ui/frontend/package.json
package_json_file: spinifex/services/spinifexui/frontend/package.json
- name: Setup Node.js
uses: actions/setup-node@v6
with:
cache: "pnpm"
cache-dependency-path: hive-ui/frontend/pnpm-lock.yaml
node-version-file: hive-ui/frontend/.nvmrc
cache-dependency-path: spinifex/services/spinifexui/frontend/pnpm-lock.yaml
node-version-file: spinifex/services/spinifexui/frontend/.nvmrc
- name: Install frontend dependencies
- name: Install Dependencies
run: pnpm install --frozen-lockfile
- name: Run Biome formatter check
- name: Run Check
run: pnpm check
- name: Build frontend
- name: Run Tests with Coverage
run: pnpm test:coverage
- name: Build Frontend
run: pnpm build
webserver:
name: Webserver Build and Test
runs-on: ubuntu-24.04
defaults:
run:
working-directory: hive-ui/webserver
steps:
- name: Checkout Code
uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
cache-dependency-path: hive-ui/webserver/go.sum
go-version-file: hive-ui/webserver/go.mod
- name: Go Vet
run: go vet ./...
- name: Static Check
run: go tool staticcheck ./...
- name: Go Vulnerability Check
run: go tool govulncheck ./...
- name: Gosec Security Scanner
run: go tool gosec -exclude-generated ./...
- name: Build
run: go build ./...
+18 -2
View File
@@ -1,9 +1,19 @@
# Terraform/OpenTofu
.terraform/
.terraform.lock.hcl
*.tfstate
*.tfstate.*
*.tfvars
!example.tfvars
scripts/iac/proxmox/clusters/
# Go workspace
go.work
go.work.sum
# Build outputs
bin/
dist/
tmp/
*.test
*.out
@@ -21,5 +31,11 @@ Thumbs.db
# Test reports
*-report.txt
# hive-ui
hive-ui/certs/
# Internal docs (moved to private mulga repo)
docs/INTERNAL.md
docs/PRIVATE.md
docs/DEVNOTES.md
docs/PLAN.md
docs/TODO.md
docs/development/
docs/internal/
-7
View File
@@ -1,7 +0,0 @@
{
"recommendations": [
"biomejs.biome",
"bradlc.vscode-tailwindcss",
"golang.go"
]
}
-36
View File
@@ -1,36 +0,0 @@
{
"files.readonlyInclude": {
"**/routeTree.gen.ts": true
},
"files.watcherExclude": {
"**/routeTree.gen.ts": true
},
"search.exclude": {
"**/routeTree.gen.ts": true
},
"[css][json][jsonc]": {
"editor.defaultFormatter": "biomejs.biome",
"editor.tabSize": 2
},
"[javascript][javascriptreact][typescript][typescriptreact]": {
"editor.codeActionsOnSave": {
"source.fixAll.biome": "explicit",
"source.organizeImports.biome": "explicit"
},
"editor.defaultFormatter": "biomejs.biome",
"editor.tabSize": 2
},
"[go]": {
"editor.defaultFormatter": "golang.go",
"editor.tabSize": 4
},
"biome.configurationPath": "./hive-ui/frontend/biome.jsonc",
"biome.requireConfiguration": true,
"biome.lsp.bin": "./hive-ui/frontend/node_modules/.bin/biome",
"editor.formatOnSave": true,
"editor.formatOnPaste": true,
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,
"files.trimTrailingWhitespace": true,
"typescript.tsdk": "./hive-ui/frontend/node_modules/typescript/lib"
}
-214
View File
@@ -1,214 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is the Hive platform - an AWS-compatible infrastructure stack for bare-metal, edge, and on-premise deployments. Hive orchestrates multiple independent components that work together to provide AWS-compatible services.
## Component Repositories
The Hive platform consists of these independent repositories:
- **[viperblock](../viperblock/)** - High-performance block storage service (EBS alternative) with WAL-backed storage
- **[predastore](../predastore/)** - S3-compatible object storage service with Reed-Solomon erasure coding
- **nbdkit/** - Network Block Device (NBD) server for exporting block devices (upstream project)
- **hive/** (this repo) - VM orchestration service and platform coordinator
Each component can be used independently or as part of the integrated Hive stack. See component-specific `CLAUDE.md` files for detailed development guidance:
- `../viperblock/CLAUDE.md` - Viperblock development guide
- `../predastore/CLAUDE.md` - Predastore development guide
## Cross-Repository Development Setup
For platform development, clone all repositories to the same parent directory:
```bash
mkdir mulga
cd mulga
git clone <hive-repo-url> hive
git clone <viperblock-repo-url> viperblock
git clone <predastore-repo-url> predastore
```
The `hive/go.mod` uses local replace directives for cross-component development:
```go
replace github.com/mulgadc/viperblock => ../viperblock
replace github.com/mulgadc/predastore => ../predastore
```
## Build Commands
### Hive (Platform Orchestrator)
```bash
make build # Build the hive binary
make test # Run Go tests
make bench # Run benchmarks
make clean # Clean build artifacts
```
### **MANDATORY: Unit Testing Requirements**
**⚠️ CRITICAL: Always run unit tests before any commit or push operation.**
```bash
# REQUIRED before any git commit or push
make test # Must pass with zero failures
# Example workflow:
make test # Verify all tests pass
git add . # Stage changes
git commit -m "..." # Only after tests pass
git push origin main # Only after tests pass
```
**Testing Policy:**
- **All unit tests MUST pass** before committing changes
- **No exceptions** - failing tests block commits
- Tests must complete without errors or panics
- Use `make test` command to run the full test suite
- If tests fail, fix issues before proceeding with git operations
### Component Build Commands
For detailed build instructions, see component-specific CLAUDE.md files:
**Viperblock** (`../viperblock/CLAUDE.md`):
```bash
cd ../viperblock && make build # Build sfs, vblock binaries + NBD plugin
```
**Predastore** (`../predastore/CLAUDE.md`):
```bash
cd ../predastore && make build # Build s3d binary
```
**NBDkit** (system dependency):
```bash
cd nbdkit && autoreconf -i && ./configure && make && make install
```
## Architecture Notes
### Message-Driven Microservices Architecture
Hive uses a **scalable message-driven architecture** that separates API handling from request processing:
```
AWS SDK (custom endpoint) → Hive API Gateway → NATS Topics → Specialized Daemons
↑ ↓
AWS-compatible response ← NATS Topics ← Processing Results
```
#### Core Components:
1. **Hive API Gateway Service** - TLS endpoint that handles AWS SDK requests
2. **Specialized Daemon Services** - Horizontally scalable workers for each AWS service type
3. **NATS Message Broker** - Topic-based routing with queue groups for load balancing
#### Current Implementation (daemon.go):
- **Gateway Logic**: `StartAWSGateway()` handles AWS API calls on port 9999
- **NATS Integration**: Uses request-response pattern: `d.natsConn.Request("ec2.runinstances", data, 30s)`
- **Queue Groups**: Load balancing via `"hive-workers"` queue group
- **Service Coordination**: All inter-service communication via NATS topics
### Inter-component Dependencies
- **Service Startup Order**: NATS → Predastore → Viperblock → NBDkit → Gateway → Daemons
- **Message Flow**: Gateway translates AWS calls to NATS messages, daemons process and respond
- **Scaling**: Multiple daemon instances can subscribe to same topics for horizontal scaling
- All Go projects use local module replacements for development
### Storage Backend Integration
- **Viperblock Integration**: EBS volumes mounted via NATS messaging (`ebs.mount` topic)
- **Predastore Integration**: S3 operations proxied through dedicated S3 daemons
- **NBDkit Integration**: Block device access for VM storage
- **Volume Lifecycle**: Coordinated through NATS between EC2 and EBS daemons
### Key Design Patterns
- **Message-Driven**: All service communication via NATS topics (ec2.*, ebs.*, s3.*, vpc.*)
- **Queue Groups**: Load balancing and fault tolerance via NATS queue groups
- **Request-Response**: 30-second timeout pattern for AWS API compatibility
- **Service Specialization**: Dedicated daemon types for each AWS service
- **Horizontal Scaling**: Multiple daemon instances handle same topic types
### NATS Topic Structure
The system uses structured NATS topics for service communication:
```bash
# EC2 Service Topics:
ec2.runinstances # Launch new instances
ec2.describeinstances # Query instance status
ec2.startinstances # Start stopped instances
ec2.stopinstances # Stop running instances
ec2.terminateinstances # Terminate instances
# EBS Service Topics:
ebs.createvolume # Create new EBS volume
ebs.attachvolume # Attach volume to instance
ebs.detachvolume # Detach volume
ebs.describevolumes # List volumes
ebs.mount # Mount volume (internal)
ebs.unmount # Unmount volume (internal)
# S3 Service Topics:
s3.createbucket # Create S3 bucket
s3.putobject # Store object
s3.getobject # Retrieve object
s3.listbuckets # List buckets
# VPC Service Topics:
vpc.createvpc # Create VPC
vpc.createsubnet # Create subnet
vpc.describevpcs # List VPCs
# Health and Discovery:
health.ec2.daemon.{id} # Daemon health heartbeats
discovery.services # Service registration
```
### Go Module Structure
All Go projects use:
- Go 1.23+ with module mode
- Local replace directives for cross-component development
- Standard build flags: `-ldflags "-s -w"` for optimized binaries
- Test environment variable: `LOG_IGNORE=1` to suppress logs during testing
### Development Workflow
#### Quick Setup (Recommended)
```bash
# 1. Clone dependencies (if not already done)
./scripts/clone-deps.sh
# 2. Setup development environment
./scripts/dev-setup.sh
# 3. Start all services
./scripts/start-dev.sh
# 4. Stop all services when done
./scripts/stop-dev.sh
```
#### Manual Development Process
1. **Cross-repo Setup**: Clone all repositories to same parent directory
2. **Service Dependencies**: Start services in order: NATS → Predastore → Viperblock → NBDkit → Gateway → Daemons
3. **Build Order**: Build components: predastore → viperblock → hive
4. **Testing**: Use `make test` in each component to verify changes
5. **Message-Driven Development**: Test NATS message flows between gateway and daemons
6. **Scaling**: Test with multiple daemon instances subscribing to same topics
#### Development Tools
- **Hot Reloading**: Use `air` for automatic restarts during development
- **TLS Certificates**: Auto-generated self-signed certificates for HTTPS endpoints
- **Log Monitoring**: Centralized logs in `data/logs/` directory
- **AWS CLI Testing**: Test endpoints with `aws --endpoint-url https://localhost:9999`
### Development Roadmap
For major feature development and architectural changes, refer to `HIVE_DEVELOPMENT_PLAN.md` which contains:
- Comprehensive roadmap for transforming Hive from proof-of-concept to beta release
- Detailed implementation phases with specific tasks and file structures
- AWS API compatibility implementation guidelines
- Development environment automation plans
- Testing and validation frameworks
When implementing new AWS services or enhancing existing functionality, follow the structured approach outlined in the development plan.
File diff suppressed because it is too large Load Diff
+2 -668
View File
@@ -1,669 +1,3 @@
# Hive Installation
# Spinifex Installation
Notes for development environment installation.
## Dependencies
The current supported OS that is validated and tested includes:
* Ubuntu 22.04
* Ubuntu 24.04
* Ubuntu 25.10
For the development preview please use one of the supported versions above.
## Download
Create the base directory for the Hive development environment.
```bash
mkdir -p ~/Development/mulga/
cd ~/Development/mulga/
git clone https://github.com/mulgadc/hive.git
```
### Quick Install
To bootstrap the dependencies of Hive in one simple step (QEMU, Go, AWS CLI)
```bash
sudo make -C hive quickinstall
```
### Manual Install
Alternatively, run the following steps to manually setup the required dependencies.
```bash
sudo add-apt-repository universe
sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev make gcc unzip xz-utils file
```
Ensure the Go toolkit is installed for version 1.25.5 or higher. Recommended to install the latest directly from [https://go.dev/dl/](https://go.dev/dl/).
Confirm go is correctly installed, and set in your $PATH.
```bash
go version
```
Hive provides AWS API/SDK layer functionality, which requires the AWS CLI tool to be installed to interface with the system.
```bash
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
```
Confirm awscli version > 2.0 is installed which is required by Hive.
[https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
```bash
aws --version
```
## Build
Setup the dev environment and package dependencies on [viperblock](https://github.com/mulgadc/viperblock/) and [predastore](https://github.com/mulgadc/predastore/).
```bash
cd hive
./scripts/clone-deps.sh # Clone viperblock + predastore repositories
./scripts/dev-setup.sh # Setup complete development environment
```
Once complete, confirm `./bin/hive` exists and executable.
# Single Node Installation
For rapid development and testing, `hive` can be installed locally as a single node instance. Follow the instructions below for a complete working environment.
## Init
When running Hive for the first time, run the init function to create the default directories for data, config files and layout required.
Example single node installation to get started, this will create a new region `ap-southeast-2` and availability zone `ap-southeast-2a` on your local instance:
```bash
./bin/hive admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
```
## Trust CA Certificate (Required)
Hive generates a local Certificate Authority (CA) during initialization. For AWS CLI and other tools to trust Hive services over HTTPS, you must add the CA certificate to your system's trust store.
```bash
sudo cp ~/hive/config/ca.pem /usr/local/share/ca-certificates/hive-ca.crt
sudo update-ca-certificates
```
Verify the certificate was added:
```bash
ls -la /usr/local/share/ca-certificates/hive-ca.crt
```
This step is required for AWS CLI 2.33+ which requires proper CA trust chains. Without this, you would need to use `--no-verify-ssl` for every AWS CLI command.
Next, set the AWS profile to use `hive` which points to the local environment.
```bash
export AWS_PROFILE=hive
```
## Launch Services
Start the core services for development.
```bash
./scripts/start-dev.sh
```
## Create SSH Key
For first install, create or import an existing key pair which can be used to launch EC2 instances.
### Import Existing Key
Import an existing key pair, replace `~/.ssh/id_rsa.pub` with your specified key.
```bash
aws ec2 import-key-pair --key-name "hive-key" --public-key-material fileb://~/.ssh/id_rsa.pub
```
If no key exists, generate one using `ssh-keygen` and repeat the command above.
```bash
ssh-keygen -t rsa
```
### Create New Key
Alternatively, create a new key pair using the AWS CLI tool and store the JSON output of the AWS SDK using the `jq` command.
```bash
aws ec2 create-key-pair \
--key-name hive-key \
| jq -r '.KeyMaterial | rtrimstr("\n")' > ~/.ssh/hive-key
```
Update permissions for the key for ssh to accept reading the file.
```bash
chmod 600 ~/.ssh/hive-key
```
Next, generate a public key from the specified private key pair.
```bash
ssh-keygen -y -f ~/.ssh/hive-key > ~/.ssh/hive-key.pub
```
### Validate New Key
```bash
aws ec2 describe-key-pairs
```
```json
{
"KeyPairs": [
{
"KeyPairId": "key-3caa0a34f53d12a3",
"KeyType": "ed25519",
"Tags": [],
"CreateTime": "2025-10-28T13:39:23.458000+00:00",
"KeyName": "hive-key",
"KeyFingerprint": "SHA256:/g/A5OkeZeSydz9WUErXYVdCt00b0VbfN6RLn2YVFAY"
}
]
}
```
## Create AMI Template
Use the hive CLI tool to import a selected OS image. Note, the source file can be compressed (e.g image.tar.gz, image.gz, image.tar.xz) and the tool will automatically extract and upload the raw OS image as an AMI after validation the disk image contains a UEFI/BIOS boot capability.
Note, when downloading OS images, use supported platforms that support the `cloud-init` feature to automatically bootstrap when using the Hive EC2 functionality to access SSH and networking services.
### Automatic Image Import
Discover available images to automatically download and install. This will pull the images from the distro official mirror and simplify the process to bootstrap a Hive installation with AMIs that include common operating systems.
```bash
./bin/hive admin images list
```
```bash
NAME | DISTRO | VERSION | ARCH | BOOT
alpine-3.22.2-x86_64 | alpine | 3.22.2 | x86_64 | bios
debian-12-arm64 | debian | 12 | arm64 | bios
debian-12-x86_64 | debian | 12 | x86_64 | bios
ubuntu-24.04-arm64 | ubuntu | 24.04 | arm64 | bios
ubuntu-24.04-x86_64 | ubuntu | 24.04 | x86_64 | bios
```
Next, choose the image you would like to import as an AMI. Choose the appropriate `x86_64` or `arm64` architecture depending on your platform.
```bash
./bin/hive admin images import --name debian-12-arm64 --force
```
Make note of the Image-ID `ami-XXX`
```bash
✅ Image import complete. Image-ID (AMI): ami-e29fcc65734aec9ea
```
Next, verify available disk images to confirm the import was successful, replace `ami-XXX` with the value from the command output.
```bash
aws ec2 describe-images --image-ids ami-XXX
```
### Manual Image Import
Using this method you can import any OS disk image. For example, download the Debian 12 image from the repository [https://cloud.debian.org/images/cloud/bookworm/latest/](https://cloud.debian.org/images/cloud/bookworm/latest/).
Download the image:
```bash
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-arm64.tar.xz -O ~/debian-12-genericcloud-arm64.tar.xz
```
Import as an AMI to the backend store:
```bash
./bin/hive admin images import --file ~/debian-12-genericcloud-arm64.tar.xz --arch arm64 --distro debian --version 12
```
Next, verify available disk images to confirm the import was successful.
```bash
aws ec2 describe-images
```
## Run Instance
Once Hive is successfully installed and bootstrapped with a system AMI and SSH keys, proceed to run an instance. Replace `ami-XXX` with your imported ImageId above.
```bash
export HIVE_AMI="ami-XXX"
```
### Query instance types available
Depending on the host platform Hive is installed for the CPU capabilities and vendor (e.g AMD, Intel, ARM), different compute instances will be available.
Query available instance types and choose an instance type available on your host with 2 vCPUs:
```bash
aws ec2 describe-instance-types
```
```json
{
"InstanceTypes": [
{
"InstanceType": "m8a.small",
"CurrentGeneration": true,
"SupportedRootDeviceTypes": [
"ebs"
],
"SupportedVirtualizationTypes": [
"hvm"
],
"Hypervisor": "kvm",
"ProcessorInfo": {
"SupportedArchitectures": [
"x86_64"
]
},
"VCpuInfo": {
"DefaultVCpus": 2
},
"MemoryInfo": {
"SizeInMiB": 1024
},
"BurstablePerformanceSupported": false
},
]
}
```
Export the instance-type:
```sh
export HIVE_INSTANCE="m8a.small"
```
Next, launch a new instance, note `hive-key` is the SSH key specified in the previous stage. Be sure to specify the instance-type that is available from your host as above.
```bash
aws ec2 run-instances \
--image-id $HIVE_AMI \
--instance-type $HIVE_INSTANCE \
--key-name hive-key \
--security-group-ids sg-0123456789abcdef0 \
--subnet-id subnet-6e7f829e \
--count 1
```
A sample response is below from the `RunInstance` request, note the `InstanceId` attribute:
```json
{
"ReservationId": "r-f101157331e261a68",
"OwnerId": "123456789012",
"Instances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"ImageId": "ami-15c3dcfe607460f15",
"State": {
"Code": 0,
"Name": "pending"
},
"KeyName": "hive-key",
"InstanceType": "t3.micro",
"LaunchTime": "2025-11-12T13:07:47.548000+00:00"
}
]
}
```
Export the instance ID for following the rest of the tutorial.
```bash
export INSTANCE_ID="i-XXX"
```
Next, validate the running instance is ready.
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
Confirm the `State.Name` attribute is set as `running`.
```json
{
"Reservations": [
{
"ReservationId": "r-f101157331e261a68",
"OwnerId": "123456789012",
"Instances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"ImageId": "ami-15c3dcfe607460f15",
"State": {
"Code": 16,
"Name": "running"
},
"KeyName": "hive-key",
"InstanceType": "t3.micro",
"LaunchTime": "2025-11-12T13:07:47.548000+00:00"
}
]
}
]
}
```
## SSH Connection (development)
For a Hive development environment (toggled off for production), a local SSH port forwaring will be active to connect directly to the instance, regardless of the VPC and network settings.
Determine the SSH port allocated.
```bash
ps auxw | grep $INSTANCE_ID
```
```bash
qemu-system-x86_64 -daemonize -pidfile /run/user/1000/i-36765eb0c6609e4d.pid -qmp unix:/run/user/1000/qmp-i-36765eb0c6609e4d2.sock,server,nowait -enable-kvm -M ubuntu -serial pty -cpu host -smp 2 -m 1024 -drive file=nbd://127.0.0.1:42653,format=raw,if=none,media=disk,id=os -drive file=nbd://127.0.0.1:44499,format=raw,if=virtio,media=cdrom,id=cloudinit -device virtio-blk-pci,drive=os,bootindex=1 -device virtio-rng-pci -device virtio-net-pci,netdev=net0 -netdev user,id=net0,hostfwd=tcp:127.0.0.1:33683-:22
```
Note the `hostfwd=tcp:127.0.0.1:33683-:22`, in this case the local port `33683` will connect to the new instance.
```bash
ssh -i ~/.ssh/hive-key ec2-user@127.0.0.1 -p 33683
...
Linux hive-vm-36765eb0 6.1.0-40-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.153-1 (2025-09-20) x86_64
ec2-user@hive-vm-36765eb0:~$
```
Congratulations, your first AMI image is imported, a new EC2 instance launched, and successfully connected via SSH for the configured SSH key, using the OS `cloud-init` procedure.
## Managing Instances
### Stop Instance
To stop a running instance gracefully:
```bash
aws ec2 stop-instances --instance-ids $INSTANCE_ID
```
```json
{
"StoppingInstances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"CurrentState": {
"Code": 64,
"Name": "stopping"
},
"PreviousState": {
"Code": 16,
"Name": "running"
}
}
]
```
Next, confirm the instance has stopped as requested.
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
```json
{
"Reservations": [
{
"ReservationId": "r-f101157331e261a68",
"OwnerId": "123456789012",
"Instances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"ImageId": "ami-15c3dcfe607460f15",
"State": {
"Code": 80,
"Name": "stopped"
},
"KeyName": "hive-key",
"InstanceType": "t3.micro",
"LaunchTime": "2025-11-12T13:07:47.548000+00:00"
}
]
}
]
}
```
### Start Instance
To start a previously stopped instance:
```bash
aws ec2 start-instances --instance-ids $INSTANCE_ID
```
```json
{
"StartingInstances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"CurrentState": {
"Code": 0,
"Name": "pending"
},
"PreviousState": {
"Code": 80,
"Name": "stopped"
}
}
]
```
Next, validate the instance is running as expected.
```bash
aws ec2 describe-instances --instance-ids "i-36765eb0c6609e4d2"
```
```json
{
"Reservations": [
{
"ReservationId": "r-f101157331e261a68",
"OwnerId": "123456789012",
"Instances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"ImageId": "ami-15c3dcfe607460f15",
"State": {
"Code": 16,
"Name": "running"
},
"KeyName": "hive-key",
"InstanceType": "t3.micro",
"LaunchTime": "2025-11-12T13:07:47.548000+00:00"
}
]
}
]
}
```
### Terminate Instance
To terminate an instance, which will first stop the instance, and on success, remove the EBS volumes and permanately remove the instance data.
```bash
aws ec2 terminate-instances --instance-ids $INSTANCE_ID
```
```json
{
"TerminatingInstances": [
{
"InstanceId": "i-36765eb0c6609e4d2",
"CurrentState": {
"Code": 16,
"Name": "running"
},
"PreviousState": {
"Code": 16,
"Name": "running"
}
}
]
}
```
Next, validate the instance is removed, this may take a few minutes depending the instance volume size.
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
On success no data will be returned, since the instance is no longer available.
# Multi-node Configuration
To create a simulate multi-node installation on a single server (e.g node1, node2, node3) review the script `./scripts/create-multi-node.sh`.
Specifically this will create 3 IPs on a specified ethernet adapter.
```bash
ip addr add 10.11.12.1/24 dev eth0
ip addr add 10.11.12.2/24 dev eth0
ip addr add 10.11.12.3/24 dev eth0
```
Next, `hive` will be installed and initiated in `$HOME/node1` for configuration files.
```bash
# Initialize node1
./bin/hive admin init \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node1 \
--bind 10.11.12.1 \
--cluster-bind 10.11.12.1 \
--port 4432 \
--hive-dir ~/node1/ \
--config-dir ~/node1/config/
```
Two other nodes will be created, joining node1 to exchange configuration files and identify as a new node.
```bash
# Join cluster
./bin/hive admin join \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node2 \
--bind 10.11.12.2 \
--cluster-bind 10.11.12.2 \
--cluster-routes 10.11.12.1:4248 \
--host 10.11.12.1:4432 \
--data-dir ~/node2/ \
--config-dir ~/node2/config/ \
# Start node2
./scripts/start-dev.sh ~/node2/
echo "Node3 Setup:"
./bin/hive admin join \
--region ap-southeast-2 \
--az ap-southeast-2a \
--node node3 \
--bind 10.11.12.3 \
--cluster-bind 10.11.12.3 \
--cluster-routes 10.11.12.1:4248 \
--host 10.11.12.1:4432 \
--data-dir ~/node3/ \
--config-dir ~/node3/config/
./scripts/start-dev.sh ~/node3/
```
Once configured, each node will have it's own storage for config, data and state in `~/node[1,2,3]` to simulate a unique node and network.
Once the multi-node is configured, follow the original installation instructions above to:
- Import an SSH key
- Clone an AMI
- Launch an instance
Note - when using the AWS CLI tool you must specify the IP address of the node (10.11.12.1, 10.11.12.2, or 10.11.12.3).
**Important**: For multi-node setups, the server certificates need to include the additional IP addresses. You may need to regenerate certificates with the `--force` flag after configuring multi-node, or ensure the CA certificate is trusted system-wide (see "Trust CA Certificate" section above).
```bash
aws --endpoint-url https://10.11.12.3:9999/ ec2 describe-instances --instance-ids i-9f5f648adc57ea46d
```
```json
{
"Reservations": [
{
"ReservationId": "r-9d26866c3b0d53bf4",
"OwnerId": "123456789012",
"Instances": [
{
"InstanceId": "i-9f5f648adc57ea46d",
"ImageId": "ami-d0de73dd18fa33ac9",
"State": {
"Code": 16,
"Name": "running"
},
"KeyName": "hive-key",
"InstanceType": "t3.micro",
"LaunchTime": "2025-11-21T10:36:42.834000+00:00"
}
]
}
]
```
For debugging, reference the node unique configuration files, e.g
```bash
tail -n 100 ~/node3/logs/hive.log
```
```bash
...
2025/11/21 22:59:32 Received message on subject: ec2.DescribeInstances
2025/11/21 22:59:32 Message data: {"DryRun":null,"Filters":null,"InstanceIds":["i-9f5f648adc57ea46d"],"MaxResults":null,"NextToken":null}
2025/11/21 22:59:32 INFO Processing DescribeInstances request from this node
2025/11/21 22:59:32 INFO handleEC2DescribeInstances completed count=1
...
```
### Stop multi-node Instance
To stop a simulated multi-node instance:
```bash
./scripts/stop-multi-node.sh
```
Note if multiple EC2 instances are running, it make take a few minutes to gracefully terminate the instance, unmount the attached EBS volume (via NBD) and push the write-ahead-log (WAL) to the S3 server (predastore).
For documentation on using and installing Spinifex see [https://docs.mulgadc.com](https://docs.mulgadc.com/)
+629 -169
View File
@@ -1,201 +1,661 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
1. Definitions.
Preamble
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
The precise terms and conditions for copying, distribution and
modification follow.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
TERMS AND CONDITIONS
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
0. Definitions.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
"This License" refers to version 3 of the GNU Affero General Public License.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
A "covered work" means either the unmodified Program or a work based
on the Program.
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
1. Source Code.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
END OF TERMS AND CONDITIONS
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
APPENDIX: How to apply the Apache License to your work.
The Corresponding Source for a work in source code form is that
same work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
2. Basic Permissions.
Copyright [yyyy] [name of copyright owner]
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
http://www.apache.org/licenses/LICENSE-2.0
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
+153 -26
View File
@@ -1,4 +1,5 @@
GO_PROJECT_NAME := hive
GO_PROJECT_NAME := spx
SHELL := /bin/bash
# Detect architecture for cross-platform support
ARCH := $(shell uname -m)
@@ -32,24 +33,69 @@ else
GO_BUILD_MOD :=
endif
# Quiet-mode filters (active when QUIET=1, set by preflight via recursive make)
ifdef QUIET
_Q = @
_COVQ = 2>&1 | { grep -Ev '^\s*(ok|PASS|\?|=== RUN|--- PASS:)\s' | grep -v 'coverage: 0\.0%' || true; }; exit $${PIPESTATUS[0]}
_RACEQ = 2>&1 | { grep -Ev '^\s*(ok|PASS|\?|=== RUN|--- PASS:)\s' || true; }; exit $${PIPESTATUS[0]}
_SECQ = >
else
_Q =
_COVQ =
_RACEQ =
_SECQ = 2>&1 | tee
endif
build:
$(MAKE) go_build
# Build spinifex-ui frontend (requires pnpm)
build-ui:
@echo -e "\n....Building spinifex-ui frontend...."
cd spinifex/services/spinifexui/frontend && pnpm build
# GO commands
VERSION ?= $(shell git describe --tags --always --dirty)
COMMIT ?= $(shell git rev-parse --short HEAD)
LDFLAGS := -s -w -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Version=$(VERSION) -X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Commit=$(COMMIT)
go_build:
@echo "\n....Building $(GO_PROJECT_NAME)"
GOEXPERIMENT=greenteagc go build $(GO_BUILD_MOD) -ldflags "-s -w" -o ./bin/$(GO_PROJECT_NAME) cmd/hive/main.go
@echo -e "\n....Building $(GO_PROJECT_NAME)"
go build $(GO_BUILD_MOD) -ldflags "$(LDFLAGS)" -o ./bin/$(GO_PROJECT_NAME) cmd/spinifex/main.go
go_run:
@echo "\n....Running $(GO_PROJECT_NAME)...."
@echo -e "\n....Running $(GO_PROJECT_NAME)...."
$(GOPATH)/bin/$(GO_PROJECT_NAME)
# Preflight — runs the same checks as GitHub Actions (format + lint + security + tests).
# Use this before committing to catch CI failures locally.
preflight:
@$(MAKE) --no-print-directory QUIET=1 check-format check-modernize vet security-check test-cover diff-coverage test-race
@echo -e "\n ✅ Preflight passed — safe to commit."
# Run unit tests
test:
@echo "\n....Running tests for $(GO_PROJECT_NAME)...."
LOG_IGNORE=1 go test -v -timeout 120s ./...
@echo -e "\n....Running tests for $(GO_PROJECT_NAME)...."
LOG_IGNORE=1 go test -timeout 120s ./spinifex/...
# Run unit tests with coverage profile
COVERPROFILE ?= coverage.out
test-cover:
@echo -e "\n....Running tests with coverage for $(GO_PROJECT_NAME)...."
$(_Q)LOG_IGNORE=1 go test -timeout 120s -coverprofile=$(COVERPROFILE) -covermode=atomic ./spinifex/... $(_COVQ)
@scripts/check-coverage.sh $(COVERPROFILE) $(QUIET)
# Run unit tests with race detector
test-race:
@echo -e "\n....Running tests with race detector for $(GO_PROJECT_NAME)...."
$(_Q)LOG_IGNORE=1 go test -race -timeout 300s ./spinifex/... $(_RACEQ)
# Check that new/changed code meets coverage threshold (runs tests first)
diff-coverage: test-cover
@QUIET=$(QUIET) scripts/diff-coverage.sh $(COVERPROFILE)
bench:
@echo "\n....Running benchmarks for $(GO_PROJECT_NAME)...."
@echo -e "\n....Running benchmarks for $(GO_PROJECT_NAME)...."
$(MAKE) easyjson
LOG_IGNORE=1 go test -benchmem -run=. -bench=. ./...
@@ -58,27 +104,31 @@ run:
$(MAKE) go_run
clean:
rm ./bin/$(GO_PROJECT_NAME)
rm -f ./bin/$(GO_PROJECT_NAME)
rm -rf spinifex/services/spinifexui/frontend/dist
install-system:
@echo "\n....Installing system dependencies for $(ARCH)...."
@echo -e "\n....Installing system dependencies for $(ARCH)...."
@echo "QEMU packages: $(QEMU_PACKAGES)"
apt-get update && sudo apt-get install -y \
sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y \
-o Dpkg::Options::="--force-confdef" \
-o Dpkg::Options::="--force-confold" \
nbdkit nbdkit-plugin-dev pkg-config $(QEMU_PACKAGES) qemu-utils qemu-kvm \
libvirt-daemon-system libvirt-clients libvirt-dev make gcc jq curl \
iproute2 netcat-openbsd openssh-client wget git unzip sudo xz-utils file
iproute2 netcat-openbsd openssh-client wget git unzip sudo xz-utils file \
ovn-central ovn-host openvswitch-switch
install-go:
@echo "\n....Installing Go 1.25.5 for $(ARCH) ($(GO_ARCH))...."
@echo -e "\n....Installing Go 1.26.1 for $(ARCH) ($(GO_ARCH))...."
@if [ ! -d "/usr/local/go" ]; then \
curl -L https://go.dev/dl/go1.25.5.linux-$(GO_ARCH).tar.gz | tar -C /usr/local -xz; \
curl -L https://go.dev/dl/go1.26.1.linux-$(GO_ARCH).tar.gz | tar -C /usr/local -xz; \
else \
echo "Go already installed in /usr/local/go"; \
fi
@echo "Go version: $$(go version)"
install-aws:
@echo "\n....Installing AWS CLI v2 for $(ARCH) ($(AWS_ARCH))...."
@echo -e "\n....Installing AWS CLI v2 for $(ARCH) ($(AWS_ARCH))...."
@if ! command -v aws >/dev/null 2>&1; then \
curl "https://awscli.amazonaws.com/awscli-exe-linux-$(AWS_ARCH).zip" -o "awscliv2.zip"; \
unzip -o awscliv2.zip; \
@@ -89,23 +139,100 @@ install-aws:
fi
quickinstall: install-system install-go install-aws
@echo "\n✅ Quickinstall complete for $(ARCH)."
@echo -e "\n✅ Quickinstall complete for $(ARCH)."
@echo " Please ensure /usr/local/go/bin is in your PATH."
@echo " Installed: Go ($(GO_ARCH)), AWS CLI ($(AWS_ARCH)), QEMU ($(QEMU_PACKAGES))"
security:
@echo "\n....Running security checks for $(GO_PROJECT_NAME)...."
# Format all Go files in place
format:
gofmt -w .
go tool govulncheck ./... > tests/govulncheck-report.txt || true
@echo "Govulncheck report saved to tests/govulncheck-report.txt"
# Check that all Go files are formatted (CI-compatible, fails on diff)
check-format:
@echo "Checking gofmt..."
@UNFORMATTED=$$(gofmt -l .); \
if [ -n "$$UNFORMATTED" ]; then \
echo "Files not formatted:"; \
echo "$$UNFORMATTED"; \
echo "Run 'make format' to fix."; \
exit 1; \
fi
@echo " gofmt ok"
go tool gosec ./... > tests/gosec-report.txt || true
@echo "Gosec report saved to tests/gosec-report.txt"
# Go vet (fails on issues, matches CI)
vet:
@echo "Running go vet..."
$(_Q)go vet ./...
@echo " go vet ok"
go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./... > tests/staticcheck-report.txt || true
@echo "Staticcheck report saved to tests/staticcheck-report.txt"
# Excluded: newexpr (replaces aws.String with new, not idiomatic for AWS SDK)
# Excluded: stringsbuilder (replaces string += in loops with strings.Builder, not worth the complexity for small loops)
GOFIX_EXCLUDE := -newexpr=false -stringsbuilder=false
go vet ./... 2>&1 | tee tests/govet-report.txt || true
@echo "Go vet report saved to tests/govet-report.txt"
# Apply go fix modernizations
modernize:
@echo "Applying go fix modernizations..."
go fix $(GOFIX_EXCLUDE) ./...
@echo " go fix applied"
.PHONY: build go_build go_run test bench run clean install-system install-go install-aws quickinstall security
# Check that code is modernized (CI-compatible, fails on diff)
check-modernize:
@echo "Checking go fix modernizations..."
@DIFF=$$(go fix $(GOFIX_EXCLUDE) -diff ./...); \
if [ -n "$$DIFF" ]; then \
echo "$$DIFF"; \
echo "Run 'make modernize' to fix."; \
exit 1; \
fi
@echo " go fix ok"
# Security checks — each tool fails the build on findings (matches CI).
# Reports are also saved to tests/ for review.
security-check:
@echo -e "\n....Running security checks for $(GO_PROJECT_NAME)...."
$(_Q)set -o pipefail && go tool govulncheck ./... $(_SECQ) tests/govulncheck-report.txt $(if $(QUIET),|| { cat tests/govulncheck-report.txt; exit 1; })
@echo " govulncheck ok"
$(_Q)set -o pipefail && go tool gosec -quiet -exclude=G204,G304,G402,G117,G703,G705,G706 -exclude-generated -exclude-dir=cmd ./... $(_SECQ) tests/gosec-report.txt $(if $(QUIET),|| { cat tests/gosec-report.txt; exit 1; })
@echo " gosec ok"
$(_Q)set -o pipefail && go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./... $(_SECQ) tests/staticcheck-report.txt $(if $(QUIET),|| { cat tests/staticcheck-report.txt; exit 1; })
@echo " staticcheck ok"
# Build release tarballs — use distro-ARCH for single arch, distro for both
distro: distro-amd64 distro-arm64
@echo ""
@echo "Distribution tarballs:"
@ls -lh dist/*.tar.gz
@echo ""
@cat dist/*.sha256
distro-amd64:
@echo "Building spinifex $(VERSION) linux/amd64..."
@mkdir -p dist/
docker buildx build \
--platform linux/amd64 \
--build-arg VERSION=$(VERSION) \
-f build/Dockerfile.distro \
--output type=local,dest=dist/amd64/ \
../
tar -czf dist/spinifex-$(VERSION)-linux-amd64.tar.gz -C dist/amd64 .
sha256sum dist/spinifex-$(VERSION)-linux-amd64.tar.gz > dist/spinifex-$(VERSION)-linux-amd64.tar.gz.sha256
distro-arm64:
@echo "Building spinifex $(VERSION) linux/arm64..."
@mkdir -p dist/
docker buildx build \
--platform linux/arm64 \
--build-arg VERSION=$(VERSION) \
-f build/Dockerfile.distro \
--output type=local,dest=dist/arm64/ \
../
tar -czf dist/spinifex-$(VERSION)-linux-arm64.tar.gz -C dist/arm64 .
sha256sum dist/spinifex-$(VERSION)-linux-arm64.tar.gz > dist/spinifex-$(VERSION)-linux-arm64.tar.gz.sha256
distro-clean:
rm -rf dist/
.PHONY: build build-ui go_build go_run preflight test test-cover test-race diff-coverage bench run clean \
install-system install-go install-aws quickinstall \
format check-format modernize check-modernize vet security-check \
distro distro-amd64 distro-arm64 distro-clean
+30 -35
View File
@@ -1,10 +1,10 @@
# Hive: An Open Source AWS-Compatible Stack for Bare-Metal, Edge, and On-Prem Deployments
# Spinifex: An Open Source AWS-Compatible Stack for Bare-Metal, Edge, and On-Prem Deployments
(Developer Preview) - Hive developed by [Mulga Defense Corporation](https://mulgadc.com/) is an open source infrastructure platform that brings the core services of AWS—like EC2, VPC, EBS, and S3—to environments where running in the cloud isn't an option. Whether you're deploying to edge sites, private data-centers, or need to operate in low-connectivity or highly contested environments, Hive gives you AWS-style workflows on your own hardware.
(Developer Preview) - Spinifex developed by [Mulga Defense Corporation](https://mulgadc.com/) is an open source infrastructure platform that brings the core services of AWS—like EC2, VPC, EBS, and S3—to environments where running in the cloud isn't an option. Whether you're deploying to edge sites, private data-centers, or need to operate in low-connectivity or highly contested environments, Spinifex gives you AWS-style workflows on your own hardware.
## What is Hive?
## What is Spinifex?
Hive replicates essential AWS primitives—virtual machines, block volumes, and object storage—using lightweight, self-contained components that are easy to deploy and integrate.
Spinifex replicates essential AWS primitives—virtual machines, block volumes, and object storage—using lightweight, self-contained components that are easy to deploy and integrate.
It’s designed for developers and operators who need:
@@ -13,13 +13,13 @@ It’s designed for developers and operators who need:
- Drop-in compatibility with tools like the AWS CLI, SDKs, and Terraform
- A secure cloud environment you control and own the entire hardware, network and software stack
You can run Hive on a few servers in a rack, a field site, or anywhere centralized cloud services aren’t feasible.
You can run Spinifex on a few servers in a rack, a field site, or anywhere centralized cloud services aren’t feasible.
## Core Components
### Hive (Compute Service – EC2 Alternative)
### Spinifex (Compute Service – EC2 Alternative)
Hive is a minimal VM orchestration layer built on top of QEMU, exposing APIs similar to EC2. It manages lifecycle operations like start, stop, and terminate, using QEMU’s QMP interface. Designed to be straightforward and scriptable, Hive lets you launch VMs using the AWS CLI, SDKs, or Terraform—without needing Kubernetes or heavyweight orchestrators. Keep in mind, you can also setup a Kubernetes environment using Hive with underlying instances.
Spinifex is a minimal VM orchestration layer built on top of QEMU, exposing APIs similar to EC2. It manages lifecycle operations like start, stop, and terminate, using QEMU’s QMP interface. Designed to be straightforward and scriptable, Spinifex lets you launch VMs using the AWS CLI, SDKs, or Terraform—without needing Kubernetes or heavyweight orchestrators. Keep in mind, you can also setup a Kubernetes environment using Spinifex with underlying instances.
- EC2-like VM management on bare metal
- Launches with cloud-init metadata support
@@ -52,44 +52,45 @@ AWS-Compatible Interfaces – Provision infrastructure with awscli, Terraform, o
- Works Offline – No reliance on centralized cloud services or external networks.
- Open Source – Licensed under Apache 2.0. Fork it, extend it, or deploy it as-is.
## Development Setup
## Installation
### For Developers
Installation requires an Ubuntu / Debian system. See the detailed documentation at [https://docs.mulgadc.com](https://docs.mulgadc.com/) for maintaining and installing Spinifex.
Get started with Hive development in minutes:
### Single node install
The installation is straightforward to set up and running on a single node for testing purposes.
```bash
# Clone the repository
git clone https://github.com/mulgadc/hive.git
cd hive
# Setup dependencies and development environment
./scripts/clone-deps.sh # Clone viperblock + predastore repositories
./scripts/dev-setup.sh # Setup complete development environment
export AWS_PROFILE=spinifex
export AWS_REGION=ap-southeast-2-sc
```
Next, follow the detailed [Installation Guide](./INSTALL.md) to bootstrap a new environment with SSH keys, AMI (OS images) and how to spin up new EC2 instances using Hive.
```bash
curl https://install.mulgadc.com/ | bash
spx admin init --region $AWS_REGION
aws ec2 describe-instance-types
```
### Development Setup
For a complete development environment see the [Source Install](https://docs.mulgadc.com/docs/source-install/) documentation
### Component Repositories
Hive coordinates these independent components:
Spinifex coordinates these independent components:
- **[Predastore](https://github.com/mulgadc/predastore)** - S3-compatible object storage
- **[Viperblock](https://github.com/mulgadc/viperblock)** - EBS-compatible block storage
Each component can be developed independently. See component-specific documentation for focused development guides.
## Detailed Installation method
See [Installation Guide](INSTALL.md) on how to setup a Hive node to get started.
Then configure your AWS CLI to point to Hive's endpoints and start launching VMs, buckets, and volumes using familiar commands and Terraform scripts.
## Development Philosophy
### Built by Engineers, For Engineers
Hive is developed by experienced infrastructure engineers with deep AWS expertise, including former AWS team members who understand the intricacies of building production-grade cloud services. Our team brings decades of combined experience from AWS, enterprise infrastructure, and edge computing environments.
Spinifex is developed by experienced infrastructure engineers with deep AWS expertise, including former AWS team members who understand the intricacies of building production-grade cloud services. Our team brings decades of combined experience from AWS, enterprise infrastructure, and edge computing environments.
**Real-World Experience:**
@@ -100,7 +101,7 @@ Hive is developed by experienced infrastructure engineers with deep AWS expertis
### AI-Assisted Development
While Hive is architected and implemented by experienced engineers, we leverage **Claude Code** (Anthropic's AI coding assistant) to accelerate certain development tasks. This approach combines human expertise with AI efficiency:
While Spinifex is architected and implemented by experienced engineers, we leverage **Claude Code** (Anthropic's AI coding assistant) to accelerate certain development tasks. This approach combines human expertise with AI efficiency:
**How We Use Claude Code:**
@@ -116,14 +117,8 @@ While Hive is architected and implemented by experienced engineers, we leverage
- **Performance Optimization**: Real-world performance tuning and benchmarking
- **Production Operations**: Deployment strategies and operational procedures
**Development Artifacts:**
- `CLAUDE.md` - Instructions for Claude Code when working on this codebase
- `HIVE_DEVELOPMENT_PLAN.md` - Comprehensive roadmap and implementation strategy
- Component-specific guidance in `viperblock/CLAUDE.md` and `predastore/CLAUDE.md`
This hybrid approach ensures Hive benefits from both proven engineering expertise and modern development acceleration, while maintaining the quality and reliability standards required for production infrastructure.
This hybrid approach ensures Spinifex benefits from both proven engineering expertise and modern development acceleration, while maintaining the quality and reliability standards required for production infrastructure.
## License
Hive is open source under the Apache 2.0 License. You're free to use, modify, and deploy it—anywhere you need reliable infrastructure without depending on centralized cloud platforms.
Spinifex is open source under the Apache 2.0 License. You're free to use, modify, and deploy it—anywhere you need reliable infrastructure without depending on centralized cloud platforms.
-444
View File
@@ -1,444 +0,0 @@
# Hive Development Roadmap
## Update Jan 2026
Big rocks:
- [DONE] Add multi-node support for predastore (S3) using reed solomon encoding
- [DONE] Implement KV store for object lookup, WAL files, offset. Use hash-ring to determine which nodes an object belongs to.
- Implement VPC support using Open vSwitch across multiple nodes, core VPC functionality included
- Add support for NVIDIA Bluefield DPU with Open vSwitch
- Implement basic IAM using NATS Jetstream as KV store, vs IAM/access-keys in local config/TOML files for beta.
- Move `daemon.go` instances.json state to Jetstream KV
- Add support using the `hive` CLI tool to provision a new user with AWS access-keys/IAM.
- Support multi-tenant operations and isolation
- Add support to include capabilities when adding a new hardware node to MulgaOS (e.g EC2 target, S3, EBS, NATs, etc) - Features can be turned on/off depending on hardware scope.
- [DONE] Add simple Web UI console, using the AWS JS SDK, communicating to local AWS gateway.
- Implement ShadCNblocks for UI framework
- Simple Go webserver, static files, easy build process.
Implementation gaps:
- EC2 - Support extended features for `run-instance`
- Volume resize of AMI. Note `nbd` does not support live resize events, instance needs to be stopped, resized, and started.
- Confirm cloud-init will resize volume on boot and configured correctly.
- Attach additional volumes
- Attach to VPC / Security group (required Open vSwitch implementation)
### Issues to investigate
- When the host disk volume is full and a new instance is launched, disk corruption will occur within the instance since the host volume is out of space. Consider a pre-flight check, check available storage on the S3 server, local viperblock for WAL cache availability and provide improved guard rails to prevent a new instance running, if available disk space is nearing to be exceeded.
Host:
```json
{"time":"2026-01-27T21:30:29.368164556+10:00","level":"ERROR","msg":"WAL sync failed","error":"write /home/hive/hive/predastore/distributed/nodes/node-2/0000000000000737.wal: no space left on device"}
```
Instance `dmesg`:
```
[ 1445.573495] EXT4-fs error (device vda1): ext4_journal_check_start:83: comm systemd-journal: Detected aborted journal
[ 1445.583565] EXT4-fs (vda1): Remounting filesystem read-only
```
### EC2
- [DONE] - describe-instances
- [DONE] - run-instances (count not implemented)
- [DONE] - start-instances
- [DONE] - stop-instances
- [DONE] - terminate-instances
- [DONE] - describe-instance-types
- [DONE] - create-key-pair
- [DONE] - delete-key-pair
- [DONE] - describe-key-pairs
- [DONE] - import-key-pair
- [DONE] - describe-images
- [DONE] - describe-regions
- [DONE] - describe-volumes
### To implement
Easier methods to implement
- attach-volume
- copy-image
- copy-snapshot
- copy-volumes
- create-image
- create-snapshot
- create-snapshots
- create-tags
- create-volume
- delete-snapshot
- describe-snapshots
- describe-subnets
- describe-tags
- describe-volume-attribute
- describe-volume-status
- detach-network-interface
- detach-volume
- get-console-output
- monitor-instances
TODO:
- allocate-address
- allocate-hosts
- assign-private-ip-addresses
- assign-private-nat-gateway-address
- associate-address
- associate-nat-gateway-address
- associate-route-server
- associate-route-table
- associate-security-group-vpc
- associate-subnet-cidr-block
- attach-internet-gateway
- attach-network-interface
- authorize-security-group-egress
- authorize-security-group-ingress
- create-customer-gateway
- create-default-subnet
- create-default-vpc
- create-dhcp-options
- create-egress-only-internet-gateway
- create-internet-gateway
- create-launch-template
- create-launch-template-version
- create-local-gateway-route
- create-local-gateway-route-table
- create-local-gateway-route-table-virtual-interface-group-association
- create-local-gateway-virtual-interface
- create-local-gateway-virtual-interface-group
- create-nat-gateway
- create-network-acl
- create-network-acl-entry
- create-public-ipv4-pool
- create-route
- create-route-server
- create-route-server-endpoint
- create-route-server-peer
- create-route-table
- create-security-group
- create-subnet
- create-subnet-cidr-reservation
- delete-coip-cidr
- delete-customer-gateway
- delete-dhcp-options
- delete-egress-only-internet-gateway
- delete-internet-gateway
- delete-local-gateway-route
- delete-local-gateway-route-table
- delete-local-gateway-route-table-virtual-interface-group-association
- delete-local-gateway-virtual-interface
- delete-local-gateway-virtual-interface-group
- delete-nat-gateway
- delete-network-acl
- delete-network-interface
- delete-network-interface-permission
- delete-public-ipv4-pool
- delete-route
- delete-route-server
- delete-route-server-endpoint
- delete-route-server-peer
- delete-route-table
- delete-security-group
- delete-subnet
- delete-subnet-cidr-reservation
- delete-tags
- delete-volume
- describe-account-attributes
- describe-addresses
- describe-customer-gateways
- describe-dhcp-options
- describe-egress-only-internet-gateways
- describe-hosts
- describe-internet-gateways
- describe-managed-prefix-lists
- describe-nat-gateways
- describe-network-acls
- describe-route-server-endpoints
- describe-route-server-peers
- describe-route-servers
- describe-route-tables
- describe-security-group-references
- describe-security-group-rules
- describe-security-groups
- describe-volumes-modifications
- detach-internet-gateway
- detach-vpn-gateway
- disable-image
- disable-route-server-propagation
- disable-serial-console-access
- enable-address-transfer
- enable-capacity-manager
- enable-ebs-encryption-by-default
- enable-image
- enable-image-block-public-access
- export-image
- get-console-screenshot
- get-instance-metadata-defaults
- get-instance-tpm-ek-pub
- get-instance-types-from-instance-requirements
- get-instance-uefi-data
- get-launch-template-data
- get-route-server-associations
- get-route-server-propagations
- get-route-server-routing-database
- get-security-groups-for-vpc
- get-serial-console-access-status
- get-snapshot-block-public-access-state
- get-spot-placement-scores
- get-subnet-cidr-reservations
- import-image
- import-snapshot
- modify-instance-attribute
- modify-instance-capacity-reservation-attributes
- modify-instance-connect-endpoint
- modify-instance-cpu-options
- modify-instance-credit-specification
- modify-instance-placement
- modify-network-interface-attribute
- modify-volume
- modify-volume-attribute
- modify-vpn-connection
- modify-vpn-connection-options
- reboot-instances
- register-image
- release-address
- release-hosts
- replace-network-acl-association
- replace-network-acl-entry
- replace-route
- replace-route-table-association
- report-instance-status
- revoke-security-group-egress
- revoke-security-group-ingress
- run-instances
### EBS
- complete-snapshot
- get-snapshot-block
- list-changed-blocks
- list-snapshot-blocks
- put-snapshot-block
- start-snapshot
### VPC
- create-vpc
- create-vpc-block-public-access-exclusion
- create-vpc-encryption-control
- create-vpc-endpoint
- create-vpc-endpoint-connection-notification
- create-vpc-endpoint-service-configuration
- create-vpc-peering-connection
- delete-vpc
- delete-vpc-block-public-access-exclusion
- delete-vpc-encryption-control
- delete-vpc-endpoint-connection-notifications
- delete-vpc-endpoint-service-configurations
- delete-vpc-endpoints
- delete-vpc-peering-connection
- associate-vpc-cidr-block
- delete-local-gateway-route-table-vpc-association
- create-local-gateway-route-table-vpc-association
- describe-security-group-vpc-associations
- describe-vpc-attribute
- describe-vpc-block-public-access-exclusions
- describe-vpc-block-public-access-options
- describe-vpc-encryption-controls
- describe-vpc-endpoint-associations
- describe-vpc-endpoint-connection-notifications
- describe-vpc-endpoint-connections
- describe-vpc-endpoint-service-configurations
- describe-vpc-endpoint-service-permissions
- describe-vpc-endpoint-services
- describe-vpc-endpoints
- describe-vpc-peering-connections
- describe-vpcs
- modify-vpc-attribute
- modify-vpc-encryption-control
- modify-vpc-endpoint
- modify-vpc-endpoint-connection-notification
- modify-vpc-endpoint-service-configuration
- modify-vpc-tenancy
### S3
- Consider moving S3 control/data plane, from predastore, to Hive format.
## Update Nov 2025
[PARTIAL] _ Implement multi-tenant support
[PARTIAL] _ Move config settings from ~/hive/\*.toml, to using Nats Jetstream for core config which can be synced between nodes.
- Implement a lightweight IAM using Nats Jetstream, vs current config files for auth settings.
[ONGOING] _ Implement Reed Solomon Encoding for Predastore (S3) objects and a WAL implementation, for storing multiple objects in a single shard (e.g 4MB), with the WAL referencing the location of each object (e.g 4kb min)
_ Implement basic KY lookup, object key sha512 (bucket/key), location to shard on S3 (obj.0000124.bin), read WAL (e.g first 4096 bytes) to determine location of the object. e.g key-1234 => obj.0000124.bin, wal header (4096 bytes) > key-1234 == offset location (seek) 16384, len 32768.
_ Read multiple nodes (e.g 5 predastore instances, k = 3 (data shards), n = 5 (total shards), n - k = 2 (parity shards) )
[ONGOING] _ Complete core scaffolding AWS SDK/API requirements (ec2 describe-instances, run-instances, etc)
- Implement UEFI support for image downloads and `qemu` exec in `vm.go`
- Confirm Alpine Linux, fails import image AMI > (run-instance) ""Failed to read block from AMI source" err="request out of range" - Block size correct?
- Improve shutdown gracefully, `./scripts/stop-dev.sh` waits 60 seconds, while qemu/nbd could still be shutting down.
- Add delete-volume support via EBS (s3 vol-\*) for terminated instance
- Add default LRU cache support for viperblock, depending on the instance type / volume size and system memory available.
## Multi-node setup
[DONE] - Development release - v0.1.0 (single node)
[PARTIAL] - Production release - v0.2.0 (multi node)
Design layout for multi-node configuration.
```
# node1
hive admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 3 --hive-dir ~/hive/
# node2
hive admin join --region ap-southeast-2 --az ap-southeast-2a --node node2 --host node1.local:8443 --data-dir ~/hive/
# node3, optionally toggle EBS/EC2/NATs support only
hive admin join --region ap-southeast-2 --az ap-southeast-2a --node node3 --host node1.local:8443 --data-dir ~/hive/ --cap ebs,ec2,nats
```
If --host is missing, the join command tries multicast broadcast to find parent (leader) node.
Set region with `--region ap-southeast-2` which will create a new Hive cluster for the specified region.
For production, it is recommended to run Hive on at least three physical nodes. In this minimum setup, Hive S3 and EBS volumes use 2 data shards and 1 parity shard RS(2,1) to distribute each chunk across the cluster. This configuration tolerates a full node failure while adding only 1.5x storage overhead.
### Overview:
- init creates a cluster-id, node-id, and a short join token, starts a tiny control server on :8443, and writes DNS hint files if available.
- init sets target size to 3 and waits until all 3 nodes join and ack.
- join nodes contact node1, present the token or cluster-id, and advertise capabilities.
- node1 appends them to the member set and immediately pushes the current settings bundle to them.
- When member count reaches 3, node1 broadcasts the final cluster settings to all nodes.
- Every node writes the same cluster.json and starts the services that match its capabilities.
Node exchange payload
```json
{
"cluster_id": "c-82d5",
"node_id": "n-rpi2",
"addr": "rpi2.local:8443",
"version": "0.1.0",
"caps": ["ec2", "s3"],
"ts": 1731388800
}
```
Settings bundle (identical on all 3 once committed)
```json
{
"cluster_id": "c-82d5",
"target_size": 3,
"members": [
{
"node_id": "n-rpi1",
"addr": "rpi1.local:8443",
"caps": ["ec2", "s3", "nats", "ebs"]
},
{ "node_id": "n-rpi2", "addr": "rpi2.local:8443", "caps": ["ec2", "s3"] },
{ "node_id": "n-rpi3", "addr": "rpi3.local:8443", "caps": ["nats", "ebs"] }
],
"services": {
"ec2": { "api_bind": ":9001" },
"s3": { "api_bind": ":9002", "replicas": 2 },
"nats": { "cluster": "c-82d5", "quorum": 2 },
"ebs": { "replicas": 3, "block_size": 4096 }
},
"epoch": 1,
"sig": "ed25519:..." // signed by rpi1 during init
}
```
## Original TODO Items → Development Plan Integration
### ✅ **Completed Integration**
All original TODO items have been incorporated into the structured development phases:
| Original TODO | Status | Integrated Into | Phase |
| ------------------------------------- | ---------- | ---------------------------------------------- | --------- |
| #1: Binary compile and install.sh | ✅ Planned | Phase 7: Task 7.1 (Binary Compilation) | Phase 7 |
| #2: Move daemon.go to services/hived/ | ✅ Planned | Phase 3: Task 3.2 (Specialized Services) | Phase 3 |
| #3: VPC with openvs-switch as `vpcd` | ✅ Planned | Phase 5: Task 5.1 (VPC with Open vSwitch) | Phase 5 |
| #4: AWS HTTP gateway (`awsd`) | ✅ Planned | Phase 3: Task 3.1 (AWS Gateway Service) | Phase 3 |
| #5: AWS SDK v2 input/output | ✅ Planned | Phase 2: Task 2.1 & Phase 3: Task 3.1 | Phase 2-3 |
| #6: `hive.service` for systemd | ✅ Planned | Phase 7: Task 7.2 (System Service Integration) | Phase 7 |
| #7: Gossip and RAFT protocols | ✅ Planned | Phase 0: Task 0.1 (Service Registry) | Phase 0 |
| #8: etcd/KV for configuration sync | ✅ Planned | Phase 0: Task 0.1 (Distributed Config) | Phase 0 |
| #9: Smithy model code generation | ✅ Planned | Phase 2: Task 2.1 (Smithy Code Generation) | Phase 2 |
## Development Phase Overview
**Phase 0**: Distributed Systems Foundation (1-2 weeks)
- Gossip, RAFT, and distributed configuration (#7, #8)
**Phase 1**: Development Environment Automation (2-3 weeks)
- Multi-service orchestration and hot reloading
**Phase 2**: AWS API Model Implementation (3-4 weeks)
- Smithy-based code generation (#9)
- AWS SDK v2 integration (#5)
**Phase 3**: Scalable Gateway and Daemon Architecture (2-3 weeks)
- AWS gateway service `awsd` (#4)
- Service refactoring to `hived` (#2)
- VPC daemon `vpcd` foundation (#3)
**Phase 4**: Service Integration and Orchestration (3-4 weeks)
- NATS clustering and cross-service coordination
**Phase 5**: Infrastructure Services (4-5 weeks)
- VPC networking with Open vSwitch (#3)
- Advanced VM features
**Phase 6**: Testing and Validation Framework (2-3 weeks)
- AWS CLI compatibility testing
**Phase 7**: Production Deployment and Packaging (2-3 weeks)
- Binary compilation and distribution (#1)
- Systemd service integration (#6)
- Production configuration management
## Service Architecture
The development plan implements these services:
- **`awsd`** - AWS API Gateway (TODO #4)
- **`hived`** - Main Hive compute daemon (TODO #2)
- **`vpcd`** - VPC networking daemon (TODO #3)
- **Predastore** - S3 service (existing)
- **Viperblock** - EBS service (existing)
## Getting Started
For development setup:
```bash
./scripts/dev-setup.sh # Setup complete development environment
./scripts/start-dev.sh # Start all services
```
For detailed implementation guidance, see [HIVE_DEVELOPMENT_PLAN.md](HIVE_DEVELOPMENT_PLAN.md).
+48
View File
@@ -0,0 +1,48 @@
ARG VERSION=dev
FROM golang:1.26-bookworm AS builder
ARG VERSION
RUN apt-get update && apt-get install -y --no-install-recommends \
nbdkit nbdkit-plugin-dev pkg-config gcc make \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build
# Copy module files for dependency caching
COPY spinifex/go.mod spinifex/go.sum ./spinifex/
COPY predastore/go.mod predastore/go.sum ./predastore/
COPY viperblock/go.mod viperblock/go.sum ./viperblock/
COPY viperblock/nbd/go.mod viperblock/nbd/go.sum ./viperblock/nbd/
COPY viperblock/nbd/libguestfs.org/nbdkit/go.mod ./viperblock/nbd/libguestfs.org/nbdkit/
# Generate go.work (gitignored, so not in checkout)
RUN cd spinifex && go work init . ../predastore ../viperblock ../viperblock/nbd ../viperblock/nbd/libguestfs.org/nbdkit
# Download dependencies (cached layer)
RUN cd spinifex && go mod download
# Copy full source
COPY spinifex/ ./spinifex/
COPY predastore/ ./predastore/
COPY viperblock/ ./viperblock/
# Build spinifex binary (spx)
RUN COMMIT=$(cd spinifex && git rev-parse --short HEAD 2>/dev/null || echo "unknown") && \
cd spinifex && go build -ldflags "-s -w \
-X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Version=${VERSION} \
-X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Commit=${COMMIT}" \
-o /out/spx cmd/spinifex/main.go
# Build nbdkit viperblock plugin (.so)
RUN cd viperblock && go build -o /out/nbdkit-viperblock-plugin.so -buildmode=c-shared nbd/viperblock.go
# Copy setup scripts and systemd units
RUN cp spinifex/scripts/setup.sh /out/setup.sh 2>/dev/null || true
RUN cp spinifex/scripts/setup-ovn.sh /out/setup-ovn.sh
RUN mkdir -p /out/systemd && cp spinifex/build/systemd/* /out/systemd/
RUN cp spinifex/build/logrotate/spinifex /out/logrotate-spinifex
FROM scratch AS export
COPY --from=builder /out/ /
+9
View File
@@ -0,0 +1,9 @@
/var/log/spinifex/*.log {
daily
rotate 14
compress
delaycompress
missingok
notifempty
copytruncate
}
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=Spinifex AWS Gateway
PartOf=spinifex.target
After=spinifex-nats.service spinifex-daemon.service
Requires=spinifex-nats.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStartPre=/bin/sh -c 'for i in $(seq 1 30); do nc -z 127.0.0.1 4222 && exit 0; sleep 1; done; exit 1'
ExecStart=/usr/local/bin/spx service awsgw start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/
Environment=SPINIFEX_AWSGW_TLS_CERT=/etc/spinifex/server.pem
Environment=SPINIFEX_AWSGW_TLS_KEY=/etc/spinifex/server.key
[Install]
WantedBy=spinifex.target
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=Spinifex Daemon (VM Orchestration)
PartOf=spinifex.target
After=spinifex-nats.service spinifex-viperblock.service
Requires=spinifex-nats.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStartPre=/bin/sh -c 'for i in $(seq 1 30); do nc -z 127.0.0.1 4222 && exit 0; sleep 1; done; exit 1'
ExecStart=/usr/local/bin/spx service spinifex start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/spinifex/
Environment=SPINIFEX_WAL_DIR=/var/lib/spinifex/spinifex/
[Install]
WantedBy=spinifex.target
+17
View File
@@ -0,0 +1,17 @@
[Unit]
Description=Spinifex NATS Message Bus
PartOf=spinifex.target
Before=spinifex-predastore.service spinifex-daemon.service spinifex-awsgw.service spinifex-vpcd.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStart=/usr/local/bin/spx service nats start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/nats/nats.conf
[Install]
WantedBy=spinifex.target
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=Spinifex Predastore (S3-compatible Object Storage)
PartOf=spinifex.target
After=spinifex-nats.service
Requires=spinifex-nats.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStartPre=/bin/sh -c 'for i in $(seq 1 30); do nc -z 127.0.0.1 4222 && exit 0; sleep 1; done; exit 1'
ExecStart=/usr/local/bin/spx service predastore start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
Environment=SPINIFEX_PREDASTORE_CONFIG_PATH=/etc/spinifex/predastore/predastore.toml
Environment=SPINIFEX_PREDASTORE_BASE_PATH=/var/lib/spinifex/predastore/
Environment=SPINIFEX_PREDASTORE_TLS_CERT=/etc/spinifex/server.pem
Environment=SPINIFEX_PREDASTORE_TLS_KEY=/etc/spinifex/server.key
[Install]
WantedBy=spinifex.target
+16
View File
@@ -0,0 +1,16 @@
[Unit]
Description=Spinifex Web UI
PartOf=spinifex.target
After=spinifex-awsgw.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStart=/usr/local/bin/spx service spinifex-ui start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
[Install]
WantedBy=spinifex.target
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=Spinifex Viperblock (Block Storage)
PartOf=spinifex.target
After=spinifex-nats.service spinifex-predastore.service
Requires=spinifex-nats.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStartPre=/bin/sh -c 'for i in $(seq 1 30); do nc -z 127.0.0.1 4222 && exit 0; sleep 1; done; exit 1'
ExecStart=/usr/local/bin/spx service viperblock start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
EnvironmentFile=/etc/spinifex/systemd.env
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
Environment=SPINIFEX_BASE_DIR=/var/lib/spinifex/viperblock/
[Install]
WantedBy=spinifex.target
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=Spinifex VPC Daemon (OVN Networking)
PartOf=spinifex.target
After=spinifex-nats.service spinifex-daemon.service
Requires=spinifex-nats.service
[Service]
Type=simple
User=spinifex
Group=spinifex
ExecStartPre=/bin/sh -c 'for i in $(seq 1 30); do nc -z 127.0.0.1 4222 && exit 0; sleep 1; done; exit 1'
ExecStart=/usr/local/bin/spx service vpcd start
Restart=on-failure
RestartSec=5
OOMScoreAdjust=-500
Environment=SPINIFEX_CONFIG_PATH=/etc/spinifex/spinifex.toml
[Install]
WantedBy=spinifex.target
+8
View File
@@ -0,0 +1,8 @@
[Unit]
Description=Spinifex Infrastructure Platform
After=network-online.target
Wants=network-online.target
Wants=spinifex-nats.service spinifex-predastore.service spinifex-viperblock.service spinifex-daemon.service spinifex-awsgw.service spinifex-vpcd.service spinifex-ui.service
[Install]
WantedBy=multi-user.target
-878
View File
@@ -1,878 +0,0 @@
/*
Copyright © 2025 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cmd
import (
"bytes"
"crypto/tls"
_ "embed"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
"github.com/mulgadc/hive/hive/admin"
"github.com/mulgadc/hive/hive/config"
"github.com/mulgadc/hive/hive/utils"
"github.com/mulgadc/viperblock/viperblock"
"github.com/mulgadc/viperblock/viperblock/backends/s3"
"github.com/mulgadc/viperblock/viperblock/v_utils"
"github.com/pelletier/go-toml/v2"
"github.com/pterm/pterm"
"github.com/spf13/cobra"
"golang.org/x/net/http2"
)
//go:embed templates/hive.toml
var hiveTomlTemplate string
//go:embed templates/awsgw.toml
var awsgwTomlTemplate string
//go:embed templates/predastore.toml
var predastoreTomlTemplate string
//go:embed templates/nats.conf
var natsConfTemplate string
var supportedArchs = map[string]bool{
"x86_64": true,
"aarch64": true, // alias for arm64
"arm64": true,
}
// TODO: Confirm suppported platform types
var supportedPlatforms = map[string]bool{
"Linux/UNIX": true,
"Windows": true,
}
var adminCmd = &cobra.Command{
Use: "admin",
Short: "Administrative commands for Hive platform management",
Long: `Administrative commands for initializing and managing the Hive platform.`,
}
var adminInitCmd = &cobra.Command{
Use: "init",
Short: "Initialize Hive platform configuration",
Long: `Initialize Hive platform by creating configuration files, generating SSL certificates,
and setting up AWS credentials. This creates the necessary directory structure and
configuration files in ~/hive/config.`,
Run: runAdminInit,
}
var adminJoinCmd = &cobra.Command{
Use: "join",
Short: "Join an existing Hive cluster",
Long: `Join an existing Hive cluster by connecting to a leader node and retrieving
the cluster configuration. This command will configure the local node to join
the cluster and participate in distributed operations.`,
Run: runAdminJoin,
}
var imagesCmd = &cobra.Command{
Use: "images",
Short: "Manage OS images",
Long: `Manage OS images for local storage and AMI creation.`,
}
var imagesImportCmd = &cobra.Command{
Use: "import",
Short: "Specify local file to import",
Long: `Create a new image from a local file`,
Run: runimagesImportCmd,
}
var imagesListCmd = &cobra.Command{
Use: "list",
Short: "List OS images to import or download",
Long: `Query the remote endpoint for common OS images available for import as AMI or locally download.`,
Run: runimagesListCmd,
}
/*
CLI ideas
hive admin images list
- fetches from remote endpoint for common/trusted images to bootstrap environment, or baked in from compile.
// If --name specified, download
hive admin images import --name debian-12-x86_64
// List available images
hive admin images list
// Manually import a path
hive admin images import --file /path/to/image --distro debian --version 12 --arch x86_64
-> x <-
*/
func init() {
rootCmd.AddCommand(adminCmd)
adminCmd.AddCommand(adminInitCmd)
adminCmd.AddCommand(adminJoinCmd)
adminCmd.AddCommand(imagesCmd)
imagesCmd.AddCommand(imagesImportCmd)
imagesCmd.AddCommand(imagesListCmd)
homeDir, _ := os.UserHomeDir()
configDir := fmt.Sprintf("%s/hive/config", homeDir)
hiveDir := fmt.Sprintf("%s/hive/", homeDir)
rootCmd.PersistentFlags().String("config-dir", configDir, "Configuration directory")
rootCmd.PersistentFlags().String("hive-dir", hiveDir, "Hive base directory")
// Flags for admin init
adminInitCmd.Flags().Bool("force", false, "Force re-initialization (overwrites existing config)")
adminInitCmd.Flags().String("region", "ap-southeast-2", "Mulga region to create")
adminInitCmd.Flags().String("az", "ap-southeast-2a", "Mulga AZ to create")
adminInitCmd.Flags().String("node", "node1", "Node name, increment for additional nodes (default, node1)")
adminInitCmd.Flags().Int("nodes", 3, "Number of nodes to expect for cluster")
adminInitCmd.Flags().String("host", "", "Leader node to join (if not specified, tries multicast discovery)")
adminInitCmd.Flags().Int("port", 4432, "Port to bind cluster services on")
adminInitCmd.Flags().String("bind", "0.0.0.0", "IP address to bind services to (e.g., 10.11.12.1 for multi-node)")
adminInitCmd.Flags().String("cluster-bind", "", "IP address to bind NATS cluster services to (e.g., 10.11.12.1 for multi-node)")
adminInitCmd.Flags().String("cluster-routes", "", "NATS cluster hosts for routing specify multiple with comma (e.g., 10.11.12.1:4248,10.11.12.2:4248 for multi-node)")
// Flags for admin join
adminJoinCmd.Flags().String("region", "ap-southeast-2", "Region for this node")
adminJoinCmd.Flags().String("az", "ap-southeast-2a", "Availability zone for this node")
adminJoinCmd.Flags().String("node", "", "Node name (required)")
adminJoinCmd.Flags().String("host", "", "Leader node host:port (e.g., node1.local:4432) (required)")
adminJoinCmd.Flags().String("data-dir", "", "Data directory for this node (default: ~/hive)")
adminJoinCmd.Flags().Int("port", 4432, "Port to bind cluster services on")
adminJoinCmd.Flags().String("bind", "0.0.0.0", "IP address to bind services to (e.g., 10.11.12.2 for multi-node on single host)")
adminJoinCmd.Flags().String("cluster-bind", "", "IP address to bind NATS cluster services to (e.g., 10.11.12.1 for multi-node)")
adminJoinCmd.Flags().String("cluster-routes", "", "NATS cluster hosts for routing specify multiple with comma (e.g., 10.11.12.1:4248,10.11.12.2:4248 for multi-node)")
adminJoinCmd.MarkFlagRequired("node")
adminJoinCmd.MarkFlagRequired("host")
imagesImportCmd.Flags().String("tmp-dir", os.TempDir(), "Temporary directory for image import processing")
imagesImportCmd.Flags().String("name", "", "Import specified image by name")
imagesImportCmd.Flags().String("file", "", "Import file from specified path (raw, qcow2, compressed)")
imagesImportCmd.Flags().String("distro", "", "Specified distro name (e.g debian)")
imagesImportCmd.Flags().String("version", "", "Specified distro version (e.g 12)")
imagesImportCmd.Flags().String("arch", "", "Specified distro arch (e.g aarch64, arm64, x86_64)")
imagesImportCmd.Flags().String("platform", "Linux/UNIX", "Specified platform (e.g Linux/UNIX, Windows)")
imagesImportCmd.Flags().Bool("force", false, "Force command execution (overwrites existing files)")
}
func runimagesImportCmd(cmd *cobra.Command, args []string) {
var image utils.Images
var imageFile string
var imageStat os.FileInfo
var err error
cfgFile, _ := cmd.Flags().GetString("config")
forceCmd, _ := cmd.Flags().GetBool("force")
ostmpDir, _ := cmd.Flags().GetString("tmp-dir")
// Use default config path
if cfgFile == "" {
homeDir, _ := os.UserHomeDir()
cfgFile = fmt.Sprintf("%s/hive/config/hive.toml", homeDir)
}
//configDir, _ := cmd.Flags().GetString("config-dir")
baseDir, _ := cmd.Flags().GetString("hive-dir")
// Strip trailing slash
baseDir = filepath.Clean(baseDir)
// Check the base dir has our images path, and correctlty init
imageDir := fmt.Sprintf("%s/images", baseDir)
if !admin.FileExists(imageDir) {
fmt.Fprintf(os.Stderr, "Image directory does not exist. Base path specified correctly? %s", imageDir)
os.Exit(1)
}
// Determine, if name specified, or file
imageName, _ := cmd.Flags().GetString("name")
if imageName != "" {
var exists bool
// Confirm the image exists
image, exists = utils.AvailableImages[imageName]
if !exists {
fmt.Fprintf(os.Stderr, "Image name not found in available images")
os.Exit(1)
}
} else {
imageFile, _ = cmd.Flags().GetString("file")
if imageFile == "" {
fmt.Fprintf(os.Stderr, "File required to import image")
os.Exit(1)
}
//imageStat, err = os.Stat(imageFile)
if err != nil {
fmt.Fprintf(os.Stderr, "File could not be found %s", err)
os.Exit(1)
}
image.Distro, _ = cmd.Flags().GetString("distro")
image.Version, _ = cmd.Flags().GetString("version")
image.Arch, _ = cmd.Flags().GetString("arch")
image.Platform, _ = cmd.Flags().GetString("platform")
}
if image.Distro == "" {
fmt.Fprintf(os.Stderr, "Specify distro name")
os.Exit(1)
}
// Check version specified
if image.Version == "" {
fmt.Fprintf(os.Stderr, "Specify image version")
os.Exit(1)
}
if !supportedArchs[image.Arch] {
fmt.Fprintf(os.Stderr, "Unsupported architecture")
os.Exit(1)
}
if !supportedPlatforms[image.Platform] {
fmt.Fprintf(os.Stderr, "Unsupported platform")
os.Exit(1)
}
// Create the specified image directory
imagePath := fmt.Sprintf("%s/%s/%s/%s", imageDir, image.Distro, image.Version, image.Arch)
// Create config directory
if err := os.MkdirAll(imagePath, 0700); err != nil {
fmt.Fprintf(os.Stderr, "Error creating config directory: %v\n", err)
os.Exit(1)
}
fmt.Printf("✅ Created config directory: %s\n", imagePath)
// Next, if the file is selected to download, fetch it, extract disk image, and save to path
if imageName != "" {
// Download the file to the image path
filename := path.Base(image.URL)
imageFile = fmt.Sprintf("%s/%s", imagePath, filename)
fmt.Printf("Downloading image %s to %s\n", image.URL, imageFile)
// If image path exists, skip
if admin.FileExists(imageFile) && !forceCmd {
fmt.Printf("Image file already exists, skipping download, use --force to overwrite: %s\n", imageFile)
} else {
err := utils.DownloadFileWithProgress(image.URL, image.Name, imageFile, 0)
if err != nil {
fmt.Printf("Download failed: %v\n", err)
os.Exit(1)
}
}
// Update image file path for later extraction
//imagePath = imageFilePath
}
// Next, validate if the image is raw, tar, gz, xv, etc. We need to upload the raw image
tmpDir, err := os.MkdirTemp(ostmpDir, "hive-image-tmp-*")
if err != nil {
fmt.Fprintf(os.Stderr, "Could not create temp dir: %v\n", err)
os.Exit(1)
}
extractedImagePath, err := utils.ExtractDiskImageFromFile(imageFile, imagePath)
fmt.Println("Extracted image to:", extractedImagePath)
if err != nil {
fmt.Fprintf(os.Stderr, "Could not extract image: %v\n", err)
os.Exit(1)
}
imageStat, err = os.Stat(extractedImagePath)
if err != nil {
fmt.Fprintf(os.Stderr, "Could not stat image: %v\n", err)
os.Exit(1)
}
// Create the specified manifest file to describe the image/AMI
manifest := viperblock.VolumeConfig{}
// Calculate the size
manifest.AMIMetadata.Name = fmt.Sprintf("ami-%s-%s-%s", image.Distro, image.Version, image.Arch)
volumeId := viperblock.GenerateVolumeID("ami", manifest.AMIMetadata.Name, "", time.Now().Unix()) // TODO: Replace with bucket
manifest.AMIMetadata.ImageID = volumeId
manifest.AMIMetadata.Description = fmt.Sprintf("%s cloud image prepared for Hive", manifest.AMIMetadata.Name)
manifest.AMIMetadata.Architecture = image.Arch
manifest.AMIMetadata.PlatformDetails = image.Platform
manifest.AMIMetadata.CreationDate = time.Now()
manifest.AMIMetadata.RootDeviceType = "ebs"
manifest.AMIMetadata.Virtualization = "hvm"
manifest.AMIMetadata.ImageOwnerAlias = "system"
manifest.AMIMetadata.VolumeSizeGiB = utils.SafeInt64ToUint64(imageStat.Size() / 1024 / 1024 / 1024)
// Volume Data
manifest.VolumeMetadata.VolumeID = volumeId // TODO: Confirm if unique, e.g vol-, if ami- used
manifest.VolumeMetadata.VolumeName = manifest.AMIMetadata.Name
manifest.VolumeMetadata.TenantID = "system"
manifest.VolumeMetadata.SizeGiB = manifest.AMIMetadata.VolumeSizeGiB
manifest.VolumeMetadata.State = "available"
manifest.VolumeMetadata.AvailabilityZone = "" // TODO: Confirm
manifest.VolumeMetadata.CreatedAt = time.Now()
manifest.VolumeMetadata.VolumeType = "gp3"
manifest.VolumeMetadata.IOPS = 1000
// Write the manifest to disk
// Save as JSON
jsonData, err := json.Marshal(manifest)
if err != nil {
fmt.Fprintf(os.Stderr, "Could not marshal manifest: %v\n", err)
os.Exit(1)
}
manifestFilename := fmt.Sprintf("%s/%s.json", imagePath, manifest.AMIMetadata.Name)
// Write to file
err = os.WriteFile(manifestFilename, jsonData, 0644)
if err != nil {
fmt.Fprintf(os.Stderr, "Could not write manifest: %v\n", err)
os.Exit(1)
}
// Upload the image to S3 (predastore)
appConfig, err := config.LoadConfig(cfgFile)
if err != nil {
fmt.Println("Error loading config file:", err)
return
}
s3Config := s3.S3Config{
VolumeName: volumeId,
VolumeSize: utils.SafeInt64ToUint64(imageStat.Size()),
Bucket: appConfig.Nodes[appConfig.Node].Predastore.Bucket,
Region: appConfig.Nodes[appConfig.Node].Predastore.Region,
AccessKey: appConfig.Nodes[appConfig.Node].AccessKey,
SecretKey: appConfig.Nodes[appConfig.Node].SecretKey,
Host: appConfig.Nodes[appConfig.Node].Predastore.Host,
}
vbConfig := viperblock.VB{
VolumeName: volumeId,
VolumeSize: utils.SafeInt64ToUint64(imageStat.Size()),
BaseDir: tmpDir,
Cache: viperblock.Cache{
Config: viperblock.CacheConfig{
Size: 0,
},
},
VolumeConfig: manifest,
}
err = v_utils.ImportDiskImage(&s3Config, &vbConfig, extractedImagePath)
if err != nil {
fmt.Fprintf(os.Stderr, "Could not import image to predastore: %v\n", err)
os.Exit(1)
}
defer os.RemoveAll(tmpDir)
fmt.Printf("✅ Image import complete. Image-ID (AMI): %s\n", volumeId)
}
// List remote images available
func runimagesListCmd(cmd *cobra.Command, args []string) {
//fmt.Println(availableImages)
tableData := pterm.TableData{
{"NAME", "DISTRO", "VERSION", "ARCH", "BOOT"},
}
// Sort A .. Z
// 1. Collect keys
keys := make([]string, 0, len(utils.AvailableImages))
for k := range utils.AvailableImages {
keys = append(keys, k)
}
// 2. Sort keys alphabetically (A→Z)
sort.Strings(keys)
// 3. Iterate in sorted order
for _, k := range keys {
img := utils.AvailableImages[k]
//for _, img := range utils.AvailableImages {
tableData = append(tableData, []string{img.Name, img.Distro, img.Version, img.Arch, img.BootMode})
}
// Create a table with the defined data.
// The table has a header and the text in the cells is right-aligned.
// The Render() method is used to print the table to the console.
pterm.DefaultTable.WithHasHeader().WithLeftAlignment().WithData(tableData).Render()
pterm.Println("To install a selected image as an AMI use:")
pterm.Println("hive admin images import --name <image-name>")
}
// TODO: Move all logic to a module, use minimal application logic in viper commands
func runAdminInit(cmd *cobra.Command, args []string) {
force, _ := cmd.Flags().GetBool("force")
configDir, _ := cmd.Flags().GetString("config-dir")
hiveRoot, _ := cmd.Flags().GetString("hive-dir")
region, _ := cmd.Flags().GetString("region")
az, _ := cmd.Flags().GetString("az")
node, _ := cmd.Flags().GetString("node")
port, _ := cmd.Flags().GetInt("port")
bindIP, _ := cmd.Flags().GetString("bind")
clusterBind, _ := cmd.Flags().GetString("cluster-bind")
clusterRoutesStr, _ := cmd.Flags().GetString("cluster-routes")
clusterRoutes := strings.Split(clusterRoutesStr, ",")
// Validate IP address format
if net.ParseIP(bindIP) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: Invalid IP address for --bind: %s\n", bindIP)
os.Exit(1)
}
// Validate port range
if port < 1 || port > 65535 {
fmt.Fprintf(os.Stderr, "❌ Error: Port must be between 1 and 65535, got: %d\n", port)
os.Exit(1)
}
fmt.Printf("Initializing Hive with bind IP: %s, port: %d\n", bindIP, port)
// Default config directory
if configDir == "" {
homeDir, err := os.UserHomeDir()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting home directory: %v\n", err)
os.Exit(1)
}
configDir = filepath.Join(homeDir, "hive", "config")
}
fmt.Println("🚀 Initializing Hive platform...")
fmt.Printf("Configuration directory: %s\n\n", configDir)
// Check if already initialized
hiveTomlPath := filepath.Join(configDir, "hive.toml")
if !force && admin.FileExists(hiveTomlPath) {
fmt.Println("⚠️ Hive already initialized!")
fmt.Printf("Config file exists: %s\n", hiveTomlPath)
fmt.Println("\nTo re-initialize, run with --force flag:")
fmt.Println(" hive admin init --force")
os.Exit(0)
}
// Create config directory
if err := os.MkdirAll(configDir, 0700); err != nil {
fmt.Fprintf(os.Stderr, "Error creating config directory: %v\n", err)
os.Exit(1)
}
fmt.Printf("✅ Created config directory: %s\n", configDir)
// Generate AWS credentials
accessKey := admin.GenerateAWSAccessKey()
secretKey := admin.GenerateAWSSecretKey()
accountID := admin.GenerateAccountID()
fmt.Println("\n🔑 Generated AWS credentials:")
fmt.Printf(" Access Key: %s\n", accessKey)
fmt.Printf(" Secret Key: %s\n", secretKey)
fmt.Printf(" Account ID: %s\n", accountID)
// Generate SSL certificates
certPath := admin.GenerateCertificatesIfNeeded(configDir, force)
// Generate NATS token
natsToken := admin.GenerateNATSToken()
fmt.Println("\n🔒 Generated NATS authentication token")
if hiveRoot == "" {
// Get home directory for data path
homeDir, _ := os.UserHomeDir()
hiveRoot = filepath.Join(homeDir, "hive")
}
// Create config files from embedded templates
fmt.Println("\n📝 Creating configuration files...")
// Create subdirectories
awsgwDir := filepath.Join(configDir, "awsgw")
predastoreDir := filepath.Join(configDir, "predastore")
natsDir := filepath.Join(configDir, "nats")
hiveDir := filepath.Join(configDir, "hive")
for _, dir := range []string{awsgwDir, predastoreDir, natsDir, hiveDir} {
if err := os.MkdirAll(dir, 0700); err != nil {
fmt.Fprintf(os.Stderr, "Error creating directory %s: %v\n", dir, err)
os.Exit(1)
}
}
portStr := fmt.Sprintf("%d", port)
configSettings := admin.ConfigSettings{
AccessKey: accessKey,
SecretKey: secretKey,
AccountID: accountID,
Region: region,
NatsToken: natsToken,
DataDir: hiveRoot,
Node: node,
Az: az,
Port: portStr,
BindIP: bindIP,
ClusterBindIP: clusterBind,
ClusterRoutes: clusterRoutes,
}
// Generate all config files
configs := []admin.ConfigFile{
{
Name: "hive.toml",
Path: hiveTomlPath,
Template: hiveTomlTemplate,
},
{
Name: filepath.Join(awsgwDir, "awsgw.toml"),
Path: filepath.Join(awsgwDir, "awsgw.toml"),
Template: awsgwTomlTemplate,
},
{
Name: filepath.Join(predastoreDir, "predastore.toml"),
Path: filepath.Join(predastoreDir, "predastore.toml"),
Template: predastoreTomlTemplate,
},
{
Name: filepath.Join(natsDir, "nats.conf"),
Path: filepath.Join(natsDir, "nats.conf"),
Template: natsConfTemplate,
},
}
err := admin.GenerateConfigFiles(configs, configSettings)
if err != nil {
fmt.Fprintf(os.Stderr, "Error generating configuration files: %v\n", err)
os.Exit(1)
}
// Update ~/.aws/credentials and ~/.aws/config
fmt.Println("\n🔧 Configuring AWS credentials...")
if err := admin.SetupAWSCredentials(accessKey, secretKey, region, certPath); err != nil {
fmt.Fprintf(os.Stderr, "Warning: Could not update AWS credentials: %v\n", err)
} else {
fmt.Println("✅ AWS credentials configured")
}
admin.CreateServiceDirectories(hiveRoot)
// Print success message
fmt.Println("\n🎉 Hive initialization complete!")
fmt.Println("\n📋 Next steps:")
fmt.Println(" 1. Start services:")
fmt.Println(" ./scripts/start-dev.sh")
fmt.Println()
fmt.Println(" 2. Test with AWS CLI:")
fmt.Println(" export AWS_PROFILE=hive")
fmt.Println(" aws ec2 describe-instances")
fmt.Println()
fmt.Println("🔗 Configuration:")
fmt.Printf(" Config file: %s\n", hiveTomlPath)
fmt.Printf(" Data directory: %s\n", hiveRoot)
fmt.Println()
}
func runAdminJoin(cmd *cobra.Command, args []string) {
node, _ := cmd.Flags().GetString("node")
leaderHost, _ := cmd.Flags().GetString("host")
region, _ := cmd.Flags().GetString("region")
az, _ := cmd.Flags().GetString("az")
dataDir, _ := cmd.Flags().GetString("data-dir")
port, _ := cmd.Flags().GetInt("port")
bindIP, _ := cmd.Flags().GetString("bind")
configDir, _ := cmd.Flags().GetString("config-dir")
clusterBind, _ := cmd.Flags().GetString("cluster-bind")
// Receive as string, split later
// TODO: Use GetStringArray
clusterRoutesStr, _ := cmd.Flags().GetString("cluster-routes")
// Supply as array to template
clusterRoutes := strings.Split(clusterRoutesStr, ",")
// Validate required parameters
if node == "" {
fmt.Fprintf(os.Stderr, "❌ Error: --node is required\n")
os.Exit(1)
}
if leaderHost == "" {
fmt.Fprintf(os.Stderr, "❌ Error: --host is required\n")
os.Exit(1)
}
// Validate IP address format
if net.ParseIP(bindIP) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: Invalid IP address for --bind: %s\n", bindIP)
os.Exit(1)
}
// Validate port range
if port < 1 || port > 65535 {
fmt.Fprintf(os.Stderr, "❌ Error: Port must be between 1 and 65535, got: %d\n", port)
os.Exit(1)
}
// Set default data directory
if dataDir == "" {
homeDir, err := os.UserHomeDir()
if err != nil {
fmt.Fprintf(os.Stderr, "❌ Error getting home directory: %v\n", err)
os.Exit(1)
}
dataDir = filepath.Join(homeDir, "hive")
}
// Set daemon host for this node
daemonHost := fmt.Sprintf("%s:%d", bindIP, port)
fmt.Println("🚀 Joining Hive cluster...")
fmt.Printf("Node: %s\n", node)
fmt.Printf("Leader: %s\n", leaderHost)
fmt.Printf("Region: %s\n", region)
fmt.Printf("AZ: %s\n", az)
fmt.Printf("Bind IP: %s\n", bindIP)
fmt.Printf("Port: %d\n\n", port)
// Create join request
joinReq := config.NodeJoinRequest{
Node: node,
Region: region,
AZ: az,
DataDir: dataDir,
DaemonHost: daemonHost,
}
reqBody, err := json.Marshal(joinReq)
if err != nil {
fmt.Fprintf(os.Stderr, "Error marshaling join request: %v\n", err)
os.Exit(1)
}
// Send join request to leader with HTTP/2 support
tr := &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true, // Skip TLS verification for self-signed certs
NextProtos: []string{"h2", "http/1.1"},
},
ForceAttemptHTTP2: true,
}
// Configure HTTP/2 support with custom TLS config
http2.ConfigureTransport(tr)
client := &http.Client{
Timeout: 30 * time.Second,
Transport: tr,
}
joinURL := fmt.Sprintf("http://%s/join", leaderHost)
resp, err := client.Post(joinURL, "application/json", bytes.NewBuffer(reqBody))
if err != nil {
fmt.Fprintf(os.Stderr, "❌ Error connecting to leader node: %v\n", err)
fmt.Fprintf(os.Stderr, "Make sure the leader node is running and accessible at %s\n", leaderHost)
os.Exit(1)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
fmt.Fprintf(os.Stderr, "❌ Error reading response body: %v\n", err)
os.Exit(1)
}
if resp.StatusCode != 200 {
var errResp config.NodeJoinResponse
json.Unmarshal(body, &errResp)
fmt.Fprintf(os.Stderr, "❌ Failed to join cluster: %s\n", errResp.Message)
os.Exit(1)
}
var joinResp config.NodeJoinResponse
if err := json.Unmarshal(body, &joinResp); err != nil {
fmt.Fprintf(os.Stderr, "Error parsing join response: %v\n", err)
os.Exit(1)
}
if !joinResp.Success {
fmt.Fprintf(os.Stderr, "❌ Failed to join cluster: %s\n", joinResp.Message)
os.Exit(1)
}
fmt.Println("✅ Successfully joined cluster!")
fmt.Printf("Epoch: %d\n", joinResp.SharedData.Epoch)
fmt.Printf("Config hash: %s\n\n", joinResp.ConfigHash)
// Save cluster config locally
// Confirm if default config directory required
if configDir == "" {
homeDir, err := os.UserHomeDir()
if err != nil {
fmt.Fprintf(os.Stderr, "Error getting home directory: %v\n", err)
os.Exit(1)
}
configDir = filepath.Join(homeDir, "hive", "config")
}
hiveTomlPath := filepath.Join(configDir, "hive.toml")
// Create config directory if needed
if err := os.MkdirAll(configDir, 0700); err != nil {
fmt.Fprintf(os.Stderr, "Error creating config directory: %v\n", err)
os.Exit(1)
}
// Construct local node config with THIS node's name at top level
localConfig := config.ClusterConfig{
Epoch: joinResp.SharedData.Epoch,
Node: node, // THIS node's name, not the leader's
Version: joinResp.SharedData.Version,
Nodes: joinResp.SharedData.Nodes,
}
// Marshal config to TOML format
configTOML, err := toml.Marshal(&localConfig)
if err != nil {
fmt.Fprintf(os.Stderr, "Error marshaling config to TOML: %v\n", err)
os.Exit(1)
}
// Write config to file
if err := os.WriteFile(hiveTomlPath, configTOML, 0600); err != nil {
fmt.Fprintf(os.Stderr, "Error writing config file: %v\n", err)
os.Exit(1)
}
fmt.Printf("✅ Config saved to: %s\n\n", hiveTomlPath)
// Generate certificates if needed
certPath := admin.GenerateCertificatesIfNeeded(configDir, false)
fmt.Printf("✅ SSL certificates available at: %s\n\n", certPath)
// Write individual node config files
portStr := fmt.Sprintf("%d", port)
configSettings := admin.ConfigSettings{
AccessKey: localConfig.Nodes[node].AccessKey,
SecretKey: localConfig.Nodes[node].SecretKey,
Region: region,
NatsToken: localConfig.Nodes[node].NATS.ACL.Token,
DataDir: dataDir,
Node: node,
Az: az,
Port: portStr,
BindIP: bindIP,
ClusterBindIP: clusterBind,
ClusterRoutes: clusterRoutes,
}
awsgwDir := filepath.Join(configDir, "awsgw")
predastoreDir := filepath.Join(configDir, "predastore")
natsDir := filepath.Join(configDir, "nats")
for _, dir := range []string{awsgwDir, predastoreDir, natsDir} {
if err := os.MkdirAll(dir, 0700); err != nil {
fmt.Fprintf(os.Stderr, "Error creating directory %s: %v\n", dir, err)
os.Exit(1)
}
}
// Generate all config files
configs := []admin.ConfigFile{
{
Name: "hive.toml",
Path: hiveTomlPath,
Template: hiveTomlTemplate,
},
{
Name: filepath.Join(awsgwDir, "awsgw.toml"),
Path: filepath.Join(awsgwDir, "awsgw.toml"),
Template: awsgwTomlTemplate,
},
{
Name: filepath.Join(predastoreDir, "predastore.toml"),
Path: filepath.Join(predastoreDir, "predastore.toml"),
Template: predastoreTomlTemplate,
},
{
Name: filepath.Join(natsDir, "nats.conf"),
Path: filepath.Join(natsDir, "nats.conf"),
Template: natsConfTemplate,
},
}
err = admin.GenerateConfigFiles(configs, configSettings)
if err != nil {
fmt.Fprintf(os.Stderr, "Error generating configuration files: %v\n", err)
os.Exit(1)
}
admin.CreateServiceDirectories(dataDir)
fmt.Println("🎉 Node successfully joined cluster!")
fmt.Println("\n📋 Next steps:")
fmt.Println(" 1. Start the hive service:")
fmt.Printf(" hive service hive start --config %s\n", hiveTomlPath)
fmt.Println()
}
-13
View File
@@ -1,13 +0,0 @@
# AWS Gateway configuration file in TOML format.
version = "1.0"
region = "{{.Region}}"
# Enable verbose debugging
debug = false
[[auth]]
access_key_id = "{{.AccessKey}}"
secret_access_key = "{{.SecretKey}}"
policy = [
{ bucket = "predastore", actions = ["s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListAllMyBuckets"] },
]
File diff suppressed because it is too large Load Diff
+195
View File
@@ -0,0 +1,195 @@
package cmd
import (
"encoding/json"
"fmt"
"os"
"strings"
"time"
"github.com/mulgadc/spinifex/spinifex/daemon"
"github.com/nats-io/nats.go"
"github.com/spf13/cobra"
)
// runClusterShutdown orchestrates a phased, coordinated shutdown of the cluster.
func runClusterShutdown(cmd *cobra.Command, args []string) {
force, _ := cmd.Flags().GetBool("force")
timeout, _ := cmd.Flags().GetDuration("timeout")
dryRun, _ := cmd.Flags().GetBool("dry-run")
cfg, nc, err := loadConfigAndConnect()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
defer nc.Close()
nodeCount := len(cfg.Nodes)
phases := []string{"gate", "drain", "storage", "persist", "infra"}
if dryRun {
fmt.Println("Cluster Shutdown Plan (dry-run)")
fmt.Printf(" Nodes: %d\n", nodeCount)
for name, nodeCfg := range cfg.Nodes {
fmt.Printf(" - %s (services: %s)\n", name, strings.Join(nodeCfg.GetServices(), ", "))
}
fmt.Printf(" Phases: %s\n", strings.Join(phases, " -> "))
fmt.Printf(" Timeout per phase: %s\n", timeout)
fmt.Printf(" Force: %v\n", force)
fmt.Println("\nPhase details:")
fmt.Println(" 1. GATE - Stop API gateway and UI, reject new work")
fmt.Println(" 2. DRAIN - Gracefully stop all VMs, persist state")
fmt.Println(" 3. STORAGE - Stop viperblock, cleanup nbdkit")
fmt.Println(" 4. PERSIST - Stop predastore")
fmt.Println(" 5. INFRA - Stop NATS, exit daemons")
return
}
fmt.Printf("Starting coordinated cluster shutdown (%d nodes)\n", nodeCount)
fmt.Printf("Phases: %s\n", strings.Join(phases, " -> "))
fmt.Printf("Timeout per phase: %s\n\n", timeout)
// Write cluster shutdown marker to KV
jsm, err := daemon.NewJetStreamManager(nc, 1)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to create JetStream manager: %v\n", err)
} else {
if err := jsm.InitClusterStateBucket(); err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to init cluster state bucket: %v\n", err)
} else {
state := &daemon.ClusterShutdownState{
Initiator: cfg.Node,
Phase: "starting",
Started: time.Now().UTC().Format(time.RFC3339),
Timeout: timeout.String(),
Force: force,
NodesTotal: nodeCount,
NodesAcked: make(map[string]string),
}
if err := jsm.WriteClusterShutdown(state); err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to write shutdown state: %v\n", err)
}
}
}
start := time.Now()
// Execute phases sequentially (except INFRA which is fire-and-forget)
for _, phase := range phases {
topic := "spinifex.cluster.shutdown." + phase
req := daemon.ShutdownRequest{
Phase: phase,
Force: force,
Timeout: int(timeout.Seconds()),
}
reqData, err := json.Marshal(req)
if err != nil {
fmt.Fprintf(os.Stderr, "Error marshaling request: %v\n", err)
os.Exit(1)
}
if phase == "infra" {
// INFRA is fire-and-forget — NATS is going down, no ACKs possible
fmt.Printf("[INFRA] Sending final shutdown to all nodes...\n")
if err := nc.Publish(topic, reqData); err != nil {
fmt.Fprintf(os.Stderr, "Error publishing infra shutdown: %v\n", err)
}
nc.Flush()
// Wait briefly for messages to propagate
time.Sleep(2 * time.Second)
fmt.Printf("[INFRA] Complete\n")
break
}
// For DRAIN phase, subscribe to progress updates
var progressSub *nats.Subscription
if phase == "drain" {
progressSub, err = nc.Subscribe("spinifex.cluster.shutdown.progress", func(msg *nats.Msg) {
var progress daemon.ShutdownProgress
if err := json.Unmarshal(msg.Data, &progress); err != nil {
return
}
fmt.Printf(" [%s] %s: %d/%d VMs remaining\n", strings.ToUpper(phase), progress.Node, progress.Remaining, progress.Total)
})
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: failed to subscribe to progress: %v\n", err)
}
}
// Collect ACKs from all nodes
phaseStart := time.Now()
fmt.Printf("[%s] Sending to %d node(s)...\n", strings.ToUpper(phase), nodeCount)
acks, err := collectShutdownACKs(nc, topic, reqData, nodeCount, timeout)
if err != nil {
fmt.Fprintf(os.Stderr, "[%s] Error: %v\n", strings.ToUpper(phase), err)
if !force {
fmt.Fprintf(os.Stderr, "Aborting shutdown. Use --force to continue despite errors.\n")
os.Exit(1)
}
}
// Unsubscribe from progress
if progressSub != nil {
progressSub.Unsubscribe()
}
// Print results
for _, ack := range acks {
if ack.Error != "" {
fmt.Printf(" [%s] %s: ERROR - %s\n", strings.ToUpper(phase), ack.Node, ack.Error)
} else if len(ack.Stopped) > 0 {
fmt.Printf(" [%s] %s: stopped %s\n", strings.ToUpper(phase), ack.Node, strings.Join(ack.Stopped, ", "))
} else {
fmt.Printf(" [%s] %s: OK\n", strings.ToUpper(phase), ack.Node)
}
}
ackedCount := len(acks)
if ackedCount < nodeCount && !force {
fmt.Fprintf(os.Stderr, "[%s] Only %d/%d nodes responded. Use --force to continue.\n",
strings.ToUpper(phase), ackedCount, nodeCount)
os.Exit(1)
}
fmt.Printf("[%s] Complete (%d/%d nodes, %s)\n\n", strings.ToUpper(phase), ackedCount, nodeCount, time.Since(phaseStart).Round(time.Millisecond))
}
fmt.Printf("Cluster shutdown complete (%s)\n", time.Since(start).Round(time.Millisecond))
}
// collectShutdownACKs publishes a shutdown request and collects ACKs from nodes.
func collectShutdownACKs(nc *nats.Conn, topic string, reqData []byte, nodeCount int, timeout time.Duration) ([]daemon.ShutdownACK, error) {
inbox := nats.NewInbox()
sub, err := nc.SubscribeSync(inbox)
if err != nil {
return nil, fmt.Errorf("failed to subscribe to inbox: %w", err)
}
defer sub.Unsubscribe()
if err := nc.PublishRequest(topic, inbox, reqData); err != nil {
return nil, fmt.Errorf("failed to publish request: %w", err)
}
nc.Flush()
var acks []daemon.ShutdownACK
deadline := time.Now().Add(timeout)
for len(acks) < nodeCount {
remaining := time.Until(deadline)
if remaining <= 0 {
break
}
msg, err := sub.NextMsg(remaining)
if err != nil {
break
}
var ack daemon.ShutdownACK
if err := json.Unmarshal(msg.Data, &ack); err != nil {
continue
}
acks = append(acks, ack)
}
return acks, nil
}
+289
View File
@@ -0,0 +1,289 @@
/*
Copyright © 2025 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cmd
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"time"
"github.com/mulgadc/spinifex/spinifex/config"
"github.com/mulgadc/spinifex/spinifex/types"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/nats-io/nats.go"
"github.com/pterm/pterm"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var getCmd = &cobra.Command{
Use: "get",
Short: "Display cluster resources",
Long: `Display cluster resources such as nodes and VMs.`,
}
var getNodesCmd = &cobra.Command{
Use: "nodes",
Short: "Display cluster nodes",
Long: `Display all physical nodes in the cluster with status, IP, region, and services.`,
Run: runGetNodes,
}
var getVMsCmd = &cobra.Command{
Use: "vms",
Aliases: []string{"instances"},
Short: "Display VMs across the cluster",
Long: `Display all VMs running across the cluster with instance type, resources, and placement.`,
Run: runGetVMs,
}
func init() {
rootCmd.AddCommand(getCmd)
getCmd.AddCommand(getNodesCmd)
getCmd.AddCommand(getVMsCmd)
getCmd.PersistentFlags().Duration("timeout", 3*time.Second, "Timeout for collecting responses from nodes")
}
// loadConfigAndConnect loads the cluster config and connects to NATS.
func loadConfigAndConnect() (*config.ClusterConfig, *nats.Conn, error) {
cfgPath := viper.GetString("config")
if cfgPath == "" {
cfgPath = DefaultConfigFile()
}
cfg, err := config.LoadConfig(cfgPath)
if err != nil {
return nil, nil, fmt.Errorf("failed to load config: %w", err)
}
// Backfill BaseDir from config file path when not set in the config.
// Config lives at <baseDir>/config/spinifex.toml, so baseDir is two levels up.
if nodeConfig, ok := cfg.Nodes[cfg.Node]; ok && nodeConfig.BaseDir == "" {
nodeConfig.BaseDir = filepath.Dir(filepath.Dir(cfgPath))
cfg.Nodes[cfg.Node] = nodeConfig
}
nodeConfig := cfg.Nodes[cfg.Node]
nc, err := utils.ConnectNATS(nodeConfig.NATS.Host, nodeConfig.NATS.ACL.Token)
if err != nil {
return nil, nil, fmt.Errorf("failed to connect to NATS: %w", err)
}
return cfg, nc, nil
}
// collectResponses publishes to a fan-out topic and collects all responses within the timeout.
func collectResponses(nc *nats.Conn, topic string, timeout time.Duration) ([][]byte, error) {
inbox := nats.NewInbox()
sub, err := nc.SubscribeSync(inbox)
if err != nil {
return nil, fmt.Errorf("failed to subscribe to inbox: %w", err)
}
defer sub.Unsubscribe()
if err := nc.PublishRequest(topic, inbox, nil); err != nil {
return nil, fmt.Errorf("failed to publish request: %w", err)
}
nc.Flush()
var responses [][]byte
deadline := time.Now().Add(timeout)
for {
remaining := time.Until(deadline)
if remaining <= 0 {
break
}
msg, err := sub.NextMsg(remaining)
if err != nil {
break // timeout or other error — done collecting
}
responses = append(responses, msg.Data)
}
return responses, nil
}
func formatUptime(seconds int64) string {
if seconds <= 0 {
return "-"
}
d := time.Duration(seconds) * time.Second
days := int(d.Hours()) / 24
hours := int(d.Hours()) % 24
mins := int(d.Minutes()) % 60
if days > 0 {
return fmt.Sprintf("%dd%dh", days, hours)
}
if hours > 0 {
return fmt.Sprintf("%dh%dm", hours, mins)
}
return fmt.Sprintf("%dm", mins)
}
func formatMemGB(gb float64) string {
if gb >= 1.0 {
return fmt.Sprintf("%.1fGi", gb)
}
return fmt.Sprintf("%dMi", int(gb*1024))
}
func runGetNodes(cmd *cobra.Command, args []string) {
cfg, nc, err := loadConfigAndConnect()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
defer nc.Close()
timeout, _ := cmd.Flags().GetDuration("timeout")
responses, err := collectResponses(nc, "spinifex.node.status", timeout)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
// Parse responses into a map by node name
respondedNodes := make(map[string]types.NodeStatusResponse)
for _, data := range responses {
var resp types.NodeStatusResponse
if err := json.Unmarshal(data, &resp); err != nil {
continue
}
respondedNodes[resp.Node] = resp
}
// Build table: union of config-known nodes + NATS responders
tableData := pterm.TableData{
{"NAME", "STATUS", "IP", "REGION", "AZ", "UPTIME", "VMs", "SERVICES"},
}
// Collect all node names: config + responded (union)
nodeSet := make(map[string]struct{})
for name := range cfg.Nodes {
nodeSet[name] = struct{}{}
}
for name := range respondedNodes {
nodeSet[name] = struct{}{}
}
nodeNames := make([]string, 0, len(nodeSet))
for name := range nodeSet {
nodeNames = append(nodeNames, name)
}
sort.Strings(nodeNames)
for _, name := range nodeNames {
nodeCfg := cfg.Nodes[name]
if resp, ok := respondedNodes[name]; ok {
tableData = append(tableData, []string{
resp.Node,
resp.Status,
resp.Host,
resp.Region,
resp.AZ,
formatUptime(resp.Uptime),
fmt.Sprintf("%d", resp.VMCount),
strings.Join(resp.Services, ","),
})
} else {
tableData = append(tableData, []string{
name,
"NotReady",
nodeCfg.Host,
nodeCfg.Region,
nodeCfg.AZ,
"-",
"-",
strings.Join(nodeCfg.GetServices(), ","),
})
}
}
pterm.DefaultTable.WithHasHeader().WithLeftAlignment().WithData(tableData).Render()
}
func runGetVMs(cmd *cobra.Command, args []string) {
_, nc, err := loadConfigAndConnect()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
defer nc.Close()
timeout, _ := cmd.Flags().GetDuration("timeout")
responses, err := collectResponses(nc, "spinifex.node.vms", timeout)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
type vmRow struct {
types.VMInfo
Node string
Host string
}
var allVMs []vmRow
for _, data := range responses {
var resp types.NodeVMsResponse
if err := json.Unmarshal(data, &resp); err != nil {
continue
}
for _, v := range resp.VMs {
allVMs = append(allVMs, vmRow{VMInfo: v, Node: resp.Node, Host: resp.Host})
}
}
if len(allVMs) == 0 {
fmt.Println("No VMs found.")
return
}
// Sort by node then instance ID
sort.Slice(allVMs, func(i, j int) bool {
if allVMs[i].Node != allVMs[j].Node {
return allVMs[i].Node < allVMs[j].Node
}
return allVMs[i].InstanceID < allVMs[j].InstanceID
})
tableData := pterm.TableData{
{"INSTANCE", "STATUS", "TYPE", "VCPU", "MEM", "NODE", "IP", "AGE"},
}
for _, v := range allVMs {
age := "-"
if v.LaunchTime > 0 {
age = formatUptime(time.Now().Unix() - v.LaunchTime)
}
tableData = append(tableData, []string{
v.InstanceID,
v.Status,
v.InstanceType,
fmt.Sprintf("%d", v.VCPU),
formatMemGB(v.MemoryGB),
v.Node,
v.Host,
age,
})
}
pterm.DefaultTable.WithHasHeader().WithLeftAlignment().WithData(tableData).Render()
}
+44
View File
@@ -0,0 +1,44 @@
package cmd
import (
"os"
"path/filepath"
)
// DefaultConfigDir returns the default configuration directory.
// Production: /etc/spinifex (when running as root or /etc/spinifex exists)
// Development: ~/spinifex/config
func DefaultConfigDir() string {
if isProductionLayout() {
return "/etc/spinifex"
}
homeDir, _ := os.UserHomeDir()
return filepath.Join(homeDir, "spinifex", "config")
}
// DefaultDataDir returns the default data directory.
// Production: /var/lib/spinifex
// Development: ~/spinifex
func DefaultDataDir() string {
if isProductionLayout() {
return "/var/lib/spinifex"
}
homeDir, _ := os.UserHomeDir()
return filepath.Join(homeDir, "spinifex")
}
// DefaultConfigFile returns the default path to spinifex.toml.
func DefaultConfigFile() string {
return filepath.Join(DefaultConfigDir(), "spinifex.toml")
}
// isProductionLayout returns true when running in a production install.
// Detected by: running as root, or /etc/spinifex directory exists.
func isProductionLayout() bool {
if os.Getuid() == 0 {
if info, err := os.Stat("/etc/spinifex"); err == nil && info.IsDir() {
return true
}
}
return false
}
+57
View File
@@ -0,0 +1,57 @@
package cmd
import (
"os"
"path/filepath"
"testing"
)
func TestDefaultConfigDir_Dev(t *testing.T) {
// Non-root user without /etc/spinifex should get dev paths
if os.Getuid() == 0 {
t.Skip("test requires non-root user")
}
homeDir, _ := os.UserHomeDir()
expected := filepath.Join(homeDir, "spinifex", "config")
got := DefaultConfigDir()
if got != expected {
t.Errorf("DefaultConfigDir() = %q, want %q", got, expected)
}
}
func TestDefaultDataDir_Dev(t *testing.T) {
if os.Getuid() == 0 {
t.Skip("test requires non-root user")
}
homeDir, _ := os.UserHomeDir()
expected := filepath.Join(homeDir, "spinifex")
got := DefaultDataDir()
if got != expected {
t.Errorf("DefaultDataDir() = %q, want %q", got, expected)
}
}
func TestDefaultConfigFile(t *testing.T) {
if os.Getuid() == 0 {
t.Skip("test requires non-root user")
}
got := DefaultConfigFile()
homeDir, _ := os.UserHomeDir()
expected := filepath.Join(homeDir, "spinifex", "config", "spinifex.toml")
if got != expected {
t.Errorf("DefaultConfigFile() = %q, want %q", got, expected)
}
}
func TestIsProductionLayout_NonRoot(t *testing.T) {
if os.Getuid() == 0 {
t.Skip("test requires non-root user")
}
if isProductionLayout() {
t.Error("isProductionLayout() = true for non-root user, want false")
}
}
@@ -19,7 +19,7 @@ import (
"fmt"
"os"
"github.com/mulgadc/hive/hive/config"
"github.com/mulgadc/spinifex/spinifex/config"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
@@ -32,20 +32,17 @@ var (
// rootCmd represents the base command when called without any subcommands
var rootCmd = &cobra.Command{
Use: "hive",
Short: "Hive - Open source AWS-compatible platform for secure edge deployments",
Use: "spx",
Short: "Spinifex - Open source AWS-compatible platform for secure edge deployments",
Long: `
__/\\\________/\\\__/\\\\\\\\\\\__/\\\________/\\\__/\\\\\\\\\\\\\\\_
_\/\\\_______\/\\\_\/////\\\///__\/\\\_______\/\\\_\/\\\///////////__
_\/\\\_______\/\\\_____\/\\\_____\//\\\______/\\\__\/\\\_____________
_\/\\\\\\\\\\\\\\\_____\/\\\______\//\\\____/\\\___\/\\\\\\\\\\\_____
_\/\\\/////////\\\_____\/\\\_______\//\\\__/\\\____\/\\\///////______
_\/\\\_______\/\\\_____\/\\\________\//\\\/\\\_____\/\\\_____________
_\/\\\_______\/\\\_____\/\\\_________\//\\\\\______\/\\\_____________
_\/\\\_______\/\\\__/\\\\\\\\\\\______\//\\\_______\/\\\\\\\\\\\\\\\_
_\///________\///__\///////////________\///________\///////////////__
_____ _____ _____ _ _ _____ ______ ________ __
/ ____| __ \_ _| \ | |_ _| ____| ____\ \ / /
| (___ | |__) || | | \| | | | | |__ | |__ \ V /
\___ \| ___/ | | | . ‘ | | | | __| | __| > <
____) | | _| |_| |\ |_| |_| | | |____ / . \
|_____/|_| |_____|_| \_|_____|_| |______/_/ \_\
Hive – Open source AWS-compatible platform for secure edge deployments.
Spinifex – Open source AWS-compatible platform for secure edge deployments.
Run EC2, VPC, S3, and EBS-like services on bare metal with full control.
Built for environments where running in the cloud isn’t an option.
Whether you’re deploying to edge sites, private data-centers, or operating
@@ -67,38 +64,38 @@ func init() {
// Global flags
rootCmd.PersistentFlags().String("config", "", "config file (required)")
viper.BindEnv("config", "HIVE_CONFIG_PATH")
viper.BindEnv("config", "SPINIFEX_CONFIG_PATH")
viper.BindPFlag("config", rootCmd.PersistentFlags().Lookup("config"))
// Authentication (access_key, secret)
rootCmd.PersistentFlags().String("access-key", "", "AWS access key (overrides config file and env)")
viper.BindEnv("access-key", "HIVE_ACCESS_KEY")
viper.BindEnv("access-key", "SPINIFEX_ACCESS_KEY")
viper.BindPFlag("access-key", rootCmd.PersistentFlags().Lookup("access-key"))
rootCmd.PersistentFlags().String("secret-key", "", "AWS secret key (overrides config file and env)")
viper.BindEnv("secret-key", "HIVE_SECRET_KEY")
viper.BindEnv("secret-key", "SPINIFEX_SECRET_KEY")
viper.BindPFlag("secret-key", rootCmd.PersistentFlags().Lookup("secret-key"))
rootCmd.PersistentFlags().String("host", "", "AWS Endpoint (overrides config file and env)")
viper.BindEnv("host", "HIVE_HOST")
viper.BindEnv("host", "SPINIFEX_HOST")
viper.BindPFlag("host", rootCmd.PersistentFlags().Lookup("host"))
// Viperblock config
rootCmd.PersistentFlags().String("base-dir", "", "Viperblock base directory (overrides config file and env)")
viper.BindEnv("base-dir", "HIVE_BASE_DIR")
viper.BindEnv("base-dir", "SPINIFEX_BASE_DIR")
viper.BindPFlag("base-dir", rootCmd.PersistentFlags().Lookup("base-dir"))
// NATS specific flags
rootCmd.PersistentFlags().String("nats-host", "", "NATS server host (overrides config file and env)")
viper.BindEnv("nats-host", "HIVE_NATS_HOST")
viper.BindEnv("nats-host", "SPINIFEX_NATS_HOST")
viper.BindPFlag("nats-host", rootCmd.PersistentFlags().Lookup("nats-host"))
rootCmd.PersistentFlags().String("nats-token", "", "NATS authentication token (overrides config file and env)")
viper.BindEnv("nats-token", "HIVE_NATS_TOKEN")
viper.BindEnv("nats-token", "SPINIFEX_NATS_TOKEN")
viper.BindPFlag("nats-token", rootCmd.PersistentFlags().Lookup("nats-token"))
rootCmd.PersistentFlags().String("nats-subject", "", "NATS subscription subject (overrides config file and env)")
viper.BindEnv("nats-subject", "HIVE_NATS_SUBJECT")
viper.BindEnv("nats-subject", "SPINIFEX_NATS_SUBJECT")
viper.BindPFlag("nats-subject", rootCmd.PersistentFlags().Lookup("nats-subject"))
// Bind flags to viper
@@ -116,7 +113,11 @@ func initConfig() {
appConfig, err = config.LoadConfig(cfgFile)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: %v\n", err)
fmt.Fprintln(os.Stderr, "Continuing with environment variables and defaults...")
// If a config file was explicitly provided, treat load failure as fatal
if cfgFile != "" {
fmt.Fprintf(os.Stderr, "Error: failed to load config %s: %v\n", cfgFile, err)
os.Exit(1)
}
// No config specified — continue with env/defaults (e.g., spx --help)
}
}
@@ -21,19 +21,21 @@ import (
"os"
"strings"
"github.com/mulgadc/hive/hive/config"
"github.com/mulgadc/hive/hive/service"
"github.com/mulgadc/hive/hive/services/nats"
"github.com/mulgadc/hive/hive/services/predastore"
"github.com/mulgadc/hive/hive/services/viperblockd"
"github.com/mulgadc/predastore/s3"
"github.com/mulgadc/spinifex/spinifex/config"
"github.com/mulgadc/spinifex/spinifex/service"
"github.com/mulgadc/spinifex/spinifex/services/nats"
"github.com/mulgadc/spinifex/spinifex/services/predastore"
"github.com/mulgadc/spinifex/spinifex/services/spinifexui"
"github.com/mulgadc/spinifex/spinifex/services/viperblockd"
"github.com/mulgadc/spinifex/spinifex/services/vpcd"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
var serviceCmd = &cobra.Command{
Use: "service",
Short: "Manage Hive services",
Short: "Manage Spinifex services",
}
var predastoreCmd = &cobra.Command{
@@ -51,9 +53,9 @@ var natsCmd = &cobra.Command{
Short: "Manage the nats service",
}
var hiveCmd = &cobra.Command{
Use: "hive",
Short: "Manage the hive service",
var spinifexCmd = &cobra.Command{
Use: "spinifex",
Short: "Manage the spinifex service",
}
var awsgwCmd = &cobra.Command{
@@ -61,6 +63,17 @@ var awsgwCmd = &cobra.Command{
Short: "Manage the awsgw (AWS gateway) service",
}
var vpcdCmd = &cobra.Command{
Use: "vpcd",
Short: "Manage the vpcd (VPC daemon) service",
}
var spinifexUICmd = &cobra.Command{
Use: "spinifex-ui",
Aliases: []string{"ui", "spinifexui"},
Short: "Manage the spinifex-ui service",
}
var predastoreStartCmd = &cobra.Command{
Use: "start",
Short: "Start the predastore service",
@@ -135,7 +148,10 @@ var predastoreStartCmd = &cobra.Command{
return
}
service.Start()
if _, err := service.Start(); err != nil {
fmt.Println("Error starting predastore service:", err)
os.Exit(1)
}
fmt.Println("Predastore service started", service)
},
@@ -154,7 +170,10 @@ var predastoreStopCmd = &cobra.Command{
return
}
service.Stop()
if err = service.Stop(); err != nil {
fmt.Println("Error stopping predastore service:", err)
os.Exit(1)
}
fmt.Println("Predastore service stopped")
@@ -188,12 +207,11 @@ var viperblockStartCmd = &cobra.Command{
// TODO: Support ENV vars, CLI, otherwise revert to config.LoadConfig()
clusterConfig, err := config.LoadConfig(cfgFile)
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
if err != nil {
fmt.Println("Error loading config file:", err)
return
}
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
natsHost := viper.GetString("nats-host")
@@ -259,6 +277,12 @@ var viperblockStartCmd = &cobra.Command{
os.Exit(1)
}
// Resolve sharded WAL setting: default false unless explicitly set to true
shardWAL := false
if nodeConfig.Viperblock.ShardWAL != nil {
shardWAL = *nodeConfig.Viperblock.ShardWAL
}
service, err := service.New("viperblock", &viperblockd.Config{
NatsHost: nodeConfig.NATS.Host,
PluginPath: pluginPath,
@@ -268,6 +292,8 @@ var viperblockStartCmd = &cobra.Command{
AccessKey: nodeConfig.Predastore.AccessKey,
SecretKey: nodeConfig.Predastore.SecretKey,
BaseDir: nodeConfig.Predastore.BaseDir,
NodeName: clusterConfig.Node,
ShardWAL: shardWAL,
})
if err != nil {
@@ -299,7 +325,10 @@ var viperblockStopCmd = &cobra.Command{
return
}
service.Stop()
if err = service.Stop(); err != nil {
fmt.Println("Error stopping viperblock service:", err)
os.Exit(1)
}
fmt.Println("Viperblock service stopped")
@@ -343,7 +372,10 @@ var natsStartCmd = &cobra.Command{
return
}
service.Start()
if _, err = service.Start(); err != nil {
fmt.Println("Error starting nats service:", err)
os.Exit(1)
}
fmt.Println("NATS service started")
},
}
@@ -361,7 +393,10 @@ var natsStopCmd = &cobra.Command{
return
}
service.Stop()
if err = service.Stop(); err != nil {
fmt.Println("Error stopping nats service:", err)
os.Exit(1)
}
fmt.Println("Nats service stopped")
},
@@ -375,12 +410,12 @@ var natsStatusCmd = &cobra.Command{
},
}
// Repeat for hive
var hiveStartCmd = &cobra.Command{
// Repeat for spinifex
var spinifexStartCmd = &cobra.Command{
Use: "start",
Short: "Start the hive service",
Short: "Start the spinifex service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Starting hive service...")
fmt.Println("Starting spinifex service...")
cfgFile := viper.GetString("config")
@@ -391,12 +426,11 @@ var hiveStartCmd = &cobra.Command{
// TODO: Support ENV vars, CLI, otherwise revert to config.LoadConfig()
clusterConfig, err := config.LoadConfig(cfgFile)
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
if err != nil {
fmt.Println("Error loading config file:", err)
return
}
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
// Overwrite defaults (CLI first, config second, env third)
baseDir := viper.GetString("base-dir")
@@ -417,47 +451,53 @@ var hiveStartCmd = &cobra.Command{
// Apply changes back to cluster config
clusterConfig.Nodes[clusterConfig.Node] = nodeConfig
svc, err := service.New("hive", clusterConfig)
svc, err := service.New("spinifex", clusterConfig)
if err != nil {
fmt.Println("Error starting hive service:", err)
fmt.Println("Error starting spinifex service:", err)
return
}
// Set config path for cluster manager
if hiveSvc, ok := svc.(interface{ SetConfigPath(string) }); ok {
hiveSvc.SetConfigPath(cfgFile)
if spxSvc, ok := svc.(interface{ SetConfigPath(string) }); ok {
spxSvc.SetConfigPath(cfgFile)
}
svc.Start()
fmt.Println("HIVE service started")
if _, err = svc.Start(); err != nil {
fmt.Println("Error starting spinifex service:", err)
os.Exit(1)
}
fmt.Println("Spinifex service started")
},
}
var hiveStopCmd = &cobra.Command{
var spinifexStopCmd = &cobra.Command{
Use: "stop",
Short: "Stop the hive service",
Short: "Stop the spinifex service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Stopping hive service...")
fmt.Println("Stopping spinifex service...")
service, err := service.New("hive", &config.ClusterConfig{})
service, err := service.New("spinifex", &config.ClusterConfig{})
if err != nil {
fmt.Println("Error stopping hive service:", err)
fmt.Println("Error stopping spinifex service:", err)
return
}
service.Stop()
if err = service.Stop(); err != nil {
fmt.Println("Error stopping spinifex service:", err)
os.Exit(1)
}
fmt.Println("Hive service stopped")
fmt.Println("Spinifex service stopped")
},
}
var hiveStatusCmd = &cobra.Command{
var spinifexStatusCmd = &cobra.Command{
Use: "status",
Short: "Get status of the hive service",
Short: "Get status of the spinifex service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Hive service status: ...")
fmt.Println("Spinifex service status: ...")
},
}
@@ -480,12 +520,11 @@ var awsgwStartCmd = &cobra.Command{
// TODO: Support ENV vars, CLI, otherwise revert to config.LoadConfig()
clusterConfig, err := config.LoadConfig(cfgFile)
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
if err != nil {
fmt.Println("Error loading config file:", err)
return
}
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
// Overwrite defaults (CLI first, config second, env third)
awsgwHost := viper.GetString("host")
@@ -509,7 +548,7 @@ var awsgwStartCmd = &cobra.Command{
baseDir := viper.GetString("base-dir")
if awsgwTlsKey != "" {
if baseDir != "" {
fmt.Println("Overwriting awsgw base-dir to:", baseDir)
nodeConfig.BaseDir = baseDir
}
@@ -524,7 +563,10 @@ var awsgwStartCmd = &cobra.Command{
return
}
service.Start()
if _, err = service.Start(); err != nil {
fmt.Println("Error starting awsgw service:", err)
os.Exit(1)
}
fmt.Println("AWSGW service started")
},
}
@@ -542,7 +584,10 @@ var awsgwStopCmd = &cobra.Command{
return
}
service.Stop()
if err = service.Stop(); err != nil {
fmt.Println("Error stopping awsgw service:", err)
os.Exit(1)
}
fmt.Println("AWSGW service stopped")
},
@@ -556,9 +601,151 @@ var awsgwStatusCmd = &cobra.Command{
},
}
var spinifexUIStartCmd = &cobra.Command{
Use: "start",
Short: "Start the spinifex-ui service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Starting spinifex-ui service...")
port := viper.GetInt("spinifex-ui-port")
host := viper.GetString("spinifex-ui-host")
tlsCert := viper.GetString("spinifex-ui-tls-cert")
tlsKey := viper.GetString("spinifex-ui-tls-key")
svc, err := service.New("spinifex-ui", &spinifexui.Config{
Port: port,
Host: host,
TLSCert: tlsCert,
TLSKey: tlsKey,
})
if err != nil {
fmt.Println("Error starting spinifex-ui service:", err)
return
}
if _, err = svc.Start(); err != nil {
fmt.Println("Error starting spinifex-ui service:", err)
os.Exit(1)
}
fmt.Println("spinifex-ui service started")
},
}
var spinifexUIStopCmd = &cobra.Command{
Use: "stop",
Short: "Stop the spinifex-ui service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Stopping spinifex-ui service...")
svc, err := service.New("spinifex-ui", &spinifexui.Config{})
if err != nil {
fmt.Println("Error stopping spinifex-ui service:", err)
return
}
if err = svc.Stop(); err != nil {
fmt.Println("Error stopping spinifex-ui service:", err)
os.Exit(1)
}
fmt.Println("spinifex-ui service stopped")
},
}
var spinifexUIStatusCmd = &cobra.Command{
Use: "status",
Short: "Get status of the spinifex-ui service",
Run: func(cmd *cobra.Command, args []string) {
svc, err := service.New("spinifex-ui", &spinifexui.Config{})
if err != nil {
fmt.Println("Error getting spinifex-ui service status:", err)
return
}
status, err := svc.Status()
if err != nil {
fmt.Println("Error getting spinifex-ui service status:", err)
return
}
fmt.Println("spinifex-ui service status:", status)
},
}
var vpcdStartCmd = &cobra.Command{
Use: "start",
Short: "Start the vpcd service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Starting vpcd service...")
cfgFile := viper.GetString("config")
if cfgFile == "" {
fmt.Println("Config file is not set")
return
}
clusterConfig, err := config.LoadConfig(cfgFile)
if err != nil {
fmt.Println("Error loading config file:", err)
return
}
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
svc, err := service.New("vpcd", &vpcd.Config{
NatsHost: nodeConfig.NATS.Host,
NatsToken: nodeConfig.NATS.ACL.Token,
OVNNBAddr: nodeConfig.VPCD.OVNNBAddr,
OVNSBAddr: nodeConfig.VPCD.OVNSBAddr,
BaseDir: nodeConfig.BaseDir,
Debug: false,
})
if err != nil {
fmt.Println("Error starting vpcd service:", err)
return
}
if _, err = svc.Start(); err != nil {
fmt.Println("Error starting vpcd service:", err)
os.Exit(1)
}
fmt.Println("vpcd service started")
},
}
var vpcdStopCmd = &cobra.Command{
Use: "stop",
Short: "Stop the vpcd service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("Stopping vpcd service...")
svc, err := service.New("vpcd", &vpcd.Config{})
if err != nil {
fmt.Println("Error stopping vpcd service:", err)
return
}
if err = svc.Stop(); err != nil {
fmt.Println("Error stopping vpcd service:", err)
os.Exit(1)
}
fmt.Println("vpcd service stopped")
},
}
var vpcdStatusCmd = &cobra.Command{
Use: "status",
Short: "Get status of the vpcd service",
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("vpcd service status: ...")
},
}
func init() {
viper.SetEnvPrefix("HIVE") // Prefix for environment variables
viper.SetEnvPrefix("SPINIFEX") // Prefix for environment variables
viper.SetEnvKeyReplacer(strings.NewReplacer("-", "_"))
viper.AutomaticEnv() // Read environment variables automatically
@@ -569,47 +756,47 @@ func init() {
// Predastore Port
predastoreCmd.PersistentFlags().Int("port", 8443, "Predastore (S3) port")
viper.BindEnv("port", "HIVE_PREDASTORE_PORT")
viper.BindEnv("port", "SPINIFEX_PREDASTORE_PORT")
viper.BindPFlag("port", predastoreCmd.PersistentFlags().Lookup("port"))
// Predastore Host
predastoreCmd.PersistentFlags().String("host", "0.0.0.0", "Predastore (S3) host")
viper.BindEnv("host", "HIVE_PREDASTORE_HOST")
viper.BindEnv("host", "SPINIFEX_PREDASTORE_HOST")
viper.BindPFlag("host", predastoreCmd.PersistentFlags().Lookup("host"))
// Base path
predastoreCmd.PersistentFlags().String("base-path", "", "Predastore (S3) base path")
viper.BindEnv("base-path", "HIVE_PREDASTORE_BASE_PATH")
viper.BindEnv("base-path", "SPINIFEX_PREDASTORE_BASE_PATH")
viper.BindPFlag("base-path", predastoreCmd.PersistentFlags().Lookup("base-path"))
// Predastore Config Path
predastoreCmd.PersistentFlags().String("config-path", "", "Predastore (S3) config path")
viper.BindEnv("config-path", "HIVE_PREDASTORE_CONFIG_PATH")
viper.BindEnv("config-path", "SPINIFEX_PREDASTORE_CONFIG_PATH")
viper.BindPFlag("config-path", predastoreCmd.PersistentFlags().Lookup("config-path"))
// Predastore Debug
predastoreCmd.PersistentFlags().Bool("debug", false, "Predastore (S3) debug")
viper.BindEnv("debug", "HIVE_PREDASTORE_DEBUG")
viper.BindEnv("debug", "SPINIFEX_PREDASTORE_DEBUG")
viper.BindPFlag("debug", predastoreCmd.PersistentFlags().Lookup("debug"))
// Predastore TLS Cert
predastoreCmd.PersistentFlags().String("tls-cert", "", "Predastore (S3) TLS certificate")
viper.BindEnv("tls-cert", "HIVE_PREDASTORE_TLS_CERT")
viper.BindEnv("tls-cert", "SPINIFEX_PREDASTORE_TLS_CERT")
viper.BindPFlag("tls-cert", predastoreCmd.PersistentFlags().Lookup("tls-cert"))
// Predastore TLS Key
predastoreCmd.PersistentFlags().String("tls-key", "", "Predastore (S3) TLS key")
viper.BindEnv("tls-key", "HIVE_PREDASTORE_TLS_KEY")
viper.BindEnv("tls-key", "SPINIFEX_PREDASTORE_TLS_KEY")
viper.BindPFlag("tls-key", predastoreCmd.PersistentFlags().Lookup("tls-key"))
// Predastore Backend
predastoreCmd.PersistentFlags().String("backend", "distributed", "Predastore (S3) backend")
viper.BindEnv("backend", "HIVE_PREDASTORE_BACKEND")
viper.BindEnv("backend", "SPINIFEX_PREDASTORE_BACKEND")
viper.BindPFlag("backend", predastoreCmd.PersistentFlags().Lookup("backend"))
// Predastore Node ID
predastoreCmd.PersistentFlags().Int("node-id", 0, "Predastore (S3) node ID")
viper.BindEnv("node-id", "HIVE_PREDASTORE_NODE_ID")
viper.BindEnv("node-id", "SPINIFEX_PREDASTORE_NODE_ID")
viper.BindPFlag("node-id", predastoreCmd.PersistentFlags().Lookup("node-id"))
// Predastore CPU Profiling
@@ -629,19 +816,19 @@ func init() {
serviceCmd.AddCommand(viperblockCmd)
viperblockCmd.PersistentFlags().String("s3-host", "0.0.0.0:8443", "Predastore (S3) host URI")
viper.BindEnv("s3-host", "HIVE_VIPERBLOCK_S3_HOST")
viper.BindEnv("s3-host", "SPINIFEX_VIPERBLOCK_S3_HOST")
viper.BindPFlag("s3-host", predastoreCmd.PersistentFlags().Lookup("s3-host"))
viperblockCmd.PersistentFlags().String("s3-bucket", "predastore", "Predastore (S3) bucket")
viper.BindEnv("s3-bucket", "HIVE_VIPERBLOCK_S3_BUCKET")
viper.BindEnv("s3-bucket", "SPINIFEX_VIPERBLOCK_S3_BUCKET")
viper.BindPFlag("s3-bucket", predastoreCmd.PersistentFlags().Lookup("s3-bucket"))
viperblockCmd.PersistentFlags().String("s3-region", "ap-southeast-2", "Predastore (S3) region")
viper.BindEnv("s3-region", "HIVE_VIPERBLOCK_S3_REGION")
viper.BindEnv("s3-region", "SPINIFEX_VIPERBLOCK_S3_REGION")
viper.BindPFlag("s3-region", predastoreCmd.PersistentFlags().Lookup("s3-region"))
viperblockCmd.PersistentFlags().String("plugin-path", "/opt/hive/lib/nbdkit-viperblock-plugin.so", "Pathname to the nbdkit viperblockplugin")
viper.BindEnv("plugin-path", "HIVE_VIPERBLOCK_PLUGIN_PATH")
viperblockCmd.PersistentFlags().String("plugin-path", "/opt/spinifex/lib/nbdkit-viperblock-plugin.so", "Pathname to the nbdkit viperblockplugin")
viper.BindEnv("plugin-path", "SPINIFEX_VIPERBLOCK_PLUGIN_PATH")
viper.BindPFlag("plugin-path", predastoreCmd.PersistentFlags().Lookup("plugin-path"))
viperblockCmd.AddCommand(viperblockStartCmd)
@@ -657,59 +844,89 @@ func init() {
// Add NATS flags
natsCmd.PersistentFlags().Int("port", 4222, "NATS server port")
viper.BindEnv("port", "HIVE_NATS_PORT")
viper.BindEnv("port", "SPINIFEX_NATS_PORT")
viper.BindPFlag("port", natsCmd.PersistentFlags().Lookup("port"))
natsCmd.PersistentFlags().String("host", "0.0.0.0", "NATS server host")
viper.BindEnv("host", "HIVE_NATS_HOST")
viper.BindEnv("host", "SPINIFEX_NATS_HOST")
viper.BindPFlag("host", natsCmd.PersistentFlags().Lookup("host"))
natsCmd.PersistentFlags().Bool("debug", false, "Enable debug logging")
viper.BindEnv("debug", "HIVE_NATS_DEBUG")
viper.BindEnv("debug", "SPINIFEX_NATS_DEBUG")
viper.BindPFlag("debug", natsCmd.PersistentFlags().Lookup("debug"))
natsCmd.PersistentFlags().String("data-dir", "", "NATS data directory")
viper.BindEnv("data-dir", "HIVE_NATS_DATA_DIR")
viper.BindEnv("data-dir", "SPINIFEX_NATS_DATA_DIR")
viper.BindPFlag("data-dir", natsCmd.PersistentFlags().Lookup("data-dir"))
natsCmd.PersistentFlags().Bool("jetstream", false, "Enable JetStream")
viper.BindEnv("jetstream", "HIVE_NATS_JETSTREAM")
viper.BindEnv("jetstream", "SPINIFEX_NATS_JETSTREAM")
viper.BindPFlag("jetstream", natsCmd.PersistentFlags().Lookup("jetstream"))
// Hive
serviceCmd.AddCommand(hiveCmd)
// Spinifex
serviceCmd.AddCommand(spinifexCmd)
hiveCmd.AddCommand(hiveStartCmd)
hiveCmd.AddCommand(hiveStopCmd)
hiveCmd.AddCommand(hiveStatusCmd)
spinifexCmd.AddCommand(spinifexStartCmd)
spinifexCmd.AddCommand(spinifexStopCmd)
spinifexCmd.AddCommand(spinifexStatusCmd)
hiveCmd.PersistentFlags().String("wal-dir", "", "Write-ahead log (WAL) directory. Place on high-speed NVMe disk, or tmpfs for development.")
viper.BindEnv("wal-dir", "HIVE_WAL_DIR")
viper.BindPFlag("wal-dir", hiveCmd.PersistentFlags().Lookup("wal-dir"))
spinifexCmd.PersistentFlags().String("wal-dir", "", "Write-ahead log (WAL) directory. Place on high-speed NVMe disk, or tmpfs for development.")
viper.BindEnv("wal-dir", "SPINIFEX_WAL_DIR")
viper.BindPFlag("wal-dir", spinifexCmd.PersistentFlags().Lookup("wal-dir"))
// AWS GW
serviceCmd.AddCommand(awsgwCmd)
awsgwCmd.PersistentFlags().String("host", "0.0.0.0:9999", "AWS Gateway server host")
viper.BindEnv("host", "HIVE_AWSGW_HOST")
viper.BindEnv("host", "SPINIFEX_AWSGW_HOST")
viper.BindPFlag("host", awsgwCmd.PersistentFlags().Lookup("host"))
// AWS GW TLS Cert
awsgwCmd.PersistentFlags().String("tls-cert", "", "AWS Gateway TLS certificate")
viper.BindEnv("tls-cert", "HIVE_AWSGW_TLS_CERT")
viper.BindEnv("tls-cert", "SPINIFEX_AWSGW_TLS_CERT")
viper.BindPFlag("tls-cert", awsgwCmd.PersistentFlags().Lookup("tls-cert"))
// AWS GW TLS Key
awsgwCmd.PersistentFlags().String("tls-key", "", "AWS Gateway TLS key")
viper.BindEnv("tls-key", "HIVE_AWSGW_TLS_KEY")
viper.BindEnv("tls-key", "SPINIFEX_AWSGW_TLS_KEY")
viper.BindPFlag("tls-key", awsgwCmd.PersistentFlags().Lookup("tls-key"))
awsgwCmd.PersistentFlags().Bool("debug", false, "AWS Gateway Debug")
viper.BindEnv("debug", "HIVE_AWSGW_DEBUG")
viper.BindEnv("debug", "SPINIFEX_AWSGW_DEBUG")
viper.BindPFlag("debug", awsgwCmd.PersistentFlags().Lookup("debug"))
awsgwCmd.AddCommand(awsgwStartCmd)
awsgwCmd.AddCommand(awsgwStopCmd)
awsgwCmd.AddCommand(awsgwStatusCmd)
// spinifex-ui
serviceCmd.AddCommand(spinifexUICmd)
spinifexUICmd.PersistentFlags().Int("port", 3000, "spinifex-ui server port")
viper.BindEnv("spinifex-ui-port", "SPINIFEX_UI_PORT")
viper.BindPFlag("spinifex-ui-port", spinifexUICmd.PersistentFlags().Lookup("port"))
spinifexUICmd.PersistentFlags().String("host", "0.0.0.0", "spinifex-ui server host")
viper.BindEnv("spinifex-ui-host", "SPINIFEX_UI_HOST")
viper.BindPFlag("spinifex-ui-host", spinifexUICmd.PersistentFlags().Lookup("host"))
spinifexUICmd.PersistentFlags().String("tls-cert", "", "TLS certificate path")
viper.BindEnv("spinifex-ui-tls-cert", "SPINIFEX_UI_TLS_CERT")
viper.BindPFlag("spinifex-ui-tls-cert", spinifexUICmd.PersistentFlags().Lookup("tls-cert"))
spinifexUICmd.PersistentFlags().String("tls-key", "", "TLS key path")
viper.BindEnv("spinifex-ui-tls-key", "SPINIFEX_UI_TLS_KEY")
viper.BindPFlag("spinifex-ui-tls-key", spinifexUICmd.PersistentFlags().Lookup("tls-key"))
spinifexUICmd.AddCommand(spinifexUIStartCmd)
spinifexUICmd.AddCommand(spinifexUIStopCmd)
spinifexUICmd.AddCommand(spinifexUIStatusCmd)
// vpcd
serviceCmd.AddCommand(vpcdCmd)
vpcdCmd.AddCommand(vpcdStartCmd)
vpcdCmd.AddCommand(vpcdStopCmd)
vpcdCmd.AddCommand(vpcdStatusCmd)
}
+6
View File
@@ -0,0 +1,6 @@
# AWS Gateway configuration file in TOML format.
version = "1.0"
region = "{{.Region}}"
# Enable verbose debugging
debug = false
@@ -1,17 +1,17 @@
# NATS Server Configuration
# Generated by Hive admin init
# Generated by Spinifex admin init
# Server configuration
{{ if .BindIP }}
{{- if .BindIP }}
listen: {{.BindIP}}:4222
{{ else }}
{{- else }}
listen: 0.0.0.0:4222
{{ end }}
{{- end }}
{{- if and .ClusterBindIP .ClusterRoutes }}
{{ if and .ClusterBindIP .ClusterRoutes }}
cluster {
name: C1
name: {{ .ClusterName }}
listen: {{.ClusterBindIP}}:4248
routes = [
@@ -20,10 +20,9 @@ cluster {
{{- end }}
]
}
{{ end }}
{{- end }}
server_name: hive-nats-{{.Node}}
server_name: spinifex-nats-{{.Node}}
# Authorization
authorization {
@@ -49,9 +48,9 @@ max_payload: 1MB
max_pending: 64MB
# Monitoring
{{ if .BindIP }}
{{- if .BindIP }}
http: {{.BindIP}}:8222
{{ else }}
{{- else }}
http_port: 8222
{{ end }}
{{- end }}
@@ -0,0 +1,55 @@
# Predastore S3-compatible storage configuration file in TOML format.
# Generated for multi-node cluster
version = "1.0"
region = "{{.Region}}"
host = "0.0.0.0"
port = 8443
debug = false
[rs]
data = 2
parity = 1
{{- range .Nodes}}
[[db]]
id = {{.ID}}
host = "{{.Host}}"
port = 6660
path = "distributed/db/node-{{.ID}}/"
access_key_id = "{{$.AccessKey}}"
secret_access_key = "{{$.SecretKey}}"
{{- if eq .ID 1}}
leader = true
{{- end}}
{{- end}}
{{- range .Nodes}}
[[nodes]]
id = {{.ID}}
host = "{{.Host}}"
port = 9991
path = "distributed/nodes/node-{{.ID}}/"
{{- end}}
[[buckets]]
name = "predastore"
region = "{{.Region}}"
type = "distributed"
public = false
encryption = ""
[[auth]]
access_key_id = "{{.AccessKey}}"
secret_access_key = "{{.SecretKey}}"
policy = [
{ bucket = "predastore", actions = ["s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListAllMyBuckets"] },
]
# IAM authentication via NATS KV (enables multi-account S3 access)
[iam]
nats_url = "nats://{{.BindIP}}:4222"
nats_token = "{{.NatsToken}}"
master_key_path = "{{.ConfigDir}}/master.key"
access_keys_bucket = "spinifex-iam-access-keys"
@@ -79,4 +79,11 @@ access_key_id = "{{.AccessKey}}"
secret_access_key = "{{.SecretKey}}"
policy = [
{ bucket = "predastore", actions = ["s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListAllMyBuckets"] },
]
]
# IAM authentication via NATS KV (enables multi-account S3 access)
[iam]
nats_url = "nats://{{if eq .BindIP "0.0.0.0"}}localhost{{else}}{{.BindIP}}{{end}}:4222"
nats_token = "{{.NatsToken}}"
master_key_path = "{{.ConfigDir}}/master.key"
access_keys_bucket = "spinifex-iam-access-keys"
@@ -5,17 +5,20 @@ node = "{{.Node}}"
[nodes.{{.Node}}]
node = "{{.Node}}"
host = "{{.BindIP}}"
region = "{{.Region}}"
az = "{{.Az}}"
{{- if .Services}}
services = [{{range $i, $s := .Services}}{{if $i}}, {{end}}"{{$s}}"{{end}}]
{{- end}}
account_id = "{{.AccountID}}"
accesskey = "{{.AccessKey}}"
secretkey = "{{.SecretKey}}"
[nodes.{{.Node}}.daemon]
host = "{{.BindIP}}:{{.Port}}"
tlskey = "config/server.key"
tlscert = "config/server.pem"
dev_networking = true
[nodes.{{.Node}}.awsgw]
host = "{{.BindIP}}:9999"
@@ -32,9 +35,30 @@ token = "{{.NatsToken}}"
[nodes.{{.Node}}.nats.sub]
[nodes.{{.Node}}.viperblock]
shardwal = false
[nodes.{{.Node}}.vpcd]
ovn_nb_addr = "{{.OVNNBAddr}}"
ovn_sb_addr = "{{.OVNSBAddr}}"
[nodes.{{.Node}}.predastore]
host = "{{.BindIP}}:8443"
region = "{{.Region}}"
bucket = "predastore"
accesskey = "{{.AccessKey}}"
secretkey = "{{.SecretKey}}"
secretkey = "{{.SecretKey}}"
{{- if gt .PredastoreNodeID 0}}
node_id = {{.PredastoreNodeID}}
{{- end}}
{{- range .RemoteNodes}}
[nodes.{{.Name}}]
node = "{{.Name}}"
host = "{{.Host}}"
region = "{{.Region}}"
az = "{{.AZ}}"
{{- if .Services}}
services = [{{range $i, $s := .Services}}{{if $i}}, {{end}}"{{$s}}"{{end}}]
{{- end}}
{{- end}}
+161
View File
@@ -0,0 +1,161 @@
/*
Copyright © 2025 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cmd
import (
"encoding/json"
"fmt"
"os"
"sort"
"time"
"github.com/mulgadc/spinifex/spinifex/types"
"github.com/pterm/pterm"
"github.com/spf13/cobra"
)
var topCmd = &cobra.Command{
Use: "top",
Short: "Display cluster resource usage",
Long: `Display resource usage (CPU, memory) for cluster nodes.`,
}
var topNodesCmd = &cobra.Command{
Use: "nodes",
Short: "Display resource usage per node",
Long: `Display CPU and memory usage per node, plus a summary of available instance types across the cluster.`,
Run: runTopNodes,
}
func init() {
rootCmd.AddCommand(topCmd)
topCmd.AddCommand(topNodesCmd)
topCmd.PersistentFlags().Duration("timeout", 3*time.Second, "Timeout for collecting responses from nodes")
}
func runTopNodes(cmd *cobra.Command, args []string) {
cfg, nc, err := loadConfigAndConnect()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
defer nc.Close()
timeout, _ := cmd.Flags().GetDuration("timeout")
responses, err := collectResponses(nc, "spinifex.node.status", timeout)
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
respondedNodes := make(map[string]types.NodeStatusResponse)
for _, data := range responses {
var resp types.NodeStatusResponse
if err := json.Unmarshal(data, &resp); err != nil {
continue
}
respondedNodes[resp.Node] = resp
}
// Node resource table: union of config-known nodes + NATS responders
nodeSet := make(map[string]struct{})
for name := range cfg.Nodes {
nodeSet[name] = struct{}{}
}
for name := range respondedNodes {
nodeSet[name] = struct{}{}
}
nodeNames := make([]string, 0, len(nodeSet))
for name := range nodeSet {
nodeNames = append(nodeNames, name)
}
sort.Strings(nodeNames)
nodeTable := pterm.TableData{
{"NAME", "CPU (used/total)", "MEM (used/total)", "VMs"},
}
// Aggregate instance type capacity across all nodes
capacityMap := make(map[string]*aggregatedCap)
for _, name := range nodeNames {
if resp, ok := respondedNodes[name]; ok {
nodeTable = append(nodeTable, []string{
resp.Node,
fmt.Sprintf("%d/%d", resp.AllocVCPU, resp.TotalVCPU),
fmt.Sprintf("%s/%s", formatMemGB(resp.AllocMemGB), formatMemGB(resp.TotalMemGB)),
fmt.Sprintf("%d", resp.VMCount),
})
for _, cap := range resp.InstanceTypes {
if agg, ok := capacityMap[cap.Name]; ok {
agg.Available += cap.Available
} else {
capacityMap[cap.Name] = &aggregatedCap{
VCPU: cap.VCPU,
MemoryGB: cap.MemoryGB,
Available: cap.Available,
}
}
}
} else {
nodeTable = append(nodeTable, []string{
name,
"-",
"-",
"-",
})
}
}
pterm.DefaultTable.WithHasHeader().WithLeftAlignment().WithData(nodeTable).Render()
// Instance type capacity summary
if len(capacityMap) == 0 {
return
}
fmt.Println()
capNames := make([]string, 0, len(capacityMap))
for name := range capacityMap {
capNames = append(capNames, name)
}
sort.Strings(capNames)
capTable := pterm.TableData{
{"INSTANCE TYPE", "AVAILABLE", "VCPU", "MEMORY"},
}
for _, name := range capNames {
agg := capacityMap[name]
capTable = append(capTable, []string{
name,
fmt.Sprintf("%d", agg.Available),
fmt.Sprintf("%d", agg.VCPU),
formatMemGB(agg.MemoryGB),
})
}
pterm.DefaultTable.WithHasHeader().WithLeftAlignment().WithData(capTable).Render()
}
type aggregatedCap struct {
VCPU int
MemoryGB float64
Available int
}
+27
View File
@@ -0,0 +1,27 @@
package cmd
import (
"fmt"
"runtime"
"github.com/spf13/cobra"
)
// Version variables set via ldflags at build time.
// Example: go build -ldflags "-X github.com/mulgadc/spinifex/cmd/spinifex/cmd.Version=v1.0.0"
var (
Version = "dev"
Commit = "unknown"
)
var versionCmd = &cobra.Command{
Use: "version",
Short: "Print the Spinifex version",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("spinifex %s (%s) %s/%s\n", Version, Commit, runtime.GOOS, runtime.GOARCH)
},
}
func init() {
rootCmd.AddCommand(versionCmd)
}
+29
View File
@@ -0,0 +1,29 @@
package cmd
import (
"testing"
)
func TestVersionVarsExist(t *testing.T) {
// Verify the version variables are set (defaults when not injected via ldflags)
if Version == "" {
t.Error("Version should not be empty")
}
if Commit == "" {
t.Error("Commit should not be empty")
}
}
func TestVersionCommand(t *testing.T) {
// Verify the version command is registered
found := false
for _, cmd := range rootCmd.Commands() {
if cmd.Use == "version" {
found = true
break
}
}
if !found {
t.Error("version command not registered on rootCmd")
}
}
+2 -2
View File
@@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
@@ -15,7 +15,7 @@ limitations under the License.
*/
package main
import "github.com/mulgadc/hive/cmd/hive/cmd"
import "github.com/mulgadc/spinifex/cmd/spinifex/cmd"
func main() {
cmd.Execute()
+44 -44
View File
@@ -1,71 +1,71 @@
# Hive Systemd Service
# Spinifex Systemd Service
This directory contains the systemd service configuration for running Hive services on Ubuntu and Debian systems.
This directory contains the systemd service configuration for running Spinifex services on Ubuntu and Debian systems.
## Files
- `hive.service` - The systemd service file that manages all Hive services
- `install-hive-service.sh` - Installation script to set up the service
- `spinifex.service` - The systemd service file that manages all Spinifex services
- `install-spinifex-service.sh` - Installation script to set up the service
- `README.md` - This documentation file
## Installation
### Prerequisites
1. Ensure Hive is installed in `/opt/hive/`
2. Ensure the `hive` binary is available at `/opt/hive/bin/hive`
1. Ensure Spinifex is installed in `/opt/spinifex/`
2. Ensure the `spx` binary is available at `/opt/spinifex/bin/spx`
3. Run the installation script as root
### Setup Configuration
```
/opt/hive/bin/hive admin init
/opt/spinifex/bin/spx admin init
```
This will init the Hive environment and create the default configuration files in the home directory, e.g `~/hive/`.
This will init the Spinifex environment and create the default configuration files in the home directory, e.g `~/spinifex/`.
```
/home/username/hive:
/home/username/spinifex:
amis config images logs nats predastore state viperblock volumes
/home/username/hive/amis:
/home/username/spinifex/amis:
/home/username/hive/config:
awsgw hive.toml nats predastore server.key server.pem
/home/username/spinifex/config:
awsgw spinifex.toml nats predastore server.key server.pem
/home/username/hive/config/awsgw:
/home/username/spinifex/config/awsgw:
awsgw.toml
/home/username/hive/config/nats:
/home/username/spinifex/config/nats:
nats.conf
/home/username/hive/config/predastore:
/home/username/spinifex/config/predastore:
predastore.toml
/home/username/hive/images:
/home/username/spinifex/images:
/home/username/hive/logs:
awsgw.log awsgw.pid hive.log hive.pid nats.log nats.pid predastore.log predastore.pid viperblock.log viperblock.pid
/home/username/spinifex/logs:
awsgw.log awsgw.pid spinifex.log spinifex.pid nats.log nats.pid predastore.log predastore.pid viperblock.log viperblock.pid
/home/username/hive/nats:
/home/username/spinifex/nats:
/home/username/hive/predastore:
/home/username/spinifex/predastore:
data
/home/username/hive/predastore/data:
/home/username/spinifex/predastore/data:
predastore
/home/ben/hive/state:
/home/ben/spinifex/state:
/home/ben/hive/viperblock:
/home/ben/spinifex/viperblock:
/home/ben/hive/volumes:
/home/ben/spinifex/volumes:
```
### Quick Installation
```bash
sudo ./config/install-hive-service.sh
sudo ./config/install-spinifex-service.sh
```
### Manual Installation
@@ -74,7 +74,7 @@ If you prefer to install manually:
1. Copy the service file:
```bash
sudo cp config/hive.service /etc/systemd/system/
sudo cp config/spinifex.service /etc/systemd/system/
```
2. Reload systemd:
@@ -84,44 +84,44 @@ If you prefer to install manually:
3. Enable the service:
```bash
sudo systemctl enable hive.service
sudo systemctl enable spinifex.service
```
## Service Management
### Start the service
```bash
sudo systemctl start hive
sudo systemctl start spinifex
```
### Stop the service
```bash
sudo systemctl stop hive
sudo systemctl stop spinifex
```
### Check service status
```bash
sudo systemctl status hive
sudo systemctl status spinifex
```
### Restart the service
```bash
sudo systemctl restart hive
sudo systemctl restart spinifex
```
### View service logs
```bash
sudo journalctl -u hive -f
sudo journalctl -u spinifex -f
```
### Disable auto-start
```bash
sudo systemctl disable hive
sudo systemctl disable spinifex
```
## Service Details
The `hive.service` file:
The `spinifex.service` file:
- Starts services in the correct order: NATS → Predastore → Viperblock
- Includes proper delays between service starts to ensure dependencies are ready
@@ -143,41 +143,41 @@ The service starts the following components in order:
### Check if the service is running
```bash
sudo systemctl is-active hive
sudo systemctl is-active spinifex
```
### View recent logs
```bash
sudo journalctl -u hive --since "1 hour ago"
sudo journalctl -u spinifex --since "1 hour ago"
```
### Check service configuration
```bash
sudo systemctl cat hive
sudo systemctl cat spinifex
```
### Verify binary exists
```bash
ls -la /opt/hive/bin/hive
ls -la /opt/spinifex/bin/spx
```
### Test service commands manually
```bash
/opt/hive/bin/hive service nats start
/opt/hive/bin/hive service predastore start
/opt/hive/bin/hive service viperblock start
/opt/spinifex/bin/spx service nats start
/opt/spinifex/bin/spx service predastore start
/opt/spinifex/bin/spx service viperblock start
```
## Customization
If you need to modify the service configuration:
1. Edit the service file: `sudo nano /etc/systemd/system/hive.service`
1. Edit the service file: `sudo nano /etc/systemd/system/spinifex.service`
2. Reload systemd: `sudo systemctl daemon-reload`
3. Restart the service: `sudo systemctl restart hive`
3. Restart the service: `sudo systemctl restart spinifex`
## Security Notes
- The service runs as root, which is necessary for system-level services
- Security hardening is enabled with `ProtectSystem=strict` and `NoNewPrivileges=true`
- Only specific paths are writable: `/opt/hive`, `/var/log`, `/var/run`
- Only specific paths are writable: `/opt/spinifex`, `/var/log`, `/var/run`
-49
View File
@@ -1,49 +0,0 @@
[Unit]
Description=Hive Services
Documentation=https://github.com/mulgadc/hive
After=network.target
Wants=network.target
[Service]
Type=oneshot
RemainAfterExit=yes
User=root
Group=root
WorkingDirectory=/opt/hive
Environment=PATH=/opt/hive/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Start NATS service
ExecStart=/opt/hive/bin/hive service nats start
ExecStartPost=/bin/sleep 3
# Start Predastore service
ExecStart=/opt/hive/bin/hive service predastore start
ExecStartPost=/bin/sleep 2
# Start Viperblock service
ExecStart=/opt/hive/bin/hive service viperblock start
# Stop services in reverse order
ExecStop=/opt/hive/bin/hive service viperblock stop
ExecStopPost=/bin/sleep 2
ExecStop=/opt/hive/bin/hive service predastore stop
ExecStopPost=/bin/sleep 2
ExecStop=/opt/hive/bin/hive service nats stop
# Restart policy
Restart=on-failure
RestartSec=10
# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/hive /var/log /var/run
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=hive
[Install]
WantedBy=multi-user.target
@@ -1,7 +1,7 @@
#!/bin/bash
# Hive Service Installation Script
# This script installs the hive systemd service on Ubuntu/Debian systems
# Spinifex Service Installation Script
# This script installs the spinifex systemd service on Ubuntu/Debian systems
set -e
@@ -38,7 +38,7 @@ fi
# Get the directory where this script is located
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SERVICE_FILE="$SCRIPT_DIR/hive.service"
SERVICE_FILE="$SCRIPT_DIR/spinifex.service"
# Check if service file exists
if [[ ! -f "$SERVICE_FILE" ]]; then
@@ -46,7 +46,7 @@ if [[ ! -f "$SERVICE_FILE" ]]; then
exit 1
fi
print_status "Installing Hive systemd service..."
print_status "Installing Spinifex systemd service..."
# Copy service file to systemd directory
cp "$SERVICE_FILE" /etc/systemd/system/
@@ -55,20 +55,20 @@ cp "$SERVICE_FILE" /etc/systemd/system/
systemctl daemon-reload
# Enable the service to start on boot
systemctl enable hive.service
systemctl enable spinifex.service
print_status "Service installed successfully!"
print_status "You can now use the following commands:"
echo " systemctl start hive # Start the service"
echo " systemctl stop hive # Stop the service"
echo " systemctl status hive # Check service status"
echo " systemctl restart hive # Restart the service"
echo " journalctl -u hive -f # View service logs"
echo " systemctl start spinifex # Start the service"
echo " systemctl stop spinifex # Stop the service"
echo " systemctl status spinifex # Check service status"
echo " systemctl restart spinifex # Restart the service"
echo " journalctl -u spinifex -f # View service logs"
# Check if hive binary exists
if [[ ! -f "/opt/hive/bin/hive" ]]; then
print_warning "Hive binary not found at /opt/hive/bin/hive"
print_warning "Please ensure hive is installed in /opt/hive before starting the service"
# Check if spinifex binary exists
if [[ ! -f "/opt/spinifex/bin/spinifex" ]]; then
print_warning "Spinifex binary not found at /opt/spinifex/bin/spinifex"
print_warning "Please ensure spinifex is installed in /opt/spinifex before starting the service"
fi
print_status "Installation complete!"
+49
View File
@@ -0,0 +1,49 @@
[Unit]
Description=Spinifex Services
Documentation=https://github.com/mulgadc/spinifex
After=network.target
Wants=network.target
[Service]
Type=oneshot
RemainAfterExit=yes
User=root
Group=root
WorkingDirectory=/opt/spinifex
Environment=PATH=/opt/spinifex/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Start NATS service
ExecStart=/opt/spinifex/bin/spinifex service nats start
ExecStartPost=/bin/sleep 3
# Start Predastore service
ExecStart=/opt/spinifex/bin/spinifex service predastore start
ExecStartPost=/bin/sleep 2
# Start Viperblock service
ExecStart=/opt/spinifex/bin/spinifex service viperblock start
# Stop services in reverse order
ExecStop=/opt/spinifex/bin/spinifex service viperblock stop
ExecStopPost=/bin/sleep 2
ExecStop=/opt/spinifex/bin/spinifex service predastore stop
ExecStopPost=/bin/sleep 2
ExecStop=/opt/spinifex/bin/spinifex service nats stop
# Restart policy
Restart=on-failure
RestartSec=10
# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/spinifex /var/log /var/run
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=spinifex
[Install]
WantedBy=multi-user.target
+479
View File
@@ -0,0 +1,479 @@
# Command Implementation Matrix
## Spinifex Admin CLI
Platform management commands not exposed via the AWS gateway API. These are CLI-only commands.
### Service Management
Service lifecycle commands for starting, stopping, and checking status of all Spinifex cluster services. Each service subcommand supports `start`, `stop`, and `status` operations.
| Command | Flags | Prerequisites | Basic Logic | Status |
|---------|-------|---------------|-------------|--------|
| `spx service predastore start` | `--port` (default: 8443), `--host` (default: 0.0.0.0), `--base-path`, `--config-path`, `--debug`, `--tls-cert`, `--tls-key`, `--backend` (distributed/filesystem, default: distributed), `--node-id` (default: 0), `--pprof`, `--pprof-output` | TLS cert/key, base-path, config-path required | Creates predastore service instance with S3-compatible storage backend → starts service | **DONE** |
| `spx service predastore stop` | — | Predastore must be running | Stops the predastore service | **DONE** |
| `spx service predastore status` | — | None | Reports predastore service status | **DONE** |
| `spx service viperblock start` | `--s3-host` (default: 0.0.0.0:8443), `--s3-bucket` (default: predastore), `--s3-region` (default: ap-southeast-2), `--plugin-path` (default: /opt/spinifex/lib/nbdkit-viperblock-plugin.so) | `--config` required, plugin-path must exist on disk | Loads cluster config → connects to NATS and Predastore → starts viperblock block storage service with NBD plugin | **DONE** |
| `spx service viperblock stop` | — | Viperblock must be running | Stops the viperblock service | **DONE** |
| `spx service viperblock status` | — | None | Reports viperblock service status | **DONE** |
| `spx service nats start` | `--port` (default: 4222), `--host` (default: 0.0.0.0), `--debug`, `--data-dir`, `--jetstream` | None | Starts embedded NATS server with optional JetStream | **DONE** |
| `spx service nats stop` | — | NATS must be running | Stops the NATS service | **DONE** |
| `spx service nats status` | — | None | Reports NATS service status | **DONE** |
| `spx service spinifex start` | `--wal-dir` | `--config` required | Loads cluster config → starts spinifex daemon (VM orchestration, NATS subscriptions, health endpoint) | **DONE** |
| `spx service spinifex stop` | — | Spinifex daemon must be running | Stops the spinifex daemon service | **DONE** |
| `spx service spinifex status` | — | None | Reports spinifex daemon service status | **DONE** |
| `spx service awsgw start` | `--host` (default: 0.0.0.0:9999), `--tls-cert`, `--tls-key`, `--debug` | `--config` required | Loads cluster config → starts AWS-compatible gateway with SigV4 auth, IAM policy enforcement, TLS | **DONE** |
| `spx service awsgw stop` | — | AWS gateway must be running | Stops the AWS gateway service | **DONE** |
| `spx service awsgw status` | — | None | Reports AWS gateway service status | **DONE** |
| `spx service spinifex-ui start` | `--port` (default: 3000), `--host` (default: 0.0.0.0), `--tls-cert`, `--tls-key` | None | Starts embedded web UI server serving the React frontend. Aliases: `ui`, `spinifexui` | **DONE** |
| `spx service spinifex-ui stop` | — | spinifex-ui must be running | Stops the spinifex-ui service | **DONE** |
| `spx service spinifex-ui status` | — | None | Reports spinifex-ui service status | **DONE** |
| `spx service vpcd start` | — | `--config` required, OVN/OVS installed | Loads cluster config → starts VPC daemon (subscribes to `vpc.*` NATS events, translates to OVN logical switches/ports/routers) | **DONE** |
| `spx service vpcd stop` | — | vpcd must be running | Stops the vpcd service | **DONE** |
| `spx service vpcd status` | — | None | Reports vpcd service status | **DONE** |
### Cluster Inspection
Operational commands for inspecting cluster state. These fan out NATS requests to all nodes and aggregate responses.
| Command | Flags | Prerequisites | Basic Logic | Output Columns | Status |
|---------|-------|---------------|-------------|----------------|--------|
| `spx get nodes` | `--timeout` (default: 3s) | Cluster must be running (NATS) | Loads config → publishes to `spinifex.node.status` fan-out topic → collects responses within timeout → merges config-known nodes with NATS responders → nodes that don't respond shown as `NotReady` | NAME, STATUS, IP, REGION, AZ, UPTIME, VMs, SERVICES | **DONE** |
| `spx get vms` | `--timeout` (default: 3s) | Cluster must be running (NATS) | Publishes to `spinifex.node.vms` fan-out topic → collects VM info from all nodes → sorts by node then instance ID → prints table. Alias: `spx get instances` | INSTANCE, STATUS, TYPE, VCPU, MEM, NODE, IP, AGE | **DONE** |
### Resource Monitoring
| Command | Flags | Prerequisites | Basic Logic | Output | Status |
|---------|-------|---------------|-------------|--------|--------|
| `spx top nodes` | `--timeout` (default: 3s) | Cluster must be running (NATS) | Publishes to `spinifex.node.status` fan-out topic → collects CPU/memory usage per node → aggregates instance type capacity across all nodes → prints two tables: per-node resource usage and cluster-wide instance type availability | Table 1: NAME, CPU (used/total), MEM (used/total), VMs. Table 2: INSTANCE TYPE, AVAILABLE, VCPU, MEMORY | **DONE** |
### Cluster Initialization
| Command | Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------|---------------|-------------|------------|--------|
| `spx admin init` | `--nodes`, `--node`, `--bind`, `--port`, `--region`, `--az`, `--cluster-name`, `--cluster-bind`, `--cluster-routes`, `--predastore-nodes`, `--services`, `--formation-timeout`, `--force` | None (first-time setup) | Generates root IAM credentials (AKIA-prefixed access key + secret) → creates master.key (AES-256, 32 bytes, 0600) → writes bootstrap.json (consumed on first start) → generates CA + server TLS certificates → creates NATS config with auth token → writes spinifex.toml, awsgw.toml, predastore.toml → configures AWS CLI `spx` profile → creates directory structure under `~/spinifex/` | 1. Init creates all config files<br>2. Root credentials printed once<br>3. master.key is 32 bytes, mode 0600<br>4. bootstrap.json consumed on first start<br>5. `--force` re-initializes existing config<br>6. AWS CLI profile `spx` auto-configured | **DONE** |
| `spx admin join` | `--host` (required), `--node` (required), `--bind`, `--port`, `--region`, `--az`, `--cluster-bind`, `--cluster-routes`, `--data-dir`, `--services` | Leader node must be running | Connects to leader node → retrieves cluster configuration → configures local node to join cluster and participate in distributed operations | 1. Join existing cluster<br>2. Missing host (error)<br>3. Missing node name (error) | **DONE** |
### Cluster Operations
| Command | Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------|---------------|-------------|------------|--------|
| `spx admin cluster shutdown` | — | Cluster must be running | Gracefully shuts down the entire cluster | 1. Shutdown running cluster<br>2. All nodes stop cleanly | **DONE** |
### Account Management
| Command | Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------|---------------|-------------|------------|--------|
| `spx admin account create` | `--name` | Cluster must be running (NATS), master key must exist | Connects to NATS → CAS loop on `spinifex-account-counter:next_id` for sequential 12-digit ID → creates Account record in `spinifex-accounts` KV → creates `admin` user in new account → creates access key for admin → creates AdministratorAccess policy (Action:*, Resource:*) → attaches policy → prints credentials | 1. Create first account (ID 000000000001)<br>2. Sequential IDs (000000000002, etc.)<br>3. Admin user has full access<br>4. Credentials printed once | **DONE** |
| `spx admin account list` | — | Cluster must be running (NATS), master key must exist | Connects to NATS → IAMService.ListAccounts() → prints table with Account ID, Name, Status, Created | 1. List all accounts<br>2. Empty list (only global account) | **DONE** |
### Image Management
| Command | Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------|---------------|-------------|------------|--------|
| `spx admin images import` | `--name` | Cluster must be running, image file must exist locally | Imports a local disk image (QCOW2) as an AMI → uploads to Predastore S3 → registers as available AMI | 1. Import valid QCOW2 image<br>2. Image appears in `describe-images` | **DONE** |
| `spx admin images list` | — | None | Lists available OS images that can be imported or downloaded | 1. List available images | **DONE** |
## AWS Commands
### EC2 - Instance Management
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `run-instances` | `--image-id`, `--instance-type`, `--count` (Min/MaxCount), `--key-name`, `--user-data`, `--subnet-id` (auto-creates ENI, assigns private IP), `--block-device-mappings` (DeviceName, VolumeSize, VolumeType, Iops, DeleteOnTermination) | `--security-group-ids`, `--placement`, `--tag-specifications`, `--dry-run`, `--client-token`, `--disable-api-termination`, `--ebs-optimized`, `--iam-instance-profile`, `--network-interfaces`, `--private-ip-address`, `--monitoring`, `--credit-specification`, `--cpu-options`, `--metadata-options`, `--launch-template`, `--hibernate-options` | `describe-images` (AMI must exist), `create-key-pair` (optional), VPC/SG (optional) | Gateway parses AWS query → NATS `ec2.runinstances` → daemon creates QEMU/KVM VM with viperblock-backed root volume via NBD → if SubnetId provided, auto-creates ENI with private IP → cloud-init injects user-data/keys → returns reservation with instance ID | 1. Launch with valid AMI and key pair<br>2. Launch with invalid AMI ID (error)<br>3. Launch with block device mappings (custom volume size)<br>4. Launch multiple instances (MinCount/MaxCount)<br>5. Launch with subnet-id (auto-creates ENI)<br>6. Invalid instance type returns error | **DONE** |
| `describe-instances` | `--instance-ids` | `--filters`, `--max-results`, `--next-token`, `--dry-run` | None | Gateway fans out NATS `ec2.DescribeInstances` to all nodes (no queue group) → each daemon returns local instances → gateway aggregates and returns combined list | 1. Describe all instances (no filter)<br>2. Describe by instance ID<br>3. Describe with filters (e.g. instance-state-name)<br>4. Instance not found returns empty set<br>5. Multi-node aggregation returns instances from all nodes | **DONE** |
| `start-instances` | `--instance-ids` | `--dry-run`, `--force` | `run-instances` (instance must exist in stopped state) | Gateway sends NATS `ec2.cmd.{instance-id}` → daemon restarts stopped QEMU process with same config → state transitions stopped→pending→running | 1. Start a stopped instance<br>2. Start already-running instance (error: IncorrectInstanceState)<br>3. Start with invalid instance ID<br>4. Verify volumes re-mount on start | **DONE** |
| `stop-instances` | `--instance-ids` | `--force`, `--hibernate`, `--dry-run` | `run-instances` (instance must be running) | Gateway sends NATS to target node → daemon issues QMP `system_powerdown` for graceful shutdown → monitors heartbeat until QEMU exits → state transitions running→stopping→stopped | 1. Graceful stop of running instance<br>2. Force stop (kills QEMU process)<br>3. Stop already-stopped instance (error)<br>4. Verify ~30s heartbeat detection | **DONE** |
| `terminate-instances` | `--instance-ids`, `DeleteOnTermination` (per-volume flag, default true) | `--dry-run` | `run-instances` (instance must exist) | Gateway sends NATS to target node → daemon kills QEMU process → cleans up NBD mounts → deletes volumes with `DeleteOnTermination=true` via `volumeService.DeleteVolume()` (S3 cleanup of vol/, vol-efi/, vol-cloudinit/) → internal volumes (EFI, cloud-init) always cleaned up via `ebs.delete` NATS → volumes with `DeleteOnTermination=false` left in available state → state→terminated | 1. Terminate running instance<br>2. Terminate stopped instance<br>3. Terminate with DeleteOnTermination=true deletes volumes<br>4. Terminate with DeleteOnTermination=false preserves volumes<br>5. Terminate already-terminated (idempotent)<br>6. Internal volumes (EFI, cloud-init) always cleaned up<br>7. Invalid instance ID | **DONE** |
| `reboot-instances` | — | `--instance-ids`, `--dry-run` | `run-instances` (instance must be running) | Gateway sends NATS to target node → daemon issues QMP `system_reset` → instance reboots without stopping | 1. Reboot running instance<br>2. Reboot stopped instance (error)<br>3. Verify instance stays in running state after reboot | **NOT STARTED** (parser test exists) |
| `describe-instance-types` | `--filters` (capacity filter only) | `--instance-types`, `--max-results`, `--next-token`, `--dry-run`, all other filters | None | Gateway fans out NATS `ec2.DescribeInstanceTypes` to all nodes → each daemon reports supported types (t3.micro/small/medium/large) with vCPU/memory specs → gateway deduplicates and returns | 1. List all instance types<br>2. Filter by specific type<br>3. Filter with `capacity=true` shows available slots<br>4. Verify vCPU/memory specs match hardware | **DONE** |
| `modify-instance-attribute` | `--instance-id`, `--instance-type`, `--user-data` | `--disable-api-termination`, `--ebs-optimized`, `--source-dest-check`, `--instance-initiated-shutdown-behavior`, `--block-device-mappings`, `--groups`, `--ena-support`, `--sriov-net-support` | Instance must be stopped (in NATS KV) | Gateway validates input (exactly one attribute per call, instance ID format) → NATS `ec2.ModifyInstanceAttribute` with `spinifex-workers` queue group → daemon loads stopped instance from JetStream KV → applies attribute change → writes back to KV → returns `{}` on success. **InstanceType**: updates vm.InstanceType, Config, and Instance fields; clears StateReason (enables recovery from instance-type-missing bug). **UserData**: stores decoded content in vm.UserData and re-encodes to base64 for RunInstancesInput (cloud-init on next start). No instance type pre-validation (matches AWS — invalid types accepted, fail at StartInstances time). | 1. Change instance type while stopped<br>2. Change user data while stopped<br>3. Modify running instance (error: NotFound — running instances not in KV)<br>4. Instance not found (error: InvalidInstanceID.NotFound)<br>5. Instance not stopped (error: IncorrectInstanceState)<br>6. Invalid instance type accepted (fails on start with InsufficientInstanceCapacity)<br>7. StateReason cleared on type change (recovery from capacity-unavailable)<br>8. Missing/malformed instance ID (error: InvalidInstanceID.Malformed)<br>9. No attribute set (error: InvalidParameterValue)<br>10. Multiple attributes in one call (error: InvalidParameterValue) | **DONE** |
| `get-console-output` | `--instance-id` | `--latest` (always returns latest), `--dry-run` | Instance must be running on a node | Gateway sends NATS `ec2.{instanceId}.GetConsoleOutput` (per-instance topic, routed to owning node) → daemon reads console log file from disk → returns last 64KB base64-encoded with timestamp. Always available regardless of serial console access setting (matches AWS behavior). | 1. Get output from running instance<br>2. Empty log file returns empty output<br>3. Instance not found (error: InvalidInstanceID.NotFound) | **DONE** |
| `monitor-instances` | — | `--instance-ids` | Instance must exist | Enable basic monitoring (CPU, disk, network) → store metrics in NATS KV → return monitoring state | 1. Enable monitoring<br>2. Verify monitoring state in describe-instances | **NOT STARTED** |
| `unmonitor-instances` | — | `--instance-ids` | Instance must exist | Disable detailed monitoring → revert to basic monitoring → return monitoring state | 1. Disable monitoring<br>2. Verify monitoring state reverts in describe-instances | **NOT STARTED** |
### EC2 - Key Pair Management
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-key-pair` | `--key-name`, `--key-type` (rsa/ed25519) | `--key-format` (pem/ppk), `--tag-specifications`, `--dry-run` | None | NATS `ec2.CreateKeyPair` → daemon generates SSH keypair → stores public key in Predastore S3 (`/bucket/ec2/{name}.pub`) → returns private key material (one-time) | 1. Create RSA key pair<br>2. Create ED25519 key pair<br>3. Duplicate key name (error: InvalidKeyPair.Duplicate)<br>4. Verify key material returned only on creation | **DONE** |
| `describe-key-pairs` | `--key-names`, `--key-pair-ids` | `--filters`, `--max-results`, `--dry-run` | None | NATS `ec2.DescribeKeyPairs` → daemon lists public keys from Predastore S3 → returns key names and fingerprints | 1. List all key pairs<br>2. Filter by key name<br>3. Filter by key pair ID<br>4. Non-existent key returns empty | **DONE** |
| `delete-key-pair` | `--key-name`, `--key-pair-id` | `--dry-run` | Key must exist | NATS `ec2.DeleteKeyPair` → daemon deletes public key from Predastore S3 → returns success | 1. Delete existing key pair<br>2. Delete non-existent key (idempotent, no error)<br>3. Verify key no longer in describe-key-pairs | **DONE** |
| `import-key-pair` | `--key-name`, `--public-key-material` | `--tag-specifications`, `--dry-run` | None | NATS `ec2.ImportKeyPair` → daemon stores provided public key in Predastore S3 → returns key name and fingerprint | 1. Import valid SSH public key<br>2. Import invalid key material (error)<br>3. Import duplicate name (error)<br>4. Verify imported key usable with run-instances | **DONE** |
### EC2 - AMI Image Management
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `describe-images` | `--image-ids` (format validation only), `--owners` (self, account ID, alias) | `--executable-users`, `--filters`, `--include-deprecated`, `--include-disabled`, `--max-results`, `--next-token`, `--dry-run` | None | NATS `ec2.DescribeImages` → daemon reads AMI metadata from Predastore S3 buckets (ami-*) → filters by ImageIds and Owners → returns image list with state, architecture, block device mappings | 1. List all images<br>2. Filter by image ID<br>3. Filter by owner (self/amazon)<br>4. Non-existent AMI returns empty<br>5. Verify metadata fields (architecture, state, rootDeviceName) | **DONE** |
| `create-image` | `--instance-id`, `--name`, `--description`, `--tag-specifications` | `--no-reboot`, `--block-device-mappings`, `--dry-run` | Instance must exist (running or stopped) | Gateway validates input → NATS `ec2.{instanceId}.CreateImage` (per-instance topic) → daemon extracts root volume → snapshots via `ebs.snapshot` NATS (running) or offline S3 copy (stopped) → creates AMI metadata in S3 (`{amiId}/config.json`) → stores tags → returns ami-ID. Duplicate AMI name validation enforced. | 1. Create image from running instance<br>2. Create image from stopped instance<br>3. Invalid instance ID (error)<br>4. Duplicate AMI name (error)<br>5. Verify new AMI appears in describe-images<br>6. Launch new instance from created AMI | **DONE** |
| `register-image` | — | `--name`, `--description`, `--architecture`, `--root-device-name`, `--virtualization-type`, `--block-device-mappings`, `--boot-mode`, `--ena-support`, `--sriov-net-support`, `--tpm-support`, `--imds-support` | Image data must exist in S3 | Create AMI metadata entry in Predastore → assign ami-ID → set state to available → return ami-ID | 1. Register with valid metadata<br>2. Missing required name (error)<br>3. Verify registered image in describe-images | **NOT STARTED** |
| `deregister-image` | — | `--image-id`, `--dry-run` | AMI must exist, no instances using it | Mark AMI metadata as deregistered in S3 → AMI no longer available for new launches → existing instances unaffected | 1. Deregister existing AMI<br>2. Deregister non-existent AMI (error)<br>3. Verify deregistered AMI not in describe-images<br>4. Existing instances from AMI still run | **NOT STARTED** |
| `copy-image` | — | `--source-image-id`, `--source-region`, `--name`, `--description`, `--encrypted`, `--kms-key-id`, `--client-token`, `--copy-image-tags`, `--tag-specifications` | Source AMI must exist | Copy AMI data between regions/nodes in Predastore → create new AMI metadata with new ami-ID → return new ami-ID | 1. Copy image within same region<br>2. Copy non-existent image (error)<br>3. Verify copied image independent of source | **NOT STARTED** |
| `import-image` | — | `--disk-containers` (Format, Url/S3Bucket+S3Key), `--description`, `--architecture`, `--platform` | S3 bucket with disk image | Download disk image from S3 → convert format (VMDK/VHD/RAW→QCOW2) → create viperblock volume → register as AMI | 1. Import QCOW2 image<br>2. Import RAW image<br>3. Invalid format (error)<br>4. Verify imported image launchable | **NOT STARTED** |
| `describe-image-attribute` | — | `--image-id`, `--attribute`, `--dry-run` | AMI must exist | Read specific attribute from AMI metadata in S3 → return attribute value | 1. Get description attribute<br>2. Get launch permission<br>3. Invalid attribute name (error) | **NOT STARTED** |
| `modify-image-attribute` | — | `--image-id`, `--launch-permission`, `--description`, `--operation-type`, `--user-ids`, `--user-groups`, `--organization-arns`, `--dry-run` | AMI must exist | Update specific attribute in AMI metadata → persist to S3 | 1. Add launch permission<br>2. Modify description<br>3. Invalid AMI (error) | **NOT STARTED** |
| `reset-image-attribute` | — | `--image-id`, `--attribute`, `--dry-run` | AMI must exist | Reset attribute to default value in AMI metadata | 1. Reset launch permission to owner-only<br>2. Invalid AMI (error) | **NOT STARTED** |
### EC2 - Volume (EBS) Management
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `describe-volumes` | `--volume-ids` (fast-path lookup), `DeleteOnTermination` (from persisted VolumeMetadata) | `--filters`, `--max-results`, `--next-token`, `--dry-run` | None | NATS `ec2.DescribeVolumes` → daemon queries viperblock for volume metadata → returns volume list with state, size, attachments, type, DeleteOnTermination flag | 1. List all volumes<br>2. Filter by volume ID<br>3. Filter by attachment state<br>4. Non-existent volume returns empty<br>5. DeleteOnTermination reflects persisted value | **DONE** |
| `modify-volume` | `--volume-id`, `--size`, `--volume-type`, `--iops` | `--throughput`, `--dry-run`, `--multi-attach-enabled` | Volume must exist | NATS `ec2.ModifyVolume` → daemon sends resize request to viperblock → NBD does not support live resize, instance must be stopped → returns modification state | 1. Increase volume size<br>2. Modify volume type<br>3. Decrease size (error - not supported)<br>4. Modify attached volume (requires stop/start) | **DONE** |
| `create-volume` | `--size`, `--availability-zone`, `--volume-type` (gp3 only), `--snapshot-id` (creates volume from snapshot) | `--iops` (hardcoded 3000), `--encrypted` (hardcoded false), `--throughput`, `--tag-specifications` | Valid AZ configured via `spinifex init` | Gateway validates input → NATS `ec2.CreateVolume` → daemon generates vol-ID via viperblock → creates volume (empty or from snapshot) of specified size → persists config.json to Predastore S3 → returns vol-ID with state=available | 1. Create 80GB gp3 volume<br>2. Boundary sizes (1 GiB min, 16384 GiB max)<br>3. Invalid AZ (error)<br>4. Verify volume in describe-volumes<br>5. Unsupported volume type (error - only gp3)<br>6. Size out of range (error)<br>7. Create from snapshot | **DONE** |
| `delete-volume` | `--volume-id` | `--dry-run` | Volume must exist and be detached (state=available) | Gateway validates vol- prefix → NATS `ec2.DeleteVolume` → daemon confirms state=available and no AttachedInstance → NATS `ebs.delete` to viperblockd (stops nbdkit/WAL) → deletes S3 objects under vol-id/, vol-id-efi/, vol-id-cloudinit/ → returns success | 1. Delete detached volume<br>2. Delete attached volume (error: VolumeInUse)<br>3. Delete non-existent volume (error: InvalidVolume.NotFound)<br>4. Verify volume gone from describe-volumes<br>5. Malformed volume ID (error: InvalidVolumeID.Malformed)<br>6. Double delete (idempotent NotFound) | **DONE** |
| `attach-volume` | `--volume-id`, `--instance-id`, `--device` (optional, auto-assigns `/dev/sd[f-p]`) | `--dry-run` | Volume must exist (available), instance must exist (running) | Gateway sends to `ec2.cmd.{instanceId}` → daemon validates volume (Predastore) → `ebs.mount` via NATS (viperblock starts NBD server) → QMP `blockdev-add` (nbd-{volId}) → QMP `device_add` (virtio-blk-pci, vdisk-{volId}) → three-phase rollback on failure → update EBSRequests + BlockDeviceMappings → persist to JetStream + Predastore → respond with VolumeAttachment | 1. Attach volume to running instance<br>2. Auto-assign device name<br>3. Attach already-attached volume (VolumeInUse)<br>4. Attach to non-existent instance (InvalidInstanceID.NotFound)<br>5. Attach to stopped instance (IncorrectInstanceState)<br>6. Volume not found (InvalidVolume.NotFound)<br>7. All device slots full (AttachmentLimitExceeded)<br>8. Volume persists across stop/start | **DONE** |
| `detach-volume` | `--volume-id`, `--instance-id` (optional, resolved via DescribeVolumes), `--device` (optional cross-check), `--force` | `--dry-run` | Volume must be attached, instance must be running | Gateway resolves InstanceId if omitted (via DescribeVolumes) → sends to `ec2.cmd.{instanceId}` → daemon validates (running, attached, not boot/EFI/CloudInit, device match) → three-phase hot-unplug: QMP `device_del` (force continues on failure) → QMP `blockdev-del` (abort if fails, preserves state to prevent double-attach) → `ebs.unmount` via NATS (best-effort) → remove from EBSRequests + BlockDeviceMappings → update volume metadata to available → persist state → respond with VolumeAttachment (state=detaching) | 1. Detach with explicit InstanceId<br>2. Detach without InstanceId (gateway resolution)<br>3. Detach with correct --device cross-check<br>4. Missing VolumeId (InvalidParameterValue)<br>5. Volume not attached (IncorrectState)<br>6. Nonexistent volume (InvalidVolume.NotFound)<br>7. Nonexistent instance (InvalidInstanceID.NotFound)<br>8. Instance not running (IncorrectInstanceState)<br>9. Device mismatch (InvalidParameterValue)<br>10. Boot volume protection (OperationNotPermitted)<br>11. Force flag (continues past device_del failure)<br>12. Volume reusability (re-attach after detach) | **DONE** |
| `describe-volume-status` | `--volume-ids` | `--filters`, `--max-results`, `--next-token`, `--dry-run` | None | Gateway validates vol- prefix → NATS `ec2.DescribeVolumeStatus` → daemon fetches VolumeConfig from Predastore S3 (parallel for specific IDs, sequential list-all for no IDs) → builds VolumeStatusItem per volume (status=ok, io-enabled=passed, io-performance=not-applicable) → returns InvalidVolume.NotFound for missing explicit IDs → skips internal sub-volumes (-efi, -cloudinit) | 1. List all volume statuses<br>2. Filter by specific volume IDs (fast path)<br>3. Non-existent volume ID returns InvalidVolume.NotFound<br>4. Invalid volume ID format (InvalidVolume.Malformed)<br>5. Internal sub-volumes excluded from listing<br>6. Nil/empty input defaults to all volumes | **DONE** |
| `describe-volumes-modifications` | — | `--volume-ids`, `--filters`, `--max-results` | None | Query pending/completed volume modifications → return modification state, progress, original/target size | 1. Check in-progress modification<br>2. Check completed modification<br>3. No modifications returns empty | **NOT STARTED** |
### EC2 - Snapshot Management
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-snapshot` | `--volume-id`, `--description`, `--tag-specifications` | `--dry-run` | Volume must exist in Predastore | Gateway validates vol- prefix → NATS `ec2.CreateSnapshot` → daemon reads VolumeConfig from Predastore → generates snap-ID via viperblock → stores SnapshotConfig (metadata-only, points to source volume) as completed → returns Snapshot with state=completed, progress=100% | 1. Snapshot from valid volume<br>2. Missing volume ID (InvalidParameterValue)<br>3. Volume not found (InvalidVolume.NotFound)<br>4. Invalid volume ID format (InvalidVolume.Malformed)<br>5. Snapshot with description<br>6. Snapshot with tag specifications | **DONE** |
| `create-snapshots` | — | `--instance-specification`, `--description`, `--tag-specifications` | Instance must exist, instance-volume attachment tracking | Create snapshots of all volumes attached to instance → return list of snapshot IDs. Blocked: requires instance-volume attachment tracking to discover which volumes to snapshot. | 1. Snapshot all volumes on instance<br>2. Instance with no volumes | **NOT STARTED** |
| `delete-snapshot` | `--snapshot-id` | `--dry-run` | Snapshot must exist | Gateway validates snap- prefix → NATS `ec2.DeleteSnapshot` → daemon verifies snapshot exists in Predastore → lists and deletes all objects under snapshot prefix → returns success | 1. Delete existing snapshot<br>2. Delete non-existent snapshot (InvalidSnapshot.NotFound)<br>3. Missing snapshot ID (InvalidParameterValue)<br>4. Invalid snapshot ID format (InvalidSnapshot.Malformed) | **DONE** |
| `describe-snapshots` | `--snapshot-ids` | `--owner-ids`, `--filters`, `--max-results`, `--dry-run` | None | Gateway validates snap- prefix on IDs → NATS `ec2.DescribeSnapshots` → daemon lists snap- prefixed objects in Predastore → reads SnapshotConfig for each → filters by snapshot ID if specified → returns snapshot list | 1. List all snapshots<br>2. Filter by snapshot ID<br>3. Empty snapshot list<br>4. Invalid snapshot ID format (InvalidSnapshot.Malformed) | **DONE** |
| `copy-snapshot` | `--source-snapshot-id`, `--source-region`, `--description` | `--encrypted`, `--dry-run` | Source snapshot must exist | Gateway validates snap- prefix + source region → NATS `ec2.CopySnapshot` → daemon reads source SnapshotConfig → generates new snap-ID → copies metadata (preserves tags, description override) → stores as completed → returns new snapshot ID | 1. Copy within same region<br>2. Copy non-existent snapshot (InvalidSnapshot.NotFound)<br>3. Missing source ID (InvalidParameterValue)<br>4. Missing source region (MissingParameter)<br>5. Copy preserves tags<br>6. Copy with description override | **DONE** |
### EC2 - Tags
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-tags` | `--resources`, `--tags` (Key=,Value=) | `--dry-run` | None | Gateway validates input (resources non-empty, tags non-empty, keys non-empty) → NATS `ec2.CreateTags` → daemon merges tags into per-resource JSON in Predastore S3 (`tags/{resourceId}.json`) → return success | 1. Tag an instance<br>2. Tag multiple resources at once<br>3. Overwrite existing tag value<br>4. Empty resources list (error: MissingParameter)<br>5. Empty tags list (error: MissingParameter)<br>6. Empty tag key (error: InvalidParameterValue) | **DONE** |
| `delete-tags` | `--resources`, `--tags` | `--dry-run` | None | Gateway validates input (resources non-empty) → NATS `ec2.DeleteTags` → daemon removes specified tag keys from resource JSON, or all tags if no keys specified → return success | 1. Delete specific tag by key<br>2. Delete all tags (no tags specified)<br>3. Empty resources list (error: MissingParameter) | **DONE** |
| `describe-tags` | `--filters` (resource-id, resource-type, key, value) | `--max-results`, `--next-token`, `--dry-run` | None | NATS `ec2.DescribeTags` → daemon lists all `tags/*.json` from Predastore S3 → applies filters (resource-id, resource-type, key, value) → returns tag list with resource type auto-detected from ID prefix | 1. List all tags<br>2. Filter by resource-id<br>3. Filter by resource-type (instance, volume, image, snapshot, vpc, subnet, security-group, route-table, internet-gateway, egress-only-internet-gateway)<br>4. Filter by key<br>5. Filter by value<br>6. Empty result when no tags exist | **DONE** |
### EC2 - Regions & Availability Zones
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `describe-regions` | *(returns configured region, endpoint, opt-in status from config — input params ignored)* | `--region-names`, `--filters`, `--all-regions`, `--dry-run` | None | Return configured region from spinifex init config with Endpoint and OptInStatus → local-only response, no NATS | 1. List all regions<br>2. Filter by region name<br>3. Verify endpoint URL returned<br>4. Verify OptInStatus returned | **DONE** |
| `describe-availability-zones` | *(returns configured AZ, region, zone ID, state, opt-in status from config — input params ignored)* | `--zone-names`, `--filters`, `--all-availability-zones` | None | Return configured AZ from spinifex init config with zone ID, state, group name, network border group → local-only response, no NATS | 1. List all AZs<br>2. Filter by zone name<br>3. Verify zone state is available<br>4. Verify region name matches config | **DONE** |
### EC2 - Account Attributes
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `describe-account-attributes` | `--attribute-names` | `--dry-run` | None | Gateway parses input → returns static account attributes: supported-platforms=VPC, default-vpc=none, max-instances=100, vpc-max-security-groups-per-interface=5, max-elastic-ips=5, vpc-max-elastic-ips=20 → local-only response, no NATS | 1. List all account attributes<br>2. Filter by attribute name<br>3. Verify all 6 attributes returned with correct values | **DONE** |
### EC2 - Account Settings (Deferred)
Persistence layer works (NATS JetStream KV), but stored values are not yet enforced by downstream services. Will be completed post-merge.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `enable-ebs-encryption-by-default` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.EnableEbsEncryptionByDefault` with `spinifex-workers` queue group → daemon stores `EbsEncryptionByDefault=true` in JetStream KV (`spinifex-ec2-account-settings` bucket) → return `EbsEncryptionByDefault=true`. Enforcement deferred: stored value not yet checked by CreateVolume/CreateSnapshot. | 1. Enable encryption<br>2. Verify via get | **STARTED** (enforcement pending) |
| `disable-ebs-encryption-by-default` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.DisableEbsEncryptionByDefault` with `spinifex-workers` queue group → daemon stores `EbsEncryptionByDefault=false` in JetStream KV → return `EbsEncryptionByDefault=false`. | 1. Disable encryption<br>2. Verify via get | **STARTED** (enforcement pending) |
| `get-ebs-encryption-by-default` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.GetEbsEncryptionByDefault` with `spinifex-workers` queue group → daemon reads `EbsEncryptionByDefault` from JetStream KV → return current state. | 1. Get default state<br>2. Verify matches last enable/disable | **STARTED** (enforcement pending) |
| `enable-serial-console-access` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.EnableSerialConsoleAccess` with `spinifex-workers` queue group → daemon stores `SerialConsoleAccessEnabled=true` in JetStream KV (`spinifex-ec2-account-settings` bucket) → return enabled=true. Stored for future interactive serial console support (not currently enforced by any operation). | 1. Enable access<br>2. Verify via get | **STARTED** (enforcement pending) |
| `disable-serial-console-access` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.DisableSerialConsoleAccess` with `spinifex-workers` queue group → daemon stores `SerialConsoleAccessEnabled=false` in JetStream KV → return enabled=false. Stored for future interactive serial console support (not currently enforced by any operation). | 1. Disable access<br>2. Verify via get | **STARTED** (enforcement pending) |
| `get-serial-console-access-status` | *(no flags needed)* | `--dry-run` | None | Gateway validates input → NATS `ec2.GetSerialConsoleAccessStatus` with `spinifex-workers` queue group → daemon reads `SerialConsoleAccessEnabled` from JetStream KV → return current state. | 1. Get current status<br>2. Verify matches last enable/disable | **DONE** |
| `enable-snapshot-block-public-access` | — | `--state` | None | Store snapshot block public access state in JetStream KV | 1. Enable with block-all-sharing<br>2. Enable with block-new-sharing | **NOT STARTED** (enforcement pending) |
| `disable-snapshot-block-public-access` | — | `--dry-run` | None | Clear snapshot block public access state in JetStream KV | 1. Disable access | **NOT STARTED** (enforcement pending) |
| `get-snapshot-block-public-access-state` | — | `--dry-run` | None | Read snapshot block public access state from JetStream KV | 1. Get current state | **NOT STARTED** (enforcement pending) |
| `enable-image-block-public-access` | — | `--image-block-public-access-state` | None | Store image block public access state in JetStream KV | 1. Enable with block-new-sharing | **NOT STARTED** (enforcement pending) |
| `disable-image-block-public-access` | — | `--dry-run` | None | Clear image block public access state in JetStream KV | 1. Disable access | **NOT STARTED** (enforcement pending) |
| `get-image-block-public-access-state` | — | `--dry-run` | None | Read image block public access state from JetStream KV | 1. Get current state | **NOT STARTED** (enforcement pending) |
| `modify-instance-metadata-defaults` | — | `--http-tokens`, `--http-put-response-hop-limit`, `--http-endpoint`, `--instance-metadata-tags` | None | Store instance metadata defaults in JetStream KV | 1. Set httpTokens=required<br>2. Set hop limit | **NOT STARTED** (enforcement pending) |
| `get-instance-metadata-defaults` | — | `--dry-run` | None | Read instance metadata defaults from JetStream KV | 1. Get current defaults | **NOT STARTED** (enforcement pending) |
### EC2 - Egress-Only Internet Gateway
Gateway routing and KV CRUD are wired up but no OVN/OVS integration exists. EIGWs are stored in `spinifex-eigw` KV bucket but have no effect on network topology (no vpcd events published, unlike IGW which publishes `vpc.igw-attach`/`vpc.igw-detach`).
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-egress-only-internet-gateway` | `--vpc-id`, `--tag-specifications` | `--client-token`, `--dry-run` | VPC must exist | Gateway validates VpcId → NATS `ec2.CreateEgressOnlyInternetGateway` → daemon generates eigw-ID → creates EgressOnlyIGWRecord in `spinifex-eigw` KV bucket with state=attached (always attached at creation, unlike IGW) → returns EgressOnlyInternetGateway with attachment | 1. Create with valid VpcId<br>2. Missing VpcId (error: MissingParameter)<br>3. Create with inline tags<br>4. Verify ID format eigw-{16hex} | **STARTED** (KV only, no OVN) |
| `delete-egress-only-internet-gateway` | `--egress-only-internet-gateway-id` | `--dry-run` | EIGW must exist | NATS `ec2.DeleteEgressOnlyInternetGateway` → daemon verifies EIGW exists in KV → deletes from `spinifex-eigw` KV → returns ReturnCode=true. No detach required (unlike IGW). | 1. Delete existing EIGW<br>2. Delete non-existent EIGW (error)<br>3. Missing ID (error: MissingParameter) | **STARTED** (KV only, no OVN) |
| `describe-egress-only-internet-gateways` | `--egress-only-internet-gateway-ids` | `--filters`, `--max-results`, `--next-token`, `--dry-run` | None | NATS `ec2.DescribeEgressOnlyInternetGateways` → daemon lists all keys from `spinifex-eigw` KV bucket → filters by ID if specified → always includes Attachments array → returns list | 1. Describe all EIGWs<br>2. Filter by specific ID<br>3. Non-existent ID returns empty list | **STARTED** (KV only, no OVN) |
### EC2 - Placement Groups
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-placement-group` | — | `--group-name`, `--strategy` (cluster/spread/partition), `--partition-count`, `--spread-level`, `--tag-specifications` | None | NATS `ec2.CreatePlacementGroup` → daemon stores placement group in JetStream KV with strategy, state=available → return group metadata | 1. Create with cluster strategy<br>2. Create with spread strategy<br>3. Create with partition strategy + partition count<br>4. Duplicate name (error)<br>5. Invalid strategy (error) | **NOT STARTED** |
| `delete-placement-group` | — | `--group-name`, `--dry-run` | Group must exist, no instances in group | NATS `ec2.DeletePlacementGroup` → daemon verifies group exists and is empty → delete from KV → return success | 1. Delete empty group<br>2. Delete group with instances (error)<br>3. Delete non-existent group (error) | **NOT STARTED** |
| `describe-placement-groups` | — | `--group-names`, `--group-ids`, `--filters` | None | NATS `ec2.DescribePlacementGroups` → daemon lists groups from KV → return list with strategy and state | 1. List all groups<br>2. Filter by name<br>3. Filter by strategy | **NOT STARTED** |
### EC2 - Dedicated Hosts
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `allocate-hosts` | — | `--availability-zone`, `--instance-type`, `--quantity`, `--auto-placement`, `--tag-specifications` | None | NATS `ec2.AllocateHosts` → daemon allocates host resources → stores in JetStream KV with h-ID → return host ID list | 1. Allocate single host<br>2. Allocate multiple hosts<br>3. Missing AZ or instance type (error) | **NOT STARTED** |
| `describe-hosts` | — | `--host-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeHosts` → daemon lists hosts from KV → return host list with capacity and instance info | 1. List all hosts<br>2. Filter by host ID<br>3. Filter by instance type | **NOT STARTED** |
| `release-hosts` | — | `--host-ids` | Host must exist, no running instances | NATS `ec2.ReleaseHosts` → daemon verifies no instances on host → delete from KV → return success/failure per host | 1. Release empty host<br>2. Release host with instances (error)<br>3. Release non-existent host (error) | **NOT STARTED** |
### EC2 - IPv4 Pools
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-public-ipv4-pool` | — | `--tag-specifications`, `--dry-run` | None | NATS `ec2.CreatePublicIpv4Pool` → daemon creates pool in JetStream KV with ipv4pool-ID → return pool ID | 1. Create pool<br>2. Verify in describe | **NOT STARTED** |
| `delete-public-ipv4-pool` | — | `--pool-id`, `--dry-run` | Pool must exist and be empty | NATS `ec2.DeletePublicIpv4Pool` → daemon verifies pool empty → delete from KV → return success | 1. Delete empty pool<br>2. Delete pool with addresses (error) | **NOT STARTED** |
| `describe-public-ipv4-pools` | — | `--pool-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribePublicIpv4Pools` → daemon lists pools from KV → return pool list with address counts | 1. List all pools<br>2. Filter by pool ID | **NOT STARTED** |
### EC2 - DHCP Options
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-dhcp-options` | — | `--dhcp-configurations` (domain-name, domain-name-servers, ntp-servers, netbios-name-servers, netbios-node-type), `--tag-specifications` | None | NATS `ec2.CreateDhcpOptions` → daemon stores DHCP option set in JetStream KV with dopt-ID → return DHCP options with configurations | 1. Create with domain-name + DNS servers<br>2. Create with NTP servers<br>3. Verify in describe | **NOT STARTED** |
| `delete-dhcp-options` | — | `--dhcp-options-id`, `--dry-run` | DHCP options must exist and not be associated with a VPC | NATS `ec2.DeleteDhcpOptions` → daemon verifies not associated → delete from KV → return success | 1. Delete unassociated options<br>2. Delete associated options (error)<br>3. Delete non-existent (error) | **NOT STARTED** |
| `describe-dhcp-options` | — | `--dhcp-options-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeDhcpOptions` → daemon lists DHCP options from KV → return list with configurations | 1. List all DHCP options<br>2. Filter by ID | **NOT STARTED** |
| `associate-dhcp-options` | — | `--dhcp-options-id`, `--vpc-id`, `--dry-run` | DHCP options and VPC must exist | NATS `ec2.AssociateDhcpOptions` → daemon links DHCP options to VPC → update VPC metadata → return success | 1. Associate with VPC<br>2. Missing DHCP options ID (error)<br>3. Missing VPC ID (error) | **NOT STARTED** |
### EC2 - Capacity Reservations
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-capacity-reservation` | — | `--instance-type`, `--instance-platform`, `--availability-zone`, `--instance-count`, `--end-date`, `--end-date-type`, `--instance-match-criteria`, `--tag-specifications` | None | NATS `ec2.CreateCapacityReservation` → daemon stores reservation in JetStream KV with cr-ID → return reservation with state=active | 1. Create with required fields<br>2. Missing instance type (error)<br>3. Verify in describe | **NOT STARTED** |
| `cancel-capacity-reservation` | — | `--capacity-reservation-id`, `--dry-run` | Reservation must exist | NATS `ec2.CancelCapacityReservation` → daemon marks reservation as cancelled in KV → return success | 1. Cancel active reservation<br>2. Cancel non-existent (error) | **NOT STARTED** |
| `describe-capacity-reservations` | — | `--capacity-reservation-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeCapacityReservations` → daemon lists reservations from KV → return list with state and counts | 1. List all reservations<br>2. Filter by ID<br>3. Filter by instance type | **NOT STARTED** |
| `modify-capacity-reservation` | — | `--capacity-reservation-id`, `--instance-count`, `--end-date`, `--end-date-type` | Reservation must exist | NATS `ec2.ModifyCapacityReservation` → daemon updates reservation in KV → return success | 1. Modify instance count<br>2. Modify end date<br>3. Missing reservation ID (error) | **NOT STARTED** |
### EC2 - Elastic IP
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `allocate-address` | — | `--domain` (vpc), `--tag-specifications`, `--dry-run` | None | NATS `ec2.AllocateAddress` → daemon allocates IP from pool → stores in JetStream KV with eipalloc-ID → return AllocationId, PublicIp, Domain | 1. Allocate EIP<br>2. Verify in describe-addresses<br>3. Allocate without --domain uses default | **NOT STARTED** |
| `release-address` | — | `--allocation-id`, `--dry-run` | EIP must exist and be disassociated | NATS `ec2.ReleaseAddress` → daemon verifies not associated → release IP back to pool → delete from KV → return success | 1. Release disassociated EIP<br>2. Release associated EIP (error)<br>3. Release non-existent (error) | **NOT STARTED** |
| `associate-address` | — | `--allocation-id`, `--instance-id` or `--network-interface-id`, `--dry-run` | EIP and target must exist | NATS `ec2.AssociateAddress` → daemon links EIP to instance/ENI → configure OVS DNAT/SNAT rules → return eipassoc-ID | 1. Associate with instance<br>2. Re-associate (moves EIP)<br>3. Associate non-existent EIP (error) | **NOT STARTED** |
| `disassociate-address` | — | `--association-id`, `--dry-run` | Association must exist | NATS `ec2.DisassociateAddress` → daemon unlinks EIP from instance/ENI → remove OVS DNAT/SNAT rules → return success | 1. Disassociate existing<br>2. Disassociate non-existent (error) | **NOT STARTED** |
| `describe-addresses` | — | `--allocation-ids`, `--filters`, `--public-ips` | None | NATS `ec2.DescribeAddresses` → daemon lists EIPs from KV → return list with association info | 1. List all EIPs<br>2. Filter by allocation ID<br>3. Filter by public IP | **NOT STARTED** |
### EC2 - NAT Gateway
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-nat-gateway` | — | `--subnet-id`, `--allocation-id`, `--connectivity-type`, `--tag-specifications` | Subnet must exist, EIP must be allocated (for public type) | NATS `ec2.CreateNatGateway` → daemon creates NAT gateway in JetStream KV with nat-ID → configure OVS SNAT rules → state=pending→available → return NatGateway | 1. Create public NAT gateway<br>2. Create private NAT gateway (no allocation-id)<br>3. Invalid subnet (error) | **NOT STARTED** |
| `delete-nat-gateway` | — | `--nat-gateway-id`, `--dry-run` | NAT gateway must exist | NATS `ec2.DeleteNatGateway` → daemon marks as deleting → remove OVS SNAT rules → delete from KV → return NatGatewayId | 1. Delete existing NAT gateway<br>2. Delete non-existent (error) | **NOT STARTED** |
| `describe-nat-gateways` | — | `--nat-gateway-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeNatGateways` → daemon lists NAT gateways from KV → return list with state, subnet, addresses | 1. List all NAT gateways<br>2. Filter by ID<br>3. Filter by VPC ID | **NOT STARTED** |
| `assign-private-nat-gateway-address` | — | `--nat-gateway-id`, `--private-ip-addresses` | NAT gateway must exist (private type) | NATS `ec2.AssignPrivateNatGatewayAddress` → daemon assigns private IPs to NAT gateway → return assigned addresses | 1. Assign address<br>2. Missing NAT gateway ID (error) | **NOT STARTED** |
| `associate-nat-gateway-address` | — | `--nat-gateway-id`, `--allocation-ids` | NAT gateway and EIPs must exist | NATS `ec2.AssociateNatGatewayAddress` → daemon associates EIPs with NAT gateway → return association info | 1. Associate EIP<br>2. Missing NAT gateway ID (error) | **NOT STARTED** |
### EC2 - VPC Networking (Core)
Requires OVN/OVS (`apt install openvswitch-switch ovn-host`). VPC/Subnet/ENI CRUD stores metadata in NATS KV and publishes events to vpcd (the network topology daemon) which translates to OVN logical switches, ports, and routers. Single AZ for Spinifex v1. IPAM uses CAS-based allocation from NATS KV (`spinifex-vpc-ipam` bucket).
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-vpc` | `--cidr-block`, `--tag-specifications` | `--instance-tenancy`, `--dry-run` | None | Gateway validates CidrBlock (required, valid CIDR, /16–/28 prefix) → NATS `ec2.CreateVpc` → daemon allocates unique VNI from `spinifex-vpc-vni-counter` KV → generates vpc-ID → stores VPCRecord in `spinifex-vpc-vpcs` KV → publishes `vpc.create` event to vpcd (creates OVN logical switch) → returns VPC with state=available | 1. Create VPC with /16 CIDR<br>2. Invalid CIDR format (error: InvalidVpcRange)<br>3. CIDR prefix outside /16–/28 (error)<br>4. Verify in describe-vpcs<br>5. Tags applied at creation | **DONE** |
| `delete-vpc` | `--vpc-id` | `--dry-run` | VPC must be empty (no subnets) | Gateway validates VpcId → NATS `ec2.DeleteVpc` → daemon checks for dependent subnets in `spinifex-vpc-subnets` KV → returns DependencyViolation if subnets exist → deletes from `spinifex-vpc-vpcs` KV → publishes `vpc.delete` event to vpcd (removes OVN resources) | 1. Delete empty VPC<br>2. Delete VPC with subnets (DependencyViolation)<br>3. Delete non-existent VPC (error) | **DONE** |
| `describe-vpcs` | `--vpc-ids` | `--filters`, `--max-results`, `--dry-run` | None | NATS `ec2.DescribeVpcs` → daemon lists all keys from `spinifex-vpc-vpcs` KV → filters by VPC IDs if specified → returns error for non-existent requested IDs → returns VPC list with CIDR, state, CIDR associations | 1. List all VPCs<br>2. Filter by VPC ID<br>3. Non-existent VPC ID returns error | **DONE** |
| `modify-vpc-attribute` | — | `--vpc-id`, `--enable-dns-support`, `--enable-dns-hostnames` | VPC must exist | NATS `ec2.ModifyVpcAttribute` → daemon updates VPC attribute in KV → return success | 1. Enable DNS support<br>2. Enable DNS hostnames<br>3. Missing VPC ID (error) | **NOT STARTED** |
| `associate-vpc-cidr-block` | — | `--vpc-id`, `--cidr-block` | VPC must exist | NATS `ec2.AssociateVpcCidrBlock` → daemon adds secondary CIDR to VPC metadata → return association | 1. Add secondary CIDR<br>2. Overlapping CIDR (error)<br>3. Max CIDR blocks exceeded (error) | **NOT STARTED** |
| `disassociate-vpc-cidr-block` | — | `--association-id` | Association must exist | NATS `ec2.DisassociateVpcCidrBlock` → daemon removes secondary CIDR from VPC → return success | 1. Remove secondary CIDR<br>2. Remove primary CIDR (error) | **NOT STARTED** |
| `create-subnet` | `--vpc-id`, `--cidr-block`, `--availability-zone`, `--tag-specifications` | `--dry-run` | VPC must exist | Gateway validates VpcId + CidrBlock (required, valid CIDR, /16–/28 prefix) → NATS `ec2.CreateSubnet` → daemon verifies parent VPC exists → validates subnet CIDR is within VPC CIDR range → checks for overlapping subnets in same VPC → calculates available IPs (total hosts - 5 AWS-reserved: .0/.1/.2/.3/.255) → generates subnet-ID → stores SubnetRecord in `spinifex-vpc-subnets` KV → publishes `vpc.create-subnet` event to vpcd → defaults AZ from config if omitted | 1. Create subnet within VPC CIDR<br>2. Subnet CIDR outside VPC range (error: InvalidSubnetRange)<br>3. Overlapping subnet CIDRs (error: InvalidSubnetConflict)<br>4. CIDR prefix outside /16–/28 (error)<br>5. Missing VPC or CIDR (error: MissingParameter) | **DONE** |
| `delete-subnet` | `--subnet-id` | `--dry-run` | Subnet must exist | NATS `ec2.DeleteSubnet` → daemon verifies subnet exists → deletes from `spinifex-vpc-subnets` KV → publishes `vpc.delete-subnet` event to vpcd | 1. Delete existing subnet<br>2. Delete non-existent subnet (error) | **DONE** |
| `describe-subnets` | `--subnet-ids`, `--filters` (vpc-id) | `--max-results`, `--dry-run` | None | NATS `ec2.DescribeSubnets` → daemon lists all keys from `spinifex-vpc-subnets` KV → applies vpc-id filter if present → filters by subnet IDs if specified → recalculates available IPs from CIDR → returns error for non-existent requested IDs | 1. List all subnets<br>2. Filter by VPC ID<br>3. Filter by subnet ID<br>4. Non-existent subnet returns error | **DONE** |
| `modify-subnet-attribute` | — | `--subnet-id`, `--map-public-ip-on-launch`, `--assign-ipv6-address-on-creation` | Subnet must exist | NATS `ec2.ModifySubnetAttribute` → daemon updates subnet attribute in KV → return success | 1. Enable auto-assign public IP<br>2. Missing subnet ID (error) | **NOT STARTED** |
| `associate-subnet-cidr-block` | — | `--subnet-id`, `--ipv6-cidr-block` | Subnet must exist | NATS `ec2.AssociateSubnetCidrBlock` → daemon adds IPv6 CIDR to subnet → return association | 1. Add IPv6 CIDR<br>2. Missing subnet ID (error) | **NOT STARTED** |
| `disassociate-subnet-cidr-block` | — | `--association-id` | Association must exist | NATS `ec2.DisassociateSubnetCidrBlock` → daemon removes IPv6 CIDR from subnet → return success | 1. Remove IPv6 CIDR<br>2. Invalid association (error) | **NOT STARTED** |
| `create-security-group` | — | `--group-name`, `--description`, `--vpc-id`, `--tag-specifications` | VPC must exist | Create security group metadata → create default OVS flow rules (deny all inbound, allow all outbound) → assign sg-ID | 1. Create SG in VPC<br>2. Duplicate name in same VPC (error)<br>3. Verify default rules | **NOT STARTED** |
| `delete-security-group` | — | `--group-id` | SG must not be in use by instances | Verify no instances reference SG → remove OVS flow rules → delete metadata | 1. Delete unused SG<br>2. Delete SG in use (error) | **NOT STARTED** |
| `describe-security-groups` | — | `--group-ids`, `--group-names`, `--filters`, `--max-results` | None | Query NATS KV for SG metadata → return SG list with rules | 1. List all SGs<br>2. Filter by VPC ID<br>3. Filter by group name | **NOT STARTED** |
| `authorize-security-group-ingress` | — | `--group-id`, `--protocol`, `--port`, `--cidr`, `--source-group` | SG must exist | Add inbound rule → create OVS OpenFlow rule → persist to metadata | 1. Allow SSH (port 22) from 0.0.0.0/0<br>2. Allow from specific CIDR<br>3. Duplicate rule (idempotent) | **NOT STARTED** |
| `authorize-security-group-egress` | — | `--group-id`, `--protocol`, `--port`, `--cidr` | SG must exist | Add outbound rule → create OVS OpenFlow rule → persist to metadata | 1. Allow HTTPS outbound<br>2. Restrict to specific CIDR | **NOT STARTED** |
| `revoke-security-group-ingress` | — | `--group-id`, `--protocol`, `--port`, `--cidr` | SG must exist, rule must exist | Remove inbound rule → delete OVS OpenFlow rule → update metadata | 1. Revoke existing rule<br>2. Revoke non-existent rule (error) | **NOT STARTED** |
| `revoke-security-group-egress` | — | `--group-id`, `--protocol`, `--port`, `--cidr` | SG must exist, rule must exist | Remove outbound rule → delete OVS OpenFlow rule → update metadata | 1. Revoke existing rule<br>2. Revoke non-existent rule (error) | **NOT STARTED** |
| `create-internet-gateway` | `--tag-specifications` | `--dry-run` | None | NATS `ec2.CreateInternetGateway` → daemon generates igw-ID → creates IGWRecord in `spinifex-igw` KV bucket with state=available (not attached) → returns InternetGateway with empty Attachments | 1. Create IGW<br>2. Verify in describe-internet-gateways<br>3. Tags applied at creation | **DONE** |
| `attach-internet-gateway` | `--internet-gateway-id`, `--vpc-id` | `--dry-run` | IGW and VPC must exist | NATS `ec2.AttachInternetGateway` → daemon validates IGW exists and is not already attached (ResourceAlreadyAssociated) → sets VpcId on record → state=attached → publishes `vpc.igw-attach` event to vpcd (creates OVN external switch, gateway port, SNAT rules) | 1. Attach IGW to VPC<br>2. Attach already-attached IGW (ResourceAlreadyAssociated)<br>3. Non-existent IGW (error)<br>4. Missing params (MissingParameter) | **DONE** |
| `detach-internet-gateway` | `--internet-gateway-id`, `--vpc-id` | `--dry-run` | IGW must be attached to specified VPC | NATS `ec2.DetachInternetGateway` → daemon validates IGW exists and is attached to specified VPC (GatewayNotAttached if wrong VPC) → clears VpcId → state=available → publishes `vpc.igw-detach` event to vpcd (removes OVN resources) | 1. Detach attached IGW<br>2. Detach from wrong VPC (GatewayNotAttached)<br>3. Detach unattached IGW (error) | **DONE** |
| `delete-internet-gateway` | `--internet-gateway-id` | `--dry-run` | IGW must be detached | NATS `ec2.DeleteInternetGateway` → daemon validates IGW exists and VpcId is empty (DependencyViolation if attached) → deletes from `spinifex-igw` KV | 1. Delete detached IGW<br>2. Delete attached IGW (DependencyViolation)<br>3. Delete non-existent (error) | **DONE** |
| `describe-internet-gateways` | `--internet-gateway-ids` | `--filters`, `--max-results`, `--dry-run` | None | NATS `ec2.DescribeInternetGateways` → daemon lists all keys from `spinifex-igw` KV → filters by IGW IDs if specified → returns list with Attachments array (if attached) | 1. List all IGWs<br>2. Filter by IGW ID<br>3. Verify attachment info | **DONE** |
### EC2 - Route Tables
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-route-table` | — | `--vpc-id`, `--tag-specifications` | VPC must exist | NATS `ec2.CreateRouteTable` → daemon creates route table in KV with rtb-ID → add default local route for VPC CIDR → return route table | 1. Create route table<br>2. Verify default local route<br>3. Missing VPC ID (error) | **NOT STARTED** |
| `delete-route-table` | — | `--route-table-id`, `--dry-run` | Route table must exist and not be main route table | NATS `ec2.DeleteRouteTable` → daemon verifies not main table → delete from KV → return success | 1. Delete non-main route table<br>2. Delete main route table (error)<br>3. Delete non-existent (error) | **NOT STARTED** |
| `describe-route-tables` | — | `--route-table-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeRouteTables` → daemon lists route tables from KV → return tables with routes | 1. List all route tables<br>2. Filter by VPC | **NOT STARTED** |
| `create-route` | — | `--route-table-id`, `--destination-cidr-block`, `--gateway-id` or `--nat-gateway-id` or `--egress-only-internet-gateway-id` | Route table and target must exist | NATS `ec2.CreateRoute` → daemon adds route entry → configure OVS flow for destination CIDR → update metadata | 1. Add internet route via IGW<br>2. Add route to NAT gateway<br>3. Add route to EIGW<br>4. Conflicting route (error) | **NOT STARTED** |
| `delete-route` | — | `--route-table-id`, `--destination-cidr-block` | Route must exist | NATS `ec2.DeleteRoute` → daemon removes route entry → remove OVS flow → update metadata | 1. Delete existing route<br>2. Delete non-existent route (error)<br>3. Delete local route (error) | **NOT STARTED** |
| `replace-route` | — | `--route-table-id`, `--destination-cidr-block`, `--gateway-id` or `--nat-gateway-id` | Route must exist | NATS `ec2.ReplaceRoute` → daemon replaces route target → update OVS flow → update metadata | 1. Replace IGW route with NAT<br>2. Replace non-existent route (error) | **NOT STARTED** |
| `associate-route-table` | — | `--route-table-id`, `--subnet-id` or `--gateway-id` | Route table and target must exist | NATS `ec2.AssociateRouteTable` → daemon links route table to subnet/gateway → return rtbassoc-ID | 1. Associate with subnet<br>2. Missing route table ID (error) | **NOT STARTED** |
| `disassociate-route-table` | — | `--association-id` | Association must exist, not main table association | NATS `ec2.DisassociateRouteTable` → daemon unlinks route table → return success | 1. Disassociate existing<br>2. Disassociate main table (error) | **NOT STARTED** |
### EC2 - VPC Peering
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-vpc-peering-connection` | — | `--vpc-id`, `--peer-vpc-id`, `--peer-owner-id`, `--peer-region`, `--tag-specifications` | Both VPCs must exist | NATS `ec2.CreateVpcPeeringConnection` → daemon creates peering in KV with pcx-ID → state=pending-acceptance → return peering connection | 1. Create peering between two VPCs<br>2. Missing VPC ID (error)<br>3. Overlapping CIDRs (error) | **NOT STARTED** |
| `accept-vpc-peering-connection` | — | `--vpc-peering-connection-id`, `--dry-run` | Peering must exist in pending-acceptance state | NATS `ec2.AcceptVpcPeeringConnection` → daemon updates state to active → configure OVS cross-VPC flows → return peering | 1. Accept pending peering<br>2. Accept already active (error) | **NOT STARTED** |
| `reject-vpc-peering-connection` | — | `--vpc-peering-connection-id`, `--dry-run` | Peering must exist in pending-acceptance state | NATS `ec2.RejectVpcPeeringConnection` → daemon updates state to rejected → return peering | 1. Reject pending peering<br>2. Reject already active (error) | **NOT STARTED** |
| `delete-vpc-peering-connection` | — | `--vpc-peering-connection-id`, `--dry-run` | Peering must exist | NATS `ec2.DeleteVpcPeeringConnection` → daemon removes OVS cross-VPC flows → delete from KV → return success | 1. Delete active peering<br>2. Delete non-existent (error) | **NOT STARTED** |
| `describe-vpc-peering-connections` | — | `--vpc-peering-connection-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeVpcPeeringConnections` → daemon lists peerings from KV → return list with state | 1. List all peerings<br>2. Filter by ID<br>3. Filter by VPC | **NOT STARTED** |
| `modify-vpc-peering-connection-options` | — | `--vpc-peering-connection-id`, `--requester-peering-connection-options`, `--accepter-peering-connection-options` | Peering must be active | NATS `ec2.ModifyVpcPeeringConnectionOptions` → daemon updates peering options in KV → return modified options | 1. Enable DNS resolution<br>2. Missing peering ID (error) | **NOT STARTED** |
### EC2 - VPC Endpoints
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-vpc-endpoint` | — | `--vpc-id`, `--service-name`, `--vpc-endpoint-type` (Gateway/Interface), `--route-table-ids`, `--subnet-ids`, `--tag-specifications` | VPC must exist | NATS `ec2.CreateVpcEndpoint` → daemon creates endpoint in KV with vpce-ID → return endpoint | 1. Create Gateway endpoint for S3<br>2. Create Interface endpoint<br>3. Missing VPC ID (error) | **NOT STARTED** |
| `delete-vpc-endpoints` | — | `--vpc-endpoint-ids`, `--dry-run` | Endpoints must exist | NATS `ec2.DeleteVpcEndpoints` → daemon deletes endpoints from KV → return success/failure per endpoint | 1. Delete existing endpoint<br>2. Delete non-existent (error) | **NOT STARTED** |
| `describe-vpc-endpoints` | — | `--vpc-endpoint-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeVpcEndpoints` → daemon lists endpoints from KV → return list with state and type | 1. List all endpoints<br>2. Filter by VPC<br>3. Filter by service name | **NOT STARTED** |
| `describe-vpc-endpoint-services` | — | `--service-names`, `--filters`, `--max-results` | None | NATS `ec2.DescribeVpcEndpointServices` → daemon returns available services (S3, DynamoDB equivalents) → return service list | 1. List available services<br>2. Filter by service name | **NOT STARTED** |
| `modify-vpc-endpoint` | — | `--vpc-endpoint-id`, `--add-route-table-ids`, `--remove-route-table-ids`, `--add-subnet-ids`, `--remove-subnet-ids` | Endpoint must exist | NATS `ec2.ModifyVpcEndpoint` → daemon updates endpoint route tables/subnets in KV → return success | 1. Add route table<br>2. Remove subnet<br>3. Missing endpoint ID (error) | **NOT STARTED** |
### EC2 - VPN Gateway & Customer Gateway
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-customer-gateway` | — | `--type` (ipsec.1), `--bgp-asn`, `--ip-address`, `--tag-specifications` | None | NATS `ec2.CreateCustomerGateway` → daemon stores in JetStream KV with cgw-ID → return customer gateway | 1. Create with BGP ASN and IP<br>2. Missing required fields (error) | **NOT STARTED** |
| `delete-customer-gateway` | — | `--customer-gateway-id`, `--dry-run` | Must exist, no VPN connections | NATS `ec2.DeleteCustomerGateway` → daemon verifies no active connections → delete from KV → return success | 1. Delete existing<br>2. Delete with active VPN (error) | **NOT STARTED** |
| `describe-customer-gateways` | — | `--customer-gateway-ids`, `--filters` | None | NATS `ec2.DescribeCustomerGateways` → daemon lists from KV → return list with BGP/IP info | 1. List all<br>2. Filter by ID | **NOT STARTED** |
| `create-vpn-gateway` | — | `--type` (ipsec.1), `--amazon-side-asn`, `--tag-specifications` | None | NATS `ec2.CreateVpnGateway` → daemon stores in JetStream KV with vgw-ID → return VPN gateway | 1. Create VPN gateway<br>2. Verify in describe | **NOT STARTED** |
| `delete-vpn-gateway` | — | `--vpn-gateway-id`, `--dry-run` | Must be detached | NATS `ec2.DeleteVpnGateway` → daemon verifies detached → delete from KV → return success | 1. Delete detached gateway<br>2. Delete attached (error) | **NOT STARTED** |
| `attach-vpn-gateway` | — | `--vpn-gateway-id`, `--vpc-id` | Both must exist | NATS `ec2.AttachVpnGateway` → daemon links VPN gateway to VPC → return attachment | 1. Attach to VPC<br>2. Already attached (error) | **NOT STARTED** |
| `detach-vpn-gateway` | — | `--vpn-gateway-id`, `--vpc-id` | Must be attached | NATS `ec2.DetachVpnGateway` → daemon unlinks VPN gateway from VPC → return success | 1. Detach from VPC<br>2. Not attached (error) | **NOT STARTED** |
| `describe-vpn-gateways` | — | `--vpn-gateway-ids`, `--filters` | None | NATS `ec2.DescribeVpnGateways` → daemon lists from KV → return list with attachment info | 1. List all<br>2. Filter by VPC attachment | **NOT STARTED** |
### EC2 - VPN Connections
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-vpn-connection` | — | `--type` (ipsec.1), `--customer-gateway-id`, `--vpn-gateway-id`, `--options`, `--tag-specifications` | Customer gateway and VPN gateway must exist | NATS `ec2.CreateVpnConnection` → daemon stores connection in JetStream KV with vpn-ID → return connection with tunnel info | 1. Create IPsec connection<br>2. Missing customer gateway (error)<br>3. Missing VPN gateway (error) | **NOT STARTED** |
| `delete-vpn-connection` | — | `--vpn-connection-id`, `--dry-run` | Connection must exist | NATS `ec2.DeleteVpnConnection` → daemon deletes from KV → return success | 1. Delete existing connection<br>2. Delete non-existent (error) | **NOT STARTED** |
| `describe-vpn-connections` | — | `--vpn-connection-ids`, `--filters` | None | NATS `ec2.DescribeVpnConnections` → daemon lists from KV → return list with state and tunnel info | 1. List all connections<br>2. Filter by ID<br>3. Filter by VPN gateway | **NOT STARTED** |
| `modify-vpn-connection` | — | `--vpn-connection-id`, `--vpn-gateway-id`, `--customer-gateway-id` | Connection must exist | NATS `ec2.ModifyVpnConnection` → daemon updates connection target in KV → return modified connection | 1. Change VPN gateway<br>2. Missing connection ID (error) | **NOT STARTED** |
| `modify-vpn-connection-options` | — | `--vpn-connection-id`, `--local-ipv4-network-cidr`, `--remote-ipv4-network-cidr` | Connection must exist | NATS `ec2.ModifyVpnConnectionOptions` → daemon updates tunnel options in KV → return modified connection | 1. Modify CIDR ranges<br>2. Missing connection ID (error) | **NOT STARTED** |
### EC2 - Network ACLs
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-network-acl` | — | `--vpc-id`, `--tag-specifications` | VPC must exist | NATS `ec2.CreateNetworkAcl` → daemon creates NACL in KV with acl-ID → add default deny-all rules → return NACL | 1. Create NACL in VPC<br>2. Missing VPC ID (error)<br>3. Verify default deny rules | **NOT STARTED** |
| `delete-network-acl` | — | `--network-acl-id`, `--dry-run` | Must not be default NACL | NATS `ec2.DeleteNetworkAcl` → daemon verifies not default → delete from KV → return success | 1. Delete non-default NACL<br>2. Delete default NACL (error) | **NOT STARTED** |
| `describe-network-acls` | — | `--network-acl-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeNetworkAcls` → daemon lists NACLs from KV → return list with entries | 1. List all NACLs<br>2. Filter by VPC | **NOT STARTED** |
| `create-network-acl-entry` | — | `--network-acl-id`, `--rule-number`, `--protocol`, `--rule-action` (allow/deny), `--cidr-block`, `--ingress`/`--egress`, `--port-range` | NACL must exist | NATS `ec2.CreateNetworkAclEntry` → daemon adds rule entry to NACL in KV → configure OVS flow → return success | 1. Add ingress allow rule<br>2. Add egress deny rule<br>3. ICMP type/code parsing<br>4. Missing NACL ID (error) | **NOT STARTED** |
| `delete-network-acl-entry` | — | `--network-acl-id`, `--rule-number`, `--ingress`/`--egress` | Entry must exist | NATS `ec2.DeleteNetworkAclEntry` → daemon removes rule from NACL → remove OVS flow → return success | 1. Delete existing entry<br>2. Delete non-existent entry (error) | **NOT STARTED** |
| `replace-network-acl-association` | — | `--association-id`, `--network-acl-id` | Both must exist | NATS `ec2.ReplaceNetworkAclAssociation` → daemon swaps NACL for subnet → return new association ID | 1. Replace association<br>2. Missing association ID (error) | **NOT STARTED** |
| `replace-network-acl-entry` | — | `--network-acl-id`, `--rule-number`, `--protocol`, `--rule-action`, `--cidr-block`, `--ingress`/`--egress`, `--port-range` | Entry must exist | NATS `ec2.ReplaceNetworkAclEntry` → daemon replaces rule in KV → update OVS flow → return success | 1. Replace existing entry<br>2. Replace non-existent entry (error) | **NOT STARTED** |
### EC2 - Prefix Lists
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-managed-prefix-list` | — | `--prefix-list-name`, `--address-family` (IPv4/IPv6), `--max-entries`, `--entries`, `--tag-specifications` | None | NATS `ec2.CreateManagedPrefixList` → daemon stores prefix list in JetStream KV with pl-ID → return prefix list with version | 1. Create with IPv4 entries<br>2. Name, address family, max entries required<br>3. Verify in describe | **NOT STARTED** |
| `delete-managed-prefix-list` | — | `--prefix-list-id`, `--dry-run` | Must exist, not referenced by security groups/route tables | NATS `ec2.DeleteManagedPrefixList` → daemon verifies not referenced → delete from KV → return success | 1. Delete unreferenced list<br>2. Delete referenced list (error) | **NOT STARTED** |
| `describe-managed-prefix-lists` | — | `--prefix-list-ids`, `--filters`, `--max-results` | None | NATS `ec2.DescribeManagedPrefixLists` → daemon lists from KV → return list with state and version | 1. List all prefix lists<br>2. Filter by ID | **NOT STARTED** |
| `modify-managed-prefix-list` | — | `--prefix-list-id`, `--current-version`, `--add-entries`, `--remove-entries`, `--prefix-list-name` | Must exist, current version must match | NATS `ec2.ModifyManagedPrefixList` → daemon validates version → apply changes → increment version → return updated list | 1. Add entries<br>2. Remove entries<br>3. Version mismatch (error) | **NOT STARTED** |
| `get-managed-prefix-list-entries` | — | `--prefix-list-id`, `--target-version`, `--max-results` | Prefix list must exist | NATS `ec2.GetManagedPrefixListEntries` → daemon reads entries from KV → return entry list | 1. List entries<br>2. List entries at specific version | **NOT STARTED** |
| `get-managed-prefix-list-associations` | — | `--prefix-list-id`, `--max-results` | Prefix list must exist | NATS `ec2.GetManagedPrefixListAssociations` → daemon finds references in SGs/route tables → return association list | 1. List associations<br>2. No associations returns empty | **NOT STARTED** |
### EC2 - Network Interfaces
ENIs are managed via `spinifex-vpc-enis` KV bucket with CAS-based IPAM allocation from `spinifex-vpc-ipam`. ENIs are auto-created by `run-instances --subnet-id` and auto-deleted on termination. MAC addresses are deterministically generated from the ENI ID. Attach/detach are internal operations used by instance lifecycle — no standalone attach/detach API yet.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-network-interface` | `--subnet-id`, `--private-ip-address`, `--description`, `--tag-specifications` | `--groups`, `--dry-run` | Subnet must exist | Gateway validates SubnetId (required) → NATS `ec2.CreateNetworkInterface` → daemon verifies subnet exists in `spinifex-vpc-subnets` KV → if no PrivateIpAddress, allocates from IPAM pool (CAS-based, up to 5 retries) → generates eni-ID → generates deterministic MAC from ENI ID → stores ENIRecord in `spinifex-vpc-enis` KV → publishes `vpc.create-port` event to vpcd (creates OVN logical port) → returns NetworkInterface with status=available | 1. Create in subnet (auto-allocate IP)<br>2. Create with specific private IP<br>3. Missing subnet ID (MissingParameter)<br>4. Non-existent subnet (error)<br>5. Tags applied at creation | **DONE** |
| `delete-network-interface` | `--network-interface-id` | `--dry-run` | Must not be attached (status != in-use) | NATS `ec2.DeleteNetworkInterface` → daemon verifies ENI exists and status is not "in-use" (InvalidNetworkInterfaceInUse) → releases IP back to IPAM pool → deletes from `spinifex-vpc-enis` KV → publishes `vpc.delete-port` event to vpcd | 1. Delete detached ENI<br>2. Delete attached ENI (InvalidNetworkInterfaceInUse)<br>3. Delete non-existent (error) | **DONE** |
| `describe-network-interfaces` | `--network-interface-ids`, `--filters` (subnet-id, vpc-id, attachment.instance-id) | `--max-results`, `--dry-run` | None | NATS `ec2.DescribeNetworkInterfaces` → daemon lists all keys from `spinifex-vpc-enis` KV → applies filters (subnet-id, vpc-id, attachment.instance-id) → filters by ENI IDs if specified → returns error for non-existent requested IDs | 1. List all ENIs<br>2. Filter by subnet-id<br>3. Filter by vpc-id<br>4. Filter by attachment.instance-id<br>5. Non-existent ENI returns error | **DONE** |
| `attach-network-interface` | — | `--network-interface-id`, `--instance-id`, `--device-index` | ENI and instance must exist | Internal operation used by `run-instances --subnet-id`. Marks ENI as "in-use", sets instance attachment, generates attachment ID. No standalone API endpoint yet. | 1. Attach ENI to instance<br>2. Already attached (error)<br>3. Missing device index (error) | **NOT STARTED** (internal only) |
| `detach-network-interface` | — | `--attachment-id`, `--force` | ENI must be attached | Internal operation used by terminate-instances. Marks ENI as "available", clears attachment info. No standalone API endpoint yet. | 1. Detach ENI<br>2. Force detach<br>3. Not attached (error) | **NOT STARTED** (internal only) |
| `modify-network-interface-attribute` | — | `--network-interface-id`, `--description`, `--source-dest-check`, `--groups` | ENI must exist | NATS `ec2.ModifyNetworkInterfaceAttribute` → daemon updates ENI attributes in KV → return success | 1. Modify description<br>2. Toggle source/dest check<br>3. Update security groups | **NOT STARTED** |
| `assign-private-ip-addresses` | — | `--network-interface-id`, `--private-ip-addresses` or `--secondary-private-ip-address-count` | ENI must exist | NATS `ec2.AssignPrivateIpAddresses` → daemon allocates IPs from subnet pool → assign to ENI → return assigned IPs | 1. Assign specific IPs<br>2. Assign by count<br>3. Cannot specify both (error) | **NOT STARTED** |
| `unassign-private-ip-addresses` | — | `--network-interface-id`, `--private-ip-addresses` | IPs must be assigned to ENI | NATS `ec2.UnassignPrivateIpAddresses` → daemon releases IPs → return success | 1. Unassign secondary IP<br>2. Unassign primary IP (error) | **NOT STARTED** |
### EC2 - Launch Templates
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-launch-template` | — | `--launch-template-name`, `--launch-template-data` (ImageId, InstanceType, etc.), `--tag-specifications` | None | NATS `ec2.CreateLaunchTemplate` → daemon stores template in JetStream KV with lt-ID → version 1 → return template metadata | 1. Create template with full config<br>2. Duplicate name (error)<br>3. Verify in describe output | **NOT STARTED** |
| `create-launch-template-version` | — | `--launch-template-id` or `--launch-template-name`, `--launch-template-data`, `--source-version` | Template must exist | NATS `ec2.CreateLaunchTemplateVersion` → daemon creates new version in KV → return version details | 1. Create version from scratch<br>2. Create version from source version<br>3. Template not found (error) | **NOT STARTED** |
| `delete-launch-template` | — | `--launch-template-id` or `--launch-template-name`, `--dry-run` | Template must exist | NATS `ec2.DeleteLaunchTemplate` → daemon deletes template and all versions from KV → return deleted template info | 1. Delete by ID<br>2. Delete by name<br>3. Non-existent template (error) | **NOT STARTED** |
| `describe-launch-templates` | — | `--launch-template-ids`, `--launch-template-names`, `--filters` | None | NATS `ec2.DescribeLaunchTemplates` → daemon lists templates from KV → return list | 1. List all templates<br>2. Filter by name<br>3. Filter by ID | **NOT STARTED** |
| `describe-launch-template-versions` | — | `--launch-template-id` or `--launch-template-name`, `--versions`, `--min-version`, `--max-version` | Template must exist | NATS `ec2.DescribeLaunchTemplateVersions` → daemon lists versions from KV → return version list with data | 1. List all versions<br>2. List specific version<br>3. Template not found (error) | **NOT STARTED** |
### EC2 - Misc Operations
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `delete-network-interface-permission` | — | `--network-interface-permission-id`, `--force` | Permission must exist | NATS `ec2.DeleteNetworkInterfacePermission` → daemon deletes permission from KV → return success | 1. Delete existing permission<br>2. Force delete | **NOT STARTED** |
| `enable-address-transfer` | — | `--allocation-id`, `--transfer-account-id` | EIP must exist | NATS `ec2.EnableAddressTransfer` → daemon stores transfer in JetStream KV with 7-day expiration → return AllocationId and TransferAccountId | 1. Enable transfer<br>2. Verify transfer stored with expiration | **NOT STARTED** |
| `disable-address-transfer` | — | `--allocation-id` | Transfer must exist | NATS `ec2.DisableAddressTransfer` → daemon updates transfer status to disabled in KV → return status | 1. Disable active transfer<br>2. Disable non-existent transfer (error) | **NOT STARTED** |
### EBS Direct API
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `start-snapshot` | — | `--volume-size`, `--parent-snapshot-id`, `--description`, `--encrypted` | None | Initialize snapshot in viperblock → return snap-ID for block-level writes | 1. Start new snapshot<br>2. Start incremental from parent | **NOT STARTED** |
| `put-snapshot-block` | — | `--snapshot-id`, `--block-index`, `--block-data`, `--checksum` | start-snapshot | Write block data to snapshot at specified index in viperblock → verify checksum | 1. Write single block<br>2. Write with bad checksum (error) | **NOT STARTED** |
| `get-snapshot-block` | — | `--snapshot-id`, `--block-index` | Snapshot must exist | Read block data from snapshot at specified index → return data with checksum | 1. Read existing block<br>2. Read sparse block (zeros) | **NOT STARTED** |
| `complete-snapshot` | — | `--snapshot-id`, `--changed-blocks-count` | start-snapshot, put-snapshot-block | Finalize snapshot → mark as completed → sync to Predastore | 1. Complete valid snapshot<br>2. Complete with wrong block count (error) | **NOT STARTED** |
| `list-snapshot-blocks` | — | `--snapshot-id`, `--max-results`, `--next-token` | Snapshot must exist | List all non-empty blocks in snapshot with tokens and sizes | 1. List blocks of populated snapshot<br>2. List blocks of empty snapshot | **NOT STARTED** |
| `list-changed-blocks` | — | `--second-snapshot-id`, `--first-snapshot-id`, `--max-results` | Snapshots must exist | Compare two snapshots → return list of differing block indexes | 1. Compare parent and child snapshots<br>2. Compare unrelated snapshots | **NOT STARTED** |
### S3 (via Predastore)
Currently S3 operations are handled directly by Predastore. Consider moving control/data plane to Spinifex format for AWS gateway integration.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `s3 mb` (CreateBucket) | — | `s3://bucket-name` | None | NATS `s3.createbucket` → Predastore creates bucket → return success | 1. Create valid bucket<br>2. Duplicate bucket name (error)<br>3. Invalid bucket name (error) | **NOT STARTED** (Predastore handles directly) |
| `s3 rb` (DeleteBucket) | — | `s3://bucket-name`, `--force` | Bucket must exist | NATS `s3.deletebucket` → verify bucket empty (or force) → Predastore deletes bucket | 1. Delete empty bucket<br>2. Delete non-empty bucket (error)<br>3. Force delete non-empty bucket | **NOT STARTED** |
| `s3 ls` (ListBuckets/Objects) | — | `s3://bucket/prefix`, `--recursive` | None | NATS `s3.listobjects` → Predastore lists objects with prefix → return list | 1. List all buckets<br>2. List objects in bucket<br>3. List with prefix filter | **NOT STARTED** |
| `s3 cp` (PutObject/GetObject) | — | `source dest`, `--recursive`, `--acl` | Bucket must exist | NATS `s3.putobject`/`s3.getobject` → Predastore stores/retrieves with Reed-Solomon encoding | 1. Upload single file<br>2. Download single file<br>3. Recursive upload/download<br>4. Large file (multipart) | **NOT STARTED** |
| `s3 rm` (DeleteObject) | — | `s3://bucket/key`, `--recursive` | Object must exist | NATS `s3.deleteobject` → Predastore deletes object shards | 1. Delete single object<br>2. Recursive delete<br>3. Delete non-existent (idempotent) | **NOT STARTED** |
| `s3 sync` | — | `source dest`, `--delete`, `--exclude`, `--include` | Bucket must exist | Compare source and dest → upload changed/new files → optionally delete removed files | 1. Sync local dir to S3<br>2. Sync with --delete<br>3. Sync with exclude filter | **NOT STARTED** |
### IAM - User Management
All IAM operations are account-scoped. Users, policies, and access keys are namespaced per account. The gateway extracts the account ID from the authenticated access key via `c.Locals("sigv4.accountId")` and passes it to all IAM service methods. Root user (account `000000000000`) bypasses policy evaluation entirely.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-user` | `--user-name`, `--path` | `--tags`, `--permissions-boundary` | None | Gateway validates UserName non-empty → IAMService.CreateUser(accountID, input) → generates AIDA-prefixed UserId → stores in KV at key `{accountID}.{userName}` → ARN includes real account ID → conditional create for race safety | 1. Create user<br>2. Duplicate name in same account (EntityAlreadyExists)<br>3. Same name in different account (succeeds)<br>4. Missing username (InvalidInput)<br>5. Default path is `/` | **DONE** |
| `get-user` | `--user-name` | — | User must exist in caller's account | Gateway validates UserName → IAMService.GetUser(accountID, input) → reads from KV at `{accountID}.{userName}` → returns User with ARN, Path, CreateDate | 1. Get existing user<br>2. Get non-existent user (NoSuchEntity)<br>3. Cannot see user in different account (NoSuchEntity) | **DONE** |
| `list-users` | `--path-prefix` | `--max-items`, `--marker` | None | IAMService.ListUsers(accountID, input) → prefix-scans KV for `{accountID}.*` → filters by path prefix if specified → returns user list scoped to caller's account | 1. List all users in account<br>2. List with path prefix filter<br>3. Empty list<br>4. Does not show other accounts' users | **DONE** |
| `delete-user` | `--user-name` | — | User must exist, no access keys attached | Gateway validates UserName → IAMService.DeleteUser(accountID, input) → checks user has no access keys → deletes from KV | 1. Delete user with no keys<br>2. Delete user with keys attached (DeleteConflict)<br>3. Delete non-existent user (NoSuchEntity) | **DONE** |
### IAM - Access Key Management
Access keys are globally unique (keyed by AccessKeyID) but associated with an account. Max 2 keys per user. Secrets encrypted with AES-256-GCM using master key.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-access-key` | `--user-name` | — | User must exist | IAMService.CreateAccessKey(accountID, input) → generates AKIA-prefixed key ID + random secret → encrypts secret with master key → stores in KV (globally unique key ID) → updates user's AccessKeys list → returns plaintext secret (one-time only) | 1. Create key for user (AKIA prefix)<br>2. Max 2 keys per user (LimitExceeded on 3rd)<br>3. Non-existent user (NoSuchEntity)<br>4. Secret only returned on creation | **DONE** |
| `list-access-keys` | `--user-name` | `--max-items`, `--marker` | None | IAMService.ListAccessKeys(accountID, input) → reads user's AccessKeys list → fetches each key's metadata → returns key IDs and status (no secrets) | 1. List keys for user<br>2. No keys returns empty list<br>3. Shows Status (Active/Inactive) but not secrets | **DONE** |
| `delete-access-key` | `--access-key-id`, `--user-name` | — | Key must exist | IAMService.DeleteAccessKey(accountID, input) → removes key from global KV → removes key ID from user's AccessKeys list | 1. Delete existing key<br>2. Delete non-existent key (NoSuchEntity) | **DONE** |
| `update-access-key` | `--access-key-id`, `--user-name`, `--status` (Active/Inactive) | — | Key must exist | IAMService.UpdateAccessKey(input) → reads key from global KV (key ID is globally unique, no accountID needed) → updates Status field → inactive keys rejected at SigV4 auth | 1. Deactivate key (Inactive)<br>2. Reactivate key (Active)<br>3. Non-existent key (NoSuchEntity)<br>4. Inactive key → auth returns InvalidClientTokenId | **DONE** |
### IAM - Policy Management
Policies are account-scoped. KV key is `{accountID}.{policyName}`. Policy documents require `Version: "2012-10-17"` and valid Statement array. Supports wildcard action matching (`ec2:*`, `ec2:Describe*`, `*`). Evaluation order: explicit Deny > explicit Allow > implicit Deny.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-policy` | `--policy-name`, `--policy-document`, `--path`, `--description` | `--tags` | None | Gateway validates PolicyName and PolicyDocument non-empty → IAMService.CreatePolicy(accountID, input) → validates JSON (Version, Statement with Effect/Action/Resource) → generates ANPA-prefixed PolicyId → stores at `{accountID}.{policyName}` → ARN includes real account ID | 1. Create with valid document<br>2. Malformed JSON (MalformedPolicyDocument)<br>3. Missing Effect/Action/Resource (MalformedPolicyDocument)<br>4. Duplicate name in same account (EntityAlreadyExists)<br>5. Same name in different account (succeeds)<br>6. With path and description | **DONE** |
| `get-policy` | `--policy-arn` | — | Policy must exist | Gateway validates PolicyArn → extracts account ID and policy name from ARN → IAMService.GetPolicy(accountID, input) → returns policy metadata (no document) with AttachmentCount | 1. Get existing policy<br>2. Non-existent policy (NoSuchEntity)<br>3. Returns metadata only, not document | **DONE** |
| `get-policy-version` | `--policy-arn`, `--version-id` | — | Policy must exist | Gateway validates PolicyArn and VersionId → IAMService.GetPolicyVersion(accountID, input) → returns PolicyVersion with URL-encoded Document, VersionId, IsDefaultVersion | 1. Get version v1 (returns document)<br>2. Non-existent version (NoSuchEntity)<br>3. Non-existent policy (NoSuchEntity) | **DONE** |
| `list-policies` | — | `--scope`, `--only-attached`, `--path-prefix`, `--max-items`, `--marker` | None | IAMService.ListPolicies(accountID, input) → prefix-scans KV for `{accountID}.*` → returns all policies in caller's account | 1. List all policies in account<br>2. Empty list<br>3. Does not show other accounts' policies | **DONE** |
| `delete-policy` | `--policy-arn` | — | Policy must exist, not attached to any user | Gateway validates PolicyArn → IAMService.DeletePolicy(accountID, input) → checks no users have this policy attached (scans account's users) → deletes from KV | 1. Delete unattached policy<br>2. Delete attached policy (DeleteConflict)<br>3. Delete non-existent (NoSuchEntity) | **DONE** |
### IAM - Policy Attachment
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `attach-user-policy` | `--user-name`, `--policy-arn` | — | User and policy must exist in caller's account | Gateway validates both params → IAMService.AttachUserPolicy(accountID, input) → verifies policy exists → adds policy ARN to user's AttachedPolicies list → idempotent (re-attach succeeds without duplicate) | 1. Attach policy to user<br>2. Already attached (idempotent, no error)<br>3. Non-existent user (NoSuchEntity)<br>4. Non-existent policy (NoSuchEntity) | **DONE** |
| `detach-user-policy` | `--user-name`, `--policy-arn` | — | Policy must be attached to user | Gateway validates both params → IAMService.DetachUserPolicy(accountID, input) → removes policy ARN from user's AttachedPolicies list | 1. Detach attached policy<br>2. Policy not attached (NoSuchEntity)<br>3. Non-existent user (NoSuchEntity) | **DONE** |
| `list-attached-user-policies` | `--user-name` | `--path-prefix`, `--max-items`, `--marker` | User must exist | IAMService.ListAttachedUserPolicies(accountID, input) → reads user's AttachedPolicies list → returns policy ARNs and names | 1. List attached policies<br>2. No policies returns empty list | **DONE** |
### IAM - Policy Evaluation
The gateway evaluates IAM policies on every request (except root user bypass). Policy evaluation is internal — not an AWS API.
| Feature | Description | Status |
|---------|-------------|--------|
| Root user bypass | Root user (`000000000000:root`) skips all policy checks | **DONE** |
| Default deny | Users with no attached policies are denied all actions | **DONE** |
| Explicit allow | Matching Allow statement grants access | **DONE** |
| Explicit deny | Deny statements override Allow (deny wins) | **DONE** |
| Wildcard matching | `*` matches all, `ec2:*` matches all EC2, `ec2:Describe*` matches prefix | **DONE** |
| Account-scoped evaluation | Policies resolved from caller's account context | **DONE** |
| EC2 action mapping | 54 EC2 actions mapped to IAM policy strings | **DONE** |
| IAM action mapping | 16 IAM actions mapped to IAM policy strings | **DONE** |
+28 -54
View File
@@ -1,6 +1,6 @@
# Hive Architecture
# Spinifex Architecture
Hive is an AWS-compatible infrastructure platform for bare-metal, edge, and on-premise deployments. It provides EC2-compatible VM orchestration backed by QEMU/KVM, with storage provided by companion projects Viperblock (EBS) and Predastore (S3).
Spinifex is an AWS-compatible infrastructure platform for bare-metal, edge, and on-premise deployments. It provides EC2-compatible VM orchestration backed by QEMU/KVM, with storage provided by companion projects Viperblock (EBS) and Predastore (S3).
## High-Level Architecture
@@ -19,7 +19,7 @@ Hive is an AWS-compatible infrastructure platform for bare-metal, edge, and on-p
│ • EC2 Query Protocol parsing • Routes to service handlers │
│ • XML response generation • NATS client connection │
│ │
│ File: hive/services/awsgw/awsgw.go │
│ File: spinifex/services/awsgw/awsgw.go │
└─────────────────────────────────────────────────────────────────────────────┘
│
│ JSON over NATS
@@ -28,18 +28,16 @@ Hive is an AWS-compatible infrastructure platform for bare-metal, edge, and on-p
│ NATS Message Broker │
│ │
│ Topics: Queue Groups: │
│ • ec2.RunInstances • hive-workers (load balanced) │
│ • ec2.RunInstances • spinifex-workers (load balanced) │
│ • ec2.DescribeInstances • No queue (fan-out to all nodes) │
│ • ec2.StartInstances │
│ • ec2.StopInstances │
│ • ec2.TerminateInstances │
│ • ec2.cmd.{instanceID} (per-instance start/stop/terminate) │
│ • ebs.mount / ebs.unmount │
└─────────────────────────────────────────────────────────────────────────────┘
│
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
│ Hive Daemon │ │ Hive Daemon │ │ Hive Daemon │
│ Spinifex Daemon │ │ Spinifex Daemon │ │ Spinifex Daemon │
│ (Node 1) │ │ (Node 2) │ │ (Node N) │
│ │ │ │ │ │
│ • NATS subscriber │ │ • NATS subscriber │ │ • NATS subscriber │
@@ -60,7 +58,7 @@ Hive is an AWS-compatible infrastructure platform for bare-metal, edge, and on-p
### 1. AWS SDK Request
Users interact with Hive using standard AWS SDKs or the AWS CLI by pointing to a custom endpoint:
Users interact with Spinifex using standard AWS SDKs or the AWS CLI by pointing to a custom endpoint:
```bash
aws --endpoint-url https://localhost:9999 --no-verify-ssl ec2 run-instances \
@@ -75,7 +73,7 @@ The AWS SDK formats this as an HTTP POST with:
### 2. AWS Gateway
The gateway (`hive/services/awsgw/awsgw.go:57-107`) is the entry point:
The gateway (`spinifex/services/awsgw/awsgw.go:57-107`) is the entry point:
```go
// awsgw.go:74 - Connect to NATS
@@ -91,7 +89,7 @@ gw := gateway.GatewayConfig{
log.Fatal(app.ListenTLS(nodeConfig.AWSGW.Host, nodeConfig.AWSGW.TLSCert, nodeConfig.AWSGW.TLSKey))
```
Request routing (`hive/gateway/gateway.go:125-156`):
Request routing (`spinifex/gateway/gateway.go:125-156`):
1. **Authentication**: SigV4 middleware validates AWS credentials (`gateway.go:104`)
2. **Service Detection**: Extracts service name (ec2, iam, account) from Authorization header (`gateway.go:158-186`)
@@ -111,7 +109,7 @@ case "iam":
### 3. EC2 Handler
The EC2 handler (`hive/gateway/ec2.go`) parses the Action parameter and delegates to specific handlers:
The EC2 handler (`spinifex/gateway/ec2.go`) parses the Action parameter and delegates to specific handlers:
```go
// ec2.go - Action routing
@@ -126,7 +124,7 @@ case "DescribeInstances":
### 4. NATS Messaging
The gateway communicates with daemons via NATS request/response (`hive/handlers/ec2/instance/service_nats.go:24-57`):
The gateway communicates with daemons via NATS request/response (`spinifex/handlers/ec2/instance/service_nats.go:24-57`):
```go
// service_nats.go:24-57
@@ -147,13 +145,13 @@ func (s *NATSInstanceService) RunInstances(input *ec2.RunInstancesInput) (*ec2.R
### 5. Daemon Processing
Daemons (`hive/daemon/daemon.go`) subscribe to NATS topics and handle requests:
Daemons (`spinifex/daemon/daemon.go`) subscribe to NATS topics and handle requests:
```go
// daemon.go:319 - Subscribe with queue group for load balancing
d.natsSubscriptions["ec2.RunInstances"], err = d.natsConn.QueueSubscribe(
"ec2.RunInstances",
"hive-workers", // Queue group - only one daemon handles each request
"spinifex-workers", // Queue group - only one daemon handles each request
d.handleEC2RunInstances,
)
@@ -164,7 +162,7 @@ d.natsSubscriptions["ec2.DescribeInstances"], err = d.natsConn.Subscribe(
)
```
**Queue Groups**: Topics subscribed with a queue group (`hive-workers`) are load-balanced - only one daemon handles each request. Topics without a queue group fan out to all daemons.
**Queue Groups**: Topics subscribed with a queue group (`spinifex-workers`) are load-balanced - only one daemon handles each request. Topics without a queue group fan out to all daemons.
### 6. VM Launch
@@ -227,21 +225,19 @@ type ResourceManager struct {
| Topic | Queue Group | Purpose |
|-------|-------------|---------|
| `ec2.RunInstances` | `hive-workers` | Launch new instances |
| `ec2.RunInstances` | `spinifex-workers` | Launch new instances |
| `ec2.DescribeInstances` | None (fan-out) | Query all nodes for instances |
| `ec2.StartInstances` | `hive-workers` | Start stopped instances |
| `ec2.StopInstances` | `hive-workers` | Stop running instances |
| `ec2.TerminateInstances` | `hive-workers` | Terminate instances |
| `ec2.CreateKeyPair` | `hive-workers` | Generate SSH keypair |
| `ec2.DescribeKeyPairs` | `hive-workers` | List SSH keypairs |
| `ec2.DescribeImages` | `hive-workers` | List AMIs |
| `ec2.cmd.{instanceID}` | None (owner only) | Per-instance commands (start/stop/terminate, attach/detach volume) |
| `ec2.CreateKeyPair` | `spinifex-workers` | Generate SSH keypair |
| `ec2.DescribeKeyPairs` | `spinifex-workers` | List SSH keypairs |
| `ec2.DescribeImages` | `spinifex-workers` | List AMIs |
| `ebs.mount` | - | Mount volume via Viperblock |
| `ebs.unmount` | - | Unmount volume |
| `hive.admin.{node}.health` | None | Node health checks |
| `spinifex.admin.{node}.health` | None | Node health checks |
### Multi-Node Aggregation
For operations that need data from all nodes (like `DescribeInstances`), the gateway uses inbox-based fan-out (`hive/gateway/ec2/instance/DescribeInstances.go`):
For operations that need data from all nodes (like `DescribeInstances`), the gateway uses inbox-based fan-out (`spinifex/gateway/ec2/instance/DescribeInstances.go`):
```go
// DescribeInstances.go:16-111
@@ -292,13 +288,13 @@ Object storage used for:
## Configuration
Cluster configuration (`hive/config/config.go`):
Cluster configuration (`spinifex/config/config.go`):
```go
type ClusterConfig struct {
Epoch uint64 // Incremented on config changes
Node string // This node's identifier
Version string // Hive version
Version string // Spinifex version
Nodes map[string]Config // Configuration for all nodes
}
@@ -316,8 +312,8 @@ type Config struct {
## File Structure
```
hive/
├── hive/
spinifex/
├── spinifex/
│ ├── services/
│ │ └── awsgw/
│ │ └── awsgw.go # AWS Gateway service entry point
@@ -344,7 +340,7 @@ hive/
│ └── qmp/
│ └── qmp.go # QEMU Machine Protocol client
└── cmd/
└── hive/
└── spinifex/
└── main.go # CLI entry point
```
@@ -355,8 +351,8 @@ hive/
1. Start NATS broker
2. Start Predastore (S3)
3. Start Viperblock (EBS)
4. Start Hive Gateway (`hive awsgw`)
5. Start Hive Daemon(s) (`hive daemon`)
4. Start Spinifex Gateway (`spx awsgw`)
5. Start Spinifex Daemon(s) (`spx daemon`)
### Testing with AWS CLI
@@ -375,25 +371,3 @@ aws --no-verify-ssl ec2 describe-instances
# Terminate instance
aws --no-verify-ssl ec2 terminate-instances --instance-ids i-xxxxx
```
## Improvements
Improve `aws ec2 stop-instances` , currently uses a ~30 sec heartbeat to detect if an instance is terminated. When the stop-instance command is sent, run the heartbeat more frequently to detect closer to realtime when the instance has stopped.
Currently a ~54 second window to detect termination.
```sh
aws ec2 stop-instances --instance-ids i-1e306bb58b99e03bd
```
```sh
2026/01/22 20:01:52 INFO QMP event event=POWERDOWN
2026/01/22 20:02:17 INFO QMP heartbeat instance=i-1e306bb58b99e03bd
2026/01/22 20:02:17 INFO QMP status status="{\"running\":true,\"singlestep\":false,\"status\":\"running\"}"
2026/01/22 20:02:47 INFO QMP heartbeat instance=i-1e306bb58b99e03bd
2026/01/22 20:02:47 ERROR Failed to send QMP command err="encode error: write unix @->/run/user/1000/qmp-i-1e306bb58b99e03bd.sock: write: broken pipe"
2026/01/22 20:02:47 INFO QMP Status - Instance stopped, exiting heartbeat id=i-1e306bb58b99e03bd
2026/01/22 20:02:47 INFO Unsubscribing from NATS subject instance=i-1e306bb58b99e03bd
2026/01/22 20:02:47 INFO Closing QMP client connection instance=i-1e306bb58b99e03bd
```
File diff suppressed because it is too large Load Diff
+3
View File
@@ -0,0 +1,3 @@
# Spinifex Documentation
This directory contains the Spinifex documentation. This is available in individual markdown files, and a generated website to aid navigation and browing documentation at: [https://docs.mulgadc.com](https://docs.mulgadc.com/)
+61
View File
@@ -0,0 +1,61 @@
---
title: "Creating Accounts"
description: "Create and manage isolated user accounts with their own resources."
category: "Administration"
tags:
- accounts
- iam
- admin
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
---
# Creating Accounts
> Create and manage isolated user accounts with their own resources.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex supports multi-tenant account isolation. Each account gets its own IAM credentials, AWS CLI profile, and isolated resource namespace.
## Instructions
## Create Account
```bash
./bin/spx admin account create --name myteam
export AWS_PROFILE=spinifex-myteam
```
## Verify
```bash
aws sts get-caller-identity
```
## Troubleshooting
## Credentials not working
Verify the AWS CLI configuration files exist and contain the correct profile:
```bash
cat ~/.aws/config
cat ~/.aws/credentials
```
Ensure the `AWS_PROFILE` environment variable matches the profile name:
```bash
echo $AWS_PROFILE
export AWS_PROFILE=spinifex-myteam
```
+120
View File
@@ -0,0 +1,120 @@
---
title: "Creating an Environment"
description: "End-to-end guide: user accounts, SSH keys, VPCs, AMIs, and launching instances."
category: "Environments"
tags:
- environment
- vpc
- ec2
- ssh
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Debian Cloud Images"
url: "https://cloud.debian.org/images/cloud/bookworm/latest/"
---
# Creating an Environment
> End-to-end guide: user accounts, SSH keys, VPCs, AMIs, and launching instances.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
This guide walks through the complete workflow for creating a production-ready Spinifex environment.
**Steps:**
1. Create a user account
2. Import or create SSH keys
3. Create a VPC and subnet
4. Select and import an AMI
5. Launch EC2 instances
6. Connect via SSH
7. Monitor instance health
## Instructions
## 1. Import SSH Key
```bash
aws ec2 import-key-pair --key-name "spinifex-key" --public-key-material fileb://~/.ssh/id_rsa.pub
```
## 2. Create VPC
```bash
aws ec2 create-vpc --cidr-block 10.200.0.0/16
export SPINIFEX_VPC="vpc-XXX"
aws ec2 create-subnet --vpc-id $SPINIFEX_VPC --cidr-block 10.200.1.0/24
export SPINIFEX_SUBNET="subnet-XXX"
```
## 3. Import AMI
```bash
./bin/spx admin images list
./bin/spx admin images import --name debian-12-x86_64
export SPINIFEX_AMI="ami-XXX"
```
## 4. Launch Instance
```bash
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 1
```
## 5. Connect via SSH
```bash
ps auxw | grep $INSTANCE_ID
# Look for: hostfwd=tcp:127.0.0.1:<port>-:22
ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
```
## Troubleshooting
## Instance stuck in pending
Check the Spinifex daemon and QEMU logs for boot errors:
```bash
ls ~/spinifex/logs/
cat ~/spinifex/logs/spinifex.log
```
Verify the AMI exists and architecture matches your host:
```bash
aws ec2 describe-images --image-ids $SPINIFEX_AMI
```
## SSH connection refused
cloud-init takes 30-60 seconds to configure networking and SSH after boot. Wait and retry.
Verify the SSH port forwarding is active:
```bash
ps auxw | grep hostfwd
```
Ensure you're using the correct key and port:
```bash
ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
```
+74
View File
@@ -0,0 +1,74 @@
---
title: "Hybrid Sync"
description: "Synchronize data between Spinifex and AWS when network connectivity is available."
category: "Migration"
tags:
- hybrid
- sync
- s3
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
---
# Hybrid Sync
> Synchronize data between Spinifex and AWS when network connectivity is available.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex's hybrid mode enables bidirectional data synchronization between local infrastructure and AWS cloud services. Ideal for intermittent connectivity environments.
## Instructions
## Push Local to Cloud
```bash
aws s3 sync s3://local-bucket/ s3://cloud-bucket/ \
--source-region spinifex --region us-east-1
```
## Pull Cloud to Local
```bash
aws s3 sync s3://cloud-bucket/ s3://local-bucket/ \
--source-region us-east-1 --region spinifex
```
## EBS Volume Backup
```bash
rsync -avz /data/ user@cloud-server:/backup/spinifex-data/
```
## Troubleshooting
## Sync fails mid-transfer
S3 sync is idempotent — re-run the same command to resume where it left off:
```bash
aws s3 sync s3://local-bucket/ s3://cloud-bucket/ \
--source-region spinifex --region us-east-1
```
Only changed or missing files will be transferred on subsequent runs.
## Permissions errors during sync
Verify your AWS credentials are configured for both the source and destination:
```bash
aws sts get-caller-identity --profile spinifex
aws sts get-caller-identity --profile default
```
Ensure the IAM user has `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket` permissions on both buckets.
@@ -0,0 +1,132 @@
---
title: "Air-Gapped Installation"
description: "Deploy Spinifex in environments without internet connectivity. Covers offline package caching, USB deployment, and package verification."
category: "Getting Started"
tags:
- install
- air-gapped
- offline
badge: advanced
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "apt-cacher-ng"
url: "https://wiki.debian.org/AptCacherNg"
---
# Air-Gapped Installation
> Deploy Spinifex in environments without internet connectivity.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
In air-gapped or disconnected environments, Spinifex can be deployed without internet access. This guide covers preparing offline packages on a connected machine, creating USB deployment media, and installing on the target server with package verification.
**Deployment methods:**
- **apt-cacher-ng** — Local APT cache for Debian/Ubuntu packages
- **Go module cache** — Pre-downloaded Go dependencies
- **USB deployment** — Portable installation media with all dependencies
- **GPG verification** — Cryptographic package integrity checks
## Instructions
## Step 1. Prepare packages (on a connected machine)
Set up a local APT cache to collect all required packages:
```bash
sudo apt install apt-cacher-ng
sudo systemctl enable --now apt-cacher-ng
```
Pre-download all Spinifex dependencies:
```bash
sudo apt install --download-only nbdkit nbdkit-plugin-dev pkg-config \
qemu-system qemu-utils qemu-kvm libvirt-daemon-system \
libvirt-clients libvirt-dev make gcc unzip xz-utils file \
ovn-central ovn-host openvswitch-switch
```
## Step 2. Create USB deployment media
```bash
mkdir -p /media/spinifex-deploy/{apt-packages,go-cache,spinifex-source}
cp /var/cache/apt/archives/*.deb /media/spinifex-deploy/apt-packages/
cp -r ~/go/pkg/mod/cache/ /media/spinifex-deploy/go-cache/
cp -r ~/Development/mulga/spinifex /media/spinifex-deploy/spinifex-source/
```
## Step 3. Verify package integrity
Sign and verify packages before transferring to the target:
```bash
gpg --import /media/spinifex-deploy/mulga-signing-key.asc
gpg --verify spinifex-v1.0.tar.gz.sig spinifex-v1.0.tar.gz
sha256sum -c /media/spinifex-deploy/checksums.sha256
```
## Step 4. Install on the target server
Mount the USB and install:
```bash
sudo mount /dev/sdb1 /mnt/usb
sudo dpkg -i /mnt/usb/apt-packages/*.deb
cp -r /mnt/usb/spinifex-source ~/Development/mulga/spinifex
cd ~/Development/mulga/spinifex && make build
```
Then follow the [Source Install](/docs/source-install) guide from Step 3 (Setup OVN) onwards.
## Troubleshooting
## Missing dependencies after dpkg install
Some packages may have unresolved dependencies. Fix with:
```bash
sudo dpkg -i /mnt/usb/apt-packages/*.deb
sudo apt-get install -f
```
The `-f` flag tells apt to fix broken dependencies using what's available locally.
## GPG verification fails
The signing key may not match or the download was corrupted. Re-import the key and verify:
```bash
gpg --import /media/spinifex-deploy/mulga-signing-key.asc
gpg --verify spinifex-v1.0.tar.gz.sig spinifex-v1.0.tar.gz
```
If verification still fails, re-download the package on the connected machine and compare checksums:
```bash
sha256sum spinifex-v1.0.tar.gz
```
## Go module cache errors
The Go module cache may have been incompletely copied. On the connected machine, re-export the full cache:
```bash
cp -r ~/go/pkg/mod/cache/ /media/spinifex-deploy/go-cache/
```
On the target, restore it:
```bash
mkdir -p ~/go/pkg/mod/
cp -r /mnt/usb/go-cache ~/go/pkg/mod/cache
```
@@ -0,0 +1,288 @@
---
title: "Multi-Node Installation"
description: "Deploy Spinifex across multiple servers to create an availability zone with high availability, data durability, and fault tolerance."
category: "Getting Started"
tags:
- install
- multi node
- cluster
badge: availability-zone
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Predastore (S3)"
url: "https://github.com/mulgadc/predastore"
- title: "Viperblock (EBS)"
url: "https://github.com/mulgadc/viperblock"
---
# Multi-Node Installation
> Deploy Spinifex across multiple servers to create an availability zone.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
A Spinifex cluster operates as a fully distributed infrastructure region — similar to an AWS Region — where multiple nodes provide high availability, data durability, and fault tolerance.
**Services distributed across nodes:**
- **NATS** — Clustered message bus for request routing and JetStream replication
- **Predastore** — Raft consensus with Reed-Solomon erasure coding (RS 2+1)
- **Viperblock** — Block storage co-located with compute nodes
- **Spinifex Daemon** — Independent AWS API request serving per node
Cluster formation is automatic. When initializing with `--nodes 3`, the init node starts a formation server and waits for peers to join. Once all nodes register, each receives the full cluster topology — credentials, CA certificates, NATS routes, Predastore peer lists — no manual configuration synchronization required.
**Network requirements:**
- Minimum 1 NIC per server
- 2 NICs recommended for production (management + overlay)
- UDP port 6081 open between hosts (Geneve tunnels)
## Instructions
## Step 1. Install Spinifex on each server
Run the binary installer on **every server** in the cluster:
```bash
curl https://install.mulgadc.com | bash
```
## Step 2. Set node IP variables
On **each server**, export the IPs for all nodes (replace with your actual IPs):
```bash
export SPINIFEX_NODE1=192.168.1.10
export SPINIFEX_NODE2=192.168.1.11
export SPINIFEX_NODE3=192.168.1.12
```
To find your server's IP:
```bash
ip addr show | grep "inet "
```
Next, define the AWS region and AZ you are deploying locally.
```bash
export AWS_REGION=us-east-1
export AWS_AZ=us-east-1a
```
## Step 3. Setup OVN networking
OVN must be configured on every server before forming the cluster. Server 1 runs OVN central (NB and SB databases) and must be set up first.
**Server 1 — OVN central + compute (run first):**
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --management --encap-ip=$SPINIFEX_NODE1
```
Verify OVN central is ready before proceeding:
```bash
sudo ovn-sbctl show
```
**Server 2 — Compute node (after server 1 is ready):**
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --ovn-remote=tcp:$SPINIFEX_NODE1:6642 --encap-ip=$SPINIFEX_NODE2
```
**Server 3 — Compute node (after server 1 is ready):**
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --ovn-remote=tcp:$SPINIFEX_NODE1:6642 --encap-ip=$SPINIFEX_NODE3
```
Verify all chassis have registered (from server 1):
```bash
sudo ovn-sbctl show
```
You should see 3 chassis entries, one per server, each with a Geneve encap IP.
## Step 4. Form the cluster
The formation process requires running init and join commands concurrently. The init node blocks until all expected nodes have joined.
**Server 1 — Initialize the cluster:**
```bash
sudo spx admin init \
--node node1 \
--nodes 3 \
--bind $SPINIFEX_NODE1 \
--cluster-bind $SPINIFEX_NODE1 \
--port 4432 \
--region $AWS_REGION \
--az $AWS_AZ
```
**Server 2 — Join the cluster (while init is running):**
```bash
spx admin join \
--node node2 \
--bind $SPINIFEX_NODE2 \
--cluster-bind $SPINIFEX_NODE2 \
--host $SPINIFEX_NODE1:4432 \
--region $AWS_REGION \
--az $AWS_AZ
```
**Server 3 — Join the cluster (while init is running):**
```bash
spx admin join \
--node node3 \
--bind $SPINIFEX_NODE3 \
--cluster-bind $SPINIFEX_NODE3 \
--host $SPINIFEX_NODE1:4432 \
--region $AWS_REGION \
--az $AWS_AZ
```
All three processes will exit with a cluster summary once formation is complete.
## Step 5. Start services and verify
Start services on **all servers**:
```bash
sudo systemctl start spinifex.target
```
From any node, verify the cluster:
```bash
spx get nodes
```
```
NAME STATUS IP REGION AZ UPTIME VMs
node1 Ready 192.168.1.10 us-east-1 us-east-1a 2m 0
node2 Ready 192.168.1.11 us-east-1 us-east-1a 2m 0
node3 Ready 192.168.1.12 us-east-1 us-east-1a 2m 0
```
## Step 6. Trust the CA certificate
On **each server**, trust the CA generated during init:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
export AWS_PROFILE=spinifex
```
## Step 7. Launch instances across the cluster
Connect to any node's AWS Gateway — all nodes serve the same cluster state:
```bash
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 3
```
Spinifex distributes instances across nodes based on available capacity. Check which node each instance landed on:
```bash
spx get vms
```
## Shutdown
A graceful shutdown coordinates all nodes — drains VMs, flushes storage, and stops services:
```bash
spx admin cluster shutdown
```
## Troubleshooting
## Nodes not joining the cluster
The init command must still be running when you execute join on other servers. If init has already exited, re-run it.
Check that all servers can reach the init node on the formation port:
```bash
curl -s http://$SPINIFEX_NODE1:4432/health
```
## OVN chassis not registering
Compute nodes need to reach OVN central on server 1. Verify from server 1:
```bash
sudo ovn-sbctl show
```
If chassis are missing, check firewall rules for port 6642:
```bash
sudo ss -tlnp | grep 6642
```
Re-run the OVN setup on the affected compute node.
## Cross-host VMs cannot communicate
This indicates a Geneve tunnel issue. Verify tunnels are established:
```bash
sudo ovs-vsctl show | grep -i geneve
```
Ensure UDP port 6081 is open between all hosts:
```bash
sudo ss -ulnp | grep 6081
```
If tunnels are missing, verify `--encap-ip` was set correctly during OVN setup.
## Health check fails on a node
Check if the daemon is running and listening on the correct IP:
```bash
curl -s http://$SPINIFEX_NODE1:4432/health
```
Review the daemon logs for errors:
```bash
cat ~/spinifex/logs/spinifex.log
```
Verify `--bind` was set to the correct IP during cluster formation.
## NATS cluster routing issues
Check NATS logs for connection errors between nodes:
```bash
grep -i "route\|cluster" ~/spinifex/logs/nats.log
```
Ensure `--cluster-bind` IPs are reachable between all servers.
+200
View File
@@ -0,0 +1,200 @@
---
title: "Installing Spinifex"
description: "Install Spinifex on a single server using the binary installer. Get a working region with EC2, VPC, EBS, and S3 in minutes."
category: "Getting Started"
tags:
- install
- single node
- quickstart
badge: quickstart
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Predastore (S3)"
url: "https://github.com/mulgadc/predastore"
- title: "Viperblock (EBS)"
url: "https://github.com/mulgadc/viperblock"
---
# Installing Spinifex
> Install Spinifex on a single server using the binary installer.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex is an open-source infrastructure platform that brings core AWS services — EC2, VPC, EBS, and S3 — to bare-metal, edge, and on-prem environments.
This guide installs Spinifex on a single server using the binary installer. For multi-server clusters, see [Multi-Node Installation](/docs/installing-spinifex-multi-node). To build from source, see [Source Install](/docs/source-install).
**Supported Operating Systems:**
- Ubuntu 22.04 / 24.04 / 25.10
- Debian 12.13
**What gets installed:**
- Spinifex daemon and CLI
- QEMU/KVM for virtual machine management
- OVN/Open vSwitch for VPC networking
- Predastore (S3-compatible object storage)
- Viperblock (EBS-compatible block storage)
- AWS CLI v2
## Instructions
## Step 1. Install Spinifex
```bash
curl https://install.mulgadc.com | bash
```
The installer downloads the Spinifex binary and bootstraps all dependencies (QEMU, OVN/OVS, AWS CLI).
## Step 2. Initialize a region
Create your first region and availability zone, replace `ap-southeast-2` with your desired region:
```bash
spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
```
## Step 3. Trust the CA certificate
Spinifex generates a local CA during initialization. Add it to your system trust store:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
## Step 4. Start services
```bash
sudo systemctl start spinifex.target
```
Set the AWS profile to use the default admin account:
```bash
export AWS_PROFILE=spinifex
```
## Step 5. Import an AMI
List available images and import one matching your architecture:
```bash
spx admin images list
spx admin images import --name debian-12-arm64
```
## Step 6. Create a VPC and launch an instance
```bash
aws ec2 create-vpc --cidr-block 10.200.0.0/16
export SPINIFEX_VPC="vpc-XXX"
aws ec2 create-subnet --vpc-id $SPINIFEX_VPC --cidr-block 10.200.1.0/24
export SPINIFEX_SUBNET="subnet-XXX"
aws ec2 import-key-pair --key-name "spinifex-key" --public-key-material fileb://~/.ssh/id_rsa.pub
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 1
```
## Step 7. Connect via SSH
Find the forwarded SSH port and connect:
```bash
ps auxw | grep hostfwd
# Look for: hostfwd=tcp:127.0.0.1:<port>-:22
ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
```
## Troubleshooting
## spx command not found
The binary is not in your PATH. Run the installer again or add the install directory to your PATH:
```bash
export PATH=$PATH:~/spinifex/bin/
```
Add this to your `~/.bashrc` or `~/.zshrc` for persistence.
## OVN services not starting
Check if the OVN controller is active:
```bash
sudo systemctl is-active ovn-controller
sudo ovn-sbctl show
```
If inactive, re-run the OVN setup that the installer performs. Check system logs for details:
```bash
journalctl -u ovn-controller --no-pager -n 20
```
## CA certificate not trusted
AWS CLI will reject HTTPS connections if the Spinifex CA is not trusted. Re-add the certificate:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
Verify it was added:
```bash
ls -la /usr/local/share/ca-certificates/spinifex-ca.crt
```
## Instance stuck in pending
Check the Spinifex daemon logs for errors:
```bash
ls ~/spinifex/logs/
cat ~/spinifex/logs/spinifex.log
```
Verify the AMI was imported successfully:
```bash
aws ec2 describe-images
```
## SSH connection refused
cloud-init takes 30-60 seconds to configure the instance after boot. Wait and retry.
Verify the SSH port forwarding is active:
```bash
ps auxw | grep hostfwd
```
If no `hostfwd` entry appears, the instance may not have started correctly. Check instance state:
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
+98
View File
@@ -0,0 +1,98 @@
---
title: "Launching Instances"
description: "Launch, manage, and connect to EC2-compatible virtual machines on Spinifex."
category: "Environments"
tags:
- ec2
- instances
- vm
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "AWS EC2 CLI"
url: "https://docs.aws.amazon.com/cli/latest/reference/ec2/"
---
# Launching Instances
> Launch, manage, and connect to EC2-compatible virtual machines on Spinifex.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex provides EC2-compatible VM management built on QEMU/KVM. Instances support cloud-init, SSH key injection, VPC networking, and standard AWS lifecycle operations.
**Supported operations:**
- `run-instances` — Launch new VMs
- `describe-instances` — Query state
- `stop-instances` / `start-instances` — Lifecycle
- `terminate-instances` — Permanent removal
## Instructions
## Launch
```bash
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 1
export INSTANCE_ID="i-XXX"
```
## Manage
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
aws ec2 stop-instances --instance-ids $INSTANCE_ID
aws ec2 start-instances --instance-ids $INSTANCE_ID
aws ec2 terminate-instances --instance-ids $INSTANCE_ID
```
## SSH (Development)
```bash
ps auxw | grep $INSTANCE_ID
ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
```
## Troubleshooting
## Instance fails to boot
Check QEMU logs for the instance and verify the AMI architecture matches your host:
```bash
cat ~/spinifex/logs/spinifex.log
aws ec2 describe-images --image-ids $SPINIFEX_AMI
```
If the AMI is for a different architecture (e.g. arm64 on an x86_64 host), import the correct image:
```bash
./bin/spx admin images list
./bin/spx admin images import --name debian-12-x86_64
```
## Cannot SSH into instance
cloud-init needs time to configure the instance after boot. Wait 30-60 seconds and retry.
Verify the SSH key was specified correctly when launching:
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
Check the `KeyName` field matches the key you're using to connect.
+102
View File
@@ -0,0 +1,102 @@
---
title: "Moving an AWS Workload to Mulga"
description: "Migrate existing AWS workloads to Spinifex using compatible APIs, SDKs, and Terraform."
category: "Migration"
tags:
- migration
- aws
- terraform
badge: new
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "AWS CLI Reference"
url: "https://docs.aws.amazon.com/cli/latest/reference/"
- title: "Terraform AWS Provider"
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
---
# Moving an AWS Workload to Mulga
> Migrate existing AWS workloads to Spinifex using compatible APIs, SDKs, and Terraform.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex provides drop-in compatibility with AWS APIs, making it possible to migrate existing workloads with minimal changes.
**Supported Services:** VPC, EC2, EBS, S3, IAM
**Compatible Tools:** AWS CLI, AWS SDKs, Terraform, any S3-compatible client
## Instructions
## Configure AWS CLI
```bash
export AWS_PROFILE=spinifex
aws ec2 describe-instances
aws s3 ls
```
## Terraform
```hcl
provider "aws" {
region = "ap-southeast-2"
access_key = "your-spinifex-access-key"
secret_key = "your-spinifex-secret-key"
endpoints {
ec2 = "https://localhost:9999"
s3 = "https://localhost:8443"
}
skip_credentials_validation = true
skip_metadata_api_check = true
skip_requesting_account_id = true
}
```
## Hybrid S3 Sync
```bash
aws s3 sync s3://local-bucket/ s3://cloud-bucket/ --source-region spinifex --region us-east-1
```
## Troubleshooting
## Terraform provider errors
Ensure all three skip flags are set in your provider configuration:
```hcl
skip_credentials_validation = true
skip_metadata_api_check = true
skip_requesting_account_id = true
```
Without these, Terraform will try to validate credentials and metadata against real AWS endpoints.
## S3 signature errors
Spinifex uses AWS Signature V4. Ensure your AWS CLI is version 2.0 or higher:
```bash
aws --version
```
If using an older version, upgrade:
```bash
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update
```
+194
View File
@@ -0,0 +1,194 @@
---
title: "Source Install"
description: "Build Spinifex from source for development, custom builds, or contributing. Includes dependency setup, compilation, and OVN configuration."
category: "Getting Started"
tags:
- install
- source
- development
badge: source
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Predastore (S3)"
url: "https://github.com/mulgadc/predastore"
- title: "Viperblock (EBS)"
url: "https://github.com/mulgadc/viperblock"
- title: "Go Downloads"
url: "https://go.dev/dl/"
- title: "AWS CLI Install"
url: "https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html"
---
# Source Install
> Build Spinifex from source for development, custom builds, or contributing.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
This guide covers building Spinifex from source code. Use this if you want to contribute to Spinifex, build custom versions, or inspect the codebase. For production deployments, the [binary installer](/docs/installing-spinifex) is recommended.
**Supported Operating Systems:**
- Ubuntu 22.04 / 24.04 / 25.10
- Debian 12.13
**Requirements:**
- Go 1.26.1+
- GCC, make, pkg-config
- QEMU/KVM
- OVN/Open vSwitch
- AWS CLI v2
## Instructions
## Step 1. Install system dependencies
### Quick Install
Bootstrap all dependencies in one step:
```bash
sudo make -C spinifex quickinstall
```
Ensure Go is in your PATH:
```bash
export PATH=$PATH:/usr/local/go/bin/
```
### Manual Install
Alternatively, install packages individually:
```bash
sudo add-apt-repository universe
sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev make gcc unzip xz-utils file ovn-central ovn-host openvswitch-switch
```
Install Go 1.26.1+ from [https://go.dev/dl/](https://go.dev/dl/) and verify:
```bash
go version
```
Install AWS CLI v2:
```bash
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version
```
## Step 2. Clone and build
```bash
mkdir -p ~/Development/mulga/
cd ~/Development/mulga/
git clone https://github.com/mulgadc/spinifex.git
cd spinifex
./scripts/clone-deps.sh # Clone viperblock + predastore
./scripts/dev-setup.sh # Setup complete dev environment
```
Confirm `./bin/spx` exists and is executable.
## Step 3. Setup OVN
For a single-node dev environment:
```bash
./scripts/setup-ovn.sh --management
```
This creates the `br-int` integration bridge, starts `ovn-controller`, configures Geneve tunnel endpoints, and enables IP forwarding.
## Step 4. Initialize
Create a region and availability zone:
```bash
./bin/spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
```
## Step 5. Trust the CA certificate
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
## Step 6. Start services
```bash
./scripts/start-dev.sh
export AWS_PROFILE=spinifex
```
## Troubleshooting
## Go not found in PATH
After installing Go, it may not be in your current shell session. Add it:
```bash
export PATH=$PATH:/usr/local/go/bin/
```
Add this to your `~/.bashrc` or `~/.zshrc` for persistence. Verify with:
```bash
go version
```
## ./bin/spx missing after build
The build did not complete successfully. Re-run the setup:
```bash
./scripts/dev-setup.sh
```
Check the output for compilation errors. Common causes are missing Go version or missing system packages.
## OVN services not starting
Verify the setup script was run with `--management`:
```bash
./scripts/setup-ovn.sh --management
```
Check OVN controller status:
```bash
sudo systemctl is-active ovn-controller
sudo ovn-sbctl show
```
## CA certificate not trusted
AWS CLI will reject HTTPS connections to Spinifex services. Re-add the certificate:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
## clone-deps.sh fails
Ensure you have access to the GitHub repositories. If using SSH and it fails, try HTTPS:
```bash
git config --global url."https://github.com/".insteadOf "git@github.com:"
./scripts/clone-deps.sh
```
+97
View File
@@ -0,0 +1,97 @@
---
title: "Spinifex Admin CLI"
description: "Complete reference for the Spinifex administration CLI. Manage accounts, nodes, VMs, and services."
category: "Administration"
tags:
- cli
- admin
- reference
badge: cli
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
---
# Spinifex Admin CLI
> Complete reference for the Spinifex administration CLI.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
The `spx` binary is the central administration tool for managing your Spinifex infrastructure. It provides commands for cluster initialization, account management, node operations, VM lifecycle, and service control.
**Binary location:** `~/Development/mulga/spinifex/bin/spx`
All services in the Spinifex platform are managed through this single binary.
## Instructions
## Account Management
```bash
./bin/spx admin account create --name myteam
export AWS_PROFILE=spinifex-myteam
```
## Node Management
```bash
./bin/spx get nodes
```
```
NAME STATUS IP REGION AZ UPTIME VMs
node1 Ready 127.0.0.1 ap-southeast-2 ap-southeast-2a 2m 0
node2 Ready 127.0.0.2 ap-southeast-2 ap-southeast-2a 2m 0
node3 Ready 127.0.0.3 ap-southeast-2 ap-southeast-2a 2m 0
```
## Monitor Resources
```bash
./bin/spx top nodes
```
## Image Management
```bash
./bin/spx admin images list
./bin/spx admin images import --name debian-12-arm64
```
## Cluster Shutdown
```bash
./bin/spx admin cluster shutdown
```
## Troubleshooting
## Permission denied running spinifex
The binary may not be executable. Fix permissions:
```bash
chmod +x ./bin/spx
```
If you get permission errors during operations, ensure you're running with appropriate privileges. Some OVN and networking commands require `sudo`.
## Services fail to start
Check the daemon logs for specific errors:
```bash
ls ~/spinifex/logs/
cat ~/spinifex/logs/spinifex.log
```
Common causes include port conflicts, missing OVN configuration, or untrusted CA certificates.
+97
View File
@@ -0,0 +1,97 @@
---
title: "VPC Networking"
description: "Create and manage Virtual Private Clouds with subnets, DHCP, and Geneve overlay networking."
category: "Environments"
tags:
- vpc
- networking
- ovn
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "OVN Architecture"
url: "https://www.ovn.org/en/"
---
# VPC Networking
> Create and manage Virtual Private Clouds with subnets, DHCP, and Geneve overlay networking.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Every EC2 instance runs inside a VPC with an isolated virtual network. Spinifex uses OVN (Open Virtual Network) to provide the networking layer.
**How it works:**
- VPC becomes an OVN logical router
- Subnet becomes an OVN logical switch with DHCP
- Each instance gets an OVN port with automatic IP assignment
- Cross-host traffic uses Geneve tunnels (UDP 6081)
## Instructions
## Create VPC
```bash
aws ec2 create-vpc --cidr-block 10.200.0.0/16
export SPINIFEX_VPC="vpc-XXX"
```
## Create Subnet
```bash
aws ec2 create-subnet --vpc-id $SPINIFEX_VPC --cidr-block 10.200.1.0/24
export SPINIFEX_SUBNET="subnet-XXX"
```
## Verify
```bash
aws ec2 describe-vpcs
aws ec2 describe-subnets
sudo ovn-nbctl lr-list
sudo ovn-nbctl ls-list
```
## Troubleshooting
## VPC creation fails
Ensure OVN services are running and the vpcd daemon is active:
```bash
sudo systemctl is-active ovn-controller
```
Check the vpcd logs for errors:
```bash
cat ~/spinifex/logs/vpcd.log
```
## Instances cannot reach each other
This typically means Geneve tunnels are not established between hosts. Verify tunnel configuration:
```bash
sudo ovs-vsctl show | grep -i geneve
```
Ensure UDP port 6081 is open between all hosts:
```bash
sudo ss -ulnp | grep 6081
```
From inside a VM, check that the private IP was assigned via DHCP:
```bash
ip addr show
```
+85 -78
View File
@@ -1,129 +1,136 @@
module github.com/mulgadc/hive
module github.com/mulgadc/spinifex
go 1.25.5
go 1.26.1
require (
github.com/aws/aws-sdk-go v1.55.8
github.com/gofiber/fiber/v2 v2.52.9
github.com/go-chi/chi/v5 v5.2.5
github.com/google/uuid v1.6.0
github.com/kdomanski/iso9660 v0.4.0
github.com/mulgadc/predastore v1.0.0
github.com/mulgadc/viperblock v1.0.2
github.com/nats-io/nats-server/v2 v2.11.6
github.com/nats-io/nats.go v1.43.0
github.com/klauspost/cpuid/v2 v2.3.0
github.com/mulgadc/predastore v1.1.0
github.com/mulgadc/viperblock v1.1.0
github.com/nats-io/nats-server/v2 v2.12.4
github.com/nats-io/nats.go v1.49.0
github.com/ovn-kubernetes/libovsdb v0.8.1
github.com/pelletier/go-toml/v2 v2.2.4
github.com/pterm/pterm v0.12.82
github.com/spf13/cobra v1.9.1
github.com/spf13/viper v1.20.1
github.com/pterm/pterm v0.12.83
github.com/spf13/cobra v1.10.2
github.com/spf13/viper v1.21.0
github.com/stretchr/testify v1.11.1
go.uber.org/automaxprocs v1.6.0
golang.org/x/net v0.47.0
gopkg.in/ini.v1 v1.67.0
gopkg.in/ini.v1 v1.67.1
)
require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
cloud.google.com/go v0.121.2 // indirect
cloud.google.com/go/auth v0.16.5 // indirect
cloud.google.com/go/compute/metadata v0.8.0 // indirect
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/anthropics/anthropic-sdk-go v1.19.0 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.2 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
github.com/BurntSushi/toml v1.6.0 // indirect
github.com/anthropics/anthropic-sdk-go v1.26.0 // indirect
github.com/antithesishq/antithesis-sdk-go v0.6.0 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/boltdb/bolt v1.3.1 // indirect
github.com/buraksezer/consistent v0.10.0 // indirect
github.com/ccojocar/zxcvbn-go v1.0.4 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cenkalti/hub v1.0.2 // indirect
github.com/cenkalti/rpc2 v1.0.5 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
github.com/containerd/console v1.0.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dgraph-io/badger/v4 v4.8.0 // indirect
github.com/dgraph-io/ristretto/v2 v2.2.0 // indirect
github.com/dgraph-io/badger/v4 v4.9.1 // indirect
github.com/dgraph-io/ristretto/v2 v2.4.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fatih/color v1.13.0 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/go-chi/chi/v5 v5.2.4 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/google/flatbuffers v25.2.10+incompatible // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.30.1 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/google/flatbuffers v25.12.19+incompatible // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/go-tpm v0.9.5 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.13 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gookit/color v1.6.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/go-hclog v1.6.2 // indirect
github.com/hashicorp/go-immutable-radix v1.0.0 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-metrics v0.5.4 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.2 // indirect
github.com/hashicorp/golang-lru v0.5.0 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.5 // indirect
github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hashicorp/raft v1.7.3 // indirect
github.com/hashicorp/raft-boltdb/v2 v2.3.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/reedsolomon v1.12.6 // indirect
github.com/klauspost/compress v1.18.4 // indirect
github.com/klauspost/reedsolomon v1.13.2 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/lithammer/fuzzysearch v1.1.8 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mattn/go-runewidth v0.0.20 // indirect
github.com/minio/crc64nvme v1.1.1 // indirect
github.com/minio/highwayhash v1.0.3 // indirect
github.com/nats-io/jwt/v2 v2.7.4 // indirect
github.com/nats-io/nkeys v0.4.11 // indirect
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nats-io/jwt/v2 v2.8.0 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/openai/openai-go/v3 v3.8.1 // indirect
github.com/openai/openai-go/v3 v3.24.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/quic-go/quic-go v0.57.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/sagikazarmark/locafero v0.7.0 // indirect
github.com/securego/gosec/v2 v2.22.11 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.12.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/prometheus/client_golang v1.23.2 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.67.5 // indirect
github.com/prometheus/procfs v0.20.1 // indirect
github.com/quic-go/quic-go v0.59.0 // indirect
github.com/sagikazarmark/locafero v0.12.0 // indirect
github.com/securego/gosec/v2 v2.24.7 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.57.0 // indirect
github.com/valyala/tcplisten v1.0.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.etcd.io/bbolt v1.3.5 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect
go.opentelemetry.io/otel v1.37.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.9.0 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0 // indirect
go.opentelemetry.io/otel v1.41.0 // indirect
go.opentelemetry.io/otel/metric v1.41.0 // indirect
go.opentelemetry.io/otel/trace v1.41.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
golang.org/x/mod v0.30.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/telemetry v0.0.0-20251111182119-bc8e575c7b54 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/text v0.31.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.39.0 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa // indirect
golang.org/x/mod v0.33.0 // indirect
golang.org/x/net v0.51.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661 // indirect
golang.org/x/term v0.40.0 // indirect
golang.org/x/text v0.34.0 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.42.0 // indirect
golang.org/x/vuln v1.1.4 // indirect
google.golang.org/genai v1.37.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c // indirect
google.golang.org/grpc v1.75.0 // indirect
google.golang.org/protobuf v1.36.8 // indirect
google.golang.org/genai v1.49.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 // indirect
google.golang.org/grpc v1.79.3 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.6.1 // indirect
honnef.co/go/tools v0.7.0 // indirect
)
tool (
+187 -173
View File
@@ -7,14 +7,14 @@ atomicgo.dev/keyboard v0.2.9/go.mod h1:BC4w9g00XkxH/f1HXhW2sXmJFOCWbKn9xrOunSFtE
atomicgo.dev/schedule v0.1.0 h1:nTthAbhZS5YZmgYbb2+DH8uQIZcTlIrd4eYr3UQxEjs=
atomicgo.dev/schedule v0.1.0/go.mod h1:xeUa3oAkiuHYh8bKiQBRojqAMq3PXXbJujjb0hw8pEU=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.121.2 h1:v2qQpN6Dx9x2NmwrqlesOt3Ys4ol5/lFZ6Mg1B7OJCg=
cloud.google.com/go v0.121.2/go.mod h1:nRFlrHq39MNVWu+zESP2PosMWA0ryJw8KUBZ2iZpxbw=
cloud.google.com/go/auth v0.16.5 h1:mFWNQ2FEVWAliEQWpAdH80omXFokmrnbDhUS9cBywsI=
cloud.google.com/go/auth v0.16.5/go.mod h1:utzRfHMP+Vv0mpOkTRQoWD2q3BatTOoWbA7gCc2dUhQ=
cloud.google.com/go/compute/metadata v0.8.0 h1:HxMRIbao8w17ZX6wBnjhcDkW6lTFpgcaobyVfZWqRLA=
cloud.google.com/go/compute/metadata v0.8.0/go.mod h1:sYOGTp851OV9bOFJ9CH7elVvyzopvWQFNNghtDQ/Biw=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.2 h1:+Nbt5Ev0xEqxlNjd6c+yYUeosQ5TtEUaNcN/3FozlaM=
cloud.google.com/go/auth v0.18.2/go.mod h1:xD+oY7gcahcu7G2SG2DsBerfFxgPAJz17zz2joOFF3M=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
github.com/MarvinJWendt/testza v0.2.1/go.mod h1:God7bhG8n6uQxwdScay+gjm9/LnO4D3kkcZX4hv9Rp8=
@@ -27,19 +27,15 @@ github.com/MarvinJWendt/testza v0.5.2 h1:53KDo64C1z/h/d/stCYCPY69bt/OSwjq5KpFNwi
github.com/MarvinJWendt/testza v0.5.2/go.mod h1:xu53QFE5sCdjtMCKk8YMQ2MnymimEctc4n3EjyIYvEY=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/OneOfOne/xxhash v1.2.2 h1:KMrpdQIwFcEqXDklaen+P1axHaj9BSKzvpUUfnHldSE=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/anthropics/anthropic-sdk-go v1.19.0 h1:mO6E+ffSzLRvR/YUH9KJC0uGw0uV8GjISIuzem//3KE=
github.com/anthropics/anthropic-sdk-go v1.19.0/go.mod h1:WTz31rIUHUHqai2UslPpw5CwXrQP3geYBioRV4WOLvE=
github.com/antithesishq/antithesis-sdk-go v0.4.3-default-no-op h1:+OSa/t11TFhqfrX0EOSqQBDJ0YlpmK0rDSiB19dg9M0=
github.com/antithesishq/antithesis-sdk-go v0.4.3-default-no-op/go.mod h1:IUpT2DPAKh6i/YhSbt6Gl3v2yvUZjmKncl7U91fup7E=
github.com/anthropics/anthropic-sdk-go v1.26.0 h1:oUTzFaUpAevfuELAP1sjL6CQJ9HHAfT7CoSYSac11PY=
github.com/anthropics/anthropic-sdk-go v1.26.0/go.mod h1:qUKmaW+uuPB64iy1l+4kOSvaLqPXnHTTBKH6RVZ7q5Q=
github.com/antithesishq/antithesis-sdk-go v0.6.0 h1:v/YViLhFYkZOEEof4AXjD5AgGnGM84YHF4RqEwp6I2g=
github.com/antithesishq/antithesis-sdk-go v0.6.0/go.mod h1:IUpT2DPAKh6i/YhSbt6Gl3v2yvUZjmKncl7U91fup7E=
github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA=
github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
@@ -85,6 +81,7 @@ github.com/aws/smithy-go v1.23.2 h1:Crv0eatJUQhaManss33hS5r40CG3ZFH+21XSkqMrIUM=
github.com/aws/smithy-go v1.23.2/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
@@ -92,13 +89,19 @@ github.com/buraksezer/consistent v0.10.0 h1:hqBgz1PvNLC5rkWcEBVAL9dFMBWz6I0VgUCW
github.com/buraksezer/consistent v0.10.0/go.mod h1:6BrVajWq7wbKZlTOUPs/XVfR8c0maujuPowduSpZqmw=
github.com/ccojocar/zxcvbn-go v1.0.4 h1:FWnCIRMXPj43ukfX000kvBZvV6raSxakYr1nzyNrUcc=
github.com/ccojocar/zxcvbn-go v1.0.4/go.mod h1:3GxGX+rHmueTUMvm5ium7irpyjmm7ikxYFOSJB21Das=
github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cenkalti/hub v1.0.2 h1:Nqv9TNaA9boeO2wQFW8o87BY3zKthtnzXmWGmJqhAV8=
github.com/cenkalti/hub v1.0.2/go.mod h1:8LAFAZcCasb83vfxatMUnZHRoQcffho2ELpHb+kaTJU=
github.com/cenkalti/rpc2 v1.0.5 h1:T6l4SS3ja3eaJfRyZrn7Oco/PSx/pr3YK5cjCgLVLTk=
github.com/cenkalti/rpc2 v1.0.5/go.mod h1:2yfU5b86vOr16+iY1jN3MvT6Kxc9Nf8j5iZWwUf7iaw=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag=
github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I=
github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
github.com/containerd/console v1.0.3/go.mod h1:7LqA/THxQ86k76b8c/EMSiaJ3h1eZkMkXar0TQ1gf3U=
github.com/containerd/console v1.0.5 h1:R0ymNeydRqH2DmakFNdmjR2k0t7UPuiOV/N/27/qqsc=
github.com/containerd/console v1.0.5/go.mod h1:YynlIjWYF8myEu6sdkwKIvGQq+cOckRm6So2avqoYAk=
@@ -106,24 +109,29 @@ github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6N
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgraph-io/badger/v4 v4.8.0 h1:JYph1ChBijCw8SLeybvPINizbDKWZ5n/GYbz2yhN/bs=
github.com/dgraph-io/badger/v4 v4.8.0/go.mod h1:U6on6e8k/RTbUWxqKR0MvugJuVmkxSNc79ap4917h4w=
github.com/dgraph-io/ristretto/v2 v2.2.0 h1:bkY3XzJcXoMuELV8F+vS8kzNgicwQFAaGINAEJdWGOM=
github.com/dgraph-io/ristretto/v2 v2.2.0/go.mod h1:RZrm63UmcBAaYWC1DotLYBmTvgkrs0+XhBd7Npn7/zI=
github.com/dgraph-io/badger/v4 v4.9.1 h1:DocZXZkg5JJHJPtUErA0ibyHxOVUDVoXLSCV6t8NC8w=
github.com/dgraph-io/badger/v4 v4.9.1/go.mod h1:5/MEx97uzdPUHR4KtkNt8asfI2T4JiEiQlV7kWUo8c0=
github.com/dgraph-io/ristretto/v2 v2.4.0 h1:I/w09yLjhdcVD2QV192UJcq8dPBaAJb9pOuMyNy0XlU=
github.com/dgraph-io/ristretto/v2 v2.4.0/go.mod h1:0KsrXtXvnv0EqnzyowllbVJB8yBonswa2lTCK2gGo9E=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da h1:aIftn67I1fkbMa512G+w+Pxci9hJPB8oMnkcP3iZF38=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w=
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/go-chi/chi/v5 v5.2.4 h1:WtFKPHwlywe8Srng8j2BhOD9312j9cGUxG1SP4V2cR4=
github.com/go-chi/chi/v5 v5.2.4/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
@@ -135,13 +143,19 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gofiber/fiber/v2 v2.52.9 h1:YjKl5DOiyP3j0mO61u3NTmK7or8GzzWzCFzkboyP5cw=
github.com/gofiber/fiber/v2 v2.52.9/go.mod h1:YEcBbO/FB+5M1IZNBP9FO3J9281zgPAreiI1oqg8nDw=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -155,8 +169,8 @@ github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/flatbuffers v25.2.10+incompatible h1:F3vclr7C3HpB1k9mxCGRMXq6FdUalZ6H/pNX4FP1v0Q=
github.com/google/flatbuffers v25.2.10+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs=
github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/go-cmdtest v0.4.1-0.20220921163831-55ab3332a786 h1:rcv+Ippz6RAtvaGgKxc+8FQIpxHgsF+HBzPyYL2cyVU=
github.com/google/go-cmdtest v0.4.1-0.20220921163831-55ab3332a786/go.mod h1:apVn/GCasLZUVpAJ6oWAuyP7Ne7CEsQbTnc0plM3m+o=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -166,21 +180,21 @@ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-tpm v0.9.5 h1:ocUmnDebX54dnW+MQWGQRbdaAcJELsa6PqZhJ48KwVU=
github.com/google/go-tpm v0.9.5/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6 h1:EEHtgt9IwisQ2AZ4pIsMjahcegHh6rmhqxzIRQIyepY=
github.com/google/pprof v0.0.0-20250820193118-f64d9cf942d6/go.mod h1:I6V7YzU0XDpsHqbsyrghnFZLO1gwK6NPTNvmetQIk9U=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/vP9vJGqPwcdqsWjOt+V8J7+bTc=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.6 h1:GW/XbdyBFQ8Qe+YAmFU9uHLo7OnF5tL52HFAgMmyrf4=
github.com/googleapis/enterprise-certificate-proxy v0.3.6/go.mod h1:MkHOF77EYAE7qfSuSS9PU6g4Nt4e11cnsDUowfwewLA=
github.com/googleapis/gax-go/v2 v2.15.0 h1:SyjDc1mGgZU5LncH8gimWo9lW1DtIfPibOG81vgd/bo=
github.com/googleapis/gax-go/v2 v2.15.0/go.mod h1:zVVkkxAQHa1RQpg9z2AUCMnKhi0Qld9rcmyfL1OZhoc=
github.com/googleapis/enterprise-certificate-proxy v0.3.13 h1:hSPAhW3NX+7HNlTsmrvU0jL75cIzxFktheceg95Nq14=
github.com/googleapis/enterprise-certificate-proxy v0.3.13/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
@@ -190,21 +204,23 @@ github.com/gookit/color v1.6.0/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmty
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-hclog v1.6.2 h1:NOtoftovWkDheyUM/8JW3QMiXyxJK3uHRK7wV04nD2I=
github.com/hashicorp/go-hclog v1.6.2/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
github.com/hashicorp/go-immutable-radix v1.0.0 h1:AKDB1HM5PWEA7i4nhcpwOrO2byshxBjXVn/J/3+z5/0=
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-immutable-radix v1.3.1 h1:DKHmCUm2hRBK510BaiZlwvpD40f8bJFeZnpfm2KLowc=
github.com/hashicorp/go-immutable-radix v1.3.1/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-metrics v0.5.4 h1:8mmPiIJkTPPEbAiV97IxdAGNdRdaWwVap1BU6elejKY=
github.com/hashicorp/go-metrics v0.5.4/go.mod h1:CG5yz4NZ/AI/aQt9Ucm/vdBnbh7fvmv4lxZ350i+QQI=
github.com/hashicorp/go-msgpack v0.5.5 h1:i9R9JSrqIz0QVLz3sz+i3YJdT7TTSLcfLLzJi9aZTuI=
github.com/hashicorp/go-msgpack v0.5.5/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-msgpack/v2 v2.1.2 h1:4Ee8FTp834e+ewB71RDrQ0VKpyFdrKOjvYtnQ/ltVj0=
github.com/hashicorp/go-msgpack/v2 v2.1.2/go.mod h1:upybraOAblm4S7rx0+jeNy+CWWhzywQsSRV5033mMu4=
github.com/hashicorp/go-msgpack/v2 v2.1.5 h1:Ue879bPnutj/hXfmUk6s/jtIK90XxgiUIcXRl656T44=
github.com/hashicorp/go-msgpack/v2 v2.1.5/go.mod h1:bjCsRXpZ7NsJdk45PoCQnzRGDaK8TKm5ZnDI/9y3J4M=
github.com/hashicorp/go-retryablehttp v0.5.3/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs=
github.com/hashicorp/go-uuid v1.0.0 h1:RS8zrF7PhGwyNPOtxSClXXj9HA8feRnJzgnI1RJCSnM=
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=
github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/raft v1.7.3 h1:DxpEqZJysHN0wK+fviai5mFcSYsCkNpFUl1xpAW8Rbo=
@@ -228,15 +244,15 @@ github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7V
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kdomanski/iso9660 v0.4.0 h1:BPKKdcINz3m0MdjIMwS0wx1nofsOjxOq8TOr45WGHFg=
github.com/kdomanski/iso9660 v0.4.0/go.mod h1:OxUSupHsO9ceI8lBLPJKWBTphLemjrCQY8LPXM7qSzU=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/reedsolomon v1.12.6 h1:8pqE9aECQG/ZFitiUD1xK/E83zwosBAZtE3UbuZM8TQ=
github.com/klauspost/reedsolomon v1.12.6/go.mod h1:ggJT9lc71Vu+cSOPBlxGvBN6TfAS77qB4fp8vJ05NSA=
github.com/klauspost/reedsolomon v1.13.2 h1:9qtQy2tKEVpVB8Pfq87ZljHZb60/LbeTQ1OxV8EGzdE=
github.com/klauspost/reedsolomon v1.13.2/go.mod h1:ggJT9lc71Vu+cSOPBlxGvBN6TfAS77qB4fp8vJ05NSA=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -247,53 +263,56 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4=
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.20 h1:WcT52H91ZUAwy8+HUkdM3THM6gXqXuLJi9O3rjcQQaQ=
github.com/mattn/go-runewidth v0.0.20/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/minio/crc64nvme v1.1.1 h1:8dwx/Pz49suywbO+auHCBpCtlW1OfpcLN7wYgVR6wAI=
github.com/minio/crc64nvme v1.1.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
github.com/minio/highwayhash v1.0.3 h1:kbnuUMoHYyVl7szWjSxJnxw11k2U709jqFPPmIUyD6Q=
github.com/minio/highwayhash v1.0.3/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76 h1:KGuD/pM2JpL9FAYvBrnBBeENKZNh6eNtjqytV6TYjnk=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mulgadc/predastore v1.0.0 h1:IZ5V/EL+k+zz0zGoTYp+1BgODb43JxHsb/kvORuydKI=
github.com/mulgadc/predastore v1.0.0/go.mod h1:x46A1z091hlZkMt3kkcuLSy2PAzSgROr6ft04qcYOU4=
github.com/mulgadc/viperblock v0.1.5 h1:vS7ImGApQvxKuU1+fJNmwOycduwr9fx/xyofxWYKPcI=
github.com/mulgadc/viperblock v0.1.5/go.mod h1:JNooDHb6ZuPvNPWaf0gChlft/D8xs5jHJFln+P3B9lQ=
github.com/mulgadc/viperblock v1.0.2 h1:l5tQ96EO7rUKbaGtJHY4TJLG/25GSxYsxv/NsSN8+ZA=
github.com/mulgadc/viperblock v1.0.2/go.mod h1:SljqW90JjI6RtBlm4Pji59A7/VQXHy8ncSQ0Ht6xKko=
github.com/mulgadc/predastore v1.1.0 h1:DJ/rUWHmb8/cmqX/Cy1ttsX9X077S5PC3vsf6FW0WXU=
github.com/mulgadc/predastore v1.1.0/go.mod h1:4AcNew+IrsSPgvXO3ALBzfsMFOJHGVdS9OUCFU8cyZ8=
github.com/mulgadc/viperblock v1.1.0 h1:afh4Ygah/pmfL7drITzEHfpNp5Gra8kzWqxaOu4QkUM=
github.com/mulgadc/viperblock v1.1.0/go.mod h1:w9V0DjUH5hMXbXsiLbDr9lpqzXBGhEiYB1BeV6TYBkw=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nats-io/jwt/v2 v2.7.4 h1:jXFuDDxs/GQjGDZGhNgH4tXzSUK6WQi2rsj4xmsNOtI=
github.com/nats-io/jwt/v2 v2.7.4/go.mod h1:me11pOkwObtcBNR8AiMrUbtVOUGkqYjMQZ6jnSdVUIA=
github.com/nats-io/nats-server/v2 v2.11.6 h1:4VXRjbTUFKEB+7UoaKL3F5Y83xC7MxPoIONOnGgpkHw=
github.com/nats-io/nats-server/v2 v2.11.6/go.mod h1:2xoztlcb4lDL5Blh1/BiukkKELXvKQ5Vy29FPVRBUYs=
github.com/nats-io/nats.go v1.43.0 h1:uRFZ2FEoRvP64+UUhaTokyS18XBCR/xM2vQZKO4i8ug=
github.com/nats-io/nats.go v1.43.0/go.mod h1:iRWIPokVIFbVijxuMQq4y9ttaBTMe0SFdlZfMDd+33g=
github.com/nats-io/nkeys v0.4.11 h1:q44qGV008kYd9W1b1nEBkNzvnWxtRSQ7A8BoqRrcfa0=
github.com/nats-io/nkeys v0.4.11/go.mod h1:szDimtgmfOi9n25JpfIdGw12tZFYXqhGxjhVxsatHVE=
github.com/nats-io/jwt/v2 v2.8.0 h1:K7uzyz50+yGZDO5o772eRE7atlcSEENpL7P+b74JV1g=
github.com/nats-io/jwt/v2 v2.8.0/go.mod h1:me11pOkwObtcBNR8AiMrUbtVOUGkqYjMQZ6jnSdVUIA=
github.com/nats-io/nats-server/v2 v2.12.4 h1:ZnT10v2LU2Xcoiy8ek9X6Se4YG8EuMfIfvAEuFVx1Ts=
github.com/nats-io/nats-server/v2 v2.12.4/go.mod h1:5MCp/pqm5SEfsvVZ31ll1088ZTwEUdvRX1Hmh/mTTDg=
github.com/nats-io/nats.go v1.49.0 h1:yh/WvY59gXqYpgl33ZI+XoVPKyut/IcEaqtsiuTJpoE=
github.com/nats-io/nats.go v1.49.0/go.mod h1:fDCn3mN5cY8HooHwE2ukiLb4p4G4ImmzvXyJt+tGwdw=
github.com/nats-io/nkeys v0.4.15 h1:JACV5jRVO9V856KOapQ7x+EY8Jo3qw1vJt/9Jpwzkk4=
github.com/nats-io/nkeys v0.4.15/go.mod h1:CpMchTXC9fxA5zrMo4KpySxNjiDVvr8ANOSZdiNfUrs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/onsi/ginkgo/v2 v2.27.2 h1:LzwLj0b89qtIy6SSASkzlNvX6WktqurSHwkk2ipF/Ns=
github.com/onsi/ginkgo/v2 v2.27.2/go.mod h1:ArE1D/XhNXBXCBkKOLkbsb2c81dQHCRcF5zwn/ykDRo=
github.com/onsi/gomega v1.38.2 h1:eZCjf2xjZAqe+LeWvKb5weQ+NcPwX84kqJ0cZNxok2A=
github.com/onsi/gomega v1.38.2/go.mod h1:W2MJcYxRGV63b418Ai34Ud0hEdTVXq9NW9+Sx6uXf3k=
github.com/openai/openai-go/v3 v3.8.1 h1:b+YWsmwqXnbpSHWQEntZAkKciBZ5CJXwL68j+l59UDg=
github.com/openai/openai-go/v3 v3.8.1/go.mod h1:UOpNxkqC9OdNXNUfpNByKOtB4jAL0EssQXq5p8gO0Xs=
github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI=
github.com/onsi/ginkgo/v2 v2.28.1/go.mod h1:CLtbVInNckU3/+gC8LzkGUb9oF+e8W8TdUsxPwvdOgE=
github.com/onsi/gomega v1.39.1 h1:1IJLAad4zjPn2PsnhH70V4DKRFlrCzGBNrNaru+Vf28=
github.com/onsi/gomega v1.39.1/go.mod h1:hL6yVALoTOxeWudERyfppUcZXjMwIMLnuSfruD2lcfg=
github.com/openai/openai-go/v3 v3.24.0 h1:08x6GnYiB+AAejTo6yzPY8RkZMJQ8NpreiOyM5QfyYU=
github.com/openai/openai-go/v3 v3.24.0/go.mod h1:cdufnVK14cWcT9qA1rRtrXx4FTRsgbDPW7Ia7SS5cZo=
github.com/ovn-kubernetes/libovsdb v0.8.1 h1:M2J8bcJt5mXCom0HqzfEtuHkT80CTSQRcYG7acT8gf4=
github.com/ovn-kubernetes/libovsdb v0.8.1/go.mod h1:ZlnHLzagmLOSvyd9qfxBIZp6wOSOw0IsRsc+6lNUGbU=
github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY=
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
@@ -303,25 +322,31 @@ github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINE
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g=
github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.11.1/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=
github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4=
github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=
github.com/prometheus/common v0.67.5 h1:pIgK94WWlQt1WLwAC5j2ynLaBRDiinoAb86HZHTUGI4=
github.com/prometheus/common v0.67.5/go.mod h1:SjE/0MzDEEAyrdr5Gqc6G+sXI67maCxzaT3A2+HqjUw=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/procfs v0.20.1 h1:XwbrGOIplXW/AU3YhIhLODXMJYyC1isLFfYCsTEycfc=
github.com/prometheus/procfs v0.20.1/go.mod h1:o9EMBZGRyvDrSPH1RqdxhojkuXstoe4UlK79eF5TGGo=
github.com/pterm/pterm v0.12.27/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.29/go.mod h1:WI3qxgvoQFFGKGjGnJR849gU0TsEOvKn5Q8LlY1U7lg=
github.com/pterm/pterm v0.12.30/go.mod h1:MOqLIyMOgmTDz9yorcYbcw+HsgoZo3BQfg2wtl3HEFE=
@@ -329,43 +354,40 @@ github.com/pterm/pterm v0.12.31/go.mod h1:32ZAWZVXD7ZfG0s8qqHXePte42kdz8ECtRyEej
github.com/pterm/pterm v0.12.33/go.mod h1:x+h2uL+n7CP/rel9+bImHD5lF3nM9vJj80k9ybiiTTE=
github.com/pterm/pterm v0.12.36/go.mod h1:NjiL09hFhT/vWjQHSj1athJpx6H8cjpHXNAK5bUw8T8=
github.com/pterm/pterm v0.12.40/go.mod h1:ffwPLwlbXxP+rxT0GsgDTzS3y3rmpAO1NMjUkGTYf8s=
github.com/pterm/pterm v0.12.82 h1:+D9wYhCaeaK0FIQoZtqbNQuNpe2lB2tajKKsTd5paVQ=
github.com/pterm/pterm v0.12.82/go.mod h1:TyuyrPjnxfwP+ccJdBTeWHtd/e0ybQHkOS/TakajZCw=
github.com/quic-go/quic-go v0.57.1 h1:25KAAR9QR8KZrCZRThWMKVAwGoiHIrNbT72ULHTuI10=
github.com/quic-go/quic-go v0.57.1/go.mod h1:ly4QBAjHA2VhdnxhojRsCUOeJwKYg+taDlos92xb1+s=
github.com/pterm/pterm v0.12.83 h1:ie+YmGmA727VuhxBlyGr74Ks+7McV6kT99IB8EU80aA=
github.com/pterm/pterm v0.12.83/go.mod h1:xlgc6bFWyJIMtmLJvGim+L7jhSReilOlOnodeIYe4Tk=
github.com/quic-go/quic-go v0.59.0 h1:OLJkp1Mlm/aS7dpKgTc6cnpynnD2Xg7C1pwL6vy/SAw=
github.com/quic-go/quic-go v0.59.0/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.7.0 h1:5MqpDsTGNDhY8sGp0Aowyf0qKsPrhewaLSsFaodPcyo=
github.com/sagikazarmark/locafero v0.7.0/go.mod h1:2za3Cg5rMaTMoG/2Ulr9AwtFaIppKXTRYnozin4aB5k=
github.com/sagikazarmark/locafero v0.12.0 h1:/NQhBAkUb4+fH1jivKHWusDYFjMOOKU88eegjfxfHb4=
github.com/sagikazarmark/locafero v0.12.0/go.mod h1:sZh36u/YSZ918v0Io+U9ogLYQJ9tLLBmM4eneO6WwsI=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/securego/gosec/v2 v2.22.11 h1:tW+weM/hCM/GX3iaCV91d5I6hqaRT2TPsFM1+USPXwg=
github.com/securego/gosec/v2 v2.22.11/go.mod h1:KE4MW/eH0GLWztkbt4/7XpyH0zJBBnu7sYB4l6Wn7Mw=
github.com/securego/gosec/v2 v2.24.7 h1:3k5yJnrhT1TTdsG0ZsnenlfCcT+7Y/+zeCPHbL7QAn8=
github.com/securego/gosec/v2 v2.24.7/go.mod h1:AdDJbjcG/XxFgVv7pW19vMNYlFM6+Q6Qy3t6lWAUcEY=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72 h1:qLC7fQah7D6K1B0ujays3HV9gkFtllcxhzImRR7ArPQ=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.12.0 h1:UcOPyRBYczmFn6yvphxkn9ZEOY65cpwGKb5mL36mrqs=
github.com/spf13/afero v1.12.0/go.mod h1:ZTlWwG4/ahT8W7T0WQ5uYmjI9duaLQGy3Q2OAl4sk/4=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v1.9.1 h1:CXSaggrXdbHK9CF+8ywj8Amf7PBRmPCOJugH954Nnlo=
github.com/spf13/cobra v1.9.1/go.mod h1:nDyEzZ8ogv936Cinf6g1RU9MRY64Ir93oCnqb9wxYW0=
github.com/spf13/pflag v1.0.6 h1:jFzHGLGAlb3ruxLB8MhbI6A8+AQX/2eW4qeyNZXNp2o=
github.com/spf13/pflag v1.0.6/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.20.1 h1:ZMi+z/lvLyPSCoNtFCpqjy0S4kPbirhpTMwl8BkW9X4=
github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqjJvu4=
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
@@ -373,7 +395,10 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
@@ -381,66 +406,57 @@ github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSW
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=
github.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.57.0 h1:Xw8SjWGEP/+wAAgyy5XTvgrWlOD1+TxbbvNADYCm1Tg=
github.com/valyala/fasthttp v1.57.0/go.mod h1:h6ZBaPRlzpZ6O3H5t2gEk1Qi33+TmLvfwgLLp0t9CpE=
github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8=
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0=
go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 h1:F7Jx+6hwnZ41NSFTO5q4LYDtJRXBf2PD0rNBkeB/lus=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0/go.mod h1:UHB22Z8QsdRDrnAtX4PntOl36ajSxcdUMt1sF7Y6E7Q=
go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
go.opentelemetry.io/otel/metric v1.37.0 h1:mvwbQS5m0tbmqML4NqK+e3aDiO02vsf/WgbsdpcPoZE=
go.opentelemetry.io/otel/metric v1.37.0/go.mod h1:04wGrZurHYKOc+RKeye86GwKiTb9FKm1WHtO+4EVr2E=
go.opentelemetry.io/otel/sdk v1.37.0 h1:ItB0QUqnjesGRvNcmAcU0LyvkVyGJ2xftD29bWdDvKI=
go.opentelemetry.io/otel/sdk v1.37.0/go.mod h1:VredYzxUvuo2q3WRcDnKDjbdvmO0sCzOvVAiY+yUkAg=
go.opentelemetry.io/otel/sdk/metric v1.37.0 h1:90lI228XrB9jCMuSdA0673aubgRobVZFhbjxHHspCPc=
go.opentelemetry.io/otel/sdk/metric v1.37.0/go.mod h1:cNen4ZWfiD37l5NhS+Keb5RXVWZWpRE+9WyVCpbo5ps=
go.opentelemetry.io/otel/trace v1.37.0 h1:HLdcFNbRQBE2imdSEgm/kwqmQj1Or1l/7bW6mxVK7z4=
go.opentelemetry.io/otel/trace v1.37.0/go.mod h1:TlgrlQ+PtQO5XFerSPUYG0JSgGyryXewPGyayAWSBS0=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0 h1:PnV4kVnw0zOmwwFkAzCN5O07fw1YOIQor120zrh0AVo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0/go.mod h1:ofAwF4uinaf8SXdVzzbL4OsxJ3VfeEg3f/F6CeF49/Y=
go.opentelemetry.io/otel v1.41.0 h1:YlEwVsGAlCvczDILpUXpIpPSL/VPugt7zHThEMLce1c=
go.opentelemetry.io/otel v1.41.0/go.mod h1:Yt4UwgEKeT05QbLwbyHXEwhnjxNO6D8L5PQP51/46dE=
go.opentelemetry.io/otel/metric v1.41.0 h1:rFnDcs4gRzBcsO9tS8LCpgR0dxg4aaxWlJxCno7JlTQ=
go.opentelemetry.io/otel/metric v1.41.0/go.mod h1:xPvCwd9pU0VN8tPZYzDZV/BMj9CM9vs00GuBjeKhJps=
go.opentelemetry.io/otel/sdk v1.41.0 h1:YPIEXKmiAwkGl3Gu1huk1aYWwtpRLeskpV+wPisxBp8=
go.opentelemetry.io/otel/sdk v1.41.0/go.mod h1:ahFdU0G5y8IxglBf0QBJXgSe7agzjE4GiTJ6HT9ud90=
go.opentelemetry.io/otel/sdk/metric v1.41.0 h1:siZQIYBAUd1rlIWQT2uCxWJxcCO7q3TriaMlf08rXw8=
go.opentelemetry.io/otel/sdk/metric v1.41.0/go.mod h1:HNBuSvT7ROaGtGI50ArdRLUnvRTRGniSUZbxiWxSO8Y=
go.opentelemetry.io/otel/trace v1.41.0 h1:Vbk2co6bhj8L59ZJ6/xFTskY+tGAbOnCtQGVVa9TIN0=
go.opentelemetry.io/otel/trace v1.41.0/go.mod h1:U1NU4ULCoxeDKc09yCWdWe+3QoyweJcISEVa1RBzOis=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
go.uber.org/multierr v1.9.0 h1:7fIwc/ZtS0q++VgcfqFDxSBZVv/Xo49/SYnDFupUwlI=
go.uber.org/multierr v1.9.0/go.mod h1:X2jQV1h+kxSjClGpnseKVIxpmcjrj7MNnI0bnlfKTVQ=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.45.0 h1:jMBrvKuj23MTlT0bQEOBcAE0mjg8mK9RXFhRH6nyF3Q=
golang.org/x/crypto v0.45.0/go.mod h1:XTGrrkGJve7CYK7J8PEww4aY7gM3qMCElcJQ8n8JdX4=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 h1:1P7xPZEwZMoBoz0Yze5Nx2/4pxj6nw9ZqHWXqP0iRgQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa h1:6Wi43P0isP8Nl8D4qJmA3VC4FswVH0RJkr5cauo67SQ=
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:PqrXSW65cXDZH0k4IeUbhmg/bcAZDbzNz3byBpKCsXo=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=
golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -451,8 +467,8 @@ golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -461,8 +477,8 @@ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -471,7 +487,6 @@ golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -488,38 +503,37 @@ golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/telemetry v0.0.0-20251111182119-bc8e575c7b54 h1:E2/AqCUMZGgd73TQkxUMcMla25GB9i/5HOdLr+uH7Vo=
golang.org/x/telemetry v0.0.0-20251111182119-bc8e575c7b54/go.mod h1:hKdjCMrbv9skySur+Nek8Hd0uJ0GuxJIoIX2payrIdQ=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661 h1:AjMsnhVxCTd9HssPcdXPhNTQ0YazDt5+52EqmcWQ0PA=
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661/go.mod h1:NuITXsA9cTiqnXtVk+/wrBT2Ja4X5hsfGOYRJ6kgYjs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.37.0 h1:8EGAD0qCmHYZg6J17DvsMy9/wJ7/D/4pV/wfnld5lTU=
golang.org/x/term v0.37.0/go.mod h1:5pB4lxRNYYVZuTLmy8oR2BH8dflOR+IbTYFD8fi3254=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=
golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=
golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k=
golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0=
golang.org/x/tools/go/expect v0.1.1-deprecated h1:jpBZDwmgPhXsKZC6WhL20P4b/wmnpsEAGHaNy0n/rJM=
golang.org/x/tools/go/expect v0.1.1-deprecated/go.mod h1:eihoPOH+FgIqa3FpoTwguz/bVUSGBlGQU67vpBeOrBY=
golang.org/x/tools/go/packages/packagestest v0.1.1-deprecated h1:1h2MnaIAIXISqTFKdENegdpAgUXz6NrPEsbIeWaBRvM=
@@ -531,12 +545,12 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genai v1.37.0 h1:dgp71k1wQ+/+APdZrN3LFgAGnVnr5IdTF1Oj0Dg+BQc=
google.golang.org/genai v1.37.0/go.mod h1:A3kkl0nyBjyFlNjgxIwKq70julKbIxpSxqKO5gw/gmk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c h1:qXWI/sQtv5UKboZ/zUk7h+mrf/lXORyI+n9DKDAusdg=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250818200422-3122310a409c/go.mod h1:gw1tLEfykwDz2ET4a12jcXt4couGAm7IwsVaTy0Sflo=
google.golang.org/grpc v1.75.0 h1:+TW+dqTd2Biwe6KKfhE5JpiYIBWq865PhKGSXiivqt4=
google.golang.org/grpc v1.75.0/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
google.golang.org/genai v1.49.0 h1:Se+QJaH2GYK1aaR1o5S38mlU2GD5FnVvP76nfkV7LH0=
google.golang.org/genai v1.49.0/go.mod h1:A3kkl0nyBjyFlNjgxIwKq70julKbIxpSxqKO5gw/gmk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 h1:ggcbiqK8WWh6l1dnltU4BgWGIGo+EVYxCaAPih/zQXQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -544,15 +558,15 @@ google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miE
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.67.1 h1:tVBILHy0R6e4wkYOn3XmiITt/hEVH4TFMYvAX2Ytz6k=
gopkg.in/ini.v1 v1.67.1/go.mod h1:x/cyOwCgZqOkJoDIJ3c1KNHMo10+nLGAhh+kn3Zizss=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
@@ -565,5 +579,5 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.6.1 h1:R094WgE8K4JirYjBaOpz/AvTyUu/3wbmAoskKN/pxTI=
honnef.co/go/tools v0.6.1/go.mod h1:3puzxxljPCe8RGJX7BIy1plGbxEOZni5mR2aXe3/uk4=
honnef.co/go/tools v0.7.0 h1:w6WUp1VbkqPEgLz4rkBzH/CSU6HkoqNLp6GstyTx3lU=
honnef.co/go/tools v0.7.0/go.mod h1:pm29oPxeP3P82ISxZDgIYeOaf9ta6Pi0EWvCFoLG2vc=
@@ -1,90 +0,0 @@
# Quick Reference
Most formatting and common issues are automatically fixed by Biome. Run `pnpm fix` before committing to ensure compliance.
## Core Principles
Write code that is **accessible, performant, type-safe, and maintainable**. Focus on clarity and explicit intent over brevity.
### Type Safety & Explicitness
- Use explicit types for function parameters and return values when they enhance clarity
- Prefer `unknown` over `any` when the type is genuinely unknown
- Use const assertions (`as const`) for immutable values and literal types
- Leverage TypeScript's type narrowing instead of type assertions
- Use meaningful variable names instead of magic numbers - extract constants with descriptive names
### Modern JavaScript/TypeScript
- Use arrow functions for callbacks and short functions
- Prefer `for...of` loops over `.forEach()` and indexed `for` loops
- Use optional chaining (`?.`) and nullish coalescing (`??`) for safer property access
- Prefer template literals over string concatenation
- Use destructuring for object and array assignments
- Use `const` by default, `let` only when reassignment is needed, never `var`
- Use interfaces for objects, otherwise use type aliases
- Prefer using function declarations over arrow functions for named functions
### Async & Promises
- Always `await` promises in async functions - don't forget to use the return value
- Use `async/await` syntax instead of promise chains for better readability
- Handle errors appropriately in async code with try-catch blocks
- Don't use async functions as Promise executors
### React & JSX
- Use function components over class components
- Call hooks at the top level only, never conditionally
- Specify all dependencies in hook dependency arrays correctly
- Use the `key` prop for elements in iterables (prefer unique IDs over array indices)
- Nest children between opening and closing tags instead of passing as props
- Don't define components inside other components
- Use semantic HTML and ARIA attributes for accessibility:
- Provide meaningful alt text for images
- Use proper heading hierarchy
- Add labels for form inputs
- Include keyboard event handlers alongside mouse events
- Use semantic elements (`<button>`, `<nav>`, etc.) instead of divs with roles
- Don't use `useMemo` or `useCallback`
- Use ref as a prop instead of `React.forwardRef`
### Error Handling & Debugging
- Remove `console.log`, `debugger`, and `alert` statements from production code
- Throw `Error` objects with descriptive messages, not strings or other values
- Use `try-catch` blocks meaningfully - don't catch errors just to rethrow them
- Prefer early returns over nested conditionals for error cases
### Code Organization
- Keep functions focused and under reasonable cognitive complexity limits
- Extract complex conditions into well-named boolean variables
- Use early returns to reduce nesting
- Prefer simple conditionals over nested ternary operators
- Group related code together and separate concerns
### Security
- Add `rel="noopener"` when using `target="_blank"` on links
- Avoid `dangerouslySetInnerHTML` unless absolutely necessary
- Don't use `eval()` or assign directly to `document.cookie`
- Validate and sanitize user input
### Performance
- Avoid spread syntax in accumulators within loops
- Use top-level regex literals instead of creating them in loops
- Prefer specific imports over namespace imports
- Avoid barrel files (index files that re-export everything)
## When Biome Can't Help
Biome's linter will catch most issues automatically. Focus your attention on:
1. **Business logic correctness** - Biome can't validate your algorithms
2. **Meaningful naming** - Use descriptive names for functions, variables, and types
3. **Architecture decisions** - Component structure, data flow, and API design
4. **Edge cases** - Handle boundary conditions and error states
5. **User experience** - Accessibility, performance, and usability considerations
6. **Documentation** - Add comments for complex logic, but prefer self-documenting code
-15
View File
@@ -1,15 +0,0 @@
{
"hooks": {
"PostToolUse": [
{
"matcher": "Write|Edit",
"hooks": [
{
"type": "command",
"command": "pnpm fix"
}
]
}
]
}
}
@@ -1,99 +0,0 @@
---
name: react-best-practices
description: React performance optimization guidelines. This skill should be used when writing, reviewing, or refactoring React code to ensure optimal performance patterns. Triggers on tasks involving React components, data fetching, bundle optimization, or performance improvements.
---
# React Best Practices
Comprehensive performance optimization guide for React applications. Contains 36 rules across 6 categories, prioritized by impact to guide automated refactoring and code generation.
## When to Apply
Reference these guidelines when:
- Writing new React components
- Implementing data fetching
- Reviewing code for performance issues
- Refactoring existing React code
- Optimizing bundle size or load times
## Rule Categories by Priority
| Priority | Category | Impact | Prefix |
|----------|----------|--------|--------|
| 1 | Eliminating Waterfalls | CRITICAL | `async-` |
| 2 | Client-Side Data Fetching | MEDIUM-HIGH | `client-` |
| 3 | Re-render Optimization | MEDIUM | `rerender-` |
| 4 | Rendering Performance | MEDIUM | `rendering-` |
| 5 | JavaScript Performance | LOW-MEDIUM | `js-` |
| 6 | Advanced Patterns | LOW | `advanced-` |
## Quick Reference
### 1. Eliminating Waterfalls (CRITICAL)
- `async-defer-await` - Move await into branches where actually used
- `async-parallel` - Use Promise.all() for independent operations
- `async-dependencies` - Use better-all for partial dependencies
### 2. Client-Side Data Fetching (MEDIUM-HIGH)
- `client-passive-event-listeners` - Use passive listeners for scroll
- `client-localstorage-schema` - Version and minimize localStorage data
### 3. Re-render Optimization (MEDIUM)
- `rerender-defer-reads` - Don't subscribe to state only used in callbacks
- `rerender-dependencies` - Use primitive dependencies in effects
- `rerender-derived-state` - Subscribe to derived booleans, not raw values
- `rerender-derived-state-no-effect` - Derive state during render, not effects
- `rerender-functional-setstate` - Use functional setState for stable callbacks
- `rerender-lazy-state-init` - Pass function to useState for expensive values
- `rerender-move-effect-to-event` - Put interaction logic in event handlers
- `rerender-transitions` - Use startTransition for non-urgent updates
- `rerender-use-ref-transient-values` - Use refs for transient frequent values
### 4. Rendering Performance (MEDIUM)
- `rendering-animate-svg-wrapper` - Animate div wrapper, not SVG element
- `rendering-content-visibility` - Use content-visibility for long lists
- `rendering-hoist-jsx` - Extract static JSX outside components
- `rendering-svg-precision` - Reduce SVG coordinate precision
- `rendering-activity` - Use Activity component for show/hide
- `rendering-conditional-render` - Use ternary, not && for conditionals
- `rendering-usetransition-loading` - Prefer useTransition for loading state
### 5. JavaScript Performance (LOW-MEDIUM)
- `js-batch-dom-css` - Group CSS changes via classes or cssText
- `js-index-maps` - Build Map for repeated lookups
- `js-cache-property-access` - Cache object properties in loops
- `js-cache-function-results` - Cache function results in module-level Map
- `js-cache-storage` - Cache localStorage/sessionStorage reads
- `js-combine-iterations` - Combine multiple filter/map into one loop
- `js-length-check-first` - Check array length before expensive comparison
- `js-early-exit` - Return early from functions
- `js-hoist-regexp` - Hoist RegExp creation outside loops
- `js-min-max-loop` - Use loop for min/max instead of sort
- `js-set-map-lookups` - Use Set/Map for O(1) lookups
- `js-tosorted-immutable` - Use toSorted() for immutability
### 6. Advanced Patterns (LOW)
- `advanced-event-handler-refs` - Store event handlers in refs
- `advanced-init-once` - Initialize app once per app load
- `advanced-use-latest` - useLatest for stable callback refs
## How to Use
Read individual rule files for detailed explanations and code examples:
```
rules/async-parallel.md
rules/bundle-barrel-imports.md
```
Each rule file contains:
- Brief explanation of why it matters
- Incorrect code example with explanation
- Correct code example with explanation
- Additional context and references
@@ -1,55 +0,0 @@
---
title: Store Event Handlers in Refs
impact: LOW
impactDescription: stable subscriptions
tags: advanced, hooks, refs, event-handlers, optimization
---
## Store Event Handlers in Refs
Store callbacks in refs when used in effects that shouldn't re-subscribe on callback changes.
**Incorrect (re-subscribes on every render):**
```tsx
function useWindowEvent(event: string, handler: (e) => void) {
useEffect(() => {
window.addEventListener(event, handler)
return () => window.removeEventListener(event, handler)
}, [event, handler])
}
```
**Correct (stable subscription):**
```tsx
function useWindowEvent(event: string, handler: (e) => void) {
const handlerRef = useRef(handler)
useEffect(() => {
handlerRef.current = handler
}, [handler])
useEffect(() => {
const listener = (e) => handlerRef.current(e)
window.addEventListener(event, listener)
return () => window.removeEventListener(event, listener)
}, [event])
}
```
**Alternative: use `useEffectEvent` if you're on latest React:**
```tsx
import { useEffectEvent } from 'react'
function useWindowEvent(event: string, handler: (e) => void) {
const onEvent = useEffectEvent(handler)
useEffect(() => {
window.addEventListener(event, onEvent)
return () => window.removeEventListener(event, onEvent)
}, [event])
}
```
`useEffectEvent` provides a cleaner API for the same pattern: it creates a stable function reference that always calls the latest version of the handler.
@@ -1,42 +0,0 @@
---
title: Initialize App Once, Not Per Mount
impact: LOW-MEDIUM
impactDescription: avoids duplicate init in development
tags: initialization, useEffect, app-startup, side-effects
---
## Initialize App Once, Not Per Mount
Do not put app-wide initialization that must run once per app load inside `useEffect([])` of a component. Components can remount and effects will re-run. Use a module-level guard or top-level init in the entry module instead.
**Incorrect (runs twice in dev, re-runs on remount):**
```tsx
function Comp() {
useEffect(() => {
loadFromStorage()
checkAuthToken()
}, [])
// ...
}
```
**Correct (once per app load):**
```tsx
let didInit = false
function Comp() {
useEffect(() => {
if (didInit) return
didInit = true
loadFromStorage()
checkAuthToken()
}, [])
// ...
}
```
Reference: [Initializing the application](https://react.dev/learn/you-might-not-need-an-effect#initializing-the-application)
@@ -1,39 +0,0 @@
---
title: useEffectEvent for Stable Callback Refs
impact: LOW
impactDescription: prevents effect re-runs
tags: advanced, hooks, useEffectEvent, refs, optimization
---
## useEffectEvent for Stable Callback Refs
Access latest values in callbacks without adding them to dependency arrays. Prevents effect re-runs while avoiding stale closures.
**Incorrect (effect re-runs on every callback change):**
```tsx
function SearchInput({ onSearch }: { onSearch: (q: string) => void }) {
const [query, setQuery] = useState('')
useEffect(() => {
const timeout = setTimeout(() => onSearch(query), 300)
return () => clearTimeout(timeout)
}, [query, onSearch])
}
```
**Correct (using React's useEffectEvent):**
```tsx
import { useEffectEvent } from 'react';
function SearchInput({ onSearch }: { onSearch: (q: string) => void }) {
const [query, setQuery] = useState('')
const onSearchEvent = useEffectEvent(onSearch)
useEffect(() => {
const timeout = setTimeout(() => onSearchEvent(query), 300)
return () => clearTimeout(timeout)
}, [query])
}
```
@@ -1,80 +0,0 @@
---
title: Defer Await Until Needed
impact: HIGH
impactDescription: avoids blocking unused code paths
tags: async, await, conditional, optimization
---
## Defer Await Until Needed
Move `await` operations into the branches where they're actually used to avoid blocking code paths that don't need them.
**Incorrect (blocks both branches):**
```typescript
async function handleRequest(userId: string, skipProcessing: boolean) {
const userData = await fetchUserData(userId)
if (skipProcessing) {
// Returns immediately but still waited for userData
return { skipped: true }
}
// Only this branch uses userData
return processUserData(userData)
}
```
**Correct (only blocks when needed):**
```typescript
async function handleRequest(userId: string, skipProcessing: boolean) {
if (skipProcessing) {
// Returns immediately without waiting
return { skipped: true }
}
// Fetch only when needed
const userData = await fetchUserData(userId)
return processUserData(userData)
}
```
**Another example (early return optimization):**
```typescript
// Incorrect: always fetches permissions
async function updateResource(resourceId: string, userId: string) {
const permissions = await fetchPermissions(userId)
const resource = await getResource(resourceId)
if (!resource) {
return { error: 'Not found' }
}
if (!permissions.canEdit) {
return { error: 'Forbidden' }
}
return await updateResourceData(resource, permissions)
}
// Correct: fetches only when needed
async function updateResource(resourceId: string, userId: string) {
const resource = await getResource(resourceId)
if (!resource) {
return { error: 'Not found' }
}
const permissions = await fetchPermissions(userId)
if (!permissions.canEdit) {
return { error: 'Forbidden' }
}
return await updateResourceData(resource, permissions)
}
```
This optimization is especially valuable when the skipped branch is frequently taken, or when the deferred operation is expensive.
@@ -1,51 +0,0 @@
---
title: Dependency-Based Parallelization
impact: CRITICAL
impactDescription: 2-10× improvement
tags: async, parallelization, dependencies, better-all
---
## Dependency-Based Parallelization
For operations with partial dependencies, use `better-all` to maximize parallelism. It automatically starts each task at the earliest possible moment.
**Incorrect (profile waits for config unnecessarily):**
```typescript
const [user, config] = await Promise.all([
fetchUser(),
fetchConfig()
])
const profile = await fetchProfile(user.id)
```
**Correct (config and profile run in parallel):**
```typescript
import { all } from 'better-all'
const { user, config, profile } = await all({
async user() { return fetchUser() },
async config() { return fetchConfig() },
async profile() {
return fetchProfile((await this.$.user).id)
}
})
```
**Alternative without extra dependencies:**
We can also create all the promises first, and do `Promise.all()` at the end.
```typescript
const userPromise = fetchUser()
const profilePromise = userPromise.then(user => fetchProfile(user.id))
const [user, config, profile] = await Promise.all([
userPromise,
fetchConfig(),
profilePromise
])
```
Reference: [https://github.com/shuding/better-all](https://github.com/shuding/better-all)
@@ -1,28 +0,0 @@
---
title: Promise.all() for Independent Operations
impact: CRITICAL
impactDescription: 2-10× improvement
tags: async, parallelization, promises, waterfalls
---
## Promise.all() for Independent Operations
When async operations have no interdependencies, execute them concurrently using `Promise.all()`.
**Incorrect (sequential execution, 3 round trips):**
```typescript
const user = await fetchUser()
const posts = await fetchPosts()
const comments = await fetchComments()
```
**Correct (parallel execution, 1 round trip):**
```typescript
const [user, posts, comments] = await Promise.all([
fetchUser(),
fetchPosts(),
fetchComments()
])
```
@@ -1,71 +0,0 @@
---
title: Version and Minimize localStorage Data
impact: MEDIUM
impactDescription: prevents schema conflicts, reduces storage size
tags: client, localStorage, storage, versioning, data-minimization
---
## Version and Minimize localStorage Data
Add version prefix to keys and store only needed fields. Prevents schema conflicts and accidental storage of sensitive data.
**Incorrect:**
```typescript
// No version, stores everything, no error handling
localStorage.setItem('userConfig', JSON.stringify(fullUserObject))
const data = localStorage.getItem('userConfig')
```
**Correct:**
```typescript
const VERSION = 'v2'
function saveConfig(config: { theme: string; language: string }) {
try {
localStorage.setItem(`userConfig:${VERSION}`, JSON.stringify(config))
} catch {
// Throws in incognito/private browsing, quota exceeded, or disabled
}
}
function loadConfig() {
try {
const data = localStorage.getItem(`userConfig:${VERSION}`)
return data ? JSON.parse(data) : null
} catch {
return null
}
}
// Migration from v1 to v2
function migrate() {
try {
const v1 = localStorage.getItem('userConfig:v1')
if (v1) {
const old = JSON.parse(v1)
saveConfig({ theme: old.darkMode ? 'dark' : 'light', language: old.lang })
localStorage.removeItem('userConfig:v1')
}
} catch {}
}
```
**Store minimal fields from server responses:**
```typescript
// User object has 20+ fields, only store what UI needs
function cachePrefs(user: FullUser) {
try {
localStorage.setItem('prefs:v1', JSON.stringify({
theme: user.preferences.theme,
notifications: user.preferences.notifications
}))
} catch {}
}
```
**Always wrap in try-catch:** `getItem()` and `setItem()` throw in incognito/private browsing (Safari, Firefox), when quota exceeded, or when disabled.
**Benefits:** Schema evolution via versioning, reduced storage size, prevents storing tokens/PII/internal flags.
@@ -1,48 +0,0 @@
---
title: Use Passive Event Listeners for Scrolling Performance
impact: MEDIUM
impactDescription: eliminates scroll delay caused by event listeners
tags: client, event-listeners, scrolling, performance, touch, wheel
---
## Use Passive Event Listeners for Scrolling Performance
Add `{ passive: true }` to touch and wheel event listeners to enable immediate scrolling. Browsers normally wait for listeners to finish to check if `preventDefault()` is called, causing scroll delay.
**Incorrect:**
```typescript
useEffect(() => {
const handleTouch = (e: TouchEvent) => console.log(e.touches[0].clientX)
const handleWheel = (e: WheelEvent) => console.log(e.deltaY)
document.addEventListener('touchstart', handleTouch)
document.addEventListener('wheel', handleWheel)
return () => {
document.removeEventListener('touchstart', handleTouch)
document.removeEventListener('wheel', handleWheel)
}
}, [])
```
**Correct:**
```typescript
useEffect(() => {
const handleTouch = (e: TouchEvent) => console.log(e.touches[0].clientX)
const handleWheel = (e: WheelEvent) => console.log(e.deltaY)
document.addEventListener('touchstart', handleTouch, { passive: true })
document.addEventListener('wheel', handleWheel, { passive: true })
return () => {
document.removeEventListener('touchstart', handleTouch)
document.removeEventListener('wheel', handleWheel)
}
}, [])
```
**Use passive when:** tracking/analytics, logging, any listener that doesn't call `preventDefault()`.
**Don't use passive when:** implementing custom swipe gestures, custom zoom controls, or any listener that needs `preventDefault()`.
@@ -1,107 +0,0 @@
---
title: Avoid Layout Thrashing
impact: MEDIUM
impactDescription: prevents forced synchronous layouts and reduces performance bottlenecks
tags: javascript, dom, css, performance, reflow, layout-thrashing
---
## Avoid Layout Thrashing
Avoid interleaving style writes with layout reads. When you read a layout property (like `offsetWidth`, `getBoundingClientRect()`, or `getComputedStyle()`) between style changes, the browser is forced to trigger a synchronous reflow.
**This is OK (browser batches style changes):**
```typescript
function updateElementStyles(element: HTMLElement) {
// Each line invalidates style, but browser batches the recalculation
element.style.width = '100px'
element.style.height = '200px'
element.style.backgroundColor = 'blue'
element.style.border = '1px solid black'
}
```
**Incorrect (interleaved reads and writes force reflows):**
```typescript
function layoutThrashing(element: HTMLElement) {
element.style.width = '100px'
const width = element.offsetWidth // Forces reflow
element.style.height = '200px'
const height = element.offsetHeight // Forces another reflow
}
```
**Correct (batch writes, then read once):**
```typescript
function updateElementStyles(element: HTMLElement) {
// Batch all writes together
element.style.width = '100px'
element.style.height = '200px'
element.style.backgroundColor = 'blue'
element.style.border = '1px solid black'
// Read after all writes are done (single reflow)
const { width, height } = element.getBoundingClientRect()
}
```
**Correct (batch reads, then writes):**
```typescript
function avoidThrashing(element: HTMLElement) {
// Read phase - all layout queries first
const rect1 = element.getBoundingClientRect()
const offsetWidth = element.offsetWidth
const offsetHeight = element.offsetHeight
// Write phase - all style changes after
element.style.width = '100px'
element.style.height = '200px'
}
```
**Better: use CSS classes**
```css
.highlighted-box {
width: 100px;
height: 200px;
background-color: blue;
border: 1px solid black;
}
```
```typescript
function updateElementStyles(element: HTMLElement) {
element.classList.add('highlighted-box')
const { width, height } = element.getBoundingClientRect()
}
```
**React example:**
```tsx
// Incorrect: interleaving style changes with layout queries
function Box({ isHighlighted }: { isHighlighted: boolean }) {
const ref = useRef<HTMLDivElement>(null)
useEffect(() => {
if (ref.current && isHighlighted) {
ref.current.style.width = '100px'
const width = ref.current.offsetWidth // Forces layout
ref.current.style.height = '200px'
}
}, [isHighlighted])
return <div ref={ref}>Content</div>
}
// Correct: toggle class
function Box({ isHighlighted }: { isHighlighted: boolean }) {
return (
<div className={isHighlighted ? 'highlighted-box' : ''}>
Content
</div>
)
}
```
Prefer CSS classes over inline styles when possible. CSS files are cached by the browser, and classes provide better separation of concerns and are easier to maintain.
See [this gist](https://gist.github.com/paulirish/5d52fb081b3570c81e3a) and [CSS Triggers](https://csstriggers.com/) for more information on layout-forcing operations.
@@ -1,80 +0,0 @@
---
title: Cache Repeated Function Calls
impact: MEDIUM
impactDescription: avoid redundant computation
tags: javascript, cache, memoization, performance
---
## Cache Repeated Function Calls
Use a module-level Map to cache function results when the same function is called repeatedly with the same inputs during render.
**Incorrect (redundant computation):**
```typescript
function ProjectList({ projects }: { projects: Project[] }) {
return (
<div>
{projects.map(project => {
// slugify() called 100+ times for same project names
const slug = slugify(project.name)
return <ProjectCard key={project.id} slug={slug} />
})}
</div>
)
}
```
**Correct (cached results):**
```typescript
// Module-level cache
const slugifyCache = new Map<string, string>()
function cachedSlugify(text: string): string {
if (slugifyCache.has(text)) {
return slugifyCache.get(text)!
}
const result = slugify(text)
slugifyCache.set(text, result)
return result
}
function ProjectList({ projects }: { projects: Project[] }) {
return (
<div>
{projects.map(project => {
// Computed only once per unique project name
const slug = cachedSlugify(project.name)
return <ProjectCard key={project.id} slug={slug} />
})}
</div>
)
}
```
**Simpler pattern for single-value functions:**
```typescript
let isLoggedInCache: boolean | null = null
function isLoggedIn(): boolean {
if (isLoggedInCache !== null) {
return isLoggedInCache
}
isLoggedInCache = document.cookie.includes('auth=')
return isLoggedInCache
}
// Clear cache when auth changes
function onAuthChange() {
isLoggedInCache = null
}
```
Use a Map (not a hook) so it works everywhere: utilities, event handlers, not just React components.
Reference: [How we made the Vercel Dashboard twice as fast](https://vercel.com/blog/how-we-made-the-vercel-dashboard-twice-as-fast)
@@ -1,28 +0,0 @@
---
title: Cache Property Access in Loops
impact: LOW-MEDIUM
impactDescription: reduces lookups
tags: javascript, loops, optimization, caching
---
## Cache Property Access in Loops
Cache object property lookups in hot paths.
**Incorrect (3 lookups × N iterations):**
```typescript
for (let i = 0; i < arr.length; i++) {
process(obj.config.settings.value)
}
```
**Correct (1 lookup total):**
```typescript
const value = obj.config.settings.value
const len = arr.length
for (let i = 0; i < len; i++) {
process(value)
}
```
@@ -1,70 +0,0 @@
---
title: Cache Storage API Calls
impact: LOW-MEDIUM
impactDescription: reduces expensive I/O
tags: javascript, localStorage, storage, caching, performance
---
## Cache Storage API Calls
`localStorage`, `sessionStorage`, and `document.cookie` are synchronous and expensive. Cache reads in memory.
**Incorrect (reads storage on every call):**
```typescript
function getTheme() {
return localStorage.getItem('theme') ?? 'light'
}
// Called 10 times = 10 storage reads
```
**Correct (Map cache):**
```typescript
const storageCache = new Map<string, string | null>()
function getLocalStorage(key: string) {
if (!storageCache.has(key)) {
storageCache.set(key, localStorage.getItem(key))
}
return storageCache.get(key)
}
function setLocalStorage(key: string, value: string) {
localStorage.setItem(key, value)
storageCache.set(key, value) // keep cache in sync
}
```
Use a Map (not a hook) so it works everywhere: utilities, event handlers, not just React components.
**Cookie caching:**
```typescript
let cookieCache: Record<string, string> | null = null
function getCookie(name: string) {
if (!cookieCache) {
cookieCache = Object.fromEntries(
document.cookie.split('; ').map(c => c.split('='))
)
}
return cookieCache[name]
}
```
**Important (invalidate on external changes):**
If storage can change externally (another tab, server-set cookies), invalidate cache:
```typescript
window.addEventListener('storage', (e) => {
if (e.key) storageCache.delete(e.key)
})
document.addEventListener('visibilitychange', () => {
if (document.visibilityState === 'visible') {
storageCache.clear()
}
})
```
@@ -1,32 +0,0 @@
---
title: Combine Multiple Array Iterations
impact: LOW-MEDIUM
impactDescription: reduces iterations
tags: javascript, arrays, loops, performance
---
## Combine Multiple Array Iterations
Multiple `.filter()` or `.map()` calls iterate the array multiple times. Combine into one loop.
**Incorrect (3 iterations):**
```typescript
const admins = users.filter(u => u.isAdmin)
const testers = users.filter(u => u.isTester)
const inactive = users.filter(u => !u.isActive)
```
**Correct (1 iteration):**
```typescript
const admins: User[] = []
const testers: User[] = []
const inactive: User[] = []
for (const user of users) {
if (user.isAdmin) admins.push(user)
if (user.isTester) testers.push(user)
if (!user.isActive) inactive.push(user)
}
```
@@ -1,50 +0,0 @@
---
title: Early Return from Functions
impact: LOW-MEDIUM
impactDescription: avoids unnecessary computation
tags: javascript, functions, optimization, early-return
---
## Early Return from Functions
Return early when result is determined to skip unnecessary processing.
**Incorrect (processes all items even after finding answer):**
```typescript
function validateUsers(users: User[]) {
let hasError = false
let errorMessage = ''
for (const user of users) {
if (!user.email) {
hasError = true
errorMessage = 'Email required'
}
if (!user.name) {
hasError = true
errorMessage = 'Name required'
}
// Continues checking all users even after error found
}
return hasError ? { valid: false, error: errorMessage } : { valid: true }
}
```
**Correct (returns immediately on first error):**
```typescript
function validateUsers(users: User[]) {
for (const user of users) {
if (!user.email) {
return { valid: false, error: 'Email required' }
}
if (!user.name) {
return { valid: false, error: 'Name required' }
}
}
return { valid: true }
}
```
@@ -1,45 +0,0 @@
---
title: Hoist RegExp Creation
impact: LOW-MEDIUM
impactDescription: avoids recreation
tags: javascript, regexp, optimization, memoization
---
## Hoist RegExp Creation
Don't create RegExp inside render. Hoist to module scope or memoize with `useMemo()`.
**Incorrect (new RegExp every render):**
```tsx
function Highlighter({ text, query }: Props) {
const regex = new RegExp(`(${query})`, 'gi')
const parts = text.split(regex)
return <>{parts.map((part, i) => ...)}</>
}
```
**Correct (memoize or hoist):**
```tsx
const EMAIL_REGEX = /^[^\s@]+@[^\s@]+\.[^\s@]+$/
function Highlighter({ text, query }: Props) {
const regex = useMemo(
() => new RegExp(`(${escapeRegex(query)})`, 'gi'),
[query]
)
const parts = text.split(regex)
return <>{parts.map((part, i) => ...)}</>
}
```
**Warning (global regex has mutable state):**
Global regex (`/g`) has mutable `lastIndex` state:
```typescript
const regex = /foo/g
regex.test('foo') // true, lastIndex = 3
regex.test('foo') // false, lastIndex = 0
```
@@ -1,37 +0,0 @@
---
title: Build Index Maps for Repeated Lookups
impact: LOW-MEDIUM
impactDescription: 1M ops to 2K ops
tags: javascript, map, indexing, optimization, performance
---
## Build Index Maps for Repeated Lookups
Multiple `.find()` calls by the same key should use a Map.
**Incorrect (O(n) per lookup):**
```typescript
function processOrders(orders: Order[], users: User[]) {
return orders.map(order => ({
...order,
user: users.find(u => u.id === order.userId)
}))
}
```
**Correct (O(1) per lookup):**
```typescript
function processOrders(orders: Order[], users: User[]) {
const userById = new Map(users.map(u => [u.id, u]))
return orders.map(order => ({
...order,
user: userById.get(order.userId)
}))
}
```
Build map once (O(n)), then all lookups are O(1).
For 1000 orders × 1000 users: 1M ops → 2K ops.
@@ -1,49 +0,0 @@
---
title: Early Length Check for Array Comparisons
impact: MEDIUM-HIGH
impactDescription: avoids expensive operations when lengths differ
tags: javascript, arrays, performance, optimization, comparison
---
## Early Length Check for Array Comparisons
When comparing arrays with expensive operations (sorting, deep equality, serialization), check lengths first. If lengths differ, the arrays cannot be equal.
In real-world applications, this optimization is especially valuable when the comparison runs in hot paths (event handlers, render loops).
**Incorrect (always runs expensive comparison):**
```typescript
function hasChanges(current: string[], original: string[]) {
// Always sorts and joins, even when lengths differ
return current.sort().join() !== original.sort().join()
}
```
Two O(n log n) sorts run even when `current.length` is 5 and `original.length` is 100. There is also overhead of joining the arrays and comparing the strings.
**Correct (O(1) length check first):**
```typescript
function hasChanges(current: string[], original: string[]) {
// Early return if lengths differ
if (current.length !== original.length) {
return true
}
// Only sort when lengths match
const currentSorted = current.toSorted()
const originalSorted = original.toSorted()
for (let i = 0; i < currentSorted.length; i++) {
if (currentSorted[i] !== originalSorted[i]) {
return true
}
}
return false
}
```
This new approach is more efficient because:
- It avoids the overhead of sorting and joining the arrays when lengths differ
- It avoids consuming memory for the joined strings (especially important for large arrays)
- It avoids mutating the original arrays
- It returns early when a difference is found
@@ -1,82 +0,0 @@
---
title: Use Loop for Min/Max Instead of Sort
impact: LOW
impactDescription: O(n) instead of O(n log n)
tags: javascript, arrays, performance, sorting, algorithms
---
## Use Loop for Min/Max Instead of Sort
Finding the smallest or largest element only requires a single pass through the array. Sorting is wasteful and slower.
**Incorrect (O(n log n) - sort to find latest):**
```typescript
interface Project {
id: string
name: string
updatedAt: number
}
function getLatestProject(projects: Project[]) {
const sorted = [...projects].sort((a, b) => b.updatedAt - a.updatedAt)
return sorted[0]
}
```
Sorts the entire array just to find the maximum value.
**Incorrect (O(n log n) - sort for oldest and newest):**
```typescript
function getOldestAndNewest(projects: Project[]) {
const sorted = [...projects].sort((a, b) => a.updatedAt - b.updatedAt)
return { oldest: sorted[0], newest: sorted[sorted.length - 1] }
}
```
Still sorts unnecessarily when only min/max are needed.
**Correct (O(n) - single loop):**
```typescript
function getLatestProject(projects: Project[]) {
if (projects.length === 0) return null
let latest = projects[0]
for (let i = 1; i < projects.length; i++) {
if (projects[i].updatedAt > latest.updatedAt) {
latest = projects[i]
}
}
return latest
}
function getOldestAndNewest(projects: Project[]) {
if (projects.length === 0) return { oldest: null, newest: null }
let oldest = projects[0]
let newest = projects[0]
for (let i = 1; i < projects.length; i++) {
if (projects[i].updatedAt < oldest.updatedAt) oldest = projects[i]
if (projects[i].updatedAt > newest.updatedAt) newest = projects[i]
}
return { oldest, newest }
}
```
Single pass through the array, no copying, no sorting.
**Alternative (Math.min/Math.max for small arrays):**
```typescript
const numbers = [5, 2, 8, 1, 9]
const min = Math.min(...numbers)
const max = Math.max(...numbers)
```
This works for small arrays, but can be slower or just throw an error for very large arrays due to spread operator limitations. Maximal array length is approximately 124000 in Chrome 143 and 638000 in Safari 18; exact numbers may vary - see [the fiddle](https://jsfiddle.net/qw1jabsx/4/). Use the loop approach for reliability.
@@ -1,24 +0,0 @@
---
title: Use Set/Map for O(1) Lookups
impact: LOW-MEDIUM
impactDescription: O(n) to O(1)
tags: javascript, set, map, data-structures, performance
---
## Use Set/Map for O(1) Lookups
Convert arrays to Set/Map for repeated membership checks.
**Incorrect (O(n) per check):**
```typescript
const allowedIds = ['a', 'b', 'c', ...]
items.filter(item => allowedIds.includes(item.id))
```
**Correct (O(1) per check):**
```typescript
const allowedIds = new Set(['a', 'b', 'c', ...])
items.filter(item => allowedIds.has(item.id))
```
@@ -1,57 +0,0 @@
---
title: Use toSorted() Instead of sort() for Immutability
impact: MEDIUM-HIGH
impactDescription: prevents mutation bugs in React state
tags: javascript, arrays, immutability, react, state, mutation
---
## Use toSorted() Instead of sort() for Immutability
`.sort()` mutates the array in place, which can cause bugs with React state and props. Use `.toSorted()` to create a new sorted array without mutation.
**Incorrect (mutates original array):**
```typescript
function UserList({ users }: { users: User[] }) {
// Mutates the users prop array!
const sorted = useMemo(
() => users.sort((a, b) => a.name.localeCompare(b.name)),
[users]
)
return <div>{sorted.map(renderUser)}</div>
}
```
**Correct (creates new array):**
```typescript
function UserList({ users }: { users: User[] }) {
// Creates new sorted array, original unchanged
const sorted = useMemo(
() => users.toSorted((a, b) => a.name.localeCompare(b.name)),
[users]
)
return <div>{sorted.map(renderUser)}</div>
}
```
**Why this matters in React:**
1. Props/state mutations break React's immutability model - React expects props and state to be treated as read-only
2. Causes stale closure bugs - Mutating arrays inside closures (callbacks, effects) can lead to unexpected behavior
**Browser support (fallback for older browsers):**
`.toSorted()` is available in all modern browsers (Chrome 110+, Safari 16+, Firefox 115+, Node.js 20+). For older environments, use spread operator:
```typescript
// Fallback for older browsers
const sorted = [...items].sort((a, b) => a.value - b.value)
```
**Other immutable array methods:**
- `.toSorted()` - immutable sort
- `.toReversed()` - immutable reverse
- `.toSpliced()` - immutable splice
- `.with()` - immutable element replacement
@@ -1,26 +0,0 @@
---
title: Use Activity Component for Show/Hide
impact: MEDIUM
impactDescription: preserves state/DOM
tags: rendering, activity, visibility, state-preservation
---
## Use Activity Component for Show/Hide
Use React's `<Activity>` to preserve state/DOM for expensive components that frequently toggle visibility.
**Usage:**
```tsx
import { Activity } from 'react'
function Dropdown({ isOpen }: Props) {
return (
<Activity mode={isOpen ? 'visible' : 'hidden'}>
<ExpensiveMenu />
</Activity>
)
}
```
Avoids expensive re-renders and state loss.
@@ -1,47 +0,0 @@
---
title: Animate SVG Wrapper Instead of SVG Element
impact: LOW
impactDescription: enables hardware acceleration
tags: rendering, svg, css, animation, performance
---
## Animate SVG Wrapper Instead of SVG Element
Many browsers don't have hardware acceleration for CSS3 animations on SVG elements. Wrap SVG in a `<div>` and animate the wrapper instead.
**Incorrect (animating SVG directly - no hardware acceleration):**
```tsx
function LoadingSpinner() {
return (
<svg
className="animate-spin"
width="24"
height="24"
viewBox="0 0 24 24"
>
<circle cx="12" cy="12" r="10" stroke="currentColor" />
</svg>
)
}
```
**Correct (animating wrapper div - hardware accelerated):**
```tsx
function LoadingSpinner() {
return (
<div className="animate-spin">
<svg
width="24"
height="24"
viewBox="0 0 24 24"
>
<circle cx="12" cy="12" r="10" stroke="currentColor" />
</svg>
</div>
)
}
```
This applies to all CSS transforms and transitions (`transform`, `opacity`, `translate`, `scale`, `rotate`). The wrapper div allows browsers to use GPU acceleration for smoother animations.
@@ -1,40 +0,0 @@
---
title: Use Explicit Conditional Rendering
impact: LOW
impactDescription: prevents rendering 0 or NaN
tags: rendering, conditional, jsx, falsy-values
---
## Use Explicit Conditional Rendering
Use explicit ternary operators (`? :`) instead of `&&` for conditional rendering when the condition can be `0`, `NaN`, or other falsy values that render.
**Incorrect (renders "0" when count is 0):**
```tsx
function Badge({ count }: { count: number }) {
return (
<div>
{count && <span className="badge">{count}</span>}
</div>
)
}
// When count = 0, renders: <div>0</div>
// When count = 5, renders: <div><span class="badge">5</span></div>
```
**Correct (renders nothing when count is 0):**
```tsx
function Badge({ count }: { count: number }) {
return (
<div>
{count > 0 ? <span className="badge">{count}</span> : null}
</div>
)
}
// When count = 0, renders: <div></div>
// When count = 5, renders: <div><span class="badge">5</span></div>
```
@@ -1,38 +0,0 @@
---
title: CSS content-visibility for Long Lists
impact: HIGH
impactDescription: faster initial render
tags: rendering, css, content-visibility, long-lists
---
## CSS content-visibility for Long Lists
Apply `content-visibility: auto` to defer off-screen rendering.
**CSS:**
```css
.message-item {
content-visibility: auto;
contain-intrinsic-size: 0 80px;
}
```
**Example:**
```tsx
function MessageList({ messages }: { messages: Message[] }) {
return (
<div className="overflow-y-auto h-screen">
{messages.map(msg => (
<div key={msg.id} className="message-item">
<Avatar user={msg.author} />
<div>{msg.content}</div>
</div>
))}
</div>
)
}
```
For 1000 messages, browser skips layout/paint for ~990 off-screen items (10× faster initial render).
@@ -1,46 +0,0 @@
---
title: Hoist Static JSX Elements
impact: LOW
impactDescription: avoids re-creation
tags: rendering, jsx, static, optimization
---
## Hoist Static JSX Elements
Extract static JSX outside components to avoid re-creation.
**Incorrect (recreates element every render):**
```tsx
function LoadingSkeleton() {
return <div className="animate-pulse h-20 bg-gray-200" />
}
function Container() {
return (
<div>
{loading && <LoadingSkeleton />}
</div>
)
}
```
**Correct (reuses same element):**
```tsx
const loadingSkeleton = (
<div className="animate-pulse h-20 bg-gray-200" />
)
function Container() {
return (
<div>
{loading && loadingSkeleton}
</div>
)
}
```
This is especially helpful for large and static SVG nodes, which can be expensive to recreate on every render.
**Note:** If your project has [React Compiler](https://react.dev/learn/react-compiler) enabled, the compiler automatically hoists static JSX elements and optimizes component re-renders, making manual hoisting unnecessary.
@@ -1,28 +0,0 @@
---
title: Optimize SVG Precision
impact: LOW
impactDescription: reduces file size
tags: rendering, svg, optimization, svgo
---
## Optimize SVG Precision
Reduce SVG coordinate precision to decrease file size. The optimal precision depends on the viewBox size, but in general reducing precision should be considered.
**Incorrect (excessive precision):**
```svg
<path d="M 10.293847 20.847362 L 30.938472 40.192837" />
```
**Correct (1 decimal place):**
```svg
<path d="M 10.3 20.8 L 30.9 40.2" />
```
**Automate with SVGO:**
```bash
npx svgo --precision=1 --multipass icon.svg
```
@@ -1,75 +0,0 @@
---
title: Use useTransition Over Manual Loading States
impact: LOW
impactDescription: reduces re-renders and improves code clarity
tags: rendering, transitions, useTransition, loading, state
---
## Use useTransition Over Manual Loading States
Use `useTransition` instead of manual `useState` for loading states. This provides built-in `isPending` state and automatically manages transitions.
**Incorrect (manual loading state):**
```tsx
function SearchResults() {
const [query, setQuery] = useState('')
const [results, setResults] = useState([])
const [isLoading, setIsLoading] = useState(false)
const handleSearch = async (value: string) => {
setIsLoading(true)
setQuery(value)
const data = await fetchResults(value)
setResults(data)
setIsLoading(false)
}
return (
<>
<input onChange={(e) => handleSearch(e.target.value)} />
{isLoading && <Spinner />}
<ResultsList results={results} />
</>
)
}
```
**Correct (useTransition with built-in pending state):**
```tsx
import { useTransition, useState } from 'react'
function SearchResults() {
const [query, setQuery] = useState('')
const [results, setResults] = useState([])
const [isPending, startTransition] = useTransition()
const handleSearch = (value: string) => {
setQuery(value) // Update input immediately
startTransition(async () => {
// Fetch and update results
const data = await fetchResults(value)
setResults(data)
})
}
return (
<>
<input onChange={(e) => handleSearch(e.target.value)} />
{isPending && <Spinner />}
<ResultsList results={results} />
</>
)
}
```
**Benefits:**
- **Automatic pending state**: No need to manually manage `setIsLoading(true/false)`
- **Error resilience**: Pending state correctly resets even if the transition throws
- **Better responsiveness**: Keeps the UI responsive during updates
- **Interrupt handling**: New transitions automatically cancel pending ones
Reference: [useTransition](https://react.dev/reference/react/useTransition)
@@ -1,39 +0,0 @@
---
title: Defer State Reads to Usage Point
impact: MEDIUM
impactDescription: avoids unnecessary subscriptions
tags: rerender, searchParams, localStorage, optimization
---
## Defer State Reads to Usage Point
Don't subscribe to dynamic state (searchParams, localStorage) if you only read it inside callbacks.
**Incorrect (subscribes to all searchParams changes):**
```tsx
function ShareButton({ chatId }: { chatId: string }) {
const searchParams = useSearchParams()
const handleShare = () => {
const ref = searchParams.get('ref')
shareChat(chatId, { ref })
}
return <button onClick={handleShare}>Share</button>
}
```
**Correct (reads on demand, no subscription):**
```tsx
function ShareButton({ chatId }: { chatId: string }) {
const handleShare = () => {
const params = new URLSearchParams(window.location.search)
const ref = params.get('ref')
shareChat(chatId, { ref })
}
return <button onClick={handleShare}>Share</button>
}
```
@@ -1,45 +0,0 @@
---
title: Narrow Effect Dependencies
impact: LOW
impactDescription: minimizes effect re-runs
tags: rerender, useEffect, dependencies, optimization
---
## Narrow Effect Dependencies
Specify primitive dependencies instead of objects to minimize effect re-runs.
**Incorrect (re-runs on any user field change):**
```tsx
useEffect(() => {
console.log(user.id)
}, [user])
```
**Correct (re-runs only when id changes):**
```tsx
useEffect(() => {
console.log(user.id)
}, [user.id])
```
**For derived state, compute outside effect:**
```tsx
// Incorrect: runs on width=767, 766, 765...
useEffect(() => {
if (width < 768) {
enableMobileMode()
}
}, [width])
// Correct: runs only on boolean transition
const isMobile = width < 768
useEffect(() => {
if (isMobile) {
enableMobileMode()
}
}, [isMobile])
```
@@ -1,40 +0,0 @@
---
title: Calculate Derived State During Rendering
impact: MEDIUM
impactDescription: avoids redundant renders and state drift
tags: rerender, derived-state, useEffect, state
---
## Calculate Derived State During Rendering
If a value can be computed from current props/state, do not store it in state or update it in an effect. Derive it during render to avoid extra renders and state drift. Do not set state in effects solely in response to prop changes; prefer derived values or keyed resets instead.
**Incorrect (redundant state and effect):**
```tsx
function Form() {
const [firstName, setFirstName] = useState('First')
const [lastName, setLastName] = useState('Last')
const [fullName, setFullName] = useState('')
useEffect(() => {
setFullName(firstName + ' ' + lastName)
}, [firstName, lastName])
return <p>{fullName}</p>
}
```
**Correct (derive during render):**
```tsx
function Form() {
const [firstName, setFirstName] = useState('First')
const [lastName, setLastName] = useState('Last')
const fullName = firstName + ' ' + lastName
return <p>{fullName}</p>
}
```
References: [You Might Not Need an Effect](https://react.dev/learn/you-might-not-need-an-effect)
@@ -1,29 +0,0 @@
---
title: Subscribe to Derived State
impact: MEDIUM
impactDescription: reduces re-render frequency
tags: rerender, derived-state, media-query, optimization
---
## Subscribe to Derived State
Subscribe to derived boolean state instead of continuous values to reduce re-render frequency.
**Incorrect (re-renders on every pixel change):**
```tsx
function Sidebar() {
const width = useWindowWidth() // updates continuously
const isMobile = width < 768
return <nav className={isMobile ? 'mobile' : 'desktop'} />
}
```
**Correct (re-renders only when boolean changes):**
```tsx
function Sidebar() {
const isMobile = useMediaQuery('(max-width: 767px)')
return <nav className={isMobile ? 'mobile' : 'desktop'} />
}
```
@@ -1,74 +0,0 @@
---
title: Use Functional setState Updates
impact: MEDIUM
impactDescription: prevents stale closures and unnecessary callback recreations
tags: react, hooks, useState, useCallback, callbacks, closures
---
## Use Functional setState Updates
When updating state based on the current state value, use the functional update form of setState instead of directly referencing the state variable. This prevents stale closures, eliminates unnecessary dependencies, and creates stable callback references.
**Incorrect (requires state as dependency):**
```tsx
function TodoList() {
const [items, setItems] = useState(initialItems)
// Callback must depend on items, recreated on every items change
const addItems = useCallback((newItems: Item[]) => {
setItems([...items, ...newItems])
}, [items]) // ❌ items dependency causes recreations
// Risk of stale closure if dependency is forgotten
const removeItem = useCallback((id: string) => {
setItems(items.filter(item => item.id !== id))
}, []) // ❌ Missing items dependency - will use stale items!
return <ItemsEditor items={items} onAdd={addItems} onRemove={removeItem} />
}
```
The first callback is recreated every time `items` changes, which can cause child components to re-render unnecessarily. The second callback has a stale closure bug—it will always reference the initial `items` value.
**Correct (stable callbacks, no stale closures):**
```tsx
function TodoList() {
const [items, setItems] = useState(initialItems)
// Stable callback, never recreated
const addItems = useCallback((newItems: Item[]) => {
setItems(curr => [...curr, ...newItems])
}, []) // ✅ No dependencies needed
// Always uses latest state, no stale closure risk
const removeItem = useCallback((id: string) => {
setItems(curr => curr.filter(item => item.id !== id))
}, []) // ✅ Safe and stable
return <ItemsEditor items={items} onAdd={addItems} onRemove={removeItem} />
}
```
**Benefits:**
1. **Stable callback references** - Callbacks don't need to be recreated when state changes
2. **No stale closures** - Always operates on the latest state value
3. **Fewer dependencies** - Simplifies dependency arrays and reduces memory leaks
4. **Prevents bugs** - Eliminates the most common source of React closure bugs
**When to use functional updates:**
- Any setState that depends on the current state value
- Inside useCallback/useMemo when state is needed
- Event handlers that reference state
- Async operations that update state
**When direct updates are fine:**
- Setting state to a static value: `setCount(0)`
- Setting state from props/arguments only: `setName(newName)`
- State doesn't depend on previous value
**Note:** If your project has [React Compiler](https://react.dev/learn/react-compiler) enabled, the compiler can automatically optimize some cases, but functional updates are still recommended for correctness and to prevent stale closure bugs.

Some files were not shown because too many files have changed in this diff Show More