Compare commits

..

96 Commits

Author SHA1 Message Date
Julian Sommer 7c6f03f0d0 chore: update action versions 2026-03-26 10:45:02 +10:00
Julian Sommer e6b48ce170 fix: docs 2026-03-26 10:29:48 +10:00
Julian Sommer fca9d018fc fix: install guide 2026-03-26 10:02:04 +10:00
Julian Sommer 4a2ae33dd9 fix: add missing assets 2026-03-26 09:56:38 +10:00
Julian Sommer 9e53443f7a fix: decrease shutdown wait time 2026-03-26 08:55:55 +10:00
Julian Sommer bf6b25842f feat: add dependabot auto updates 2026-03-26 08:36:50 +10:00
Ben Duncan 5b442e8eef chore: increase pseudo multi-node E2E timeout to 35m
External networking adds overhead (DHCP, tap setup/teardown) and the
shutdown drain phase needs more time with tap networking cleanup.
2026-03-25 20:54:46 +10:00
Ben Duncan a3c4819c1c fix: enable external networking in pseudo multi-node E2E
The pseudo multi-node test was not passing --external-mode/--external-pool
to spx admin init, so instances never got public IPs. SSH then failed
trying to use hostfwd which doesn't exist with tap networking.

- init_leader_node: pass SPX_EXTERNAL_POOL as --external-mode=pool flags
- e2e.yml: set SPX_EXTERNAL_POOL=192.168.1.220-229 for pseudo test run
2026-03-25 20:26:52 +10:00
Ben Duncan 70199f410a fix: pseudo multi-node E2E uses public IP + nbdkit for node detection
After enabling tap networking (public subnet), QEMU no longer uses
hostfwd bindings. The pseudo multi-node test was still grepping for
hostfwd to detect instance distribution and SSH endpoints.

- check_instance_distribution: count nbdkit root-volume processes per
  node IP instead of hostfwd-bound QEMU processes
- get_instance_node: match QEMU nbd socket → nbdkit host= IP to
  determine which simulated node hosts each instance
- count_instances_per_node: use get_instance_node instead of hostfwd
- SSH connectivity: check PublicIpAddress first (port 22), fall back
  to hostfwd — same pattern as single-node and multi-node E2E
2026-03-25 19:53:00 +10:00
Ben Duncan 7cdf630cf7 feat: propagate network config via formation server to joining nodes (mulga-759)
admin init builds a NetworkConfig from external networking flags and
passes it to the formation server. When joining nodes poll /formation/status,
they receive the cluster-wide [network] section (external mode, pools,
gateway, DNS, bootstrap VPC IDs). Each joining node auto-detects its
own external_interface and wan_bridge locally.

Fixes multi-node clusters having no [network] section in spinifex.toml
on secondary nodes, which caused no public IP assignment and broke
guest SSH in E2E tests.
2026-03-25 17:06:21 +10:00
Ben Duncan fb8c42e934 fix: poll for NAT rule cleanup instead of fixed sleep + hard fail
The dnat_and_snat rule is removed asynchronously via NATS after instance
termination. A 2s sleep was insufficient. Now polls for up to 30s and
downgrades to WARN if the rule persists (non-fatal — cleanup happens
when the VPC is deleted).
2026-03-25 16:35:15 +10:00
Ben Duncan cc46cf746e fix: multi-node E2E SSH via public IP when external networking enabled
Same fix as single-node: detect PublicIpAddress from describe-instances,
SSH via public IP on port 22. Falls back to QEMU hostfwd on the hosting
node when no public IP is available.
2026-03-25 16:05:54 +10:00
Ben Duncan e65f00c86a fix: set TenantID on root volume created by RunInstances (mulga-758)
GenerateVolumes did not set TenantID on the VolumeMetadata for root
volumes cloned from AMIs. After the pre-Phase4 tenant fallback was
removed (ae316db), describe-volumes filters volumes by exact tenantID
match — root volumes with empty TenantID became invisible to the
owning account, breaking E2E Phase 5b volume lifecycle tests.
2026-03-25 16:01:18 +10:00
Ben Duncan 85f51b8084 Merge branch 'vpc-public-subnet' into dev 2026-03-25 15:43:08 +10:00
Ben Duncan 4749a37725 fix: SSH via public IP instead of QEMU hostfwd when external networking enabled
When external_mode is set, dev_networking=false (no QEMU hostfwd NIC).
E2E tests now detect the instance's public IP from describe-instances
and SSH via port 22 on the public IP. Falls back to hostfwd only when
no public IP is available (overlay-only mode).

Applies to Phase 5a-ii, 7b, 7c-pre, and 8b SSH tests.
2026-03-25 15:34:33 +10:00
Ben Duncan 982a7b6b45 feat: SSH via public IP in E2E public subnet test
Verify full networking path: host → public IP → OVN DNAT → VM.
Test outbound internet from VM via OVN SNAT (ping 8.8.8.8).
Falls back to API-only verification if public IP unreachable.
2026-03-25 15:33:21 +10:00
Julian Sommer b81fdb5bc0 chore: frontend deps 2026-03-25 15:26:50 +10:00
Julian Sommer 71e82b22a7 chore: bump deps 2026-03-25 15:24:21 +10:00
Ben Duncan 21c4754336 feat: E2E tests for public/private subnet isolation + remove legacy nat mode
- Add Phase 8b to run-e2e.sh: public subnet instance gets PublicIpAddress
  and dnat_and_snat rule; private subnet instance gets neither; no blanket
  VPC CIDR SNAT (validates mulga-754 fix)
- Remove legacy external_mode="nat" — only "pool" with source=static or
  source=dhcp. Admin init defaults to pool/dhcp when WAN detected.
- Enable dev_networking always (was disabled with external mode, breaking
  E2E SSH tests)
- Update bootstrap.sh + e2e.yml: SPX_EXTERNAL_POOL env var configures
  static IP pool for CI (230-240 single-node, 240-250 multi-node)
- Fix run-vpc-e2e.sh SNAT assertion for mulga-754 (no blanket SNAT)
- Add unit tests for parseDHCPCDLeasedIP, ReconcileFromKV edge cases,
  DHCP pool config validation to meet 60% diff coverage threshold
2026-03-25 15:22:18 +10:00
Julian Sommer ec29cc2745 Revert "fix: remove InsecureSkipVerify (#87)"
This reverts commit a66b1a3d48.
2026-03-25 15:19:48 +10:00
Julian Sommer 5d31f05dba refactor: migrate ObjectStore from ListObjects V1 to V2
Replace all ListObjects (V1) usage with ListObjectsV2 across the
objectstore interface, both implementations, all 10 call sites, and
tests. V2 uses ContinuationToken instead of Marker for pagination
and adds KeyCount to responses. The frontend and Predastore already
speak V2 — this aligns the Go backend.
2026-03-25 14:44:13 +10:00
Julian Sommer 73e5905092 chore: bump viperblock version 2026-03-25 14:17:15 +10:00
Ben Duncan edcc53b947 Merge commit '820f620e34006699de1105859301146aaf491ca5' into vpc-public-subnet
# Conflicts:
#	cmd/spinifex/cmd/admin.go
#	docs/installing-spinifex/README.md
#	scripts/setup.sh
#	spinifex/admin/admin.go
#	spinifex/admin/network_detect.go
#	spinifex/handlers/iam/service_impl.go
2026-03-25 14:16:47 +10:00
Julian Sommer 820f620e34 feat: aws tf template 2026-03-25 13:54:24 +10:00
Ben Duncan 3d73dd180b fix: add dhcpcd-base to system dependencies for DHCP pool mode
DHCP pool mode requires dhcpcd-base (/usr/sbin/dhcpcd) to obtain
leases from the router. Added to Makefile install-system, setup.sh
binary installer, and source-install docs.

Closes: mulga-753
2026-03-25 12:05:21 +10:00
Ben Duncan 7bfb032f90 fix: remove blanket VPC CIDR SNAT from IGW — private subnet isolation (AWS parity)
handleIGWAttach and reconcileIGW created a blanket snat rule for the
entire VPC CIDR, allowing private subnet instances (no public IP) to
route outbound through the IGW. AWS behavior: only instances with
public IPs route through IGW via per-VM dnat_and_snat rules. Private
subnets require a NAT Gateway for outbound internet.

Also fixes pre-existing lint issues in external_ipam_dhcp.go (gofmt,
modernize, nolintlint).

Closes: mulga-754
2026-03-25 12:05:15 +10:00
Julian Sommer 15d6b0aa20 fix: multi node e2e decrease ssh timeout 2026-03-25 11:53:52 +10:00
Julian Sommer a66b1a3d48 fix: remove InsecureSkipVerify (#87) 2026-03-25 12:51:34 +11:00
Ben Duncan cbbd81bc76 feat: DHCP pool mode — public IPs from router DHCP instead of static range
Adds source="dhcp" option to external pool config. When set, ExternalIPAM
obtains public IPs from the upstream router's DHCP server (via dhcpcd on
br-ext) instead of allocating from a static range_start..range_end.

The VM behavior is identical to static pool mode: private VPC IP from OVN
DHCP, public IP via DNAT+SNAT, full DescribeInstances support. The only
difference is where the public IP comes from.

Two separate DHCP conversations:
- Host ↔ Router DHCP (br-ext): "give me a public IP for this VM's NAT"
- VM ↔ OVN DHCP (br-int): "give me my private VPC IP"

Tested: DHCP pool mode on single-node (ben-banksia). VM got public IP
192.168.1.77 from router DHCP, SSH works, outbound internet works.
2026-03-25 10:53:29 +10:00
Julian Sommer 8c4adf08a3 chore: bump predastore to v1.1.2 2026-03-25 10:52:08 +10:00
Julian Sommer c0b89e6d8c fix: path 2026-03-25 10:35:30 +10:00
Ben Duncan dd86805514 docs: add DHCP pool mode to vpc-networking guide
Explains static vs DHCP IP sources in pool mode, the two-DHCP-conversations
architecture, when to use each, and adds a DHCP homelab deployment example.
Also updates bridge setup section for veth pair approach.
2026-03-25 10:30:42 +10:00
Julian Sommer ad1339505d fix: dump vpcd log 2026-03-25 10:09:39 +10:00
Julian Sommer ba7e6a5c67 fix: redirect image import to dev null 2026-03-25 10:04:16 +10:00
Ben Duncan 4d56565c3f feat: veth pair bridge — link Linux bridge to OVS without conversion
setup-ovn.sh now detects whether the default route is on a Linux bridge
or OVS bridge. When it's a Linux bridge (e.g. br-wan from cloud-init),
instead of destructively converting it, the script creates an OVS bridge
(br-ext) and links them with a veth pair. The Linux bridge keeps its IP,
routes, and ports untouched — no WAN interruption.

Also fixes all pre-existing lint issues (gofmt, ineffassign, errchkjson,
gosec, modernize, whitespace) to bring golangci-lint to 0 issues.

Tested: pool mode with static IP range on single-node (ben-banksia).
VM gets public IP 192.168.1.201, SSH works, outbound internet works.
2026-03-25 10:01:37 +10:00
Julian Sommer ae316dba47 fix: remove pre-Phase4 volume tenant fallback
Simplify all 6 tenant-check sites in volume handlers from
`if accountID != "" && tenantID != "" && tenantID != accountID`
to `if tenantID != accountID`. No pre-Phase4 volumes with empty
TenantID exist in any environment — this code was unreachable.
Remove vol-legacy test fixture and empty-accountID-sees-all assertion.
2026-03-25 09:59:42 +10:00
Julian Sommer bdf0522895 feat: admin init auto install cert 2026-03-25 09:06:50 +10:00
dependabot[bot] e347f0a75e chore: bump github.com/nats-io/nats-server/v2 (#86) 2026-03-25 08:14:15 +10:00
Ben Duncan c2992ebe5e fix: reset-dev-env.sh — pass ADMIN_INIT_ARGS to second admin init --force
The second admin init --force was called without external networking
flags, overwriting the pool/nat config from the first call with
auto-detected NAT+DHCP defaults.
2026-03-24 17:09:48 +10:00
Ben Duncan ffca7330c5 fix: skip external IPAM init in NAT mode — only pool mode needs per-VM allocation
In NAT mode, all VMs share a single gateway IP for SNAT. No per-VM
public IPs are allocated. Previously, external IPAM was initialized
for any external_mode != "", causing noisy AllocateIP failures (empty
pool range) on every instance launch.

Now IPAM is only initialized when external_mode == "pool". In NAT mode,
d.externalIPAM stays nil, cleanly skipping the auto-assign public IP
path. SNAT topology is still created by vpcd via the pool's gateway_ip.

Part of mulga-737 (Phase 16).
2026-03-24 16:27:28 +10:00
Ben Duncan a21a275776 docs: update vpc-networking README for br-wan + new setup-ovn.sh flags
Replace all br-external references with br-wan. Update setup-ovn.sh
command examples to use --wan-bridge/--wan-iface/--macvlan instead of
--external-bridge/--external-iface/--direct.

Part of mulga-742.
2026-03-24 16:19:19 +10:00
Ben Duncan 78bc0d989d docs: update install guides + setup.sh for new setup-ovn.sh flags
- All guides: OVN setup before admin init (consistent ordering)
- Single-node + source-install: Replace --external-bridge/--external-iface
  with auto-detection + --wan-bridge/--wan-iface/--macvlan options
- Multi-node: Simplify examples (auto-detect WAN bridge), show override
  options in prose
- setup.sh: Update "next steps" to match single-node guide, add
  --wan-bridge/--macvlan guidance for physical NIC setups

Part of mulga-742.
2026-03-24 16:17:35 +10:00
Ben Duncan 8c6f66670a feat: add wan_bridge config field, update vpcd detection + network_detect
- config.go: Add WanBridge field to VPCDConfig (default "br-wan")
- spinifex.toml template: Add wan_bridge under [nodes.X.vpcd]
- admin.go: Auto-detect WAN bridge name from default route interface,
  add detectedWanBridge helper
- service.go: Wire WanBridge from cluster config to vpcd Config
- vpcd.go: Add WanBridge to Config, log it on startup, update comments
- network_detect.go: Fix isVirtualInterface to accept the default route
  device as a parameter instead of hardcoding br-external. Bridges like
  br-wan are now correctly treated as physical when they carry the
  default route.

Part of mulga-741.
2026-03-24 16:02:22 +10:00
Ben Duncan 1fca43c9f4 refactor: reset-dev-env.sh + stop-dev.sh — use new setup-ovn.sh flags
reset-dev-env.sh: Replace --external-bridge/--external-iface/--direct
with --wan-bridge/--wan-iface/--macvlan. Auto-detect when WAN interface
is already a bridge (tofu-cluster) vs physical NIC.

stop-dev.sh: Replace hardcoded br-external check with dynamic scan of
all OVS bridges — any non-br-int bridge with an IP means OVS is
infrastructure and should not be stopped.

Part of mulga-740.
2026-03-24 15:58:49 +10:00
Ben Duncan 34b50b11fb refactor: setup-ovn.sh — replace br-external with configurable WAN bridge
Replace --external-bridge/--external-iface/--direct with:
- --wan-bridge=NAME: OVS bridge for WAN traffic (auto-detected from
  default route if it's already a bridge)
- --wan-iface=NAME: physical NIC to add to the WAN bridge
- --macvlan: SSH-safe macvlan mode for single-NIC hosts

Auto-detection logic:
- If default route goes through a bridge (br-wan) → use it directly
- If default route is a physical NIC → stop with guidance (never
  auto-move a NIC into OVS — may drop SSH)

Bridge-mappings updated: external:$WAN_BRIDGE (was external:br-external).
OVN logical network name stays "external" (no topology.go change needed).

Encap IP now prefers br-vpc IP when available (dedicated VPC data plane).

Part of mulga-739. Blocks VPC E2E work.
2026-03-24 15:46:49 +10:00
Julian Sommer 8f81ac29a4 chore: update license references 2026-03-24 15:43:11 +10:00
Julian Sommer eefed70658 docs: add iam 2026-03-24 15:16:33 +10:00
Ben Duncan 216ca9e710 feat: Phase 12b + Phase 16 — KV reconcile on startup + NAT mode support
Phase 12b: vpcd now reconciles OVN topology from NATS KV on startup
(Pass 2). After subscribing to topics, it reads all VPCs, subnets,
IGW attachments, and ENIs from KV and ensures matching OVN objects
exist. Handles reboots, OVN DB loss, and missed events. All operations
are idempotent.

Phase 16 prep: Add --gateway-ip flag to admin init for NAT mode
(specifies the OVN gateway router's external IP without DHCP).
Update reset-dev-env.sh to support EXTERNAL_MODE=nat env var for
NAT/DHCP mode testing alongside the default pool mode.
2026-03-24 15:03:04 +10:00
Julian Sommer fefd5ed9e8 chore: update to typescript 6 2026-03-24 14:29:55 +10:00
Julian Sommer d47f2ab4fd fix: a11y 2026-03-24 14:02:26 +10:00
Julian Sommer 58a9d5741f fix: missing ui tests 2026-03-24 13:45:48 +10:00
Julian Sommer 625a78c529 feat: placement groups frontend 2026-03-24 13:39:04 +10:00
Ben Duncan 88baf5e202 feat: direct bridge mode — prefer over macvlan when WAN NIC != SSH NIC
Auto-detect bridge topology: if a dedicated WAN NIC exists (not used for
SSH), add it directly to br-external instead of creating a macvlan.
Direct bridge enables distributed NAT (DNAT on VM's own chassis) and
eliminates macvlan workarounds (MAC alignment, gratuitous ARPs,
centralized NAT bottleneck).

- topology.go: bridge-mode-aware NAT and localnet port options
- vpcd.go: auto-detect bridge mode, skip MAC alignment for direct bridge
- setup-ovn.sh: --direct flag for physical NIC on br-external
- reset-dev-env.sh: SSH NIC detection via SSH_CONNECTION
2026-03-24 01:57:54 +00:00
Julian Sommer 6379a2098a fix: lint errors 2026-03-24 11:55:42 +10:00
Julian Sommer 638828f636 feat: add leaders to get nodes command 2026-03-24 11:50:28 +10:00
Julian Sommer 0fbf4537c0 chore: bump viperblock 2026-03-24 11:24:40 +10:00
Julian Sommer 1b5108d379 chore: bump frontend deps 2026-03-24 10:45:01 +10:00
Julian Sommer bab7b1dea3 fix: enable last rules 2026-03-24 10:28:20 +10:00
Julian Sommer 30b08ca984 fix: use nullish coalescing 2026-03-24 10:20:11 +10:00
Julian Sommer 3467e0cf22 feat: oxlint type checked rules 2026-03-24 10:14:31 +10:00
Julian Sommer 35d0df5b56 feat: placement groups (#85) 2026-03-24 10:52:00 +11:00
Julian Sommer 648b36d672 fix: add agent browser 2026-03-24 09:25:48 +10:00
Ben Duncan d6062ed550 docs: document direct bridge vs macvlan modes for br-external
Direct bridge (WAN NIC added directly to OVS) is preferred when a
dedicated WAN NIC exists. Enables distributed NAT, no MAC workarounds,
host can reach VMs. Macvlan is fallback for single-NIC hosts only.
2026-03-23 11:36:15 +00:00
Ben Duncan 93aaf0f991 fix: single-node E2E — OVN SB cleanup, chassis naming, macvlan NAT, DNS
Three blockers fixed for VPC public subnet E2E on single-node:

1. OVN SB stale data: reset-dev-env.sh now deletes both NB+SB DB files
   instead of individual NB cleanup. Eliminates ovn-controller commit
   failure loop from stale chassis/port binding entries.

2. Chassis name mismatch: reset-dev-env.sh passes --chassis-id=chassis-node1
   to setup-ovn.sh, aligning OVS system-id with vpcd's chassis-{nodeName}.

3. Macvlan unicast filtering: switched to centralized NAT (no external_mac
   on DNAT rules), set nat-addresses=router on localnet ports, and vpcd
   aligns macvlan MAC with OVN gateway router MAC on startup. Macvlan only
   delivers unicast matching its own MAC — this ensures inbound traffic
   from the LAN reaches OVS/OVN.

Also fixed: DNS for VMs. admin init auto-detects host DNS servers via
resolvectl, stores them in spinifex.toml dns_servers field, and vpcd
propagates them to OVN DHCP options. Fallback: 8.8.8.8 + 1.1.1.1.

Validated E2E: inbound ping, SSH, outbound ping, DNS, curl, apt update.
2026-03-23 10:59:08 +00:00
Ben Duncan f34452883c refactor: move bootstrap config to admin init, add IGW ID, eliminate race
- admin init now pre-generates VPC/subnet/IGW IDs and writes [bootstrap]
  to spinifex.toml before services ever start. No race condition.
- EnsureDefaultVPC accepts optional BootstrapIDs to use pre-determined
  IDs instead of generating random ones.
- CreateInternetGatewayWithID allows daemon to use the bootstrap IGW ID.
- Removed writeBootstrapConfig from daemon (no longer needed).
- Fixed bootstrap account_id: uses admin account (000000000001) not
  system account (000000000000).
- reconcileIGW now sets spinifex:igw_id in OVN external IDs.
2026-03-23 06:57:25 +00:00
Ben Duncan c21f3dc188 feat: vpcd OVN reconciliation on startup + default subnet resolution
Phase 11b: RunInstances now auto-resolves the default subnet when no
SubnetId is specified, matching AWS behavior. Previously fell back to
SLIRP networking.

Phase 12: vpcd reconciles OVN topology from [bootstrap] config on startup.
The daemon writes a [bootstrap] section to spinifex.toml with the default
VPC/subnet IDs on first run. vpcd reads this on startup and creates any
missing OVN objects (router, switch, DHCP, IGW topology, SNAT, gateway
chassis). All operations are idempotent — safe on every restart.

This solves the bootstrap ordering problem where admin init creates VPC
records before vpcd is running, so NATS events are missed.

EnsureDefaultVPC now returns DefaultVPCInfo with VPC/subnet IDs.
2026-03-23 06:40:19 +00:00
Ben Duncan daae6ca6b8 refactor: remove single-nic bridge mode, standardize on macvlan
The single-nic IP migration path was fragile and caused connectivity
issues during setup. macvlan is now the only bridge strategy — always
creates spx-ext-{nic} on the WAN NIC. Removed --single-nic flag,
SingleNIC config/detection fields, and updated all docs.
2026-03-23 01:12:22 +00:00
Ben Duncan 32276c7cc7 feat: wire EIP + SG gateway actions, fix gateway chassis scheduling, default VPC infra
Phase 8a: Wire all 5 EIP gateway actions (AllocateAddress, ReleaseAddress,
AssociateAddress, DisassociateAddress, DescribeAddresses) end-to-end through
gateway → NATS → daemon → EIP service.

Phase 8b: Wire all 7 SG gateway actions (Create/Delete/Describe SecurityGroup,
Authorize/Revoke Ingress/Egress) through the same pipeline.

Critical fix: handleIGWAttach was not calling SetGatewayChassis on the gateway
router port. Without this, OVN has no chassis binding for external traffic —
SNAT/default route exist logically but packets have nowhere to go physically.
This was the root cause of outbound routing failure (can't ping 8.8.8.8).

Default VPC now matches AWS behavior: auto-creates IGW (attached),
MapPublicIpOnLaunch=true on default subnet, and a default security group
with intra-SG inbound allow + all outbound allow.

Comprehensive VPC networking docs rewrite covering modes, bridge setup,
configuration, security groups, EIPs, OVN reference, and troubleshooting.
2026-03-21 10:26:59 +00:00
Ben Duncan 30394b735b Removed legacy dir 2026-03-21 09:36:28 +00:00
Ben Duncan a25a4cec34 Merge commit '9b64c1911fc7e1a2442c376fb2cba0f5ece9a913' into vpc-public-subnet 2026-03-20 07:28:15 +00:00
Ben Duncan f6c6660f15 fix: create default VPC for admin account, not just system account
EnsureDefaultVPC was only called for GlobalAccountID (000000000000).
The admin account (000000000001) from admin init never got a default
VPC because seedAdminAccount didn't publish iam.account.created.

Fix: daemon startup now ensures default VPC for both system and admin
accounts. Also publish iam.account.created from seedAdminAccount so
any future account-creation listeners fire correctly.
2026-03-20 07:19:05 +00:00
Ben Duncan 704874c0c7 docs: restructure install guides + new common setup guide
All 3 install guides (binary, multi-node, source) now end at the same
verification point: aws ec2 describe-instance-types. Installation is
complete when that command returns instance types.

New docs/setting-up-your-cluster/README.md is the common guide all 3
install paths point to. Covers: AMI import, SSH key creation, VPC with
public subnet (IGW + modify-subnet-attribute), instance launch with
public IP, SSH via public IP, instance lifecycle management.

Removed SSH-via-hostfwd from all guides — public subnet is the default
connectivity path. Private subnet documented as an alternative.
2026-03-20 07:10:33 +00:00
Ben Duncan 665093343c fix: preserve DNS resolution after IP migration to br-external
When setup-ovn.sh migrates the WAN IP from a physical NIC to
br-external, systemd-resolved loses its DNS association (DNS servers
are per-interface). This breaks hostname resolution for everything
on the host — git, apt, curl, SSH all fail.

Now captures DNS config from the original interface before migration
and re-associates it with br-external via resolvectl. Falls back to
8.8.8.8 if no DNS was configured on the original interface.
2026-03-20 06:57:18 +00:00
Ben Duncan bb9e8db386 fix: stop-dev.sh must not stop OVS when br-external has WAN IP
When setup-ovn.sh migrates the host's WAN IP to br-external, stopping
openvswitch-switch destroys the bridge and the host loses all network
connectivity — unreachable via SSH, requires console reboot.

Now checks if br-external has an IP before stopping OVS. If it does,
skip the OVS stop and only stop ovn-controller + ovn-central. OVS is
infrastructure — bridges must survive service stop/start cycles.

Discovered during 3-node E2E testing: stop-dev.sh killed all 3 nodes'
WAN connectivity simultaneously.
2026-03-20 06:55:23 +00:00
Ben Duncan 0d4a70ef28 fix: remove verbose next-steps from admin init, disable dev_networking with external mode
Admin init output was duplicating install guide steps. Simplified to
just show completion message. Install guides already have all steps.

Set dev_networking=false when external_mode is configured — the hostfwd
NIC adds a confusing second interface inside VMs and is not needed when
public subnets provide WAN-routable IPs.
2026-03-20 06:45:11 +00:00
Ben Duncan 946429d792 feat: Phase 8c — daemon auto-assign public IP on RunInstances
Wire public IP auto-assignment end-to-end:
- RunInstances checks MapPublicIpOnLaunch on subnet, allocates external
  IP from ExternalIPAM, updates ENI record, publishes vpc.add-nat for
  dnat_and_snat rule, returns PublicIpAddress in response
- TerminateInstances releases public IP back to pool, publishes
  vpc.delete-nat to remove NAT rule before ENI deletion
- DescribeInstances populates PublicIpAddress from VM struct

Add ExternalIPAM initialization in daemon startup from cluster
NetworkConfig. Add GetSubnet and UpdateENIPublicIP helpers. Add
PublicIP/PublicIPPool fields to vm.VM struct. Fix network auto-detect
to recognize br-external as WAN interface.
2026-03-20 06:41:32 +00:00
Ben Duncan a554fe4ecf docs: rewrite single + multi-node install guides
Single-node:
- Add auto-detected network topology + external networking section
- Replace hostfwd SSH with VPC overlay private IP SSH
- Add setup-ovn.sh step with --dhcp flag
- Fix undefined variables ($SPINIFEX_AMI, $SPINIFEX_SUBNET)
- Use bash variable capture for VPC/subnet/AMI IDs
- Add three-tier external networking table
- Fix step headings (## → ###)

Multi-node:
- Move CA certificate trust to Step 5 (before starting services)
- Add --external-bridge --dhcp to all OVN setup commands
- Add AMI import step (was missing entirely)
- Add VPC + subnet creation step with variable capture
- Add SSH via private IP step
- Fix sudo inconsistency (all formation commands use sudo)
- Add external networking section with pool upgrade path
- Add DHCP troubleshooting section
- Add port requirements to overview
2026-03-20 05:58:38 +00:00
Ben Duncan 2af3bf5287 feat: auto-detect network topology + DHCP gateway in admin init
spx admin init now works with zero external networking flags:
- Auto-detects WAN interface, gateway, prefix from ip route
- Auto-detects single-NIC vs multi-NIC mode
- Defaults to nat mode with DHCP for gateway IP
- Prints detected topology with interface roles

setup-ovn.sh gains --dhcp flag: obtains a DHCP lease on the
macvlan/bridge interface for the OVN gateway SNAT address, then
writes gateway_ip into spinifex.toml. Tested on VM: macvlan gets
separate IP from router DHCP, host keeps its original IP.

Three tiers of external networking:
- Zero flags: auto nat+DHCP (VMs get outbound internet)
- --external-pool=start-end: pool mode (VMs get public IPs)
- --no-external: overlay only (no internet for VMs)
2026-03-20 05:45:30 +00:00
Ben Duncan 1f38213d4b feat: add external networking flags to spx admin init
Add --external-mode, --external-iface, --single-nic, --external-pool,
--external-gateway, --external-prefix-len flags to admin init. Config
is now generated with the correct [network] section and per-node vpcd
external_interface/single_nic — no manual spinifex.toml editing needed.

Single-node single-NIC:
  spx admin init --external-mode=pool --external-iface=eth0 --single-nic \
    --external-pool=192.168.1.150-192.168.1.250 --external-gateway=192.168.1.1

Single-node two-NIC:
  spx admin init --external-mode=pool --external-iface=eth1 \
    --external-pool=192.168.1.150-192.168.1.250 --external-gateway=192.168.1.1

Next steps output now shows the correct setup-ovn.sh command with
matching flags.
2026-03-20 05:24:06 +00:00
Ben Duncan 1ee7012ba9 feat: Phase 7 — single-NIC macvlan bridge strategy
Add --single-nic flag to setup-ovn.sh. When used with --external-bridge,
creates a macvlan sub-interface (spx-ext-{iface}) in bridge mode off the
host's NIC instead of adding the NIC directly to br-external. The host
keeps its IP — no migration, SSH-safe.

Strategy selection:
- Dedicated NIC: --external-iface=eth1 (add directly)
- Single NIC:    --external-iface=eth0 --single-nic (macvlan)
- IP migration:  --external-iface=eth0 (legacy, no --single-nic)

Add single_nic bool field to VPCDConfig for per-node config. Different
nodes in a cluster can use different strategies. Unit test verifies
single_nic parsing alongside external_interface.
2026-03-20 05:15:35 +00:00
Ben Duncan b6e99947e1 feat: Phase 5 — unit tests for ACL, security groups, public subnet, EIP
Add 18 unit tests covering:
- ACL match expression compiler (8 tests): TCP port/range, ICMP, UDP
  range, all-protocols, SG-ref source, egress, no-source filter
- Security group Port Group lifecycle (3 tests): create, delete, update
  with ACL generation via NATS events + MockOVNClient
- ModifySubnetAttribute + MapPublicIpOnLaunch (2 tests): default false,
  toggle to true via ModifySubnetAttribute
- Elastic IP lifecycle (5 tests): allocate, allocate from specific pool,
  release, release-while-associated error, describe
2026-03-20 04:35:52 +00:00
Ben Duncan 5481ef4361 feat: Phase 4 — security groups (OVN ACLs + Port Groups)
Add security group CRUD handlers: CreateSecurityGroup, DeleteSecurityGroup,
DescribeSecurityGroups, AuthorizeSecurityGroupIngress/Egress,
RevokeSecurityGroupIngress/Egress. SG records stored in NATS KV.

Add ACL match expression compiler (acl.go) translating AWS SG rules to
OVN ACL match strings for TCP/UDP port/ranges, ICMP, all-protocols, CIDR
filters, and intra-SG references.

Add vpcd security.go handlers for OVN Port Group lifecycle and ACL
management via vpc.create-sg/delete-sg/update-sg NATS topics. Extend
OVNClient interface with CreatePortGroup, DeletePortGroup,
SetPortGroupPorts, AddACL, ClearACLs. Add PortGroup and ACL to nbdb
model.
2026-03-20 04:24:49 +00:00
Ben Duncan ee06105c56 feat: Phase 3 — Elastic IPs (AllocateAddress, AssociateAddress)
Add EIP service with AllocateAddress, ReleaseAddress, AssociateAddress,
DisassociateAddress, and DescribeAddresses. Uses ExternalIPAM for IP
allocation from admin-defined pools. Publishes vpc.add-nat/delete-nat
events for OVN dnat_and_snat rules.

EIP records stored in spinifex-vpc-elastic-ips NATS KV bucket with
account-scoped keys. AssociateAddress replaces auto-assigned public IPs
when present.
2026-03-20 04:21:46 +00:00
Ben Duncan 150bf246a4 feat: Phase 2 — public subnets + auto-assign public IPs
Add MapPublicIpOnLaunch field to SubnetRecord with ModifySubnetAttribute
API handler. Add PublicIpAddress/PublicIpPool fields to ENIRecord for
tracking auto-assigned public IPs.

Add vpc.add-nat / vpc.delete-nat NATS topics and handlers in vpcd
topology for creating distributed dnat_and_snat NAT rules (with
LogicalPort + ExternalMAC for chassis-local NAT processing).

Wire ModifySubnetAttribute through gateway → daemon → VPC service.
2026-03-20 04:19:28 +00:00
Julian Sommer 9b64c1911f ci: add pull_request triggers to CI workflows
Enable PR-triggered CI so required status checks can gate merges.
Spinifex runs on all PRs (no path filter) to avoid blocking non-Go PRs.
UI workflow keeps path filters as it's not a required check.
2026-03-20 14:08:12 +10:00
Ben Duncan c1cdfc07c3 feat: Phase 1 completion — external IPAM, br-external setup, config template
Add external IPAM module (CAS-based IP allocation from admin-defined
pools in NATS KV). Supports multi-pool with region/AZ scoping, pool
fallback logic, gateway IP reservation, and concurrent allocation via
CAS retries. 12 unit tests covering allocation, release, exhaustion,
CAS conflict, multi-pool, fallback, validation, and idempotent init.

Add --external-bridge flag to setup-ovn.sh: creates OVS bridge
br-external, adds WAN NIC, migrates host IP to bridge, and sets
ovn-bridge-mappings=external:br-external for OVN localnet ports.

Add commented [network] section and external_interface to spinifex.toml
config template for discoverability.
2026-03-20 03:59:47 +00:00
Ben Duncan c6af189638 Merge commit '02931567ac207bd074d8b2efda6e523ab36540d9' into vpc-public-subnet 2026-03-20 03:47:50 +00:00
Julian Sommer 02931567ac release: update go.mod for v0.4.2 2026-03-20 13:09:48 +10:00
dependabot[bot] c4890c80d3 build(deps): bump fast-xml-parser (#83) 2026-03-20 12:48:53 +11:00
Julian Sommer d2d3e52522 feat: migrate to golangci-lint (#82) 2026-03-20 12:14:42 +11:00
Julian Sommer 42cb6dbed0 fix: add better error message 2026-03-20 11:12:32 +10:00
Julian Sommer 6cdeea3aa8 chore: more tests 2026-03-20 09:27:21 +10:00
Julian Sommer 4884b68b51 feat: version bootstrap file 2026-03-20 09:12:42 +10:00
Julian Sommer 726528bd9d feat: version jetstream buckets 2026-03-20 09:07:40 +10:00
Julian Sommer 593920ce7d feat: E2E test redistribution and dev-setup cleanup
Redistribute E2E tests between single-node and multi-node suites, slim
down pseudo multi-node suite, and remove directory creation from
dev-setup.sh so spx admin init is the single source of truth for
~/spinifex/ setup.
2026-03-20 08:24:15 +10:00
Ben Duncan 5e30d12f76 feat: Phase 1 — external network config + IGW gateway router fix
Add cluster-wide [network] config with external_mode and multi-pool
support (ExternalPool with region/AZ scoping). Per-node vpcd gets
external_interface for WAN NIC name.

Fix handleIGWAttach to use real external IPs from the configured pool
instead of link-local 169.254.0.1/30. Gateway router port now gets
the pool's gateway IP with correct prefix length, SNAT uses the real
external IP, and the default route points to the actual WAN gateway.

Includes pool selection logic (AZ-scoped → region-scoped → unscoped
fallback) and unit tests for config parsing, IGW with external pool,
explicit gateway IP, and pool selection.
2026-03-19 11:34:30 +00:00
294 changed files with 18146 additions and 4432 deletions
+45
View File
@@ -0,0 +1,45 @@
version: 2
updates:
- package-ecosystem: gomod
target-branch: dev
directory: /
schedule:
interval: weekly
day: monday
timezone: Australia/Sydney
cooldown:
default-days: 1
groups:
go-dependencies:
patterns:
- "*"
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
day: monday
timezone: Australia/Sydney
ignore:
- dependency-name: "*"
update-types:
- "version-update:semver-minor"
- "version-update:semver-patch"
groups:
github-actions:
patterns:
- "*"
- package-ecosystem: npm
target-branch: dev
directory: /spinifex/services/spinifexui/frontend
schedule:
interval: weekly
day: monday
timezone: Australia/Sydney
cooldown:
default-days: 1
groups:
npm-dependencies:
patterns:
- "*"
+4 -2
View File
@@ -61,6 +61,7 @@ jobs:
run: |
SPX_PREBUILT=true \
SPX_ARTIFACT_TAR="${RUNNER_TEMP}/spinifex-artifacts.tar" \
SPX_EXTERNAL_POOL="192.168.1.230-192.168.1.240" \
./bootstrap.sh env3 >> "$TOFU_LOG" 2>&1
- name: Run E2E Tests
@@ -84,7 +85,7 @@ jobs:
e2e_pseudo_multi:
name: Pseudo Multi-Node E2E
runs-on: [self-hosted, ci-pseudo]
timeout-minutes: 25
timeout-minutes: 30
env:
TOFU_LOG: /tmp/e2e-tofu-pseudo-${{ github.run_id }}.log
steps:
@@ -134,7 +135,7 @@ jobs:
SSH_KEY=$(echo "$INVENTORY" | jq -r '.ssh_key_path')
SSH_KEY="${SSH_KEY/#\~/$HOME}"
ssh -o StrictHostKeyChecking=no -i "$SSH_KEY" tf-user@${WAN_IP} \
'cd ~/Development/mulga/spinifex && ./tests/e2e/run-pseudo-multinode-e2e.sh'
'cd ~/Development/mulga/spinifex && SPX_EXTERNAL_POOL="192.168.1.220-192.168.1.229" ./tests/e2e/run-pseudo-multinode-e2e.sh'
- name: Teardown
if: always()
@@ -187,6 +188,7 @@ jobs:
run: |
SPX_PREBUILT=true \
SPX_ARTIFACT_TAR="${RUNNER_TEMP}/spinifex-artifacts.tar" \
SPX_EXTERNAL_POOL="192.168.1.240-192.168.1.250" \
./bootstrap.sh env2 >> "$TOFU_LOG" 2>&1
- name: Run E2E Tests
+6 -6
View File
@@ -37,7 +37,7 @@ jobs:
run: cd spinifex && make distro-amd64 VERSION="$VERSION"
- name: Upload amd64 Artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: distro-amd64
path: |
@@ -74,7 +74,7 @@ jobs:
run: cd spinifex && make distro-arm64 VERSION="$VERSION"
- name: Upload arm64 Artifacts
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: distro-arm64
path: |
@@ -86,13 +86,13 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Download amd64 Artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: distro-amd64
path: artifacts
- name: Download arm64 Artifacts
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: distro-arm64
path: artifacts
@@ -108,7 +108,7 @@ jobs:
generate_release_notes: true
- name: Upload Tarball for Verify Job
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: release-assets
path: artifacts/
@@ -140,7 +140,7 @@ jobs:
path: mulga
- name: Download Release Assets
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: release-assets
path: release-assets
+11 -10
View File
@@ -11,6 +11,10 @@ on:
- go.sum
- Makefile
- .github/workflows/spinifex.yml
pull_request:
branches:
- main
- dev
workflow_dispatch:
jobs:
@@ -120,14 +124,11 @@ jobs:
- name: Setup Go Workspace
run: go work init ./spinifex ./viperblock ./predastore
- name: Check Format
run: make -C spinifex check-format
- name: Lint
uses: golangci/golangci-lint-action@v9
with:
version: v2.11.3
working-directory: spinifex
- name: Check Modernize
run: make -C spinifex check-modernize
- name: Go Vet
run: make -C spinifex vet
- name: Security Checks
run: make -C spinifex security-check
- name: Govulncheck
run: make -C spinifex govulncheck
+7
View File
@@ -8,6 +8,13 @@ on:
paths:
- spinifex/services/spinifexui/frontend/**
- .github/workflows/ui.yml
pull_request:
branches:
- main
- dev
paths:
- spinifex/services/spinifexui/frontend/**
- .github/workflows/ui.yml
workflow_dispatch:
jobs:
+1
View File
@@ -14,6 +14,7 @@ go.work.sum
# Build outputs
bin/
dist/
!spinifex/services/spinifexui/frontend/dist/
tmp/
*.test
*.out
+128
View File
@@ -0,0 +1,128 @@
version: "2"
formatters:
enable:
- gofmt
settings:
gofmt:
simplify: false # modernize linter covers gofmt -s simplifications (Go 1.23+)
linters:
enable:
- asasalint # catches pass []any as any in variadic func(...any)
- asciicheck # checks that code does not contain non-ASCII identifiers
- bidichk # checks for dangerous unicode bidi character sequences
- bodyclose # checks whether HTTP response body is closed successfully
- canonicalheader # checks whether net/http.Header uses canonical header keys
- decorder # checks declaration order of types, consts, vars, and funcs
- durationcheck # checks for two durations multiplied together
- embeddedstructfieldcheck # checks embedded sync.Mutex/RWMutex in structs
- errcheck # checks for unchecked errors in type assertions and I/O
- errchkjson # checks types passed to json encoding functions, flags omittable error checks
- errname # checks that sentinel errors are prefixed with Err, error types suffixed with Error
- exptostd # detects golang.org/x/exp/ functions replaceable by stdlib
- fatcontext # detects nested contexts in loops
- gocheckcompilerdirectives # validates go compiler directive comments (//go:)
- goprintffuncname # checks that printf-like functions are named with f at the end
- gosec # inspects source code for security problems
- govet # reports suspicious constructs (Printf alignment, shadow, etc.)
- ineffassign # detects assignments to existing variables that are not used
- makezero # finds slice declarations with non-zero initial length followed by append
- mirror # catches wrong mirror patterns in bytes/strings usage
- modernize # modernizes code to use newer Go idioms (go fix analyzers)
- nakedret # finds naked returns in functions
- nilerr # catches code that returns nil even after checking err != nil
- nilnesserr # catches returning a different nil error variable after err != nil check
- nolintlint # reports ill-formed or insufficient nolint directives
- predeclared # catches shadowing of Go builtins (len, cap, error, etc.)
- reassign # checks that package variables are not reassigned
- recvcheck # checks for receiver type consistency (pointer vs value)
- sloglint # enforces consistent log/slog usage (key-value pairs, no globals)
- staticcheck # extensive static analysis checks
- unconvert # removes unnecessary type conversions
- usestdlibvars # detects hardcoded values that should use stdlib constants
- usetesting # catches os.TempDir() etc. in tests — should use t.TempDir()
- wastedassign # finds assignments to variables that are never subsequently read
- whitespace # detects leading and trailing whitespace in functions
settings:
embeddedstructfieldcheck:
forbid-mutex: true
errcheck:
check-type-assertions: true
exclude-functions:
- (*github.com/nats-io/nats.go.Subscription).Unsubscribe # cleanup-only, errors not actionable
- (*github.com/kdomanski/iso9660.ImageWriter).Cleanup # deferred cleanup, errors not actionable
govet:
enable-all: true
disable:
- fieldalignment
- shadow # too noisy to retrofit — all 50 findings are err re-declarations
- unusedwrite # false positives on struct field assignments before marshal/send
staticcheck:
checks:
- all
- -ST1000 # package comment
- -ST1003 # naming conventions (Id→ID etc.) — too noisy to retrofit
- -ST1012 # error var naming — errname covers this + error type suffixes
- -ST1013 # HTTP status constants — usestdlibvars is broader
- -ST1016 # receiver name consistency (too noisy to retrofit)
- -ST1020 # godoc on exported types — not enforcing yet
- -ST1021 # godoc on exported type groups — not enforcing yet
- -ST1022 # godoc on exported vars/consts — not enforcing yet
- -SA1019 # deprecated usage (handle separately)
- -SA4006 # value overwritten — ineffassign + wastedassign cover this better
- -SA9005 # ineffective break (staticcheck false positives)
gosec:
excludes:
- G104 # unchecked errors — errcheck is stricter
- G116 # bidi chars — bidichk is more thorough
- G117 # use of net/http serve functions (we handle TLS separately)
- G703 # path traversal taint analysis — false positives on config file loading
- G705 # XSS taint analysis — false positives on internal error responses
- G706 # log injection taint analysis — false positives on structured slog usage
modernize:
disable:
- newexpr # don't replace aws.String() with new() — AWS SDK idiom
- stringsbuilder # don't replace small string concatenations with strings.Builder
nakedret:
max-func-lines: 0
nolintlint:
require-explanation: true
require-specific: true
reassign:
patterns:
- ".*"
exclusions:
presets:
- std-error-handling
- common-false-positives
rules:
- path: '_test\.go'
linters:
- bodyclose
- errcheck
- errchkjson
- gosec
# spinifex-specific: gosec exclusions
- path: "cmd/"
linters:
- errcheck # CLI glue code — viper/cobra/pterm errors not actionable
- gosec # matches current -exclude-dir=cmd
- text: "G204"
linters: [gosec] # subprocess execution — legitimate in VM orchestration
- text: "G304"
linters: [gosec] # file path from variable — necessary for config loading
- text: "G402"
linters: [gosec] # TLS config — we handle TLS ourselves
+16 -54
View File
@@ -67,10 +67,10 @@ go_run:
@echo -e "\n....Running $(GO_PROJECT_NAME)...."
$(GOPATH)/bin/$(GO_PROJECT_NAME)
# Preflight — runs the same checks as GitHub Actions (format + lint + security + tests).
# Preflight — runs the same checks as GitHub Actions (lint + vuln + tests).
# Use this before committing to catch CI failures locally.
preflight:
@$(MAKE) --no-print-directory QUIET=1 check-format check-modernize vet security-check test-cover diff-coverage test-race
@$(MAKE) --no-print-directory QUIET=1 lint govulncheck test-cover diff-coverage test-race
@echo -e "\n ✅ Preflight passed — safe to commit."
# Run unit tests
@@ -116,7 +116,7 @@ install-system:
nbdkit nbdkit-plugin-dev pkg-config $(QEMU_PACKAGES) qemu-utils qemu-kvm \
libvirt-daemon-system libvirt-clients libvirt-dev make gcc jq curl \
iproute2 netcat-openbsd openssh-client wget git unzip sudo xz-utils file \
ovn-central ovn-host openvswitch-switch
ovn-central ovn-host openvswitch-switch dhcpcd-base
install-go:
@echo -e "\n....Installing Go 1.26.1 for $(ARCH) ($(GO_ARCH))...."
@@ -143,59 +143,21 @@ quickinstall: install-system install-go install-aws
@echo " Please ensure /usr/local/go/bin is in your PATH."
@echo " Installed: Go ($(GO_ARCH)), AWS CLI ($(AWS_ARCH)), QEMU ($(QEMU_PACKAGES))"
# Format all Go files in place
format:
gofmt -w .
# Lint all Go code via golangci-lint (replaces check-format, vet, gosec, staticcheck)
lint:
@echo "Running golangci-lint..."
$(_Q)golangci-lint run ./...
@echo " golangci-lint ok"
# Check that all Go files are formatted (CI-compatible, fails on diff)
check-format:
@echo "Checking gofmt..."
@UNFORMATTED=$$(gofmt -l .); \
if [ -n "$$UNFORMATTED" ]; then \
echo "Files not formatted:"; \
echo "$$UNFORMATTED"; \
echo "Run 'make format' to fix."; \
exit 1; \
fi
@echo " gofmt ok"
# Auto-fix all linter issues that have fixers
fix:
golangci-lint run --fix ./...
# Go vet (fails on issues, matches CI)
vet:
@echo "Running go vet..."
$(_Q)go vet ./...
@echo " go vet ok"
# Excluded: newexpr (replaces aws.String with new, not idiomatic for AWS SDK)
# Excluded: stringsbuilder (replaces string += in loops with strings.Builder, not worth the complexity for small loops)
GOFIX_EXCLUDE := -newexpr=false -stringsbuilder=false
# Apply go fix modernizations
modernize:
@echo "Applying go fix modernizations..."
go fix $(GOFIX_EXCLUDE) ./...
@echo " go fix applied"
# Check that code is modernized (CI-compatible, fails on diff)
check-modernize:
@echo "Checking go fix modernizations..."
@DIFF=$$(go fix $(GOFIX_EXCLUDE) -diff ./...); \
if [ -n "$$DIFF" ]; then \
echo "$$DIFF"; \
echo "Run 'make modernize' to fix."; \
exit 1; \
fi
@echo " go fix ok"
# Security checks — each tool fails the build on findings (matches CI).
# Reports are also saved to tests/ for review.
security-check:
@echo -e "\n....Running security checks for $(GO_PROJECT_NAME)...."
$(_Q)set -o pipefail && go tool govulncheck ./... $(_SECQ) tests/govulncheck-report.txt $(if $(QUIET),|| { cat tests/govulncheck-report.txt; exit 1; })
# Govulncheck — dependency vulnerability scanning (not covered by golangci-lint)
govulncheck:
@echo "Running govulncheck..."
$(_Q)go tool govulncheck ./...
@echo " govulncheck ok"
$(_Q)set -o pipefail && go tool gosec -quiet -exclude=G204,G304,G402,G117,G703,G705,G706 -exclude-generated -exclude-dir=cmd ./... $(_SECQ) tests/gosec-report.txt $(if $(QUIET),|| { cat tests/gosec-report.txt; exit 1; })
@echo " gosec ok"
$(_Q)set -o pipefail && go tool staticcheck -checks="all,-ST1000,-ST1003,-ST1016,-ST1020,-ST1021,-ST1022,-SA1019,-SA9005" ./... $(_SECQ) tests/staticcheck-report.txt $(if $(QUIET),|| { cat tests/staticcheck-report.txt; exit 1; })
@echo " staticcheck ok"
# Build release tarballs — use distro-ARCH for single arch, distro for both
distro: distro-amd64 distro-arm64
@@ -234,5 +196,5 @@ distro-clean:
.PHONY: build build-ui go_build go_run preflight test test-cover test-race diff-coverage bench run clean \
install-system install-go install-aws quickinstall \
format check-format modernize check-modernize vet security-check \
lint fix govulncheck \
distro distro-amd64 distro-arm64 distro-clean
+12 -11
View File
@@ -50,25 +50,26 @@ AWS-Compatible Interfaces – Provision infrastructure with awscli, Terraform, o
- Designed for Control – Run on your own terms, whether that’s in a datacenter, remote site, or sensitive environment.
- Minimal Dependencies – Each service is standalone and avoids complex orchestration layers.
- Works Offline – No reliance on centralized cloud services or external networks.
- Open Source – Licensed under Apache 2.0. Fork it, extend it, or deploy it as-is.
- Open Source – Licensed under AGPL 3.0. Fork it, extend it, or deploy it as-is.
## Installation
Installation requires an Ubuntu / Debian system. See the detailed documentation at [https://docs.mulgadc.com](https://docs.mulgadc.com/) for maintaining and installing Spinifex.
Installation requires an Ubuntu / Debian system. See the detailed documentation at [https://docs.mulgadc.com](https://docs.mulgadc.com) for maintaining and installing Spinifex.
### Single node install
### Single Node Install
The installation is straightforward to set up and running on a single node for testing purposes.
```bash
curl https://install.mulgadc.com | bash
sudo /usr/local/share/spinifex/setup-ovn.sh --management
sudo spx admin init
sudo systemctl start spinifex.target
export AWS_PROFILE=spinifex
export AWS_REGION=ap-southeast-2-sc
```
```bash
curl https://install.mulgadc.com/ | bash
spx admin init --region $AWS_REGION
aws ec2 describe-instance-types
```
@@ -121,4 +122,4 @@ This hybrid approach ensures Spinifex benefits from both proven engineering expe
## License
Spinifex is open source under the Apache 2.0 License. You're free to use, modify, and deploy it—anywhere you need reliable infrastructure without depending on centralized cloud platforms.
Spinifex is open source under the [GNU Affero General Public License v3.0](LICENSE). You're free to use, modify, and deploy it—anywhere you need reliable infrastructure without depending on centralized cloud platforms.
+424 -34
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package cmd
@@ -26,6 +27,7 @@ import (
"net"
"net/http"
"os"
"os/exec"
"path"
"path/filepath"
"sort"
@@ -215,6 +217,16 @@ func init() {
adminInitCmd.Flags().String("cluster-name", "spinifex", "NATS cluster name")
adminInitCmd.Flags().StringSlice("services", nil, "Services this node runs (default: all). Valid: nats,predastore,viperblock,daemon,awsgw,ui")
// External networking flags
adminInitCmd.Flags().String("external-mode", "", "External network mode: 'pool' (default when WAN detected) or '' (disabled)")
adminInitCmd.Flags().String("external-iface", "", "WAN NIC for br-external (auto-detected from default route)")
adminInitCmd.Flags().String("external-source", "", "Pool IP source: 'static' (default, uses --external-pool range) or 'dhcp' (from router DHCP)")
adminInitCmd.Flags().String("external-pool", "", "External IP pool range as start-end (e.g., 192.168.1.150-192.168.1.250)")
adminInitCmd.Flags().String("external-gateway", "", "WAN gateway IP (auto-detected from default route)")
adminInitCmd.Flags().String("gateway-ip", "", "OVN gateway router's external IP for SNAT (default: pool range_start for pool mode, required for nat mode without DHCP)")
adminInitCmd.Flags().Int("external-prefix-len", 24, "External pool subnet prefix length (auto-detected)")
adminInitCmd.Flags().Bool("no-external", false, "Disable external networking (overlay-only, no internet for VMs)")
// Flags for admin join
adminJoinCmd.Flags().String("region", "ap-southeast-2", "Region for this node")
adminJoinCmd.Flags().String("az", "ap-southeast-2a", "Availability zone for this node")
@@ -238,11 +250,9 @@ func init() {
imagesImportCmd.Flags().String("arch", "", "Specified distro arch (e.g aarch64, arm64, x86_64)")
imagesImportCmd.Flags().String("platform", "Linux/UNIX", "Specified platform (e.g Linux/UNIX, Windows)")
imagesImportCmd.Flags().Bool("force", false, "Force command execution (overwrites existing files)")
}
func runimagesImportCmd(cmd *cobra.Command, args []string) {
var image utils.Images
var imageFile string
@@ -268,7 +278,8 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
imageDir := fmt.Sprintf("%s/images", baseDir)
if !admin.FileExists(imageDir) {
fmt.Fprintf(os.Stderr, "Image directory does not exist. Base path specified correctly? %s", imageDir)
fmt.Fprintf(os.Stderr, "Error: image directory does not exist: %s\n\n", imageDir)
fmt.Fprintf(os.Stderr, "Run 'spx admin init' first to initialize the Spinifex platform.\n")
os.Exit(1)
}
@@ -276,7 +287,6 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
imageName, _ := cmd.Flags().GetString("name")
if imageName != "" {
var exists bool
// Confirm the image exists
image, exists = utils.AvailableImages[imageName]
@@ -285,9 +295,7 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
fmt.Fprintf(os.Stderr, "Image name not found in available images")
os.Exit(1)
}
} else {
imageFile, _ = cmd.Flags().GetString("file")
if imageFile == "" {
@@ -304,7 +312,6 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
image.Version, _ = cmd.Flags().GetString("version")
image.Arch, _ = cmd.Flags().GetString("arch")
image.Platform, _ = cmd.Flags().GetString("platform")
}
if image.Distro == "" {
@@ -351,14 +358,12 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
if admin.FileExists(imageFile) && !forceCmd {
fmt.Printf("Image file already exists, skipping download, use --force to overwrite: %s\n", imageFile)
} else {
err := utils.DownloadFileWithProgress(image.URL, image.Name, imageFile, 0)
if err != nil {
fmt.Printf("Download failed: %v\n", err)
os.Exit(1)
}
}
// Update image file path for later extraction
@@ -475,12 +480,10 @@ func runimagesImportCmd(cmd *cobra.Command, args []string) {
defer os.RemoveAll(tmpDir)
fmt.Printf("✅ Image import complete. Image-ID (AMI): %s\n", volumeId)
}
// List remote images available
func runimagesListCmd(cmd *cobra.Command, args []string) {
//fmt.Println(availableImages)
tableData := pterm.TableData{
@@ -499,7 +502,6 @@ func runimagesListCmd(cmd *cobra.Command, args []string) {
// 3. Iterate in sorted order
for _, k := range keys {
img := utils.AvailableImages[k]
//for _, img := range utils.AvailableImages {
@@ -518,6 +520,12 @@ func runimagesListCmd(cmd *cobra.Command, args []string) {
// TODO: Move all logic to a module, use minimal application logic in viper commands
func runAdminInit(cmd *cobra.Command, args []string) {
if os.Getuid() != 0 {
fmt.Fprintln(os.Stderr, "⚠️ Not running as root — CA certificate will not be installed to system trust store.")
fmt.Fprintln(os.Stderr, " Run with sudo, or manually: sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt && sudo update-ca-certificates")
fmt.Fprintln(os.Stderr)
}
force, _ := cmd.Flags().GetBool("force")
configDir, _ := cmd.Flags().GetString("config-dir")
spxRoot, _ := cmd.Flags().GetString("spinifex-dir")
@@ -538,6 +546,124 @@ func runAdminInit(cmd *cobra.Command, args []string) {
clusterName, _ := cmd.Flags().GetString("cluster-name")
services, _ := cmd.Flags().GetStringSlice("services")
// External networking flags
externalMode, _ := cmd.Flags().GetString("external-mode")
externalIface, _ := cmd.Flags().GetString("external-iface")
externalSource, _ := cmd.Flags().GetString("external-source")
externalPool, _ := cmd.Flags().GetString("external-pool")
externalGateway, _ := cmd.Flags().GetString("external-gateway")
externalPrefixLen, _ := cmd.Flags().GetInt("external-prefix-len")
gatewayIP, _ := cmd.Flags().GetString("gateway-ip")
noExternal, _ := cmd.Flags().GetBool("no-external")
// Auto-detect network topology
var poolStart, poolEnd string
var detectedNet *admin.DetectedNetwork
if !noExternal {
detected, err := admin.DetectNetwork()
if err != nil {
fmt.Fprintf(os.Stderr, "⚠️ Network auto-detection failed: %v\n", err)
fmt.Fprintf(os.Stderr, " Use --no-external to skip, or specify flags manually.\n")
} else {
detectedNet = detected
// Print detected topology
fmt.Println("\n🔍 Detected network topology:")
fmt.Printf(" %-14s %-18s %-20s %-16s %s\n", "Interface", "IP", "Subnet", "Gateway", "Role")
for _, iface := range detected.Interfaces {
gw := "—"
if iface.Gateway != "" {
gw = iface.Gateway
}
fmt.Printf(" %-14s %-18s %-20s %-16s %s\n", iface.Name, iface.IP, iface.Subnet, gw, strings.ToUpper(iface.Role))
}
if detected.LANCount == 0 {
fmt.Println("\n Mode: single-NIC (macvlan for external bridge)")
} else {
fmt.Printf("\n Mode: %d LAN + 1 WAN (macvlan for external bridge)\n", detected.LANCount)
}
// Apply auto-detected values when flags not explicitly set
if detected.WAN != nil {
if externalIface == "" {
externalIface = detected.WAN.Name
}
if externalGateway == "" {
externalGateway = detected.WAN.Gateway
}
if !cmd.Flags().Changed("external-prefix-len") {
externalPrefixLen = detected.WAN.PrefixLen
}
// Default mode: always "pool" — with static range if --external-pool
// is given, otherwise with source=dhcp (gateway IP from router DHCP)
if externalMode == "" && !cmd.Flags().Changed("external-mode") {
externalMode = "pool"
if externalPool == "" && externalSource == "" {
externalSource = "dhcp"
}
}
}
}
}
// Validate external networking flags
if externalMode != "" && externalMode != "pool" {
fmt.Fprintf(os.Stderr, "❌ Error: --external-mode must be 'pool' or empty, got: %s\n", externalMode)
os.Exit(1)
}
if externalMode == "pool" {
if externalSource == "dhcp" {
// DHCP source: no static range needed, just gateway
if externalGateway == "" {
fmt.Fprintf(os.Stderr, "❌ Error: --external-gateway is required when --external-mode=pool\n")
os.Exit(1)
}
} else {
// Static source (default): need pool range
if externalPool == "" {
fmt.Fprintf(os.Stderr, "❌ Error: --external-pool is required when --external-mode=pool (e.g., 192.168.1.150-192.168.1.250)\n")
fmt.Fprintf(os.Stderr, " Or use --external-source=dhcp to get IPs from router DHCP\n")
if detectedNet != nil && detectedNet.WAN != nil {
sugStart, sugEnd := admin.SuggestPoolRange(detectedNet.WAN)
fmt.Fprintf(os.Stderr, " Suggested: --external-pool=%s-%s\n", sugStart, sugEnd)
}
os.Exit(1)
}
if externalGateway == "" {
fmt.Fprintf(os.Stderr, "❌ Error: --external-gateway is required when --external-mode=pool\n")
os.Exit(1)
}
parts := strings.SplitN(externalPool, "-", 2)
if len(parts) != 2 || net.ParseIP(parts[0]) == nil || net.ParseIP(parts[1]) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: --external-pool must be start-end IPs (e.g., 192.168.1.150-192.168.1.250), got: %s\n", externalPool)
os.Exit(1)
}
poolStart, poolEnd = parts[0], parts[1]
}
}
if externalGateway != "" && net.ParseIP(externalGateway) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: --external-gateway is not a valid IP: %s\n", externalGateway)
os.Exit(1)
}
if gatewayIP != "" && net.ParseIP(gatewayIP) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: --gateway-ip is not a valid IP: %s\n", gatewayIP)
os.Exit(1)
}
// Detect DNS servers from the host for VM DHCP
var dnsServers []string
if externalMode != "" {
dnsServers = detectDNSServers(externalIface)
if len(dnsServers) > 0 {
fmt.Printf(" DNS servers: %s\n", strings.Join(dnsServers, ", "))
}
}
// For pool/dhcp source, the gateway IP is obtained via DHCP on the
// macvlan/bridge interface (no static pool range or explicit gateway-ip).
useExternalDHCP := externalMode == "pool" && externalSource == "dhcp" && gatewayIP == ""
// Validate IP address format
if net.ParseIP(bindIP) == nil {
fmt.Fprintf(os.Stderr, "❌ Error: Invalid IP address for --bind: %s\n", bindIP)
@@ -619,6 +745,9 @@ func runAdminInit(cmd *cobra.Command, args []string) {
// Generate SSL certificates (with bind IP in SANs for multi-node support)
certPath := admin.GenerateCertificatesIfNeeded(configDir, force, bindIP)
// Install CA certificate into system trust store
installCACertificate(filepath.Join(configDir, "ca.pem"))
// Generate NATS token
natsToken, err := admin.GenerateNATSToken()
if err != nil {
@@ -636,9 +765,35 @@ func runAdminInit(cmd *cobra.Command, args []string) {
isMultiNode := nodes >= 2 && bindIP != "0.0.0.0"
if isMultiNode {
// Build cluster-wide network config for propagation to joining nodes
var networkConfig *formation.NetworkConfig
if externalMode != "" {
bootstrapVpcId := utils.GenerateResourceID("vpc")
bootstrapSubnetId := utils.GenerateResourceID("subnet")
bootstrapIgwId := utils.GenerateResourceID("igw")
networkConfig = &formation.NetworkConfig{
ExternalMode: externalMode,
ExternalDHCP: useExternalDHCP,
PoolName: "wan",
PoolSource: externalSource,
PoolStart: poolStart,
PoolEnd: poolEnd,
PoolGateway: externalGateway,
PoolGatewayIP: gatewayIP,
PoolPrefixLen: externalPrefixLen,
PoolDNSServers: dnsServers,
BootstrapAccountId: admin.DefaultAccountID(),
BootstrapVpcId: bootstrapVpcId,
BootstrapSubnetId: bootstrapSubnetId,
BootstrapIgwId: bootstrapIgwId,
BootstrapCidr: handlers_ec2_vpc.DefaultVPCCidr,
BootstrapSubnetCidr: handlers_ec2_vpc.DefaultSubnetCidr,
}
}
runAdminInitMultiNode(cmd, accessKey, secretKey, accountID, natsToken, clusterName,
configDir, spxRoot, certPath, region, az, node, bindIP, clusterBind,
port, nodes, formationTimeoutStr, services)
port, nodes, formationTimeoutStr, services, networkConfig)
return
}
@@ -705,6 +860,14 @@ func runAdminInit(cmd *cobra.Command, args []string) {
fmt.Printf("✅ Created: multi-node predastore.toml (node ID: %d)\n", predastoreNodeID)
}
// Pre-generate default VPC/subnet/IGW IDs for bootstrap config.
// These are written to [bootstrap] in spinifex.toml so vpcd can
// create OVN topology on first boot. The daemon uses the same IDs
// when it creates the records in NATS KV via EnsureDefaultVPC.
bootstrapVpcId := utils.GenerateResourceID("vpc")
bootstrapSubnetId := utils.GenerateResourceID("subnet")
bootstrapIgwId := utils.GenerateResourceID("igw")
configSettings := admin.ConfigSettings{
AccessKey: accessKey,
SecretKey: secretKey,
@@ -727,6 +890,42 @@ func runAdminInit(cmd *cobra.Command, args []string) {
OVNNBAddr: "tcp:127.0.0.1:6641",
OVNSBAddr: "tcp:127.0.0.1:6642",
ExternalMode: externalMode,
ExternalIface: externalIface,
WanBridge: detectedWanBridge(detectedNet),
ExternalDHCP: useExternalDHCP,
PoolName: "wan",
PoolSource: externalSource,
PoolStart: poolStart,
PoolEnd: poolEnd,
PoolGateway: externalGateway,
PoolGatewayIP: gatewayIP,
PoolPrefixLen: externalPrefixLen,
PoolDNSServers: dnsServers,
BootstrapAccountId: admin.DefaultAccountID(),
BootstrapVpcId: bootstrapVpcId,
BootstrapSubnetId: bootstrapSubnetId,
BootstrapIgwId: bootstrapIgwId,
BootstrapCidr: handlers_ec2_vpc.DefaultVPCCidr,
BootstrapSubnetCidr: handlers_ec2_vpc.DefaultSubnetCidr,
}
// Print external networking summary
if externalMode != "" {
fmt.Printf("\n📡 External networking: %s\n", externalMode)
fmt.Printf(" WAN interface: %s\n", externalIface)
if poolStart != "" {
fmt.Printf(" Source: static (IP range)\n")
fmt.Printf(" IP pool: %s - %s\n", poolStart, poolEnd)
fmt.Printf(" ⚠️ Ensure %s-%s is excluded from your router's DHCP range.\n", poolStart, poolEnd)
} else if useExternalDHCP {
fmt.Printf(" Source: dhcp (from router DHCP)\n")
fmt.Println(" Gateway IP: DHCP (obtained during OVN setup)")
} else if gatewayIP != "" {
fmt.Printf(" Gateway IP: %s (static)\n", gatewayIP)
}
}
// Generate config files
@@ -769,13 +968,6 @@ func runAdminInit(cmd *cobra.Command, args []string) {
// Print success message
fmt.Println("\n🎉 Spinifex initialization complete!")
fmt.Println("\n📋 Next steps:")
fmt.Println(" 1. Start services:")
fmt.Println(" ./scripts/start-dev.sh")
fmt.Println()
fmt.Println(" 2. Test with AWS CLI:")
fmt.Println(" export AWS_PROFILE=spinifex")
fmt.Println(" aws ec2 describe-instances")
fmt.Println()
fmt.Println("🔗 Configuration:")
fmt.Printf(" Config file: %s\n", spinifexTomlPath)
@@ -788,8 +980,7 @@ func runAdminInit(cmd *cobra.Command, args []string) {
// then generates configs with complete cluster topology.
func runAdminInitMultiNode(cmd *cobra.Command, accessKey, secretKey, accountID, natsToken, clusterName,
configDir, spxRoot, certPath, region, az, node, bindIP, clusterBind string,
port, expectedNodes int, formationTimeoutStr string, services []string) {
port, expectedNodes int, formationTimeoutStr string, services []string, networkConfig *formation.NetworkConfig) {
formationTimeout, err := time.ParseDuration(formationTimeoutStr)
if err != nil {
fmt.Fprintf(os.Stderr, "❌ Error: Invalid --formation-timeout: %v\n", err)
@@ -833,7 +1024,7 @@ func runAdminInitMultiNode(cmd *cobra.Command, accessKey, secretKey, accountID,
AdminSecretKey: bootstrapResult.AdminSecretKey,
}
fs := formation.NewFormationServer(expectedNodes, creds, string(caCertData), string(caKeyData))
fs := formation.NewFormationServer(expectedNodes, creds, string(caCertData), string(caKeyData), networkConfig)
// Include master key in formation server for distribution to joining nodes
fs.SetMasterKey(base64.StdEncoding.EncodeToString(masterKey))
@@ -941,6 +1132,34 @@ func runAdminInitMultiNode(cmd *cobra.Command, accessKey, secretKey, accountID,
OVNSBAddr: "tcp:127.0.0.1:6642",
}
// Apply cluster-wide network config to init node's config
if networkConfig != nil {
configSettings.ExternalMode = networkConfig.ExternalMode
configSettings.ExternalDHCP = networkConfig.ExternalDHCP
configSettings.PoolName = networkConfig.PoolName
configSettings.PoolSource = networkConfig.PoolSource
configSettings.PoolStart = networkConfig.PoolStart
configSettings.PoolEnd = networkConfig.PoolEnd
configSettings.PoolGateway = networkConfig.PoolGateway
configSettings.PoolGatewayIP = networkConfig.PoolGatewayIP
configSettings.PoolPrefixLen = networkConfig.PoolPrefixLen
configSettings.PoolDNSServers = networkConfig.PoolDNSServers
configSettings.BootstrapAccountId = networkConfig.BootstrapAccountId
configSettings.BootstrapVpcId = networkConfig.BootstrapVpcId
configSettings.BootstrapSubnetId = networkConfig.BootstrapSubnetId
configSettings.BootstrapIgwId = networkConfig.BootstrapIgwId
configSettings.BootstrapCidr = networkConfig.BootstrapCidr
configSettings.BootstrapSubnetCidr = networkConfig.BootstrapSubnetCidr
// Auto-detect local network topology for per-node config
detected, err := admin.DetectNetwork()
if err == nil && detected.WAN != nil {
configSettings.ExternalIface = detected.WAN.Name
configSettings.WanBridge = detectedWanBridge(detected)
}
}
// Generate config files
configs := []admin.ConfigFile{
{Name: "spinifex.toml", Path: spinifexTomlPath, Template: spinifexTomlTemplate},
@@ -991,6 +1210,11 @@ func runAdminInitMultiNode(cmd *cobra.Command, accessKey, secretKey, accountID,
}
func runAdminJoin(cmd *cobra.Command, args []string) {
if os.Getuid() != 0 {
fmt.Fprintln(os.Stderr, "⚠️ Not running as root — CA certificate will not be installed to system trust store.")
fmt.Fprintln(os.Stderr)
}
node, _ := cmd.Flags().GetString("node")
leaderHost, _ := cmd.Flags().GetString("host")
region, _ := cmd.Flags().GetString("region")
@@ -1166,6 +1390,9 @@ func runAdminJoin(cmd *cobra.Command, args []string) {
}
fmt.Printf("✅ CA certificate received from leader: %s\n", caCertPath)
// Install CA certificate into system trust store
installCACertificate(caCertPath)
// Extract and write master key from formation server
if statusResp.MasterKey == "" {
fmt.Fprintf(os.Stderr, "❌ Error: formation server did not return master key\n")
@@ -1264,6 +1491,37 @@ func runAdminJoin(cmd *cobra.Command, args []string) {
OVNSBAddr: fmt.Sprintf("tcp:%s:6642", leaderIP),
}
// Apply cluster-wide network config from leader
if statusResp.NetworkConfig != nil {
nc := statusResp.NetworkConfig
configSettings.ExternalMode = nc.ExternalMode
configSettings.ExternalDHCP = nc.ExternalDHCP
configSettings.PoolName = nc.PoolName
configSettings.PoolSource = nc.PoolSource
configSettings.PoolStart = nc.PoolStart
configSettings.PoolEnd = nc.PoolEnd
configSettings.PoolGateway = nc.PoolGateway
configSettings.PoolGatewayIP = nc.PoolGatewayIP
configSettings.PoolPrefixLen = nc.PoolPrefixLen
configSettings.PoolDNSServers = nc.PoolDNSServers
configSettings.BootstrapAccountId = nc.BootstrapAccountId
configSettings.BootstrapVpcId = nc.BootstrapVpcId
configSettings.BootstrapSubnetId = nc.BootstrapSubnetId
configSettings.BootstrapIgwId = nc.BootstrapIgwId
configSettings.BootstrapCidr = nc.BootstrapCidr
configSettings.BootstrapSubnetCidr = nc.BootstrapSubnetCidr
// Auto-detect local network topology for per-node config
if nc.ExternalMode != "" {
detected, err := admin.DetectNetwork()
if err == nil && detected.WAN != nil {
configSettings.ExternalIface = detected.WAN.Name
configSettings.WanBridge = detectedWanBridge(detected)
}
}
}
// Generate config files
configs := []admin.ConfigFile{
{Name: "spinifex.toml", Path: spinifexTomlPath, Template: spinifexTomlTemplate},
@@ -1381,7 +1639,7 @@ func runAccountCreate(cmd *cobra.Command, args []string) {
vpcSvc, vpcErr := handlers_ec2_vpc.NewVPCServiceImplWithNATS(&nodeConfig, nc)
if vpcErr != nil {
fmt.Fprintf(os.Stderr, "Warning: could not create default VPC service: %v\n", vpcErr)
} else if vpcErr = vpcSvc.EnsureDefaultVPC(accountID); vpcErr != nil {
} else if _, vpcErr = vpcSvc.EnsureDefaultVPC(accountID); vpcErr != nil {
fmt.Fprintf(os.Stderr, "Warning: could not create default VPC: %v\n", vpcErr)
}
@@ -1546,6 +1804,7 @@ func writeBootstrapFilesWithAdmin(configDir string, masterKey []byte, accessKey,
}
bd := &handlers_iam.BootstrapData{
Version: handlers_iam.BootstrapVersion,
AccessKeyID: accessKey,
EncryptedSecret: encryptedSecret,
AccountID: accountID,
@@ -1560,3 +1819,134 @@ func writeBootstrapFilesWithAdmin(configDir string, masterKey []byte, accessKey,
return handlers_iam.SaveBootstrapData(filepath.Join(configDir, "bootstrap.json"), bd)
}
// detectedWanBridge returns the OVS bridge name that OVN bridge-mappings should
// use for WAN traffic. If the WAN interface is already an OVS bridge, returns
// its name directly. If it's a Linux bridge, returns "br-ext" — setup-ovn.sh
// will create this OVS bridge and link it to the Linux bridge via a veth pair.
// If it's a physical NIC, returns "br-wan" as the default.
func detectedWanBridge(detected *admin.DetectedNetwork) string {
if detected == nil || detected.WAN == nil {
return ""
}
name := detected.WAN.Name
if strings.HasPrefix(name, "br-") {
// Check if it's an OVS bridge (use directly) or Linux bridge (need br-ext)
if out, err := exec.Command("sudo", "ovs-vsctl", "br-exists", name).CombinedOutput(); err == nil && len(out) == 0 {
return name // OVS bridge — use directly
}
// Linux bridge — setup-ovn.sh creates br-ext + veth pair
return "br-ext"
}
return "br-wan"
}
// detectDNSServers auto-detects DNS servers from the host for the specified
// interface. Uses resolvectl (systemd-resolved) first, then falls back to
// /etc/resolv.conf. Returns up to 3 servers. Falls back to public DNS if none found.
func detectDNSServers(iface string) []string {
// Try resolvectl for the specific link first (most reliable on modern systems)
if iface != "" {
out, err := exec.Command("resolvectl", "dns", iface).CombinedOutput()
if err == nil {
servers := parseDNSFromResolvectl(string(out))
if len(servers) > 0 {
return servers
}
}
}
// Try resolvectl global
out, err := exec.Command("resolvectl", "dns").CombinedOutput()
if err == nil {
servers := parseDNSFromResolvectl(string(out))
if len(servers) > 0 {
return servers
}
}
// Fall back to /etc/resolv.conf
data, err := os.ReadFile("/etc/resolv.conf")
if err == nil {
var servers []string
for line := range strings.SplitSeq(string(data), "\n") {
line = strings.TrimSpace(line)
if strings.HasPrefix(line, "nameserver ") {
ip := strings.TrimSpace(strings.TrimPrefix(line, "nameserver"))
// Skip localhost (systemd-resolved stub)
if ip != "127.0.0.53" && ip != "127.0.0.1" && net.ParseIP(ip) != nil {
servers = append(servers, ip)
}
}
}
if len(servers) > 0 {
if len(servers) > 3 {
servers = servers[:3]
}
return servers
}
}
// Fallback to well-known public DNS
return []string{"8.8.8.8", "1.1.1.1"}
}
// parseDNSFromResolvectl extracts IP addresses from resolvectl dns output.
// Format: "Link 2 (enp0s3): 192.168.1.1 8.8.8.8 1.1.1.1"
func parseDNSFromResolvectl(output string) []string {
var servers []string
for line := range strings.SplitSeq(output, "\n") {
// Find the colon separator, IPs come after it
_, after, ok := strings.Cut(line, ":")
if !ok {
continue
}
fields := strings.FieldsSeq(after)
for f := range fields {
if net.ParseIP(f) != nil && f != "127.0.0.53" && f != "127.0.0.1" {
servers = append(servers, f)
}
}
}
if len(servers) > 3 {
servers = servers[:3]
}
return servers
}
// installCACertificate copies the Spinifex CA certificate into the system
// trust store and runs update-ca-certificates so TLS clients (AWS CLI, etc.)
// trust the self-signed gateway certificate without extra configuration.
func installCACertificate(caPemPath string) {
if os.Getuid() != 0 {
return
}
const systemCertPath = "/usr/local/share/ca-certificates/spinifex-ca.crt"
data, err := os.ReadFile(caPemPath)
if err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not read CA certificate %s: %v\n", caPemPath, err)
return
}
if err := os.MkdirAll(filepath.Dir(systemCertPath), 0755); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not create certificate directory: %v\n", err)
return
}
if err := os.WriteFile(systemCertPath, data, 0644); err != nil {
fmt.Fprintf(os.Stderr, "Warning: could not install CA certificate: %v\n", err)
return
}
cmd := exec.Command("update-ca-certificates")
cmd.Stdout = io.Discard
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
fmt.Fprintf(os.Stderr, "Warning: update-ca-certificates failed: %v\n", err)
return
}
fmt.Printf("✅ CA certificate installed to system trust store\n")
}
+29 -11
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package cmd
@@ -138,6 +139,20 @@ func formatUptime(seconds int64) string {
return fmt.Sprintf("%dm", mins)
}
func formatRoles(resp types.NodeStatusResponse) string {
var roles []string
if resp.NATSRole != "" {
roles = append(roles, "nats:"+resp.NATSRole)
}
if resp.PredastoreRole != "" {
roles = append(roles, "predastore:"+resp.PredastoreRole)
}
if len(roles) == 0 {
return "-"
}
return strings.Join(roles, ",")
}
func formatMemGB(gb float64) string {
if gb >= 1.0 {
return fmt.Sprintf("%.1fGi", gb)
@@ -172,7 +187,7 @@ func runGetNodes(cmd *cobra.Command, args []string) {
// Build table: union of config-known nodes + NATS responders
tableData := pterm.TableData{
{"NAME", "STATUS", "IP", "REGION", "AZ", "UPTIME", "VMs", "SERVICES"},
{"NAME", "STATUS", "ROLES", "IP", "REGION", "AZ", "UPTIME", "VMs", "SERVICES"},
}
// Collect all node names: config + responded (union)
@@ -195,6 +210,7 @@ func runGetNodes(cmd *cobra.Command, args []string) {
tableData = append(tableData, []string{
resp.Node,
resp.Status,
formatRoles(resp),
resp.Host,
resp.Region,
resp.AZ,
@@ -206,6 +222,7 @@ func runGetNodes(cmd *cobra.Command, args []string) {
tableData = append(tableData, []string{
name,
"NotReady",
"-",
nodeCfg.Host,
nodeCfg.Region,
nodeCfg.AZ,
@@ -236,6 +253,7 @@ func runGetVMs(cmd *cobra.Command, args []string) {
type vmRow struct {
types.VMInfo
Node string
Host string
}
+49
View File
@@ -0,0 +1,49 @@
package cmd
import (
"testing"
"github.com/mulgadc/spinifex/spinifex/types"
"github.com/stretchr/testify/assert"
)
func TestFormatRoles(t *testing.T) {
tests := []struct {
name string
resp types.NodeStatusResponse
want string
}{
{
name: "no roles",
resp: types.NodeStatusResponse{},
want: "-",
},
{
name: "nats leader only",
resp: types.NodeStatusResponse{NATSRole: "leader"},
want: "nats:leader",
},
{
name: "predastore follower only",
resp: types.NodeStatusResponse{PredastoreRole: "follower"},
want: "predastore:follower",
},
{
name: "both roles",
resp: types.NodeStatusResponse{NATSRole: "leader", PredastoreRole: "follower"},
want: "nats:leader,predastore:follower",
},
{
name: "both leaders",
resp: types.NodeStatusResponse{NATSRole: "leader", PredastoreRole: "leader"},
want: "nats:leader,predastore:leader",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := formatRoles(tt.resp)
assert.Equal(t, tt.want, got)
})
}
}
+18 -5
View File
@@ -2,18 +2,18 @@ package cmd
import (
"os"
"os/user"
"path/filepath"
)
// DefaultConfigDir returns the default configuration directory.
// Production: /etc/spinifex (when running as root or /etc/spinifex exists)
// Production: /etc/spinifex (when running as root AND /etc/spinifex exists)
// Development: ~/spinifex/config
func DefaultConfigDir() string {
if isProductionLayout() {
return "/etc/spinifex"
}
homeDir, _ := os.UserHomeDir()
return filepath.Join(homeDir, "spinifex", "config")
return filepath.Join(realUserHomeDir(), "spinifex", "config")
}
// DefaultDataDir returns the default data directory.
@@ -23,8 +23,20 @@ func DefaultDataDir() string {
if isProductionLayout() {
return "/var/lib/spinifex"
}
return filepath.Join(realUserHomeDir(), "spinifex")
}
// realUserHomeDir returns the home directory of the real (non-sudo) user.
// When running under sudo, SUDO_USER is set to the invoking user — resolve
// their home directory so config/data land in the right place.
func realUserHomeDir() string {
if sudoUser := os.Getenv("SUDO_USER"); sudoUser != "" {
if u, err := user.Lookup(sudoUser); err == nil {
return u.HomeDir
}
}
homeDir, _ := os.UserHomeDir()
return filepath.Join(homeDir, "spinifex")
return homeDir
}
// DefaultConfigFile returns the default path to spinifex.toml.
@@ -33,7 +45,8 @@ func DefaultConfigFile() string {
}
// isProductionLayout returns true when running in a production install.
// Detected by: running as root, or /etc/spinifex directory exists.
// Detected by: running as root AND /etc/spinifex directory already exists
// (created by setup.sh during binary install).
func isProductionLayout() bool {
if os.Getuid() == 0 {
if info, err := os.Stat("/etc/spinifex"); err == nil && info.IsDir() {
+11 -11
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package cmd
@@ -102,7 +103,6 @@ func init() {
//viper.BindPFlag("nats.host", rootCmd.PersistentFlags().Lookup("nats-host"))
//viper.BindPFlag("nats.acl.token", rootCmd.PersistentFlags().Lookup("nats-token"))
//viper.BindPFlag("nats.sub.subject", rootCmd.PersistentFlags().Lookup("nats-subject"))
}
// initConfig reads in config file and ENV variables if set.
+58 -18
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package cmd
@@ -694,13 +695,54 @@ var vpcdStartCmd = &cobra.Command{
nodeConfig := clusterConfig.Nodes[clusterConfig.Node]
// Map cluster-wide external pools to vpcd config
var extPools []vpcd.ExternalPoolConfig
for _, p := range clusterConfig.Network.ExternalPools {
extPools = append(extPools, vpcd.ExternalPoolConfig{
Name: p.Name,
RangeStart: p.RangeStart,
RangeEnd: p.RangeEnd,
Gateway: p.Gateway,
GatewayIP: p.GatewayIP,
PrefixLen: p.PrefixLen,
DNSServers: p.DNSServers,
Region: p.Region,
AZ: p.AZ,
})
}
// Derive OVN chassis names from cluster node names (format: "chassis-{hostname}")
var chassisNames []string
for nodeName := range clusterConfig.Nodes {
chassisNames = append(chassisNames, "chassis-"+nodeName)
}
var bootstrap *vpcd.BootstrapVPC
if clusterConfig.Bootstrap.VpcId != "" {
bootstrap = &vpcd.BootstrapVPC{
AccountID: clusterConfig.Bootstrap.AccountID,
VpcId: clusterConfig.Bootstrap.VpcId,
SubnetId: clusterConfig.Bootstrap.SubnetId,
IgwId: clusterConfig.Bootstrap.IgwId,
Cidr: clusterConfig.Bootstrap.Cidr,
SubnetCidr: clusterConfig.Bootstrap.SubnetCidr,
}
}
svc, err := service.New("vpcd", &vpcd.Config{
NatsHost: nodeConfig.NATS.Host,
NatsToken: nodeConfig.NATS.ACL.Token,
OVNNBAddr: nodeConfig.VPCD.OVNNBAddr,
OVNSBAddr: nodeConfig.VPCD.OVNSBAddr,
BaseDir: nodeConfig.BaseDir,
Debug: false,
NatsHost: nodeConfig.NATS.Host,
NatsToken: nodeConfig.NATS.ACL.Token,
OVNNBAddr: nodeConfig.VPCD.OVNNBAddr,
OVNSBAddr: nodeConfig.VPCD.OVNSBAddr,
BaseDir: nodeConfig.BaseDir,
Debug: false,
ExternalMode: clusterConfig.Network.ExternalMode,
ExternalPools: extPools,
ChassisNames: chassisNames,
Bootstrap: bootstrap,
ExternalInterface: nodeConfig.VPCD.ExternalInterface,
WanBridge: nodeConfig.VPCD.WanBridge,
BridgeMode: nodeConfig.VPCD.BridgeMode,
})
if err != nil {
fmt.Println("Error starting vpcd service:", err)
@@ -744,7 +786,6 @@ var vpcdStatusCmd = &cobra.Command{
}
func init() {
viper.SetEnvPrefix("SPINIFEX") // Prefix for environment variables
viper.SetEnvKeyReplacer(strings.NewReplacer("-", "_"))
@@ -928,5 +969,4 @@ func init() {
vpcdCmd.AddCommand(vpcdStartCmd)
vpcdCmd.AddCommand(vpcdStopCmd)
vpcdCmd.AddCommand(vpcdStatusCmd)
}
+47 -1
View File
@@ -2,6 +2,42 @@
version = "1.0"
epoch = 1
node = "{{.Node}}"
{{- if .ExternalMode}}
[network]
external_mode = "{{.ExternalMode}}"
{{- if .ExternalDHCP}}
external_dhcp = true
{{- end}}
[[network.external_pools]]
name = "{{.PoolName}}"
{{- if .PoolSource}}
source = "{{.PoolSource}}"
{{- end}}
{{- if .PoolStart}}
range_start = "{{.PoolStart}}"
range_end = "{{.PoolEnd}}"
{{- end}}
gateway = "{{.PoolGateway}}"
{{- if .PoolGatewayIP}}
gateway_ip = "{{.PoolGatewayIP}}"
{{- end}}
prefix_len = {{if .PoolPrefixLen}}{{.PoolPrefixLen}}{{else}}24{{end}}
{{- if .PoolDNSServers}}
dns_servers = [{{range $i, $s := .PoolDNSServers}}{{if $i}}, {{end}}"{{$s}}"{{end}}]
{{- end}}
{{- end}}
{{- if .BootstrapVpcId}}
[bootstrap]
account_id = "{{.BootstrapAccountId}}"
vpc_id = "{{.BootstrapVpcId}}"
subnet_id = "{{.BootstrapSubnetId}}"
igw_id = "{{.BootstrapIgwId}}"
cidr = "{{.BootstrapCidr}}"
subnet_cidr = "{{.BootstrapSubnetCidr}}"
{{- end}}
[nodes.{{.Node}}]
node = "{{.Node}}"
@@ -18,7 +54,11 @@ account_id = "{{.AccountID}}"
host = "{{.BindIP}}:{{.Port}}"
tlskey = "config/server.key"
tlscert = "config/server.pem"
{{- if .ExternalMode}}
dev_networking = false
{{- else}}
dev_networking = true
{{- end}}
[nodes.{{.Node}}.awsgw]
host = "{{.BindIP}}:9999"
@@ -41,6 +81,12 @@ shardwal = false
[nodes.{{.Node}}.vpcd]
ovn_nb_addr = "{{.OVNNBAddr}}"
ovn_sb_addr = "{{.OVNSBAddr}}"
{{- if .ExternalIface}}
external_interface = "{{.ExternalIface}}"
{{- end}}
{{- if .WanBridge}}
wan_bridge = "{{.WanBridge}}"
{{- end}}
[nodes.{{.Node}}.predastore]
host = "{{.BindIP}}:8443"
@@ -61,4 +107,4 @@ az = "{{.AZ}}"
{{- if .Services}}
services = [{{range $i, $s := .Services}}{{if $i}}, {{end}}"{{$s}}"{{end}}]
{{- end}}
{{- end}}
{{- end}}
+11 -10
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package cmd
+11 -10
View File
@@ -1,17 +1,18 @@
/*
Copyright © 2025 Mulga Defense Corporation
Copyright © 2026 Mulga Defense Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
http://www.apache.org/licenses/LICENSE-2.0
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package main
-7
View File
@@ -1,7 +0,0 @@
#!/bin/sh
# Linux
#genisoimage -output seed.iso -volid cidata -joliet -rock user-data meta-data
# MacOS
mkisofs -output seed.iso -volid cidata -joliet -rock meta-data user-data
-5
View File
@@ -1,5 +0,0 @@
#!/bin/sh
# Example to launch nbdkit
./server/nbdkit -p 10810 --pidfile /tmp/vb-vol-1-efi.pid ./plugins/golang/examples/ramdisk/nbdkit-goramdisk-plugin.so -v -f size=67108864 volume=vol-50c968e8dbcc74bd2-efi bucket=predastore region=ap-southeast-2 access_key="AKIAIOSFODNN7EXAMPLE" secret_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" base_dir="/tmp/vb/" host="https://127.0.0.1:8443" cache_size=1
-28
View File
@@ -1,28 +0,0 @@
#!/bin/sh
qemu-system-aarch64 \
-nographic \
-M virt,highmem=off \
-accel hvf \
-cpu host \
-smp 4 \
-m 3000 \
-drive file=/Users/benduncan/qemu/QEMU_EFI.img,if=pflash,format=raw \
-device virtio-blk-pci,drive=debian,bootindex=1 \
-drive if=none,media=disk,id=debian,format=raw,file=nbd://192.168.64.5:10809/default \
-drive format=raw,file=nbd://192.168.64.5:10810/default,if=pflash \
-drive file=nbd://192.168.64.5:10811/default,format=raw,if=virtio \
-device qemu-xhci \
-device usb-kbd \
-device usb-tablet \
-device intel-hda \
-device hda-duplex \
-netdev user,id=net0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=net0 \
-device virtio-rng-pci
# ./server/nbdkit -p 10809 --pidfile /tmp/vb-vol-1.pid ./plugins/golang/examples/ramdisk/nbdkit-goramdisk-plugin.so -v -f size=4294967296 volume=vol-e660ff0065ad21f74 bucket=predastore region=ap-southeast-2 access_key="AKIAIOSFODNN7EXAMPLE" secret_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" base_dir="/tmp/vb/" host="https://127.0.0.1:8443" cache_size=20
# ./server/nbdkit -p 10810 --pidfile /tmp/vb-vol-1-efi.pid ./plugins/golang/examples/ramdisk/nbdkit-goramdisk-plugin.so -v -f size=67108864 volume=vol-e660ff0065ad21f74-efi bucket=predastore region=ap-southeast-2 access_key="AKIAIOSFODNN7EXAMPLE" secret_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" base_dir="/tmp/vb/" host="https://127.0.0.1:8443" cache_size=1
# ./server/nbdkit -p 10811 --pidfile /tmp/vb-vol-1-cloudinit.pid ./plugins/golang/examples/ramdisk/nbdkit-goramdisk-plugin.so -v -f size=1048576 volume=vol-e660ff0065ad21f74-cloudinit bucket=predastore region=ap-southeast-2 access_key="AKIAIOSFODNN7EXAMPLE" secret_key="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" base_dir="/tmp/vb/" host="https://127.0.0.1:8443" cache_size=1
+7 -5
View File
@@ -80,12 +80,12 @@ Operational commands for inspecting cluster state. These fan out NATS requests t
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `run-instances` | `--image-id`, `--instance-type`, `--count` (Min/MaxCount), `--key-name`, `--user-data`, `--subnet-id` (auto-creates ENI, assigns private IP), `--block-device-mappings` (DeviceName, VolumeSize, VolumeType, Iops, DeleteOnTermination) | `--security-group-ids`, `--placement`, `--tag-specifications`, `--dry-run`, `--client-token`, `--disable-api-termination`, `--ebs-optimized`, `--iam-instance-profile`, `--network-interfaces`, `--private-ip-address`, `--monitoring`, `--credit-specification`, `--cpu-options`, `--metadata-options`, `--launch-template`, `--hibernate-options` | `describe-images` (AMI must exist), `create-key-pair` (optional), VPC/SG (optional) | Gateway parses AWS query → NATS `ec2.runinstances` → daemon creates QEMU/KVM VM with viperblock-backed root volume via NBD → if SubnetId provided, auto-creates ENI with private IP → cloud-init injects user-data/keys → returns reservation with instance ID | 1. Launch with valid AMI and key pair<br>2. Launch with invalid AMI ID (error)<br>3. Launch with block device mappings (custom volume size)<br>4. Launch multiple instances (MinCount/MaxCount)<br>5. Launch with subnet-id (auto-creates ENI)<br>6. Invalid instance type returns error | **DONE** |
| `run-instances` | `--image-id`, `--instance-type`, `--count` (Min/MaxCount), `--key-name`, `--user-data`, `--subnet-id` (auto-creates ENI, assigns private IP), `--block-device-mappings` (DeviceName, VolumeSize, VolumeType, Iops, DeleteOnTermination), `--placement` (GroupName only — routes via spread or cluster strategy) | `--security-group-ids`, `--tag-specifications`, `--dry-run`, `--client-token`, `--disable-api-termination`, `--ebs-optimized`, `--iam-instance-profile`, `--network-interfaces`, `--private-ip-address`, `--monitoring`, `--credit-specification`, `--cpu-options`, `--metadata-options`, `--launch-template`, `--hibernate-options` | `describe-images` (AMI must exist), `create-key-pair` (optional), VPC/SG (optional) | Gateway parses AWS query → if Placement.GroupName set, looks up strategy: spread → `distributeInstancesSpread()` (1 instance per node, atomic CAS reservation), cluster → `distributeInstancesCluster()` (pin all to single node); otherwise NATS `ec2.runinstances` → daemon creates QEMU/KVM VM with viperblock-backed root volume via NBD → if SubnetId provided, auto-creates ENI with private IP → cloud-init injects user-data/keys → on termination, removes instance from placement group → returns reservation with instance ID | 1. Launch with valid AMI and key pair<br>2. Launch with invalid AMI ID (error)<br>3. Launch with block device mappings (custom volume size)<br>4. Launch multiple instances (MinCount/MaxCount)<br>5. Launch with subnet-id (auto-creates ENI)<br>6. Invalid instance type returns error<br>7. Launch with spread placement group (1 per node)<br>8. Launch with cluster placement group (all on one node)<br>9. Insufficient capacity for placement group (error) | **DONE** |
| `describe-instances` | `--instance-ids` | `--filters`, `--max-results`, `--next-token`, `--dry-run` | None | Gateway fans out NATS `ec2.DescribeInstances` to all nodes (no queue group) → each daemon returns local instances → gateway aggregates and returns combined list | 1. Describe all instances (no filter)<br>2. Describe by instance ID<br>3. Describe with filters (e.g. instance-state-name)<br>4. Instance not found returns empty set<br>5. Multi-node aggregation returns instances from all nodes | **DONE** |
| `start-instances` | `--instance-ids` | `--dry-run`, `--force` | `run-instances` (instance must exist in stopped state) | Gateway sends NATS `ec2.cmd.{instance-id}` → daemon restarts stopped QEMU process with same config → state transitions stopped→pending→running | 1. Start a stopped instance<br>2. Start already-running instance (error: IncorrectInstanceState)<br>3. Start with invalid instance ID<br>4. Verify volumes re-mount on start | **DONE** |
| `stop-instances` | `--instance-ids` | `--force`, `--hibernate`, `--dry-run` | `run-instances` (instance must be running) | Gateway sends NATS to target node → daemon issues QMP `system_powerdown` for graceful shutdown → monitors heartbeat until QEMU exits → state transitions running→stopping→stopped | 1. Graceful stop of running instance<br>2. Force stop (kills QEMU process)<br>3. Stop already-stopped instance (error)<br>4. Verify ~30s heartbeat detection | **DONE** |
| `terminate-instances` | `--instance-ids`, `DeleteOnTermination` (per-volume flag, default true) | `--dry-run` | `run-instances` (instance must exist) | Gateway sends NATS to target node → daemon kills QEMU process → cleans up NBD mounts → deletes volumes with `DeleteOnTermination=true` via `volumeService.DeleteVolume()` (S3 cleanup of vol/, vol-efi/, vol-cloudinit/) → internal volumes (EFI, cloud-init) always cleaned up via `ebs.delete` NATS → volumes with `DeleteOnTermination=false` left in available state → state→terminated | 1. Terminate running instance<br>2. Terminate stopped instance<br>3. Terminate with DeleteOnTermination=true deletes volumes<br>4. Terminate with DeleteOnTermination=false preserves volumes<br>5. Terminate already-terminated (idempotent)<br>6. Internal volumes (EFI, cloud-init) always cleaned up<br>7. Invalid instance ID | **DONE** |
| `reboot-instances` | — | `--instance-ids`, `--dry-run` | `run-instances` (instance must be running) | Gateway sends NATS to target node → daemon issues QMP `system_reset` → instance reboots without stopping | 1. Reboot running instance<br>2. Reboot stopped instance (error)<br>3. Verify instance stays in running state after reboot | **NOT STARTED** (parser test exists) |
| `reboot-instances` | `--instance-ids` | `--dry-run` | `run-instances` (instance must be running) | Gateway validates instance IDs → sends EC2InstanceCommand with `RebootInstance=true` via NATS `ec2.cmd.{instanceId}` → daemon validates instance is in StateRunning (returns IncorrectInstanceState if stopped) → issues QMP `system_reset` → instance reboots without stopping, stays in running state | 1. Reboot running instance<br>2. Reboot multiple instances<br>3. Reboot stopped instance (error: IncorrectInstanceState)<br>4. Instance not found (error: InvalidInstanceID.NotFound)<br>5. Verify instance stays in running state after reboot | **DONE** |
| `describe-instance-types` | `--filters` (capacity filter only) | `--instance-types`, `--max-results`, `--next-token`, `--dry-run`, all other filters | None | Gateway fans out NATS `ec2.DescribeInstanceTypes` to all nodes → each daemon reports supported types (t3.micro/small/medium/large) with vCPU/memory specs → gateway deduplicates and returns | 1. List all instance types<br>2. Filter by specific type<br>3. Filter with `capacity=true` shows available slots<br>4. Verify vCPU/memory specs match hardware | **DONE** |
| `modify-instance-attribute` | `--instance-id`, `--instance-type`, `--user-data` | `--disable-api-termination`, `--ebs-optimized`, `--source-dest-check`, `--instance-initiated-shutdown-behavior`, `--block-device-mappings`, `--groups`, `--ena-support`, `--sriov-net-support` | Instance must be stopped (in NATS KV) | Gateway validates input (exactly one attribute per call, instance ID format) → NATS `ec2.ModifyInstanceAttribute` with `spinifex-workers` queue group → daemon loads stopped instance from JetStream KV → applies attribute change → writes back to KV → returns `{}` on success. **InstanceType**: updates vm.InstanceType, Config, and Instance fields; clears StateReason (enables recovery from instance-type-missing bug). **UserData**: stores decoded content in vm.UserData and re-encodes to base64 for RunInstancesInput (cloud-init on next start). No instance type pre-validation (matches AWS — invalid types accepted, fail at StartInstances time). | 1. Change instance type while stopped<br>2. Change user data while stopped<br>3. Modify running instance (error: NotFound — running instances not in KV)<br>4. Instance not found (error: InvalidInstanceID.NotFound)<br>5. Instance not stopped (error: IncorrectInstanceState)<br>6. Invalid instance type accepted (fails on start with InsufficientInstanceCapacity)<br>7. StateReason cleared on type change (recovery from capacity-unavailable)<br>8. Missing/malformed instance ID (error: InvalidInstanceID.Malformed)<br>9. No attribute set (error: InvalidParameterValue)<br>10. Multiple attributes in one call (error: InvalidParameterValue) | **DONE** |
| `get-console-output` | `--instance-id` | `--latest` (always returns latest), `--dry-run` | Instance must be running on a node | Gateway sends NATS `ec2.{instanceId}.GetConsoleOutput` (per-instance topic, routed to owning node) → daemon reads console log file from disk → returns last 64KB base64-encoded with timestamp. Always available regardless of serial console access setting (matches AWS behavior). | 1. Get output from running instance<br>2. Empty log file returns empty output<br>3. Instance not found (error: InvalidInstanceID.NotFound) | **DONE** |
@@ -192,11 +192,13 @@ Gateway routing and KV CRUD are wired up but no OVN/OVS integration exists. EIGW
### EC2 - Placement Groups
Placement groups are stored in `spinifex-placement-groups` JetStream KV bucket, keyed by `{accountID}.{groupName}`. All mutations use CAS (compare-and-swap) with 5-retry limit for optimistic concurrency. Supported strategies: **spread** (strict 1 instance per node) and **cluster** (all instances pinned to single node). Partition strategy is rejected. Internal NATS operations (ReserveSpreadNodes, FinalizeSpreadInstances, ReleaseSpreadNodes, RemoveInstanceFromPlacementGroup, ReserveClusterNode, FinalizeClusterInstances) support atomic multi-node coordination for RunInstances and TerminateInstances.
| Command | Implemented Flags | Missing Flags | Prerequisites | Basic Logic | Test Cases | Status |
|---------|-------------------|---------------|---------------|-------------|------------|--------|
| `create-placement-group` | — | `--group-name`, `--strategy` (cluster/spread/partition), `--partition-count`, `--spread-level`, `--tag-specifications` | None | NATS `ec2.CreatePlacementGroup` → daemon stores placement group in JetStream KV with strategy, state=available → return group metadata | 1. Create with cluster strategy<br>2. Create with spread strategy<br>3. Create with partition strategy + partition count<br>4. Duplicate name (error)<br>5. Invalid strategy (error) | **NOT STARTED** |
| `delete-placement-group` | — | `--group-name`, `--dry-run` | Group must exist, no instances in group | NATS `ec2.DeletePlacementGroup` → daemon verifies group exists and is empty → delete from KV → return success | 1. Delete empty group<br>2. Delete group with instances (error)<br>3. Delete non-existent group (error) | **NOT STARTED** |
| `describe-placement-groups` | — | `--group-names`, `--group-ids`, `--filters` | None | NATS `ec2.DescribePlacementGroups` → daemon lists groups from KV → return list with strategy and state | 1. List all groups<br>2. Filter by name<br>3. Filter by strategy | **NOT STARTED** |
| `create-placement-group` | `--group-name`, `--strategy` (spread/cluster) | `--partition-count`, `--spread-level` (hardcoded to "host"), `--tag-specifications`, `--dry-run` | None | Gateway validates GroupName + Strategy → NATS `ec2.CreatePlacementGroup` with `spinifex-workers` queue group → daemon generates pg-ID (pg-{16hex}) → stores PlacementGroupRecord in `spinifex-placement-groups` KV with state=available, SpreadLevel="host" for spread → rejects partition strategy (InvalidParameterValue) → rejects duplicate names (InvalidPlacementGroup.Duplicate) → returns PlacementGroup | 1. Create with spread strategy<br>2. Create with cluster strategy<br>3. Create with partition strategy (error: InvalidParameterValue)<br>4. Duplicate name (error: InvalidPlacementGroup.Duplicate)<br>5. Missing GroupName (error)<br>6. Missing Strategy (error) | **DONE** |
| `delete-placement-group` | `--group-name` | `--dry-run` | Group must exist, no instances in group | NATS `ec2.DeletePlacementGroup` → daemon verifies group exists (InvalidPlacementGroup.Unknown) → counts instances in NodeInstances map → rejects if non-empty (InvalidPlacementGroup.InUse) → deletes from KV → returns success | 1. Delete empty group<br>2. Delete group with instances (error: InvalidPlacementGroup.InUse)<br>3. Delete non-existent group (error: InvalidPlacementGroup.Unknown) | **DONE** |
| `describe-placement-groups` | `--group-names`, `--group-ids`, `--filters` (strategy, state, spread-level, group-name) | `--dry-run` | None | NATS `ec2.DescribePlacementGroups` → daemon lists all keys from `spinifex-placement-groups` KV → filters by GroupNames, GroupIds, or Filters → returns list with Strategy, State, SpreadLevel, GroupId | 1. List all groups<br>2. Filter by name<br>3. Filter by group ID<br>4. Filter by strategy<br>5. Filter by state<br>6. Empty result when no groups exist | **DONE** |
### EC2 - Dedicated Hosts
+1 -1
View File
@@ -44,7 +44,7 @@ aws sts get-caller-identity
## Troubleshooting
## Credentials not working
## Credentials Not Working
Verify the AWS CLI configuration files exist and contain the correct profile:
+2 -12
View File
@@ -30,16 +30,6 @@ resources:
This guide walks through the complete workflow for creating a production-ready Spinifex environment.
**Steps:**
1. Create a user account
2. Import or create SSH keys
3. Create a VPC and subnet
4. Select and import an AMI
5. Launch EC2 instances
6. Connect via SSH
7. Monitor instance health
## Instructions
## 1. Import SSH Key
@@ -88,7 +78,7 @@ ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
## Troubleshooting
## Instance stuck in pending
## Instance Stuck in Pending
Check the Spinifex daemon and QEMU logs for boot errors:
@@ -103,7 +93,7 @@ Verify the AMI exists and architecture matches your host:
aws ec2 describe-images --image-ids $SPINIFEX_AMI
```
## SSH connection refused
## SSH Connection Refused
cloud-init takes 30-60 seconds to configure networking and SSH after boot. Wait and retry.
+2 -2
View File
@@ -51,7 +51,7 @@ rsync -avz /data/ user@cloud-server:/backup/spinifex-data/
## Troubleshooting
## Sync fails mid-transfer
## Sync Fails Mid Transfer
S3 sync is idempotent — re-run the same command to resume where it left off:
@@ -62,7 +62,7 @@ aws s3 sync s3://local-bucket/ s3://cloud-bucket/ \
Only changed or missing files will be transferred on subsequent runs.
## Permissions errors during sync
## Permission Errors During Sync
Verify your AWS credentials are configured for both the source and destination:
+546
View File
@@ -0,0 +1,546 @@
---
title: "IAM Users and Policies"
description: "Create IAM users, manage access keys, and control permissions with policies."
category: "Administration"
tags:
- iam
- users
- policies
- access keys
- security
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "AWS IAM CLI Reference"
url: "https://docs.aws.amazon.com/cli/latest/reference/iam/"
- title: "IAM Policy Reference"
url: "https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html"
---
# IAM Users and Policies
> Create IAM users, manage access keys, and control permissions with policies.
## Table of Contents
- [Overview](#overview)
- [Prerequisites](#prerequisites)
- [Users](#users)
- [Access Keys](#access-keys)
- [Policies](#policies)
- [Attaching Policies](#attaching-policies)
- [Putting It All Together](#putting-it-all-together)
- [Command Reference](#command-reference)
- [Troubleshooting](#troubleshooting)
---
## Overview
Spinifex implements AWS-compatible IAM with 16 operations covering user management, access key lifecycle, policy CRUD, and policy attachment. All IAM resources are scoped to the account that creates them — users in one account cannot see or modify resources in another.
When you create an account with `spx admin account create`, Spinifex bootstraps a root user with an `AdministratorAccess` policy and writes the credentials to `~/.aws/credentials`. From there, you use the standard AWS CLI to manage additional users and permissions.
**How authentication works:** Every AWS CLI request is signed with SigV4 using an access key pair. The gateway verifies the signature, resolves the caller's account, and evaluates attached policies before routing the request. The root user (account `000000000000`) bypasses policy evaluation entirely.
## Instructions
## Prerequisites
- A running Spinifex cluster with the AWS gateway started
- An account created via `spx admin account create` (see [Creating Accounts](/docs/creating-accounts))
- AWS CLI configured with the account's profile:
```bash
export AWS_PROFILE=spinifex-myteam
aws sts get-caller-identity
```
## Users
### Create a User
```bash
aws iam create-user --user-name alice
```
The response includes the user's ARN, unique ID, and creation date:
```json
{
"User": {
"UserName": "alice",
"UserId": "AIDA1A2B3C4D5E6F7890",
"Arn": "arn:aws:iam::000000000001:user/alice",
"Path": "/",
"CreateDate": "2026-03-24T10:00:00Z"
}
}
```
Use `--path` to organise users into hierarchical groups:
```bash
aws iam create-user --user-name bob --path /developers/
```
### Get a User
```bash
aws iam get-user --user-name alice
```
### List Users
```bash
aws iam list-users
```
Filter by path prefix:
```bash
aws iam list-users --path-prefix /developers/
```
### Delete a User
Before deleting a user, remove all access keys and detach all policies:
```bash
# Remove access keys
aws iam list-access-keys --user-name alice
aws iam delete-access-key --user-name alice --access-key-id AKIA...
# Detach policies
aws iam list-attached-user-policies --user-name alice
aws iam detach-user-policy --user-name alice \
--policy-arn arn:aws:iam::000000000001:policy/MyPolicy
# Now delete
aws iam delete-user --user-name alice
```
## Access Keys
Access keys are how users authenticate with the AWS CLI. Each user can have up to **2 access keys** at a time, allowing key rotation without downtime.
### Create an Access Key
```bash
aws iam create-access-key --user-name alice
```
```json
{
"AccessKey": {
"UserName": "alice",
"AccessKeyId": "AKIA1A2B3C4D5E6F7890ABCD",
"Status": "Active",
"SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"CreateDate": "2026-03-24T10:05:00Z"
}
}
```
> **The secret access key is only shown once.** Save it immediately. If lost, delete the key and create a new one.
Add the credentials to `~/.aws/credentials` under a new profile and set `AWS_PROFILE` to use them.
### List Access Keys
```bash
aws iam list-access-keys --user-name alice
```
### Rotate Access Keys
Create a second key, update your configuration, then delete the old one:
```bash
# 1. Create new key (while old key still works)
aws iam create-access-key --user-name alice
# 2. Update ~/.aws/credentials with the new key
# 3. Verify the new key works
AWS_PROFILE=spinifex-alice aws sts get-caller-identity
# 4. Delete the old key
aws iam delete-access-key --user-name alice --access-key-id AKIA_OLD_KEY_ID
```
### Deactivate an Access Key
Temporarily disable a key without deleting it:
```bash
aws iam update-access-key --user-name alice \
--access-key-id AKIA1A2B3C4D5E6F7890ABCD \
--status Inactive
```
Reactivate it later:
```bash
aws iam update-access-key --user-name alice \
--access-key-id AKIA1A2B3C4D5E6F7890ABCD \
--status Active
```
## Policies
Policies are JSON documents that define what actions a user can perform on which resources.
### Policy Document Format
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["ec2:RunInstances", "ec2:DescribeInstances"],
"Resource": "*"
}
]
}
```
Each statement has:
- **Effect** — `Allow` or `Deny`
- **Action** — Service actions (e.g. `ec2:RunInstances`, `s3:GetObject`). Supports wildcards: `ec2:*`, `s3:Get*`, or `*` for all actions.
- **Resource** — Target resources. Use `*` for all resources.
**Evaluation order:** An explicit `Deny` always wins. If no statement matches, access is denied by default.
### Create a Policy
Save the policy document to a file, then create the policy:
```bash
cat > /tmp/ec2-readonly.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:DescribeImages",
"ec2:DescribeKeyPairs",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeVpcs"
],
"Resource": "*"
}
]
}
EOF
aws iam create-policy \
--policy-name EC2ReadOnly \
--policy-document file:///tmp/ec2-readonly.json
```
```json
{
"Policy": {
"PolicyName": "EC2ReadOnly",
"PolicyId": "ANPA1A2B3C4D5E6F7890",
"Arn": "arn:aws:iam::000000000001:policy/EC2ReadOnly",
"Path": "/",
"DefaultVersionId": "v1",
"CreateDate": "2026-03-24T10:10:00Z"
}
}
```
### Common Policy Examples
**Full administrator access:**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}
```
**S3 read/write (for hybrid sync, backups):**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket",
"s3:DeleteObject"
],
"Resource": "*"
}
]
}
```
**EC2 operator (launch and manage instances, no VPC changes):**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:RunInstances",
"ec2:StartInstances",
"ec2:StopInstances",
"ec2:TerminateInstances",
"ec2:DescribeInstances",
"ec2:DescribeImages"
],
"Resource": "*"
}
]
}
```
**Deny terminate (attach alongside a broader Allow to prevent accidental termination):**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Action": "ec2:TerminateInstances",
"Resource": "*"
}
]
}
```
### Get a Policy
```bash
aws iam get-policy \
--policy-arn arn:aws:iam::000000000001:policy/EC2ReadOnly
```
### Get the Policy Document
Use `get-policy-version` with version `v1` to retrieve the actual JSON document:
```bash
aws iam get-policy-version \
--policy-arn arn:aws:iam::000000000001:policy/EC2ReadOnly \
--version-id v1
```
### List Policies
```bash
aws iam list-policies
```
### Delete a Policy
A policy must be detached from all users before it can be deleted:
```bash
aws iam delete-policy \
--policy-arn arn:aws:iam::000000000001:policy/EC2ReadOnly
```
## Attaching Policies
Policies have no effect until attached to a user.
### Attach a Policy to a User
```bash
aws iam attach-user-policy --user-name alice \
--policy-arn arn:aws:iam::000000000001:policy/EC2ReadOnly
```
### List a User's Policies
```bash
aws iam list-attached-user-policies --user-name alice
```
### Detach a Policy
```bash
aws iam detach-user-policy --user-name alice \
--policy-arn arn:aws:iam::000000000001:policy/EC2ReadOnly
```
## Putting It All Together
A complete workflow for onboarding a developer with scoped EC2 and S3 access:
```bash
# 1. Create the user
aws iam create-user --user-name dev-carol --path /developers/
# 2. Create an access key
aws iam create-access-key --user-name dev-carol
# Save the AccessKeyId and SecretAccessKey from the output
# 3. Create a scoped policy
cat > /tmp/dev-policy.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2Access",
"Effect": "Allow",
"Action": ["ec2:*"],
"Resource": "*"
},
{
"Sid": "S3ReadOnly",
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:ListBucket"],
"Resource": "*"
},
{
"Sid": "NoTerminate",
"Effect": "Deny",
"Action": "ec2:TerminateInstances",
"Resource": "*"
}
]
}
EOF
aws iam create-policy \
--policy-name DeveloperAccess \
--policy-document file:///tmp/dev-policy.json
# 4. Attach the policy
aws iam attach-user-policy --user-name dev-carol \
--policy-arn arn:aws:iam::000000000001:policy/DeveloperAccess
# 5. Configure the developer's AWS CLI
aws configure --profile spinifex-carol
# Enter the access key and secret from step 2
# 6. Verify
AWS_PROFILE=spinifex-carol aws ec2 describe-instances
AWS_PROFILE=spinifex-carol aws ec2 terminate-instances --instance-ids i-123
# ^ This will be denied by the NoTerminate statement
```
## Command Reference
| Command | Key Parameters | Notes |
|---------|---------------|-------|
| `create-user` | `--user-name`, `--path` | Path default: `/` |
| `get-user` | `--user-name` | |
| `list-users` | `--path-prefix` | |
| `delete-user` | `--user-name` | Must have no keys or policies |
| `create-access-key` | `--user-name` | Max 2 per user; secret shown once |
| `list-access-keys` | `--user-name` | |
| `delete-access-key` | `--user-name`, `--access-key-id` | |
| `update-access-key` | `--user-name`, `--access-key-id`, `--status` | `Active` or `Inactive` |
| `create-policy` | `--policy-name`, `--policy-document` | Max 6144 bytes |
| `get-policy` | `--policy-arn` | Metadata only |
| `get-policy-version` | `--policy-arn`, `--version-id` | Use `v1`; includes document |
| `list-policies` | | |
| `delete-policy` | `--policy-arn` | Must not be attached |
| `attach-user-policy` | `--user-name`, `--policy-arn` | Idempotent |
| `detach-user-policy` | `--user-name`, `--policy-arn` | |
| `list-attached-user-policies` | `--user-name` | |
## Troubleshooting
### AccessDenied on IAM Commands
The calling user must have IAM permissions. Attach a policy with `iam:*` actions, or use the root account profile:
```bash
export AWS_PROFILE=spinifex-myteam
aws iam list-users
```
### InvalidClientTokenId
The access key is either inactive or does not exist. Check the key status:
```bash
aws iam list-access-keys --user-name alice
```
If the key shows `Inactive`, reactivate it:
```bash
aws iam update-access-key --user-name alice \
--access-key-id AKIA... --status Active
```
If the key was deleted, create a new one.
### DeleteConflict When Deleting a User
The user still has access keys or attached policies. Remove them first:
```bash
# Check for access keys
aws iam list-access-keys --user-name alice
# Check for attached policies
aws iam list-attached-user-policies --user-name alice
```
### DeleteConflict When Deleting a Policy
The policy is still attached to one or more users. Detach it from all users before deleting:
```bash
# Find who has it attached, then detach
aws iam detach-user-policy --user-name alice \
--policy-arn arn:aws:iam::000000000001:policy/MyPolicy
aws iam delete-policy \
--policy-arn arn:aws:iam::000000000001:policy/MyPolicy
```
### LimitExceeded When Creating Access Keys
Each user can have at most 2 access keys. Delete an existing key before creating a new one:
```bash
aws iam list-access-keys --user-name alice
aws iam delete-access-key --user-name alice --access-key-id AKIA_OLD
aws iam create-access-key --user-name alice
```
### MalformedPolicyDocument
The policy JSON is invalid. Check that:
- `Version` is exactly `"2012-10-17"`
- At least one `Statement` exists
- Each statement has `Effect` (`Allow`/`Deny`), `Action`, and `Resource`
- The document is under 6144 bytes
- The JSON is well-formed (no trailing commas, correct quoting)
+7 -8
View File
@@ -6,7 +6,6 @@ tags:
- install
- air-gapped
- offline
badge: advanced
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
@@ -38,7 +37,7 @@ In air-gapped or disconnected environments, Spinifex can be deployed without int
## Instructions
## Step 1. Prepare packages (on a connected machine)
## Step 1. Prepare Packages (on a connected machine)
Set up a local APT cache to collect all required packages:
@@ -56,7 +55,7 @@ sudo apt install --download-only nbdkit nbdkit-plugin-dev pkg-config \
ovn-central ovn-host openvswitch-switch
```
## Step 2. Create USB deployment media
## Step 2. Create USB Deployment Media
```bash
mkdir -p /media/spinifex-deploy/{apt-packages,go-cache,spinifex-source}
@@ -65,7 +64,7 @@ cp -r ~/go/pkg/mod/cache/ /media/spinifex-deploy/go-cache/
cp -r ~/Development/mulga/spinifex /media/spinifex-deploy/spinifex-source/
```
## Step 3. Verify package integrity
## Step 3. Verify Package Integrity
Sign and verify packages before transferring to the target:
@@ -75,7 +74,7 @@ gpg --verify spinifex-v1.0.tar.gz.sig spinifex-v1.0.tar.gz
sha256sum -c /media/spinifex-deploy/checksums.sha256
```
## Step 4. Install on the target server
## Step 4. Install on the Target Server
Mount the USB and install:
@@ -90,7 +89,7 @@ Then follow the [Source Install](/docs/source-install) guide from Step 3 (Setup
## Troubleshooting
## Missing dependencies after dpkg install
### Missing Dependencies After dpkg Install
Some packages may have unresolved dependencies. Fix with:
@@ -101,7 +100,7 @@ sudo apt-get install -f
The `-f` flag tells apt to fix broken dependencies using what's available locally.
## GPG verification fails
### GPG Verification Fails
The signing key may not match or the download was corrupted. Re-import the key and verify:
@@ -116,7 +115,7 @@ If verification still fails, re-download the package on the connected machine an
sha256sum spinifex-v1.0.tar.gz
```
## Go module cache errors
### Go Module Cache Errors
The Go module cache may have been incompletely copied. On the connected machine, re-export the full cache:
+74 -185
View File
@@ -6,7 +6,6 @@ tags:
- install
- multi node
- cluster
badge: availability-zone
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
@@ -30,259 +29,149 @@ resources:
## Overview
A Spinifex cluster operates as a fully distributed infrastructure region — similar to an AWS Region — where multiple nodes provide high availability, data durability, and fault tolerance.
**Services distributed across nodes:**
- **NATS** — Clustered message bus for request routing and JetStream replication
- **Predastore** — Raft consensus with Reed-Solomon erasure coding (RS 2+1)
- **Viperblock** — Block storage co-located with compute nodes
- **Spinifex Daemon** — Independent AWS API request serving per node
Cluster formation is automatic. When initializing with `--nodes 3`, the init node starts a formation server and waits for peers to join. Once all nodes register, each receives the full cluster topology — credentials, CA certificates, NATS routes, Predastore peer lists — no manual configuration synchronization required.
A Spinifex cluster distributes services across multiple servers for high availability, data durability, and fault tolerance. Cluster formation is automatic — the init node waits for peers to join, then distributes credentials, CA certificates, and configuration.
**Network requirements:**
- Minimum 1 NIC per server
- 2 NICs recommended for production (management + overlay)
- Minimum 1 NIC per server (2 recommended for production)
- UDP port 6081 open between hosts (Geneve tunnels)
- TCP ports 4222, 4248, 6641, 6642 open between hosts (NATS, OVN)
## Instructions
## Step 1. Install Spinifex on each server
Run the binary installer on **every server** in the cluster:
## Step 1. Install Spinifex on Each Server
```bash
curl https://install.mulgadc.com | bash
```
## Step 2. Set node IP variables
## Step 2. Set Node IP Variables
On **each server**, export the IPs for all nodes (replace with your actual IPs):
On **each server**, export the management IPs for all nodes:
```bash
export SPINIFEX_NODE1=192.168.1.10
export SPINIFEX_NODE2=192.168.1.11
export SPINIFEX_NODE3=192.168.1.12
```
To find your server's IP:
```bash
ip addr show | grep "inet "
```
Next, define the AWS region and AZ you are deploying locally.
```bash
export AWS_REGION=us-east-1
export AWS_AZ=us-east-1a
```
## Step 3. Setup OVN networking
## Step 3. Setup OVN Networking
OVN must be configured on every server before forming the cluster. Server 1 runs OVN central (NB and SB databases) and must be set up first.
Server 1 runs OVN central and must be set up first. If your WAN interface is already a bridge (e.g. `br-wan`), setup-ovn.sh auto-detects it. Otherwise use `--wan-bridge=br-wan --wan-iface=eth1` (dedicated WAN NIC) or `--macvlan --wan-iface=eth0` (single-NIC).
**Server 1 — OVN central + compute (run first):**
**Server 1:**
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --management --encap-ip=$SPINIFEX_NODE1
sudo /usr/local/share/spinifex/setup-ovn.sh \
--management \
--encap-ip=$SPINIFEX_NODE1
```
Verify OVN central is ready before proceeding:
**Server 2** (after server 1 is ready):
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh \
--ovn-remote=tcp:$SPINIFEX_NODE1:6642 \
--encap-ip=$SPINIFEX_NODE2
```
**Server 3** (after server 1 is ready):
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh \
--ovn-remote=tcp:$SPINIFEX_NODE1:6642 \
--encap-ip=$SPINIFEX_NODE3
```
Verify all 3 chassis registered:
```bash
sudo ovn-sbctl show
```
**Server 2 — Compute node (after server 1 is ready):**
## Step 4. Form the Cluster
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --ovn-remote=tcp:$SPINIFEX_NODE1:6642 --encap-ip=$SPINIFEX_NODE2
```
Run init and join concurrently — init blocks until all nodes join.
**Server 3 — Compute node (after server 1 is ready):**
```bash
sudo /usr/local/share/spinifex/setup-ovn.sh --ovn-remote=tcp:$SPINIFEX_NODE1:6642 --encap-ip=$SPINIFEX_NODE3
```
Verify all chassis have registered (from server 1):
```bash
sudo ovn-sbctl show
```
You should see 3 chassis entries, one per server, each with a Geneve encap IP.
## Step 4. Form the cluster
The formation process requires running init and join commands concurrently. The init node blocks until all expected nodes have joined.
**Server 1 — Initialize the cluster:**
**Server 1 — Initialize:**
```bash
sudo spx admin init \
--node node1 \
--nodes 3 \
--bind $SPINIFEX_NODE1 \
--cluster-bind $SPINIFEX_NODE1 \
--port 4432 \
--region $AWS_REGION \
--az $AWS_AZ
--node node1 --nodes 3 \
--bind $SPINIFEX_NODE1 --cluster-bind $SPINIFEX_NODE1 \
--port 4432 --region $AWS_REGION --az $AWS_AZ
```
**Server 2 — Join the cluster (while init is running):**
Save the admin credentials — they will not be shown again.
**Server 2 — Join** (while init is running):
```bash
spx admin join \
--node node2 \
--bind $SPINIFEX_NODE2 \
--cluster-bind $SPINIFEX_NODE2 \
--host $SPINIFEX_NODE1:4432 \
--region $AWS_REGION \
--az $AWS_AZ
sudo spx admin join \
--node node2 --bind $SPINIFEX_NODE2 --cluster-bind $SPINIFEX_NODE2 \
--host $SPINIFEX_NODE1:4432 --region $AWS_REGION --az $AWS_AZ
```
**Server 3 — Join the cluster (while init is running):**
**Server 3 — Join** (while init is running):
```bash
spx admin join \
--node node3 \
--bind $SPINIFEX_NODE3 \
--cluster-bind $SPINIFEX_NODE3 \
--host $SPINIFEX_NODE1:4432 \
--region $AWS_REGION \
--az $AWS_AZ
sudo spx admin join \
--node node3 --bind $SPINIFEX_NODE3 --cluster-bind $SPINIFEX_NODE3 \
--host $SPINIFEX_NODE1:4432 --region $AWS_REGION --az $AWS_AZ
```
All three processes will exit with a cluster summary once formation is complete.
## Step 5. Start Services
## Step 5. Start services and verify
Start services on **all servers**:
On **all servers**:
```bash
sudo systemctl start spinifex.target
```
From any node, verify the cluster:
## Step 6. Verify
From any node:
```bash
spx get nodes
export AWS_PROFILE=spinifex
aws ec2 describe-instance-types
```
```
NAME STATUS IP REGION AZ UPTIME VMs
node1 Ready 192.168.1.10 us-east-1 us-east-1a 2m 0
node2 Ready 192.168.1.11 us-east-1 us-east-1a 2m 0
node3 Ready 192.168.1.12 us-east-1 us-east-1a 2m 0
If this returns a list of available instance types, your cluster is working.
**Congratulations! Your Spinifex cluster is installed.**
Continue to [Setting Up Your Cluster](/docs/setting-up-your-cluster) to import an AMI, create a VPC, and launch your first instance.
## Troubleshooting
### Nodes Not Joining
The init command must still be running when join executes. If init exited, re-run with `--force`.
```bash
curl -s http://$SPINIFEX_NODE1:4432/health
```
## Step 6. Trust the CA certificate
### OVN Chassis Not Registering
On **each server**, trust the CA generated during init:
```bash
sudo ovn-sbctl show
sudo ss -tlnp | grep 6642
```
### CA Certificate Not Trusted
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
export AWS_PROFILE=spinifex
```
## Step 7. Launch instances across the cluster
Connect to any node's AWS Gateway — all nodes serve the same cluster state:
```bash
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 3
```
Spinifex distributes instances across nodes based on available capacity. Check which node each instance landed on:
```bash
spx get vms
```
## Shutdown
A graceful shutdown coordinates all nodes — drains VMs, flushes storage, and stops services:
```bash
spx admin cluster shutdown
```
## Troubleshooting
## Nodes not joining the cluster
The init command must still be running when you execute join on other servers. If init has already exited, re-run it.
Check that all servers can reach the init node on the formation port:
```bash
curl -s http://$SPINIFEX_NODE1:4432/health
```
## OVN chassis not registering
Compute nodes need to reach OVN central on server 1. Verify from server 1:
```bash
sudo ovn-sbctl show
```
If chassis are missing, check firewall rules for port 6642:
```bash
sudo ss -tlnp | grep 6642
```
Re-run the OVN setup on the affected compute node.
## Cross-host VMs cannot communicate
This indicates a Geneve tunnel issue. Verify tunnels are established:
### Cross-Host VMs Cannot Communicate
```bash
sudo ovs-vsctl show | grep -i geneve
```
Ensure UDP port 6081 is open between all hosts:
```bash
sudo ss -ulnp | grep 6081
```
If tunnels are missing, verify `--encap-ip` was set correctly during OVN setup.
## Health check fails on a node
Check if the daemon is running and listening on the correct IP:
```bash
curl -s http://$SPINIFEX_NODE1:4432/health
```
Review the daemon logs for errors:
```bash
cat ~/spinifex/logs/spinifex.log
```
Verify `--bind` was set to the correct IP during cluster formation.
## NATS cluster routing issues
Check NATS logs for connection errors between nodes:
```bash
grep -i "route\|cluster" ~/spinifex/logs/nats.log
```
Ensure `--cluster-bind` IPs are reachable between all servers.
+33 -106
View File
@@ -1,12 +1,11 @@
---
title: "Installing Spinifex"
description: "Install Spinifex on a single server using the binary installer. Get a working region with EC2, VPC, EBS, and S3 in minutes."
description: "Install Spinifex on a single server using the binary installer."
category: "Getting Started"
tags:
- install
- single node
- quickstart
badge: quickstart
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
@@ -37,7 +36,7 @@ This guide installs Spinifex on a single server using the binary installer. For
**Supported Operating Systems:**
- Ubuntu 22.04 / 24.04 / 25.10
- Debian 12.13
- Debian 12 / 13
**What gets installed:**
@@ -58,143 +57,71 @@ curl https://install.mulgadc.com | bash
The installer downloads the Spinifex binary and bootstraps all dependencies (QEMU, OVN/OVS, AWS CLI).
## Step 2. Initialize a region
## Step 2. Setup OVN Networking
Create your first region and availability zone, replace `ap-southeast-2` with your desired region:
If your WAN interface is already a bridge (e.g. `br-wan`), setup-ovn.sh auto-detects it:
```bash
spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
sudo /usr/local/share/spinifex/setup-ovn.sh --management
```
## Step 3. Trust the CA certificate
Spinifex generates a local CA during initialization. Add it to your system trust store:
If your WAN is a physical NIC, choose one:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
# Dedicated WAN NIC (not your SSH connection):
sudo /usr/local/share/spinifex/setup-ovn.sh --management --wan-bridge=br-wan --wan-iface=eth1
# Single-NIC host (SSH-safe macvlan):
sudo /usr/local/share/spinifex/setup-ovn.sh --management --macvlan --wan-iface=enp0s3
```
## Step 4. Start services
## Step 3. Initialize
```bash
sudo spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
```
This auto-detects your network topology, generates configuration and TLS certificates, installs the CA into the system trust store, and configures AWS CLI credentials. Save the admin credentials printed during init — they will not be shown again.
## Step 4. Start Services
```bash
sudo systemctl start spinifex.target
```
Set the AWS profile to use the default admin account:
## Step 5. Verify
```bash
export AWS_PROFILE=spinifex
aws ec2 describe-instance-types
```
## Step 5. Import an AMI
If this returns a list of available instance types, your installation is working.
List available images and import one matching your architecture:
**Congratulations! Spinifex is installed.**
```bash
spx admin images list
spx admin images import --name debian-12-arm64
```
## Step 6. Create a VPC and launch an instance
```bash
aws ec2 create-vpc --cidr-block 10.200.0.0/16
export SPINIFEX_VPC="vpc-XXX"
aws ec2 create-subnet --vpc-id $SPINIFEX_VPC --cidr-block 10.200.1.0/24
export SPINIFEX_SUBNET="subnet-XXX"
aws ec2 import-key-pair --key-name "spinifex-key" --public-key-material fileb://~/.ssh/id_rsa.pub
aws ec2 run-instances \
--image-id $SPINIFEX_AMI \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SPINIFEX_SUBNET \
--count 1
```
## Step 7. Connect via SSH
Find the forwarded SSH port and connect:
```bash
ps auxw | grep hostfwd
# Look for: hostfwd=tcp:127.0.0.1:<port>-:22
ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
```
Continue to [Setting Up Your Cluster](/docs/setting-up-your-cluster) to import an AMI, create a VPC, and launch your first instance.
## Troubleshooting
## spx command not found
The binary is not in your PATH. Run the installer again or add the install directory to your PATH:
### spx Command Not Found
```bash
export PATH=$PATH:~/spinifex/bin/
export PATH=$PATH:/usr/local/bin
```
Add this to your `~/.bashrc` or `~/.zshrc` for persistence.
### CA Certificate Not Trusted
## OVN services not starting
Check if the OVN controller is active:
`sudo spx admin init` installs the CA automatically. If you need to re-install it manually:
```bash
sudo systemctl is-active ovn-controller
sudo ovn-sbctl show
```
If inactive, re-run the OVN setup that the installer performs. Check system logs for details:
```bash
journalctl -u ovn-controller --no-pager -n 20
```
## CA certificate not trusted
AWS CLI will reject HTTPS connections if the Spinifex CA is not trusted. Re-add the certificate:
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo cp /etc/spinifex/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
Verify it was added:
### OVN Services Not Starting
```bash
ls -la /usr/local/share/ca-certificates/spinifex-ca.crt
```
## Instance stuck in pending
Check the Spinifex daemon logs for errors:
```bash
ls ~/spinifex/logs/
cat ~/spinifex/logs/spinifex.log
```
Verify the AMI was imported successfully:
```bash
aws ec2 describe-images
```
## SSH connection refused
cloud-init takes 30-60 seconds to configure the instance after boot. Wait and retry.
Verify the SSH port forwarding is active:
```bash
ps auxw | grep hostfwd
```
If no `hostfwd` entry appears, the instance may not have started correctly. Check instance state:
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
sudo systemctl is-active ovn-controller
journalctl -u ovn-controller --no-pager -n 20
```
+2 -2
View File
@@ -69,7 +69,7 @@ ssh -i ~/.ssh/spinifex-key ec2-user@127.0.0.1 -p <port>
## Troubleshooting
## Instance fails to boot
### Instance Fails to Boot
Check QEMU logs for the instance and verify the AMI architecture matches your host:
@@ -85,7 +85,7 @@ If the AMI is for a different architecture (e.g. arm64 on an x86_64 host), impor
./bin/spx admin images import --name debian-12-x86_64
```
## Cannot SSH into instance
### Cannot SSH Into Instance
cloud-init needs time to configure the instance after boot. Wait 30-60 seconds and retry.
+2 -3
View File
@@ -6,7 +6,6 @@ tags:
- migration
- aws
- terraform
badge: new
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
@@ -73,7 +72,7 @@ aws s3 sync s3://local-bucket/ s3://cloud-bucket/ --source-region spinifex --reg
## Troubleshooting
## Terraform provider errors
## Terraform Provider Errors
Ensure all three skip flags are set in your provider configuration:
@@ -85,7 +84,7 @@ skip_requesting_account_id = true
Without these, Terraform will try to validate credentials and metadata against real AWS endpoints.
## S3 signature errors
## S3 Signature Errors
Spinifex uses AWS Signature V4. Ensure your AWS CLI is version 2.0 or higher:
+336
View File
@@ -0,0 +1,336 @@
---
title: "Setting Up Your Cluster"
description: "Import an AMI, create a VPC with a public subnet, and launch your first EC2 instance on Spinifex."
category: "Getting Started"
tags:
- setup
- vpc
- ec2
- quickstart
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "AWS EC2 CLI Reference"
url: "https://docs.aws.amazon.com/cli/latest/reference/ec2/"
---
# Setting Up Your Cluster
> Import an AMI, create a VPC with a public subnet, and launch your first instance.
## Table of Contents
- [1. Import an AMI](#1-import-an-ami)
- [2. Create an SSH Key](#2-create-an-ssh-key)
- [3. Create a VPC and Public Subnet](#3-create-a-vpc-and-public-subnet)
- [4. Launch an Instance](#4-launch-an-instance)
- [5. Connect via SSH](#5-connect-via-ssh)
- [6. Managing Instances](#6-managing-instances)
- [Troubleshooting](#troubleshooting)
---
## Overview
This guide assumes Spinifex is already installed and running. If not, follow one of the installation guides first:
- [Binary Install (Single Node)](/docs/installing-spinifex)
- [Multi-Node Installation](/docs/installing-spinifex-multi-node)
- [Source Install](/docs/source-install)
## Instructions
## Prerequisites
Ensure the AWS profile is set:
```bash
export AWS_PROFILE=spinifex
```
## 1. Import an AMI
List available images and import one matching your architecture:
```bash
spx admin images list
```
```
NAME | DISTRO | VERSION | ARCH | BOOT
debian-12-arm64 | debian | 12 | arm64 | bios
debian-12-x86_64 | debian | 12 | x86_64 | bios
ubuntu-24.04-arm64 | ubuntu | 24.04 | arm64 | bios
ubuntu-24.04-x86_64 | ubuntu | 24.04 | x86_64 | bios
```
Import an image:
```bash
spx admin images import --name ubuntu-24.04-x86_64
```
Or import a local image file:
```bash
spx admin images import --file ~/images/ubuntu-24.04.img --distro ubuntu --version 24.04 --arch x86_64
```
Verify the import and note the AMI ID:
```bash
aws ec2 describe-images
```
## 2. Create an SSH Key
### Option A: Import an existing key
```bash
aws ec2 import-key-pair \
--key-name "spinifex-key" \
--public-key-material fileb://~/.ssh/id_rsa.pub
```
### Option B: Create a new key pair
```bash
aws ec2 create-key-pair --key-name spinifex-key \
| jq -r '.KeyMaterial | rtrimstr("\n")' > ~/.ssh/spinifex-key
chmod 600 ~/.ssh/spinifex-key
ssh-keygen -y -f ~/.ssh/spinifex-key > ~/.ssh/spinifex-key.pub
```
Verify:
```bash
aws ec2 describe-key-pairs
```
## 3. Create a VPC and Public Subnet
### Create a VPC
```bash
VPC_ID=$(aws ec2 create-vpc --cidr-block 10.200.0.0/16 \
--query 'Vpc.VpcId' --output text)
echo "VPC: $VPC_ID"
```
### Create an Internet Gateway
An Internet Gateway enables instances in public subnets to reach the internet and be reachable from the LAN/WAN.
```bash
IGW_ID=$(aws ec2 create-internet-gateway \
--query 'InternetGateway.InternetGatewayId' --output text)
aws ec2 attach-internet-gateway \
--internet-gateway-id $IGW_ID \
--vpc-id $VPC_ID
echo "IGW: $IGW_ID (attached to $VPC_ID)"
```
### Create a Public Subnet
A public subnet auto-assigns a routable IP to each instance, making it directly reachable from your network.
```bash
SUBNET_ID=$(aws ec2 create-subnet \
--vpc-id $VPC_ID \
--cidr-block 10.200.1.0/24 \
--query 'Subnet.SubnetId' --output text)
# Enable auto-assign public IP
aws ec2 modify-subnet-attribute \
--subnet-id $SUBNET_ID \
--map-public-ip-on-launch
echo "Subnet: $SUBNET_ID (public)"
```
### Verify
```bash
aws ec2 describe-vpcs --vpc-ids $VPC_ID
aws ec2 describe-subnets --subnet-ids $SUBNET_ID
```
## 4. Launch an Instance
Query available instance types for your hardware:
```bash
aws ec2 describe-instance-types
```
Launch an instance in the public subnet:
```bash
AMI_ID=$(aws ec2 describe-images --query 'Images[0].ImageId' --output text)
INSTANCE_ID=$(aws ec2 run-instances \
--image-id $AMI_ID \
--instance-type t3.small \
--key-name spinifex-key \
--subnet-id $SUBNET_ID \
--count 1 \
--query 'Instances[0].InstanceId' --output text)
echo "Instance: $INSTANCE_ID"
```
Wait for the instance to reach `running` state:
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID \
--query 'Reservations[0].Instances[0].[State.Name, PrivateIpAddress, PublicIpAddress]' \
--output text
```
Expected output:
```
running 10.200.1.4 192.168.1.155
```
The instance has both a private IP (VPC overlay) and a public IP (from your external pool, routable on your network).
## 5. Connect via SSH
SSH directly to the instance's public IP:
```bash
PUBLIC_IP=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID \
--query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
ssh -i ~/.ssh/spinifex-key ec2-user@$PUBLIC_IP
```
> **Note:** cloud-init takes 30-60 seconds to configure the instance. If SSH is refused, wait and retry.
Once connected, verify the instance has internet access:
```bash
curl -s http://ifconfig.me
```
This should return the instance's public IP or the gateway's SNAT address.
## 6. Managing Instances
### Stop
```bash
aws ec2 stop-instances --instance-ids $INSTANCE_ID
```
### Start
```bash
aws ec2 start-instances --instance-ids $INSTANCE_ID
```
### Terminate
```bash
aws ec2 terminate-instances --instance-ids $INSTANCE_ID
```
### Console Output
View the instance's serial console log (useful for debugging boot issues):
```bash
aws ec2 get-console-output --instance-id $INSTANCE_ID \
--query 'Output' --output text
```
### Multi-Node: Check Instance Placement
On a multi-node cluster, instances are distributed across nodes:
```bash
spx get vms
```
## Additional Options
### Private Subnets (No Public IP)
Create a subnet without `--map-public-ip-on-launch`:
```bash
PRIVATE_SUBNET=$(aws ec2 create-subnet \
--vpc-id $VPC_ID \
--cidr-block 10.200.2.0/24 \
--query 'Subnet.SubnetId' --output text)
```
Instances in private subnets get a private IP only. They can still reach the internet via SNAT through the VPC router, but are not directly reachable from your network.
### Multiple Accounts
Create isolated accounts with their own resources:
```bash
spx admin account create --name myteam
export AWS_PROFILE=spinifex-myteam
```
### Web UI
The Spinifex web interface is available at `https://localhost:3000` when services are running.
## Troubleshooting
### Instance Stuck in Pending
```bash
cat ~/spinifex/logs/spinifex.log
aws ec2 describe-images
```
### SSH Connection Refused
cloud-init needs 30-60 seconds after boot. Check instance state:
```bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID
```
### No Public IP Assigned
Verify the subnet has `MapPublicIpOnLaunch` enabled:
```bash
aws ec2 describe-subnets --subnet-ids $SUBNET_ID
```
If `MapPublicIpOnLaunch` is false:
```bash
aws ec2 modify-subnet-attribute --subnet-id $SUBNET_ID --map-public-ip-on-launch
```
Also verify an Internet Gateway is attached to the VPC:
```bash
aws ec2 describe-internet-gateways
```
### Instance Has No Internet Access
Check the VPC router's NAT rules (from the host):
```bash
sudo ovn-nbctl lr-nat-list $(sudo ovn-nbctl lr-list | awk '{print $2}' | head -1)
```
Verify the default route exists:
```bash
sudo ovn-nbctl lr-route-list $(sudo ovn-nbctl lr-list | awk '{print $2}' | head -1)
```
+52 -122
View File
@@ -1,23 +1,16 @@
---
title: "Source Install"
description: "Build Spinifex from source for development, custom builds, or contributing. Includes dependency setup, compilation, and OVN configuration."
description: "Build Spinifex from source for development, custom builds, or contributing."
category: "Getting Started"
tags:
- install
- source
- development
badge: source
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Predastore (S3)"
url: "https://github.com/mulgadc/predastore"
- title: "Viperblock (EBS)"
url: "https://github.com/mulgadc/viperblock"
- title: "Go Downloads"
url: "https://go.dev/dl/"
- title: "AWS CLI Install"
url: "https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html"
---
# Source Install
@@ -34,161 +27,98 @@ resources:
## Overview
This guide covers building Spinifex from source code. Use this if you want to contribute to Spinifex, build custom versions, or inspect the codebase. For production deployments, the [binary installer](/docs/installing-spinifex) is recommended.
**Supported Operating Systems:**
- Ubuntu 22.04 / 24.04 / 25.10
- Debian 12.13
This guide builds Spinifex from source. For production deployments, the [binary installer](/docs/installing-spinifex) is recommended.
**Requirements:**
- Ubuntu 22.04+ / Debian 12+
- Go 1.26.1+
- GCC, make, pkg-config
- QEMU/KVM
- OVN/Open vSwitch
- AWS CLI v2
- QEMU/KVM, OVN/Open vSwitch, AWS CLI v2, dhcpcd-base
## Instructions
## Step 1. Install system dependencies
### Quick Install
Bootstrap all dependencies in one step:
## Step 1. Install Dependencies
```bash
sudo make -C spinifex quickinstall
```
Ensure Go is in your PATH:
```bash
export PATH=$PATH:/usr/local/go/bin/
```
### Manual Install
Alternatively, install packages individually:
```bash
sudo add-apt-repository universe
sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev make gcc unzip xz-utils file ovn-central ovn-host openvswitch-switch
```
Install Go 1.26.1+ from [https://go.dev/dl/](https://go.dev/dl/) and verify:
```bash
go version
```
Install AWS CLI v2:
```bash
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version
```
## Step 2. Clone and build
```bash
mkdir -p ~/Development/mulga/
cd ~/Development/mulga/
mkdir -p ~/Development/mulga && cd ~/Development/mulga
git clone https://github.com/mulgadc/spinifex.git
cd spinifex
./scripts/clone-deps.sh # Clone viperblock + predastore
./scripts/dev-setup.sh # Setup complete dev environment
sudo make -C spinifex quickinstall
export PATH=$PATH:/usr/local/go/bin
```
Confirm `./bin/spx` exists and is executable.
## Step 2. Clone and Build
## Step 3. Setup OVN
```bash
cd spinifex
./scripts/clone-deps.sh
./scripts/dev-setup.sh
```
For a single-node dev environment:
Confirm `./bin/spx` exists.
## Step 3. Initialize
```bash
sudo ./bin/spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
```
Save the admin credentials printed during init.
## Step 4. Setup OVN
If your WAN interface is already a bridge (e.g. `br-wan`), setup-ovn.sh auto-detects it:
```bash
./scripts/setup-ovn.sh --management
```
This creates the `br-int` integration bridge, starts `ovn-controller`, configures Geneve tunnel endpoints, and enables IP forwarding.
## Step 4. Initialize
Create a region and availability zone:
If your WAN is a physical NIC, choose one:
```bash
./bin/spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1
# Dedicated WAN NIC (not your SSH connection):
./scripts/setup-ovn.sh --management --wan-bridge=br-wan --wan-iface=eth1
# Single-NIC host (SSH-safe macvlan):
./scripts/setup-ovn.sh --management --macvlan --wan-iface=enp0s3
```
## Step 5. Trust the CA certificate
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
## Step 6. Start services
## Step 5. Start Services
```bash
./scripts/start-dev.sh
export AWS_PROFILE=spinifex
```
## Step 6. Verify
```bash
export AWS_PROFILE=spinifex
aws ec2 describe-instance-types
```
If this returns a list of available instance types, your installation is working.
**Congratulations! Spinifex is installed from source.**
Continue to [Setting Up Your Cluster](/docs/setting-up-your-cluster) to import an AMI, create a VPC, and launch your first instance.
## Troubleshooting
## Go not found in PATH
After installing Go, it may not be in your current shell session. Add it:
### Go Not Found in PATH
```bash
export PATH=$PATH:/usr/local/go/bin/
export PATH=$PATH:/usr/local/go/bin
```
Add this to your `~/.bashrc` or `~/.zshrc` for persistence. Verify with:
```bash
go version
```
## ./bin/spx missing after build
The build did not complete successfully. Re-run the setup:
### ./bin/spx Missing After Build
```bash
./scripts/dev-setup.sh
```
Check the output for compilation errors. Common causes are missing Go version or missing system packages.
## OVN services not starting
Verify the setup script was run with `--management`:
```bash
./scripts/setup-ovn.sh --management
```
Check OVN controller status:
```bash
sudo systemctl is-active ovn-controller
sudo ovn-sbctl show
```
## CA certificate not trusted
AWS CLI will reject HTTPS connections to Spinifex services. Re-add the certificate:
### CA Certificate Not Trusted
```bash
sudo cp ~/spinifex/config/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt
sudo update-ca-certificates
```
## clone-deps.sh fails
Ensure you have access to the GitHub repositories. If using SSH and it fails, try HTTPS:
```bash
git config --global url."https://github.com/".insteadOf "git@github.com:"
./scripts/clone-deps.sh
```
+2 -3
View File
@@ -6,7 +6,6 @@ tags:
- cli
- admin
- reference
badge: cli
resources:
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
@@ -75,7 +74,7 @@ node3 Ready 127.0.0.3 ap-southeast-2 ap-southeast-2a 2m 0
## Troubleshooting
## Permission denied running spinifex
### Permission Denied Running Spinifex
The binary may not be executable. Fix permissions:
@@ -85,7 +84,7 @@ chmod +x ./bin/spx
If you get permission errors during operations, ensure you're running with appropriate privileges. Some OVN and networking commands require `sudo`.
## Services fail to start
### Services Fail to Start
Check the daemon logs for specific errors:
File diff suppressed because it is too large Load Diff
+24 -51
View File
@@ -8,12 +8,12 @@ require (
github.com/google/uuid v1.6.0
github.com/kdomanski/iso9660 v0.4.0
github.com/klauspost/cpuid/v2 v2.3.0
github.com/mulgadc/predastore v1.1.0
github.com/mulgadc/viperblock v1.1.0
github.com/nats-io/nats-server/v2 v2.12.4
github.com/mulgadc/predastore v1.1.3
github.com/mulgadc/viperblock v1.1.5
github.com/nats-io/nats-server/v2 v2.12.6
github.com/nats-io/nats.go v1.49.0
github.com/ovn-kubernetes/libovsdb v0.8.1
github.com/pelletier/go-toml/v2 v2.2.4
github.com/pelletier/go-toml/v2 v2.3.0
github.com/pterm/pterm v0.12.83
github.com/spf13/cobra v1.10.2
github.com/spf13/viper v1.21.0
@@ -25,17 +25,11 @@ require (
atomicgo.dev/cursor v0.2.0 // indirect
atomicgo.dev/keyboard v0.2.9 // indirect
atomicgo.dev/schedule v0.1.0 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.2 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
github.com/BurntSushi/toml v1.6.0 // indirect
github.com/anthropics/anthropic-sdk-go v1.26.0 // indirect
github.com/antithesishq/antithesis-sdk-go v0.6.0 // indirect
github.com/antithesishq/antithesis-sdk-go v0.7.0 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/boltdb/bolt v1.3.1 // indirect
github.com/buraksezer/consistent v0.10.0 // indirect
github.com/ccojocar/zxcvbn-go v1.0.4 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cenkalti/hub v1.0.2 // indirect
github.com/cenkalti/rpc2 v1.0.5 // indirect
@@ -46,8 +40,7 @@ require (
github.com/dgraph-io/badger/v4 v4.9.1 // indirect
github.com/dgraph-io/ristretto/v2 v2.4.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fatih/color v1.19.0 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
github.com/go-logr/logr v1.4.3 // indirect
@@ -57,13 +50,8 @@ require (
github.com/go-playground/validator/v10 v10.30.1 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/google/flatbuffers v25.12.19+incompatible // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.13 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gookit/color v1.6.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-metrics v0.5.4 // indirect
@@ -74,20 +62,19 @@ require (
github.com/hashicorp/raft-boltdb/v2 v2.3.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/klauspost/compress v1.18.4 // indirect
github.com/klauspost/reedsolomon v1.13.2 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/reedsolomon v1.13.3 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/lithammer/fuzzysearch v1.1.8 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.21 // indirect
github.com/minio/crc64nvme v1.1.1 // indirect
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/nats-io/jwt/v2 v2.8.0 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/openai/openai-go/v3 v3.24.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.23.2 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
@@ -95,46 +82,32 @@ require (
github.com/prometheus/procfs v0.20.1 // indirect
github.com/quic-go/quic-go v0.59.0 // indirect
github.com/sagikazarmark/locafero v0.12.0 // indirect
github.com/securego/gosec/v2 v2.24.7 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0 // indirect
go.opentelemetry.io/otel v1.41.0 // indirect
go.opentelemetry.io/otel/metric v1.41.0 // indirect
go.opentelemetry.io/otel/trace v1.41.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.opentelemetry.io/otel v1.42.0 // indirect
go.opentelemetry.io/otel/metric v1.42.0 // indirect
go.opentelemetry.io/otel/trace v1.42.0 // indirect
go.yaml.in/yaml/v2 v2.4.4 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa // indirect
golang.org/x/crypto v0.49.0 // indirect
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa // indirect
golang.org/x/mod v0.33.0 // indirect
golang.org/x/net v0.51.0 // indirect
golang.org/x/sync v0.19.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/net v0.52.0 // indirect
golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.42.0 // indirect
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661 // indirect
golang.org/x/term v0.40.0 // indirect
golang.org/x/text v0.34.0 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/term v0.41.0 // indirect
golang.org/x/text v0.35.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/tools v0.42.0 // indirect
golang.org/x/vuln v1.1.4 // indirect
google.golang.org/genai v1.49.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 // indirect
google.golang.org/grpc v1.79.3 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.7.0 // indirect
)
tool (
github.com/securego/gosec/v2/cmd/gosec
golang.org/x/vuln/cmd/govulncheck
honnef.co/go/tools/cmd/staticcheck
)
tool golang.org/x/vuln/cmd/govulncheck
+44 -116
View File
@@ -7,14 +7,6 @@ atomicgo.dev/keyboard v0.2.9/go.mod h1:BC4w9g00XkxH/f1HXhW2sXmJFOCWbKn9xrOunSFtE
atomicgo.dev/schedule v0.1.0 h1:nTthAbhZS5YZmgYbb2+DH8uQIZcTlIrd4eYr3UQxEjs=
atomicgo.dev/schedule v0.1.0/go.mod h1:xeUa3oAkiuHYh8bKiQBRojqAMq3PXXbJujjb0hw8pEU=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.2 h1:+Nbt5Ev0xEqxlNjd6c+yYUeosQ5TtEUaNcN/3FozlaM=
cloud.google.com/go/auth v0.18.2/go.mod h1:xD+oY7gcahcu7G2SG2DsBerfFxgPAJz17zz2joOFF3M=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
github.com/MarvinJWendt/testza v0.1.0/go.mod h1:7AxNvlfeHP7Z/hDQ5JtE3OKYT3XFUeLCDE2DQninSqs=
github.com/MarvinJWendt/testza v0.2.1/go.mod h1:God7bhG8n6uQxwdScay+gjm9/LnO4D3kkcZX4hv9Rp8=
@@ -25,17 +17,13 @@ github.com/MarvinJWendt/testza v0.3.0/go.mod h1:eFcL4I0idjtIx8P9C6KkAuLgATNKpX4/
github.com/MarvinJWendt/testza v0.4.2/go.mod h1:mSdhXiKH8sg/gQehJ63bINcCKp7RtYewEjXsvsVUPbE=
github.com/MarvinJWendt/testza v0.5.2 h1:53KDo64C1z/h/d/stCYCPY69bt/OSwjq5KpFNwi+zB4=
github.com/MarvinJWendt/testza v0.5.2/go.mod h1:xu53QFE5sCdjtMCKk8YMQ2MnymimEctc4n3EjyIYvEY=
github.com/Masterminds/semver/v3 v3.4.0 h1:Zog+i5UMtVoCU8oKka5P7i9q9HgrJeGzI9SA1Xbatp0=
github.com/Masterminds/semver/v3 v3.4.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/anthropics/anthropic-sdk-go v1.26.0 h1:oUTzFaUpAevfuELAP1sjL6CQJ9HHAfT7CoSYSac11PY=
github.com/anthropics/anthropic-sdk-go v1.26.0/go.mod h1:qUKmaW+uuPB64iy1l+4kOSvaLqPXnHTTBKH6RVZ7q5Q=
github.com/antithesishq/antithesis-sdk-go v0.6.0 h1:v/YViLhFYkZOEEof4AXjD5AgGnGM84YHF4RqEwp6I2g=
github.com/antithesishq/antithesis-sdk-go v0.6.0/go.mod h1:IUpT2DPAKh6i/YhSbt6Gl3v2yvUZjmKncl7U91fup7E=
github.com/antithesishq/antithesis-sdk-go v0.7.0 h1:uWDG8BqLD1lI2ps38WDz2vXflrTX2+vLX0SvZtztJtE=
github.com/antithesishq/antithesis-sdk-go v0.7.0/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA=
github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4=
github.com/atomicgo/cursor v0.0.1/go.mod h1:cBON2QmmrysudxNBFthvMtN32r3jxVRIvzkUiF/RuIk=
@@ -87,8 +75,6 @@ github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
github.com/buraksezer/consistent v0.10.0 h1:hqBgz1PvNLC5rkWcEBVAL9dFMBWz6I0VgUCW25rrZlU=
github.com/buraksezer/consistent v0.10.0/go.mod h1:6BrVajWq7wbKZlTOUPs/XVfR8c0maujuPowduSpZqmw=
github.com/ccojocar/zxcvbn-go v1.0.4 h1:FWnCIRMXPj43ukfX000kvBZvV6raSxakYr1nzyNrUcc=
github.com/ccojocar/zxcvbn-go v1.0.4/go.mod h1:3GxGX+rHmueTUMvm5ium7irpyjmm7ikxYFOSJB21Das=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cenkalti/hub v1.0.2 h1:Nqv9TNaA9boeO2wQFW8o87BY3zKthtnzXmWGmJqhAV8=
@@ -115,15 +101,11 @@ github.com/dgraph-io/ristretto/v2 v2.4.0 h1:I/w09yLjhdcVD2QV192UJcq8dPBaAJb9pOuM
github.com/dgraph-io/ristretto/v2 v2.4.0/go.mod h1:0KsrXtXvnv0EqnzyowllbVJB8yBonswa2lTCK2gGo9E=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da h1:aIftn67I1fkbMa512G+w+Pxci9hJPB8oMnkcP3iZF38=
github.com/dgryski/go-farm v0.0.0-20240924180020-3414d57e47da/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/dnaeon/go-vcr v1.2.0 h1:zHCHvJYTMh1N7xnV7zf1m1GPBF9Ad0Jk/whtQ1663qI=
github.com/dnaeon/go-vcr v1.2.0/go.mod h1:R4UdLID7HZT3taECzJs4YgbbH6PIGXB6W/sc5OLb6RQ=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
@@ -152,8 +134,6 @@ github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
@@ -167,8 +147,6 @@ github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:W
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs=
github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
github.com/google/go-cmdtest v0.4.1-0.20220921163831-55ab3332a786 h1:rcv+Ippz6RAtvaGgKxc+8FQIpxHgsF+HBzPyYL2cyVU=
@@ -183,26 +161,16 @@ github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83 h1:z2ogiKUYzX5Is6zr/vP9vJGqPwcdqsWjOt+V8J7+bTc=
github.com/google/pprof v0.0.0-20260115054156-294ebfa9ad83/go.mod h1:MxpfABSjhmINe3F1It9d+8exIHFvUqtLIRCdOGNXqiI=
github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.13 h1:hSPAhW3NX+7HNlTsmrvU0jL75cIzxFktheceg95Nq14=
github.com/googleapis/enterprise-certificate-proxy v0.3.13/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gookit/assert v0.1.1 h1:lh3GcawXe/p+cU7ESTZ5Ui3Sm/x8JWpIis4/1aF0mY0=
github.com/gookit/assert v0.1.1/go.mod h1:jS5bmIVQZTIwk42uXl4lyj4iaaxx32tqH16CFj0VX2E=
github.com/gookit/color v1.4.2/go.mod h1:fqRyamkC1W8uxl+lxCQxOT09l/vYfZ+QeiX3rKQHCoQ=
github.com/gookit/color v1.5.0/go.mod h1:43aQb+Zerm/BWh2GnrgOQm7ffz7tvQXEKV6BFMl7wAo=
github.com/gookit/color v1.6.0 h1:JjJXBTk1ETNyqyilJhkTXJYYigHG24TM9Xa2M1xAhRA=
github.com/gookit/color v1.6.0/go.mod h1:9ACFc7/1IpHGBW8RwuDm/0YEnhg3dwwXpoMsmtyHfjs=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
@@ -244,15 +212,15 @@ github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7V
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kdomanski/iso9660 v0.4.0 h1:BPKKdcINz3m0MdjIMwS0wx1nofsOjxOq8TOr45WGHFg=
github.com/kdomanski/iso9660 v0.4.0/go.mod h1:OxUSupHsO9ceI8lBLPJKWBTphLemjrCQY8LPXM7qSzU=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/klauspost/cpuid/v2 v2.0.10/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/reedsolomon v1.13.2 h1:9qtQy2tKEVpVB8Pfq87ZljHZb60/LbeTQ1OxV8EGzdE=
github.com/klauspost/reedsolomon v1.13.2/go.mod h1:ggJT9lc71Vu+cSOPBlxGvBN6TfAS77qB4fp8vJ05NSA=
github.com/klauspost/reedsolomon v1.13.3 h1:01GwnO2xoCSaM0ShP4qwl+FsHg3csFShC6Tu/RS1ji0=
github.com/klauspost/reedsolomon v1.13.3/go.mod h1:yjqqjgMTQkBUHSG97/rm4zipffCNbCiZcB3kTqr++sQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -276,47 +244,41 @@ github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27k
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.20 h1:WcT52H91ZUAwy8+HUkdM3THM6gXqXuLJi9O3rjcQQaQ=
github.com/mattn/go-runewidth v0.0.20/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/mattn/go-runewidth v0.0.21 h1:jJKAZiQH+2mIinzCJIaIG9Be1+0NR+5sz/lYEEjdM8w=
github.com/mattn/go-runewidth v0.0.21/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/minio/crc64nvme v1.1.1 h1:8dwx/Pz49suywbO+auHCBpCtlW1OfpcLN7wYgVR6wAI=
github.com/minio/crc64nvme v1.1.1/go.mod h1:eVfm2fAzLlxMdUGc0EEBGSMmPwmXD5XiNRpnu9J3bvg=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76 h1:KGuD/pM2JpL9FAYvBrnBBeENKZNh6eNtjqytV6TYjnk=
github.com/minio/highwayhash v1.0.4-0.20251030100505-070ab1a87a76/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mulgadc/predastore v1.1.0 h1:DJ/rUWHmb8/cmqX/Cy1ttsX9X077S5PC3vsf6FW0WXU=
github.com/mulgadc/predastore v1.1.0/go.mod h1:4AcNew+IrsSPgvXO3ALBzfsMFOJHGVdS9OUCFU8cyZ8=
github.com/mulgadc/viperblock v1.1.0 h1:afh4Ygah/pmfL7drITzEHfpNp5Gra8kzWqxaOu4QkUM=
github.com/mulgadc/viperblock v1.1.0/go.mod h1:w9V0DjUH5hMXbXsiLbDr9lpqzXBGhEiYB1BeV6TYBkw=
github.com/mulgadc/predastore v1.1.3 h1:WddAOXmGJFQ7tRG+5B3yz/tCX46yW1CmbVL07y+tyBU=
github.com/mulgadc/predastore v1.1.3/go.mod h1:YP4TaathWczm9weoI8YomZ1/IgnFB9WJowkHPidh6rw=
github.com/mulgadc/viperblock v1.1.5 h1:ojmoJQrY7VkDGzdQj07w2F7ICRJX9RlGAKyn87Wa55o=
github.com/mulgadc/viperblock v1.1.5/go.mod h1:utoIQLwCmPujOBaT0mL5NhW3m2fwjQPQiQRJH6+Wtk8=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nats-io/jwt/v2 v2.8.0 h1:K7uzyz50+yGZDO5o772eRE7atlcSEENpL7P+b74JV1g=
github.com/nats-io/jwt/v2 v2.8.0/go.mod h1:me11pOkwObtcBNR8AiMrUbtVOUGkqYjMQZ6jnSdVUIA=
github.com/nats-io/nats-server/v2 v2.12.4 h1:ZnT10v2LU2Xcoiy8ek9X6Se4YG8EuMfIfvAEuFVx1Ts=
github.com/nats-io/nats-server/v2 v2.12.4/go.mod h1:5MCp/pqm5SEfsvVZ31ll1088ZTwEUdvRX1Hmh/mTTDg=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.12.6 h1:Egbx9Vl7Ch8wTtpXPGqbehkZ+IncKqShUxvrt1+Enc8=
github.com/nats-io/nats-server/v2 v2.12.6/go.mod h1:4HPlrvtmSO3yd7KcElDNMx9kv5EBJBnJJzQPptXlheo=
github.com/nats-io/nats.go v1.49.0 h1:yh/WvY59gXqYpgl33ZI+XoVPKyut/IcEaqtsiuTJpoE=
github.com/nats-io/nats.go v1.49.0/go.mod h1:fDCn3mN5cY8HooHwE2ukiLb4p4G4ImmzvXyJt+tGwdw=
github.com/nats-io/nkeys v0.4.15 h1:JACV5jRVO9V856KOapQ7x+EY8Jo3qw1vJt/9Jpwzkk4=
github.com/nats-io/nkeys v0.4.15/go.mod h1:CpMchTXC9fxA5zrMo4KpySxNjiDVvr8ANOSZdiNfUrs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/onsi/ginkgo/v2 v2.28.1 h1:S4hj+HbZp40fNKuLUQOYLDgZLwNUVn19N3Atb98NCyI=
github.com/onsi/ginkgo/v2 v2.28.1/go.mod h1:CLtbVInNckU3/+gC8LzkGUb9oF+e8W8TdUsxPwvdOgE=
github.com/onsi/gomega v1.39.1 h1:1IJLAad4zjPn2PsnhH70V4DKRFlrCzGBNrNaru+Vf28=
github.com/onsi/gomega v1.39.1/go.mod h1:hL6yVALoTOxeWudERyfppUcZXjMwIMLnuSfruD2lcfg=
github.com/openai/openai-go/v3 v3.24.0 h1:08x6GnYiB+AAejTo6yzPY8RkZMJQ8NpreiOyM5QfyYU=
github.com/openai/openai-go/v3 v3.24.0/go.mod h1:cdufnVK14cWcT9qA1rRtrXx4FTRsgbDPW7Ia7SS5cZo=
github.com/ovn-kubernetes/libovsdb v0.8.1 h1:M2J8bcJt5mXCom0HqzfEtuHkT80CTSQRcYG7acT8gf4=
github.com/ovn-kubernetes/libovsdb v0.8.1/go.mod h1:ZlnHLzagmLOSvyd9qfxBIZp6wOSOw0IsRsc+6lNUGbU=
github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY=
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -364,10 +326,6 @@ github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.12.0 h1:/NQhBAkUb4+fH1jivKHWusDYFjMOOKU88eegjfxfHb4=
github.com/sagikazarmark/locafero v0.12.0/go.mod h1:sZh36u/YSZ918v0Io+U9ogLYQJ9tLLBmM4eneO6WwsI=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/securego/gosec/v2 v2.24.7 h1:3k5yJnrhT1TTdsG0ZsnenlfCcT+7Y/+zeCPHbL7QAn8=
github.com/securego/gosec/v2 v2.24.7/go.mod h1:AdDJbjcG/XxFgVv7pW19vMNYlFM6+Q6Qy3t6lWAUcEY=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
@@ -388,7 +346,6 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
@@ -403,17 +360,6 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=
github.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
github.com/xo/terminfo v0.0.0-20210125001918-ca9a967f8778/go.mod h1:2MuV+tbUrU1zIOPMxZ5EncGwgmMJsa+9ucAQZXxsObs=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
@@ -423,36 +369,28 @@ go.etcd.io/bbolt v1.4.3 h1:dEadXpI6G79deX5prL3QRNP6JB8UxVkqo4UPnHaNXJo=
go.etcd.io/bbolt v1.4.3/go.mod h1:tKQlpPaYCVFctUIgFKFnAlvbmB3tpy1vkTnDWohtc0E=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0 h1:PnV4kVnw0zOmwwFkAzCN5O07fw1YOIQor120zrh0AVo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.66.0/go.mod h1:ofAwF4uinaf8SXdVzzbL4OsxJ3VfeEg3f/F6CeF49/Y=
go.opentelemetry.io/otel v1.41.0 h1:YlEwVsGAlCvczDILpUXpIpPSL/VPugt7zHThEMLce1c=
go.opentelemetry.io/otel v1.41.0/go.mod h1:Yt4UwgEKeT05QbLwbyHXEwhnjxNO6D8L5PQP51/46dE=
go.opentelemetry.io/otel/metric v1.41.0 h1:rFnDcs4gRzBcsO9tS8LCpgR0dxg4aaxWlJxCno7JlTQ=
go.opentelemetry.io/otel/metric v1.41.0/go.mod h1:xPvCwd9pU0VN8tPZYzDZV/BMj9CM9vs00GuBjeKhJps=
go.opentelemetry.io/otel/sdk v1.41.0 h1:YPIEXKmiAwkGl3Gu1huk1aYWwtpRLeskpV+wPisxBp8=
go.opentelemetry.io/otel/sdk v1.41.0/go.mod h1:ahFdU0G5y8IxglBf0QBJXgSe7agzjE4GiTJ6HT9ud90=
go.opentelemetry.io/otel/sdk/metric v1.41.0 h1:siZQIYBAUd1rlIWQT2uCxWJxcCO7q3TriaMlf08rXw8=
go.opentelemetry.io/otel/sdk/metric v1.41.0/go.mod h1:HNBuSvT7ROaGtGI50ArdRLUnvRTRGniSUZbxiWxSO8Y=
go.opentelemetry.io/otel/trace v1.41.0 h1:Vbk2co6bhj8L59ZJ6/xFTskY+tGAbOnCtQGVVa9TIN0=
go.opentelemetry.io/otel/trace v1.41.0/go.mod h1:U1NU4ULCoxeDKc09yCWdWe+3QoyweJcISEVa1RBzOis=
go.opentelemetry.io/otel v1.42.0 h1:lSQGzTgVR3+sgJDAU/7/ZMjN9Z+vUip7leaqBKy4sho=
go.opentelemetry.io/otel v1.42.0/go.mod h1:lJNsdRMxCUIWuMlVJWzecSMuNjE7dOYyWlqOXWkdqCc=
go.opentelemetry.io/otel/metric v1.42.0 h1:2jXG+3oZLNXEPfNmnpxKDeZsFI5o4J+nz6xUlaFdF/4=
go.opentelemetry.io/otel/metric v1.42.0/go.mod h1:RlUN/7vTU7Ao/diDkEpQpnz3/92J9ko05BIwxYa2SSI=
go.opentelemetry.io/otel/trace v1.42.0 h1:OUCgIPt+mzOnaUTpOQcBiM/PLQ/Op7oq6g4LenLmOYY=
go.opentelemetry.io/otel/trace v1.42.0/go.mod h1:f3K9S+IFqnumBkKhRJMeaZeNk9epyhnCmQh/EysQCdc=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko=
go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ=
golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE=
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa h1:6Wi43P0isP8Nl8D4qJmA3VC4FswVH0RJkr5cauo67SQ=
golang.org/x/exp/typeparams v0.0.0-20260218203240-3dfff04db8fa/go.mod h1:PqrXSW65cXDZH0k4IeUbhmg/bcAZDbzNz3byBpKCsXo=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
@@ -467,8 +405,8 @@ golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -477,8 +415,8 @@ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -507,8 +445,8 @@ golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661 h1:AjMsnhVxCTd9HssPcdXPhNTQ0YazDt5+52EqmcWQ0PA=
golang.org/x/telemetry v0.0.0-20260304144227-18da59047661/go.mod h1:NuITXsA9cTiqnXtVk+/wrBT2Ja4X5hsfGOYRJ6kgYjs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
@@ -516,18 +454,18 @@ golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9sn
golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM=
golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=
golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
@@ -542,15 +480,7 @@ golang.org/x/vuln v1.1.4 h1:Ju8QsuyhX3Hk8ma3CesTbO8vfJD9EvUBgHvkxHBzj0I=
golang.org/x/vuln v1.1.4/go.mod h1:F+45wmU18ym/ca5PLTPLsSzr2KppzswxPP603ldA67s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genai v1.49.0 h1:Se+QJaH2GYK1aaR1o5S38mlU2GD5FnVvP76nfkV7LH0=
google.golang.org/genai v1.49.0/go.mod h1:A3kkl0nyBjyFlNjgxIwKq70julKbIxpSxqKO5gw/gmk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171 h1:ggcbiqK8WWh6l1dnltU4BgWGIGo+EVYxCaAPih/zQXQ=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260226221140-a57be14db171/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -579,5 +509,3 @@ gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.7.0 h1:w6WUp1VbkqPEgLz4rkBzH/CSU6HkoqNLp6GstyTx3lU=
honnef.co/go/tools v0.7.0/go.mod h1:pm29oPxeP3P82ISxZDgIYeOaf9ta6Pi0EWvCFoLG2vc=
+1 -1
View File
@@ -5,7 +5,7 @@ set -euo pipefail
# Usage: scripts/check-coverage.sh <coverprofile> [quiet]
PROFILE="${1:?Usage: check-coverage.sh <coverprofile> [quiet]}"
MIN=65.0
MIN=60.0
QUIET="${2:-}"
if [[ ! -s "$PROFILE" ]]; then
+10 -32
View File
@@ -13,17 +13,9 @@ MULGA_ROOT="$(cd "$PROJECT_ROOT/.." && pwd)"
NATS_PORT=4222
PREDASTORE_PORT=8443
SPINIFEX_GATEWAY_PORT=9999
DATA_DIR="$HOME/spinifex"
# Use CONFIG_DIR environment variable if set, otherwise default to ~/spinifex/config
CONFIG_DIR="${CONFIG_DIR:-$HOME/spinifex/config}"
echo "🚀 Setting up Spinifex development environment..."
echo "Project root: $PROJECT_ROOT"
echo "Data directory: $DATA_DIR"
# Create necessary directories
mkdir -p "$DATA_DIR"/{nats,predastore,viperblock,logs,spinifex}
mkdir -p "$CONFIG_DIR"
# Function to check if command exists
command_exists() {
@@ -148,19 +140,6 @@ cd "$PROJECT_ROOT"
make build
echo "✅ Spinifex built successfully"
# Initialize Spinifex configuration using admin init
#echo ""
#echo "🔐 Initializing Spinifex configuration..."
#if [[ ! -f "$CONFIG_DIR/spinifex.toml" ]]; then
# echo "📋 Running spx admin init..."
# ./bin/spx admin init --config-dir "$CONFIG_DIR"
# echo "✅ Spinifex configuration initialized"
#else
# echo "✅ Spinifex configuration already exists"
# echo " To re-initialize, run: ./bin/spx admin init --force"
#fi
# Build components
echo ""
echo "🔨 Building components..."
@@ -188,23 +167,22 @@ fi
echo ""
echo "🎉 Development environment setup complete!"
echo ""
echo "When running Spinifex for the first time, run the init function to create the"
echo "default directories for data, config files and layout required."
echo "./bin/spx admin init"
echo "📋 Next steps:"
echo ""
echo "🚀 To start the development environment:"
echo " ./scripts/start-dev.sh"
echo " 1. Setup OVN networking:"
echo " ./scripts/setup-ovn.sh --management"
echo ""
echo "🛑 To stop the development environment:"
echo " ./scripts/stop-dev.sh"
echo " 2. Initialize Spinifex (creates ~/spinifex/ directories, config, certs, and installs CA):"
echo " sudo ./bin/spx admin init --region ap-southeast-2 --az ap-southeast-2a --node node1 --nodes 1"
echo ""
echo " 3. Start services:"
echo " ./scripts/start-dev.sh"
echo ""
echo "🔧 Development endpoints:"
echo " - Spinifex Gateway: https://localhost:$SPINIFEX_GATEWAY_PORT"
echo " - Predastore S3: https://localhost:$PREDASTORE_PORT"
echo " - NATS: nats://localhost:$NATS_PORT"
echo ""
echo "📊 Monitor logs:"
echo " tail -f $DATA_DIR/logs/*.log"
echo ""
echo "🧪 Test with AWS CLI:"
echo " aws --endpoint-url https://localhost:$SPINIFEX_GATEWAY_PORT --no-verify-ssl ec2 describe-instances"
echo " export AWS_PROFILE=spinifex"
echo " aws ec2 describe-instances"
+1 -1
View File
@@ -13,7 +13,7 @@ set -euo pipefail
PROFILE=""
BASE_REF=""
THRESHOLD=65
THRESHOLD=60
QUIET="${QUIET:-}"
while [[ $# -gt 0 ]]; do
+8 -8
View File
@@ -26,10 +26,10 @@ variable "region" {
# Set these environment variables or use terraform.tfvars:
# export AWS_ACCESS_KEY_ID=your_access_key
# export AWS_SECRET_ACCESS_KEY=your_secret_key
# export TF_VAR_mulgaos_endpoint=http://localhost:8080
# export TF_VAR_mulgaos_endpoint=https://localhost:9999
variable "mulgaos_endpoint" {
type = string
default = "http://localhost:8080"
default = "https://localhost:9999"
description = "MulgaOS API endpoint (awsd gateway)"
}
@@ -57,15 +57,15 @@ data "aws_availability_zones" "available" {
state = "available"
}
# Debian 12 AMI (official Debian account)
# For MulgaOS, you may need to import a local AMI first
# Debian 12 AMI
# For MulgaOS, need to import a local AMI first
data "aws_ami" "debian12" {
most_recent = true
owners = ["136693071363"] # Debian official
owners = ["000000000000"] # Spinifex system images (on AWS use "136693071363" for Debian official)
filter {
name = "name"
values = ["debian-12-amd64-*"]
values = ["*debian-12*"]
}
filter {
@@ -96,9 +96,9 @@ resource "local_file" "hive_test_pem" {
file_permission = "0600"
}
# 1: VPC 10.1.1.0/16
# 1: VPC 10.1.0.0/16
resource "aws_vpc" "main" {
cidr_block = "10.1.1.0/16"
cidr_block = "10.1.0.0/16"
enable_dns_hostnames = true
enable_dns_support = true
+224 -26
View File
@@ -1,9 +1,34 @@
#!/bin/bash
# Reset dev environment (will purge all data)
# This script is for single-node dev environments only.
set -euo pipefail
CONFIG_FILE="$HOME/spinifex/config/spinifex.toml"
# --- Guard: refuse to run on multi-node clusters ---
if [ -f "$CONFIG_FILE" ]; then
NODE_COUNT=$(grep -cE '^\[nodes\.[^.]+\]' "$CONFIG_FILE")
if [ "$NODE_COUNT" -gt 1 ]; then
echo "❌ Multi-node cluster detected ($NODE_COUNT nodes in $CONFIG_FILE)."
echo " This script only supports single-node dev environments."
echo " Reset each node individually or use 'spx admin cluster shutdown'."
exit 1
fi
fi
# Save region from existing config before we delete everything
REGION="ap-southeast-2"
if [ -f "$CONFIG_FILE" ]; then
SAVED_REGION=$(grep -E '^\s*region\s*=' "$CONFIG_FILE" | tail -1 | sed 's/.*=\s*"\(.*\)"/\1/')
if [ -n "$SAVED_REGION" ]; then
REGION="$SAVED_REGION"
fi
fi
echo "Using region: $REGION"
# --- Shutdown services ---
echo "Shutting down services"
if ! ./scripts/stop-dev.sh; then
@@ -11,47 +36,220 @@ if ! ./scripts/stop-dev.sh; then
exit 1
fi
echo "Removing data"
# Verify no spinifex-related processes are still running
PROCESS_PATTERN='(bin/spx|spinifex-ui|nats-server|predastore|viperblock|vpcd)'
remaining=$(pgrep -af "$PROCESS_PATTERN" | grep -v "reset-dev-env.sh" || true)
if [ -n "$remaining" ]; then
echo "Waiting for remaining processes to exit..."
timeout=30
elapsed=0
while pgrep -af "$PROCESS_PATTERN" | grep -qv "reset-dev-env.sh" 2>/dev/null; do
if [ $elapsed -ge $timeout ]; then
echo "❌ Processes still running after ${timeout}s:"
pgrep -af "$PROCESS_PATTERN" | grep -v "reset-dev-env.sh" || true
echo " Kill them manually and re-run this script."
exit 1
fi
sleep 1
elapsed=$((elapsed + 1))
done
fi
rm -rf ~/spinifex/predastore/
# Verify no QEMU VMs are running
if pgrep -x qemu-system-x86_64 > /dev/null 2>&1; then
echo "❌ QEMU instances still running. Cannot reset while VMs are active."
echo " Run './scripts/stop-dev.sh' or kill them manually."
exit 1
fi
rm -rf ~/spinifex/spinifex/*
echo "All services confirmed stopped"
rm -rf ~/spinifex/viperblock/*
# --- Clean OVS/OVN ---
echo "Removing OVS bridges and config"
rm -rf ~/spinifex/nats/*
if command -v ovs-vsctl >/dev/null 2>&1; then
# Ensure OVS is running so we can clean up
sudo systemctl start openvswitch-switch 2>/dev/null || true
sleep 1
rm -rf ~/spinifex/images/*
# Delete all OVS bridges (br-int, br-external, etc.)
for br in $(sudo ovs-vsctl list-br 2>/dev/null); do
echo " Deleting bridge: $br"
sudo ovs-vsctl --if-exists del-br "$br"
done
# Clear OVN external_ids
sudo ovs-vsctl --if-exists clear Open_vSwitch . external_ids 2>/dev/null || true
echo " Cleared OVS external_ids"
# Stop OVS again after cleanup
sudo systemctl stop openvswitch-switch 2>/dev/null || true
fi
# Clean OVN databases (both Northbound and Southbound)
# Delete the DB files outright — setup-ovn.sh will restart ovn-central with fresh
# empty databases. This eliminates stale SB state (chassis entries, port bindings,
# datapath bindings) that accumulates across resets and causes ovn-controller to
# enter a commit failure loop ("OVNSB commit failed, force recompute").
echo "Removing OVN database files"
sudo systemctl stop ovn-central 2>/dev/null || true
sudo systemctl stop ovn-controller 2>/dev/null || true
if [ -d /var/lib/ovn ]; then
sudo rm -f /var/lib/ovn/ovnnb_db.db /var/lib/ovn/ovnsb_db.db
echo " Deleted /var/lib/ovn/ovn{nb,sb}_db.db"
else
echo " /var/lib/ovn not found, skipping OVN DB cleanup"
fi
# Remove veth pair created by setup-ovn.sh (veth mode — Linux bridge ↔ OVS bridge)
if ip link show veth-wan-br >/dev/null 2>&1; then
echo " Deleting veth pair: veth-wan-br ↔ veth-wan-ovs"
sudo ip link del veth-wan-br 2>/dev/null || true
fi
# Remove macvlan interfaces created by setup-ovn.sh
for iface in $(ip -o link show type macvlan 2>/dev/null | awk -F': ' '{print $2}' | grep '^spx-ext-'); do
echo " Deleting macvlan: $iface"
sudo ip link del "$iface" 2>/dev/null || true
done
# --- Wipe data ---
echo "Removing ~/spinifex"
rm -rf ~/spinifex
# --- Re-initialize ---
# Detect WAN interface (default route).
WAN_IFACE=$(ip -4 route show default | awk '{print $5}' | head -1)
WAN_GW=$(ip -4 route show default | awk '{print $3}' | head -1)
echo "Detected WAN interface: ${WAN_IFACE:-none}, gateway: ${WAN_GW:-none}"
# Check if WAN interface is already a bridge (tofu-cluster / production setup).
# If so, setup-ovn.sh auto-detects it. If it's a physical NIC, we need to
# decide between direct bridge and macvlan based on SSH safety.
WAN_IS_BRIDGE=false
if [ -n "$WAN_IFACE" ]; then
if ip -d link show "$WAN_IFACE" 2>/dev/null | grep -q "bridge"; then
WAN_IS_BRIDGE=true
fi
fi
# Build setup-ovn.sh flags based on detected topology
SETUP_OVN_FLAGS=""
if [ "$WAN_IS_BRIDGE" = true ]; then
# WAN is already a bridge (e.g. br-wan from cloud-init) — setup-ovn.sh
# auto-detects it, no explicit flags needed.
echo " WAN is a bridge: $WAN_IFACE (auto-detected)"
elif [ -n "$WAN_IFACE" ]; then
# WAN is a physical NIC. Detect SSH NIC to decide bridge strategy.
SSH_NIC=""
if [ -n "${SSH_CONNECTION:-}" ]; then
SSH_IP=$(echo "$SSH_CONNECTION" | awk '{print $3}')
SSH_NIC=$(ip -o -4 addr show | awk -v ip="$SSH_IP" '$0 ~ ip"/" {print $2}' | head -1)
fi
if [ -z "$SSH_NIC" ]; then
SSH_NIC="$WAN_IFACE"
fi
if [ "$WAN_IFACE" != "$SSH_NIC" ]; then
SETUP_OVN_FLAGS="--wan-bridge=br-wan --wan-iface=$WAN_IFACE"
echo " Bridge mode: direct (WAN=$WAN_IFACE != SSH=$SSH_NIC)"
else
SETUP_OVN_FLAGS="--macvlan --wan-iface=$WAN_IFACE"
echo " Bridge mode: macvlan (WAN=$WAN_IFACE == SSH=$SSH_NIC)"
fi
fi
# Chassis ID must match what vpcd derives from the node name in spinifex.toml.
# vpcd generates "chassis-{nodeName}" (e.g., chassis-node1) and uses it to schedule
# gateway chassis in the NB DB. ovn-controller registers in the SB DB using the OVS
# system-id. If they don't match, gateway traffic has no host to schedule to.
NODE_NAME="node1"
CHASSIS_ID="chassis-${NODE_NAME}"
echo "Re-initializing OVN (chassis-id: $CHASSIS_ID)"
./scripts/setup-ovn.sh --management $SETUP_OVN_FLAGS --chassis-id="$CHASSIS_ID"
echo "Initializing platform"
ADMIN_INIT_ARGS="--region $REGION --az ${REGION}a --node node1 --nodes 1"
# External networking mode: set EXTERNAL_MODE=nat for NAT/DHCP mode (outbound-only),
# or leave unset / EXTERNAL_MODE=pool for pool mode (per-VM public IPs).
EXTERNAL_MODE="${EXTERNAL_MODE:-pool}"
if [ -n "$WAN_IFACE" ] && [ -n "$WAN_GW" ]; then
WAN_IP=$(ip -4 -o addr show "$WAN_IFACE" 2>/dev/null | awk '{print $4}' | cut -d/ -f1 | head -1)
WAN_PREFIX=$(ip -4 -o addr show "$WAN_IFACE" 2>/dev/null | awk '{print $4}' | cut -d/ -f2 | head -1)
if [ -z "$WAN_PREFIX" ]; then WAN_PREFIX=24; fi
if [ "$EXTERNAL_MODE" = "nat" ]; then
# NAT mode: outbound-only via shared SNAT. Use a single IP from the
# high end of the WAN subnet as the gateway IP. No per-VM public IPs.
IFS='.' read -r o1 o2 o3 o4 <<< "$WAN_GW"
GATEWAY_IP="${GATEWAY_IP:-${o1}.${o2}.${o3}.200}"
echo " External mode: nat (gateway IP: $GATEWAY_IP, WAN gateway: $WAN_GW)"
ADMIN_INIT_ARGS="$ADMIN_INIT_ARGS --external-mode=nat --gateway-ip=${GATEWAY_IP} --external-gateway=${WAN_GW} --external-prefix-len=${WAN_PREFIX}"
else
# Pool mode: per-VM public IPs from a static range
IFS='.' read -r o1 o2 o3 o4 <<< "$WAN_GW"
POOL_START="${o1}.${o2}.${o3}.200"
POOL_END="${o1}.${o2}.${o3}.250"
echo " External pool: $POOL_START - $POOL_END (gateway: $WAN_GW)"
ADMIN_INIT_ARGS="$ADMIN_INIT_ARGS --external-mode=pool --external-pool=${POOL_START}-${POOL_END} --external-gateway=${WAN_GW} --external-prefix-len=${WAN_PREFIX}"
fi
fi
./bin/spx admin init $ADMIN_INIT_ARGS
# Re-initialize with --force to regenerate credentials, certs, config, and
# update ~/.aws/credentials. Must carry the same args so external networking
# config isn't lost.
echo "Re-initializing platform..."
./bin/spx admin init --force $ADMIN_INIT_ARGS
# Generate SSH key if it doesn't exist
if [ ! -f ~/.ssh/spinifex-key.pub ]; then
echo "Generating SSH key pair..."
ssh-keygen -t ed25519 -f ~/.ssh/spinifex-key -N ""
fi
# Enable pprof for development
PPROF_ENABLED=1 PPROF_OUTPUT=/tmp/spinifex-vm.prof ./scripts/start-dev.sh --build
# Follow INSTALL.md steps
echo "Importing AMI"
export AWS_PROFILE=spinifex
# Step 1 SSH key
# Import SSH key
echo "Importing SSH key"
aws ec2 import-key-pair --key-name "spinifex-key" --public-key-material fileb://~/.ssh/spinifex-key.pub
echo "Validating key pair imported"
aws ec2 describe-key-pairs
# Detect architecture
ARCH=$(uname -m)
if [ "$ARCH" = "x86_64" ]; then
IMAGE_NAME="ubuntu-24.04-x86_64"
elif [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
IMAGE_NAME="ubuntu-24.04-arm64"
else
echo "Warning: Unknown architecture $ARCH, defaulting to x86_64"
IMAGE_NAME="ubuntu-24.04-x86_64"
fi
# Import AMI
echo "Importing AMI"
echo "Detected architecture: $ARCH, using image: $IMAGE_NAME"
./bin/spx admin images import --name "$IMAGE_NAME"
LOCAL_IMAGE="$HOME/images/ubuntu-24.04.img"
if [ -f "$LOCAL_IMAGE" ]; then
echo "Using local image: $LOCAL_IMAGE"
ARCH=$(uname -m)
if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
IMG_ARCH="arm64"
else
IMG_ARCH="x86_64"
fi
./bin/spx admin images import --file "$LOCAL_IMAGE" --distro ubuntu --version 24.04 --arch "$IMG_ARCH"
else
# Download by name based on architecture
ARCH=$(uname -m)
if [ "$ARCH" = "x86_64" ]; then
IMAGE_NAME="ubuntu-24.04-x86_64"
elif [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
IMAGE_NAME="ubuntu-24.04-arm64"
else
echo "Warning: Unknown architecture $ARCH, defaulting to x86_64"
IMAGE_NAME="ubuntu-24.04-x86_64"
fi
echo "Downloading image: $IMAGE_NAME"
./bin/spx admin images import --name "$IMAGE_NAME"
fi
aws ec2 describe-images
echo "Reset successful, fresh AMI imported, proceed to creating instances"
echo "Reset complete, fresh AMI imported, proceed to creating instances"
+390 -29
View File
@@ -6,34 +6,55 @@
# 1. Installs OVN/OVS packages (if not present)
# 2. Enables required services (openvswitch-switch, ovn-controller)
# 3. Creates br-int with secure fail-mode
# 4. Configures OVS external_ids for OVN chassis identity
# 5. Applies sysctl tuning for overlay networking
# 4. Configures WAN bridge for public subnet uplink (auto-detected or manual)
# 5. Configures OVS external_ids for OVN chassis identity
# 6. Applies sysctl tuning for overlay networking
#
# Usage:
# ./scripts/setup-ovn.sh [options]
#
# Options:
# --management Also start OVN central services (NB DB, SB DB, ovn-northd)
# --ovn-remote OVN SB DB address (default: tcp:127.0.0.1:6642)
# --encap-ip Geneve tunnel endpoint IP (default: auto-detect)
# --chassis-id OVN chassis identifier (default: hostname)
# --management Also start OVN central services (NB DB, SB DB, ovn-northd)
# --wan-bridge=NAME OVS bridge for WAN traffic (default: auto-detect from default route)
# --wan-iface=NAME Physical NIC to add to the WAN bridge (use with --wan-bridge or --macvlan)
# --macvlan Create macvlan off --wan-iface instead of moving NIC directly.
# SSH-safe for single-NIC hosts where WAN NIC carries SSH.
# --dhcp Obtain gateway IP via DHCP on the WAN bridge interface
# --ovn-remote=ADDR OVN SB DB address (default: tcp:127.0.0.1:6642)
# --encap-ip=IP Geneve tunnel endpoint IP (default: auto-detect)
# --chassis-id=ID OVN chassis identifier (default: chassis-$(hostname -s))
#
# WAN Bridge Auto-Detection:
# When no --wan-bridge is given, the script checks the default route interface:
# - If it's an OVS bridge → use it directly for bridge-mappings
# - If it's a Linux bridge → create OVS br-ext + veth pair to link them
# (non-destructive, Linux bridge keeps IP/routes, no interruption)
# - If it's a physical NIC → stop and print guidance (cannot safely move NIC)
#
# Examples:
# # Single-node development (management + compute on same host):
# # WAN is already on a bridge (tofu-cluster, production):
# ./scripts/setup-ovn.sh --management
#
# # Dedicated WAN NIC (not your SSH NIC — you take responsibility):
# ./scripts/setup-ovn.sh --management --wan-bridge=br-wan --wan-iface=eth1
#
# # Single-NIC host (SSH-safe macvlan):
# ./scripts/setup-ovn.sh --management --macvlan --wan-iface=eth0
#
# # Compute node joining an existing cluster:
# ./scripts/setup-ovn.sh --ovn-remote=tcp:10.0.0.1:6642 --encap-ip=10.0.0.2
#
# # Multi-node dev cluster (node1 = management):
# Node 1: ./scripts/setup-ovn.sh --management --encap-ip=127.0.0.1
# Node 2: ./scripts/setup-ovn.sh --ovn-remote=tcp:127.0.0.1:6642 --encap-ip=127.0.0.2
# Node 3: ./scripts/setup-ovn.sh --ovn-remote=tcp:127.0.0.1:6642 --encap-ip=127.0.0.3
# # No WAN bridge (overlay-only, no public subnet):
# ./scripts/setup-ovn.sh --management --encap-ip=10.0.0.1
set -e
# Defaults
MANAGEMENT=false
WAN_BRIDGE=""
WAN_IFACE=""
MACVLAN_MODE=false
EXTERNAL_DHCP=false
OVN_REMOTE="tcp:127.0.0.1:6642"
ENCAP_IP=""
CHASSIS_ID=""
@@ -41,12 +62,16 @@ CHASSIS_ID=""
# Parse arguments
for arg in "$@"; do
case "$arg" in
--management) MANAGEMENT=true ;;
--ovn-remote=*) OVN_REMOTE="${arg#*=}" ;;
--encap-ip=*) ENCAP_IP="${arg#*=}" ;;
--chassis-id=*) CHASSIS_ID="${arg#*=}" ;;
--management) MANAGEMENT=true ;;
--macvlan) MACVLAN_MODE=true ;;
--dhcp) EXTERNAL_DHCP=true ;;
--wan-bridge=*) WAN_BRIDGE="${arg#*=}" ;;
--wan-iface=*) WAN_IFACE="${arg#*=}" ;;
--ovn-remote=*) OVN_REMOTE="${arg#*=}" ;;
--encap-ip=*) ENCAP_IP="${arg#*=}" ;;
--chassis-id=*) CHASSIS_ID="${arg#*=}" ;;
--help|-h)
head -30 "$0" | tail -28
head -44 "$0" | tail -42
exit 0
;;
*)
@@ -56,13 +81,121 @@ for arg in "$@"; do
esac
done
# --- WAN bridge auto-detection ---
# Determine the WAN bridge name and how to set it up.
WAN_BRIDGE_MODE="" # "existing", "veth", "direct", "macvlan", or ""
LINUX_BRIDGE="" # Set when WAN_BRIDGE_MODE="veth" — the Linux bridge behind the veth pair
detect_wan_bridge() {
# If --wan-bridge was explicitly given, use it
if [ -n "$WAN_BRIDGE" ]; then
if [ "$MACVLAN_MODE" = true ] && [ -n "$WAN_IFACE" ]; then
WAN_BRIDGE_MODE="macvlan"
elif [ -n "$WAN_IFACE" ]; then
WAN_BRIDGE_MODE="direct"
elif sudo ovs-vsctl br-exists "$WAN_BRIDGE" 2>/dev/null; then
WAN_BRIDGE_MODE="existing"
else
# Bridge doesn't exist yet and no --wan-iface — create empty OVS bridge
WAN_BRIDGE_MODE="existing"
fi
return
fi
# If --macvlan was given without --wan-bridge, we need --wan-iface
if [ "$MACVLAN_MODE" = true ]; then
if [ -z "$WAN_IFACE" ]; then
echo "ERROR: --macvlan requires --wan-iface=<NIC>"
exit 1
fi
WAN_BRIDGE="br-wan"
WAN_BRIDGE_MODE="macvlan"
return
fi
# Auto-detect: find the default route interface
local default_dev
default_dev=$(ip -4 route show default 2>/dev/null | awk '{for(i=1;i<=NF;i++) if($i=="dev") print $(i+1)}' | head -1)
if [ -z "$default_dev" ]; then
echo " No default route found — no WAN bridge configured"
echo " (VMs will not have external connectivity)"
return
fi
# Check if the default route device is a bridge (Linux or OVS)
local is_bridge=false
if ip -d link show "$default_dev" 2>/dev/null | grep -q "bridge"; then
is_bridge=true
fi
if sudo ovs-vsctl br-exists "$default_dev" 2>/dev/null; then
is_bridge=true
fi
if [ "$is_bridge" = true ]; then
if sudo ovs-vsctl br-exists "$default_dev" 2>/dev/null; then
# Already an OVS bridge — use it directly for bridge-mappings
WAN_BRIDGE="$default_dev"
WAN_BRIDGE_MODE="existing"
echo " Auto-detected WAN bridge: $WAN_BRIDGE (OVS bridge, default route)"
else
# Linux bridge — link to OVS via veth pair (non-destructive)
LINUX_BRIDGE="$default_dev"
WAN_BRIDGE="br-ext"
WAN_BRIDGE_MODE="veth"
echo " Auto-detected Linux bridge: $LINUX_BRIDGE (default route)"
echo " Will create OVS bridge br-ext + veth pair to link them"
fi
return
fi
# Default route is a physical NIC — cannot safely move it to OVS
# because it might be carrying SSH.
local wan_ip
wan_ip=$(ip -4 -o addr show "$default_dev" 2>/dev/null | awk '{print $4}' | cut -d/ -f1 | head -1)
echo ""
echo "============================================================"
echo " WAN interface '$default_dev' ($wan_ip) is a physical NIC."
echo " Cannot auto-create a bridge — this may drop your connection."
echo ""
echo " Options:"
echo ""
echo " 1. Create a WAN bridge first (e.g. via netplan), then re-run:"
echo " ./scripts/setup-ovn.sh --management"
echo ""
echo " 2. Dedicated WAN NIC (NOT your SSH connection):"
echo " ./scripts/setup-ovn.sh --management --wan-bridge=br-wan --wan-iface=$default_dev"
echo ""
echo " 3. Single-NIC host (SSH-safe macvlan):"
echo " ./scripts/setup-ovn.sh --management --macvlan --wan-iface=$default_dev"
echo ""
echo " 4. No external networking (overlay-only):"
echo " ./scripts/setup-ovn.sh --management --encap-ip=$wan_ip"
echo "============================================================"
echo ""
exit 1
}
detect_wan_bridge
# Auto-detect encap IP if not specified
if [ -z "$ENCAP_IP" ]; then
ENCAP_IP=$(ip -4 route get 8.8.8.8 2>/dev/null | awk '/src/{print $7}' | head -1)
if [ -z "$ENCAP_IP" ]; then
ENCAP_IP="127.0.0.1"
# Prefer br-vpc IP if it exists (dedicated VPC data plane)
if ip -4 addr show br-vpc >/dev/null 2>&1; then
ENCAP_IP=$(ip -4 -o addr show br-vpc 2>/dev/null | awk '{print $4}' | cut -d/ -f1 | head -1)
if [ -n "$ENCAP_IP" ]; then
echo "Auto-detected encap IP from br-vpc: $ENCAP_IP"
fi
fi
# Fall back to default route source IP
if [ -z "$ENCAP_IP" ]; then
ENCAP_IP=$(ip -4 route get 8.8.8.8 2>/dev/null | awk '/src/{print $7}' | head -1)
if [ -z "$ENCAP_IP" ]; then
ENCAP_IP="127.0.0.1"
fi
echo "Auto-detected encap IP: $ENCAP_IP"
fi
echo "Auto-detected encap IP: $ENCAP_IP"
fi
# Auto-detect chassis ID if not specified
@@ -72,10 +205,21 @@ if [ -z "$CHASSIS_ID" ]; then
fi
echo "=== Spinifex OVN Compute Node Setup ==="
echo " Management node: $MANAGEMENT"
echo " OVN Remote (SB): $OVN_REMOTE"
echo " Encap IP: $ENCAP_IP"
echo " Chassis ID: $CHASSIS_ID"
echo " Management node: $MANAGEMENT"
if [ -n "$WAN_BRIDGE" ]; then
echo " WAN bridge: $WAN_BRIDGE ($WAN_BRIDGE_MODE)"
if [ -n "$LINUX_BRIDGE" ]; then
echo " Linux bridge: $LINUX_BRIDGE (linked via veth pair)"
fi
if [ -n "$WAN_IFACE" ]; then
echo " WAN interface: $WAN_IFACE"
fi
else
echo " WAN bridge: none (overlay-only)"
fi
echo " OVN Remote (SB): $OVN_REMOTE"
echo " Encap IP: $ENCAP_IP"
echo " Chassis ID: $CHASSIS_ID"
echo ""
# --- Step 1: Install packages ---
@@ -118,6 +262,15 @@ if [ "$MANAGEMENT" = true ]; then
sudo systemctl start ovn-central
echo " ovn-central: started (NB DB + SB DB + ovn-northd)"
# Wait for OVN NB DB socket to become available
for i in $(seq 1 15); do
if sudo ovn-nbctl --timeout=2 get-connection >/dev/null 2>&1; then
break
fi
echo " Waiting for OVN NB DB... ($i/15)"
sleep 1
done
# Allow remote connections to NB and SB databases
sudo ovn-nbctl set-connection ptcp:6641
sudo ovn-sbctl set-connection ptcp:6642
@@ -135,15 +288,189 @@ sudo ovs-vsctl set Bridge br-int other-config:disable-in-band=true
sudo ip link set br-int up
echo " br-int: created, fail-mode=secure, up"
# --- Step 3b: Configure WAN bridge for public subnet uplink ---
if [ -n "$WAN_BRIDGE" ]; then
echo ""
echo "Step 3b: Configuring WAN bridge ($WAN_BRIDGE) for public subnet uplink..."
case "$WAN_BRIDGE_MODE" in
existing)
# Already an OVS bridge (from a previous run or explicit --wan-bridge).
if ! sudo ovs-vsctl br-exists "$WAN_BRIDGE" 2>/dev/null; then
sudo ovs-vsctl --may-exist add-br "$WAN_BRIDGE"
echo " created OVS bridge: $WAN_BRIDGE"
fi
sudo ip link set "$WAN_BRIDGE" up
echo " $WAN_BRIDGE: OVS bridge, up"
;;
veth)
# Linux bridge detected (e.g. br-wan from cloud-init/netplan).
# OVN bridge-mappings require an OVS bridge. Rather than converting
# the Linux bridge (destructive, causes WAN interruption), we create
# a separate OVS bridge and link them with a veth pair:
#
# br-wan (Linux, keeps IP/routes) ←→ veth pair ←→ br-ext (OVS, for OVN)
#
# No network interruption. The Linux bridge is untouched.
# Create OVS bridge
if ! sudo ovs-vsctl br-exists "$WAN_BRIDGE" 2>/dev/null; then
sudo ovs-vsctl --may-exist add-br "$WAN_BRIDGE"
echo " created OVS bridge: $WAN_BRIDGE"
fi
sudo ip link set "$WAN_BRIDGE" up
# Create veth pair (idempotent)
if ! ip link show veth-wan-br >/dev/null 2>&1; then
sudo ip link add veth-wan-br type veth peer name veth-wan-ovs
echo " created veth pair: veth-wan-br ↔ veth-wan-ovs"
else
echo " veth pair already exists: veth-wan-br ↔ veth-wan-ovs"
fi
# Enslave veth-wan-br to the Linux bridge
if ! ip link show veth-wan-br 2>/dev/null | grep -q "master $LINUX_BRIDGE"; then
sudo ip link set veth-wan-br master "$LINUX_BRIDGE"
echo " veth-wan-br → $LINUX_BRIDGE (Linux bridge)"
fi
sudo ip link set veth-wan-br up
# Add veth-wan-ovs to the OVS bridge
if ! sudo ovs-vsctl port-to-br veth-wan-ovs >/dev/null 2>&1; then
sudo ovs-vsctl --may-exist add-port "$WAN_BRIDGE" veth-wan-ovs
echo " veth-wan-ovs → $WAN_BRIDGE (OVS bridge)"
fi
sudo ip link set veth-wan-ovs up
echo " $LINUX_BRIDGE (Linux) ↔ veth pair ↔ $WAN_BRIDGE (OVS)"
echo " $LINUX_BRIDGE keeps its IP and routes — no interruption"
;;
direct)
# Add WAN NIC directly to OVS bridge. The NIC becomes an OVS slave —
# its IP (if any) is no longer reachable from the host. The user has
# confirmed this NIC is NOT their SSH connection.
if ! ip link show "$WAN_IFACE" >/dev/null 2>&1; then
echo " ERROR: interface $WAN_IFACE does not exist"
echo " Available interfaces:"
ip -o link show | awk -F': ' '{print " " $2}'
exit 1
fi
sudo ovs-vsctl --may-exist add-br "$WAN_BRIDGE"
sudo ip link set "$WAN_BRIDGE" up
if sudo ovs-vsctl port-to-br "$WAN_IFACE" >/dev/null 2>&1; then
echo " $WAN_IFACE already on $(sudo ovs-vsctl port-to-br "$WAN_IFACE")"
else
sudo ovs-vsctl --may-exist add-port "$WAN_BRIDGE" "$WAN_IFACE"
echo " added $WAN_IFACE directly to $WAN_BRIDGE"
fi
sudo ip link set "$WAN_IFACE" up
echo " $WAN_BRIDGE: direct bridge on $WAN_IFACE"
echo " NOTE: $WAN_IFACE is now an OVS port — no host IP on this NIC"
;;
macvlan)
# Create a macvlan sub-interface in bridge mode off the WAN NIC.
# The host keeps its IP on the parent NIC — SSH-safe. OVN localnet
# traffic flows through the macvlan to the physical wire.
if ! ip link show "$WAN_IFACE" >/dev/null 2>&1; then
echo " ERROR: interface $WAN_IFACE does not exist"
echo " Available interfaces:"
ip -o link show | awk -F': ' '{print " " $2}'
exit 1
fi
MACVLAN_NAME="spx-ext-${WAN_IFACE}"
sudo ovs-vsctl --may-exist add-br "$WAN_BRIDGE"
sudo ip link set "$WAN_BRIDGE" up
if ip link show "$MACVLAN_NAME" >/dev/null 2>&1; then
echo " macvlan $MACVLAN_NAME already exists"
else
sudo ip link add "$MACVLAN_NAME" link "$WAN_IFACE" type macvlan mode bridge
echo " created macvlan: $MACVLAN_NAME (bridge mode) on $WAN_IFACE"
fi
sudo ip link set "$MACVLAN_NAME" up
sudo ovs-vsctl --may-exist add-port "$WAN_BRIDGE" "$MACVLAN_NAME"
echo " $WAN_BRIDGE: macvlan port $MACVLAN_NAME on $WAN_IFACE"
echo " NOTE: host keeps its IP on $WAN_IFACE (SSH-safe)"
echo " QUIRK: host cannot reach VMs at their public IPs (macvlan isolation)"
;;
esac
# --- DHCP: obtain gateway IP for OVN SNAT ---
if [ "$EXTERNAL_DHCP" = true ]; then
echo ""
echo "Step 3c: Obtaining external gateway IP via DHCP..."
# For macvlan mode, DHCP on the macvlan interface (it has L2 access to WAN).
# For direct/existing bridge, DHCP on the bridge itself.
if [ "$WAN_BRIDGE_MODE" = "macvlan" ]; then
DHCP_IFACE="spx-ext-${WAN_IFACE}"
else
DHCP_IFACE="$WAN_BRIDGE"
fi
# Run DHCP client to get a lease
if command -v dhcpcd >/dev/null 2>&1; then
sudo dhcpcd --waitip=4 --timeout 15 "$DHCP_IFACE" 2>/dev/null || true
elif command -v dhclient >/dev/null 2>&1; then
sudo dhclient -1 -timeout 15 "$DHCP_IFACE" 2>/dev/null || true
else
echo " WARNING: no DHCP client found (dhcpcd or dhclient)"
echo " Install dhcpcd-base or isc-dhcp-client, or set gateway_ip manually"
fi
# Read the obtained IP
DHCP_IP=$(ip -4 addr show dev "$DHCP_IFACE" 2>/dev/null | awk '/inet /{print $2}' | head -1 | cut -d/ -f1)
if [ -n "$DHCP_IP" ]; then
echo " DHCP obtained: $DHCP_IP on $DHCP_IFACE"
# Write the gateway IP to the spinifex config so vpcd can use it
CONFIG_DIR="${CONFIG_DIR:-$HOME/spinifex/config}"
CONFIG_FILE="$CONFIG_DIR/spinifex.toml"
if [ -f "$CONFIG_FILE" ]; then
if grep -q "gateway_ip" "$CONFIG_FILE"; then
sed -i "s/gateway_ip.*/gateway_ip = \"$DHCP_IP\"/" "$CONFIG_FILE"
else
sed -i "/^gateway *=.*/a gateway_ip = \"$DHCP_IP\"" "$CONFIG_FILE"
fi
echo " Updated $CONFIG_FILE with gateway_ip = $DHCP_IP"
else
echo " WARNING: $CONFIG_FILE not found — set gateway_ip manually"
fi
else
echo " WARNING: DHCP failed to obtain IP on $DHCP_IFACE"
echo " VMs will not have external connectivity until gateway_ip is configured"
fi
fi
fi
# --- Step 4: Configure OVN external_ids ---
echo ""
echo "Step 4: Setting OVS external_ids for OVN..."
sudo ovs-vsctl set Open_vSwitch . \
external_ids:system-id="$CHASSIS_ID" \
external_ids:ovn-remote="$OVN_REMOTE" \
external_ids:ovn-encap-ip="$ENCAP_IP" \
external_ids:ovn-encap-type="geneve"
if [ -n "$WAN_BRIDGE" ]; then
BRIDGE_MAPPINGS="external:${WAN_BRIDGE}"
sudo ovs-vsctl set Open_vSwitch . \
external_ids:system-id="$CHASSIS_ID" \
external_ids:ovn-remote="$OVN_REMOTE" \
external_ids:ovn-encap-ip="$ENCAP_IP" \
external_ids:ovn-encap-type="geneve" \
external_ids:ovn-bridge-mappings="$BRIDGE_MAPPINGS"
echo " ovn-bridge-mappings: $BRIDGE_MAPPINGS"
else
sudo ovs-vsctl set Open_vSwitch . \
external_ids:system-id="$CHASSIS_ID" \
external_ids:ovn-remote="$OVN_REMOTE" \
external_ids:ovn-encap-ip="$ENCAP_IP" \
external_ids:ovn-encap-type="geneve"
fi
echo " system-id: $CHASSIS_ID"
echo " ovn-remote: $OVN_REMOTE"
@@ -311,6 +638,40 @@ else
OK=false
fi
# Check WAN bridge (only if configured)
if [ -n "$WAN_BRIDGE" ]; then
if sudo ovs-vsctl br-exists "$WAN_BRIDGE"; then
echo " $WAN_BRIDGE:$(printf '%*s' $((15 - ${#WAN_BRIDGE})) '') OK"
if [ "$WAN_BRIDGE_MODE" = "veth" ]; then
if ip link show veth-wan-br >/dev/null 2>&1 && ip link show veth-wan-ovs >/dev/null 2>&1; then
echo " veth pair: OK (veth-wan-br ↔ veth-wan-ovs)"
echo " linux bridge: $LINUX_BRIDGE (untouched)"
else
echo " veth pair: FAILED (veth-wan-br/veth-wan-ovs not found)"
OK=false
fi
elif [ "$WAN_BRIDGE_MODE" = "direct" ]; then
if sudo ovs-vsctl port-to-br "$WAN_IFACE" >/dev/null 2>&1; then
echo " direct bridge: OK ($WAN_IFACE on $WAN_BRIDGE)"
else
echo " direct bridge: FAILED ($WAN_IFACE not on $WAN_BRIDGE)"
OK=false
fi
elif [ "$WAN_BRIDGE_MODE" = "macvlan" ]; then
MACVLAN_NAME="spx-ext-${WAN_IFACE}"
if ip link show "$MACVLAN_NAME" >/dev/null 2>&1; then
echo " macvlan: OK ($MACVLAN_NAME)"
else
echo " macvlan: FAILED ($MACVLAN_NAME not found)"
OK=false
fi
fi
else
echo " $WAN_BRIDGE:$(printf '%*s' $((15 - ${#WAN_BRIDGE})) '') FAILED"
OK=false
fi
fi
# Check ovn-controller
if sudo ovs-appctl -t ovn-controller version >/dev/null 2>&1 || systemctl is-active --quiet ovn-controller 2>/dev/null; then
echo " ovn-controller: OK"
+8 -6
View File
@@ -127,7 +127,7 @@ install_apt_deps() {
$QEMU_PACKAGES qemu-utils qemu-kvm \
libvirt-daemon-system libvirt-clients \
jq curl iproute2 netcat-openbsd wget unzip xz-utils file \
ovn-central ovn-host openvswitch-switch \
ovn-central ovn-host openvswitch-switch dhcpcd-base \
> /dev/null
info "System dependencies installed"
@@ -342,18 +342,20 @@ print_summary() {
echo ""
echo " 1. Setup OVN networking:"
echo " sudo /usr/local/share/spinifex/setup-ovn.sh --management"
echo " # If WAN is a physical NIC (not a bridge), add:"
echo " # --wan-bridge=br-wan --wan-iface=<WAN_NIC> (dedicated WAN NIC)"
echo " # --macvlan --wan-iface=<WAN_NIC> (single-NIC, SSH-safe)"
echo ""
echo " 2. Initialize a region:"
echo " sudo spx admin init --region <region> --az <az> --node node1 --nodes 1"
echo ""
echo " 3. Trust the CA certificate:"
echo " sudo cp /etc/spinifex/ca.pem /usr/local/share/ca-certificates/spinifex-ca.crt"
echo " sudo update-ca-certificates"
echo " 2. Setup OVN networking:"
echo " sudo /usr/local/share/spinifex/setup-ovn.sh --management"
echo ""
echo " 4. Start services:"
echo " 3. Start services:"
echo " sudo systemctl start spinifex.target"
echo ""
echo " 5. Verify:"
echo " 4. Verify:"
echo " export AWS_PROFILE=spinifex"
echo " aws ec2 describe-instance-types"
echo ""
+31 -1
View File
@@ -150,13 +150,43 @@ has_service "predastore" && stop_service "predastore" "$PID_DIR"
has_service "nats" && stop_service "nats" "$PID_DIR"
# Stop OVN networking (prevents idle CPU burn when spinifex isn't running)
# CRITICAL: Do NOT stop openvswitch-switch if OVS carries WAN traffic.
# This can happen when an OVS bridge has a WAN IP (direct bridge mode) or
# when a veth pair links an OVS bridge to a Linux WAN bridge (veth mode).
# Stopping OVS would destroy the bridge and break WAN connectivity.
# OVS is infrastructure — only stop ovn-controller and ovn-central.
if pidof systemd >/dev/null 2>&1; then
echo "🛑 Stopping OVN networking..."
sudo systemctl stop ovn-controller 2>/dev/null && echo "✅ ovn-controller stopped" || true
if systemctl is-active --quiet ovn-central 2>/dev/null; then
sudo systemctl stop ovn-central 2>/dev/null && echo "✅ ovn-central stopped" || true
fi
sudo systemctl stop openvswitch-switch 2>/dev/null && echo "✅ openvswitch-switch stopped" || true
# Only stop OVS if tearing it down won't kill WAN connectivity.
# Two cases where OVS is load-bearing for WAN:
# 1. An OVS bridge has a WAN IP directly (direct bridge mode)
# 2. A veth pair links a Linux bridge to an OVS bridge (veth mode) —
# stopping OVS destroys the OVS bridge end, breaking the link
WAN_BRIDGE_HAS_IP=false
VETH_LINKS_WAN=false
for br in $(sudo ovs-vsctl list-br 2>/dev/null); do
if [ "$br" = "br-int" ]; then continue; fi
if ip -4 addr show "$br" 2>/dev/null | grep -q "inet "; then
WAN_BRIDGE_HAS_IP=true
break
fi
done
# Check if a veth pair links an OVS bridge to a Linux bridge with a WAN IP
if ip link show veth-wan-br >/dev/null 2>&1; then
VETH_LINKS_WAN=true
fi
if [ "$WAN_BRIDGE_HAS_IP" = true ]; then
echo "⚠️ Skipping openvswitch-switch stop ($br has WAN IP — stopping would kill connectivity)"
elif [ "$VETH_LINKS_WAN" = true ]; then
echo "⚠️ Skipping openvswitch-switch stop (veth pair links OVS to WAN bridge)"
else
sudo systemctl stop openvswitch-switch 2>/dev/null && echo "✅ openvswitch-switch stopped" || true
fi
fi
echo ""
+32 -10
View File
@@ -64,8 +64,31 @@ type ConfigSettings struct {
OVNNBAddr string
OVNSBAddr string
// External networking for public subnets
ExternalMode string // "pool" or "" (disabled)
ExternalIface string // WAN NIC name (e.g., "eth0", "eth1")
WanBridge string // OVS bridge for WAN traffic (default "br-wan")
ExternalDHCP bool // Obtain gateway IP via DHCP on macvlan/bridge
PoolName string // External pool name (e.g., "wan")
PoolSource string // IP source: "static" (default) or "dhcp" (from router DHCP)
PoolStart string // First IP in external pool range (static source only)
PoolEnd string // Last IP in external pool range (static source only)
PoolGateway string // WAN gateway IP
PoolGatewayIP string // Explicit SNAT IP (for nat mode without DHCP)
PoolPrefixLen int // Subnet prefix length (default 24)
PoolDNSServers []string // DNS servers for VM DHCP (auto-detected from host)
// Other nodes in the cluster (for config source of truth)
RemoteNodes []RemoteNode
// Bootstrap: pre-generated default VPC IDs for vpcd reconciliation.
// Written by admin init so [bootstrap] exists before services start.
BootstrapAccountId string
BootstrapVpcId string
BootstrapSubnetId string
BootstrapIgwId string
BootstrapCidr string
BootstrapSubnetCidr string
}
// PredastoreNodeConfig describes a single Predastore node for multi-node config generation.
@@ -81,7 +104,6 @@ type ConfigFile struct {
}
func GenerateConfigFiles(configs []ConfigFile, configSettings ConfigSettings) error {
for _, cfg := range configs {
if err := GenerateConfigFile(cfg.Path, cfg.Template, configSettings); err != nil {
return fmt.Errorf("error creating %s: %v", cfg.Name, err)
@@ -148,11 +170,13 @@ func GenerateCertificatesIfNeeded(configDir string, force bool, bindIP string) (
fmt.Printf(" Certificate: %s\n", serverCertPath)
fmt.Printf(" Key: %s\n", serverKeyPath)
// Print instructions for adding CA to system trust store
fmt.Println("\n📋 To trust the Spinifex CA system-wide (recommended):")
fmt.Printf(" sudo cp %s /usr/local/share/ca-certificates/spinifex-ca.crt\n", caCertPath)
fmt.Println(" sudo update-ca-certificates")
fmt.Println("\n This allows AWS CLI and other tools to trust Spinifex services automatically.")
// Print manual instructions only when not root (root gets auto-install)
if os.Getuid() != 0 {
fmt.Println("\n📋 To trust the Spinifex CA system-wide (recommended):")
fmt.Printf(" sudo cp %s /usr/local/share/ca-certificates/spinifex-ca.crt\n", caCertPath)
fmt.Println(" sudo update-ca-certificates")
fmt.Println("\n This allows AWS CLI and other tools to trust Spinifex services automatically.")
}
} else {
fmt.Println("\n✅ CA and SSL certificates already exist")
}
@@ -178,7 +202,6 @@ func GenerateServerCertOnly(configDir string, bindIP string) error {
}
func CreateServiceDirectories(spxRoot string) {
// Create additional directories
dirs := []string{
filepath.Join(spxRoot, "images"),
@@ -194,7 +217,6 @@ func CreateServiceDirectories(spxRoot string) {
fmt.Println("\n📁 Creating directory structure...")
for _, dir := range dirs {
// Check if directory exists
if _, err := os.Stat(dir); os.IsNotExist(err) {
if err := os.MkdirAll(dir, 0750); err != nil {
@@ -203,7 +225,6 @@ func CreateServiceDirectories(spxRoot string) {
}
}
fmt.Printf("✅ Directory structure created in %s\n", spxRoot)
}
// Helper functions
@@ -239,7 +260,8 @@ func ChownRecursive(path, username string) {
_ = fs.WalkDir(root.FS(), ".", func(p string, d fs.DirEntry, err error) error {
if err != nil {
return nil
slog.Debug("chown walk: skipping inaccessible entry", "path", p, "err", err)
return nil // best-effort: continue past inaccessible entries
}
fullPath := filepath.Join(path, p)
if chownErr := os.Lchown(fullPath, uid, gid); chownErr != nil {
+205
View File
@@ -0,0 +1,205 @@
package admin
import (
"fmt"
"net"
"os/exec"
"strings"
"github.com/mulgadc/spinifex/spinifex/utils"
)
// DetectedInterface represents a network interface discovered on the host.
type DetectedInterface struct {
Name string // e.g., "enp0s3"
IP string // e.g., "192.168.1.21"
Subnet string // e.g., "192.168.0.0/23"
PrefixLen int // e.g., 23
Gateway string // non-empty only for WAN (default route)
Role string // "wan", "lan"
}
// DetectedNetwork holds the auto-detected network topology.
type DetectedNetwork struct {
Interfaces []DetectedInterface
WAN *DetectedInterface // The interface with the default route
LANCount int // Number of non-WAN interfaces
}
// DetectNetwork auto-detects the host's network topology from routing table.
// It identifies the WAN interface (default route) and LAN interfaces.
func DetectNetwork() (*DetectedNetwork, error) {
// Get default route: "default via 192.168.1.1 dev enp0s3 ..."
out, err := exec.Command("ip", "-4", "route", "show", "default").Output()
if err != nil {
return nil, fmt.Errorf("failed to get default route: %w", err)
}
defaultRoute := strings.TrimSpace(string(out))
if defaultRoute == "" {
return nil, fmt.Errorf("no default route found — host has no internet connectivity")
}
// Parse: "default via 192.168.1.1 dev enp0s3 ..."
var wanGateway, wanIface string
fields := strings.Fields(defaultRoute)
for i, f := range fields {
if f == "via" && i+1 < len(fields) {
wanGateway = fields[i+1]
}
if f == "dev" && i+1 < len(fields) {
wanIface = fields[i+1]
}
}
if wanIface == "" || wanGateway == "" {
return nil, fmt.Errorf("could not parse default route: %s", defaultRoute)
}
// Get all routes to find interfaces and their subnets
allRoutes, err := exec.Command("ip", "-4", "route", "show", "proto", "kernel", "scope", "link").Output()
if err != nil {
return nil, fmt.Errorf("failed to get routes: %w", err)
}
// Parse routes: "192.168.0.0/23 dev enp0s3 ... src 192.168.1.21"
seen := make(map[string]bool)
var interfaces []DetectedInterface
for line := range strings.SplitSeq(strings.TrimSpace(string(allRoutes)), "\n") {
if line == "" {
continue
}
fields := strings.Fields(line)
if len(fields) < 3 {
continue
}
subnet := fields[0]
var ifaceName, srcIP string
for i, f := range fields {
if f == "dev" && i+1 < len(fields) {
ifaceName = fields[i+1]
}
if f == "src" && i+1 < len(fields) {
srcIP = fields[i+1]
}
}
if ifaceName == "" || srcIP == "" {
continue
}
// Skip virtual/docker/ovs interfaces (but allow the default route device)
if isVirtualInterface(ifaceName, wanIface) {
continue
}
// Skip duplicates (same interface may have multiple routes)
if seen[ifaceName] {
continue
}
seen[ifaceName] = true
prefixLen := 24 // default
if _, ipNet, err := net.ParseCIDR(subnet); err == nil {
ones, _ := ipNet.Mask.Size()
prefixLen = ones
}
iface := DetectedInterface{
Name: ifaceName,
IP: srcIP,
Subnet: subnet,
PrefixLen: prefixLen,
}
if ifaceName == wanIface {
iface.Gateway = wanGateway
iface.Role = "wan"
} else {
iface.Role = "lan"
}
interfaces = append(interfaces, iface)
}
if len(interfaces) == 0 {
return nil, fmt.Errorf("no usable network interfaces found")
}
result := &DetectedNetwork{
Interfaces: interfaces,
}
for i := range interfaces {
if interfaces[i].Role == "wan" {
result.WAN = &interfaces[i]
} else {
result.LANCount++
}
}
return result, nil
}
// SuggestPoolRange suggests a default external IP pool range based on the
// WAN subnet. Places the pool at the high end of the subnet to avoid
// collisions with typical DHCP ranges (which start low).
func SuggestPoolRange(wan *DetectedInterface) (start, end string) {
_, ipNet, err := net.ParseCIDR(wan.Subnet)
if err != nil {
return "", ""
}
// Calculate broadcast address
ip := ipNet.IP.To4()
mask := ipNet.Mask
broadcast := make(net.IP, 4)
for i := range ip {
broadcast[i] = ip[i] | ^mask[i]
}
// Pool: broadcast - 100 to broadcast - 5
// e.g., for 192.168.0.0/23 (broadcast 192.168.1.255):
// start = 192.168.1.155, end = 192.168.1.250
endIP := make(net.IP, 4)
copy(endIP, broadcast)
endIP[3] -= 5 // Leave some room at top
startIP := make(net.IP, 4)
copy(startIP, endIP)
// Go back 100 IPs
carry := 100
for i := 3; i >= 0 && carry > 0; i-- {
val := int(startIP[i]) - carry
if val < 0 {
startIP[i] = utils.SafeIntToUint8(256 + val)
carry = 1
} else {
startIP[i] = utils.SafeIntToUint8(val)
carry = 0
}
}
return startIP.String(), endIP.String()
}
// isVirtualInterface returns true for interfaces that shouldn't be
// considered as physical NICs. The defaultRouteDev parameter is the
// interface carrying the default route — it's always considered physical
// even if its name matches a virtual prefix (e.g., br-wan).
func isVirtualInterface(name string, defaultRouteDev string) bool {
// The default route interface is always considered physical — it's the
// WAN uplink, even if it's a bridge (br-wan, br-external, etc.).
if defaultRouteDev != "" && name == defaultRouteDev {
return false
}
prefixes := []string{
"docker", "br-", "veth", "ovs", "virbr",
"tun", "tap", "tailscale", "wg", "lo",
}
for _, p := range prefixes {
if strings.HasPrefix(name, p) {
return true
}
}
return false
}
+55 -7
View File
@@ -5,15 +5,51 @@ import (
"log/slog"
"os"
"slices"
"strings"
"github.com/spf13/viper"
)
type ClusterConfig struct {
Epoch uint64 `mapstructure:"epoch"` // bump when leader commits changes
Node string `mapstructure:"node"` // my node name
Version string `mapstructure:"version"` // spinifex version
Nodes map[string]Config `mapstructure:"nodes"` // full config for every node
Epoch uint64 `mapstructure:"epoch"` // bump when leader commits changes
Node string `mapstructure:"node"` // my node name
Version string `mapstructure:"version"` // spinifex version
Network NetworkConfig `mapstructure:"network"` // cluster-wide external network settings
Bootstrap BootstrapConfig `mapstructure:"bootstrap"` // default VPC IDs for OVN reconciliation
Nodes map[string]Config `mapstructure:"nodes"` // full config for every node
}
// ExternalPool defines a range of routable IPs that Spinifex manages for public subnets.
type ExternalPool struct {
Name string `mapstructure:"name"` // Pool identifier (e.g., "wan", "dc1-primary")
Source string `mapstructure:"source"` // IP source: "static" (default) or "dhcp" (from router DHCP)
RangeStart string `mapstructure:"range_start"` // First IP in range (static source only)
RangeEnd string `mapstructure:"range_end"` // Last IP in range (static source only)
Gateway string `mapstructure:"gateway"` // WAN default gateway (next hop for 0.0.0.0/0)
GatewayIP string `mapstructure:"gateway_ip"` // OVN router external IP (override; defaults to first IP in range)
PrefixLen int `mapstructure:"prefix_len"` // Subnet mask (default 24)
DNSServers []string `mapstructure:"dns_servers"` // DNS servers for VM DHCP (auto-detected from host; fallback: 8.8.8.8, 1.1.1.1)
Region string `mapstructure:"region"` // Scope to region (optional — empty means any region)
AZ string `mapstructure:"az"` // Scope to AZ (optional — empty means any AZ in region)
}
// NetworkConfig holds cluster-wide external network settings.
type NetworkConfig struct {
ExternalMode string `mapstructure:"external_mode"` // "pool" or "" (disabled)
ExternalDHCP bool `mapstructure:"external_dhcp"` // Gateway IP obtained via DHCP (pool/dhcp source)
ExternalPools []ExternalPool `mapstructure:"external_pools"` // One or more IP pools
}
// BootstrapConfig holds the default VPC infrastructure IDs written by admin init.
// vpcd reads this on startup to ensure OVN topology exists for the bootstrap VPC,
// covering the case where admin init ran before services were started.
type BootstrapConfig struct {
AccountID string `mapstructure:"account_id"`
VpcId string `mapstructure:"vpc_id"`
SubnetId string `mapstructure:"subnet_id"`
IgwId string `mapstructure:"igw_id"`
Cidr string `mapstructure:"cidr"`
SubnetCidr string `mapstructure:"subnet_cidr"`
}
// Config holds all configuration for the application
@@ -53,8 +89,11 @@ type ViperblockConfig struct {
// VPCDConfig holds the VPC daemon (vpcd) configuration.
type VPCDConfig struct {
OVNNBAddr string `mapstructure:"ovn_nb_addr"` // OVN Northbound DB address (e.g., "tcp:127.0.0.1:6641")
OVNSBAddr string `mapstructure:"ovn_sb_addr"` // OVN Southbound DB address (e.g., "tcp:127.0.0.1:6642")
OVNNBAddr string `mapstructure:"ovn_nb_addr"` // OVN Northbound DB address (e.g., "tcp:127.0.0.1:6641")
OVNSBAddr string `mapstructure:"ovn_sb_addr"` // OVN Southbound DB address (e.g., "tcp:127.0.0.1:6642")
ExternalInterface string `mapstructure:"external_interface"` // WAN NIC name (e.g., "eth1", "enp0s3") — the physical NIC on the WAN bridge
WanBridge string `mapstructure:"wan_bridge"` // OVS bridge for WAN traffic (default "br-wan", maps to OVN "external" network)
BridgeMode string `mapstructure:"bridge_mode"` // "direct" or "macvlan" (auto-detected if empty)
}
type PredastoreConfig struct {
@@ -148,7 +187,6 @@ func LoadConfig(configPath string) (*ClusterConfig, error) {
return nil, fmt.Errorf("error reading config file: %w", err)
}
//fmt.Fprintf(os.Stderr, "Using config file: %s\n", viper.ConfigFileUsed())
} else {
fmt.Fprintf(os.Stderr, "Config file not found: %s, using environment variables and defaults\n", configPath)
}
@@ -160,5 +198,15 @@ func LoadConfig(configPath string) (*ClusterConfig, error) {
return nil, fmt.Errorf("error unmarshaling config: %w", err)
}
// Normalize the local node's bind address: 0.0.0.0 means "listen on all
// interfaces" but is not a valid connect address. Only rewrite for the
// local node — remote nodes use real IPs that must not be changed.
if local, ok := config.Nodes[config.Node]; ok {
if strings.HasPrefix(local.Predastore.Host, "0.0.0.0") {
local.Predastore.Host = strings.Replace(local.Predastore.Host, "0.0.0.0", "127.0.0.1", 1)
config.Nodes[config.Node] = local
}
}
return &config, nil
}
+136 -1
View File
@@ -205,7 +205,7 @@ expected_nodes = 3
assert.Equal(t, "0.0.0.0:4222", n.NATS.Host)
assert.Equal(t, "secret-token", n.NATS.ACL.Token)
assert.Equal(t, "test-subject", n.NATS.Sub.Subject)
assert.Equal(t, "0.0.0.0:8443", n.Predastore.Host)
assert.Equal(t, "127.0.0.1:8443", n.Predastore.Host) // 0.0.0.0 normalized to loopback
assert.Equal(t, "mybucket", n.Predastore.Bucket)
assert.Equal(t, "AK", n.Predastore.AccessKey)
assert.Equal(t, "/data", n.Predastore.BaseDir)
@@ -300,6 +300,141 @@ func TestNodeBaseDir_EmptyBaseDir(t *testing.T) {
assert.Equal(t, "", cc.NodeBaseDir())
}
// Tests for NetworkConfig (external pools)
func TestLoadConfig_NetworkExternalPools(t *testing.T) {
resetViper(t)
dir := t.TempDir()
path := filepath.Join(dir, "spinifex.toml")
toml := `
node = "n1"
[network]
external_mode = "pool"
[[network.external_pools]]
name = "wan"
range_start = "192.168.1.150"
range_end = "192.168.1.250"
gateway = "192.168.1.1"
prefix_len = 24
[[network.external_pools]]
name = "overflow"
range_start = "10.0.0.2"
range_end = "10.0.0.254"
gateway = "10.0.0.1"
prefix_len = 24
region = "us-east-1"
az = "us-east-1a"
[nodes.n1]
region = "us-east-1"
[nodes.n1.vpcd]
ovn_nb_addr = "tcp:127.0.0.1:6641"
external_interface = "enp0s3"
`
require.NoError(t, os.WriteFile(path, []byte(toml), 0600))
cfg, err := LoadConfig(path)
require.NoError(t, err)
assert.Equal(t, "pool", cfg.Network.ExternalMode)
require.Len(t, cfg.Network.ExternalPools, 2)
wan := cfg.Network.ExternalPools[0]
assert.Equal(t, "wan", wan.Name)
assert.Equal(t, "192.168.1.150", wan.RangeStart)
assert.Equal(t, "192.168.1.250", wan.RangeEnd)
assert.Equal(t, "192.168.1.1", wan.Gateway)
assert.Equal(t, 24, wan.PrefixLen)
assert.Empty(t, wan.Region)
assert.Empty(t, wan.AZ)
overflow := cfg.Network.ExternalPools[1]
assert.Equal(t, "overflow", overflow.Name)
assert.Equal(t, "us-east-1", overflow.Region)
assert.Equal(t, "us-east-1a", overflow.AZ)
n := cfg.Nodes["n1"]
assert.Equal(t, "enp0s3", n.VPCD.ExternalInterface)
}
func TestLoadConfig_NetworkPoolDHCPMode(t *testing.T) {
resetViper(t)
dir := t.TempDir()
path := filepath.Join(dir, "spinifex.toml")
toml := `
node = "n1"
[network]
external_mode = "pool"
[[network.external_pools]]
name = "wan"
source = "dhcp"
gateway = "192.168.1.1"
gateway_ip = "192.168.1.100"
prefix_len = 24
[nodes.n1]
region = "us-east-1"
`
require.NoError(t, os.WriteFile(path, []byte(toml), 0600))
cfg, err := LoadConfig(path)
require.NoError(t, err)
assert.Equal(t, "pool", cfg.Network.ExternalMode)
require.Len(t, cfg.Network.ExternalPools, 1)
assert.Equal(t, "dhcp", cfg.Network.ExternalPools[0].Source)
assert.Equal(t, "192.168.1.100", cfg.Network.ExternalPools[0].GatewayIP)
}
func TestLoadConfig_NetworkDisabledByDefault(t *testing.T) {
resetViper(t)
dir := t.TempDir()
path := filepath.Join(dir, "spinifex.toml")
require.NoError(t, os.WriteFile(path, []byte(`node = "n1"`), 0600))
cfg, err := LoadConfig(path)
require.NoError(t, err)
assert.Empty(t, cfg.Network.ExternalMode)
assert.Empty(t, cfg.Network.ExternalPools)
}
func TestLoadConfig_ExternalInterfacePerNode(t *testing.T) {
resetViper(t)
dir := t.TempDir()
path := filepath.Join(dir, "spinifex.toml")
toml := `
node = "n1"
[nodes.n1.vpcd]
external_interface = "eth1"
[nodes.n2.vpcd]
external_interface = "eno2"
[nodes.n3.vpcd]
external_interface = "enp3s0"
`
require.NoError(t, os.WriteFile(path, []byte(toml), 0600))
cfg, err := LoadConfig(path)
require.NoError(t, err)
assert.Equal(t, "eth1", cfg.Nodes["n1"].VPCD.ExternalInterface)
assert.Equal(t, "eno2", cfg.Nodes["n2"].VPCD.ExternalInterface)
assert.Equal(t, "enp3s0", cfg.Nodes["n3"].VPCD.ExternalInterface)
}
// Tests for ViperblockConfig
func TestLoadConfig_ViperblockShardWAL_Explicit(t *testing.T) {
+196 -69
View File
@@ -28,14 +28,17 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/go-chi/chi/v5"
"github.com/mulgadc/spinifex/spinifex/admin"
"github.com/mulgadc/spinifex/spinifex/awserrors"
"github.com/mulgadc/spinifex/spinifex/config"
handlers_ec2_account "github.com/mulgadc/spinifex/spinifex/handlers/ec2/account"
handlers_ec2_eigw "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eigw"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
handlers_ec2_igw "github.com/mulgadc/spinifex/spinifex/handlers/ec2/igw"
handlers_ec2_image "github.com/mulgadc/spinifex/spinifex/handlers/ec2/image"
handlers_ec2_instance "github.com/mulgadc/spinifex/spinifex/handlers/ec2/instance"
handlers_ec2_key "github.com/mulgadc/spinifex/spinifex/handlers/ec2/key"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
handlers_ec2_snapshot "github.com/mulgadc/spinifex/spinifex/handlers/ec2/snapshot"
handlers_ec2_tags "github.com/mulgadc/spinifex/spinifex/handlers/ec2/tags"
handlers_ec2_volume "github.com/mulgadc/spinifex/spinifex/handlers/ec2/volume"
@@ -86,28 +89,32 @@ type ResourceManager struct {
natsConn *nats.Conn
instanceSubs map[string]*nats.Subscription
handler nats.MsgHandler
nodeID string // node identifier for node-specific topic subscriptions
}
// Daemon represents the main daemon service
type Daemon struct {
node string
clusterConfig *config.ClusterConfig
config *config.Config
natsConn *nats.Conn
resourceMgr *ResourceManager
instanceService *handlers_ec2_instance.InstanceServiceImpl
keyService *handlers_ec2_key.KeyServiceImpl
imageService *handlers_ec2_image.ImageServiceImpl
volumeService *handlers_ec2_volume.VolumeServiceImpl
accountService *handlers_ec2_account.AccountSettingsServiceImpl
snapshotService *handlers_ec2_snapshot.SnapshotServiceImpl
tagsService *handlers_ec2_tags.TagsServiceImpl
eigwService *handlers_ec2_eigw.EgressOnlyIGWServiceImpl
igwService *handlers_ec2_igw.IGWServiceImpl
vpcService *handlers_ec2_vpc.VPCServiceImpl
ctx context.Context
cancel context.CancelFunc
shutdownWg sync.WaitGroup
node string
clusterConfig *config.ClusterConfig
config *config.Config
natsConn *nats.Conn
resourceMgr *ResourceManager
instanceService *handlers_ec2_instance.InstanceServiceImpl
keyService *handlers_ec2_key.KeyServiceImpl
imageService *handlers_ec2_image.ImageServiceImpl
volumeService *handlers_ec2_volume.VolumeServiceImpl
accountService *handlers_ec2_account.AccountSettingsServiceImpl
snapshotService *handlers_ec2_snapshot.SnapshotServiceImpl
tagsService *handlers_ec2_tags.TagsServiceImpl
eigwService *handlers_ec2_eigw.EgressOnlyIGWServiceImpl
igwService *handlers_ec2_igw.IGWServiceImpl
placementGroupService *handlers_ec2_placementgroup.PlacementGroupServiceImpl
vpcService *handlers_ec2_vpc.VPCServiceImpl
eipService *handlers_ec2_eip.EIPServiceImpl
externalIPAM *handlers_ec2_vpc.ExternalIPAM
ctx context.Context
cancel context.CancelFunc
shutdownWg sync.WaitGroup
// Local VM Instances
Instances vm.Instances
@@ -302,13 +309,13 @@ func (rm *ResourceManager) GetResourceStats() (totalVCPU int, totalMemGB float64
remainingMem := rm.availableMem - rm.allocatedMem
for _, it := range rm.instanceTypes {
cap := resourceStatsForType(remainingVCPU, remainingMem, it)
if cap.VCPU == 0 || cap.MemoryGB == 0 {
typeCap := resourceStatsForType(remainingVCPU, remainingMem, it)
if typeCap.VCPU == 0 || typeCap.MemoryGB == 0 {
continue
}
caps = append(caps, cap)
caps = append(caps, typeCap)
}
return
return totalVCPU, totalMemGB, allocVCPU, allocMemGB, caps
}
// SetConfigPath sets the configuration file path for cluster management
@@ -386,15 +393,37 @@ func (d *Daemon) subscribeAll() error {
{"ec2.DescribeInternetGateways", d.handleEC2DescribeInternetGateways, "spinifex-workers"},
{"ec2.AttachInternetGateway", d.handleEC2AttachInternetGateway, "spinifex-workers"},
{"ec2.DetachInternetGateway", d.handleEC2DetachInternetGateway, "spinifex-workers"},
{"ec2.CreatePlacementGroup", d.handleEC2CreatePlacementGroup, "spinifex-workers"},
{"ec2.DeletePlacementGroup", d.handleEC2DeletePlacementGroup, "spinifex-workers"},
{"ec2.DescribePlacementGroups", d.handleEC2DescribePlacementGroups, "spinifex-workers"},
{"ec2.ReserveSpreadNodes", d.handleEC2ReserveSpreadNodes, "spinifex-workers"},
{"ec2.FinalizeSpreadInstances", d.handleEC2FinalizeSpreadInstances, "spinifex-workers"},
{"ec2.ReleaseSpreadNodes", d.handleEC2ReleaseSpreadNodes, "spinifex-workers"},
{"ec2.RemoveInstanceFromPlacementGroup", d.handleEC2RemoveInstanceFromPlacementGroup, "spinifex-workers"},
{"ec2.ReserveClusterNode", d.handleEC2ReserveClusterNode, "spinifex-workers"},
{"ec2.FinalizeClusterInstances", d.handleEC2FinalizeClusterInstances, "spinifex-workers"},
{"ec2.CreateVpc", d.handleEC2CreateVpc, "spinifex-workers"},
{"ec2.DeleteVpc", d.handleEC2DeleteVpc, "spinifex-workers"},
{"ec2.DescribeVpcs", d.handleEC2DescribeVpcs, "spinifex-workers"},
{"ec2.CreateSubnet", d.handleEC2CreateSubnet, "spinifex-workers"},
{"ec2.DeleteSubnet", d.handleEC2DeleteSubnet, "spinifex-workers"},
{"ec2.DescribeSubnets", d.handleEC2DescribeSubnets, "spinifex-workers"},
{"ec2.ModifySubnetAttribute", d.handleEC2ModifySubnetAttribute, "spinifex-workers"},
{"ec2.CreateNetworkInterface", d.handleEC2CreateNetworkInterface, "spinifex-workers"},
{"ec2.DeleteNetworkInterface", d.handleEC2DeleteNetworkInterface, "spinifex-workers"},
{"ec2.DescribeNetworkInterfaces", d.handleEC2DescribeNetworkInterfaces, "spinifex-workers"},
{"ec2.CreateSecurityGroup", d.handleEC2CreateSecurityGroup, "spinifex-workers"},
{"ec2.DeleteSecurityGroup", d.handleEC2DeleteSecurityGroup, "spinifex-workers"},
{"ec2.DescribeSecurityGroups", d.handleEC2DescribeSecurityGroups, "spinifex-workers"},
{"ec2.AuthorizeSecurityGroupIngress", d.handleEC2AuthorizeSecurityGroupIngress, "spinifex-workers"},
{"ec2.AuthorizeSecurityGroupEgress", d.handleEC2AuthorizeSecurityGroupEgress, "spinifex-workers"},
{"ec2.RevokeSecurityGroupIngress", d.handleEC2RevokeSecurityGroupIngress, "spinifex-workers"},
{"ec2.RevokeSecurityGroupEgress", d.handleEC2RevokeSecurityGroupEgress, "spinifex-workers"},
{"ec2.AllocateAddress", d.handleEC2AllocateAddress, "spinifex-workers"},
{"ec2.ReleaseAddress", d.handleEC2ReleaseAddress, "spinifex-workers"},
{"ec2.AssociateAddress", d.handleEC2AssociateAddress, "spinifex-workers"},
{"ec2.DisassociateAddress", d.handleEC2DisassociateAddress, "spinifex-workers"},
{"ec2.DescribeAddresses", d.handleEC2DescribeAddresses, "spinifex-workers"},
{"ec2.ModifyInstanceAttribute", d.handleEC2ModifyInstanceAttribute, "spinifex-workers"},
{"ec2.start", d.handleEC2StartStoppedInstance, "spinifex-workers"},
{"ec2.terminate", d.handleEC2TerminateStoppedInstance, "spinifex-workers"},
@@ -504,6 +533,13 @@ func (d *Daemon) Start() error {
return fmt.Errorf("failed to initialize IGW service: %w", err)
}
d.placementGroupService, err = initServiceWithRetry("placement group service", func() (*handlers_ec2_placementgroup.PlacementGroupServiceImpl, error) {
return handlers_ec2_placementgroup.NewPlacementGroupServiceImplWithNATS(d.config, d.natsConn)
})
if err != nil {
return fmt.Errorf("failed to initialize placement group service: %w", err)
}
d.vpcService, err = initServiceWithRetry("VPC service", func() (*handlers_ec2_vpc.VPCServiceImpl, error) {
return handlers_ec2_vpc.NewVPCServiceImplWithNATS(d.config, d.natsConn)
})
@@ -511,6 +547,53 @@ func (d *Daemon) Start() error {
return fmt.Errorf("failed to initialize VPC service: %w", err)
}
// Initialize external IPAM if pool mode is configured (per-VM public IPs).
// NAT mode only uses SNAT via a shared gateway IP — no per-VM allocation needed.
if d.clusterConfig != nil && d.clusterConfig.Network.ExternalMode == "pool" {
js, jsErr := d.natsConn.JetStream()
if jsErr != nil {
slog.Warn("Failed to get JetStream for external IPAM", "err", jsErr)
} else {
var pools []handlers_ec2_vpc.ExternalPoolConfig
// Resolve WAN bridge name for DHCP pools
wanBridge := ""
if node, ok := d.clusterConfig.Nodes[d.clusterConfig.Node]; ok {
wanBridge = node.VPCD.WanBridge
}
for _, p := range d.clusterConfig.Network.ExternalPools {
pools = append(pools, handlers_ec2_vpc.ExternalPoolConfig{
Name: p.Name,
Source: p.Source,
RangeStart: p.RangeStart,
RangeEnd: p.RangeEnd,
Gateway: p.Gateway,
GatewayIP: p.GatewayIP,
PrefixLen: p.PrefixLen,
Region: p.Region,
AZ: p.AZ,
WanBridge: wanBridge,
})
}
d.externalIPAM, err = handlers_ec2_vpc.NewExternalIPAM(js, pools)
if err != nil {
slog.Warn("Failed to initialize external IPAM", "err", err)
} else {
slog.Info("External IPAM initialized", "mode", d.clusterConfig.Network.ExternalMode, "pools", len(pools))
}
}
}
// Initialize EIP service if external IPAM is available
if d.externalIPAM != nil && d.vpcService != nil {
eipSvc, eipErr := handlers_ec2_eip.NewEIPServiceImpl(d.natsConn, d.externalIPAM, d.vpcService)
if eipErr != nil {
slog.Warn("Failed to initialize EIP service", "err", eipErr)
} else {
d.eipService = eipSvc
slog.Info("EIP service initialized")
}
}
d.accountService, err = initServiceWithRetry("account settings service", func() (*handlers_ec2_account.AccountSettingsServiceImpl, error) {
return handlers_ec2_account.NewAccountSettingsServiceImplWithNATS(d.config, d.natsConn)
})
@@ -518,11 +601,25 @@ func (d *Daemon) Start() error {
return fmt.Errorf("failed to initialize account settings service: %w", err)
}
// Ensure default VPC exists (matches AWS: every account has a default VPC)
// Ensure default VPC exists for system and admin accounts
// (matches AWS: every account has a default VPC with IGW + default SG)
if d.vpcService != nil {
if err := d.vpcService.EnsureDefaultVPC(utils.GlobalAccountID); err != nil {
slog.Error("Failed to ensure default VPC", "error", err)
for _, accountID := range []string{utils.GlobalAccountID, admin.DefaultAccountID()} {
// Pass bootstrap IDs for the admin account so EnsureDefaultVPC uses
// the same IDs that admin init wrote to [bootstrap] in spinifex.toml.
var opts []handlers_ec2_vpc.BootstrapIDs
if accountID == admin.DefaultAccountID() && d.clusterConfig != nil && d.clusterConfig.Bootstrap.VpcId != "" {
opts = append(opts, handlers_ec2_vpc.BootstrapIDs{
VpcId: d.clusterConfig.Bootstrap.VpcId,
SubnetId: d.clusterConfig.Bootstrap.SubnetId,
})
}
if _, err := d.vpcService.EnsureDefaultVPC(accountID, opts...); err != nil {
slog.Error("Failed to ensure default VPC", "accountID", accountID, "error", err)
}
}
// Ensure default VPC has an IGW and default security group
d.ensureDefaultVPCInfrastructure()
}
// Initialize network plumber for VPC tap device management
@@ -546,7 +643,7 @@ func (d *Daemon) Start() error {
// Initialize dynamic per-instance-type subscriptions for capacity-aware routing.
// Each instance type gets its own NATS topic (ec2.RunInstances.{type}) so requests
// are only routed to nodes with available capacity.
d.resourceMgr.initSubscriptions(d.natsConn, d.handleEC2RunInstances)
d.resourceMgr.initSubscriptions(d.natsConn, d.handleEC2RunInstances, d.node)
d.startHeartbeat()
d.startPendingWatchdog()
@@ -1131,7 +1228,6 @@ func (d *Daemon) saveClusterConfig() error {
// ClusterManager starts the HTTP cluster management server
func (d *Daemon) ClusterManager() error {
// Get daemon host from config
daemonHost := d.config.Daemon.Host
if daemonHost == "" {
@@ -1145,7 +1241,7 @@ func (d *Daemon) ClusterManager() error {
configHash, err := d.computeConfigHash()
if err != nil {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(500)
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte(`{"error":"failed to compute config hash"}`))
return
}
@@ -1202,7 +1298,7 @@ func (d *Daemon) ClusterManager() error {
var req types.NodeJoinRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(400)
w.WriteHeader(http.StatusBadRequest)
if err := json.NewEncoder(w).Encode(types.NodeJoinResponse{
Success: false,
Message: "invalid request body",
@@ -1217,7 +1313,7 @@ func (d *Daemon) ClusterManager() error {
// Validate request
if req.Node == "" || req.Region == "" || req.AZ == "" {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(400)
w.WriteHeader(http.StatusBadRequest)
if err := json.NewEncoder(w).Encode(types.NodeJoinResponse{
Success: false,
Message: "node, region, and az are required",
@@ -1230,7 +1326,7 @@ func (d *Daemon) ClusterManager() error {
// Check if node already exists
if _, exists := d.clusterConfig.Nodes[req.Node]; exists {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(409)
w.WriteHeader(http.StatusConflict)
if err := json.NewEncoder(w).Encode(types.NodeJoinResponse{
Success: false,
Message: fmt.Sprintf("node %s already exists in cluster", req.Node),
@@ -1265,7 +1361,7 @@ func (d *Daemon) ClusterManager() error {
if err := d.saveClusterConfig(); err != nil {
slog.Error("Failed to save cluster config", "error", err)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(500)
w.WriteHeader(http.StatusInternalServerError)
if err := json.NewEncoder(w).Encode(types.NodeJoinResponse{
Success: false,
Message: "failed to save cluster config",
@@ -1405,7 +1501,6 @@ func (d *Daemon) LoadState() error {
}
func (d *Daemon) SendQMPCommand(q *qmp.QMPClient, cmd qmp.QMPCommand, instanceId string) (*qmp.QMPResponse, error) {
// Confirm QMP client is initialized
if q == nil || q.Encoder == nil || q.Decoder == nil {
return nil, fmt.Errorf("QMP client is not initialized")
@@ -1419,7 +1514,7 @@ func (d *Daemon) SendQMPCommand(q *qmp.QMPClient, cmd qmp.QMPCommand, instanceId
if err := q.Conn.SetReadDeadline(time.Now().Add(30 * time.Second)); err != nil {
return nil, fmt.Errorf("set read deadline: %w", err)
}
defer q.Conn.SetReadDeadline(time.Time{}) // clear deadline after command
defer func() { _ = q.Conn.SetReadDeadline(time.Time{}) }() // clear deadline after command
if err := q.Encoder.Encode(cmd); err != nil {
return nil, fmt.Errorf("encode error: %w", err)
@@ -1446,7 +1541,10 @@ func (d *Daemon) SendQMPCommand(q *qmp.QMPClient, cmd qmp.QMPCommand, instanceId
return nil, fmt.Errorf("QMP error: %s: %s", errObj["class"], errObj["desc"])
}
if _, ok := msg["return"]; ok {
respBytes, _ := json.Marshal(msg)
respBytes, err := json.Marshal(msg)
if err != nil {
return nil, fmt.Errorf("marshal QMP response: %w", err)
}
var resp qmp.QMPResponse
if err := json.Unmarshal(respBytes, &resp); err != nil {
return nil, fmt.Errorf("unmarshal error: %w", err)
@@ -1457,15 +1555,12 @@ func (d *Daemon) SendQMPCommand(q *qmp.QMPClient, cmd qmp.QMPCommand, instanceId
}
func (d *Daemon) stopInstance(instances map[string]*vm.VM, deleteVolume bool) error {
// Signal to shutdown each VM
var wg sync.WaitGroup
// Run asynchronously within a worker group
for _, instance := range instances {
wg.Go(func() {
// Send shutdown command - if it fails, VM may already be dead, continue with cleanup
_, err := d.SendQMPCommand(instance.QMPClient, qmp.QMPCommand{Execute: "system_powerdown"}, instance.ID)
if err != nil {
@@ -1473,8 +1568,11 @@ func (d *Daemon) stopInstance(instances map[string]*vm.VM, deleteVolume bool) er
// Don't return - continue with cleanup
}
// Wait for PID file removal (or check if already gone)
err = utils.WaitForPidFileRemoval(instance.ID, 60*time.Second)
// Wait for PID file removal (or check if already gone).
// 20s is enough for a graceful ACPI shutdown — if the guest hasn't
// responded to system_powerdown by then, it won't (e.g. still booting,
// no ACPI handler). Force-kill at that point rather than wasting 60s.
err = utils.WaitForPidFileRemoval(instance.ID, 20*time.Second)
if err != nil {
slog.Warn("Timeout waiting for PID file removal", "id", instance.ID, "err", err)
@@ -1495,7 +1593,6 @@ func (d *Daemon) stopInstance(instances map[string]*vm.VM, deleteVolume bool) er
defer instance.EBSRequests.Mu.Unlock()
for _, ebsRequest := range instance.EBSRequests.Requests {
// Send the volume payload as JSON
ebsUnMountRequest, err := json.Marshal(ebsRequest)
@@ -1567,6 +1664,24 @@ func (d *Daemon) stopInstance(instances map[string]*vm.VM, deleteVolume bool) er
}
}
// Release public IP before deleting ENI
if deleteVolume && instance.PublicIP != "" && instance.PublicIPPool != "" && d.externalIPAM != nil {
// Publish vpc.delete-nat to remove dnat_and_snat rule
portName := "port-" + instance.ENIId
vpcId := ""
if instance.Instance != nil && instance.Instance.VpcId != nil {
vpcId = *instance.Instance.VpcId
}
d.publishNATEvent("vpc.delete-nat", vpcId, instance.PublicIP, "", portName, "")
// Release IP back to pool
if err := d.externalIPAM.ReleaseIP(instance.PublicIPPool, instance.PublicIP); err != nil {
slog.Warn("Failed to release public IP on termination", "ip", instance.PublicIP, "pool", instance.PublicIPPool, "err", err)
} else {
slog.Info("Released public IP on termination", "ip", instance.PublicIP, "instanceId", instance.ID)
}
}
// On termination, delete the auto-created ENI (releases IP back to IPAM,
// publishes vpc.delete-port for vpcd). On stop, ENI persists (AWS behavior).
if deleteVolume && instance.ENIId != "" && d.vpcService != nil {
@@ -1615,12 +1730,10 @@ func (d *Daemon) stopInstance(instances map[string]*vm.VM, deleteVolume bool) er
}
}
return nil
}
func (d *Daemon) setupShutdown() {
d.shutdownWg.Go(func() {
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM, syscall.SIGHUP)
@@ -1650,7 +1763,6 @@ func (d *Daemon) setupShutdown() {
if err := sub.Unsubscribe(); err != nil {
slog.Error("Error unsubscribing from NATS", "err", err)
}
}
// Write shutdown marker to cluster state KV
@@ -1682,7 +1794,6 @@ func (d *Daemon) setupShutdown() {
}
func (d *Daemon) CreateQMPClient(instance *vm.VM) (err error) {
// Create a new QMP client to communicate with the instance
instance.QMPClient, err = qmp.NewQMPClient(instance.Config.QMPSocket)
@@ -1734,11 +1845,9 @@ func (d *Daemon) CreateQMPClient(instance *vm.VM) (err error) {
}()
return nil
}
func (d *Daemon) LaunchInstance(instance *vm.VM) (err error) {
// Abort if instance is no longer in a launchable state (e.g., terminated
// by a concurrent request while waiting in the launch queue).
d.Instances.Mu.Lock()
@@ -1749,7 +1858,7 @@ func (d *Daemon) LaunchInstance(instance *vm.VM) (err error) {
}
// First, confirm if the instance is already running
pid, err := utils.ReadPidFile(instance.ID)
pid, _ := utils.ReadPidFile(instance.ID)
if pid > 0 {
process, err := os.FindProcess(pid)
@@ -1974,7 +2083,6 @@ func (d *Daemon) startPendingWatchdog() {
}
func (d *Daemon) StartInstance(instance *vm.VM) error {
pidFile, err := utils.GeneratePidFile(instance.ID)
if err != nil {
@@ -2025,7 +2133,6 @@ func (d *Daemon) StartInstance(instance *vm.VM) error {
instance.EBSRequests.Mu.Lock()
for _, v := range instance.EBSRequests.Requests {
drive := vm.Drive{}
// Use the NBDURI from mount response - contains socket path or TCP address
@@ -2293,12 +2400,10 @@ func (d *Daemon) ebsTopic(action string) string {
// MountVolumes mounts the volumes for an instance
func (d *Daemon) MountVolumes(instance *vm.VM) error {
instance.EBSRequests.Mu.Lock()
defer instance.EBSRequests.Mu.Unlock()
for k, v := range instance.EBSRequests.Requests {
// Send the volume payload as JSON
ebsMountRequest, err := json.Marshal(v)
@@ -2327,21 +2432,17 @@ func (d *Daemon) MountVolumes(instance *vm.VM) error {
}
if ebsMountResponse.Error == "" {
slog.Debug("Mounted volume successfully", "response", ebsMountResponse.URI)
// Append the NBD URI to the request
instance.EBSRequests.Requests[k].NBDURI = ebsMountResponse.URI
} else {
slog.Error("Failed to mount volume", "error", ebsMountResponse.Error)
return fmt.Errorf("failed to mount volume: %s", ebsMountResponse.Error)
}
}
return nil
}
// rollbackEBSMount sends an ebs.unmount request to undo a previously successful ebs.mount.
@@ -2481,17 +2582,21 @@ func (rm *ResourceManager) deallocate(instanceType *ec2.InstanceTypeInfo) {
// initSubscriptions sets up dynamic per-instance-type NATS subscriptions.
// Called once during daemon startup after NATS is connected.
func (rm *ResourceManager) initSubscriptions(nc *nats.Conn, handler nats.MsgHandler) {
func (rm *ResourceManager) initSubscriptions(nc *nats.Conn, handler nats.MsgHandler, nodeID string) {
rm.natsConn = nc
rm.handler = handler
rm.nodeID = nodeID
rm.instanceSubs = make(map[string]*nats.Subscription)
rm.updateInstanceSubscriptions()
}
// updateInstanceSubscriptions recalculates which instance types can fit on this
// node and subscribes/unsubscribes from the corresponding NATS topics. Each type
// gets its own topic (ec2.RunInstances.{type}) with the spinifex-workers queue group,
// so NATS only routes requests to nodes that can actually serve them.
// gets two topics:
// - ec2.RunInstances.{type} with spinifex-workers queue group (load-balanced, for single-instance launches)
// - ec2.RunInstances.{type}.{nodeId} without queue group (targeted, for multi-node distribution)
//
// Both use the same handler. NATS only routes requests to nodes with available capacity.
func (rm *ResourceManager) updateInstanceSubscriptions() {
if rm.natsConn == nil {
return
@@ -2501,24 +2606,46 @@ func (rm *ResourceManager) updateInstanceSubscriptions() {
defer rm.subsMu.Unlock()
for typeName, typeInfo := range rm.instanceTypes {
topic := fmt.Sprintf("ec2.RunInstances.%s", typeName)
queueTopic := fmt.Sprintf("ec2.RunInstances.%s", typeName)
canFit := rm.canAllocate(typeInfo, 1) >= 1
_, subscribed := rm.instanceSubs[topic]
// Queue group subscription (load-balanced across nodes)
_, subscribed := rm.instanceSubs[queueTopic]
if canFit && !subscribed {
sub, err := rm.natsConn.QueueSubscribe(topic, "spinifex-workers", rm.handler)
sub, err := rm.natsConn.QueueSubscribe(queueTopic, "spinifex-workers", rm.handler)
if err != nil {
slog.Error("Failed to subscribe to instance type topic", "topic", topic, "err", err)
slog.Error("Failed to subscribe to instance type topic", "topic", queueTopic, "err", err)
continue
}
rm.instanceSubs[topic] = sub
slog.Debug("Subscribed to instance type", "topic", topic)
rm.instanceSubs[queueTopic] = sub
slog.Debug("Subscribed to instance type", "topic", queueTopic)
} else if !canFit && subscribed {
if err := rm.instanceSubs[topic].Unsubscribe(); err != nil {
slog.Error("Failed to unsubscribe from instance type topic", "topic", topic, "err", err)
if err := rm.instanceSubs[queueTopic].Unsubscribe(); err != nil {
slog.Error("Failed to unsubscribe from instance type topic", "topic", queueTopic, "err", err)
}
delete(rm.instanceSubs, queueTopic)
slog.Info("Unsubscribed from instance type (capacity full)", "topic", queueTopic)
}
// Node-specific subscription (targeted routing for multi-node distribution)
if rm.nodeID != "" {
nodeTopic := fmt.Sprintf("ec2.RunInstances.%s.%s", typeName, rm.nodeID)
_, nodeSubscribed := rm.instanceSubs[nodeTopic]
if canFit && !nodeSubscribed {
sub, err := rm.natsConn.Subscribe(nodeTopic, rm.handler)
if err != nil {
slog.Error("Failed to subscribe to node-specific topic", "topic", nodeTopic, "err", err)
continue
}
rm.instanceSubs[nodeTopic] = sub
slog.Debug("Subscribed to node-specific instance type", "topic", nodeTopic)
} else if !canFit && nodeSubscribed {
if err := rm.instanceSubs[nodeTopic].Unsubscribe(); err != nil {
slog.Error("Failed to unsubscribe from node-specific topic", "topic", nodeTopic, "err", err)
}
delete(rm.instanceSubs, nodeTopic)
slog.Info("Unsubscribed from node-specific instance type (capacity full)", "topic", nodeTopic)
}
delete(rm.instanceSubs, topic)
slog.Info("Unsubscribed from instance type (capacity full)", "topic", topic)
}
}
}
+103
View File
@@ -1,10 +1,13 @@
package daemon
import (
"crypto/tls"
"encoding/json"
"fmt"
"log/slog"
"net"
"net/http"
"sync"
"time"
"github.com/mulgadc/spinifex/spinifex/awserrors"
@@ -175,9 +178,109 @@ func (d *Daemon) handleNodeStatus(msg *nats.Msg) {
InstanceTypes: caps,
}
// Query service roles concurrently to halve worst-case latency (500ms vs 1s).
var wg sync.WaitGroup
wg.Add(2)
go func() { defer wg.Done(); resp.NATSRole = d.queryNATSRole() }()
go func() { defer wg.Done(); resp.PredastoreRole = d.queryPredastoreRole() }()
wg.Wait()
respondWithJSON(msg, resp)
}
const (
roleLeader = "leader"
roleFollower = "follower"
natsMonitorPort = 8222
predastoreDBPort = 6660
)
// queryNATSRole queries the local NATS monitoring endpoint to determine this
// node's JetStream meta-leader status. Returns "leader", "follower", or "".
func (d *Daemon) queryNATSRole() string {
if !d.config.HasService("nats") {
return ""
}
url := fmt.Sprintf("http://%s:%d/varz", d.daemonIP(), natsMonitorPort)
return fetchNATSRole(url, roleHTTPClient)
}
// queryPredastoreRole queries the local Predastore DB status endpoint to
// determine this node's Raft role. Returns "leader", "follower", or "".
func (d *Daemon) queryPredastoreRole() string {
if !d.config.HasService("predastore") {
return ""
}
url := fmt.Sprintf("https://%s:%d/status", d.daemonIP(), predastoreDBPort)
return fetchPredastoreRole(url, roleTLSHTTPClient)
}
var roleHTTPClient = &http.Client{Timeout: 500 * time.Millisecond}
var roleTLSHTTPClient = &http.Client{
Timeout: 500 * time.Millisecond,
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // local self-signed cert
},
}
// fetchNATSRole queries a NATS /varz endpoint and returns "leader", "follower", or "".
func fetchNATSRole(url string, client *http.Client) string {
resp, err := client.Get(url) //nolint:noctx // internal monitoring call
if err != nil {
slog.Debug("Failed to query NATS monitoring", "err", err)
return ""
}
defer resp.Body.Close()
var varz struct {
ServerName string `json:"server_name"`
JetStream struct {
Meta struct {
Leader string `json:"leader"`
ClusterSize int `json:"cluster_size"`
} `json:"meta"`
} `json:"jetstream"`
}
if err := json.NewDecoder(resp.Body).Decode(&varz); err != nil {
slog.Debug("Failed to decode NATS varz", "err", err)
return ""
}
// Single node or no meta cluster — this node is the leader by default
if varz.JetStream.Meta.ClusterSize <= 1 {
return roleLeader
}
if varz.JetStream.Meta.Leader == varz.ServerName {
return roleLeader
}
return roleFollower
}
// fetchPredastoreRole queries a Predastore /status endpoint and returns "leader", "follower", or "".
func fetchPredastoreRole(url string, client *http.Client) string {
resp, err := client.Get(url) //nolint:noctx // internal monitoring call
if err != nil {
slog.Debug("Failed to query Predastore status", "err", err)
return ""
}
defer resp.Body.Close()
var status struct {
IsLeader bool `json:"is_leader"`
}
if err := json.NewDecoder(resp.Body).Decode(&status); err != nil {
slog.Debug("Failed to decode Predastore status", "err", err)
return ""
}
if status.IsLeader {
return roleLeader
}
return roleFollower
}
// handleNodeVMs responds with the list of VMs running on this node.
// Used by the CLI: spx get vms.
func (d *Daemon) handleNodeVMs(msg *nats.Msg) {
+23
View File
@@ -0,0 +1,23 @@
package daemon
import "github.com/nats-io/nats.go"
func (d *Daemon) handleEC2AllocateAddress(msg *nats.Msg) {
handleNATSRequest(msg, d.eipService.AllocateAddress)
}
func (d *Daemon) handleEC2ReleaseAddress(msg *nats.Msg) {
handleNATSRequest(msg, d.eipService.ReleaseAddress)
}
func (d *Daemon) handleEC2AssociateAddress(msg *nats.Msg) {
handleNATSRequest(msg, d.eipService.AssociateAddress)
}
func (d *Daemon) handleEC2DisassociateAddress(msg *nats.Msg) {
handleNATSRequest(msg, d.eipService.DisassociateAddress)
}
func (d *Daemon) handleEC2DescribeAddresses(msg *nats.Msg) {
handleNATSRequest(msg, d.eipService.DescribeAddresses)
}
+125 -1
View File
@@ -11,6 +11,7 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/mulgadc/spinifex/spinifex/qmp"
"github.com/mulgadc/spinifex/spinifex/types"
"github.com/mulgadc/spinifex/spinifex/utils"
@@ -156,6 +157,15 @@ func (d *Daemon) handleEC2RunInstances(msg *nats.Msg) {
continue
}
// Resolve default subnet when none specified (matches AWS behavior)
if (runInstancesInput.SubnetId == nil || *runInstancesInput.SubnetId == "") && d.vpcService != nil {
defaultSubnet, dsErr := d.vpcService.GetDefaultSubnet(accountID)
if dsErr == nil {
runInstancesInput.SubnetId = aws.String(defaultSubnet.SubnetId)
slog.Info("Resolved default subnet for instance", "instanceId", instance.ID, "subnetId", defaultSubnet.SubnetId)
}
}
// Auto-create ENI when SubnetId is provided (matches AWS behavior)
if runInstancesInput.SubnetId != nil && *runInstancesInput.SubnetId != "" && d.vpcService != nil {
eniOut, eniErr := d.vpcService.CreateNetworkInterface(&ec2.CreateNetworkInterfaceInput{
@@ -196,6 +206,41 @@ func (d *Daemon) handleEC2RunInstances(msg *nats.Msg) {
"privateIp", *eni.PrivateIpAddress,
"mac", instance.ENIMac,
)
// Auto-assign public IP if subnet has MapPublicIpOnLaunch and external IPAM is available
if d.externalIPAM != nil {
subnet, subErr := d.vpcService.GetSubnet(accountID, *runInstancesInput.SubnetId)
if subErr == nil && subnet.MapPublicIpOnLaunch {
region := ""
az := ""
if d.config != nil {
region = d.config.Region
az = d.config.AZ
}
publicIP, poolName, allocErr := d.externalIPAM.AllocateIP(region, az, "auto_assign", *eni.NetworkInterfaceId, instance.ID)
if allocErr != nil {
slog.Warn("Failed to allocate public IP for instance", "instanceId", instance.ID, "err", allocErr)
} else {
// Update ENI record with public IP
if updateErr := d.vpcService.UpdateENIPublicIP(accountID, *eni.NetworkInterfaceId, publicIP, poolName); updateErr != nil {
slog.Warn("Failed to update ENI with public IP", "eniId", *eni.NetworkInterfaceId, "err", updateErr)
}
// Publish vpc.add-nat for dnat_and_snat rule
portName := "port-" + *eni.NetworkInterfaceId
d.publishNATEvent("vpc.add-nat", *eni.VpcId, publicIP, *eni.PrivateIpAddress, portName, *eni.MacAddress)
// Set on ec2Instance response
ec2Instance.PublicIpAddress = aws.String(publicIP)
instance.PublicIP = publicIP
instance.PublicIPPool = poolName
slog.Info("Auto-assigned public IP",
"instanceId", instance.ID,
"publicIp", publicIP,
"privateIp", *eni.PrivateIpAddress,
"pool", poolName,
)
}
}
}
}
instances = append(instances, instance)
@@ -227,10 +272,14 @@ func (d *Daemon) handleEC2RunInstances(msg *nats.Msg) {
reservation.SetOwnerId(accountID)
reservation.Instances = allEC2Instances
// Store reservation reference and account ID in all VMs
// Store reservation reference, account ID, and placement group in all VMs
for _, instance := range instances {
instance.Reservation = &reservation
instance.AccountID = accountID
if runInstancesInput.Placement != nil && runInstancesInput.Placement.GroupName != nil && *runInstancesInput.Placement.GroupName != "" {
instance.PlacementGroupName = *runInstancesInput.Placement.GroupName
instance.PlacementGroupNode = d.node
}
}
// Respond to NATS immediately with reservation (instances are provisioning)
@@ -484,6 +533,18 @@ func (d *Daemon) handleStopOrTerminateInstance(msg *nats.Msg, command types.EC2I
}
slog.Info("Instance "+string(finalState), "id", inst.ID)
// Remove instance from placement group on terminate
if isTerminate && inst.PlacementGroupName != "" && d.placementGroupService != nil {
if _, pgErr := d.placementGroupService.RemoveInstance(&handlers_ec2_placementgroup.RemoveInstanceInput{
GroupName: inst.PlacementGroupName,
NodeName: inst.PlacementGroupNode,
InstanceID: inst.ID,
}, inst.AccountID); pgErr != nil {
slog.Error("Failed to remove instance from placement group",
"instanceId", inst.ID, "groupName", inst.PlacementGroupName, "err", pgErr)
}
}
if d.jsManager != nil {
if isTerminate {
// Write to terminated KV bucket (auto-expires after 1 hour via TTL).
@@ -628,6 +689,11 @@ func (d *Daemon) handleEC2DescribeInstances(msg *nats.Msg) {
instanceCopy := *instance.Instance
instanceCopy.State = &ec2.InstanceState{}
// Populate PublicIpAddress from VM if stored
if instance.PublicIP != "" && instanceCopy.PublicIpAddress == nil {
instanceCopy.PublicIpAddress = aws.String(instance.PublicIP)
}
// Map internal status to EC2 state codes using the centralized mapping
if info, ok := vm.EC2StateCodes[instance.Status]; ok {
instanceCopy.State.SetCode(info.Code)
@@ -639,6 +705,14 @@ func (d *Daemon) handleEC2DescribeInstances(msg *nats.Msg) {
instanceCopy.State.SetName("pending")
}
// Populate Placement if instance belongs to a placement group
if instance.PlacementGroupName != "" {
instanceCopy.Placement = &ec2.Placement{
GroupName: aws.String(instance.PlacementGroupName),
AvailabilityZone: aws.String(d.config.AZ),
}
}
// Add instance to its reservation
reservationMap[resID].Instances = append(reservationMap[resID].Instances, &instanceCopy)
}
@@ -899,6 +973,34 @@ func (d *Daemon) handleEC2TerminateStoppedInstance(msg *nats.Msg) {
}
instance.EBSRequests.Mu.Unlock()
// Release public IP before termination
if instance.PublicIP != "" && instance.PublicIPPool != "" && d.externalIPAM != nil {
portName := "port-" + instance.ENIId
vpcId := ""
if instance.Instance != nil && instance.Instance.VpcId != nil {
vpcId = *instance.Instance.VpcId
}
d.publishNATEvent("vpc.delete-nat", vpcId, instance.PublicIP, "", portName, "")
if err := d.externalIPAM.ReleaseIP(instance.PublicIPPool, instance.PublicIP); err != nil {
slog.Warn("handleEC2TerminateStoppedInstance: failed to release public IP", "ip", instance.PublicIP, "pool", instance.PublicIPPool, "err", err)
} else {
slog.Info("handleEC2TerminateStoppedInstance: released public IP", "ip", instance.PublicIP, "instanceId", req.InstanceID)
}
}
// Delete ENI if present
if instance.ENIId != "" && d.vpcService != nil {
_, eniErr := d.vpcService.DeleteNetworkInterface(&ec2.DeleteNetworkInterfaceInput{
NetworkInterfaceId: &instance.ENIId,
}, instance.AccountID)
if eniErr != nil {
slog.Error("handleEC2TerminateStoppedInstance: failed to delete ENI", "eni", instance.ENIId, "err", eniErr)
} else {
slog.Info("handleEC2TerminateStoppedInstance: deleted ENI", "eni", instance.ENIId, "instanceId", req.InstanceID)
}
}
// Write to terminated KV bucket FIRST so the instance is visible in DescribeInstances.
// If this fails, the instance remains in the stopped bucket (safe to retry).
instance.Status = vm.StateTerminated
@@ -1115,3 +1217,25 @@ func (d *Daemon) handleEC2ModifyInstanceAttribute(msg *nats.Msg) {
slog.Error("Failed to respond to NATS request", "err", err)
}
}
// publishNATEvent publishes a NAT lifecycle event (vpc.add-nat or vpc.delete-nat) to NATS.
func (d *Daemon) publishNATEvent(topic, vpcId, externalIP, logicalIP, portName, mac string) {
if d.natsConn == nil {
return
}
evt := struct {
VpcId string `json:"vpc_id"`
ExternalIP string `json:"external_ip"`
LogicalIP string `json:"logical_ip"`
PortName string `json:"port_name"`
MAC string `json:"mac"`
}{VpcId: vpcId, ExternalIP: externalIP, LogicalIP: logicalIP, PortName: portName, MAC: mac}
data, err := json.Marshal(evt)
if err != nil {
slog.Error("Failed to marshal NAT event", "topic", topic, "err", err)
return
}
if err := d.natsConn.Publish(topic, data); err != nil {
slog.Error("Failed to publish NAT event", "topic", topic, "err", err)
}
}
@@ -0,0 +1,39 @@
package daemon
import "github.com/nats-io/nats.go"
func (d *Daemon) handleEC2CreatePlacementGroup(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.CreatePlacementGroup)
}
func (d *Daemon) handleEC2DeletePlacementGroup(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.DeletePlacementGroup)
}
func (d *Daemon) handleEC2DescribePlacementGroups(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.DescribePlacementGroups)
}
func (d *Daemon) handleEC2ReserveSpreadNodes(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.ReserveSpreadNodes)
}
func (d *Daemon) handleEC2FinalizeSpreadInstances(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.FinalizeSpreadInstances)
}
func (d *Daemon) handleEC2ReleaseSpreadNodes(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.ReleaseSpreadNodes)
}
func (d *Daemon) handleEC2RemoveInstanceFromPlacementGroup(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.RemoveInstance)
}
func (d *Daemon) handleEC2ReserveClusterNode(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.ReserveClusterNode)
}
func (d *Daemon) handleEC2FinalizeClusterInstances(msg *nats.Msg) {
handleNATSRequest(msg, d.placementGroupService.FinalizeClusterInstances)
}
+31
View File
@@ -0,0 +1,31 @@
package daemon
import "github.com/nats-io/nats.go"
func (d *Daemon) handleEC2CreateSecurityGroup(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.CreateSecurityGroup)
}
func (d *Daemon) handleEC2DeleteSecurityGroup(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.DeleteSecurityGroup)
}
func (d *Daemon) handleEC2DescribeSecurityGroups(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.DescribeSecurityGroups)
}
func (d *Daemon) handleEC2AuthorizeSecurityGroupIngress(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.AuthorizeSecurityGroupIngress)
}
func (d *Daemon) handleEC2AuthorizeSecurityGroupEgress(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.AuthorizeSecurityGroupEgress)
}
func (d *Daemon) handleEC2RevokeSecurityGroupIngress(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.RevokeSecurityGroupIngress)
}
func (d *Daemon) handleEC2RevokeSecurityGroupEgress(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.RevokeSecurityGroupEgress)
}
+128 -1
View File
@@ -4,6 +4,9 @@ import (
"encoding/json"
"log/slog"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/admin"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/nats-io/nats.go"
)
@@ -31,6 +34,10 @@ func (d *Daemon) handleEC2DescribeSubnets(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.DescribeSubnets)
}
func (d *Daemon) handleEC2ModifySubnetAttribute(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.ModifySubnetAttribute)
}
func (d *Daemon) handleEC2CreateNetworkInterface(msg *nats.Msg) {
handleNATSRequest(msg, d.vpcService.CreateNetworkInterface)
}
@@ -56,8 +63,128 @@ func (d *Daemon) handleAccountCreated(msg *nats.Msg) {
slog.Error("Account creation event has empty account ID")
return
}
if err := d.vpcService.EnsureDefaultVPC(evt.AccountID); err != nil {
if _, err := d.vpcService.EnsureDefaultVPC(evt.AccountID); err != nil {
slog.Error("Failed to create default VPC for new account",
"accountID", evt.AccountID, "error", err)
}
d.ensureDefaultVPCInfrastructure()
}
// ensureDefaultVPCInfrastructure creates an IGW and default security group for
// each default VPC that doesn't already have them. Matches AWS behavior where
// the default VPC comes with an attached IGW and a default security group.
func (d *Daemon) ensureDefaultVPCInfrastructure() {
if d.igwService == nil || d.vpcService == nil {
return
}
for _, accountID := range []string{utils.GlobalAccountID, admin.DefaultAccountID()} {
// Find the default VPC for this account
descOut, err := d.vpcService.DescribeVpcs(&ec2.DescribeVpcsInput{}, accountID)
if err != nil {
continue
}
var defaultVpcId string
for _, vpc := range descOut.Vpcs {
if vpc.IsDefault != nil && *vpc.IsDefault {
defaultVpcId = *vpc.VpcId
break
}
}
if defaultVpcId == "" {
continue
}
// Check if IGW already attached
igwOut, err := d.igwService.DescribeInternetGateways(&ec2.DescribeInternetGatewaysInput{}, accountID)
if err != nil {
continue
}
hasIGW := false
for _, igw := range igwOut.InternetGateways {
for _, att := range igw.Attachments {
if att.VpcId != nil && *att.VpcId == defaultVpcId {
hasIGW = true
break
}
}
}
if !hasIGW {
// Create and attach an IGW — use bootstrap ID if available
var createOut *ec2.CreateInternetGatewayOutput
var err error
if accountID == admin.DefaultAccountID() && d.clusterConfig != nil && d.clusterConfig.Bootstrap.IgwId != "" {
createOut, err = d.igwService.CreateInternetGatewayWithID(&ec2.CreateInternetGatewayInput{}, accountID, d.clusterConfig.Bootstrap.IgwId)
} else {
createOut, err = d.igwService.CreateInternetGateway(&ec2.CreateInternetGatewayInput{}, accountID)
}
if err != nil {
slog.Error("Failed to create default IGW", "accountID", accountID, "err", err)
continue
}
igwId := *createOut.InternetGateway.InternetGatewayId
_, err = d.igwService.AttachInternetGateway(&ec2.AttachInternetGatewayInput{
InternetGatewayId: &igwId,
VpcId: &defaultVpcId,
}, accountID)
if err != nil {
slog.Error("Failed to attach default IGW", "igwId", igwId, "vpcId", defaultVpcId, "err", err)
} else {
slog.Info("Attached default IGW to default VPC", "igwId", igwId, "vpcId", defaultVpcId, "accountID", accountID)
}
}
// Check if default security group exists for this VPC
sgOut, err := d.vpcService.DescribeSecurityGroups(&ec2.DescribeSecurityGroupsInput{}, accountID)
if err != nil {
continue
}
hasDefaultSG := false
for _, sg := range sgOut.SecurityGroups {
if sg.VpcId != nil && *sg.VpcId == defaultVpcId && sg.GroupName != nil && *sg.GroupName == "default" {
hasDefaultSG = true
break
}
}
if !hasDefaultSG {
desc := "default VPC security group"
groupName := "default"
createSGOut, err := d.vpcService.CreateSecurityGroup(&ec2.CreateSecurityGroupInput{
GroupName: &groupName,
Description: &desc,
VpcId: &defaultVpcId,
}, accountID)
if err != nil {
slog.Error("Failed to create default security group", "vpcId", defaultVpcId, "err", err)
continue
}
sgId := *createSGOut.GroupId
// Default SG rules (AWS behavior):
// - Allow all inbound from same SG (self-referencing)
// - Allow all outbound to 0.0.0.0/0
allProto := "-1"
allCidr := "0.0.0.0/0"
_, _ = d.vpcService.AuthorizeSecurityGroupIngress(&ec2.AuthorizeSecurityGroupIngressInput{
GroupId: &sgId,
IpPermissions: []*ec2.IpPermission{
{
IpProtocol: &allProto,
UserIdGroupPairs: []*ec2.UserIdGroupPair{{GroupId: &sgId}},
},
},
}, accountID)
_, _ = d.vpcService.AuthorizeSecurityGroupEgress(&ec2.AuthorizeSecurityGroupEgressInput{
GroupId: &sgId,
IpPermissions: []*ec2.IpPermission{
{
IpProtocol: &allProto,
IpRanges: []*ec2.IpRange{{CidrIp: &allCidr}},
},
},
}, accountID)
slog.Info("Created default security group for default VPC",
"groupId", sgId, "vpcId", defaultVpcId, "accountID", accountID)
}
}
}
+13 -21
View File
@@ -31,12 +31,7 @@ import (
// createTestDaemon creates a test daemon instance with minimal configuration
func createTestDaemon(t *testing.T, natsURL string) *Daemon {
// Create a temporary directory for test data
tmpDir, err := os.MkdirTemp("", "spx-daemon-test-*")
require.NoError(t, err, "Failed to create temp directory")
t.Cleanup(func() {
os.RemoveAll(tmpDir)
})
tmpDir := t.TempDir()
// New cluster config
clusterCfg := &config.ClusterConfig{
@@ -345,9 +340,7 @@ func TestHandleEC2RunInstances_ResourceManagement(t *testing.T) {
// TestDaemon_Initialization tests daemon initialization
func TestDaemon_Initialization(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "spx-daemon-init-test-*")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)
tmpDir := t.TempDir()
// New cluster config
clusterCfg := &config.ClusterConfig{
@@ -859,9 +852,7 @@ func TestDaemon_BootAllocation(t *testing.T) {
natsURL := sharedJSNATSURL
// Create daemon temp directory
tmpDir, err := os.MkdirTemp("", "spx-daemon-boot-test-*")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)
tmpDir := t.TempDir()
// Create test VMs with one running and one stopped instance
vms := map[string]*vm.VM{
@@ -1403,7 +1394,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
// Count how many types actually fit on this machine
fittableTypes := 0
@@ -1413,8 +1404,9 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
}
}
assert.Equal(t, fittableTypes, len(rm.instanceSubs),
"should subscribe to all instance types that fit")
// Each fittable type gets 2 subscriptions: queue group + node-specific
assert.Equal(t, fittableTypes*2, len(rm.instanceSubs),
"should subscribe to all instance types that fit (queue + node-specific)")
assert.Greater(t, len(rm.instanceSubs), 0,
"should subscribe to at least some instance types")
@@ -1433,7 +1425,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
initialCount := len(rm.instanceSubs)
require.Greater(t, initialCount, 0)
@@ -1458,7 +1450,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
expectedCount := len(rm.instanceSubs)
@@ -1489,7 +1481,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
// Leave only 2 vCPUs and 1 GB free — enough for nano/micro but not larger types
rm.mu.Lock()
@@ -1501,7 +1493,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
// Count subscribed types — should be less than total but more than zero
assert.Greater(t, len(rm.instanceSubs), 0,
"should still be subscribed to small instance types")
assert.Less(t, len(rm.instanceSubs), len(rm.instanceTypes),
assert.Less(t, len(rm.instanceSubs), len(rm.instanceTypes)*2,
"should not be subscribed to large instance types")
// Verify nano (0.5 GB) and micro (1 GB) are subscribed
@@ -1518,7 +1510,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
initialCount := len(rm.instanceSubs)
require.Greater(t, initialCount, 0)
@@ -1562,7 +1554,7 @@ func TestInstanceTypeSubscriptions(t *testing.T) {
defer nc.Close()
handler := func(msg *nats.Msg) {}
rm.initSubscriptions(nc, handler)
rm.initSubscriptions(nc, handler, "test-node")
// Fill the node completely
rm.mu.Lock()
+29 -3
View File
@@ -9,6 +9,7 @@ import (
"sync"
"time"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/mulgadc/spinifex/spinifex/vm"
"github.com/nats-io/nats.go"
)
@@ -26,6 +27,11 @@ const (
TerminatedInstanceBucket = "spinifex-terminated-instances"
// TerminatedInstancePrefix is the key prefix for terminated instances
TerminatedInstancePrefix = "terminated."
// Schema versions for daemon KV buckets
InstanceStateBucketVersion = 1
ClusterStateBucketVersion = 1
TerminatedInstanceBucketVersion = 1
)
// JetStreamManager manages JetStream KV store operations for instance state
@@ -77,6 +83,9 @@ func (m *JetStreamManager) InitKVBucket() error {
}
m.kv = kv
if err := utils.WriteVersion(kv, InstanceStateBucketVersion); err != nil {
return fmt.Errorf("write version to %s: %w", InstanceStateBucket, err)
}
return nil
}
@@ -104,6 +113,9 @@ func (m *JetStreamManager) InitClusterStateBucket() error {
}
m.clusterKV = kv
if err := utils.WriteVersion(kv, ClusterStateBucketVersion); err != nil {
return fmt.Errorf("write version to %s: %w", ClusterStateBucket, err)
}
return nil
}
@@ -132,6 +144,9 @@ func (m *JetStreamManager) InitTerminatedInstanceBucket() error {
}
m.terminatedKV = kv
if err := utils.WriteVersion(kv, TerminatedInstanceBucketVersion); err != nil {
return fmt.Errorf("write version to %s: %w", TerminatedInstanceBucket, err)
}
return nil
}
@@ -157,7 +172,8 @@ func isStreamUnavailable(err error) bool {
// recoverBucket attempts to reconnect to or re-create a KV bucket after the
// underlying JetStream stream was lost during cluster formation.
// Returns the recovered KV handle directly so callers avoid a racy re-read.
func (m *JetStreamManager) recoverBucket(cfg *nats.KeyValueConfig, field *nats.KeyValue) (nats.KeyValue, error) {
// When a bucket is recreated, the schema version is re-stamped.
func (m *JetStreamManager) recoverBucket(cfg *nats.KeyValueConfig, field *nats.KeyValue, version int) (nats.KeyValue, error) {
m.kvMu.Lock()
defer m.kvMu.Unlock()
@@ -183,6 +199,10 @@ func (m *JetStreamManager) recoverBucket(cfg *nats.KeyValueConfig, field *nats.K
return nil, err
}
if err := utils.WriteVersion(kv, version); err != nil {
slog.Error("Failed to write version to recreated bucket", "bucket", cfg.Bucket, "err", err)
}
*field = kv
slog.Info("KV bucket recreated successfully", "bucket", cfg.Bucket)
return kv, nil
@@ -192,7 +212,7 @@ func (m *JetStreamManager) recoverKVBucket() (nats.KeyValue, error) {
return m.recoverBucket(&nats.KeyValueConfig{
Bucket: InstanceStateBucket,
Description: "Spinifex instance state storage",
}, &m.kv)
}, &m.kv, InstanceStateBucketVersion)
}
func (m *JetStreamManager) recoverTerminatedKVBucket() (nats.KeyValue, error) {
@@ -200,7 +220,7 @@ func (m *JetStreamManager) recoverTerminatedKVBucket() (nats.KeyValue, error) {
Bucket: TerminatedInstanceBucket,
Description: "Terminated instances (auto-expire after 1 hour)",
TTL: 1 * time.Hour,
}, &m.terminatedKV)
}, &m.terminatedKV, TerminatedInstanceBucketVersion)
}
// Heartbeat represents a daemon's periodic health status published to cluster KV.
@@ -662,6 +682,9 @@ func (m *JetStreamManager) ListStoppedInstances() ([]*vm.VM, error) {
var instances []*vm.VM
for _, key := range keys {
if key == utils.VersionKey {
continue
}
if !strings.HasPrefix(key, StoppedInstancePrefix) {
continue
}
@@ -751,6 +774,9 @@ func (m *JetStreamManager) ListTerminatedInstances() ([]*vm.VM, error) {
var instances []*vm.VM
for _, key := range keys {
if key == utils.VersionKey {
continue
}
if !strings.HasPrefix(key, TerminatedInstancePrefix) {
continue
}
+56
View File
@@ -5,6 +5,7 @@ import (
"errors"
"testing"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/mulgadc/spinifex/spinifex/vm"
"github.com/nats-io/nats.go"
"github.com/stretchr/testify/assert"
@@ -1208,3 +1209,58 @@ func TestJetStreamManager_ListTerminatedInstances_RecoverAfterStreamLost(t *test
require.NoError(t, err, "ListTerminatedInstances should recover after stream loss")
assert.Empty(t, instances)
}
func TestJetStreamManager_InitBuckets_WritesVersion(t *testing.T) {
nc, err := nats.Connect(sharedJSNATSURL)
require.NoError(t, err)
defer nc.Close()
jsm, err := NewJetStreamManager(nc, 1)
require.NoError(t, err)
require.NoError(t, jsm.InitKVBucket())
require.NoError(t, jsm.InitClusterStateBucket())
require.NoError(t, jsm.InitTerminatedInstanceBucket())
v, err := utils.ReadVersion(jsm.kv)
require.NoError(t, err)
assert.Equal(t, InstanceStateBucketVersion, v)
v, err = utils.ReadVersion(jsm.clusterKV)
require.NoError(t, err)
assert.Equal(t, ClusterStateBucketVersion, v)
v, err = utils.ReadVersion(jsm.terminatedKV)
require.NoError(t, err)
assert.Equal(t, TerminatedInstanceBucketVersion, v)
}
func TestJetStreamManager_ListStoppedInstances_SkipsVersionKey(t *testing.T) {
nc, err := nats.Connect(sharedJSNATSURL)
require.NoError(t, err)
defer nc.Close()
jsm, err := NewJetStreamManager(nc, 1)
require.NoError(t, err)
require.NoError(t, jsm.InitKVBucket())
// _version key is written by InitKVBucket; listing should not include it
instances, err := jsm.ListStoppedInstances()
require.NoError(t, err)
assert.Empty(t, instances)
}
func TestJetStreamManager_ListTerminatedInstances_SkipsVersionKey(t *testing.T) {
nc, err := nats.Connect(sharedJSNATSURL)
require.NoError(t, err)
defer nc.Close()
jsm, err := NewJetStreamManager(nc, 1)
require.NoError(t, err)
require.NoError(t, jsm.InitTerminatedInstanceBucket())
// _version key is written by InitTerminatedInstanceBucket; listing should not include it
instances, err := jsm.ListTerminatedInstances()
require.NoError(t, err)
assert.Empty(t, instances)
}
+1 -1
View File
@@ -403,7 +403,7 @@ func TestGenerateDevMAC_Format(t *testing.T) {
t.Errorf("generateDevMAC(%q) = %q, expected ':' at pos %d", id, mac, i)
}
} else {
if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f')) {
if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
t.Errorf("generateDevMAC(%q) = %q, invalid hex char '%c' at pos %d", id, mac, c, i)
}
}
+63
View File
@@ -0,0 +1,63 @@
package daemon
import (
"testing"
"github.com/mulgadc/spinifex/spinifex/utils"
)
func TestIsInstanceVisible(t *testing.T) {
tests := []struct {
name string
callerAccountID string
ownerAccountID string
expected bool
}{
{
name: "same account",
callerAccountID: "123456789012",
ownerAccountID: "123456789012",
expected: true,
},
{
name: "different account",
callerAccountID: "123456789012",
ownerAccountID: "999999999999",
expected: false,
},
{
name: "pre-Phase4 instance visible to root",
callerAccountID: utils.GlobalAccountID,
ownerAccountID: "",
expected: true,
},
{
name: "pre-Phase4 instance hidden from non-root",
callerAccountID: "123456789012",
ownerAccountID: "",
expected: false,
},
{
name: "root accessing owned instance",
callerAccountID: utils.GlobalAccountID,
ownerAccountID: utils.GlobalAccountID,
expected: true,
},
{
name: "root accessing other account instance",
callerAccountID: utils.GlobalAccountID,
ownerAccountID: "123456789012",
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isInstanceVisible(tt.callerAccountID, tt.ownerAccountID)
if got != tt.expected {
t.Errorf("isInstanceVisible(%q, %q) = %v, want %v",
tt.callerAccountID, tt.ownerAccountID, got, tt.expected)
}
})
}
}
-1
View File
@@ -16,7 +16,6 @@ var errInsufficientCapacity = errors.New("insufficient capacity to satisfy MinCo
// or side effects.
func canAllocateCount(availVCPU, allocVCPU int, availMem, allocMem float64,
vCPUs int64, memMiB int64, maxCount int) int {
remainingVCPU := availVCPU - allocVCPU
remainingMem := availMem - allocMem
memoryGB := float64(memMiB) / 1024.0
+5 -5
View File
@@ -201,11 +201,11 @@ func TestResourceStatsForType(t *testing.T) {
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cap := resourceStatsForType(tc.remainVCPU, tc.remainMem, tc.it)
assert.Equal(t, tc.wantName, cap.Name)
assert.Equal(t, tc.wantVCPU, cap.VCPU)
assert.InDelta(t, tc.wantMemGB, cap.MemoryGB, 0.001)
assert.Equal(t, tc.wantAvail, cap.Available)
typeCap := resourceStatsForType(tc.remainVCPU, tc.remainMem, tc.it)
assert.Equal(t, tc.wantName, typeCap.Name)
assert.Equal(t, tc.wantVCPU, typeCap.VCPU)
assert.InDelta(t, tc.wantMemGB, typeCap.MemoryGB, 0.001)
assert.Equal(t, tc.wantAvail, typeCap.Available)
})
}
}
+113
View File
@@ -0,0 +1,113 @@
package daemon
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/stretchr/testify/assert"
)
func TestFetchNATSRole_SingleNode(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{
"server_name": "spinifex-nats-node1",
"jetstream": {
"meta": {
"leader": "",
"cluster_size": 0
}
}
}`))
}))
defer srv.Close()
role := fetchNATSRole(srv.URL, srv.Client())
assert.Equal(t, "leader", role)
}
func TestFetchNATSRole_ClusterLeader(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{
"server_name": "spinifex-nats-node1",
"jetstream": {
"meta": {
"leader": "spinifex-nats-node1",
"cluster_size": 3
}
}
}`))
}))
defer srv.Close()
role := fetchNATSRole(srv.URL, srv.Client())
assert.Equal(t, "leader", role)
}
func TestFetchNATSRole_ClusterFollower(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{
"server_name": "spinifex-nats-node2",
"jetstream": {
"meta": {
"leader": "spinifex-nats-node1",
"cluster_size": 3
}
}
}`))
}))
defer srv.Close()
role := fetchNATSRole(srv.URL, srv.Client())
assert.Equal(t, "follower", role)
}
func TestFetchNATSRole_Unreachable(t *testing.T) {
role := fetchNATSRole("http://127.0.0.1:1/varz", &http.Client{})
assert.Equal(t, "", role)
}
func TestFetchNATSRole_BadJSON(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`not json`))
}))
defer srv.Close()
role := fetchNATSRole(srv.URL, srv.Client())
assert.Equal(t, "", role)
}
func TestFetchPredastoreRole_Leader(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{"node_id":"1","state":"Leader","is_leader":true}`))
}))
defer srv.Close()
role := fetchPredastoreRole(srv.URL, srv.Client())
assert.Equal(t, "leader", role)
}
func TestFetchPredastoreRole_Follower(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{"node_id":"2","state":"Follower","is_leader":false}`))
}))
defer srv.Close()
role := fetchPredastoreRole(srv.URL, srv.Client())
assert.Equal(t, "follower", role)
}
func TestFetchPredastoreRole_Unreachable(t *testing.T) {
role := fetchPredastoreRole("https://127.0.0.1:1/status", &http.Client{})
assert.Equal(t, "", role)
}
func TestFetchPredastoreRole_BadJSON(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`not json`))
}))
defer srv.Close()
role := fetchPredastoreRole(srv.URL, srv.Client())
assert.Equal(t, "", role)
}
+4 -9
View File
@@ -3,7 +3,6 @@ package daemon
import (
"fmt"
"net"
"os"
"path/filepath"
"sync"
"testing"
@@ -24,9 +23,7 @@ import (
func createDaemonWithJetStream(t *testing.T) *Daemon {
t.Helper()
jsTmpDir, err := os.MkdirTemp("", "nats-js-state-test-*")
require.NoError(t, err)
t.Cleanup(func() { os.RemoveAll(jsTmpDir) })
jsTmpDir := t.TempDir()
opts := &server.Options{
Host: "127.0.0.1",
@@ -44,9 +41,7 @@ func createDaemonWithJetStream(t *testing.T) *Daemon {
natsURL := ns.ClientURL()
tmpDir, err := os.MkdirTemp("", "spx-state-test-*")
require.NoError(t, err)
t.Cleanup(func() { os.RemoveAll(tmpDir) })
tmpDir := t.TempDir()
clusterCfg := &config.ClusterConfig{
Node: "node-1",
@@ -260,7 +255,7 @@ func TestEveryState_CanReachTerminated(t *testing.T) {
continue
}
visited := map[vm.InstanceState]bool{start: true}
visited := map[vm.InstanceState]bool{start: true} //nolint:exhaustive // BFS seed — populated dynamically
queue := []vm.InstanceState{start}
found := false
@@ -406,7 +401,7 @@ func TestTransitionState_ConcurrentTransitions(t *testing.T) {
// reachableStates returns all states reachable from start via BFS.
func reachableStates(start vm.InstanceState) map[vm.InstanceState]bool {
visited := map[vm.InstanceState]bool{start: true}
visited := map[vm.InstanceState]bool{start: true} //nolint:exhaustive // BFS seed — populated dynamically
queue := []vm.InstanceState{start}
for len(queue) > 0 {
current := queue[0]
+50 -24
View File
@@ -38,14 +38,37 @@ type JoinResponse struct {
// StatusResponse is returned by the formation server status endpoint.
type StatusResponse struct {
Complete bool `json:"complete"`
Joined int `json:"joined"`
Expected int `json:"expected"`
Nodes map[string]NodeInfo `json:"nodes,omitempty"`
Credentials *SharedCredentials `json:"credentials,omitempty"`
CACert string `json:"ca_cert,omitempty"`
CAKey string `json:"ca_key,omitempty"`
MasterKey string `json:"master_key,omitempty"`
Complete bool `json:"complete"`
Joined int `json:"joined"`
Expected int `json:"expected"`
Nodes map[string]NodeInfo `json:"nodes,omitempty"`
Credentials *SharedCredentials `json:"credentials,omitempty"`
CACert string `json:"ca_cert,omitempty"`
CAKey string `json:"ca_key,omitempty"`
MasterKey string `json:"master_key,omitempty"`
NetworkConfig *NetworkConfig `json:"network_config,omitempty"`
}
// NetworkConfig holds the cluster-wide external networking configuration
// propagated from the init node to joining nodes during formation.
type NetworkConfig struct {
ExternalMode string `json:"external_mode"`
ExternalDHCP bool `json:"external_dhcp,omitempty"`
PoolName string `json:"pool_name"`
PoolSource string `json:"pool_source,omitempty"`
PoolStart string `json:"pool_start,omitempty"`
PoolEnd string `json:"pool_end,omitempty"`
PoolGateway string `json:"pool_gateway"`
PoolGatewayIP string `json:"pool_gateway_ip,omitempty"`
PoolPrefixLen int `json:"pool_prefix_len"`
PoolDNSServers []string `json:"pool_dns_servers,omitempty"`
BootstrapAccountId string `json:"bootstrap_account_id,omitempty"`
BootstrapVpcId string `json:"bootstrap_vpc_id,omitempty"`
BootstrapSubnetId string `json:"bootstrap_subnet_id,omitempty"`
BootstrapIgwId string `json:"bootstrap_igw_id,omitempty"`
BootstrapCidr string `json:"bootstrap_cidr,omitempty"`
BootstrapSubnetCidr string `json:"bootstrap_subnet_cidr,omitempty"`
}
// SharedCredentials contains the cluster-wide credentials distributed during formation.
@@ -67,26 +90,28 @@ type SharedCredentials struct {
// nodes have joined, the done channel is closed and full cluster data (credentials,
// CA, node list) becomes available via GET /formation/status.
type FormationServer struct {
mu sync.RWMutex
expected int
nodes map[string]NodeInfo
credentials *SharedCredentials
caCert string
caKey string
masterKey string
done chan struct{}
server *http.Server
mu sync.RWMutex
expected int
nodes map[string]NodeInfo
credentials *SharedCredentials
caCert string
caKey string
masterKey string
networkConfig *NetworkConfig
done chan struct{}
server *http.Server
}
// NewFormationServer creates a new formation server expecting the given number of nodes.
func NewFormationServer(expected int, creds *SharedCredentials, caCert, caKey string) *FormationServer {
func NewFormationServer(expected int, creds *SharedCredentials, caCert, caKey string, networkConfig *NetworkConfig) *FormationServer {
return &FormationServer{
expected: expected,
nodes: make(map[string]NodeInfo),
credentials: creds,
caCert: caCert,
caKey: caKey,
done: make(chan struct{}),
expected: expected,
nodes: make(map[string]NodeInfo),
credentials: creds,
caCert: caCert,
caKey: caKey,
networkConfig: networkConfig,
done: make(chan struct{}),
}
}
@@ -252,6 +277,7 @@ func (fs *FormationServer) handleStatus(w http.ResponseWriter, r *http.Request)
resp.CACert = fs.caCert
resp.CAKey = fs.caKey
resp.MasterKey = fs.masterKey
resp.NetworkConfig = fs.networkConfig
}
writeJSON(w, http.StatusOK, resp)
+14 -14
View File
@@ -35,7 +35,7 @@ func testNode(name, ip string) NodeInfo {
func TestNewFormationServer(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "ca-cert-pem", "ca-key-pem")
fs := NewFormationServer(3, testCreds(), "ca-cert-pem", "ca-key-pem", nil)
assert.Equal(t, 3, fs.expected)
assert.Empty(t, fs.nodes)
@@ -44,7 +44,7 @@ func TestNewFormationServer(t *testing.T) {
func TestRegisterNode(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
err := fs.RegisterNode(testNode("node1", "10.0.0.1"))
require.NoError(t, err)
@@ -56,7 +56,7 @@ func TestRegisterNode(t *testing.T) {
func TestRegisterDuplicateName(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
require.NoError(t, fs.RegisterNode(testNode("node1", "10.0.0.1")))
err := fs.RegisterNode(testNode("node1", "10.0.0.2"))
@@ -66,7 +66,7 @@ func TestRegisterDuplicateName(t *testing.T) {
func TestRegisterDuplicateIP(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
require.NoError(t, fs.RegisterNode(testNode("node1", "10.0.0.1")))
err := fs.RegisterNode(testNode("node2", "10.0.0.1"))
@@ -76,7 +76,7 @@ func TestRegisterDuplicateIP(t *testing.T) {
func TestIsComplete(t *testing.T) {
t.Parallel()
fs := NewFormationServer(2, testCreds(), "", "")
fs := NewFormationServer(2, testCreds(), "", "", nil)
assert.False(t, fs.IsComplete())
require.NoError(t, fs.RegisterNode(testNode("node1", "10.0.0.1")))
@@ -87,7 +87,7 @@ func TestIsComplete(t *testing.T) {
func TestWaitForCompletion(t *testing.T) {
t.Parallel()
fs := NewFormationServer(2, testCreds(), "", "")
fs := NewFormationServer(2, testCreds(), "", "", nil)
go func() {
time.Sleep(50 * time.Millisecond)
@@ -102,7 +102,7 @@ func TestWaitForCompletion(t *testing.T) {
func TestWaitForCompletionTimeout(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
require.NoError(t, fs.RegisterNode(testNode("node1", "10.0.0.1")))
@@ -122,7 +122,7 @@ func testServer(t *testing.T, fs *FormationServer) *httptest.Server {
func TestJoinEndpoint(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
ts := testServer(t, fs)
defer ts.Close()
@@ -142,7 +142,7 @@ func TestJoinEndpoint(t *testing.T) {
func TestJoinEndpointDuplicate(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "", "")
fs := NewFormationServer(3, testCreds(), "", "", nil)
ts := testServer(t, fs)
defer ts.Close()
@@ -161,7 +161,7 @@ func TestJoinEndpointDuplicate(t *testing.T) {
func TestStatusEndpointIncomplete(t *testing.T) {
t.Parallel()
fs := NewFormationServer(3, testCreds(), "ca-cert", "ca-key")
fs := NewFormationServer(3, testCreds(), "ca-cert", "ca-key", nil)
ts := testServer(t, fs)
defer ts.Close()
@@ -184,7 +184,7 @@ func TestStatusEndpointIncomplete(t *testing.T) {
func TestStatusEndpointComplete(t *testing.T) {
t.Parallel()
creds := testCreds()
fs := NewFormationServer(2, creds, "ca-cert-data", "ca-key-data")
fs := NewFormationServer(2, creds, "ca-cert-data", "ca-key-data", nil)
ts := testServer(t, fs)
defer ts.Close()
@@ -209,7 +209,7 @@ func TestStatusEndpointComplete(t *testing.T) {
func TestStatusEndpointMasterKey(t *testing.T) {
t.Parallel()
fs := NewFormationServer(2, testCreds(), "ca-cert", "ca-key")
fs := NewFormationServer(2, testCreds(), "ca-cert", "ca-key", nil)
fs.SetMasterKey("dGVzdC1tYXN0ZXIta2V5LWJhc2U2NC1lbmNvZGVk")
ts := testServer(t, fs)
defer ts.Close()
@@ -237,7 +237,7 @@ func TestStatusEndpointMasterKey(t *testing.T) {
func TestHealthEndpoint(t *testing.T) {
t.Parallel()
fs := NewFormationServer(1, testCreds(), "", "")
fs := NewFormationServer(1, testCreds(), "", "", nil)
ts := testServer(t, fs)
defer ts.Close()
@@ -292,7 +292,7 @@ func TestBuildPredastoreNodes(t *testing.T) {
func TestFullFormationFlow(t *testing.T) {
t.Parallel()
creds := testCreds()
fs := NewFormationServer(3, creds, "-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----", "-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----")
fs := NewFormationServer(3, creds, "-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----", "-----BEGIN PRIVATE KEY-----\ntest\n-----END PRIVATE KEY-----", nil)
ts := testServer(t, fs)
defer ts.Close()
-1
View File
@@ -3,6 +3,5 @@ package gateway
import "net/http"
func (gw *GatewayConfig) Account_Request(w http.ResponseWriter, r *http.Request) error {
return nil
}
+2 -2
View File
@@ -28,7 +28,7 @@ func (gw *GatewayConfig) SigV4AuthMiddleware() func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Skip OPTIONS requests (CORS preflight)
if r.Method == "OPTIONS" {
if r.Method == http.MethodOptions {
next.ServeHTTP(w, r)
return
}
@@ -301,7 +301,7 @@ func canonicalHeaderName(header string) string {
// writeSigV4Error writes an EC2-compatible XML error response for authentication failures.
func (gw *GatewayConfig) writeSigV4Error(w http.ResponseWriter, r *http.Request, errorCode string) {
requestID := r.Header.Get("x-amz-request-id")
requestID := r.Header.Get("X-Amz-Request-Id")
if requestID == "" {
requestID = uuid.NewString()
}
+74 -72
View File
@@ -211,12 +211,12 @@ func setupTestApp(accessKey, secretKey string) http.Handler {
func TestSigV4Auth_NoAuthorizationHeader(t *testing.T) {
handler := setupTestApp(testAccessKey, testSecretKey)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
@@ -242,14 +242,14 @@ func TestSigV4Auth_MalformedHeader(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", tc.authHeader)
req.Header.Set("X-Amz-Date", time.Now().UTC().Format(auth.TimeFormat))
resp := doRequest(handler, req)
if resp.StatusCode != 400 {
if resp.StatusCode != http.StatusBadRequest {
t.Errorf("Expected status 400, got %d", resp.StatusCode)
}
@@ -269,14 +269,14 @@ func TestSigV4Auth_InvalidAccessKey(t *testing.T) {
"INVALID_ACCESS_KEY", testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
@@ -295,14 +295,14 @@ func TestSigV4Auth_InvalidSignature(t *testing.T) {
testAccessKey, "WRONG_SECRET_KEY", testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
@@ -320,14 +320,14 @@ func TestSigV4Auth_ValidSignature(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -336,13 +336,13 @@ func TestSigV4Auth_ValidSignature(t *testing.T) {
func TestSigV4Auth_OptionsSkipsAuth(t *testing.T) {
handler := setupTestApp(testAccessKey, testSecretKey)
req := httptest.NewRequest("OPTIONS", "/", nil)
req := httptest.NewRequest(http.MethodOptions, "/", nil)
req.Host = "localhost:9999"
// No Authorization header
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
t.Errorf("Expected status 200 for OPTIONS, got %d", resp.StatusCode)
}
}
@@ -356,7 +356,7 @@ func TestSigV4Auth_ValidSignatureWithBody(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
@@ -364,7 +364,7 @@ func TestSigV4Auth_ValidSignatureWithBody(t *testing.T) {
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(respBody))
}
@@ -379,14 +379,14 @@ func TestSigV4Auth_ValidSignatureWithQueryString(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/?"+queryString, nil)
req := httptest.NewRequest(http.MethodGet, "/?"+queryString, nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(respBody))
}
@@ -427,14 +427,14 @@ func TestSigV4Auth_InactiveAccessKey(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403 for inactive key, got %d", resp.StatusCode)
}
@@ -462,14 +462,14 @@ func TestSigV4Auth_NilIAMService(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 500 {
if resp.StatusCode != http.StatusInternalServerError {
t.Errorf("Expected status 500 for nil IAM service, got %d", resp.StatusCode)
}
@@ -515,14 +515,14 @@ func TestSigV4Auth_CallerIdentity(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Fatalf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -650,14 +650,14 @@ func TestSigV4Auth_DecryptFailure(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 500 {
if resp.StatusCode != http.StatusInternalServerError {
t.Errorf("Expected status 500 for decrypt failure, got %d", resp.StatusCode)
}
@@ -786,14 +786,14 @@ func TestSigV4Auth_ExpiredTimestamp(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService, past,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
@@ -813,14 +813,14 @@ func TestSigV4Auth_FutureTimestamp(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService, future,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected status 403, got %d", resp.StatusCode)
}
@@ -840,14 +840,14 @@ func TestSigV4Auth_TimestampWithinSkew(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService, recent,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -878,18 +878,18 @@ func TestSigV4Auth_ClockSkewBoundary(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService, at,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if tc.expectPass && resp.StatusCode != 200 {
if tc.expectPass && resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected 200, got %d, body: %s", resp.StatusCode, string(body))
}
if !tc.expectPass && resp.StatusCode != 403 {
if !tc.expectPass && resp.StatusCode != http.StatusForbidden {
t.Errorf("Expected 403, got %d", resp.StatusCode)
}
})
@@ -904,14 +904,14 @@ func TestSigV4Auth_MissingXAmzDate(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
// Deliberately omit X-Amz-Date
resp := doRequest(handler, req)
if resp.StatusCode != 400 {
if resp.StatusCode != http.StatusBadRequest {
t.Errorf("Expected status 400, got %d", resp.StatusCode)
}
@@ -929,14 +929,14 @@ func TestSigV4Auth_MalformedTimestamp(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", "not-a-valid-date")
resp := doRequest(handler, req)
if resp.StatusCode != 400 {
if resp.StatusCode != http.StatusBadRequest {
t.Errorf("Expected status 400, got %d", resp.StatusCode)
}
@@ -965,7 +965,7 @@ func TestWriteSigV4Error_ResponseFormat(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.errorCode, func(t *testing.T) {
w := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
gw.writeSigV4Error(w, req, tc.errorCode)
resp := w.Result()
@@ -1002,8 +1002,8 @@ func TestWriteSigV4Error_PreservesRequestID(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, Region: testRegion}
w := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/", nil)
req.Header.Set("x-amz-request-id", "test-request-id-123")
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Header.Set("X-Amz-Request-Id", "test-request-id-123")
gw.writeSigV4Error(w, req, awserrors.ErrorIncompleteSignature)
@@ -1059,14 +1059,14 @@ func TestSigV4Auth_ContextPropagation(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Fatalf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1142,7 +1142,7 @@ func TestSigV4Auth_ContextDoesNotLeakBetweenRequests(t *testing.T) {
"GET", "/", "", "",
testAccessKey, testSecretKey, testRegion, testService,
)
req1 := httptest.NewRequest("GET", "/", nil)
req1 := httptest.NewRequest(http.MethodGet, "/", nil)
req1.Host = "localhost:9999"
req1.Header.Set("Authorization", authHeader1)
req1.Header.Set("X-Amz-Date", timestamp1)
@@ -1158,7 +1158,7 @@ func TestSigV4Auth_ContextDoesNotLeakBetweenRequests(t *testing.T) {
"GET", "/", "", "",
secondKey, testSecretKey, testRegion, testService,
)
req2 := httptest.NewRequest("GET", "/", nil)
req2 := httptest.NewRequest(http.MethodGet, "/", nil)
req2.Host = "localhost:9999"
req2.Header.Set("Authorization", authHeader2)
req2.Header.Set("X-Amz-Date", timestamp2)
@@ -1191,14 +1191,14 @@ func TestSigV4Auth_LookupAccessKeyUnexpectedError(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(r, req)
if resp.StatusCode != 500 {
if resp.StatusCode != http.StatusInternalServerError {
t.Errorf("Expected status 500 for unexpected lookup error, got %d", resp.StatusCode)
}
@@ -1230,14 +1230,14 @@ func TestSigV4Auth_PathWithSpecialCharacters(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", tc.reqPath, nil)
req := httptest.NewRequest(http.MethodGet, tc.reqPath, nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1291,7 +1291,7 @@ func TestSigV4Auth_SignedContentType(t *testing.T) {
headers, "content-type;host;x-amz-date",
)
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
@@ -1299,7 +1299,7 @@ func TestSigV4Auth_SignedContentType(t *testing.T) {
resp := doRequest(r, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(respBody))
}
@@ -1314,14 +1314,14 @@ func TestSigV4Auth_EmptyBodyPOST(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("POST", "/", strings.NewReader(""))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(""))
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1388,7 +1388,7 @@ func TestComputeSignatureWithSecret_SpecialURIPaths(t *testing.T) {
if reqPath == "" {
reqPath = "/"
}
req := httptest.NewRequest("GET", reqPath, nil)
req := httptest.NewRequest(http.MethodGet, reqPath, nil)
req.Host = "localhost:9999"
req.Header.Set("X-Amz-Date", timestamp)
@@ -1412,7 +1412,7 @@ func TestComputeSignatureWithSecret_Determinism(t *testing.T) {
var first string
for i := range 10 {
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("X-Amz-Date", timestamp)
@@ -1442,7 +1442,7 @@ func TestComputeSignatureWithSecret_MultipartContentType(t *testing.T) {
headers, "content-type;host;x-amz-date",
)
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
@@ -1450,7 +1450,7 @@ func TestComputeSignatureWithSecret_MultipartContentType(t *testing.T) {
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200 for multipart content-type, got %d, body: %s", resp.StatusCode, string(respBody))
}
@@ -1475,14 +1475,14 @@ func TestSigV4Auth_QueryStringEdgeCases(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/?"+tc.queryString, nil)
req := httptest.NewRequest(http.MethodGet, "/?"+tc.queryString, nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1495,6 +1495,7 @@ func TestSigV4Auth_QueryStringEdgeCases(t *testing.T) {
// policyMockIAMService extends mockIAMService with configurable GetUserPolicies.
type policyMockIAMService struct {
mockIAMService
getUserPoliciesFn func(accountID, userName string) ([]handlers_iam.PolicyDocument, error)
}
@@ -1508,6 +1509,7 @@ func (m *policyMockIAMService) GetUserPolicies(accountID, userName string) ([]ha
// errorLookupMockIAMService returns a configurable error from LookupAccessKey.
type errorLookupMockIAMService struct {
mockIAMService
err error
}
@@ -1566,14 +1568,14 @@ func TestCheckPolicy_NonRootNoPolicies_Denied(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 403, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1632,14 +1634,14 @@ func TestCheckPolicy_NonRootWithAllow_Passes(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1698,14 +1700,14 @@ func TestCheckPolicy_NonRootWithExplicitDeny_Denied(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 403, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1765,14 +1767,14 @@ func TestCheckPolicy_RootGlobalAccount_Bypasses(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 200 {
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 200 (root bypass), got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1813,14 +1815,14 @@ func TestCheckPolicy_MissingAccountID_InternalError(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 500 {
if resp.StatusCode != http.StatusInternalServerError {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 500, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1863,14 +1865,14 @@ func TestCheckPolicy_GetUserPoliciesError_InternalError(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
resp := doRequest(handler, req)
if resp.StatusCode != 500 {
if resp.StatusCode != http.StatusInternalServerError {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 500, got %d, body: %s", resp.StatusCode, string(body))
}
@@ -1915,7 +1917,7 @@ func TestCheckPolicy_RootNonGlobalAccount_StillEvaluated(t *testing.T) {
testAccessKey, testSecretKey, testRegion, testService,
)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Host = "localhost:9999"
req.Header.Set("Authorization", authHeader)
req.Header.Set("X-Amz-Date", timestamp)
@@ -1924,7 +1926,7 @@ func TestCheckPolicy_RootNonGlobalAccount_StillEvaluated(t *testing.T) {
// root on non-global account is evaluated by the policy engine like any
// other user. With no policies attached, the default deny applies.
if resp.StatusCode != 403 {
if resp.StatusCode != http.StatusForbidden {
body, _ := io.ReadAll(resp.Body)
t.Errorf("Expected status 403, got %d, body: %s", resp.StatusCode, string(body))
}
+50
View File
@@ -14,10 +14,12 @@ import (
"github.com/mulgadc/spinifex/spinifex/awserrors"
gateway_ec2_account "github.com/mulgadc/spinifex/spinifex/gateway/ec2/account"
gateway_ec2_eigw "github.com/mulgadc/spinifex/spinifex/gateway/ec2/eigw"
gateway_ec2_eip "github.com/mulgadc/spinifex/spinifex/gateway/ec2/eip"
gateway_ec2_igw "github.com/mulgadc/spinifex/spinifex/gateway/ec2/igw"
gateway_ec2_image "github.com/mulgadc/spinifex/spinifex/gateway/ec2/image"
gateway_ec2_instance "github.com/mulgadc/spinifex/spinifex/gateway/ec2/instance"
gateway_ec2_key "github.com/mulgadc/spinifex/spinifex/gateway/ec2/key"
gateway_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/gateway/ec2/placementgroup"
gateway_ec2_snapshot "github.com/mulgadc/spinifex/spinifex/gateway/ec2/snapshot"
gateway_ec2_tags "github.com/mulgadc/spinifex/spinifex/gateway/ec2/tags"
gateway_ec2_volume "github.com/mulgadc/spinifex/spinifex/gateway/ec2/volume"
@@ -197,6 +199,15 @@ var ec2Actions = map[string]EC2Handler{
"DescribeEgressOnlyInternetGateways": ec2Handler(func(input *ec2.DescribeEgressOnlyInternetGatewaysInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eigw.DescribeEgressOnlyInternetGateways(input, gw.NATSConn, accountID)
}),
"CreatePlacementGroup": ec2Handler(func(input *ec2.CreatePlacementGroupInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_placementgroup.CreatePlacementGroup(input, gw.NATSConn, accountID)
}),
"DeletePlacementGroup": ec2Handler(func(input *ec2.DeletePlacementGroupInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_placementgroup.DeletePlacementGroup(input, gw.NATSConn, accountID)
}),
"DescribePlacementGroups": ec2Handler(func(input *ec2.DescribePlacementGroupsInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_placementgroup.DescribePlacementGroups(input, gw.NATSConn, accountID)
}),
"CreateVpc": ec2Handler(func(input *ec2.CreateVpcInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.CreateVpc(input, gw.NATSConn, accountID)
}),
@@ -215,6 +226,9 @@ var ec2Actions = map[string]EC2Handler{
"DescribeSubnets": ec2Handler(func(input *ec2.DescribeSubnetsInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.DescribeSubnets(input, gw.NATSConn, accountID)
}),
"ModifySubnetAttribute": ec2Handler(func(input *ec2.ModifySubnetAttributeInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.ModifySubnetAttribute(input, gw.NATSConn, accountID)
}),
"CreateNetworkInterface": ec2Handler(func(input *ec2.CreateNetworkInterfaceInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.CreateNetworkInterface(input, gw.NATSConn, accountID)
}),
@@ -224,6 +238,42 @@ var ec2Actions = map[string]EC2Handler{
"DescribeNetworkInterfaces": ec2Handler(func(input *ec2.DescribeNetworkInterfacesInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.DescribeNetworkInterfaces(input, gw.NATSConn, accountID)
}),
"CreateSecurityGroup": ec2Handler(func(input *ec2.CreateSecurityGroupInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.CreateSecurityGroup(input, gw.NATSConn, accountID)
}),
"DeleteSecurityGroup": ec2Handler(func(input *ec2.DeleteSecurityGroupInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.DeleteSecurityGroup(input, gw.NATSConn, accountID)
}),
"DescribeSecurityGroups": ec2Handler(func(input *ec2.DescribeSecurityGroupsInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.DescribeSecurityGroups(input, gw.NATSConn, accountID)
}),
"AuthorizeSecurityGroupIngress": ec2Handler(func(input *ec2.AuthorizeSecurityGroupIngressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.AuthorizeSecurityGroupIngress(input, gw.NATSConn, accountID)
}),
"AuthorizeSecurityGroupEgress": ec2Handler(func(input *ec2.AuthorizeSecurityGroupEgressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.AuthorizeSecurityGroupEgress(input, gw.NATSConn, accountID)
}),
"RevokeSecurityGroupIngress": ec2Handler(func(input *ec2.RevokeSecurityGroupIngressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.RevokeSecurityGroupIngress(input, gw.NATSConn, accountID)
}),
"RevokeSecurityGroupEgress": ec2Handler(func(input *ec2.RevokeSecurityGroupEgressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_vpc.RevokeSecurityGroupEgress(input, gw.NATSConn, accountID)
}),
"AllocateAddress": ec2Handler(func(input *ec2.AllocateAddressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eip.AllocateAddress(input, gw.NATSConn, accountID)
}),
"ReleaseAddress": ec2Handler(func(input *ec2.ReleaseAddressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eip.ReleaseAddress(input, gw.NATSConn, accountID)
}),
"AssociateAddress": ec2Handler(func(input *ec2.AssociateAddressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eip.AssociateAddress(input, gw.NATSConn, accountID)
}),
"DisassociateAddress": ec2Handler(func(input *ec2.DisassociateAddressInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eip.DisassociateAddress(input, gw.NATSConn, accountID)
}),
"DescribeAddresses": ec2Handler(func(input *ec2.DescribeAddressesInput, gw *GatewayConfig, accountID string) (any, error) {
return gateway_ec2_eip.DescribeAddresses(input, gw.NATSConn, accountID)
}),
}
// ec2LocalActions are actions that don't require a NATS connection.
@@ -0,0 +1,27 @@
package gateway_ec2_eip
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
"github.com/nats-io/nats.go"
)
// AllocateAddress handles the EC2 AllocateAddress API call.
func AllocateAddress(input *ec2.AllocateAddressInput, natsConn *nats.Conn, accountID string) (ec2.AllocateAddressOutput, error) {
var output ec2.AllocateAddressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
svc := handlers_ec2_eip.NewNATSEIPService(natsConn)
result, err := svc.AllocateAddress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,30 @@
package gateway_ec2_eip
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
"github.com/nats-io/nats.go"
)
// AssociateAddress handles the EC2 AssociateAddress API call.
func AssociateAddress(input *ec2.AssociateAddressInput, natsConn *nats.Conn, accountID string) (ec2.AssociateAddressOutput, error) {
var output ec2.AssociateAddressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.AllocationId == nil || *input.AllocationId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_eip.NewNATSEIPService(natsConn)
result, err := svc.AssociateAddress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,27 @@
package gateway_ec2_eip
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
"github.com/nats-io/nats.go"
)
// DescribeAddresses handles the EC2 DescribeAddresses API call.
func DescribeAddresses(input *ec2.DescribeAddressesInput, natsConn *nats.Conn, accountID string) (ec2.DescribeAddressesOutput, error) {
var output ec2.DescribeAddressesOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
svc := handlers_ec2_eip.NewNATSEIPService(natsConn)
result, err := svc.DescribeAddresses(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,30 @@
package gateway_ec2_eip
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
"github.com/nats-io/nats.go"
)
// DisassociateAddress handles the EC2 DisassociateAddress API call.
func DisassociateAddress(input *ec2.DisassociateAddressInput, natsConn *nats.Conn, accountID string) (ec2.DisassociateAddressOutput, error) {
var output ec2.DisassociateAddressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.AssociationId == nil || *input.AssociationId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_eip.NewNATSEIPService(natsConn)
result, err := svc.DisassociateAddress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,30 @@
package gateway_ec2_eip
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_eip "github.com/mulgadc/spinifex/spinifex/handlers/ec2/eip"
"github.com/nats-io/nats.go"
)
// ReleaseAddress handles the EC2 ReleaseAddress API call.
func ReleaseAddress(input *ec2.ReleaseAddressInput, natsConn *nats.Conn, accountID string) (ec2.ReleaseAddressOutput, error) {
var output ec2.ReleaseAddressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.AllocationId == nil || *input.AllocationId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_eip.NewNATSEIPService(natsConn)
result, err := svc.ReleaseAddress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
+1 -2
View File
@@ -24,11 +24,10 @@ func ValidateDescribeImagesInput(input *ec2.DescribeImagesInput) (err error) {
}
}
return
return err
}
func DescribeImages(input *ec2.DescribeImagesInput, natsConn *nats.Conn, accountID string) (output ec2.DescribeImagesOutput, err error) {
// Validate input
err = ValidateDescribeImagesInput(input)
@@ -57,7 +57,10 @@ func RebootInstances(input *ec2.RebootInstancesInput, natsConn *nats.Conn, accou
describeInput := &ec2.DescribeInstancesInput{
InstanceIds: []*string{&instanceID},
}
describeData, _ := json.Marshal(describeInput)
describeData, err := json.Marshal(describeInput)
if err != nil {
return nil, fmt.Errorf("marshal describe input: %w", err)
}
if reservations := queryInstanceBucket(natsConn, "ec2.DescribeStoppedInstances", describeData, accountID); len(reservations) > 0 {
return nil, errors.New(awserrors.ErrorIncorrectInstanceState)
}
+64 -17
View File
@@ -5,11 +5,11 @@ import (
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/mulgadc/spinifex/spinifex/utils"
handlers_ec2_instance "github.com/mulgadc/spinifex/spinifex/handlers/ec2/instance"
"github.com/nats-io/nats.go"
)
@@ -53,15 +53,12 @@ func ValidateRunInstancesInput(input *ec2.RunInstancesInput) (err error) {
if !strings.HasPrefix(*input.ImageId, "ami-") {
return errors.New(awserrors.ErrorInvalidAMIIDMalformed)
}
return
return err
}
func RunInstances(input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID string) (reservation ec2.Reservation, err error) {
// Validate input
err = ValidateRunInstancesInput(input)
@@ -69,28 +66,78 @@ func RunInstances(input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID s
return reservation, err
}
// Create NATS-based instance service
service := handlers_ec2_instance.NewNATSInstanceService(natsConn)
// Placement group routing: when a placement group is specified, validate it
// and route based on its strategy (spread or cluster).
groupName := placementGroupName(input)
if groupName != "" {
strategy, err := lookupPlacementGroupStrategy(natsConn, accountID, groupName)
if err != nil {
return reservation, err
}
// Call the service directly (no need for JSON marshaling/unmarshaling in same process)
reservationPtr, err := service.RunInstances(input, accountID)
switch strategy {
case ec2.PlacementStrategySpread:
reservationPtr, err := distributeInstancesSpread(input, natsConn, accountID, groupName)
if err != nil {
return reservation, err
}
return *reservationPtr, nil
case ec2.PlacementStrategyCluster:
reservationPtr, err := distributeInstancesCluster(input, natsConn, accountID, groupName)
if err != nil {
return reservation, err
}
return *reservationPtr, nil
default:
return reservation, errors.New(awserrors.ErrorInvalidParameterValue)
}
}
// Capacity-aware routing: query all nodes for capacity and distribute
// instances across nodes with best-effort spread. This applies to both
// single-instance (count=1) and batch (count>1) launches, ensuring fair
// distribution across the cluster.
reservationPtr, err := distributeInstances(input, natsConn, accountID)
if err != nil {
if errors.Is(err, nats.ErrNoResponders) {
// ErrNoResponders means no daemon subscribes to ec2.RunInstances.{type}.
// This happens when either: (a) the type is unknown, or (b) all nodes
// are at capacity. Query DescribeInstanceTypes to differentiate.
// When no nodes have capacity, distinguish between "unknown instance type"
// and "all nodes full" by checking DescribeInstanceTypes.
if err.Error() == awserrors.ErrorInsufficientInstanceCapacity {
if !isKnownInstanceType(natsConn, *input.InstanceType) {
return reservation, errors.New(awserrors.ErrorInvalidInstanceType)
}
return reservation, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
return reservation, err
}
// Dereference pointer to return value
return *reservationPtr, nil
}
// placementGroupName extracts the placement group name from RunInstancesInput.
func placementGroupName(input *ec2.RunInstancesInput) string {
if input.Placement != nil && input.Placement.GroupName != nil {
return aws.StringValue(input.Placement.GroupName)
}
return ""
}
// lookupPlacementGroupStrategy validates that a placement group exists and returns its strategy.
func lookupPlacementGroupStrategy(natsConn *nats.Conn, accountID, groupName string) (string, error) {
pgSvc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
out, err := pgSvc.DescribePlacementGroups(&ec2.DescribePlacementGroupsInput{
GroupNames: []*string{aws.String(groupName)},
}, accountID)
if err != nil {
return "", err
}
if len(out.PlacementGroups) == 0 {
return "", errors.New(awserrors.ErrorInvalidPlacementGroupUnknown)
}
pg := out.PlacementGroups[0]
if pg.State == nil || *pg.State != ec2.PlacementGroupStateAvailable {
return "", errors.New(awserrors.ErrorInvalidPlacementGroupUnknown)
}
return aws.StringValue(pg.Strategy), nil
}
// isKnownInstanceType checks whether any daemon recognizes the given instance type.
func isKnownInstanceType(natsConn *nats.Conn, instanceType string) bool {
result, err := utils.NATSRequest[ec2.DescribeInstanceTypesOutput](
@@ -23,7 +23,6 @@ var defaults = ec2.RunInstancesInput{
}
func TestParseRunInstances(t *testing.T) {
// Group multiple tests
tests := []struct {
name string
@@ -214,5 +213,4 @@ func TestParseRunInstances(t *testing.T) {
}
// Additional test
}
+555
View File
@@ -0,0 +1,555 @@
package gateway_ec2_instance
import (
"encoding/json"
"errors"
"fmt"
"log/slog"
"math/rand"
"sort"
"sync"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/mulgadc/spinifex/spinifex/types"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/nats-io/nats.go"
)
// nodeAllocation tracks how many instances to launch on a specific node.
type nodeAllocation struct {
NodeID string
Available int // capacity for the requested instance type
Assigned int // instances assigned to this node
}
// distributeInstances implements the best-effort spread algorithm for multi-node
// instance distribution. It queries cluster capacity, filters eligible nodes,
// distributes instances across nodes (1 per node first, then pack extras onto
// nodes with most remaining capacity), launches in parallel, and handles
// partial failures with rollback.
//
// Returns the merged reservation on success or an error.
func distributeInstances(input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID string) (*ec2.Reservation, error) {
instanceType := aws.StringValue(input.InstanceType)
minCount := int(aws.Int64Value(input.MinCount))
maxCount := int(aws.Int64Value(input.MaxCount))
// Step 1: Query capacity from all nodes via fan-out
nodes, err := queryNodeCapacity(natsConn, instanceType)
if err != nil {
return nil, err
}
if len(nodes) == 0 {
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
// Step 2: Calculate total capacity and check feasibility
totalCapacity := 0
for _, n := range nodes {
totalCapacity += n.Available
}
if totalCapacity < minCount {
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
// Step 3: Determine launch count (capped to MaxCount and available capacity)
launchCount := min(maxCount, totalCapacity)
// Step 4: Distribute instances across nodes using best-effort spread
allocations := spreadAllocate(nodes, launchCount)
// Step 5: Launch instances on each node in parallel
results := launchOnNodes(allocations, input, natsConn, accountID)
// Step 6: Aggregate results and handle partial failure
return aggregateResults(results, minCount, natsConn, accountID)
}
// queryNodeCapacity fans out spinifex.node.status to all daemons and returns
// eligible nodes (those with Available >= 1 for the requested instance type),
// sorted by available capacity descending with random tiebreaking for fair
// distribution among equal-capacity nodes.
//
// Uses a collection window: after the first response arrives, only waits an
// additional 200ms for remaining responses (instead of the full 3s timeout).
func queryNodeCapacity(natsConn *nats.Conn, instanceType string) ([]nodeAllocation, error) {
inbox := nats.NewInbox()
sub, err := natsConn.SubscribeSync(inbox)
if err != nil {
return nil, fmt.Errorf("failed to create inbox: %w", err)
}
defer sub.Unsubscribe()
pubMsg := nats.NewMsg("spinifex.node.status")
pubMsg.Reply = inbox
pubMsg.Data = []byte("{}")
if err := natsConn.PublishMsg(pubMsg); err != nil {
return nil, fmt.Errorf("failed to publish node status request: %w", err)
}
const (
initialTimeout = 3 * time.Second
collectWindow = 200 * time.Millisecond
)
deadline := time.Now().Add(initialTimeout)
gotFirst := false
var nodes []nodeAllocation
for time.Now().Before(deadline) {
remaining := time.Until(deadline)
if remaining <= 0 {
break
}
msg, err := sub.NextMsg(remaining)
if err != nil {
if err == nats.ErrTimeout {
break
}
slog.Debug("queryNodeCapacity: error receiving message", "err", err)
break
}
var status types.NodeStatusResponse
if err := json.Unmarshal(msg.Data, &status); err != nil {
slog.Debug("queryNodeCapacity: failed to unmarshal response", "err", err)
continue
}
if status.Node == "" {
slog.Debug("queryNodeCapacity: skipping response with empty node ID")
continue
}
// Find capacity for the requested instance type on this node
for _, cap := range status.InstanceTypes {
if cap.Name == instanceType && cap.Available >= 1 {
nodes = append(nodes, nodeAllocation{
NodeID: status.Node,
Available: cap.Available,
})
break
}
}
// After the first valid response, tighten the deadline so we don't
// wait the full 3s for stragglers.
if !gotFirst {
gotFirst = true
collectDeadline := time.Now().Add(collectWindow)
if collectDeadline.Before(deadline) {
deadline = collectDeadline
}
}
}
// Shuffle first for random tiebreaking, then stable-sort by capacity
// descending. This ensures fair distribution among equal-capacity nodes.
rand.Shuffle(len(nodes), func(i, j int) {
nodes[i], nodes[j] = nodes[j], nodes[i]
})
sort.SliceStable(nodes, func(i, j int) bool {
return nodes[i].Available > nodes[j].Available
})
return nodes, nil
}
// spreadAllocate distributes count instances across nodes using best-effort spread:
// - Round 1: assign 1 instance to each node (up to count)
// - Round 2+: assign remaining to nodes with most remaining capacity
func spreadAllocate(nodes []nodeAllocation, count int) []nodeAllocation {
// Make a working copy
allocs := make([]nodeAllocation, len(nodes))
copy(allocs, nodes)
remaining := count
// Round 1: 1 instance per node
for i := range allocs {
if remaining <= 0 {
break
}
allocs[i].Assigned = 1
remaining--
}
// Round 2+: pack remaining onto nodes with most remaining capacity.
// Ties are broken by preferring the node with fewer assigned instances,
// which produces a more balanced spread.
for remaining > 0 {
bestIdx := -1
bestRemaining := 0
for i, a := range allocs {
nodeRemaining := a.Available - a.Assigned
if nodeRemaining > bestRemaining {
bestRemaining = nodeRemaining
bestIdx = i
} else if nodeRemaining == bestRemaining && nodeRemaining > 0 &&
(bestIdx < 0 || a.Assigned < allocs[bestIdx].Assigned) {
bestIdx = i
}
}
if bestIdx < 0 {
break // no more capacity anywhere
}
allocs[bestIdx].Assigned++
remaining--
}
// Filter out nodes with no assignments
result := make([]nodeAllocation, 0, len(allocs))
for _, a := range allocs {
if a.Assigned > 0 {
result = append(result, a)
}
}
return result
}
// nodeLaunchResult holds the outcome of launching instances on a single node.
type nodeLaunchResult struct {
NodeID string
Reservation *ec2.Reservation
Err error
}
// launchOnNodes sends targeted RunInstances requests to specific nodes in parallel.
// Each node gets MinCount=MaxCount=assignedCount so the daemon treats it as all-or-nothing.
func launchOnNodes(allocations []nodeAllocation, input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID string) []nodeLaunchResult {
instanceType := aws.StringValue(input.InstanceType)
results := make([]nodeLaunchResult, len(allocations))
var wg sync.WaitGroup
for i, alloc := range allocations {
wg.Add(1)
go func(idx int, a nodeAllocation) {
defer wg.Done()
// Build per-node input with MinCount=MaxCount=assigned
nodeInput := *input
nodeInput.MinCount = aws.Int64(int64(a.Assigned))
nodeInput.MaxCount = aws.Int64(int64(a.Assigned))
topic := fmt.Sprintf("ec2.RunInstances.%s.%s", instanceType, a.NodeID)
reservation, err := utils.NATSRequest[ec2.Reservation](natsConn, topic, &nodeInput, 5*time.Minute, accountID)
if err != nil {
results[idx] = nodeLaunchResult{NodeID: a.NodeID, Err: fmt.Errorf("launch on %s: %w", a.NodeID, err)}
return
}
results[idx] = nodeLaunchResult{NodeID: a.NodeID, Reservation: reservation}
}(i, alloc)
}
wg.Wait()
return results
}
// aggregateResults merges successful node launches into a single reservation.
// If total launched instances < minCount, all successfully launched instances
// are terminated (rollback) and InsufficientInstanceCapacity is returned.
func aggregateResults(results []nodeLaunchResult, minCount int, natsConn *nats.Conn, accountID string) (*ec2.Reservation, error) {
var allInstances []*ec2.Instance
var reservationID *string
for _, r := range results {
if r.Err != nil {
slog.Warn("distributeInstances: node launch failed", "node", r.NodeID, "err", r.Err)
continue
}
if r.Reservation != nil {
allInstances = append(allInstances, r.Reservation.Instances...)
if reservationID == nil {
reservationID = r.Reservation.ReservationId
}
}
}
totalLaunched := len(allInstances)
if totalLaunched < minCount {
// Rollback: terminate all successfully launched instances
if totalLaunched > 0 {
rollbackInstances(allInstances, natsConn, accountID)
}
// Propagate specific client errors (e.g. InvalidAMIID.NotFound) instead
// of the generic InsufficientInstanceCapacity.
if clientErr := extractClientError(results); clientErr != nil {
return nil, clientErr
}
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
return &ec2.Reservation{
ReservationId: reservationID,
Instances: allInstances,
}, nil
}
// distributeInstancesSpread implements strict 1-per-node spread for placement groups.
// It queries capacity, reserves unused nodes via CAS, launches 1 instance per node,
// and finalizes or rolls back the placement group record.
func distributeInstancesSpread(input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID string, groupName string) (*ec2.Reservation, error) {
instanceType := aws.StringValue(input.InstanceType)
minCount := int(aws.Int64Value(input.MinCount))
maxCount := int(aws.Int64Value(input.MaxCount))
pgSvc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
// Step 1: Query capacity from all nodes
nodes, err := queryNodeCapacity(natsConn, instanceType)
if err != nil {
return nil, err
}
if len(nodes) == 0 {
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
// Build list of eligible node IDs
eligibleNodeIDs := make([]string, len(nodes))
for i, n := range nodes {
eligibleNodeIDs[i] = n.NodeID
}
// Step 2: Reserve nodes atomically (CAS-based, handles concurrent launches)
reserveOut, err := pgSvc.ReserveSpreadNodes(&handlers_ec2_placementgroup.ReserveSpreadNodesInput{
GroupName: groupName,
EligibleNodes: eligibleNodeIDs,
MinCount: minCount,
MaxCount: maxCount,
}, accountID)
if err != nil {
return nil, err
}
reservedNodes := reserveOut.ReservedNodes
// Step 3: Build allocations (1 instance per reserved node)
allocations := make([]nodeAllocation, len(reservedNodes))
for i, nodeID := range reservedNodes {
allocations[i] = nodeAllocation{NodeID: nodeID, Assigned: 1}
}
// Step 4: Launch instances on reserved nodes in parallel
results := launchOnNodes(allocations, input, natsConn, accountID)
// Step 5: Collect results
var allInstances []*ec2.Instance
var reservationID *string
nodeInstances := make(map[string][]string)
var failedNodes []string
for _, r := range results {
if r.Err != nil {
slog.Warn("distributeInstancesSpread: node launch failed", "node", r.NodeID, "err", r.Err)
failedNodes = append(failedNodes, r.NodeID)
continue
}
if r.Reservation != nil {
for _, inst := range r.Reservation.Instances {
allInstances = append(allInstances, inst)
if inst.InstanceId != nil {
nodeInstances[r.NodeID] = append(nodeInstances[r.NodeID], *inst.InstanceId)
}
}
if reservationID == nil {
reservationID = r.Reservation.ReservationId
}
}
}
totalLaunched := len(allInstances)
if totalLaunched < minCount {
// Rollback: terminate launched instances and release all reserved nodes
if totalLaunched > 0 {
rollbackInstances(allInstances, natsConn, accountID)
}
// Release all reserved nodes from the placement group
if _, err := pgSvc.ReleaseSpreadNodes(&handlers_ec2_placementgroup.ReleaseSpreadNodesInput{
GroupName: groupName,
Nodes: reservedNodes,
}, accountID); err != nil {
slog.Error("distributeInstancesSpread: failed to release nodes after rollback", "err", err)
}
if clientErr := extractClientError(results); clientErr != nil {
return nil, clientErr
}
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
// Step 6: Finalize — replace placeholders with actual instance IDs.
// If finalization fails, the placement group record has stale placeholders;
// roll back the launched instances to avoid untracked instances.
if _, err := pgSvc.FinalizeSpreadInstances(&handlers_ec2_placementgroup.FinalizeSpreadInstancesInput{
GroupName: groupName,
NodeInstances: nodeInstances,
}, accountID); err != nil {
slog.Error("distributeInstancesSpread: finalize failed, rolling back instances", "err", err)
rollbackInstances(allInstances, natsConn, accountID)
// Release all reserved nodes (both succeeded and failed)
allReleaseNodes := append(reservedNodes[:0:0], reservedNodes...)
if _, releaseErr := pgSvc.ReleaseSpreadNodes(&handlers_ec2_placementgroup.ReleaseSpreadNodesInput{
GroupName: groupName,
Nodes: allReleaseNodes,
}, accountID); releaseErr != nil {
slog.Error("distributeInstancesSpread: failed to release nodes after finalize rollback", "err", releaseErr)
}
return nil, fmt.Errorf("failed to finalize placement group record: %w", err)
}
// Release any failed nodes that didn't launch
if len(failedNodes) > 0 {
if _, err := pgSvc.ReleaseSpreadNodes(&handlers_ec2_placementgroup.ReleaseSpreadNodesInput{
GroupName: groupName,
Nodes: failedNodes,
}, accountID); err != nil {
slog.Error("distributeInstancesSpread: failed to release failed nodes", "err", err)
}
}
return &ec2.Reservation{
ReservationId: reservationID,
Instances: allInstances,
}, nil
}
// distributeInstancesCluster implements cluster placement group routing.
// All instances are pinned to a single node. If the group already has instances,
// subsequent launches go to the same node. If empty, picks the node with most capacity.
func distributeInstancesCluster(input *ec2.RunInstancesInput, natsConn *nats.Conn, accountID string, groupName string) (*ec2.Reservation, error) {
instanceType := aws.StringValue(input.InstanceType)
minCount := int(aws.Int64Value(input.MinCount))
maxCount := int(aws.Int64Value(input.MaxCount))
pgSvc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
// Step 1: Query capacity from all nodes
nodes, err := queryNodeCapacity(natsConn, instanceType)
if err != nil {
return nil, err
}
// Build eligible node IDs (already sorted by capacity desc)
eligibleNodeIDs := make([]string, len(nodes))
for i, n := range nodes {
eligibleNodeIDs[i] = n.NodeID
}
// Step 2: Reserve the cluster target node (CAS-based for empty groups)
reserveOut, err := pgSvc.ReserveClusterNode(&handlers_ec2_placementgroup.ReserveClusterNodeInput{
GroupName: groupName,
EligibleNodes: eligibleNodeIDs,
}, accountID)
if err != nil {
return nil, err
}
targetNode := reserveOut.TargetNode
// Step 3: Check capacity on the target node
var targetCapacity int
for _, n := range nodes {
if n.NodeID == targetNode {
targetCapacity = n.Available
break
}
}
if targetCapacity < minCount {
return nil, errors.New(awserrors.ErrorInsufficientInstanceCapacity)
}
// Cap launch count to available capacity and MaxCount
launchCount := min(maxCount, targetCapacity)
// Step 4: Targeted launch — all instances to the pinned node
allocations := []nodeAllocation{{
NodeID: targetNode,
Assigned: launchCount,
}}
results := launchOnNodes(allocations, input, natsConn, accountID)
// Step 5: Handle result (single node, no partial failure logic needed)
if results[0].Err != nil {
if clientErr := extractClientError(results); clientErr != nil {
return nil, clientErr
}
return nil, results[0].Err
}
reservation := results[0].Reservation
if reservation == nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
// Step 6: CAS-update record with launched instance IDs
nodeInstances := make(map[string][]string)
for _, inst := range reservation.Instances {
if inst.InstanceId != nil {
nodeInstances[targetNode] = append(nodeInstances[targetNode], *inst.InstanceId)
}
}
if _, err := pgSvc.FinalizeClusterInstances(&handlers_ec2_placementgroup.FinalizeClusterInstancesInput{
GroupName: groupName,
NodeInstances: nodeInstances,
}, accountID); err != nil {
slog.Error("distributeInstancesCluster: finalize failed, rolling back instances", "err", err)
rollbackInstances(reservation.Instances, natsConn, accountID)
return nil, fmt.Errorf("failed to finalize cluster placement group record: %w", err)
}
return reservation, nil
}
// extractClientError scans node launch results for specific client validation
// errors (e.g. InvalidAMIID.NotFound) that should be propagated instead of the
// generic InsufficientInstanceCapacity. Returns the first match, or nil.
func extractClientError(results []nodeLaunchResult) error {
for _, r := range results {
if r.Err == nil {
continue
}
inner := errors.Unwrap(r.Err)
if inner == nil {
continue
}
switch inner.Error() {
case awserrors.ErrorInvalidAMIIDNotFound,
awserrors.ErrorInvalidAMIIDMalformed,
awserrors.ErrorInvalidAMIIDUnavailable,
awserrors.ErrorInvalidKeyPairNotFound,
awserrors.ErrorMissingParameter:
return inner
}
}
return nil
}
// rollbackInstances terminates all instances from a failed multi-node launch.
func rollbackInstances(instances []*ec2.Instance, natsConn *nats.Conn, accountID string) {
var ids []*string
for _, inst := range instances {
if inst.InstanceId != nil {
ids = append(ids, inst.InstanceId)
}
}
if len(ids) == 0 {
return
}
slog.Warn("distributeInstances: rolling back launched instances", "count", len(ids))
_, err := TerminateInstances(&ec2.TerminateInstancesInput{
InstanceIds: ids,
}, natsConn, accountID)
if err != nil {
slog.Error("distributeInstances: rollback failed", "err", err)
}
}
File diff suppressed because it is too large Load Diff
+1 -2
View File
@@ -19,11 +19,10 @@ func ValidateCreateKeyPairInput(input *ec2.CreateKeyPairInput) (err error) {
return errors.New(awserrors.ErrorMissingParameter)
}
return
return err
}
func CreateKeyPair(input *ec2.CreateKeyPairInput, natsConn *nats.Conn, accountID string) (output ec2.CreateKeyPairOutput, err error) {
// Validate input
err = ValidateCreateKeyPairInput(input)
+1 -2
View File
@@ -20,11 +20,10 @@ func ValidateDeleteKeyPairInput(input *ec2.DeleteKeyPairInput) (err error) {
return errors.New(awserrors.ErrorMissingParameter)
}
return
return err
}
func DeleteKeyPair(input *ec2.DeleteKeyPairInput, natsConn *nats.Conn, accountID string) (output ec2.DeleteKeyPairOutput, err error) {
// Validate input
err = ValidateDeleteKeyPairInput(input)
+1 -1
View File
@@ -22,7 +22,7 @@ func ValidateImportKeyPairInput(input *ec2.ImportKeyPairInput) (err error) {
return errors.New(awserrors.ErrorMissingParameter)
}
return
return err
}
func ImportKeyPair(input *ec2.ImportKeyPairInput, natsConn *nats.Conn, accountID string) (output ec2.ImportKeyPairOutput, err error) {
@@ -0,0 +1,33 @@
package gateway_ec2_placementgroup
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/nats-io/nats.go"
)
// CreatePlacementGroup handles the EC2 CreatePlacementGroup API call.
func CreatePlacementGroup(input *ec2.CreatePlacementGroupInput, natsConn *nats.Conn, accountID string) (ec2.CreatePlacementGroupOutput, error) {
var output ec2.CreatePlacementGroupOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupName == nil || *input.GroupName == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
if input.Strategy == nil || *input.Strategy == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
result, err := svc.CreatePlacementGroup(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,30 @@
package gateway_ec2_placementgroup
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/nats-io/nats.go"
)
// DeletePlacementGroup handles the EC2 DeletePlacementGroup API call.
func DeletePlacementGroup(input *ec2.DeletePlacementGroupInput, natsConn *nats.Conn, accountID string) (ec2.DeletePlacementGroupOutput, error) {
var output ec2.DeletePlacementGroupOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupName == nil || *input.GroupName == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
result, err := svc.DeletePlacementGroup(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,20 @@
package gateway_ec2_placementgroup
import (
"github.com/aws/aws-sdk-go/service/ec2"
handlers_ec2_placementgroup "github.com/mulgadc/spinifex/spinifex/handlers/ec2/placementgroup"
"github.com/nats-io/nats.go"
)
// DescribePlacementGroups handles the EC2 DescribePlacementGroups API call.
func DescribePlacementGroups(input *ec2.DescribePlacementGroupsInput, natsConn *nats.Conn, accountID string) (ec2.DescribePlacementGroupsOutput, error) {
var output ec2.DescribePlacementGroupsOutput
svc := handlers_ec2_placementgroup.NewNATSPlacementGroupService(natsConn)
result, err := svc.DescribePlacementGroups(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
+1 -1
View File
@@ -39,7 +39,7 @@ func DetachVolume(input *ec2.DetachVolumeInput, natsConn *nats.Conn, accountID s
volumeID := *input.VolumeId
// Resolve InstanceId if not provided by looking up volume metadata
instanceID := ""
var instanceID string
if input.InstanceId != nil && *input.InstanceId != "" {
instanceID = *input.InstanceId
} else {
@@ -0,0 +1,119 @@
package gateway_ec2_vpc
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_vpc "github.com/mulgadc/spinifex/spinifex/handlers/ec2/vpc"
"github.com/nats-io/nats.go"
)
func CreateSecurityGroup(input *ec2.CreateSecurityGroupInput, natsConn *nats.Conn, accountID string) (ec2.CreateSecurityGroupOutput, error) {
var output ec2.CreateSecurityGroupOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupName == nil || *input.GroupName == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.CreateSecurityGroup(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func DeleteSecurityGroup(input *ec2.DeleteSecurityGroupInput, natsConn *nats.Conn, accountID string) (ec2.DeleteSecurityGroupOutput, error) {
var output ec2.DeleteSecurityGroupOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupId == nil || *input.GroupId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.DeleteSecurityGroup(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func DescribeSecurityGroups(input *ec2.DescribeSecurityGroupsInput, natsConn *nats.Conn, accountID string) (ec2.DescribeSecurityGroupsOutput, error) {
var output ec2.DescribeSecurityGroupsOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.DescribeSecurityGroups(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func AuthorizeSecurityGroupIngress(input *ec2.AuthorizeSecurityGroupIngressInput, natsConn *nats.Conn, accountID string) (ec2.AuthorizeSecurityGroupIngressOutput, error) {
var output ec2.AuthorizeSecurityGroupIngressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupId == nil || *input.GroupId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.AuthorizeSecurityGroupIngress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func AuthorizeSecurityGroupEgress(input *ec2.AuthorizeSecurityGroupEgressInput, natsConn *nats.Conn, accountID string) (ec2.AuthorizeSecurityGroupEgressOutput, error) {
var output ec2.AuthorizeSecurityGroupEgressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupId == nil || *input.GroupId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.AuthorizeSecurityGroupEgress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func RevokeSecurityGroupIngress(input *ec2.RevokeSecurityGroupIngressInput, natsConn *nats.Conn, accountID string) (ec2.RevokeSecurityGroupIngressOutput, error) {
var output ec2.RevokeSecurityGroupIngressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupId == nil || *input.GroupId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.RevokeSecurityGroupIngress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
func RevokeSecurityGroupEgress(input *ec2.RevokeSecurityGroupEgressInput, natsConn *nats.Conn, accountID string) (ec2.RevokeSecurityGroupEgressOutput, error) {
var output ec2.RevokeSecurityGroupEgressOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.GroupId == nil || *input.GroupId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.RevokeSecurityGroupEgress(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -0,0 +1,30 @@
package gateway_ec2_vpc
import (
"errors"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_vpc "github.com/mulgadc/spinifex/spinifex/handlers/ec2/vpc"
"github.com/nats-io/nats.go"
)
// ModifySubnetAttribute handles the EC2 ModifySubnetAttribute API call
func ModifySubnetAttribute(input *ec2.ModifySubnetAttributeInput, natsConn *nats.Conn, accountID string) (ec2.ModifySubnetAttributeOutput, error) {
var output ec2.ModifySubnetAttributeOutput
if input == nil {
return output, errors.New(awserrors.ErrorInvalidParameterValue)
}
if input.SubnetId == nil || *input.SubnetId == "" {
return output, errors.New(awserrors.ErrorMissingParameter)
}
svc := handlers_ec2_vpc.NewNATSVPCService(natsConn)
result, err := svc.ModifySubnetAttribute(input, accountID)
if err != nil {
return output, err
}
return *result, nil
}
@@ -6,7 +6,6 @@ import (
)
func DescribeAvailabilityZones(input *ec2.DescribeAvailabilityZonesInput, region string, az string) (output *ec2.DescribeAvailabilityZonesOutput, err error) {
output = &ec2.DescribeAvailabilityZonesOutput{
AvailabilityZones: []*ec2.AvailabilityZone{
{
@@ -27,7 +26,6 @@ func DescribeAvailabilityZones(input *ec2.DescribeAvailabilityZonesInput, region
}
func DescribeRegions(input *ec2.DescribeRegionsInput, region string) (output *ec2.DescribeRegionsOutput, err error) {
output = &ec2.DescribeRegionsOutput{
Regions: []*ec2.Region{
{
+3 -7
View File
@@ -68,7 +68,6 @@ type ErrorDetail struct {
}
func (gw *GatewayConfig) SetupRoutes() http.Handler {
var logLevel slog.Level
if gw.Debug {
@@ -159,7 +158,7 @@ func corsMiddleware(next http.Handler) http.Handler {
w.Header().Set("Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,HEAD,OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "*")
w.Header().Set("Access-Control-Allow-Credentials", "true")
if r.Method == "OPTIONS" {
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
@@ -173,7 +172,6 @@ func corsMiddleware(next http.Handler) http.Handler {
// AWS_ENDPOINT_URL=https://localhost:9999/ aws --no-verify-ssl ec2 describe-instances
func (gw *GatewayConfig) Request(w http.ResponseWriter, r *http.Request) {
// Route the request to the appropriate endpoint (e.g EC2, IAM, etc)
svc, err := gw.GetService(r)
slog.Info("Request", "service", svc, "method", r.Method, "path", r.URL.Path)
@@ -291,7 +289,7 @@ func (gw *GatewayConfig) ErrorHandler(w http.ResponseWriter, r *http.Request, er
// Get the request ID
var requestId = uuid.NewString()
if rid := r.Header.Get("x-amz-request-id"); rid != "" {
if rid := r.Header.Get("X-Amz-Request-Id"); rid != "" {
requestId = rid
}
@@ -346,7 +344,6 @@ func ParseAWSQueryArgs(query string) map[string]string {
}
func GenerateEC2ErrorResponse(code, message, requestID string) (output []byte) {
errorXml := ErrorResponse{
Errors: Errors{
Error: ErrorDetail{
@@ -405,7 +402,6 @@ func GenerateIAMErrorResponse(code, message, requestID string) (output []byte) {
}
func ParseArgsToStruct(input *any, args map[string]string) (err error) {
// Generated from input shape: RunInstancesRequest
err = awsec2query.QueryParamsToStruct(args, input)
@@ -414,7 +410,6 @@ func ParseArgsToStruct(input *any, args map[string]string) (err error) {
}
return nil
}
// DiscoverActiveNodes discovers the number of active spinifex daemon nodes in the cluster
@@ -487,6 +482,7 @@ func (gw *GatewayConfig) DiscoverActiveNodes() int {
// statusWriter wraps http.ResponseWriter to capture the status code.
type statusWriter struct {
http.ResponseWriter
status int
}
+40 -35
View File
@@ -175,7 +175,7 @@ func TestErrorHandler_IAMService(t *testing.T) {
gw.ErrorHandler(w, r, errors.New(awserrors.ErrorIAMNoSuchEntity))
})
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
resp := doRequest(handler, req)
assert.Equal(t, 404, resp.StatusCode)
@@ -197,7 +197,7 @@ func TestErrorHandler_UnknownError(t *testing.T) {
gw.ErrorHandler(w, r, errors.New("SomeCompletelyBogusError"))
})
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
resp := doRequest(handler, req)
assert.Equal(t, 500, resp.StatusCode)
@@ -218,7 +218,7 @@ func TestErrorHandler_EC2Service(t *testing.T) {
gw.ErrorHandler(w, r, errors.New(awserrors.ErrorInvalidParameterValue))
})
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
resp := doRequest(handler, req)
assert.Equal(t, 400, resp.StatusCode)
@@ -240,8 +240,8 @@ func TestErrorHandler_UsesRequestIDHeader(t *testing.T) {
gw.ErrorHandler(w, r, errors.New(awserrors.ErrorInternalError))
})
req := httptest.NewRequest("POST", "/", nil)
req.Header.Set("x-amz-request-id", "custom-req-id-123")
req := httptest.NewRequest(http.MethodPost, "/", nil)
req.Header.Set("X-Amz-Request-Id", "custom-req-id-123")
resp := doRequest(handler, req)
body, _ := io.ReadAll(resp.Body)
@@ -257,7 +257,7 @@ func TestErrorHandler_ContentTypeXML(t *testing.T) {
gw.ErrorHandler(w, r, errors.New(awserrors.ErrorInternalError))
})
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
resp := doRequest(handler, req)
assert.Equal(t, "application/xml", resp.Header.Get("Content-Type"))
}
@@ -493,7 +493,7 @@ func TestGetService(t *testing.T) {
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
if tc.ctxVal != nil {
ctx := context.WithValue(req.Context(), ctxService, tc.ctxVal)
req = req.WithContext(ctx)
@@ -514,7 +514,7 @@ func TestGetService(t *testing.T) {
func TestRequest_NoServiceContext(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
w := httptest.NewRecorder()
gw.Request(w, req)
@@ -527,7 +527,7 @@ func TestRequest_NoServiceContext(t *testing.T) {
func TestRequest_UnsupportedService(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxService, "s3")
req = req.WithContext(ctx)
w := httptest.NewRecorder()
@@ -542,7 +542,7 @@ func TestRequest_UnsupportedService(t *testing.T) {
func TestRequest_EC2MissingAction(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true}
req := httptest.NewRequest("POST", "/", strings.NewReader(""))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(""))
ctx := context.WithValue(req.Context(), ctxService, "ec2")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -558,7 +558,7 @@ func TestRequest_EC2MissingAction(t *testing.T) {
func TestRequest_IAMNilService(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: nil}
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser&UserName=test"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=test"))
ctx := context.WithValue(req.Context(), ctxService, "iam")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -574,7 +574,7 @@ func TestRequest_IAMNilService(t *testing.T) {
func TestRequest_AccountReturns200(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxService, "account")
req = req.WithContext(ctx)
w := httptest.NewRecorder()
@@ -587,7 +587,7 @@ func TestRequest_AccountReturns200(t *testing.T) {
// setupEC2Request creates an http.Request with EC2 service context and optional account ID.
func setupEC2Request(body string, accountID string) *http.Request {
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(body))
ctx := context.WithValue(req.Context(), ctxService, "ec2")
if accountID != "" {
ctx = context.WithValue(ctx, ctxAccountID, accountID)
@@ -697,7 +697,7 @@ func TestEC2Request_DescribeAvailabilityZones(t *testing.T) {
func TestCheckPolicy_NilIAMService(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: nil}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
err := gw.checkPolicy(req, "ec2", "DescribeInstances")
assert.NoError(t, err)
@@ -705,7 +705,7 @@ func TestCheckPolicy_NilIAMService(t *testing.T) {
func TestCheckPolicy_NoIdentityInContext(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: &mockIAMService{}}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
err := gw.checkPolicy(req, "ec2", "DescribeInstances")
assert.NoError(t, err)
@@ -713,7 +713,7 @@ func TestCheckPolicy_NoIdentityInContext(t *testing.T) {
func TestCheckPolicy_RootUserGlobalAccount(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: &mockIAMService{}}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "root")
ctx = context.WithValue(ctx, ctxAccountID, "000000000000") // GlobalAccountID
req = req.WithContext(ctx)
@@ -736,7 +736,7 @@ func TestCheckPolicy_NonRootAllowPolicy(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -759,7 +759,7 @@ func TestCheckPolicy_NonRootDenyPolicy(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -776,7 +776,7 @@ func TestCheckPolicy_NonRootNoPolicies(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -793,7 +793,7 @@ func TestCheckPolicy_GetUserPoliciesError(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -805,7 +805,7 @@ func TestCheckPolicy_GetUserPoliciesError(t *testing.T) {
func TestCheckPolicy_EmptyIdentity(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: &mockIAMService{}}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -816,7 +816,7 @@ func TestCheckPolicy_EmptyIdentity(t *testing.T) {
func TestCheckPolicy_MissingAccountID(t *testing.T) {
gw := &GatewayConfig{DisableLogging: true, IAMService: &mockIAMService{}}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
// No account ID
req = req.WithContext(ctx)
@@ -835,7 +835,7 @@ func TestCheckPolicy_NATSTransientRetriesAllAttempts(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -865,7 +865,7 @@ func TestCheckPolicy_NATSTransientRetriesThenSucceeds(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -882,7 +882,7 @@ func TestCheckPolicy_NonTransientErrorStillFails(t *testing.T) {
},
}
gw := &GatewayConfig{DisableLogging: true, IAMService: mock}
req := httptest.NewRequest("POST", "/", nil)
req := httptest.NewRequest(http.MethodPost, "/", nil)
ctx := context.WithValue(req.Context(), ctxIdentity, "alice")
ctx = context.WithValue(ctx, ctxAccountID, "123456789012")
req = req.WithContext(ctx)
@@ -904,11 +904,11 @@ func TestCorsMiddleware_GETRequest(t *testing.T) {
nextCalled := false
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
nextCalled = true
w.WriteHeader(200)
w.WriteHeader(http.StatusOK)
})
handler := corsMiddleware(next)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -926,7 +926,7 @@ func TestCorsMiddleware_OPTIONSRequest(t *testing.T) {
})
handler := corsMiddleware(next)
req := httptest.NewRequest("OPTIONS", "/", nil)
req := httptest.NewRequest(http.MethodOptions, "/", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -937,7 +937,7 @@ func TestCorsMiddleware_OPTIONSRequest(t *testing.T) {
func TestCorsMiddleware_LocalIPOrigin(t *testing.T) {
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
w.WriteHeader(http.StatusOK)
})
handler := corsMiddleware(next)
@@ -948,7 +948,7 @@ func TestCorsMiddleware_LocalIPOrigin(t *testing.T) {
if origin == "https://localhost:3000" {
continue
}
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Header.Set("Origin", origin)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -959,11 +959,11 @@ func TestCorsMiddleware_LocalIPOrigin(t *testing.T) {
func TestCorsMiddleware_UnknownOrigin(t *testing.T) {
next := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
w.WriteHeader(http.StatusOK)
})
handler := corsMiddleware(next)
req := httptest.NewRequest("GET", "/", nil)
req := httptest.NewRequest(http.MethodGet, "/", nil)
req.Header.Set("Origin", "https://evil.example.com")
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -979,7 +979,7 @@ func TestSlogRequestLogger_CallsNext(t *testing.T) {
})
handler := slogRequestLogger(next)
req := httptest.NewRequest("GET", "/test", nil)
req := httptest.NewRequest(http.MethodGet, "/test", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -993,7 +993,7 @@ func TestSlogRequestLogger_CapturesStatusCode(t *testing.T) {
})
handler := slogRequestLogger(next)
req := httptest.NewRequest("GET", "/missing", nil)
req := httptest.NewRequest(http.MethodGet, "/missing", nil)
w := httptest.NewRecorder()
handler.ServeHTTP(w, req)
@@ -1020,9 +1020,14 @@ func TestEC2ActionMapCompleteness(t *testing.T) {
"DescribeInternetGateways", "AttachInternetGateway", "DetachInternetGateway",
"CreateEgressOnlyInternetGateway", "DeleteEgressOnlyInternetGateway",
"DescribeEgressOnlyInternetGateways",
"CreatePlacementGroup", "DeletePlacementGroup", "DescribePlacementGroups",
"CreateVpc", "DeleteVpc", "DescribeVpcs",
"CreateSubnet", "DeleteSubnet", "DescribeSubnets",
"CreateSubnet", "DeleteSubnet", "DescribeSubnets", "ModifySubnetAttribute",
"CreateNetworkInterface", "DeleteNetworkInterface", "DescribeNetworkInterfaces",
"CreateSecurityGroup", "DeleteSecurityGroup", "DescribeSecurityGroups",
"AuthorizeSecurityGroupIngress", "AuthorizeSecurityGroupEgress",
"RevokeSecurityGroupIngress", "RevokeSecurityGroupEgress",
"AllocateAddress", "ReleaseAddress", "AssociateAddress", "DisassociateAddress", "DescribeAddresses",
}
for _, action := range expectedActions {
+8 -8
View File
@@ -167,7 +167,7 @@ func TestIAMRequest_CreateUser_Success(t *testing.T) {
}
handler := setupIAMRequestHandler(svc)
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -193,7 +193,7 @@ func TestIAMRequest_ListUsers_Success(t *testing.T) {
}
handler := setupIAMRequestHandler(svc)
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=ListUsers"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=ListUsers"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -209,7 +209,7 @@ func TestIAMRequest_ListUsers_Success(t *testing.T) {
func TestIAMRequest_UnknownAction(t *testing.T) {
handler := setupIAMRequestHandler(&flexMockIAMService{})
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=DoesNotExist"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=DoesNotExist"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -222,7 +222,7 @@ func TestIAMRequest_UnknownAction(t *testing.T) {
func TestIAMRequest_EmptyAction(t *testing.T) {
handler := setupIAMRequestHandler(&flexMockIAMService{})
req := httptest.NewRequest("POST", "/", strings.NewReader("UserName=alice"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("UserName=alice"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -247,7 +247,7 @@ func TestIAMRequest_NilService(t *testing.T) {
}
})
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -265,7 +265,7 @@ func TestIAMRequest_ServiceError(t *testing.T) {
}
handler := setupIAMRequestHandler(svc)
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -291,7 +291,7 @@ func TestIAMRequest_MissingAccountID(t *testing.T) {
}
})
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser&UserName=alice"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
@@ -305,7 +305,7 @@ func TestIAMRequest_ValidationError(t *testing.T) {
// CreateUser with missing UserName should return MissingParameter
handler := setupIAMRequestHandler(&flexMockIAMService{})
req := httptest.NewRequest("POST", "/", strings.NewReader("Action=CreateUser"))
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader("Action=CreateUser"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp := doRequest(handler, req)
+1 -1
View File
@@ -31,7 +31,7 @@ func EvaluateAccess(identity, action, resource string, policies []handlers_iam.P
hasAllow := false
for i := range policies {
for j := range policies[i].Statement {
stmt := &policies[i].Statement[j]
stmt := &policies[i].Statement[j] //nolint:gosec // G602 false positive: j is bounded by range
if !matchesAny(stmt.Action, action) {
continue
@@ -14,9 +14,10 @@ import (
)
const (
KVBucketAccountSettings = "spinifex-ec2-account-settings"
KeyEbsEncryptionDefault = "ebs-encryption-default"
KeySerialConsoleAccess = "serial-console-access"
KVBucketAccountSettings = "spinifex-ec2-account-settings"
KVBucketAccountSettingsVersion = 1
KeyEbsEncryptionDefault = "ebs-encryption-default"
KeySerialConsoleAccess = "serial-console-access"
)
// AccountSettingsRecord represents stored account settings
@@ -43,6 +44,9 @@ func NewAccountSettingsServiceImplWithNATS(cfg *config.Config, natsConn *nats.Co
if err != nil {
return nil, fmt.Errorf("failed to create account settings KV bucket: %w", err)
}
if err := utils.WriteVersion(settingsKV, KVBucketAccountSettingsVersion); err != nil {
return nil, fmt.Errorf("write version to %s: %w", KVBucketAccountSettings, err)
}
slog.Info("Account settings service initialized with JetStream KV", "bucket", KVBucketAccountSettings)
+10 -1
View File
@@ -20,7 +20,10 @@ import (
// Ensure EgressOnlyIGWServiceImpl implements EgressOnlyIGWService
var _ EgressOnlyIGWService = (*EgressOnlyIGWServiceImpl)(nil)
const KVBucketEgressOnlyIGW = "spinifex-eigw"
const (
KVBucketEgressOnlyIGW = "spinifex-eigw"
KVBucketEgressOnlyIGWVersion = 1
)
// EgressOnlyIGWRecord represents a stored Egress-only Internet Gateway
type EgressOnlyIGWRecord struct {
@@ -49,6 +52,9 @@ func NewEgressOnlyIGWServiceImplWithNATS(cfg *config.Config, natsConn *nats.Conn
if err != nil {
return nil, fmt.Errorf("failed to create KV bucket %s: %w", KVBucketEgressOnlyIGW, err)
}
if err := utils.WriteVersion(eigwKV, KVBucketEgressOnlyIGWVersion); err != nil {
return nil, fmt.Errorf("write version to %s: %w", KVBucketEgressOnlyIGW, err)
}
// Get or create VPC KV bucket for cross-resource ownership validation
vpcKV, err := utils.GetOrCreateKVBucket(js, handlers_ec2_vpc.KVBucketVPCs, 10)
@@ -161,6 +167,9 @@ func (s *EgressOnlyIGWServiceImpl) DescribeEgressOnlyInternetGateways(input *ec2
}
for _, key := range keys {
if key == utils.VersionKey {
continue
}
if !strings.HasPrefix(key, prefix) {
continue
}
+12
View File
@@ -0,0 +1,12 @@
package handlers_ec2_eip
import "github.com/aws/aws-sdk-go/service/ec2"
// EIPService defines the interface for Elastic IP address operations.
type EIPService interface {
AllocateAddress(input *ec2.AllocateAddressInput, accountID string) (*ec2.AllocateAddressOutput, error)
ReleaseAddress(input *ec2.ReleaseAddressInput, accountID string) (*ec2.ReleaseAddressOutput, error)
AssociateAddress(input *ec2.AssociateAddressInput, accountID string) (*ec2.AssociateAddressOutput, error)
DisassociateAddress(input *ec2.DisassociateAddressInput, accountID string) (*ec2.DisassociateAddressOutput, error)
DescribeAddresses(input *ec2.DescribeAddressesInput, accountID string) (*ec2.DescribeAddressesOutput, error)
}
+519
View File
@@ -0,0 +1,519 @@
package handlers_ec2_eip
import (
"encoding/json"
"errors"
"fmt"
"log/slog"
"strings"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/mulgadc/spinifex/spinifex/awserrors"
handlers_ec2_vpc "github.com/mulgadc/spinifex/spinifex/handlers/ec2/vpc"
"github.com/mulgadc/spinifex/spinifex/utils"
"github.com/nats-io/nats.go"
)
// Ensure EIPServiceImpl implements EIPService.
var _ EIPService = (*EIPServiceImpl)(nil)
// EIPServiceImpl implements Elastic IP operations with NATS JetStream persistence.
type EIPServiceImpl struct {
eipKV nats.KeyValue
externalIPAM *handlers_ec2_vpc.ExternalIPAM
vpcService handlers_ec2_vpc.VPCService
natsConn *nats.Conn
}
// natEvent is the payload published to vpc.add-nat / vpc.delete-nat topics.
type natEvent struct {
VpcId string `json:"vpc_id"`
ExternalIP string `json:"external_ip"`
LogicalIP string `json:"logical_ip"`
PortName string `json:"port_name"`
MAC string `json:"mac"`
}
// NewEIPServiceImpl creates a new EIP service backed by NATS JetStream KV.
func NewEIPServiceImpl(natsConn *nats.Conn, externalIPAM *handlers_ec2_vpc.ExternalIPAM, vpcService handlers_ec2_vpc.VPCService) (*EIPServiceImpl, error) {
js, err := natsConn.JetStream()
if err != nil {
return nil, fmt.Errorf("failed to get JetStream context: %w", err)
}
eipKV, err := utils.GetOrCreateKVBucket(js, KVBucketEIPs, 10)
if err != nil {
return nil, fmt.Errorf("failed to create KV bucket %s: %w", KVBucketEIPs, err)
}
if err := utils.WriteVersion(eipKV, KVBucketEIPsVersion); err != nil {
return nil, fmt.Errorf("write version to %s: %w", KVBucketEIPs, err)
}
slog.Info("EIP service initialized with JetStream KV", "bucket", KVBucketEIPs)
return &EIPServiceImpl{
eipKV: eipKV,
externalIPAM: externalIPAM,
vpcService: vpcService,
natsConn: natsConn,
}, nil
}
// AllocateAddress allocates a new Elastic IP from the external IPAM pool.
func (s *EIPServiceImpl) AllocateAddress(input *ec2.AllocateAddressInput, accountID string) (*ec2.AllocateAddressOutput, error) {
allocID := utils.GenerateResourceID("eipalloc")
var publicIP, poolName string
var err error
if input.PublicIpv4Pool != nil && *input.PublicIpv4Pool != "" {
// Allocate from a specific named pool.
poolName = *input.PublicIpv4Pool
publicIP, err = s.externalIPAM.AllocateFromPool(poolName, "elastic_ip", "", "")
if err != nil {
slog.Error("AllocateAddress: IPAM pool allocation failed", "pool", poolName, "err", err)
return nil, errors.New(awserrors.ErrorInsufficientAddressCapacity)
}
} else {
// Allocate from the best pool matching region/AZ (empty strings = global fallback).
region := ""
az := ""
publicIP, poolName, err = s.externalIPAM.AllocateIP(region, az, "elastic_ip", "", "")
if err != nil {
slog.Error("AllocateAddress: IPAM allocation failed", "err", err)
return nil, errors.New(awserrors.ErrorInsufficientAddressCapacity)
}
}
record := EIPRecord{
AllocationId: allocID,
PublicIp: publicIP,
PoolName: poolName,
State: "allocated",
Tags: make(map[string]string),
CreatedAt: time.Now(),
}
// Store tags from input.
for _, tagSpec := range input.TagSpecifications {
if tagSpec.ResourceType != nil && *tagSpec.ResourceType == "elastic-ip" {
for _, tag := range tagSpec.Tags {
if tag.Key != nil && tag.Value != nil {
record.Tags[*tag.Key] = *tag.Value
}
}
}
}
data, err := json.Marshal(record)
if err != nil {
return nil, fmt.Errorf("failed to marshal EIP record: %w", err)
}
if _, err := s.eipKV.Put(utils.AccountKey(accountID, allocID), data); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
slog.Info("AllocateAddress completed", "allocationId", allocID, "publicIp", publicIP, "pool", poolName, "accountID", accountID)
return &ec2.AllocateAddressOutput{
AllocationId: aws.String(allocID),
PublicIp: aws.String(publicIP),
Domain: aws.String("vpc"),
PublicIpv4Pool: aws.String(poolName),
}, nil
}
// ReleaseAddress releases a previously allocated Elastic IP back to the IPAM pool.
func (s *EIPServiceImpl) ReleaseAddress(input *ec2.ReleaseAddressInput, accountID string) (*ec2.ReleaseAddressOutput, error) {
if input.AllocationId == nil || *input.AllocationId == "" {
return nil, errors.New(awserrors.ErrorMissingParameter)
}
allocID := *input.AllocationId
key := utils.AccountKey(accountID, allocID)
entry, err := s.eipKV.Get(key)
if err != nil {
return nil, errors.New(awserrors.ErrorInvalidAllocationIDNotFound)
}
var record EIPRecord
if err := json.Unmarshal(entry.Value(), &record); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
// Cannot release an EIP that is still associated.
if record.State == "associated" {
return nil, errors.New(awserrors.ErrorInvalidAddressLocked)
}
// Release IP back to IPAM pool.
if err := s.externalIPAM.ReleaseIP(record.PoolName, record.PublicIp); err != nil {
slog.Warn("Failed to release IP back to IPAM pool", "allocationId", allocID, "ip", record.PublicIp, "pool", record.PoolName, "err", err)
}
if err := s.eipKV.Delete(key); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
slog.Info("ReleaseAddress completed", "allocationId", allocID, "publicIp", record.PublicIp, "accountID", accountID)
return &ec2.ReleaseAddressOutput{}, nil
}
// AssociateAddress associates an Elastic IP with an ENI or instance.
func (s *EIPServiceImpl) AssociateAddress(input *ec2.AssociateAddressInput, accountID string) (*ec2.AssociateAddressOutput, error) {
if input.AllocationId == nil || *input.AllocationId == "" {
return nil, errors.New(awserrors.ErrorMissingParameter)
}
allocID := *input.AllocationId
key := utils.AccountKey(accountID, allocID)
entry, err := s.eipKV.Get(key)
if err != nil {
return nil, errors.New(awserrors.ErrorInvalidAllocationIDNotFound)
}
var record EIPRecord
if err := json.Unmarshal(entry.Value(), &record); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
// Resolve the target ENI. Either by direct NetworkInterfaceId or by InstanceId lookup.
var eniID, instanceID, privateIP, vpcID, macAddr string
if input.NetworkInterfaceId != nil && *input.NetworkInterfaceId != "" {
eniID = *input.NetworkInterfaceId
eni, err := s.lookupENI(accountID, eniID)
if err != nil {
return nil, err
}
instanceID = eni.InstanceId
privateIP = eni.PrivateIpAddress
vpcID = eni.VpcId
macAddr = eni.MacAddress
} else if input.InstanceId != nil && *input.InstanceId != "" {
instanceID = *input.InstanceId
eni, err := s.lookupENIByInstance(accountID, instanceID)
if err != nil {
return nil, err
}
eniID = eni.NetworkInterfaceId
privateIP = eni.PrivateIpAddress
vpcID = eni.VpcId
macAddr = eni.MacAddress
} else {
return nil, errors.New(awserrors.ErrorMissingParameter)
}
// Allow override of private IP if specified.
if input.PrivateIpAddress != nil && *input.PrivateIpAddress != "" {
privateIP = *input.PrivateIpAddress
}
associationID := utils.GenerateResourceID("eipassoc")
record.AssociationId = associationID
record.ENIId = eniID
record.InstanceId = instanceID
record.PrivateIp = privateIP
record.VpcId = vpcID
record.State = "associated"
data, err := json.Marshal(record)
if err != nil {
return nil, fmt.Errorf("failed to marshal EIP record: %w", err)
}
if _, err := s.eipKV.Update(key, data, entry.Revision()); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
// Publish vpc.add-nat event (fire-and-forget).
s.publishNATEvent("vpc.add-nat", vpcID, record.PublicIp, privateIP, eniID, macAddr)
slog.Info("AssociateAddress completed",
"allocationId", allocID,
"associationId", associationID,
"eniId", eniID,
"instanceId", instanceID,
"publicIp", record.PublicIp,
"privateIp", privateIP,
"accountID", accountID)
return &ec2.AssociateAddressOutput{
AssociationId: aws.String(associationID),
}, nil
}
// DisassociateAddress removes an Elastic IP association from an ENI.
func (s *EIPServiceImpl) DisassociateAddress(input *ec2.DisassociateAddressInput, accountID string) (*ec2.DisassociateAddressOutput, error) {
if input.AssociationId == nil || *input.AssociationId == "" {
return nil, errors.New(awserrors.ErrorMissingParameter)
}
associationID := *input.AssociationId
// Find the EIP record by association ID.
record, key, revision, err := s.findByAssociationID(accountID, associationID)
if err != nil {
return nil, err
}
// Publish vpc.delete-nat event before clearing association (fire-and-forget).
if record.ENIId != "" {
eni, lookupErr := s.lookupENI(accountID, record.ENIId)
macAddr := ""
if lookupErr == nil {
macAddr = eni.MacAddress
}
s.publishNATEvent("vpc.delete-nat", record.VpcId, record.PublicIp, record.PrivateIp, record.ENIId, macAddr)
}
// Clear association fields, revert to "allocated" state.
record.AssociationId = ""
record.ENIId = ""
record.InstanceId = ""
record.PrivateIp = ""
record.VpcId = ""
record.State = "allocated"
data, err := json.Marshal(record)
if err != nil {
return nil, fmt.Errorf("failed to marshal EIP record: %w", err)
}
if _, err := s.eipKV.Update(key, data, revision); err != nil {
return nil, errors.New(awserrors.ErrorServerInternal)
}
slog.Info("DisassociateAddress completed", "associationId", associationID, "accountID", accountID)
return &ec2.DisassociateAddressOutput{}, nil
}
// DescribeAddresses lists Elastic IPs with optional filtering by allocation ID.
func (s *EIPServiceImpl) DescribeAddresses(input *ec2.DescribeAddressesInput, accountID string) (*ec2.DescribeAddressesOutput, error) {
allocIDs := make(map[string]bool)
for _, id := range input.AllocationIds {
if id != nil {
allocIDs[*id] = true
}
}
publicIPs := make(map[string]bool)
for _, ip := range input.PublicIps {
if ip != nil {
publicIPs[*ip] = true
}
}
prefix := accountID + "."
keys, err := s.eipKV.Keys()
if err != nil && !errors.Is(err, nats.ErrNoKeysFound) {
return nil, errors.New(awserrors.ErrorServerInternal)
}
var addresses []*ec2.Address
for _, k := range keys {
if k == utils.VersionKey {
continue
}
if !strings.HasPrefix(k, prefix) {
continue
}
entry, err := s.eipKV.Get(k)
if err != nil {
slog.Warn("Failed to get EIP record", "key", k, "error", err)
continue
}
var record EIPRecord
if err := json.Unmarshal(entry.Value(), &record); err != nil {
slog.Warn("Failed to unmarshal EIP record", "key", k, "error", err)
continue
}
if len(allocIDs) > 0 && !allocIDs[record.AllocationId] {
continue
}
if len(publicIPs) > 0 && !publicIPs[record.PublicIp] {
continue
}
addresses = append(addresses, s.eipRecordToEC2(&record))
}
// If specific allocation IDs were requested but not found, return error.
if len(allocIDs) > 0 {
found := make(map[string]bool)
for _, addr := range addresses {
if addr.AllocationId != nil {
found[*addr.AllocationId] = true
}
}
for id := range allocIDs {
if !found[id] {
return nil, errors.New(awserrors.ErrorInvalidAllocationIDNotFound)
}
}
}
slog.Info("DescribeAddresses completed", "count", len(addresses), "accountID", accountID)
return &ec2.DescribeAddressesOutput{
Addresses: addresses,
}, nil
}
// lookupENI retrieves an ENI record by its ID using the VPC service.
func (s *EIPServiceImpl) lookupENI(accountID, eniID string) (*handlers_ec2_vpc.ENIRecord, error) {
output, err := s.vpcService.DescribeNetworkInterfaces(&ec2.DescribeNetworkInterfacesInput{
NetworkInterfaceIds: []*string{aws.String(eniID)},
}, accountID)
if err != nil {
return nil, err
}
if len(output.NetworkInterfaces) == 0 {
return nil, errors.New(awserrors.ErrorInvalidNetworkInterfaceIDNotFound)
}
ni := output.NetworkInterfaces[0]
record := &handlers_ec2_vpc.ENIRecord{
NetworkInterfaceId: aws.StringValue(ni.NetworkInterfaceId),
PrivateIpAddress: aws.StringValue(ni.PrivateIpAddress),
VpcId: aws.StringValue(ni.VpcId),
MacAddress: aws.StringValue(ni.MacAddress),
SubnetId: aws.StringValue(ni.SubnetId),
}
if ni.Attachment != nil && ni.Attachment.InstanceId != nil {
record.InstanceId = aws.StringValue(ni.Attachment.InstanceId)
}
return record, nil
}
// lookupENIByInstance finds the primary ENI for an instance.
func (s *EIPServiceImpl) lookupENIByInstance(accountID, instanceID string) (*handlers_ec2_vpc.ENIRecord, error) {
output, err := s.vpcService.DescribeNetworkInterfaces(&ec2.DescribeNetworkInterfacesInput{
Filters: []*ec2.Filter{
{
Name: aws.String("attachment.instance-id"),
Values: []*string{aws.String(instanceID)},
},
},
}, accountID)
if err != nil {
return nil, err
}
if len(output.NetworkInterfaces) == 0 {
return nil, errors.New(awserrors.ErrorInvalidInstanceIDNotFound)
}
// Use the first ENI (primary).
ni := output.NetworkInterfaces[0]
record := &handlers_ec2_vpc.ENIRecord{
NetworkInterfaceId: aws.StringValue(ni.NetworkInterfaceId),
PrivateIpAddress: aws.StringValue(ni.PrivateIpAddress),
VpcId: aws.StringValue(ni.VpcId),
MacAddress: aws.StringValue(ni.MacAddress),
SubnetId: aws.StringValue(ni.SubnetId),
}
if ni.Attachment != nil && ni.Attachment.InstanceId != nil {
record.InstanceId = aws.StringValue(ni.Attachment.InstanceId)
}
return record, nil
}
// findByAssociationID scans EIP records to find one matching the given association ID.
func (s *EIPServiceImpl) findByAssociationID(accountID, associationID string) (*EIPRecord, string, uint64, error) {
prefix := accountID + "."
keys, err := s.eipKV.Keys()
if err != nil {
return nil, "", 0, errors.New(awserrors.ErrorServerInternal)
}
for _, k := range keys {
if k == utils.VersionKey {
continue
}
if !strings.HasPrefix(k, prefix) {
continue
}
entry, err := s.eipKV.Get(k)
if err != nil {
continue
}
var record EIPRecord
if err := json.Unmarshal(entry.Value(), &record); err != nil {
continue
}
if record.AssociationId == associationID {
return &record, k, entry.Revision(), nil
}
}
return nil, "", 0, errors.New(awserrors.ErrorInvalidAssociationIDNotFound)
}
// publishNATEvent publishes a NAT lifecycle event to NATS for vpcd consumption.
// This is fire-and-forget; errors are logged but do not fail the API response.
func (s *EIPServiceImpl) publishNATEvent(topic, vpcID, externalIP, logicalIP, eniID, mac string) {
if s.natsConn == nil {
return
}
evt := natEvent{
VpcId: vpcID,
ExternalIP: externalIP,
LogicalIP: logicalIP,
PortName: eniID,
MAC: mac,
}
data, err := json.Marshal(evt)
if err != nil {
slog.Error("Failed to marshal NAT event", "topic", topic, "err", err)
return
}
if err := s.natsConn.Publish(topic, data); err != nil {
slog.Error("Failed to publish NAT event", "topic", topic, "err", err)
}
}
// eipRecordToEC2 converts an EIPRecord to an EC2 Address.
func (s *EIPServiceImpl) eipRecordToEC2(record *EIPRecord) *ec2.Address {
addr := &ec2.Address{
AllocationId: aws.String(record.AllocationId),
PublicIp: aws.String(record.PublicIp),
Domain: aws.String("vpc"),
PublicIpv4Pool: aws.String(record.PoolName),
}
if record.AssociationId != "" {
addr.AssociationId = aws.String(record.AssociationId)
}
if record.ENIId != "" {
addr.NetworkInterfaceId = aws.String(record.ENIId)
}
if record.InstanceId != "" {
addr.InstanceId = aws.String(record.InstanceId)
}
if record.PrivateIp != "" {
addr.PrivateIpAddress = aws.String(record.PrivateIp)
}
if len(record.Tags) > 0 {
tags := make([]*ec2.Tag, 0, len(record.Tags))
for k, v := range record.Tags {
tags = append(tags, &ec2.Tag{Key: aws.String(k), Value: aws.String(v)})
}
addr.Tags = tags
}
return addr
}

Some files were not shown because too many files have changed in this diff Show More