spinifex/docs/COMMANDS.md

Command Implementation Matrix

Spinifex Admin CLI

Platform management commands not exposed via the AWS gateway API. These are CLI-only commands.

Service Management

Services run under systemd. Units live in spinifex/build/systemd/, installed by scripts/setup.sh on install and reconciled by spx admin upgrade.

Units

Every service is PartOf=spinifex.target and ordered by its own After=/Wants=.

Unit	ExecStart	Description
spinifex.target	—	The whole stack: every service below plus the watchdog timer, openvswitch and ovn-controller
spinifex-nats.service	spx service nats start	NATS message bus with JetStream. Config at /etc/spinifex/nats/nats.conf; dependants block on wait-for-nats.sh as an ExecStartPre
spinifex-predastore.service	spx service predastore start	S3-compatible object storage, serving that host's nodes and the S3 gate among them
spinifex-northstar.service	spx service northstar start	Authoritative DNS for *.spx3.net, recursive via upstream forwarders, syncing zones from its S3 bucket
spinifex-viperblock.service	spx service viperblock start	Block storage with the NBD plugin. KillMode=process: nbdkit serving a guest survives a restart of this unit
spinifex-daemon.service	spx service spinifex start	VM orchestration, NATS subscriptions, health endpoint. KillMode=process: guest QEMU survives a restart and the new daemon reattaches. SIGHUP reloads config
spinifex-awsgw.service	spx service awsgw start	AWS-compatible gateway: SigV4 auth, IAM policy enforcement, TLS. Also serves the Private Admin API
spinifex-vpcd.service	spx service vpcd start	VPC daemon: translates vpc.* NATS events to OVN logical switches, ports and routers
spinifex-ui.service	spx service spinifex-ui start	Web UI serving the React frontend
spinifex-qmp-collector.service	spx service qmp-collector start	Polls per-VM telemetry QMP sockets and tap counters, publishes to NATS metrics.ec2.*
spinifex-shutdown.service	/bin/true; the work is its ExecStop	Runs spx admin node drain --local --unless-restarting. After= every service, so it stops first and drains guests while the stack is still up
spinifex-nats-watchdog.service / .timer	nats-js-watchdog.sh	JetStream liveness probe every 30s. try-restarts NATS when JetStream stops accepting writes while the process stays alive — what Restart=on-failure cannot catch
spinifex-firewall.service	spinifex-firewall-apply	Oneshot nftables input policy, WantedBy=sysinit.target rather than the target. No ExecStop: remove it with nft delete table inet spinifex_filter
regenerate-ssh-host-keys.service	ssh-keygen -A	Oneshot for AMI-built nodes, conditional on the host key being absent

Operations

Command	Description
sudo systemctl start spinifex.target	Starts the stack in dependency order
sudo systemctl stop spinifex.target	Stops the stack, draining guests first via spinifex-shutdown.service
sudo systemctl restart spinifex.target	Restarts the stack. Does not drain: QEMU and nbdkit survive on KillMode=process and the daemon reattaches
sudo systemctl restart spinifex-daemon	Restarts one service. Required after a binary swap — services keep executing the replaced inode until they restart (spx admin preflight reports this)
systemctl status 'spinifex-*'	Per-service state, PID, memory and recent log lines
journalctl -u spinifex-daemon -f	Follows one service; -u 'spinifex-*' follows all
sudo systemctl kill -s HUP spinifex-daemon	Reloads daemon config without a restart
sudo systemctl edit spinifex-daemon	Drop-in override. Preferred over editing the installed unit, which spx admin upgrade then reports as operator-modified and refuses to touch

A cluster-wide stop is spx admin cluster shutdown, not a systemctl loop across nodes — it sequences GATE → DRAIN → STORAGE → PERSIST → INFRA with a per-phase ACK from every node.

Service configuration

Under systemd the services take no flags: each unit supplies Environment= entries, and the flags below are reachable as their SPINIFEX_* equivalents. Precedence is flag, then env var, then the node's spinifex.toml section, then the default. Environment files are /etc/spinifex/telemetry.env (all services), /etc/spinifex/systemd.env (viperblock's plugin path) and /etc/spinifex/host.env (the daemon's SPINIFEX_RESERVED_MEM_GB).

Service	Flag	Environment variable	Default / notes
predastore	--port	SPINIFEX_PREDASTORE_PORT	8443, the host's S3 gate port
predastore	--host	SPINIFEX_PREDASTORE_HOST	0.0.0.0, the address the host's sockets bind
predastore	--base-path	SPINIFEX_PREDASTORE_BASE_PATH	Directory holding the service pid file; the unit sets SPINIFEX_BASE_DIR
predastore	--config-path	SPINIFEX_PREDASTORE_CONFIG_PATH	Required
predastore	--tls-cert / --tls-key	SPINIFEX_PREDASTORE_TLS_CERT / _TLS_KEY	Required
predastore	--encryption-key-file	SPINIFEX_PREDASTORE_ENCRYPTION_KEY_FILE	Required
predastore	--host-id	SPINIFEX_PREDASTORE_HOST_ID	Required; names a [[host]] in the predastore config. When SPINIFEX_CONFIG_PATH points at a cluster spinifex.toml, host/port/host-id default to that node's [nodes.<node>.predastore] section rather than the defaults above. host_id absent from spinifex.toml resolves to 0, which names no host and is rejected
viperblock	--s3-host	SPINIFEX_VIPERBLOCK_S3_HOST	0.0.0.0:8443
viperblock	--s3-bucket	SPINIFEX_VIPERBLOCK_S3_BUCKET	predastore
viperblock	--s3-region	SPINIFEX_VIPERBLOCK_S3_REGION	ap-southeast-2
viperblock	--plugin-path	SPINIFEX_VIPERBLOCK_PLUGIN_PATH	/opt/spinifex/lib/nbdkit-viperblock-plugin.so; set in /etc/spinifex/systemd.env
viperblock	--encryption-key-file	SPINIFEX_VIPERBLOCK_ENCRYPTION_KEY_FILE	32-byte AES-256 master key for at-rest encryption; empty disables it
viperblock	--debug	SPINIFEX_VIPERBLOCK_DEBUG
nats	--port / --host / --debug	SPINIFEX_NATS_PORT / SPINIFEX_NATS_HOST / SPINIFEX_NATS_DEBUG	4222 / 0.0.0.0
nats	--data-dir / --jetstream	SPINIFEX_NATS_DATA_DIR / SPINIFEX_NATS_JETSTREAM	The unit sets neither: it points SPINIFEX_CONFIG_PATH at nats.conf, which carries both
spinifex (daemon)	--wal-dir	SPINIFEX_WAL_DIR	The unit sets it alongside SPINIFEX_BASE_DIR
awsgw	--host	SPINIFEX_AWSGW_HOST	0.0.0.0:9999
awsgw	--tls-cert / --tls-key	SPINIFEX_AWSGW_TLS_CERT / _TLS_KEY	/etc/spinifex/server.pem and server.key
awsgw	--debug	SPINIFEX_AWSGW_DEBUG
spinifex-ui	--port / --host	SPINIFEX_UI_PORT / SPINIFEX_UI_HOST	3000 / 0.0.0.0. Command aliases: ui, spinifexui
spinifex-ui	--tls-cert / --tls-key	SPINIFEX_UI_TLS_CERT / SPINIFEX_UI_TLS_KEY	/etc/spinifex/server.pem and server.key
spinifex-ui	--base-dir	SPINIFEX_UI_BASE_DIR	/var/lib/spinifex/spinifex-ui/
northstar	--northstar-config	SPINIFEX_NORTHSTAR_CONFIG	Overrides nodes.<node>.northstar.config_path

Cluster Inspection

Operational commands for inspecting cluster state. These fan out NATS requests to all nodes and aggregate responses.

Command	Flags	Description
spx get nodes	--timeout (default: 3s)	Loads config → publishes to spinifex.node.status fan-out topic → collects responses within timeout → merges config-known nodes with NATS responders → nodes that don't respond shown as NotReady
spx get vms	--timeout (default: 3s)	Publishes to spinifex.node.vms fan-out topic → collects VM info from all nodes → sorts by node then instance ID → prints table. Alias: spx get instances
spx get dhcp-leases	--timeout (default: 3s), --orphans (show only leases whose owning resource is gone), --skip-owner-check (skip the owner lookup; faster, OWNER reads as unchecked)	Lists the upstream DHCP leases vpcd holds for source="dhcp" external pools, with the resource each was taken for and whether that resource still exists. An OWNER of gone is a leaked address — nothing renews it on purpose and nothing will release it, so it stays out of the upstream scope until its TTL expires. unknown means the owning service could not be reached, which is not the same finding. Alias: spx get leases

Resource Monitoring

Command	Flags	Description
spx top nodes	--timeout (default: 3s)	Publishes to spinifex.node.status fan-out topic → collects CPU/memory usage per node → aggregates instance type capacity across all nodes → prints two tables: per-node resource usage and cluster-wide instance type availability

Cluster Initialization

Command	Flags	Description
spx admin init	--nodes, --node, --bind, --advertise (external IP off-host clients dial; auto-detected from the default route), --port, --region, --az, --cluster-name, --cluster-bind, --cluster-routes, --predastore-nodes, --predastore-compaction-interval (seconds; test clusters set a short one), --services, --formation-timeout, --token-ttl, --force, --skip-host-dns, --email (operator address for update and security notices), --no-telemetry, --gpu-passthrough, --ipsec (default true; encrypt intra-AZ Geneve via OVN native IPsec cluster-wide — disable only for a trusted single-rack lab), --host (leader to join; falls back to multicast discovery), --external-mode (pool | nat routed NAT for non-bridgeable uplinks, single-node; pair with setup-ovn.sh --nat-uplink; note: system instances (ECS/EKS/load-balancer agents) are unsupported in nat v1 — they require pool mode), --external-iface (WAN NIC for br-external, auto-detected), --external-source (dhcp | static), --external-pool (start-end), --external-prefix-len (default 24), --external-gateway (auto-detected), --external-bind-bridge (default br-wan when --external-source=dhcp), --gateway-ip (OVN gateway router's external IP for SNAT; defaults to the pool's range start, required for nat mode without DHCP)	Generates root IAM credentials (AKIA-prefixed access key + secret) → creates master.key (AES-256, 32 bytes, 0600) → writes bootstrap.json (consumed on first start) → generates CA + server TLS certificates → generates join token (written to join-token file, displayed in join command) → creates NATS config with auth token → writes spinifex.toml, awsgw.toml, predastore.toml → configures AWS CLI spx profile → creates directory structure under ~/spinifex/ → points the host resolver at this node's local northstar so the Spinifex zones (ELB/EKS names) resolve from the node itself (opt out with --skip-host-dns). --force on an already-initialized node is idempotent for crypto: it preserves the existing keys (master.key, predastore/viperblock encryption keys), system + admin credentials, and CA, and refreshes only the config files and the CA-signed server certificate (so a changed bind IP / SANs is picked up without breaking already-joined nodes or CA-baked AMIs). A genuine clean slate requires removing the data dirs first.
spx admin join	--host (required), --node (required), --token (required), --bind, --advertise (defaults to --bind, or the auto-detected WAN IP when --bind is 0.0.0.0), --port, --region, --az, --cluster-bind, --cluster-routes, --data-dir, --services, --skip-host-dns, --email, --no-telemetry, --predastore-compaction-interval, --join-timeout (how long to keep retrying while the formation server is unreachable, default 20m), --force (join even though this node is already initialized, discarding its own CA and master key)	Connects to leader node with join token (Authorization: Bearer header) → retrieves cluster configuration → configures local node to join cluster and participate in distributed operations → points the host resolver at this node's local northstar so the Spinifex zones resolve from the node itself (opt out with --skip-host-dns)

Version

Command	Flags	Description
spx version	—	Prints Spinifex version, commit hash, OS, and architecture (populated via build-time ldflags)

Cluster Operations

Command	Flags	Description
spx admin cluster shutdown	--force (shutdown even if nodes don't respond), --timeout (max wait per phase, default 120s), --dry-run (print phase plan without executing)	Performs coordinated, phased shutdown of entire cluster. Phases execute in order: GATE (stop API/UI) → DRAIN (stop VMs) → STORAGE (stop viperblock) → PERSIST (stop predastore) → INFRA (stop NATS/daemon). Each phase waits for all nodes to ACK before proceeding. Uses JetStream state tracking.
spx admin cluster drain-dhcp	--timeout (reply-collection window, default 30s)	Asks each vpcd to DHCPRELEASE every external-pool DHCP lease it currently holds, returning them to the upstream DHCP server. Run on teardown before stopping services — an env reset otherwise strands held leases upstream until their TTL expires, eventually exhausting the upstream scope. Best-effort: warns and exits 0 if the cluster is already down.
spx admin node drain --local	--local (drain the local node only, required), --timeout (max wait per phase, default 120s), --unless-restarting (drain on a real host shutdown or a plain target stop, and skip only when a restart of the stack is already queued; unset, it always drains)	Runs the GATE and DRAIN phases against the local node only: powers down its guests via QMP and unmounts their volumes (flushing the viperblock WAL) while every service is still up. STORAGE/PERSIST/INFRA are left to systemd's ordered teardown. Wired as the ExecStop of spinifex-shutdown.service, so systemctl stop spinifex.target and host reboot/poweroff drain guests before any storage service stops.

Certificate Management

Command	Flags	Description
spx admin cert renew	--extra-ip (additional IPs for SANs), --extra-dns (additional DNS names for SANs)	Reads existing CA → regenerates server certificate with all current network interface IPs and machine hostname in SANs → writes new cert. Use after adding a new network interface or changing IP addresses.
spx admin cert create-tenant-ca	--domain (required, repeatable — permitted domains baked into the CA's name constraints), --regenerate (replace an existing tenant CA, requires confirmation), --yes (skip the --regenerate confirmation prompt)	Creates the tenant root CA that ACM's PRIVATE_CA validation mode signs leaf certificates from — a separate, independent root from the platform CA. Idempotent: running it again against an existing tenant CA reports the current state rather than regenerating; --regenerate is required to replace the root, which invalidates every device's existing trust.
spx admin cert force-renew	--arn (required, certificate ARN), --account (defaults to the bootstrap account)	Reissues an ACM certificate under its existing ARN immediately, instead of waiting for the renewal worker's proportional window. Only PRIVATE_CA (tenant-CA-issued) certificates can be force-renewed; imported and public ones are refused. Talks to the running daemon over NATS and takes the same per-certificate lease the scheduled worker uses, so it is safe against a live deployment. It exists mainly to verify the ELBv2 fan-out — that every load balancer referencing the certificate re-renders with the new leaf — without waiting out the real window.

Node Diagnostics

Command	Flags	Description
spx admin preflight	--quiet (print only non-OK rows)	Reports whether this host has the privileged helper scripts and sudoers grants the installed build depends on. spx version knows the build but not the host: setup.sh and setup-ovn.sh install those assets, and a binary-swap deploy (make deploy, or a manual copy) skips them. It also reports any running service still executing a replaced binary — overwriting /usr/local/bin/spx while spinifex.target is up does not move services onto it, so each keeps executing the replaced inode until it restarts and a node can serve a mixture of two builds indefinitely. Checks compare content, not existence, so a present-but-stale asset is caught. Exits non-zero if any asset is Missing, Stale or Ungranted.
spx admin node js-probe	--timeout (default: 10s)	Round-trips a canary key through the cluster-state KV bucket and exits non-zero if the write or the read-back fails. JetStream can latch a write error per message block and keep serving reads — nats-server's filestore records the failure on the block and returns it on every later write, and nothing clears it short of a restart. The HTTP health endpoint reports the server-wide JetStream state, which that latch never touches, so only an actual write observes it.
spx admin banner	--boot-check	Writes the node information banner to /etc/issue (shown before login on the physical/serial console) and /etc/motd (shown after SSH login). With --boot-check it also detects a management IP that has changed since last boot and updates /etc/spinifex/node.conf accordingly.

Storage Inspection

Command	Flags	Description
spx admin volumes orphans	—	Lists volumes the storage provider holds that the control plane has no record of. Their blocks consume space but they have no API handle, so they can be neither described nor deleted through the EC2 API. Reports only, never deletes: the evidence that an orphan is still wanted is the record that went missing, so removal is an operator decision taken per volume.
spx admin volumes unsealed	—	Lists volumes a node has open, or had open, without a confirmed seal to the backend — the node named holds writes predastore may not have. Starting such a volume elsewhere is allowed, and is preferred over leaving the instance unable to run, but it opens from the last checkpoint that did reach the backend. This is the list of volumes where that would cost something.

Upgrade Management

Command	Flags	Description
spx admin upgrade	--yes (apply without prompting), --dry-run (report pending config and unit changes without applying), --units-only (reconcile systemd units only, skip config migrations), --skip-units (apply config migrations only, skip unit reconciliation), --config-dir (persistent, default: ~/spinifex/config), --spinifex-dir (persistent, default: ~/spinifex)	Reads current config versions from registry → computes pending config migrations (from→to per target) → reads installed systemd unit versions and compares each against the copy embedded in the binary (systemd.Reconcile with DryRun: true) → prints both summaries → prompts for confirmation unless --yes or --dry-run → runs migrate.DefaultRegistry.RunAllConfig() to apply migrations to config files → runs systemd.Reconcile for real to install missing units and replace stale ones (after a timestamped backup), then systemctl daemon-reload. Units whose installed marker version matches the embedded one but whose content differs are left untouched and reported as operator-modified — never overwritten. Writing /etc/systemd/system requires root; without it, drift is still reported and the command exits non-zero rather than writing partially. Never restarts services — a unit change applies to that service's next stop. Intended for upgrades between Spinifex versions, including nodes that were upgraded by swapping the spx binary directly rather than by re-running the installer. Operators can skip migrations during install by setting INSTALL_SPINIFEX_SKIP_MIGRATE=1, then run spx admin upgrade manually to review and apply. After completion, services must be restarted with sudo systemctl restart spinifex.target to pick up config changes. Invoked non-interactively by scripts/setup.sh with --yes.

Account Management

Command	Flags	Description
spx admin account create	--name (required)	Connects to NATS → refuses if an account already holds the name (case- and whitespace-insensitive) → reserves the name in spinifex-account-names KV → CAS loop on spinifex-account-counter:next_id for sequential 12-digit ID → creates Account record in spinifex-accounts KV → creates default VPC → creates admin user, AdministratorAccess policy (Action:, Resource:) and one access key → prints credentials and writes a spinifex-<name> AWS CLI profile
spx admin account create --remote	--name (required), --endpoint (default: this node's AWS gateway), --region (default: this node's region), --ca-bundle (default: this node's CA), --client-token (default: generated), --source (default: spx-cli)	Same account, created through POST /admin/CreateAccount over SigV4 instead of NATS — the path the public signup form uses, so a shell can exercise it. Credentials come from the standard AWS chain (env vars or AWS_PROFILE), never from the cluster master key; the caller must be an IAM user in the super-admin account holding spinifex:CreateAccount. Reuse the printed --client-token to retry: a fresh token would create a second account.
spx admin account list	—	Connects to NATS → IAMService.ListAccounts() → prints table with Account ID, Name, Status, Created
spx admin account list --remote	--endpoint (default: this node's AWS gateway), --region (default: this node's region), --ca-bundle (default: this node's CA)	Same listing read through POST /admin/ListAccounts over SigV4 instead of NATS, so an off-cluster operator or the load-test harness sees its tenants without SSH access to a node. Credentials come from the standard AWS chain; the caller must be an IAM user in the super-admin account holding spinifex:ListAccounts.
spx admin account quota get <account-id>	--endpoint (default: this node's AWS gateway), --region (default: this node's region), --ca-bundle (default: this node's CA)	Reads the limits in force for one account through POST /admin/GetAccountQuota over SigV4, and reports for each dimension whether it came from the [quota] block in awsgw.toml (config) or from an override stored against the account (override). Credentials come from the standard AWS chain; the caller must be an IAM user in the super-admin account holding spinifex:GetAccountQuota.
spx admin account quota set <account-id>	--vcpus, --vpcs, --subnets, --eips, --volumes, --volumes-gib, --rds-instances, --load-balancers, --clear, plus --endpoint, --region, --ca-bundle	Overrides individual limits for one account through POST /admin/PutAccountQuota, leaving every dimension not named inheriting the configured baseline. -1 uncaps a dimension; 0 is a real limit that denies every request on it. --clear removes the whole override record and cannot be combined with a limit flag. The account must already exist. Prints the resulting limits, so a caller never has to guess what it applied. The caller must hold spinifex:PutAccountQuota.
spx admin account delete <account-id>	--dry-run, --force, --yes (requires --name), --name (account name, must match the stored record)	Prints the inventory it is about to remove → confirms the account name (prompted unless --name is given) → marks the account TERMINATING, which blocks its credentials cluster-wide → deletes its resources in dependency order (compute, attachments, storage, network, platform, identity, account), waiting for each stage to actually drain before starting the next → releases the name reservation and quota counter → deletes the account record. Anything that will not delete is reported as stuck with the reason, and the account stays TERMINATING so the residue keeps an owner. 000000000000 and 000000000001 are refused. --dry-run lists and changes nothing. The compute stage reaps ECS clusters, EKS nodegroups and clusters, and RDS instances before it reaps plain instances, because their backing VMs belong to the system account and a tenant-scoped instance listing cannot see them. The storage stage reaps the account's S3 buckets: uploads in flight are aborted, then the objects are removed a page at a time, then the bucket. Only buckets the account owns are ever listed, so the shared platform buckets are out of reach rather than filtered out. Platform-managed serving endpoints are reaped ahead of instances, because only their own delete releases the ENI and attached data volume alongside the VM, and spot instance requests are cancelled first so a live request cannot launch a replacement for something already terminated. The network stage reaps load balancers, then target groups, then NAT gateways and egress-only internet gateways before the subnets, security groups and addresses they hold. The platform stage reaps ACM certificates.
spx admin account delete --force	as above	Escalates only for resources the ordinary API refuses: it clears a volume attachment in the control plane without the guest's cooperation (ec2.ForceDetachVolume), hard-destroys after a graceful attempt times out, and treats an already-missing resource as deleted. It never reorders the stages and never widens scope beyond the account. Use it once the guest is being destroyed — this is the escape from a volume stuck attached to an instance that will not terminate, where neither can otherwise be deleted.
spx admin account delete --remote	as above, plus --endpoint, --region, --ca-bundle, --client-token	Same teardown driven through POST /admin/DeleteAccount over SigV4 instead of NATS, then follows DescribeAccountDeletion to completion, printing each stage as it lands. Credentials come from the standard AWS chain; the caller must be an IAM user in the super-admin account holding spinifex:DeleteAccount and spinifex:DescribeAccountDeletion.
spx admin principal create <name>	--grant (repeatable or comma-separated; default: every admin method — CreateAccount, DeleteAccount, DescribeAccountDeletion, GetAccountQuota, ListAccounts, PutAccountQuota)	Creates an IAM user in the super-admin account with an inline spx-admin-methods policy allowing the requested methods, revokes any access key left from a previous run, and mints one new key. Each method is named rather than granted as spinifex:*, so a later addition to the admin surface is not authorised by an existing key. The principal's grants are exactly that one policy: re-running with fewer --grant values narrows it, and any other inline policy on the user is removed, so a rename cannot leave a wider grant behind. The secret is printed once; store it in a secret manager or an AWS profile, never in a config file. admin is refused as a name — it is the account's own administrator and rotating its key would revoke the bootstrap credential.
spx admin principal list	—	Lists every principal in the super-admin account with its access-key count and what it may call. An attached policy is reported by name (AdministratorAccess (attached)) rather than expanded, which is how an unscoped principal is spotted.
spx admin principal revoke <name>	—	Deletes every access key the named principal holds, leaving the user and its policy in place. Immediate and cluster-wide; re-issue with principal create.
spx admin principal delete <name>	—	Removes the principal entirely: its access keys, its inline policies and the user. admin is refused, as it is for create. Use revoke instead when the principal should stay and only its credential should stop working.
spx admin principal audit	—	Reports roles in the super-admin account whose trust policy names the account, its root ARN, or a wildcard. STS does not evaluate the caller's identity policy on AssumeRole, so any such role is assumable by a scoped admin principal, which then inherits its permissions. Exits non-zero if any role is flagged.

Image Management

Command	Flags	Description
spx admin images import	--name, --file, --distro, --version, --arch, --platform, --boot-mode (bios/uefi/uefi-preferred), --tag, --force, --skip-verify, --ami-name (override the registered DescribeImages name, which otherwise defaults to ami-{distro}-{version}-{arch}; used for locally-built appliances such as --ami-name spinifex-eks-node), --tmp-dir (default: /tmp)	Catalog imports (--name) download the image, fetch the catalog Checksum URL, verify the SHA-256/SHA-512 digest, and inherit BootMode from the catalog entry. --boot-mode overrides the catalog value when set. Mismatch fails closed; the cached file is left on disk and --force re-downloads. --file imports skip checksum verification (operator-supplied media is outside Spinifex's trust boundary, the skip is logged at INFO for audit) and require an explicit --boot-mode because there is no catalog metadata to inherit from. --skip-verify bypasses verification for catalog imports and emits a WARN slog + stderr notice; use only for debugging or when upstream mirrors are confirmed-broken. Import never modifies the image: the bytes written to storage are the bytes that were verified. A guest that needs to trust the deployment CA for in-guest SDK-over-TLS calls fetches it at runtime from http://169.254.169.254/spinifex/ca.pem (see the IMDS section), so a CA rotation reaches guests without re-importing anything.
spx admin images list	—	Lists available OS images that can be imported or downloaded
spx admin images promote	--image-id (required), --yes	Reads ami-<id>/config.json, validates the AMI is account-owned, then rewrites ImageOwnerAlias to "system" in-place. No block data is copied. The change takes effect immediately — the AMI becomes visible to all accounts via DescribeImages. Prompts for confirmation (skipped with --yes). Already-system AMIs are refused.
spx admin images remove	--image-id (required), --force, --yes	Loads ami-<id>/config.json, walks transitive dependents — copied snapshots whose VolumeID == imageID, volumes whose SnapshotID references the internal snap-ami-<id> or any derived snap, and account AMIs created via CopyImage whose SnapshotID is a derived snap — then prompts (skipped with --yes) before deleting ami-<id>/config.json (the DescribeImages barrier) followed by the rest of ami-<id>/ and snap-ami-<id>/. Account-owned AMIs are refused with a hint pointing at aws ec2 deregister-image + aws ec2 delete-snapshot. --force bypasses the dependency, ownership and config-corrupt checks for salvage of orphaned blocks.

GPU Management

Command	Flags	Description
spx admin gpu status	--node (default: local node)	Queries spinifex.node.status fan-out → finds the target node response → prints Node, GPU hardware (model list or "none detected"), IOMMU state, vfio-pci state, passthrough enabled/disabled, and GPU pool allocation (allocated/total). Also lists GPU-capable instance types when passthrough is active.
spx admin gpu enable	—	Checks current passthrough state via NATS → errors if already enabled or prerequisites not met (directs to setup) → writes gpu_passthrough = true to spinifex.toml via admin.SetGPUPassthrough → sends SIGHUP to spinifex-daemon via systemctl kill -s HUP → polls node status for up to 30 s until daemon confirms new state → prints final gpu status output. Must be run directly on the target host.
spx admin gpu disable	—	Checks current passthrough state via NATS → errors if already disabled or if AllocGPUs > 0 (must terminate GPU instances first) → writes gpu_passthrough = false → sends SIGHUP to spinifex-daemon → polls for up to 30 s → prints final gpu status output. Must be run directly on the target host.
spx admin gpu setup	—	Idempotent host configuration for GPU passthrough. Steps (skipped if already applied): detect GPUs via gpu.Discover → collect PCI IDs for all IOMMU-group siblings → check/enable IOMMU in GRUB (intel_iommu=on iommu=pt or amd_iommu=on iommu=pt) → write vfio udev rule (/etc/udev/rules.d/99-spinifex-vfio.rules) → blacklist nouveau (/etc/modprobe.d/blacklist-nouveau.conf) → blacklist amdgpu if AMD GPU present (/etc/modprobe.d/blacklist-amdgpu.conf) → write vfio-pci early binding config (/etc/modprobe.d/vfio-pci.conf) → add vfio modules to initramfs. If any change requires a reboot: runs update-initramfs -u and exits with reboot instructions. After reboot: verifies vfio_pci module is loaded → verifies each GPU is bound to vfio-pci (binds explicitly via driver_override if unbound) → calls gpu enable to activate passthrough.
spx admin gpu mig status	— (must be run on target host)	Runs gpu.Discover() locally → for each GPU prints: PCI address, model, MIG capability, MIG mode (enabled/disabled/N/A); for GPUs with MIG enabled lists active MIG slices with GI ID, profile name, and mdev path.
spx admin gpu mig enable	--profile <name> (required, e.g. 1g.10gb), --gpu <pci-addr> (optional, default: all MIG-capable GPUs)	Checks no GPU instances running (NATS) → discovers MIG-capable GPUs via gpu.Discover() (filtered by --gpu if set) → enables MIG mode on each target (gpu.EnableMIGMode) → lists available profiles (gpu.ListProfiles) and validates requested profile name → destroys any existing instances (gpu.DestroyAllInstances) → creates new instances filling GPU capacity (gpu.CreateInstances) → writes mig_profile to spinifex.toml via admin.SetMIGProfile → sends SIGHUP to spinifex-daemon. Must be run directly on the target host.
spx admin gpu mig disable	--gpu <pci-addr> (optional, default: all MIG-capable GPUs)	Checks no GPU instances running (NATS) → discovers MIG-capable GPUs via gpu.Discover() (filtered by --gpu if set) → destroys all GPU instances (gpu.DestroyAllInstances) → disables MIG mode (gpu.DisableMIGMode) → clears mig_profile in spinifex.toml via admin.SetMIGProfile → sends SIGHUP to spinifex-daemon. Must be run directly on the target host.

EKS Control-Plane Disaster Recovery

Command	Flags	Description
spx admin eks restore-snapshot	--cluster (required), --snapshot (optional, defaults to the latest snapshot in predastore), --account (optional, defaults to the bootstrap account)	Single-CP total-loss DR path (fail-safe): validates the snapshot exists in predastore BEFORE any mutation (a typo'd/missing key hard-fails, never resets into an empty datastore) → launches a fresh control-plane VM as a cluster-init seed (replaying the persisted create-time launch template) → sets a required-snapshot RecoveryDirective (cluster-reset) so the boot-time recovery agent aborts rather than resets-into-empty if it cannot fetch the snapshot → persists the replacement in cluster meta BEFORE re-pointing the NLB (so an NLB failure is convergeable by the reconciler, returned as a provisional status, not a hard error) → re-points the cluster NLB's apiserver and konnectivity target groups from the old CP's ENI to the new one → fences the old CP with retries, failing loudly if it cannot be confirmed terminated (split-brain guard). Any failure before the meta commit unwinds the fresh CP (terminate + clear directive) so a re-run does not stack a second resetting control plane. The returned status is provisional — success means the sequence completed, not that etcd is restored and serving; verify cluster health. HA clusters (a spread with a potentially surviving quorum) are rejected — recover those via quorum reformation instead.

Private Admin API

Not an AWS API: a Spinifex-internal surface on the AWS gateway, JSON in and JSON out, one method per path segment at POST /admin/<Method>. It exists so a self-service signup form, a provisioning system or a load-test harness can create and remove tenant accounts without an operator shell on a node.

Requests are SigV4-signed with service=spinifex and the gateway's configured region (region in awsgw.toml; production is us-west-1). The caller must be an IAM user in the super-admin account (000000000001) holding spinifex:<Method> — assumed-role sessions are refused outright. Every gate denies with the same AccessDenied, so a caller cannot learn which one it failed. Every response carries X-Amzn-RequestId, repeated as requestId in an error body.

Method	Request	Response	Notes
POST /admin/CreateAccount	name (email address, required), clientToken (32–128 chars of [A-Za-z0-9_-], required), source (free-form provenance tag, max 64 chars)	accountId, accountName, adminUser, accessKeyId, secretAccessKey, defaultVpcId	Creates a tenant account with an admin user, an AdministratorAccess policy, one access key and a default VPC. The secret is returned once; replaying the same clientToken within 24 hours returns the identical response and is the only way to re-obtain it.
POST /admin/DeleteAccount	accountId (12 digits, required), accountName (must match the stored record), clientToken (32–128 chars), force, dryRun	deletionId, accountId, state; plus inventory when dryRun	Marks the account TERMINATING and tears down everything it owns in dependency order, then deletes the account record. Returns as soon as the teardown starts — it takes minutes, so progress comes from DescribeAccountDeletion. dryRun needs neither the name nor a token, answers inline and changes nothing. One job per account: a second call with a different token gets OperationInProgress while one is running. 000000000000 and 000000000001 return AccessDenied, the same answer an unauthorized caller gets.
POST /admin/DescribeAccountDeletion	accountId (12 digits, required)	deletionId, state (RUNNING, COMPLETED, FAILED), startedAt, updatedAt, finishedAt, stages[] with per-stage deleted[], stuck[] and elapsed, error	The record outlives the account, so this stays answerable after the deletion completes. NoSuchEntity means no teardown was ever started for that account. A RUNNING job whose updatedAt has not advanced for five minutes is treated as abandoned — the gateway running it restarted — and a fresh DeleteAccount takes it over.
POST /admin/ListAccounts	—	accounts[] of accountId, accountName, status, createdAt; count	Enumerates every account. No credential material, and no paging: [signup] max_accounts bounds the listing. status is how a teardown stuck in TERMINATING is noticed.
POST /admin/GetAccountQuota	accountId (12 digits, required)	per-dimension limits, each with the source it came from	Reads the limits in force for one account, reporting for each dimension whether the value came from the [quota] block in awsgw.toml (config) or from an override stored against the account (override). Driven by spx admin account quota get.
POST /admin/PutAccountQuota	accountId (12 digits, required), the limit dimensions to override, or a clear directive	the resulting limits	Overrides individual limits for one account, leaving every dimension not named inheriting the configured baseline. -1 uncaps a dimension; 0 is a real limit that denies every request on it. The account must already exist. Driven by spx admin account quota set.

Errors are {"error":{"code":…,"message":…},"requestId":…}. OperationInProgress (409), ServiceUnavailable (503) and InternalError (500) are retryable with backoff, always reusing the same clientToken — a fresh token after a failure creates a duplicate account. AccountAlreadyExists (409), IdempotentParameterMismatch (400), LimitExceeded (400, [signup] max_accounts reached), InvalidParameterValue, MissingParameter, InvalidRequest, InvalidAction, MethodNotAllowed (405) and AccessDenied (403) are not.

Each method is a separate grant, and spx admin principal create decides which a credential holds. spx admin principal create operator grants all six — the operator and load-test credential. spx admin principal create signup --grant CreateAccount grants one, which is what an internet-facing signup form should hold: that key cannot reach deletion, so a compromise of the form's host cannot remove a tenant. Separate principals are the point rather than an accident: revoking either costs nothing else.

The surface is unreachable until an operator runs spx admin principal create; there is no config toggle and no default credential for it. The super-admin account's own admin user does hold these grants already, through the AdministratorAccess policy every account's administrator carries (Action: "*"), so its bootstrap key can call the surface without any further setup — but it is unscoped and unrotatable without locking out the operator, which is why automation should use a named principal. Revocation is spx admin principal revoke <name> (or aws iam delete-access-key), which is immediate and cluster-wide. [signup] max_accounts in awsgw.toml caps how many accounts may exist; it defaults to 128 when the section is absent, and 0 means uncapped.

AWS-Compatible API

EC2 — Instance Management

Command	Implemented Flags	Missing Flags	Status
run-instances	--image-id, --instance-type, --count, --key-name, --user-data, --subnet-id, --security-group-ids, --tag-specifications (instance-scoped), --block-device-mappings (DeviceName, VolumeSize, VolumeType, Iops, DeleteOnTermination), --placement (GroupName), --iam-instance-profile (Name/Arn), --capacity-reservation-specification (CapacityReservationTarget.CapacityReservationId, targeted-by-id only), --metadata-options (HttpPutResponseHopLimit; HttpTokens required/optional, defaulting to required except on Windows images), --launch-template (LaunchTemplateId/LaunchTemplateName, Version — resolves $Default/$Latest; direct params override the template)	--dry-run, --client-token, --disable-api-termination, --ebs-optimized, --network-interfaces, --private-ip-address, --monitoring, --credit-specification, --cpu-options, --hibernate-options	DONE
describe-instances	--instance-ids, --filters (instance-state-name, instance-id, instance-type, vpc-id, subnet-id, tag:*, tag-key, tag-value)	--max-results, --next-token, --dry-run	DONE
start-instances	--instance-ids	--dry-run, --force	DONE
stop-instances	--instance-ids	--force, --hibernate, --dry-run	DONE
terminate-instances	--instance-ids, DeleteOnTermination (per-volume)	--dry-run	DONE
reboot-instances	--instance-ids	--dry-run	DONE
describe-instance-types	--filters (capacity only)	--instance-types, --max-results, --next-token, --dry-run, other filters	DONE
modify-instance-attribute	--instance-id, --instance-type, --user-data, --disable-api-termination	--ebs-optimized, --source-dest-check, --instance-initiated-shutdown-behavior, --block-device-mappings, --groups, --ena-support, --sriov-net-support	DONE
get-console-output	--instance-id	--latest, --dry-run	DONE
get-password-data	--instance-id	--dry-run	DONE
describe-instance-attribute	--instance-id, --attribute (instanceType, userData, disableApiTermination, instanceInitiatedShutdownBehavior, disableApiStop, ebsOptimized, enaSupport, sourceDestCheck, rootDeviceName, kernel, ramdisk)	--dry-run	DONE
modify-instance-metadata-options	--instance-id, --http-put-response-hop-limit (1–64), --http-tokens (required/optional), --http-endpoint (enabled), --http-protocol-ipv6/--instance-metadata-tags (disabled) — unmodelled values return UnsupportedOperation	--dry-run	DONE
describe-instance-credit-specifications	--instance-ids	--filters, --max-results, --dry-run	DONE (stub — always returns standard)
describe-instance-status	--instance-ids, --include-all-instances, --filters (availability-zone, instance-state-code, instance-state-name, tag:*)	--max-results, --next-token, --dry-run, event/instance-status/system-status filters	DONE (static health)
monitor-instances	--instance-ids	--dry-run	DONE (60s detailed telemetry; state is enabled/disabled only, never pending/disabling)
unmonitor-instances	--instance-ids	--dry-run	DONE (returns the instance to the 300s basic tier)

EC2 — Spot Instances

Spot Instance Requests (SIRs) are a mock over the on-demand run-instances path: a request synchronously launches real VMs on the operator's own compute and is then reported active/fulfilled. There is no spot market — no bidding, price rejection, interruption, or reclamation, and instances are never reclaimed.

Command	Implemented Flags	Missing Flags	Status
request-spot-instances	--instance-count (default 1), --type (one-time/persistent — stored, behaviour identical), --spot-price (echoed only), --client-token, --launch-specification (ImageId, InstanceType, KeyName, SubnetId, SecurityGroupIds, UserData, BlockDeviceMappings, IamInstanceProfile, Placement.GroupName, NetworkInterfaces), --tag-specifications (spot-instances-request)	--valid-from, --valid-until, --launch-group, --availability-zone-group, --block-duration-minutes, --instance-interruption-behavior, --dry-run	DONE (mock)
describe-spot-instance-requests	--spot-instance-request-ids, --filters (spot-instance-request-id, state, instance-id, launch.image-id, launch.instance-type, launch.key-name, type, launched-availability-zone, tag-key, tag:*)	--max-results, --next-token, --dry-run	DONE
cancel-spot-instance-requests	--spot-instance-request-ids	--dry-run	DONE
describe-spot-price-history	—	all	NOT STARTED (InvalidAction) — on owned hardware there is no spot/on-demand price differential, so any synthetic price would be misleading rather than helpful

EC2 — IAM Instance Profile Associations

Command	Implemented Flags	Missing Flags	Status
associate-iam-instance-profile	--instance-id, --iam-instance-profile (Name/Arn)	--dry-run	DONE
disassociate-iam-instance-profile	--association-id	--dry-run	DONE
replace-iam-instance-profile-association	--association-id, --iam-instance-profile (Name/Arn)	--dry-run	DONE
describe-iam-instance-profile-associations	--association-ids, --filters (instance-id, state)	--max-results, --next-token, --dry-run	DONE

EC2 — Key Pairs

Command	Implemented Flags	Missing Flags	Status
create-key-pair	--key-name, --key-type (rsa/ed25519), --tag-specifications	--key-format, --dry-run	DONE
describe-key-pairs	--key-names, --key-pair-ids, --filters (key-pair-id, key-name, fingerprint, tag:*)	--max-results, --dry-run	DONE
delete-key-pair	--key-name, --key-pair-id	--dry-run	DONE
import-key-pair	--key-name, --public-key-material, --tag-specifications	--dry-run	DONE

EC2 — AMI Images

Command	Implemented Flags	Missing Flags	Status
describe-images	--image-ids, --owners (self/account-id/alias), --filters (name, state, architecture, image-id, is-public, owner-id, description, image-type, tag:*)	--executable-users, --include-deprecated, --include-disabled, --max-results, --next-token, --dry-run	DONE
create-image	--instance-id, --name, --description, --tag-specifications	--no-reboot, --block-device-mappings, --dry-run	DONE
register-image	--name, --description, --architecture (x86_64/arm64/i386), --root-device-name, --virtualization-type (hvm), --boot-mode (bios/uefi/uefi-preferred), --block-device-mappings (root w/ Ebs.SnapshotId+VolumeSize), --tag-specifications	--billing-products, --uefi-data	DONE
deregister-image	--image-id	--dry-run	DONE
copy-image	--source-image-id, --source-region, --name, --description, --client-token, --copy-image-tags, --tag-specifications (image only)	--encrypted, --kms-key-id, --destination-outpost-arn, --dry-run	DONE — metadata-only, no block copy; the new snapshot inherits the source VolumeID
describe-image-attribute	--image-id, --attribute (description, blockDeviceMapping)	--dry-run, other attributes (launchPermission, bootMode, kernel, ramdisk, sriovNetSupport, productCodes, tpmSupport, uefiData, imdsSupport, lastLaunchedTime, deregistrationProtection)	DONE
modify-image-attribute	--image-id, --description (top-level or structured)	--launch-permission, --imds-support, --operation-type, --user-ids, --user-groups, --organization-arns, --product-codes, --dry-run, other --attribute values	DONE
reset-image-attribute	--image-id, --attribute description	--attribute launchPermission, --dry-run	DONE
import-image	—	--disk-containers, --description, --architecture, --platform	NOT STARTED

EC2 — Volumes (EBS)

Command	Implemented Flags	Missing Flags	Status
describe-volumes	--volume-ids, --filters (volume-id, status, size, volume-type, attachment.instance-id, attachment.status, attachment.device, availability-zone, tag:*), persisted DeleteOnTermination	--max-results, --next-token, --dry-run	DONE
create-volume	--size, --availability-zone, --volume-type (gp3), --snapshot-id, --tag-specifications, --iops, --throughput	--encrypted (hardcoded false)	DONE
delete-volume	--volume-id	--dry-run	DONE
modify-volume	--volume-id, --size, --volume-type, --iops	--throughput, --dry-run, --multi-attach-enabled	DONE
attach-volume	--volume-id, --instance-id, --device (auto-assigns /dev/sd[f-p])	--dry-run	DONE
detach-volume	--volume-id, --instance-id (optional), --device, --force	--dry-run	DONE
describe-volume-status	--volume-ids, --filters (volume-id, volume-status.status, availability-zone)	--max-results, --next-token, --dry-run	DONE
describe-volumes-modifications	--volume-ids, --filters (modification-state, original-iops, original-size, original-volume-type, start-time, target-iops, target-size, target-volume-type, volume-id)	--max-results, --next-token, --dry-run	DONE

EC2 — Snapshots

Command	Implemented Flags	Missing Flags	Status
create-snapshot	--volume-id, --description, --tag-specifications	--dry-run	DONE
delete-snapshot	--snapshot-id	--dry-run	DONE
describe-snapshots	--snapshot-ids, --filters (snapshot-id, status, volume-id, volume-size, owner-id, tag:*)	--owner-ids, --max-results, --dry-run	DONE
copy-snapshot	--source-snapshot-id, --source-region, --description	--encrypted, --dry-run	DONE
create-snapshots	—	--instance-specification, --description, --tag-specifications	NOT STARTED

EC2 — Tags

Command	Implemented Flags	Missing Flags	Status
create-tags	--resources, --tags	--dry-run	DONE
delete-tags	--resources, --tags	--dry-run	DONE
describe-tags	--filters (resource-id, resource-type, key, value)	--max-results, --next-token, --dry-run	DONE

EC2 — Regions, AZs, Account Attributes

Command	Implemented Flags	Missing Flags	Status
describe-regions	— (returns configured region)	--region-names, --filters, --all-regions, --dry-run	DONE
describe-availability-zones	— (returns configured AZ)	--zone-names, --filters, --all-availability-zones	DONE
describe-account-attributes	--attribute-names	--dry-run	DONE

EC2 — Account Settings

Persistence works but stored values are not yet enforced by downstream services.

Command	Implemented Flags	Missing Flags	Status
enable-ebs-encryption-by-default	—	--dry-run	STARTED (enforcement pending)
disable-ebs-encryption-by-default	—	--dry-run	STARTED (enforcement pending)
get-ebs-encryption-by-default	—	--dry-run	STARTED (enforcement pending)
enable-serial-console-access	—	--dry-run	STARTED (enforcement pending)
disable-serial-console-access	—	--dry-run	STARTED (enforcement pending)
get-serial-console-access-status	—	--dry-run	DONE
enable-snapshot-block-public-access	—	--state	NOT STARTED
disable-snapshot-block-public-access	—	--dry-run	NOT STARTED
get-snapshot-block-public-access-state	—	--dry-run	NOT STARTED
enable-image-block-public-access	—	--image-block-public-access-state	NOT STARTED
disable-image-block-public-access	—	--dry-run	NOT STARTED
get-image-block-public-access-state	—	--dry-run	NOT STARTED
modify-instance-metadata-defaults	—	--http-tokens, --http-put-response-hop-limit, --http-endpoint, --instance-metadata-tags	NOT STARTED — account-level; blocked on an aws-sdk-go bump, since the gateway's generic handler needs the typed input struct to route it
get-instance-metadata-defaults	—	--dry-run	NOT STARTED — account-level; blocked on an aws-sdk-go bump, as InstanceMetadataDefaultsResponse is absent from our version

EC2 — VPC Core

Command	Implemented Flags	Missing Flags	Status
create-vpc	--cidr-block, --tag-specifications	--instance-tenancy, --dry-run	DONE
delete-vpc	--vpc-id	--dry-run	DONE
describe-vpcs	--vpc-ids, --filters (vpc-id, state, cidr-block, is-default, owner-id, tag:*)	--max-results, --dry-run	DONE
modify-vpc-attribute	--vpc-id, --enable-dns-hostnames, --enable-dns-support, --enable-network-address-usage-metrics	--dry-run	DONE
describe-vpc-attribute	--vpc-id, --attribute (enableDnsHostnames, enableDnsSupport, enableNetworkAddressUsageMetrics)	--dry-run	DONE
associate-vpc-cidr-block	—	--vpc-id, --cidr-block	NOT STARTED
disassociate-vpc-cidr-block	—	--association-id	NOT STARTED
create-subnet	--vpc-id, --cidr-block, --availability-zone, --tag-specifications	--dry-run	DONE
delete-subnet	--subnet-id	--dry-run	DONE
describe-subnets	--subnet-ids, --filters (vpc-id, subnet-id, availability-zone, cidr-block, state, default-for-az, tag:*)	--max-results, --dry-run	DONE
modify-subnet-attribute	--subnet-id, --map-public-ip-on-launch	--assign-ipv6-address-on-creation, --dry-run	DONE
associate-subnet-cidr-block	—	--subnet-id, --ipv6-cidr-block	NOT STARTED
disassociate-subnet-cidr-block	—	--association-id	NOT STARTED

EC2 — Security Groups

Command	Implemented Flags	Missing Flags	Status
create-security-group	--group-name, --description, --vpc-id, --tag-specifications	--dry-run	DONE
delete-security-group	--group-id	--dry-run	DONE
describe-security-groups	--group-ids, --filters (vpc-id, group-name, group-id, description, ip-permission.cidr, tag:*)	--group-names, --max-results, --dry-run	DONE
authorize-security-group-ingress	--group-id, --ip-permissions	--dry-run	DONE
authorize-security-group-egress	--group-id, --ip-permissions	--dry-run	DONE
revoke-security-group-ingress	--group-id, --ip-permissions	--dry-run	DONE
revoke-security-group-egress	--group-id, --ip-permissions	--dry-run	DONE
describe-security-group-rules	--filters (group-id, security-group-rule-id, tag:*, tag-key), --security-group-rule-ids	--max-results, --next-token, --dry-run	DONE

EC2 — Internet Gateway

Command	Implemented Flags	Missing Flags	Status
create-internet-gateway	--tag-specifications	--dry-run	DONE
attach-internet-gateway	--internet-gateway-id, --vpc-id	--dry-run	DONE
detach-internet-gateway	--internet-gateway-id, --vpc-id	--dry-run	DONE
delete-internet-gateway	--internet-gateway-id	--dry-run	DONE
describe-internet-gateways	--internet-gateway-ids, --filters (internet-gateway-id, attachment.vpc-id, attachment.state, tag:*)	--max-results, --dry-run	DONE

EC2 — Egress-Only Internet Gateway

KV CRUD only — no OVN/OVS integration. EIGWs are stored but have no effect on network topology. Implementation is blocked on platform-wide IPv6 support.

Command	Implemented Flags	Missing Flags	Status
create-egress-only-internet-gateway	--vpc-id, --tag-specifications	--client-token, --dry-run	STARTED (KV only, no OVN)
delete-egress-only-internet-gateway	--egress-only-internet-gateway-id	--dry-run	STARTED (KV only, no OVN)
describe-egress-only-internet-gateways	--egress-only-internet-gateway-ids, --filters (egress-only-internet-gateway-id, tag:*)	--max-results, --next-token, --dry-run	STARTED (KV only, no OVN)

EC2 — Route Tables

Command	Implemented Flags	Missing Flags	Status
create-route-table	--vpc-id, --tag-specifications	--dry-run	DONE
delete-route-table	--route-table-id	--dry-run	DONE
describe-route-tables	--route-table-ids, --filters (vpc-id, route-table-id, association.main, association.route-table-association-id, association.subnet-id, route.destination-cidr-block, route.gateway-id)	--max-results, --next-token, --dry-run	DONE
create-route	--route-table-id, --destination-cidr-block, --gateway-id, --nat-gateway-id	--egress-only-internet-gateway-id, --vpc-peering-connection-id, --dry-run	DONE
delete-route	--route-table-id, --destination-cidr-block	--dry-run	DONE
replace-route	--route-table-id, --destination-cidr-block, --gateway-id	--nat-gateway-id, --dry-run	DONE
associate-route-table	--route-table-id, --subnet-id	--gateway-id, --dry-run	DONE
disassociate-route-table	--association-id	--dry-run	DONE
replace-route-table-association	--association-id, --route-table-id	--dry-run	DONE

EC2 — Network Interfaces (ENIs)

Command	Implemented Flags	Missing Flags	Status
create-network-interface	--subnet-id, --private-ip-address, --description, --tag-specifications	--groups, --dry-run	DONE
delete-network-interface	--network-interface-id	--dry-run	DONE
describe-network-interfaces	--network-interface-ids, --filters (subnet-id, vpc-id, attachment.instance-id)	--max-results, --dry-run	DONE
modify-network-interface-attribute	—	--network-interface-id, --description, --groups	DONE
attach-network-interface	--network-interface-id, --instance-id, --device-index (all required)	--dry-run, --network-card-index, --ena-srd-specification	DONE — hot-plugged into a running instance
detach-network-interface	--attachment-id (required), --force	--dry-run	DONE
assign-private-ip-addresses	—	--network-interface-id, --private-ip-addresses, --secondary-private-ip-address-count	NOT STARTED
unassign-private-ip-addresses	—	--network-interface-id, --private-ip-addresses	NOT STARTED

EC2 — Elastic IP

EIP handlers are always registered. Without a public IPAM pool (external mode disabled, or nat without a public pool), describe-addresses returns an empty list and mutating commands return UnsupportedOperation. In nat mode a public pool (--external-pool or --external-source=dhcp at init) enables the full EIP surface with host-delivered ingress.

Command	Implemented Flags	Missing Flags	Status
allocate-address	--public-ipv4-pool, --tag-specifications	--domain, --dry-run	DONE
release-address	--allocation-id	--dry-run	DONE
associate-address	--allocation-id, --network-interface-id, --instance-id, --private-ip-address	--dry-run, --allow-reassociation	DONE
disassociate-address	--association-id	--dry-run	DONE
describe-addresses	--allocation-ids, --public-ips, --filters (allocation-id, public-ip, instance-id, association-id, domain, tag:*)	--dry-run	DONE
describe-addresses-attribute	--allocation-ids	--attribute, --dry-run, --max-results, --next-token	DONE

EC2 — NAT Gateway

Command	Implemented Flags	Missing Flags	Status
create-nat-gateway	--subnet-id, --allocation-id, --tag-specifications	--connectivity-type, --dry-run	DONE
delete-nat-gateway	--nat-gateway-id	--dry-run	DONE
describe-nat-gateways	--nat-gateway-ids, --filters (vpc-id, state)	--max-results, --next-token, --dry-run	DONE
assign-private-nat-gateway-address	—	--nat-gateway-id, --private-ip-addresses	NOT STARTED
associate-nat-gateway-address	—	--nat-gateway-id, --allocation-ids	NOT STARTED

EC2 — Placement Groups

Command	Implemented Flags	Missing Flags	Status
create-placement-group	--group-name, --strategy (spread/cluster), --tag-specifications	--partition-count, --spread-level, --dry-run	DONE
delete-placement-group	--group-name	--dry-run	DONE
describe-placement-groups	--group-names, --group-ids, --filters (strategy, state, spread-level, group-name, tag:*, tag-key, tag-value)	--dry-run	DONE

EC2 — VPC Peering

Command	Implemented Flags	Missing Flags	Status
create-vpc-peering-connection	—	--vpc-id, --peer-vpc-id, --peer-owner-id, --peer-region, --tag-specifications	NOT STARTED
accept-vpc-peering-connection	—	--vpc-peering-connection-id, --dry-run	NOT STARTED
reject-vpc-peering-connection	—	--vpc-peering-connection-id, --dry-run	NOT STARTED
delete-vpc-peering-connection	—	--vpc-peering-connection-id, --dry-run	NOT STARTED
describe-vpc-peering-connections	—	--vpc-peering-connection-ids, --filters, --max-results	NOT STARTED
modify-vpc-peering-connection-options	—	--vpc-peering-connection-id, --requester-peering-connection-options, --accepter-peering-connection-options	NOT STARTED

EC2 — VPC Endpoints

Command	Implemented Flags	Missing Flags	Status
create-vpc-endpoint	—	--vpc-id, --service-name, --vpc-endpoint-type, --route-table-ids, --subnet-ids, --tag-specifications	NOT STARTED
delete-vpc-endpoints	—	--vpc-endpoint-ids, --dry-run	NOT STARTED
describe-vpc-endpoints	—	--vpc-endpoint-ids, --filters, --max-results	NOT STARTED
describe-vpc-endpoint-services	—	--service-names, --filters, --max-results	NOT STARTED
modify-vpc-endpoint	—	--vpc-endpoint-id, --add-route-table-ids, --remove-route-table-ids, --add-subnet-ids, --remove-subnet-ids	NOT STARTED

EC2 — VPN & Customer Gateway

Command	Implemented Flags	Missing Flags	Status
create-customer-gateway	—	--type, --bgp-asn, --ip-address, --tag-specifications	NOT STARTED
delete-customer-gateway	—	--customer-gateway-id, --dry-run	NOT STARTED
describe-customer-gateways	—	--customer-gateway-ids, --filters	NOT STARTED
create-vpn-gateway	—	--type, --amazon-side-asn, --tag-specifications	NOT STARTED
delete-vpn-gateway	—	--vpn-gateway-id, --dry-run	NOT STARTED
attach-vpn-gateway	—	--vpn-gateway-id, --vpc-id	NOT STARTED
detach-vpn-gateway	—	--vpn-gateway-id, --vpc-id	NOT STARTED
describe-vpn-gateways	—	--vpn-gateway-ids, --filters	NOT STARTED
create-vpn-connection	—	--type, --customer-gateway-id, --vpn-gateway-id, --options, --tag-specifications	NOT STARTED
delete-vpn-connection	—	--vpn-connection-id, --dry-run	NOT STARTED
describe-vpn-connections	—	--vpn-connection-ids, --filters	NOT STARTED
modify-vpn-connection	—	--vpn-connection-id, --vpn-gateway-id, --customer-gateway-id	NOT STARTED
modify-vpn-connection-options	—	--vpn-connection-id, --local-ipv4-network-cidr, --remote-ipv4-network-cidr	NOT STARTED

EC2 — Network ACLs

Command	Implemented Flags	Missing Flags	Status
create-network-acl	—	--vpc-id, --tag-specifications	NOT STARTED
delete-network-acl	—	--network-acl-id, --dry-run	NOT STARTED
describe-network-acls	—	--network-acl-ids, --filters, --max-results	NOT STARTED
create-network-acl-entry	—	--network-acl-id, --rule-number, --protocol, --rule-action, --cidr-block, --ingress/--egress, --port-range	NOT STARTED
delete-network-acl-entry	—	--network-acl-id, --rule-number, --ingress/--egress	NOT STARTED
replace-network-acl-association	—	--association-id, --network-acl-id	NOT STARTED
replace-network-acl-entry	—	--network-acl-id, --rule-number, --protocol, --rule-action, --cidr-block, --ingress/--egress, --port-range	NOT STARTED

EC2 — Prefix Lists

Command	Implemented Flags	Missing Flags	Status
create-managed-prefix-list	—	--prefix-list-name, --address-family, --max-entries, --entries, --tag-specifications	NOT STARTED
delete-managed-prefix-list	—	--prefix-list-id, --dry-run	NOT STARTED
describe-managed-prefix-lists	—	--prefix-list-ids, --filters, --max-results	NOT STARTED
modify-managed-prefix-list	—	--prefix-list-id, --current-version, --add-entries, --remove-entries, --prefix-list-name	NOT STARTED
get-managed-prefix-list-entries	—	--prefix-list-id, --target-version, --max-results	NOT STARTED
get-managed-prefix-list-associations	—	--prefix-list-id, --max-results	NOT STARTED

EC2 — Launch Templates

Command	Implemented Flags	Missing Flags	Status
create-launch-template	--launch-template-name, --launch-template-data (full nested RequestLaunchTemplateData), --version-description, --tag-specifications (launch-template-scoped), --dry-run (no-op)	--client-token (idempotency)	DONE
create-launch-template-version	--launch-template-id/--launch-template-name, --launch-template-data, --source-version (clone-and-override), --version-description, --dry-run (no-op)	--client-token, --resolve-alias	DONE
delete-launch-template	--launch-template-id/--launch-template-name, --dry-run (no-op)	—	DONE
delete-launch-template-versions	--launch-template-id/--launch-template-name, --versions (rejects the current default version), --dry-run (no-op)	—	DONE
modify-launch-template	--launch-template-id/--launch-template-name, --default-version, --dry-run (no-op)	—	DONE
describe-launch-templates	--launch-template-ids, --launch-template-names, --filters (launch-template-id, launch-template-name, create-time, tag:*, tag-key)	--max-results, --next-token, --dry-run	DONE
describe-launch-template-versions	--launch-template-id/--launch-template-name, --versions ($Default/$Latest/numeric), --min-version, --max-version, --filters (is-default-version, image-id, instance-type, kernel-id, ram-disk-id, ebs-optimized)	--max-results, --next-token, --dry-run, --resolve-alias	DONE

EC2 — Dedicated Hosts, IPv4 Pools, DHCP, Capacity Reservations

Command	Implemented Flags	Missing Flags	Status
allocate-hosts	—	--availability-zone, --instance-type, --quantity, --auto-placement, --tag-specifications	NOT STARTED
describe-hosts	—	--host-ids, --filters, --max-results	NOT STARTED
release-hosts	—	--host-ids	NOT STARTED
create-public-ipv4-pool	—	--tag-specifications, --dry-run	NOT STARTED
delete-public-ipv4-pool	—	--pool-id, --dry-run	NOT STARTED
describe-public-ipv4-pools	—	--pool-ids, --filters, --max-results	NOT STARTED
create-dhcp-options	—	--dhcp-configurations, --tag-specifications	NOT STARTED
delete-dhcp-options	—	--dhcp-options-id, --dry-run	NOT STARTED
describe-dhcp-options	—	--dhcp-options-ids, --filters, --max-results	NOT STARTED
associate-dhcp-options	—	--dhcp-options-id, --vpc-id, --dry-run	NOT STARTED
create-capacity-reservation	--instance-type, --instance-count, --availability-zone, --instance-platform, --instance-match-criteria, --tenancy (default only), --dry-run	--end-date, --end-date-type (unlimited only), --availability-zone-id, --tag-specifications	DONE
cancel-capacity-reservation	--capacity-reservation-id, --dry-run	—	DONE
describe-capacity-reservations	--capacity-reservation-ids, --filters	--max-results, --next-token	DONE
modify-capacity-reservation	—	--capacity-reservation-id, --instance-count, --end-date, --end-date-type	NOT STARTED

EC2 — Misc

Command	Implemented Flags	Missing Flags	Status
delete-network-interface-permission	—	--network-interface-permission-id, --force	NOT STARTED
enable-address-transfer	—	--allocation-id, --transfer-account-id	NOT STARTED
disable-address-transfer	—	--allocation-id	NOT STARTED

EBS Direct API

Command	Implemented Flags	Missing Flags	Status
start-snapshot	—	--volume-size, --parent-snapshot-id, --description, --encrypted	NOT STARTED
put-snapshot-block	—	--snapshot-id, --block-index, --block-data, --checksum	NOT STARTED
get-snapshot-block	—	--snapshot-id, --block-index	NOT STARTED
complete-snapshot	—	--snapshot-id, --changed-blocks-count	NOT STARTED
list-snapshot-blocks	—	--snapshot-id, --max-results, --next-token	NOT STARTED
list-changed-blocks	—	--second-snapshot-id, --first-snapshot-id, --max-results	NOT STARTED

---

IAM

All IAM operations are account-scoped. Root user (account 000000000000) bypasses policy evaluation entirely.

IAM — Users

Command	Implemented Flags	Missing Flags	Status
create-user	--user-name, --path	--tags, --permissions-boundary	DONE
get-user	--user-name	—	DONE
list-users	--path-prefix	--max-items, --marker	DONE
delete-user	--user-name	—	DONE
update-user	—	--user-name, --new-path, --new-user-name	NOT STARTED
put-user-policy	--user-name, --policy-name, --policy-document	—	DONE
get-user-policy	--user-name, --policy-name	—	DONE
delete-user-policy	--user-name, --policy-name	—	DONE
list-user-policies	--user-name	--max-items, --marker	DONE
tag-user	--user-name, --tags	—	DONE
untag-user	--user-name, --tag-keys	—	DONE
list-user-tags	--user-name	--max-items, --marker	DONE
put-user-permissions-boundary	—	--user-name, --permissions-boundary	NOT STARTED
delete-user-permissions-boundary	—	--user-name	NOT STARTED
create-login-profile	—	--user-name, --password	NOT STARTED
get-login-profile	—	--user-name	NOT STARTED
update-login-profile	—	--user-name, --password	NOT STARTED
delete-login-profile	—	--user-name	NOT STARTED
change-password	—	--old-password, --new-password	NOT STARTED

IAM — Access Keys

Command	Implemented Flags	Missing Flags	Status
create-access-key	--user-name	—	DONE
list-access-keys	--user-name	--max-items, --marker	DONE
delete-access-key	--access-key-id, --user-name	—	DONE
update-access-key	--access-key-id, --user-name, --status (Active/Inactive)	—	DONE
get-access-key-last-used	—	--access-key-id	NOT STARTED

IAM — Policies

Command	Implemented Flags	Missing Flags	Status
create-policy	--policy-name, --policy-document, --path, --description	--tags	DONE
get-policy	--policy-arn	—	DONE
get-policy-version	--policy-arn, --version-id	—	DONE
list-policy-versions	--policy-arn	--max-items, --marker	DONE
list-policies	—	--scope, --only-attached, --path-prefix, --max-items, --marker	DONE
delete-policy	--policy-arn	—	DONE
attach-user-policy	--user-name, --policy-arn	—	DONE
detach-user-policy	--user-name, --policy-arn	—	DONE
list-attached-user-policies	--user-name	--path-prefix, --max-items, --marker	DONE
create-policy-version	—	--policy-arn, --policy-document, --set-as-default	NOT STARTED
delete-policy-version	—	--policy-arn, --version-id	NOT STARTED
set-default-policy-version	—	--policy-arn, --version-id	NOT STARTED
list-entities-for-policy	—	--policy-arn, --entity-filter, --path-prefix, --policy-usage-filter	NOT STARTED
tag-policy	--policy-arn, --tags	—	DONE
untag-policy	--policy-arn, --tag-keys	—	DONE
list-policy-tags	--policy-arn	--max-items, --marker	DONE
generate-service-last-accessed-details	—	--arn, --granularity	NOT STARTED
get-service-last-accessed-details	—	--job-id	NOT STARTED
get-service-last-accessed-details-with-entities	—	--job-id, --service-namespace	NOT STARTED
list-policies-granting-service-access	—	--arn, --service-namespaces	NOT STARTED

IAM — Roles

Command	Implemented Flags	Missing Flags	Status
create-role	--role-name, --assume-role-policy-document, --path, --description, --max-session-duration, --tags	--permissions-boundary	DONE
get-role	--role-name	—	DONE
list-roles	--path-prefix	--max-items, --marker	DONE
delete-role	--role-name	—	DONE
update-role	--role-name, --description, --max-session-duration	—	DONE
update-assume-role-policy	--role-name, --policy-document	—	DONE
attach-role-policy	--role-name, --policy-arn	—	DONE
detach-role-policy	--role-name, --policy-arn	—	DONE
list-attached-role-policies	--role-name, --path-prefix	--max-items, --marker	DONE
list-role-policies	--role-name	--max-items, --marker	DONE
put-role-policy	--role-name, --policy-name, --policy-document	—	DONE
get-role-policy	--role-name, --policy-name	—	DONE (document returned as raw JSON, not URL-encoded)
delete-role-policy	--role-name, --policy-name	—	DONE
put-role-permissions-boundary	—	--role-name, --permissions-boundary	NOT STARTED
delete-role-permissions-boundary	—	--role-name	NOT STARTED
tag-role	--role-name, --tags	—	DONE
untag-role	--role-name, --tag-keys	—	DONE
list-role-tags	--role-name	--max-items, --marker	DONE
update-role-description	—	--role-name, --description	NOT STARTED

IAM — Instance Profiles

Command	Implemented Flags	Missing Flags	Status
create-instance-profile	--instance-profile-name, --path, --tags	—	DONE
get-instance-profile	--instance-profile-name	—	DONE
list-instance-profiles	--path-prefix	--max-items, --marker	DONE
list-instance-profiles-for-role	--role-name	--max-items, --marker	DONE
delete-instance-profile	--instance-profile-name	—	DONE
add-role-to-instance-profile	--instance-profile-name, --role-name	—	DONE
remove-role-from-instance-profile	--instance-profile-name, --role-name	—	DONE
tag-instance-profile	--instance-profile-name, --tags	—	DONE
untag-instance-profile	--instance-profile-name, --tag-keys	—	DONE
list-instance-profile-tags	--instance-profile-name	--max-items, --marker	DONE

IAM — OIDC Providers

Command	Implemented Flags	Missing Flags	Status
create-open-id-connect-provider	--url, --client-id-list, --thumbprint-list, --tags	—	DONE
get-open-id-connect-provider	--open-id-connect-provider-arn	—	DONE
list-open-id-connect-providers	—	—	DONE
delete-open-id-connect-provider	--open-id-connect-provider-arn	—	DONE
add-client-id-to-open-id-connect-provider	—	--open-id-connect-provider-arn, --client-id	NOT STARTED
remove-client-id-from-open-id-connect-provider	—	--open-id-connect-provider-arn, --client-id	NOT STARTED
update-open-id-connect-provider-thumbprint	—	--open-id-connect-provider-arn, --thumbprint-list	NOT STARTED
tag-open-id-connect-provider / untag-open-id-connect-provider	--open-id-connect-provider-arn, --tags/--tag-keys	—	DONE
list-open-id-connect-provider-tags	--open-id-connect-provider-arn	--max-items, --marker	DONE

IAM — Groups

Command	Implemented Flags	Missing Flags	Status
create-group	--group-name, --path	—	DONE
get-group	--group-name	—	DONE
list-groups	--path-prefix	--max-items, --marker	DONE
delete-group	--group-name	—	DONE
update-group	—	--group-name, --new-path, --new-group-name	NOT STARTED
add-user-to-group	--group-name, --user-name	—	DONE
remove-user-from-group	--group-name, --user-name	—	DONE
list-groups-for-user	--user-name	--max-items, --marker	DONE
attach-group-policy	--group-name, --policy-arn	—	DONE
detach-group-policy	--group-name, --policy-arn	—	DONE
list-attached-group-policies	--group-name, --path-prefix	--max-items, --marker	DONE
put-group-policy	--group-name, --policy-name, --policy-document	—	DONE
get-group-policy	--group-name, --policy-name	—	DONE
delete-group-policy	--group-name, --policy-name	—	DONE
list-group-policies	--group-name	--max-items, --marker	DONE

IAM — Account

Command	Implemented Flags	Missing Flags	Status
get-account-summary	—	—	DONE

---

STS

Command	Implemented Flags	Missing Flags	Status
get-caller-identity	—	—	DONE
assume-role	--role-arn, --role-session-name, --duration-seconds (900–min(role MaxSessionDuration, 43200))	--policy, --policy-arns (→ PackedPolicyTooLarge); --tags, --transitive-tag-keys (→ InvalidParameterValue); --serial-number, --token-code (→ InvalidParameterValue); --external-id, --source-identity (accepted and logged, not enforced — no Condition evaluator in v1)	DONE
get-session-token	--duration-seconds (900–129600, default 43200 = 12h; clamped, not rejected)	--serial-number, --token-code (MFA → InvalidParameterValue)	DONE
assume-role-with-web-identity	--role-arn, --role-session-name, --web-identity-token, --duration-seconds (900–43200, default 3600)	--provider-id; --policy, --policy-arns (→ PackedPolicyTooLarge)	DONE
assume-role-with-saml	—	--role-arn, --principal-arn, --saml-assertion, --policy, --policy-arns, --duration-seconds	NOT STARTED
get-access-key-info	—	--access-key-id	NOT STARTED
get-federation-token	—	--name, --policy, --policy-arns, --duration-seconds, --tags	NOT STARTED
decode-authorization-message	—	--encoded-message	NOT STARTED

Trust policies (AssumeRolePolicyDocument) reject NotPrincipal, NotAction, empty-string Action elements, and empty Principal blocks at write time (MalformedPolicyDocument). Condition blocks are rejected except on sts:AssumeRoleWithWebIdentity with StringEquals (IRSA), which v1 evaluates at assume time ({iss}:sub, {iss}:aud); anything wider is rejected to avoid silent over-grant.

---

IMDS (Instance Metadata Service)

Available at 169.254.169.254 from inside every running guest VM, matching AWS. The endpoint is reached from within a guest over plain HTTP, with no in-VM agent to install.

IMDSv2 by default. Every read requires a session token unless the instance opted into IMDSv1. Obtain a token with a PUT /latest/api/token carrying X-aws-ec2-metadata-token-ttl-seconds (1–21600), then send it back in X-aws-ec2-metadata-token on every read. A tokenless (v1-style) GET returns 401 Unauthorized with an empty body unless the requesting ENI's instance carries MetadataOptions.HttpTokens=optional.

IMDSv1 is opt-in per instance via --metadata-options HttpTokens=optional on run-instances or modify-instance-metadata-options, on any platform, exactly as AWS does. The launch default is required everywhere except Windows images, which default to optional because cloudbase-init has no IMDSv2 token support in any release and would otherwise never read metadata at all. --http-endpoint disabled, --http-protocol-ipv6 enabled and --instance-metadata-tags enabled are still rejected with UnsupportedOperation.

# Inside the guest VM:
TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" \
    -H "X-aws-ec2-metadata-token-ttl-seconds: 60")
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
    http://169.254.169.254/latest/meta-data/instance-id

# Tokenless GET → 401 Unauthorized (empty body)
curl -i http://169.254.169.254/latest/meta-data/instance-id

IMDS — Deployment CA

A guest that calls a Spinifex HTTPS endpoint with an AWS SDK needs the deployment CA in its trust store. The CA is served from the IMDS responder at /spinifex/ca.pem — deliberately outside the /latest tree, because the AWS-compatible surface stays byte-identical to EC2's and cloud-init must never descend into a Spinifex-only path.

The route is token-free: a CA certificate is public material (the console already serves it unauthenticated at /api/ca.pem), and requiring the IMDSv2 handshake would push a token dance into every bootstrap snippet. The X-Forwarded-For SSRF guard still applies, as on every other path.

#cloud-config
runcmd:
  # Debian/Ubuntu
  - curl -fsS http://169.254.169.254/spinifex/ca.pem -o /usr/local/share/ca-certificates/spinifex.crt
  - update-ca-certificates

RHEL/Rocky write to /etc/pki/ca-trust/source/anchors/spinifex.crt and run update-ca-trust; stock Alpine has no updater or anchor directory, so append the PEM to /etc/ssl/certs/ca-certificates.crt. Rotating ca.pem on a node is served immediately — vpcd re-reads the file when it changes.

IMDS — Supported Paths

Path	Method	Source	Status
/latest/api/token	PUT	Issues an ENI-bound IMDSv2 token; X-aws-ec2-metadata-token-ttl-seconds ∈ [1, 21600] required	DONE
/	GET	Supported API-version list (2021-07-15, latest); token-gated	DONE
/latest	GET	Top-level tree listing (dynamic, meta-data, user-data)	DONE
/<date>/...	GET/PUT	Any dated API version aliases to /latest (cloud-init parity)	DONE
/latest/meta-data/	GET	Directory listing of supported children	DONE
/latest/meta-data/instance-id	GET	vm.ID	DONE
/latest/meta-data/instance-type	GET	vm.InstanceType	DONE
/latest/meta-data/ami-id	GET	launch ImageId	DONE
/latest/meta-data/ami-launch-index	GET	Per-instance launch index (0..n-1), contiguous on partial failure	DONE
/latest/meta-data/reservation-id	GET	DescribeInstances Reservation.ReservationId	DONE
/latest/meta-data/instance-life-cycle	GET	spot for a spot-launched instance, else on-demand; defaults to on-demand on a resolution miss (never 404, the leaf is always advertised)	DONE
/latest/meta-data/local-ipv4	GET	ENIRecord.PrivateIpAddress (== request source IP)	DONE
/latest/meta-data/public-ipv4	GET	EIP, else instance public IP; empty body if none	DONE
/latest/meta-data/public-hostname	GET	Mirrors public-ipv4; 404 when no public IP	DONE
/latest/meta-data/mac	GET	ENIRecord.MacAddress	DONE
/latest/meta-data/security-groups	GET	ENIRecord.SecurityGroupIds, newline-separated	DONE
/latest/meta-data/hostname, /local-hostname	GET	Synthesised ip-<dashed-ip>.<region>.compute.internal	DONE
/latest/meta-data/placement/availability-zone	GET	ENIRecord.AvailabilityZone	DONE
/latest/meta-data/placement/region	GET	Derived from AZ (trailing letter stripped)	DONE
/latest/meta-data/services/{domain,partition}	GET	Static: amazonaws.com / aws	DONE
/latest/meta-data/iam/info	GET	{InstanceProfileArn, InstanceProfileId}; 404 if no profile	DONE
/latest/meta-data/iam/security-credentials/	GET	Role name(s) under the profile, one per line; empty body if none	DONE
/latest/meta-data/iam/security-credentials/<role>	GET	STS AssumeRoleForInstance → ASIA-prefixed temporary credential JSON	DONE
/latest/meta-data/public-keys/	GET	0=<keyName> from the launch key pair; 404 if none	DONE
/latest/meta-data/public-keys/0/	GET	openssh-key (format list for index 0)	DONE
/latest/meta-data/public-keys/0/openssh-key	GET	Launch SSH public key, live-fetched from the key store; 404 if the key was deleted, 500 on backend fault	DONE
/latest/user-data	GET	vm.UserData; 404 if none	DONE
/spinifex/ca.pem	GET	Deployment CA PEM (application/x-pem-file); token-free, outside the AWS tree; 404 when no CA is configured	DONE
/latest/dynamic	GET	Lists instance-identity/	DONE
/latest/dynamic/instance-identity	GET	Lists document (signed forms listed when the signing key lands)	DONE
/latest/dynamic/instance-identity/document	GET	Unsigned identity document from resolved ENI + instance facts	DONE
/latest/dynamic/instance-identity/{signature,pkcs7,rsa2048}	GET	Signed forms; need a per-cluster signing key	NOT STARTED (404; lands with EKS IRSA)
/latest/meta-data/network/interfaces/macs/<mac>/...	GET	Primary ENI subtree: mac, device-number, interface-id, owner-id, subnet-id, vpc-id, local-ipv4s, local-hostname, security-group-ids, security-groups, subnet-ipv4-cidr-block, vpc-ipv4-cidr-block(s), and public-ipv4s/public-hostname when an EIP is attached	DONE (single-NIC; multi-ENI deferred)
/latest/meta-data/tags/instance/<key>	GET	Instance-tag metadata; gated on InstanceMetadataTags enablement	NOT STARTED (404)
/latest/meta-data/block-device-mapping/...	GET	ami/root/ebsN/ephemeralN device map	NOT STARTED (404)
/latest/meta-data/placement/{group-name,partition-number,availability-zone-id,host-id}	GET	Placement extras beyond availability-zone/region	NOT STARTED (404)
/latest/meta-data/instance-action	GET	none unless interruptible instances ship	NOT STARTED (404)
/latest/meta-data/spot/{instance-action,termination-time}	GET	404 is the faithful steady state for a never-interrupted spot instance ("no action scheduled"); a 200 body would trigger interruption handling in pollers (AWS Node Termination Handler / Karpenter). Not advertised in the spot/ listing	DONE (404 by contract)

---

ELBv2 (Application & Network Load Balancer)

The data plane uses a system-managed LB VM, launched automatically during create-load-balancer. Application Load Balancers run HAProxy (L7: rules, fixed-response, redirect, HTTP/HTTPS). Network Load Balancers run nginx stream (L4: TCP, UDP, TLS, TCP_UDP) — HAProxy cannot load-balance UDP. The agent selects the engine from the Engine field on the config-delivery response.

ELBv2 — Load Balancers

Command	Implemented Flags	Missing Flags	Status
create-load-balancer	--name, --subnets, --security-groups, --scheme (internet-facing/internal), --tags, --ip-address-type (ipv4)	--type (hardcoded application), --customer-owned-ipv4-pool, --dry-run	DONE
delete-load-balancer	--load-balancer-arn	--dry-run	DONE
describe-load-balancers	--load-balancer-arns, --names	--page-size, --marker, --dry-run	DONE

ELBv2 — Target Groups

Command	Implemented Flags	Missing Flags	Status
create-target-group	--name, --protocol (HTTP), --port, --vpc-id, --target-type (instance), --health-check-protocol, --health-check-port, --health-check-path, --health-check-interval-seconds, --health-check-timeout-seconds, --healthy-threshold-count, --unhealthy-threshold-count, --matcher, --tags	--health-check-enabled, --protocol-version, --ip-address-type, --dry-run	DONE
delete-target-group	--target-group-arn	--dry-run	DONE
describe-target-groups	--target-group-arns, --names, --load-balancer-arn	--page-size, --marker, --dry-run	DONE

ELBv2 — Targets

Command	Implemented Flags	Missing Flags	Status
register-targets	--target-group-arn, --targets (Id, Port)	--dry-run	DONE
deregister-targets	--target-group-arn, --targets (Id, Port)	--dry-run	DONE
describe-target-health	--target-group-arn, --targets	--include	DONE

ELBv2 — Listeners

Command	Implemented Flags	Missing Flags	Status
create-listener	--load-balancer-arn, --default-actions (Type=forward, TargetGroupArn), --protocol (HTTP/HTTPS/TLS), --port, --certificates (HTTPS/TLS), --ssl-policy	--alpn-policy, --mutual-authentication, --dry-run	DONE
modify-listener	--listener-arn, --protocol, --port, --default-actions, --certificates, --ssl-policy	--alpn-policy, --mutual-authentication, --dry-run	DONE
delete-listener	--listener-arn	--dry-run	DONE
describe-listeners	--load-balancer-arn, --listener-arns	--page-size, --marker, --dry-run	DONE

ELBv2 — Listener Rules

Command	Implemented Flags	Missing Flags	Status
create-rule	--listener-arn, --priority (1–50000), --conditions (host-header, path-pattern, http-header, http-request-method, query-string, source-ip), --actions (forward, redirect, fixed-response), --tags	--dry-run	DONE
modify-rule	--rule-arn, --conditions, --actions	--dry-run	DONE
delete-rule	--rule-arn	--dry-run	DONE
describe-rules	--listener-arn, --rule-arns	--page-size, --marker (parsed, not enforced)	DONE
set-rule-priorities	--rule-priorities (RuleArn, Priority)	—	DONE

A synthetic default rule is derived from the listener's DefaultActions.

ELBv2 — Listener Certificates & SSL Policies

Command	Implemented Flags	Missing Flags	Status
add-listener-certificates	--listener-arn, --certificates	--dry-run	DONE
remove-listener-certificates	--listener-arn, --certificates	--dry-run	DONE
describe-listener-certificates	--listener-arn	--page-size, --marker	DONE
describe-ssl-policies	--names	--load-balancer-type, --page-size, --marker	DONE (static catalog: ELBSecurityPolicy-FS-1-2-Res-2019-08, ELBSecurityPolicy-TLS13-1-2-2021-06 — metadata only, no in-platform TLS termination)

The default certificate cannot be added/removed via these calls — set it on the listener.

ELBv2 — Attributes & Modify

Command	Implemented Flags	Missing Flags	Status
describe-load-balancer-attributes	--load-balancer-arn	—	DONE (stored values over per-type defaults)
modify-load-balancer-attributes	--load-balancer-arn, --attributes	—	DONE (unknown keys → ValidationError)
describe-target-group-attributes	--target-group-arn	—	DONE
modify-target-group-attributes	--target-group-arn, --attributes (incl. deregistration_delay.timeout_seconds, stickiness.*)	—	DONE
modify-target-group	--target-group-arn, --health-check-*, --matcher	--target-type/--protocol/--vpc-id (immutable)	DONE
describe-listener-attributes	--listener-arn	—	DONE (stub — returns empty; not persisted)
modify-listener-attributes	--listener-arn, --attributes	—	DONE (stub — echoes input; not persisted)

ELBv2 — Network & Security

Command	Implemented Flags	Missing Flags	Status
set-security-groups	--load-balancer-arn, --security-groups	--enforce-security-group-inbound-rules-on-private-link-traffic	DONE (ALB only — NLB → InvalidConfigurationRequest)
set-subnets	--load-balancer-arn, --subnets, --subnet-mappings	--ip-address-type	DONE (live ENI add/remove with rollback)
set-ip-address-type	--load-balancer-arn, --ip-address-type (ipv4)	dualstack/IPv6 (rejected)	DONE

ELBv2 — Tags

Command	Implemented Flags	Missing Flags	Status
describe-tags	--resource-arns (loadbalancer, targetgroup, listener)	—	DONE
add-tags	--resource-arns, --tags	—	DONE
remove-tags	--resource-arns, --tag-keys	—	DONE

ELBv2 — Not Yet Implemented

Feature	Priority	Status
In-platform HTTPS/TLS termination (cert + SSL-policy APIs exist; data-plane TLS not terminated)	High	NOT STARTED
ALPN policy, mutual TLS (mTLS)	Medium	NOT STARTED
Listener attribute persistence (Describe/ModifyListenerAttributes are stubs)	Medium	NOT STARTED
Active health checking (API-driven, vs. HAProxy/nginx-only today)	Medium	NOT STARTED
IP and Lambda target types	Low	NOT STARTED
S3 access log delivery	Low	NOT STARTED
WAF integration	Low	NOT STARTED

---

ACM (AWS Certificate Manager)

Spinifex both stores externally-issued certificates (import-certificate) and issues its own (request-certificate) for ELBv2 listener references. Certs are account-scoped; describe/delete enforce ownership, and delete-certificate refuses with ResourceInUseException while any load balancer listener still references the ARN — no force flag, matching AWS.

request-certificate mints a CertificateArn immediately and returns PENDING_VALIDATION; it never issues inline. The validation mode is derived from deployment state, never configured:

Mode	Who writes the DNS record	ResourceRecord returned	Renewal	Status
PROVIDER_API	Spinifex, via the operator's DNS provider API	none	automatic	request accepted, PENDING_VALIDATION — the DNS-01 order is driven by a later worker
MANUAL_TXT	the operator, by hand or Terraform	TXT, rotates per order	manual — INELIGIBLE	request accepted, PENDING_VALIDATION — same as above
CNAME_DELEGATION	operator once, then Spinifex	CNAME, stable	automatic	deferred — never selected yet (northstar cannot serve public authoritative queries); an ARN-stable delegation token is minted on every managed certificate now so this lands as a non-breaking addition
PRIVATE_CA	nobody — no validation	none	automatic	DONE — issues synchronously against the tenant CA, no domain outside its name constraints

PROVIDER_API is selected when a DNS provider credential is configured; MANUAL_TXT when northstar hosts the zone; otherwise PRIVATE_CA — the only option for a deployment with no real, publicly delegated domain. Terraform's canonical aws_acm_certificate → aws_route53_record → aws_acm_certificate_validation → aws_lb_listener flow works unmodified in every mode: where Spinifex owns the record write, no ResourceRecord is emitted, so for_each over domain_validation_options yields zero records and aws_acm_certificate_validation still blocks correctly by polling until ISSUED.

Command	Implemented Flags	Missing Flags	Status
import-certificate	--certificate, --private-key, --certificate-chain, --certificate-arn (re-import)	--tags	DONE
describe-certificate	--certificate-arn	—	DONE
list-certificates	—	--certificate-statuses, --includes, --max-items, --next-token	DONE
get-certificate	--certificate-arn	—	DONE — returns the leaf certificate and its chain
delete-certificate	--certificate-arn	—	DONE
request-certificate	--domain-name, --subject-alternative-names, --tags	--validation-method, --certificate-authority-arn, --options, --idempotency-token	PARTIAL — PRIVATE_CA issues synchronously; PROVIDER_API/MANUAL_TXT are accepted and correctly shaped but stay PENDING_VALIDATION until a later issuance worker lands
add-tags-to-certificate / list-tags-for-certificate / remove-tags-from-certificate	--certificate-arn, --tags/--tag-keys	—	DONE
export-certificate	—	--certificate-arn, --passphrase	NOT STARTED

---

RDS (PostgreSQL and MariaDB)

Each DB instance is one dedicated system-owned VM running the engine directly, launched from the spinifex-rds-postgres or spinifex-rds-mariadb AMI, tagged spinifex:managed-by=rds and therefore hidden from the customer's EC2 API. The engine is reached over a customer-account ENI injected into a subnet of the DB subnet group, so the endpoint is private — reachable from inside the VPC only, from any subnet of it rather than the endpoint ENI's own. Endpoint.Address is {db-instance-identifier}.{account-id}.{region}.rds.{base-domain} where northstar is configured, and the endpoint ENI's private IP where it is not; the IP is stable across VM replacement either way.

The endpoint is the only path to the engine. The DB VM has two other NICs that no customer security group governs — a system NIC in a shared RDS system VPC, which is how the in-guest agent reaches the control plane, and a management NIC on the host's management bridge — and the engine binds neither. It binds the endpoint ENI's address alone, so the port is not open on them at all rather than open and gated. The system NIC additionally carries a dedicated region-wide security group that authorizes no ingress; agent traffic is outbound and its replies are matched by connection state. PostgreSQL's generated pg_hba.conf admits the customer VPC's address range and nothing wider, so a client outside the VPC is refused authentication even if it somehow reached the socket. MariaDB has no equivalent file: its analogue is the host pattern on a grant, which lives in the data volume and would follow a restore into another VPC, so for that engine the bind is the whole boundary.

Everything below applies to both engines unless it names one. Engine is fixed at create — there is no in-place engine change, no cross-engine snapshot restore and no migration between the two.

	postgres	mariadb
Version	18	11.8
EngineVersion accepted	18, or omitted	11.8, or omitted
Default port	5432	3306
Parameter-group family	postgres18	mariadb11.8
Implicit default group	default.postgres18	default.mariadb11.8
DBName maximum	63 characters	64 characters
MasterUsername maximum	63 characters	80 characters
Reserved usernames	postgres, rdsadmin, rds_superuser, and the pg_ prefix	root, mysql, mariadb.sys, rdsadmin, PUBLIC, and the mysql. and mariadb. prefixes
Catalog size	53 parameters	56 parameters

mysql is not an accepted engine and is not an alias for mariadb. MariaDB is offered under its own AWS engine name, exactly as AWS RDS offers it. Naming mysql fails with InvalidParameterValue; a client — including Terraform's aws_db_instance — must set engine = "mariadb". Aliasing would report an engine and a version the instance is not running, and the discrepancy would propagate into DescribeDBInstances, parameter-group families and snapshot metadata.

• Engine version: pinned per engine, and an EngineVersion naming anything but the pin is rejected. A minor version such as 18.4 or 11.8.8 is rejected as well: the AMI does not promise any particular minor, so accepting one would be a promise the platform cannot keep. There is no in-place upgrade.
• Instance classes: db.t3.{micro,small,medium,large} and db.m5.{large,xlarge} — a naming facade over the platform's EC2 sizing table. Any other class is rejected at create. db.X is the EC2 instance type X, so a client reads a class's vCPU count and memory from ec2:describe-instance-types for the identically named type; an orderable option carries neither, because AWS's own shape has no field for them. describe-orderable-db-instance-options reports only the classes the cluster's nodes can actually run, which on a Graviton-only cluster is none of the six — a class that would fail at launch is not offered.
• Storage: gp3 only, 20–65536 GiB, always encrypted with the cluster key. Grow-only, and a grow is stop/start with downtime — the volume cannot be resized while attached.
• DBName: optional; an omitted name creates no initial database. When supplied it must begin with a letter and hold only letters, digits and underscores, within the per-engine limit above. The rule is narrower than either engine's own, because the guest interpolates the name into a CREATE DATABASE.
• TLS v1.3: required by default on both engines — rds.force_ssl on postgres18 and require_secure_transport on mariadb11.8, both boolean, modifiable and dynamic, both defaulting to 1. Setting either to 0 in a parameter group restores plaintext without a reboot. Defaulting on is a deliberate divergence for MariaDB, which AWS leaves off.
• What enforcement requires is that the connection is encrypted, not that the client validates the certificate: sslmode=require satisfies it and does no validation at all. The engine serves a per-instance certificate signed by the cluster CA carrying both the ENI IP and the DNS name in its SAN set, so sslmode=verify-full — or MariaDB's --ssl-ca plus --ssl-verify-server-cert — verifies by name or by address; a client that wants to fetches the CA from the IMDS responder at http://169.254.169.254/spinifex/ca.pem. A deployment holding no cluster CA cannot serve TLS at all, so a group asking for enforcement is refused with InvalidParameterCombination, and the guest will not start an engine that could not honour it.go-sql-driver/mysql (no TLS without a tls DSN parameter) do not.
• PostgreSQL master user: administrative but not a PostgreSQL superuser, as on AWS. It gets CREATEDB and CREATEROLE, owns the initial database, and holds rds_superuser (pg_monitor, pg_signal_backend, pg_checkpoint) with ADMIN OPTION, so it can grant that set onward. It cannot COPY ... FROM/TO PROGRAM, touch server-side files, run ALTER SYSTEM, or install untrusted extensions — those need the cluster superuser, which no customer credential is. Trusted extensions install normally in a database it owns.
• MariaDB master user: created as '<name>'@'%' with global RELOAD, PROCESS, SHOW DATABASES, REPLICATION SLAVE and REPLICATION CLIENT, while application DML, DDL, routines, events, triggers and delegation are granted only on customer databases. MariaDB has no deny or partial-revoke mechanism, so granting those object privileges ON *.* would also grant them on mysql.global_priv and let the master acquire SUPER, FILE or SHUTDOWN. The initial DBName is granted automatically. Create another customer database with CALL _spinifex_rds.create_database('name'); plain CREATE DATABASE, arbitrary account administration and access to mysql.* are intentionally unavailable on the stock MariaDB build.
• MariaDB storage engine: default_storage_engine is platform-owned and pinned to InnoDB, and is absent from the parameter catalog — a deliberate divergence from AWS, which exposes it as modifiable. Only InnoDB keeps a redo log, so only InnoDB is covered by the snapshot guarantee. An explicit ENGINE=Aria or ENGINE=MyISAM on CREATE TABLE still works — the limitation is documented rather than prevented — but such a table has no redo log, so a snapshot taken without a successful quiesce can leave it inconsistent with no way back. Use InnoDB, as AWS also recommends.
• Backups: daily COW snapshots of the data volume inside PreferredBackupWindow. Retention defaults to 7 days and caps at 7; 0 disables automated backups. No point-in-time recovery, and MariaDB's binary log is off.
• Availability: single-AZ. Engine crashes restart in-guest and VM crashes on a live host restart via ec2-health-restart, but an instance whose host is lost is reported failed with a reason in StatusInfos and needs operator recovery.

Statuses: creating, available, modifying, backing-up, rebooting, stopping, stopped, starting, deleting, failed. (recovering is defined in the state machine but unreachable until auto-recovery lands.)

RDS — DB Instances

Command	Implemented Flags	Missing Flags	Status
create-db-instance	--db-instance-identifier, --engine (postgres, mariadb), --engine-version (the engine's pin, or omitted), --db-instance-class, --allocated-storage (20–65536 GiB), --storage-type (gp3), --master-username, --master-user-password, --db-name (letters, digits and underscores, beginning with a letter, max 63 on postgres and 64 on mariadb), --port (1150–65535), --db-subnet-group-name (unnamed → the account's default VPC subnet), --vpc-security-group-ids, --db-parameter-group-name, --backup-retention-period (0–7, default 7), --preferred-backup-window, --preferred-maintenance-window (unnamed → assigned from the identifier), --deletion-protection, --tags, --storage-encrypted (true only)	--auto-minor-version-upgrade, --copy-tags-to-snapshot, Performance Insights / Enhanced Monitoring flags (all accepted, no-op); see "Rejected Parameters" for those that fail loudly	DONE
describe-db-instances	--db-instance-identifier	--filters, --max-records, --marker (parsed, not applied)	DONE
modify-db-instance	--db-instance-identifier, --master-user-password, --allocated-storage (grow only, stop/start), --db-instance-class (VM replace), --db-parameter-group-name, --vpc-security-group-ids, --deletion-protection, --backup-retention-period, --preferred-backup-window, --preferred-maintenance-window, --apply-immediately	--new-db-instance-identifier, --engine-version, --db-port-number, --db-subnet-group-name, --max-allocated-storage, --ca-certificate-identifier (all rejected, not ignored)	DONE
delete-db-instance	--db-instance-identifier, --skip-final-snapshot, --final-db-snapshot-identifier (exactly one is required)	--delete-automated-backups (no-op — automated snapshots are always purged with the instance)	DONE — DeletionProtection blocks the call; a final snapshot retains the data volume until that snapshot is deleted
reboot-db-instance	--db-instance-identifier	--force-failover (rejected — single-AZ)	DONE — applies parameters marked pending-reboot
stop-db-instance	--db-instance-identifier	--db-snapshot-identifier (rejected)	DONE — the data volume and endpoint ENI are retained
start-db-instance	--db-instance-identifier	—	DONE — fresh VM, same data volume, same ENI and IP

RDS — Snapshots & Restore

Command	Implemented Flags	Missing Flags	Status
create-db-snapshot	--db-snapshot-identifier, --db-instance-identifier, --tags	—	DONE — the agent quiesces the engine first (pg_backup_start on postgres, BACKUP STAGE on mariadb); a failed quiesce falls back to a crash-consistent snapshot and writes a DescribeEvents warning naming what that engine does and does not recover
describe-db-snapshots	--db-snapshot-identifier, --db-instance-identifier, --snapshot-type (manual, automated)	--filters, --dbi-resource-id, --include-shared, --include-public (rejected), --max-records, --marker	DONE
delete-db-snapshot	--db-snapshot-identifier	—	DONE — refused while a restored volume still references the snapshot; the last snapshot released reclaims a retained data volume
restore-db-instance-from-db-snapshot	--db-instance-identifier, --db-snapshot-identifier, --db-instance-class, --allocated-storage (≥ the snapshot's), --storage-type (gp3), --port, --db-subnet-group-name, --vpc-security-group-ids, --db-parameter-group-name, --deletion-protection, --tags, --engine (must match the snapshot)	--db-name (rejected when it differs from the snapshot's), plus the create rejections	DONE — unnamed fields are inherited from the snapshot; the master password comes from the restored datadir
copy-db-snapshot	—	all	NOT STARTED (InvalidAction)
modify-db-snapshot-attribute / describe-db-snapshot-attributes	—	all	NOT STARTED (InvalidAction) — cross-account snapshot sharing

RDS — Automated Backups

Command	Implemented Flags	Missing Flags	Status
describe-db-instance-automated-backups	--db-instance-identifier	--filters, --dbi-resource-id, --db-instance-automated-backups-arn (rejected)	DONE — one entry per instance with backups enabled; the individual snapshots are listed by describe-db-snapshots --snapshot-type automated

RestoreWindow and LatestRestorableTime are deliberately absent: backups are discrete daily snapshots, and reporting a window would imply recovery to any instant inside it.

RDS — Subnet Groups

Command	Implemented Flags	Missing Flags	Status
create-db-subnet-group	--db-subnet-group-name, --db-subnet-group-description, --subnet-ids, --tags	—	DONE (any subnet count — every subnet reports the single spinifexz1 zone)
describe-db-subnet-groups	--db-subnet-group-name	--filters, --max-records, --marker	DONE
delete-db-subnet-group	--db-subnet-group-name	—	DONE — refused while any DB instance still names the group, including one that is deleting
modify-db-subnet-group	—	all	NOT STARTED (InvalidAction)

RDS — Parameter Groups

Command	Implemented Flags	Missing Flags	Status
create-db-parameter-group	--db-parameter-group-name, --db-parameter-group-family (postgres18 or mariadb11.8), --description, --tags	—	DONE — names beginning with default. are reserved. --db-parameter-group-family defaults to postgres18 when omitted, so a MariaDB group must name its family explicitly or the mismatch surfaces one call later, at create-db-instance
describe-db-parameter-groups	--db-parameter-group-name	--filters, --max-records, --marker	DONE — every engine's implicit default group is always listed, so both default.postgres18 and default.mariadb11.8 appear in every account
modify-db-parameter-group	--db-parameter-group-name, --parameters (ParameterName, ParameterValue, ApplyMethod)	—	DONE — the whole batch is validated before anything is written and propagated to attached instances; immediate on a static parameter is rejected, as AWS does
describe-db-parameters	--db-parameter-group-name, --source (user, engine-default)	--filters, --max-records, --marker	DONE — the catalog of the group's own family: 53 parameters on postgres18, 56 on mariadb11.8. Memory defaults are computed per instance class and reported as literals
delete-db-parameter-group	--db-parameter-group-name	—	DONE — refused for a default group and while any instance references it
reset-db-parameter-group	—	all	NOT STARTED (InvalidAction)

A group takes effect on an instance when modify-db-instance --db-parameter-group-name attaches it — immediately with --apply-immediately, otherwise at the next maintenance window. Later group edits propagate to every attached instance. Dynamic parameters are written into the engine's config and reloaded live; static ones are recorded pending-reboot and applied by reboot-db-instance.

A group's family must match the instance's engine. A postgres18 group cannot be attached to a MariaDB instance and a mariadb11.8 group cannot be attached to a PostgreSQL one, on create, on modify, on a deferred apply or on a snapshot restore; the mismatch is refused with InvalidParameterCombination. The catalogs are per-family too, so modify-db-parameter-group will not store one engine's parameter name into the other's group. Each catalog is a curated, validated subset rather than the engine's full variable set, and the platform-owned settings — the port, the datadir, the socket, the bind address, the certificate and key paths, and on MariaDB secure_file_priv, log_bin, default_storage_engine and the InnoDB buffer-pool chunk settings — are absent from it rather than present-and-unmodifiable, because they are not the customer's to set. The exception is a setting AWS exposes as modifiable that this platform pins: the TLS floor is present as ssl_min_protocol_version and tls_version and reported as not modifiable, so a Terraform plan that sets it is refused as policy rather than reading as a platform gap.

RDS — Tags

Command	Implemented Flags	Missing Flags	Status
add-tags-to-resource	--resource-name (db, snapshot, subgrp, pg ARNs), --tags	—	DONE
remove-tags-from-resource	--resource-name, --tag-keys	—	DONE
list-tags-for-resource	--resource-name	--filters	DONE — describe-db-instances reports the same tags in TagList

RDS — Events

Command	Implemented Flags	Missing Flags	Status
describe-events	--source-type (db-instance, db-snapshot), --source-identifier, --event-categories, --duration, --start-time, --end-time, --max-records	--marker	DONE — 100-event ring per resource, 14-day retention, one-hour default window

RDS — Engine Versions & Orderable Options

Command	Implemented Flags	Missing Flags	Status
describe-db-engine-versions	--engine, --engine-version, --db-parameter-group-family, --filters (engine, engine-version, db-parameter-group-family, status)	--default-only, --include-all, --list-supported-character-sets, --list-supported-timezones (accepted, provably no-ops), --max-records (accepted, no effect), --marker (rejected)	DONE — one row per engine
describe-orderable-db-instance-options	--engine (required), --engine-version, --db-instance-class, --license-model, --vpc, --filters (engine, engine-version, db-instance-class, license-model, vpc)	--max-records (accepted, no effect), --marker, --availability-zone-group (both rejected)	DONE — the engines crossed with the classes the cluster's nodes can run

Both are read from the tables the create path validates against, so an engine pin bump or a class-map change moves these answers with it. Neither paginates — at most 2 engine versions and 12 orderable options — so no Marker is ever returned, and one supplied is rejected rather than answered as page one, where the other RDS describes parse and ignore it.

• AvailabilityZones and AvailableProcessorFeatures are always empty, and are gaps only in appearance: --availability-zone is rejected on create and --processor-features is ignored, so naming either here would advertise a knob that does nothing.
• Only the classes the cluster's nodes can run are offered. A cluster that answers nothing is ServerInternal, never the full class list; one that answers and runs none of them is an empty list. Free capacity is deliberately not consulted — exhausted capacity is a create-time failure, and filtering on it would make a Terraform data source return a different answer on every plan.

Two further deliberate divergences: the four describe-db-engine-versions booleans are accepted as provable identities rather than the "parsed, not applied" the other describes document, because one version per engine and two empty lists leave them nothing to change; and --filters on describe-orderable-db-instance-options is applied where AWS documents it unsupported. Both actions reject an unrecognised filter name, and a non-boolean vpc value, with InvalidParameterValue rather than ignoring it — stricter than AWS rather than looser.

RDS — Rejected Parameters

Policy: a parameter whose omission would create a false safety, security or availability guarantee is rejected with InvalidParameterValue rather than silently dropped. Parameters that are merely inert — AutoMinorVersionUpgrade, CopyTagsToSnapshot, DeleteAutomatedBackups, Performance Insights and Enhanced Monitoring fields — are accepted as no-ops.

Parameter	Why it is rejected
MultiAZ=true	Single-AZ platform; a standby would not exist
PubliclyAccessible=true	The endpoint is a private VPC address
StorageEncrypted=false	Unencrypted storage is not offered
EnableIAMDatabaseAuthentication	IAM database authentication is not implemented
Iops, StorageThroughput, StorageType ≠ gp3	Provisioned performance classes are not implemented
KmsKeyId, TdeCredentialArn	Storage is encrypted with the cluster key, not a customer-managed one
AvailabilityZone	The platform exposes a single zone
AvailabilityZoneGroup (orderable options)	It selects a zone or local-zone group, and naming a zone is already refused
DBSecurityGroups	EC2-Classic security groups — use VpcSecurityGroupIds
DBClusterIdentifier, DBClusterSnapshotIdentifier	Clustered engines are not offered
EnableCloudwatchLogsExports	Log export is not implemented
EngineVersion other than the engine's pin, Engine on modify	No in-place engine or version change
Engine=mysql (and Aurora engines)	Oracle MySQL is not offered; mariadb is a distinct engine, not an alias for it
NewDBInstanceIdentifier	The identifier is the DNS label and the KV key
DBPortNumber, DBSubnetGroupName on modify	Both would move the endpoint
MaxAllocatedStorage	Storage autoscaling is not implemented
ManageMasterUserPassword, RotateMasterUserPassword	Secrets Manager integration is not offered
CACertificateIdentifier	The serving certificate is minted from the cluster CA
Domain, DomainFqdn	Active Directory domain join is not offered
OptionGroupName	Option groups are not offered
CustomIamInstanceProfile	The DB VM's instance profile is platform-owned
EnableCustomerOwnedIp	An Outposts feature
ForceFailover (reboot)	No standby to fail over to
DBSnapshotIdentifier (stop)	Snapshot-on-stop is not implemented

RDS — Not Yet Implemented

Recognised actions below return OperationNotSupported, so a client sees "not offered" rather than a typo'd action name. Everything else in the rds namespace returns InvalidAction.

Feature	Actions	Priority	Status
Oracle MySQL engine	—	Low	NOT PLANNED — MariaDB is offered instead, under its own mariadb engine name; see the note above on why it is not aliased onto mysql
Point-in-time recovery (WAL archiving to predastore on postgres, binlog on mariadb)	restore-db-instance-to-point-in-time	High	NOT STARTED
Auto-recovery from node loss (failure is detected, not repaired)	—	High	NOT STARTED
Read replicas	create-db-instance-read-replica, promote-read-replica	Medium	NOT STARTED
Aurora / DB clusters	create-db-cluster, modify-db-cluster, delete-db-cluster, describe-db-clusters, failover-db-cluster	Low	NOT STARTED
Option groups	create-option-group, modify-option-group, delete-option-group, describe-option-groups	Low	NOT STARTED
Multi-AZ standby, online (no-downtime) storage grow, storage autoscaling, IAM database auth, Performance Insights, Enhanced Monitoring, log exports, per-tenant private DNS zones, enforced TLS, cross-engine snapshot restore, in-place migration between the two engines	—	—	NOT STARTED

IAM: AmazonRDSFullAccess and AmazonRDSReadOnlyAccess are available as managed policies. They grant rds: verb prefixes rather than rds:*, because rds:* would also appear to grant the internal agent actions the gateway reserves for a DB VM's own role.

---

ECR (Elastic Container Registry)

Repository metadata is served over the AWS API on the gateway endpoint; image data moves over the OCI Distribution /v2/ endpoint on that same host, authenticated by the bearer token get-login-password mints.

Command	Status
create-repository, delete-repository, describe-repositories	DONE
describe-images, list-images, batch-get-image, batch-delete-image, put-image	DONE
put-image-tag-mutability	DONE
get-authorization-token	DONE — mints the short-lived bearer token used by docker login
get-repository-policy, set-repository-policy, delete-repository-policy	DONE
get-lifecycle-policy, put-lifecycle-policy, delete-lifecycle-policy, get-lifecycle-policy-preview, start-lifecycle-policy-preview	DONE
list-tags-for-resource	DONE
tag-resource, untag-resource	NOT STARTED (registered stub)
batch-check-layer-availability, initiate-layer-upload, upload-layer-part, complete-layer-upload, get-download-url-for-layer	NOT STARTED (registered stub) — layer transfer is served by the OCI /v2/ endpoint
describe-registry, get-registry-policy, put-registry-policy, put-replication-configuration, replicate-image	NOT STARTED (registered stub) — single-registry deployment, no cross-region replication
list-repositories	NOT STARTED (registered stub) — use describe-repositories
start-image-scan, describe-image-scan-findings, put-image-scanning-configuration, get-image-scanning-configuration, get-registry-scanning-configuration, put-registry-scanning-configuration, batch-get-repository-scanning-configuration	NOT STARTED (OperationNotSupported) — vulnerability scanning is not offered
Pull-through cache rules, repository creation templates, image replication status	NOT STARTED (InvalidAction)

---

ECS (Elastic Container Service)

Clusters, services and tasks run on EC2 container instances; there is no Fargate launch type.

Command	Status
create-cluster, delete-cluster, describe-clusters, list-clusters	DONE
create-service, update-service, delete-service, describe-services, list-services	DONE
register-task-definition, deregister-task-definition, describe-task-definition, list-task-definitions	DONE
run-task, start-task, stop-task, describe-tasks, list-tasks	DONE
register-container-instance, deregister-container-instance, describe-container-instances, list-container-instances, update-container-instances-state	DONE
create-capacity-provider, delete-capacity-provider, describe-capacity-providers, put-cluster-capacity-providers	DONE
submit-task-state-change	DONE — reported by the in-guest agent
tag-resource, untag-resource, list-tags-for-resource	DONE
update-cluster, list-account-settings, put-account-setting, list-services-by-namespace, list-task-definition-families	NOT STARTED (registered stub)
Task sets (blue/green): create-task-set, update-task-set, delete-task-set, describe-task-sets, update-service-primary-task-set	NOT STARTED (InvalidAction)
execute-command, get-task-protection, update-task-protection	NOT STARTED (InvalidAction)
Attributes: put-attributes, delete-attributes, list-attributes	NOT STARTED (InvalidAction)
update-capacity-provider, update-cluster-settings, update-container-agent	NOT STARTED (InvalidAction)
delete-account-setting, put-account-setting-default, delete-task-definitions	NOT STARTED (InvalidAction)
discover-poll-endpoint, submit-container-state-change, submit-attachment-state-changes	NOT STARTED (InvalidAction)

---

EKS (Elastic Kubernetes Service)

REST-JSON, dispatched by HTTP method + path rather than by an action parameter, so the routing table in gateway/eks.go is the authoritative list. Clusters run K3s on control-plane VMs; nodegroups are EC2 instances joined to them.

Command	Status
create-cluster, describe-cluster, list-clusters, delete-cluster	DONE
create-nodegroup, describe-nodegroup, list-nodegroups, delete-nodegroup, update-nodegroup-config, update-nodegroup-version	DONE
create-addon, describe-addon, describe-addon-versions, list-addons, update-addon, delete-addon	DONE
create-access-entry, describe-access-entry, list-access-entries, update-access-entry, delete-access-entry	DONE
associate-access-policy, disassociate-access-policy, list-associated-access-policies, list-access-policies	DONE
tag-resource, untag-resource, list-tags-for-resource	DONE for cluster ARNs; other EKS resource ARNs return NotImplemented
update-cluster-config, update-cluster-version	NOT STARTED (NotImplemented)
Identity provider configs: associate-identity-provider-config, describe-identity-provider-config, list-identity-provider-configs, disassociate-identity-provider-config	NOT STARTED (NotImplemented)

Four routes on this surface are internal, not AWS actions, and are not callable with an ordinary tenant credential: PublishInternal (the control-plane VM broker, relaying bootstrap/state POSTs onto eks.bus.*/eks.state.*), WebhookTokenReview (the eks-token-webhook posting bearer tokens for host-side resolution), ListInternalAddons (the on-VM addon-sync agent fetching its staged manifests) and GetRecoveryDirective (the on-VM k3s-recovery agent reading its per-member directive at boot). They name the target account in the path rather than taking it from the credential, so each requires a control-plane instance-role session bound to the named cluster — a broad eks:* grant does not reach them.

Disaster recovery for a lost single control plane is an operator command, not an API call: see spx admin eks restore-snapshot above.

---

Resource Groups Tagging API (tagging)

JSON-1.1, dispatched from the action suffix of X-Amz-Target: ResourceGroupsTaggingAPI_20170126.<Action>.

Command	Status
get-resources	DONE — authorizes against "*", which is correct here rather than a fail-open: the Resource Groups Tagging API defines no resource types of its own. A two-way completeness test pins that, so a later action cannot be added to this surface with a silent account-wide grant
get-tag-keys, get-tag-values, tag-resources, untag-resources	NOT STARTED (InvalidAction)

---

CloudWatch (Basic Monitoring)

Command	Implemented Flags	Missing Flags	Status
put-metric-data	—	--namespace, --metric-data	NOT STARTED
get-metric-statistics	—	--namespace, --metric-name, --start-time, --end-time, --period, --statistics, --dimensions	NOT STARTED
list-metrics	—	--namespace, --metric-name, --dimensions, --recently-active	NOT STARTED
describe-alarms	—	--alarm-names, --alarm-name-prefix, --state-value, --action-prefix	NOT STARTED
put-metric-alarm	—	--alarm-name, --namespace, --metric-name, --statistic, --period, --evaluation-periods, --threshold, --comparison-operator, --alarm-actions, --dimensions	NOT STARTED
delete-alarms	—	--alarm-names	NOT STARTED

---

Auto Scaling

Command	Implemented Flags	Missing Flags	Status
create-auto-scaling-group	—	--auto-scaling-group-name, --launch-template, --min-size, --max-size, --desired-capacity, --vpc-zone-identifier, --target-group-arns, --health-check-type, --health-check-grace-period, --tags	NOT STARTED
update-auto-scaling-group	—	--auto-scaling-group-name, --min-size, --max-size, --desired-capacity, --launch-template, --health-check-type, --health-check-grace-period	NOT STARTED
delete-auto-scaling-group	—	--auto-scaling-group-name, --force-delete	NOT STARTED
describe-auto-scaling-groups	—	--auto-scaling-group-names, --filters, --max-records	NOT STARTED
set-desired-capacity	—	--auto-scaling-group-name, --desired-capacity, --honor-cooldown	NOT STARTED
describe-auto-scaling-instances	—	--instance-ids, --max-records	NOT STARTED
put-scaling-policy	—	--auto-scaling-group-name, --policy-name, --policy-type, --target-tracking-configuration, --scaling-adjustment, --cooldown	NOT STARTED
delete-scaling-policy	—	--auto-scaling-group-name, --policy-name	NOT STARTED