Commit Graph

68 Commits

Author SHA1 Message Date
joshsiv-mulga 9e77181f21 Merge origin/dev into refactor/spinifex-package-boundaries 2026-10-08 09:28:43 +10:00
Julian Sommer b3b066562e chore: add comment to all exported types 2026-10-07 13:23:42 +10:00
joshsiv-mulga 7003168bc5 refactor: move network packages under the network domain
Move spinifex/network/* to spinifex/domains/network/* as one enclosing
move so the L0-L5 layer invariants keep classifying every package from
a single root. Package names and dependency direction are unchanged;
invariant scan roots, the RG-10 sudo allow-list path, CODEOWNERS and
the repo-root walk in the IMDS netns unit test follow the new path.
2026-10-07 08:15:48 +10:00
joshsiv-mulga 83615f1f47 refactor: move aws xml response helpers 2026-10-02 14:26:42 +10:00
joshsiv-mulga 6ac02a8865 refactor: move cluster replication policy to state foundation 2026-10-02 09:43:17 +10:00
joshsiv-mulga dbc62562c8 refactor: move object storage adapter to providers 2026-10-01 14:16:25 +10:00
joshsiv-mulga 3f91a3dbcd Merge remote-tracking branch 'origin/dev' into refactor/spinifex-package-boundaries
# Conflicts:
#	spinifex/daemon/daemon.go
#	spinifex/foundation/aws/arn/arn.go
#	spinifex/foundation/aws/arn/arn_test.go
#	spinifex/foundation/state/kvlease/bucket.go
#	spinifex/foundation/state/kvlease/bucket_test.go
#	spinifex/foundation/state/kvstore/wait.go
#	spinifex/foundation/state/kvstore/wait_test.go
#	spinifex/foundation/state/kvutil/bucket_creation_invariants_test.go
#	spinifex/foundation/state/kvutil/export_test.go
#	spinifex/foundation/state/kvutil/kvutil.go
#	spinifex/foundation/state/kvutil/kvutil_test.go
#	spinifex/foundation/state/kvutil/replicas.go
#	spinifex/foundation/state/kvutil/replicas_health_test.go
#	spinifex/foundation/state/kvutil/replicas_test.go
#	spinifex/foundation/state/kvutil/state_test.go
#	spinifex/gateway/auth.go
#	spinifex/gateway/bedrock/logging.go
#	spinifex/gateway/ec2_authz_test.go
#	spinifex/gateway/ecrapi/authz.go
#	spinifex/gateway/eks.go
#	spinifex/gateway/gateway_test.go
#	spinifex/handlers/ec2/eigw/service_impl.go
#	spinifex/handlers/ec2/igw/service_impl.go
#	spinifex/handlers/ec2/instance/service_impl.go
#	spinifex/handlers/ec2/launchtemplate/service_impl_test.go
#	spinifex/handlers/ec2/placementgroup/service_impl.go
#	spinifex/handlers/ec2/placementgroup/service_impl_test.go
#	spinifex/handlers/ec2/routetable/service_impl.go
#	spinifex/handlers/ec2/spotinstance/service_impl.go
#	spinifex/handlers/ec2/volume/service_impl.go
#	spinifex/handlers/ec2/vpc/eni.go
#	spinifex/handlers/ec2/vpc/security_group.go
#	spinifex/handlers/ec2/vpc/service_impl.go
#	spinifex/handlers/ecs/tags.go
#	spinifex/handlers/eks/k3s_ha_control_plane.go
#	spinifex/handlers/iam/groups.go
#	spinifex/handlers/iam/retry.go
#	spinifex/handlers/rds/subnetgroup_test.go
#	spinifex/handlers/sts/assume_role.go
#	spinifex/ingress/aws/query/parser_fuzz_test.go
#	spinifex/runtime/compute/cache/cache_test.go
#	spinifex/runtime/roles/nats/servername.go
#	spinifex/services/awsgw/awsgw.go
#	spinifex/services/viperblockd/volume_dirty.go
#	spinifex/utils/account.go
2026-10-01 13:31:45 +10:00
Julian Sommer 810aa019bf chore: use strings.Cut 2026-09-30 11:48:27 +10:00
Julian Sommer 610ee9bab0 chore: use slices.Sorted 2026-09-30 11:44:17 +10:00
joshsiv-mulga d87326f644 refactor: move listener inventory to test kit 2026-09-29 14:56:39 +10:00
joshsiv-mulga 94ddc91a18 refactor: move EBS provider contract to providers 2026-09-29 14:48:08 +10:00
joshsiv-mulga cbbf252e68 refactor: consolidate test helpers in internal testkit 2026-09-29 14:39:42 +10:00
Julian Sommer 410faf3c83 fix: EC2 UnauthorizedOperation, MaxResults ranges and tag resource types as AWS answers them (#1134) 2026-09-29 12:06:48 +10:00
Julian Sommer bfad1dc004 test: target fixture resources in generated conformance requests
Salts restart per operation, so tag, untag and update cases only reached a resource when the same-numbered create case had made one. Create an IAM, ECS and ECR fixture per run and point every operation but a create or delete at it, and give ECR's policy and manifest documents values AWS accepts.
2026-09-29 11:14:37 +10:00
Julian Sommer b3be052311 test: remove integration conformance suite artefacts
Clear the warnings the conformance suite caused itself, so what remains is a real divergence from AWS:

- Seed conditionally required members into the base request of RDS CreateDBInstance and RestoreDBInstanceFromDBSnapshot, ECS CreateCapacityProvider and CreateService, and EC2 CreateCapacityReservation.
- Give loosely modelled members values AWS accepts: ELBv2 ProtocolVersion and Matcher.HttpCode, EKS policyArn, addonName and access-entry type, EC2 SourceRegion and STS AccessKeyId.
- Import a CA-signed certificate, key and chain in ACM ImportCertificate.
- Rewrite ACM's lookahead domain pattern as RE2 so RequestCertificate is generated, and give the integration ACM service a tenant CA permitting example.com.
- Run the service daemon-lite behind the cross-tenant boundary test.
- Accept ELBv2's common parameter validation errors in responses.
2026-09-29 11:05:22 +10:00
joshsiv-mulga 573ee67610 refactor: move load generator to internal testkit 2026-09-29 10:40:47 +10:00
Julian Sommer 2898181837 fix: test packages 2026-09-28 16:01:49 +10:00
Julian Sommer 2d2d2f27ed feat: model-driven request conformance in the integration suite (#1122) 2026-09-28 14:14:15 +10:00
Julian Sommer 0c3403889c feat: load AWS conformance models from api-models-aws (#1121) 2026-09-28 12:21:05 +10:00
Julian Sommer c473365f1a refactor: move new test files to external test packages
Tests added since main used in-package test packages, failing the
test-package check. Convert them to <pkg>_test with export_test.go
hooks; the ecs-agent main package keeps an in-package marker.
2026-09-28 10:54:58 +10:00
Josh Siviour 451df1a8cd docs: publish AWS operation coverage inventory (#1075) 2026-09-17 21:33:53 +10:00
Julian Sommer c0d54e6131 docs: publish one column of implemented operations and nothing else 2026-09-15 11:27:17 +10:00
Julian Sommer 881a66ab57 docs: publish only the operations the platform serves 2026-09-15 11:22:17 +10:00
Julian Sommer b89334d76f docs: keep the not-applicable operations out of the internal gap list
The internal report listed every unimplemented operation together, so the 87
EC2 operations declared not applicable sat in the same list as the work, where
anything reading it for candidates would pick them up.

They are now listed apart, grouped under the reason they were declared with,
and the counts line reports them as not applicable rather than as gaps.
2026-09-15 11:04:35 +10:00
Julian Sommer 3b523498fc docs: stop publishing planned RDS work as never
Multi-AZ clusters, read replicas and point-in-time restore were declared not
applicable, which says the platform will never serve them. All three are
planned, so the pages claimed something the roadmap contradicts.

Only a declaration now publishes that claim. A handler that refuses an
operation today establishes nothing about tomorrow, so it reads as the gap it
is rather than a permanent one.
2026-09-15 10:56:29 +10:00
Julian Sommer c24636c734 docs: put the not-applicable operations back in the one table
Splitting them into a second table repeated a reason on every row that shared
it and separated an operation from the rest of the surface it belongs to. One
table carries every operation the platform answers, served or refused, and the
reasons return to numbered footnotes below it.
2026-09-15 10:53:37 +10:00
Julian Sommer c8d42ac160 docs: publish S3 API coverage
S3 was the one served surface with no coverage page. The gateway has no
operation-name dispatch table for it, so there was nothing to compare against
the model.

Predastore now declares the operations each of its S3 routes answers, so the
generator compares that inventory the same way it compares the other nine
services. The page credits Predastore rather than Spinifex, and its intro
separates what is routed from what conforms, pointing at the ceph/s3-tests
report for the second.
2026-09-15 10:51:19 +10:00
Julian Sommer a3fbf82b9b chore: fix coverage text 2026-09-15 10:37:02 +10:00
Julian Sommer 05830f3499 docs: total the served operations on the coverage index
The index listed a count per service and left the reader to add them up. It
now opens with the total and closes the table with it, counting only the
services it links.
2026-09-15 10:31:01 +10:00
Julian Sommer e2fb37c0ac docs: publish only the AWS operations Spinifex serves
The coverage pages led with a percentage and an "x of y" count, and every
table listed the modelled operations Spinifex does not implement, which reads
as a scorecard against AWS rather than a statement of what the platform serves.

A published page now opens with the operation count alone, tables the
operations the gateway serves, and carries a separate not-applicable table with
each reason inline instead of footnotes. The unimplemented operations and the
counts move to docs/coverage/internal-report.md, which no docs-site slug points
at. A service with no implemented operations has no publishable page.
2026-09-15 10:28:35 +10:00
Julian Sommer ba98509c5b fix: simplify mulga-docs build step by collapsing coverage paths + hardware 2026-09-14 12:41:02 +10:00
Julian Sommer 613231fd18 feat: hide internal commands from compatibility page 2026-09-14 12:13:32 +10:00
Julian Sommer 19a047bccb fix: footnotes instead of notes for compatibility 2026-09-14 12:05:32 +10:00
Julian Sommer bba735497f fix: add not applicable to some actions 2026-09-14 11:58:21 +10:00
Julian Sommer 29faaa78fd chore: update compatibility pages 2026-09-14 11:49:46 +10:00
Julian Sommer fb6494da01 feat: coverage docs 2026-09-14 11:42:01 +10:00
Julian Sommer 2265aee120 feat: add eks to aws coverage 2026-09-14 11:30:55 +10:00
Josh Siviour 31eb3bc672 fix: fail a credential refresh instead of extending a dead key's expiry (#995)
ec2rolecreds implements aws.HandleFailRefreshCredentialsCacheStrategy, which
aws.CredentialsCache prefers over returning the refresh error. When the cached
credential is expired or within five minutes of expiring and the refresh fails,
it hands back the same access key, secret and session token with Expires pushed
a random five to fifteen minutes out, reported as a success. Every later
refresh fails the same way, so ecs-agent signs with a credential the control
plane has already rejected, indefinitely, and nothing surfaces it.

The cache picks its strategies by asserting on the provider's dynamic type, so
returning aws.CredentialsProvider from NewProvider does not disarm it. Wrap the
provider in a type whose method set omits HandleFailToRefresh, and the cache
falls through to returning the error. AdjustExpiresBy is forwarded on purpose:
it only fires under an ExpiryWindow, which no caller sets today, but dropping
it silently would change behaviour for whoever sets one.

Fixed in imdscreds rather than at the one exposed call site, so a future caller
that wraps this provider in a cache is safe by construction.
ecr-credential-provider was never affected: it calls Retrieve uncached.

Two comments claimed the hazard was ec2rolecreds capping Expires to an hour.
That cap only ever shortens a too-far-future expiry and is unrelated.
2026-09-04 12:00:31 +10:00
Julian Sommer 95afec6a76 refactor: use bluebottle/pkg/fipsboot for the FIPS 140-3 startup guard (#911) 2026-08-27 14:38:36 +10:00
Julian Sommer 97acfbbd80 refactor: use bluebottle/pkg/tlsconfig for the TLS curve allowlist (#910) 2026-08-27 14:31:29 +10:00
Julian Sommer 8e01e76dc0 fix: disable embedlit 2026-08-20 10:52:03 +10:00
Julian Sommer f21db26816 chore: block panic 2026-08-19 15:03:47 +10:00
Julian Sommer 6b71fb876d fix: use stdlib sort 2026-08-13 11:22:15 +10:00
Josh Siviour fd8be8a814 refactor(test): route duplicated NATS bootstraps through testutil (#776)
* refactor(test): route duplicated NATS bootstraps through testutil

Thirteen test sites hand-rolled a NATS or JetStream server bootstrap that
spinifex/testutil already provides, and one seedKV clone duplicated
testutil.SeedKV. Route them through the shared helpers.

Sites whose options genuinely diverge are left alone: fixed ports needed for
reconnect and restart scenarios, custom TLS and token auth, a custom
MaxPayload, and TestMain entry points that have no testing.TB for t.Cleanup.
The natgw and routetable seedKV clones use CreateOrUpdateKeyValue rather than
CreateKeyValue, so their failure behaviour differs and they stay as they are.

Also moves eight test files to external test packages where they declare no
helper used by a sibling in the same package. Coverage is unaffected: Go
compiles internal and external test packages into one binary per package, so
attribution is identical either way.

* chore(test): require new test files to use an external test package

Adds a preflight check that fails when a newly added test file declares an
in-package test. External test packages bind tests to the exported API and
let packages share test doubles, which an in-package double cannot be. Only
added files are checked, so the existing in-package tests are untouched.

e2e and integration suites are exempt: both share a package-level fixture
built in main_test.go, so their files cannot move individually. A file that
needs unexported access can opt out with a //test:in-package comment.
2026-08-12 22:38:15 +10:00
Julian Sommer 8f930e6275 refactor: consume the shared packages from bluebottle
Takes iampolicy, masterkey and sigv4 from github.com/mulgadc/bluebottle rather
than predastore, so the control plane no longer depends on the S3 server for
shared auth and policy code.
2026-08-12 10:50:48 +10:00
Julian Sommer 456a60717b feat: add RDS to awsmodel coverage 2026-08-06 12:00:02 +10:00
Julian Sommer e3b5701f15 fix: authorize RDS data-volume formatting by exact volume identity (#737) 2026-08-06 11:12:01 +10:00
Julian Sommer 8c07f80032 chore: remove the half line breaks 2026-08-06 11:10:15 +10:00
Josh Siviour 688ce5ccfd feat: add AWS model conformance coverage (#736) 2026-08-05 18:46:37 +10:00
Julian Sommer 615578a2e7 refactor: simplify RDS internals (#692) 2026-08-04 14:42:45 +10:00