From cf9e8ce0d054b84eb6fe83585bab08d569dbd15c Mon Sep 17 00:00:00 2001 From: joshsiv-mulga Date: Fri, 9 Oct 2026 19:23:17 +1000 Subject: [PATCH] docs: record the network contract, projection client and utils removal --- contracts/network/v1/README.md | 5 +++-- docs/PACKAGE_BOUNDARY_MIGRATION.md | 14 +++++++++++--- docs/service-interfaces.yaml | 6 +++--- spinifex/domains/network/projection/client.go | 11 +++-------- spinifex/domains/network/projection/golden_test.go | 3 +-- 5 files changed, 21 insertions(+), 18 deletions(-) diff --git a/contracts/network/v1/README.md b/contracts/network/v1/README.md index b69bfcdb0..64636f88b 100644 --- a/contracts/network/v1/README.md +++ b/contracts/network/v1/README.md @@ -43,8 +43,9 @@ against. | `vpc.delete-sg` | `SecurityGroupEvent` | `AckEnvelope` | request/reply, 5s, idempotent | `vpcd-workers` | | `vpc.update-sg` | `SecurityGroupEvent` | `AckEnvelope` | request/reply, 5s | `vpcd-workers` | -Producer for every route above is the daemon (`spinifex-daemon`), by way of -`domains/ec2/*`, `handlers/elbv2`, `handlers/eks` or `spinifex/utils`. +Producer for every route above is the daemon (`spinifex-daemon`), through the +typed client in `domains/network/projection`; `vpc.add-system-egress` has no +current producer. Consumer for every route is `vpcd`, by way of `domains/network/subscribers`. This package is the sole declared owner of every subject and type above, including `vpc.igw-attach`/`vpc.igw-detach`, which were previously also diff --git a/docs/PACKAGE_BOUNDARY_MIGRATION.md b/docs/PACKAGE_BOUNDARY_MIGRATION.md index 2ee94d83d..0abb36124 100644 --- a/docs/PACKAGE_BOUNDARY_MIGRATION.md +++ b/docs/PACKAGE_BOUNDARY_MIGRATION.md @@ -321,6 +321,11 @@ while the branch is in flight; they do not describe the umbrella repository's | `53697bc76` | `spinifex/services/viperblockd/vbwire` (fence and abandon subjects and payloads, `DirtyBucket`, key rule, `DirtyRecord`) | `contracts/viperblockd/legacy/v1` (`fence.go`, `abandon.go`, `dirty_marker.go`) | These routes and the dirty marker cross the daemon–viperblockd process boundary and, for the marker, persist across nodes, so they are deployed legacy contract vocabulary. The characterisation tests moved with them, literals unchanged. The README records the approved dirty-marker boundary: the adapter owns it, only the adapter writes, recovery on any node and `spx admin` (read-only) consume it, no TTL, N/N-1, no renames or reinterpretation, and a breaking change needs a successor contract. `docs/service-interfaces.yaml` now lists both routes for the daemon and viperblockd. | | `f17e93aed` | `spinifex/services/viperblockd` (lease, fence, dirty-marker writes, nbdkit recovery, provider handlers, `Config`) and `vbwire` I/O helpers | `spinifex/providers/ebs/viperblock` and `spinifex/providers/ebs/viperblock/dirty` | The Viperblock EBS adapter is a provider; it and `internal/testkit/vbscan` are the only importers of the Viperblock library. `dirty` exposes only `ListUnsealedVolumes`; it never creates the bucket (unchanged behaviour), and write, clear and purge stay unexported in the adapter. `operator/cli` uses the reader, so it is not a raw KV client. `vbwire` is deleted and its frozen-list and allowlist entries removed. The adapter loads its key through `bluebottle/pkg/masterkey`, so `utils/encryption.go` was not needed. | | `7e70222f1` | `spinifex/services/viperblockd` (role supervision, opt-in live data-path tests) | `spinifex/runtime/roles/viperblockd` | ADR-0001 maps `services/*` to `runtime/roles/*`; what remained is an 82-line role. It was the last package under `services/`, which now holds nothing. Full preflight (excluding the recorded Go 1.27.0 advisories) and `manifest-check` passed for each of the four commits. | +| `7a45e7a3e` | (characterisation only) | — | Pins the subject, JSON, reply envelope, delivery mode and queue group of all 22 inbound `vpc.*` network-realisation routes where they lived. `docs/service-interfaces.yaml` gains six live routes it was missing (IGW routes, subnet egress gating, system egress). | +| `77ac0f0f7` | (new contract) | `contracts/network/v1` | Versioned vocabulary for the `vpc.*` control-plane-to-network-realisation boundary: one file per route family, the `{success,error}` acknowledgement envelope and the `vpcd-workers` queue group. Network owns the meaning and wire contract because it owns OVN and host realisation; EC2 and managed-service domains own their AWS-visible transitions and decide when to project them. The README states it does not move `vpcd` and changes no lifecycle or reliability semantics. | +| `105048594` | Subjects and payload types in `domains/network/subscribers`, `domains/ec2/{vpc,eip,natgw}`, `handlers/eks` and `contracts/ec2/v1/igw.go` | `contracts/network/v1` | Producers and the subscribers use one vocabulary; the duplicates are deleted, including the IGW attach/detach declarations in `contracts/ec2/v1` (network is the sole owner). Wire bytes are unchanged: `networkv1.SecurityGroupRule` carries the publisher's nine fields with its tags (the subscriber-side golden now pins the `"rule_id"` the publisher always sent), and both NAT-gateway publisher shapes marshal as before. `domains/ec2/vpc.SGRule` stays because it is EC2's persisted rule record, converted at the publish boundary. | +| `3aa82953e`, `46f448963` | `spinifex/utils/vpcd_event.go` and the raw `vpc.*` publishes in EC2, ELBv2, EKS, the daemon and `domains/network/systemvpc` | `spinifex/domains/network/projection` | A typed projection client with one method per operation and delivery mode; callers pass no subject strings. Timeouts (5s, 45s NAT add, 15s NAT delete), the NAT add barrier, the ErrNoResponders retry (3 attempts, 500ms) and non-fatal teardown are unchanged, and the inconsistent NAT paths keep their current mode under honest names (recorded below). The IGW, NAT-gateway, IGW-route and egress-gate publishes previously logged their own `WarnContext` messages with `subnetId`; they now log the shared publish warning without request context or subnet, a diagnostic loss only. | +| `fd88e8ceb` | `spinifex/utils/encryption.go` (`LoadViperblockMasterKey`) | `spinifex/domains/ec2/ebs/encryption` (`Enabled`) | EC2's AWS-visible EBS encryption posture, a temporary reading of the provider's key-file setting rather than an encryption implementation. Cache, error text and each caller's handling are unchanged (image reports unencrypted on a load failure; instance and volume return InternalError). `utils/` is deleted; the layering test retires `utils` and `services` and fails if either regains files. The S7 subject-prefix scan now walks `contracts/network`, where the `vpc.*` literals live; after the projection move it had silently stopped listing them. Full preflight (excluding the recorded Go 1.27.0 advisories) and `manifest-check` passed for each commit. | The EC2 contract row is intentionally different from the directory moves: it creates a compatibility boundary. `ec2.cmd.*` retains its deployed one-token NATS @@ -404,7 +409,10 @@ These residuals were recorded by individual slices and must be resolved once the - ACM control-plane registration: `domains/acm/awsapi.Deps.NATS` (`registration.go`) is likewise a raw `*nats.Conn` carried by every `Handler`, for the same reason; it must not be broadened further meanwhile. - `cmd/aws-model-coverage` imports `ingress/aws/dispatch`, `domains/ecr/awsapi`, `domains/acm/awsapi` and `gateway` to read ECR's and ACM's declared inventories. It is a non-serving tool and the only named exception to the ADR-0002 composition invariant; it must not construct a registration. - Ingress pipeline (ADR-0002 phase 4): the request-audit record, the trace action enricher, the throttle key functions and the gateway request-context keys stay in `gateway` until the auth/IAM and request-context boundary is designed. SigV4 authentication writes the audit record and context keys from four places, so moving them now would export exactly the setters ADR-0002 says must not be exported solely to make a directory move possible. -- `utils`: `vpcd_event.go` (vpcd request/ack protocol and NAT helpers) awaits a network contract, and `encryption.go` (the Viperblock master-key loader) awaits a Viperblock adapter owner, since `providers/ebs` is provider-neutral. Importers: `domains/ec2/{eip,instance,vpc}` and `handlers/elbv2/service_impl.go` use the `vpcd_event.go` helpers (`PublishNATEvent`, `AddNAT`, `RequestEvent`), and `domains/ec2/{image,instance,volume}` use `LoadViperblockMasterKey`; each edge clears when its file moves to its owner. +- NAT projection delivery is inconsistent and is preserved as found: EIP `AssociateAddress` sends `vpc.add-nat` unacknowledged (`AnnounceNATAdd`), skipping the 45s OVN barrier every other add path waits on, and `DeleteNetworkInterface` sends `vpc.delete-nat` unacknowledged (`AnnounceNATDelete`), skipping the ack and the ErrNoResponders retry. The reconciler's orphan sweep is the only backstop for both. Clears when those call sites use `AddNAT`/`AddNATBestEffort` and `RemoveNAT`, a reliability change of its own. +- `vpc.add-system-egress` has a `vpcd` subscriber and a contract entry but no producer (its former `domains/ec2/eip` publisher is gone); `handlers/eks` still publishes the delete. Either a producer is missing or the add route is dead; it needs an owner decision. +- EBS encryption posture: `domains/ec2/ebs/encryption` reads the Viperblock key-file setting to decide what EC2 reports. Clears when `providers/ebs` reports each volume's encryption posture and EC2 stops reading key files. +- `foundation/messaging/nats/nats_trace.go` still names its OpenTelemetry tracer `github.com/mulgadc/spinifex/spinifex/utils`. It is an emitted instrumentation-scope name that dashboards may filter on, so renaming it is a telemetry change, not a path fix. - `runtime/compute/cache.New`: two construction-time panics carry a lint exemption pending a constructor that returns an error. - `domains/ec2/systeminstance`: managed-service parent metadata is only the `ManagedBy` tag value. - `gateway/bedrock`: holds the Ochre domain core (model catalogue, provisioned throughput, guardrail, access, credential, usage, price, logging and weights stores), the inference engine (provider, vLLM, Anthropic, embeddings, reranker, admission) and the Bedrock action layer in one package with internal cycles. `domains/ochre` imports it, which is the ADR step 7 Ochre gateway import. Splitting it needs a plan: domain core and stores to `domains/ochre/`, inference to an Ochre-owned package, and only the action layer to `domains/ochre/awsapi`. The top-level `gateway/bedrock*.go` HTTP handlers additionally wait for the ingress registration seam. @@ -509,8 +517,8 @@ series above; "needs decision" is open. | `gateway/spx` | `promote_image.go` and `spx.go` dispatch (`storage.go` already consumes `contracts/operator/v1`). | Needs decision: which package owns operator-plane action dispatch registration — an `ingress/aws`-style seam for the operator plane, or `gateway` stays its home. | | `gateway/tagging` | The cross-service tagging API (`GetResources`) and its authorization. | Decided: a new `domains/tagging` boundary owns the cross-resource tagging API and aggregation; individual resource domains keep ownership of their own tag storage and expose narrow tagging capabilities. Stays debt until those per-domain capabilities exist; clears when `domains/tagging` aggregates only through them. | | `services/viperblockd` | — | Done. The adapter is `providers/ebs/viperblock` (with the read-only `dirty` reader), the shared vocabulary is `contracts/viperblockd/legacy/v1`, and the role is `runtime/roles/viperblockd`. `services/` is empty. | -| `utils` | `vpcd_event.go`, `encryption.go` (`ec2_subject.go` moved to `contracts/ec2/v1` as `EnsureDefaultVpcSubject`; its request and reply bodies are still anonymous structs on each side). | Decided: `vpcd_event.go` is a serialised cross-process contract (its `natEvent` payload crosses a NATS request-reply boundary on `vpc.add-nat`/`vpc.delete-nat`), so `domains/network` owns its meaning and its versioned wire form goes under `contracts/network/v1`. `ec2_subject.go` (`SubjectEnsureDefaultVpc`, a NATS request/reply subject between the gateway and the daemon) has `domains/ec2` as its subject-semantics owner, with `contracts/ec2/v1` as its eventual home since it crosses a process boundary. `encryption.go` (the Viperblock master-key loader) joins the Viperblock adapter at `providers/ebs/viperblock`, not generic EBS or foundation code. Blocked on a capability decision: its only callers are `domains/ec2/{image,instance,volume}`, which test the key only to derive the AWS-visible `Encrypted` posture; moving it would make EC2 import a Viperblock-specific provider. It needs an encryption-posture field in the `providers/ebs` contract or an EC2-owned configuration predicate first. The adapter itself loads its key directly with `masterkey.LoadShared` and never calls this cached loader, so the loader serves only EC2. | -| `vpcd` | `imds_instance_state.go` reads `daemon.LocalState`, `ReadLocalState`, `LocalStatePath`, `NewJetStreamManager` and `InstanceStateBucket` directly (recorded above). | Needs a daemon-owned local-state capability or contract vpcd can consume instead of the `daemon` package. | +| `utils` | — | Done and deleted. `ec2_subject.go` went to `contracts/ec2/v1`, the vpcd protocol to `contracts/network/v1` with the typed client in `domains/network/projection`, and the master-key loader to `domains/ec2/ebs/encryption`. | +| `vpcd` | `imds_instance_state.go` reads `daemon.LocalState`, `ReadLocalState`, `LocalStatePath`, `NewJetStreamManager` and `InstanceStateBucket` directly (recorded above). | Its inter-process boundary now exists (`contracts/network/v1`, with `domains/network/subscribers` as the consumer and `domains/network/projection` as the producer client). Moving `vpcd` to `runtime/roles/vpcd` remains blocked on a separate design task: a daemon-owned local-state capability or contract vpcd can consume instead of the `daemon` package. | | `daemon` | Reserved as `runtime/daemon`'s home only; not split or moved. | Deferred: Q-28's write-side consolidation is a prerequisite ADR-0001 names explicitly. | | `admin` | `DefaultAccountID`/`DefaultAccountName` (consumed by `daemon`, `gateway`, `runtime/roles/awsgw`, `operator/cli`), `GenerateAWSAccessKey`/`GenerateAWSSecretKey` (`handlers/iam`, `operator/cli`) and `images_promote.go` (`daemon`, `gateway/spx`, `operator/cli`). The operator use cases moved to `operator/admin`. | Deferred to their owners: IAM/account bootstrap, IAM credentials, and the existing `domains/ec2/image`. | | `accountteardown` | Direct calls into `domains/ec2/awsapi/*` action adapters (recorded above). | Deferred: the ADR's `accountteardown` row replaces this with domain-owned cleanup capabilities. | diff --git a/docs/service-interfaces.yaml b/docs/service-interfaces.yaml index f5a8ad410..b9a75c5b4 100644 --- a/docs/service-interfaces.yaml +++ b/docs/service-interfaces.yaml @@ -206,9 +206,9 @@ services: - "vpc.delete-igw-route" - "vpc.gate-subnet-egress" - "vpc.ungate-subnet-egress" - # Egress-only reroute for a single system instance (handlers/eks and, - # historically, domains/ec2/eip). Fire-and-forget. - - "vpc.add-system-egress" + # Egress-only reroute removal for a single system instance + # (handlers/eks). Fire-and-forget. vpc.add-system-egress has no + # current producer. - "vpc.delete-system-egress" # → viperblockd # The migrated legacy ebs.config and ebs.sync payloads and routes are diff --git a/spinifex/domains/network/projection/client.go b/spinifex/domains/network/projection/client.go index 5c905f233..193752ac5 100644 --- a/spinifex/domains/network/projection/client.go +++ b/spinifex/domains/network/projection/client.go @@ -1,12 +1,7 @@ // Package projection is network's authorised view onto the vpc.* NATS -// realisation boundary described by contracts/network/v1. It wraps every -// subject in an operation-specific method so callers never pass a subject -// string or a bare event value: the method name states the delivery mode -// (strict, best-effort, unacknowledged) that today's call sites rely on. -// -// This package moves no behaviour: every timeout, retry and barrier here is -// the one spinifex/utils/vpcd_event.go and each call site already had. It -// replaces spinifex/utils/vpcd_event.go, which no longer exists. +// realisation boundary described by contracts/network/v1. Each subject has +// operation-specific methods, so callers never pass a subject string, and the +// method name states the delivery mode (strict, best-effort, unacknowledged). package projection import ( diff --git a/spinifex/domains/network/projection/golden_test.go b/spinifex/domains/network/projection/golden_test.go index 79b0f0bc5..5913a4378 100644 --- a/spinifex/domains/network/projection/golden_test.go +++ b/spinifex/domains/network/projection/golden_test.go @@ -10,8 +10,7 @@ import ( "github.com/stretchr/testify/require" ) -// This file characterizes the vpc.add-nat / vpc.delete-nat wire behaviour -// that spinifex/utils/vpcd_event.go pinned before this package replaced it: +// This file characterizes the vpc.add-nat / vpc.delete-nat wire behaviour: // the exact JSON AddNAT/RemoveNAT put on the wire, the {success,error} ack // envelope the internal request helper expects, and the delete-nat // retry-on-ErrNoResponders behaviour. in-package because request is