5 Commits

Author	SHA1	Message	Date
Josh Siviour	b5879f44ff	ADR-0001 package-boundary migration checkpoint (#1190) ... * refactor: move telemetry file naming to contract * docs: record telemetry contract move * refactor: move aws xml response helpers * docs: record aws xml helper move * refactor: move aws tag projections to foundation * docs: record aws tag projection move * refactor: move account key to state foundation * docs: record account key state move * refactor: move host privilege escalation to runtime * docs: record host privilege move * chore: resolve lint findings from package-boundary moves * refactor: move process supervision helpers to runtime host * docs: record process supervision move * refactor: move nbd endpoint helpers to compute runtime * docs: record nbd endpoint move * refactor: move image catalogue to operator * docs: record image catalogue move * refactor: move resource id generation to aws foundation * docs: record resource id move * refactor: move predastore role to runtime roles * docs: point service manifest at acm domain * docs: record predastore role move * refactor: move northstar role to runtime roles * docs: record northstar role move * refactor: move awsgw role to runtime roles * docs: record awsgw role move * refactor: move error payload helpers to aws errors * docs: record error payload move * refactor: move account identifiers to aws identifiers * docs: record account identifier move * refactor: move NATS transport to messaging foundation * docs: record NATS transport move * refactor: move system instance contract to EC2 domain * docs: record system instance move * refactor: move system VPC builder to network domain * docs: record system VPC move * refactor: move instance metadata service to EC2 domain * docs: record guest metadata move and dependency-rule debt * refactor: move DNS control plane to DNS domain * docs: record DNS domain move * refactor: move base zone bootstrap to DNS domain * docs: record base zone bootstrap move * refactor: replace SystemAccountID with GlobalAccountID * docs: record SystemAccountID removal * refactor: extract CA key-pair loader to foundation pki * docs: record foundation pki extraction * refactor: move tenant CA paths to ACM domain * docs: record tenant CA path move * refactor: move IPsec certificate handling to network host * docs: record IPsec certificate move * refactor: move Ochre serving lifecycle to the Ochre domain handlers/bedrock becomes domains/ochre. Ochre is the platform service; Bedrock names only its AWS API surface. Persisted and observable identifiers keep their bedrock names. * docs: record Ochre domain move * refactor: move Ochre vector store to the Ochre domain handlers/ochrevector becomes domains/ochre/vector. Callers import it as ochrevector because vector is a common local name in the gateway. * docs: record Ochre vector store move * refactor: move Hugging Face Hub client to the Ochre domain gateway/bedrock/hfhub becomes domains/ochre/hfhub. Its only caller is the spx admin ochre weights pull. * docs: record Hugging Face Hub client move and Ochre gateway debt * refactor: move EC2 tag mutation helpers to the EC2 domain utils/tagmirror.go becomes domains/ec2/tagmirror. It is EC2-wide tag mutation behaviour, and the tags resource package cannot host it without a resource-to-resource import cycle. * docs: record EC2 tag mutation move and migrate split finding * refactor: move network packages under the network domain Move spinifex/network/* to spinifex/domains/network/* as one enclosing move so the L0-L5 layer invariants keep classifying every package from a single root. Package names and dependency direction are unchanged; invariant scan roots, the RG-10 sudo allow-list path, CODEOWNERS and the repo-root walk in the IMDS netns unit test follow the new path. * docs: record network domain move * refactor: move standalone EC2 resource packages to the EC2 domain Move handlers/ec2/{account,key,launchtemplate,placementgroup, spotinstance,volumestate} to domains/ec2/<resource>. Each package is named after its directory and callers import it as ec2<resource>; volumestate already carried its directory name. * docs: record standalone EC2 resource moves * refactor: move EC2 VPC networking packages to the EC2 domain Move handlers/ec2/{vpc,eigw,eip,igw,natgw,routetable} to domains/ec2/<resource>, each named after its directory and imported as ec2<resource>. The reconcile drift-watch source scan now matches the ec2<resource> bucket qualifiers; its rule is unchanged. * docs: record EC2 VPC networking moves * refactor: move EC2 compute and storage packages to the EC2 domain Move handlers/ec2/{instance,volume,snapshot,image,tags} to domains/ec2/<resource>, each named after its directory and imported as ec2<resource>, which empties handlers/ec2. The daemon's e2e suite selection now lists the moved EC2 resource paths in place of handlers/ec2, so EC2-only changes select the daemon suites again. * docs: record EC2 compute and storage moves * refactor: move ECR credential provider to the EKS agent tree Move the kubelet exec credential provider implementation from cmd/ecr-credential-provider to agents/eks/credentialprovider, exposing one Main(in, out) entry point. The binary keeps its name and FIPS boot import as a thin main. Moving out of cmd/ brings the code under gosec; the one G101 hit on the kubelet API version constant is annotated. * docs: record ECR credential provider move * refactor: move EKS gateway publish helper to the EKS agent tree Move the eks-gateway-publish implementation to agents/eks/gatewaypublish behind Run(ctx, Config) error. The binary keeps flag definitions, environment defaults, parsing, its log prefix and exit code; each former fatal site returns an error with the same text. The retry wait now also honours context cancellation, which the binary never triggers. * docs: record EKS gateway publish move * refactor: move EKS gateway fetch helper to the EKS agent tree Move the eks-gateway-fetch implementation to agents/eks/gatewayfetch behind Run(ctx, Config) error, writing its TSV to Config.Out. The binary keeps flag definitions, environment defaults, parsing, its log prefix and exit code; each former fatal site returns an error with the same text. * docs: record EKS gateway fetch move * refactor: move EKS webhook cert helper to the EKS agent tree cmd/eks-webhook-cert keeps flag parsing, the log prefix and the exit; agents/eks/webhookcert owns Config and Run. Output, error text, file modes and exit codes are unchanged. * docs: record EKS webhook cert move * refactor: move EKS konnectivity cert helper to the EKS agent tree cmd/eks-konnectivity-cert keeps flag parsing, the log prefix and the exit; agents/eks/konnectivitycert owns Config and Run. Output, error text, file modes and exit codes are unchanged. * docs: record EKS konnectivity cert move * refactor: move EKS token webhook to the EKS agent tree cmd/eks-token-webhook keeps the -addr flag, environment-derived config, logging setup and exits; agents/eks/tokenwebhook owns Config and Run. The handlers/eks TokenReview result type stays as recorded temporary debt. * docs: record EKS token webhook move * refactor: move ECS agent runtime and credentials packages to the ECS agent tree * docs: record ECS agent subpackage move * refactor: move ECS agent implementation to the ECS agent tree cmd/ecs-agent keeps process lifecycle: signal context, the separate New and Run calls, both failure log lines, the exit and the ldflags-set build version, now passed in through Config.AgentVersion. agents/ecs/agent owns Config, LoadConfig and the agent. * docs: record ECS agent move * refactor: move RDS guest agent to agents/rds/agent * docs: record RDS guest agent move * refactor: move installer packages to operator/installer * docs: record installer move * refactor: move spx command tree to operator/cli Moving the command tree out of cmd/ brings it under errcheck and gosec. Flag and config wiring now fails loudly instead of silently dropping a binding, listing commands fail when they cannot write their table, and the formation server's unauthenticated TLS is annotated as a known gap. * docs: record spx command tree move * chore: accept known vulnerabilities with an expiry The vulnerability scan failed preflight on a containerd advisory that dev already carries and that is deferred to a separate dependency update. The gate now passes only reachable findings listed with an expiry date, fails on any new one or on an expired acceptance, and warns when an acceptance is no longer needed. * refactor: move EC2 account action adapters to domains/ec2/awsapi * refactor: move EC2 capacityreservation action adapters to domains/ec2/awsapi * refactor: move EC2 key action adapters to domains/ec2/awsapi * refactor: move EC2 launchtemplate action adapters to domains/ec2/awsapi * refactor: move EC2 placementgroup action adapters to domains/ec2/awsapi * docs: record EC2 standalone action adapter moves * refactor: move EC2 zone action adapters to domains/ec2/awsapi * refactor: move EC2 vpc action adapters to domains/ec2/awsapi * refactor: move EC2 eigw action adapters to domains/ec2/awsapi * refactor: move EC2 eip action adapters to domains/ec2/awsapi * refactor: move EC2 igw action adapters to domains/ec2/awsapi * refactor: move EC2 natgw action adapters to domains/ec2/awsapi * refactor: move EC2 routetable action adapters to domains/ec2/awsapi * docs: record EC2 VPC networking action adapter moves * refactor: move EC2 image action adapters to domains/ec2/awsapi * refactor: move EC2 snapshot action adapters to domains/ec2/awsapi * refactor: move EC2 volume action adapters to domains/ec2/awsapi * refactor: move EC2 tags action adapters to domains/ec2/awsapi * docs: record EC2 storage and tag action adapter moves * refactor: move EC2 resource authorization scopes to domains/ec2/awsapi * refactor: move EC2 idempotency adapter to domains/ec2/awsapi * refactor: move EC2 instance action adapters to domains/ec2/awsapi * refactor: move EC2 spotinstance action adapters to domains/ec2/awsapi * docs: record EC2 instance action adapter moves * refactor: move AWS REST-JSON router to ingress/aws/rest * refactor: move SigV4 auth-failure rate limiter to ingress/aws/ratelimit * docs: record AWS ingress helper moves and tooling conformance * docs: correct historical lifecycle references * test: characterize ECR control-plane dispatch before registration seam * feat: add AWS ingress service registration contract * docs: record ingress registration phases 0 and 1 * refactor: route registered AWS services through the ingress dispatch registry The gateway now consumes ingress/aws/dispatch. A registered service is accepted by the auth gate, resolves its own action label, renders errors in its declared envelope and is dispatched with an authenticated Invocation (account, region, policy authorisation and a lazy verified actor). Every other service keeps the existing switch, now named as the legacy path. The awsgw role wires an empty registry and refuses to start if a registered name is also served by the legacy table. No service is registered yet, so behaviour for every existing service is unchanged. * docs: record ingress dispatch registry consumption * refactor: register the ECR control plane from its domain domains/ecr/awsapi now registers the "ecr" AWS JSON 1.1 surface with the ingress dispatch registry: action resolution, resource scope, authorisation through the invocation, the composed capabilities and the success writer all live with the domain. The awsgw role is the only serving composition point; the gateway no longer imports the ECR API package and ECR leaves the legacy service table, JSON error table and dispatch switch. The OCI /v2 route is unchanged. Operation coverage reads ECR's declared inventory, which the registration uses verbatim; coverage.json is byte-identical. * docs: record ECR control-plane registration * fix: compose the ECR control plane in the integration harness The harness stopped wiring ECR's composed capabilities when the authorization-token action was composed, so every integration test that creates a repository or mints a token got ServerInternal. It now composes ECR as the awsgw role does, sharing the OCI registry, meta store and token issuer the /v2 data plane uses. * docs: record ingress pipeline deferral * refactor: extract AWS error envelope rendering to ingress ingress/aws/envelope owns the EC2, IAM-style and S3 XML error envelopes, the AWS JSON 1.1 error body and X-Amzn-Errortype value, the JSON-protocol header detector and the two error content types. Each shape keeps its own builder. The gateway error writers, ErrorHandler and the legacy per-service envelope selection stay in gateway and call the new package; gateway/eks renders its JSON errors through it directly. A 140-case status/header/body golden comparison across services and error writers was identical before and after. * docs: record error envelope extraction * refactor: move AWS service selection into ingress dispatch Selector answers served, action, error protocol, registered entry and startup overlap. The gateway supplies its legacy tables through the dispatch.Legacy interface; legacy dispatch and invocation building stay in gateway. * docs: record ingress service selection move * test: characterize ACM control-plane dispatch before registration Pins full-pipeline error envelopes, a successful action, target parsing agreement between dispatch and telemetry, authorization before the NATS call, 413 and 503 paths, throttle/trace/audit action and the inventory. * refactor: register ACM control plane from its domain domains/acm/awsapi registers the acm JSON 1.1 surface through the ingress dispatch registry, wired by the awsgw role. The gateway drops ACM_Request, the ACM action table and every legacy acm entry; the import invariant now covers each registered domain. * docs: record ACM registration * docs: record ECS, EKS and RDS source inventory and first-resource ranking * docs: record EKS control-plane secrets persisted in the cluster record * chore: drop containerd advisory acceptance now that dev ships v2.4.1 * test: characterize RDS DB subnet group lifecycle before extraction * docs: record RDS DB subnet group lifecycle contract * refactor: extract the RDS DB subnet group lifecycle owner domains/rds/subnetgroup owns the db-subnet-groups/{name} record, member subnet validation and the delete in-use guard. handlers/rds keeps AWS validation, projection and paging and binds two consumer-owned capabilities: the owner's Dependants (instances naming the group) and the instance side's subnetGroupPlacement. Stored shape and behaviour are unchanged. * docs: record RDS DB subnet group owner extraction * docs: correct RDS subnet group tagging debt and conformance scope * test: characterize ECS task definition lifecycle before extraction * docs: record ECS task definition lifecycle contract * refactor: extract ECS task definition owner into domains/ecs/taskdefinition Move the task definition record, container types, keys, reference parsing, ARN helpers and the register/deregister/resolve/list logic into a resource owner package. Stored JSON field names and KV keys are unchanged, and revision allocation keeps its existing non-atomic read-then-write. The owner declares its own PortMapping and SystemControl persisted types so it no longer depends on handlers/ecs/bus; the handler translates them onto the assign payload. Task and service launches reach task definitions only through a launch-side taskDefinitionResolver. AWS conversion, the guest projection and tags stay in handlers/ecs, and gateway authz only switches to the moved reference/ARN helpers. List handlers/ecs and the new package under spinifex_daemon so ECS-only changes select e2e-ecs. * docs: record ECS task definition owner extraction * test: characterize EKS access entry lifecycle before extraction * docs: record EKS access entry lifecycle contract * refactor: extract EKS access entry owner into domains/eks/access Move the access entry record, associated policy and scope types, keys, PrincipalARNHash, the policy catalogue and group mapping into a resource owner package. Stored JSON field names and KV keys are unchanged. Owner is the only writer: Create, SeedCreatorAdmin, Update, Associate, Disassociate and Delete carry the existing logic verbatim, and the put, compare-and-set and delete primitives are private. The handler keeps AWS validation, the cluster existence check, error mapping and projection. Token review still reads records through the exported Get, and the cluster prefix purge still sweeps entry keys. List handlers/eks and the new package under spinifex_daemon so EKS-only changes select the EKS suites. * docs: record EKS access entry owner extraction * test: characterize RDS engine catalogue before extraction * fix: return InvalidParameterValue for a missing RDS master password ValidateMasterUserPassword built the empty-password error with a bare errors.New carrying the code only in its text, which awserrors does not resolve, so CreateDBInstance without MasterUserPassword returned HTTP 500 ServerInternal. Build it with awserrors.Errorf like the other branches so the gateway returns 400 InvalidParameterValue, and cover the full gateway to RDS service path with a regression test. * docs: record RDS engine catalogue boundary and evidence * refactor: extract RDS engine catalogue into domains/rds/engine Move engine identity and validation, the per-engine parameter catalogues, effective-parameter resolution, db.* instance classes and engine/orderable selection into an engine package. Behaviour, error codes and ordering are unchanged. The package imports no EC2 package: instance-class memory comes from an immutable Sizing value built from an engine-declared InstanceTypes interface, backed by the fixed instance-type table in handlers/rds. Effective parameters leave as an engine-owned Setting, converted to Parameter in handlers/rds. AWS output building, paging, apply-method mapping, storage limits and the gateway runnable check stay in the adapter path. The RDS agent declares its own EngineCatalog interface and types and receives an engine-backed implementation from cmd/rds-agent, so agent code does not import the catalogue. Cross-engine tests use the two real engines instead of a registered fake. * docs: record RDS engine catalogue owner move * test: characterize RDS DB parameter group lifecycle before extraction * docs: record RDS DB parameter group lifecycle contract * refactor: extract RDS DB parameter group owner Move the parameter group and override records, keys and listing into domains/rds/parametergroup behind an Owner. Delete asks an owner-declared Dependants interface; instance-side resolution reads through its own parameterGroupValues interface. SetOverrides refuses an unknown group so no override key is written without its group. Stored shape, keys, public errors and propagation are unchanged. * docs: record RDS DB parameter group owner extraction * test: characterize EKS add-on lifecycle and guest wire Pin the persisted record and manifest layout, create/update/delete and status-report transitions, the ignored client token and report version, the missing serviceAccountRoleArn PassRole check, and the exact internal add-ons HTTP body and agent TSV output as they stand today. * docs: record EKS add-on lifecycle contract * fix: require iam:PassRole for EKS add-on service account role CreateAddon and UpdateAddon now pass serviceAccountRoleArn through the existing EKS PassedRoleARNs path, so the caller needs iam:PassRole on the role for eks.amazonaws.com, matching CreateCluster and CreateNodegroup. * docs: record EKS add-on PassRole check * refactor: extract EKS add-on owner Move the add-on catalogue, record and manifest keys, CAS, status transitions and staging into domains/eks/addon behind an Owner that is the sole writer. The handler keeps AWS validation, error mapping and projection; the cluster reconciler delegates reports through the narrow addonReports interface; node groups call EnsureGPUDevicePlugin. Stored keys, JSON shape, wire types and lifecycle behaviour are unchanged. * docs: record EKS add-on owner extraction * refactor: extract EKS add-on guest wire contract Move the add-on status subject, delivery phase, status report, staged manifest and internal add-ons response into contracts/eks/v1 and use it from the handler, gateway and gatewayfetch, replacing the agent's duplicate. Wire bytes are unchanged, including the internal add-ons body that the SDK renderer emits with Go field names; compatibility tests pin the JSON, the guest printf form and the TSV. * docs: record EKS add-on wire contract extraction * docs: update EKS add-on inventory and remaining edges * docs: draft EKS add-on lifecycle correction design for review * chore: run contracts tests in the default test, coverage and race targets * docs: apply review decisions to EKS add-on lifecycle design * test: characterize EKS add-on v1 compatibility with added fields Pin which owner writes drop unknown record and manifest fields, that reads and deletes ignore or erase them, that one undecodable entry fails a listing, and that v1 wire consumers ignore extra fields. * docs: add EKS add-on executor, bundle, v1 compatibility and release inventories * docs: draft EKS add-on v2 guest contract design for review * docs: qualify historical architecture references * docs: apply review decisions and ADR-0006 to EKS add-on v2 design * fix(eks): bind internal report publishing to the caller's cluster PublishInternal was authorised by IAM policy alone, with its resource built from the body accountId and the URL cluster. The control-plane role grants it on "*", so any control-plane VM session, any tenant holding eks:*, or static system credentials could publish cluster state, bootstrap and add-on reports for another cluster or account. PublishInternal now passes the existing internal-route gate: the caller must be a control-plane instance-role session that is a member of the URL cluster in the body account. The gate and the handler decode the body through one shared decoder, so the account checked is the account the subject is built from. Every unauthorised target returns the same AccessDenied, and invalid account or cluster names are treated as not found. Valid publishes keep their subjects and payload bytes. * fix(eks): bind webhook token review to the caller's cluster WebhookTokenReview took its account from the request body and was authorised by IAM policy alone, with the control-plane role granting it on "*". A control-plane VM, any tenant holding eks:*, or static system credentials could resolve a token against another cluster or account and learn the Kubernetes groups it maps to there. Error responses also distinguished absent accounts from absent clusters. WebhookTokenReview now passes the same internal-route gate as PublishInternal: a control-plane instance-role session that is a member of the URL cluster in the body account. The gate and the handler share one body decoder, each body-scoped internal route must declare its reader, and every unauthorised target returns the same AccessDenied without reaching STS verification. Same-cluster responses are unchanged. * fix(eks): validate add-on serviceAccountRoleArn syntax CreateAddon and UpdateAddon stored serviceAccountRoleArn without checking it was an ARN, and the guest renders the value into the add-on's manifests. A non-empty value must now be syntactically an IAM role ARN (12-digit account, no region, role resource, IAM name and path rules); anything else returns InvalidParameterException before any record is written or manifest staged. The syntax check is foundation/aws/arn.ValidateRoleARN. iam:PassRole at the gateway remains the permission and existence check. Stored values are not rewritten, and an UpdateAddon that omits the field still succeeds on a record holding a legacy value. * docs: record EKS internal-route and role-ARN security corrections * docs: add EKS add-on release membership and activation case study * fix: point UI build, CI and lint rules at moved package paths * docs: record EKS add-on activation review decisions * refactor: move quota admission to domains/admission/quota Relocate the quota service, live counters and vCPU records from handlers/quota to domains/admission/quota. Package renamed from handlers_quota to quota; callers with a local quota identifier import it as admissionquota. KV buckets, subjects and error codes are unchanged, and the package's existing debt edges move with it unchanged. * docs: reconcile package-boundary record for ADR-0001 close-out Add the missing slice rows (the commits that dissolved spinifex/types, the ECR repository projection, the stale-path fix and the quota move), correct stale entries, record the dependency-rule breaks found in the import graph with removal conditions, add the close-out table of remaining legacy paths with their decided target owners, recount the mulga-side follow-ups, and fix stale old-path references in docs and comments. * test: move KV migration engine tests to foundation/state/migrate The Registry and RunKV tests exercise foundation/state/migrate exclusively but lived in spinifex/migrate's test file. Relocate them, with their NATS helpers, to the package that owns the engine. No test name, assertion or behaviour changes. * refactor: move config and object-store upgrade workflow to operator/upgrade Relocate spinifex/migrate, the operator-driven config-file and object-store upgrade workflow behind spx admin upgrade, to operator/upgrade. Package renamed to upgrade (imported as opupgrade where a local upgrade identifier exists). It stays distinct from foundation/state/migrate, the generic KV migration engine; no registry types, migration names, versions, file formats or command names change. * docs: record migrate move to operator/upgrade * test: enforce ADR-0001 layering with a blocking ratchet Add spinifex/architecture/layering, a test-only package that parses the module's import graph, maps every production package to an ADR-0001 layer and checks each dependency rule as its own subtest. Existing violations are allow-listed against their recorded debt; a new forbidden edge, an unmapped package, a stale allowlist entry or a new package under a frozen legacy root fails local preflight and CI. Record the nine edges the test found missing from the migration record (quota -> EC2 instance types, STS/Ochre/ELBv2 -> IAM, ELBv2 cross-domain, installer UI -> admin), name the importers of the utils residuals and gateway/ec2_idempotency.go, and correct the handlers/iam consumer list. * refactor: drop the legacy handlers_ecr import alias Import domains/ecr under its package name, or as ecrdomain where the file also imports the AWS SDK ecr package, matching acmdomain. Also fix a stale daemon file reference in the integration ECR daemon comment. * chore: mark spf13/pflag as a direct dependency operator/cli/wiring.go imports pflag directly. Same version; no upgrade. * docs: record test-package-check and nilaway as unverified gates * refactor: move the layering test to module-root architecture/ The ADR-0001 layering ratchet is repository governance tooling, not Spinifex product code, so it sits beside tests/ rather than under the spinifex/ product tree. make test, test-cover and test-race now include ./architecture/..., and the graph walk excludes architecture/. * fix: update paths left stale by the package moves The MariaDB image stamp test read the RDS engine from handlers/rds/engine_mariadb.go, so test-images failed in CI; it now reads domains/rds/engine/mariadb.go. Comments citing moved files now name their new locations. * refactor: move the default-VPC subject into contracts/ec2/v1 utils.SubjectEnsureDefaultVpc becomes ec2v1.EnsureDefaultVpcSubject, the request/reply route between gateway account creation and the daemon. Only the subject moves; the request and reply bodies are still anonymous structs on each side and are recorded as such in the contract README. The layering freeze now allows two utils files. * docs: qualify the migrate split and record admin and Viperblock findings The migrate row is a split, not an exact ADR-0001 map row, and needs a map clarification. Correct the stale admin consumers, record the operator-only admin files as movable, the vbwire prerequisite for the Viperblock adapter and the capability decision encryption.go needs. * refactor: move operator-only admin code to operator/admin Config generation, certificate issuance, host discovery, telemetry, email validation and image import/removal/orphan-volume tooling move to operator/admin. Legacy admin keeps only the symbols with non-operator consumers: default account identifiers, AWS credential generators and system image promotion. foundation/pki and the tenant ACM CA are unchanged. The readAMI helper and the removal test fixtures are duplicated privately in legacy admin rather than shared across the split. The now-stale operator/installer/ui -> admin layering allowlist entry is removed. * docs: record operator/admin split in package-boundary migration * test: pin viperblockd fence, abandon and dirty-marker wire shapes Characterization tests against the current vbwire package, before it moves to contracts/viperblockd/legacy/v1: both subject forms for the fenced and abandon routes, golden JSON for VolumeFencedEvent, VolumeAbandonRequest, VolumeAbandonResponse and DirtyRecord, the dirty bucket name, and the volume-key pattern shared by the lease and dirty stores. * refactor: move viperblockd fence, abandon and dirty-marker vocabulary to contracts Move the fence and abandon subjects/DTOs, the dirty bucket name, DirtyRecord and the shared volume-key pattern out of vbwire and into contracts/viperblockd/legacy/v1, alongside the package's existing mount/unmount/delete/sync/config-update routes. vbwire keeps only the I/O helpers (BindDirtyBucket, ListUnsealedVolumes), now built on the contract vocabulary instead of defining it. Record the approved dirty-marker ownership boundary in the contract README: the Viperblock adapter is the sole producer, viperblockd was only its transitional custodian. Document the fence and abandon routes and the dirty bucket in docs/service-interfaces.yaml. * refactor: split the Viperblock EBS adapter out of services/viperblockd Move the EBS provider adapter (volume lease, fence, dirty marker, nbdkit recovery, provider handlers, encryption key loading) to providers/ebs/viperblock. services/viperblockd keeps only process lifecycle and the NATS subscription wiring that starts and stops it. The dirty-bucket read path moves to providers/ebs/viperblock/dirty as a read-only ListUnsealedVolumes capability; the adapter's write, clear and purge logic stays unexported in the adapter package. Delete services/viperblockd/vbwire now that its vocabulary lives in contracts/viperblockd/legacy/v1 and its I/O helpers live in dirty. Update operator/cli, the runtime service registry, qemunbdd's config comment and the integration test harness to the new import paths, and drop the now-stale architecture/layering allowlist and frozen-package entries for the deleted package. Behaviour, persisted keys and NATS subjects are unchanged. * refactor: move the viperblockd role to runtime/roles/viperblockd With the EBS adapter in providers/ebs/viperblock, services/viperblockd holds only role supervision and the opt-in live data-path tests, so it joins the other runtime roles. It was the last package under services/; the frozen legacy list drops it. * docs: record viperblockd contract, adapter and role moves * test: characterise vpc.* network-realisation routes Pin every vpc.* subject, JSON payload shape and the request/reply {success,error} ack envelope exactly as they exist today, ahead of introducing contracts/network/v1. Covers the 22 subjects registered by domains/network/subscribers, the vpc.igw-attach/detach subjects and InternetGatewayEvent already owned by contracts/ec2/v1, the utils.AddNAT/PublishNATEvent client wire behaviour including the delete-nat retry-on-ErrNoResponders budget, and the EC2-side NATEvent and SGEvent/SGRule duplicates in domains/ec2/vpc. Also adds the six vpc.* route-table and system-egress subjects that were missing from service-interfaces.yaml despite being live production routes (vpc.add/delete-igw-route, vpc.gate/ungate-subnet- egress, vpc.add/delete-system-egress). * feat: add contracts/network/v1 for vpc.* realisation routes Declare the versioned wire vocabulary for the vpc.* control-plane-to- network-realisation boundary: subjects, payload types, the shared {success,error} ack envelope and the vpcd-workers queue group, one file per route family. No NATS I/O, no consumer switched over yet; the package coexists with the fragmented originals pinned in the prior characterization commit until slice 3 migrates call sites. * refactor: use contracts/network/v1 for vpc.* producers and consumers Switch every vpc.* producer (domains/ec2/vpc, eip, instance, natgw, routetable, igw, handlers/elbv2, handlers/eks, daemon, systemvpc) and the network/subscribers consumer onto contracts/network/v1 subjects, event types and the shared AckEnvelope. contracts/network/v1 becomes the sole owner of vpc.igw-attach/detach, replacing the duplicate definitions in contracts/ec2/v1. Deletes the duplicated local subject constants and event structs: subscribers' Topic* consts and event structs, vpc.NATEvent and SGEvent, eip's natEvent, natgw's natGatewayEvent, and eks's systemEgressEvent. domains/ec2/vpc.SGRule survives because EC2 persists it directly in SecurityGroupRecord; contracts/network/v1.SecurityGroupRule was expanded in slice 2 to match it field-for-field so wire bytes are unchanged. No lifecycle or reliability changes: the NAT add barrier, delete retry-on-ErrNoResponders, and the inconsistent per-call-site NAT delivery modes (EIP AssociateAddress and DeleteNetworkInterface both stay fire-and-forget with no ack) are preserved exactly, as are the golden wire literals pinned in the characterization tests. * refactor: replace utils/vpcd_event.go with a typed network projection client Every vpc.* call site now goes through domains/network/projection's operation-specific methods instead of utils.AddNAT, utils.PublishNATEvent or utils.RequestEvent, so no caller passes a bare subject string. Each method states today's delivery mode by name: AddNAT stays strict, AddNATBestEffort and AnnounceNATAdd/AnnounceNATDelete keep the gaps those call sites already had, and RemoveNAT keeps the ErrNoResponders retry. utils/vpcd_event.go and its tests are deleted; the behaviour they pinned (wire shape, ack envelope, retry-on-no-responders) moved into the projection package's own golden and nat tests, unchanged. utils/ now holds only encryption.go, which this does not touch. * refactor: route the remaining fire-and-forget vpc.* producers through projection CreateVPC/DeleteVPC, CreateSubnet/DeleteSubnet, AttachInternetGateway/ DetachInternetGateway, AddNATGateway/RemoveNATGateway, AddIGWRoute/ RemoveIGWRoute, GateSubnetEgress/UngateSubnetEgress and RemoveSystemEgress were still publishing through natsmsg.PublishEvent or a raw nc.Publish call with a bare subject string, left over from the mechanical slice 3 pass. Each now goes through its named projection.Client method instead, so every vpc.* producer in the tree uses the typed client. * refactor: move EBS encryption posture to domains/ec2/ebs/encryption and delete utils The master-key loader was used only by EC2 image, instance and volume to derive the AWS-visible Encrypted flag. It becomes ebsencryption.Enabled, EC2's temporary reading of the provider's key-file setting; the cache, empty-path meaning, error text and each caller's error handling are unchanged (image still reports unencrypted when the key will not load). Its successor is a providers/ebs capability reporting volume encryption posture. utils/ is now empty and deleted. The layering test drops utils and services from the legacy roots, removes their allowlist entries and fails if either retired root regains files. The S7 subject scan follows the vpc.* literals from utils to contracts/network so its gap list stays complete. * docs: record the network contract, projection client and utils removal * test: characterise instance-state persistence, recovery and generation handling Pin the current local-file bytes and schema failure modes, the instance bucket configuration, record key and wire form, the bucket version gate, generation preservation across every KV writer and the VM conversion, and what a returning node does for a missing, foreign, terminal or unreadable canonical record. These describe current behaviour, not the ADR-0007 target. * test: freeze direct instance-state consumers Add an AST ratchet over production code that fails when a new file names the daemon's instance-state bucket, record keys, local-file API, the raw vm.InstanceRecord or its conversion, or a JetStreamManager instance-state method. Today's 73 uses form a shrink-only allowlist keyed to the slice 0 inventory; stale entries fail, and a synthetic-source self-test proves the matcher detects each kind of use and ignores declarations. * docs: record ADR-0007 slice 0 instance-state inventory Record the instance-state persistence formats, the nineteen direct consumers by process and access kind, the characterised recovery and schema outcomes, the generation-preservation matrix and the ratchet that freezes the consumer set. Nothing conforms to ADR-0007 yet and vpcd stays blocked. * test: narrow the instance-state ratchet to instance-state access Stop freezing the JetStreamManager type and constructor, which also serve cluster state; flag only instance-state methods, bucket and key identifiers, record codecs and local-state access. The cluster-state-only consumers in daemon_mgmt_ip.go and operator/cli/cluster.go leave the allowlist (73 to 67). The self-test now proves a cluster-state use is ignored and an instance-state method call is caught. Record the owner's disposition of the KV-unavailable restore result in the migration record. * docs: record the narrowed instance-state ratchet commit * docs: design the instance-state transition record and activation gate * docs: record the instance-state transition-record design commit * docs: record slice 1a activation-gate decisions * feat: add instance transition-record codec with format marker Adds DecodeTransitionRecord and TransitionRecord.Encode in the record's owner package. A top-level format member selects the shape: absent or 1 decodes the legacy record and is never given a marker on write; 2 also decodes the new assignment and observation sections; an integer above 2 decodes its legacy fields read-only and Encode refuses it; anything else is refused on decode. Encode applies only the typed paths that differ from the decode-time snapshot onto the input, so unknown members at every object level and unchanged known fields keep their read form. A changed array is replaced whole. Decode refuses malformed JSON, duplicate keys at any depth, keys that alias a known field by case, trailing data and a non-object record, each with its own sentinel. No production reader or writer uses the codec yet; the new types are not part of vm.InstanceRecord, so every existing record encodes unchanged. The codec file is registered as an owner file of the instance-state ratchet. * docs: record slice 1b step 1 codec and awsgw rollout qualification Records the codec added in e62f921d2 as design Section 10 (C1 to C7): its surface, preservation at every object level, the changed-array limitation, marker handling, the rejection classes and its capable-writers-only scope. C2 supersedes P1's top-level-only preservation for the codec. G19 records that old read-only gateway consumers remain safe but DescribeInstanceStatus gains unconfirmed/degraded visibility only after awsgw upgrades. The migration record gains the commit row and states that no production path uses the codec yet. * refactor: move the instance transition-record codec to domains/ec2/instance/state The codec defines how the canonical persisted instance record evolves, with its assignment and observation sections, which is EC2 instance ownership rather than node-local VM supervision. It moves unchanged into its own package and keeps wrapping the legacy vm.InstanceRecord alias and its spec and status types, which stay in runtime/compute/vm for now. The domain-to-runtime import is recorded as new dependency-rule debt with its removal condition and allow-listed against that entry. The instance-state ratchet now names the new file as an owner file; its allowlist is unchanged. * fix: refuse array changes that would drop element extensions A changed array is replaced whole by the writer's value, which silently lost any member a newer writer had added inside its elements. Encode now inspects the array as read: if any element carries a member its typed schema does not know, at any depth, it returns ErrRecordExtensionConflict naming the array and the first such member. Arrays without such members are still replaced whole, unchanged arrays are still kept verbatim, and arrays inside unknown members are never inspected. * docs: record the codec move and the array extension-conflict rule Records 8db9771fb and c617914a1. T1 and Section 10 now name domains/ec2/instance/state as the eventual owner of the persisted instance record and its evolution, with vm.InstanceRecord as the temporary legacy type it wraps. C3 replaces the changed-array limitation with the refusal: a changed array with element extensions fails with ErrRecordExtensionConflict and is never silently dropped. P2 stays a conservative rule until finalisation. * fix: refuse removals that would drop record extension fields A capable writer may no longer remove or replace a known object, map entry or array when its stored value carries members the writer's schema does not know. The check that guarded changed arrays now runs wherever the merge drops or overwrites a stored value instead of merging into it: a known member deleted or set to null, a map entry removed or stored under a key the typed decode rewrote, a value of another JSON kind, and a changed array. Encode returns ErrRecordExtensionConflict naming the changed container and the first unknown member. Removal stays allowed when the stored value has no extensions, and a change merged member by member still keeps unknown members. * docs: record the uniform extension-conflict rule Adds design clause C8 for 0d7b4266d: a capable writer may not remove or replace a known object, map entry or array whose stored value carries unknown structured extension fields; removal without extensions stays allowed. C3 and C5 point at it. The migration record gains the commit row, updated subsection wording and a pointer-bump follow-up deferring make docs-portal to the Mulga gitlink PR so the portal evaluates the proposed Spinifex revision. * refactor: inject vpcd's IMDS instance-state readers from the instance state boundary * docs: cite the vpcd reader injection commit in the migration record * refactor: move vpcd to runtime/roles/vpcd * docs: cite the vpcd move commit in the migration record * refactor: move the ECS agent wire vocabulary to contracts/ecs/v1 Move handlers/ecs/bus to contracts/ecs/v1 (package ecsv1) with no legacy shim. Subject strings, JSON fields and types are byte-identical; the old-agent compatibility test moves unchanged and golden pins cover the remaining message types. The README route table records that only service-reconcile has a live producer today; the agent reuses these payload types through the gateway actions. service-interfaces.yaml lists ecs.bus.* under the daemon. * refactor: move system-image promotion to domains/ec2/image Move images_promote.go and its tests from admin to domains/ec2/image: SystemOwnerAlias, PromoteImageOpts/PromoteImageResult, PromoteSystemImage, GetAMIMetadata and the private snapshot reader. Storage, ordering, errors and promotion behaviour are unchanged; the daemon, gateway/spx and operator CLI callers import ec2image. admin now holds only account IDs and key generators. * docs: record the ECS contract and image-promotion moves and the remaining-roots matrix * refactor: move the bootstrap account identifiers to foundation/aws/identifiers DefaultAccountID and DefaultAccountName join GlobalAccountID as the fixed, platform-reserved identity of the first human-facing bootstrap account. Names and values are unchanged and stay fixed. Callers in daemon, gateway, awsgw, accountteardown and the operator CLI use awsidentifiers; admin now holds only the IAM credential generators. * docs: cite the bootstrap account identifier move in the migration record * refactor: inject the STS cluster JWKS lookup from handlers/eks handlers/sts declares ClusterJWKSResolver and keeps JWKS decoding and InvalidIdentityToken mapping; handlers/eks implements it as OIDCJWKSReader beside the bucket and key layout helpers. awsgw and the integration harness inject the reader. STS no longer imports any EKS package, including tests, and the handlers/sts -> handlers/eks layering exception is removed. * docs: cite the STS JWKS resolver cut in the migration record * refactor: move the AWS credential-format generators to foundation/aws/credentials and retire admin GenerateAWSAccessKey and GenerateAWSSecretKey move unchanged to foundation/aws/credentials as a stateless format primitive. handlers/iam and operator/cli import it directly. The admin legacy root is deleted and joins the retired roots so it cannot return. * docs: cite the admin retirement in the migration record * revert: remove the uncalled ADR-0007 codec and design stack from the ADR-0001 checkpoint The instance transition-record codec, its tests and the slice 1a design had no production caller. They leave this package-boundary checkpoint and will be reapplied on a dedicated ADR-0007 branch. Slice 0 characterisation and the vpcd IMDS readers in domains/ec2/instance/state stay. * docs: cite the ADR-0007 codec revert in the migration record * fix: declare the ECS action subjects in service-interfaces.yaml Declaring ecs.bus.* made ecs a managed prefix for manifest-lint, which then failed on the 34 ECS action subjects the daemon registers. Declare them as ecs.*, the same wildcard convention as rds.*. * chore: run manifest-check and manifest-lint in preflight CI runs both, but preflight did not, so manifest drift could pass local checks and still fail Build and Test. * docs: record resumed preflight enforcement and dev merge ports * fix: restore dev changes dropped by earlier merges Reapplies the slices.ContainsFunc rewrite in nodeFitsCustomerInstance and the DeleteDBSubnetGroup doc comment, both lost when their files were refactored. * docs: record accountteardown as a frozen legacy root pending cleanup capabilities --------- Co-authored-by: Julian Sommer <115516918+juliansommer@users.noreply.github.com>	2026-10-15 14:42:53 +10:00
Ben Duncan	8409799cbf	fix: default the OCI helper to the DEFAULT profile (#1174) ... * fix: default the OCI helper to the DEFAULT profile oci_env.py shipped our own internal Oracle compartment profile as the default for every reader, with a second internal name as a fallback. A fresh tenancy's config holds only `[DEFAULT]`, so the shipped default was a name nobody else can have, and the fallback chain could land a deployment in whichever tenancy a different profile happened to point at. The profile is now `DEFAULT`, overridable by `OCI_CLI_PROFILE` or `--profile`, and a named profile the config does not hold is an error rather than a fallback. `DEFAULT` needs its own existence test, because configparser treats it as the fallback for every other section and `has_section` denies it exists even when `oci setup config` wrote nothing else. `eip-move-test.sh` and the config table in the README carried the same name. CI reads its credential from the environment rather than from a profile, so nothing there referenced it. * docs: look the OCI compartment OCID up by name The guide asked the reader to paste `ocid1.compartment.oc1..xxx` three times, and the `oci` CLI it already requires can resolve the name instead. The lookup needs its own check. `oci iam compartment list --query ... | [0]` exits 0 and prints nothing when the name matches nothing, so an unset `$CID` would otherwise reach Terraform and fail with an OCI error naming neither the compartment nor the cause. Two compartments in one tenancy can share a name, which the listing command shows and the lookup cannot resolve, so the paste form stays as the answer for that case. Records why the tenancy root is the wrong value: the step 2.1 policy is scoped to whatever `$CID` holds. * Revert "docs: look the OCI compartment OCID up by name" This reverts commit 5443149cce. * Reapply "docs: look the OCI compartment OCID up by name" This reverts commit 1535d94c8a. * fix: stop validate-topology.sh failing on macOS bash `tf_principal_var` is empty on every run that does not pass `--instance-principal`, and bash before 4.4 treats `"${arr[@]}"` on an empty array as an unbound variable under `set -u`. macOS ships bash 3.2, so the documented deployment command aborted at the tf_vars assembly with `tf_principal_var[@]: unbound variable` before building anything. Assigning the array was not enough, which is why the earlier `HOSTS=()` fix did not cover this: the error is in the expansion, not the reference. Both sites now expand only behind a length test, which is valid on every version. `capture_journals` had the same shape for `HOSTS`, reachable when a run fails before the apply. Reproduced and fixed against bash 3.2.57 in a container, which emits the identical message, and the script now runs to the terraform call on 3.2 for vm-single, vm-single-principal and `--instance-principal`, including the `--keep` cleanup path with no hosts. * docs: record the bash 3.x requirement in the OCI scripts These three are the scripts the OCI guide tells an operator to run, and macOS ships bash 3.2, so a bash 4 construct in any of them breaks a deployment rather than a test. The constraint has now been rediscovered twice, through declare -A and mapfile, and then through an empty array expanded under set -u. Names the three traps at the top of each file so the next change does not have to find them by failing on someone's Mac. * fix: stream the workbook suite instead of hiding it in a file The nine workbooks ran with stdout redirected to `workbooks.log`, so a run wedged inside one printed nothing until the whole suite finished. A CI job then sat silent for hours with no way to tell which workbook it was in, or whether it was working at all. The output is now teed, so the file the per-workbook table is parsed from is unchanged while the run is visible live. Each of the driver's own RUN/PASS/FAIL lines is stamped with elapsed seconds and the workbook name, which is what identifies the slow one without reading the log. `pipefail` carries the ssh exit status through the pipe, so a failed suite still takes the branch that records the table and captures the journals. * fix: grant the OCI policy the subnet verb AssignPrivateIP requires An OCI node with a pool-mode external interface formed, reported healthy, and then refused every launch wanting a public address with a 404 NotAuthorizedOrNotFound from AssignPrivateIP. AssignPrivateIP calls CreatePrivateIp, which Oracle documents as needing PRIVATE_IP_CREATE, PRIVATE_IP_ASSIGN, VNIC_ASSIGN and SUBNET_ATTACH, and DeletePrivateIp as needing SUBNET_DETACH. The policy granted use vnics, manage private-ips and manage public-ips, and no subnet permission at all, so SUBNET_ATTACH was never held. Only the subnets use verb carries it. Measured on a live node as the instance principal: GetVnic and ListPrivateIps succeed, GetSubnet and CreatePrivateIp both 404. Authentication, the dynamic group's matching rule and the compartment were all correct, so the credential was authentic and unauthorised. The documented operator policy for the API-key route carried the same three statements, so neither credential route could allocate. A node now proves its own credential at pool start by making one read in each permission class, and logs the missing verb if one fails. The topology gate waited for "resolved the external VNIC", which comes from the metadata service and needs no IAM policy, so an unauthorised node passed the gate and failed an hour later inside a workbook as capacity. It now waits for the authorisation line and fails immediately on a refusal. Reads rather than a real assign-and-release round trip: a round trip also fails on a full subnet, and reporting that as an authorisation fault sends an operator to IAM for a capacity problem. * docs: say what a principal run costs in a tenancy with no dynamic group No principal topology can pass there, and the only previous hint was one clause about OCI_INSTANCE_PRINCIPAL needing the group to exist. * test: cover the subnet read and the authorisation verdict Diff coverage was 39.1% against a 60% gate. Three of the four gaps were real. GetSubnet on the SDK-backed client had no test, so the request shape was unverified: addressing the collection rather than one subnet by OCID would answer from a different permission and report a credential as working when it is not. Also pins that a sparse body is not an error and that the unauthorised 404 arrives as ErrNotFound, which is what stops the check retrying a verdict that will never change. The fake's GetSubnet was exercised only from the ocinet tests, and coverage is recorded per package under test, so it read as zero-covered. reportAuthorisation now has a test asserting the exact log text, because validate-topology.sh greps for it and a rename would quietly turn the gate into one that never fires. The six lines still uncovered are inside FromPoolConfig, which needs a live JetStream to reach. Diff coverage 87.0%. * fix: refuse a run whose tfvars file contradicts the compartment it was given Terraform ranks *.auto.tfvars above TF_VAR_*, so a leftover file wins silently over an operator targeting a different tenancy by environment, and the guide tells every operator to write one. Hit for real: a stale terraform.auto.tfvars naming a different tenancy's compartment and a bare-metal shape overrode TF_VAR_compartment_ocid, and the plan was for 11 resources in a compartment nobody had chosen. Only a dry run caught it. The run now stops and names both values rather than building. * fix: collect orphaned OCI public IPs and name the limit that was hit A reserved public IP attached to nothing is on no VNIC, so the reconcile that exists to find leaked addresses could not see the ones that matter. It also ran only at startup, while an address is leaked on release and on teardown. Twenty accumulated in one compartment and exhausted the tenancy's limit of 50, after which every launch needing a public address aborted. Reconcile now lists reserved public IPs in the compartment and collects those that carry our prefix, are attached to nothing, are named by no binding and are older than a grace period. All four are needed: an allocated but unassociated Elastic IP is detached too and is the customer's to keep, and a young detached address is what another node's CreatePublicIP looks like until it attaches. A pool with no binding record is not swept at all, since that is indistinguishable from a store that cannot be read. The pass now repeats on a ticker rather than running once at startup. Both OCI ceilings also resolved to InsufficientAddressCapacity, which tells an operator the region is out of addresses when the fix is to release addresses they already hold. AWS separates them, so a tenancy out of reserved public IPs is now AddressLimitExceeded and a full VNIC is PrivateIpAddressLimitExceeded. * feat: run the OCI principal e2e in a tenancy that can hold its dynamic group vm-single-principal needs the tenancy's dynamic group, and creating one is a tenancy-root write that a shared test tenancy does not grant us. The topology could never pass there however correct the code was, so it now takes its own credential from OCI_PRINCIPAL_*, all six or none. A partial set is refused before the build rather than at the apply, because it would build one tenancy's instances against another's compartment. With none set the topology is recorded SKIPPED instead of spending a provision to fail. The sweep takes the same credential, or a leftover principal deployment is destroyed against a tenancy that never held it and reported clean. Two defects found while proving it: --dry-run reported FAIL on every topology with a tree source, because the dry run skips the build and then refuses the tarball it would have produced. The flag operators are told to run first could not pass. The dry run truncated its plan to the last 30 lines, which is the outputs block. That dropped the resource count and every compartment and shape line, so the one check a dry run exists for could not be made from its output. It now writes the whole plan and prints the summary. Removing a deployment is documented as leaving public IPs behind. Spinifex allocates them on the running node, so they are in no Terraform state and destroy cannot see them; twenty accumulated and exhausted a tenancy's limit, which then refused every launch in every compartment. * fix: canonicalise the Opc-Next-Page header in the pagination test CI's linter rejects a non-canonical header name and the local preflight does not, so this passed here and failed there.	2026-10-06 20:56:16 -05:00
Ben Duncan	7c56855526	fix(oci): both credential routes in the guide, and a nightly topology that proves the principal one (#1173) ... * docs(oci): offer both credential routes, and default the guide to the release Step 2 was one route only, the scoped API key, with the instance principal mentioned in a TIP at the end of it. It is now "2. OCI Credentials for Spinifex" with the two routes as 2.1 and 2.2, a table to choose between them, and the instance principal first because it leaves no key on any node. The guide also assumed the dev branch throughout. `git switch dev` and `--channel dev` are now an optional note for installing a pre-release, and the documented path is the published release. * test(oci): add a vm-single-principal topology, and cover the credential branch Instance-principal authentication was broken for a release while the OCI nightly stayed green, because the run's two axes never crossed: every topology that built the address allocator authenticated with an API key, and every run that authenticated as the instance was dispatched with no external pool, so the allocator was never constructed and the credential never used. vm-single-principal closes that. It is vm-single with the credential fixed by the topology rather than by a separate switch, so the name alone decides it, and pairing it with --no-external-pool is refused rather than run: without the allocator there is nothing for it to exercise. It joins the nightly's default list, and naming it alone is the way to test this one thing in a quarter of the full run. A unit test covers the same branch without a tenancy. newClient now reaches its two constructors through package variables a test can swap, which is the only way to observe the choice off an OCI instance - both real constructors need either an instance identity or a key file on disk. * fix(oci): let the tfvars choose the credential, and stop the guide hardcoding paths The deploy driver passed instance_principal as a -var, which outranks a tfvars file, so a deployment that set it there had it silently replaced with off and the nodes went looking for a key. Same defect class as the shape and node count, and the same fix: the flag is a choice about one run and still outranks everything, and without it the deployment's own tfvars decides. The driver now reads instance_principal from terraform.tfvars and *.auto.tfvars for its pre-apply gates, taking the last assignment as Terraform does, and checks that guess against a new Terraform output once the apply has run. They have to agree, because the credential handoff is skipped on the strength of it and a wrong answer leaves a node with no credential at all. The guide now carries one export rather than two more steps. SPX_PRINCIPAL is set in step 2.1, expanded into the tfvars step 4 already writes, and read back from there in step 5, so an instance-principal deployment runs the same command as a key-based one. Getting into the node needed an address copied out of the log. Every topology now writes its hosts file, not only the multi-node one that hands it to install-node.sh, and the run prints the ssh and destroy commands when --keep leaves the infrastructure up. The guide exports TOPOLOGY once, so the paths read the same for one VM, three, or bare metal. * fix(oci): print the ssh key path as ~ rather than expanded The --keep line read ssh -i /home/you/.ssh/oci-spx, which is the invoking user's home and differs on every machine. Collapsed to ~ when the key is under $HOME, left alone when it is not.	2026-10-06 13:15:56 -05:00
Ben Duncan	4fbde4a665	docs(oci): ship the credential installer, and rewrite both OCI guides in the house voice (#1166) ... * docs(oci): rewrite both OCI guides in the house voice Seventy-five em dashes across the two guides, with four of them in headings where GitHub and the docs site slugify them differently and a cross-link resolves on neither. Recast as commas, colons and separate sentences. The silent-credential failure was stated five times as the same triple, "forms, passes every health check, and then refuses every launch". It now reads in full once, under Credentials, and the other four refer back to it instead of echoing the skeleton. Also broken up: the parallel stack in "Why run Spinifex on OCI", several five-sentence paragraphs, and a handful of clauses that leaned on a dash where a full stop was clearer. No technical content changed. Every inline-code token is byte-identical before and after, every fenced block is unchanged apart from one comment line, and the four reworded headings carry no inbound anchor links. The docs build reports zero warnings for either page and both render at the same size as before. * docs(oci): open the install guide with what Spinifex is, as the other install guides do * docs(oci): ship the credential installer instead of asking for a hook The guide told the reader to write their own executable to put the OCI API key on each node, which is work nobody should have to do to install a product, and the step was unclear about where the scoping actually happens. `scripts/terraform/oci-spx/spx-oci-config.sh` now does it. It takes `validate-topology.sh`'s hook contract, `hook <ssh-key> <host>...`, so it drops straight into `--credential-hook` and is also runnable on its own against an existing cluster. It resolves the credential in four steps and takes the first complete one: the `OCI_SPX_*` environment variables, the `[spinifex]` profile in `~/.oci/config`, `~/.oci/oci_api_key_spx.pem` with the Terraform profile's identifiers, then the Terraform credential itself with a warning that it is wider than the allocator needs. Steps 2 to 4 go through `scripts/oci_env.py`, the same resolver the apply uses, so the credential on the nodes and the one that built the infrastructure cannot come from different profiles. The fingerprint is checked against the key before anything is written, because a mismatched pair is otherwise an OCI 401 at the first allocation, long after the deploy has reported success. The node's filename stays `oci_api_key.pem` while the workstation's is `oci_api_key_spx.pem`: the suffix exists only to keep the Spinifex key off whatever `[DEFAULT]` uses, and a node has no other OCI key. Nothing reads the name anyway, since the daemon learns the path from `key_file` inside `/etc/spinifex/oci/config`. Key material reaches the nodes over SSH, in the remote shell's stdin rather than its arguments where `ps` would expose it, and goes into neither user-data nor Terraform state. `--credential-hook` still accepts any executable with that argument shape, so a tenancy that keeps credentials in a vault can substitute its own. Also in the guide: the policy statements name the compartment by `compartment id` rather than by name, which is the form that resolves from a compartment the policy is not attached at; `$CID` is consistently a compartment OCID; and the table of contents matches the headings again.	2026-10-05 16:18:00 -05:00
Ben Duncan	cf17866fed	docs(oci): publish both OCI guides on the docs site (#1165) ... * docs(oci): publish both OCI guides on the docs site The site extracts only four H2 sections and drops everything else, so both guides would have published nearly empty: the deployment guide matched none of the four, and the reference doc only its troubleshooting. Insert the section markers the other published docs already use, keeping every heading at the level it had so no anchor moves. The architecture reference becomes oci-architecture/README.md, because a page is always <slug>/README.md and a flat URL takes the last path segment -- plain "architecture" would have published at /docs/architecture. Two headings lose an em dash. GitHub renders it as a double hyphen in the fragment and the site's slugify drops it entirely, so the two disagreed and the cross-links resolved on neither. * docs(oci): the OCI guides sit in their own Cloud Install section Their category frontmatter named Install, which is the section they are moving out of.	2026-10-04 15:18:02 -05:00