Commit Graph

2882 Commits

Author SHA1 Message Date
joshsiv-mulga 09646388ed revert: remove the uncalled ADR-0007 codec and design stack from the ADR-0001 checkpoint
The instance transition-record codec, its tests and the slice 1a design
had no production caller. They leave this package-boundary checkpoint
and will be reapplied on a dedicated ADR-0007 branch. Slice 0
characterisation and the vpcd IMDS readers in
domains/ec2/instance/state stay.
2026-10-11 23:06:51 +10:00
joshsiv-mulga c419fb3490 docs: cite the admin retirement in the migration record 2026-10-11 22:54:42 +10:00
joshsiv-mulga 0368e861a9 refactor: move the AWS credential-format generators to foundation/aws/credentials and retire admin
GenerateAWSAccessKey and GenerateAWSSecretKey move unchanged to
foundation/aws/credentials as a stateless format primitive. handlers/iam
and operator/cli import it directly. The admin legacy root is deleted
and joins the retired roots so it cannot return.
2026-10-11 22:54:42 +10:00
joshsiv-mulga 87cfc96335 docs: cite the STS JWKS resolver cut in the migration record 2026-10-11 21:59:51 +10:00
joshsiv-mulga 254d898a3f refactor: inject the STS cluster JWKS lookup from handlers/eks
handlers/sts declares ClusterJWKSResolver and keeps JWKS decoding and
InvalidIdentityToken mapping; handlers/eks implements it as
OIDCJWKSReader beside the bucket and key layout helpers. awsgw and the
integration harness inject the reader. STS no longer imports any EKS
package, including tests, and the handlers/sts -> handlers/eks layering
exception is removed.
2026-10-11 21:59:47 +10:00
joshsiv-mulga 6d0eddf0a4 docs: cite the bootstrap account identifier move in the migration record 2026-10-11 16:01:04 +10:00
joshsiv-mulga 3fdaa54d99 refactor: move the bootstrap account identifiers to foundation/aws/identifiers
DefaultAccountID and DefaultAccountName join GlobalAccountID as the fixed,
platform-reserved identity of the first human-facing bootstrap account.
Names and values are unchanged and stay fixed. Callers in daemon, gateway,
awsgw, accountteardown and the operator CLI use awsidentifiers; admin now
holds only the IAM credential generators.
2026-10-11 16:01:04 +10:00
joshsiv-mulga 947ca45245 docs: record the ECS contract and image-promotion moves and the remaining-roots matrix 2026-10-11 14:16:29 +10:00
joshsiv-mulga 263c581678 refactor: move system-image promotion to domains/ec2/image
Move images_promote.go and its tests from admin to domains/ec2/image:
SystemOwnerAlias, PromoteImageOpts/PromoteImageResult, PromoteSystemImage,
GetAMIMetadata and the private snapshot reader. Storage, ordering, errors and
promotion behaviour are unchanged; the daemon, gateway/spx and operator CLI
callers import ec2image. admin now holds only account IDs and key generators.
2026-10-11 13:00:20 +10:00
joshsiv-mulga a130f6b8ee refactor: move the ECS agent wire vocabulary to contracts/ecs/v1
Move handlers/ecs/bus to contracts/ecs/v1 (package ecsv1) with no legacy
shim. Subject strings, JSON fields and types are byte-identical; the old-agent
compatibility test moves unchanged and golden pins cover the remaining
message types. The README route table records that only service-reconcile
has a live producer today; the agent reuses these payload types through the
gateway actions. service-interfaces.yaml lists ecs.bus.* under the daemon.
2026-10-11 13:00:20 +10:00
joshsiv-mulga a8233794ab docs: cite the vpcd move commit in the migration record 2026-10-11 10:38:17 +10:00
joshsiv-mulga 7bf6c10d79 refactor: move vpcd to runtime/roles/vpcd 2026-10-11 10:38:17 +10:00
joshsiv-mulga eb53eded04 docs: cite the vpcd reader injection commit in the migration record 2026-10-11 09:50:56 +10:00
joshsiv-mulga e40e699e5f refactor: inject vpcd's IMDS instance-state readers from the instance state boundary 2026-10-11 09:49:44 +10:00
joshsiv-mulga c3f302423e docs: record the uniform extension-conflict rule
Adds design clause C8 for 0d7b4266d: a capable writer may not remove or
replace a known object, map entry or array whose stored value carries unknown
structured extension fields; removal without extensions stays allowed. C3 and
C5 point at it. The migration record gains the commit row, updated subsection
wording and a pointer-bump follow-up deferring make docs-portal to the Mulga
gitlink PR so the portal evaluates the proposed Spinifex revision.
2026-10-11 08:59:45 +10:00
joshsiv-mulga 0d7b4266d3 fix: refuse removals that would drop record extension fields
A capable writer may no longer remove or replace a known object, map entry or
array when its stored value carries members the writer's schema does not know.
The check that guarded changed arrays now runs wherever the merge drops or
overwrites a stored value instead of merging into it: a known member deleted or
set to null, a map entry removed or stored under a key the typed decode
rewrote, a value of another JSON kind, and a changed array. Encode returns
ErrRecordExtensionConflict naming the changed container and the first unknown
member. Removal stays allowed when the stored value has no extensions, and a
change merged member by member still keeps unknown members.
2026-10-11 08:59:14 +10:00
joshsiv-mulga c6fdc239ec docs: record the codec move and the array extension-conflict rule
Records 8db9771fb and c617914a1. T1 and Section 10 now name
domains/ec2/instance/state as the eventual owner of the persisted instance
record and its evolution, with vm.InstanceRecord as the temporary legacy type
it wraps. C3 replaces the changed-array limitation with the refusal: a changed
array with element extensions fails with ErrRecordExtensionConflict and is never
silently dropped. P2 stays a conservative rule until finalisation.
2026-10-11 08:42:08 +10:00
joshsiv-mulga c617914a1a fix: refuse array changes that would drop element extensions
A changed array is replaced whole by the writer's value, which silently lost
any member a newer writer had added inside its elements. Encode now inspects
the array as read: if any element carries a member its typed schema does not
know, at any depth, it returns ErrRecordExtensionConflict naming the array and
the first such member. Arrays without such members are still replaced whole,
unchanged arrays are still kept verbatim, and arrays inside unknown members are
never inspected.
2026-10-11 08:41:36 +10:00
joshsiv-mulga 8db9771fbf refactor: move the instance transition-record codec to domains/ec2/instance/state
The codec defines how the canonical persisted instance record evolves, with
its assignment and observation sections, which is EC2 instance ownership rather
than node-local VM supervision. It moves unchanged into its own package and
keeps wrapping the legacy vm.InstanceRecord alias and its spec and status
types, which stay in runtime/compute/vm for now.

The domain-to-runtime import is recorded as new dependency-rule debt with its
removal condition and allow-listed against that entry. The instance-state
ratchet now names the new file as an owner file; its allowlist is unchanged.
2026-10-11 08:37:56 +10:00
joshsiv-mulga f821e3eed8 docs: record slice 1b step 1 codec and awsgw rollout qualification
Records the codec added in e62f921d2 as design Section 10 (C1 to C7): its
surface, preservation at every object level, the changed-array limitation,
marker handling, the rejection classes and its capable-writers-only scope. C2
supersedes P1's top-level-only preservation for the codec. G19 records that
old read-only gateway consumers remain safe but DescribeInstanceStatus gains
unconfirmed/degraded visibility only after awsgw upgrades. The migration record
gains the commit row and states that no production path uses the codec yet.
2026-10-10 22:34:50 +10:00
joshsiv-mulga e62f921d2c feat: add instance transition-record codec with format marker
Adds DecodeTransitionRecord and TransitionRecord.Encode in the record's owner
package. A top-level format member selects the shape: absent or 1 decodes the
legacy record and is never given a marker on write; 2 also decodes the new
assignment and observation sections; an integer above 2 decodes its legacy
fields read-only and Encode refuses it; anything else is refused on decode.

Encode applies only the typed paths that differ from the decode-time snapshot
onto the input, so unknown members at every object level and unchanged known
fields keep their read form. A changed array is replaced whole. Decode refuses
malformed JSON, duplicate keys at any depth, keys that alias a known field by
case, trailing data and a non-object record, each with its own sentinel.

No production reader or writer uses the codec yet; the new types are not
part of vm.InstanceRecord, so every existing record encodes unchanged. The
codec file is registered as an owner file of the instance-state ratchet.
2026-10-10 22:33:49 +10:00
joshsiv-mulga 8fe3ea55d2 docs: record slice 1a activation-gate decisions 2026-10-10 22:16:28 +10:00
joshsiv-mulga daff8094b8 docs: record the instance-state transition-record design commit 2026-10-10 16:46:32 +10:00
joshsiv-mulga e9e11d531e docs: design the instance-state transition record and activation gate 2026-10-10 16:46:26 +10:00
joshsiv-mulga 830f704899 docs: record the narrowed instance-state ratchet commit 2026-10-10 16:38:50 +10:00
joshsiv-mulga 2a196f9ba9 test: narrow the instance-state ratchet to instance-state access
Stop freezing the JetStreamManager type and constructor, which also serve
cluster state; flag only instance-state methods, bucket and key identifiers,
record codecs and local-state access. The cluster-state-only consumers in
daemon_mgmt_ip.go and operator/cli/cluster.go leave the allowlist (73 to 67).
The self-test now proves a cluster-state use is ignored and an instance-state
method call is caught. Record the owner's disposition of the KV-unavailable
restore result in the migration record.
2026-10-10 16:38:47 +10:00
joshsiv-mulga c04222b27a docs: record ADR-0007 slice 0 instance-state inventory
Record the instance-state persistence formats, the nineteen direct
consumers by process and access kind, the characterised recovery and
schema outcomes, the generation-preservation matrix and the ratchet that
freezes the consumer set. Nothing conforms to ADR-0007 yet and vpcd stays
blocked.
2026-10-10 13:14:37 +10:00
joshsiv-mulga 6fa09d336c test: freeze direct instance-state consumers
Add an AST ratchet over production code that fails when a new file names
the daemon's instance-state bucket, record keys, local-file API, the raw
vm.InstanceRecord or its conversion, or a JetStreamManager instance-state
method. Today's 73 uses form a shrink-only allowlist keyed to the slice 0
inventory; stale entries fail, and a synthetic-source self-test proves the
matcher detects each kind of use and ignores declarations.
2026-10-10 13:11:10 +10:00
joshsiv-mulga 38fbd14993 test: characterise instance-state persistence, recovery and generation handling
Pin the current local-file bytes and schema failure modes, the instance
bucket configuration, record key and wire form, the bucket version gate,
generation preservation across every KV writer and the VM conversion, and
what a returning node does for a missing, foreign, terminal or unreadable
canonical record. These describe current behaviour, not the ADR-0007 target.
2026-10-10 13:06:57 +10:00
joshsiv-mulga cf9e8ce0d0 docs: record the network contract, projection client and utils removal 2026-10-09 19:23:17 +10:00
joshsiv-mulga fd88e8ceb0 refactor: move EBS encryption posture to domains/ec2/ebs/encryption and delete utils
The master-key loader was used only by EC2 image, instance and volume to
derive the AWS-visible Encrypted flag. It becomes ebsencryption.Enabled,
EC2's temporary reading of the provider's key-file setting; the cache,
empty-path meaning, error text and each caller's error handling are
unchanged (image still reports unencrypted when the key will not load).
Its successor is a providers/ebs capability reporting volume encryption
posture.

utils/ is now empty and deleted. The layering test drops utils and
services from the legacy roots, removes their allowlist entries and fails
if either retired root regains files. The S7 subject scan follows the
vpc.* literals from utils to contracts/network so its gap list stays
complete.
2026-10-09 19:21:34 +10:00
joshsiv-mulga 46f4489631 refactor: route the remaining fire-and-forget vpc.* producers through projection
CreateVPC/DeleteVPC, CreateSubnet/DeleteSubnet, AttachInternetGateway/
DetachInternetGateway, AddNATGateway/RemoveNATGateway, AddIGWRoute/
RemoveIGWRoute, GateSubnetEgress/UngateSubnetEgress and RemoveSystemEgress
were still publishing through natsmsg.PublishEvent or a raw nc.Publish
call with a bare subject string, left over from the mechanical slice 3
pass. Each now goes through its named projection.Client method instead,
so every vpc.* producer in the tree uses the typed client.
2026-10-09 19:13:52 +10:00
joshsiv-mulga 3aa82953e4 refactor: replace utils/vpcd_event.go with a typed network projection client
Every vpc.* call site now goes through domains/network/projection's
operation-specific methods instead of utils.AddNAT, utils.PublishNATEvent
or utils.RequestEvent, so no caller passes a bare subject string. Each
method states today's delivery mode by name: AddNAT stays strict,
AddNATBestEffort and AnnounceNATAdd/AnnounceNATDelete keep the gaps those
call sites already had, and RemoveNAT keeps the ErrNoResponders retry.

utils/vpcd_event.go and its tests are deleted; the behaviour they pinned
(wire shape, ack envelope, retry-on-no-responders) moved into the
projection package's own golden and nat tests, unchanged. utils/ now
holds only encryption.go, which this does not touch.
2026-10-09 19:02:32 +10:00
joshsiv-mulga 1050485944 refactor: use contracts/network/v1 for vpc.* producers and consumers
Switch every vpc.* producer (domains/ec2/vpc, eip, instance, natgw,
routetable, igw, handlers/elbv2, handlers/eks, daemon, systemvpc) and the
network/subscribers consumer onto contracts/network/v1 subjects, event
types and the shared AckEnvelope. contracts/network/v1 becomes the sole
owner of vpc.igw-attach/detach, replacing the duplicate definitions in
contracts/ec2/v1.

Deletes the duplicated local subject constants and event structs:
subscribers' Topic* consts and event structs, vpc.NATEvent and SGEvent,
eip's natEvent, natgw's natGatewayEvent, and eks's systemEgressEvent.
domains/ec2/vpc.SGRule survives because EC2 persists it directly in
SecurityGroupRecord; contracts/network/v1.SecurityGroupRule was expanded
in slice 2 to match it field-for-field so wire bytes are unchanged.

No lifecycle or reliability changes: the NAT add barrier, delete
retry-on-ErrNoResponders, and the inconsistent per-call-site NAT
delivery modes (EIP AssociateAddress and DeleteNetworkInterface both stay
fire-and-forget with no ack) are preserved exactly, as are the golden
wire literals pinned in the characterization tests.
2026-10-09 18:39:02 +10:00
joshsiv-mulga 77ac0f0f78 feat: add contracts/network/v1 for vpc.* realisation routes
Declare the versioned wire vocabulary for the vpc.* control-plane-to-
network-realisation boundary: subjects, payload types, the shared
{success,error} ack envelope and the vpcd-workers queue group, one
file per route family. No NATS I/O, no consumer switched over yet;
the package coexists with the fragmented originals pinned in the
prior characterization commit until slice 3 migrates call sites.
2026-10-09 16:33:03 +10:00
joshsiv-mulga 7a45e7a3e7 test: characterise vpc.* network-realisation routes
Pin every vpc.* subject, JSON payload shape and the request/reply
{success,error} ack envelope exactly as they exist today, ahead of
introducing contracts/network/v1. Covers the 22 subjects registered
by domains/network/subscribers, the vpc.igw-attach/detach subjects
and InternetGatewayEvent already owned by contracts/ec2/v1, the
utils.AddNAT/PublishNATEvent client wire behaviour including the
delete-nat retry-on-ErrNoResponders budget, and the EC2-side NATEvent
and SGEvent/SGRule duplicates in domains/ec2/vpc.

Also adds the six vpc.* route-table and system-egress subjects that
were missing from service-interfaces.yaml despite being live
production routes (vpc.add/delete-igw-route, vpc.gate/ungate-subnet-
egress, vpc.add/delete-system-egress).
2026-10-09 16:25:29 +10:00
joshsiv-mulga 08b98016bd docs: record viperblockd contract, adapter and role moves 2026-10-09 15:57:49 +10:00
joshsiv-mulga 7e70222f17 refactor: move the viperblockd role to runtime/roles/viperblockd
With the EBS adapter in providers/ebs/viperblock, services/viperblockd
holds only role supervision and the opt-in live data-path tests, so it
joins the other runtime roles. It was the last package under services/;
the frozen legacy list drops it.
2026-10-09 15:57:19 +10:00
joshsiv-mulga f17e93aedf refactor: split the Viperblock EBS adapter out of services/viperblockd
Move the EBS provider adapter (volume lease, fence, dirty marker,
nbdkit recovery, provider handlers, encryption key loading) to
providers/ebs/viperblock. services/viperblockd keeps only process
lifecycle and the NATS subscription wiring that starts and stops it.

The dirty-bucket read path moves to providers/ebs/viperblock/dirty as
a read-only ListUnsealedVolumes capability; the adapter's write, clear
and purge logic stays unexported in the adapter package. Delete
services/viperblockd/vbwire now that its vocabulary lives in
contracts/viperblockd/legacy/v1 and its I/O helpers live in dirty.

Update operator/cli, the runtime service registry, qemunbdd's config
comment and the integration test harness to the new import paths, and
drop the now-stale architecture/layering allowlist and frozen-package
entries for the deleted package. Behaviour, persisted keys and NATS
subjects are unchanged.
2026-10-09 15:52:29 +10:00
joshsiv-mulga 53697bc762 refactor: move viperblockd fence, abandon and dirty-marker vocabulary to contracts
Move the fence and abandon subjects/DTOs, the dirty bucket name,
DirtyRecord and the shared volume-key pattern out of vbwire and into
contracts/viperblockd/legacy/v1, alongside the package's existing
mount/unmount/delete/sync/config-update routes. vbwire keeps only the
I/O helpers (BindDirtyBucket, ListUnsealedVolumes), now built on the
contract vocabulary instead of defining it.

Record the approved dirty-marker ownership boundary in the contract
README: the Viperblock adapter is the sole producer, viperblockd was
only its transitional custodian. Document the fence and abandon routes
and the dirty bucket in docs/service-interfaces.yaml.
2026-10-09 15:09:22 +10:00
joshsiv-mulga 29f9ad07f1 test: pin viperblockd fence, abandon and dirty-marker wire shapes
Characterization tests against the current vbwire package, before it
moves to contracts/viperblockd/legacy/v1: both subject forms for the
fenced and abandon routes, golden JSON for VolumeFencedEvent,
VolumeAbandonRequest, VolumeAbandonResponse and DirtyRecord, the dirty
bucket name, and the volume-key pattern shared by the lease and dirty
stores.
2026-10-09 14:57:35 +10:00
joshsiv-mulga c2dfba8a60 docs: record operator/admin split in package-boundary migration 2026-10-09 14:51:38 +10:00
joshsiv-mulga 6b01f39e78 refactor: move operator-only admin code to operator/admin
Config generation, certificate issuance, host discovery, telemetry,
email validation and image import/removal/orphan-volume tooling move
to operator/admin. Legacy admin keeps only the symbols with
non-operator consumers: default account identifiers, AWS credential
generators and system image promotion.

foundation/pki and the tenant ACM CA are unchanged. The readAMI
helper and the removal test fixtures are duplicated privately in
legacy admin rather than shared across the split. The now-stale
operator/installer/ui -> admin layering allowlist entry is removed.
2026-10-09 14:51:14 +10:00
joshsiv-mulga abf734ef18 docs: qualify the migrate split and record admin and Viperblock findings
The migrate row is a split, not an exact ADR-0001 map row, and needs a map
clarification. Correct the stale admin consumers, record the operator-only
admin files as movable, the vbwire prerequisite for the Viperblock adapter
and the capability decision encryption.go needs.
2026-10-09 13:08:16 +10:00
joshsiv-mulga 8bf162fb73 refactor: move the default-VPC subject into contracts/ec2/v1
utils.SubjectEnsureDefaultVpc becomes ec2v1.EnsureDefaultVpcSubject, the
request/reply route between gateway account creation and the daemon.
Only the subject moves; the request and reply bodies are still anonymous
structs on each side and are recorded as such in the contract README.
The layering freeze now allows two utils files.
2026-10-09 13:07:40 +10:00
joshsiv-mulga 2bacd5ba22 fix: update paths left stale by the package moves
The MariaDB image stamp test read the RDS engine from
handlers/rds/engine_mariadb.go, so test-images failed in CI; it now reads
domains/rds/engine/mariadb.go. Comments citing moved files now name
their new locations.
2026-10-09 12:47:58 +10:00
joshsiv-mulga 382699c563 refactor: move the layering test to module-root architecture/
The ADR-0001 layering ratchet is repository governance tooling, not
Spinifex product code, so it sits beside tests/ rather than under the
spinifex/ product tree. make test, test-cover and test-race now include
./architecture/..., and the graph walk excludes architecture/.
2026-10-09 12:36:47 +10:00
joshsiv-mulga 66b37e7b5d docs: record test-package-check and nilaway as unverified gates 2026-10-09 12:29:56 +10:00
joshsiv-mulga d4415515ec chore: mark spf13/pflag as a direct dependency
operator/cli/wiring.go imports pflag directly. Same version; no upgrade.
2026-10-09 12:29:56 +10:00
joshsiv-mulga c3c8e28f86 refactor: drop the legacy handlers_ecr import alias
Import domains/ecr under its package name, or as ecrdomain where the
file also imports the AWS SDK ecr package, matching acmdomain. Also fix
a stale daemon file reference in the integration ECR daemon comment.
2026-10-09 12:17:01 +10:00