From c6fdc239ecc4d57bbb5740eb554d77965f0376ff Mon Sep 17 00:00:00 2001 From: joshsiv-mulga Date: Sun, 11 Oct 2026 08:42:08 +1000 Subject: [PATCH] docs: record the codec move and the array extension-conflict rule Records 8db9771fb and c617914a1. T1 and Section 10 now name domains/ec2/instance/state as the eventual owner of the persisted instance record and its evolution, with vm.InstanceRecord as the temporary legacy type it wraps. C3 replaces the changed-array limitation with the refusal: a changed array with element extensions fails with ErrRecordExtensionConflict and is never silently dropped. P2 stays a conservative rule until finalisation. --- docs/PACKAGE_BOUNDARY_MIGRATION.md | 9 +++++--- ...instance-state-transition-record-design.md | 22 ++++++++++++++----- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/docs/PACKAGE_BOUNDARY_MIGRATION.md b/docs/PACKAGE_BOUNDARY_MIGRATION.md index a3189e7d6..78df6442e 100644 --- a/docs/PACKAGE_BOUNDARY_MIGRATION.md +++ b/docs/PACKAGE_BOUNDARY_MIGRATION.md @@ -331,6 +331,8 @@ while the branch is in flight; they do not describe the umbrella repository's | `2a196f9ba` | (governance only) | — | ADR-0007 slice 0 ratchet narrowed to instance-state access: the `JetStreamManager` type and constructor are no longer matched, the cluster-state consumers INV-09 and INV-19 leave the allowlist, and 67 entries remain. | | `e9e11d531` | (design only) | — | ADR-0007 slice 1a: `docs/package-boundary/instance-state-transition-record-design.md` designs the transitional `i.` record (per-record `format` marker, `assignment` with generation, epoch and token, fenced `observation`, legacy mirror), the writer authority table, top-level unknown-member preservation, the activation gate (per-slot capability proof over the daemon and `vpcd`, narrowed by the recorded user decisions D1 to D5; Inactive, `PREPARED`, `ACTIVE`, `DEACTIVATING`, `FINALISED`; stamp raise against R0 restarts; fail closed when unreadable), the journal v2 rules and the slice 1b order. No code, schema, writer or test changed; no conformance claimed. | | `e62f921d2` | (new owner code only) | `spinifex/runtime/compute/vm/record_codec.go` | ADR-0007 slice 1b step 1: the instance transition-record codec (`DecodeTransitionRecord`, `TransitionRecord.Encode`) with the per-record `format` marker, the format-2 `assignment` and `observation` types, preservation of unknown members at every object level, and typed rejection of malformed, duplicate-key, case-alias, trailing-data and invalid-marker input. `vm.InstanceRecord` and every existing writer are unchanged, so wire bytes are unchanged (`TestCurrentBehaviour_RecordKeyAndWireForm` passes with its literal unchanged). No production reader or writer uses the codec yet and nothing conforms yet. The ratchet gains the codec as an owner file; the allowlist is unchanged. Full preflight passed except the recorded Go 1.27.0 advisories. | +| `8db9771fb` | `spinifex/runtime/compute/vm/record_codec.go` | `spinifex/domains/ec2/instance/state` | The codec moves unchanged to the package that is the eventual owner of the persisted instance record and its evolution; `vm.InstanceRecord` and its spec and status types stay in `runtime/compute/vm` as the temporary legacy types it wraps. The domain-to-runtime import is recorded below as `domains/ec2/instance/state` (dependency) debt and allow-listed against it; the instance-state ratchet names the new owner file and its allowlist is unchanged. Full preflight passed except the recorded Go 1.27.0 advisories. | +| `c617914a1` | (codec behaviour only) | — | A changed array whose elements as read carry an unknown member at any depth is refused with `ErrRecordExtensionConflict` instead of being replaced and losing the member; unchanged arrays and changed arrays without element extensions behave as before. Still no production caller. Full preflight passed except the recorded Go 1.27.0 advisories. | The EC2 contract row is intentionally different from the directory moves: it creates a compatibility boundary. `ec2.cmd.*` retains its deployed one-token NATS @@ -623,11 +625,12 @@ After narrowing, a `runtime/roles/awsgw` file constructing a manager and calling ### Slice 1b step 1: record codec -`e62f921d2` adds the record codec in the owner package; the design records it as Section 10 (C1 to C7) of `docs/package-boundary/instance-state-transition-record-design.md`. +`e62f921d2` adds the record codec, `8db9771fb` moves it to `domains/ec2/instance/state` and `c617914a1` adds the extension-conflict refusal; the design records it as Section 10 (C1 to C7) of `docs/package-boundary/instance-state-transition-record-design.md`. +`domains/ec2/instance/state` is the eventual owner of the persisted instance record and its evolution; `vm.InstanceRecord` in `runtime/compute/vm` is the temporary legacy type it wraps. No production reader or writer uses it, so every row of the generation matrix above still reads as characterised, and no path conforms to ADR-0007 yet. The codec protects capable writers only: a deployed binary still decodes into the typed record and drops unknown members, and excluding such writers remains the activation gate's job. -Within the codec, an unchanged array keeps unknown members inside its elements, and a changed array is replaced whole and loses them. -`architecture/instancestate` lists `spinifex/runtime/compute/vm/record_codec.go` as an owner file; no allowlist entry was added. +Within the codec, an unchanged array keeps unknown members inside its elements; a changed array with element extensions is refused with a typed extension-conflict error, never silently dropped, and a changed array without them is replaced whole. +`architecture/instancestate` lists `spinifex/domains/ec2/instance/state/record_codec.go` as an owner file; no instance-state allowlist entry was added. Operator-visible rollout qualification (design clause G19): old read-only gateway consumers remain safe, but DescribeInstanceStatus gains unconfirmed/degraded visibility only after awsgw upgrades. diff --git a/docs/package-boundary/instance-state-transition-record-design.md b/docs/package-boundary/instance-state-transition-record-design.md index 07ced9cb7..c2a4104c2 100644 --- a/docs/package-boundary/instance-state-transition-record-design.md +++ b/docs/package-boundary/instance-state-transition-record-design.md @@ -50,6 +50,7 @@ All of it is from the slice-0 inventory in `docs/PACKAGE_BOUNDARY_MIGRATION.md` **T1. Key and owner.** The record stays at `i.` in `spinifex-instance-state` and, once terminal, in `spinifex-terminated-instances`. Its codec, keys and format marker belong to the EC2 instance owner (ADR-0007 S1); slice 2 moves them behind `domains/ec2/instance`, and this design does not require that move first. +`domains/ec2/instance/state` is the eventual owner of the persisted record and its evolution, and holds the codec from slice 1b step 1 (C1); `vm.InstanceRecord` and its spec and status types in `runtime/compute/vm` are the temporary legacy types it wraps. Rationale: the key migration already landed (live 3→4); re-keying again would add a second crossing to every consumer INV-03 to INV-18. **T2. Format marker.** @@ -183,6 +184,7 @@ Superseded in part by C2: by user requirement the step 1 codec preserves unknown **P2. Future format changes add top-level members.** Because P1 preserves only top-level members, a later format adds sections at the top level and never adds a field inside `metadata`, `spec`, `status`, `assignment` or `observation` that an older writer must keep. A change that cannot follow P2 raises `format` and relies on T2's refusal. +C2 lets capable writers preserve deeper additions, but P2 keeps future format evolution disciplined while pre-expand writers exist, and it is revisited only at finalisation. **P3. Preservation alone is insufficient.** R0 binaries are deployed and cannot be patched: they drop every unknown member (E1), erase generations (E2, E3), keep writing through open handles (E5) and are ordinary writers on every node (INV-03, INV-05 to INV-08, INV-10). @@ -407,7 +409,9 @@ The user approved this design and decided the former open questions as follows. ## 10. Slice 1b step 1: record codec (C-series) -Implemented at `e62f921d2` in `spinifex/runtime/compute/vm/record_codec.go`, the owner package of `vm.InstanceRecord` today (T1); tests are in `record_codec_test.go` beside it. +Implemented at `e62f921d2` and moved at `8db9771fb` to `spinifex/domains/ec2/instance/state/record_codec.go`, with tests in `record_codec_test.go` beside it. +`domains/ec2/instance/state` is the eventual owner of the persisted record and its evolution (T1); `vm.InstanceRecord`, `vm.InstanceSpec` and `vm.InstanceStatus` stay in `runtime/compute/vm` as the temporary legacy types the codec wraps. +That import is recorded dependency-rule debt in `docs/PACKAGE_BOUNDARY_MIGRATION.md`, cleared when those types move into `domains/ec2/instance/state`. **C1. Codec surface.** `DecodeTransitionRecord` returns a `TransitionRecord` holding the typed `Record` (`vm.InstanceRecord`), the format-2 `Assignment` (T4) and `Observation` (T5), the decoded input and a snapshot of the typed fields as decoded. @@ -424,9 +428,11 @@ Writing back the value a field already had is indistinguishable from not changin Numbers are kept as their literal text, so integers beyond 2^53 and unknown decimal members do not lose precision. A typed map merges per entry; a key the typed decode normalised is written in the typed form only when that map changed. -**C3. Array limitation.** +**C3. Changed arrays.** An array whose typed value is unchanged is written back as read, including unknown members inside its elements. -An array the writer changed is replaced whole by the writer's value, so unknown members inside its elements are lost. +An array the writer changed is replaced whole by the writer's value only when no element of the array as read carries an unknown member at any depth. +Otherwise `Encode` refuses with `ErrRecordExtensionConflict`, naming the array's path and the first unknown member's path, so a changed array with element extensions is never silently dropped (`c617914a1`). +Arrays inside unknown members are never inspected and are always retained. No keyed merge is attempted; a format change that needs element-level preservation in a changing array must raise `format` (T2). **C4. Format marker handling.** @@ -450,6 +456,8 @@ Each is a distinct sentinel matched with `errors.Is`. | `ErrRecordCaseAliasKey` | A key that is not a known name but equals one under Unicode case folding at the same level, which `encoding/json` would otherwise decode into the known field | | `ErrRecordFormatNotInteger`, `ErrRecordFormatOutOfRange` | An invalid marker (C4) | +`Encode` has its own refusals: `ErrRecordFormatAhead` (C4), `ErrRecordSectionsNeedFormat` (C4) and `ErrRecordExtensionConflict` (C3). + Duplicates and case aliases are detected by a `json.Decoder` token walk against a schema reflected from the typed shape, because `encoding/json` keeps the last duplicate and matches names case-insensitively. Two unknown members that differ only in case from each other are both kept. @@ -461,6 +469,8 @@ Excluding R0 writers remains the activation gate's job (P3, G3, G16). Section 8 step 1 lists that slice-0 test as flipped; it does not flip in this step, because no writer uses the codec, and flips only when step 2 routes writers through it. **C7. Evidence.** -Table-driven tests cover the unmarked v1 round trip, single-path mutation, nested unknown members beside a mutated sibling, raw-form retention against a re-encoding precondition, `omitempty` removal, mutation shadowing, both array cases, the v2 round trip and section mutation, read-only newer formats, every rejection class and integer precision. -`FuzzTransitionRecordRoundTrip`, seeded with the golden inputs, asserts that decode never panics and that an accepted, unmutated record encodes to a semantically equal document; a 120 s run (about 940,000 executions) found no failure. -`architecture/instancestate` lists `record_codec.go` as an owner file; the allowlist is unchanged. +Table-driven tests cover the unmarked v1 round trip, single-path mutation, nested unknown members beside a mutated sibling, raw-form retention against a re-encoding precondition, `omitempty` removal, mutation shadowing, the array cases (unchanged with element extensions, changed with known members only, changed scalars, and the extension conflict at element level and one level deeper), the v2 round trip and section mutation, read-only newer formats, every rejection class and integer precision. +`FuzzTransitionRecordRoundTrip`, seeded with the golden inputs, asserts that decode never panics and that an accepted, unmutated record encodes to a semantically equal document; a 120 s run (about 940,000 executions) found no failure, and a 60 s rerun after the conflict check (about 660,000 executions) found none. +An unmutated record cannot raise the extension conflict, so the fuzz invariant is unchanged. +`architecture/instancestate` lists `spinifex/domains/ec2/instance/state/record_codec.go` as an owner file; its allowlist is unchanged. +`architecture/layering` allows the edge to `runtime/compute/vm` against the recorded debt entry for `domains/ec2/instance/state`.