diff --git a/docs/PACKAGE_BOUNDARY_MIGRATION.md b/docs/PACKAGE_BOUNDARY_MIGRATION.md index 665371586..b9f13161b 100644 --- a/docs/PACKAGE_BOUNDARY_MIGRATION.md +++ b/docs/PACKAGE_BOUNDARY_MIGRATION.md @@ -300,6 +300,7 @@ while the branch is in flight; they do not describe the umbrella repository's | `ca1dd9aa9` | — | — | RDS DB parameter group characterization before extraction (ADR-0003 S1, ADR-0004): persisted meta and override field names, pre-existing JSON decodes, default groups synthesised and never stored, repeated delete is not found, a pending attachment blocks delete, a dynamic `pending-reboot` override is still applied live, propagation reaches a stopped instance. Present contract and gaps in `docs/package-boundary/rds-db-parameter-group-lifecycle.md`; propagation to instances is recorded as instance-side realization outside the group, and no conformance is claimed. | | `d9c3f3ca6` | `spinifex/handlers/rds` (`DBParameterGroupRecord`, `DBParameterRecord`, the parameter-group key helpers, `ListDBParameterGroupNames`, `ListDBParameterOverrides`, `getDBParameterGroup` and the create/modify/delete writes) | `spinifex/domains/rds/parametergroup` (`Owner`, `Record`, `Override`, `MetaKey`, `ParamsPrefix`, `ParamKey`, `Prefix`) | ADR-0004 owner extraction (contract in `docs/package-boundary/rds-db-parameter-group-lifecycle.md`). Stored keys and JSON field names unchanged. Delete reaches instances through the owner-declared `Dependants`; instance-side resolution reads through the instance-declared `parameterGroupValues`. `SetOverrides` refuses an unknown group. The handler keeps AWS validation (it refuses every `default.*` name; the owner refuses only a real engine default), projection, paging and propagation. Proves ownership and dependency direction only; no conformance claimed. | | `52de5e517` | — | — | EKS add-on characterization before extraction (ADR-0003 S1, ADR-0004): persisted record and manifest keys and field names, durable CREATING and staged descriptor, installer failure as CREATE_FAILED (update too), update returning Successful while UPDATING, mismatched-version reports accepted, delete erasing both keys while answering DELETING, the ignored client token, no PassRole check on serviceAccountRoleArn, and the exact internal add-ons body (Go field names, not the declared json tags) and agent TSV. Present contract and gaps in `docs/package-boundary/eks-addon-lifecycle.md`; no conformance claimed. | +| `d89963054` | `spinifex/handlers/eks` (`addon_catalog.go`, `addon_state.go`, `addon_installer.go`, the add-on key helpers, the create/update/delete transitions, `markAddonFailed`, `nextAddonStatus` and report application) | `spinifex/domains/eks/addon` (`Owner`, `Record`, `Manifest`, `Status`, `Spec`, `Installer`, `StagingInstaller`, `Report`, `Key`, `ManifestKey`, `Prefix`) | ADR-0004 owner extraction (contract in `docs/package-boundary/eks-addon-lifecycle.md`). The owner is the sole writer; stored keys and JSON fields unchanged. The handler checks cluster existence (`acctKVForCluster`) and keeps AWS validation, error mapping and projection. `ClusterReconciler` remains the subscriber host and delegates through the reconciler-declared `addonReports`; node groups call `EnsureGPUDevicePlugin`. Guest wire types stay in `handlers/eks`. Behaviour preserving: proves ownership and dependency direction only; every lifecycle gap remains. | The EC2 contract row is intentionally different from the directory moves: it creates a compatibility boundary. `ec2.cmd.*` retains its deployed one-token NATS @@ -401,6 +402,7 @@ These residuals were recorded by individual slices and must be resolved once the - `handlers/ecs` → `domains/ecs/taskdefinition` (legacy edges): `tags.go` reads and writes the revision record through its generic tag registry; this clears when ECS tagging invokes an owner-provided tag mutation capability, and moving `tags.go` to `domains/ecs/awsapi` alone does not clear it. `taskDefinitionResolver` returns the persisted `Record` to task and service launch code; when the task and service owners are extracted, replace the record crossing that boundary with a consumer-owned resolved task-definition (launch-spec) projection. Moving packages has not made the record private. - `domains/ecs/taskdefinition` (deliberate duplication, not debt): private `getJSON`, `putJSON` and `keysWithPrefix` copy the raw-KV helpers in `handlers/ecs` so the extraction keeps ECS's get/put/list error and recovery semantics. Revisit only if a later, tested state abstraction proves it preserves those semantics; do not migrate ECS to `kvstore` merely to remove them. - `handlers/eks` → `domains/eks/access` (legacy edges): `ResolveTokenReview` opens the EKS bucket in the gateway process and reads records through the exported `access.Get`; this clears when token review asks the owner across a contract rather than opening the bucket itself. `DeleteClusterPrefix` erases access-entry keys as part of its blanket cluster sweep; this clears when cluster delete invokes an owner-provided removal of the cluster's entries. The owner returns the persisted `Record` to the AWS adapter and token review; replace it with consumer-owned projections when those consumers are extracted. +- `handlers/eks` → `domains/eks/addon` (legacy edges): `DeleteClusterPrefix` erases add-on records and manifests in its blanket cluster sweep; this clears when cluster delete invokes an add-on-owned teardown capability. `stageGPUDeviceAddon` is a temporary node-group-to-add-on orchestration edge: best-effort, errors logged and swallowed, never undone on node-group delete; it clears when the node-group owner is extracted and the dependency is designed. `ClusterReconciler` hosts the add-on status subscription; it clears when add-on delivery observation has its own host under the lifecycle correction. Cluster existence is checked in the handler, not through an owner-declared capability; it moves into the owner when cluster admission is defined (today status is ignored). Guest wire types (`AddonStatusReport`, `AddonDeliveryPhase`, `AddonStatusSubject`, `StagedAddonManifest`) remain in `handlers/eks` and are duplicated in `agents/eks/gatewayfetch`; this clears when they move to `contracts/eks/v1`. - `agents/rds/agent` (dependency, narrowed): engine metadata now arrives through the agent-declared `EngineCatalog`, but the agent still imports `handlers/rds` for the command protocol, wire and record types (`Command*`, `Parameter`, `EngineHealth*`, `BootstrapMode*`) and API output types. This clears when those move behind a guest/controller wire contract. - `gateway/rds` → `handlers/rds` (`InstanceSizing`): the gateway obtains its `Sizing` through the RDS handler package's adapter rather than its own composition root. This clears when the gateway's RDS composition moves to `runtime/roles/awsgw` and builds `Sizing` there. - `docs/service-interfaces.yaml` (coverage gap): no `ecs.*` subjects are listed and the manifest lint does not cover them, so ECS publisher/subscriber edges play no part in suite selection. Add them only after tracing the actual publishers and subscribers. diff --git a/docs/package-boundary/eks-addon-lifecycle.md b/docs/package-boundary/eks-addon-lifecycle.md index 9973effff..28435a023 100644 --- a/docs/package-boundary/eks-addon-lifecycle.md +++ b/docs/package-boundary/eks-addon-lifecycle.md @@ -17,24 +17,24 @@ There is no generation, so the record mixes desired fields with the latest obser |---|---|---| | Identity | Cluster name plus add-on name; one add-on per name per cluster. ARN `arn:aws:eks:{region}:{account}:addon/{cluster}/{addon}` (`foundation/aws/arn/eks.go` `FormatEKSAddon`), deterministic by name, with no per-incarnation suffix. The gateway authorizer derives the same ARN (`gateway/eks/authz.go` `addonARN`). | `handlers/eks/addon_lifecycle_test.go` `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `gateway/eks/authz_test.go` `TestResourceARNsFidelity` | | Scope | Account and Region, within one cluster: keys live under `clusters/{cluster}/addons/` in the `eks-account-{account}` bucket. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestDeleteClusterPrefix_SweepsAddonRecordsAndManifestsScopedToCluster` | -| Durable record | `clusters/{cluster}/addons/{addon}` holds `AddonRecord` (`handlers/eks/addon_state.go`): `addonName`, `addonVersion`, `status`, `serviceAccountRoleArn`, `configurationValues`, `health`, `arn`, `tags`, `createdAt`, `modifiedAt` (the optional ones `omitempty`). `clusters/{cluster}/addons/{addon}/manifest` holds `StagedAddonManifest` (`handlers/eks/addon_installer.go`): `addonName`, `addonVersion`, `serviceAccountRoleArn`, `configurationValues`. Listing skips the manifest sub-key by segment count. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestAddonRecord_DecodesHandWrittenJSON`, `handlers/eks/addon_state_test.go` `TestAddonRecord_RoundTrip`, `TestListAddonRecords_SkipsManifestSubKeys` | -| Writers | Record: `CreateAddon` (unconditional put), `UpdateAddon` and `markAddonFailed` (revision CAS), `DeleteAddon` via `DeleteAddonRecord` (get then delete), `ClusterReconciler.applyAddonStatusReport` (revision CAS), `stageGPUDeviceAddon` through `CreateAddon`, and the cluster prefix purge (`DeleteClusterPrefix`). Manifest: `stagingInstaller.Install` (unconditional put on create and update), `stagingInstaller.Uninstall`, `DeleteAddonRecord` and the prefix purge. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestUpdateAddon_ReportsSuccessfulWhileRecordIsUpdating`, `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest`, `TestDeleteClusterPrefix_SweepsAddonRecordsAndManifestsScopedToCluster`, `handlers/eks/addon_state_test.go` `TestCasUpdateAddon`, `TestDeleteAddonRecord_RemovesRecordAndManifest` | -| Readers | `ListAddons`, `DescribeAddon`, `CreateAddon` (existence check) and the reconciler read the record. `ListStagedAddonManifests` reads every `/manifest` key for the cluster; it is served on `eks.ListStagedAddonManifests` (`daemon/daemon.go`) and reached only through the gateway's internal-addons route. | `handlers/eks/addons_test.go` `TestAddon_DescribeListDelete`, `TestListStagedAddonManifests` | -| Catalogue | Static in-binary catalogue (`handlers/eks/addon_catalog.go`): `aws-load-balancer-controller`, `argocd`, `aws-ebs-csi-driver`, and hidden-but-creatable `nvidia-device-plugin` and `spinifex-noop`; one version each, the first being the default. Create and update reject unknown names and versions with `InvalidParameterValue`. | `handlers/eks/addon_catalog_test.go`, `handlers/eks/addons_test.go` `TestDescribeAddonVersions_ReturnsCatalog`, `TestCreateAddon_UnknownAddonRejected`, `TestCreateAddon_UnknownVersionRejected`, `TestUpdateAddon_UnknownVersionRejected` | +| Durable record | `clusters/{cluster}/addons/{addon}` holds `Record` (`domains/eks/addon/record.go`): `addonName`, `addonVersion`, `status`, `serviceAccountRoleArn`, `configurationValues`, `health`, `arn`, `tags`, `createdAt`, `modifiedAt` (the optional ones `omitempty`). `clusters/{cluster}/addons/{addon}/manifest` holds `Manifest` (`domains/eks/addon/record.go`): `addonName`, `addonVersion`, `serviceAccountRoleArn`, `configurationValues`. Listing skips the manifest sub-key by segment count. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestAddonRecord_DecodesHandWrittenJSON`, `domains/eks/addon/store_test.go` `TestRecord_RoundTrip`, `TestList_SkipsManifestSubKeys` | +| Writers | Record: the owner in `domains/eks/addon/owner.go`, reached from the `CreateAddon`, `UpdateAddon` and `DeleteAddon` handlers, the cluster reconciler's report relay and the GPU node group launch: `Owner.Create` (unconditional put), `Owner.Update` and its failure marking (revision CAS), `Owner.Delete` (get then delete), `Owner.ApplyReport` (revision CAS), `Owner.EnsureGPUDevicePlugin` through `Owner.Create`; and the cluster prefix purge (`DeleteClusterPrefix`). Manifest: `StagingInstaller.Install` (unconditional put on create and update), `StagingInstaller.Uninstall` (`domains/eks/addon/delivery.go`), `Owner.Delete` and the prefix purge. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestUpdateAddon_ReportsSuccessfulWhileRecordIsUpdating`, `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest`, `TestDeleteClusterPrefix_SweepsAddonRecordsAndManifestsScopedToCluster`, `domains/eks/addon/store_test.go` `TestCasUpdate`, `TestDeleteRecord_RemovesRecordAndManifest` | +| Readers | `ListAddons`, `DescribeAddon`, `CreateAddon` (existence check) and the reconciler read the record. `ListStagedAddonManifests` reads every `/manifest` key for the cluster (`addon.ListManifests`); it is served on `eks.ListStagedAddonManifests` (`daemon/daemon.go`) and reached only through the gateway's internal-addons route. | `handlers/eks/addons_test.go` `TestAddon_DescribeListDelete`, `TestListStagedAddonManifests` | +| Catalogue | Static in-binary catalogue (`domains/eks/addon/catalog.go`): `aws-load-balancer-controller`, `argocd`, `aws-ebs-csi-driver`, and hidden-but-creatable `nvidia-device-plugin` and `spinifex-noop`; one version each, the first being the default. Create and update reject unknown names and versions with `InvalidParameterValue`. | `domains/eks/addon/catalog_test.go`, `handlers/eks/addons_test.go` `TestDescribeAddonVersions_ReturnsCatalog`, `TestCreateAddon_UnknownAddonRejected`, `TestCreateAddon_UnknownVersionRejected`, `TestUpdateAddon_UnknownVersionRejected` | | Desired state | Version, `serviceAccountRoleArn`, `configurationValues` and create-time tags. `resolveConflicts`, `podIdentityAssociations` and `clientRequestToken` are accepted and dropped; `preserve` on delete is ignored. `configurationValues` reaches the guest but the agent discards it (`mulga-eks-addon-sync.sh` `sync_once`), so it has no effect. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest` | | Observed state | `status` and `health` are overwritten by guest reports on the same record as the desired fields. `health` is the latest failure message, projected as one `AddonIssue` with only `message`. Statuses held: `CREATING`, `ACTIVE`, `UPDATING`, `DEGRADED`, `CREATE_FAILED`; `DELETING` is defined but never stored. | `TestAddonRecord_DecodesHandWrittenJSON`, `handlers/eks/addon_status_reconciler_test.go` `TestApplyAddonStatusReport` | | Generation | None. Reports carry `version` and `ts`, and neither is read: any report for the add-on name applies to whatever is currently desired. | `TestAddonStatusReport_MismatchedVersionIsAccepted` | | Create | Validate against the catalogue, require the cluster meta (`acctKVForCluster`), get then put a `CREATING` record, then stage the manifest. Returns `CREATING`. | `TestAddonPersistedLayout_RecordAndManifestKeysAndFieldNames`, `handlers/eks/addons_test.go` `TestCreateAddon_CreatesStagingAndDefaultsVersion`, `TestStagingInstaller_StagesManifest`, `TestCreateAddon_UnknownClusterIsNotFound` | | Update | Validate the version, CAS the new version, config and role onto the record with status `UPDATING`, re-stage the manifest, and return an `Update` with status `Successful`, type `AddonUpdate` and `id` equal to the add-on ARN (the same for every update). Allowed from any stored status. | `TestUpdateAddon_ReportsSuccessfulWhileRecordIsUpdating`, `handlers/eks/addons_test.go` `TestUpdateAddon_ChangesVersionAndReinstalls`, `TestUpdateAddon_MissingIsNotFound` | -| Readiness | `ready` maps any non-`ACTIVE` status to `ACTIVE` and clears `health`; `applied` and unknown phases change nothing. The agent reports `ready` when K3s records the add-on's `gvks` annotation, or after a 90 s grace once the bundle's workloads are rolled out; the check is per rendered file, not per version. | `handlers/eks/addon_status_reconciler_test.go` `TestNextAddonStatus`, `TestApplyAddonStatusReport`, `tests/e2e/eks/eks_test.go` `TestEKS/AddonDelivery` | +| Readiness | `ready` maps any non-`ACTIVE` status to `ACTIVE` and clears `health`; `applied` and unknown phases change nothing. The agent reports `ready` when K3s records the add-on's `gvks` annotation, or after a 90 s grace once the bundle's workloads are rolled out; the check is per rendered file, not per version. | `domains/eks/addon/owner_test.go` `TestNextStatus`, `TestOwnerApplyReport`, `handlers/eks/addon_status_reconciler_test.go` `TestApplyAddonStatusReport`, `tests/e2e/eks/eks_test.go` `TestEKS/AddonDelivery` | | Failure | Staging failure on create or update marks the record `CREATE_FAILED` with the error as `health` and returns the raw error. A `failed` report maps `ACTIVE` to `DEGRADED`, keeps `DEGRADED` and `CREATE_FAILED`, and maps anything else (including `UPDATING`) to `CREATE_FAILED`. Neither failure state is terminal: a later `ready` report returns the add-on to `ACTIVE`. | `TestCreateAddon_InstallerFailureLeavesDurableCreateFailedRecord`, `TestUpdateAddon_InstallerFailureMarksCreateFailed`, `TestAddonStatusReport_MismatchedVersionIsAccepted`, `handlers/eks/addons_test.go` `TestCreateAddon_InstallerFailureMarksCreateFailed` | -| Delete | Read the record, build a `DELETING` response from it, delete the manifest, then delete the record and manifest again in `DeleteAddonRecord`. Both keys are gone when the call returns. The guest later notices the missing manifest, removes the rendered file and runs `kubectl delete --wait=false`; it sends no report, so nothing observes removal. | `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest`, `TestListStagedAddonManifests`, `tests/e2e/eks/eks_test.go` `TestEKS/AddonDelivery` | +| Delete | Read the record, build a `DELETING` response from it, delete the manifest, then delete the record and manifest again (`Owner.Delete`). Both keys are gone when the call returns. The guest later notices the missing manifest, removes the rendered file and runs `kubectl delete --wait=false`; it sends no report, so nothing observes removal. | `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest`, `TestListStagedAddonManifests`, `tests/e2e/eks/eks_test.go` `TestEKS/AddonDelivery` | | Guest fetch | `GET /clusters/{cluster}/internal-addons/{accountId}` (`gateway/eks.go`, `ListInternalAddons`) is admitted only for a control-plane agent serving that cluster (`AuthorizeInternal`), relays to `eks.ListStagedAddonManifests`, and wraps the result as `internalAddonsOutput`. The body is written by the SDK REST-JSON marshaller, which ignores the `json` tags: it is `{"Addons":[{"AddonName","AddonVersion","ServiceAccountRoleArn","ConfigurationValues"}]}` with empty strings present, sorted by add-on name. | `gateway/eks_addon_wire_test.go` `TestEKSRequest_InternalAddonsWireBody`, `gateway/eks_authz_test.go` `TestEKSRequest_InternalRoutesAdmitTheCPAgentPrincipal`, `gateway/eks/internal_authz_test.go` `TestAuthorizeInternal_RejectsClusterTheCallerDoesNotServe` | | TSV conversion | `eks-gateway-fetch -resource addons` (`agents/eks/gatewayfetch`) decodes the body into its own copy of the shape (matching the Go field names only because `encoding/json` is case-insensitive) and writes `name\tversion\troleArn\tbase64(config)` per add-on; an empty set writes nothing. It retries the GET up to 30 times at 5 s. | `agents/eks/gatewayfetch/addons_wire_test.go` `TestEmitAddonsTSV_GatewayWireBody`, `agents/eks/gatewayfetch/emit_test.go` `TestEmitAddonsTSV`, `TestEmitAddonsTSV_Empty`, `agents/eks/gatewayfetch/fetch_test.go` `TestRun_RendersAddons`, `TestRun_FailsAfterAttemptBudget` | | Guest render and report | Every 30 s (`ADDON_SYNC_INTERVAL`) the agent renders each staged add-on, substituting the role ARN into web-identity env when set, then publishes `{"addon","version","phase","message","ts"}` through `eks-gateway-publish -channel addon`. The gateway relays the raw payload to core NATS `eks.addon.{account}.{cluster}.status` (`gateway/eks/internal_publish.go`). A fetch failure keeps the current render. | `gateway/eks/internal_publish_test.go` `TestPublishInternal_RelaysToSubjects`, `agents/eks/gatewaypublish/publish_test.go` `TestRun_PublishesWrappedPayload`, `TestRun_FailsAfterAttemptBudget` | | Retry meaning | Reports are level-triggered: each tick re-sends the current phase for every staged add-on, so a lost report is repeated on the next tick. `eks-gateway-publish` retries the POST up to 30 times at 5 s and the agent ignores its failure. Core NATS keeps nothing: a report published while no reconciler holds the cluster's lease, or after the reconciler stops on a terminal cluster status, is dropped. | none | | Report ordering | The reconciler subscribes plainly (no queue group) on the lease holder and applies reports in arrival order through a revision CAS. Nothing orders a report against a later update: a report produced for the previous version settles the new `UPDATING` record, and a stale `failed` turns it `CREATE_FAILED`. A report for a deleted add-on is a no-op. | `TestAddonStatusReport_MismatchedVersionIsAccepted`, `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest` | -| Cross-resource dependency | A GPU node group's launch calls `stageGPUDeviceAddon`, which calls `CreateAddon` for `nvidia-device-plugin` and ignores `ResourceInUse` and every other error (logged only). Deleting the node group never unstages it, and an existing record in any status, including `CREATE_FAILED`, blocks re-staging. | `handlers/eks/nodegroup_test.go` `TestCreateNodegroup_GPUNodegroupStagesDevicePluginAddon`, `TestCreateNodegroup_NonGPUNodegroupDoesNotStageDevicePluginAddon`, `tests/e2e/gpu/eks_gpu_test.go` `TestEKSGPUPodExposure` | +| Cross-resource dependency | A GPU node group's launch calls `stageGPUDeviceAddon`, which checks the cluster exists and calls `Owner.EnsureGPUDevicePlugin` for `nvidia-device-plugin`; an existing record is a no-op and every other error is logged only. Deleting the node group never unstages it, and an existing record in any status, including `CREATE_FAILED`, blocks re-staging. | `handlers/eks/nodegroup_test.go` `TestCreateNodegroup_GPUNodegroupStagesDevicePluginAddon`, `domains/eks/addon/owner_test.go` `TestOwnerEnsureGPUDevicePlugin_StagesDefaultVersionOnceAndKeepsAnyExistingRecord`, `TestCreateNodegroup_NonGPUNodegroupDoesNotStageDevicePluginAddon`, `tests/e2e/gpu/eks_gpu_test.go` `TestEKSGPUPodExposure` | | Idempotency | `clientRequestToken` is ignored. A duplicate create, including an SDK retry with the same token, returns `ResourceInUseException`. A repeated update re-stages and returns another `Successful` with the same id. A repeated delete returns `ResourceNotFoundException`. | `TestCreateAddon_RepeatedClientRequestTokenIsResourceInUse`, `TestUpdateAddon_ReportsSuccessfulWhileRecordIsUpdating`, `TestDeleteAddon_ReturnsDeletingAfterErasingRecordAndManifest`, `handlers/eks/addons_test.go` `TestCreateAddon_CreatesStagingAndDefaultsVersion` | | Absent resource | An unknown cluster returns `ResourceNotFoundException` for every action, including the internal manifest listing. A missing add-on returns `ResourceNotFoundException` from describe, update and delete. | `handlers/eks/addons_test.go` `TestCreateAddon_UnknownClusterIsNotFound`, `TestDescribeDeleteAddon_MissingIsNotFound`, `TestUpdateAddon_MissingIsNotFound`, `TestListStagedAddonManifests` | | Dependencies | Only the cluster meta's existence is checked; cluster status is not read, so add-ons can be created on a `DELETING` cluster. | `TestAddonActionsIgnoreClusterLifecycleStatus` | @@ -45,7 +45,7 @@ There is no generation, so the record mixes desired fields with the latest obser ## Gaps -1. Create is get-then-put (`handlers/eks/addons.go` `CreateAddon`), so concurrent creates of one add-on both succeed and the later write wins; internal NATS redelivery of a create returns `ResourceInUseException`. +1. Create is get-then-put (`domains/eks/addon/owner.go` `Owner.Create`), so concurrent creates of one add-on both succeed and the later write wins; internal NATS redelivery of a create returns `ResourceInUseException`. 2. No client-token support: `clientRequestToken` is dropped on create and update, so an SDK retry of a successful create fails instead of returning the add-on as AWS does. 3. Delivery reports ignore their supplied `version` (and `ts`), so a stale or mismatched report settles the current desired state; there is no generation or incarnation fence. 4. `UpdateAddon` returns `Successful` while the add-on remains `UPDATING`, and the update `id` is the add-on ARN. AWS returns an in-progress update with its own id, tracked through `DescribeUpdate`; not verified against live AWS. @@ -55,9 +55,9 @@ There is no generation, so the record mixes desired fields with the latest obser 8. Fixed: `serviceAccountRoleArn` now requires `iam:PassRole` on `CreateAddon` and `UpdateAddon`, matching `CreateCluster`/`CreateNodegroup` (`gateway/eks/passrole.go`). 9. AWS `UPDATE_FAILED` and `DELETE_FAILED` are not represented: a failed update becomes `CREATE_FAILED` and a failed delete leaves the prior status. 10. The add-on ARN lacks the per-incarnation suffix AWS appends, so a recreated add-on reuses its predecessor's ARN. This is a compatibility gap; extraction must keep the current ARN. -11. The guest/controller wire shape is duplicated and unversioned: `StagedAddonManifest` and `internalAddonsOutput` in the handler and gateway, `stagedAddon` in `agents/eks/gatewayfetch`, `AddonStatusReport` in `handlers/eks/state_report.go` and a `printf` in the shell agent. The HTTP body uses Go field names because the REST-JSON marshaller ignores the `json` tags, and the agent decodes it only through case-insensitive matching. +11. The guest/controller wire shape is duplicated and unversioned: `Manifest` in the add-on owner, `StagedAddonManifest` and `internalAddonsOutput` in the handler and gateway, `stagedAddon` in `agents/eks/gatewayfetch`, `AddonStatusReport` in `handlers/eks/state_report.go` and a `printf` in the shell agent. The HTTP body uses Go field names because the REST-JSON marshaller ignores the `json` tags, and the agent decodes it only through case-insensitive matching. 12. Reports travel over core NATS to a lease-held subscription, so they are lost whenever no reconciler runs for the cluster; delivery relies on the agent re-sending every tick. 13. The record mixes desired fields with the latest observation (`status`, `health`), and `ready` lifts `CREATE_FAILED` back to `ACTIVE`. -14. GPU auto-staging crosses from the node group into the add-on through the public `CreateAddon`, swallows every error, and is never undone. +14. GPU auto-staging crosses from the node group into the add-on through `Owner.EnsureGPUDevicePlugin`, swallows every error, and is never undone. 15. `ListAddons` ignores `maxResults` and `nextToken`; tag actions on add-on ARNs return `NotImplemented`. 16. No ADR-0003 S5 failure or recovery evidence.