diff --git a/docs/release-control/v6/internal/records/api-runtime-decomposition-2026-08-21.md b/docs/release-control/v6/internal/records/api-runtime-decomposition-2026-08-21.md index 43fe11f17..cdee194c5 100644 --- a/docs/release-control/v6/internal/records/api-runtime-decomposition-2026-08-21.md +++ b/docs/release-control/v6/internal/records/api-runtime-decomposition-2026-08-21.md @@ -4,7 +4,7 @@ - Subsystem: `api-contracts` - Baseline commit: `9dac68fd6` - Boundary commit: `b70a658c8992158b8d0f8a6408053afa2915a8d7` -- Result: production decomposition passed; PVE acceleration remains open +- Result: production decomposition and chart/resource PVE acceleration passed ## Production Boundaries @@ -104,12 +104,135 @@ resource/router integration tests, including: - `TestLoad_500Node_ConcurrentResources` - `TestLoad_500Node_MixedEndpoints` -The chart handlers and response builders currently span `internal/api/router.go` -and `internal/api/types.go`; the resource query path shares one -`ResourceHandlers` type with action lifecycle and operator-state mutation. +At that boundary revision, chart handlers and response builders still spanned +`internal/api/router.go` and `internal/api/types.go`; the resource query path +still shared one `ResourceHandlers` type with action lifecycle and +operator-state mutation. Moving only the expensive tests, copying list filters into a new package, or wrapping root behavior in callbacks would create an artificial boundary and is not acceptable. The next decomposition must separate the production chart and resource-query services, give each a lower-level monitor/registry interface, move their full contract/load tests with them, and leave root with route composition and cross-domain proof only. + +## Chart and Resource Query Continuation + +The continuation was mapped from `418402bf9eeb4ddb5a2f437ac9b93290bc96f103`, +then rebased onto the corrected release selector at +`1327dddad5200f07271e16abdf4dd83fa1f2eb4f`. Implementation commit +`2e4bd36ec5fd0121a2a4cc3564be89eeb9d5c750` completed the two production +boundaries identified above: + +- `internal/api/chartapi` owns all six chart handlers, monitor history queries, + aggregation/downsampling, response contracts, per-tenant response caches, + singleflight coordination, and the chart handler/contract suite. +- `internal/api/resourceapi` owns registry construction, tenant stores, + unified seed ingestion, list/detail/facet/timeline queries, discovery and + metrics projections, response contracts, and the 500-node resource load + proof. +- Root Router and `ResourceHandlers` retain source-compatible delegates and + type aliases. Root continues to own route authorization, action lifecycle, + operator-state mutation, and cross-domain integration proof. Neither domain + package imports root or delegates behavior back through generic callbacks. + +The root package changed from 587 to 581 Go files and from 3,755 to 3,645 +top-level tests. Exactly 110 existing test/benchmark entry points moved with +their production domains: 35 to `chartapi` and 75 to `resourceapi`. The moved +resource set includes `TestLoad_500Node_ConcurrentResources`; the moved chart +set includes 17 end-to-end handler tests that previously ran serially in the +root test binary. + +The exact local warm correctness run used the same command form as the earlier +record: + +```sh +/usr/bin/time -lp sh -c 'go test -count=1 ./internal/api/...' +``` + +It passed in `118.77s` wall, `177.00s` user, and `30.73s` system with +`3,455,729,664` bytes maximum RSS. Average CPU occupancy was 1.749 cores +(`17.49%` of the 10-logical-CPU local host). Concurrent package elapsed times +were `114.725s` root, `96.313s` config, `22.665s` chart, and `8.971s` +resource. The prior recorded local boundary run was `153.96s`; because the +continuation baseline also includes later mainline release fixes, that +`35.19s` difference is supporting evidence rather than the controlled PVE +before/after result. + +Full race qualification passed with no detector report or panic: + +```sh +/usr/bin/time -lp sh -c 'go test -race -timeout 20m -count=1 ./internal/api/...' +``` + +The race run used `667.59s` wall, `885.43s` user, `105.78s` system, and +`3,728,080,896` bytes maximum RSS. Package elapsed times were `649.623s` root, +`123.379s` chart, `104.904s` config, and `14.749s` resource. + +A static equivalence audit extracted every top-level `Test`, `Benchmark`, and +`Example` function name from the continuation baseline and the completed tree. +Both sets contain 4,038 unique names; the set difference is zero in both +directions. This complements executable contract qualification by proving the +package move did not rename, delete, or replace any existing test entry point. + +### Controlled PVE release qualification + +The final comparison ran on `pulse-dev`, Linux amd64 with Go 1.26.5 and 8 +vCPUs. Exact committed bundles were checked out detached beneath +`/tmp/pulse-api-chart-resource.vxQAJx`; no primary checkout or runner service +was modified. Both measured revisions used warmed race build and test caches, +a fresh isolated data-root value, and the same fixed two-shard release plan: + +```sh +/usr/bin/time -v -o baseline-steady.time \ + ./scripts/run-release-backend-tests.sh \ + --data-root /tmp/pulse-api-chart-resource.vxQAJx/data-baseline-steady \ + --api-shards 2 + +/usr/bin/time -v -o after.time \ + ./scripts/run-release-backend-tests.sh \ + --data-root /tmp/pulse-api-chart-resource.vxQAJx/data-after-fresh \ + --api-shards 2 +``` + +The explicit shard count removes a discovered environmental confound: the +first baseline saw 11,116 MiB available and auto-selected two shards, while an +immediate split attempt saw 9,591 MiB and auto-selected one. That one-shard +attempt was terminated and excluded. A subsequent matched attempt was also +excluded after its reused data-root exposed an asymmetric `resourceapi` test +cache hit. The recorded pair is steady-state warm on both sides, used distinct +fresh data-root values, saw 10,392 and 10,377 MiB available respectively, and +produced the same two shards with one exact-name batch per shard. + +| Warm release gate | Wall | User | System | Average cores | 8-vCPU capacity | Maximum RSS | Result | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | --- | +| Baseline `1327dddad` | 13:46.65 | 1,545.57s | 148.79s | 2.050 | 25.62% | 6,381,264 KiB | pass | +| Chart/resource split `2e4bd36ec` | 11:16.05 | 1,369.46s | 138.27s | 2.230 | 27.88% | 6,526,072 KiB | pass | + +The production split reduced the warm release wall time by 150.60 seconds +(18.22%) and total CPU time by 186.63 seconds (11.01%). Average occupancy rose +from 2.050 to 2.230 cores, or 2.26 percentage points of worker capacity. Peak +RSS increased by 144,808 KiB (2.27%). Both release runs completed with two API +`PASS` results, no race report, and the terminal +`Race-enabled backend release tests passed with 2 API shard(s).` marker. + +The split release log reports native warm package results for both +`internal/api/chartapi` and `internal/api/resourceapi`; the uncached local race +run above remains the execution proof for every moved test. Live checkpoints +also showed the shorter root shard finishing near 5m13s after the split versus +about 7m in the steady baseline. The remaining root shard therefore remains +the release critical path even though the architectural cut materially reduced +it. + +Preserved local evidence files and checksums: + +- `/tmp/pulse-api-next-cut.Cxp7CT/baseline-steady.time`: + `d6673f5a0f06a076e8fc661695b813129a831c5900eaf6d0bd74a122b6363f11` +- `/tmp/pulse-api-next-cut.Cxp7CT/baseline-steady.log`: + `1e83269680567e1d92e361cfde26414e6bc6a815f44710a0353b94dfaeac3784` +- `/tmp/pulse-api-next-cut.Cxp7CT/after.time`: + `c9570f253460e360432bacd9803b23f703a73baa93929d6cfce06c65dfce4f71` +- `/tmp/pulse-api-next-cut.Cxp7CT/after.log`: + `5a8dfa99f184514467669990dd667e22507a04602809d2a1d48895c897964c00` + +After the measurements, the worker had no matching release-backend, API race, +or Docker build process. The temporary remote files and logs were preserved. diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md index a381ac580..6965f8164 100644 --- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md +++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md @@ -3338,10 +3338,11 @@ runtime mock rewiring only through the internal `demo_fixtures` entitlement, and browser-facing lifecycle surfaces must not infer or persist that internal grant from public runtime-capabilities or presentation-policy payloads. Shared workload-chart reads that lifecycle surfaces reuse must stay -presentation-only on that same boundary: `internal/api/router.go` may batch -those reads in parallel, but it must request only the canonical rendered -metric set for workload cards instead of widening the hot path back to -fetch-all metrics on behalf of install or reporting callers. +presentation-only on that same boundary: +`internal/api/chartapi/service.go` may batch those reads in parallel, but it +must request only the canonical rendered metric set for workload cards instead +of widening the hot path back to fetch-all metrics on behalf of install or +reporting callers. The same presentation-only rule applies when shared infrastructure-summary or workloads-summary chart routes serve a short cached response for repeated org/range/scope requests: lifecycle-adjacent surfaces may render those charts @@ -3438,10 +3439,11 @@ platform inventory or recovery context, they must consume `internal/mock/fixture_graph.go` and its graph-owned projections instead of reintroducing snapshot-only or platform-only helper exports. Lifecycle-adjacent summary chart consumers may still depend on shared -`internal/api/router.go` transport, but any synthetic mock series on that path -must resolve through canonical `resourceType` and `resourceID` identities -rather than lifecycle-local seed prefixes, so platform handoff surfaces do not -see a different recent tail than the runtime mock inventory they describe. +`internal/api/chartapi/service.go` transport, but any synthetic mock series on +that path must resolve through canonical `resourceType` and `resourceID` +identities rather than lifecycle-local seed prefixes, so platform handoff +surfaces do not see a different recent tail than the runtime mock inventory +they describe. When those lifecycle-adjacent surfaces call `/api/charts/infrastructure`, the shared `metrics` filter contract must stay authoritative through the backend batch loader as well, so quickstart or install readouts that only render CPU diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.md index 53827172a..9906d7c1e 100644 --- a/docs/release-control/v6/internal/subsystems/api-contracts.md +++ b/docs/release-control/v6/internal/subsystems/api-contracts.md @@ -3143,18 +3143,19 @@ a new API state machine, queue contract, or verification-accounting field. hidden. Upgrade prompts, trial nudges, monitored-system migration guidance, usage counts, billing identity, and plan metadata must therefore not depend on hidden commercial routes surviving the public demo boundary. -37. Keep the storage summary route in `internal/api/router.go` as the +37. Keep the storage summary route in `internal/api/chartapi/service.go` as the canonical storage summary contract across dashboard and storage consumers. - `internal/api/router.go`, - `internal/api/contract_test.go`, and shared frontend consumers must expose - pooled storage history through one response keyed by canonical + `internal/api/chartapi/service.go`, + `internal/api/chartapi/service_test.go`, and shared frontend consumers must + expose pooled storage history through one response keyed by canonical metrics-target IDs, preserve millisecond chart timestamps, and avoid reconstructing storage summary behavior from per-pool `/api/metrics-store/history` fan-out. 38. Keep infrastructure summary metric filtering canonical on that same shared API surface. `frontend-modern/src/api/charts.ts`, - `internal/api/router_routes_monitoring.go`, `internal/api/router.go`, - `internal/api/types.go`, and `internal/api/contract_test.go` must route + `internal/api/router_routes_monitoring.go`, + `internal/api/chartapi/service.go`, `internal/api/chartapi/types.go`, and + `internal/api/chartapi/service_test.go` must route optional infrastructure-summary `metrics` filters through one governed transport contract, so route-owned consumers can request only the series they render without inventing a second summary endpoint or silently @@ -3172,8 +3173,8 @@ a new API state machine, queue contract, or verification-accounting field. target join keys. New summary payloads must be owned by their product route and pinned in the API contract there. 40. Keep mock and demo chart reads on the same canonical unified snapshot as - the rest of the API surface. `internal/api/router.go`, - `internal/api/contract_test.go`, and chart consumers must route + the rest of the API surface. `internal/api/chartapi/service.go`, + `internal/api/chartapi/service_test.go`, and chart consumers must route `/api/charts`, `/api/charts/infrastructure`, and `/api/storage-charts` through `GetUnifiedReadStateOrSnapshot()` whenever mock or demo presentation is active, so VMware, storage, and infrastructure series stay @@ -7552,11 +7553,11 @@ store-backed metric reads across workload types, with no payload shape change. That endpoint now also carries an explicit API p95 budget under the same store-backed mixed-workload fixture used to verify the batched hot path. That same summary-chart contract now also owns synthetic mock fallback -identity. When `internal/api/router.go` needs to synthesize summary history -for workloads, infrastructure, or storage cards, it must key those series by -canonical `resourceType`, `resourceID`, and `metricType` instead of ad hoc -seed-prefix bounds, so all time ranges and runtime mock samples stay on one -governed timeline. +identity. When `internal/api/chartapi/service.go` needs to synthesize summary +history for workloads, infrastructure, or storage cards, it must key those +series by canonical `resourceType`, `resourceID`, and `metricType` instead of +ad hoc seed-prefix bounds, so all time ranges and runtime mock samples stay on +one governed timeline. Frontend AI API clients now also normalize `402 Payment Required` responses for optional paywalled collections into explicit empty states, so Pulse Pro gating does not become a transport error path during page bootstrap. @@ -8249,7 +8250,7 @@ resource ownership for tenant-scoped requests: once a tenant state provider implements `UnifiedResourceSnapshotForTenant`, `/api/resources` may not fall back to raw tenant `StateSnapshot` seeding when that unified seed is empty. That same mock/runtime contract now also governs chart payloads under -`internal/api/router.go`: when demo or mock presentation is enabled, +`internal/api/chartapi/service.go`: when demo or mock presentation is enabled, `/api/charts`, `/api/charts/infrastructure`, and `/api/storage-charts` must read through `GetUnifiedReadStateOrSnapshot()` so chart payloads use the same canonical mock unified-resource snapshot as `/api/resources` and `/api/state` diff --git a/docs/release-control/v6/internal/subsystems/storage-recovery.md b/docs/release-control/v6/internal/subsystems/storage-recovery.md index 68ffcaa56..7d0d02649 100644 --- a/docs/release-control/v6/internal/subsystems/storage-recovery.md +++ b/docs/release-control/v6/internal/subsystems/storage-recovery.md @@ -1581,11 +1581,11 @@ recovery scope, or a storage/recovery-owned secret source. agent-lifecycle owned. 34. Keep workload chart downsampling and short response caching presentation-only on that same adjacent shared API boundary. When `internal/api/chartapi/service.go` caps mixed-cadence workload history into equal-time buckets or serves a cached workload-summary payload for repeated operator-facing workload-card requests, storage and recovery may consume the resulting visual context only; they must not reinterpret those shaped chart samples, cached timestamps, or cache hits as recovery freshness windows, backup cadence, or restore evidence. The same adjacent chart boundary now covers compact storage capacity - transport. `internal/api/router.go` may batch only the canonical `used` - and `avail` storage series for `/api/charts/storage-summary`, but storage - and recovery must not treat the omitted `usage` or `total` series as lost - recovery truth or widen that compact route back into the full storage-page - payload. + transport. `internal/api/chartapi/service.go` may batch only the canonical + `used` and `avail` storage series for `/api/charts/storage-summary`, but + storage and recovery must not treat the omitted `usage` or `total` series + as lost recovery truth or widen that compact route back into the full + storage-page payload. That same adjacent API boundary also owns summary-request minimization: storage/recovery-adjacent consumers may rely on filtered infrastructure or guest summary payloads, but they must not widen a scoped chart request back @@ -3137,11 +3137,11 @@ clusters monitored without PVE storage entries do not lose their only capacity row. Raw pool accounting stays on the Ceph tab's cluster drawer, which remains the canonical home for per-pool stored/available bytes. Shared chart transport that storage and recovery coexist with must also stay -on rendered-metric budgets. When `internal/api/router.go` batches workload -history for adjacent overview or shared summary cards, it may parallelize the -provider reads, but it must not widen the shared hot path to disk read/write -or fetch-all metrics just because storage or recovery also mount nearby chart -shells. +on rendered-metric budgets. When `internal/api/chartapi/service.go` batches +workload history for adjacent overview or shared summary cards, it may +parallelize the provider reads, but it must not widen the shared hot path to +disk read/write or fetch-all metrics just because storage or recovery also +mount nearby chart shells. That adjacent shared chart transport may also expose host-agent or Proxmox node CPU temperature as `metric=temperature` for node drawers. Storage and recovery may consume the surrounding context, but they must not reinterpret that