Bound state refresh and recovery ingest ownership

This commit is contained in:
rcourtman
2026-07-23 22:23:30 +01:00
parent d618fb8b74
commit 87fe506c0a
12 changed files with 676 additions and 163 deletions
@@ -253,7 +253,15 @@ not a parallel API payload contract. `internal/api/agent_handlers_base.go` and
`BroadcastCurrentState` / `BroadcastCurrentStateToTenant`; they must not build
or retain full frontend-state payloads at the handler boundary. The WebSocket
hub owns tenant-aware state resolution after coalescing, through the same state
getter that backs the canonical `/api/state` payload.
getter that backs the canonical `/api/state` payload. The hub must serialize
whole-state resolution and JSON construction across coalesced broadcasts,
initial-client delivery, and explicit client data requests; cancel delayed or
queued work when its client leaves; and join every state producer before
closing client channels. Reconnect, request, or invalidation churn must not
multiply concurrent clones of that canonical payload or race shutdown sends
against channel closure. Ordinary unregister and slow-client eviction must use
the same synchronized send/close ownership; recovering a send-on-closed panic
is not a substitute for a race-free channel lifecycle.
Docker and Podman app-container CPU payloads expose two API facts with
different meanings: canonical resource metrics and `/api/metrics-store/history`
@@ -70,7 +70,10 @@ must be pruned to the retained group set after a completed poll, while
preserving cache metadata only for groups still observed or intentionally reused
after transient datastore failures. Recovery-point ingestion started by backup
polling must be serialized and coalesced so slow store writes cannot retain one
full backup point batch per poll cycle.
full backup point batch per poll cycle. Complete authoritative enumerations
coalesce only within the same provider, ID-prefix, and instance scope; distinct
source scopes and non-reconciling event batches remain FIFO so bounding memory
does not discard independent recovery facts.
Removed host-agent reconnect blocks are identity-scoped: matching may use the
canonical host ID or token-qualified machine/hostname continuity, but must never
block a distinct live host by hostname alone.
@@ -345,7 +348,9 @@ data cannot wrap into a fabricated healthy value.
worker count rather than datastore cardinality. Per-group cache timestamps
must be removed when successful group discovery no longer retains that
group, and recovery-store ingestion must be a bounded latest-batch pipeline
rather than an untracked goroutine per poll.
rather than an untracked goroutine per poll. Latest-batch replacement applies
only to complete enumerations with the same provider, ID-prefix, and instance
scope; distinct scopes and event batches must remain independently queued.
12. Add or change agentless availability monitoring only through the
poll-provider path. `internal/monitoring/availability_poller.go` owns ICMP,
TCP, and HTTP probes, provider health, scheduler task construction, and
@@ -193,7 +193,12 @@ change may globally weaken the Task 03 lifecycle-state idempotency invariant.
paths must enqueue coalesced current-state invalidations instead of building
full frontend-state payloads at every signal. The hub owns delayed
tenant-aware state resolution so superseded signals do not retain large
state snapshots.
state snapshots. Full state resolution and serialization must run through
one cancellable build slot across broadcast, initial-client delivery, and
client-requested refresh delivery: reconnect churn must cancel work for
clients that have already left, and active reconnects or refreshes must not
overlap whole-state clones with each other or with the coalesced broadcast
worker.
The operations-loop status endpoint is performance-adjacent because it
aggregates fleet, action-audit, workflow-starter, AI-usage, and
external-agent activity evidence for every request. Starter and contextual
@@ -3990,7 +3990,9 @@ per cycle and grew `resource_changes` without bound (issue #1496, demo
outage 2026-07-08). Retention pruning must also enforce a hard row cap on
`resource_changes` (`maxResourceChangesRows` in `store.go`) so a
pathological writer cannot grow the table unbounded inside the time-based
retention window.
retention window. Store shutdown must close the retention signal exactly once,
join the retention loop, and only then close the single-connection SQLite
pool; a concurrent pruning pass must never outlive its database owner.
Action plans in `actions.go` still keep stale-plan protection to the canonical
`resourceVersion`, `policyVersion`, and `planHash` fields, so stale execution
checks stay in the shared resource action model rather than provider-local