The 2026-08-25 evening perf pass re-profiled the remaining frontend
costs on the 50-node mock rig after d5440ff43. Idle main-thread burn on
a throttled phone (~10.5s of long tasks per 30s on the worst-case mock)
and residual warm tab-entry cost both decomposed into per-changed-row
work that ignores what actually changed: every metrics tick deep-cloned,
re-canonicalized, and fully re-merged each patched row, and both store
commits then deep-unwrapped the merged rows again through whole-row
keyed reconciles.
The server's resource delta is a JSON merge patch, so the change shape
is already known. applyResourceStateDelta now records the top-level
keys each patch touched (platformData expanded one level), the
connection store publishes them with the resource revision and unions
them across the bounded history and the hidden-tab deferral set, and
mergeCanonicalResourceDeltaSnapshot takes a fast path for changed
non-host rows whose keys stay within the pass-through metric fields,
the proxmox facet mirror, and the four platformData metric mirror
leaves: the previous display row with just those subtrees cloned in
(a manual plain-data clone — structuredClone's per-invocation setup
dominates at this size). Both commit sites write fast rows as per-key
subtree patches (nested reconcile for records, direct sets for
primitives and platformData leaves) instead of whole-row reconciles,
and the connection store commits aligned ticks per index. Any other
change shape — additions, removals, repeated patches in one frame,
structural keys, agent rows (host coalescing) — keeps the full
clone-canonicalize-merge path, and the fast output is pinned
content-equivalent to it in resourceStateAdapters tests; a websocket
store test pins the per-key change shapes, the meta-history union, and
its unknown-shape contamination.
Measured on the pulse-dev rig (VM-to-VM, 50-node/1500-resource mock,
worst-case RandomMetrics, vs two d5440ff43 baseline runs): mobile-4x
idle long tasks 10.8/10.2s -> 3.9s per 30s; warm Alerts entry 5-rep
median long-task total 3.85s -> 1.52s; mobile warm Overview return
3.8/1.7s -> 1.0s; desktop Backups entry settle 1.06/1.20s -> 0.71s;
warm Storage/Overview sub-tab switch medians flat within rep spread;
desktop cold load and /api/state size unchanged.
Browser-verified on the rig build of this tree: live ticks render
through the fast path with per-cell DOM-vs-store consistency at two
samples 30s apart, zero console errors, desktop and 390px/4x mobile.
The 2026-08-25 audit follow-up profiled the remaining frontend costs on
the 50-node mock rig. Entering a tab more than one websocket tick behind
the shared cache fell off the incremental path onto a full-estate
deep-unwrap merge — a 2.1s unwrap dominated warm Alerts entry at 4x
throttle — and the client-facing resource stream re-shipped ~3KB of
mostly-static metadata per resource on every snapshot, REST recovery,
and reconnect (governed gap resource-payload-static-metadata).
The connection store now keeps a bounded per-tick changed-id history and
serves unions from it, so an instance that mounts or resumes a few
revisions behind catches up with a delta merge over the union instead of
remerging the estate. Uncovered gaps and full-snapshot commits still
fall back to keyed full reconciliation.
Broadcast payloads dedupe the estate's distinct capability blobs into a
content-addressed state-level capabilityCatalog referenced per resource
by capabilitiesRef; default-posture resources (internal sensitivity,
cloud-summary routing, no redactions) omit policy and aiSafeSummary; and
canonicalIdentity.aliases no longer duplicates supersededIds. Ingestion
reverses all three: refs expand through the catalog as per-row clones
(reconcile mutates adopted objects in place), missing policies
synthesize the default posture so a policy:null transition patch cannot
strand a stale governed policy, and identity alias resolution consults
supersededIds explicitly. Slimming edits only the per-broadcast copy,
never stored monitor state, and the delta engine's generic top-level
field diff carries catalog changes in the same frame as the first
resource referencing a new entry.
Rig A/B on the pinned 50-node mock (same VM as the morning baseline,
single runs, +-30% variance; RandomMetrics mutates nearly every row per
tick, so this estate is the worst case for the catch-up union): mobile
4x Alerts entry settles 3.9s -> 2.0s with long tasks 5.5s -> 3.3s,
idle-30s long tasks drop 12.6s -> 10.8s mobile and 3.4s -> 2.8s
desktop, remaining steps flat within variance. Sparse-delta estates
gain more, because their catch-up unions stay small.
/api/state on the pinned mock estate drops 4.75MB -> 4.09MB (-13.9%);
the capability catalog is 7 entries totalling 2.5KB where 946 resources
previously inlined the blobs, and 806 default-posture resources shed
policy and aiSafeSummary. Browser-verified against a live build of this
tree: byte-identical UI outcomes vs the unmodified baseline across
overview/storage/docker/alerts probes at both viewports, with docker
container lifecycle actions enabled through catalog-expanded
capabilities. pulse-mobile and pulse-enterprise do not read the slimmed
fields; the AI runtime consumes the internal model, which is unchanged.
Contracts for monitoring, api-contracts, unified-resources,
performance-and-scalability, alerts, agent-lifecycle, and
storage-recovery are amended with the slimmed wire shape, the ingestion
synthesis obligations, the catch-up history, and superseded-id
resolution.
The 2026-08-25 frontend perf audit measured 21-31% idle main-thread
saturation on phones (4x throttle, 50-node mock estate) with the dominant
costs in the websocket tick pipeline rather than rendering.
useUnifiedResources deep-unwrapped the entire resource store on every
websocket tick in every realtime hook instance (1.7s per 20s idle window on
the Alerts page alone). The realtime effect now skips the store read
entirely when the tick's revision is already applied, and the merging
instance unwraps only the subtrees the delta merge will clone: flagged ids,
host-merge members, and ids absent from the shared cache.
mergeCanonicalResourceDeltaSnapshot re-cloned and re-canonicalized every
agent resource on every tick because a host-merge group could change
without its surviving member being flagged. Host-key groups now refresh
only when a member id is in the delta; a flagged id absent from the
incoming snapshot (removal, or a partner a previous coalesce folded away)
conservatively refreshes every group.
useWorkloads rebuilt every row object each refresh, churning row identity
estate-wide per metrics tick. Rows whose serialized form is unchanged now
reuse the previous object via WeakMap-cached signatures, and refreshes
that change nothing return the previous array identity. The joined
signature cache field this replaces is removed.
Remeasured (mobile 4x, dev attribution, 20s windows): Alerts-page tick
unwrap 1713ms -> 200ms, Overview app busy -27%, workload signature cost
-72%. Contracts for unified-resources, performance-and-scalability, and
storage-recovery are amended to state the group-scoped coalesce refresh
and lazy store-read obligations.
allowHostAgentReenrollLocked trusted only the durable continuity store
when one was attached, so a removal block living only in persisted
monitor state (written by releases predating the store) could never be
cleared: every fresh re-enroll read as already-consumed and the host
was rejected with 400 on every report indefinitely, regardless of new
tokens or reinstalls.
Refs #1772
With the adaptive scheduler disabled (the default), every planning pass
on the main poll tick rebuilt each instance task with NextRun=now and
Upsert overwrote the pending slot, so availability targets polled at
the tick cadence regardless of their configured interval. Reproduced
at exactly 10s polling on a 60s ICMP target; c70431caa only covered
the adaptive-scheduler path. Planning now preserves a pending future
slot, tightening it only when a freshly shortened interval justifies
an earlier run.
Refs #1745
Every candidate_lanes record now carries demand_evidence: dated pointers
to the demand signals justifying the proposed lane (issues, discussions,
support threads, telemetry findings, or a demand-ledger entry in
pulse-pro/FEATURE_REQUESTS.md), or an explicit named-bet declaration for
maintainer-originated lanes with no external signal yet. The schema
requires the field, status_audit rejects missing or empty lists and
surfaces the evidence in the candidate_lanes and candidate_lane_queue
pretty blocks, and the canonical development protocol documents the
rule. No data migration needed; candidate_lanes is currently empty.
PR #1715 fixed the discovery symptom of forked host identities
(0e8f63cdd): lookups now resolve across the '<base>-<hex>' fork
spellings. The identity defect underneath remains: a re-enrollment
under a new token forks the host and the only heal path
(hostRenameHealSource) requires the same token plus a changed hostname
plus three missed health windows, so a re-enrollment fork fails two of
the three conditions structurally and is held open forever. Every
identity consumer must resolve equivalent spellings indefinitely or
silently miss records.
Healing at the source is clone-safety territory (#1584), so it is
registered as a triaged coverage gap for an owned agent-lifecycle
slice rather than an opportunistic patch. The evidence record carries
the verified heal-path conditions and the field-observed fork shapes,
including the truncated double fork.
Follow-up to 6686cdce2: the 100-default/500-max list page bounds existed
as separate untyped constants in internal/api and internal/recovery/store,
relying on a contract clause to keep them aligned. Export
DefaultListPageLimit / MaxListPageLimit from internal/recovery/model
(re-exported through internal/recovery) and alias both consumers to them,
so drift is impossible by construction. store_test.go and
recovery_handlers_test.go pin both consumers to the shared constants;
the three pagination-meta contract clauses now name the single source.
Telemetry could see only saved connections, so an install that tried to
reach a node and could not was indistinguishable from one that never
opened the add-node dialog. Both report zero configured connections and
stall at the same activation stage. Fleet data shows that population is
real and concentrated three to one in container deployments, and nothing
recorded whether those installs attempted a connection at all.
Record node connection test attempts and failures in a bounded,
day-bucketed tally in the config directory, pruned to a 31-day retention
window, and report both over the install-ID rotation window as
node_test_attempts_30d and node_test_failures_30d.
Recording starts only once a request carries a target and credentials, so
an incomplete form is never counted as a node that could not be reached.
A host string that turns out to be unusable does count, because the
attempt was made and it failed. Only the add-node dialog endpoint is
instrumented: instrumenting the unused test-config endpoint as well would
double-count a single operator action.
The tally holds counts alone. Hosts, credentials, and error text never
enter it, which is why it is plain JSON rather than encrypted history.
The deployment-installability verification policy routed
frontend-modern/vite.config.ts through the dev-runtime orchestration
proof set, all of which exercise the unbuilt hot-dev runtime. No
accepted proof could observe production build output, which is why
c4af728c0 (preload posture change) needed
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT.
Split vite.config.ts into its own frontend-build-output path policy:
the accepted set keeps every dev-runtime proof, so dev-server-facing
edits are unchanged, and adds
frontend-modern/scripts/check-bundle-size.mjs, which now also asserts
the built index.html posture the contract clause pins: modulepreload
links limited to the entry's static imports (no lazy route chunks) and
import map integrity coverage of every built JS asset. The guard test
pins the new policy's accepted set.
Verified against the built output: flipping preloadDynamicChunks to
true fails the check with 47 lazy-chunk preload violations; the
healthy build passes. Full canonical-governance chain run locally, all
exit 0.
/api/recovery/points and /api/recovery/rollups clamp the requested page
size to [100 default, 500 max] in both the mock paginators and the store
paths, but the meta block was computed from the raw query value. A client
requesting limit=1000 with 1200 rollups was told totalPages=2 while the
server served 3 pages of 500, so iterating totalPages silently dropped
rollups; limit<=0 reported totalPages=1 at an effective limit of 100.
Normalize page and limit once at parse time, compute meta from the
normalized values, and echo the effective limit. Contract clause 34 in
api-contracts.md pins the obligation; storage-recovery and agent-lifecycle
record the boundary alignment and adjacency; recovery_handlers_test.go
pins above-max and non-positive limit meta.
The browser performance audit measured the client-facing resource
stream as a primary scalability cost at estate scale: a 4.73MB
single-frame snapshot at 1,508 resources, ~3KB of mostly-static
metadata per resource (only 10 distinct capabilities blobs across the
whole estate, identity alias history, aiSafeSummary prose, and policy
routing shipped to every browser), and ~1s of blocked main thread per
delta on desktop hardware. Slimming the stream is a wire-format change
with consumers beyond frontend-modern (pulse-mobile OTA compatibility,
Pro surfaces, AI runtime), so it is registered as a triaged coverage
gap for an owned slice instead of an opportunistic patch. Evidence
record carries the measured payload composition.
The SRI plugin's default preloadDynamicChunks injected modulepreload
links for all ~50 lazy chunks into the built index.html — 3.1MB raw
(4.2MB decoded): every Settings panel, Alerts, Chat, AI Intelligence,
all six platform pages, and both locale bundles fetched and compiled up
front, defeating route-level code splitting exactly where it matters
(slow devices, tab-evicted mobile browsers that cold start often).
Disable dynamic-chunk preloading; the entry keeps its three static
vendor preloads and dynamic-import integrity stays enforced through the
generated import map integrity block (112 entries verified in the built
output). Measured on a 4x-CPU-throttled mobile profile against the
50-node mock estate: cold-load long tasks 32s -> 18s, scripts fetched
99 -> 62, decoded JS 4.2MB -> 2.3MB.
Contract-Neutral: build-output preload posture change: deployment-installability's dev-runtime orchestration proof files all run the unbuilt hot-dev runtime and cannot observe built index.html preloads; verified instead by direct built-output inspection (0 route-chunk preloads, 112 importmap integrity entries) and 4x-throttle before/after measurement; substantive contract clause ships in this same commit
The primaryTabs/utilityTabs memos in AppLayout rebuild their arrays from
live store reads; activeAlerts is replaced wholesale whenever a state
frame carries it, so every reference-keyed <For> consumer (desktop tab
strip and the mobile bottom bar) tore down and recreated all nav buttons
even when nothing visible changed. Probed on the 50-node mock estate:
3 of 4 mobile bottom-bar buttons and 4 of 10 desktop tabs were detached
within 40s of websocket frames; taps landing mid-rebuild die silently.
Reuse previous tab object references (and the previous array identity)
when a rebuilt list is structurally unchanged, so downstream <For>s keep
their DOM. With the fix the same 40s probe keeps every nav element
connected on both form factors. MobileNavBar's proof suite now pins the
DOM-identity behavior and App.architecture pins the stabilizer wiring;
contract notes recorded in frontend-primitives, cloud-paid, and
ai-runtime.
Project freshly minted installer command intent onto the stable host before returning remote config, and consume that intent once so later admin policy changes remain authoritative.
rcourtman