The advisory Benchmarks check failed four pull requests on
`NormalizeSegment` alone in the last 48 paired runs: #2217, #2225, #2252
and #2253. None of them touched the normaliser. Nine other runs moved
the same rows down by more than 10%. The values are quantized (EPYC
7763: `short_name` 2.50 or 3.12ns, `uuid` about 29.6 or 40.5ns). Go
aligns these functions to 32 bytes, so a change anywhere earlier in the
binary moves them between the halves of a 64-byte block.
This drops `BenchmarkNormalizeSegment`. Its guarantees move to
deterministic tests:
- Classification was already pinned by `TestNormalizeSegment`,
`TestLooksLikeUUID` and `TestIsNumeric`.
- The new `TestNormalizeSegment_DoesNotAllocate` pins zero allocations
for the five former inputs. It stores each result in a package-level
sink. Without the sink, an inlinable `normalizeSegment` could keep a
short-segment copy on the stack, and the test would pass.
Timing regressions still show in `BenchmarkNormalizeRoute`, because
every path runs its segments through `normalizeSegment`.
### Proof
These are paired runs of the CI runner's recipe on pulse-dev. The
package list is narrowed to `./internal/api/` and the sample count
raised to 30, because the shared host's load spread these rows by up to
25%.
| Run | `check-bench-regression.sh` | Flagged |
|---|---|---|
| `looksLikeUUID` scans twice | exit 1 | only
`NormalizeRoute/with_uuid`, +13.7% (p=0.005) |
| `normalizeSegment` does its work twice | exit 1 | only
`NormalizeRoute/with_uuid`, +21.9% (p=0.030) |
| candidate vs copy of itself | exit 0 | none |
| candidate vs itself + unrelated init-time function (32-byte shift) |
exit 0 | none (`with_uuid` +9.2%, under threshold) |
| old tree vs itself + same function | exit 1 | all five
`NormalizeSegment` rows, +25% to +40% (p=0.000) |
The deterministic test fails with one allocation per call when the
pass-through return uses `strings.Clone` or `string([]byte(seg))`.
The margin for a UUID-scan slowdown is thin. Doubling the scan reads as
roughly +11% to +14% on `with_uuid`. Layout alone has moved that row by
up to 9.2%, both on CI and in the probe above.
The advisory Benchmarks check failed four pull requests on
NormalizeSegment alone in the last 48 paired runs: #2217, #2225, #2252
and #2253. None of them touched the normaliser. On #2252 all five
sub-benchmarks rose 25-39% together. Nine other runs moved the rows
down by more than 10% instead, -20% to -28% on #2228, #2230 and #2242.
Those 13 runs used EPYC 7763, EPYC 9V74 and Intel Xeon runners.
The values are quantized. On EPYC 7763 short_name is 2.50ns or 3.12ns
and uuid is about 29.6ns or 40.5ns, a cycle or two per call, or one
per byte of the UUID scan. Go aligns these functions to 32 bytes, so a
change anywhere earlier in the binary can move them between the halves
of a 64-byte block. A 32-byte shift is enough: on pulse-dev, adding an
unrelated init-time function ahead of the normaliser flipped that
parity and failed the old tree the way #2252 did, all five rows +25% to
+40% at p=0.000 and nothing else flagged. More samples cannot separate
that from a real change, and batching does not help, because every
call repeats the same instructions.
The route-level rows do not behave this way. Over the same 48 runs no
NormalizeRoute row except the root case retired in #2255 moved by more
than 10%. with_uuid came closest at 9.2%.
The guarantees the benchmark carried now live in deterministic tests.
TestNormalizeSegment, TestLooksLikeUUID and TestIsNumeric already pin
classification. TestNormalizeSegment_DoesNotAllocate pins zero
allocations for the five former inputs, and fails with one allocation
per call when the pass-through return copies the segment. It stores
each result in a package-level sink: when normalizeSegment was cut
down to an inlinable copy, a discarded result let the compiler keep
short segments on the stack, and the test missed the allocation.
Timing regressions still show in BenchmarkNormalizeRoute, since every
path runs its segments through normalizeSegment. The proofs below are
paired runs of the CI runner's recipe on pulse-dev, narrowed to
./internal/api/ and raised to 30 samples, because the shared host's
load spread these rows by up to 25%, against about 1% on CI runners.
- looksLikeUUID scanning twice: check-bench-regression.sh exited 1,
flagging only NormalizeRoute/with_uuid, +13.7% (p=0.005).
- normalizeSegment doing its work twice: exited 1, flagging only
NormalizeRoute/with_uuid, +21.9% (p=0.030).
- the candidate against a copy of itself: exited 0.
- the candidate against itself plus the same init-time function:
exited 0. The route rows still moved 4-5%, and with_uuid 9.2%
(p=0.000).
The margin for a UUID-scan slowdown is thin. On EPYC 7763 the scan is
about 30ns of a 280ns with_uuid call, so doubling it reads as roughly
+11% to +14%, while layout alone has moved with_uuid by up to 9.2% both
on CI and in the probe above. Giving UUIDs a larger share of a route
would make that row flip the way this one did; a smaller share would
hide the slowdown.
## What changed
Make provider Plan checkout and billing returns report an active paid
plan only after the running control plane serves the matching refreshed
licence. Handle already-applied, trailing, and cancelled billing returns
without falsely claiming a charge or losing a valid plan confirmation.
The portal tests cover these return states.
## Why
A checkout return alone is not proof that the provider's refreshed
licence is active. The Plan page must tell providers what is actually
running, particularly when a payment callback is delayed, repeated, or
cancelled. This is the reviewed provider Plan return correction and its
follow-up verification.
## Included source
`Pulse` main after `ab0ab381a0f1b4e1ffbcc38537667870dec46960` through
`2dd856e9aa7b155032f102bb96fa59e08f582599`. The reviewed local tip
`cb26fb96ec002a40c52b618a6247fd931ff64b4a` remains an ancestor. Its
provider Plan return changes include `d1ea9ce0`, `8a59a8df`, `ac721a26`,
`a631fc80` and their review/integration merges. Commit `2dd856e9`
additively merges current upstream and regenerates the portal
bundle/manifest from the combined source. No previously reviewed commit
was rewritten.
## Validation and limits
The reviewed Plan candidate passed an offline exact-lock VM check: npm
installation, 25 focused Plan tests, typecheck, all 129 portal frontend
tests, build, and generated-file byte comparison. Its Go portal test
passed in a separate exact-source VM, and earlier offline browser
inspection covered its unchanged Plan paths. For this upstream merge,
the publisher verified a clean worktree, no diff whitespace errors, and
that the committed manifest hash matches every combined portal build
input. The exact post-merge source-proof VM was busy on two attempts; no
post-merge VM pass is claimed. Required hosted checks and exact-head
landing validation must complete before merge. This source proposal is
not live payment acceptance, deployment, or release readiness.
[How Pulse handles
triage](https://github.com/rcourtman/Pulse/blob/main/docs/AI_TRANSPARENCY.md)
Preserve the reviewed provider Plan return history and incorporate current upstream changes. Rebuild the portal bundle and manifest from the combined source.
Change-source: pulse-maintainer
The advisory Benchmarks check keeps failing unrelated pull requests on
`NormalizeRoute/root`: alone on #2228, #2242, #2245 and #2250 (and PR
1973 on 2026-09-08), and beside other results on #2230 and #2244.
**Why it isn't a signal.** The `"/"` path is a length compare and a
constant return. In the last 19 paired PR runs, the 11 on AMD EPYC 7763
measured it at exactly 2.18 ns or 2.50 ns on both sides, each within
0–1%. That is about one CPU cycle. In five of those runs, base and
candidate landed on different values: +14% four times and −13% once
(#2252), though none touched the normaliser. A stable one-cycle shift
passes p<0.05 every time, so more samples can't filter it. A looser
threshold would blunt the gate for every other benchmark.
**Why batching doesn't fix it.** On an amd64 host, a 64-call batch
spread as widely in relative terms as a single call, because every call
repeats the same few instructions.
**Change.**
- Drop the `root` sub-benchmark.
- Add `TestNormalizeRoute_RootFastPathDoesNotAllocate`, which pins the
property that matters: `"/"` and `""` return before splitting. With the
fast path removed, `TestNormalizeRoute` still passes, but the new test
fails with one allocation per call. It also passes under `-race`.
**Real regressions stay detectable.** I ran a paired comparison through
`scripts/run-ci-benchmarks.sh` (10 alternating samples, 100 ms, scoped
to the HTTP metrics benchmarks), with `normalizeRoute` made to do its
work twice:
| Benchmark | Result |
|---|---|
| `NormalizeRoute/*` (6 sub-benchmarks) | +89% to +107%, p=0.000 |
| `NormalizeRoute_Parallel` | +87% |
| `FullMiddlewarePath` | +54% |
| B/op and allocs/op | +100% |
`check-bench-regression.sh` exited 1. The same tree compared with itself
exited 0 with no changes.
**Not in this change.** `NormalizeSegment` shows the same layout flip.
It failed #2252 on its own (+25% to +39%), and in the paired run above
it moved −20% to −38% even though its code was untouched. Its uuid flip
is about 10 ns, so it needs its own fix rather than a small ns floor.
The advisory Benchmarks check kept failing unrelated pull requests on
NormalizeRoute/root alone (#2228, #2242, #2245, #2250, and PR 1973 on
2026-09-08), and flagged it beside other results on #2230 and #2244.
The "/" path is a length compare and a constant return.
In the last 19 paired PR runs, the 11 on AMD EPYC 7763 measured root
at exactly 2.18ns or 2.50ns on both sides, each within 0-1%. The two
values are about one CPU cycle apart. In five of those runs base and
candidate landed on different values, four times up (+14%) and once
down (-13%), though none touched the normaliser. The earlier
investigation found the benchmark loop identical and the normaliser
differing only in relocated addresses. More samples cannot separate a
stable one-cycle shift from a real change, and a looser threshold
would blunt the gate for everything else.
Batching root calls does not escape it. On an amd64 host a 64-call
batch spread as widely in relative terms as a single call, because
every call repeats the same few instructions.
What the path must guarantee is that "/" and "" return before
splitting. TestNormalizeRoute_RootFastPathDoesNotAllocate pins that
deterministically: with the fast path removed, TestNormalizeRoute
still passes, but the new test fails with one allocation per call.
Real regressions stay visible. A paired run using the CI runner's own
alternating recipe (10 samples, 100ms), with normalizeRoute made to do
its work twice, flagged every remaining NormalizeRoute sub-benchmark
at +89% to +107%, FullMiddlewarePath at +54%, and bytes and allocs at
+100%. check-bench-regression.sh exited 1. The same tree compared with
itself exited 0.
A provider-hosted control plane refused to start once its licence was
past
expiry and the 7-day grace, because startup validated it with
ValidateLicense. So at the next restart after a 60-day evaluation ran
out,
or after a paying provider's subscription lapsed, the portal went down,
and
the portal's Plan tab is the only place a provider can buy or renew.
Reproduced on the walkthrough lab.
Startup now accepts an authentic, key-bound licence even when lapsed
(ValidateLicenseAllowingLapse), preferring a current licence and
otherwise
the one that expired later, and logs the lapse. It unlocks nothing:
client
runtimes still verify the provider licence in every lease with
ValidateLicense and drop MSP capabilities, new clients are refused with
provider_msp_license_lapsed, and the refresher still refuses a lapsed
licence. The Plan panel says the evaluation or plan ended and offers the
plans, and provider-msp status reports license_lapsed without failing so
a
lapsed install can still upgrade.
The same run showed the portal printing control plane error codes such
as
provider_msp_license_lapsed instead of their message. The portal now
prefers an error's message over its code, which also fixes
already_subscribed and checkout_unavailable in the Plan tab.
Replaces #2241 and #2250, replayed onto current main as one commit.
Every accepted agent report refreshed the canonical resource store, and
every refresh covers the whole estate, so refresh work grew with agent
count × estate size (#2199). #2235 made each pass linear and cheaper.
This PR bounds how many passes reports can trigger.
**Behaviour.** Accepted host, Docker and Kubernetes reports, including
rejected-report liveness, refresh through
`refreshUnifiedResourceStoreAfterAgentReport`:
- The first report after a quiet window is published before its handler
returns, the same as today.
- Reports within 2 s of the previous refresh fold into one trailing
refresh at the window's end. It reads the latest state and broadcasts it
to websocket clients.
- 2 s matches `readPathRegistryFreshness`, the staleness the read paths
already accept.
- Agent removal and host-agent evaluation still refresh immediately, so
removed inventory retires at once.
- `Monitor.Stop` cancels a pending trailing refresh and waits for a
running one before closing stores.
- Monitors built without `New` keep a zero window and refresh on every
report.
**Cost bound.** At 2,080 resources one refresh costs about 82 ms (after
#2235). 200 agents reporting every 30 s drive 6.7 refreshes/s, roughly
55% of a core; coalesced, reports trigger at most one refresh per 2 s,
about 4%, however many agents report.
**Contract change.** The agent-lifecycle contract promised that an
accepted report is visible when its handler returns. It is now visible
within the 2 s window. The founder approved that trade on 24 September
2026, and the contract records it.
**Tests**
- `TestAgentReportRefreshFoldsReportsWithinWindow`: the leading report
is published synchronously; a burst inside the window is deferred; one
trailing refresh publishes the burst's latest value; after a quiet
window the next report is synchronous again. It fails when the throttle
is removed.
- `TestStopCancelsPendingAgentReportRefresh`: fails if stop does not
cancel the pending refresh.
- The issue-1753 end-to-end test posts two reports back to back, so it
now waits for the published rows instead of reading them at once.
**Verification on pulse-dev**
- `internal/monitoring` runs in 112 s against 107 s on #2235's head. No
test is slower by more than 1.2 s.
- `internal/unifiedresources`, `internal/websocket` and
`internal/api/...` pass. `TestServerInfoEndpointReportsDevelopment`
fails in that source export with and without this change, because the
export has no git checkout.
Refs #2199
Follow-up to #2234. Patrol stopped calling
`ChangeDetector.DetectChanges` when it moved to agentic execution
(1b1c9bb2a3). Recent-change context comes from the unified-resource
timeline, and the detector survives only as a fallback reader of the
legacy `ai_changes.json` that older releases wrote. #2234 removed the
detector's unjoined save. The detection code itself was still there, and
only tests used it.
## Removed
- `DetectChanges` and `detectResourceChanges`, which diffed snapshots.
- `ResourceSnapshot` and its `internal/ai` alias.
- Change-ID generation, and the seven description formatters with their
`formatBytes`/`formatFloat` helpers.
- The detector's lock. The history is loaded once at construction and
never modified, so reads need no lock.
- Tests of the detection logic, its heap stability
(`change_detector_memory_regression_test.go`) and ID uniqueness
(`changes_concurrency_test.go`).
## Unchanged
- `NewChangeDetector`, legacy loading and trimming.
- The read APIs: `GetChangesForResource`, `GetRecentChanges` and
`GetChangesSummary`.
- The `ChangeType` constants, which label the legacy entries.
- `intToString`, which the incident and remediation code share.
## Tests
The `internal/ai` tests that used detection to seed the fallback now
write a legacy `ai_changes.json` and load it. That is the path
production reads.
To check that the new seeding really exercises the load path, I broke
legacy loading through an overlay. 10 of the 12 re-seeded tests then
fail. The other two assert that the fallback is left out of a
type-scoped patrol, or that it loses to the canonical timeline, so empty
history also satisfies them.
That run also exposed three weak assertions, now tightened:
- Two formatter comparisons passed against an empty detector because `""
== ""`. They now reject empty output.
- One summary test used `t.Error` and then indexed `[0]`. It now uses
`t.Fatal`.
The ai-runtime contract now describes the detector as a legacy-history
reader.
A provider-hosted control plane refused to start once its licence was past
expiry and the 7-day grace, because startup validated it with
ValidateLicense. So at the next restart after a 60-day evaluation ran out,
or after a paying provider's subscription lapsed, the portal went down, and
the portal's Plan tab is the only place a provider can buy or renew.
Reproduced on the walkthrough lab.
Startup now accepts an authentic, key-bound licence even when lapsed
(ValidateLicenseAllowingLapse), preferring a current licence and otherwise
the one that expired later, and logs the lapse. It unlocks nothing: client
runtimes still verify the provider licence in every lease with
ValidateLicense and drop MSP capabilities, new clients are refused with
provider_msp_license_lapsed, and the refresher still refuses a lapsed
licence. The Plan panel says the evaluation or plan ended and offers the
plans, and provider-msp status reports license_lapsed without failing so a
lapsed install can still upgrade.
The same run showed the portal printing control plane error codes such as
provider_msp_license_lapsed instead of their message. The portal now
prefers an error's message over its code, which also fixes
already_subscribed and checkout_unavailable in the Plan tab.
setup.sh resolves the image pins in .env to digests once, so upgrade.sh
only ever re-pulled the release a provider first installed. Moving to a
new release meant finding the new digests by hand, so none of a
release's
fixes, and no new feature such as the portal Plan tab, reached an
existing install.
Run from a newly extracted release bundle, upgrade.sh now works on the
existing install: it resolves the bundle's control-plane and runtime
tags
to digests, runs the status gate, preflight and verified backup with the
new release's control plane through a shell override while nothing on
disk changes, then keeps a copy of .env, installs the bundle files and
writes the new pins before starting the new images. --keep-image-pins
keeps hand-set pins.
It also pulls the tenant runtime image before the first status check,
which otherwise failed every upgrade to a new CP_PULSE_IMAGE with "not
present locally", including the documented pin-editing route.
Replaces #2243, which closed when its branch was renamed.
With no custom logo, Settings > General > Application branding previewed
the header with a hand-drawn stand-in: a blue circle holding a "●" text
character, which does not look like the Pulse logo. The header itself
falls back to PulseBrandMark (blue disc, white ring, white centre dot),
so the preview misrepresented what the header shows. Render the same
PulseBrandMark at the header's size; it follows the app theme in light
and dark.
Replaces #2240, which closed when its branch was renamed.
While a standalone agent stays offline across a restart (up to the
72-hour continuity window), every canonical read-state lookup adds it
back from host continuity by building a whole registry clone
(`readStateWithRecords`). Alert evaluation makes one lookup per resource
per poll, so in that state each poll rebuilt the registry once per
resource. Each lookup also called `GetLiveHostsSnapshot` twice, and that
went through `State.GetSnapshot()`, deep-copying every VM and container
just to return the hosts.
**Changes**
- **Overlay cache.** The adapter keeps the last continuity overlay,
keyed by:
- the registry pointer;
- `LastRebuiltAt` (every replace and in-place supplemental ingest
updates it);
- a fingerprint of the requested records. The fingerprint leaves out
`Resource.UpdatedAt`, which `resourceFromHost` stamps with `time.Now()`
at build time.
It reuses the overlay for at most 2 s, because the overlay evaluates
source staleness when it is built. The live adapter is never mutated.
- **Hosts-only copy.** `GetLiveHostsSnapshot` now copies only hosts,
through the existing `State.GetHosts()`. The result is identical,
because `GetSnapshot` sets `Hosts: cloneHosts(s.Hosts)`. This also
removes a full state copy from each agent report (`ApplyHostReport`),
each config fetch and every other caller.
- **Single host copy per lookup.**
`readStateWithStandaloneHostContinuity` takes the host list once instead
of twice.
**One read-state lookup** with one offline standalone agent (synthetic,
8-node clusters, pulse-dev):
| Resources | main | this PR |
|---|---|---|
| 130 | 3.0 ms / 1.9 MB | 13 µs / 10 KB |
| 520 | 15.9 ms / 11.8 MB | 25 µs / 29 KB |
| 1,040 | 41 ms / 28 MB | 39 µs / 54 KB |
| 2,080 | 136 ms / 95 MB | 74 µs / 104 KB |
**Tests.** Each fails when its mechanism is reverted.
- `TestHostContinuityOverlayReusedWithinRegistryGeneration`: reuse
within a generation, including records rebuilt with a fresh `UpdatedAt`;
rebuild after a new generation or different records; the 2 s age bound;
and no mutation of the live adapter.
- `TestStandaloneHostContinuityReadStateReusedAcrossLookups`: end to end
through `GetUnifiedReadStateOrSnapshot` with a real continuity store.
- `TestGetLiveHostsSnapshotCopiesOnlyHosts`: 1 host with 1,000 VMs must
not allocate per guest. The old path made 1,010 allocations.
**Verification on pulse-dev:** `internal/unifiedresources`,
`internal/monitoring`, the `internal/api/...` subpackages and
`cmd/pulse-control-plane` pass.
`TestServerInfoEndpointReportsDevelopment` fails in that source export
with and without this change, because the export has no git checkout.
Refs #2199
Every accepted agent report sends a websocket state invalidation, and
each state broadcast rebuilds and diffs the full frontend state:
| Resources | One broadcast (`BuildFrontendState` + JSON) |
|---|---|
| 130 | 7.5 ms |
| 1,040 | 62 ms |
| 2,080 | 132 ms |
The hub's keyed snapshot adds about 10 ms per 1,000 resources.
The hub coalesced invalidations with a 100 ms debounce that **restarted
on every signal**. That caused two problems:
- Reports spaced more than 100 ms apart (typical for agents on
independent 30 s timers) each produced a full broadcast. With 200 agents
that is roughly 3.4 broadcasts/s, about half a core whenever a browser
is open.
- Signals closer together than the window postponed the broadcast until
they stopped.
**Change: coalescing is now a throttle.**
- The first invalidation after a quiet period still broadcasts after the
100 ms window.
- Later invalidations for the same audience (all clients, or one tenant)
replace the pending request without rescheduling it.
- They broadcast no sooner than `defaultStateBroadcastInterval` (2 s)
after that audience's previous broadcast. That caps broadcast work near
7% of a core at the same scale.
2 s matches the report-driven store refresh interval (#2242), so faster
broadcasts mostly re-sent the same resources. Broadcasts are still
skipped when no client is connected.
**Tests.** Each fails with the old restart-on-signal, unspaced
behaviour.
- `TestStateBroadcastsSpacedByInterval`: the second broadcast waits for
the interval and carries the latest of a burst; the burst produces only
one broadcast.
- `TestTenantStateBroadcastsSpacedByInterval`: the same for tenant
broadcasts.
- `TestStateBroadcastNotStarvedBySteadySignals`: signals every 10 ms
against a 60 ms window still produce broadcasts. The old debounce
produced none in 600 ms.
**Verification on pulse-dev:** `internal/websocket` (existing coalescing
tests unchanged), `internal/monitoring`, `pkg/server` and the
`internal/api/...` subpackages pass.
`TestServerInfoEndpointReportsDevelopment` fails in that source export
with and without this change, because the export has no git checkout.
Refs #2199
Patrol stopped calling DetectChanges when it moved to agentic execution
(1b1c9bb2a3). Recent-change context comes from the unified-resource
timeline, and the detector survives only as a fallback reader of the
legacy ai_changes.json that older releases wrote. The previous change
removed the detector's unjoined save; the detection itself was still
there, kept alive only by tests.
Remove DetectChanges, its snapshot diffing, ResourceSnapshot (and its
internal/ai alias), change-ID generation, the description formatters and
the detector's lock. The history is now loaded once and never modified,
so reads need no lock. The read APIs, trimming and legacy loading are
unchanged.
Tests that used detection to seed the fallback now write a legacy
ai_changes.json and load it, the same path production reads. With
loading broken, 10 of the 12 re-seeded internal/ai tests fail; the other
two assert that the fallback is absent or loses to the canonical
timeline. Two formatter comparisons now reject empty output, which let
them pass against an empty detector. Tests of the detection logic, its
heap stability and ID uniqueness go with it.
Agents poll `/api/agents/agent/{id}/config` every minute, and every
successful poll was written as a signed `agent_config_fetch` audit
event. On a five-agent homelab install that was 483,000 of 483,101 audit
rows (250 MB) over 75 days, burying the 64 logins and 25 CSRF failures
the log exists to show. At the default 90-day retention it comes to
roughly 60 MB per agent, so an install with hundreds of agents would
carry gigabytes of routine polling in its audit log.
**New rule**
- Every **failed** fetch is still audited (missing scope, unknown agent,
signing or serialisation failure).
- A **successful** fetch is audited only when it is new audit
information:
- the agent's first delivery after Pulse starts;
- a different token;
- a different delivered config (`desiredConfig` hash);
- an unchanged delivery last recorded 24 hours earlier.
- Recorded successes now carry `config=<desiredConfig hash>` and
`reason=first_since_start|token_changed|config_changed|daily` in their
details.
That is about 1,440 rows per agent per day down to a handful.
The tracker lives on the handler, is keyed by org and agent, and is
bounded to 4,096 entries. Handlers built without the constructor keep
the old audit-every-success behaviour.
**Existing rows** are left to age out under retention rather than
deleted, because removing entries from a security log would look like
tampering.
**Docs and contracts.** `AUDIT_LOGGING.md` and its shipped mirror
describe the new rule. The api-contracts and agent-lifecycle contracts
record it. storage-recovery gets a boundary note, because it lists
`internal/api/` as an extension point: recovery actions keep their own
action audits, receipts and verification.
**Tests**
- `TestAgentConfigFetchAuditTrackerRecordsOnlyNewDeliveries` walks first
delivery, unchanged poll, config change, token rotation, and the 24-hour
boundary, plus org isolation and the nil tracker.
- `TestAgentConfigFetchAuditsNewDeliveriesAndEveryFailure` makes five
successful fetches around one config change and gets two success events,
and two failed fetches give two failure events. Without the tracker it
records all five successes.
**Verification on pulse-dev:** `internal/api/...` passes.
`TestServerInfoEndpointReportsDevelopment` fails in that source export
with and without this change, because the export has no git checkout.
The agent registration journey posts a second host report about a
second after the first and read /api/state at once. Report-driven store
refreshes now fold into one trailing refresh per 2-second window, so
the update is published within the window rather than before the report
response returns, and the journey read the first report's 42.5% CPU
instead of 75% (Core E2E "Agent registration lifecycle"). Poll
/api/state for up to 10 seconds until the update appears, then check
the remaining fields as before.
Every accepted agent report sends a state invalidation, and each state
broadcast rebuilds and diffs the full frontend state, about 130 ms at
2,080 synthetic resources. The hub coalesced invalidations with a 100 ms
debounce that restarted on every signal, so reports spaced more than
100 ms apart each produced a broadcast, while signals arriving closer
than the window could postpone broadcasts until they stopped (#2199).
With 200 agents reporting every 30 seconds, that was roughly 3.4
broadcasts a second, about half a core whenever a browser was open.
Coalescing is now a throttle. The first invalidation after a quiet
period still broadcasts after the 100 ms window. Later invalidations for
the same audience (all clients, or one tenant) replace the pending
request without rescheduling it, and broadcast no sooner than 2 seconds
after that audience's previous broadcast. That caps broadcast work near
7% of a core at the same scale. Two seconds matches the report-driven
store refresh interval, so faster broadcasts mostly re-sent the same
resources.
Refs #2199
While a standalone agent stayed offline across a restart, every canonical
read-state lookup overlaid it from host continuity by building a whole
registry clone. Alert evaluation makes one lookup per resource per poll, so
for up to 72 hours each poll rebuilt the registry once per resource. Each
lookup also called GetLiveHostsSnapshot twice, and that deep-copied every
guest in the monitor state to return the hosts.
The adapter now keeps the last continuity overlay, keyed by registry
pointer, LastRebuiltAt, and a fingerprint of the requested records that
leaves out the build-time UpdatedAt stamp. It reuses the overlay for at
most two seconds, because the overlay evaluates source staleness when it
is built, and it never mutates the live adapter. GetLiveHostsSnapshot now
copies only hosts through State.GetHosts, which also removes a full state
copy from every agent report and config fetch.
With one offline standalone agent, a read-state lookup at 2,080 synthetic
resources went from 136 ms and 95 MB to 74 us and 104 KB.
Refs #2199
setup.sh resolves the image pins in .env to digests once, so upgrade.sh
only ever re-pulled the release a provider first installed. Moving to a
new release meant finding the new digests by hand, so none of a release's
fixes, and no new feature such as the portal Plan tab, reached an
existing install.
Run from a newly extracted release bundle, upgrade.sh now works on the
existing install: it resolves the bundle's control-plane and runtime tags
to digests, runs the status gate, preflight and verified backup with the
new release's control plane through a shell override while nothing on
disk changes, then keeps a copy of .env, installs the bundle files and
writes the new pins before starting the new images. --keep-image-pins
keeps hand-set pins.
It also pulls the tenant runtime image before the first status check,
which otherwise failed every upgrade to a new CP_PULSE_IMAGE with "not
present locally", including the documented pin-editing route.
Agents poll /api/agents/agent/{id}/config every minute, and every
successful poll was written as a signed agent_config_fetch audit event.
On a five-agent homelab that was 483,000 of 483,101 audit rows (250 MB)
over 75 days, burying the 64 logins and 25 CSRF failures the log exists
to show; at the default 90-day retention it is roughly 60 MB per agent.
Every failed fetch is still audited. A successful fetch is audited when
it is new audit information: the agent's first delivery after Pulse
starts, a different token or desiredConfig hash, or an unchanged
delivery last recorded 24 hours earlier. Recorded successes now carry
the delivered config hash and the reason. The tracker is per handler,
keyed by org and agent, and bounded to 4,096 entries.
Existing rows are left to age out under retention rather than deleted,
since removing entries from a security log would look like tampering.
AUDIT_LOGGING.md and the API and agent-lifecycle contracts describe the
new rule.
Every accepted agent report refreshed the canonical resource store, and
every refresh covers the whole estate, so refresh work grew with agent
count times estate size (#2199). The previous commit made each pass
linear and cheaper; this bounds how many passes reports can trigger.
Accepted host, Docker and Kubernetes reports, including rejected-report
liveness, now refresh through a throttle. The first report after a
quiet window is published before its handler returns. Reports within 2
seconds of the previous refresh fold into one trailing refresh at the
window's end, which reads the latest state and broadcasts it. Two
seconds matches readPathRegistryFreshness, the staleness read paths
already accept. Agent removal and host-agent evaluation still refresh
immediately. Monitor.Stop cancels a pending trailing refresh and waits
for a running one before closing stores. Monitors built without New
keep a zero window and refresh on every report.
At 2,080 resources a refresh costs about 82 ms, so 200 agents reporting
every 30 seconds went from roughly 55% of a core in report-driven
refreshes to at most about 4%, however many agents report.
This relaxes the agent-lifecycle guarantee that a report is visible
when its handler returns: it is now visible within the window. The
founder approved that trade on 24 September 2026 and the contract
records it. The issue-1753 end-to-end test posts two reports back to
back, so it now waits for the published rows instead of reading them
at once.
Refs #2199
With no custom logo, Settings > General > Application branding previewed
the header with a hand-drawn stand-in: a blue circle holding a "●" text
character, which does not look like the Pulse logo. The header itself
falls back to PulseBrandMark (blue disc, white ring, white centre dot),
so the preview misrepresented what the header shows. Render the same
PulseBrandMark at the header's size; it follows the app theme in light
and dark.
Setup's summary tells every new provider to run the install proof before
their first real client. Its temporary workspaces are created on the
provider's own account, so they count against the client limit. On a
two-client evaluation that already had a client, the proof created its
first workspace, was refused the second, and returned no report, so
install-proof's cleanup had nothing to remove and "Provider MSP Proof
01" stayed behind in one of the evaluation's two slots. Found on the
walkthrough lab.
The proof now checks for room before creating anything and says why it
cannot run, and a proof that fails part-way returns the IDs it created
so install-proof removes them. On the lab, with 3 of 3 slots in use,
install-proof now refuses up front and the client count is unchanged.
Every accepted agent report refreshes the canonical resource store, and
two parts of that pass did more work than the estate warranted (#2199).
Guest ingest resolved each guest's parent node through a fallback that
walked every resource, which made each rebuild quadratic. Every
agent-producing source is ingested before the VM and container loops,
so the registry now buckets agents by node name for those loops, as
buildChildCounts already did. It rebuilds the buckets whenever an
ingest yields an agent and skips entries a merge replaced, so parent
resolution matches the full scan exactly. On 8-node clusters one
rebuild went from 74 to 42 ms at 2,080 resources and now grows
linearly.
The refresh pass then cloned the whole registry up to six times, once
in each metric sync and again for the alert sync. A pass now wraps the
store in resourceSnapshotStore, which clones once and forwards
metrics-target resolution, so every consumer reads one generation.
Together, one synthetic agent-report refresh drops from 188 to 82 ms at
2,080 resources and from 90 to 41 ms at 1,040, with allocations per
report falling from 93 MB to 50 MB. Refreshes stay synchronous, as the
agent-lifecycle contract requires; each pass still covers the whole
estate, and coalescing reports would need that contract changed first.
Refs #2199
DetectChanges launched an untracked goroutine to rewrite ai_changes.json.
Tests backed by a temp directory slept 20ms or 100ms hoping it had
finished; when the goroutine started late it wrote into a directory that
cleanup was removing, which is the latent TempDir race already fixed for
IncidentStore and KnowledgeStore.
Nothing in the runtime has called DetectChanges since patrol moved to
agentic execution (1b1c9bb2a3). Recent-change context comes from the
unified-resource timeline, and the detector is only its read fallback, so
the save goroutine was reachable only from tests. Joining it would harden
a dead legacy writer, so remove it along with saveToDisk and the unused
save-scheduling fields. The detector still loads legacy history at
construction; detected changes stay in memory.
Replace the sleeping tests with TestChangeDetector_LegacyHistoryIsReadOnly,
which fails 210/300 runs against the old async writer. With the save
delayed 0-150ms, 349 of 600 old-code saves ran after their test's
directory was removed; none remain. Also drop two sleeps after
RemediationLog.Log, which already persists before returning.
TestProviderMSPLicenseRefreshAppliesShorterPaidPeriod compared the
refresher's UTC expiry ("...Z") with the expected time formatted in the
machine's local zone ("...+01:00"), so it failed on any machine not set
to UTC, including a maintainer Mac on British Summer Time, while CI in
UTC passed. Format the expectation in UTC. It now passes under
Europe/London, America/Los_Angeles and UTC.
Exercise two provider installations with the same tenant ID, unowned-network collision and cleanup isolation. Gate the release-line push before packet freeze and repeat against digest-verified candidate images during container qualification. Update the deployment-installability contract and pin both workflow gates in the release-asset verification suite.
Change-source: pulse-maintainer
Metric identity lived in idx_metrics_lookup, ordered (type, id, metric,
tier, timestamp). Once a series retains history, that order gives every
series its own insertion point, so each poll's commit rewrites one index
leaf page per series through the WAL and again at checkpoint. On a
three-node homelab running v6.4.5-rc.2 that came to 330-607 KB/s of disk
writes, about 10.6 KB per ~60-byte sample and 27-50 GB a day (#1966).
idx_metrics_query_all already holds the same five columns time-major, so
one poll's samples for a resource share pages. It now carries uniqueness
and serves every read, and the metric-major lookup index is dropped. With
an hour of retained history for 480 series, 30 polls write 6,022 WAL
frames instead of 25,310 (4.2x fewer); the empty-table issue-1124 estate
drops from 36,308 frames to 20,223, and its ceiling tightens from 40,000
to 23,000.
The tier-reconciliation overlap probe now names the identity index too;
left to the planner it seeks on (tier, timestamp) alone once the
metric-major tree is gone, which TestRetainedQueryPlansUseIndexes caught.
Metric-filtered reads now scan a resource's other metrics in the window:
a single-metric 500-node query went from 45 to 59 us and the 24-hour
50-disk smart_temp batch from about 0.25 to 0.49 s. All-metric dashboard
reads are unchanged. Continuous write volume is the user-visible harm, so
the contract records that trade explicitly.
The migration reuses the crash-safe swap: databases whose identity is
already enforced by a unique index defer the rebuild to startup
maintenance and swap in one transaction, and a database with no unique
index migrates synchronously. Keeping the name idx_metrics_query_all
means a downgrade rebuilds only the lookup index and leaves no orphan.
Refs #1966
Carry the focused Plan return repair alongside unrelated monitoring integration. Keep paid state contingent on a served matching plan after licence refresh.
Change-source: pulse-maintainer
Every IncidentStore mutation queues a coalesced saveAsync goroutine
that later runs MkdirAll on the data directory and writes its .tmp
file. Nothing tracked those goroutines, so a t.TempDir-backed test
could finish while one was still queued. The save then either wrote
into the directory while TempDir's RemoveAll was deleting it (CI:
"unlinkat ...: directory not empty" in
TestIncidentQueryCanonicalOccurrencesAndNotes/memory, run 35932105100)
or recreated the directory after cleanup and leaked it. The test's
synchronous saveToDisk did not help: it cannot join a save that is
already queued.
Track queued saves with a WaitGroup and add an unexported flush(),
mirroring KnowledgeStore (47cadcc188). The disk-backed incident tests
now join through flush in place of polling, a 20ms sleep or waiting
for a save count. The canonical-occurrences test reloads from
RecordNote's own asynchronous save instead of a manual checkpoint.
TestIncidentStoreFlushJoinsQueuedSave holds the save lock to prove
flush waits for a queued save that has not started.
With a 0-3ms delay injected before each save, the old code leaked a
recreated data directory in 233 of 300 runs; with this change, none.
Checkpoint timing, coalescing and the stored format are unchanged.
Carry the reviewed checkout and billing-return confirmation fix on the assigned source after the Core registry merge. Keep the paid-plan announcement tied to the running control-plane response and retain the embedded portal bundle.
Change-source: pulse-maintainer
Keep canonical change classification while avoiding two full List clones on accepted ingest. Document the registry and monitoring boundaries, add exact verification tests, and retain 1,000-resource benchmarks for #2199.
Change-source: pulse-maintainer
rcourtman