Free self-hosted sessions now see reactive commercial surfaces: paid-feature
nav items stay visible with panel-owned inline gates (Relay precedent extended
to reporting, roles, users, audit log, audit webhooks), gate CTAs render, and
Plans & Billing is discoverable and carries an MSP/provider section with the
free 60-day evaluation. Adds a one-shot business-estate card for authenticated
free sessions whose monitored estate crosses business-scale thresholds
(>=5 PVE nodes, >=10 Docker hosts, or >=3 VMware hosts); the flag rides
sessionCapabilities only, never the pre-auth presentation policy, so estate
size cannot leak to anonymous visitors. Demo mode and white-label runtimes
(including MSP tenant containers) keep full commercial suppression; the
multi_tenant organization nav items deliberately stay hidden.
Decision record supersedes the 2026-04-25 opt-in record:
docs/release-control/v6/internal/records/self-hosted-commercial-surfaces-revision-2026-08-07.md
RA5 summary, contract pins, nav integration pins, and the RA5 browser spec
are updated to the revised invariant in the same commit.
The server now echoes its version on unified-agent report acks, and the
agent nudges its auto-updater the moment an ack carries a newer version.
After a server upgrade, agents converge within one report interval
instead of waiting out the hourly update check, so the "older Pulse
agent" notice self-resolves in seconds once upgraded agents report in.
The hourly loop stays as the retry and backstop path. Nudges dedupe per
server version, refuse downgrades, skip disabled and development-mode
updaters, and never fire from observer destination acks — only the
authoritative server may steer an agent's updater, and a nudged check
re-validates against the server and runs the existing checksum and
self-test pipeline before swapping binaries. Agents deployed before
this change still converge on their old hourly cadence once; every
upgrade after that lands within a report cycle.
Contract deltas recorded in agent-lifecycle and api-contracts, with
boundary notes in security-privacy (no update authority in the echo),
performance-and-scalability (no steady-state work), and
storage-recovery (nothing persisted).
Traefik took the whole operator .env via env_file, so the internet-facing
container's environment carried CP_ADMIN_KEY and the entitlement signing
private key, neither of which it needs. It now receives only ACME/DNS
material: the CF token by explicit passthrough plus an optional
dns-credentials.env that setup.sh creates 0600.
The same wiring removes the silent Cloudflare-only narrowing: the resolver
provider is now ACME_DNS_PROVIDER (default cloudflare) via Traefik's env
override, the CF token is required only for the default provider, and any
other Traefik dnsChallenge provider supplies its credential variables through
dns-credentials.env. Pinned by
TestProviderMSPTraefikEnvIsMinimalAndDNSProviderOverridable; contract delta
in deployment-installability.
restoreProviderMSPArchiveFile copied each tar entry with an unbounded
io.Copy, so the only limit on what a restore wrote to disk was the size
the archive declared for itself. A gzip bomb, a PAX sparse entry claiming
a huge logical size, or a corrupt stream could fill the target volume.
Bound extraction the way readProviderMSPBackupManifestBytes already
bounds the manifest: a per-entry cap and a cumulative cap across the
whole restore, both enforced against the bytes actually copied rather
than the declared header size. That size comes from the archive, so it
is only good for an early reject, never as the bound. An entry that
overruns fails the restore and its partial file is removed, rather than
being silently truncated into a file that looks complete.
A restore that failed partway had already deleted whatever it replaced,
leaving a half-populated control plane that looks bootable and forcing
the retry to use replace. Roll the partial restore back to an empty
target instead, and say so in the error.
Follows the zip-slip fix in the same function (CodeQL alert 314).
The #1601 follow-on: per-container alert overrides were keyed by Docker
container ID (docker:{host}/{containerID}), which changes on every
recreate, so each image update silently re-armed alerts the user had
switched off and left a dead entry behind in alerts.json — the unbounded
growth that pushed the reporter's config past the old 64KB body cap
(raised in 38434a513). The v6 thresholds UI additionally wrote keys from
the unified hash id (docker:{host}/app-container-{16hex}), which the
evaluator never read at all.
Overrides now key on stable identity, docker:{host}/{containerName}:
- The evaluator resolves the name key first and falls back to the legacy
container-ID key so pre-migration entries keep working
(evaluateDockerContainer, checkDockerContainerState, the
container-update resolver, and reevaluateActiveAlertsLocked).
- MigrateDockerContainerOverrideKeys runs in the monitor sync next to
MigrateCanonicalOverrideKeys, driven by the unified resource snapshot:
it re-homes live legacy-ID and unified-hash keys onto the name key and
prunes orphaned ID-shaped entries, ending the per-update orphan
accumulation. Name-keyed entries for absent containers are kept so a
recreate under the same name still honours them.
- The UI candidate chain (single implementation in alertOverridesModel)
now leads with docker:{host}/{name} and trails the container-ID, short
ID, unified-hash and slash-tail forms, so rows bind pre-existing
overrides of every historical shape and the next save re-homes them.
Rows carry overrideStorageId/overrideIdCandidates so toggle,
connectivity, offline-state, edit and remove all write the stable key.
- The ignored-containers card copy now documents the wildcard forms
(runner-*, *-dev, *staging*) shipped in b5fa6a9af, under the title
"Ignored container patterns".
Contract deltas: alerts, frontend-primitives, monitoring, and
unified-resources now pin the name-keyed override identity, the single
frontend candidate-chain owner, the sync-cadence migration, and the
resource-facet-backed table identity respectively.
Verified live against a mock instance: a UI toggle persists
docker:{host}/loki and binds back after reload, and seeded
legacy/hash/orphan keys converge to name keys on disk within two sync
ticks. go test ./internal/alerts/... ./internal/monitoring/... green;
recreate survival pinned in
TestDockerContainerOverrideSurvivesContainerRecreate.
Refs #1601
Fixes#1685.
The "Last refresh" clock in the app shell was built with a hardcoded en-US
locale and hour12 true, so every reader saw a US 12 hour clock regardless of
their system settings. #1279 already fixed this once in App.tsx; the v6
rewrite that moved the logic into useAppRuntimeState.ts reinstated the
hardcoded form, and it has shipped that way since GA. The reporter spotted
the regression and named the original PR.
Swept the rest of the frontend for the same defect. One other site remained,
the alert history day group full-date label, which rendered "Thursday,
August 6, 2026" to a reader whose own rows are ordered day-month. Both now
pass undefined so the runtime resolves the viewer's locale and clock
convention.
Because this is a regression that already survived one fix, add a
canonical-shared/no-hardcoded-format-locale audit rule covering
toLocaleString, toLocaleDateString, toLocaleTimeString and the Intl
constructors, with an allowFiles escape for any call that genuinely needs a
fixed locale. Confirmed the rule fires on the exact reintroduced regression
and passes once reverted.
Verified in an en-GB browser: the footer now reads 20:52:55 rather than
8:52:55 PM, and the history day header title reads "Thursday, 6 August 2026".
Contract-Neutral: Contract deltas staged where the change actually lands: frontend-primitives.md gains the shared date/number formatting-locale rule and names the audit that enforces it, and alerts.md extends the existing timestamp paragraph to the day group label. Residual demands are inapplicable. cloud-paid.md names useAppRuntimeState.ts for hosted org-context bootstrap and licence boundaries and performance-and-scalability.md names it as an app-shell performance boundary; grepped both and neither documents date formatting, and swapping a locale argument moves neither boundary. The alerts frontend surface proof is the same fixed policy list as a0113b964 and does not name alertHistoryModel.branchcov.test.ts, which is the file that actually covers this model; editing an unrelated listed proof would be fabrication.
The vSphere adapter filled `Resource.Tags` with six fixed strings on every
resource — `vmware`, `vsphere`, `<kind>`, `source:vcenter`,
`connection:<name>`, `power:<state>` — and never read vCenter's own tag and
category system. Every VM in an estate returned a byte-identical set, so the
workload Tags column rendered the same dots on every row and filtering on any
of them selected everything. Commit 6b78feba8 default-hid the column and said
in as many words that the hide was a stopgap awaiting this fix.
`internal/vmware/client_tags.go` reads the CIS tagging service. That is a
different endpoint family from the `/api/vcenter/...` inventory reads, but the
same vSphere Automation API, so it reuses the caller's `/api/session` token
rather than opening and managing a second session. Associations come from one
batched `list-attached-tags-on-objects` POST per bounded object batch, never a
per-object request; tag and category names resolve through a client-scoped
catalog with a 10-minute TTL, so a steady-state refresh of a tagged estate
costs only the association reads while a rename still converges without a
restart. A vCenter without the tagging service, or an account without the tag
read privilege, degrades into a `tags` stage enrichment issue and leaves the
inventory untagged; it never fails the refresh.
The provenance strings stay. `Resource.Tags` is the only keyword set
`resourceSearchMatch.ts`, the `?tags=` resources filter, and saved
report-schedule tag filters read — `collectSearchCandidates` gathers no
`technology`, `type`, or `platformScopes` candidate — so dropping "vmware" or
"vsphere" would silently stop matching searches and saved filters that depend
on them. Real vCenter labels are appended to that set, never substituted for
it.
Because that flat set is deliberately mixed, it is the wrong source for a
per-row Tags cell. Real tags therefore also land on a canonical `VMware.Tags`
facet that carries vCenter's category alongside each name, and
`useWorkloads.ts` maps `WorkloadGuest.tags` from that facet for any resource
carrying VMware metadata — including the empty case, so a vSphere VM nobody
tagged renders an empty cell instead of falling back to the provenance dots.
vCenter tag names are unique only inside their category, so the flat label is
`category:name`: two categories may each hold a "Production".
With the column carrying per-row meaning again, `tags` leaves
VMWARE_WORKLOAD_DEFAULT_HIDDEN_COLUMN_IDS and the `defaultHiddenMigrationIds`
retirement list, and the state-model test that pinned the stopgap now pins its
absence. No un-hide migration ships alongside it: 6b78feba8 is on main but no tag
contains it, so the stopgap never shipped and no install carries the
auto-hidden preference. That holds only while the two stay together — the
migration writes the hide into each user's saved preference on first load,
so an rc cut from main carrying the stopgap without this commit would make
an explicit un-hide path necessary.
Mock fixtures carry uneven tag coverage — several categories on some objects,
one on others, none on the rest — because a uniform fixture set would hide
exactly the defect this data exists to catch.
Verified against a mock estate built from this branch: `/api/resources`
returns provenance plus real labels on the flat set and only real labels on
`vmware.tags`; the Tags column renders 2-4 dots per tagged VM and none for
untagged ones; a dot's tooltip reads `Backup:Nightly`, and clicking it
searches `tags:Backup:Nightly` and narrows 18 VMs to the 3 that carry it.
Contract deltas: performance-and-scalability.md Extension Point 17 replaces
the stopgap paragraph with the two-surface tag contract and the bounded
tag-read budget; unified-resources.md states the keyword-union vs facet split
and that a present-but-empty facet means "no operator tags" rather than a
fallback; storage-recovery.md extends its VMware descriptive-only boundary to
`vmware.tags`, because vCenter tag vocabularies read like protection policy
(`Backup:Nightly`) and a label the operator wrote must never satisfy a
coverage or compliance verdict that recovery-owned evidence should decide.
Resolves every open code scanning alert on the repository. Dependabot and
secret scanning were already clear.
SMART temperature truncation (alerts 312, 313). parseRawValue returns a
64-bit raw attribute value, but DiskSMART.Temperature is an int, which is
32 bits wide on the 386 and arm release builds Pulse ships. The range check
ran after the narrowing conversion, so a raw value of 4294967316 truncated
to 20 and was published as a plausible 20 degree reading.
validSMARTTemperature64 now gates the conversion.
Provider MSP restore archive names (alert 314). cleanProviderMSPArchiveName
rejected a leading "../" but not a bare "..", which path.Clean produces from
entries such as ".." and "a/../..". pathIsInside caught the escape
downstream, so this was not exploitable, but the sanitizer now rejects it
outright instead of depending on a second gate.
TrueNAS device paths (alert 315). vdev.Device is supplied by the appliance,
concatenated into a path and published verbatim on ZFSDevice.Path, so values
like "//evil.example.com/share" and "/\evil.example.com" passed straight
through. devicePath now drops traversal segments and backslashes and
collapses a leading double slash. The alert's open-redirect framing does not
apply here, there is no redirect sink on this path, but the value is
untrusted input rendered as a path and is worth normalising.
Patrol readiness cache key (alert 311). The key is persisted to
ai_patrol_model_readiness.json and embedded an unkeyed SHA-256 of the Ollama
Basic Auth username and password. That password is chosen by a human, so
anyone holding the evidence file could recover it offline at two SHA-256
operations per guess. The fingerprint is now HMAC-SHA256 keyed with a
32-byte per-install salt stored beside the evidence at mode 600. Credential
rotation still invalidates the cache and the key still survives a restart.
Each fix carries a regression test confirmed to fail against the previous
implementation.
monitoring.md carries the one warranted contract refinement. It already
required SMART temperature selection to accept only plausible readings, and
that rule now states the width at which plausibility is decided.
Contract-Neutral: CodeQL security fixes with no public-contract delta and no payload change. monitoring.md carries the one warranted refinement (SMART plausibility decided at 64-bit width). Residual demands are inapplicable: ai-runtime readiness prose documents interruption semantics, not cache-key derivation, and the credential-invalidation contract is unchanged; cloud-paid and deployment-installability contracts never name archive-entry sanitisation; agent-lifecycle owns smartctl.go but its SMART temperature prose lives in the staged monitoring.md.
Fixes#1687. Also addresses the locale half of #1685.
The per-row Resource button in Alerts > History rendered its panel as a
page-level sibling in HistoryTab, between the filters card and the table.
Reproduced at 1280px: scrolled 3200px into the history, clicking the button
opened the panel 2595px above the top of the viewport. The only visible
effect was the row list shifting down as the panel was inserted above it, so
the button read as dead and the reporter could not reach the incident detail
or the absolute timestamps it carries.
Render the panel inline under the row instead, matching the neighbouring
Timeline button, in both the desktop table and the phone card list. Because
several alerts can share one resource, the panel state now carries the
originating rowKey and each row renders it only on a match, which also keeps
exactly one panel open at a time. Re-triggering the same row closes it.
The resource resolver moves from a HistoryTab prop onto the history state.
useAlertHistoryState already receives getResource; re-exposing it avoids
threading the lookup through the table section, group row, alert row and
mobile list now that the panel mounts in four places rather than one.
Row timestamps were built with a hardcoded 'en-US' locale while the rest of
this feature already formats through the viewer's locale, so a European
reader saw "05:19 AM" for 05:19. They also showed clock time only, with the
date available just in the day group header, which scrolls out of sight in a
long history. Both surfaces now format through the viewer's locale and carry
the full absolute date and time as a title, with the formatters owned by the
history state so table and mobile cannot drift.
Verified against mock data at 1280px and 390px: the panel opens in view
under the clicked row and card, a non-owning row renders nothing, timestamps
render 23:50 with a "Thursday, 6 August 2026 at 23:50:18" title across all
98 rendered rows, and there is no horizontal overflow at phone width.
Contract-Neutral: Contract deltas staged in alerts.md (inline resource-incident panel placement, rowKey targeting, locale-aware row timestamps) and frontend-primitives.md rule 35 plus its prose (resource resolver moves from the tab prop chain to the history state). Residual demand is an 'alerts frontend surface proof' from a fixed policy list that does not name the three test files this change actually exercises: __tests__/useAlertHistoryState.test.tsx, __tests__/AlertHistoryMobileList.test.tsx and __tests__/HistoryTab.test.tsx, all staged with new assertions covering the panel placement, the toggle, and the timestamp title. Editing an unrelated listed proof file would be fabrication.
Follow-up audit after the vSphere Backup fix, sweeping every platform page
and tab for columns that say the same thing on every row.
Avail (both Proxmox and vSphere, default-visible): the cell renders nothing
at all until an availability check is linked to that workload. Availability
checks are opt-in per resource, so any install without one showed an empty
column under an "Avail" header on every row. Gate it on live data via a new
`hasAvailabilityData` accessor rather than on a stored preference: writing a
hidden preference would overwrite the user's own choice the first time a
probe appeared. The column now returns by itself when data arrives, and the
accessor reads the unfiltered guest set so narrowing the table by search or
status never makes it vanish.
Tags (vSphere, default-visible): every VM rendered six dots carrying the
identical tag set, because `internal/vmware/provider.go` fills `Resource.Tags`
with fixed provenance strings (`vmware`, `vsphere`, `vm`, `source:vcenter`,
`connection:<name>`, `power:<state>`) rather than reading vCenter's tag API.
Five are constant across an estate and the sixth restates the power state the
status filter already owns, so filtering on any of them selects everything.
Default-hide it on the vSphere scope, with the same one-time migration used
for `backup` so existing preferences are retired too. Proxmox keeps the column
since its tags are genuine per-guest labels.
The Tags hide is a stopgap, not the canonical fix. vCenter does expose a real
tag/category system; the adapter simply does not read it. The tags stay in the
payload because search and facet counts consume them, and the hide should be
removed once the adapter ingests real vCenter tags.
Also checked and left alone: AI Context on vSphere reports the backend's own
`unsupported` discovery state as "N/A", which is honest and already
default-hidden. Docker, Kubernetes, TrueNAS, Machines and every Proxmox and
vSphere sub-tab had no single-value columns.
Contract-Neutral: Contract delta staged in performance-and-scalability.md Extension Point 17 (vSphere tags stopgap + the hasAvailabilityData gate). Residual demand is a Workloads hot-path perf proof; inapplicable because this only changes which columns are offered at render time and adds no per-row or per-frame work, so no hot-path proof file legitimately changes.
The workload table's Backup column reads only `resource.proxmox.lastBackup`
(useWorkloads.ts), so on vSphere it renders "None" on every row forever.
d929ac647 added `backup` to VMWARE_WORKLOAD_DEFAULT_HIDDEN_COLUMN_IDS, but a
default only applies to users with no saved preference for that storage scope.
Anyone who had touched the Columns control on the vSphere page before that
commit kept the column and still sees a full column of "None".
useColumnVisibility already carries the mechanism for exactly this case: a
one-time default-hidden migration that hides a newly-defaulted column once for
users with an existing preference, and records a marker so a deliberate
re-show sticks. useWorkloadsControlsState was passing only `aiContext`, so
`backup` never migrated. Add it.
The migration is guarded on the id already being in that scope's
effectiveDefaultHidden set, so Proxmox, the only platform whose adapter
populates lastBackup, keeps the column visible with real backup ages.
Contract-Neutral: Contract delta staged in performance-and-scalability.md Extension Point 17 (defaultHiddenMigrationIds ownership). Residual demand is a Workloads hot-path perf proof; inapplicable because this adds one id to a startup-time default-hidden migration array and no per-row or per-frame work, so no hot-path proof file legitimately changes.
The installer compares the agent binary it downloaded against the server that
served it, stripping a leading "v" so "v6.0.4" and "6.0.4" match. It did not
strip semver build metadata, so a server built from a working tree reporting
"6.2.0-rc.8+git.46.g98a638e00.dirty" never matched the "v6.2.0-rc.8" agent it
had just served, and the mismatch warning fired on every correct development
install.
This is the warning's whole job, so a false positive is expensive. It is the
only client-side signal that a stale agent was downloaded, and because it
always fired it read as background noise. That is exactly how a genuinely
stale v6.0.5 agent was installed on a live host earlier today: the warning
was there, above the install output, and looked like the one that always
appears.
Strip build metadata from both sides before comparing, keeping the prerelease
suffix because 6.2.0-rc.8 and 6.2.0 are genuinely different releases. This is
the same release-identity reduction the server applies when deciding whether a
local agent artifact is fresh enough to serve; the contracts now state that
one definition governs both ends rather than leaving each side to invent its
own.
Guarded by a test that pins both normalisation steps and exercises the
comparison across the cases that matter: the dev-server shape that used to
warn wrongly, the stale-download shape that must still warn, and a prerelease
against its release. Verified to fail when either strip is removed.
/download/pulse-agent served whatever agent artifact sat on disk with no
relation to the running build. Local agent binaries are build outputs that
nothing refreshes on their own, so they go stale silently: a dev backend was
found serving a four-week-old v6.0.5 agent while reporting 6.2.0-rc.8.
Staleness is not cosmetic. The installer renders its service wrapper from the
server's current template, so an agent predating a flag that template now
passes exits immediately with "flag provided but not defined" and crash-loops
under its watchdog. That is how a real host lost its agent: the version
mismatch was reported only as an installer warning, after download, easy to
read as noise.
Validation already scanned the binary for its report-contract endpoints, so
the version check joins that same single pass and rejects a binary that does
not carry this server's agent version. Refusal is loud where the old warning
was quiet: a dev server answers 404 naming the stale path and the build
command, and a published release falls through to the existing release-asset
proxy and fetches the matching version, which makes production self-healing
rather than silently downgraded.
The expected version resolves through updates.GetCurrentVersion rather than
the compiled-in serverVersion. The first cut of this guard used serverVersion
and was inert on exactly the builds that need it: the enterprise binary
compiles in "dev-pro", no version parser accepts it, and the check disabled
itself. It passed its unit tests and still served the stale binary; only
replaying the real v6.0.5 artifact through the running server exposed it.
"dev-pro" is now pinned in the version table with that reasoning attached.
The shared download-test fixture built a binary carrying the report endpoint
but no version string, which a real agent always has, so it now stamps the
expected version. Verified non-vacuous in both directions: those tests fail
with the guard active and the thin fixture, and pass with a faithful one.
Contract-Neutral: storage-recovery is pulled in only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: it constrains which agent binary /download/pulse-agent serves, touching no storage provider, backup target, recovery repository or protected-workload evidence. The agent-lifecycle, deployment-installability and api-contracts deltas staged here cover every boundary the change actually moves.
Completes the wrapper-teardown rule across the remaining branches. The QNAP
install and both uninstall paths still used a bare pkill -f
"start-pulse-agent.sh" and still stopped the agent before its wrapper.
The bare pattern is narrower than it looks and wider than it should be. It
does NOT match a co-installed agent's supervisor, so the sibling case was
already safe; what it does match is anything where the unescaped dot stands in
for another character and the unbounded tail keeps going, including a .bak
copy of the wrapper and an editor session holding it open. Escaping the dot
and bounding the far end removes both without narrowing the intended match.
Ordering is the more consequential half. A wrapper is a watchdog, so stopping
the agent while its wrapper still loops only races the respawn. QNAP and the
uninstall paths now stop the supervisor first, which is what the contracts
already required of every branch that writes and launches a wrapper.
Uninstall keeps a deliberately broader match than install, with no leading
path separator, so it still reaches a wrapper invoked by a relative path or
stranded at a superseded location. Both contracts now carry that distinction
and the teardown ordering rule, which each had stated only for install.
Guarded by two tests that pin every wrapper kill in the file rather than one
branch: one requires the escaped dot and the bounded tail everywhere, the
other walks each stop block and fails if an agent kill precedes its wrapper
kill. Both were confirmed to fail against the pre-fix QNAP block.
The Unraid install path killed the running agent but never the wrapper
supervising it, then appended a second wrapper at the end of the install. The
survivor and the newcomer both loop trying to own the same agent id, and
because the old wrapper is a watchdog it respawns the agent mid-install with
the previous binary and arguments. Observed on a live Unraid host: a
supervisor from a July install was still running beside the one the reinstall
had just started.
Nothing reports this as a failure. It presents later as an agent that
restarts on its own or reverts to superseded arguments.
Stop the wrapper first, then the agent: killing a supervised agent while its
wrapper still loops only races the respawn. The pattern matches the trailing
path segment so a wrapper left at an older storage location is caught too,
with the dot escaped and the far end bounded so a co-installed agent's
supervisor (start-pulse-agent-prod.sh) is not.
The QNAP branch already stopped its wrapper, which is what made the Unraid
omission visible; the contracts now require every wrapper-writing branch to
own the same teardown.
Guarded by two tests: one pins that the Unraid branch stops the wrapper and
does so before the agent, the other pins that the wrapper pattern spares a
sibling supervisor, with a premise check that the loose pattern really does
match so neither assertion can pass vacuously.
pkill -f matches the whole command line and "^" only anchors the start, so
"^/usr/local/bin/pulse-agent" also matches "/usr/local/bin/pulse-agent-prod".
On a host running a second agent whose binary name shares the prefix, every
install, every upgrade, and every restart of the generated Unraid wrapper
silently killed the other agent too. Confirmed on a live dual-agent Unraid
box: the old pattern matched both the dev agent and the production dogfood
agent, the bounded pattern matches only its own.
The wrapper is the worst of the three because restarting through it is the
documented runbook step, so the collateral kill repeats every time an
operator follows it.
Bound the far end of each binary-anchored pattern with ([[:space:]]|$), and
swap the bare pkill -9 -f "pulse-agent" for -x on the exact process name,
which keeps that site's deliberate path-agnostic intent while excluding the
sibling. The pkill -x sites were already safe and are unchanged.
Guarded by two tests: one pins that no binary-anchored pkill in the installer
is left unbounded, the other exercises POSIX ERE semantics against the two
command lines a dual-agent host presents, including a premise check that the
unbounded pattern really does match the sibling so the assertion cannot pass
vacuously.
Mock mode suspends pull-based collection outright, but push-based agent
reports were never given the same treatment, so a real machine still landed
in monitor state while the unified read path substituted the mock snapshot
over the top. The hosts were hidden, everything downstream was not: a real
Unraid box raised a live storage-topology alert next to fixture data, and
its identity persisted through host continuity.
Three vectors, each closed at its source.
Agent ingest now drops real reports while mock mode is on. ApplyHostReport,
ApplyDockerReport and ApplyKubernetesReport acknowledge the report with the
reporting agent's own identity and touch no state, so nothing raises alerts,
persists continuity, records metrics or feeds the online/offline sweep. The
acknowledgement stays a success so a real agent does not read a demo server
as an outage and retry-storm it.
recentStandaloneHostContinuityEntries returns nothing in mock mode. Those
entries are written to disk from real reports and outlive the toggle, and
every consumer injects them after the read path has already substituted the
mock snapshot, so a machine that reported before mock mode was enabled came
back by its real hostname. There is no real-polling exception here: agent
ingest is not gated on PULSE_MOCK_KEEP_REAL_POLLING and the read state is
mock either way.
Active-alert restore is now opt-out, and mock mode opts out. SetMockMode
already clears active alerts when the toggle flips, but a process booting
with mock mode already enabled never ran that path and restored real alerts
from active-alerts.json.
TestHostedTenantAgentInstallTokenCannotReportToOtherTenant used mock mode as
scaffolding. Under the ingest guard both tenants would be empty and its
isolation assertion would pass without exercising the boundary, so it now
runs in real mode. Every new test pairs the mock assertion with a real-mode
one for the same reason.
pct exec runs with PATH=/sbin:/bin:/usr/sbin:/usr/bin, which does not
include /usr/local/bin where the pulse binary is linked. Every first-run
instruction that invoked a bare `pulse` through pct exec therefore failed
with exit 127, including the command the installer itself prints at the
end of an LXC install.
Reproduced on Proxmox VE 9.1.9 and 8.3.0:
pct exec <ctid> -- pulse --version
-> exit 127, Failed to exec "pulse"
pct exec <ctid> -- /usr/local/bin/pulse --version
-> Pulse v6.2.0-rc.8
The installer now prints $BINARY_LINK_PATH and $UPDATE_HELPER_PATH rather
than a bare name, so both stay correct when a custom service name moves
them. The update helper was only affected for custom service names, since
the default /bin/update already sits on the pct exec PATH.
Docs updated in INSTALL.md, FAQ.md and TROUBLESHOOTING.md plus their
frontend-modern/public/docs mirrors. The setup wizard prints the same
broken command and is fixed separately.
Mock mode never initialises real PVE/PBS/PMG clients and does not run the
platform pollers, so every configured real source sat in the connections
ledger at "awaiting first poll" forever. That published real connection
names and addresses through /api/connections while the rest of the payload
was authored fixtures, and surfaced them on monitoring copy: the Proxmox
workloads empty state rendered "Collection pending: minipc" next to three
mock nodes.
/api/config/nodes already substitutes mock entries wholesale in mock mode
and rejects node mutations outright, so the ledger was the one surface that
had not been brought in line. Move the mock-mode input shaping into
applyMockLedgerInputs in platform_mock_connections.go, which already owns
the mock vSphere, TrueNAS, and availability ledger fixtures, and drop the
config and persistence derived sources there. PULSE_MOCK_KEEP_REAL_POLLING
keeps the previous behaviour, since those sources do collect under it.
Proof is at the payload level rather than the aggregator inputs: the new
handler test asserts the served /api/connections body contains no real
source by name or address, and was verified red before the fix.
Contract-Neutral: agent-lifecycle is named only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: agent rows come from the monitor hosts snapshot, which is untouched. The api-contracts and storage-recovery deltas cover the boundaries actually moved. No payload field was added or changed, so the backend-API-payload proof list does not apply; the handler-level payload proof is TestConnectionsHandleListDropsRealSourcesInMockMode in internal/api/connections_handlers_mock_test.go, verified red without the fix.
The agent_runtime verify-only build pointed its registry buildcache at
ghcr.io/<owner>/pulse-agent, a package no workflow publishes. A cache ref
creates the package it points at, so every release recreated it and the
repository's Packages sidebar advertised an empty pulse-agent entry that
reads like a pullable agent image sitting beside the real pulse,
pulse-control-plane and pulse-chart/pulse packages. The only release tag it
ever carried is a stray v6.0.0-rc.3.
Move the cache to ghcr.io/<owner>/pulse:agent-buildcache, alongside the
runtime stage's existing pulse:buildcache tag, so it lives in the package
that actually ships and shares backend-builder layers with the runtime
build.
TestAgentBuildCacheDoesNotResurrectPulseAgentPackage locks this in. It scans
every workflow for registry-qualified refs to the pulse-agent package,
collapsing ${{ }} expressions first so an interpolated owner cannot hide one
behind its spaces, and asserts both the new cache target and the chart's
published agent image default. Verified red on a restored buildcache ref and
red again on a ref hidden behind ${{ github.repository_owner }}.
Two stale claims corrected while here. The chart values comment said the old
agent.image default was never published, when it did carry v6.0.0-rc.3. The
deployment-installability contract called rcourtman/pulse the only image
publish-docker.yml pushes, when the same workflow also pushes the MSP
pulse-control-plane image. The contract now also forbids buildcache
references to the pulse-agent package, which is what let this regress.
Deleting the orphaned package itself remains a manual GHCR step.
The Recovery and Snapshot Age threshold sections rendered two Global
Defaults surfaces at once, the live editor row plus a synthetic
read-only resource row of the same values. Both looked editable but
the synthetic row's pencil editor saved through findThresholdResource,
which never finds the pseudo-resource, so its edits were dropped
silently. Reported in #1680 by FabienD74. The synthetic rows are gone;
the always-live editor row is the single surface in both table and
card layouts. The dead editScope field and its unreachable saveEdit
branches go with them.
The size columns move from Recovery to Snapshot Age where they belong.
BackupAlertConfig has no size fields, so on Recovery they always
rendered an inert Off. SnapshotAlertConfig sizes are live in the alert
engine (backup_snapshot.go) but were previously invisible and, worse,
unpersistable. The records now carry the normalized warningSizeGiB and
criticalSizeGiB metric keys the column editor reads and writes, and
applyAlertsConfigToSnapshot plus buildAlertsConfigurationPayload carry
the size fields through load and save instead of stripping them.
Verified end to end against the mock dev stack, PUT then GET returns
the saved sizes and a reload shows them.
Guests in the VMs & Containers section now sort by display name with
vmid as tiebreaker. The old vmid-first order looked random because the
rows do not render the vmid, which is what the sorting complaint in
#1680 was about.
Dropping the phantom columns and the per-row bulk-edit checkbox also
brings the Recovery table under the desktop width budget, so it
renders as a table at 1280px and up instead of falling back to cards.
Seeded mock history recorded docker hosts with cpu, memory and disk only,
while the synthetic generator used past the seed window emits the full guest
metric set. The result was an inversion across chart ranges: a docker host had
no diskread, diskwrite, netin or netout history at 5m through 24h and full
history at 7d and 30d. Real docker hosts report both through the agent, so the
seed now covers the same series the generator does and every range agrees.
The guardrail test asserts seeded coverage matches the synthetic metric set
rather than a hand-listed set, so a future series added to one path cannot
quietly skip the other.
Mock history is seeded for 48h, so chart windows longer than that fall through
to the synthetic generator in mock_chart_history.go. That generator produced
cpu, memory, disk and the I/O pairs but never memoryused, so a 7d workloads
read returned 64 points for every other series and zero for memoryused. The
memory column in host-capacity mode had no series to draw at all, which reads
as a broken column rather than missing mock data. Real installs are unaffected:
the live PVE tick writes memoryused to the metrics store and the store rollup
groups by metric_type without an allowlist.
The synthetic generator now derives memoryused from the sampled memory
percentage and the fixture memory capacity, the same derivation live mock ticks
and the seeder already use, so the series stays continuous across the seed
boundary. Capacity comes from a new fixture registry beside the existing metric
role registry rather than a per-call fixture graph clone. Docker containers and
pods stay out of it, matching the Proxmox-only memoryused contract.
The workloads and Proxmox nodes trend cells drew every percent series on a
fixed 0-100 window inside a 16px cell, so any series living in the bottom of
that range rendered on top of the axis rule and read as missing data. A guest's
share of host memory is single digits by construction: at 2% the path sat at
y=15.7 against a baseline at y=16, which is what "memory column trends not
working" looks like. Idle guests hit the same wall on CPU.
Percent series now scale to their own peak like the I/O series already do,
zero-floored, with a 5% floor ceiling so idle noise stays flat rather than
amplified, and still capped at 100. Bars mode remains the level view and every
cell keeps its current-value label, so nothing loses the absolute reading.
Advance the governed install, Docker, and Helm metadata together; add the RC9 release packet for the post-RC8 notification, lifecycle, and resource-read fixes; and record the existing mobile-candidate compatibility decision.
ResourceHandlers.getStore opens a SQLite handle per org and caches it for the
process lifetime, and nothing ever closed them. CleanupTenant already released
patrol, AI, RBAC, license and monitor-adapter state for a deleted org but left
the resource store open, so an offboarded tenant kept its file descriptors and
its unified_resources.db-wal/-shm files alive, and its directory could not be
fully removed.
CloseTenantStore releases and evicts one org's store and is now called from
CleanupTenant alongside the other per-tenant teardown. CloseStores releases all
of them, exposed as Router.ShutdownResourceStores next to the existing
Shutdown*/Stop* helpers.
Found while investigating the flaky hosted-tenant test. It is not what made that
test flaky, which was a detached guest-metadata write fixed separately, but the
leak is real on its own: verified by observing that -wal and -shm sidecars, which
exist only while a connection is open, survived the test before this change and
do not after it.
persistGuestIdentity spawned a detached goroutine per changed guest to write
guest_metadata.json, with a comment noting it avoided blocking the monitor.
Nothing tracked those goroutines, so neither Monitor.Stop nor
MultiTenantMonitor.Stop could wait for them and a queued write could land after
shutdown. In hosted mode that means a write into a tenant directory that
offboarding is already removing, and a stray guest_metadata.json.tmp left
behind when the atomic write is interrupted.
The store now owns the goroutine. SetAsync tracks the write on a WaitGroup and
WaitForPendingWrites drains it under a bounded timeout matching
tenantMonitorShutdownTimeout, so a wedged store cannot hold up tenant teardown.
Monitor.Stop drains before closing the metrics store.
This is what made TestHostedTenantAgentInstallTokenCannotReportToOtherTenant
flaky: t.TempDir cleanup raced a queued write into orgs/client-b and failed
with "directory not empty". The test itself is unchanged, because it was never
a test bug. A goroutine dump at cleanup time showed the writers still live,
created by persistGuestIdentity, blocked on the store mutex.
Verified causally rather than by observation alone: the target test fails 0/4
with the drain removed and passes 8/8 with it, against 2/3 failures on the
unmodified baseline. The regression tests fail if SetAsync stops tracking its
goroutine.
Note for a future pass, deliberately not changed here: each changed guest still
triggers a full-file save, so one poll cycle over N changed guests does N
marshals and N atomic writes that serialize on the store mutex anyway. Fixing
that means coalescing at the call site and is a behavioural change beyond this
defect.
Fixes#1681Fixes#1682Fixes#1683
Contract-Neutral: Notification grouping initialization and alert-config propagation do not alter the broadly referenced agent-lifecycle or storage-recovery contracts; primary alerts, notifications, API, and monitoring contracts and regression proofs are updated.
Schema v6 shipped audit_logging_persistent and audit_events_30d as Pro adoption
signals. Neither discriminated. pkg/server installs the SQLite audit logger on
every install for defense in depth and gates only the read/export endpoints, so
the boolean was true on all 8 installs that had taken rc.8 and 0 rows in the
retained table have ever had it false. The event count measured that background
write volume: three of those eight unlicensed community installs were pegged at
the receiver's 100000 clamp ceiling, with the rest between 4863 and 67509.
Schema v7 replaces both with audit_reads_30d, a count of requests that cleared
the license gate on an audit read or export surface. A read requires a human
action, so unlike store presence or write volume it cannot settle into a
constant. The recorder is wrapped INSIDE RequireLicenseFeature so unentitled
requests never count, and the persisted marker carries a timestamp and a coarse
activity class from a fixed allowlist. Query filters, actors, ranges, and every
audit row read stay on the install.
The retired columns are left in the live database. They hold real rc.8 rows and
migrations only add, so dropping them would be a pointless risk; nothing writes
them once the receiver struct loses the fields.
Adds the guard this class needed. LicensedFeatureAdoptionFields registers every
field that exists to measure licensed-feature adoption, and
TestLicensedFeatureAdoptionFieldsDiscriminate builds an unused install through
the real production snapshot paths, installs a real SQLite audit logger exactly
as pkg/server does, records a baseline audit event, and fails if any registered
field is non-zero. Pinning a console logger there would have made the guard pass
while the payload lied, so it deliberately does not. The guard was verified by
reintroducing the v6 sourcing and confirming it fails with the field named.
A companion test pins the three retired fields so they cannot return under
their old names.
This is the third instance of one bug class. v6 removed
pulse_intelligence_patrol_autofixes_30d, hardcoded to zero with no increment
site, and then introduced two fields that were constant in the other direction.
Three occurrences is a guard, not a habit.
Verified end to end on a running unlicensed install: the payload that reported
audit_logging_persistent true under v6 now reports audit_reads_30d 0, and
seeding two in-window reads, one outside the window, and one with an invalid
activity class yields 2.
Six of the eight Pro-exclusive features had no telemetry field at all, so
there was no way to answer whether RBAC, audit logging, scheduled reporting,
agent profiles, alert-triggered AI, or Kubernetes AI were being used by the
installs paying for them. Schema v6 adds nine content-free adoption signals:
alert_ai_enabled AIConfig.IsAlertTriggeredAnalysisEnabled()
rbac_custom_roles non-built-in roles, per org
rbac_user_assignments user-to-role assignments, per org
audit_logging_persistent a persistent audit store is active, not console
audit_events_30d audit events retained inside the window
report_schedules configured scheduled reports
report_schedules_enabled scheduled reports switched on
report_schedules_run_30d schedules whose last run falls inside the window
agent_profiles configured agent profiles
Counts only. Role names, permissions, usernames, schedule names, delivery
recipients, report scope, profile names, and every audit event field stay on
the install. kubernetes_ai needs no field of its own: it is derivable at read
time from alert_ai_enabled combined with the existing kubernetes_clusters
count, and a dedicated field would be redundant.
Config-sourced signals are read through applyLicensedFeatureConfigSnapshot;
RBAC and audit live behind the router and are read through
Router.ApplyLicensedFeatureTelemetrySnapshot. The RBAC read goes through a new
TenantRBACProvider.PeekManager so a background telemetry read can never
provision an RBAC store for an org that has never used RBAC.
Also removes pulse_intelligence_patrol_autofixes_30d and the AutoFixCount
field behind it. patrol_run.go hardcoded AutoFixCount to 0 and no increment
site existed anywhere in the tree, so the counter was zero in all 233,364
retained production pings. That was a wiring bug, not evidence that nobody
uses Patrol fixes; governed fixes are delivered through the approved-action
pipeline, which is already instrumented. The field was plumbed through run
records, history persistence, the Assistant handoff, and telemetry while being
structurally incapable of holding a non-zero value.
Verified end to end against a running install rather than only in unit tests,
which is precisely the check the autofix counter never had: seeding three
report schedules (two enabled, one last run inside the window) and two agent
profiles produced report_schedules 3, report_schedules_enabled 2,
report_schedules_run_30d 1, agent_profiles 2 in the Settings telemetry
preview, and signing in moved audit_events_30d to 1.
The private receiver landed first in pulse-pro 78ff7dd so the new fields are
accepted on arrival.
Three defects on 2026-08-05 shared one root. A settings capability and the
route it describes each answered "is this caller an admin" from their own copy
of the rule, and the copies had drifted, so the frontend gated tabs on
promises the routes did not keep. 28fd2d1c1 had four capabilities reporting
true against a 403. 7d066f1d5 had three guards refusing callers the settings
routes admitted. 563a3aa06 had the platform admin route disagreeing with the
billingAdmin capability it publishes.
Obligation 35 states the rule the code now follows. A session admin decision
is sessionUserCarriesAdminPrivileges reached through ensureAdminSession or the
status snapshot rather than re-derived, an org-scoped caller is never an
instance administrator, and authorizer output alone cannot establish a
capability while DefaultAuthorizer allows everything.
It also records why a source read is not sufficient evidence here. The gate is
frequently a wrapper two calls above the handler, which is how a capability
looks ungated when it is not, and cost a retracted claim before it cost
anything else.
Appended rather than renumbered so existing pins stay valid.
rcourtman