Follow-up audit after the vSphere Backup fix, sweeping every platform page
and tab for columns that say the same thing on every row.
Avail (both Proxmox and vSphere, default-visible): the cell renders nothing
at all until an availability check is linked to that workload. Availability
checks are opt-in per resource, so any install without one showed an empty
column under an "Avail" header on every row. Gate it on live data via a new
`hasAvailabilityData` accessor rather than on a stored preference: writing a
hidden preference would overwrite the user's own choice the first time a
probe appeared. The column now returns by itself when data arrives, and the
accessor reads the unfiltered guest set so narrowing the table by search or
status never makes it vanish.
Tags (vSphere, default-visible): every VM rendered six dots carrying the
identical tag set, because `internal/vmware/provider.go` fills `Resource.Tags`
with fixed provenance strings (`vmware`, `vsphere`, `vm`, `source:vcenter`,
`connection:<name>`, `power:<state>`) rather than reading vCenter's tag API.
Five are constant across an estate and the sixth restates the power state the
status filter already owns, so filtering on any of them selects everything.
Default-hide it on the vSphere scope, with the same one-time migration used
for `backup` so existing preferences are retired too. Proxmox keeps the column
since its tags are genuine per-guest labels.
The Tags hide is a stopgap, not the canonical fix. vCenter does expose a real
tag/category system; the adapter simply does not read it. The tags stay in the
payload because search and facet counts consume them, and the hide should be
removed once the adapter ingests real vCenter tags.
Also checked and left alone: AI Context on vSphere reports the backend's own
`unsupported` discovery state as "N/A", which is honest and already
default-hidden. Docker, Kubernetes, TrueNAS, Machines and every Proxmox and
vSphere sub-tab had no single-value columns.
Contract-Neutral: Contract delta staged in performance-and-scalability.md Extension Point 17 (vSphere tags stopgap + the hasAvailabilityData gate). Residual demand is a Workloads hot-path perf proof; inapplicable because this only changes which columns are offered at render time and adds no per-row or per-frame work, so no hot-path proof file legitimately changes.
The workload table's Backup column reads only `resource.proxmox.lastBackup`
(useWorkloads.ts), so on vSphere it renders "None" on every row forever.
d929ac647 added `backup` to VMWARE_WORKLOAD_DEFAULT_HIDDEN_COLUMN_IDS, but a
default only applies to users with no saved preference for that storage scope.
Anyone who had touched the Columns control on the vSphere page before that
commit kept the column and still sees a full column of "None".
useColumnVisibility already carries the mechanism for exactly this case: a
one-time default-hidden migration that hides a newly-defaulted column once for
users with an existing preference, and records a marker so a deliberate
re-show sticks. useWorkloadsControlsState was passing only `aiContext`, so
`backup` never migrated. Add it.
The migration is guarded on the id already being in that scope's
effectiveDefaultHidden set, so Proxmox, the only platform whose adapter
populates lastBackup, keeps the column visible with real backup ages.
Contract-Neutral: Contract delta staged in performance-and-scalability.md Extension Point 17 (defaultHiddenMigrationIds ownership). Residual demand is a Workloads hot-path perf proof; inapplicable because this adds one id to a startup-time default-hidden migration array and no per-row or per-frame work, so no hot-path proof file legitimately changes.
The installer compares the agent binary it downloaded against the server that
served it, stripping a leading "v" so "v6.0.4" and "6.0.4" match. It did not
strip semver build metadata, so a server built from a working tree reporting
"6.2.0-rc.8+git.46.g98a638e00.dirty" never matched the "v6.2.0-rc.8" agent it
had just served, and the mismatch warning fired on every correct development
install.
This is the warning's whole job, so a false positive is expensive. It is the
only client-side signal that a stale agent was downloaded, and because it
always fired it read as background noise. That is exactly how a genuinely
stale v6.0.5 agent was installed on a live host earlier today: the warning
was there, above the install output, and looked like the one that always
appears.
Strip build metadata from both sides before comparing, keeping the prerelease
suffix because 6.2.0-rc.8 and 6.2.0 are genuinely different releases. This is
the same release-identity reduction the server applies when deciding whether a
local agent artifact is fresh enough to serve; the contracts now state that
one definition governs both ends rather than leaving each side to invent its
own.
Guarded by a test that pins both normalisation steps and exercises the
comparison across the cases that matter: the dev-server shape that used to
warn wrongly, the stale-download shape that must still warn, and a prerelease
against its release. Verified to fail when either strip is removed.
/download/pulse-agent served whatever agent artifact sat on disk with no
relation to the running build. Local agent binaries are build outputs that
nothing refreshes on their own, so they go stale silently: a dev backend was
found serving a four-week-old v6.0.5 agent while reporting 6.2.0-rc.8.
Staleness is not cosmetic. The installer renders its service wrapper from the
server's current template, so an agent predating a flag that template now
passes exits immediately with "flag provided but not defined" and crash-loops
under its watchdog. That is how a real host lost its agent: the version
mismatch was reported only as an installer warning, after download, easy to
read as noise.
Validation already scanned the binary for its report-contract endpoints, so
the version check joins that same single pass and rejects a binary that does
not carry this server's agent version. Refusal is loud where the old warning
was quiet: a dev server answers 404 naming the stale path and the build
command, and a published release falls through to the existing release-asset
proxy and fetches the matching version, which makes production self-healing
rather than silently downgraded.
The expected version resolves through updates.GetCurrentVersion rather than
the compiled-in serverVersion. The first cut of this guard used serverVersion
and was inert on exactly the builds that need it: the enterprise binary
compiles in "dev-pro", no version parser accepts it, and the check disabled
itself. It passed its unit tests and still served the stale binary; only
replaying the real v6.0.5 artifact through the running server exposed it.
"dev-pro" is now pinned in the version table with that reasoning attached.
The shared download-test fixture built a binary carrying the report endpoint
but no version string, which a real agent always has, so it now stamps the
expected version. Verified non-vacuous in both directions: those tests fail
with the guard active and the thin fixture, and pass with a faithful one.
Contract-Neutral: storage-recovery is pulled in only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: it constrains which agent binary /download/pulse-agent serves, touching no storage provider, backup target, recovery repository or protected-workload evidence. The agent-lifecycle, deployment-installability and api-contracts deltas staged here cover every boundary the change actually moves.
Completes the wrapper-teardown rule across the remaining branches. The QNAP
install and both uninstall paths still used a bare pkill -f
"start-pulse-agent.sh" and still stopped the agent before its wrapper.
The bare pattern is narrower than it looks and wider than it should be. It
does NOT match a co-installed agent's supervisor, so the sibling case was
already safe; what it does match is anything where the unescaped dot stands in
for another character and the unbounded tail keeps going, including a .bak
copy of the wrapper and an editor session holding it open. Escaping the dot
and bounding the far end removes both without narrowing the intended match.
Ordering is the more consequential half. A wrapper is a watchdog, so stopping
the agent while its wrapper still loops only races the respawn. QNAP and the
uninstall paths now stop the supervisor first, which is what the contracts
already required of every branch that writes and launches a wrapper.
Uninstall keeps a deliberately broader match than install, with no leading
path separator, so it still reaches a wrapper invoked by a relative path or
stranded at a superseded location. Both contracts now carry that distinction
and the teardown ordering rule, which each had stated only for install.
Guarded by two tests that pin every wrapper kill in the file rather than one
branch: one requires the escaped dot and the bounded tail everywhere, the
other walks each stop block and fails if an agent kill precedes its wrapper
kill. Both were confirmed to fail against the pre-fix QNAP block.
The Unraid install path killed the running agent but never the wrapper
supervising it, then appended a second wrapper at the end of the install. The
survivor and the newcomer both loop trying to own the same agent id, and
because the old wrapper is a watchdog it respawns the agent mid-install with
the previous binary and arguments. Observed on a live Unraid host: a
supervisor from a July install was still running beside the one the reinstall
had just started.
Nothing reports this as a failure. It presents later as an agent that
restarts on its own or reverts to superseded arguments.
Stop the wrapper first, then the agent: killing a supervised agent while its
wrapper still loops only races the respawn. The pattern matches the trailing
path segment so a wrapper left at an older storage location is caught too,
with the dot escaped and the far end bounded so a co-installed agent's
supervisor (start-pulse-agent-prod.sh) is not.
The QNAP branch already stopped its wrapper, which is what made the Unraid
omission visible; the contracts now require every wrapper-writing branch to
own the same teardown.
Guarded by two tests: one pins that the Unraid branch stops the wrapper and
does so before the agent, the other pins that the wrapper pattern spares a
sibling supervisor, with a premise check that the loose pattern really does
match so neither assertion can pass vacuously.
pkill -f matches the whole command line and "^" only anchors the start, so
"^/usr/local/bin/pulse-agent" also matches "/usr/local/bin/pulse-agent-prod".
On a host running a second agent whose binary name shares the prefix, every
install, every upgrade, and every restart of the generated Unraid wrapper
silently killed the other agent too. Confirmed on a live dual-agent Unraid
box: the old pattern matched both the dev agent and the production dogfood
agent, the bounded pattern matches only its own.
The wrapper is the worst of the three because restarting through it is the
documented runbook step, so the collateral kill repeats every time an
operator follows it.
Bound the far end of each binary-anchored pattern with ([[:space:]]|$), and
swap the bare pkill -9 -f "pulse-agent" for -x on the exact process name,
which keeps that site's deliberate path-agnostic intent while excluding the
sibling. The pkill -x sites were already safe and are unchanged.
Guarded by two tests: one pins that no binary-anchored pkill in the installer
is left unbounded, the other exercises POSIX ERE semantics against the two
command lines a dual-agent host presents, including a premise check that the
unbounded pattern really does match the sibling so the assertion cannot pass
vacuously.
Mock mode suspends pull-based collection outright, but push-based agent
reports were never given the same treatment, so a real machine still landed
in monitor state while the unified read path substituted the mock snapshot
over the top. The hosts were hidden, everything downstream was not: a real
Unraid box raised a live storage-topology alert next to fixture data, and
its identity persisted through host continuity.
Three vectors, each closed at its source.
Agent ingest now drops real reports while mock mode is on. ApplyHostReport,
ApplyDockerReport and ApplyKubernetesReport acknowledge the report with the
reporting agent's own identity and touch no state, so nothing raises alerts,
persists continuity, records metrics or feeds the online/offline sweep. The
acknowledgement stays a success so a real agent does not read a demo server
as an outage and retry-storm it.
recentStandaloneHostContinuityEntries returns nothing in mock mode. Those
entries are written to disk from real reports and outlive the toggle, and
every consumer injects them after the read path has already substituted the
mock snapshot, so a machine that reported before mock mode was enabled came
back by its real hostname. There is no real-polling exception here: agent
ingest is not gated on PULSE_MOCK_KEEP_REAL_POLLING and the read state is
mock either way.
Active-alert restore is now opt-out, and mock mode opts out. SetMockMode
already clears active alerts when the toggle flips, but a process booting
with mock mode already enabled never ran that path and restored real alerts
from active-alerts.json.
TestHostedTenantAgentInstallTokenCannotReportToOtherTenant used mock mode as
scaffolding. Under the ingest guard both tenants would be empty and its
isolation assertion would pass without exercising the boundary, so it now
runs in real mode. Every new test pairs the mock assertion with a real-mode
one for the same reason.
pct exec runs with PATH=/sbin:/bin:/usr/sbin:/usr/bin, which does not
include /usr/local/bin where the pulse binary is linked. Every first-run
instruction that invoked a bare `pulse` through pct exec therefore failed
with exit 127, including the command the installer itself prints at the
end of an LXC install.
Reproduced on Proxmox VE 9.1.9 and 8.3.0:
pct exec <ctid> -- pulse --version
-> exit 127, Failed to exec "pulse"
pct exec <ctid> -- /usr/local/bin/pulse --version
-> Pulse v6.2.0-rc.8
The installer now prints $BINARY_LINK_PATH and $UPDATE_HELPER_PATH rather
than a bare name, so both stay correct when a custom service name moves
them. The update helper was only affected for custom service names, since
the default /bin/update already sits on the pct exec PATH.
Docs updated in INSTALL.md, FAQ.md and TROUBLESHOOTING.md plus their
frontend-modern/public/docs mirrors. The setup wizard prints the same
broken command and is fixed separately.
Mock mode never initialises real PVE/PBS/PMG clients and does not run the
platform pollers, so every configured real source sat in the connections
ledger at "awaiting first poll" forever. That published real connection
names and addresses through /api/connections while the rest of the payload
was authored fixtures, and surfaced them on monitoring copy: the Proxmox
workloads empty state rendered "Collection pending: minipc" next to three
mock nodes.
/api/config/nodes already substitutes mock entries wholesale in mock mode
and rejects node mutations outright, so the ledger was the one surface that
had not been brought in line. Move the mock-mode input shaping into
applyMockLedgerInputs in platform_mock_connections.go, which already owns
the mock vSphere, TrueNAS, and availability ledger fixtures, and drop the
config and persistence derived sources there. PULSE_MOCK_KEEP_REAL_POLLING
keeps the previous behaviour, since those sources do collect under it.
Proof is at the payload level rather than the aggregator inputs: the new
handler test asserts the served /api/connections body contains no real
source by name or address, and was verified red before the fix.
Contract-Neutral: agent-lifecycle is named only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: agent rows come from the monitor hosts snapshot, which is untouched. The api-contracts and storage-recovery deltas cover the boundaries actually moved. No payload field was added or changed, so the backend-API-payload proof list does not apply; the handler-level payload proof is TestConnectionsHandleListDropsRealSourcesInMockMode in internal/api/connections_handlers_mock_test.go, verified red without the fix.
The agent_runtime verify-only build pointed its registry buildcache at
ghcr.io/<owner>/pulse-agent, a package no workflow publishes. A cache ref
creates the package it points at, so every release recreated it and the
repository's Packages sidebar advertised an empty pulse-agent entry that
reads like a pullable agent image sitting beside the real pulse,
pulse-control-plane and pulse-chart/pulse packages. The only release tag it
ever carried is a stray v6.0.0-rc.3.
Move the cache to ghcr.io/<owner>/pulse:agent-buildcache, alongside the
runtime stage's existing pulse:buildcache tag, so it lives in the package
that actually ships and shares backend-builder layers with the runtime
build.
TestAgentBuildCacheDoesNotResurrectPulseAgentPackage locks this in. It scans
every workflow for registry-qualified refs to the pulse-agent package,
collapsing ${{ }} expressions first so an interpolated owner cannot hide one
behind its spaces, and asserts both the new cache target and the chart's
published agent image default. Verified red on a restored buildcache ref and
red again on a ref hidden behind ${{ github.repository_owner }}.
Two stale claims corrected while here. The chart values comment said the old
agent.image default was never published, when it did carry v6.0.0-rc.3. The
deployment-installability contract called rcourtman/pulse the only image
publish-docker.yml pushes, when the same workflow also pushes the MSP
pulse-control-plane image. The contract now also forbids buildcache
references to the pulse-agent package, which is what let this regress.
Deleting the orphaned package itself remains a manual GHCR step.
The Recovery and Snapshot Age threshold sections rendered two Global
Defaults surfaces at once, the live editor row plus a synthetic
read-only resource row of the same values. Both looked editable but
the synthetic row's pencil editor saved through findThresholdResource,
which never finds the pseudo-resource, so its edits were dropped
silently. Reported in #1680 by FabienD74. The synthetic rows are gone;
the always-live editor row is the single surface in both table and
card layouts. The dead editScope field and its unreachable saveEdit
branches go with them.
The size columns move from Recovery to Snapshot Age where they belong.
BackupAlertConfig has no size fields, so on Recovery they always
rendered an inert Off. SnapshotAlertConfig sizes are live in the alert
engine (backup_snapshot.go) but were previously invisible and, worse,
unpersistable. The records now carry the normalized warningSizeGiB and
criticalSizeGiB metric keys the column editor reads and writes, and
applyAlertsConfigToSnapshot plus buildAlertsConfigurationPayload carry
the size fields through load and save instead of stripping them.
Verified end to end against the mock dev stack, PUT then GET returns
the saved sizes and a reload shows them.
Guests in the VMs & Containers section now sort by display name with
vmid as tiebreaker. The old vmid-first order looked random because the
rows do not render the vmid, which is what the sorting complaint in
#1680 was about.
Dropping the phantom columns and the per-row bulk-edit checkbox also
brings the Recovery table under the desktop width budget, so it
renders as a table at 1280px and up instead of falling back to cards.
Seeded mock history recorded docker hosts with cpu, memory and disk only,
while the synthetic generator used past the seed window emits the full guest
metric set. The result was an inversion across chart ranges: a docker host had
no diskread, diskwrite, netin or netout history at 5m through 24h and full
history at 7d and 30d. Real docker hosts report both through the agent, so the
seed now covers the same series the generator does and every range agrees.
The guardrail test asserts seeded coverage matches the synthetic metric set
rather than a hand-listed set, so a future series added to one path cannot
quietly skip the other.
Mock history is seeded for 48h, so chart windows longer than that fall through
to the synthetic generator in mock_chart_history.go. That generator produced
cpu, memory, disk and the I/O pairs but never memoryused, so a 7d workloads
read returned 64 points for every other series and zero for memoryused. The
memory column in host-capacity mode had no series to draw at all, which reads
as a broken column rather than missing mock data. Real installs are unaffected:
the live PVE tick writes memoryused to the metrics store and the store rollup
groups by metric_type without an allowlist.
The synthetic generator now derives memoryused from the sampled memory
percentage and the fixture memory capacity, the same derivation live mock ticks
and the seeder already use, so the series stays continuous across the seed
boundary. Capacity comes from a new fixture registry beside the existing metric
role registry rather than a per-call fixture graph clone. Docker containers and
pods stay out of it, matching the Proxmox-only memoryused contract.
The workloads and Proxmox nodes trend cells drew every percent series on a
fixed 0-100 window inside a 16px cell, so any series living in the bottom of
that range rendered on top of the axis rule and read as missing data. A guest's
share of host memory is single digits by construction: at 2% the path sat at
y=15.7 against a baseline at y=16, which is what "memory column trends not
working" looks like. Idle guests hit the same wall on CPU.
Percent series now scale to their own peak like the I/O series already do,
zero-floored, with a 5% floor ceiling so idle noise stays flat rather than
amplified, and still capped at 100. Bars mode remains the level view and every
cell keeps its current-value label, so nothing loses the absolute reading.
Advance the governed install, Docker, and Helm metadata together; add the RC9 release packet for the post-RC8 notification, lifecycle, and resource-read fixes; and record the existing mobile-candidate compatibility decision.
ResourceHandlers.getStore opens a SQLite handle per org and caches it for the
process lifetime, and nothing ever closed them. CleanupTenant already released
patrol, AI, RBAC, license and monitor-adapter state for a deleted org but left
the resource store open, so an offboarded tenant kept its file descriptors and
its unified_resources.db-wal/-shm files alive, and its directory could not be
fully removed.
CloseTenantStore releases and evicts one org's store and is now called from
CleanupTenant alongside the other per-tenant teardown. CloseStores releases all
of them, exposed as Router.ShutdownResourceStores next to the existing
Shutdown*/Stop* helpers.
Found while investigating the flaky hosted-tenant test. It is not what made that
test flaky, which was a detached guest-metadata write fixed separately, but the
leak is real on its own: verified by observing that -wal and -shm sidecars, which
exist only while a connection is open, survived the test before this change and
do not after it.
persistGuestIdentity spawned a detached goroutine per changed guest to write
guest_metadata.json, with a comment noting it avoided blocking the monitor.
Nothing tracked those goroutines, so neither Monitor.Stop nor
MultiTenantMonitor.Stop could wait for them and a queued write could land after
shutdown. In hosted mode that means a write into a tenant directory that
offboarding is already removing, and a stray guest_metadata.json.tmp left
behind when the atomic write is interrupted.
The store now owns the goroutine. SetAsync tracks the write on a WaitGroup and
WaitForPendingWrites drains it under a bounded timeout matching
tenantMonitorShutdownTimeout, so a wedged store cannot hold up tenant teardown.
Monitor.Stop drains before closing the metrics store.
This is what made TestHostedTenantAgentInstallTokenCannotReportToOtherTenant
flaky: t.TempDir cleanup raced a queued write into orgs/client-b and failed
with "directory not empty". The test itself is unchanged, because it was never
a test bug. A goroutine dump at cleanup time showed the writers still live,
created by persistGuestIdentity, blocked on the store mutex.
Verified causally rather than by observation alone: the target test fails 0/4
with the drain removed and passes 8/8 with it, against 2/3 failures on the
unmodified baseline. The regression tests fail if SetAsync stops tracking its
goroutine.
Note for a future pass, deliberately not changed here: each changed guest still
triggers a full-file save, so one poll cycle over N changed guests does N
marshals and N atomic writes that serialize on the store mutex anyway. Fixing
that means coalescing at the call site and is a behavioural change beyond this
defect.
Fixes#1681Fixes#1682Fixes#1683
Contract-Neutral: Notification grouping initialization and alert-config propagation do not alter the broadly referenced agent-lifecycle or storage-recovery contracts; primary alerts, notifications, API, and monitoring contracts and regression proofs are updated.
Schema v6 shipped audit_logging_persistent and audit_events_30d as Pro adoption
signals. Neither discriminated. pkg/server installs the SQLite audit logger on
every install for defense in depth and gates only the read/export endpoints, so
the boolean was true on all 8 installs that had taken rc.8 and 0 rows in the
retained table have ever had it false. The event count measured that background
write volume: three of those eight unlicensed community installs were pegged at
the receiver's 100000 clamp ceiling, with the rest between 4863 and 67509.
Schema v7 replaces both with audit_reads_30d, a count of requests that cleared
the license gate on an audit read or export surface. A read requires a human
action, so unlike store presence or write volume it cannot settle into a
constant. The recorder is wrapped INSIDE RequireLicenseFeature so unentitled
requests never count, and the persisted marker carries a timestamp and a coarse
activity class from a fixed allowlist. Query filters, actors, ranges, and every
audit row read stay on the install.
The retired columns are left in the live database. They hold real rc.8 rows and
migrations only add, so dropping them would be a pointless risk; nothing writes
them once the receiver struct loses the fields.
Adds the guard this class needed. LicensedFeatureAdoptionFields registers every
field that exists to measure licensed-feature adoption, and
TestLicensedFeatureAdoptionFieldsDiscriminate builds an unused install through
the real production snapshot paths, installs a real SQLite audit logger exactly
as pkg/server does, records a baseline audit event, and fails if any registered
field is non-zero. Pinning a console logger there would have made the guard pass
while the payload lied, so it deliberately does not. The guard was verified by
reintroducing the v6 sourcing and confirming it fails with the field named.
A companion test pins the three retired fields so they cannot return under
their old names.
This is the third instance of one bug class. v6 removed
pulse_intelligence_patrol_autofixes_30d, hardcoded to zero with no increment
site, and then introduced two fields that were constant in the other direction.
Three occurrences is a guard, not a habit.
Verified end to end on a running unlicensed install: the payload that reported
audit_logging_persistent true under v6 now reports audit_reads_30d 0, and
seeding two in-window reads, one outside the window, and one with an invalid
activity class yields 2.
Six of the eight Pro-exclusive features had no telemetry field at all, so
there was no way to answer whether RBAC, audit logging, scheduled reporting,
agent profiles, alert-triggered AI, or Kubernetes AI were being used by the
installs paying for them. Schema v6 adds nine content-free adoption signals:
alert_ai_enabled AIConfig.IsAlertTriggeredAnalysisEnabled()
rbac_custom_roles non-built-in roles, per org
rbac_user_assignments user-to-role assignments, per org
audit_logging_persistent a persistent audit store is active, not console
audit_events_30d audit events retained inside the window
report_schedules configured scheduled reports
report_schedules_enabled scheduled reports switched on
report_schedules_run_30d schedules whose last run falls inside the window
agent_profiles configured agent profiles
Counts only. Role names, permissions, usernames, schedule names, delivery
recipients, report scope, profile names, and every audit event field stay on
the install. kubernetes_ai needs no field of its own: it is derivable at read
time from alert_ai_enabled combined with the existing kubernetes_clusters
count, and a dedicated field would be redundant.
Config-sourced signals are read through applyLicensedFeatureConfigSnapshot;
RBAC and audit live behind the router and are read through
Router.ApplyLicensedFeatureTelemetrySnapshot. The RBAC read goes through a new
TenantRBACProvider.PeekManager so a background telemetry read can never
provision an RBAC store for an org that has never used RBAC.
Also removes pulse_intelligence_patrol_autofixes_30d and the AutoFixCount
field behind it. patrol_run.go hardcoded AutoFixCount to 0 and no increment
site existed anywhere in the tree, so the counter was zero in all 233,364
retained production pings. That was a wiring bug, not evidence that nobody
uses Patrol fixes; governed fixes are delivered through the approved-action
pipeline, which is already instrumented. The field was plumbed through run
records, history persistence, the Assistant handoff, and telemetry while being
structurally incapable of holding a non-zero value.
Verified end to end against a running install rather than only in unit tests,
which is precisely the check the autofix counter never had: seeding three
report schedules (two enabled, one last run inside the window) and two agent
profiles produced report_schedules 3, report_schedules_enabled 2,
report_schedules_run_30d 1, agent_profiles 2 in the Settings telemetry
preview, and signing in moved audit_events_30d to 1.
The private receiver landed first in pulse-pro 78ff7dd so the new fields are
accepted on arrival.
Three defects on 2026-08-05 shared one root. A settings capability and the
route it describes each answered "is this caller an admin" from their own copy
of the rule, and the copies had drifted, so the frontend gated tabs on
promises the routes did not keep. 28fd2d1c1 had four capabilities reporting
true against a 403. 7d066f1d5 had three guards refusing callers the settings
routes admitted. 563a3aa06 had the platform admin route disagreeing with the
billingAdmin capability it publishes.
Obligation 35 states the rule the code now follows. A session admin decision
is sessionUserCarriesAdminPrivileges reached through ensureAdminSession or the
status snapshot rather than re-derived, an org-scoped caller is never an
instance administrator, and authorizer output alone cannot establish a
capability while DefaultAuthorizer allows everything.
It also records why a source read is not sufficient evidence here. The gate is
frequently a wrapper two calls above the handler, which is how a capability
looks ungated when it is not, and cost a retracted claim before it cost
anything else.
Appended rather than renumbered so existing pins stay valid.
TrueNAS SCALE catalog apps ship one-shot init containers from ixSystems'
own base images (permissions, postgres_upgrade, pgvecto_upgrade). They run
to completion and stay exited for the life of the app, so essentially every
installed app produced a standing CRITICAL that could never clear.
TrueNAS already separates a completed workload from a failed one before
Pulse sees it. Its app state machine reports a container that exits with a
normal exit code as EXITED and one that exits abnormally as CRASHED, then
rolls any CRASHED container up into an app-level CRASHED. Pulse was
treating EXITED as a failure, which inverted the platform's own semantics
and fired precisely on the apps TrueNAS reported RUNNING.
EXITED now raises nothing. CRASHED still raises a per-container incident so
the app-level CRASHED names the failing service, which is the case that
was actually worth alerting on.
The same false assumption sat in the app's rendered container state, which
read app.Containers[0] and so let whichever init container app.query
happened to return first decide how a healthy app displayed. That now
collapses the workloads using the precedence TrueNAS itself uses.
Refs #1677
The mock:on and mock:off scripts used GNU 'sed -i' syntax, which fails on
the default macOS BSD sed. The '||' fallback then appended a fresh
PULSE_MOCK_MODE line on every invocation, so a local .env accumulated three
contradictory entries.
They also targeted the repo-root .env, which hot-dev.sh does not consult
when choosing the data directory. hot-dev.sh reads the canonical flag from
tmp/dev-config/.env, written authoritatively by toggle-mock.sh. The npm
wrappers were therefore a no-op that corrupted .env as a side effect, while
leaving the operator believing mock mode had switched.
Point them at toggle-mock.sh, which handles BSD sed, writes the canonical
file, syncs the runtime env and restarts the managed runtime. Add
mock:status and mock:edit; the hot-dev startup banner already advertised
'npm run mock:edit', which did not exist.
Extend the deployment-installability contract to cover the mock wrappers
alongside the existing repo-root dev entry rules, and add a regression test
asserting the wrappers delegate rather than rewriting the flag inline, and
that every mock command named in the hot-dev banner exists.
rcourtman