Free self-hosted sessions now see reactive commercial surfaces: paid-feature
nav items stay visible with panel-owned inline gates (Relay precedent extended
to reporting, roles, users, audit log, audit webhooks), gate CTAs render, and
Plans & Billing is discoverable and carries an MSP/provider section with the
free 60-day evaluation. Adds a one-shot business-estate card for authenticated
free sessions whose monitored estate crosses business-scale thresholds
(>=5 PVE nodes, >=10 Docker hosts, or >=3 VMware hosts); the flag rides
sessionCapabilities only, never the pre-auth presentation policy, so estate
size cannot leak to anonymous visitors. Demo mode and white-label runtimes
(including MSP tenant containers) keep full commercial suppression; the
multi_tenant organization nav items deliberately stay hidden.
Decision record supersedes the 2026-04-25 opt-in record:
docs/release-control/v6/internal/records/self-hosted-commercial-surfaces-revision-2026-08-07.md
RA5 summary, contract pins, nav integration pins, and the RA5 browser spec
are updated to the revised invariant in the same commit.
The server now echoes its version on unified-agent report acks, and the
agent nudges its auto-updater the moment an ack carries a newer version.
After a server upgrade, agents converge within one report interval
instead of waiting out the hourly update check, so the "older Pulse
agent" notice self-resolves in seconds once upgraded agents report in.
The hourly loop stays as the retry and backstop path. Nudges dedupe per
server version, refuse downgrades, skip disabled and development-mode
updaters, and never fire from observer destination acks — only the
authoritative server may steer an agent's updater, and a nudged check
re-validates against the server and runs the existing checksum and
self-test pipeline before swapping binaries. Agents deployed before
this change still converge on their old hourly cadence once; every
upgrade after that lands within a report cycle.
Contract deltas recorded in agent-lifecycle and api-contracts, with
boundary notes in security-privacy (no update authority in the echo),
performance-and-scalability (no steady-state work), and
storage-recovery (nothing persisted).
A pulse-agent installed on a Proxmox node merges into one canonical
resource whose primary ID is node:{sourceID}, so the agent:{hostID}
reference the alert evaluator uses (hostResourceID) matched nothing in
the registry. Per-resource grace overrides and operator-state lookups
then silently resolved to factory policy: alerts fired with no grace
while the policy preview, which receives the canonical ID directly,
showed the override working. Index the prefixed agent ref as a
canonical identity alias so runtime resolution lands on the same
resource the override was saved against.
Refs #1497
Contract-Neutral: behavioral fix#1497: index agent: ref as canonical alias, no public contract shape change
restoreProviderMSPArchiveFile copied each tar entry with an unbounded
io.Copy, so the only limit on what a restore wrote to disk was the size
the archive declared for itself. A gzip bomb, a PAX sparse entry claiming
a huge logical size, or a corrupt stream could fill the target volume.
Bound extraction the way readProviderMSPBackupManifestBytes already
bounds the manifest: a per-entry cap and a cumulative cap across the
whole restore, both enforced against the bytes actually copied rather
than the declared header size. That size comes from the archive, so it
is only good for an early reject, never as the bound. An entry that
overruns fails the restore and its partial file is removed, rather than
being silently truncated into a file that looks complete.
A restore that failed partway had already deleted whatever it replaced,
leaving a half-populated control plane that looks bootable and forcing
the retry to use replace. Roll the partial restore back to an empty
target instead, and say so in the error.
Follows the zip-slip fix in the same function (CodeQL alert 314).
The #1601 follow-on: per-container alert overrides were keyed by Docker
container ID (docker:{host}/{containerID}), which changes on every
recreate, so each image update silently re-armed alerts the user had
switched off and left a dead entry behind in alerts.json — the unbounded
growth that pushed the reporter's config past the old 64KB body cap
(raised in 38434a513). The v6 thresholds UI additionally wrote keys from
the unified hash id (docker:{host}/app-container-{16hex}), which the
evaluator never read at all.
Overrides now key on stable identity, docker:{host}/{containerName}:
- The evaluator resolves the name key first and falls back to the legacy
container-ID key so pre-migration entries keep working
(evaluateDockerContainer, checkDockerContainerState, the
container-update resolver, and reevaluateActiveAlertsLocked).
- MigrateDockerContainerOverrideKeys runs in the monitor sync next to
MigrateCanonicalOverrideKeys, driven by the unified resource snapshot:
it re-homes live legacy-ID and unified-hash keys onto the name key and
prunes orphaned ID-shaped entries, ending the per-update orphan
accumulation. Name-keyed entries for absent containers are kept so a
recreate under the same name still honours them.
- The UI candidate chain (single implementation in alertOverridesModel)
now leads with docker:{host}/{name} and trails the container-ID, short
ID, unified-hash and slash-tail forms, so rows bind pre-existing
overrides of every historical shape and the next save re-homes them.
Rows carry overrideStorageId/overrideIdCandidates so toggle,
connectivity, offline-state, edit and remove all write the stable key.
- The ignored-containers card copy now documents the wildcard forms
(runner-*, *-dev, *staging*) shipped in b5fa6a9af, under the title
"Ignored container patterns".
Contract deltas: alerts, frontend-primitives, monitoring, and
unified-resources now pin the name-keyed override identity, the single
frontend candidate-chain owner, the sync-cadence migration, and the
resource-facet-backed table identity respectively.
Verified live against a mock instance: a UI toggle persists
docker:{host}/loki and binds back after reload, and seeded
legacy/hash/orphan keys converge to name keys on disk within two sync
ticks. go test ./internal/alerts/... ./internal/monitoring/... green;
recreate survival pinned in
TestDockerContainerOverrideSurvivesContainerRecreate.
Refs #1601
The PVE and PBS branches of handleCanonicalAutoRegister each carried a
verbatim copy of the identity-matching ladder — exact host, then TLS
fingerprint conflict rejection, resolved host identity, then DHCP
continuity by node name plus token. golangci-lint's dupl flagged the pair.
Extract one matcher, findCanonicalAutoRegisterMatch, over a
canonicalAutoRegisterCandidate identity view that both instance types
project onto. Behavior is unchanged: the same ladder in the same order,
the same log messages and fields, and the same (index, preserveHost)
result, with the previous "break with preserveHost still false" paths
becoming explicit returns.
golangci-lint run ./... green; full go test ./internal/api/ green.
Contract-Neutral: dupl-only refactor of handleCanonicalAutoRegister: PVE/PBS identity-matching ladder hoisted into one helper, zero public-contract or behavioral delta
The vSphere adapter filled `Resource.Tags` with six fixed strings on every
resource — `vmware`, `vsphere`, `<kind>`, `source:vcenter`,
`connection:<name>`, `power:<state>` — and never read vCenter's own tag and
category system. Every VM in an estate returned a byte-identical set, so the
workload Tags column rendered the same dots on every row and filtering on any
of them selected everything. Commit 6b78feba8 default-hid the column and said
in as many words that the hide was a stopgap awaiting this fix.
`internal/vmware/client_tags.go` reads the CIS tagging service. That is a
different endpoint family from the `/api/vcenter/...` inventory reads, but the
same vSphere Automation API, so it reuses the caller's `/api/session` token
rather than opening and managing a second session. Associations come from one
batched `list-attached-tags-on-objects` POST per bounded object batch, never a
per-object request; tag and category names resolve through a client-scoped
catalog with a 10-minute TTL, so a steady-state refresh of a tagged estate
costs only the association reads while a rename still converges without a
restart. A vCenter without the tagging service, or an account without the tag
read privilege, degrades into a `tags` stage enrichment issue and leaves the
inventory untagged; it never fails the refresh.
The provenance strings stay. `Resource.Tags` is the only keyword set
`resourceSearchMatch.ts`, the `?tags=` resources filter, and saved
report-schedule tag filters read — `collectSearchCandidates` gathers no
`technology`, `type`, or `platformScopes` candidate — so dropping "vmware" or
"vsphere" would silently stop matching searches and saved filters that depend
on them. Real vCenter labels are appended to that set, never substituted for
it.
Because that flat set is deliberately mixed, it is the wrong source for a
per-row Tags cell. Real tags therefore also land on a canonical `VMware.Tags`
facet that carries vCenter's category alongside each name, and
`useWorkloads.ts` maps `WorkloadGuest.tags` from that facet for any resource
carrying VMware metadata — including the empty case, so a vSphere VM nobody
tagged renders an empty cell instead of falling back to the provenance dots.
vCenter tag names are unique only inside their category, so the flat label is
`category:name`: two categories may each hold a "Production".
With the column carrying per-row meaning again, `tags` leaves
VMWARE_WORKLOAD_DEFAULT_HIDDEN_COLUMN_IDS and the `defaultHiddenMigrationIds`
retirement list, and the state-model test that pinned the stopgap now pins its
absence. No un-hide migration ships alongside it: 6b78feba8 is on main but no tag
contains it, so the stopgap never shipped and no install carries the
auto-hidden preference. That holds only while the two stay together — the
migration writes the hide into each user's saved preference on first load,
so an rc cut from main carrying the stopgap without this commit would make
an explicit un-hide path necessary.
Mock fixtures carry uneven tag coverage — several categories on some objects,
one on others, none on the rest — because a uniform fixture set would hide
exactly the defect this data exists to catch.
Verified against a mock estate built from this branch: `/api/resources`
returns provenance plus real labels on the flat set and only real labels on
`vmware.tags`; the Tags column renders 2-4 dots per tagged VM and none for
untagged ones; a dot's tooltip reads `Backup:Nightly`, and clicking it
searches `tags:Backup:Nightly` and narrows 18 VMs to the 3 that carry it.
Contract deltas: performance-and-scalability.md Extension Point 17 replaces
the stopgap paragraph with the two-surface tag contract and the bounded
tag-read budget; unified-resources.md states the keyword-union vs facet split
and that a present-but-empty facet means "no operator tags" rather than a
fallback; storage-recovery.md extends its VMware descriptive-only boundary to
`vmware.tags`, because vCenter tag vocabularies read like protection policy
(`Backup:Nightly`) and a label the operator wrote must never satisfy a
coverage or compliance verdict that recovery-owned evidence should decide.
Resolves every open code scanning alert on the repository. Dependabot and
secret scanning were already clear.
SMART temperature truncation (alerts 312, 313). parseRawValue returns a
64-bit raw attribute value, but DiskSMART.Temperature is an int, which is
32 bits wide on the 386 and arm release builds Pulse ships. The range check
ran after the narrowing conversion, so a raw value of 4294967316 truncated
to 20 and was published as a plausible 20 degree reading.
validSMARTTemperature64 now gates the conversion.
Provider MSP restore archive names (alert 314). cleanProviderMSPArchiveName
rejected a leading "../" but not a bare "..", which path.Clean produces from
entries such as ".." and "a/../..". pathIsInside caught the escape
downstream, so this was not exploitable, but the sanitizer now rejects it
outright instead of depending on a second gate.
TrueNAS device paths (alert 315). vdev.Device is supplied by the appliance,
concatenated into a path and published verbatim on ZFSDevice.Path, so values
like "//evil.example.com/share" and "/\evil.example.com" passed straight
through. devicePath now drops traversal segments and backslashes and
collapses a leading double slash. The alert's open-redirect framing does not
apply here, there is no redirect sink on this path, but the value is
untrusted input rendered as a path and is worth normalising.
Patrol readiness cache key (alert 311). The key is persisted to
ai_patrol_model_readiness.json and embedded an unkeyed SHA-256 of the Ollama
Basic Auth username and password. That password is chosen by a human, so
anyone holding the evidence file could recover it offline at two SHA-256
operations per guess. The fingerprint is now HMAC-SHA256 keyed with a
32-byte per-install salt stored beside the evidence at mode 600. Credential
rotation still invalidates the cache and the key still survives a restart.
Each fix carries a regression test confirmed to fail against the previous
implementation.
monitoring.md carries the one warranted contract refinement. It already
required SMART temperature selection to accept only plausible readings, and
that rule now states the width at which plausibility is decided.
Contract-Neutral: CodeQL security fixes with no public-contract delta and no payload change. monitoring.md carries the one warranted refinement (SMART plausibility decided at 64-bit width). Residual demands are inapplicable: ai-runtime readiness prose documents interruption semantics, not cache-key derivation, and the credential-invalidation contract is unchanged; cloud-paid and deployment-installability contracts never name archive-entry sanitisation; agent-lifecycle owns smartctl.go but its SMART temperature prose lives in the staged monitoring.md.
Bare entries keep their historical prefix semantics. Entries may now
also use the wildcard forms already established by the PBS datastore
excluder, so *-dev matches a suffix, *staging* a substring, and
runner-* an explicit prefix. Wildcard-only entries are skipped, that
job belongs to DisableAllDockerContainers.
Suffix matching is what the reporter on #1601 was approximating with
hundreds of per-container disable toggles, which is also what pushed
their config past the old request body cap.
Refs #1601
Contract-Neutral: extend docker ignored-prefix matching with wildcard forms; no schema or payload changes
Alert config grows one Overrides entry per toggled resource, so the
64KB cap on PUT /api/alerts/config rejected saves from instances with
a few hundred disabled containers with 'http: request body too large'.
The 32KB cap on bulk acknowledge/clear failed ack-all during large
alert floods, the exact situation it exists for. Intent policies carry
per-resource rules with the same scaling shape.
All four now share a 1MB bound, which still caps memory per request
but no longer rejects legitimate fleet-sized payloads. Endpoint tests
pin a >64KB config save and a >32KB bulk ack at 200.
Refs #1601
Contract-Neutral: raise alert config and bulk ack request body caps; no payload shape or field changes
The 25 July GLM swarm generated these and they were never collected, unlike
the 0712 through 0724 batches already in main. They cover apiClient retry and
abort handling, the prompt-secret model boundary sanitizer, AI tool
normalisation, Patrol handoff, threshold table state, the audit log panel, the
connections ledger, licence and resource-badge presentation, and the AI
intelligence store.
Verified against current main before harvesting rather than trusting their
age: 36 Go tests and 197 frontend tests pass, go vet is clean, and both
batches were re-run after formatting.
/download/pulse-agent served whatever agent artifact sat on disk with no
relation to the running build. Local agent binaries are build outputs that
nothing refreshes on their own, so they go stale silently: a dev backend was
found serving a four-week-old v6.0.5 agent while reporting 6.2.0-rc.8.
Staleness is not cosmetic. The installer renders its service wrapper from the
server's current template, so an agent predating a flag that template now
passes exits immediately with "flag provided but not defined" and crash-loops
under its watchdog. That is how a real host lost its agent: the version
mismatch was reported only as an installer warning, after download, easy to
read as noise.
Validation already scanned the binary for its report-contract endpoints, so
the version check joins that same single pass and rejects a binary that does
not carry this server's agent version. Refusal is loud where the old warning
was quiet: a dev server answers 404 naming the stale path and the build
command, and a published release falls through to the existing release-asset
proxy and fetches the matching version, which makes production self-healing
rather than silently downgraded.
The expected version resolves through updates.GetCurrentVersion rather than
the compiled-in serverVersion. The first cut of this guard used serverVersion
and was inert on exactly the builds that need it: the enterprise binary
compiles in "dev-pro", no version parser accepts it, and the check disabled
itself. It passed its unit tests and still served the stale binary; only
replaying the real v6.0.5 artifact through the running server exposed it.
"dev-pro" is now pinned in the version table with that reasoning attached.
The shared download-test fixture built a binary carrying the report endpoint
but no version string, which a real agent always has, so it now stamps the
expected version. Verified non-vacuous in both directions: those tests fail
with the guard active and the thin fixture, and pass with a faithful one.
Contract-Neutral: storage-recovery is pulled in only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: it constrains which agent binary /download/pulse-agent serves, touching no storage provider, backup target, recovery repository or protected-workload evidence. The agent-lifecycle, deployment-installability and api-contracts deltas staged here cover every boundary the change actually moves.
Mock mode suspends pull-based collection outright, but push-based agent
reports were never given the same treatment, so a real machine still landed
in monitor state while the unified read path substituted the mock snapshot
over the top. The hosts were hidden, everything downstream was not: a real
Unraid box raised a live storage-topology alert next to fixture data, and
its identity persisted through host continuity.
Three vectors, each closed at its source.
Agent ingest now drops real reports while mock mode is on. ApplyHostReport,
ApplyDockerReport and ApplyKubernetesReport acknowledge the report with the
reporting agent's own identity and touch no state, so nothing raises alerts,
persists continuity, records metrics or feeds the online/offline sweep. The
acknowledgement stays a success so a real agent does not read a demo server
as an outage and retry-storm it.
recentStandaloneHostContinuityEntries returns nothing in mock mode. Those
entries are written to disk from real reports and outlive the toggle, and
every consumer injects them after the read path has already substituted the
mock snapshot, so a machine that reported before mock mode was enabled came
back by its real hostname. There is no real-polling exception here: agent
ingest is not gated on PULSE_MOCK_KEEP_REAL_POLLING and the read state is
mock either way.
Active-alert restore is now opt-out, and mock mode opts out. SetMockMode
already clears active alerts when the toggle flips, but a process booting
with mock mode already enabled never ran that path and restored real alerts
from active-alerts.json.
TestHostedTenantAgentInstallTokenCannotReportToOtherTenant used mock mode as
scaffolding. Under the ingest guard both tenants would be empty and its
isolation assertion would pass without exercising the boundary, so it now
runs in real mode. Every new test pairs the mock assertion with a real-mode
one for the same reason.
Mock mode never initialises real PVE/PBS/PMG clients and does not run the
platform pollers, so every configured real source sat in the connections
ledger at "awaiting first poll" forever. That published real connection
names and addresses through /api/connections while the rest of the payload
was authored fixtures, and surfaced them on monitoring copy: the Proxmox
workloads empty state rendered "Collection pending: minipc" next to three
mock nodes.
/api/config/nodes already substitutes mock entries wholesale in mock mode
and rejects node mutations outright, so the ledger was the one surface that
had not been brought in line. Move the mock-mode input shaping into
applyMockLedgerInputs in platform_mock_connections.go, which already owns
the mock vSphere, TrueNAS, and availability ledger fixtures, and drop the
config and persistence derived sources there. PULSE_MOCK_KEEP_REAL_POLLING
keeps the previous behaviour, since those sources do collect under it.
Proof is at the payload level rather than the aggregator inputs: the new
handler test asserts the served /api/connections body contains no real
source by name or address, and was verified red before the fix.
Contract-Neutral: agent-lifecycle is named only by the broad internal/api/ Extension Points prefix and this change does not move that boundary: agent rows come from the monitor hosts snapshot, which is untouched. The api-contracts and storage-recovery deltas cover the boundaries actually moved. No payload field was added or changed, so the backend-API-payload proof list does not apply; the handler-level payload proof is TestConnectionsHandleListDropsRealSourcesInMockMode in internal/api/connections_handlers_mock_test.go, verified red without the fix.
Seeded mock history recorded docker hosts with cpu, memory and disk only,
while the synthetic generator used past the seed window emits the full guest
metric set. The result was an inversion across chart ranges: a docker host had
no diskread, diskwrite, netin or netout history at 5m through 24h and full
history at 7d and 30d. Real docker hosts report both through the agent, so the
seed now covers the same series the generator does and every range agrees.
The guardrail test asserts seeded coverage matches the synthetic metric set
rather than a hand-listed set, so a future series added to one path cannot
quietly skip the other.
Mock history is seeded for 48h, so chart windows longer than that fall through
to the synthetic generator in mock_chart_history.go. That generator produced
cpu, memory, disk and the I/O pairs but never memoryused, so a 7d workloads
read returned 64 points for every other series and zero for memoryused. The
memory column in host-capacity mode had no series to draw at all, which reads
as a broken column rather than missing mock data. Real installs are unaffected:
the live PVE tick writes memoryused to the metrics store and the store rollup
groups by metric_type without an allowlist.
The synthetic generator now derives memoryused from the sampled memory
percentage and the fixture memory capacity, the same derivation live mock ticks
and the seeder already use, so the series stays continuous across the seed
boundary. Capacity comes from a new fixture registry beside the existing metric
role registry rather than a per-call fixture graph clone. Docker containers and
pods stay out of it, matching the Proxmox-only memoryused contract.
Update source-shape guards for the shared per-generation resource list and cached mock unified view. Demote the onboarding E2E spec after its rate-limit/shared-state retry flake on main so it continues running without reddening the advisory verdict.
Contract-Neutral: advisory CI guardrails and probation tier bookkeeping only; no runtime or public contract delta
ResourceHandlers.getStore opens a SQLite handle per org and caches it for the
process lifetime, and nothing ever closed them. CleanupTenant already released
patrol, AI, RBAC, license and monitor-adapter state for a deleted org but left
the resource store open, so an offboarded tenant kept its file descriptors and
its unified_resources.db-wal/-shm files alive, and its directory could not be
fully removed.
CloseTenantStore releases and evicts one org's store and is now called from
CleanupTenant alongside the other per-tenant teardown. CloseStores releases all
of them, exposed as Router.ShutdownResourceStores next to the existing
Shutdown*/Stop* helpers.
Found while investigating the flaky hosted-tenant test. It is not what made that
test flaky, which was a detached guest-metadata write fixed separately, but the
leak is real on its own: verified by observing that -wal and -shm sidecars, which
exist only while a connection is open, survived the test before this change and
do not after it.
persistGuestIdentity spawned a detached goroutine per changed guest to write
guest_metadata.json, with a comment noting it avoided blocking the monitor.
Nothing tracked those goroutines, so neither Monitor.Stop nor
MultiTenantMonitor.Stop could wait for them and a queued write could land after
shutdown. In hosted mode that means a write into a tenant directory that
offboarding is already removing, and a stray guest_metadata.json.tmp left
behind when the atomic write is interrupted.
The store now owns the goroutine. SetAsync tracks the write on a WaitGroup and
WaitForPendingWrites drains it under a bounded timeout matching
tenantMonitorShutdownTimeout, so a wedged store cannot hold up tenant teardown.
Monitor.Stop drains before closing the metrics store.
This is what made TestHostedTenantAgentInstallTokenCannotReportToOtherTenant
flaky: t.TempDir cleanup raced a queued write into orgs/client-b and failed
with "directory not empty". The test itself is unchanged, because it was never
a test bug. A goroutine dump at cleanup time showed the writers still live,
created by persistGuestIdentity, blocked on the store mutex.
Verified causally rather than by observation alone: the target test fails 0/4
with the drain removed and passes 8/8 with it, against 2/3 failures on the
unmodified baseline. The regression tests fail if SetAsync stops tracking its
goroutine.
Note for a future pass, deliberately not changed here: each changed guest still
triggers a full-file save, so one poll cycle over N changed guests does N
marshals and N atomic writes that serialize on the store mutex anyway. Fixing
that means coalescing at the call site and is a behavioural change beyond this
defect.
Fixes#1681Fixes#1682Fixes#1683
Contract-Neutral: Notification grouping initialization and alert-config propagation do not alter the broadly referenced agent-lifecycle or storage-recovery contracts; primary alerts, notifications, API, and monitoring contracts and regression proofs are updated.
Every list-shaped resources request deep-cloned the whole registry:
HandleListResources via ListForPresentation, the storage summary and
incidents handlers and the k8s namespaces handler via List/ListByType,
and the stats handler once more. With the frontend polling three pages
plus summaries every few seconds, the same unchanged world was cloned
dozens of times between snapshot updates.
Cache the raw and presentation lists on the existing per-generation
registry cache entry (same invalidation: entries rebuild when the seed
lastUpdate moves) and hand requests a flat top-level copy instead. Every
decorator in the request pipeline was audited to write only top-level
fields on request-owned elements: action availability, discovery
targets and readiness, metrics targets, canonical metadata refresh, and
contract types all assign freshly built values. The one nested writer —
the PMG list prune, which cleared relay domains through the shared
pointer — now clones the PMG struct before clearing, with a regression
test pinning both the clone-on-write and the cache's immunity to
request decoration. Read-only consumers (stats aggregation, storage
filtering, namespace counting) use the shared list with no copy at all.
Single-resource lookups (presentationResourceByReference and its
callers) still deep-clone per lookup; they are cold paths and stay
as-is.
Contract-Neutral: per-generation shared resource lists: response bytes pinned by contract tests, no payload delta
Schema v6 shipped audit_logging_persistent and audit_events_30d as Pro adoption
signals. Neither discriminated. pkg/server installs the SQLite audit logger on
every install for defense in depth and gates only the read/export endpoints, so
the boolean was true on all 8 installs that had taken rc.8 and 0 rows in the
retained table have ever had it false. The event count measured that background
write volume: three of those eight unlicensed community installs were pegged at
the receiver's 100000 clamp ceiling, with the rest between 4863 and 67509.
Schema v7 replaces both with audit_reads_30d, a count of requests that cleared
the license gate on an audit read or export surface. A read requires a human
action, so unlike store presence or write volume it cannot settle into a
constant. The recorder is wrapped INSIDE RequireLicenseFeature so unentitled
requests never count, and the persisted marker carries a timestamp and a coarse
activity class from a fixed allowlist. Query filters, actors, ranges, and every
audit row read stay on the install.
The retired columns are left in the live database. They hold real rc.8 rows and
migrations only add, so dropping them would be a pointless risk; nothing writes
them once the receiver struct loses the fields.
Adds the guard this class needed. LicensedFeatureAdoptionFields registers every
field that exists to measure licensed-feature adoption, and
TestLicensedFeatureAdoptionFieldsDiscriminate builds an unused install through
the real production snapshot paths, installs a real SQLite audit logger exactly
as pkg/server does, records a baseline audit event, and fails if any registered
field is non-zero. Pinning a console logger there would have made the guard pass
while the payload lied, so it deliberately does not. The guard was verified by
reintroducing the v6 sourcing and confirming it fails with the field named.
A companion test pins the three retired fields so they cannot return under
their old names.
This is the third instance of one bug class. v6 removed
pulse_intelligence_patrol_autofixes_30d, hardcoded to zero with no increment
site, and then introduced two fields that were constant in the other direction.
Three occurrences is a guard, not a habit.
Verified end to end on a running unlicensed install: the payload that reported
audit_logging_persistent true under v6 now reports audit_reads_30d 0, and
seeding two in-window reads, one outside the window, and one with an invalid
activity class yields 2.
In mock mode every unified read-state access built two throwaway
registries: mock.UnifiedResourceSnapshot constructed one to derive the
resource list, and the monitor's currentUnifiedStateView ingested that
list into another, deep-cloning all resources both ways. Chart requests,
broadcasts, alert evaluation, and API reads each repaid that full cost —
the dominant share of the demo's 76TB/9.5d allocation churn, since every
one of those reads runs against a world that only changes on the 2-second
mock tick.
Introduce fixtureDataVersion, a token that advances on every observable
mock-graph change (metric ticks and the structural changes that bump
fixtureRevision, which stays structural-only so seeded trend history
remains reusable). Memoize the package-level UnifiedResourceSnapshot and
the monitor's mock-branch state view against it, so consumers between
ticks share one immutable build. Sharing mirrors the semantics the
persistent-store ReadState path has always had in real mode: all
consumers were audited — they ingest (which clones), copy before
top-level writes, or build fresh outputs. Real-mode paths are untouched.
Contract-Neutral: mock snapshot memoization: identical data served from cache, no contract delta
Six of the eight Pro-exclusive features had no telemetry field at all, so
there was no way to answer whether RBAC, audit logging, scheduled reporting,
agent profiles, alert-triggered AI, or Kubernetes AI were being used by the
installs paying for them. Schema v6 adds nine content-free adoption signals:
alert_ai_enabled AIConfig.IsAlertTriggeredAnalysisEnabled()
rbac_custom_roles non-built-in roles, per org
rbac_user_assignments user-to-role assignments, per org
audit_logging_persistent a persistent audit store is active, not console
audit_events_30d audit events retained inside the window
report_schedules configured scheduled reports
report_schedules_enabled scheduled reports switched on
report_schedules_run_30d schedules whose last run falls inside the window
agent_profiles configured agent profiles
Counts only. Role names, permissions, usernames, schedule names, delivery
recipients, report scope, profile names, and every audit event field stay on
the install. kubernetes_ai needs no field of its own: it is derivable at read
time from alert_ai_enabled combined with the existing kubernetes_clusters
count, and a dedicated field would be redundant.
Config-sourced signals are read through applyLicensedFeatureConfigSnapshot;
RBAC and audit live behind the router and are read through
Router.ApplyLicensedFeatureTelemetrySnapshot. The RBAC read goes through a new
TenantRBACProvider.PeekManager so a background telemetry read can never
provision an RBAC store for an org that has never used RBAC.
Also removes pulse_intelligence_patrol_autofixes_30d and the AutoFixCount
field behind it. patrol_run.go hardcoded AutoFixCount to 0 and no increment
site existed anywhere in the tree, so the counter was zero in all 233,364
retained production pings. That was a wiring bug, not evidence that nobody
uses Patrol fixes; governed fixes are delivered through the approved-action
pipeline, which is already instrumented. The field was plumbed through run
records, history persistence, the Assistant handoff, and telemetry while being
structurally incapable of holding a non-zero value.
Verified end to end against a running install rather than only in unit tests,
which is precisely the check the autofix counter never had: seeding three
report schedules (two enabled, one last run inside the window) and two agent
profiles produced report_schedules 3, report_schedules_enabled 2,
report_schedules_run_30d 1, agent_profiles 2 in the Settings telemetry
preview, and signing in moved audit_events_30d to 1.
The private receiver landed first in pulse-pro 78ff7dd so the new fields are
accepted on arrival.
Mock mode seeded 90 days of synthetic history for every resource at
startup. Measured on the demo droplet with identical entity counts, that
put the process near 1GB of Go heap before the first tick; a 6h seed
runs at ~200MB. Every local dev instance and CI run in mock mode paid
that gigabyte silently, and on 1GB hosts it OOMed the box.
48h keeps day-scale charts fully populated. Long-range chart work opts
in with PULSE_MOCK_TRENDS_SEED_DURATION, which is how the public demo
already pins its own seed (6h).
Contract-Neutral: mock seed default resize: dev-only mock path, no contract delta
A live heap profile of the demo (9.5 days uptime) showed 76TB of
cumulative allocations, with inferMetricRole accounting for 12% of the
total: the classifier table and its keyword slices were rebuilt on every
call, and normalizeMetricRoleTokens constructed a fresh strings.Replacer
per token, paying the lazy trie build each time. Both run per resource
per 2-second mock tick, so the demo spent a measurable share of its
single vCPU feeding the garbage collector.
Both structures are static; make them package-level.
Contract-Neutral: mock allocation hoists: behavior-identical, no contract delta
canAccessPlatformAdminSurface publishes billingAdmin for any instance
administrator. RequirePlatformAdmin compared the session user against
cfg.AuthUser alone, so on an instance whose only administrators are SSO
principals the UI offered the surface and the route refused it. Same
capability against enforcement split as 28fd2d1c1, on the hosted routes.
The session branch now uses sessionUserCarriesAdminPrivileges, which is what
the capability already resolves to.
A straight swap would have been worse than the bug. That helper treats any SSO
principal as an administrator when no local admin is configured, and a hosted
control plane authenticates its tenants by SSO, so on a control plane with no
local admin every tenant would have become a platform admin. The session
branch is therefore also gated on the request not being org-scoped, matching
what ensureAdminSession and the security status snapshot already do. Removing
that gate lets an org-scoped tenant session reach the surface with a 200,
which the parity test pins.
The org-scope test itself was written inline in two places and is now one
helper, sessionIsOrgScoped, so the instance-versus-tenant boundary has a
single definition rather than a copy per caller.
RequireOrgOwnerOrPlatformAdmin is untouched. It has no session branch in its
platform-admin switch by design and requires org ownership instead.
Contract-Neutral: behavioral fix on existing routes, no request or response shape change; platform admin route aligned with the billingAdmin capability it already publishes
On the OIDC-only pattern there is no local admin, so SSO principals are the
only administrators the instance has. ensureAdminSession already knows that:
sessionUserCarriesAdminPrivileges admits the configured admin, anyone holding
an RBAC admin grant, and any SSO principal when no local admin is configured.
Three guards did not use it. They compared the session username against
cfg.AuthUser directly, which on those instances is empty, so they could admit
nobody at all. The same operator was admitted by the settings routes and
refused by discovery, by public URL capture, and by config export and import,
which is three answers to one question.
Verified against an unlicensed OIDC-only router before and after. Before,
sessionUserCarriesAdminPrivileges returned true for the SSO owner while
canCapturePublicURL and discovery isAdminRequest returned false and
/api/config/export returned 403. After, all four agree.
This does not widen anything. On an instance that does configure a local
admin, an unrelated SSO principal is still not an administrator and every one
of these guards still refuses them, which the parity test pins in both
directions.
RequirePlatformAdmin is deliberately left alone. Its stricter session rule is
documented as intentional for the hosted control plane rather than an
oversight, and loosening it is a different decision from this one.
Contract-Neutral: behavioral fix on existing routes, no request or response shape change; three guards switched to the canonical session-admin helper they should already have used
handleChangePassword refuses a non-admin proxy user outright. The session
branch of that rule was never written, so any authenticated session reached
the current-password comparison.
Knowing the current password is the real gate on the change, so this was not
an escalation for someone who did not already know it. What it did give an
authenticated non-admin was an oracle: submit a guess at the local admin
password and read the answer off the 401, under the login rate limit and
lockout rather than outside them. With the password in hand the same caller
could then change it and lock the administrator out.
ensureAdminSession is a no-op for requests carrying no session cookie, so the
Basic Auth path this handler was largely written around is unaffected, and it
admits SSO principals on instances with no local admin.
Found by sweeping every proxy-admin check for a missing session counterpart.
The other 43 call sites were either identity extraction rather than
authorization, already carried a session branch, or were compensated
downstream by ensureSettingsScope.
Contract-Neutral: behavioral hardening on an existing route, no request or response shape change; session callers now held to the same admin rule the proxy branch already applied
The read deadline was refreshed only by protocol pong frames, so a client
whose ping/pong control frames were delayed more than 6 seconds or eaten
by a middlebox (Cloudflare edge, AV proxies) was killed every 60 seconds
even while its 30-second JSON heartbeats were arriving. Demo logs show
the signature clearly: sessions dying at exactly 60s/114s/168s, i.e.
zero, one, or two pongs before execution, while the user watched the
badge cycle through Reconnecting on an otherwise healthy connection.
Refresh the read deadline on every successful read, widen it to 90s so a
background tab throttled to one heartbeat per minute still survives, and
ping every 30s so pong-only clients tolerate two lost round trips.
Contract-Neutral: websocket keepalive hardening: read-deadline/ping cadence only, no payload or contract delta
A settings capability is a promise the routes have to keep. Without an RBAC
licence the authorizer allows every action, so capabilities derived from it
alone reported true while the matching route stayed gated by
ensureSettingsScope and in turn ensureAdminSession.
canAccessPermissionSurface already refused to trust the authorizer for a
non-admin proxy caller. The session half of that rule was never written, so a
non-admin session on an unlicensed instance was told apiAccessRead,
apiAccessWrite, singleSignOnRead and singleSignOnWrite were all available. The
nav gates on exactly those flags, so the API Access and Single Sign-On tabs
rendered, their first request came back 403, and the user got an error toast
on a tab they were never able to use. Everything routed through
canAccessAdminSurface was already correct, which is why authenticationRead and
the audit surfaces behaved and these two did not.
The fallback uses snapshot.sessionIsAdmin, which derives from the same
sessionUserCarriesAdminPrivileges the routes enforce, so the capability now
matches the answer the route will give. That keeps the OIDC-only pattern
working, where SSO principals are the instance's only administrators.
Nothing was reachable that should not have been. This is a capability
reporting fix, not an access control one.
Refs #1672
Contract-Neutral: settingsCapabilities JSON shape is unchanged (same 14 fields, same types); this corrects a wrong value returned to non-admin sessions when no RBAC authorizer is registered, no public-contract delta
TrueNAS SCALE catalog apps ship one-shot init containers from ixSystems'
own base images (permissions, postgres_upgrade, pgvecto_upgrade). They run
to completion and stay exited for the life of the app, so essentially every
installed app produced a standing CRITICAL that could never clear.
TrueNAS already separates a completed workload from a failed one before
Pulse sees it. Its app state machine reports a container that exits with a
normal exit code as EXITED and one that exits abnormally as CRASHED, then
rolls any CRASHED container up into an app-level CRASHED. Pulse was
treating EXITED as a failure, which inverted the platform's own semantics
and fired precisely on the apps TrueNAS reported RUNNING.
EXITED now raises nothing. CRASHED still raises a per-container incident so
the app-level CRASHED names the failing service, which is the case that
was actually worth alerting on.
The same false assumption sat in the app's rendered container state, which
read app.Containers[0] and so let whichever init container app.query
happened to return first decide how a healthy app displayed. That now
collapses the workloads using the precedence TrueNAS itself uses.
Refs #1677
A full monitor reload (triggered by every node settings save) starts
from empty host state, and the physical disk poll runs on the first
cycle. Evaluating disk alerts before a node's agent has re-reported
treats the agent's --disk-exclude patterns as absent, so excluded disks
fired wearout/health alerts that the next cycle resolved. Defer disk
alert evaluation for unlinked nodes during a two minute settle window
after monitor start.
Refs #1674
Contract-Neutral: behavioral fix: defer disk alert evaluation until host-agent linkage settles after reload, no contract delta (#1674)
The three deep-dive documents describe enumerations and tuned constants that
live in code. Prose cannot notice when a fifth state or an eighth signal type
is added next to it, so the documents would have rotted silently.
These guards derive the truth from the source rather than asserting literal
strings. A test that only checks the document contains RESOLVING keeps
passing when a new state appears, which is the drift actually worth catching.
Covered, in both directions where the set is enumerable. Session states and
tool kinds against internal/ai/chat/fsm.go and the ToolCallKind String
method. Patrol signal types and the five tuned signal thresholds against
internal/ai/patrol_signals.go. The investigation attempt limit and cooldown
against internal/ai/findings.go. The look-before-asking bound and the tool
concurrency cap against internal/ai/chat/agentic.go. Identifiers named in the
structured errors section against the declared error codes and tool names.
Error codes are checked document to code only. The document names a
representative sample rather than all 45 declared codes, so requiring the
reverse would force a table that churns on every addition while catching
nothing a reader cares about. A rename or removal still fails.
Writing the guard immediately found a real mistake in the document it
guards. ErrCodeFSMBlocked is FSM_BLOCKED, not fsm_blocked as the prose
claimed, corrected here.
Each guard was verified to fail by adding a fifth session state, adding a
signal type, and changing a threshold, then confirming the failure named the
document and the value.
Contract-Neutral: test and documentation only
The evaluation mode added earlier today was hollow. An unlicensed control
plane starts, but release-build client runtimes only trust entitlement
leases chained to a Pulse-signed licence, so its client workspaces ran
without the capabilities the provider was evaluating. Standing the stack
up proved the portal and the isolation boundary and nothing else.
setup.sh now requests a capped evaluation licence from the licence server
when no licence path is set, sending only the public half of the key it
generated locally. It degrades rather than blocks: a missing key, an
unreachable server, or a licence-free response leaves the install
unlicensed with an explicit warning, an existing licence on disk is
reused, and PULSE_PROVIDER_MSP_SKIP_EVAL_LICENSE skips it for air-gapped
hosts. Guarded with an if-test rather than a trailing true inside the
command substitution, because the derive helper calls die and exit in a
subshell is not a status that can be caught, so setup.sh aborted under
set -e.
Second fix. The lease capability ceiling was selected by licence
presence, so an unlicensed provider control plane fell through to the
Pulse-hosted branch and minted leases claiming relay, mobile and push,
which a provider deployment cannot serve and which previously caused
repeating relay registration failures in client runtimes. The ceiling now
follows hosting via SetProviderHosted. providerChained keeps its narrower
meaning of having a licence available to embed.
Third. Corrects a sentence I wrote into cloud-paid.md this afternoon
claiming msp_eval carries the same MSP capabilities. A plan version
selects the workspace cap; it does not entitle a workspace.
The regression test was negative-tested by reverting the ceiling selector
and confirming it catches relay. All three setup.sh degradation paths
were exercised directly. Licence server side is pulse-pro 7f6a319 and is
not live until the next deploy-license-server run.
99ad8c2c4 routed collectDiskIO through fsfilters.IsVirtualBlockDevice to
stop ZFS zvols being collected. That helper answers "can this device report
SMART", which is the wrong question for I/O accounting: it also matches
vd* and xvd*, so every agent running inside a KVM, Proxmox or Xen VM lost
host disk I/O entirely, since vda is the real disk on those machines. It
also dropped md, nbd, rbd, drbd and pmem, none of which were part of the
report.
Add IsNonPhysicalDiskIODevice for the I/O question and use it here. It
covers only loop, ram, zram, dm- and zd, so the zvol fix for #1671 stands
while everything that was collected before 99ad8c2c4 is collected again.
Tests assert vda and xvda survive the I/O filter while still being excluded
from SMART, so the two predicates cannot be conflated again.
Contract-Neutral: behavioural regression fix, no contract delta
Refs #1671
rcourtman