Discovering Docker inside Proxmox LXC guests previously required an
undocumented environment variable plus a hand-edited systemd unit,
which is how the surface kept shipping invisible. The opt-in is now a
persisted system setting with an admin-only toggle in Settings →
System → General, presented as its own "Docker in Proxmox LXCs" card
under the existing Docker/Podman updates card, with the requirements
(node agent with command execution) and data-collection bounds stated
inline.
The setting rides the admin-gated system settings endpoint
(RequireAdmin + settings:write, matching every other server-wide
toggle). Flipping it persists first, then applies to the runtime
config and fires a reconfigure hook so the router rebuilds the
monitor's Docker checker and inventory collector immediately — no
restart. PULSE_ENABLE_PROXMOX_GUEST_DOCKER_INVENTORY remains
authoritative: when set it locks the toggle (EnvironmentLockBadge in
the UI, HTTP 400 from the API), and an unrelated settings save can
never clobber the runtime value. The opt-in guardrail test now pins
the settings path, the env lock, and the persisted field.
Installer and node setup-guide callouts now point at the toggle first
with the environment variable as the locking override, and the docs,
their public mirrors, and the agent-lifecycle subsystem contract say
the same. New i18n strings shipped for en/de/es.
Verified live in mock mode: toggle off/on from the UI drives the
collector teardown/setup log lines, the value survives a server
restart, and all three touched surfaces render at desktop and mobile
widths (receipt in frontend-modern/browser-verification.json).
The pct exec socket probe that discovers Docker inside LXC guests
failed invisibly: errors logged at debug only, and a failed probe never
set DockerCheckedAt, so the guest stayed in first-check state and was
re-probed on every poll forever. On a node where lxc-attach could not
enter unprivileged guests that meant ~49 pct execs per minute of
steady-state churn, with nothing at default log level to say why the
Proxmox page showed no Docker.
Track consecutive probe failures per guest in the monitor. The first
failure of a streak, and the point where the backoff reaches its cap,
log at warn with the guest, node, and error; repeats stay at debug.
Failed probes retry with exponential backoff from one minute to a
thirty-minute ceiling, a success clears the streak with an info log,
and reconfiguring the checker resets all streaks so command execution
enabled from the UI at runtime still retries immediately. Entries not
refreshed for a day belong to deleted guests and are pruned.
PULSE_PROXMOX_GUEST_DOCKER_INVENTORY_VMIDS now gates the socket probe
as well as inventory collection. It previously only limited inventory,
so opting into specific guests still probed every running LXC on every
cycle; now guests outside the allowlist are never pct exec'd at all.
Subsystem contract and configuration docs updated to match.
lxc-attach into an unprivileged guest writes /proc/<pid>/uid_map, which
needs CAP_SETUID in the parent user namespace. NoNewPrivileges drops
CAP_SETUID from the effective set and also stops lxc-attach falling back
to the setuid newuidmap/newgidmap helpers, so the socket probe dies with
"write_id_mapping: 61 Operation not permitted".
install.sh already relaxed NoNewPrivileges for this, but only when the
agent was installed with --enable-commands. Command execution is also
togglable from the server afterwards: applyRemoteConfig starts the
command client without rewriting the unit. An agent installed without
the flag and switched on later therefore ends up able to run commands
and unable to attach to unprivileged guests, so Docker inside every
unprivileged LXC disappears from the Proxmox page. The probe failure is
logged at debug level and retried on every poll, so the surface looks
empty rather than broken while the agent re-probes the whole guest list.
Grant CAP_SETUID/CAP_SETGID to any PVE agent rather than gating on the
install-time flag, so the later toggle lands on a unit that can attach.
Ambient capabilities restore exactly the privilege lxc-attach needs and
leave the rest of the sandbox intact; the existing install-time
relaxation is unchanged.
Verified on a live PVE node. With the hardened unit the probe succeeded
only on the three privileged guests and failed on every unprivileged
one. After the ambient grant CapEff regained CAP_SETUID and both
unprivileged Docker guests were discovered, taking that node from one
Docker LXC to three.
An agent whose token authenticated seconds ago cannot be in a
credential outage, yet the fleet doctor flagged it critical with
"missing or revoked" whenever the token id missed the server's token
registry view, and the offered repair-authentication loop cannot fix a
stale server-side registry view. When the subject's own row shows the
judged token authenticating within the freshness window, emit a
warning-severity registry-stale verdict naming the id and the last
authentication instead. Genuinely stale rows keep the critical verdict.
Refs #1730
Saved views stored a named URL query string in localStorage under
`pulse:filterbar:saved-views:<key>` and offered it back from a "Saved"
control on eight surfaces. A saved view was only ever the page's URL:
`saveCurrent` snapshotted `window.location.search` and `applyView`
navigated back to it.
The browser already does that, better. A bookmark syncs across devices,
survives clearing site data, is searchable, can be foldered, and can be
shared with someone else. The localStorage copy did none of those, so a
"saved" view silently vanished on a new browser or a cleared cache.
There is also no evidence anyone used it. It shipped 2026-05-01 in
v6.0.5 and no Pulse issue, comment, or discussion has mentioned it since.
#1510 is the telling case: filed two months after it shipped, by a user
who wanted their Proxmox status filter to survive a tab switch, which is
exactly the job this feature claimed. They did not reach for it, and the
fix that satisfied them was sticky route state (d20c556db).
Removed the hook, the menu, the `savedViewsKey` prop, and the per-surface
keys. URL-backed filters and sticky route state are untouched, so a
filtered page is still a shareable, bookmarkable link. On mobile, a
surface with no other auxiliary control no longer renders an action row
that existed only to hold the Saved button.
Subsystem contracts keep the URL-ownership rule this feature depended on
and now state it as the primary contract, with a note not to reintroduce
an in-app view library. A guardrail asserts the FilterBar barrel does not
re-export SavedViews.
User-visible removal: needs a release-note line at the next cut.
The Assistant overlay and its backdrop reserved a hardcoded
calc(5rem + env(safe-area-inset-bottom)) for a bar that measures 45px. At
375x812 the backdrop ended at y=732 while the bar starts at 767, leaving a
35px band that was neither dimmed nor click-blocked: page content showed
through and stayed interactive outside an open modal.
The bar's height is content-driven and already includes the safe-area inset
via pb-safe, so no caller can hardcode it correctly. Five sites had tried:
the Assistant panel, its backdrop, the star banner, and both halves of the
.filter-bottom-nav-aware-panel rule, which also carried a derived 6rem.
MobileNavBar now publishes its measured height as --pulse-mobile-nav-height
and those five sites read it. Publishing happens in onMount rather than the
ref, because Solid runs refs before the node is in the document where
offsetHeight is 0; reading after mount forces layout and gets the real value
on first paint. A ResizeObserver covers content-driven changes and a resize
listener covers the xl breakpoint, where the bar swaps between its height and
0 by display alone.
The declared :root value is only a pre-measurement fallback and deliberately
under-estimates. Over-reserving is what produces the un-dimmed band; reserving
slightly less is harmless because the opaque bar paints over it.
Verified at 375x812 and 1280x800: backdrop and panel now both end at 767
against a bar top of 767, gap 0; the point 5px above the bar hit-tests to the
Assistant panel rather than page content; the filter panel resolves to 45px on
mobile; and the star banner's md:bottom-4 still wins at 16px.
Adds the regression proofs the shape guard asks for, including an
architecture test that fails if any runtime source reintroduces a literal
bar height, and records the ownership boundary in the frontend-primitives
and ai-runtime contracts.
Two counters could not answer the question they exist for.
pulse_intelligence_patrol_runs_30d was counted from the operator-facing
patrol run history, which is capped at MaxPatrolRunHistory (100). On any
install patrolling on a normal schedule that cap is reached within hours
of a thirty-day window opening, so the field reported the cap rather than
the run count, and the calls-per-run ratio derived from it was inflated by
a censored denominator. Patrol run history now carries an uncapped daily
tally beside the capped list, advanced from a persisted high-water mark so
repeated full-list saves cannot double count, pruned to 31 days, and read
in preference to the list. An install with no tally yet falls back to the
list, so the counter never regresses on upgrade.
Schema v8 split agent-side pre-mutation refusals into target-change,
prerequisite and contract categories so they would stop hiding in "other".
In production all three are zero fleet-wide and "other" still absorbs every
refusal, because the dominant reason code is the legacy preflight_refused
aggregate recorded when an agent sends no machine reason code at all.
Agents older than the typed refusal contract report every refusal that way,
so the split is starved rather than broken, and folding the two together
made those two states indistinguishable. Schema v9 counts uncoded refusals
separately.
Contract-Neutral: Release-note copy correction for the v6.3.0-rc.3 packet. Restores the 'Read-only observers' Highlights bullet that build_release_assets_test.go pins for the 6.3 packet and that an earlier trim dropped, and lists the alert-delivery repairs under Fixed where the changelog already carried them. Both governed gates verified: render_release_body validates, and scripts/installtests passes. No contract, runtime or version-pin change.
Contract-Neutral: Release-note copy fix for the v6.3.0-rc.3 packet already prepared in 2f6f2396b. Trims Highlights to the three bullets the canonical render_release_body validator permits and keeps each under the 140-character cap. No contract, runtime or version-pin change; deployment-installability.md and the version pins are unchanged from the prep commit.
Contract-Neutral: Routine release-preparation cut for v6.3.0-rc.3. deployment-installability.md IS staged with the substantive active-prerelease and mobile-decision updates for this cut. The residual demand is a verification artifact, and the registry-listed proof scripts/installtests/build_release_assets_test.go plus install_docker_sh_test.go already cover this change and pass against it; they validate the packet dynamically from the repo-root VERSION rather than pinning a literal, so they have no diff to stage. Both were run green immediately before this commit and caught two real errors in the mobile-decision clauses, which are fixed here.
Pulse could tell that its own delivery had stopped and had no way to say so.
Queue health was computed for one API endpoint and rendered on the alerts
surfaces, and that was the whole escalation path: a destination that stops
delivering cannot announce itself through a notification, so an operator who
does not open the alerts page learns nothing.
Telemetry: 78 installs retry every notification to the maxAttempts ceiling and
deliver almost nothing, 155,967 attempts against 221 deliveries in a week. 50
have delivered nothing in 30 days and 28 more delivered normally at some point
and silently stopped. The dominant terminal failure classes are authentication
and configuration, so most are fixable in minutes once someone knows.
Move the queue-health rule into internal/notifications as ClassifyQueueHealth
and DeliveryHealth, next to the queue that produces the counts, and delegate
the API handler to it so the endpoint and the new evaluator cannot drift into
two rules. A queue that cannot be read reports unavailable rather than healthy,
because silence is the failure mode being guarded against.
Monitoring evaluates delivery health on the poll ticker and raises or clears
the notification-delivery system alert, which puts it in the alert list and the
navigation badge. Reading queue health costs a SQLite query and the poll
cadence can be seconds, so the evaluation is throttled to five minutes.
RaiseSystemAlert is idempotent for an unchanged condition, so the timer neither
re-notifies nor accumulates alerts.
The alert is a warning rather than critical. It is real, but a new
critical-by-default alert appearing on upgrade across the affected installs is
a bigger change than this should make on its own.
The new Monitor field is deliberately named to fit inside the existing struct
alignment column. A longer name makes gofmt re-pad the whole block, which
breaks the canonical guardrail tests that pin those field declarations
verbatim.
Proof: delivery_health_test.go pins that in-flight work and retries stay
healthy while retained terminal failures do not, and that an unreadable queue
is never reported healthy. system_alerts_test.go pins the message for each
outcome including singular and plural, that it points at the destinations
surface, and that the throttle honours its interval. internal/monitoring,
internal/notifications and internal/alerts suites all pass.
Contract-Neutral: Staged contract deltas cover every boundary this change actually moves: notifications.md gains delivery_health.go plus the ClassifyQueueHealth rule, and monitoring.md gains system_alerts.go plus the system-alert evaluation extension point. The residual demands are inapplicable. agent-lifecycle is demanded only because monitor.go is one of its canonical files, and its verification artifact is host-agent deletion and re-enrollment proof, which this change does not touch: the diff to monitor.go is one struct field and one call in the poll loop. api-contracts and storage-recovery are demanded only because internal/api/notifications.go falls under a broad internal/api reference, and that file's change is a pure delegation of an internal helper to notifications.ClassifyQueueHealth with a byte-identical HTTP response and no route or payload delta.
Every alert type is scoped to a monitored resource: a node, a disk, a
container, a host. That leaves no way to report a condition whose subject is
Pulse, and the condition that needs it most is broken notification delivery,
where the channel that would carry the warning is the thing that failed.
Telemetry shows what that costs. 78 installs retry every notification to the
maxAttempts ceiling and deliver almost nothing, 155,967 attempts against 221
deliveries in a week; 50 have delivered nothing in 30 days and 28 more
delivered normally at some point and silently stopped. The dominant terminal
failure classes are authentication and configuration, so most are fixable in
minutes by an operator who has no way to learn there is anything to fix. Until
now the only evidence lived on a config tab, because a delivery failure cannot
announce itself through delivery.
Add RaiseSystemAlert and ClearSystemAlert. A system alert carries the stable
pulse-system- identity prefix so repeated raises update one alert instead of
accumulating, sets no ResourceID so surfaces skip resource-linked affordances
rather than offering a link to nothing, and stamps systemAlert metadata for
surfaces that want to tell the two apart. Raising is idempotent for an
unchanged condition and only re-notifies on a change of level or message, so an
evaluator on a timer cannot turn into the notification storm this is meant to
warn about. Routing through the ordinary pipeline is the point: the alert
reaches the alert list and the navigation badge, which is the escalation path
that does not depend on delivery working.
This commit adds the capability and its contract entry. Nothing raises a
notification-delivery alert yet: the evaluator needs a package holding both the
notification queue and the alert manager, and internal/monitoring plus
pkg/server are held by another agent's path claim. The evaluator and its UI
verification follow.
Proof: four cases in alerts_test.go pin the stable identity and absent
ResourceID, the idempotence that keeps a repeated raise from re-notifying, the
level change that does re-notify while still updating a single alert, the clear
path including clearing something absent, and the rejection of a blank type.
Full internal/alerts suite passes under -race.
Adding a PVE/PBS source through the Connect via API path discarded the
name typed in the add dialog: the setup bootstrap only carried type and
host, and auto-registration named the new connection after the node's
self-reported hostname. A user asking for a cluster named enacon got a
connection called pve01 and could only rename it after adoption.
Carry the typed name on the one-time setup token
(SetupTokenRecord.DesiredName) via an optional name field on
/api/setup-script-url, and let the setup-token-authenticated
auto-register completion name a newly created connection from it,
falling back to the hostname-derived default when absent. Dedup and
cluster-member adoption identity stay hostname/candidate based, so the
carried name cannot fork an existing registration.
Contract-Neutral: Customer-facing punctuation cleanup only; no API, lifecycle, policy, or subsystem behavior changed. The owning frontend copy contract and lint enforcement are updated.
The shell fetched `/api/state` on every sign-in for two reasons: to learn
whether the session was authenticated, and to have something to render
before the socket connected. On a real estate that is 96% resources, and
the fetch sat on the critical path ahead of the websocket, which could not
begin connecting until it returned.
Neither reason survives. The session probe is now `/api/state/summary`,
which passes the same auth and monitoring-read scope checks and returns a
few hundred bytes. Navigation resolves from the platform admission facet.
Nothing else in the shell read the payload: `state()` had exactly three
consumers, two of them navigation and one an alert-count badge that the
socket supplies.
`/api/state` itself is untouched. It is the recovery path for a snapshot
too large for a websocket frame, which is precisely what the largest
estates need, so shrinking the endpoint would have removed the escape
hatch for the installs that depend on it most.
The seeded bootstrap state is gone rather than left empty. Keeping it
would have been actively unsafe: `runtimeStateResolved()` meant "a payload
arrived", so a bootstrap that no longer carried resources would still read
as resolved and navigation would classify an empty estate, hiding every
platform tab. Resolution is now websocket-backed, and an empty estate is
answered by the facet admitting nothing, which is a resolved answer rather
than an absent one.
Verified against two backends built from this change: a populated estate
resolves all six tabs, lands on the Proxmox overview and renders every
platform page, and an estate with zero resources shows no platform tabs in
the navigation landmark and lands on the infrastructure settings page.
Both issue one summary probe and zero `/api/state` requests.
Known regression: the alert-count badge is empty for the sub-second window
before the socket delivers alerts, where the bootstrap payload used to
supply it. The alerts store is not socket-synced, so reading the badge
from it would have traded a brief empty count for a lastingly stale one.
Tests that pinned the old mechanism were rewritten to their subjects: the
SSO and proxy-auth cases keep their auth assertions, and the empty-estate
distinction now asserts the facet admitting nothing rather than a
full-state payload arriving.
Contracts: performance-and-scalability records that the bootstrap is
estate-independent and that `/api/state` stays the recovery path rather
than a bootstrap surface; cloud-paid records the tenant probe.
The shell read the admission facet once at sign-in, which left two ways
for navigation to be wrong.
Switching organization was the serious one, and it was a defect in the
change that introduced the facet. The switch clears bootstrap state but
left admission untouched, so between the switch and the arrival of the new
tenant's runtime state the shell would render the outgoing tenant's
platform tabs. Admission is now cleared synchronously on the switch,
before anything can render from it, and refetched for the new tenant.
The second is a reconnect: an estate can gain or lose a platform while the
socket is down, so admission is refetched when the socket comes back. That
was verified by stopping the backend mid-session and restarting it, not by
toggling the browser offline, which leaves an established socket open and
silently proves nothing.
No polling is introduced. Live runtime state stays authoritative once it
arrives and tracks the estate itself, so a steady-state session makes no
further admission requests; that was measured rather than assumed.
I dropped a periodic backstop I had planned for sessions whose websocket is
blocked. The premise was wrong: such a session still receives runtime state
from the REST bootstrap, so navigation is already resolved and the timer
never fires. Verified against a socket-blocked session before removing it,
rather than shipping a session-long timer for a case that does not occur.
Contracts: cloud-paid gains the tenant-switch rule, and
performance-and-scalability records that admission refresh is event-driven
rather than polled.
The shell decided which platform pages exist by classifying every resource
in the runtime state payload, so navigation could not resolve until an
estate-sized payload had arrived. It now reads the platformAdmission facet
from the canonical resource contract, which answers the same question from
a one-resource request: 2KB against 314KB on the estate this was verified
on.
Precedence is deliberate. Live runtime state stays authoritative once it
arrives, because it tracks an estate that can gain a platform after the
facet was read; the facet only resolves navigation before that payload
exists. The two agree by construction, so the handover is not visible. A
server that does not report the facet keeps the previous behaviour rather
than hiding platforms the estate really has, and a partial facet payload is
rejected outright, since a missing flag would read as a hidden platform.
Verified against a backend built from the commit that publishes the facet:
navigation resolves with all six tabs, every platform page renders its own
rows, and the shipped facet equals the client classifier over the same
658-resource estate.
The architecture pin on the authenticated bootstrap gains the new loader.
Its invariant, that branding loads inside that bootstrap, is unchanged.
This does not yet free the state payload: it remains the steady-state
source for navigation, and other shell surfaces still read it.
Contracts: performance-and-scalability gains the rule that admission cost
must not scale with estate size; cloud-paid, ai-runtime and
storage-recovery gain the matching clauses for tenant scoping, hosted
surfaces and TrueNAS visibility.
The app shell decides which primary platform pages exist by classifying
every resource in the legacy full-state payload, which is why it has to
download that payload before it can render navigation. This publishes the
same answer as a `platformAdmission` facet on the canonical resource
aggregations, so admission has one definition instead of two that can
drift.
Counts cannot answer it, which is the whole reason this is a facet rather
than a client-side tally over `bySource`. A TrueNAS or Proxmox host
reports through the agent source and carries the "agent" platform scope,
so a count-based derivation admits the standalone page for an estate that
has no Pulse agent in it at all. Ownership is per-resource evidence, so it
is evaluated per resource here.
Verified against the live client classifier over real estates rather than
by inspection: the facet and the classifier agree on all six pages across
ten estates each for a 652-resource and a 216-resource estate, including
the cases that break a count-based derivation (Proxmox-only, TrueNAS-only,
vSphere-only and provider-owned-agents-only estates all correctly withhold
the standalone page, while a genuine Pulse agent admits it).
The tenant-fallback contract pin gains the new field. Its invariant, that
an empty tenant returns no resources rather than seeding from the raw
snapshot, is unchanged and still pinned; an empty estate admits nothing.
No caller reads the facet yet. Moving the shell onto it is a separate
change, gated by the same parity comparison.
Contracts: unified-resources and api-contracts gain the facet and its
derivation rule; agent-lifecycle and storage-recovery gain the ownership
clause, since an agent-typed host owned by a provider must not admit the
standalone page.
rcourtman