## Why
The web chat window and its backend routes are retired (#3126, #3137),
so the `pulse_assistant` surface now only serves Pulse Mobile's chat.
Its manifest description still called it "the contextual explanation,
approval, and handoff surface for Patrol findings". That text is public
in three places: the overview at the top of `docs/AI.md`, the MCP README
surface block, and the External agents settings panel.
## What changes
- **Manifest description.** It now reads "The chat surface Pulse Mobile
uses for operator questions over the same governed tools, approvals, and
verification; the in-app web chat window is retired." The generated docs
blocks follow it. The surface ID, label, affordances and tools are
unchanged.
- **Browser fallback.** `frontend-modern/src/api/agentCapabilities.ts`
no longer gives an undeclared Assistant surface `interactiveQuestions`.
That matches the backend default of tools only.
- **Tests and contracts.** The Go, Python docs-policy and frontend tests
that pinned the old text and fallback are updated. The settings panel
test now asserts that no "Interactive questions" badge appears. The
ai-runtime and api-contracts contracts carry dated notes.
Checked on a private stack built from this branch, at desktop and phone
sizes:
- **What it shows:** under External agents → Developer details → Patrol
access model, Pulse Assistant has "Native surface" and "Actions" badges
and the new description, and no "Interactive questions" badge appears.
- **Layout and errors:** no horizontal overflow and no console errors.
The receipt is included.
## Not in this PR
- The dead `highlight.js` chunk rule in `vite.config.ts`. It matches
nothing since the package was removed. The commit guard requires a
build-proof file change for any edit there, which isn't worth it for a
no-op.
With the web chat window and its backend routes retired (#3126,
#3137), the pulse_assistant surface only serves Pulse Mobile's chat.
Its manifest description still called it the explanation and handoff
surface for Patrol findings, and that text is public: the docs/AI.md
overview, the MCP README surface block, and the External agents
settings panel all show it. It now names the Pulse Mobile role and says
the web chat is retired.
The browser's fallback affordances for an Assistant surface with none
declared still offered interactive questions. They now match the
backend default of tools only, so no client path advertises a question
tool nobody can answer.
The macOS Intel native job killed the DSM Upstart suite at its single
one-minute alarm after sixteen controls had passed (main 39883120153,
PR 39952586718). Passing runs take 41s on Intel, 23s on ARM and 14s on
Linux, so the budget measured runner speed, not a stuck teardown. The
Legacy suite sits at 35s against the same alarm.
Run each discovered systemd, OpenRC, legacy and Upstart control through
the shared helper under its existing bound, as the NAS and TrueNAS suites
already do. Required controls stay mandatory, and skipped, absent or
failed controls still fail. Installer deadlines are unchanged.
## Why
The in-app Assistant chat window was retired in #3126. Several backend
routes existed only for that window, and Pulse Mobile, the remaining
chat client, never called them. This removes them and the code that only
they reached.
## What changes
- **Routes removed:**
- `GET /api/ai/assistant/surface-tools`
- `POST /api/ai/workflow-prompts/render`
- `POST /api/ai/question/{id}/answer`
- the `/summarize`, `/fork`, `/undo`, `/redo`, `/steer`, `/diff`,
`/revert` and `/unrevert` sub-resources under `/api/ai/sessions/{id}`.
These now return `404` after the existing `ai:chat` scope check.
- **Code removed behind them:**
- the handlers and their `AIService` methods;
- the chat service's fork, undo, redo, steer, summarize, answer-question
and surface-tool methods;
- the session store's fork and turn-undo operations and the persisted
redo stack (`can_redo` leaves session summaries);
- session compaction, the agent loop's steering inbox and
question-answer wait, the `steered` message flag, and the `question` and
`steer_applied` stream events.
- **The question tool is no longer offered.** Interactive chat still
offered `pulse_question`, and Pulse Mobile has no way to answer it, so a
Mobile turn that called it waited until the request ended. Now:
- no profile offers the tool;
- the agent loop answers any call with a refusal telling the model to
ask in its reply text, the same path Patrol runs already took;
- the chat system prompt no longer mentions a clarification tool;
- the agent capabilities manifest's Pulse Assistant surface no longer
declares the `interactiveQuestions` affordance, so the prompt's
affordance text, the fallback governance names, and the generated
`docs/AI.md` and `cmd/pulse-mcp/README.md` surface blocks drop it. The
External agents settings panel stops showing an "Interactive questions"
affordance for Pulse Assistant. The shared `pulse_question` declaration
stays.
- **Docs and contracts:**
- `docs/API.md` lists the remaining routes under "Chat Sessions (Pulse
Mobile)", including the rename route Mobile already used.
- The ai-runtime and api-contracts rules for the removed workflows are
rewritten, and the dependent contracts carry dated notes.
## Unchanged
- `/api/ai/chat` and the session list, create, rename, delete, messages
and abort routes, which are the Pulse Mobile relay route list.
- `/api/ai/status`, which the dev health check uses.
- `/api/ai/workflow-prompts/activity`, which Patrol uses.
- Patrol investigations, provider routing, model-boundary sanitizing and
approvals.
- Old session files: they load as before and drop the redo stack and
`steered` flags on their next write.
## Not in this PR
- The Pulse Assistant surface description in the manifest, which still
calls it a handoff surface for Patrol findings. It now describes Pulse
Mobile chat and can be reworded with the surface contract when Mobile
retires.
- The browser's legacy fallback in
`frontend-modern/src/api/agentCapabilities.ts`. It still gives Pulse
Assistant `interactiveQuestions` when a manifest declares no affordances
at all. A same-version server always declares them, so that path is
unused; it changes with the surface description.
- The dead `highlight.js` chunk rule in `vite.config.ts`.
## Release note
> Removed API routes the retired in-app chat window used: Assistant
surface tools, workflow-prompt rendering, question answers, and the
session summarize, fork, undo, redo, steer, diff, revert and unrevert
routes. Pulse Mobile's chat is unaffected.
The in-app Assistant chat window was retired in #3126. Several backend
routes existed only for it, and Pulse Mobile, the remaining chat client,
never called them: assistant surface-tools, workflow-prompt render,
question answers, and the session summarize, fork, undo, redo, steer,
diff, revert and unrevert sub-resources. This removes them along with
the handlers, service and session-store methods, the persisted redo
stack, session compaction, mid-turn steering and the question-answer
wait behind them.
The question tool needed more than a route removal. Interactive chat
still offered pulse_question, and Pulse Mobile has no way to answer it,
so a Mobile turn that called it waited until the request ended. No
profile offers it now. The agentic loop answers any call with a refusal
that tells the model to ask in its reply text, the path Patrol runs
already took, and the chat prompt drops its clarification-tool advice.
The capabilities manifest's Assistant surface stops declaring
interactive questions, so the generated docs and fallback governance
names follow it.
/api/ai/chat and the session list, create, rename, delete, messages and
abort routes Pulse Mobile uses are unchanged, as is /api/ai/status. Old
session files load as before and drop the redo stack and steered flag
on their next write. docs/API.md lists the remaining chat routes, and
the subsystem contracts record the removal.
The agent installer checked an --action-token-file's permissions with
GNU-only stat -c. BSD stat on macOS rejects it, the mode read came back
empty, and every action token file was refused as group- or
world-accessible. It now uses the existing portable_path_mode helper,
which the collector token path already relies on.
The ingress test runs both token options through the real installer
functions and the compiled token reader; on the macOS runner it is the
proof for this fix.
Contract-Neutral: portability fix restoring the documented --action-token-file option; no contract change
Pulse Assistant was a chat drawer beside the monitoring UI. Operators who
want to talk to their infrastructure already use Claude Code, Codex, or
another MCP client, and Pulse MCP gives those agents the same governed
tools. Keeping a second, weaker chat in the web app cost a 66 kB chunk,
a launcher on every page, hand-off buttons on alerts, findings, approvals
and drawers, and thousands of lines of OpenCode-parity drawer code, for a
feature the maintainer judged ineffective.
This removes the browser chat and everything that only fed it:
- The drawer, its launcher (desktop and phone), the command-palette
Assistant commands, the Escape and blocking-dialog shell plumbing, and
the chat store, client, dev stream fixture and generated SSE types.
- Every "Open in Assistant", "Explain with Assistant", "Discuss" and
"Ask Assistant" hand-off on alerts, incident timelines, Patrol findings,
approvals, run history, the attention workbench and resource drawers,
with their context models and the chat-only helpers they used.
- The Assistant settings page and its session maintenance section. The
page becomes "External agents" at /settings/pulse-intelligence/agents;
the old /assistant path, ?tab=assistant and system-ai-assistant redirect
there with the hash kept.
Alert Investigate now starts a targeted Patrol run (resource, alert ID and
type) and says where the findings land, and it ignores repeat clicks while
the request is in flight. The run route needs an admin with ai:execute, so
security status gains settingsCapabilities.patrolRun, derived from the same
snapshot as the other admin capabilities but following the route for
org-scoped admins, organization-bound tokens and a cookie beside a token,
and Investigate is offered only where it is true. The Patrol
tool transcript block moves to components/patrol. A small aiAvailability
store holds the session assistantEnabled fact that gates Patrol entry
points and Investigate; it now clears and refreshes on organization switch,
refreshes when organization loading settles on another org and after
provider removal, and ignores answers that arrive after a newer one.
The backend chat service stays: Patrol investigations run on it and
Pulse Mobile chat uses it until 31 March 2027, so /api/ai/chat, the
session endpoints and the stream contract are unchanged. A new contract
snapshot pins the steer_applied event that only the deleted TypeScript
projection used to cover.
highlight.js, used only by the chat's code highlighter, leaves the
dependencies, as does the DOMPurify carve-out that kept its fence hints.
Contracts, registry and readiness proofs drop the retired surfaces; the
ai-runtime contract loses about 1,600 lines of drawer rules while keeping
every backend session, stream and provider rule. The production build
ships about 100 kB less gzipped JavaScript.
The PMG help says to compare the same time window in PMG, and its docs
check pinned the old drawer's `stats()?.timeframe` read. The Mail tab is
now headless mail-flow content inside the shared resource drawer and
names the window on its Mail flow card from the same field
(`stats?.timeframe`); the check now accepts either spelling, so the
help still has to describe a window the drawer actually shows.
The earlier drawer changes left a few surfaces that still looked or
behaved differently, plus code nothing used any more:
- Proxmox Mail Gateway had two drawers for one gateway; the row now
opens the shared resource drawer with mail flow as its Mail tab. The
Ceph cluster drawer gains the canonical header, Overview (alerts,
Worth checking, capacity history, cards, closed Details) and Manage,
and its dot follows the row instead of reading grey on missing health.
- The Kubernetes cluster drawer's Deployments tab repeated the Workloads
table with dead-end rows; it is gone, and Namespaces opens the
Workloads tab scoped to the cluster and namespace with one action.
The Docker Swarm drawer tab could not be reached from any page; gone.
- On phones, windowed tables (Kubernetes deployments, pods) and the
Storage tab's grouped pools stopped at 70-90% of the width: a
colspan=100 spacer or colspan=99 group row created phantom columns
that took the width hidden columns left. Those rows now span exactly
the visible header cells.
- The node and Docker host drawers gain the guest drawer's Open action;
the Docker host withholds readings while offline or stale, lists
operator state first in Manage and shows its action history. A
guest's physical disks open the same disk Overview as the Storage tab,
with the guest agent's disk temperature alerts.
- One "No active alerts" line, one timestamp row and wrapping alert
subjects replace per-drawer copies; PBS and PMG alerts match on their
instance id so drawers find them without a metrics target.
- Removes the old disk detail and its presentation code, the Swarm and
Kubernetes Deployments drawers and their presentation helpers, and
drawer state nothing read.
The Kubernetes events, ConfigMap/Secret/RBAC and policy rows now build
record drawers in the page chunk, which measures 21732 bytes gzipped on a
fresh build of this branch. Its recorded baseline (18267) predated
earlier growth on the Kubernetes tabs, so the chunk crossed its limit by
0.71 kB. Lazy-loading the record panels was tried and made the chunk
larger (their shared pieces already live in common chunks), so this
records the measured size and leaves every other chunk's baseline
unchanged.
Two docs checks pinned the old hand-written markup: replication help
matched `>Guest</dt>`-style labels, and Agent Doctor help matched a
"· Privilege:" sentence fragment. The same labels are now shared detail
rows and section labels; the checks match those, so the help still has
to name labels the drawers actually show.
The German and Spanish getting-started guides told paid customers to
activate under Settings -> Plans -> Existing purchases. That heading
left the app in June (27ba85ddc6); the key field now sits under
Settings -> Plans & Billing -> License recovery, so customers following
the guide looked for a section that does not exist. Update both guides,
their shipped mirrors, and the machine-facing string the localized docs
test pins.
Release candidate cut from main at 045e7af651. It carries every change merged since stable v6.5.0.
Contract-Neutral: Release-only v6.6.0-rc.6 version, release notes and distribution metadata; no public install or runtime contract changes.
Change-source: pulse-maintainer
Go 1.26.10 (20 October) distrusts the compromised eMudhra intermediate
CAs in crypto/x509 (CVE-2026-111943). Binaries built with 1.26.9 still
accept chains through them, so every verified outbound TLS connection
Pulse makes (licence, updates, webhooks, AI providers) stays open to
interception by a holder of those keys. govulncheck does not flag a
trust-store change as a called symbol, so the scheduled scan stayed
green while the gap existed.
Advance the governed toolchain floor and every pin together: go.mod,
the toolchain installer and release-script guards, the source-built
container stages (manifest-list digests for golang:1.26.10-alpine and
the Debian dev container image), the root-pair diagnostic, and their
tests.
Vite 6.4.3 bundles a launch-editor with three advisories
(GHSA-587h-7wmj-h5pm, GHSA-r825-w8mm-5pxw, GHSA-698v-4r3x-3vcv) that
failed the daily advisory watch. 6.4.4 bundles the fixed launch-editor
and 6.4.5 adds the dev-server server.fs query check. Both reach only
the local dev server, not the built product. The lockfile changes only
the vite entry, and dependencySecurity.test.ts now pins the floor.
Promote v6.6.0-rc.5 to stable after its 24-hour soak. Only release metadata changes.
Contract-Neutral: Release-only v6.6.0 version, release notes and distribution metadata; no public install or runtime contract changes.
Change-source: pulse-maintainer
Memory grew release after release with nothing comparing it (#1803):
a 256 MB LXC that ran v5 for a year needed 1 GB by v6.3. The existing
memory test only checks growth within one run.
The daily watch builds the latest stable release and main from source,
runs both for 15 minutes against the same small mock estate, and splits
steady-state resident memory into the Go heap, memory outside Go and
file pages. It fails past both 15% and 16 MiB, uploads both heap
profiles with a per-function growth list, and keeps one issue open
while it fails. It runs on a schedule rather than in the pull request
benchmarks job because a 15-minute whole-server run is too slow and
noisy to gate every PR. Mock fixtures live in the heap, so the summary
says to check internal/mock first.
Checked end to end on pulse-dev: v6.5.0 against this branch measured
123.5 and 117.2 MiB, no regression; a synthetic 40% growth exits 1.
Related to #1803
Release candidate cut from main at 1c2f23aef6. It carries every change merged since stable v6.5.0.
Contract-Neutral: Release-only v6.6.0-rc.5 version, release notes and distribution metadata; no public install or runtime contract changes.
Change-source: pulse-maintainer
Keep monitoring readings, trusted connections and recovery guidance
reliable when observations are stale, guest responses are incomplete or
agent recovery is interrupted.
Qualified source: `b6e56c5baa2ad995264faa80877543300bc10273`. All nine
[required source
checks](https://github.com/rcourtman/Pulse/pull/2888/checks), the
[native
matrix](https://github.com/rcourtman/Pulse/actions/runs/39148879516) and
[Core E2E](https://github.com/rcourtman/Pulse/actions/runs/39148879470)
passed. Intel accounted for all 545 installer roots, with 532 passes, 13
explicit platform skips and no failures. Its remaining installer phase
passed in 18m48s, followed by native agent build and execution. The
matrix covers Linux x64/ARM64, macOS Intel/ARM64 and Windows x64
execution plus the FreeBSD cross-build contract.
- Preserve reading provenance, measured zero, stored History and VM/LXC
memory interpretation. Selected-agent staleness cannot renew a retained
observation or invalidate independent provider readings.
- Reject incomplete or foreign guest responses and retain saved
certificate trust, scoped access, private installer input, native NAS
teardown and bounded recovery guidance.
- Keep initial Proxmox login within its caller and total client timeout
while accepted-session renewal remains shared. Preserve the PBS owner's
context outcome when cancellation races a response, prevent a cancelled
fallback and reject late session publication.
- Assert troubleshooting workflows and destinations semantically. Run
every discovered Python safety control and native Go root independently
with existing fixture, operation, root and job deadlines. Script-smoke
and native verification share one control suite, including real
compiled-Go failure, exit-zero, timeout and exact-selector controls.
- Integrate current main while retaining selected-agent freshness, guest
memory interpretation, native GNU prerequisites, API concurrency
controls and both parents' subsystem contracts.
Worker proof passed 113 focused frontend cases, API and guest-transport
race checks, all 59 Python fixture controls, production frontend build
and full frontend lint/types. The exhaustive worker installer inventory
accounted for 551 roots: 535 passed and 16 explicitly skipped for
platform reasons. The eight consolidated runner controls passed through
the script-smoke harness and native Go wrapper. Authentication proof
passed all 340 Proxmox tests, 108 PBS tests and the affected monitoring
cancellation controls under the race detector. The new PBS test first
failed all ten JSON/form cases on unchanged runtime, then passed with
the correction. Status, ownership and normal commit checks passed.
Browser proof uses production components and CSS with synthetic
observations and endpoints. Drawer History passed at 1280×900, 1024×768
and 390×844. Private installation passed at 1280×900 and 390×900 in both
themes. It covers stale/current/unknown readings, provider caveats,
stored History and keyboard inspection, wrapped signing keys, token
copy/dismissal and Last seen advancing without another lookup. Final
screenshots were inspected, the source-bound receipt covers the combined
runtime files, and owned browsers and servers were closed.
Retain the [previous Frontend
assertions](https://github.com/rcourtman/Pulse/actions/runs/38660106123/job/116179237804),
[Intel
timeouts](https://github.com/rcourtman/Pulse/actions/runs/39138980077/job/117709536519),
[script-smoke integration
failure](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117728549560)
and [PBS cancellation
race](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117729442632)
with their diagnosed causes and corrected proof. The final matrix above
qualifies the combined corrections.
The [earlier advisory benchmark
comparison](https://github.com/rcourtman/Pulse/actions/runs/39138980114/job/117709718583)
reported three legacy history-filter regressions. A worker repeat
reproduced one. Longer controls showed wide timing variation even
between byte-identical binaries. The [45a82ea03c
comparison](https://github.com/rcourtman/Pulse/actions/runs/39145479921/job/117728549510)
passed against a newer baseline. The [final
comparison](https://github.com/rcourtman/Pulse/actions/runs/39148879492/job/117738375776)
completed execution but failed on test-only empty/legacy filtering,
2.945 to 3.263ns, +10.82% (p<.001, ten samples). Production empty
filtering improved 16.70%, interleaved filtering rose 2.96%, and history
allocation counts were unchanged. The filter and benchmark sources are
byte-identical across the final comparison. This does not establish a
cause or performance closure. Artifact 12145720151 retains full paired
samples, comparison and metadata. The overall Build and Test workflow is
red from this advisory job while all nine protected required checks
pass. The existing L1 reliability follow-up owns repeatability and
attribution. No benchmark threshold, protected check or release gate
changed.
Main's benchmark/SLO test and documentation changes in PR3053 compose
without conflict. The candidate benchmark artifact records synthetic
merge c21333c676ac004935c9683027490203d8f869ea with tree
fa059ae37b, combining this head and main
52b40fd611. A bounded worker composition
proof also executed all three chart benchmark cases under the race
detector with source hashes unchanged. Its SLO test explicitly skipped
under the race detector, and one-iteration execution supplies no
performance comparison.
This proposal qualifies source delivery. Release inclusion, installed
upgrade/recovery, guest thaw, fresh writes on every backed-up filesystem
and workload liveness remain separate outcomes. Reliability keeps its
existing priority. Recovery/change feature expansion stays deferred.
Related to #1099, #2439, #2619, #2822, #2897, #2757, #2511, #2513,
#1788, #2755 and #2140.
Script smoke still asserted the removed grouped runner while native
verification used a separate suite. Share the canonical execution controls
so both entrypoints reject lost checks, false passes and changed deadlines.
The chart benchmark warmed its response cache and then timed a fresh
payload-retaining recorder. That hid store-backed work behind test buffer
allocation and made the regression signal ambiguous.
Measure cached handler work, fresh-service computation and real loopback
HTTP transfer separately. Wait for store startup maintenance, preserve the
original failed comparison and record the paired diagnostic limits. Make
the adjacent SLO exercise uncached chart generation at its existing budget.
Refs #3052
Compile and discover one exact test binary per native workflow phase, then
execute every discovered root check independently. This removes inventory
growth from another check's ten-minute Go alarm while preserving fixture,
installer and overall CI job deadlines.
Retain exhaustive phase coverage and reject missing completion, failed checks,
unexpected exit zero and timeouts using real compiled Go execution controls.
Register the runner with deployment installability and retain the failed Intel
and advisory benchmark evidence in the existing reliability record.
The NAS suite exhausts its aggregate alarm after fifteen successful
controls on macOS Intel. Run each discovered NAS and updater control
under its existing bound, retaining the native phase's ten-minute alarm
and all installer operation deadlines.
Keep required controls mandatory and reject skipped, absent or failed
execution. Exercise discovery and false-pass rejection with real Python
controls, including a newly added method.
Preserve selected memory freshness and the VM/LXC interpretation.
Retain both native tool contracts and current API concurrency controls.
Qualify this source separately from release and installed recovery.
Contract-Neutral: Integrate already-governed parents without a new contract change. Incoming verification files already match main. Fresh integration tests and source-bound browser proof cover the combined memory and installer surfaces.
Require the existing troubleshooting jobs and their destinations without
making additional help entries break unrelated safety checks. Keep the
opening backup warning scoped to its actual entry point.
Run closed service and middleware models through the containing fixture's
selected Python interpreter without site startup. Retain every safety
scenario and deadline while qualifying the pending maintenance source.
Refs #2888
Contract-Neutral: Test-only navigation assertions and confined model interpreter selection. Production behavior and safety deadlines are unchanged.
Configured report schedules and recent attempts do not show successful
delivery or repeat use. Add durable aggregate reporting counters to the
existing opt-out-controlled heartbeat, covering resources, VM inventory,
backup health and Patrol digests.
Count successful generation, saved files, SMTP acceptance, definite
delivery failures and uncertain acceptance separately. Track distinct
successful saved-schedule periods and active days, including manual
downloads. Deduplicate retries and same-period reruns for repeat-period
counts. The adoption report distinguishes unmeasured senders and partial
observation from measured zero use. Generation across multiple days is
reported separately from successful repeat periods because it can
include retries.
No report contents, recipients, raw schedule or tenant identifiers,
scopes, errors or date ranges enter these new telemetry fields. Mock
activity is suppressed. Pricing, billing, customer entitlements and the
existing outbound opt-out remain unchanged. Update the schema-v20
disclosure and privacy document.
The shared telemetry observer has an exact-path adapter declaration.
Boundary regression fixtures reject sibling reporting implementation and
private licensing imports.
Validation: reporting persistence, API delivery/retry/restart and
successful-write tests, server snapshot tests, the full telemetry
package, 45 adoption-report tests, frontend type-check and build, and
governance audits passed. The three telemetry browser tests and a
separate pixel interaction check passed against the final source at
1280, 390 and 768 pixels, including saved opt-out states and the privacy
table. Source-bound browser receipt included.
Companion receiver: rcourtman/pulse-pro#472. Receiver deployment and a
client release are separate activation steps. This change alone does not
start production collection.
A scheduled retry across UTC days can generate another artifact without
completing a new successful reporting period. Keep multi-day generation as
an activity observation rather than inferred repeat use, while retaining
successful distinct-period counts as their own signal.
Exercise both cross-day generation without success and multiple successful
periods on one day so the adoption summary cannot conflate these metrics.
The usage observer records aggregate outcomes through the existing settings
store without implementing a paid renderer or entitlement decision. Declare
its exact shared-adapter path so the repository audit can distinguish it
from new private reporting implementation.
Exercise the real audit against sibling paths and private licensing imports
to retain those restrictions for the observer.
Record successful response writes, scheduled generation, file saves and SMTP
outcomes in a durable content-free tally. Distinct active days include manual
exports, while saved-schedule periods deduplicate retries and monthly reuse.
Publish bounded scalars through the existing opt-out-controlled heartbeat and
report missing coverage explicitly. Preserve pricing and customer entitlements.
Release candidate cut from main at 0d5b877425. It carries every change merged since stable v6.5.0.
Contract-Neutral: Release-only v6.6.0-rc.4 version, release notes and distribution metadata; no public install or runtime contract changes.
Change-source: pulse-maintainer
Retain backup execution outcomes and collection coverage in each
organization's recovery store so provider inventory changes and restarts
cannot erase report evidence. Keep missing and incomplete history distinct
from an observed period with no attempts.
Reuse existing report scopes, calendar schedules, branding and delivery
recovery for client PDF and CSV reports. Register the paid renderer through
the existing private runtime boundary. Pricing and grants are unchanged.
VM inventory was available only as an on-demand download. It can now be
scheduled weekly or monthly as CSV, emailed or saved on the Pulse
server. Each attempt exports all current VMs in the organization,
including newly discovered VMs, through the existing exporter.
Historical performance-report limits and windows remain separate.
The schedule uses existing delivery configuration checks, persisted
success/failure/uncertain states, bounded retries and deliberate
recovery controls. Pricing, feature gates, retention entitlements and
existing customer grants are unchanged.
Live checks also reproduced two unattended inventory defects: reporting
retained a stopped monitor after configuration reload, and canonical
snapshot reads relied on dashboard reads to publish polled VM inventory.
Reporting now receives reload replacements through an atomic manager
reference. Canonical snapshot reads reuse the existing publication
boundary, including its freshness fast path, provider/mode fences and
nonblocking warm reads.
Validation:
- 105 focused frontend tests; full type check, lint, production build
and bundle budgets.
- Race checks for reporting, inventory, reload, publication and
organization isolation; clean private runtime build.
- Current runtime browser checks on `/settings/support/reporting` and
`/alerts/notifications` at 1440×1000, 820×844 and 390×844, with actual
pixel review. The first inventory read exports 61 normally polled VMs;
real MIME CSV matches the download; a newly discovered 62nd VM enters
the next export.
- Local SMTP acceptance, 451 retry across a real restart, permanent
rejection, uncertain acknowledgement and deliberate recovery; disk
delivery, edit/reload, disable, details and delete.
- Staged status, registry, contract, completion and exact-content
browser guards pass. Regression proof is registered against the specific
runtime paths it covers.
Reuse the canonical inventory exporter and report delivery workflow so
operators can email or save a current organization-wide inventory weekly
or monthly. Keep delivery checks, bounded retries and recovery states
shared, with existing pricing and customer entitlements unchanged.
Refresh the shared inventory for unattended reads and rebind reporting
after monitor reload so schedules use the current infrastructure state.
The canonical completion guard checks each commit on its own; the
delivery commit relies on the following commit for its contracts and
proof files, so record that exact pair.
The canonical completion guard checks each commit on its own; the
classifier commit relies on the following commit for its ai-runtime
contract, so record that exact pair.
ensureAdminSession, the session check behind RequireAdmin and the
settings scope helpers, admitted anyone who could manage the
organization named in the request. That was meant for the org-bound
settings pages, but it applied to every admin route. On a multi-tenant
install an organization owner could therefore rotate the instance
administrator's API token and receive its secret, widen any token's
scopes, add or replace SSO providers, change system settings, relay
and log level, apply or roll back updates, restart the service, and
read instance logs and diagnostics.
ensureAdminSession now admits only instance administrators: the
configured admin or an instance RBAC admin grant, whichever org the
request names. Routes whose handlers act only on the request
organization opt in explicitly through RequireOrgAdmin,
ensureOrgAdminSession or ensureOrgSettingsWriteScope, which also admit
that organization's owners and admins. Those routes cover agents,
nodes, connections, availability, TrueNAS and VMware, agent profiles,
install commands, deploy status, configuration transfer and the
org-bound relay mobile token. Anything that does not opt in fails
closed.
Reporting stays instance administration, and the security status
snapshot stops offering it to org managers, because report generation
still reads the instance's default metrics store. Notifications,
LAN discovery and the action-capability fallback also stay instance
administration. The two credential routes no longer need their own
check, so ensureInstanceAdminSession is folded into this rule.
The canonical completion guard checks each commit on its own. The
admin split commit relies on the following commit for its dependent
agent-lifecycle and security-privacy contract updates and the route
inventory proof, so record that exact pair.
The route inventory treated RequireOrgAdmin routes as unauthenticated
"bare" routes, and the agent diagnostics contract and the API
reference anchors still named RequireAdmin for routes that now opt
into organization authority. The security-privacy contract still said
org managers receive reportingRead through ensureAdminSession; it now
names the org-bound routes' check and records that instance
administration refuses organization owners. The agent-lifecycle
contract notes that agent routes declare that authority explicitly.
The non-required **Unified Agent Native Verification** workflow has
failed on `main` (and most PRs) in both macOS jobs since 12 Oct (first
failing main run 38285211868 on c3524d373a; last green 38273335159 on
552e838749). The single cause is
`TestInstallSHUpstartTeardownRequiresObservedQuiescence`: 13 Python
subtests in `agent_upstart_teardown.py` fail with `ln: illegal option --
T`.
`scripts/install.sh` uses `ln -T -- <pending> <override>` and `ln -T --
<retired> <conf>` in the DSM Upstart teardown (added in the DSM
retirement series, a7271f5840 / b7b0eb0b01). That path only runs on
Synology DSM (Linux), so the installer is correct. The harness runs
those functions with the host `ln`, which on macOS runners is BSD `ln`
without `-T`.
The workflow already supplies GNU `stat/chmod/chown/mv/tar` (plus Bash
5) for Linux-only installer fixtures on macOS. This adds `ln` to that
set and extends `TestNativeInstallerFixtureToolsAreRealAndStepLocal` to
require it (missing/non-GNU scenarios, PATH binding, and real `ln -T`
no-replacement checks against file and directory targets).
Verification:
- Reproduced locally on macOS: the harness fails 13/15 with BSD `ln` and
passes 15/15 when `ln -T` is available.
- `TestNativeInstallerFixtureToolsAreRealAndStepLocal`: fails against
the old workflow (real tools, missing ln, non-GNU ln) and passes with
this change (Linux).
- The macOS jobs themselves are only exercised by this PR's CI run.
Commit a5baa7563b stopped Patrol reading cache-inclusive Proxmox memory
as guest pressure (issue #2762, a QEMU VM). It also put
`cluster-resources` in that cache-inclusive group for every guest type.
An LXC without a linked Pulse agent only ever gets `cluster-resources`,
so Patrol dropped memory pressure for all agentless containers:
- a container at 97% of its limit got no triage flag, forecast or
anomaly
- the seed said "guest pressure unknown; may include reclaimable cache"
- older memory alerts went to a billed model review on every run, and
any resolve verdict was thrown away
**That premise is wrong for LXC.** On both PVE 9 (`master`) and PVE 8
(`stable-bookworm` / `stable-8`), Proxmox subtracts page cache before
reporting a container's memory:
- `pve-common` `src/PVE/CGroup.pm` `get_memory_stat`: `$res->{mem} =
$mem - $stat->{file};` on cgroup v2 (`memory.current`), and `$res->{mem}
= $mem - $stat->{total_cache};` on v1 (`memory.usage_in_bytes`).
- `pve-container` `src/PVE/LXC.pm` `vmstatus` stores that value with
`$d->{mem} = int($mem->{mem});`. `PVE::LXC::CGroup` inherits
`get_memory_stat` from `PVE::CGroup` without overriding it.
- `pve-manager` `pvestatd` `update_lxc_status` writes `$d->{mem}` into
RRD field 8. `/cluster/resources` (`PVE/API2/Cluster.pm` →
`PVE::API2Tools::extract_vm_stats`) returns `$entry->{mem} = ($d->[8] ||
0) + 0;`.
A QEMU guest's listing `mem` is the host's view of guest RAM, which does
include the guest's cache, so it stays pressure-unknown.
**Change**
- `unifiedresources.QualifyGuestMemoryForKind` now takes the guest kind:
other platform, Proxmox VM or Proxmox container.
- A current, dated `cluster-resources` reading counts as known pressure
for a Proxmox container and stays cache-inclusive and unknown for QEMU.
- `status-mem`, `status-freemem` and `derived-total-minus-used` are
unchanged for every guest type.
- `GuestMemoryEvidenceForResource` takes the kind from the resource
type, so the Patrol inventory, the observer runtime and AI query
projections agree.
- `QualifyGuestMemory(bool)` keeps the conservative QEMU reading for
callers that don't know the guest type.
- The wrong comments in `calculateLXCMemory` and the LXC monitor tests
are corrected.
- Everything else in a5baa7563b still applies: retained, undated,
unknown and unavailable readings still cannot establish pressure or
recovery.
**Tests**
- `TestPatrolAgentlessLXCListingMemoryIsPressure` (new) builds an LXC
and a QEMU guest at 97%, both on `cluster-resources`, through legacy and
canonical read state. The LXC gets a memory triage flag and alert review
reads its value; the QEMU guest gets neither.
- `TestGuestMemoryClusterResourcesIsTypeAware` (new) covers both guest
kinds, retained readings, the other cache-inclusive sources and the
resource-type mapping.
- The `cluster-cache` fixture in `TestPatrolGuestMemoryEvidence` now
expects pressure for the container variant. It previously encoded the
wrong behaviour.
- Against origin/main's production files (`go test -overlay`), the new
and updated Patrol tests fail; with this change they pass.
Refs #2762
Supersedes #3011, which failed the documentation recipe test that pinned
the old `QualifyGuestMemory(` call in Patrol; this commit also updates
that pin to `QualifyGuestMemoryForKind(`.
`scripts/installtests/agent_upstart_teardown.py` created its fixture job
directory with a bare `mkdir`. Under a caller umask of 002 (the default
on Ubuntu hosts with user-private groups, pulse-dev included) that
directory came out 775. The installer's `trusted_lifecycle_regular_file`
check then correctly refused the fixture config, and running the harness
directly failed 23 subtests. The Go driver
`TestInstallSHUpstartTeardownRequiresObservedQuiescence` still passed,
because the installtests package already pins umask 022 for its children
(`umask_unix_test.go`). So the bug only shows up when the harness runs
on its own.
The fix gives the job directory an explicit 0755 mode. The sibling
TrueNAS and uninstall-state harnesses already create their trust-checked
directories with explicit modes. The production functions under test
still run with the caller's umask. They `chmod 644` what they write, so
nothing in production depends on umask.
Checked on pulse-dev (python3 3.12, go 1.26.5):
| run | umask 0002 | umask 0022 | umask 0077 |
| --- | --- | --- | --- |
| harness before | FAILED (failures=23, errors=1) | OK (15) | OK (15) |
| harness after | OK (15) | OK (15) | OK (15) |
| Go driver after | ok | ok | — |
The other Python harnesses in `scripts/installtests/` (legacy, OpenRC,
systemd, TrueNAS, uninstall-state) already pass under umask 0002, so
they are unchanged.
Test-only change.
An existing collector can retain its legacy TLS bypass after the host
trusts the Pulse certificate. Updating it with action-runner enrollment
then incorrectly rejects normal system-root trust and enters recovery
before creating the runner service.
Keep collector/download settings separate from verified runner TLS.
HTTPS runner configuration continues to use system roots, a configured
CA or an exact certificate pin and never inherits the collector bypass.
Activation, cancellation, revocation and service recovery retain their
existing boundaries.
Validation: the new executable regression fails against the previous
installer and passes with the fix. It covers recovered insecure state,
private configuration persistence and the real runner credential
lifecycle binary over TLS. System roots, custom CA and exact pin
succeed. Untrusted certificates and incorrect pins receive no
credential-bearing request. The installer Action Runner suite, runner
binary tests and hostagent Action Runner suite pass.
Agent Doctor enrollment also passes in the current built app at 1440,
820 and 390 pixels wide, covering details disclosure, issue
failure/pending/success, private command copying, one-time reveal,
clearing and reload. Browser endpoints use synthetic fixtures. No native
installed-host or published-release acceptance is claimed.
The additional native workflow passes on Linux x64 and ARM64. Its macOS
Intel and ARM64 jobs fail the unchanged Upstart retirement contract
because the host `ln` rejects `-T`. The same failure occurs in [the
earlier main
run](https://github.com/rcourtman/Pulse/actions/runs/38899584939). This
change does not touch that fixture or the exercised retirement
functions.
## Summary
Unattended updates on small Proxmox LXCs were being OOM-killed. The
server installer staged the release archive (~280 MB for v6.5.0) and its
extracted tree (~750 MB) under `/tmp`. On Debian 13 and many LXC
templates `/tmp` is tmpfs, and tmpfs pages count against the container's
memory cgroup. The reporter's host log shows the `/lxc/118` cgroup at
its 1 GiB limit with ~820 MB of shmem, the kernel OOM-killing the
container's `systemd` (PID 1), and Proxmox then stopping the container.
`install.sh` now downloads and extracts into owner-only hidden
directories under the install directory:
- The install directory is always writable from the hardened
`pulse-update.service` sandbox.
- The staged binary already lives there.
- The disk headroom preflight now measures the filesystem the update
actually stages on.
The Go updater already prefers its data directory over `/tmp` for the
same reason.
Two follow-ups that come with staging on disk:
- **Stale staging sweep.** An interrupted run's staging (~1 GB) now
survives reboots. Staging setup removes stale `.pulse-extract-XXXXXX`
and `.pulse-<release>-linux-<arch>-XXXXXX` directories under the install
directory. It only removes real directories with those exact names,
never symlinks or files, and only when their ctime is more than 6 hours
old. `install.sh` has no update lock, so age is what proves no live run
still owns them.
- **Ownership.** Every recursive `chown -R pulse:pulse "$INSTALL_DIR"`
(archive install, source build, `setup_directories`) now goes through
`chown_install_dir_for_pulse`, which skips staging. The payload that
root still reads is never handed to the service account.
The `Failed to print table: Broken pipe` line in the report came from
`systemctl list-unit-files | grep -q` in v6.5.0's service discovery.
`main` already reads that inventory in full, and it was not the cause of
the failure.
## Tests
- New Go tests `TestRootInstallScriptSweepsOnlyStaleServerStaging`
(fresh staging kept, stale staging removed; look-alike names, files,
symlinks and their targets kept) and
`TestRootInstallScriptPulseOwnershipExcludesServerStaging` (staging
skipped, no raw recursive install-dir chown left).
- New `test_archive_extraction_stages_on_the_install_filesystem` (server
installer stop harness) and
`test_download_stages_archive_on_the_install_filesystem` (temporary
inputs harness) fail against the current installer and pass with this
change.
- `scripts/tests/test_server_installer_stop.py`,
`test_server_installer_temporary_inputs.py`,
`test_server_service_discovery.py`, and the `TestRootInstall*` tests in
`scripts/installtests` pass (run on Linux).
Refs #2785
agent_upstart_teardown.py created its fixture job directory with a bare
mkdir, so under a caller umask of 002 (the default on pulse-dev and other
Ubuntu user-private-group hosts) the directory came out 775. The
installer's trusted_lifecycle_regular_file check correctly refuses a
config whose parent is group-writable, and 23 subtests failed when the
harness ran directly. The Go driver only passed because the
installtests package pins umask 022 for its children.
Give the job directory an explicit 0755 mode, matching the sibling
TrueNAS and uninstall-state harnesses, which already create their
trust-checked directories with explicit modes. The production code
under test still runs with the caller's umask; it sets 644 on the files
it writes, so the harness now passes under 002, 022 and 077.
Patrol treated every guest's cluster-resources memory as cache-inclusive,
so an LXC without a linked Pulse agent, which only ever gets that source,
lost memory pressure entirely: no triage flag, forecast or anomaly at 97%
of its limit, a "may include reclaimable cache" caveat in the seed, and
older memory alerts sent to model review every run with any resolve
verdict discarded.
The premise holds for QEMU but not for LXC. Proxmox's container mem is
cgroup usage with page cache already subtracted: PVE::CGroup's
get_memory_stat returns memory.current minus memory.stat file (cgroup v2)
or usage_in_bytes minus total_cache (v1), PVE::LXC::vmstatus reports it
as mem, and pvestatd feeds that value to the RRD that /cluster/resources
reads. A QEMU guest's listing mem is the host's view of guest RAM and
does count the guest's cache.
Qualification now takes the guest kind. A current, dated cluster-resources
reading establishes pressure for a Proxmox container and stays unknown for
QEMU; status-mem, status-freemem and derived-total-minus-used stay
cache-inclusive for both, and retained readings still cannot prove
pressure. The typed read state derives the kind from the resource type,
so Patrol, the observer and AI queries agree. The calculateLXCMemory
comment that called the listing value cache-inclusive is corrected.
Contract-Neutral: monitoring change is a corrected comment in calculateLXCMemory and its tests; LXC memory collection and sources are unchanged
The DSM Upstart removal path links its manual override and restores a
retired job config with GNU `ln -T --`, a hard link that never replaces
or descends into an existing target. Its Python harness executes those
production functions with the host's ln. On the macOS runners that is
BSD ln, which rejects -T ("illegal option -- T"), so 13 subtests of
TestInstallSHUpstartTeardownRequiresObservedQuiescence have failed on
both macOS jobs since the DSM retirement series landed on 12 Oct.
The production code is right: that path only runs on Synology DSM
(Linux), never on a macOS agent host. The gap is the macOS fixture
toolset, which already supplies GNU stat/chmod/chown/mv/tar for these
Linux-only installer contracts but not ln. Add ln to that step and make
the fixture-tool contract test bind it and exercise real `ln -T`
no-replacement semantics (file and directory targets).
Contract-Neutral: CI fixture tooling only; macOS runners gain the GNU ln the existing DSM Upstart harness already assumes, no installer or lifecycle contract change
The server installer downloaded the release archive and extracted it
under /tmp. Debian 13 and many LXC templates mount /tmp as tmpfs, whose
pages are charged to the container's memory cgroup. A v6.5.0 archive is
~280 MB and extracts to ~750 MB, so an unattended update on a 1 GiB LXC
filled the cgroup with ~820 MB of shmem; the kernel OOM-killed the
container's systemd and Proxmox stopped the container mid-update.
Download and extract into owner-only hidden directories under the
install directory instead. It is always writable from the hardened
pulse-update.service sandbox, it is where the staged binary already
lives, and the disk headroom preflight now measures the filesystem the
update actually stages on. The Go updater already prefers its data
directory over /tmp for the same reason.
Unlike /tmp, the install filesystem keeps an interrupted run's staging
across reboots. Staging setup now removes stale staging directories that
match only the exact owner-only names, never symlinks or files, and only
once their ctime is over six hours old: install.sh has no update lock,
so age is what proves no live run still owns them. Every recursive
pulse chown of the install directory now goes through one helper that
skips staging, so the payload root still reads is never handed to the
service account and the ~750 MB tree is not rewritten for nothing.
The "Failed to print table: Broken pipe" line in the report was
systemctl list-unit-files | grep -q in v6.5.0's service discovery; main
already reads that inventory in full, and it was not the failure.
Contract-Neutral: Defect fix inside the existing private server-installer inputs and update-space boundaries; staging moves from tmpfs-prone /tmp to the install filesystem with no public contract change
Legacy updates recover the collector's insecure setting even after the host
trusts the server certificate. Do not reject verified runner system trust
because of that unrelated setting. Keep HTTPS runner verification enabled
and preserve custom CA and exact fingerprint trust.
Exercise recovered configuration and the real credential lifecycle client
against TLS, including untrusted certificates and mismatched pins that must
receive no credential-bearing request.
rcourtman