v6.1.0-rc.4 shipped the attention workbench gated on monitoring:read
only, so mobile relay tokens (which carry just relay:mobile-access) got
403 on alert sync, finding detail, and acknowledge the moment a server
upgraded to rc.4. The attention routes supersede the legacy patrol
findings routes, which accepted the mobile capability, so they join the
governed relay mobile runtime route inventory: reads accept
relay:mobile-access alongside monitoring:read (legacy ai:execute),
mutations alongside monitoring:write.
The 1-run-per-hour Community cadence gate on manual Patrol runs keyed off
lastFullPatrol, which is stamped on every completed run including errored
ones. Debugging a broken provider therefore cost an hour per attempt,
raised in discussion #1571. The gate now keys off the most recent
successful full run from history, matching the success-aware skip logic
the startup path already uses.
Contract-Neutral: Behavioral fix: Community manual-Patrol cadence gate now ignores failed runs; no API payload or endpoint change (#1571)
The advisor endpoint capped the whole four-turn evaluation at a fixed two
minutes, ignoring the operator's provider request timeout. Local runtimes
like LM Studio that legitimately spend minutes on prompt processing per
turn were cut off mid-run, reported in discussion #1571 on rc.4. The
handler budget now scales to four streaming turns at the configured
timeout plus slack, floored at the old two minutes.
Contract-Neutral: Behavioral fix: Patrol readiness advisor HTTP budget now scales with configured provider timeout; no API payload or endpoint change (#1571)
The connections aggregator, alerts and grouping helpers plus the monitoring
group-key, availability-key and read-state mapper helpers were all at zero
coverage. Each of these is a pure value-in to value-out function on a path
that decides how a connection is grouped, whether an alert snapshot is
emitted, and how host thermal state crosses the read-state boundary, so a
silent regression there is invisible until it reaches a user.
Adds branch coverage for buildPMGConnection, connectionTypeForAlerts,
snapshotConnectionsForAlerts, uniqueMatch, connectedInfrastructureGroupKey,
availabilityConnectionKey, ConnectionHealthKey, pingArgs,
hostThermalStateFromReadStateView, latestMetricPoint and MemorySourceTrust,
including nil and whitespace arms, candidate precedence order, and clone
independence in both directions for the mapper. Every named target moves
from zero to full statement coverage apart from the pingArgs arms that are
unreachable on this platform.
Test-only change.
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT=test-only branch coverage, no contract surface touched
One vCenter connection spans many ESXi hosts, so the Infrastructure source
row now lists them the way Proxmox cluster rows list their nodes: an
expandable member list with per-host state, aliases, and last-seen. Members
are API-side composition only — no primary marker, no agent connection, and
the member subtitle reads 'vSphere host' instead of cluster-node wording.
TrueNAS keeps no member composition because that connection monitors exactly
one machine.
Mock mode now feeds its vCenter and TrueNAS fixtures into the connections
aggregator (only when no real instances are configured), so the mock ledger
shows the same platform source rows a real deployment would instead of
omitting vSphere and TrueNAS entirely.
vSphere ESXi hosts and TrueNAS boxes rendered as standalone rows under
'Pulse Agent hosts' in Infrastructure settings even though no Pulse Agent
runs on them. The rows were unmanageable duplicates of their owning platform
connection: they carry no credentials, pause, or remove semantics, and they
can never attach to their vSphere/TrueNAS system because attachment requires
a shared host while the machine hostname differs from the vCenter address.
They also inflated connected-system counts.
buildConnections now skips hosts with a non-empty IntegrationSource, so the
ledger and grouped systems represent those machines solely through their
owning platform connection. Per-machine visibility is unchanged on the
vSphere/TrueNAS platform pages and Machines, which read the unified fabric
directly. Connection.integrationSource stays declared as defense-in-depth
for agent-only client workflows. No alert impact: agent-type rows were
already dropped from alert snapshots.
The connections ledger derives agent rows from the unified fabric, which
includes machines whose telemetry comes from platform integrations (vSphere
ESXi hosts, TrueNAS). Agent Doctor rendered every one as a permanent
'Unknown / no structured reason' row, while agents the ledger does not carry
(Docker-only, Kubernetes-only) were silently dropped from the fleet view.
- Expose HostView.IntegrationSource() (source-set based: only SourceAgent
ingest counts, since integration providers fabricate an Agent payload) and
plumb it through models.Host to the connections ledger as the optional
integrationSource field.
- Agent Doctor skips integration-backed connections and appends
diagnostics-only agents, honoring scope, so the doctor covers exactly the
real Pulse Agent fleet.
- Update readiness agent checks no longer count integration-backed machines
as registered agents.
- Humanize doctor copy: plain-language stale message with '10m 2s'-style
durations, offline wording without enum leakage, no 'Supported target:
Unknown' cell when no target is published, host-local command banner only
when a command is actually offered, and a compact non-zero summary strip.
Contracts updated for unified-resources, monitoring, api-contracts,
agent-lifecycle, and dependent storage-recovery; verification via
views_test.go, monitor_host_agents_test.go, state_host_test.go,
contract_test.go, and the frontend connections API test.
Contract-Neutral: The first-session agent install handoff keeps the existing API and setup contracts while binding runtime handlers to the Router-owned canonical config through startup and monitor reloads.
Contract-Neutral: RC qualification fixes preserve existing public API, tenant, monitoring, and organization contracts while correcting canonical runtime ownership and test fixtures.
GatePatrolAutonomyUpdate validates and consumes the activation
acknowledgement, then re-marshals the settings for the downstream
handler - but the marshaled body still carried acknowledgement_id.
The Pro autonomy handler decodes the body with DisallowUnknownFields
and has no such field, so every Autopilot (full mode) activation on a
Pro build failed with 400 Invalid request body. Lower autonomy levels
forward the original body untouched, which is why only full mode broke.
Clear the consumed acknowledgement before normalizing and hand it to
the downstream handler via request context only. The new regression
test strict-decodes the forwarded body with the enterprise handler's
exact field set.
Contract-Neutral: Autopilot activation gate normalization fix; endpoint contract unchanged
A routine Docker container update took five clicks across two in-row
confirmations and a two-phase Approve then Run dialog, and the review
screen said "Rollback declared: No" while its own intended-change text
promised automatic backup and rollback (#1564 feedback).
- Capabilities can declare rollback support; the planner carries it
into plan.rollbackAvailable instead of hard-coding false, so the
container update review now truthfully says rollback is available.
- Container update is classed AutoAuthorizeLowRisk (it ships backup and
rollback; restart was already low_risk). The action projection
exposes the read-time capability class plus resolved blast-radius
names, none of which enter plan identity.
- The review dialog collapses pending approval for low-risk
capabilities into one Approve and run control that records the
approval and dispatches execution from a single confirmation; other
capabilities keep the two-phase flow. Blast radius renders resource
names with IDs demoted.
- The in-row Update button plans on first click; the dialog is the
confirmation surface, so the Confirm hop and its cancel affordance
are gone.
Routine update ceremony is now Update, Approve and run, Close, with
the durable action record unchanged.
Field report (Johannes Strasser, 6.1.0-rc.3 mobile chat). The assistant
could find an LXC container but told him it had no tools to restart it.
Relay-mobile tokens carry only the relay access scope, which does not
satisfy the execute-authority check, so mobile chat sessions were offered
read tools only. The gap was one-sided: mobile could already approve and
execute planned actions, it just could not propose them.
Relay-mobile callers now hold execute authority, and the chat entrypoint
pins their sessions to approval-required mode, so every mobile-originated
action lands on the approval queue regardless of the configured control
level. Web sessions are unchanged.
Tool-heavy chat histories exceed the relay proxy's 47KB response cap, so
the mobile app could never load big conversations over Remote Access.
GET /api/ai/sessions/{id}/messages now accepts compact=1 (strips tool
outputs, results and thought signatures while keeping tool_use_id
linkage) and max_bytes=N (returns the newest messages that fit the
budget; a single oversized message is truncated with a notice). The
mobile client sends compact=1&max_bytes=40960 and falls back to a
halving limit ladder when older servers answer 413.
The Patrol finding-notification work (23695681a, e0e454abf) added
patrol_finding_notifications_enabled and
patrol_finding_notify_min_severity to the AI settings payload but did
not update the six JSON snapshot pins, leaving Build and Test red on
every push since. The fields are intended API surface; the snapshots
now include them.
Contract-Neutral: snapshot refresh for the intended Patrol finding-notification fields added in 23695681a/e0e454abf; no behavior change
The desired side of the /api/connections command-policy comparison is
contractually the effective config served to the agent after token scope
and binding checks, but a host whose recorded TokenID no longer resolved
to a live API token skipped sanitization entirely and kept the raw
profile desire. A stale binding, typical after a token was revoked and
the agent reinstalled, then presented as Command policy mismatch even
though the served runtime config could never enable commands and the
operator had nothing to fix (seen on issue #1564). When API tokens
exist and the host's token cannot be resolved, desired command policy
now fails closed to disabled.
The contract delta also records the install-token mint semantics that
landed with the Add Pulse Agent scope fix, closing the deferred
agent-lifecycle delta noted on that commit's contract-neutral trailer.
Contract-Neutral: fail-closed bugfix bringing the connections command-policy view into conformance with the existing agent-lifecycle contract clause; the staged agent-lifecycle delta records the semantics, api-contracts/storage-recovery have no real delta
Completes #1369. The delivery path landed in 23695681a with config-only
gating; this surfaces the two fields so operators can turn finding
notifications off or restrict them to critical findings without editing
config by hand.
The AI settings GET response and update request carry
patrol_finding_notifications_enabled and
patrol_finding_notify_min_severity, with the same warning-or-critical
validation the alert-trigger severity field uses. The Patrol settings
page gains a Notifications card between Triggers and Model readiness,
mirroring the alert-trigger toggle-plus-severity pattern, with the
severity select shown only while notifications are enabled.
Exercised end to end against a scratch backend in mock mode. A severity
change and an explicit opt-out both survive save, reload, and the GET
round trip, and the opt-out hides the severity select immediately.
Patrol findings previously reached operators only through the Relay
mobile push path. Anyone relying on the email, webhook, or Apprise
destinations they already configured for alerts got no proactive signal
when Patrol detected a problem and had to open /patrol to learn about
it. This is the delivery half of #1369.
Each newly stored warning or critical finding now also flows through a
FindingNotifyCallback wired in the router, which projects the finding
into the alert shape the notification manager delivers. The callback
fires only on the genuinely-new path in recordFindingWithInvestigation,
so a finding notifies at most once per lifetime regardless of how many
later runs re-detect it, and SendAlert's own per-ID cooldown backstops
that. Demo mode never notifies.
Gating lives in AIConfig via patrol_finding_notifications_enabled
(default on, matching the long-standing default for mobile push) and
patrol_finding_notify_min_severity (warning or critical, default
warning). The enabled flag persists without omitempty so an explicit
opt-out survives reload while pre-existing configs inherit the default.
The settings surface for these fields follows in a separate commit once
the AI settings handler is free.
The Add Pulse Agent flow composed a fixed scope list in the frontend,
so ticking Enable Pulse command execution added --enable-commands to
the install command while the token it shipped with never carried
agent:exec, and the command channel rejected every registration
(issues #1586, #1564, confirmed by a reporter's server log). Even with
the scope, the generic token had no binding metadata, so the first-use
binding gate would have refused it next.
The install token is now minted through POST /api/agent-install-command
with type 'host'. The server decides the scopes from enableCommands at
mint time, stamps the install_type/issued_via metadata that makes the
token eligible for first-use command-channel binding
(canBindProxmoxAgentInstallExecToken renamed canBindAgentInstallExecToken
and extended to the host install type), and returns the sanitized token
record. The frontend regenerates the token when the checkbox toggles,
since scopes cannot be upgraded on an existing token, and revokes the
superseded token so toggling does not accumulate orphans.
Contract-Neutral: install-token mint bugfix (#1586, #1564): checkbox-promised exec scope now real; agent-lifecycle contract delta deferred because the contract docs carry another agent's uncommitted WIP on the shared tree
Two diagnosability gaps found by exercising the docker-update vertical
live with a real agent, each of which turns 'approved action cannot
execute' into a dead end with no visible cause:
- The durable operation-receipt store failing to open (e.g. unwritable
state dir) silently registered the agent with receipt version 0; the
only symptom was a server-side claim that the agent was too old. The
agent now logs the real cause and state dir loudly at startup.
- The server readiness copy asserted 'older agent version' for any
receipt-version mismatch. It now names both causes (old version or
unavailable state directory) and points at the agent logs.
(The related docker collect-cycle watchdog landed separately in
a0f75b1bb with a hung-daemon reproducer.)
Contract-Neutral: agent-side diagnosability: receipt-store failure warning + honest readiness copy; no public contract delta
The 30d window said 18 of 26 approved action attempts failed but
carried zero cause information. Every approved attempt that is not a
verified success now lands in exactly one content-free bucket:
- pre_dispatch: terminally refused before dispatch (plan drift, expiry,
emergency stop, policy authorization)
- execution: dispatched execution failed or ended inconclusive
- unverified: execution succeeded but outcome verification was not
confirmed (reads as success in the UI, counted as failure here)
- stuck_executing: still executing over an hour after dispatch
plus one sanitized machine reason code for the most recent failure.
Successes, failure buckets, and recent in-flight attempts partition the
attempt count, so the next window explains its own gap.
RefuseActionExecution now persists the specific refusal code
(plan_drift, action_plan_expired, ...) as the canonical execution
reason code instead of the generic pre_dispatch_refused, so audit truth
and telemetry distinguish refusal causes without message parsing.
Privacy disclosures updated in both PRIVACY.md copies; the adoption
report script surfaces the new counters.
Contract-Neutral: additive content-free telemetry counters + specific pre-dispatch refusal reason codes; privacy disclosures updated in-commit per security-privacy extension point
Production telemetry (30d) showed 18 of 26 approved governed-action
attempts never reaching verified success. The dominant stranding path:
HandleExecuteAction dispatched under the HTTP request context, so a
reverse proxy or browser dropping the connection mid-operation (docker
image pulls run minutes; proxies cut at 60-100s) cancelled the committed
dispatch and left the action executing forever — recovery only ran at
restart or agent re-registration, which never fires while the agent
stays healthily connected.
- Execute now dispatches under context.WithoutCancel: a committed
dispatch runs to its terminal state regardless of the caller's
connection, bounded by the per-operation transport timeout.
- A standing 2-minute recovery loop re-drives durable-dispatch
reconciliation so any abandoned executing action (timeout, send
failure, crash) converges from the agent's durable receipt instead
of waiting for a reconnect that may never come.
Verified live: docker-update executed with the client aborting at 3s;
the agent completed pull+recreate+verify and the action reached
completed/verified with the replacement container running.
Contract-Neutral: behavioral fix: detach approved-action dispatch from client ctx + periodic recovery loop; no public contract delta
Telemetry shows thousands of installs configure an AI provider but almost
none ever use the interactive Assistant. A live first-session exercise
(fresh install, Ollama qwen3:8b quickstart) found why: after enabling,
nothing changes on screen — the launcher and handoff buttons are gated on
sessionCapabilities.assistantEnabled, which was only read at page load;
the empty transcript was blank; and the blessed Ollama+qwen3:8b path
reported Patrol degraded while telling the user to pull the model they
had just selected.
- Setup-modal success now opens the Assistant drawer, and the AI settings
save paths refresh the assistantEnabled capability in place
(aiChatStore.refreshEnabledFromServer) so entry points appear without a
reload; toasts point at the Assistant instead of back at settings.
- The empty transcript owns a plain-language welcome and three suggested
prompts that dispatch as real turns (ASSISTANT_SUGGESTED_PROMPTS).
- Patrol static readiness: the blessed Ollama Patrol model is Ready;
other Ollama models keep the warning, now naming the selected model.
- pulse_summarize fleet argument errors instruct the model to enumerate
resources itself instead of interrogating the operator (observed live:
'how is my machine doing?' ended in a resource-ID elicitation).
Contracts: ai-runtime and frontend-primitives Current State updated.
Tests: full internal/ai + internal/api suites green; vitest ChatMessages,
AISettings, aiChat store, and settingsArchitecture suites green; flow
verified live end-to-end.
make lint (dupl) flagged two clone pairs: shouldRestartAIChat vs
aiSettingsUpdateTouchesPatrolReadiness shared the forty-field provider
config block, and the approved/rejected action decision-ID collectors in
Pulse Intelligence telemetry differed only by lifecycle state and audit
predicate.
Extract aiSettingsUpdateTouchesProviderConfig for the shared field
check and parameterize pulseIntelligenceActionDecisionIDs by state,
label, and audit predicate, keeping the two named wrappers. The
decision-telemetry contract tests now pin the wrapper call sites, which
still fix the lifecycle state and audit predicate per decision kind.
Contract-Neutral: behavior-preserving dedup of internal helpers to clear dupl lint findings; no public-contract delta
The raw-command endpoint answers 410 Gone unconditionally; the old
implementation below the retirement return was unreachable (govet) and
kept normalizeRunCommandApprovalTarget, normalizeAIExecuteTargetType,
and AIRunCommandRequest alive with no live callers. Remove them and the
tests that only exercised the removed helpers.
Contract-Neutral: remove unreachable dead code below the 410 retirement return in HandleRunCommand; endpoint behavior unchanged
'This agent does not support durable typed-operation receipts' told users
nothing they could act on. Say the agent is on an older version and how to
resolve it. Applies to docker container, host update and host storage
cleanup executors.
Every ID token verification failure rendered the issuer-mismatch
advice, sending users with audience or clock problems down the wrong
path and telling users with a genuine issuer mismatch nothing they had
not already checked (#1533). Map the distinct verification failures
(issuer, audience, expiry) to their own error codes and give each
accurate login-page copy pointing at the server log's got/want detail.
A full monitor reload tears the state down and rebuilds it empty, so
agent-reported host rows vanish until each agent's next report lands.
Config fetches in that window 404ed with a perfectly valid token, which
showed up as rare correlated agent_config_fetch failures across
unrelated agents (#1570).
When the live snapshot has no match, resolve the host from the
persisted continuity store with the same semantics as the live path. A
report-scoped token resolves only its bound host and manage-scoped
tokens resolve by host ID. Deliberately removed hosts stay 404 because
removal deletes their continuity entry.
RecoverExecutingActions existed with full test coverage but had no
production caller, so any typed action mid-dispatch across a server
restart (container update, start/stop/restart, host update, storage
cleanup) stayed in the executing state forever and sat in the Actions
inbox as live work, even after the agent persisted its terminal durable
receipt. Reproduced live on the dev instance with a Docker container
update (act_bf77dfe860ad3d8e4e0a91dc8eb83b44).
The router now runs a bounded, serialized recovery pass per organization
from a startup background worker, and again whenever an agent
(re)registers on the agentexec command server via a new registration
notifier, because a receipt-pending attempt can only be reconciled while
the owning agent is connected. Both triggers reuse the existing
query-only reconciliation semantics; nothing gains a resend authority.
Task 07 owns this residual; the api-contracts and agent-lifecycle
subsystem contracts now record the production trigger. The
rg-07-durable-delivery gate suite stays green, and a new router-level
test pins that a receipt-pending executing action completes from the
agent receipt without a second dispatch.
rcourtman