The desired side of the /api/connections command-policy comparison is
contractually the effective config served to the agent after token scope
and binding checks, but a host whose recorded TokenID no longer resolved
to a live API token skipped sanitization entirely and kept the raw
profile desire. A stale binding, typical after a token was revoked and
the agent reinstalled, then presented as Command policy mismatch even
though the served runtime config could never enable commands and the
operator had nothing to fix (seen on issue #1564). When API tokens
exist and the host's token cannot be resolved, desired command policy
now fails closed to disabled.
The contract delta also records the install-token mint semantics that
landed with the Add Pulse Agent scope fix, closing the deferred
agent-lifecycle delta noted on that commit's contract-neutral trailer.
Contract-Neutral: fail-closed bugfix bringing the connections command-policy view into conformance with the existing agent-lifecycle contract clause; the staged agent-lifecycle delta records the semantics, api-contracts/storage-recovery have no real delta
Keys linked to Readonly Admin users return 403 on endpoints Pulse
polls through the deprecated REST bridge on TrueNAS SCALE 25.10
(issue #1589); note the Full Admin workaround until the WebSocket API
migration (#1550) lands.
Contract-Neutral: user docs note for the TrueNAS 25.10 role limitation (#1589)
wilddev65's rc.2 retest confirmed the connection-keying fix split the two
same-serial CORE boxes but left four symptoms. All four trace to Pulse
reading fields the TrueNAS API has never served, which the pre-fix
identity flapping had masked behind bigger breakage:
- Datasets all showed Offline: pool.dataset.query returns no "mounted"
field on any TrueNAS version (CORE 13 and SCALE both strip it from the
property allowlist), and the parser read absence as unmounted. A listed
dataset now defaults to mounted unless "locked" (encrypted, key
unloaded) or an explicit mounted=false says otherwise.
- Disks all showed Attention/Unknown: disk.query carries no status or
smart_status field either, so every disk landed on UNKNOWN health,
which the storage table bucketed as Attention. Per-disk ZFS state
(ONLINE/DEGRADED/FAULTED/...) and pool membership are now derived from
the pool.query topology that both platforms attach unconditionally --
disk.query's extra.pools cannot cross the REST bridge, whose query
params turn into filters that silently match nothing. Disks therefore
nest under their pools again, and a genuinely unreadable health renders
as neutral Unknown instead of amber Attention.
- Disk temperatures were blank: disk.temperatures takes parameters, so
REST v2.0 has only ever served it as POST with a body keyed by param
name; the client's GET failed on every version, and CORE (no JSON-RPC
endpoint) has no reporting fallback to rescue it. The ladder now tries
native JSON-RPC reporting first and falls back to the POST form, and
the provider surfaces API-reported disk temps as SMART sensor entries
so the host Thermals card lists disks for API-backed systems.
- The storage table ordered identically-named pools (a DR pair both
named "tank") by map iteration order, flipping between refreshes;
ordering now tie-breaks on system name then id.
Verified end-to-end against two stub TrueNAS CORE 13 REST servers
reporting the same placeholder DMI serial: storage tab nests both
systems' pools/datasets/disks correctly with temperatures and Healthy
states, and the system drawer Thermals card lists per-disk readings.
Completes the first-session elicitation fix that a3f8b18bf started at
the tool layer. The second observed failure mode ("Are there any alerts
I should look at?" on Ollama qwen3:8b) had the model calling
pulse_question BEFORE any tool call, so no tool error copy and no tool
description can reach it — the resolve-before-asking prompt policy
(945ed2198) needs runtime teeth for small local models.
The agentic loop now refuses an interactive pulse_question issued
before the run has attempted any real tool call: the model gets an
error tool result steering it to read-only enumeration
(pulse_summarize {"action":"fleet"} and the alert tools need no
parameters), no question card or clarify event reaches the stream,
sibling tool calls from the same provider turn keep processing instead
of tripping the interactive-set skip path, any queued tool attempt
satisfies the gate for the rest of the run, and it fails open after 2
refusals so an unanswerable prompt cannot livelock. Non-interactive
profiles keep their existing separate block. The system prompt
discloses the gate; question-flow tests now open with a look turn the
way real runs must; a corpus scenario pins the stream promise (a
natural first question produces an answer, never a clarification card).
Live-verified on a real Ollama qwen3:8b scratch instance (real local
agent, not mock): both natural openers stream tool-backed answers with
zero question events. Full ./internal/ai/... suite green. ai-runtime
contract updated in-commit.
A fresh install's natural first question ("how is my machine doing?",
observed live with Ollama qwen3:8b) made the model call pulse_summarize
action=fleet without resource_ids; the tool errored and the agentic loop
surfaced a structured question asking a first-run user for "the
comma-separated list of resource IDs" — pure jargon, funnel-killing
(GitHub discussion #1042). Two root fixes:
pulse_summarize self-targets. action=fleet with resource_ids omitted now
enumerates the known fleet from the executor's unified resource provider
(infrastructure parents first, then guests, then storage; deduped;
bounded at the existing 50-resource cap with a truncation note). Both
modes resolve what models actually pass — canonical unified IDs and
unambiguous names — onto the reporting request shape the way the API
path's resolveReportSubject does: the canonical ID stays ResourceID
(findings/recovery keying) and the resolved metrics target rides
MetricsResourceID, so store queries find data instead of silently
returning zero points. Reporting types are classified from the unified
resource (agent-backed hosts "agent", pure Proxmox nodes "node" — the
documented target-type exception — Docker hosts "docker-host");
resource_type becomes an optional filter/default. Remaining error paths
instruct the model to enumerate or retry and forbid asking the operator
for resource IDs.
Ask-user policy covers tool-argument recovery. The system prompt's
resolve-before-asking section (945e2198's target policy) now extends to
failed tool calls: missing/invalid arguments are self-recovered
(enumerate, retry), and internal identifiers are never valid questions.
The pulse_question description carries the same prohibition so the
structured clarification surface cannot become an identifier elicitation
channel on small local models.
ai-runtime and api-contracts contracts updated in-commit (the
subsystem_lookup line pin follows the api-contracts insertion); full
./internal/ai/... and ./internal/agentcapabilities/... suites green.
Same defect class as 64fb3d198 (pkg/securityutil): the SSRF-pinned webhook
dialer resolved the host and dialed only the first permitted IP, so a host
resolving to ::1 ahead of 127.0.0.1 while the receiver listens on one
loopback family — or a multi-A-record host with a dead leading address —
got a hard connection failure even though curl works. The dialer now tries
each permitted resolved IP in resolution order; every candidate still
passes the private-IP/allowlist validation, so rebinding protection is
unchanged.
Evaluated migrating onto securityutil.NewRestrictedOutboundHTTPClient and
rejected a straight swap: the webhook client's isPrivateIP blocks a wider
range set (CGNAT, benchmarking, TEST-NET), its redirect policy allows
re-validated cross-origin hops where securityutil enforces same-origin,
Proxy is deliberately nil, and the private-IP allowlist is a live per-CIDR
runtime hook — a swap would loosen the SSRF policy and change behavior.
Adds a resolver test seam on NotificationManager and a regression test
pinning the ::1-first fallback; notifications contract records the
multi-IP dial obligation.
Closes chip task_5e91afd9.
Production telemetry showed 1,437 installs on the external-agent surface but
only 2 on the Pulse MCP adapter. Live verification proved the pulse_mcp
telemetry surface attribution is honest (adapter tools/call records
surface=pulse_mcp; header-less calls record agent_api), so the gap is real -
but the funnel comparison is also misleading: agent_api counts any scoped
API-token REST consumer (scripts, dashboards polling /api/config/nodes or
/api/ai/patrol/findings), not BYO AI agents. Recorded that interpretation
constraint on PulseIntelligenceExternalAgentEvidence.
The genuine product gap: the in-app connector setup (Settings -> Pulse
Intelligence -> Assistant -> External agents) was invisible to sidebar search.
Searching 'mcp', 'claude', 'opencode', or 'connector' returned 'No settings
found' because search matched only nav labels and header descriptions.
- Add search-only keywords to SettingsNavItem and match them in
filteredTabGroups; the Assistant item now carries mcp / model context
protocol / external agent(s) / claude / opencode / connector / pulse-mcp.
- Name external agent (MCP) connectors in the Assistant header description
(EN/DE/ES kept in sync), so the page header states what the page contains.
- Pin the search behavior with parameterized useSettingsAccess tests plus a
settingsArchitecture source proof; record the keyword channel in the
frontend-primitives contract (obligation 41) and the product-copy allowance
in the security-privacy i18n extension point.
The 30d window said 18 of 26 approved action attempts failed but
carried zero cause information. Every approved attempt that is not a
verified success now lands in exactly one content-free bucket:
- pre_dispatch: terminally refused before dispatch (plan drift, expiry,
emergency stop, policy authorization)
- execution: dispatched execution failed or ended inconclusive
- unverified: execution succeeded but outcome verification was not
confirmed (reads as success in the UI, counted as failure here)
- stuck_executing: still executing over an hour after dispatch
plus one sanitized machine reason code for the most recent failure.
Successes, failure buckets, and recent in-flight attempts partition the
attempt count, so the next window explains its own gap.
RefuseActionExecution now persists the specific refusal code
(plan_drift, action_plan_expired, ...) as the canonical execution
reason code instead of the generic pre_dispatch_refused, so audit truth
and telemetry distinguish refusal causes without message parsing.
Privacy disclosures updated in both PRIVACY.md copies; the adoption
report script surfaces the new counters.
Contract-Neutral: additive content-free telemetry counters + specific pre-dispatch refusal reason codes; privacy disclosures updated in-commit per security-privacy extension point
Production telemetry shows 17 weekly-active installs with Relay enabled out
of 6,225 — the weakest adoption of any feature flag. A funnel walk from a
fresh session showed why: the only Relay surface was the last item of the
last Settings section, hidden entirely from unlicensed installs, described
in transport jargon, and never mentioned on the alert Notifications page
where "get alerts on my phone" actually lives.
- Alerts → Notifications: new "Mobile push notifications" destinations
section (AlertPushDestinationsSection) presenting push via Pulse Mobile
alongside email/Apprise/webhooks. Licensed installs get a pointer to the
canonical /settings/system-relay panel; unlicensed installs get the shared
FeatureGateSection with the relay upgrade destination, suppressed when
upgrade prompts are hidden. Page description now names mobile push.
- Settings nav: system-relay no longer sets hideWhenUnavailable — the
Remote Access entry stays visible to free installs and the panel-owned
upgrade gate communicates the paid boundary. relayRead capability hiding
is unchanged.
- Remote Access copy rewritten around user value: push notifications and
remote monitoring through Pulse Mobile with no port forwarding or VPN;
pairing help drops "provisions a dedicated relay access credential"
jargon, names iOS + Android early access, and states that download-page
install links appear after activation-key entry. EN/ES/DE headers updated.
- Contracts updated in-commit: alerts.md (destinations catalog extension
point), frontend-primitives.md (system-relay is not a deep-link-only
flow), cloud-paid.md (value-first relay presentation obligation).
Contract-Neutral: i18n messages edit changes settings-header/alerts-destinations copy only; no security-privacy delta; alerts/cloud-paid/frontend-primitives contracts updated in-commit
The restricted outbound dialer resolved a hostname, validated the results,
then dialed only the first permitted IP. When localhost resolves to ::1
ahead of 127.0.0.1 and the target service binds only 127.0.0.1 (Ollama's
default), the dial failed with 'connect: connection refused' even though
curl and browsers connect fine via address-family fallback. This broke the
AI provider setup quickstart: the blessed http://localhost:11434 Ollama URL
failed Pulse's connection test on IPv6-first hosts (setup-friction class of
#847/#1003).
Try each permitted IP from the validating resolution in order until one
connects. Every candidate is still validated against the outbound policy
from the same resolution, so the DNS-rebinding pinning guarantee is
unchanged. Fixes all consumers of the shared client (Ollama provider, OIDC,
SSO, connection probes, availability poller); api-contracts wording updated
to match. Live-verified against Ollama 0.32.1 bound to 127.0.0.1 only:
TestConnection via http://localhost:11434 failed before, passes after.
Production telemetry (30d) shows Patrol averaging ~9.4 billed AI calls
per run while only ~1-2 come from the analysis passes. The remainder is
the alert auto-resolve review: one QuickAnalysis call per active alert
(>=10min old) on every run, re-asking the model about the same standing
alert every interval even when the trigger condition demonstrably still
holds.
Two structural changes, both resolution-neutral:
- Still-firing gate: skip the model review for alerts whose current
snapshot still shows the trigger condition (metric at/above threshold,
offline resource still offline). A correct review could only answer
KEEP, so the question is not worth a billed call. The gate never
resolves anything locally - uncertain cases (unknown types, missing
resources, unmapped metrics) still go to the model, which retains
sole authority over resolution.
- Batched review: the remaining candidates are reviewed in one model
call per 20 alerts (numbered verdict lines, unparseable -> KEEP,
bare-RESOLVE fallback for single-alert batches) instead of one call
per alert.
Steady-state runs with standing alerts drop from N review calls to 0;
runs where conditions may have cleared pay ceil(K/20) instead of K.
QuickAnalysisRequest gains a TargetType tag (recorded on the usage
event) so cost telemetry can decompose alert-review spend from the
main patrol pass going forward. Contract updated in-commit
(ai-runtime: model-owned resolve direction, cost-gated keep direction,
batched review, fail-safe parsing, alert_autoresolve usage tagging).
Production telemetry (7d, 2026-07-17) shows a persistent tail of live
installs on 6.0.0-rc.x and 6.0.3/6.0.4 while 6.0.5 has been stable since
2026-07-09. Root cause for the reachable stranding class: release
selection trusted GitHub's created_at ordering, and this repo interleaves
v5-line maintenance releases with v6 releases (v5.1.36 was created the
day before v6.0.5). Whenever a v5.1.x release is the most recently
created stable:
- stable channel (getLatestReleaseForChannel) returned the first
non-prerelease in list order -> "no update" for every v6 install
until the next v6 release ships
- the RSS rate-limit fallback returned the first feed entry matching the
channel, same failure, both channels
- the unattended updater trusted /releases/latest, which GitHub defines
as most-recently-created stable -> timer no-ops in the same window
All three now select the highest eligible version: the API path tracks
newest stable + newest prerelease in one pass (prerelease = GitHub flag
OR tag shape), the feed fallback picks the max matching the channel, and
pulse-auto-update.sh scans /releases pairing each tag with its own
draft/prerelease flags before the fail-closed shape filter, keeping
/releases/latest as fallback. helm-chart-* tags fail semver parsing and
are dropped everywhere.
Verified not broken (pinned by new tests): rc-channel installs are
offered the newer stable (6.0.0-rc.6 -> 6.0.5) and move onto the next rc
line when one opens (-> 6.1.0-rc.2); the auto-update prerelease filter
(fail-closed is_prerelease_tag, metadata-flag refusal) still refuses
prerelease targets on stable. Go tests fail on the pre-fix code
(returned 5.1.37 / "no update").
Contract: deployment-installability now pins version-max release
selection with proofs in manager_stranded_upgrade_test.go and
pulse_auto_update_test.go.
Telemetry shows thousands of installs configure an AI provider but almost
none ever use the interactive Assistant. A live first-session exercise
(fresh install, Ollama qwen3:8b quickstart) found why: after enabling,
nothing changes on screen — the launcher and handoff buttons are gated on
sessionCapabilities.assistantEnabled, which was only read at page load;
the empty transcript was blank; and the blessed Ollama+qwen3:8b path
reported Patrol degraded while telling the user to pull the model they
had just selected.
- Setup-modal success now opens the Assistant drawer, and the AI settings
save paths refresh the assistantEnabled capability in place
(aiChatStore.refreshEnabledFromServer) so entry points appear without a
reload; toasts point at the Assistant instead of back at settings.
- The empty transcript owns a plain-language welcome and three suggested
prompts that dispatch as real turns (ASSISTANT_SUGGESTED_PROMPTS).
- Patrol static readiness: the blessed Ollama Patrol model is Ready;
other Ollama models keep the warning, now naming the selected model.
- pulse_summarize fleet argument errors instruct the model to enumerate
resources itself instead of interrogating the operator (observed live:
'how is my machine doing?' ended in a resource-ID elicitation).
Contracts: ai-runtime and frontend-primitives Current State updated.
Tests: full internal/ai + internal/api suites green; vitest ChatMessages,
AISettings, aiChat store, and settingsArchitecture suites green; flow
verified live end-to-end.
Production telemetry (30d) shows the Pulse Intelligence funnel collapsing at
governed actions: 332 installs enabled chat actions but only 23 created an
action plan. Root cause: both action dials default off, and an install in the
default Watch only mode has no forward path — findings on unlocked installs
show neither an Investigate button (gated on non-monitor modes) nor any nudge
(the Pro handoff only renders when the capability is plan-locked), and the
empty Actions inbox never says why it is empty.
- Patrol findings on unlocked Watch-only installs now offer 'Switch to Ask
first' in the expanded row (same slot as the Pro handoff), driving
handleAutonomyChange('approval') so the Investigate action appears in place.
- The empty Actions Open tab now explains that Watch only never queues fixes
and points at the Patrol mode switch, or names the Pro capability on locked
installs (presentation-policy aware, upgrade link suppressed with
hideUpgrade).
- Contracts updated: patrol-intelligence (expanded-row forward-path rule),
unified-resources (Actions calm-state producer explanation), api-contracts
(canonical autonomy read/save reuse), frontend-primitives (guidance stays in
governed presentation helpers).
Verified live on a scratch instance (locked drawer keeps the Pro handoff with
no nudge; empty inbox renders the locked explanation) plus DOM tests for the
unlocked slot, which needs the Pro runtime to exercise live.
Every mock-mode boot (e2e/CI managed backends, qual runs, demo containers)
was sending startup/heartbeat pings describing the synthetic fixture fleet,
registering as real installs in the license-server telemetry DB and
dominating the VMware/Kubernetes/TrueNAS adoption aggregates (346 of 6,225
weekly-active installs matched the mock signature, incl. 120 of 227 on
6.1.0-rc.1).
- internal/telemetry: route startup and heartbeat sends through sendEvent,
which drops the ping while mock.IsMockEnabled() is true; checked per event
so runtime mock toggles take effect immediately. Disclosed in the package
doc and both PRIVACY.md copies.
- scripts/telemetry_adoption_report.py: exclude historical mock-fleet rows
(kubernetes_pods = 120×N with vmware_hosts = 7×N, the internal/mock
fixture template) by default, report the excluded row/install counts, and
add --include-mock-fleet to audit them. Versions through 6.1.0-rc.2 keep
pinging the mock fleet until upgraded, so report-side filtering stays
necessary.
- security-privacy contract: record the mock-mode suppression boundary and
the report-side mock-fleet exclusion as subsystem invariants.
- Tests: Go sendEvent suppression/sending pair against an httptest endpoint;
Python signature-matcher and summarize_rows exclusion coverage.
Cloned VMs that keep the same /etc/machine-id collapse into a single
Docker host in Pulse, with each clone's report silently overwriting the
other (#1584). The server now tracks per-identity hostname and machine
ID observations and flags a conflict when a value flaps back to one
already seen inside a 15-minute window, a signature a one-time rename
never produces. The conflict rides the DockerHost model through unified
resources, and the Docker page shows a warning naming the flapping
hostnames with the machine-id remedy. The warning self-clears once one
clone stops reporting for the window.
rcourtman