Contract-Neutral: RC qualification fixes preserve existing public API, tenant, monitoring, and organization contracts while correcting canonical runtime ownership and test fixtures.
Cover pure functions left at 0% after the canonical Operational Trust and
unified-resource work, surfaced by a fresh coverage probe.
- unifiedresources views.go accessors for K8sNodeView, DockerContainerView,
PodView, K8sDeploymentView, HostView, DockerHostView, K8sClusterView and
the smaller ContainerView.Pool / NodeView.IsClusterMember /
PhysicalDiskView.MetricResourceID / PBSInstanceView.Datastores /
PMGInstanceView.InstanceID accessors. Each exercises the nil-receiver and
nil-nested defensive arms, the populated projection, and slice/map
clone-independence.
- unifiedresources clone.go seven Ceph deep-clone helpers, asserting value
equality, mutation independence, and nil/empty inputs.
- agentcapabilities firstStringPayloadValue, MCPManifestPromptProjectionSupported
and JSONRPCError.Error.
- ai/tools data_types NormalizeCollections (three receivers),
ValidateCurrentResourceAvailable and ErrExecutionContextUnavailable.Error.
Test-only. Every target function moved 0% to covered. No source changes.
Contract-Neutral: test-only branch-coverage tests; no source or contract changes
GatePatrolAutonomyUpdate validates and consumes the activation
acknowledgement, then re-marshals the settings for the downstream
handler - but the marshaled body still carried acknowledgement_id.
The Pro autonomy handler decodes the body with DisallowUnknownFields
and has no such field, so every Autopilot (full mode) activation on a
Pro build failed with 400 Invalid request body. Lower autonomy levels
forward the original body untouched, which is why only full mode broke.
Clear the consumed acknowledgement before normalizing and hand it to
the downstream handler via request context only. The new regression
test strict-decodes the forwarded body with the enterprise handler's
exact field set.
Contract-Neutral: Autopilot activation gate normalization fix; endpoint contract unchanged
Test-only wave, contract-neutral. New *_branchcov0719pm_test.go files cover
previously-uncovered pure value-in/value-out helpers, each verified to move
its target functions from 0% to full coverage:
- internal/agentcontext: formatKubernetesServicePorts (empty/single/cap/overflow
arms) and addMetricFact (nil-metric, percent/value/ratio arms) now 100%.
- pkg/reporting: reportLogoTypeFromPath 0->100, reportLogoTypeFromData 28.6->100,
scaledLogoSize 70->90 (extension and aspect branches).
- internal/cloudcp/email: RenderMagicLinkEmail 0->80 (render success path; the
compile-time template-error arm is unreachable and left uncovered).
- internal/recovery: recoveryDetailString (nil map, missing key, non-string,
string arms) and recoveryPointObservedAt 40->100.
- internal/ai/tools: ErrStrictResolution/ErrRoutingMismatch ToToolResponse 100.
- internal/ai/providers: every NormalizeCollections receiver 0->100.
No source or existing test modified.
Cover pure functions the recent canonical Operational Trust and protection
posture work landed with no unit test:
- recovery/model posture: enum Valid, struct Clone/Validate/Payload and the
normalize/cloneTime/validOutcome/sortedUniqueStrings/compareProviderStates
helpers (20 funcs, all 0%->100%)
- operationaltrust contracts: EvidencePayloadRef/Acknowledgement/Suppression/
LifecycleTransition Validate plus LifecycleTransition/NotificationLink Clone
- cloudcp/proxytrust: ClientIP, ExtractRemoteIP, rightMostUntrustedForwardedIP
and IsTrustedProxyIP forwarded-header and CIDR parsing
- mockmodel: NormalizeBlendWeight, SeriesForTimestamps, seriesForProfile,
diskIOValue and flatValue deterministic seeded math
- updatesignature: DecodePrivateKey and HasTrustedPublicKeys
- recovery keys: ProxmoxPBSGuestLooseContinuityKey branch guards
- vmware SourceID and truenas availableAppLogContainers formatting
Test-only, contract-neutral. New *_branchcov0719_test.go files only; no source
or existing test touched.
New *_branchcov0718_test.go files extend the Go coverage wave to three more
areas of previously-uncovered pure logic. Covered areas include the domain-type
NormalizeCollections normalizers in models.go, the cloud control-plane pure
helpers for hosted-entitlement org-id normalization, magic-link URL redaction
and default MSP backup paths, and the actionlifecycle dispatch-attempt context
round-trip plus the typed-error Error and Unwrap methods.
Test-only, with no runtime or subsystem-contract change. Impure targets that
need a live Docker manager or tenant registry were skipped per the purity gate
and noted in the wave reports. Verified in a clean worktree at HEAD with go vet
and package tests green, gofmt clean, and every covered target moved from 0
percent toward full coverage.
New *_branchcov0718_test.go files raise coverage of previously-uncovered pure
functions across ten packages. Covered areas include securityutil SSRF and URL
validation, truenas path and telemetry parse helpers, storagehealth SMART and
physical-disk risk assessment, vmware inventory sort keys and error classifiers,
servicediscovery token filtering and readiness, telemetry evidence-from-history,
models ToFrontend converters and frontend NormalizeCollections normalizers,
actionplanner type predicates and canonical resource-id sort, config API-token
accessors, and licensing state accessors.
Test-only, with no runtime or subsystem-contract change. Verified in a clean
worktree at HEAD with go vet and package tests green, gofmt clean, and every
named target function moved from 0 percent to covered.
A routine Docker container update took five clicks across two in-row
confirmations and a two-phase Approve then Run dialog, and the review
screen said "Rollback declared: No" while its own intended-change text
promised automatic backup and rollback (#1564 feedback).
- Capabilities can declare rollback support; the planner carries it
into plan.rollbackAvailable instead of hard-coding false, so the
container update review now truthfully says rollback is available.
- Container update is classed AutoAuthorizeLowRisk (it ships backup and
rollback; restart was already low_risk). The action projection
exposes the read-time capability class plus resolved blast-radius
names, none of which enter plan identity.
- The review dialog collapses pending approval for low-risk
capabilities into one Approve and run control that records the
approval and dispatches execution from a single confirmation; other
capabilities keep the two-phase flow. Blast radius renders resource
names with IDs demoted.
- The in-row Update button plans on first click; the dialog is the
confirmation surface, so the Confirm hop and its cancel affordance
are gone.
Routine update ceremony is now Update, Approve and run, Close, with
the durable action record unchanged.
Wiping the agent state dir regenerates the agent ID and a fresh install
command mints a fresh token, so Docker host identity resolution correctly
refuses to adopt the old record (a foreign token must never take over a
live host) and creates a new one. Nothing ever reaped the old record, so
it sat in state forever with a stale agent version, stale containers and
stale image digests. That is the source of the outdated-agent banner that
contradicts the update dialog in #1586, the doubled container rows in
#1564, and plausibly the digest-mismatch preflight refusal there too.
Reap a matching record only when it stopped reporting before the
superseding token was minted, the same explicit-reenroll-intent rule the
removal block uses (#1581). A record that is still reporting keeps
advancing LastSeen and is never touched, and supersession sets no
resurrection block.
Refs #1586#1564
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT: docker host state bug fix; live Codex agent WIP holds the contract docs
Field report (Johannes Strasser, 6.1.0-rc.3 mobile chat). The assistant
could find an LXC container but told him it had no tools to restart it.
Relay-mobile tokens carry only the relay access scope, which does not
satisfy the execute-authority check, so mobile chat sessions were offered
read tools only. The gap was one-sided: mobile could already approve and
execute planned actions, it just could not propose them.
Relay-mobile callers now hold execute authority, and the chat entrypoint
pins their sessions to approval-required mode, so every mobile-originated
action lands on the approval queue regardless of the configured control
level. Web sessions are unchanged.
Proxmox guests advertise reboot while container platforms advertise
restart, and the Assistant tool schema plus the resolved-resource action
lists teach models both words. A pulse_control call with the wrong
spelling failed with CapabilityNotFound even though the equivalent
capability was advertised. Plan requests now follow the resource's own
vocabulary when the requested verb is missing but its lifecycle synonym
is advertised; the rewritten name flows into the plan, the audit record
and the executor. Non-synonym verbs still fail closed.
Tool-heavy chat histories exceed the relay proxy's 47KB response cap, so
the mobile app could never load big conversations over Remote Access.
GET /api/ai/sessions/{id}/messages now accepts compact=1 (strips tool
outputs, results and thought signatures while keeping tool_use_id
linkage) and max_bytes=N (returns the newest messages that fit the
budget; a single oversized message is truncated with a notice). The
mobile client sends compact=1&max_bytes=40960 and falls back to a
halving limit ladder when older servers answer 413.
Percentage metric specs derive their critical escalation as trigger+10
capped at 99. A user-set trigger of 99 or 100 produced critical <=
trigger, which failed spec validation and silently dropped the whole
metric from evaluation while logging a warn every poll cycle. Omit the
escalation threshold instead so the warning-level threshold keeps
working.
Seen in the wild as the 'Skipping invalid canonical unified metric
spec' warn spam in #1593's diagnostics.
Contract-Neutral: behavior fix with regression test; alerts contract docs carry another agent's WIP on the shared tree, delta deferred
A PBS datastore reached the unified registry twice, once from the PBS
snapshot adapter under the canonical <instance-id>/<name> source ID and
once from the PBS poller's models.Storage conversion under the legacy
<instance-id>-<name> storage ID. The thresholds page rendered two cards
per datastore and their alert overrides split across both key formats,
with only the legacy key honoured at evaluation time.
Skip the redundant storage-conversion ingest when the canonical
datastore resource is already registered, carry the canonical ID as a
storage alias so evaluation accepts overrides in either format, and add
the legacy dash key to the frontend candidate list so pre-existing
overrides stay bound to the single surviving card and re-home onto the
canonical key on the next save.
Fixes#1591
Contract-Neutral: behavior fix with regression tests across registry/alerts/frontend; contract docs carry another agent's WIP on the shared tree, delta deferred
The PVE backup poll classified storage as PBS-backed by the pbs- name
prefix as well as the storage type. A dir storage merely named pbs-*
(content backup, is_mountpoint) lost all of its vzdump backups from the
Backups view whenever a direct PBS connection was also configured,
because the poll skipped its content as 'PBS is authoritative'. Classify
by storage type only.
Fixes#1592
Contract-Neutral: behavior fix with regression test; monitoring contract docs carry another agent's WIP on the shared tree, delta deferred
The canonical alert evaluation reactivation branch read and mutated
recentlyResolved and resolvedAlias while holding only m.mu, while the
broadcaster and recovery paths guarded them with resolvedMutex, so the
two lock domains did not exclude each other and a flapping resource
re-firing within the five-minute cooldown concurrently with the state
broadcast crashed the daemon with a concurrent map access fault
(issue #1590, reported with exact sites and a race repro).
Both eval sites now take resolvedMutex for the lookup and removal only,
keeping history and dispatch outside the lock. GetResolvedAlert takes
the write lock because getResolvedAlertNoLock backfills resolvedAlias
on a canonical-identity miss, which made even read-locked lookups
racy against each other. The lock ordering documentation now states
the real rule the cleanup path already relied on: resolvedMutex is
subordinate to m.mu and may be acquired while holding it, never the
reverse.
Contract-Neutral: crash bugfix restoring the documented resolved-map lock guarantee (#1590); no contract surface changes
The Patrol finding-notification work (23695681a, e0e454abf) added
patrol_finding_notifications_enabled and
patrol_finding_notify_min_severity to the AI settings payload but did
not update the six JSON snapshot pins, leaving Build and Test red on
every push since. The fields are intended API surface; the snapshots
now include them.
Contract-Neutral: snapshot refresh for the intended Patrol finding-notification fields added in 23695681a/e0e454abf; no behavior change
The desired side of the /api/connections command-policy comparison is
contractually the effective config served to the agent after token scope
and binding checks, but a host whose recorded TokenID no longer resolved
to a live API token skipped sanitization entirely and kept the raw
profile desire. A stale binding, typical after a token was revoked and
the agent reinstalled, then presented as Command policy mismatch even
though the served runtime config could never enable commands and the
operator had nothing to fix (seen on issue #1564). When API tokens
exist and the host's token cannot be resolved, desired command policy
now fails closed to disabled.
The contract delta also records the install-token mint semantics that
landed with the Add Pulse Agent scope fix, closing the deferred
agent-lifecycle delta noted on that commit's contract-neutral trailer.
Contract-Neutral: fail-closed bugfix bringing the connections command-policy view into conformance with the existing agent-lifecycle contract clause; the staged agent-lifecycle delta records the semantics, api-contracts/storage-recovery have no real delta
Completes #1369. The delivery path landed in 23695681a with config-only
gating; this surfaces the two fields so operators can turn finding
notifications off or restrict them to critical findings without editing
config by hand.
The AI settings GET response and update request carry
patrol_finding_notifications_enabled and
patrol_finding_notify_min_severity, with the same warning-or-critical
validation the alert-trigger severity field uses. The Patrol settings
page gains a Notifications card between Triggers and Model readiness,
mirroring the alert-trigger toggle-plus-severity pattern, with the
severity select shown only while notifications are enabled.
Exercised end to end against a scratch backend in mock mode. A severity
change and an explicit opt-out both survive save, reload, and the GET
round trip, and the opt-out hides the severity select immediately.
Patrol findings previously reached operators only through the Relay
mobile push path. Anyone relying on the email, webhook, or Apprise
destinations they already configured for alerts got no proactive signal
when Patrol detected a problem and had to open /patrol to learn about
it. This is the delivery half of #1369.
Each newly stored warning or critical finding now also flows through a
FindingNotifyCallback wired in the router, which projects the finding
into the alert shape the notification manager delivers. The callback
fires only on the genuinely-new path in recordFindingWithInvestigation,
so a finding notifies at most once per lifetime regardless of how many
later runs re-detect it, and SendAlert's own per-ID cooldown backstops
that. Demo mode never notifies.
Gating lives in AIConfig via patrol_finding_notifications_enabled
(default on, matching the long-standing default for mobile push) and
patrol_finding_notify_min_severity (warning or critical, default
warning). The enabled flag persists without omitempty so an explicit
opt-out survives reload while pre-existing configs inherit the default.
The settings surface for these fields follows in a separate commit once
the AI settings handler is free.
The Add Pulse Agent flow composed a fixed scope list in the frontend,
so ticking Enable Pulse command execution added --enable-commands to
the install command while the token it shipped with never carried
agent:exec, and the command channel rejected every registration
(issues #1586, #1564, confirmed by a reporter's server log). Even with
the scope, the generic token had no binding metadata, so the first-use
binding gate would have refused it next.
The install token is now minted through POST /api/agent-install-command
with type 'host'. The server decides the scopes from enableCommands at
mint time, stamps the install_type/issued_via metadata that makes the
token eligible for first-use command-channel binding
(canBindProxmoxAgentInstallExecToken renamed canBindAgentInstallExecToken
and extended to the host install type), and returns the sanitized token
record. The frontend regenerates the token when the checkbox toggles,
since scopes cannot be upgraded on an existing token, and revokes the
superseded token so toggling does not accumulate orphans.
Contract-Neutral: install-token mint bugfix (#1586, #1564): checkbox-promised exec scope now real; agent-lifecycle contract delta deferred because the contract docs carry another agent's uncommitted WIP on the shared tree
The rejection told operators to re-run the agent installer, but the
token minted by the original install command cannot gain the exec scope
after the fact and install tokens are single use, so re-running the
same command loops forever (issues #1586, #1564). Name the actual
recovery step instead.
Contract-Neutral: reworded operator-facing rejection copy (#1586, #1564); no behavior or contract change
A pool built from /dev/disk/by-id/nvme-MODEL_SERIAL_1-part1 references
(systemd appends the _1 namespace token) made serialFromByID take the
last underscore token, so it extracted the namespace digit instead of
the serial and the disk fell back to the generic 'ZFS' usage string
while an identically-built node without the suffix resolved its pool
name. Reported with exact zpool output in issue #1540.
Contract-Neutral: bugfix restoring intended disk-to-pool annotation (#1540); no contract surface changes
The server-side disk-to-pool matcher learned nvme-eui.<hex> zpool member
references in 4dcc18fbd, but the host agent keeps its own copy of the
matching for SMART pool annotation and never derived a key from the
nvme-eui form, nor stripped the eui. prefix smartctl reports on NVMe
WWNs. systemd's nvme by-id links can also carry a trailing _<n>
namespace suffix (nvme-MODEL_SERIAL_1), which made the serial
derivation return the namespace digit instead of the serial. Both gaps
left a pool member unmatched so the disk fell back to the generic
usage string (issue #1540).
Contract-Neutral: bugfix restoring intended disk-to-pool annotation (#1540); no contract surface changes
wilddev65's rc.2 retest confirmed the connection-keying fix split the two
same-serial CORE boxes but left four symptoms. All four trace to Pulse
reading fields the TrueNAS API has never served, which the pre-fix
identity flapping had masked behind bigger breakage:
- Datasets all showed Offline: pool.dataset.query returns no "mounted"
field on any TrueNAS version (CORE 13 and SCALE both strip it from the
property allowlist), and the parser read absence as unmounted. A listed
dataset now defaults to mounted unless "locked" (encrypted, key
unloaded) or an explicit mounted=false says otherwise.
- Disks all showed Attention/Unknown: disk.query carries no status or
smart_status field either, so every disk landed on UNKNOWN health,
which the storage table bucketed as Attention. Per-disk ZFS state
(ONLINE/DEGRADED/FAULTED/...) and pool membership are now derived from
the pool.query topology that both platforms attach unconditionally --
disk.query's extra.pools cannot cross the REST bridge, whose query
params turn into filters that silently match nothing. Disks therefore
nest under their pools again, and a genuinely unreadable health renders
as neutral Unknown instead of amber Attention.
- Disk temperatures were blank: disk.temperatures takes parameters, so
REST v2.0 has only ever served it as POST with a body keyed by param
name; the client's GET failed on every version, and CORE (no JSON-RPC
endpoint) has no reporting fallback to rescue it. The ladder now tries
native JSON-RPC reporting first and falls back to the POST form, and
the provider surfaces API-reported disk temps as SMART sensor entries
so the host Thermals card lists disks for API-backed systems.
- The storage table ordered identically-named pools (a DR pair both
named "tank") by map iteration order, flipping between refreshes;
ordering now tie-breaks on system name then id.
Verified end-to-end against two stub TrueNAS CORE 13 REST servers
reporting the same placeholder DMI serial: storage tab nests both
systems' pools/datasets/disks correctly with temperatures and Healthy
states, and the system drawer Thermals card lists per-disk readings.
Completes the first-session elicitation fix that a3f8b18bf started at
the tool layer. The second observed failure mode ("Are there any alerts
I should look at?" on Ollama qwen3:8b) had the model calling
pulse_question BEFORE any tool call, so no tool error copy and no tool
description can reach it — the resolve-before-asking prompt policy
(945ed2198) needs runtime teeth for small local models.
The agentic loop now refuses an interactive pulse_question issued
before the run has attempted any real tool call: the model gets an
error tool result steering it to read-only enumeration
(pulse_summarize {"action":"fleet"} and the alert tools need no
parameters), no question card or clarify event reaches the stream,
sibling tool calls from the same provider turn keep processing instead
of tripping the interactive-set skip path, any queued tool attempt
satisfies the gate for the rest of the run, and it fails open after 2
refusals so an unanswerable prompt cannot livelock. Non-interactive
profiles keep their existing separate block. The system prompt
discloses the gate; question-flow tests now open with a look turn the
way real runs must; a corpus scenario pins the stream promise (a
natural first question produces an answer, never a clarification card).
Live-verified on a real Ollama qwen3:8b scratch instance (real local
agent, not mock): both natural openers stream tool-backed answers with
zero question events. Full ./internal/ai/... suite green. ai-runtime
contract updated in-commit.
A fresh install's natural first question ("how is my machine doing?",
observed live with Ollama qwen3:8b) made the model call pulse_summarize
action=fleet without resource_ids; the tool errored and the agentic loop
surfaced a structured question asking a first-run user for "the
comma-separated list of resource IDs" — pure jargon, funnel-killing
(GitHub discussion #1042). Two root fixes:
pulse_summarize self-targets. action=fleet with resource_ids omitted now
enumerates the known fleet from the executor's unified resource provider
(infrastructure parents first, then guests, then storage; deduped;
bounded at the existing 50-resource cap with a truncation note). Both
modes resolve what models actually pass — canonical unified IDs and
unambiguous names — onto the reporting request shape the way the API
path's resolveReportSubject does: the canonical ID stays ResourceID
(findings/recovery keying) and the resolved metrics target rides
MetricsResourceID, so store queries find data instead of silently
returning zero points. Reporting types are classified from the unified
resource (agent-backed hosts "agent", pure Proxmox nodes "node" — the
documented target-type exception — Docker hosts "docker-host");
resource_type becomes an optional filter/default. Remaining error paths
instruct the model to enumerate or retry and forbid asking the operator
for resource IDs.
Ask-user policy covers tool-argument recovery. The system prompt's
resolve-before-asking section (945e2198's target policy) now extends to
failed tool calls: missing/invalid arguments are self-recovered
(enumerate, retry), and internal identifiers are never valid questions.
The pulse_question description carries the same prohibition so the
structured clarification surface cannot become an identifier elicitation
channel on small local models.
ai-runtime and api-contracts contracts updated in-commit (the
subsystem_lookup line pin follows the api-contracts insertion); full
./internal/ai/... and ./internal/agentcapabilities/... suites green.
Same defect class as 64fb3d198 (pkg/securityutil): the SSRF-pinned webhook
dialer resolved the host and dialed only the first permitted IP, so a host
resolving to ::1 ahead of 127.0.0.1 while the receiver listens on one
loopback family — or a multi-A-record host with a dead leading address —
got a hard connection failure even though curl works. The dialer now tries
each permitted resolved IP in resolution order; every candidate still
passes the private-IP/allowlist validation, so rebinding protection is
unchanged.
Evaluated migrating onto securityutil.NewRestrictedOutboundHTTPClient and
rejected a straight swap: the webhook client's isPrivateIP blocks a wider
range set (CGNAT, benchmarking, TEST-NET), its redirect policy allows
re-validated cross-origin hops where securityutil enforces same-origin,
Proxy is deliberately nil, and the private-IP allowlist is a live per-CIDR
runtime hook — a swap would loosen the SSRF policy and change behavior.
Adds a resolver test seam on NotificationManager and a regression test
pinning the ::1-first fallback; notifications contract records the
multi-IP dial obligation.
Closes chip task_5e91afd9.
Production telemetry showed 1,437 installs on the external-agent surface but
only 2 on the Pulse MCP adapter. Live verification proved the pulse_mcp
telemetry surface attribution is honest (adapter tools/call records
surface=pulse_mcp; header-less calls record agent_api), so the gap is real -
but the funnel comparison is also misleading: agent_api counts any scoped
API-token REST consumer (scripts, dashboards polling /api/config/nodes or
/api/ai/patrol/findings), not BYO AI agents. Recorded that interpretation
constraint on PulseIntelligenceExternalAgentEvidence.
The genuine product gap: the in-app connector setup (Settings -> Pulse
Intelligence -> Assistant -> External agents) was invisible to sidebar search.
Searching 'mcp', 'claude', 'opencode', or 'connector' returned 'No settings
found' because search matched only nav labels and header descriptions.
- Add search-only keywords to SettingsNavItem and match them in
filteredTabGroups; the Assistant item now carries mcp / model context
protocol / external agent(s) / claude / opencode / connector / pulse-mcp.
- Name external agent (MCP) connectors in the Assistant header description
(EN/DE/ES kept in sync), so the page header states what the page contains.
- Pin the search behavior with parameterized useSettingsAccess tests plus a
settingsArchitecture source proof; record the keyword channel in the
frontend-primitives contract (obligation 41) and the product-copy allowance
in the security-privacy i18n extension point.
Two diagnosability gaps found by exercising the docker-update vertical
live with a real agent, each of which turns 'approved action cannot
execute' into a dead end with no visible cause:
- The durable operation-receipt store failing to open (e.g. unwritable
state dir) silently registered the agent with receipt version 0; the
only symptom was a server-side claim that the agent was too old. The
agent now logs the real cause and state dir loudly at startup.
- The server readiness copy asserted 'older agent version' for any
receipt-version mismatch. It now names both causes (old version or
unavailable state directory) and points at the agent logs.
(The related docker collect-cycle watchdog landed separately in
a0f75b1bb with a hung-daemon reproducer.)
Contract-Neutral: agent-side diagnosability: receipt-store failure warning + honest readiness copy; no public contract delta
The 30d window said 18 of 26 approved action attempts failed but
carried zero cause information. Every approved attempt that is not a
verified success now lands in exactly one content-free bucket:
- pre_dispatch: terminally refused before dispatch (plan drift, expiry,
emergency stop, policy authorization)
- execution: dispatched execution failed or ended inconclusive
- unverified: execution succeeded but outcome verification was not
confirmed (reads as success in the UI, counted as failure here)
- stuck_executing: still executing over an hour after dispatch
plus one sanitized machine reason code for the most recent failure.
Successes, failure buckets, and recent in-flight attempts partition the
attempt count, so the next window explains its own gap.
RefuseActionExecution now persists the specific refusal code
(plan_drift, action_plan_expired, ...) as the canonical execution
reason code instead of the generic pre_dispatch_refused, so audit truth
and telemetry distinguish refusal causes without message parsing.
Privacy disclosures updated in both PRIVACY.md copies; the adoption
report script surfaces the new counters.
Contract-Neutral: additive content-free telemetry counters + specific pre-dispatch refusal reason codes; privacy disclosures updated in-commit per security-privacy extension point
Production telemetry (30d) showed 18 of 26 approved governed-action
attempts never reaching verified success. The dominant stranding path:
HandleExecuteAction dispatched under the HTTP request context, so a
reverse proxy or browser dropping the connection mid-operation (docker
image pulls run minutes; proxies cut at 60-100s) cancelled the committed
dispatch and left the action executing forever — recovery only ran at
restart or agent re-registration, which never fires while the agent
stays healthily connected.
- Execute now dispatches under context.WithoutCancel: a committed
dispatch runs to its terminal state regardless of the caller's
connection, bounded by the per-operation transport timeout.
- A standing 2-minute recovery loop re-drives durable-dispatch
reconciliation so any abandoned executing action (timeout, send
failure, crash) converges from the agent's durable receipt instead
of waiting for a reconnect that may never come.
Verified live: docker-update executed with the client aborting at 3s;
the agent completed pull+recreate+verify and the action reached
completed/verified with the replacement container running.
Contract-Neutral: behavioral fix: detach approved-action dispatch from client ctx + periodic recovery loop; no public contract delta
Follow-up to the 2026-07-17 live docker-update exercise where a DiskUsage
roundtrip against a colima daemon parked 6+ minutes even though
dockerCallWithRetry wraps every call in a 20s context.WithTimeout.
Investigation result: context deadline propagation through moby client
v0.5.0 (request.go, API-version negotiation) and the otelhttp transport
wrapper is intact. Reproducing with a deliberately hung unix-socket
daemon aborts DiskUsage at the deadline in both hang shapes (pre-header
and mid-body), so there is no client-library bug to fix or file
upstream; the production stall's root cause remains environmental
(deadline timer never fired process-side).
Containment and diagnosis:
- buildReport now runs under dockerCollectCycleTimeout (5m) so a wedged
cycle can never stall the module indefinitely, plus an independent
watchdog timer that logs an error with a full goroutine dump if the
cycle outlives even that deadline - capturing exactly the evidence
that was missing from the original incident.
- hung_daemon_deadline_test.go pins that a context deadline aborts the
real moby client against a hung unix-socket daemon (pre-header and
mid-body stalls), guarding future moby/otelhttp upgrades.
The incident note referenced a dockerCollectCycleTimeout watchdog as
already added; it did not exist on any branch - this commit is that
containment, landed for real.
Contract-Neutral: dockeragent collect-cycle watchdog containment: timeout plumbing only, no collection-semantics or contract-surface delta
Production telemetry (30d) shows Patrol averaging ~9.4 billed AI calls
per run while only ~1-2 come from the analysis passes. The remainder is
the alert auto-resolve review: one QuickAnalysis call per active alert
(>=10min old) on every run, re-asking the model about the same standing
alert every interval even when the trigger condition demonstrably still
holds.
Two structural changes, both resolution-neutral:
- Still-firing gate: skip the model review for alerts whose current
snapshot still shows the trigger condition (metric at/above threshold,
offline resource still offline). A correct review could only answer
KEEP, so the question is not worth a billed call. The gate never
resolves anything locally - uncertain cases (unknown types, missing
resources, unmapped metrics) still go to the model, which retains
sole authority over resolution.
- Batched review: the remaining candidates are reviewed in one model
call per 20 alerts (numbered verdict lines, unparseable -> KEEP,
bare-RESOLVE fallback for single-alert batches) instead of one call
per alert.
Steady-state runs with standing alerts drop from N review calls to 0;
runs where conditions may have cleared pay ceil(K/20) instead of K.
QuickAnalysisRequest gains a TargetType tag (recorded on the usage
event) so cost telemetry can decompose alert-review spend from the
main patrol pass going forward. Contract updated in-commit
(ai-runtime: model-owned resolve direction, cost-gated keep direction,
batched review, fail-safe parsing, alert_autoresolve usage tagging).
Production telemetry (7d, 2026-07-17) shows a persistent tail of live
installs on 6.0.0-rc.x and 6.0.3/6.0.4 while 6.0.5 has been stable since
2026-07-09. Root cause for the reachable stranding class: release
selection trusted GitHub's created_at ordering, and this repo interleaves
v5-line maintenance releases with v6 releases (v5.1.36 was created the
day before v6.0.5). Whenever a v5.1.x release is the most recently
created stable:
- stable channel (getLatestReleaseForChannel) returned the first
non-prerelease in list order -> "no update" for every v6 install
until the next v6 release ships
- the RSS rate-limit fallback returned the first feed entry matching the
channel, same failure, both channels
- the unattended updater trusted /releases/latest, which GitHub defines
as most-recently-created stable -> timer no-ops in the same window
All three now select the highest eligible version: the API path tracks
newest stable + newest prerelease in one pass (prerelease = GitHub flag
OR tag shape), the feed fallback picks the max matching the channel, and
pulse-auto-update.sh scans /releases pairing each tag with its own
draft/prerelease flags before the fail-closed shape filter, keeping
/releases/latest as fallback. helm-chart-* tags fail semver parsing and
are dropped everywhere.
Verified not broken (pinned by new tests): rc-channel installs are
offered the newer stable (6.0.0-rc.6 -> 6.0.5) and move onto the next rc
line when one opens (-> 6.1.0-rc.2); the auto-update prerelease filter
(fail-closed is_prerelease_tag, metadata-flag refusal) still refuses
prerelease targets on stable. Go tests fail on the pre-fix code
(returned 5.1.37 / "no update").
Contract: deployment-installability now pins version-max release
selection with proofs in manager_stranded_upgrade_test.go and
pulse_auto_update_test.go.
Telemetry shows thousands of installs configure an AI provider but almost
none ever use the interactive Assistant. A live first-session exercise
(fresh install, Ollama qwen3:8b quickstart) found why: after enabling,
nothing changes on screen — the launcher and handoff buttons are gated on
sessionCapabilities.assistantEnabled, which was only read at page load;
the empty transcript was blank; and the blessed Ollama+qwen3:8b path
reported Patrol degraded while telling the user to pull the model they
had just selected.
- Setup-modal success now opens the Assistant drawer, and the AI settings
save paths refresh the assistantEnabled capability in place
(aiChatStore.refreshEnabledFromServer) so entry points appear without a
reload; toasts point at the Assistant instead of back at settings.
- The empty transcript owns a plain-language welcome and three suggested
prompts that dispatch as real turns (ASSISTANT_SUGGESTED_PROMPTS).
- Patrol static readiness: the blessed Ollama Patrol model is Ready;
other Ollama models keep the warning, now naming the selected model.
- pulse_summarize fleet argument errors instruct the model to enumerate
resources itself instead of interrogating the operator (observed live:
'how is my machine doing?' ended in a resource-ID elicitation).
Contracts: ai-runtime and frontend-primitives Current State updated.
Tests: full internal/ai + internal/api suites green; vitest ChatMessages,
AISettings, aiChat store, and settingsArchitecture suites green; flow
verified live end-to-end.
rcourtman