Patrol's operator-state projection dereferenced the one-shot
MaintenanceStartAt/EndAt whenever IsInMaintenanceAt was true. A
recurring window also reports active but leaves those fields nil, so
the first finding Patrol evaluated during a recurring occurrence
panicked the provider. The projection now takes its bounds from the
active occurrence.
Alerts already silence guests under a host window scoped to
resource_and_descendants, but Patrol read only the exact resource, so
the same planned work dismissed a guest's alerts and still raised its
Patrol findings. Both now resolve maintenance through one rule,
unifiedresources.ActiveMaintenanceCoverageAt: own window plus
descendant-scoped ancestor windows, latest active end wins. Only
maintenance is inherited; monitoring mode, lifecycle and remediation
locks stay exact-resource state. An inherited dismissal says the window
is on a parent resource and names it on the lifecycle event, and the
Manage help now says descendants' new Patrol findings are dismissed
too.
Expected offline is availability-specific everywhere else: the canonical
ResourceOperatorState doc says it "suppresses availability noise", the
alert runtime suppresses only offline and availability signals, and both
monitoring-policy controls tell the operator it hides availability noise
only, with Mute all as the way to stop Alerts and Patrol. Patrol was the
outlier. Its projection returned a suppression cause for expected_offline
(and the legacy IntentionallyOffline flag) regardless of the finding, so
a disk, memory, backup or security finding on an expected-offline
resource was auto-acknowledged exactly as if the resource were muted.
The suppression decision is now per finding. Retired and muted still
silence every finding; expected_offline and the legacy flag (when no
MonitoringMode is projected, as in the alert runtime) silence only
findings in the shared down condition class that alert mirroring already
uses. The wake path judges the re-detected finding with the same helper,
so findings auto-acknowledged under the old behaviour wake on their next
detection.
That classifier matched "offline" in any reliability title, which would
have read a SMART "offline uncorrectable sectors" report as the resource
being down and hidden a failing disk. SMART findings now fall outside the
down class, which also stops them mirroring an offline alert.
A new test pins the split (availability silenced; disk, memory and SMART
findings visible) with muted as the negative control, plus a muted to
expected-offline transition that wakes only non-availability findings.
Each test fails when the old behaviour, the wake-path scoping or the
SMART guard is reverted.
Keep selection on the registry ID through context, sessions and command
routing so same-name resources cannot cross provider boundaries.
Retain native executor coordinates, legacy inputs that resolve uniquely,
policy redaction and paid execution modes.
Refs #2681
Pro sells Patrol investigation, but every install starts in Watch only
and stays there unless the operator finds the mode control. In the paid
telemetry read on 12 October, 20 of 35 Pro installs with Patrol on ran
Watch only, and none of those 20 ran an investigation in 30 days, so
the feature they paid for never ran.
An install that never saved a Patrol mode now runs Ask first when it
holds a positive investigation licence. Patrol investigates new and
re-detected warning and critical findings and proposes a fix; nothing
changes without approval. The default only applies to a positive
licence answer: a missing licence checker counts as unlicensed, so a
startup window cannot spend provider budget on a free install. Free
installs and explicit choices are unchanged.
The Patrol header says so once, with the cost (the operator's AI
provider and 30-day budget) and two actions that save Ask first or
Watch only. The notice never renders on a plan-locked install. The
header also stops claiming "Watching your infrastructure" while Patrol
cannot run yet and reads "Patrol is not running yet" instead.
Reverts #2863 before any release ships it. The live Pro pricing and
Patrol pages, in English, German and Spanish, sell both modes ("applies
safe fixes in the mode you choose: Ask first, Safe auto-fix, or
Autopilot"), and paying installs chose them. Shipping the retirement
would have silently downgraded those installs while the sales page still
promised the modes.
If the modes are retired, that needs a dated notice: stop selling them
first, warn installs on those modes in-app, and then re-ship the change.
It should not arrive through an engineering cleanup.
The receipt #2863 landed stays in place; this commit adds the receipt
for the restored content.
Patrol could run some fixes on its own in two paid modes: assisted (Safe
auto-fix) and full (Autopilot, behind a server-owned acknowledgement).
Schema-v17 telemetry for the week to 2026-10-11 counts 1 assisted and 2
full installs out of 612 reporting Patrol, 4 executed Patrol actions in
30 days, all on one install, and no GitHub request for more autonomy,
while the automatic path carried its own policy leases, acknowledgement
store, per-resource allowlist, release gate and Colima lab. Investigation
is what paid installs use, so it stays.
Patrol now has two modes. Watch only reports findings. Ask first
investigates and proposes fixes, and every fix waits in Actions for an
operator to approve and run it.
- A saved assisted, full or autonomous level reads as approval, and the
update routes refuse those levels with 400 invalid_autonomy_level.
- The core Patrol action broker only plans: Plan and Submit persist the
plan and return. ExecuteUnderPolicy, the policy authorization lease,
the policy admission store methods, the Autopilot acknowledgement
persistence, routes and dialog, and the per-resource automatic-action
allowlist (store column now written as NULL) are gone.
GetPatrolFullModeUnlocked stays on the extension contract and reports
false.
- Historical action records keep their policy lease and tenant/resource
reason codes decodable, labelled as retired modes.
- The Patrol inbox no longer splits issues into "continuing without a
decision"; every current issue is an operator decision.
- Release gate RG-06 now proves Patrol never executes without an operator
decision (integration tier) instead of limited unattended autonomy,
and RG-02/RG-04 point at the surviving human-admission proofs.
- Pro plan copy, AI docs, privacy telemetry wording and the subsystem
contracts describe the two modes.
Compose current main e2f4bdff03 additively. Retain both subsystem records and the held change-age control while withdrawing only the obsolete Discovery identification test. Reviewed identities and ancestry, canonical guest History ownership and upstream feature removal remain intact.
Change-source: pulse-maintainer
AI Discovery ran a separate model-backed service scanner beside Patrol
and the Assistant: its own store, scheduler, model route, API, tool and
drawer UI. No telemetry counter ever measured its use (the telemetry
discovery_enabled flag OR'd it with network subnet discovery), the
maintainer judged it a flop, and the context it produced is already
available to the models through pulse_query, pulse_read and the
unified-resource context sections.
This deletes internal/servicediscovery, internal/ai/infradiscovery, the
pulse_discovery tool, the /api/discovery routes, the mock Discovery
fixtures, the Discovery tabs in the resource, guest, node and Docker host
drawers, the identified-service cards, readiness badges, provenance
markers, suggested web-interface URLs, the Workloads AI Context column,
the chat "Discovery is off" hint and the AI Service Context settings page.
The old settings route redirects to Assistant settings. The model-bound
sanitizer allow-list existed only to pass Discovery context through, so it
goes too, along with the CopyableCodeRow primitive that only the
Discovery tab used.
Network subnet discovery (/api/discover, the Infrastructure discovery
settings) and the discoveryTarget agent coordinates are separate features
and stay. Stored discovery_enabled, discovery_interval_hours and
discovery_model AI settings are ignored on load; the settings API decoder
is lenient, so older frontends saving those fields still succeed.
The retained race attempt reached the completed agent loop but exceeded the five-second service deadline. Keep that adverse result and the deadline; cancel and observe the worker before temporary-session cleanup, and retain goroutine stacks if either completion or cancellation fails. This prevents failure-path work from silently outliving its fixture and makes the unresolved stall diagnosable.
Contract-Neutral: Test-only cancellation, bounded cleanup and failure diagnostics; production streaming, persistence, steering and completion deadlines are unchanged.
Change-source: pulse-maintainer
The complete offline Assistant suite exposed a live models.dev fetch in a unit test. Use the existing URL-injection seam with a loopback JSON fixture, assert every decoded provider and fuzzy cache key, and verify TTL reuse and recent versus old model classification. Isolate both tests from the singleton without changing production code.
Contract-Neutral: Test-only loopback fixtures replace live network and catalogue dependence with stricter cache assertions; no runtime, API, model-catalogue or product behaviour change.
Change-source: pulse-maintainer
pulse_control only plans typed resource actions into Pulse Actions
(executeControlResource -> typedActionPlanner). The handlers that sat
behind the retired model-authored command, file and container paths
stayed in the tree with no production caller: run command, guest
control, app-container control, Docker control and container update,
Kubernetes scale/restart/delete-pod/exec, and file append/write.
registerFileTools was never called either, so pulse_file_edit is not
registered at all; calls to it are denied by the mutation registry
before any tool lookup.
A whole-program reachability run over cmd/... lists each of them as
unreachable, and so is everything only they used: approval-record and
preflight authoring, the command and native-action audit lanes with
their remediation-lock gate and plan-hash helpers, the Docker host
resolvers, and the file write verification. The executor's autonomous
flag, whose only readers were those handlers, goes with them (the
executor's SetAutonomousMode and the flag argument of SetContext).
Delete them with their tests and the constants, types and test helpers
they stranded. The contract prose that described the removed broker now
says that plan freshness and the operator remediation lock are enforced
by internal/actionlifecycle. Two source pins that named deleted code go
(the publishActionCompleted dispatch sites and the pulse_file_edit
declaration), the standards tests drop their expectations of an
ActionPlan construction in action_audit.go and of the legacy app_container
rejection in the deleted file-edit test, and the RA35 evidence entry for
the deleted run-command test goes with them.
Left alone on purpose: the saved autonomous control level and its
entitlement and approval-admission checks, which stay as legacy
compatibility, Service.SetAutonomousMode and the chat clamp,
SetOnActionCompleted and its router wiring (no dispatcher is left, the
contract prose says so), and the protected-guests plumbing.
Rebuilt on current main: the first version had 213 commits of drift.
Two conflicts, both on code this change deletes. tools_control.go takes
the deletion over the Protected guests cleanup that landed inside
executeRunCommand. action_audit_remediation_lock_test.go goes with the
audit lanes, including the test added for the dispatch gate's read
through a succession; that gate is deleted here, and
TestLockMovedBySuccessionStillRefusesTheIDThePreviousGenerationLists in
internal/actionlifecycle covers the same refusal on the live path.
Snapshot recovery must preserve identity, lifecycle events and acknowledgement fields. Canonical query coverage uses a fresh observation window on each read and is deliberately not checkpointed.
Change-source: pulse-maintainer
Eight imported snapshots caused eight filtered history queries and thirty-six identity reads although replay discards each result. Share the unchanged occurrence repair with a write-only materialisation entrypoint; keep EnsureAlertOccurrence and user queries on their existing canonical read-repair path.
Preserve snapshot fallback, bounded count and age retention, checkpoint recovery and read-error behaviour. Shutdown worker ownership and the reported memory baseline remain separate outcomes.
Contract-Neutral: Internal replay cost correction only; alert occurrences, canonical history, checkpoint and retention formats, user reads and notification policy are unchanged.
Change-source: pulse-maintainer
A rebuild applies canonical-ID successions to the store, which moves the
predecessor's operator row to the successor, before it publishes the
registry generation that lists the successor. For the length of the
rebuild the previous generation keeps serving the old ID, and the planner,
the Patrol broker and both dispatch gates read the row of that ID. The row
had gone, so a lock the operator set did not refuse an action planned or
dispatched against it. Every kind of succession reaches this window:
pin-driven, Proxmox guest eras, and the host and Docker reporter eras.
GetResourceOperatorState now follows the recorded successions when an ID
holds no row, in both stores, through one walker. It answers as the ID the
chain ends on would, with that row's remediation block alone, so a lock
the operator lifted on the successor stays lifted, a chain that loops
ends, and nothing is written. The SQLite walk is one transaction, because
a succession that commits between two of its statements can move the lock
onto an ID the walk has already passed.
Delaying the move was rejected: later steps of the same rebuild read the
store, and plans already issued against the old ID would not be
protected. Seven tests that used a missing read to prove a row had moved
now read the rows directly. An operator edit to the retired ID inside the
window still does not reach the successor; the contract names it as a
limit.
Settings > Assistant offered a Protected guests list (protected_guests)
that was saved and pushed into the Assistant tool executor, but its only
reader was executeControlGuest, which nothing calls. pulse_control plans
through the shared action lifecycle and never consulted the list, so a
guest named in it (by VMID and by name) was planned like any other. The
"legacy" label added in #2765 states that limit; this removes the
setting instead of keeping a field that cannot do its job.
The canonical way to keep Pulse off a resource is its operator lock
(Never auto-remediate, or a Retired lifecycle). It is keyed by canonical
resource id, applies to every platform, and the action lifecycle refuses
dispatch for it even for approved plans. The one gap was planning: the
lifecycle plans without reading operator state, so the Assistant could
leave a plan in Actions that can never run. assistantTypedActionPlanner
now reads the state through the tenant-scoped lifecycle store (the same
read Patrol's broker uses) and declines before Plan, and pulse_control
reports the block with how to clear it. A state that cannot be read does
not refuse planning, because dispatch re-checks.
Removed: AIConfig.ProtectedGuests and its getter, the settings API
field, the executor field/setter/config, the chat service plumbing, the
Settings field and form state, and the legacy copy, pins and prose from
#2765. A stored value still loads (tolerant decode) and is dropped the
next time the settings are written; older PUT bodies that send the key
are ignored. The Settings panel points at the guest's Manage tab.
Merge exact Web tip b908be2e55 with launch upstream 0289e8b648. Preserve both independent memory-evidence and Assistant-control contract sections in two internal documentation conflicts; all reviewed frontend content remains byte-identical. Retained sensitive race, browser, build and completed source observations are adequate for the changed control boundary. Broad affected-package race remains unproved after its recorded observation timeout; no aggregate pass or publication is claimed.
Change-source: pulse-maintainer
The current canonical resource planner does not consult the retained guest list. Label that compatibility data honestly and link its accessible limitation without discarding saved policy. Keep the review-only mode menu within the composer on phones, with geometry and resize controls.
Reconcile the old autonomous-mode test with the explicit core-owned investigation flag, preserving saved preferences and legacy approvals. Mounted settings, parent-discriminating copy/menu checks, help mirrors and final-byte browser receipts cover the existing flow.
Change-source: pulse-maintainer
Use used-byte fixtures consistent with each selected percentage and assert that actual get results keep percentage, bytes and total together. This strengthens the same-sample positive controls without changing runtime code or the parent-proven qualification boundaries.
Change-source: pulse-maintainer
Carry selected memory source, state and original time through existing queries, Assistant context and condensed facts. Missing usage remains unknown, measured zero stays numeric, and resource refreshes cannot renew a local observer's retained memory sample. Historical numeric percentages retain explicit source limitations. Shared qualification keeps the existing Patrol, collector, threshold and guest-command contracts intact.
Change-source: pulse-maintainer
Keep the Read-only description within the existing copy rules. Bind plan-only checks to the actual canonical fixture row, and distinguish the existing informational disabled-mode reply from unauthorised execution errors while still requiring zero planner/native calls.
Record a fresh parent-bound browser pass for the changed description; existing runtime and saved-policy semantics are unchanged.
Change-source: pulse-maintainer
Contract-Neutral: Copy-style punctuation correction and synthetic plan fixture alignment only; no control, settings, entitlement, approval or execution contract delta.
Project interactive Assistant controls to Read-only and Ask first without migrating legacy stored preferences or relaxing entitlement, legacy command approval, scoped execution or Patrol policy. Preserve stored policy even when an older client echoes the projected mode on an unrelated save. Explain saved Actions plans and operator review in existing Settings, Chat and help.
Add saved-client and encrypted-config compatibility, authenticated plan-only access, active-stream and scope regression controls with parent comparisons. Record inspected final-byte desktop/phone browser proof and the affected subsystem contracts.
Change-source: pulse-maintainer
Respect the existing guest-analysis opt-out when adding the bounded uncertainty section. Keep typed-read-state tests as real current cache-aware positives and assert the complete qualified row rather than an unlabelled percentage. Retain the first full-suite failures as evidence; do not change thresholds, bypass unknown provenance or replay the completed parent comparison.
Change-source: pulse-maintainer
Carry selected memory source, original observation and absence through full and condensed seeds, deterministic flags, forecasts, anomalies and recovery verification. Cache-inclusive hypervisor estimates and retained readings cannot prove guest pressure or recovery. Keep measured zero, real pressure and unrelated platform readings, and use native guest runtime state for eligibility.
Add connected parent-comparable controls for issue #2762 and preserve existing collection, safety admission, thresholds and History. This is source interpretation repair, not a claim of installed cause or recovery.
Change-source: pulse-maintainer
Preserve exact retained Core ancestry while retaining the accepted Assistant target, webhook confidentiality, log-viewer and installer repairs. Qualify the composed source against the original seven-package, inventory-control and real embed/API/Patrol recipe; source acceptance remains separate from native recovery.
Change-source: pulse-maintainer
Bind session-derived targets only when current inventory has one canonical match. Preserve explicit canonical IDs over stale session aliases and return conflicting candidates before planning. Keep unique lookup, session registration, capability policy, approval and execution boundaries intact. Listener-free control tests and exact-parent regressions verify this repair; it does not establish native recovery.
Change-source: pulse-maintainer
Preserve the exact prior Core history alongside current canonical Patrol resource-owner resolution and recovery cleanup. Runtime source retains the TrueNAS alert-policy repair and sequential repeated-name Docker assertions; current bytes require isolated qualification.
Change-source: pulse-maintainer
Merge exact upstream 7f466fc183 without re-parenting its reviewed history. Retain both independent contract sections in the two documentation conflicts; no frontend resolution or runtime correction.
Change-source: pulse-maintainer
Host filesystem alerts identify a disk child that Patrol does not collect as
a separate resource. Bind an exact collected alert to its current canonical
owner through the shared inventory contract, including linked agents, and
retain its filesystem, threshold and breach evidence in the scoped prompt.
Keep missing owners, ambiguous source identities and uncollected child
subjects fail-closed. Add evaluator-to-handler, repeated scope resolution
and browser admission coverage.
Reconcile the absent runtime portion of PR2693 aaf5398480 on current assigned source. TrueNAS disk heat follows its disk override, TrueNAS-wide default and inherited per-type policy rather than the system synthetic agent ID. Preserve agent-host overrides, Off, clear thresholds, alert switches, retained-reading limits and canonical identity. Include discriminating resolver, adapter, Patrol and Assistant tests; leave the frontend stack to Web.
Change-source: pulse-maintainer
Reject ambiguous names and VMIDs before reading provider configuration, honour partial placement constraints, and read current inventory rather than stale state. Use the chosen typed guest view for policy metadata so shared names cannot select another installation. Preserve direct reads, original provider errors, guest safety and authority; cover both VM and LXC paths.
Change-source: pulse-maintainer
Additively incorporate exact upstream main 533027ee06 without rewriting accepted maintenance. Resolve the storage-recovery contract insertion by retaining both the reviewed silent-source History fence and upstream SAML/method constraints. No runtime conflict resolution is introduced; final-content evidence and exact-head checks remain required before publication.
Change-source: pulse-maintainer
## Why
Since 4edf5f8265 (2026-07-11), `ChatRequest` in
`internal/api/ai_handler.go` has no `autonomous_mode` field. Go's
decoder drops it and `HandleChat` passes approval-required mode into the
chat service for every `/api/ai/chat` request (pinned by
`TestHandleChat_CannotElevateAutonomousModePerRequest`). The browser
still put `autonomous_mode` in the request body from its send options,
and the eval runner kept a per-step override that sent it too. Both read
as if the client chose the execution mode, but the server ignored them.
## What changes
- `AIChatAPI.chat` loses the `autonomousMode` positional parameter and
no longer writes `body.autonomous_mode`.
- `SendMessageOptions.autonomousMode`,
`ChatMessageRequestContext.autonomousMode`, and the clone, retry,
restore and Explain send-option copies are removed.
- `aiChatStore.context.autonomousMode` stays as a drawer-only flag.
Scoped handoffs still show "Chat: Ask first" and "Approval required
before any action." and hide the chat-actions warning.
- `internal/ai/eval` drops `Step.AutonomousMode`; the envelope test now
asserts the field is absent.
- `scripts/check-patrol-assistant-journey.mjs` asserts the Explain
request has no `autonomous_mode`.
- Contract prose (ai-runtime, api-contracts, agent-lifecycle,
storage-recovery, patrol-intelligence, frontend-primitives) no longer
describes `autonomous_mode:false` as a per-request override sent by the
browser. It now describes the server-set mode and the drawer disclosure.
No backend runtime change. The handler test proving a stale client's
`autonomous_mode:true` cannot elevate a request stays.
## Verification
- Private mock stack built from this branch, 1440x900 and 390x844. Every
`/api/ai/chat` body was captured with a page-level fetch wrapper.
- Plain chat with the saved level at Chat actions: body is `{prompt}`.
The pill and warning are unchanged.
- Patrol "Explain with Assistant": body keys are `handoff_context`,
`handoff_resources`, `prompt`. The drawer shows "Approval required
before any action." and "Chat: Ask first", with no warning.
- Follow-up turn: body keys are `model`, `prompt`, `session_id`, and the
disclosure persists.
- `go test ./internal/ai/eval/` passed.
- On pulse-dev: `tsc --noEmit` passed. `vitest` over
`src/components/AI`, `src/api/__tests__` and the aiChat store test
passed 2531 tests. In that parallel sweep, `AIChat.test.tsx` hit its 10s
`beforeAll` import timeout; run alone it passed 211/211.
- Read-only second-opinion review found no runtime regressions. It
flagged two contract sentences (undo/retry restoring an
"approval/autonomous override", and an overbroad "the body carries no
field"); both are fixed in this commit.
Additively integrate exact upstream 1a094646d7 while retaining starting reviewed 19a8358a49. Keep reviewed UUID incident/event IDs and upstream non-wrapping atomic sequences for other memory IDs. Compose upstream shared SMART/Unraid temperatures with reviewed pending context and observation-gap interruption; pin Unraid-only continuity and both grace paths. Retain all browser, completion and security validation limits until evidenced.
Change-source: pulse-maintainer
Since 4edf5f8265, ChatRequest in internal/api/ai_handler.go has no
autonomous_mode field: Go's decoder drops it and HandleChat passes
approval-required mode into the chat service for every /api/ai/chat
request. The browser still serialized autonomous_mode from its send
options, and the eval runner kept a per-step override that sent it too,
so both read as if the client chose the execution mode while the server
ignored it.
Drop the field from AIChatAPI.chat, SendMessageOptions,
ChatMessageRequestContext and the clone, retry and restore helpers, and
Step.AutonomousMode from internal/ai/eval. aiChatStore.context's
autonomousMode stays as a drawer-only flag, so scoped handoffs still show
"Ask first" and "Approval required before any action." Tests now assert
the request carries no autonomous_mode; the backend test proving a stale
client's autonomous_mode:true cannot elevate a request stays.
Contract prose that described autonomous_mode:false as a per-request
override set by the browser now describes the server-set mode and the
drawer disclosure.
Each tenant monitor owns its own IncidentStore, but incident and
incident-event IDs came from package-level int64 counters bumped with ++
under nothing but the calling store's mutex. Two tenants recording
alerts at once raced on them: go test -race failed
TestLeavingMockModeClearsEveryRunningTenantMonitor on PR 2698 at
incidents.go:1753. The remediation record counter had the same shape,
and remediation logs are created per org.
All six internal/ai/memory ID generators now mint through one atomic
helper. The numeric suffix also stops wrapping at 1000: even with atomic
counters, one generator minting more than a thousand IDs in a
wall-clock second repeated an ID, and incident notes find their incident
by ID. The prefix and timestamp layout is unchanged, and intToString
lost its last callers.
ids_test.go mints 2,400 IDs from every generator on 16 goroutines and
records alerts into two tenant stores concurrently. Against the old
files it reports DATA RACE at incidents.go:1753/1760 and
remediation.go:355, and every generator mints duplicate or wrapped IDs.
Separate tenant stores previously raced on process-wide counters and reused identifiers every thousand events in one second. Use the existing UUID dependency without mutable generator state. Cover concurrent bursts, cross-store lifecycle and by-ID note ownership, plus legacy JSON references and reloads.
Contract-Neutral: Correct uniqueness of existing opaque incident and event references; prefixes and persisted legacy references remain supported without API or schema changes.
Change-source: pulse-maintainer
#2636 made the alert disk temperature policy the only judge of disk
heat, but it resolved that policy by disk type alone. Disk temperature
alerts (`CheckHost`) evaluate each SMART disk under the thresholds of
the host agent that reports it. An explicit Disk Temp override on that
host, or one inherited from its linked node or guest, replaces the
per-type trigger, and a host whose alerts are switched off raises
nothing. So a host with an 80C Disk Temp override still read **Running
Hot** on Physical Disks at the 70C NVMe trigger. Patrol flagged it too,
and a host with alerts switched off still read hot. The Alerts
thresholds page promises the opposite: "Setting a Disk Temp override on
a host above replaces these for all of that host's disks".
## What changes
- **Backend policy:** `alerts.Manager.HostDiskTemperatureThreshold(host,
diskType)` resolves the policy for a host the way `CheckHost` does, from
the same helpers (`resolveHostThresholdsNoLock`,
`hostThresholdOverrideNoLock`, `hostDiskTemperatureThresholdNoLock`). A
disabled override gives no threshold.
`DiskTemperatureThreshold(diskType)` is now its hostless case.
- **Patrol and the AI chat context:**
`ThresholdProvider.GetDiskTemperatureThreshold(host, diskType)`. Each
unified physical disk finds its host by walking its parent to the agent
machine, through the pool for Unraid array and cache disks.
- **Physical Disks, the disk detail, the pool drawer's linked disks and
the guest drawer** pass the reporting machine's override keys to
`resolveDiskTemperatureDisplayThresholds`. Keys follow `CheckHost`'s
order: agent ID, canonical resource ID, then the merged node or guest.
The resolver returns null (uncoloured, never hot) for an override or
agent default that switches the host's alerts off.
The browser rebuilds `CheckHost`'s key chain from what it loads, so a
few identity edge cases can still differ from alerts and Patrol (listed
in `storage-recovery.md`): on the Storage page, a disk whose agent an
operator manually linked into a VM; an override inherited through agent
link hints the API does not publish (an unmerged linked node or guest),
or stored under a linked node's own canonical ID; and guest keys in the
guest drawer for an agent merged into a guest it is not automatically
linked to. Overrides set on the agent's own row, or on the node or guest
it is merged with, reach every surface.
## Verification
- `TestHostDiskTemperatureThresholdMatchesCheckHostOverrides` checks
each case against what `CheckHost` actually fires: no override, host
override, host override without Disk Temp, an override under the
canonical resource the agent ID resolves to, linked-node and
linked-guest overrides, a host override shadowing a node one, a
switched-off Disk Temp, and a disabled host, linked node or linked
guest.
- `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride` uses a
real registry with two agents, one overridden. A mutation that drops the
owner lookup fails it.
- Frontend tests cover the resolver, the owner walk and key order,
DiskList with three hosts, the presentation map, pool-drawer linked
disks and the guest drawer.
- Private mock stack: three NVMe disks pinned at 72C. Before (main
frontend): all three read Running Hot. After:
- pve1, with an 80C override, reads Healthy (green).
- pve2 still reads Running Hot.
- pve3, with alerts off, is uncoloured.
- Moving pve2's override onto its linked node ID turns it Healthy.
- Checked at 1440 and 375 with no overflow. The receipt is in
`frontend-modern/browser-verification/`.
- A read-only second-opinion review ran twice. Round 1's findings
(frontend key order, guest drawer canonical ID, disabled agent default,
doc wording) are fixed in the first commit. Round 2's are the parity
cases and the documented browser limits in the second commit.
The storage pool rows the AI tools build (pulse_storage pools and
pulse_query storage pools) derived both enabled and active from "status
is not offline". Storage that reported itself unavailable, such as
Proxmox storage with active=0, reached Patrol and Assistant as
administratively disabled even when it is enabled in Proxmox.
Enabled now comes from the storage facet's own flag. Active needs the
facet's flag and a row that is not offline, because PBS datastores
report Active even while unavailable. Rows without a storage facet keep
the status-derived value.
TrueNAS pools and datasets were the only storage producers that left
both flags at their zero value, so reading the facet alone would have
turned every TrueNAS pool into a disabled one. They now report Enabled,
and Active from the pool state or the dataset's mount and lock state.
Disk temperature alerts evaluate each SMART disk under the thresholds of
the host agent that reports it. An explicit Disk Temp override on that
host, or one inherited from its linked node or guest, replaces the
per-type trigger, and a host whose alerts are switched off raises none.
Since #2636 the Physical Disks verdict and Patrol judged heat by disk
type alone. A host with an 80C override still read Running Hot at the
70C NVMe trigger, and a host with its alerts off still read hot.
- alerts.Manager.HostDiskTemperatureThreshold resolves the policy for a
host exactly as CheckHost does, from the same helpers.
DiskTemperatureThreshold is its hostless case.
- Patrol and the AI chat context find each disk's host by walking its
parent to the agent machine (through the pool for Unraid disks) and
ask the threshold adapter for that host's limits.
- Physical Disks, the disk detail, the pool drawer's linked disks and
the guest drawer pass the reporting machine's override keys to the
thresholds resolver, in the order CheckHost reads them: agent ID,
canonical ID, then the merged node or guest. The resolver treats an
override or agent default that switches alerts off as no heat.
Two cases still fall back to the per-type trigger in the browser while
alerts and Patrol apply the override: on the Storage page, a disk whose
agent was manually linked into a VM, and an override an agent inherits
from a guest it is linked to without a merge.
PVE disk health and wearout alerts reference the disk by instance, node
and device path, so a physical disk's history never showed its own SMART
and wearout alerts. Binding the path reference failed review: a binding
belongs to the reference string, so any disk that takes over the path
inherits every row and every read of it.
Own each lifecycle row instead. The alert now records the disk's WWN
next to its serial, and the history writer puts the row under the disk
the registry knows by that hardware identity, wherever it sits now. The
registry only translates identity into a canonical ID, so a generation
that still places another disk at the path cannot claim the row, and no
check precedes a later write. A disk not yet in inventory, or removed,
gets the ID the registry mints for its identity. Rows without usable or
unambiguous identity stay under the reference, as do rows journaled
before this change; journal rows are never rewritten.
Reads by the reference still return every row raised under it, through
the alert identifier index, and the incident projection keeps owned rows
under their alert reference, so the Alerts history Resource action and
incident timelines show the same occurrences as before. A read by the
disk returns its own occurrences with their saved shells and notes.
Follows #2591. A node alert that moves to its linked Pulse agent closes
with a `moved_to_agent` resolution (#2580). Two readers still presented
that close as a recovery.
**Performance reports.** The node, VM and container enrichers copied
recently resolved alerts into `reporting.AlertInfo` without the
resolution, so a report generated in the five-minute window after a
handover read HEALTHY, "All systems operating normally", counted the
alert as "triggered and resolved", and sent the AI narrator `resolved:
true`.
- Report rows now carry the alert's resource and resolution
(`reportActiveAlertInfo` / `reportResolvedAlertInfo`).
- `pkg/reporting` has one rule (`unresolvedAlerts`): an alert that
closed without recovering stays live unless the report also shows its
successor's reading for that metric, meaning an open alert or a recovery
since the handover. That way the condition is counted once, and fleets
match handovers across all their resources.
- The single-resource PDF card reads WARNING (or CRITICAL at the alert's
level) with "Alert moved to <agent> - check the agent for the current
reading". The active list names the agent, the alerts table shows
"Moved" with the engine's summary as a note, and recommendations say to
check the agent.
- Fleets count the alert as active and name the agent. Their all-clear
pattern and "No fleet-wide action required" no longer appear while any
alert is active.
- The narrator payload carries `resolved: false`, the summary and
`successor_alert_listed`, with prompt rules for both.
**Resource history.** `alert_resolved` changes were labelled by kind
alone ("Alert resolved", green) with that label prefixed to the
headline. `getResourceChangePresentation` now reads the
`alert_resolution` metadata: the change reads "Alert moved" in the
neutral blue of the history badge, and its summary is the whole headline
in the drawer, `ResourceChangeSummary` and Patrol's assessment handoff.
**Verification**
- A private mock stack running this branch, with a local-only overlay
that closes a real pve1 memory alert through `clearAlertWithResolution`.
Within the window, `/api/admin/reports/generate?resourceType=node`
rendered CRITICAL with the handover message and a "Moved" row, while the
ordinary CPU recovery stayed green.
- In the browser, a full drawer (Availability checks, Fleet view) showed
the producer-made rows as "Alert moved" next to "Alert resolved" at 1280
and 375. A Proxmox node's own drawer uses the `table-row` presentation,
which fetches no change history (see below).
- Tests in registered proof files: `pkg/reporting/alert_state_test.go`
(verdict, successor timing, fleet pool, PDF text and width),
`internal/api/contract_test.go`, `internal/ai/report_narrator_test.go`,
`resourceChangePresentation.test.ts`,
`ResourceDetailDrawer.history.test.tsx` and
`patrolInvestigationContextModel.test.ts`. Each failed with the fix
reverted.
- Second opinion: two read-only review rounds. The temporal match,
fleet-wide pool, narrator dedupe signal, fleet all-clear fallbacks and
contract wording came from them.
**Not fixed here (separate changes)**
- Reports requested as `agent` carry no alert rows at all. That is how
the Reports page addresses every Proxmox node in v6, so such a report
reads HEALTHY even with active critical alerts. This change makes the
shared rule correct for when those reports attach the machine's alerts.
- Node and guest alert lifecycle changes are recorded under legacy alert
resource IDs, which canonical history queries never match. Facet kind
counts still group moved closes under "Alert resolved".
Patrol verifies a disk-high finding as recovered once the disk's
temperature cools to its alert clear value. A disk in standby, or one
whose host agent stopped reporting, has no current reading, and a
missing reading counted as cooled. A finding raised for a SATA disk at
56C would close as soon as the disk spun down, while its temperature
alert stayed open.
When a disk has no current reading and its last known reading is still
at or above the clear value, verification now reports the recovery as
unknown instead of proven. A disk last seen cool still recovers.
Pulse decided whether a physical disk was too hot in three places with
three thresholds. Disk risk added temperature_high at a flat 60/70C for
every disk type and drove the disk status and the Physical Disks Health
verdict. Patrol flagged anything over 55C and the AI chat context listed
disks from 50C. Alerts and the Temp column used the per-type thresholds
users tune in Alerts (NVMe 70, SAS 65, SATA 55 by default). An NVMe at
63C read Running Hot beside a green Temp cell. A SATA disk at 56C read
Healthy beside a red one. A user who raised the NVMe trigger to 75 still
got a hot verdict at 70.
The alert disk temperature policy now decides disk heat.
alerts.Manager.DiskTemperatureThreshold exposes the per-type resolution
CheckHost already applies.
- Disk risk no longer judges temperature. Every registry that rebuilds
risk lacks the alert configuration, so no rule there could honour it.
A hot disk keeps an online status.
- The Health verdict reads Running Hot when a current reading reaches
the disk type's trigger. It uses the thresholds object that colours
the Temp cell, so the two cannot disagree. The amber band below the
trigger stays a cell colour, not a verdict.
- The TrueNAS storage table lost the heat it used to get from risk. It
now judges heat the same way: an Attention row with the reason, and a
warning dot for phones.
- Patrol triage, its disk issue filter, finding recovery and the chat
context's attention list judge the collected reading against the same
trigger, including scoped runs and a provider set before Patrol
starts.
- A switched-off agent Disk Temp default resolves to no thresholds,
which the displays that read them now treat as off.
Per-host Disk Temp overrides do not reach these surfaces yet. TrueNAS
disk alerts, the TrueNAS disk drawer and PDF reports keep their own
fixed temperatures for now.
A container reported without an ID alerted under its Docker host's own
reference (docker:<host>). Its alerts showed as host alerts, joined the
host's resource history, were never cleared when the container left the
report, and made the host's Patrol scope alias ambiguous. A Swarm service
without an ID put its normalized name in the service-ID slot, so a name
equal to another service's ID shared that service's alerts and history,
and the history resolver had to treat the whole cluster as a conflict.
Both now alert under their name, marked in the ID position
(docker:<host>/name:<name>, docker:<host>/service/name:<name>). Docker
and Swarm IDs never contain a colon, so these cannot equal a host or ID
reference, and one with neither an ID nor a name raises no alert. The
policy switches, cleanup, Patrol aliases and the history resolver use the
same builders and parse the same shapes, so the resolver no longer needs
the cluster-wide conflict. A pre-upgrade container alert left on the
host's reference is cleared on the host's next report, and a pending
image update recorded there keeps its age.
Canonical state IDs were split at the first "::", but Docker hosts that
collide are disambiguated as "<base>::<suffix>". Every container alert
of such a host was rewritten to the reference of the host named <base>,
so it joined that host's history and skipped its own host's cleanup.
State IDs now split before the alert's recorded spec ID, or after its
own resource ID, so alerts saved that way also recover on restore.
Normalization keeps a disk's last-known temperature when the current
observation did not collect one (a disk in standby, a host agent past
its reporting lease) and marks it non-available. Several readers still
took PhysicalDisk.Temperature at face value. The AI chat context listed
a standby disk as needing attention for heat. Patrol raised health
issues and triage flags from the retained value. The AI query and
disk-health tools handed it to the model as the disk's temperature.
Performance reports and the reporting runtime snapshot tabulated it as
measured, and the disk temperature charts padded a series out to now
with it.
Each of them now goes through diskinventory.TemperatureCollected. AI
surfaces keep a retained value as context, labelled last known with
its reason. Reports and the charts leave it out. monitor.go's private
copy of the predicate is replaced by the shared helper.
A threshold alert stays open below its trigger until the reading reaches
the clear level and holds there for the recovery delay, while its message
keeps the last breach. #2567 added the live status and moved node, guest
and Mail Gateway drawers onto it, but the Alerts page card, the open
incident in the History panel and Patrol's attention summary still said
"Node temperature at 95.0°C" while the node read 72°C.
The Alerts card now leads with the reading now and what clears the alert,
keeps the last breach as hover text, and labels the alert and clear
levels in the metric's unit, temperatures in the viewer's °C or °F.
Without a live status it labels a level only for metric types whose unit
is known, so a Mail Gateway queue-age alert no longer reads "limit: 30%"
for a 30 minute threshold. The resource incident panel describes only the
open occurrence whose alert id and start match a live alert, the incident
store's own occurrence key, rechecked on the shared clock; closed
occurrences keep their recorded message.
Patrol's attention summary is built server-side because the Patrol inbox,
the Assistant handoff and the mobile app all read plainLanguageSummary.
A latched or recovering alert with a fresh status gets the live reading
in the frontend helper's words; a breaching alert keeps its message,
which tracks the breach, and a stale status falls back to it.
Every disk alert on a guest carries the guest's name, so the live
reading names the disk from metadata.label as the per-disk message does;
without it two disks on one guest read the same. Recovery progress now
rounds down, so 4m50s into a 5 minute delay no longer reads "5 minutes so
far" while the alert is still open, and the server copy rounds the way the
frontend does (toFixed and Math.round ties), so Patrol and the card agree.
From the sm breakpoint the card's text column keeps a 16rem floor. Before,
it shrank to its longest word beside one row of action buttons, so at a
1024px window every card's text sat in a 110-160px column; the buttons now
wrap instead.
Refs #2068
A node alert that moves to its linked Pulse agent closes with a
moved_to_agent resolution (#2580), and #2591 taught Patrol, alert
history and the assistant to say so. Two readers still showed the
close as a recovery.
Performance reports dropped the resolution when copying recently
resolved alerts into reporting.AlertInfo. A node report generated in
the five-minute window after a handover read HEALTHY, "All systems
operating normally", counted the alert as triggered and resolved, and
told the AI narrator resolved: true, although nothing had seen the
condition clear. Report rows now carry the resolution and the alert's
resource. pkg/reporting treats an alert that closed without recovering
as still live unless the report also shows its successor's reading for
that metric (an open alert, or a recovery since the handover), so a
handover never reads as healthy and one condition is counted once;
fleet reports match handovers across all their resources. The PDF
card, Quick Stats, active list and heuristic narrative say where the
alert went and to check the agent, the alerts table shows "Moved" with
the engine's summary, fleets count it as active and name the agent,
and the narrator gets resolved: false, the summary and whether the
successor's alert is listed, with prompt rules for both.
Resource change readers labelled every alert_resolved change "Alert
resolved" in green and prefixed the headline with it, so a moved close
read "Alert resolved: Alert moved to pve1 (Host Agent). This is not a
recovery...". getResourceChangePresentation now reads the
alert_resolution metadata: such a change is "Alert moved" in the
neutral blue of the history badge, and its summary is the whole
headline in the drawer, ResourceChangeSummary and Patrol's assessment
handoff.
Reports requested as agent, which is how the Reports page addresses
every Proxmox node, still carry no alert rows at all, and node alert
history is stored under legacy node IDs that the drawer never queries;
both are separate fixes.
rcourtman