Merge origin/main into fix/saml-acs-request-binding

The SAML stack this branch builds on appends its boundary notes to
storage-recovery.md at the anchor main used for the linked-agent stale
flag note. Both sets of notes are kept, the SAML ones first.

Contract-Neutral: Merge of origin/main into the SAML stack: main's changes were governed when they landed, and the only resolution keeps both sets of storage-recovery boundary notes
This commit is contained in:
rcourtman
2026-10-08 00:13:11 +01:00
137 changed files with 5918 additions and 2835 deletions
+3 -1
View File
@@ -891,7 +891,9 @@ For an existing incident whose notifications should be paused without clearing
it, use the snooze and unsnooze endpoints below.
- `GET /api/alerts/config`
- `PUT /api/alerts/config`
- `PUT /api/alerts/config` — replaces only the top-level keys the body
carries; a key left out keeps its stored value, and the merged config is
normalized as before. The body must be a JSON object.
- `GET /api/alerts/deadman/config` — returns only whether an external watchdog
is configured; `pingUrl` is `***REDACTED***` when present and never returns
the credential-bearing URL
@@ -248,6 +248,18 @@ Without comparable identity (a SAS, SCSI or USB replacement with no WWN on one
side) the row still matches. Report admission and the SMART wire contract are
unchanged; the regression is
`TestHostAgentSMARTRowForSwappedOutDiskDoesNotLendItsIdentity`.
A PVE disk that a still-reporting linked Agent's SMART row matches is marked
`AgentSMARTReported` (internal poll evidence on `models.PhysicalDisk`, never
serialized); disks the poller builds from that report when the Proxmox disk
query fails go through the same merge, and an Agent whose lease lapsed owns
none. The Agent then owns the disk's temperature alert
through `CheckHost`, and the PVE disk temperature check closes its own as
moved to the Agent, so one disk never alerts twice. Report admission, the SMART
wire contract and Agent authority are unchanged;
`TestMergeHostAgentSMARTIntoDisksMarksDisksTheAgentReports` in
`internal/monitoring/physical_disk_roundtrip_test.go` and
`TestPhysicalDiskAgentSMARTReportedStaysInternal` in
`internal/models/deepcopy_test.go` pin the marker.
Assistant historical metric wiring uses the current monitor's retained store
and registry metrics coordinates. Historical reads do not alter enrollment,
@@ -3678,6 +3690,14 @@ Agent` secondary handoff against the live setup wizard instead of relying
## Current State
### Alert settings saves do not touch agent lifecycle
`PUT /api/alerts/config` in `internal/api/alerting/alerts.go` now keeps stored
values for the top-level keys a client leaves out. Agent alert defaults
(`agentDefaults`) follow the same rule as every other key; enrollment,
reporting leases, commands and agent removal are unchanged.
`internal/api/alerting/alerts_test.go` pins the handler.
### VM guest execution admission (backup precaution)
`AgentRegisterPayload.guest_exec_guard_version` is optional; version 1 on a
@@ -4712,6 +4732,12 @@ agent-lifecycle operations. If those routes are called directly, the API must
fail them as unsupported rather than presenting file diffs or reverts as
agent command rollback, enrollment repair, update rollback, or fleet-control
authority.
Assistant session mutations (`/abort`, `/summarize`, `/fork`, `/undo`,
`/redo`, `/steer` under `/api/ai/sessions/{id}`) accept only `POST`; any other
method that reaches those handlers gets `405` before touching session state.
That method guard
is AI-runtime transport hardening in `internal/api/ai_handler.go`; it adds no
agent command, enrollment, update, or fleet-control path.
That same shared dependency now also assumes hosted cloud handoff authorizes
tenant org access before browser lifecycle continues. Lifecycle-adjacent opens
into hosted workspaces may depend on `internal/api/cloud_handoff_handlers.go`,
@@ -8974,3 +9000,11 @@ like any other write instead of always reaching its handler. Agent command
websocket handshakes to `/api/agent/ws` are HTTP/1.1 `GET` requests, the only
transport its gorilla upgrader serves, and still connect on a demo instance.
No agent registration, enrolment, install, update or removal path changed.
### Demo mode hides the pprof routes
`internal/api/demo_mode_operations.go` changed only so the demo guard on a
`DEMO_MODE` instance answers `/debug/pprof` and every path below it with `404`
for every method. No agent route lives under `/debug/`, so agent registration,
enrolment, install, update, removal, report ingest and `/api/agent/ws`
handshakes are unchanged on a demo instance.
@@ -3556,6 +3556,12 @@ query...`, and `Reading storage...` before streamed tool arguments are
answer (nothing streaming, no error block, no pending approval or
question, and a user prompt available to re-send) because the undo
boundary only operates on the last durable turn.
Session mutations stay POST-only at the handler: abort, summarize, fork,
undo, redo, and steer refuse every other method with `405` and
`Allow: POST` before resolving the Assistant service. A `GET` or `HEAD`
that the demo-mode read-only guard or the CSRF check admits as a read
therefore cannot cancel a running turn, compact or rewind a transcript,
or persist a fork.
The empty Assistant drawer may surface recent non-empty sessions as direct
resume actions using the backend session list already owned by the drawer;
it must not create a parallel recent-chat store or product-authored prompt
@@ -6723,6 +6729,11 @@ seed prompt correlations now flow through the shared AI intelligence facade
first, so the detector remains an implementation detail behind one canonical
correlation access path instead of being routed directly by handlers or prompt
builders.
The browser does not read the global learned-correlation list:
`frontend-modern/src/api/ai.ts` carries no client for
`/api/ai/intelligence/correlations`, and neither the shared AI intelligence
store nor the Patrol page loads it, because no page renders it. The resource
drawer reads per-resource correlations from the resource-intelligence payload.
AI-facing policy metadata must also be cloned through the shared unified-
resource policy helper so chat and tools consumers do not maintain their own
policy copy logic. Chat mention prefetch now calls that shared helper directly
@@ -1114,6 +1114,43 @@ history and affirmative recovery. This is source-fixture lifecycle proof, not
an installed registry/destination result or a diagnosis of a reporter's daily
resolve/reopen cycle.
### A settings save keeps the alert settings it did not send
`PUT /api/alerts/config` applies the body to the stored configuration through
`Manager.ApplyConfigUpdate` and `ApplyAlertConfigUpdate` in
`internal/alerts/config/update.go`: a top-level key the client sent replaces
that setting whole, a key it left out keeps the stored value, and the result
is normalized as any update is. The thresholds page has no control for
flapping detection (`flappingEnabled` and its window, threshold and cooldown),
alert TTL cleanup (`maxAlertAgeDays`, `maxAcknowledgedAgeDays`,
`autoAcknowledgeAfterHours`) or `customRules`, so it never sends them. The
handler used to decode the body into a zero `AlertConfig`, so every thresholds
save turned flapping detection and TTL cleanup off, reset the flapping tuning
to its defaults, dropped custom rules, and persisted that. The page's
`flapping`, `aggregation` and `ioNormalization` payload objects are not
`AlertConfig` fields and never enabled anything. A stored config keeps what it
holds, including values an earlier save zeroed; nothing rewrites them.
The body is decoded on its own, so sent keys mean exactly what the decoder
makes of them, duplicates and case-folded keys included. Unsent fields are
copied from a JSON round trip of the stored config, so the result shares no
maps or slices with it. `ApplyConfigUpdate` reads, merges and applies under
the manager lock, so two concurrent partial saves cannot revert each other in
memory, and returns a JSON-cloned snapshot so that persistence, which
normalizes the config it is handed in place, never writes into the live
config's maps.
`internal/alerts/config/update_test.go` covers unsent, explicit-off,
case-folded, duplicate-key, malformed and full-body updates, a sent object
replacing its stored value whole (a sent `truenasDiskDefaults: {}` does not
inherit the stored temperature) and the no-sharing rule.
`TestUpdateConfigKeepsSettingsAThresholdsSaveDidNotSend` and
`TestApplyConfigUpdateKeepsConcurrentPartialUpdates` in
`config_validation_test.go` run saves through the manager, and
`TestUpdateAlertConfig_KeepsStoredValuesForUnsentKeys` in
`internal/api/alerting/alerts_test.go` drives the handler against a real
manager.
### Confirmed empty storage is recovery evidence
Static storage capacity evaluation must admit a zero usage observation when
@@ -1473,7 +1510,8 @@ That same guest-threshold owner also governs guest-derived lifecycle and
posture alerts. Snapshot age, backup age, powered-off state, and
configuration-change reevaluation must all construct a canonical lightweight
guest snapshot and route threshold resolution through the shared
guest-defaults → filter-driven custom rules → guest-override chain.
guest-defaults → filter-driven custom rules → guest-override → `pulse-relaxed`
chain.
That canonical guest context must preserve the live guest name and tags for
snapshot and backup posture evaluation. Ignored prefixes, `pulse-no-alerts`,
configured ignored tags, and required-tag filtering must resolve through the
@@ -1483,6 +1521,27 @@ operator's suppression policy.
Passing `nil` guest context or resolving only overrides/defaults is forbidden
because it silently bypasses custom guest rules and makes guest lifecycle
alerting diverge from running-guest metric truth.
The `pulse-relaxed` step lives in `effectiveAlertPolicyNoLock`, not in
`CheckGuest`, so polling and config-save reevaluation judge a relaxed guest's
alert by one threshold. Reevaluation rebuilds the guest from alert metadata,
so `guestSnapshotFromAlert` reads back the `tags` the evaluator recorded
(`[]string` live, `[]interface{}` after a restart). Without them a save judged
the alert by unrelaxed thresholds, resolved one raised under an unset
threshold's relaxed floor with a recovery notification, and the next poll
raised it again. Those tags must stay current: `metricOptionsWithTags` drops
the key from an open alert once its resource has no tags, or a removed
`pulse-relaxed` would keep judging the alert by the relaxed trigger. The tag
lifts CPU, memory and disk triggers to a 95/92/95 floor and gives an unset
threshold that floor, but an Off threshold (trigger <= 0) stays Off: relaxing
a guest must never raise an alert its config turned off. A filesystem's own
override still replaces the relaxed guest disk threshold for that filesystem,
on save as while polling. Proof:
`TestConfigSaveJudgesRelaxedGuestAlertByRelaxedThresholds`,
`TestConfigSaveKeepsRelaxedGuestFilesystemOverride`,
`TestPulseRelaxedKeepsOffGuestThresholdOff` and the Off case of
`TestApplyRelaxedGuestThresholds` in `internal/alerts/alerts_test.go`, and
`TestCheckGuestDropsRemovedTagsFromOpenAlerts` in
`internal/alerts/unified_eval_test.go`.
That same guest-alert owner also has to retire per-disk guest alerts when the
guest stops, disk alerting is disabled, or the reported disk set changes.
Canonical guest disk identity is only valid while the guest still exposes that
@@ -1851,14 +1910,84 @@ A disk is hot from its trigger. Alerts and Patrol findings stay open until the
reading falls to the clear value, or under the trigger when there is no band
below it. The Physical Disks verdict and the TrueNAS Health cell judge only the
current reading against the trigger. PDF reports colour disk temperatures by
the same thresholds. Judges that still differ: TrueNAS disk temperature alerts
use `TrueNASDiskDefaults.Temperature` (a flat 55/50), and the TrueNAS disk
drawer tones the reading from a fixed 55C.
the same thresholds.
`TestDiskTemperatureThresholdMatchesCheckHostPolicy` in
`internal/alerts/threshold_resolution_shared_test.go` pins per-type resolution, a raised
NVMe trigger that `CheckHost` also honours, the copy, the disabled default and
the nil manager.
TrueNAS disk temperature alerts follow the policy too:
`trueNASDiskTemperatureDefaultNoLock`
(`internal/alerts/truenas_disk_temperature.go`) is the type-default tier of
`effectiveAlertPolicyNoLock` for `truenas-disk`, so an unset
`TrueNASDiskDefaults.Temperature` resolves per disk type (the
`alertPolicyQuery.DiskType` the unified input carries from
`PhysicalDisk.DiskType`). A TrueNAS-wide value the user saved under TrueNAS
Disks replaces it for every TrueNAS disk, as a host Disk Temp override does for
an agent's disks, 0 meaning off, and a per-disk override beats both. The alert
records `diskType` so a config save re-judges it per type; one without it is
held to the lowest per-type trigger. A retained reading
(`diskinventory.TemperatureCollected` false) is no evidence and raises nothing.
The factory value is unset. `normalizeTrueNASDiskTemperature` drops a stored
55/50 only from a config without `TrueNASDiskTemperatureByType`, because the
thresholds page wrote that flat factory value back on every save; the page
writes the marker, so a 55 typed later is kept. Every config `GetConfig`
returns carries the marker, so a client that reads and writes back keeps it.
Without the marker (an old saved file, or a page loaded before the upgrade)
55/50 is read as that unchosen factory value; a client building a config from
scratch with an explicit TrueNAS-wide 55 has to send the marker. The TrueNAS
Disks Global Defaults cell reads `By type` while unset
(`globalDefaultFallbacks` on `ResourceTable`,
`resolveAlertResourceGlobalDefaultCell`), or Off when the agent Disk Temp
default switches the policy off, and switching it on then stages an explicit
TrueNAS-wide 55 because unset would stay off. Each disk row inherits its type's
trigger from the unsaved editor state (`resolveTrueNASDiskTemperatureDefault`).
The TrueNAS disk drawer tones a current reading from the same per-type trigger
(`getDiskTemperatureThresholds` passed into `buildTrueNASDetailSections`), so it
no longer judges heat from a fixed 55C. Judges that still differ: the TrueNAS
storage table and drawer do not apply a TrueNAS-wide value or per-disk
override.
`TestTrueNASDiskTemperatureAlertsFollowDiskTemperaturePolicy` and
`TestTrueNASDiskTemperatureAlertIgnoresRetainedReading` in
`internal/alerts/unified_eval_test.go` pin the tiers and retained readings;
`TestTrueNASDiskTemperatureAlertsReevaluatePerDiskType` and
`TestTrueNASDiskTemperatureAlertWithoutDiskTypeHeldAtLowestTrigger` in
`internal/alerts/alerts_test.go` pin re-judging on a config save;
`TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues` pins the
saved-config migration.
Proxmox physical disks alert on temperature under the same policy.
`CheckProxmoxDiskTemperature` (`internal/alerts/proxmox_disk_temperature.go`)
raises a `diskTemperature` metric alert under the disk's PVE alert reference
(`ProxmoxPhysicalDiskAlertResourceID`, resource type `proxmox-disk`), judged by
`proxmoxDiskTemperatureThresholdNoLock`: the disk type's `DiskTempByType` entry
under the agent Disk Temp default, which switches it off for every type. It
carries the disk's serial, WWN and `diskType` like the health and wearout
alerts, so `ProxmoxPhysicalDiskAlertIdentifiers` lists its
`<reference>::metric-threshold:diskTemperature` identifier for history
ownership, and `reevaluateActiveAlertsLocked` re-judges it per disk type (the
lowest per-type trigger when the type was never recorded) instead of falling
through to guest thresholds that have no disk temperature. A disk a
still-reporting linked agent lists in its SMART report is the agent's: its
alert closes with the `moved_to_agent` resolution and `CheckHost` raises the
agent's own. The handover happens on the next disk poll, so for that one
interval an agent newly linked to a disk with an open PVE alert can hold its
own alert beside it. A lapsed agent owns nothing, so the node's own readings
keep alerting while its agent is silent. An
excluded disk, or a policy switched off, closes the alert; a poll without a
current reading holds it. Like the health and wearout alerts, a disk that
leaves the inventory keeps its alert until the 24-hour stale sweep. The
`proxmox-disk` alert delay defaults to the factory 5 seconds, so a warning
waits out the same 300-second noisy-gauge stability window agent and TrueNAS
disk temperature warnings do, and a critical reading bypasses it.
`TestProxmoxDiskTemperatureAlertsFollowDiskTemperaturePolicy`,
`TestProxmoxDiskTemperatureAlertOwnershipAndHolds`,
`TestProxmoxDiskTemperatureWarningWaitsOutStabilityWindow`,
`TestProxmoxDiskTemperatureAlertsReevaluatePerDiskType` and
`TestCheckDiskHealthAlertsCarryHistoryOwnershipIdentity` in
`internal/alerts/alerts_test.go` pin the policy, ownership, holds, re-judging
and identifiers.
### Agent disk temperature alerts clear when their disk leaves the report
`CheckHost` also clears a disk temperature alert, and drops its pending
@@ -3180,6 +3309,76 @@ ownership is `internal/alerts/alert_identity_migration_test.go`,
`internal/monitoring/monitor_alert_override_migration_test.go`, and
`frontend-modern/src/features/alerts/__tests__/alertsConfigurationModel.snapshot.test.ts`.
### Alert config snapshots are owned copies
`Manager.GetConfig()` returns `AlertConfig.Clone()`, a typed deep copy that
shares no map, slice or pointer with the live config (custom rule filter
values are copied as decoded JSON values). Callers encode, persist
and edit that snapshot outside `m.mu`: the GET config handler, alert
activation, the identity migration, backup evaluation and diagnostics. It used
to be a shallow copy, so `SaveAlertConfig` normalizing it in place, or
`UpdateConfig` re-normalizing the overrides of a re-applied snapshot, wrote the
live maps while evaluation or another encoder read them, and Go aborted Pulse
with a concurrent map write. `SaveAlertConfig` also clones before normalizing,
because a config just handed to `UpdateConfig` is the live config. Persistence
keeps an explicit `0` type-level delay as no delay on save and load, matching
`NormalizeTimeThresholds`. It used to store and load it as the 5-second factory
delay, which the noisy-gauge rule stretches to 300 seconds for memory and
temperature, and the shared map write was what made the running manager agree.
Pulse saves never wrote a `0`, so only a newly sent or hand-edited `0` changes
meaning. `internal/alerts/config/clone_test.go`
fills every field and fails when a new reference field is not copied.
Regression ownership is `internal/alerts/resolved_lock_discipline_test.go` and
`internal/config/persistence_alert_ownership_test.go`.
### Alert evaluation owns the thresholds it normalizes
Host, guest and Docker container evaluation copy what they read from
`m.config` while holding `m.mu`, and only then fill in defaults.
`CheckHost` and `CheckGuest` clone a per-disk
override's `Disk` threshold before `ensureHysteresisThreshold` sets a missing
`Clear` to the trigger minus 5. They used to normalize the live override's
pointer after unlocking, so a `GetConfig()` clone under the read lock raced
the write, and the saved override changed to whatever the last poll filled in.
The live override now keeps the clear it was saved with, and the config API
returns that value. Docker container evaluation resolves its thresholds from
cloned `DockerDefaults` plus the override under the same lock, and reads the
restart-loop and memory-limit settings under the lock. It used to point into
`m.config.DockerDefaults` with no lock while `UpdateConfig` replaced
`m.config`. The test-only `getThresholdForMetricFromConfig`, which normalized
in place, is gone. `internal/alerts/alerts_test.go`
`TestAlertEvaluationLeavesTheLiveConfigToTheLock` runs host, guest and Docker
evaluation against `GetConfig()` and `UpdateConfig(GetConfig())` under
`-race`.
### Config-save reevaluation judges a filesystem alert by its own threshold
`UpdateConfig` re-judges every active alert, and a filesystem usage alert's
threshold resolves through the same per-filesystem helper its evaluator uses.
`hostDiskUsageThresholdNoLock` resolves an agent filesystem
(`agent:<host>/disk:<label>`): its own override, then the `DiskFillByType`
threshold for a hardware type inferred from the device while the host's disk
alerting is on, then the host's disk threshold. `guestDiskUsageThresholdNoLock`
resolves a guest filesystem (`<guestID>-disk-<key>`): its `guest-disk:`
override, then the guest's disk threshold. `CheckHost` and `CheckGuest` call
them under the read lock, and `resolveHostAlertThresholdsNoLock` and
`resolveGuestAlertThresholdsNoLock` call them for reevaluation. Both return an
override or per-type threshold as an owned copy. A guest filesystem alert
resolves its guest's threshold, override and identity-based custom rules by the
canonical `instance:node:vmid` guest ID that `parseGuestAlertIdentity` reads
from its resource ID. Custom rules that filter on live metrics still see no
readings there, as for every reevaluated guest alert. Reevaluation used to
judge agent filesystems by the host threshold, and guest filesystems without
their `guest-disk:` override or the guest's own override, because the
filesystem's resource ID names no guest. Any settings save resolved an alert a
per-disk threshold of 70 raised at 80 under a default of 90, notifying
recovery, and the next poll raised it again as a new alert.
`internal/alerts/threshold_resolution_shared_test.go`
`TestConfigSaveJudgesFilesystemAlertsByTheirOwnThreshold` covers an agent
filesystem override, an NVMe fill threshold, a guest filesystem override and a
guest override: an unchanged save must keep the alert, and a save lifting the
trigger above the reading must resolve it.
### Versioned alert-intent policy
The alerts runtime owns one versioned alert-intent document and its durable
@@ -2090,6 +2090,20 @@ payload shape change when the portal presents compact client rows.
index; clients must treat the endpoint's `accepted:true` as inbox
receipt, not delivery. Browser clients must use the shared
`AIChatAPI.steerSession(sessionId, { prompt, clientMessageId })` helper.
Every Assistant session mutation sub-resource under
`/api/ai/sessions/{id}` (`/abort`, `/summarize`, `/fork`, `/undo`,
`/redo`, `/steer`) accepts only `POST`. Any other method that reaches the
handler returns `405` with `Allow: POST` through
`requireAssistantSessionMutationMethod` in `internal/api/ai_handler.go`,
before the session service is resolved (earlier layers still answer
first: `Router.ServeHTTP` returns `200` for `OPTIONS` preflight, and
auth, scope, CSRF, and demo-mode guards keep their own refusals). The
session router dispatches these sub-resources by path alone, and the
demo-mode read-only guard and the CSRF check both treat `GET` and `HEAD`
as reads, so the method contract belongs to the mutating handler rather
than to its callers. `GET /api/ai/sessions/{id}/messages` stays the read
path, and the unsupported `/diff`, `/revert`, and `/unrevert` handlers
keep answering `501` for every method that reaches them.
OpenCode-style file diff/revert session routes are deliberately not part
of Pulse's supported Assistant session contract: Pulse sessions do not own
local code-file edits, and infrastructure mutations must be reviewed
@@ -3192,7 +3206,7 @@ a new API state machine, queue contract, or verification-accounting field.
regressions keep the current findings empty state out of green all-clear copy
even when the active finding count is zero and the latest summary score is
otherwise healthy
and the Patrol Current work assessment behavior, so the same governed AI summary contract decides whether the workspace leads with verified health, issues detected, coverage incomplete, or another attention state instead of letting count-only page fragments emit a stale `No issues found` conclusion
and the Patrol Current work empty-state behavior, so once runtime state (running, blocked, disabled, unavailable) and stale coverage have taken precedence, the governed AI summary's overall health, together with run history, decides whether the current-findings empty state reads `No current issues`, `Check needed`, or `Patrol needs review` instead of letting count-only page fragments emit a stale `No issues found` conclusion
and the Patrol Current work copy boundary, so the API-owned health,
finding, run, and control facts remain semantic inputs while browser-visible
empty and descriptive text stays operator-facing: what will appear there,
@@ -3250,8 +3264,8 @@ a new API state machine, queue contract, or verification-accounting field.
without adding a new API payload or page-level proof surface; those process details
remain available in expanded context, selected run records, Assistant
handoff context, or API diagnostics when they are actually needed
and the Patrol verification summary derived from run history, so the page also states whether recent Patrol evidence came from a successful full patrol or only from scoped/erroring runs instead of leaving verification scope implicit
and the same-day activity-mix explanation derived from that governed run history, so when a recent full patrol is followed by alert-triggered or anomaly-triggered scoped work the verification surface can explain the mix directly instead of reconstructing it from page-local timing heuristics
and the Patrol verification posture derived from run history (`getPatrolVerificationPresentation`), which today only withholds the current-findings empty state's all-clear when the latest completed full patrol ended with errors or only targeted or follow-up runs completed; the page shows no aggregate verification-posture summary (expanded run-history entries still show each run's own scope and coverage), and any surface that adds one must take it from that run-history posture rather than page-local heuristics
and the same-day activity-mix label that posture computes from the same governed run history, which no surface renders today; one that does must explain a recent full patrol followed by alert-triggered or anomaly-triggered scoped work from that label instead of reconstructing the mix from page-local timing heuristics
and the Patrol status recency split, so `last_patrol_at` remains reserved for completed full Patrol sweeps while scoped runs and verification checks advance `last_activity_at` without claiming a fresh full-estate verification pass
and the monitor-context Patrol coverage posture boundary, so Proxmox
overview and other monitor-first launch pages must not consume API-owned
@@ -3304,7 +3318,7 @@ a new API state machine, queue contract, or verification-accounting field.
and the `/api/ai/intelligence/changes` route plus `internal/api/contract_test.go`, so the canonical recent-changes endpoint stays on the same intelligence facade and contract snapshot instead of bypassing the shared timeline source
and the canonical policy-posture snapshot derived from unified resources, so sensitivity, routing, and redaction counts stay owned by the same AI summary contract instead of being reconstructed as a page-local governance rollup
and the resource-intelligence payload carried by the drawer AI card, so the resource-detail surface stays on one canonical intelligence contract instead of introducing a separate detail endpoint
and the learned-correlation payload loaded into the shared AI intelligence store, so the Patrol intelligence page and the AI summary page consume the same governed correlation slice instead of each page fetching its own copy
and the global learned-correlation route `/api/ai/intelligence/correlations`, which has no frontend client today: no page renders that list, so the shared AI intelligence store neither loads nor holds it and the resource drawer reads per-resource correlations from the resource-intelligence payload; a surface that later renders the global list must add one store-owned load instead of a page-local fetch
and the shared dashboard-load bundle inside `frontend-modern/src/stores/aiIntelligence.ts`, so the page orchestration stays on the store-owned bundle instead of enumerating the AI fetches inline
and the Patrol page refresh lifecycle in `frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts`, so slow or stalled secondary reads from that shared dashboard-load bundle may continue resolving in the background while the operator-facing Patrol refresh control remains generation-aware, timeout-bounded, and reusable once Patrol findings and status are already visible
and the Patrol header support drawer in `frontend-modern/src/features/patrol/PatrolIntelligenceHeader.tsx`, so API-owned Patrol status and trigger facts can feed a secondary Schedule & model surface without turning provider model, schedule, trigger tuning, or background-only runtime-policy pauses into the primary Patrol control decision
@@ -3325,6 +3339,7 @@ a new API state machine, queue contract, or verification-accounting field.
action artifact so the Assistant runtime can refresh finding and action posture
from IDs and safe summaries instead of relying on pasted chat text
and the dedicated `frontend-modern/src/stores/aiIntelligenceSummaryModel.ts` owner, so recent-change counts and governed policy-posture fallbacks normalize once at the shared store boundary instead of as Patrol-hook-local payload repair
and the Patrol findings list the state hook reads from `aiIntelligenceStore.patrolFindings`, so `usePatrolIntelligenceState.ts` does not re-derive page-level severity tallies from it: the hook's unread `summaryStats` tally has been removed, the hook derives no client-side severity tally from that list today (the backend's per-severity `findings_count` reaches it only inside the AI summary payload it passes through), and the Activity tab's `This week` card takes its open-by-severity counts from `GET /api/ai/patrol/digest` instead of recounting that list
and the shared `frontend-modern/src/components/Infrastructure/ResourceCorrelationSummary.tsx` card, so learned correlations and correlation context stay rendered through one governed frontend card instead of separate page-local list loops
and the same shared correlation card's ordering and truncation rule, so callers pass raw correlations instead of encoding their own top-N sort behavior
and the shared `frontend-modern/src/components/Infrastructure/ResourceChangeSummary.tsx` and `frontend-modern/src/components/Infrastructure/ResourceCorrelationSummary.tsx` cards' optional `buildResourceHref` input, so a surface that needs resource-filter links passes them into the shared card instead of rebuilding infrastructure URLs inline; the cards carry no link default since the cross-resource drilldown links were retired, and render resource labels as plain text
@@ -3458,9 +3473,9 @@ a new API state machine, queue contract, or verification-accounting field.
17. Keep Patrol intelligence summary transport semantics single-voiced: the canonical overall-health payload and Patrol run-history payload together must support one primary assessment plus one explicit verification explanation, and frontend consumers must not need to derive a second compact assessment or verification verdict row from the same payloads beneath the primary assessment strip.
That same transport split supports Patrol assessment and action metadata
for any consumer without adding another API field (the Patrol page renders
no assessment strip today): the frontend summary
contract derives compact state from existing overall-health, run-history,
active finding, runtime, and pending-approval facts, so the API remains the
no assessment strip today): a consumer that needs compact state derives
it from existing overall-health, run-history, active finding, runtime, and
pending-approval facts (no shared frontend presenter does so today), so the API remains the
source of facts while the configured LLM owns next-step reasoning. Those
action references map back to existing API-backed Patrol controls and
approval/finding filters; summary transport must not become a new execution
@@ -4172,17 +4187,19 @@ the authoritative analysis outcome.
plumbing.
The same page-header recency line also reads
`PatrolRecencyPresentation.resourcesCheckedLabel` — derived by
`getPatrolRecencyPresentation` from
`PatrolRunRecord.resources_checked` and the latest completed run outcome —
`getPatrolRecencyPresentation` from the latest completed run's
`PatrolRunRecord.resources_checked` —
so the operator sees coverage alongside recency without needing a
new API surface. The label stays optional when coverage is zero, says
verified only for successful full patrols, and uses neutral checked wording
for errored full patrols or scoped activity.
The Patrol Current work empty-state copy uses that same run-history-backed
coverage truth when reconciling AI summary coverage factors: a successful
full run with non-zero `resources_checked` suppresses stale
coverage-incomplete wording, while scoped, missing, zero-coverage, or errored
runs still allow the coverage caveat to surface.
new API surface. The label stays optional when coverage is zero and reads
`checked N resources` for any completed run, whatever its type or outcome;
it never says verified.
The Patrol current-findings empty state does not reconcile AI summary
coverage factors against run history: outside runtime and stale-coverage
states, any `coverage` factor in `overall_health` withholds the all-clear,
even after a successful full run
with non-zero `resources_checked`, and run history adds its own caveat when
the latest completed full run ended with errors or only targeted or
follow-up runs completed.
and the Assistant finding-context request contract, so `/api/ai/chat`
payloads carrying `finding_id` may hydrate a structured investigation
summary from the unified finding, but raw action commands must stay
@@ -4308,10 +4325,10 @@ the authoritative analysis outcome.
control level and paid Patrol settings currently allowed by runtime
entitlements.
11. Treat Patrol summary supporting metrics as readouts, not reinterpretations: when frontend consumers derive cards such as active findings, criticals, warnings, or fixes from the canonical payloads, those cards must stay numeric and must not synthesize new assessment labels like `Issues detected` or verification labels like `Partial verification` beneath the primary summary contract
12. Treat active Patrol runtime transport as compatible with factual activity surfaces: when the runtime is currently running, frontend consumers may surface in-progress activity context, but they must not replace the activity strip with a second assessment verdict derived from runtime state alone
13. Treat Patrol recency as a singular transport-driven fact: once header metadata, verification copy, or the findings footer already present the governed Patrol timing context, frontend summary consumers must not derive an extra timing pill from the same payloads inside any primary summary card
12. Treat active Patrol runtime transport as compatible with factual activity surfaces: when the runtime is currently running, frontend consumers may surface in-progress activity context, but they must not add a second assessment verdict derived from runtime state alone
13. Treat Patrol recency as a singular transport-driven fact: the page-header recency line presents the governed Patrol timing context, so frontend summary consumers must not derive an extra timing pill from the same payloads inside any primary summary card that returns
14. Treat Patrol findings counts as a singular supporting surface as well: when the summary shell already exposes count cards for active findings, warnings, criticals, and fixes, the primary assessment card must not repeat those same payload-derived counts as secondary badges
15. Treat Patrol schedule and recency as header-owned metadata on the main Patrol page: findings empty-state consumers should not receive or restate `next_patrol_at`, `last_patrol_at`, `last_activity_at`, or interval timing once those transport fields are already presented by the primary header and verification shell
15. Treat Patrol schedule and recency as header-owned metadata on the main Patrol page: findings empty-state consumers should not receive or restate `next_patrol_at`, `last_patrol_at`, `last_activity_at`, or interval timing once those transport fields are already presented by the page header
16. Keep recovery payload filters canonical across `/api/recovery/rollups`, `/api/recovery/points`, `/api/recovery/series`, and `/api/recovery/facets`: when `internal/api/recovery_handlers.go` adds a governed recovery filter or display field such as provider-neutral `itemType`, the same normalized transport must land across all four endpoints and the contract tests must pin both outbound payload shape and accepted query aliases in the same slice
17. Keep recovery platform-query vocabulary canonical across that same `/api/recovery/*` surface: operator-facing transport must emit `platform` as the canonical query field, accepted legacy `provider` aliases must remain compatibility-only input, and `internal/api/contract_test.go` must pin that fallback behavior in the same slice as any handler change
18. Keep recovery payload platform vocabulary canonical across that same `/api/recovery/*` surface: point payloads must expose `platform`, rollup payloads must expose `platforms`, and any compatibility `provider` / `providers` aliases must remain secondary fallback fields rather than replacing the shared response model
@@ -4846,6 +4863,10 @@ transition and evidence times) from the frontend-primitives shared
relative-time clock. That is presentation over the same `GET
/api/ai/patrol/status`, run history and `GET /api/ai/patrol/attention/{id}`
payloads: no field, request or polling cadence changes.
The attention detail's suppression `expiresAt` and a finding's `remind_at` and
`snoozed_until` from `GET /api/ai/patrol/findings` are future times, so the
page presents them as countdowns from the same clock ("in 1d") instead of
ages. That is presentation only: no field, request or polling cadence changes.
### Resource-list facets preserve scoped navigation evidence
@@ -7270,12 +7291,14 @@ introducing VMware-only mention payloads or provider-local inventory reads
under `/api/vmware/*`. Runtime-specific container/app mentions remain shared
unified-resource mentions as well; VMware network inventory does not create a
provider-local mention family.
That same `/api/ai/chat` payload boundary owns per-request execution-mode
overrides. Dashboard Pulse Brief and other scoped handoffs may include
`autonomous_mode:false` on the chat request to force approval-required command
execution for that exchange, but the transport must treat the field as a
request override only and must not mutate the user's persistent AI control
setting.
That same `/api/ai/chat` payload boundary owns request-local execution mode,
and the server sets it. `ChatRequest` in `internal/api/ai_handler.go` has no
`autonomous_mode` field, so the `autonomous_mode:false` that scoped handoffs
such as an explain-this-issue request still send is dropped at decode, and the
handler passes approval-required mode into the chat service for every
exchange; an Autonomous control level runs as Controlled for that request.
That clamp is request-local and must not mutate the user's persistent AI
control setting.
That same chat transport boundary owns new-session anchoring. When the request
omits `session_id`, the handler may generate and stream a session ID
immediately so the browser can anchor the visible turn, but that generated ID is
@@ -7329,7 +7352,7 @@ not serialize the model-only `handoff_context`, runtime failure detail, action
preflight/result bodies, remediation descriptions, raw commands, or approval
command payloads, and it must not preserve Patrol-authored next-step
recommendation fields from legacy handoffs.
Patrol finding handoffs are stricter than ordinary chat requests: when a request
Patrol finding handoffs keep that clamp on their own path: when a request
carries a non-empty `finding_id` or resolves to model-only Patrol briefing,
resource, or action context, `internal/api/ai_handler.go` must clamp the
request-local autonomous mode to false even if the caller supplied
@@ -7449,11 +7472,13 @@ The same AI resource-intelligence payload now also carries dependency and
dependent correlation arrays plus correlation evidence, so the drawer can render
canonical correlation context from the shared AI contract instead of inferring it
from the relationship facet payload alone.
The same AI frontend client now also loads `/api/ai/intelligence/correlations`
through the shared `frontend-modern/src/stores/aiIntelligence.ts` store for
the Patrol intelligence page and the AI summary page, so the
learned-correlation list is governed by the same API contract that backs the
resource drawer's correlation evidence instead of being fetched as page-local state.
The frontend does not load `/api/ai/intelligence/correlations`. No page renders
the global learned-correlation list, so `frontend-modern/src/api/ai.ts` carries
no client for it, and neither the dashboard load bundle in
`frontend-modern/src/stores/aiIntelligence.ts` nor the Patrol refresh path
fetches it. The resource drawer's correlation evidence comes from the
resource-intelligence payload above. The route stays an API surface behind the
`ai:execute` scope.
That correlations route now reads through the canonical AI intelligence
facade first, so the handler and its payload keep the detector behind one
shared access layer instead of routing directly to Patrol-local correlation
@@ -7784,6 +7809,23 @@ read-only demo account. That same hidden read-side boundary includes `GET` and
`HEAD` reads for `/api/admin/users` and manual discovery at `/api/discover`;
public demo mode may block writes generically, but it must not reveal that
admin-user inventory or manual-discovery read routes exist.
The same policy hides the Go runtime profiling family for every method: on a
`DEMO_MODE` instance the demo guard answers `/debug/pprof`, `/debug/pprof/`
and every path below it with `404`, ahead of route authentication.
`registerDebugRoutes` gates those handlers on `RequireAdmin` plus a
`settings:read` scope that session logins skip, so a demo that signs visitors
in as the configured admin would otherwise hand every visitor goroutine and
heap dumps, the process command line, and CPU profiles, execution traces and
forced garbage collection that cost the whole process. `PULSE_PPROF_DISABLED`
still removes the routes on any instance.
`TestPublicDemoAdminOperationsPolicyHidesEveryDebugRoute` fails when a literal
route the parsed router files register under `/debug/`, with or without a
method prefix, escapes that policy for any common method.
`TestContract_DemoModeHidesPprofFromDemoAdminSession` in
`internal/api/contract_test.go` signs in as that admin through the full
router, proves the same session still reads `/debug/pprof/cmdline` below the
demo guard, and gets `404` through it for eight pprof paths with `GET`,
`HEAD`, `OPTIONS` and `POST`.
That generic write block keys on the request method alone. The demo
middleware admits `GET`, `HEAD` and `OPTIONS` plus the login, logout, AI
execute and OIDC login/callback allowlist; HTTP/1.1 websocket handshakes for
@@ -10424,6 +10466,22 @@ boundaries. Alert evaluation after that save remains alerts-owned and must use
the persisted trigger/recovery pair plus a valid derived critical threshold;
the API must not invent a TrueNAS-specific threshold sidecar.
`PUT /api/alerts/config` replaces only the top-level keys the body carries
and keeps the stored value of every key it leaves out, so a client that does
not know a setting cannot turn it off by leaving it out: the thresholds page
has no flapping-detection or alert-TTL controls, and an older browser bundle
or API script predates later fields. A sent key replaces its stored value
whole, so `"overrides": {}` removes every override rather than merging, and
the merged config then goes through the same normalization as any update (for
example, activation state is never cleared). The read, merge and apply run
under the alert manager's lock, so two concurrent partial saves cannot revert
each other in memory; the handler runs one save at a time and persists a
snapshot of the applied config, so successful saves through this endpoint
reach `alerts.json` in the order they were applied. A body that is not a JSON object, `null` included, is a 400 and changes
nothing. `TestUpdateAlertConfig_KeepsStoredValuesForUnsentKeys`
and `TestUpdateAlertConfig_RejectsNonObjectBody` in
`internal/api/alerting/alerts_test.go` prove the transport.
### Alert intent and UDP availability transport
`GET /api/alerts/intent-policies` and
@@ -294,6 +294,14 @@ expose only a raw reason. A same-VM fresh observation withdraws the notice and
updates values without remounting the row. Permission, disabled-agent and actual
unavailable cases retain their own explanations.
When a linked Pulse agent's filesystems fill in for a VM with no Proxmox guest
filesystems (`disksFromAgent`), the reading follows that agent instead of the
Proxmox reason: current while it reports, last known once it is stale, and
unavailable once the guest is stopped. `getWorkloadGuestDiskRead` in
`workloadGuestPresentation.ts` owns that rule for the workload row's disk cell,
the guest drawer's filesystem rows and its History, so the three cannot
disagree about one guest; `workloadGuestPresentation.test.ts` pins it.
Verification: `workloadGuestPresentation.test.ts`, the mounted `GuestRow`,
`StackedDiskBar` and `GuestDrawerOverview.filesystems` regressions,
and `browser-tests/guest-disk-deferral.cjs` cover the fixed reasons, retained/no
@@ -2116,6 +2124,18 @@ the same record; the table renders defaults once, in both desktop table and
narrow card layouts, and per-section metric columns must resolve to metric
keys the shared column normalizer produces so the defaults editor reads and
writes the same record keys the section persists.
A global default that another setting decides while it is unset passes a
`globalDefaultFallbacks` entry for that metric (label, title, whether the other
setting is itself off, and the value switching on stages when it is). The
TrueNAS Disks temperature uses it to read `By type`, following Disk
temperature by type, with `features/alerts/thresholds/trueNASDiskTemperature.ts`
giving each disk row its type's trigger from the unsaved editor state.
`resolveAlertResourceGlobalDefaultCell` decides the cell for both layouts:
unset follows the fallback, switching on from Off returns to unset unless the
fallback is off, and an unset default with no fallback stays Off as the engine
reads it. `ResourceTable.tsx` hands its props to the shared table state through
getters, so the Custom badge and reset control follow defaults that load after
the table mounts.
Platform sub-routes that add native provider inventory must stay on the shared
platform page and table primitives. The vSphere Networks surface routes through
`/vmware/networks`, the shared platform tab model, the command palette
@@ -4498,12 +4518,12 @@ The recency line beside the header actions also renders coverage
alongside time when the canonical `getPatrolRecencyPresentation` helper
returns `resourcesCheckedLabel` from the latest completed run. Render code
must gate on `<Show when={recency().resourcesCheckedLabel}>` (truthy) so
zero-coverage runs do not surface a misleading coverage phrase, failed or
scoped runs use neutral checked wording, and only successful full patrols read
as verified. Any Patrol assessment shell must pass the same run-history facts
into `getPatrolAssessmentPresentation` so assessment coverage caveats do not
contradict the header's verified full-run coverage state; the page renders no
assessment shell today.
zero-coverage runs do not surface a misleading coverage phrase. The label
reads `checked N resources` for every completed run with a positive count
(full, targeted, follow-up, or errored) and never says `verified`. The page
renders no assessment shell today; if one returns, its coverage caveats must
come from the same run-history facts so they do not contradict the coverage
the header line shows.
The same header row may surface `Trigger status` when
`getPatrolTriggerStatusSummary` returns a runtime-relevant value from the
Patrol status payload. That text is page-owned operational metadata inside the
@@ -4900,6 +4920,16 @@ or not fully verified.
The same empty-state helper must consume Patrol trust-history evidence so a
historical regression reads as history review context, not as a current issue
and not as a healthy all-clear.
`frontend-modern/scripts/canonical-platform-audit.mjs` carries no Patrol
summary-card rule. Its `canonical-patrol/no-local-summary-card-presentation`
regex matched only text shaped like the deleted summary card's tint ternaries
on `summaryStats().criticalFindings`, `warningFindings` and `fixedCount`. The
Patrol state hook's `summaryStats` accessor has had no reader since that card
was deleted, so the accessor and the rule are both gone.
`frontend-modern/src/utils/patrolSummaryPresentation.ts` is off the audit
allowlist too: it now holds only the recency and verification presenters,
which no audit rule needs to exempt, so the audit scans it like any other
module.
The Patrol page renders no summary shell today. If one returns, the same
hierarchy applies inside it: once its primary assessment strip states Patrol's
current risk and verification basis,
@@ -4915,10 +4945,10 @@ That same summary shell should also keep the shared Pulse surface neutral:
severity belongs in compact accents, inline readouts, and badges rather than
turning the whole assessment into a tinted warning banner, nested card, or
hero-style block that breaks the surrounding operator workflow.
That same summary-shell rule also applies to timing metadata: if the header,
verification card, or findings footer already presents the governed Patrol
activity timestamp, the summary chip row must not add another recency badge
that competes with those owned timing surfaces.
That same summary-shell rule also applies to timing metadata: the header
recency line already presents the governed Patrol activity timestamp, so a
returning summary chip row must not add another recency badge that competes
with it.
The same default-readout rule applies to collapsed Patrol issue rows:
`MetadataBadge` may carry severity, recurrence, and active decision/work states,
but the default Patrol page must not render raw lifecycle or investigation
@@ -5118,10 +5148,10 @@ KPI, problem-resource, or card shells. Workload-table and guest-row fallback
copy that lives under `frontend-modern/src/components/Workloads/` must keep
using `frontend-modern/src/utils/workloadEmptyStatePresentation.ts` and
`frontend-modern/src/utils/workloadGuestPresentation.ts`. New route-level empty
states, tone mapping, or compact issue copy must extend the shared
`emptyStatePresentation`, `semanticTonePresentation`, and
`problemResourcePresentation` helpers instead of reviving deleted
dashboard-only KPI, metric, storage, recovery, or trend presentation helpers.
states or tone mapping must extend the shared `emptyStatePresentation` and
`semanticTonePresentation` helpers instead of reviving deleted dashboard-only
KPI, metric, storage, recovery, problem-resource, or trend presentation
helpers.
That shell must also stay passive with respect to data ownership: future
overview trend cards may render summary-range controls and operator-facing
empty or error copy only after they have a governed owner, and they must not
@@ -5245,7 +5275,19 @@ too. Every read of the clock returns the wall clock; the 30-second tick only
tells readers to re-read, so a cell that mounts between ticks never measures
from a stale time and a timestamp from the last few seconds never reads as a
future time.
The rule has one deliberate exception. The age of a latest reading (last used,
Future times have their own formatter. `formatRelativeTime` is past-only: it
reads a time ahead of `now` as "just now" (compact) or "0s ago", which is right
for clock skew on something already observed and wrong for an expiry, a
reminder or a schedule.
Those go through `formatTimeUntil` (`frontend-modern/src/utils/format.ts`),
which counts down ("in 3h", "in 1d"), rounds to the nearest unit so a duration
just chosen reads as chosen, and returns `dueText` (default "now") once the
time arrives. A countdown target does not change while its surface stays open,
so the countdown passes `now` from the shared clock, as the Patrol suppression
expiry and the Patrol findings reminder and snooze lines do. The replication
Next sync column and the Patrol header's next-check `CountdownTimer` keep their
own minute- and second-precision countdowns.
The relative-age rule has one deliberate exception. The age of a latest reading (last used,
last seen, last success, last checked) on data the surface reads once and does
not re-read stays the age at read time, because a moving age over a snapshot
that never refreshes claims the reading stopped when it may not have. Such a
@@ -5268,8 +5310,8 @@ than hardcoded gray palettes; non-gray typed tones may retain their state color
vocabulary so success, warning, danger, info, and platform-adjacent metadata do
not collapse into visually identical chips.
Patrol run-history labels follow this state-badge boundary:
Patrol may derive the status label and typed variant in
`patrolRunPresentation.ts` or `patrolSummaryPresentation.ts`, but
Patrol derives the status label and typed variant in
`patrolRunPresentation.ts`, but
`RunHistoryEntry.tsx` must render visible state badges through
`StatusIndicatorBadge` rather than `runStatus.badgeClass` or a local span.
The shared segmented selector now follows that same owner split.
@@ -5760,7 +5802,15 @@ the surface-alt detail row shell locally. The content shell must clip
horizontal paint below the large breakpoint without becoming a scroll
container, reset the parent table's `whitespace-nowrap` inheritance, and allow
its descendants to shrink, then restore visible overflow for the static
desktop layout. Long operator-state copy must wrap inside the shared row border
desktop layout. Below that breakpoint the default content shell is capped at
the table scroll shell's content-box inline size (`max-w-[100cqi]`; the shell
is a size container), never a viewport estimate of the page chrome, so a phone
drawer does not leave an empty strip beside its content (the retired
`100vw-3.5rem` cap left 27px at 390px) and is never wider than the table's
visible area. Callers that pass their own `contentClass` own their width.
Fixed-layout cells clip overflow, so the sticky offset is inert and horizontal
scrolling still moves the drawer with its cell.
Long operator-state copy must wrap inside the shared row border
instead of painting beneath adjacent controls or disappearing at the clip edge.
When focused detail content is removed, `InlineDetailTableRow` restores focus
to its current `aria-controls` disclosure with `preventScroll`; live refresh,
@@ -6400,20 +6450,6 @@ rollups, points, or facets payloads arrive, it must keep that selected
platform present in the option set so the shared `LabeledFilterSelect` shows
the owned value immediately instead of flashing back to `All Platforms` until
recovery data warms.
`frontend-modern/src/utils/problemResourcePresentation.ts` now also belongs to
that same dashboard overview boundary so the problem-resource severity contract
stays shared with `ProblemResourcesTable.tsx` instead of floating as an
unowned helper.
Problem-resource table readability belongs to that same owner. Repeated rows
may collapse only when they share the same governed display label, resource
type, and problem signal; the header count and Pulse Brief counts must continue
to represent the underlying affected resources, and grouped links must route to
the broad owning surface rather than inventing a synthetic resource target.
Problem Resources and Pulse Brief wording must not amplify generic
status-shaped names such as `storage (offline)` into first-viewport prose or
grouped-row sublabels; when the resource name is only a type plus status, the
surface should summarize the type-level issue in operator language instead of
repeating raw backend-shaped labels.
The retired dashboard action queue must not be reintroduced as a compact
Patrol or infrastructure issue panel. Patrol-owned runtime findings remain
governed by `frontend-modern/src/utils/aiFindingPresentation.ts` and their
@@ -6766,25 +6802,28 @@ same-day scoped follow-up work, that summary shell should also carry a compact
activity-mix explanation rather than forcing operators to infer why Patrol
looked busy from a second competing status band.
That explanation belongs on the verification surface itself when operators are
reconciling `Recently verified` copy against same-day scoped Patrol bursts; the
reconciling `Recently checked` copy against same-day scoped Patrol bursts; the
supporting activity context may complement the readout, but it is not
sufficient as the only explanation path.
That same shell rule also owns Patrol recency labels. Shared Patrol header and
status-shell surfaces must keep `Last full patrol` tied only to the full-sweep
transport fact and use `Last activity` for scoped or verification work instead
of collapsing both timestamps back into a generic `Last run` label. Coverage
phrases on those recency surfaces must come from the Patrol recency presenter
instead of hardcoding verified wording in the shell.
That same run-history ownership applies to assessment caveats: Patrol summary
shells should not present `Recent coverage is incomplete` when the shared
recency/verification helpers already prove a successful full patrol with
non-zero resource coverage.
Patrol recency labels and coverage phrases come from the Patrol recency
presenter (`getPatrolRecencyPresentation`), never from hardcoded shell copy.
The header labels the latest completed run of any type `Last check`; without
run history it uses `Last check` for the `last_patrol_at` full-sweep transport
fact and `Last activity` when `last_activity_at` is newer, and it never
collapses them into a generic `Last run` label. Its coverage phrase reads
`checked N resources` and never says `verified`.
If a summary shell returns, it should not present `Recent coverage is
incomplete` when run history shows a successful full patrol with non-zero
resource coverage.
That same Patrol shell ownership includes refresh affordance state:
`frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts` must keep
operator refresh controls generation-aware, timeout-bounded, and separate from
background polling state, so a slow supporting intelligence read cannot make the
shared Patrol header Refresh Patrol action spin indefinitely or stay disabled
while Patrol findings and status remain visible.
Neither the Patrol load and poll path (the store's `loadDashboardData` bundle)
nor the Retry path's background supporting reads fetch the global
learned-correlation list, which no Patrol surface shows.
That same Patrol shell should make scoped trigger policy legible without
another navigation step. `frontend-modern/src/features/patrol/PatrolIntelligenceHeader.tsx`
should keep actionable scoped-trigger state legible without promoting
@@ -1825,6 +1825,29 @@ boundary around it:
matches node names only, so it relies on fixture node names being unique
across instances, which they are.
`checkPhysicalDiskAlerts` also hands each disk's temperature to
`CheckProxmoxDiskTemperature`, so a Proxmox disk that only the node's sensors
report alerts under the disk temperature policy, as its Running Hot verdict
already says it should. It passes the reading only when this poll collected it
(`collectedPhysicalDiskTemperature`, `diskinventory.TemperatureCollected`), so
a retained last-known value holds the alert rather than judging it, and it
passes `AgentSMARTReported`, which `mergeHostAgentSMARTIntoDisks` sets on a
disk a still-reporting linked agent lists in its SMART report (disks built
from that report when the Proxmox query fails go through the same merge). A
disk the agent lists is the agent's: `CheckHost` raises its temperature
alert, and the PVE disk check closes its own as moved to the agent, so no
disk alerts twice. Once the agent's lease lapses (`ExpireHostTelemetry` marks
the host offline) its retained rows still enrich the disk, but ownership
returns to the PVE check, which judges the node's own current readings; an
agent newly linked to a disk with an open PVE alert takes it over on the next
disk poll. An excluded device closes its temperature
alert with the health and wearout ones. Fixture disks carry no agent SMART
merge, so on a mock estate every hot disk on an online node alerts.
`TestMergeHostAgentSMARTIntoDisksMarksDisksTheAgentReports` and
`TestCheckPhysicalDiskAlertsRaisesProxmoxDiskTemperatureAlerts` in
`internal/monitoring/physical_disk_roundtrip_test.go` pin the marker,
the collected reading, agent ownership and exclusion.
Switching mock mode fences the alert evaluations that read mode-dependent data
(`mockModeFence`, `internal/monitoring/mock_mode_fence.go`). `GetState`, the
fixture graph, the unified read view, the recovery rollups and the connection
@@ -4491,6 +4514,13 @@ disk usage when the last VM snapshot is still recent guest-agent truth rather
than an already carried-forward fallback. That keeps transient guest-agent or
status-call failures from regressing a VM back to misleading allocated-disk
data while still avoiding indefinite replay of stale disk summaries.
The previous VM that `previousVMFromView` builds from the unified read state
carries disk usage only when it is Proxmox's own reading. When a manually
linked Pulse agent's disk filled in because Proxmox had no guest filesystems
(`VMView.DiskFromLinkedAgent`), carrying it would relabel the agent's value as
a Proxmox `prev-` read, and with Proxmox outranking the agent on guests that
copy would then freeze over the agent's live disk.
`TestPreviousVMFromViewKeepsLinkedAgentDiskOutOfProxmoxCarry` pins both owners.
That compatibility boundary also applies to historical snapshot labels that may
still exist in tests, live in-memory state, or pre-canonical diagnostic paths:
legacy aliases such as `rrd-available`, `rrd-data`, `node-status-available`,
@@ -827,11 +827,12 @@ clear`, `Found N new issues`, `Fixed N issues`, `N issues still open`, or
and raw tool traces are forensic context and must stay secondary to what
Patrol did.
2. Keep Patrol-specific copy and badge logic inside the governed Patrol presentation helpers instead of page-local branches
Patrol assessment copy must not present an all-clear health prediction while
active Patrol findings or Patrol runtime issues are still present. The
canonical summary helper owns that conflict resolution so the visible
assessment title, description, and compact metrics all speak from the same
current findings state. Patrol-owned runtime issues must stay
Patrol renders no assessment copy today. If it returns, it must not present
an all-clear health prediction while active Patrol findings or Patrol
runtime issues are still present, and one shared presenter must own that
conflict resolution so the visible assessment title, description, and
compact metrics all speak from the same current findings state.
Patrol-owned runtime issues must stay
distinct from infrastructure findings in assessment copy rather than being
described as infrastructure warning findings about Patrol itself.
Historical Patrol trust regressions must also suppress a green all-clear in
@@ -842,13 +843,24 @@ clear`, `Found N new issues`, `Fixed N issues`, `N issues still open`, or
tolerance and otherwise allows two configured Patrol intervals before
warning, so deliberately slower schedules are not mislabeled while an old
default-schedule result cannot remain green indefinitely.
Assessment coverage caveats must also reconcile against current run-history
proof: a stale coverage factor or prediction must not claim recent coverage
is incomplete when the latest completed full Patrol run successfully checked
real resources.
Recency coverage copy must also come from the shared presentation helper:
use verified wording only for successful full patrols, and use neutral
checked wording for failed full patrols or scoped activity.
No assessment readout renders today, so nothing reconciles a coverage
caveat against run history: once runtime states (running, blocked,
disabled, unavailable) and stale coverage have taken precedence, the
current-findings empty state withholds the all-clear (`Check needed`)
whenever the overall-health summary carries any
`coverage` factor, including the backend `Recent Patrol errors` factor that
can outlast a successful full Patrol run until three consecutive clean full
runs lead the coverage window or the erroring runs leave it
(`summarizeRecentPatrolCoverage` in `internal/ai/intelligence.go`), and
whenever the latest completed full run ended with errors or only targeted or
follow-up runs completed. If an assessment readout returns, its coverage
caveat must reconcile against current run-history proof: a stale coverage
factor or prediction must not claim recent coverage is incomplete when the
latest completed full Patrol run successfully checked real resources.
Recency coverage copy comes from the shared presentation helper
(`getPatrolRecencyPresentation`): `checked N resources` for the latest
completed run with a positive count, whatever its type or error state. It
never says `verified`.
3. Update this contract whenever a new Patrol-specific page, store, helper, or presentation component becomes canonical runtime surface area
4. Keep retired hosted-model and trial-like Patrol acquisition copy out of the
normal self-hosted GA app. Patrol may parse legacy transport fields, but
@@ -1496,15 +1508,29 @@ provider/settings affordances.
The feature surface now also keeps the same shell/runtime split internally:
`frontend-modern/src/features/patrol/PatrolIntelligenceSurface.tsx` owns feature
composition, the Patrol-owned section files under
`frontend-modern/src/features/patrol/` own the header, banner, summary, and
workspace render surfaces, and
`frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts` owns Patrol
state, transport, polling, and effect lifecycle. The shell and section surfaces
must not re-accumulate Patrol API calls, timer orchestration, or store refresh
semantics, and `frontend-modern/src/stores/aiIntelligenceSummaryModel.ts` now
`frontend-modern/src/features/patrol/` own the header, banner, workspace, and
tab-panel render surfaces (the Inbox attention list, Protection objectives, and
the Activity tab's `This week` digest and recent work), and
`frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts` owns the
page's shared Patrol state, transport, polling, and effect lifecycle. The shell
and the header, banner, and workspace surfaces must not re-accumulate those
Patrol API calls, timer orchestration, or store refresh semantics; the
attention list, objectives, recent work, and `This week` panels each own their
own typed read and refresh instead, and `frontend-modern/src/stores/aiIntelligenceSummaryModel.ts` now
owns the canonical summary normalization so Patrol consumers inherit one
governed recent-change and policy-posture snapshot instead of reintroducing
hook-local fallback logic.
The hook no longer returns the summary card's severity and fix tallies. Its
`summaryStats` accessor (critical, warning, and total active findings plus
fixed outcomes, counted client-side over the Patrol findings list) lost its
only reader when the Patrol summary card was deleted and has been removed,
together with the `canonical-patrol/no-local-summary-card-presentation` rule in
`frontend-modern/scripts/canonical-platform-audit.mjs`, whose regex matched only
text shaped like the deleted card's tint ternaries on
`summaryStats().criticalFindings`, `warningFindings` and `fixedCount`. The open-issue severity counts the page shows today come from
the Activity tab's `This week` card, which reads them from
`GET /api/ai/patrol/digest` (see that card's section below); a count surface
that returns must not revive a hook-local tally of the findings list.
That same Patrol hook boundary now consumes shared AI settings/model truth
through `frontend-modern/src/stores/aiRuntimeState.ts` instead of mounting its
own `/api/settings/ai` or `/api/ai/models` reads. Patrol-specific state still
@@ -1571,12 +1597,20 @@ compatible `handoff_metadata.kind=patrol_configuration_failure` plus only the
runtime-failure boolean needed for drawer/session presentation, so they restore
as a Patrol control issue without carrying raw provider, credential, command,
or retry payloads into the browser.
Successful provider-model saves that return
`patrol_readiness.status=not_ready` are still Patrol control issues, not silent
successes: the Patrol popover must keep the saved provider/model visible,
render a `Patrol control needs attention` inline state with the returned
readiness cause and summary; any Assistant handoff from that state must
describe a Patrol control issue rather than a save failure.
Successful Pulse Intelligence settings saves whose returned
`patrol_readiness.status` is `not_ready` or `warning` are still Patrol control
issues, not silent successes: Settings > Provider & Models, Patrol, Assistant
and Service context keep the saved non-secret values in the form and raise a
warning
toast (`<page> settings saved, but Patrol is not ready` or `... is degraded`)
with the readiness summary, provider and model instead of a success toast
(`getAISettingsPatrolReadinessSaveMessage` in
`frontend-modern/src/components/Settings/useAISettingsState.ts`). A Patrol
on/off toggle save whose response reports `not_ready` raises the fixed warning
`Patrol setting was saved, but Patrol is not ready to run.` in either
direction, because readiness is evaluated independently of `patrol_enabled`.
Patrol renders no inline control-issue state or Assistant handoff for either
case.
A separate browser proof in
`tests/integration/tests/78-monitor-first-patrol-workbench.spec.ts` keeps the
authenticated launch workbench monitor-first. When the runtime has monitored
@@ -1675,55 +1709,49 @@ must not show Pro trial CTAs, upgrade links, paid helper copy, or a plan-lock
upsell banner in the default Patrol workflow unless hosted mode, an explicit
commercial handoff, or an active entitlement makes those actions relevant.
That degraded empty-state copy must also interpret the finding state rather
than simply replaying the primary assessment sentence verbatim: when coverage
than replaying an `overall_health.prediction` sentence verbatim: when coverage
is incomplete, the findings panel should tell the operator to run Patrol to
complete coverage before trusting the all-clear. The page must not duplicate
the assessment prediction as a second independent status surface above current
issues.
Secondary metric strips must not render `No issues found` when the same
governed overall-health summary says coverage is incomplete or health still
requires attention; compact assessment derivation belongs to the shared
`frontend-modern/src/utils/patrolSummaryPresentation.ts` helper for Assistant,
history, and explicit context consumers rather than the default Patrol page.
Any explicit context assessment shell must also stay inside the shared Pulse
card language. It should use the same neutral `bg-surface` page-card base as
adjacent Pulse surfaces, carrying severity through compact header accents, icon
treatments, and badges instead of tinting an entire full-width assessment panel
as a one-off warning banner.
That supporting score chip must also avoid overstating infrastructure truth.
When the current state is dominated by incomplete coverage or Patrol-owned
runtime failures, the chip should read as an `Assessment` grade rather than an
`Health` grade; `Health` belongs to verified healthy infrastructure states.
That same helper also owns explicit assessment explanations. Assessment
consumers must not pair an `Issues detected` headline with a raw coverage-only
`overall_health.prediction` sentence from a separate source; when active
findings and incomplete verification are both true, the assessment should
describe both in one canonical message.
That same assessment contract must also distinguish Patrol-owned runtime
findings from infrastructure findings. When the only active Patrol findings are
synthetic Patrol service/runtime conditions such as the `ai-service`
provider-credit failure, the top assessment should read as a Patrol runtime
issue rather than implying infrastructure issues were detected across the
estate. When there is exactly one active Patrol runtime finding, that same
assessment copy should name the concrete runtime failure, such as
`Provider billing or quota issue`, instead of reducing the state to a generic count of
runtime findings.
That primary assessment must also expose a single visible recommended next
step derived from the same Patrol summary contract, not a page-local helper.
`frontend-modern/src/utils/patrolSummaryPresentation.ts` owns that decision
from pending governed approvals, Patrol runtime issues, active infrastructure
findings, verification posture, and the current assessment tone. Pending
approvals take priority over general triage, coverage-incomplete states must
ask for full verification before any all-clear claim, runtime impairment must
point the operator back to restoring Patrol visibility, and verified healthy
states should fall back to continued scheduled monitoring rather than
inviting generic Assistant chat.
When that recommendation has an immediate governed operator path, the same
helper must also declare the bounded action kind. The summary card may map
those action kinds only to existing Patrol controls: run a full Patrol,
review approval-scoped findings, review active findings, or open Patrol
provider settings. It must not
invent page-local remediation, approval, or execution authority from
No shared presenter derives a compact assessment, assessment shell, score
chip, assessment explanation, or recommended next step today. The helpers that
did (`getPatrolAssessmentPresentation`, `getPatrolAssessmentShellPresentation`,
`getPatrolScoreChipLabel`, `getPatrolCompactAssessmentLabel`,
`getPatrolSummaryMetricState`, `getPatrolAssessmentAction` and their siblings)
have had no production caller since the Patrol summary card was deleted
(several lost theirs months earlier) and are gone;
`frontend-modern/src/utils/patrolSummaryPresentation.ts` now owns only the
header recency line (`getPatrolRecencyPresentation`) and the run-history
verification posture the current-findings empty state reads
(`getPatrolVerificationPresentation`). If a summary card, metric strip, or
assessment readout returns, it must derive its state from one shared
Patrol-owned presenter rather than page-local logic, and keep these rules.
A secondary metric strip must not render `No issues found` while the governed
overall-health summary says coverage is incomplete or health still requires
attention. An assessment shell stays inside the shared Pulse card language:
the neutral `bg-surface` page-card base, with severity carried by compact
header accents, icon treatments, and badges rather than a tinted full-width
warning banner. A supporting score chip reads as an `Assessment` grade rather
than a `Health` grade when incomplete coverage or Patrol-owned runtime failures
dominate; `Health` belongs to verified healthy infrastructure states. An
`Issues detected` headline is never paired with a raw coverage-only
`overall_health.prediction` sentence from a separate source; active findings
and incomplete verification are described together in one message. When the
only active Patrol findings are synthetic Patrol service/runtime conditions
such as the `ai-service` provider-credit failure, the readout names a Patrol
runtime issue rather than implying infrastructure issues across the estate,
and with exactly one such finding it names the concrete failure, such as
`Provider billing or quota issue`. Any recommended next step comes from that
same presenter: pending approvals take priority over general triage,
coverage-incomplete states ask for a full check before any all-clear, runtime
impairment points the operator back to restoring Patrol visibility, and
healthy states fall back to continued scheduled monitoring rather than
inviting generic Assistant chat. When that recommendation has an immediate
governed operator path, the presenter declares a bounded action kind that maps
only to existing Patrol controls (run a full Patrol, review approval-scoped
findings, review active findings, or open Patrol provider settings), never to
page-local remediation, approval, or execution authority invented from
recommendation text.
If a summary card returns, that same runtime-owned assessment must expose the
fix path directly. When the primary Patrol issue is a Patrol runtime/provider
@@ -1801,8 +1829,8 @@ dismiss, resolve, or suppress controls for them. The correct operator path is
to fix Patrol provider configuration in Pulse Intelligence > Provider & Models
settings and rerun Patrol, optionally adding context notes, rather than hiding
the runtime issue.
That same runtime-versus-infrastructure split must carry through the summary
metrics strip as well. When Patrol-owned runtime issues are active, the
If a summary metrics strip returns, that same runtime-versus-infrastructure
split must carry through it. When Patrol-owned runtime issues are active, the
supporting metrics must stop counting them under generic infrastructure
`Warnings` or `Active findings`; the strip should break out `Runtime issues`
separately and reserve infrastructure finding counts for actual estate issues.
@@ -1810,16 +1838,18 @@ The findings list must respect that same trust priority. When Patrol-owned
runtime issues share a severity tier with ordinary infrastructure findings, the
runtime issue should sort first within that tier so Patrol blindness is not
buried under same-severity estate warnings.
The summary recency chip must follow the same governed scope distinction. When
the latest completed activity was only a scoped run, the summary should label
that timestamp as `Last activity` instead of `Last patrol`; `Last full patrol`
belongs only to the most recent completed full Patrol run.
That same distinction is transport-backed. `last_patrol_at` names the last
completed full Patrol sweep, while `last_activity_at` may advance on scoped
work or fix-verification checks without claiming a new full verification pass.
That same recency contract also applies to the header metadata row. The top
header must not revert to a generic `Last:` timestamp when the rest of Patrol
is explicitly distinguishing activity from full verification recency.
Patrol renders no summary recency chip today; the page-header recency line is
the only Patrol recency surface, and `getPatrolRecencyPresentation` labels it.
When run history holds a completed run, the line reads `Last check` for the
most recent one of any type: full, targeted, or follow-up. Otherwise it falls
back to the status transport: `Last check` with `last_patrol_at` when that is
the only timestamp, when it is at least as recent as `last_activity_at`, or when
only the activity timestamp fails to parse; otherwise `Last activity` with
`last_activity_at`. The transport keeps the
distinction: `last_patrol_at` names the last completed full Patrol sweep,
while `last_activity_at` may advance on scoped work or fix-verification checks
without claiming a new full verification pass. The header must not revert to
a generic `Last:` timestamp.
If Patrol renders a summary surface again, it must also avoid reintroducing a
second compact assessment or verification layer beneath the primary card. Supporting metric strips
belong to counts and outcomes such as active findings, critical findings,
@@ -1828,8 +1858,8 @@ verification labels in a second row that competes with the primary governed
assessment and verification copy above.
The same supporting-chip rule applies to timing: such a primary summary card
may show health and active-finding support, but it should not add another recency
pill once header metadata, verification, and findings footer already carry the
governed activity/verification timestamps.
pill, because the header recency line already carries the governed activity
timestamp.
That same Patrol-owned run-history surface must keep platform-backed system
counts canonical too. When the backend distinguishes API-backed TrueNAS
systems from unified-agent hosts in Patrol run history, the run-history chips
@@ -1844,16 +1874,20 @@ split also applies to findings counts: the primary assessment card may keep
health as supporting context, but active
findings, warning counts, and critical counts belong to the supporting metric
strip rather than being repeated as duplicate badges inside the primary card.
That same summary surface must also explain what Patrol actually checked
without turning coverage mechanics into the default product language. Recent
run history should drive a visible check summary that tells the operator
whether Patrol recently completed a clean broad check, only ran targeted alert
or anomaly-triggered checks, or ended its most recent broad check with errors,
so the page does not leave trust and coverage as implicit background knowledge.
When same-day run history shows both a recent broad check and a burst of
targeted or follow-up activity, that same surface should expose the recent
Patrol renders no check summary today. `getPatrolVerificationPresentation`
still classifies recent run history (a clean full check, a full check that
ended with errors, only targeted or follow-up checks, or no completed check)
and computes a same-day activity-mix label, but its only reader is the
current-findings empty state, which uses just its warning tone to withhold the
all-clear; the header recency line carries the visible coverage phrase. If a
check summary returns with that summary surface, it must explain what Patrol
actually checked without turning coverage mechanics into the default product
language: whether Patrol recently completed a clean broad check, only ran
targeted alert or anomaly-triggered checks, or ended its most recent broad
check with errors. When same-day run history shows both a recent broad check
and a burst of targeted or follow-up activity, that surface should expose the
activity mix in check language instead of asking operators to reconcile a
`Recently verified` headline with a busy Patrol strip elsewhere on the page.
`Recently checked` headline with a busy Patrol strip elsewhere on the page.
Fix-verification checks belong to that same explanation layer as follow-up
checks, not as evidence of a fresh full-estate sweep.
The same hierarchy keeps correlations, recent changes, and policy posture off
@@ -1880,8 +1914,9 @@ activity facts belong in run history, selected finding details, or explicit
secondary context, not as a second Patrol verdict label.
That same activity explanation should handle noisy Patrol behavior concretely.
When same-day history shows a mix of full sweeps, verification checks, and
scoped alert- or anomaly-triggered patrols, the verification/activity surface
should expose a compact breakdown of those run categories instead of leaving
scoped alert- or anomaly-triggered patrols, a returning verification/activity
surface should expose a compact breakdown of those run categories (Patrol
renders none today) instead of leaving
operators to infer why Patrol looked busy from an undifferentiated run count
alone.
The same operational layer should also surface scoped-trigger state directly.
@@ -2154,8 +2189,10 @@ requester identity as safe metadata, without copying the approval command
payload into Assistant.
That same store now owns the Patrol dashboard load bundle as well, so the
page refresh path stays aligned on a single orchestrated AI bundle instead of
repeating the individual summary, findings, approval, and correlation fetches
inline.
repeating the individual summary, findings, circuit-breaker, and approval
fetches inline. Neither that bundle nor the Patrol refresh path fetches the
global learned-correlation list: the Patrol page renders no correlations, so
the store holds no copy of that list.
The shared
`frontend-modern/src/components/Infrastructure/ResourceChangeSummary.tsx` and
`frontend-modern/src/components/Infrastructure/ResourceCorrelationSummary.tsx`
@@ -2326,10 +2363,10 @@ focused on product title, recency, and route controls instead of repeating
the same trust counters above the workspace tabs.
The same page-header recency line must also surface the coverage
signal from the most recent completed run via
`PatrolRecencyPresentation.resourcesChecked`, populated by
`getPatrolRecencyPresentation` from
`PatrolRunRecord.resources_checked`. The render reads "Last full
patrol: 3m ago — checked 47 resources" so operators see both
`PatrolRecencyPresentation.resourcesCheckedLabel` (raw count in
`resourcesChecked`), populated by `getPatrolRecencyPresentation` from
`PatrolRunRecord.resources_checked`. The render reads "Last check:
3m ago · checked 47 resources" so operators see both
temporal recency and coverage in one line. The field stays optional
(omitted when zero) so a degenerate run that completed without
checking any resources does not render a misleading "checked 0
@@ -2655,6 +2692,36 @@ clock while it stays open. Its `Last seen` and flapping window are latest
readings of a record that may have moved since, so they stay the ages at
read time, and the queue rows, summary, recent work, run history and digest
re-derive on their own 30- or 60-second re-reads.
Future times in Patrol count down instead of reading as ages, because the
past-only `formatRelativeTime` reads each of them as "just now" for their whole
duration. The attention detail's suppression line counts down to the expiry
with the exact time on hover and names the state expiry restores in the
early-end control's words, from the same
`getAttentionSuppressionEndPresentation` split on
`operationalRecord.acknowledgement`: an open issue reads `Returns to active
attention in 1d.`, one marked reviewed reads `Suppression ends in 1d. It stays
reviewed.` (expiry restores Acknowledged, which is not active attention), and a
passed expiry reads `Suppression has ended.`
(`getAttentionSuppressionExpiryLabel`). The findings list reads `Reminding in
6d` for a will_fix_later reminder and `snoozed, returns in 3h` for a snooze,
both through `formatTimeUntil` on the shared clock with the exact time on
hover. A reminder reads `Reminder overdue` once the panel's overdue id list
holds it. That list is sampled on the shared clock and is what the Overdue
commitments chip and filter count, so the label, chip and filter agree: a row
mounted or re-read between the deadline and the tick that adds it reads
`Reminder due`, and a row already on screen keeps its last countdown until that
tick. The list only notifies when its ids change, so a clock tick that moves no
deadline past now does not rebuild the open Overdue rows. The hourly reminder
sweep then brings the finding back whether or not it still trips. An ended
snooze reads `snooze ended` until the next findings re-read drops the snoozed
status: `GET /api/ai/patrol/findings` sends no status, so the frontend
`aiIntelligence` store (`normalizeFindingStatus`) derives `snoozed` on each
read only while `snoozed_until` is ahead, and the finding then shows as Active
unless it has since resolved or been dismissed.
`getFindingReminderPresentation` and `getFindingSnoozePresentation` in
`aiFindingPresentation.ts` own the findings copy.
`PatrolAttentionWorkbench.test.tsx`, `FindingsPanel.links.test.tsx` and
`FindingsPanel.test.ts` pin it.
### This week card answers what Patrol did for the customer
@@ -2712,9 +2712,26 @@ unavailable once the guest is stopped, whether its status or the Proxmox
`vm-stopped` reason says so, and Proxmox's own guest filesystems keep their
Proxmox read state. The two flags ride the existing row mapping and
add no query or poll; `GuestDrawer.test.tsx` and `useWorkloads.test.ts` pin
both sources. The workload table's disk cell and the drawer History still read
the aggregate disk metric and the Proxmox reason, and the agent's OS, addresses
and physical disk health carry no stale cue yet.
both sources. The workload table's disk cell (`useGuestRowState.ts`,
`GuestRow.tsx`) and the drawer History (`guestDrawerModel.ts`) judge the same
guest through the drawer rows' rule, `getWorkloadGuestDiskRead` in
`workloadGuestPresentation.ts`: a silent agent's bars carry the Last known
label, a live agent's bars stay current under Proxmox `agent-not-running`, and
History reads the aggregate disk metric as current, last known or unavailable
the same way. The rule covers the aggregate because Proxmox records no guest
disk usage without guest filesystems, so a manually linked guest then keeps the
agent's disk metric, and the poller no longer carries that agent reading
forward as Proxmox's own `prev-` read (see the monitoring contract).
`TestResourceRegistry_ManualGuestAgentLinkDiskMetricFollowsProxmoxGuestFilesystems`
pins that merge; `GuestRow.test.tsx` and `GuestDrawer.test.tsx` pin the cell and
History. On the websocket path (`resourceStateAdapters.ts`) a native Proxmox VM
facet's new guest-read outcome drops filesystems it no longer reports, a native
agent facet without `disks` drops the agent's, and the agent facet always
carries `stale`, so the rows switch source and lose the Last known label when
the agent resumes, without a reload. A stale Proxmox source
can still lose the aggregate to a live agent while its own filesystem rows
remain; such a guest stays on the Proxmox rule. The agent's OS, addresses and
physical disk health carry no stale cue yet.
Guest, node, and Docker-host drawer headers follow the same frontend-primitives dependency
boundary for collapse: Workloads owns which inline row is selected and the
close handler, while
@@ -1413,6 +1413,12 @@ state only. A restored prompt or restored message count may help an operator
continue a protected-item investigation, but it must not become backup coverage
evidence, recovery freshness, restore entitlement, storage-owner identity,
approval policy, or a provider-local recovery command handoff.
The POST-only guard on Assistant session mutations (`/abort`,
`/summarize`, `/fork`, `/undo`, `/redo`, `/steer`; `405` with `Allow: POST`
for any other method that reaches the handler) is transport hardening in
`internal/api/ai_handler.go`.
It opens no storage, backup, or recovery read or write path, and a rejected
request never reaches session state.
Approved Assistant tool execution through `internal/api/router_routes_ai_relay.go`
is also adjacent API/AI action plumbing for storage/recovery consumers.
`AssistantToolExecutor` / `ApprovedAssistantToolExecutor` may execute an already
@@ -3002,6 +3008,15 @@ recovery evidence. Unremembered overflow deliveries remain auditable with
`reason=capacity`; cache expiry, overflow, or restart is not a successful
storage operation or a reason to delete historical events.
### Alert settings saves keep unsent keys without a new storage path
`PUT /api/alerts/config` now applies only the top-level keys the body carries
and keeps stored values for the rest, so a thresholds save no longer turns off
alert TTL cleanup. It still writes `alerts.json` through the existing
`SaveAlertConfig` path, opens no storage or recovery route, and leaves backup
and snapshot alert settings under the same replace-what-was-sent rule as every
other key. `internal/api/alerting/alerts_test.go` pins the handler.
### Container diagnostics shares private bootstrap transport (1 October 2026)
Recovery-adjacent diagnostics now reuse the canonical complete installer
@@ -6905,6 +6920,15 @@ expires after ten minutes. Nothing is persisted, so a restart during the IdP
round trip refuses the returning Response and the user signs in again. No
storage, retention, backup, migration or recovery path is added or moved.
### A linked agent's stale flag and guest disk owner open no storage path
`internal/unifiedresources/types.go` now always sends `AgentData.Stale`, so a
resumed agent clears its stopped-reporting mark in browsers that merge agent
facets field by field, and `VMView.DiskFromLinkedAgent` keeps the poller from
carrying a linked agent's guest disk forward as a Proxmox read. Both describe
which source's reading is current; no backup, retention, migration or recovery
path is added or moved.
### Demo write guard ignores websocket upgrade headers
`internal/api/demo_middleware.go` no longer exempts every request carrying
@@ -6915,3 +6939,11 @@ allowlist it gets the generic demo `403`, or `404` on a hidden route, and that
includes config import at `/api/config/import`. HTTP/1.1 websocket handshakes
are `GET` requests and still connect. No storage, retention, backup, migration
or recovery path is added or moved.
### Demo mode hides the pprof routes
`internal/api/demo_mode_operations.go` adds the Go runtime profiling family
(`/debug/pprof` and every path below it, every method) to the public-demo
hidden routes, so the demo guard on a `DEMO_MODE` instance answers it with
`404`. That removes a diagnostic read surface on demo instances only; no
storage, retention, backup, migration or recovery path is added or moved.
@@ -2303,6 +2303,16 @@ evidence of what the array was doing when the agent went quiet, but without
live status colour. Where a retained rebuild percentage is shown, it reads
"Rebuild was at N%" with no speed or progress bar, and a member that was not
healthy names its state in its badge.
A Pulse agent manually linked into a guest carries the same signal there.
`AgentData.Stale` is always on the wire, because browsers merge agent facets
field by field (`resourceStateAdapters.ts`) and would keep a resumed agent's
omitted false as true; `TestAgentDataAlwaysSendsStale` pins it. That always
present `stale` also marks a native agent facet, so one without `disks` has
withdrawn them, and a native Proxmox VM facet's new guest-read outcome likewise
drops `disks` it no longer reports; either way the other source's filesystems
take over without a reload (`resourceStateAdapters.test.ts`). `VMView.DiskFromLinkedAgent` names the agent
as the owner of a guest's selected disk metric, which the monitoring poller
reads before it carries a Proxmox disk reading forward.
For SMART disk
temperatures on rows that still render, the provenance travels on
`agent.sensors.smart[].collection`: the Machines temperature cell
@@ -4614,26 +4624,29 @@ reconstructing a separate type-token summary in the emitter.
The same AI resource-intelligence payload now also carries canonical
correlation evidence from the shared detector, so the drawer can show learned
edge patterns alongside the dependency relationships without rebuilding correlation
reasoning from raw events. The Patrol intelligence page now also renders that
correlation evidence through the shared
reasoning from raw events. The drawer renders that correlation evidence
through the shared
`frontend-modern/src/components/Infrastructure/ResourceCorrelationSummary.tsx`
card, so the same learned-edge list stays governed by one frontend surface
instead of separate page-local implementations. That shared card also owns
card, and is that card's only caller today; the Patrol page renders no learned
correlations and the frontend does not load the global correlation list. Any later surface
that shows learned edges must reuse that card instead of a page-local
implementation. That shared card also owns
the first-class relationship-map surface for canonical `resource.relationships`,
the correlation ordering, and the truncation rule, so callers pass raw
relationships and correlation lists instead of encoding their own sort or
top-N behavior.
Canonical parent edges now also originate in this subsystem: `ParentID` is
folded into the facet relationship set through
`ResourceRelationshipsWithCanonicalParent` before any drawer or Patrol
consumer renders a relationship map, so pages do not rederive parent topology
`ResourceRelationshipsWithCanonicalParent` before any consumer renders a
relationship map, so pages do not rederive parent topology
from raw resource fields or invent relationship-map fallbacks locally.
The same surfaces now also render recent changes through the shared
The drawer renders recent changes through the shared
`frontend-modern/src/components/Infrastructure/ResourceChangeSummary.tsx`
card, so canonical timeline wording and ordering stay governed by one
frontend feed instead of separate page-local loops. Callers may suppress
resource-change metadata badges only for compact operator-context surfaces such
as Patrol's supporting context; the shared card still owns headline/reason
resource-change metadata badges only for compact operator-context surfaces;
no caller does today, and the Patrol page renders no recent changes. The
shared card still owns headline/reason
dedupe so prefixed backend reasons do not render as duplicated visible copy.
Assistant finding handoffs are part of that same timeline contract: when the AI
runtime needs recent changes for product-originated handoff resources, it should
@@ -5577,12 +5590,14 @@ wording stays aligned with the drawer's recent-change cards and timeline.
Timeline cards in that drawer surface change metadata when it is present, so
the history view preserves the richer provenance already carried by the
unified-resource model instead of flattening those fields away.
The same Infrastructure resource-only links now also default through the
shared `frontend-modern/src/components/Infrastructure/ResourceChangeSummary.tsx`
The shared `frontend-modern/src/components/Infrastructure/ResourceChangeSummary.tsx`
and `frontend-modern/src/components/Infrastructure/ResourceCorrelationSummary.tsx`
cards from the Patrol page, resource drawer, and problem-resource dashboard
panels, so canonical resource-filter path construction stays owned by the
shared summary cards rather than being duplicated per surface.
cards keep resource links behind an optional `buildResourceHref` input with no
default, following the 2026-05-16 cross-resource drilldown retirement above.
The resource drawer passes none, so its change and correlation labels render
as plain text; a surface that needs resource links must pass a platform-route
builder into the shared card rather than rebuilding resource-filter paths per
surface.
Platform tables supply the resource-label resolver to the resource drawer
through `PlatformResourceDetailTableRow`. `createPlatformResourceLabelResolver(...)`
in `frontend-modern/src/features/platformPage/PlatformResourceDetailTableRow.tsx`
@@ -6073,7 +6088,12 @@ the reference and is never retried. An owned row records its alert's
reference as `alert_resource_id`. A read by the reference returns the rows
journaled under it plus the rows owned away from it, matched through the
indexed alert identifier (`ProxmoxPhysicalDiskAlertIdentifiers`) and that
recorded reference, and every count uses the same predicate. The Alerts
recorded reference, and every count uses the same predicate. Those
identifiers cover the health and wearout specs and the disk temperature
metric spec (`<reference>::metric-threshold:diskTemperature`), since PVE disks
also raise temperature alerts under the same reference
(`TestProxmoxDiskTemperatureAlertRowsFollowRecordedHardwareIdentity` in
`internal/unifiedresources/store_test.go`). The Alerts
history Resource action therefore still lists every occurrence raised under
the path, across the disks that held it, and alert-centric reads keep their
occurrences (see the AI runtime contract's incident-history queries). Rows
@@ -6229,6 +6249,13 @@ presentation. `frontend-modern/src/components/Infrastructure/`
placement, signal, and snapshot context through the canonical resource drawer
and debug/source sections rather than introducing a VMware-only detail route,
drawer tab, or provider-local investigation shell.
The TrueNAS physical-disk drawer judges a current temperature reading by the
disk temperature policy, like the TrueNAS storage table:
`useResourceDetailDrawerDerivedState.ts` passes the alerts store's
`getDiskTemperatureThresholds` into `buildTrueNASDetailSections`, and the
Temperature row takes a warning tone only when `isPhysicalDiskRunningHot`
says the reading reached its disk type's alert trigger. A retained reading
keeps its muted last-known row, and without a resolver heat is not judged.
That same infrastructure consumer boundary also owns source selection
continuity. Settings infrastructure panels and platform/runtime pages must
keep canonical sources such as `truenas` and `availability` present in their
@@ -0,0 +1,40 @@
{
"version": 1,
"base_sha": "bdf0f76b7bf61d550b1cb8574497eef75b354acb",
"verified_at": "2026-10-07T14:17:03Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/api/ai.ts",
"frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts",
"frontend-modern/src/stores/aiIntelligence.ts"
],
"content_sha256": {
"frontend-modern/src/api/ai.ts": "03a3c67b18121618f60a3bde3718a27d0be94b9cbfd96328340d0f8d5b8909a0",
"frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts": "087cb28ff8c235f9c4b03ad3115021a40b6952b3a3c5844dbfc3eaa57d2753c0",
"frontend-modern/src/stores/aiIntelligence.ts": "bd947cc05d31c9ad126f6c992555bf8cffa8ee23eae88598485fa9fb9d812f29"
},
"routes": [
"/patrol"
],
"viewports": [
{
"width": 1440,
"height": 900
},
{
"width": 390,
"height": 844
}
],
"states": [
"Private DEMO_MODE mock stack (demo/demo, PULSE_TELEMETRY=false) serving an embedded production build of this tree on base bdf0f76b7b, plus a before build of unmodified main on a second port for comparison; headless Chromium at 1440x900 and 390x844.",
"Patrol Inbox renders unchanged at both widths: Watch only mode row, Check now and Settings, Inbox/Protection/Activity tabs, the 7-decision inbox with Start review, Review 1 approval and Refresh. No console errors after the change.",
"Before: one GET /api/ai/intelligence/correlations per initial Patrol load, browser reload, 60 s poll tick and tab hide/show at both widths. After: zero in every phase. Every other /api/ai/* read in each phase is identical before and after.",
"The production bundle no longer contains the /ai/intelligence/correlations string or the 'Failed to load correlations' log (control string /ai/intelligence still present); check-bundle-size passes with no chunk renamed.",
"Not covered in the browser: the Patrol could not refresh banner's Retry path, which every Patrol read swallows so it cannot be forced from the network; it calls the same background reader edited here and is covered by unit tests."
],
"interactions": [
"Signed in through the demo auto sign-in, opened /patrol, reloaded the page, waited 68 s for the 60 s poll, and simulated a tab hide then show (visibilitychange) to trigger the visibility reload, counting /api/ai/* requests per phase with Playwright.",
"Repeated the load, reload and tab hide/show phases on the final rebased build at both widths and inspected the screenshots."
]
}
@@ -0,0 +1,60 @@
{
"version": 1,
"base_sha": "48c2d6cdce8d019151e3567873de88e21b76bb22",
"verified_at": "2026-10-07T19:21:21Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx",
"frontend-modern/src/components/Workloads/GuestRow.tsx",
"frontend-modern/src/components/Workloads/guestDrawerModel.ts",
"frontend-modern/src/components/Workloads/useGuestRowState.ts",
"frontend-modern/src/utils/resourceStateAdapters.ts",
"frontend-modern/src/utils/workloadGuestPresentation.ts"
],
"content_sha256": {
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx": "1cf03b76a8d8732db0fda9b2ec30c0245c350eadcb36a36909493a9f4d6e7831",
"frontend-modern/src/components/Workloads/GuestRow.tsx": "c7b8bb81f158d8d64195be65fa69b8ee18019414e602043c5413693efc06d1d6",
"frontend-modern/src/components/Workloads/guestDrawerModel.ts": "a30e4435523c0d30f618eb7f0e803cfba9a3a5203851d642f1d5b25c2f33b10d",
"frontend-modern/src/components/Workloads/useGuestRowState.ts": "1c41975ec9447f3657cd4e1cf0e9fc6ce187fdca18e7f050f882ac5d483379c9",
"frontend-modern/src/utils/resourceStateAdapters.ts": "35e7e01344e2f78cfcf550cc0b277aa06e42c544a110a6c33b054d97d9b16765",
"frontend-modern/src/utils/workloadGuestPresentation.ts": "3d1458ee02891047f722d643e74978fac6ec4edae5a24f5eb1808331de2e3b8d"
},
"routes": [
"/proxmox?q=checkout-web-01",
"/proxmox?q=checkout-web-02",
"/proxmox?q=analytics-batch-01",
"/proxmox?q=checkout-web-193",
"/standalone"
],
"viewports": [
{
"width": 1440,
"height": 900
},
{
"width": 1920,
"height": 1000
},
{
"width": 390,
"height": 844
}
],
"states": [
"Private mock stack (PULSE_MOCK_MODE, embedded production build of this tree on origin/main dace9295c6, rebased onto 48c2d6cdce which changes none of the verified frontend sources) with a verification-only generator overlay: Proxmox guest filesystems stripped to usage -1 + agent-not-running (or vm-stopped) for checkout-web-01/02/193, and standalone agents Apollo-114 and Meridian-731 held offline from a control file",
"Agents linked into VMs through POST /api/resources/{vm}/link: Apollo-114 into checkout-web-01 (silent), Spectrum-709 into checkout-web-02 (live), Meridian-848 into analytics-batch-01 (Proxmox filesystems kept, control), Meridian-731 into stopped checkout-web-193 (silent); /api/resources confirmed agent.stale, agent.disks, absent proxmox.disks and the agent-sourced metrics.disk",
"Mock-mode websocket state does not apply manual links (record-ingested resources skip applyManualLinks; chipped separately), so guest passes ran the Proxmox page on the linked REST resources with the websocket held by Playwright routeWebSocket",
"Before (main frontend + backend, same data): checkout-web-01 and checkout-web-02 disk cells read '- Unavailable' with 'Guest agent not running' and History deferred the disk on that message at 1440 and 390",
"After, live agent (checkout-web-02): agent filesystem bars with no read-status label, nothing deferred in History, drawer filesystem rows current (1440 and 390)",
"After, silent agent (checkout-web-01): bars plus 'Last known' (phone 'Prior') with 'Using last known disk stats. The Pulse Agent in this guest stopped reporting.', drawer filesystem rows last known, History disk deferred on the same message (1440 and 390)",
"After, stopped guest (checkout-web-193): '- Unavailable' / 'N/A' with the vm-stopped message in cell, drawer and History; analytics-batch-01 with Proxmox filesystems unchanged",
"Machines (/standalone at 1920): Apollo-114 held silent read '(stale)' with the stopped-reporting title; after releasing the hold mid-session without a reload, main kept '(stale)' although /api/state had dropped stale, while this build received stale:false and cleared it with metrics resuming",
"No page errors or console errors on /proxmox (guest rows, drawers, History) and /standalone, with and without the websocket"
],
"interactions": [
"Playwright 1440x900 and 390x844: open /proxmox?q=<guest> for four guests, read the disk cell text, read-status notice and bar, open the guest drawer (phone: second visible cell), read the filesystem status line, switch to History and read the disk current/last-known/deferred legends; screenshots of rows and History",
"Same pass against a before binary built from main's frontend and backend on the same data dir",
"Playwright 1920x1000 on /standalone: watch Apollo-114 while the control file releases its silent hold, polling /api/state for the agent's stale flag and re-reading the row without reloading",
"API: GET /api/resources/{vm} and /api/state before and after linking and after backend restarts"
]
}
@@ -0,0 +1,69 @@
{
"version": 1,
"base_sha": "bdf0f76b7bf61d550b1cb8574497eef75b354acb",
"verified_at": "2026-10-07T14:11:27Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/Alerts/AlertResourceTableDesktop.tsx",
"frontend-modern/src/components/Alerts/AlertResourceTableMobile.tsx",
"frontend-modern/src/components/Alerts/ResourceTable.tsx",
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx",
"frontend-modern/src/components/Alerts/ThresholdsTableTrueNASTab.tsx",
"frontend-modern/src/components/Alerts/alertResourceTableModel.ts",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerTrueNASModel.ts",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts",
"frontend-modern/src/features/alerts/alertsConfigurationModel.ts",
"frontend-modern/src/features/alerts/thresholds/hooks/useThresholdsPlatformData.ts",
"frontend-modern/src/features/alerts/thresholds/trueNASDiskTemperature.ts",
"frontend-modern/src/types/alerts.ts",
"frontend-modern/src/utils/alertThresholdDefaults.ts"
],
"content_sha256": {
"frontend-modern/src/components/Alerts/AlertResourceTableDesktop.tsx": "4d09dd97adc2868889e256455c207b6b9089ea311713878f0d6ff00efb6b7123",
"frontend-modern/src/components/Alerts/AlertResourceTableMobile.tsx": "ceaf136da9361a5b1756350626a833da4dfebcb4f211c7269d7d89d37fb7e085",
"frontend-modern/src/components/Alerts/ResourceTable.tsx": "7e80a6c789b3069fcadcb86e0a6689e5319e7b0ac5d5bfa1a596520daec2d7e1",
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx": "fc4e9f4d05c0bf5799acbbe64c37b901e104869ff120819342432563789ad0e9",
"frontend-modern/src/components/Alerts/ThresholdsTableTrueNASTab.tsx": "a22b86c28b65ec18ba81171fb1206a24107a0d97db19d7dd488bdfd0a689a956",
"frontend-modern/src/components/Alerts/alertResourceTableModel.ts": "5881aeaf7ee575589997bf67fb713a01784ce7ca16ecb5a5eaea77f36010837a",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerTrueNASModel.ts": "b60ba59688d6ea0cabfb2b413b8adcaa85c8d41524c1d1f0df9bdb9e6d85733c",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts": "63b07de3f354f8328298348a8892ad73fde2f11ec6768e9c0f6fc37eb4c43516",
"frontend-modern/src/features/alerts/alertsConfigurationModel.ts": "0e116e65c992a72e21d700835b0cb0db7173be9ec9c4980382958506d4e75976",
"frontend-modern/src/features/alerts/thresholds/hooks/useThresholdsPlatformData.ts": "8525b0b012198ff87e29063908a9907b7fa1080cd84187202b1cd3cf198a3b46",
"frontend-modern/src/features/alerts/thresholds/trueNASDiskTemperature.ts": "80433d42dd4658a1449c0f0c378d7fa52cd6dfab117d626195099768e750352f",
"frontend-modern/src/types/alerts.ts": "d072175ab98e3785eae643d655be4c2d54cac0eddc8b55803a2841a312062e5a",
"frontend-modern/src/utils/alertThresholdDefaults.ts": "077e291d5c224063a373a5e9f5555d9b470f585801b0cfdd9dc1acacf233af28"
},
"routes": [
"/alerts/thresholds/truenas",
"/truenas/storage"
],
"viewports": [
{
"width": 1366,
"height": 900
},
{
"width": 375,
"height": 812
}
],
"states": [
"Private mock stack (8 nodes, mock TrueNAS truenas-main) with a local-only generator overlay pinning TrueNAS disk temperatures: nvme0n1 60C, nvme1n1 72C, sda 57C (SATA), sdc 63C (SAS); after = this branch, before = main logic in the same tree",
"Thresholds > TrueNAS > Disks with no saved TrueNAS-wide value: header reads 'Defaults: NVMe 70\u00b0C \u00b7 SAS 65\u00b0C \u00b7 SATA 55\u00b0C', note 'Unless set here, each disk alerts at Disk temperature by type, set under Agents.', Global Defaults Temp cell empty with placeholder 'By type' (fits after widening to w-20) and On, disk rows nvme0n1/nvme1n1 70\u00b0C, sda/sdb 55\u00b0C, sdc-sdf 65\u00b0C",
"Same section at 375px: Global Defaults card shows TEMP \u00b0C 'By type' with On, disk cards show their type triggers, no horizontal overflow",
"Agent Disk Temp default switched off through the API: header 'Defaults: all metric alerts off', Global Defaults cell Off with title 'Off because the agent Disk Temp default under Agents is off...', disk rows Off",
"Saved TrueNAS-wide 62: header 'Defaults: Temp 62\u00b0C', every disk row 62\u00b0C, Custom badge and reset control after reload; saved 55 survives save and reload as 55 (stored with truenasDiskTemperatureByType)",
"Alerts with factory per-type thresholds: after stack raises nvme1n1 72 (trigger 70) and sda 57 (trigger 55) with diskType metadata, none for nvme0n1 60 or sdc 63; before (main) raised all four at a flat 55",
"TrueNAS storage table agrees: Attention 'Disk temperature is 72\u00b0C, at or above its 70\u00b0C alert threshold.' on nvme1n1 and '57\u00b0C ... 55\u00b0C' on sda, none on nvme0n1",
"TrueNAS disk drawer (Platform details > Show TrueNAS > Health): nvme0n1 Temperature 60\u00b0C in default text colour (main toned it amber from the fixed 55C); sda 57\u00b0C amber rgb(252,211,77) at 1366 and at 375 with no overflow"
],
"interactions": [
"Typed 62 into the TrueNAS Disks Global Defaults Temp cell, saved, reloaded: API truenasDiskDefaults {trigger 62, clear 57}, cell shows 62 with Custom",
"Triple-clicked and typed 55, saved, reloaded: API keeps {55, 50}, cell shows 55 with Custom and the reset control",
"Clicked Reset to factory defaults: cell returns to 'By type'; saved: API truenasDiskDefaults {}",
"On the 375px card, clicked On to switch the default Off, saved: API stores {trigger 0, clear 0} and the after stack's TrueNAS disk alerts resolve; clicked again: returns to 'By type', saved: {}",
"With the agent Disk Temp default off, clicked the Off badge: stages an explicit 55 (Custom, rows 55\u00b0C); discarded",
"Loaded an old saved config (flat 55/50, no marker) with the new binary: GET returns truenasDiskDefaults {} and truenasDiskTemperatureByType true",
"Expanded nvme0n1 and sda rows on /truenas/storage, opened Platform details and Show TrueNAS, read the Temperature row value and computed colour at 1366 and 375"
]
}
@@ -0,0 +1,52 @@
{
"version": 1,
"base_sha": "7eb32f318100fef045d63414fd63304c62acfdef",
"verified_at": "2026-10-07T16:55:00Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/shared/InlineDetailTableRow.tsx"
],
"content_sha256": {
"frontend-modern/src/components/shared/InlineDetailTableRow.tsx": "e5b27178cb3268132595943058849b3f619da22fcbd7916dfc301d4e852ef244"
},
"routes": [
"/standalone/availability",
"/standalone/machines",
"/proxmox",
"/docker",
"/kubernetes"
],
"viewports": [
{
"width": 1440,
"height": 900
},
{
"width": 1000,
"height": 800
},
{
"width": 768,
"height": 1024
},
{
"width": 390,
"height": 844
},
{
"width": 320,
"height": 700
}
],
"states": [
"Private stacks served by Vite from this tree (Playwright 1.56.1 headless Chromium): a non-mock stack with one HTTP availability check, then a mock estate (3 Proxmox nodes, 2 Docker hosts, 1 Kubernetes cluster, 2 generic hosts).",
"Before (origin/main class max-w-[calc(100vw-3.5rem)]): at 390px every opened drawer (availability check, Proxmox node, Proxmox guest, Docker host, Docker container, Kubernetes node, Machines agent) was 334px inside a 361px visible table, leaving a 27px empty strip; at 768px a table forced wider than its shell got a 712px drawer in a 695px visible area.",
"After (max-w-[100cqi]): drawer content equals the scroll shell's visible width at 320 (291px), 390 (361px), 768 (695px) and 1000 (916px) with no right gap, including the forced-wider table; 1440px keeps the static lg layout (max-width none, 1320px). The availability drawer's hostname now shows in full instead of truncating.",
"Long unwrapped identity chips (e.g. proxmox-lxc-docker:Production West:pve1:107) were already clipped at the content edge before (Kubernetes 20px, Docker 8px and 83px) and are clipped less after (7px, 70px); chip wrapping is a separate follow-up.",
"Sticky positioning is inert in both versions because .table-fixed td sets overflow hidden; not changed here. This is not an installed-build, physical-phone or release acceptance."
],
"interactions": [
"Opened each drawer by its disclosure button (row tap at phone width where the button is hidden), measured the scroll shell, detail cell, content shell and every descendant's right edge, and captured screenshots at 390px before and after.",
"Forced a 900px table inside its shell at 390px and 768px and scrolled it 300px sideways to compare drawer width against the visible area for both classes."
]
}
@@ -0,0 +1,51 @@
{
"version": 1,
"base_sha": "7eb32f318100fef045d63414fd63304c62acfdef",
"verified_at": "2026-10-07T16:47:38Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/AI/FindingsPanel.tsx",
"frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx",
"frontend-modern/src/utils/aiFindingPresentation.ts",
"frontend-modern/src/utils/format.ts"
],
"content_sha256": {
"frontend-modern/src/components/AI/FindingsPanel.tsx": "6b50ebbf3bbe24e31eebdb3c8be8ac6ca9c08405352facbfcd294a1361d8a2f5",
"frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx": "9055b69986cd4d320fd2194dba2819c0643e63326724952fd1021325530e9f5b",
"frontend-modern/src/utils/aiFindingPresentation.ts": "29e212a8d0aed4c1eb17697bbed86519ed82f64bc2940239db4a79470a23a090",
"frontend-modern/src/utils/format.ts": "abaec8a1f40c159e9a58702b726ccaea196bf6b72825c97addbee00af3bd7809"
},
"routes": [
"/patrol",
"/patrol/activity"
],
"viewports": [
{
"width": 1440,
"height": 900
},
{
"width": 390,
"height": 844
}
],
"states": [
"Private mock stack (backend :7793 built from this tree on origin/main 7eb32f3181, Vite :5303 serving it) with Playwright page.clock installed before load and the data held: the /ws proxy dropped every server frame and answered the client heartbeat with a data-free pong, every GET /api/ read was replayed from its cached body (Patrol findings reads left pending), and the clock was moved with setSystemTime plus a 31-second runFor so only the shared 30-second tick fired; one websocket connection and no attention detail re-read for the whole hold",
"Unreviewed suppressed issue (ops-prometheus Docker service health, suppressed through the real form for the default 24 hours; the form's duration read 'Return it to active attention after'): Lifecycle reads 'Returns to active attention in 1d.' with hover 'Suppressed until 8 Oct 17:36' and button 'Return to active attention', then 'in 20h.' at +4h, 'in 29m.' at +23h20m (28m at 390) and 'Suppression has ended.' at +25h, at 1440 and 390",
"Reviewed then suppressed issue (esxi-07.lab.local VMware host health, Mark reviewed then suppressed for 24 hours from its detail; the form's duration read 'End suppression after'): Lifecycle reads 'Suppression ends in 1d. It stays reviewed.' with button 'End suppression', then 'in 20h' at +4h, 'in 29m' at +23h20m (28m at 390) and 'Suppression has ended.' at +25h, at 1440 and 390",
"Patrol activity, Finding options and history, Resolved: mail-gateway-us dismissed through Finding options, Dismiss: Later reads '(will fix later) \u00b7 Reminding in 7d' with hover 'Reminder due 14 Oct 17:36. Pulse brings this finding back on its first reminder check after that.'; at +7d1m it reads 'Reminder overdue' and the 'Overdue commitments (1)' chip appears on that tick with findings reads still pending, and the chip filters to that finding, at 1440 and 390",
"Between ticks: page clock set 5 seconds past that reminder's deadline with no timer fired, then switching from Active to Resolved mounts the row reading 'Reminder due' with no Overdue chip; one 31-second runFor later the row reads 'Reminder overdue' and the 'Overdue commitments (1)' chip appears together, at 1440 and 390",
"Same list, dr-vault snoozed for 1 hour through POST /api/ai/patrol/snooze (Patrol rows hide the snooze buttons): reads 'snoozed, returns in 49m' with hover 'Snoozed until 7 Oct 18:36', 'in 9m' at +40m (8m at 390) and 'snooze ended' at +70m; after releasing the hold the next findings read lists dr-vault under Active, at 1440 and 390",
"No 'just now' on any future time at either viewport (the only 'just now' left is the attention detail's read-time Last seen age)",
"Same-session controls under the running Vite: with FindingsPanel.tsx and aiFindingPresentation.ts swapped to the first draft of this change, the between-ticks row read 'Reminder overdue' with no Overdue chip; with FindingsPanel.tsx, PatrolAttentionWorkbench.tsx, aiFindingPresentation.ts and format.ts swapped back to origin/main, the reviewed issue read 'Returns to active attention just now.' before and after +4h, the reminder 'Reminding just now' before and after +7d with no Overdue chip, and the snooze 'snoozed until just now'"
],
"interactions": [
"Opened a decision by its Patrol attention link, filled the suppression reason and submitted Suppress temporarily with the default 24 hours",
"Opened a second decision by its attention link, clicked Mark reviewed, reopened it and suppressed it for 24 hours",
"Opened Finding options and history, opened Finding options on a finding, chose Dismiss: Later and Confirm Dismiss",
"Opened Reviewed and suppressed, opened each suppressed issue from the list, read the Lifecycle line, its hover title and the early-end button",
"Switched the findings filter between Active and Resolved, read reminder and snooze hover titles, stepped the page clock and clicked the Overdue commitments chip",
"Released the hold and switched to Active to see the ended snooze re-derived",
"Repeated every pass at 1440x900 and 390x844, then reran with the first draft and with origin/main's components as controls"
]
}
@@ -0,0 +1,39 @@
{
"version": 1,
"base_sha": "cb5ffc2efe2b7d821f59eb26406d0616cf62e23d",
"verified_at": "2026-10-07T14:53:12Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/features/patrol/PatrolIntelligenceHeader.tsx",
"frontend-modern/src/utils/patrolSummaryPresentation.ts"
],
"content_sha256": {
"frontend-modern/src/features/patrol/PatrolIntelligenceHeader.tsx": "c7a603bb6741502acb0972bd9be241de089bf8106e88042ab6a16a8f4932ef48",
"frontend-modern/src/utils/patrolSummaryPresentation.ts": "6e7e9dd5a6e49c12aa90431bbad3650272798a9f216630fcc02392887d73a262"
},
"routes": [
"/patrol",
"/patrol (Activity > Finding options and history > Open work)"
],
"viewports": [
{
"width": 1440,
"height": 1000
},
{
"width": 390,
"height": 844
}
],
"states": [
"Embedded production builds (vite build into internal/api/frontend-modern/dist, go build) of the parent tree (cb5ffc2efe) and this tree, each on a private DEMO_MODE mock stack with a fresh data dir; same Playwright script against both.",
"Header recency line from getPatrolRecencyPresentation: clean full run 'Last check: 4m ago \u00b7 checked 1377 resources'; targeted-only run '... \u00b7 checked 3 resources'; full run with 2 errors '... \u00b7 checked 40 resources'. No 'verified' wording in any state.",
"Findings empty state through getPatrolVerificationPresentation (findings, runs and overall_health stubbed via page.route): clean full run + grade A -> 'No current issues / Checked 1377 resources.'; targeted-only run -> 'Check needed / Run Patrol to check everything and refresh open work.'; full run with errors -> 'Check needed'; grade A with a 'Recent Patrol errors' coverage factor after a clean full run -> 'Check needed' (pre-existing behaviour, filed separately).",
"Parent and this tree produced identical recency and empty-state text in all 8 scenario x viewport cases; no horizontal overflow; no console errors.",
"Dist proof: 187 files each side; 93 byte-identical, 93 differ only in chunk-hash references, index CSS drops exactly one rule (.bg-emerald-50/60, used only by the deleted assessment shell); check-bundle-size.mjs passes."
],
"interactions": [
"Demo sign-in, navigate to /patrol, read the header recency line, open the Activity tab, expand 'Finding options and history', read the Open work empty state; desktop mouse and phone touch contexts.",
"Screenshots of the header and empty state captured and inspected at both widths."
]
}
@@ -0,0 +1,39 @@
{
"version": 1,
"base_sha": "8e1a564d7e229bda51c021c7e6bde920e0ca0596",
"verified_at": "2026-10-07T14:35:42Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx"
],
"content_sha256": {
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx": "3052a190a76e885e98ed6601b3e83c8959e1c656d12270b87e074fffa95860ac"
},
"routes": [
"/alerts/thresholds/systems",
"/alerts/overview",
"/proxmox/storage?tab=disks"
],
"viewports": [
{
"width": 1366,
"height": 900
},
{
"width": 375,
"height": 812
}
],
"states": [
"Private mock stack (8 PVE nodes) on this branch with a local-only generator overlay pinning PVE disks: pve1 NVMe 63C, pve2 SATA 56C, pve3 NVMe 72C, pve3 SAS 63C, pve3 SATA 66C",
"Alerts > Thresholds > Systems (Agents section): 'Disk temperature by type' card reads 'Alert trigger in \u00b0C for each disk type, for agent, TrueNAS and Proxmox disks. ... a TrueNAS Disks value replaces them for TrueNAS disks.' at 1366, and at 375 the card spans 34-341px with no horizontal overflow",
"Alerts overview: 'Samsung 870 EVO 1TB (/dev/nvme2) (Temperature) CRITICAL on West Production C \u2014 Temperature 66\u00b0C, above the 55\u00b0C alert level; Clears once it drops to 50\u00b0C or lower and stays there for 5 minutes', Alert level 55\u00b0C, Clear level 50\u00b0C",
"Warning-level PVE disk temperature alerts (SATA 56C trigger 55, NVMe 72C trigger 70) appear after the 300s stability window (first breach 15:26:32, firing by 15:31:46); the critical 66C fires at once; none for NVMe 63C or SAS 63C",
"Physical Disks filtered to West Production: exactly the three alerting disks read Running Hot (Crucial MX500 56\u00b0C, WD Black SN850X 72\u00b0C, Samsung 870 EVO 66\u00b0C); the 870 EVO drawer reads 'Running Hot \u2014 Disk temperature is 66\u00b0C, at or above its 55\u00b0C alert threshold.'"
],
"interactions": [
"Opened Alerts overview and read the PVE disk temperature alert card",
"Filtered Physical Disks by 'West Production C', expanded the Samsung 870 EVO row",
"Opened Thresholds > Systems, read the Disk temperature by type card at 1366 and at 375"
]
}
@@ -0,0 +1,41 @@
{
"version": 1,
"base_sha": "421d1e5423e5b83b7923ea8c3b133ad520cd5f60",
"verified_at": "2026-10-07T15:23:47Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts"
],
"content_sha256": {
"frontend-modern/src/features/patrol/usePatrolIntelligenceState.ts": "0fef3795b2a6e2bcb1a979583deceeabd8536868adcb08eb383425dd97bb85ad"
},
"routes": [
"/patrol",
"/patrol (Activity tab)",
"/patrol (Protection tab)",
"/patrol?attention=<first attention row>"
],
"viewports": [
{
"width": 1440,
"height": 900
},
{
"width": 390,
"height": 844
}
],
"states": [
"Embedded production build of this tree (vite build into internal/api/frontend-modern/dist, go build) on a private DEMO_MODE mock stack (port 7693, fresh data dir, telemetry off).",
"Inbox: tablist Inbox/Protection/Activity with Inbox selected; '50 decisions need you' attention list with 5 rows (for example 'Open clickhouse-2 · Docker container memory limit'); header recency line 'Last check: just now · checked 1377 resources'.",
"Activity: 'This week' digest card ('Patrol runs 11', 'New issues 6', '6 still open (2 critical, 4 warning)') above Recent work; Protection panel renders.",
"No console errors and no page errors at either width.",
"Dist proof against the parent tree (built at 333e51d2de, whose frontend-modern tree is identical to the parent 421d1e5423): 187 files each side; 94 byte-identical, 92 differ only in chunk-hash references and SRI strings; only assets/AIIntelligence-*.js differs in code, and after normalising hashes and 1-3 character identifiers its only residual is the removed summaryStats closure and its summaryStats entry in the returned state object. check-bundle-size.mjs passes."
],
"interactions": [
"Demo auto sign-in, navigate to /patrol, read the header and Inbox attention rows.",
"Click the first attention row: URL gains ?attention=<id> and the detail panel opens; Escape, reload /patrol.",
"Click Activity, Protection, then Inbox tabs; Inbox re-selects.",
"Same script in a 1440x900 desktop context and a 390x844 phone context; screenshots of Inbox, opened row and Activity inspected at both widths."
]
}
+3 -1
View File
@@ -891,7 +891,9 @@ For an existing incident whose notifications should be paused without clearing
it, use the snooze and unsnooze endpoints below.
- `GET /api/alerts/config`
- `PUT /api/alerts/config`
- `PUT /api/alerts/config` — replaces only the top-level keys the body
carries; a key left out keeps its stored value, and the merged config is
normalized as before. The body must be a JSON object.
- `GET /api/alerts/deadman/config` — returns only whether an external watchdog
is configured; `pingUrl` is `***REDACTED***` when present and never returns
the credential-bearing URL
@@ -130,7 +130,6 @@ const ALLOWLIST = new Set([
'src/utils/remediationPresentation.ts',
'src/utils/patrolEmptyStatePresentation.ts',
'src/utils/patrolRunPresentation.ts',
'src/utils/patrolSummaryPresentation.ts',
'src/utils/rbacPresentation.ts',
'src/utils/rbacPermissions.ts',
'src/utils/systemLogsPresentation.ts',
@@ -581,13 +580,6 @@ const HELPER_RULES = [
message:
'Do not define local AI quickstart credits badge presentation in page code. Use @/utils/aiQuickstartPresentation instead.',
},
{
rule: 'canonical-patrol/no-local-summary-card-presentation',
regex:
/\bsummaryStats\(\)\.criticalFindings\s*>\s*0\b[\s\S]{0,320}bg-red-50[\s\S]{0,320}text-red-600|\bsummaryStats\(\)\.warningFindings\s*>\s*0\b[\s\S]{0,320}bg-amber-50[\s\S]{0,320}text-amber-600|\bsummaryStats\(\)\.fixedCount\s*>\s*0\b[\s\S]{0,320}bg-green-50[\s\S]{0,320}text-green-600/g,
message:
'Do not define local patrol summary card presentation in page code. Use @/utils/patrolSummaryPresentation instead.',
},
{
rule: 'canonical-patrol/no-local-empty-state-copy',
regex:
@@ -266,12 +266,6 @@ describe('AIAPI', () => {
expect(apiFetchJSONMock).toHaveBeenCalledWith(
'/api/ai/intelligence?resource_id=vm%2F100%3Ffilter%3Dall',
);
apiFetchJSONMock.mockResolvedValueOnce({} as any);
await AIAPI.getCorrelations('storage/1?filter=all');
expect(apiFetchJSONMock).toHaveBeenCalledWith(
'/api/ai/intelligence/correlations?resource_id=storage%2F1%3Ffilter%3Dall',
);
});
it('treats 402 responses from optional AI paywalled collections as empty state', async () => {
-8
View File
@@ -20,7 +20,6 @@ import type {
} from '@/types/ai';
import type {
AnomaliesResponse,
CorrelationsResponse,
IntelligenceSummary,
LearningStatusResponse,
ResourceIntelligence,
@@ -154,13 +153,6 @@ export class AIAPI {
return (await this.fetchIntelligence(resourceId)) as ResourceIntelligence;
}
static async getCorrelations(resourceId?: string): Promise<CorrelationsResponse> {
const params = resourceId ? `?resource_id=${encodeURIComponent(resourceId)}` : '';
return apiFetchJSON(
`${this.baseUrl}/ai/intelligence/correlations${params}`,
) as Promise<CorrelationsResponse>;
}
// Analyze a Kubernetes cluster with AI
static async analyzeKubernetesCluster(clusterId: string): Promise<AIExecuteResponse> {
return apiFetchJSON(`${this.baseUrl}/ai/kubernetes/analyze`, {
@@ -37,6 +37,7 @@ import { AIAPI, type RemediationPlan } from '@/api/ai';
import { createSuppressionRuleFromFinding, reinvestigateFinding } from '@/api/patrol';
import type { PatrolRunRecord, PatrolRuntimeState, PatrolAutonomyLevel } from '@/api/patrol';
import { formatRelativeTime } from '@/utils/format';
import { useRelativeTimeNow } from '@/utils/relativeTimeClock';
import {
getFindingAlertIdentifier,
hasTriggeringAlert,
@@ -82,6 +83,8 @@ import {
getFindingSubjectPresentation,
getFindingTitlePresentation,
getFindingRecencyPresentation,
getFindingReminderPresentation,
getFindingSnoozePresentation,
getFindingEvidencePresentation,
hasFindingInvestigationDetails,
hasFindingInvestigationHandoffPointer,
@@ -192,6 +195,10 @@ function getFindingSuppressionRuleScope(finding: UnifiedFinding) {
export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
const location = useLocation();
const [filter, setFilter] = createSignal<FindingsPanelFilter>(props.filterOverride ?? 'active');
// A reminder or snooze end never changes while its row stays on screen, so
// the countdown to it, and whether a reminder is overdue, read the shared
// clock.
const now = useRelativeTimeNow();
const [sortBy, setSortBy] = createSignal<'severity' | 'time'>('severity');
const [expandedId, setExpandedId] = createSignal<string | null>(null);
let panelRoot: HTMLDivElement | undefined;
@@ -336,6 +343,32 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
}
});
// Will_fix_later commitments whose RemindAt deadline has passed. The Go
// store's proactive sweep (SweepWillFixLaterReminders) wakes these on a 1h
// timer; until then the Overdue commitments chip and filter surface them on
// demand. The list is sampled on the shared clock and notifies only when its
// ids change, so a tick that moves no deadline past now does not rebuild the
// filtered rows. Each row's "Reminder overdue" label reads the same list, so
// a row that mounts between ticks agrees with the chip and filter.
const overdueFindingIds = createMemo(
() => {
const nowMs = now();
return sourceFindings()
.filter((f) => {
if (f.dismissedReason !== 'will_fix_later' || !f.remindAt) return false;
const due = Date.parse(f.remindAt);
return Number.isFinite(due) && due <= nowMs;
})
.map((f) => f.id);
},
[],
{
equals: (prev, next) =>
prev.length === next.length && prev.every((id, index) => id === next[index]),
},
);
const overdueFindingIdSet = createMemo(() => new Set(overdueFindingIds()));
// Filter and sort findings
const filteredFindings = createMemo(() => {
if (hasUnknownRunSnapshot()) {
@@ -363,22 +396,8 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
const approvalFindingIds = new Set(sourceFindingsWithPendingApprovals().map((f) => f.id));
findings = findings.filter((f) => approvalFindingIds.has(f.id));
} else if (filter() === 'overdue') {
// Overdue surfaces will_fix_later commitments whose RemindAt
// deadline has already passed. The Go store's proactive sweep
// (SweepWillFixLaterReminders) wakes these on a 1h timer, but
// this filter lets the operator see them on demand — including
// any that the sweep has not yet promoted on this cycle.
const nowMs = Date.now();
findings = findings.filter((f) => {
if (f.dismissedReason !== 'will_fix_later') {
return false;
}
if (!f.remindAt) {
return false;
}
const due = Date.parse(f.remindAt);
return Number.isFinite(due) && due <= nowMs;
});
const overdueIds = overdueFindingIdSet();
findings = findings.filter((f) => overdueIds.has(f.id));
}
// Filter by specific finding IDs if provided
@@ -466,17 +485,8 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
? scopedPendingApprovalCount()
: sourceFindingsWithPendingApprovals().length,
}));
// Count of will_fix_later commitments whose RemindAt deadline has
// already passed. Drives the Overdue commitments chip below.
const overdueCount = createMemo(() => {
const nowMs = Date.now();
return sourceFindings().filter((f) => {
if (f.dismissedReason !== 'will_fix_later') return false;
if (!f.remindAt) return false;
const due = Date.parse(f.remindAt);
return Number.isFinite(due) && due <= nowMs;
}).length;
});
// Drives the Overdue commitments chip below.
const overdueCount = createMemo(() => overdueFindingIds().length);
// Active findings that restate an active alert (backend-stamped mirror).
// They are demoted into a collapsed group below the list rather than shown
// as separate items: the alert already owns that condition on the Patrol
@@ -1277,18 +1287,27 @@ export const FindingsPanel: Component<FindingsPanelProps> = (props) => {
{formatOperatorStateDismissCauseLabel(getOperatorStateDismissCause(finding))}
</span>
</Show>
<Show when={finding.dismissedReason === 'will_fix_later' && finding.remindAt}>
<span
class="ml-2 text-amber-600 dark:text-amber-400"
title="Pulse will surface this finding again on this date if it is still tripping."
>
{' · '}Reminding {formatTime(finding.remindAt!)}
</span>
<Show
when={getFindingReminderPresentation(
finding,
now(),
overdueFindingIdSet().has(finding.id),
)}
>
{(reminder) => (
<span class="ml-2 text-amber-600 dark:text-amber-400" title={reminder().title}>
{' · '}
{reminder().label}
</span>
)}
</Show>
<Show when={finding.status === 'snoozed' && finding.snoozedUntil}>
<span class="ml-2 text-blue-500 dark:text-blue-400">
{' · '}snoozed until {formatTime(finding.snoozedUntil!)}
</span>
<Show when={getFindingSnoozePresentation(finding, now())}>
{(snooze) => (
<span class="ml-2 text-blue-500 dark:text-blue-400" title={snooze().title}>
{' · '}
{snooze().label}
</span>
)}
</Show>
<Show when={finding.acknowledgedAt && finding.status === 'active'}>
<span class="ml-2 text-muted">
@@ -1,8 +1,9 @@
import { fireEvent, render, screen, waitFor } from '@solidjs/testing-library';
import type { JSX } from 'solid-js';
import { createSignal, type JSX } from 'solid-js';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { FindingsPanel } from '../FindingsPanel';
import { RELATIVE_TIME_TICK_MS } from '@/utils/relativeTimeClock';
const mockState = vi.hoisted(() => {
const loadFindings = vi.fn();
@@ -771,3 +772,163 @@ describe('FindingsPanel investigate action gating', () => {
expect(busyButton).toBeDisabled();
});
});
describe('FindingsPanel future times', () => {
const hour = 60 * 60 * 1000;
beforeEach(() => {
mockState.findings = [...mockState.initialFindings];
mockState.loadPatrolFindings.mockClear();
mockState.findingsLoading = false;
mockState.findingsError = null;
mockState.patrolFindingsLoading = false;
mockState.patrolFindingsError = null;
mockState.patrolPendingApprovals = [];
});
it('counts reminders and snoozes down on the shared clock instead of reading just now', () => {
vi.useFakeTimers({
toFake: ['Date', 'setInterval', 'clearInterval'],
now: Date.parse('2026-03-30T12:00:00Z'),
});
mockState.patrolFindings = [
{
...mockState.initialPatrolFindings[0],
id: 'finding-later',
title: 'Backup job keeps slipping',
status: 'dismissed',
dismissedReason: 'will_fix_later',
remindAt: '2026-04-06T12:00:00Z',
},
{
...mockState.initialPatrolFindings[0],
id: 'finding-snoozed',
title: 'Swap pressure on vm-100',
status: 'snoozed',
snoozedUntil: '2026-03-30T15:00:00Z',
},
];
try {
render(() => <FindingsPanel findingsSource="patrol" filterOverride="resolved" />);
const reminder = screen.getByText(/Reminding in 7d/);
const snooze = screen.getByText(/snoozed, returns in 3h/);
expect(reminder).toHaveAttribute(
'title',
expect.stringMatching(
/^Reminder due .+\. Pulse brings this finding back on its first reminder check after that\.$/,
),
);
expect(snooze).toHaveAttribute('title', expect.stringMatching(/^Snoozed until /));
expect(screen.queryByText(/just now/)).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: /Overdue commitments/ })).not.toBeInTheDocument();
// The findings are not re-read: only the shared clock moves.
vi.advanceTimersByTime(2 * hour);
expect(snooze).toHaveTextContent('snoozed, returns in 1h');
vi.advanceTimersByTime(hour);
expect(snooze).toHaveTextContent('snooze ended');
vi.setSystemTime(Date.parse('2026-04-06T12:30:00Z'));
vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);
expect(reminder).toHaveTextContent('Reminder overdue');
// The Overdue commitments chip reads the same clock as the label, with no
// findings re-read.
expect(screen.getByRole('button', { name: /Overdue commitments \(1\)/ })).toBeInTheDocument();
expect(reminder).toHaveAttribute('title', expect.stringMatching(/^Reminder was due /));
} finally {
vi.useRealTimers();
}
});
it('keeps overdue rows mounted across clock ticks and adds a reminder once it falls due', () => {
vi.useFakeTimers({
toFake: ['Date', 'setInterval', 'clearInterval'],
now: Date.parse('2026-03-30T12:00:00Z'),
});
mockState.patrolFindings = [
{
...mockState.initialPatrolFindings[0],
id: 'finding-overdue',
title: 'Certificate renewal keeps slipping',
status: 'dismissed',
dismissedReason: 'will_fix_later',
remindAt: '2026-03-30T11:00:00Z',
},
{
...mockState.initialPatrolFindings[0],
id: 'finding-due-soon',
title: 'Backup job keeps slipping',
status: 'dismissed',
dismissedReason: 'will_fix_later',
remindAt: '2026-03-30T12:10:00Z',
},
];
try {
render(() => <FindingsPanel findingsSource="patrol" filterOverride="resolved" />);
fireEvent.click(screen.getByRole('button', { name: /Overdue commitments \(1\)/ }));
const review = screen.getByRole('button', {
name: 'Review issue for Certificate renewal keeps slipping',
});
review.focus();
expect(screen.queryByText('Backup job keeps slipping')).not.toBeInTheDocument();
// A tick that moves no deadline past now leaves the rows alone.
vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);
expect(review.isConnected).toBe(true);
expect(document.activeElement).toBe(review);
// Once the second reminder falls due it joins the open Overdue filter
// and the chip count, on the same clock as the row labels.
vi.advanceTimersByTime(10 * 60 * 1000);
expect(screen.getByText('Backup job keeps slipping')).toBeInTheDocument();
expect(screen.getByRole('button', { name: /Overdue commitments \(2\)/ })).toBeInTheDocument();
} finally {
vi.useRealTimers();
}
});
it('calls a reminder overdue only once the Overdue commitments list holds it', () => {
vi.useFakeTimers({
toFake: ['Date', 'setInterval', 'clearInterval'],
now: Date.parse('2026-03-30T12:00:00Z'),
});
mockState.patrolFindings = [
{
...mockState.initialPatrolFindings[0],
id: 'finding-due',
title: 'Certificate renewal keeps slipping',
status: 'dismissed',
dismissedReason: 'will_fix_later',
remindAt: '2026-03-30T12:00:05Z',
},
];
const [filter, setFilter] = createSignal<'active' | 'resolved'>('active');
try {
render(() => <FindingsPanel findingsSource="patrol" filterOverride={filter()} />);
expect(screen.queryByText('Certificate renewal keeps slipping')).not.toBeInTheDocument();
// The deadline passes between ticks of the shared clock, and the row
// first mounts before the next tick: it must not call itself overdue
// while the chip and filter do not count it.
vi.setSystemTime(Date.parse('2026-03-30T12:00:10Z'));
setFilter('resolved');
expect(screen.getByText('Certificate renewal keeps slipping')).toBeInTheDocument();
expect(screen.getByText(/Reminder due/)).toBeInTheDocument();
expect(screen.queryByText(/Reminder overdue/)).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: /Overdue commitments/ })).not.toBeInTheDocument();
// The next tick moves the label, the chip and the filter together.
vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);
expect(screen.getByText(/Reminder overdue/)).toBeInTheDocument();
expect(screen.getByRole('button', { name: /Overdue commitments \(1\)/ })).toBeInTheDocument();
} finally {
vi.useRealTimers();
}
});
});
@@ -22,6 +22,8 @@ import {
getFindingSeveritySortOrder,
getFindingResolutionReason,
getFindingRecencyPresentation,
getFindingReminderPresentation,
getFindingSnoozePresentation,
getFindingLoopStateBadgeClasses,
getFindingLoopStateBadgeTone,
getFindingSeverityBadgeClasses,
@@ -369,11 +371,14 @@ describe('FindingsPanel assistant handoff', () => {
// Once a finding is dismissed as will_fix_later, the row must surface the
// pending reminder so the operator knows the commitment exists; otherwise
// the only place the deadline lives is the lifecycle log. The amber tone
// signals "pending operator action" rather than a generic muted note.
expect(findingsPanelSource).toContain(
"finding.dismissedReason === 'will_fix_later' && finding.remindAt",
// signals "pending operator action" rather than a generic muted note. The
// deadline is in the future, so it counts down on the shared clock rather
// than going through the past-only age formatter, and it reads overdue from
// the same list as the Overdue commitments chip.
expect(findingsPanelSource).toMatch(
/getFindingReminderPresentation\(\s*finding,\s*now\(\),\s*overdueFindingIdSet\(\)\.has\(finding\.id\),?\s*\)/,
);
expect(findingsPanelSource).toContain('Reminding {formatTime(finding.remindAt!)}');
expect(findingsPanelSource).not.toContain('formatTime(finding.remindAt');
expect(findingsPanelSource).toContain('text-amber-600 dark:text-amber-400');
});
@@ -652,6 +657,92 @@ describe('aiFindingPresentation', () => {
});
});
describe('findingReminderAndSnoozePresentation', () => {
const now = Date.parse('2026-03-25T12:00:00Z');
it('counts down to a will_fix_later reminder instead of reading it as just now', () => {
const reminder = getFindingReminderPresentation(
{ dismissedReason: 'will_fix_later', remindAt: '2026-03-31T12:00:00Z' },
now,
false,
);
expect(reminder?.label).toBe('Reminding in 6d');
expect(reminder?.title).toMatch(
/^Reminder due .+\. Pulse brings this finding back on its first reminder check after that\.$/,
);
expect(reminder?.title).not.toContain('still tripping');
expect(
getFindingReminderPresentation(
{ dismissedReason: 'will_fix_later', remindAt: '2026-03-25T15:00:00Z' },
now,
false,
)?.label,
).toBe('Reminding in 3h');
});
it('calls a reminder overdue once the Overdue commitments list holds it', () => {
const reminder = getFindingReminderPresentation(
{ dismissedReason: 'will_fix_later', remindAt: '2026-03-25T11:00:00Z' },
now,
true,
);
expect(reminder?.label).toBe('Reminder overdue');
expect(reminder?.title).toMatch(
/^Reminder was due .+\. Pulse brings this finding back on its next reminder check\.$/,
);
// Past the deadline but not yet on the list (it is sampled on the next
// clock tick), so the chip and filter would not count it yet.
expect(
getFindingReminderPresentation(
{ dismissedReason: 'will_fix_later', remindAt: '2026-03-25T11:59:50Z' },
now,
false,
)?.label,
).toBe('Reminder due');
});
it('shows no reminder for other dismissals or an unusable remind time', () => {
expect(
getFindingReminderPresentation(
{ dismissedReason: 'not_an_issue', remindAt: '2026-03-31T12:00:00Z' },
now,
false,
),
).toBeNull();
expect(
getFindingReminderPresentation({ dismissedReason: 'will_fix_later' }, now, false),
).toBeNull();
expect(
getFindingReminderPresentation(
{ dismissedReason: 'will_fix_later', remindAt: 'not a date' },
now,
true,
),
).toBeNull();
});
it('counts down to the end of a snooze and keeps the exact time on hover', () => {
const snooze = getFindingSnoozePresentation(
{ status: 'snoozed', snoozedUntil: '2026-03-25T15:00:00Z' },
now,
);
expect(snooze?.label).toBe('snoozed, returns in 3h');
expect(snooze?.title).toMatch(/^Snoozed until .+/);
expect(
getFindingSnoozePresentation(
{ status: 'snoozed', snoozedUntil: '2026-03-25T11:00:00Z' },
now,
)?.label,
).toBe('snooze ended');
expect(
getFindingSnoozePresentation(
{ status: 'active', snoozedUntil: '2026-03-25T15:00:00Z' },
now,
),
).toBeNull();
});
});
describe('patrolFindingClassification', () => {
it('describes grouped Patrol findings as issues instead of internal signals', () => {
expect(getPatrolFindingIssueCountLabel(0)).toBe('0 issues');
@@ -997,7 +1088,8 @@ describe('aiFindingPresentation', () => {
it('uses explicit textual separators in finding metadata instead of relying on visual spacing', () => {
expect(findingsPanelSource).toContain("{' · '}acknowledged");
expect(findingsPanelSource).toContain("{' · '}last investigated");
expect(findingsPanelSource).toContain("{' · '}snoozed until");
expect(findingsPanelSource).toMatch(/\{' · '\}\s*\{snooze\(\)\.label\}/);
expect(findingsPanelSource).toMatch(/\{' · '\}\s*\{reminder\(\)\.label\}/);
});
it('keeps the Open work heading free of a duplicate count badge', () => {
@@ -43,12 +43,11 @@ import {
ALERT_RESOURCE_METRIC_OFF_VALUE,
getAlertResourceColumnKind,
getAlertResourceColumnHeaderTooltip,
getAlertResourceEnabledDefault,
getAlertResourceMetricBounds,
getAlertResourceMetricDelayOverride,
getAlertResourceMetricStep,
isAlertResourceMetricOff,
normalizeAlertResourceMetricKey,
resolveAlertResourceGlobalDefaultCell,
} from './alertResourceTableModel';
import type { OfflineState, Resource, ResourceTableProps } from './ResourceTable';
@@ -210,7 +209,10 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
{(column) => (
<TableHead
class={`${getPlatformTableHeadClassForKind(getAlertResourceColumnKind(column))} whitespace-normal wrap-break-word`}
title={getAlertResourceColumnHeaderTooltip(column)}
title={
props.table.columnTooltips?.[column] ??
getAlertResourceColumnHeaderTooltip(column)
}
>
{column}
</TableHead>
@@ -281,10 +283,14 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
{(column) => {
const metric = normalizeAlertResourceMetricKey(column);
const bounds = getAlertResourceMetricBounds(metric);
const value = () => props.table.globalDefaults?.[metric] ?? 0;
const value = () => props.table.globalDefaults?.[metric];
const fallback = () => props.table.globalDefaultFallbacks?.[metric];
// An unset default and a stored 0 are both disabled to the
// alert engine, so neither may render as On.
const isOff = () => isAlertResourceMetricOff(value());
// alert engine, so neither may render as On, unless the
// metric's unset default follows another setting.
const cell = () =>
resolveAlertResourceGlobalDefaultCell(metric, value(), fallback());
const isOff = () => cell().isOff;
return (
<TableCell
@@ -297,8 +303,12 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
min={bounds.min}
max={bounds.max}
step={getAlertResourceMetricStep(metric)}
value={isOff() ? '' : value()}
placeholder={getAlertResourceTableMetricPlaceholder(isOff())}
value={isOff() || cell().follows ? '' : (value() ?? '')}
placeholder={
cell().follows && !isOff()
? fallback()?.label
: getAlertResourceTableMetricPlaceholder(isOff())
}
disabled={isOff()}
onInput={(e) => {
const nextValue = parseFloat(e.currentTarget.value);
@@ -318,12 +328,16 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
}));
props.table.setHasUnsavedChanges?.(true);
}}
class={`w-16 px-2 py-0.5 text-sm text-center border rounded ${
class={`${cell().follows && !isOff() ? 'w-20' : 'w-16'} px-2 py-0.5 text-sm text-center border rounded ${
isOff()
? 'border-border bg-surface-alt text-muted italic placeholder: dark:placeholder: placeholder:opacity-60 pointer-events-none'
: 'border-border text-base-content focus:border-blue-500 focus:ring-1 focus:ring-blue-500'
}`}
title={getAlertResourceTableMetricInputTitle(isOff())}
title={
cell().follows
? fallback()?.title
: getAlertResourceTableMetricInputTitle(isOff())
}
/>
<Show when={isOff()}>
<button
@@ -332,7 +346,7 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
onClick={() => {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: getAlertResourceEnabledDefault(metric),
[metric]: cell().enableValue,
}));
props.table.setHasUnsavedChanges?.(true);
}}
@@ -348,7 +362,7 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: isOff()
? getAlertResourceEnabledDefault(metric)
? cell().enableValue
: ALERT_RESOURCE_METRIC_OFF_VALUE,
}));
props.table.setHasUnsavedChanges?.(true);
@@ -26,7 +26,6 @@ import {
ALERT_RESOURCE_METRIC_OFF_VALUE,
alertResourceSupportsMetric,
buildAlertResourceEditPayload,
getAlertResourceEnabledDefault,
getAlertResourceLabel,
getAlertResourceMetricBounds,
getAlertResourceMetricDisplayValue,
@@ -34,6 +33,7 @@ import {
isAlertResourceMetricOff,
isAlertResourceMetricOverridden,
normalizeAlertResourceMetricKey,
resolveAlertResourceGlobalDefaultCell,
resolveAlertResourceMetricEnableValue,
} from './alertResourceTableModel';
import type { Resource } from '@/features/alerts/thresholds/tableTypes';
@@ -212,10 +212,14 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
{(column) => {
const metric = normalizeAlertResourceMetricKey(column);
const bounds = getAlertResourceMetricBounds(metric);
const value = () => props.table.globalDefaults?.[metric] ?? 0;
const value = () => props.table.globalDefaults?.[metric];
const fallback = () => props.table.globalDefaultFallbacks?.[metric];
// An unset default and a stored 0 are both disabled to the alert
// engine, so neither may render as On.
const isOff = () => isAlertResourceMetricOff(value());
// engine, so neither may render as On, unless the metric's unset
// default follows another setting.
const cell = () =>
resolveAlertResourceGlobalDefaultCell(metric, value(), fallback());
const isOff = () => cell().isOff;
return (
<div class="p-2 bg-surface rounded-sm border border-border-subtle flex flex-col gap-1">
@@ -227,8 +231,13 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
min={bounds.min}
max={bounds.max}
step={getAlertResourceMetricStep(metric)}
value={isOff() ? '' : value()}
placeholder={getAlertResourceTableMetricPlaceholder(isOff())}
value={isOff() || cell().follows ? '' : (value() ?? '')}
placeholder={
cell().follows && !isOff()
? fallback()?.label
: getAlertResourceTableMetricPlaceholder(isOff())
}
title={cell().follows ? fallback()?.title : undefined}
disabled={isOff()}
class={`min-h-11 w-full rounded-sm border p-1 text-center text-sm sm:min-h-0 ${isOff() ? 'bg-surface-hover' : ' border-border'}`}
onInput={(e) => {
@@ -257,7 +266,7 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
onClick={() => {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: getAlertResourceEnabledDefault(metric),
[metric]: cell().enableValue,
}));
props.table.setHasUnsavedChanges?.(true);
}}
@@ -271,7 +280,7 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: isOff()
? getAlertResourceEnabledDefault(metric)
? cell().enableValue
: ALERT_RESOURCE_METRIC_OFF_VALUE,
}));
props.table.setHasUnsavedChanges?.(true);
@@ -1,4 +1,5 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { createSignal } from 'solid-js';
import { cleanup, fireEvent, render, screen } from '@solidjs/testing-library';
import { ALERT_BULK_EDIT_CLEAR_LABEL } from '@/utils/alertBulkEditPresentation';
import resourceTableSource from '@/components/Alerts/ResourceTable.tsx?raw';
@@ -1007,6 +1008,144 @@ describe('ResourceTable', () => {
expect(updater({ cpu: 0 })).toEqual({ cpu: 80 });
});
it.each([
['desktop', false],
['mobile', true],
])('lets an unset %s global default follow its fallback', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: undefined },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
// Unset reads On and names where the value comes from.
expect(screen.getByTestId('status-badge')).toHaveTextContent('On');
const input = screen.getByPlaceholderText('By type') as HTMLInputElement;
expect(input.value).toBe('');
expect(input.disabled).toBe(false);
expect(input.title).toBe('Follows Disk temperature by type');
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: undefined })).toEqual({ temperature: -1 });
});
it.each([
['desktop', false],
['mobile', true],
])('switches an Off %s fallback default back to following', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: -1 },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByTestId('status-badge')).toHaveTextContent('Off');
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: -1 })).toEqual({ temperature: undefined });
});
it.each([
['desktop', false],
['mobile', true],
])('reads an unset %s default Off when its fallback is off', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: undefined },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Off by type', off: true, enableValue: 55 },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByTestId('status-badge')).toHaveTextContent('Off');
expect(screen.queryByPlaceholderText('By type')).toBeNull();
// Unset would still be off, so switching on stages an explicit value.
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: undefined })).toEqual({ temperature: 55 });
});
it('shows a set fallback global default as its number', () => {
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: 62 },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults: vi.fn(),
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByDisplayValue('62')).toBeInTheDocument();
expect(screen.queryByPlaceholderText('By type')).toBeNull();
});
it('follows global defaults that arrive after the table mounts', () => {
const [globalDefaults, setGlobalDefaults] = createSignal<Record<string, number | undefined>>({
temperature: undefined,
});
const props = makeProps({
resources: [],
columns: ['Temp °C'],
factoryDefaults: { temperature: undefined },
onResetDefaults: vi.fn(),
setGlobalDefaults: vi.fn(),
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} globalDefaults={globalDefaults()} />);
expect(screen.queryByText('Custom')).toBeNull();
// The saved config loads after mount.
setGlobalDefaults({ temperature: 62 });
expect(screen.getByText('Custom')).toBeInTheDocument();
});
it('uses a table column tooltip over the generic one', () => {
const props = makeProps({
resources: [],
columns: ['Temp °C'],
columnTooltips: { 'Temp °C': 'Disk temperature that raises an alert.' },
});
render(() => <ResourceTable {...props} />);
expect(screen.getByText('Temp °C', { selector: 'th' })).toHaveAttribute(
'title',
'Disk temperature that raises an alert.',
);
});
it('stages an explicit threshold from the mobile row editor', () => {
mockIsMobile.mockReturnValue(true);
const setEditingThresholds = vi.fn();
@@ -11,6 +11,7 @@ import { ActionIconButton } from '@/components/shared/Button';
import { useObservedElementWidth } from '@/hooks/useObservedElementWidth';
import { useAlertResourceTableState } from './useAlertResourceTableState';
import type { GroupHeaderMeta, Resource } from '@/features/alerts/thresholds/tableTypes';
import type { AlertResourceGlobalDefaultFallback } from './alertResourceTableModel';
import { AlertResourceTableDesktop } from './AlertResourceTableDesktop';
import { AlertResourceTableMobile } from './AlertResourceTableMobile';
@@ -49,6 +50,10 @@ export interface ResourceTableProps {
formatMetricValue: (metric: string, value: number | undefined) => string;
hasActiveAlert: (resourceId: string, metric: string) => boolean;
globalDefaults?: Record<string, number | undefined>;
/** Metrics whose unset global default follows another setting. */
globalDefaultFallbacks?: Record<string, AlertResourceGlobalDefaultFallback>;
/** Header tooltips that replace the generic one for a column label. */
columnTooltips?: Record<string, string>;
setGlobalDefaults?: (
value:
| Record<string, number | undefined>
@@ -97,10 +102,20 @@ export function ResourceTable(props: ResourceTableProps) {
someSelected,
clearSelectedIds,
} = useAlertResourceTableState({
resources: props.resources,
groupedResources: props.groupedResources,
globalDefaults: props.globalDefaults,
factoryDefaults: props.factoryDefaults,
// Getters keep the state reactive: saved defaults arrive after mount, and
// the Custom badge and reset control must follow them.
get resources() {
return props.resources;
},
get groupedResources() {
return props.groupedResources;
},
get globalDefaults() {
return props.globalDefaults;
},
get factoryDefaults() {
return props.factoryDefaults;
},
});
const totalColumnCount = createMemo(
() =>
@@ -82,9 +82,10 @@ export function ThresholdsTableAgentsResourcesSection(props: ThresholdsTableSect
<Card padding="md" tone="card" class="mt-4">
<h3 class="text-sm font-semibold text-base-content">Disk temperature by type</h3>
<p class="mt-1 text-xs text-muted">
Alert trigger in °C for each disk type. Warning colors start 5°C below the trigger.
Setting a Disk Temp override on a host above replaces these for all of that host's
disks.
Alert trigger in °C for each disk type, for agent, TrueNAS and Proxmox disks. Warning
colors start 5°C below the trigger. A Disk Temp override on a host above replaces
these for all of that host's disks, and a TrueNAS Disks value replaces them for
TrueNAS disks.
</p>
<div class="mt-4 grid gap-4 sm:grid-cols-3">
<For each={DISK_TEMP_TYPE_FIELDS}>
@@ -8,6 +8,9 @@ import { CollapsibleSection } from './Thresholds/sections/CollapsibleSection';
import { formatMetricValue } from '@/features/alerts/thresholds/helpers';
import type { ThresholdsTableSectionProps } from '@/features/alerts/thresholds/thresholdsTableSectionProps';
import type { Resource } from '@/features/alerts/thresholds/tableTypes';
import { getDiskTemperatureByTypeItems } from '@/features/alerts/thresholds/trueNASDiskTemperature';
import { FACTORY_AGENT_DEFAULTS } from '@/utils/alertThresholdDefaults';
import type { AlertResourceGlobalDefaultFallback } from './alertResourceTableModel';
import { getAlertThresholdsDefaultsSummary } from '@/utils/alertThresholdsSectionPresentation';
const TRUENAS_SYSTEM_COLUMNS = [
@@ -22,6 +25,7 @@ const TRUENAS_SYSTEM_COLUMNS = [
];
const TRUENAS_STORAGE_COLUMNS = ['Usage %'];
const TRUENAS_DISK_COLUMNS = ['Temp °C'];
const TRUENAS_DISK_COLUMN_TOOLTIPS = { 'Temp °C': 'Disk temperature that raises an alert.' };
function TrueNASResourceSection(
props: ThresholdsTableSectionProps & {
@@ -39,6 +43,10 @@ function TrueNASResourceSection(
| ((prev: Record<string, number | undefined>) => Record<string, number | undefined>),
) => void;
onResetDefaults?: () => void;
defaultsSummary?: string;
globalDefaultFallbacks?: Record<string, AlertResourceGlobalDefaultFallback>;
columnTooltips?: Record<string, string>;
note?: string;
},
) {
const { state, tableProps } = props;
@@ -48,7 +56,9 @@ function TrueNASResourceSection(
<CollapsibleSection
id={props.id}
title={props.title}
defaultsSummary={getAlertThresholdsDefaultsSummary(props.columns, props.defaults)}
defaultsSummary={
props.defaultsSummary ?? getAlertThresholdsDefaultsSummary(props.columns, props.defaults)
}
resourceCount={props.resources().length}
collapsed={state.isCollapsed(props.id)}
onToggle={() => state.toggleSection(props.id)}
@@ -57,6 +67,9 @@ function TrueNASResourceSection(
emptyMessage="No TrueNAS alert targets match the current filters."
>
<div ref={state.registerSection(props.id)} class="scroll-mt-24">
<Show when={props.note}>
<p class="mb-3 text-xs text-muted">{props.note}</p>
</Show>
<ResourceTable
title=""
onConfigureResourceIntent={tableProps.onConfigureResourceIntent}
@@ -79,6 +92,8 @@ function TrueNASResourceSection(
formatMetricValue={formatMetricValue}
hasActiveAlert={state.hasActiveAlert}
globalDefaults={props.defaults}
globalDefaultFallbacks={props.globalDefaultFallbacks}
columnTooltips={props.columnTooltips}
setGlobalDefaults={props.setGlobalDefaults}
setHasUnsavedChanges={tableProps.setHasUnsavedChanges}
globalDisableFlag={tableProps.disableAllTrueNAS}
@@ -104,6 +119,38 @@ export function ThresholdsTableTrueNASTab(props: ThresholdsTableSectionProps) {
const trueNASDefaults = () => props.tableProps.trueNASDefaults ?? {};
const trueNASStorageDefaults = () => ({ usage: trueNASDefaults().usage ?? 85 });
const trueNASDiskDefaults = () => props.tableProps.trueNASDiskDefaults ?? {};
const diskTemperatureByType = () =>
getDiskTemperatureByTypeItems({
agentDiskTemperature: props.tableProps.agentDefaults.diskTemperature,
diskTempByType: props.tableProps.diskTempByType,
});
// Unset, TrueNAS disks follow Disk temperature by type, so the header names
// the per-type triggers rather than one number.
const trueNASDiskDefaultsSummary = () => {
if (trueNASDiskDefaults().temperature !== undefined) return undefined;
return getAlertThresholdsDefaultsSummary(
TRUENAS_DISK_COLUMNS,
diskTemperatureByType().length > 0 ? {} : { temperature: 0 },
{ ruleItems: diskTemperatureByType() },
);
};
// When the agent Disk Temp default is off, Disk temperature by type is off
// too, so the unset cell reads Off and switching it on stages a TrueNAS-wide
// value at the factory disk temperature trigger.
const trueNASDiskTemperatureFallback = (): Record<string, AlertResourceGlobalDefaultFallback> => {
const items = diskTemperatureByType();
return {
temperature: {
label: 'By type',
title:
items.length > 0
? `Follows Disk temperature by type under Agents: ${items.join(', ')}. Enter a value to use one temperature for every TrueNAS disk.`
: 'Off because the agent Disk Temp default under Agents is off. Turn it on here to alert on TrueNAS disks anyway.',
off: items.length === 0,
enableValue: FACTORY_AGENT_DEFAULTS.diskTemperature,
},
};
};
return (
<>
@@ -175,6 +222,10 @@ export function ThresholdsTableTrueNASTab(props: ThresholdsTableSectionProps) {
icon={<HardDrive class="w-5 h-5" />}
typeKey="truenas-disk"
defaults={trueNASDiskDefaults()}
defaultsSummary={trueNASDiskDefaultsSummary()}
globalDefaultFallbacks={trueNASDiskTemperatureFallback()}
columnTooltips={TRUENAS_DISK_COLUMN_TOOLTIPS}
note="Unless set here, each disk alerts at Disk temperature by type, set under Agents."
factoryDefaults={props.tableProps.factoryTrueNASDiskDefaults}
setGlobalDefaults={props.tableProps.setTrueNASDiskDefaults}
onResetDefaults={props.tableProps.resetTrueNASDiskDefaults}
@@ -457,6 +457,25 @@ describe('ThresholdsTable navigation and redirection', () => {
});
});
it('says Disk temperature by type covers agent, TrueNAS and Proxmox disks', async () => {
setPathname('/alerts/thresholds/systems');
const host: Agent = {
id: 'legacy-h1',
hostname: 'legacy-host',
displayName: 'Legacy Host',
status: 'online',
lastSeen: 123,
memory: { total: 100, used: 50, free: 50, usage: 50 },
};
render(() => <ThresholdsTable {...(baseProps() as any)} agents={[host]} />);
const heading = await screen.findByText('Disk temperature by type');
expect(heading.parentElement?.textContent).toContain(
'for each disk type, for agent, TrueNAS and Proxmox disks',
);
});
it('navigates to correct route when tabs are clicked', () => {
render(() => <ThresholdsTable {...(baseProps() as any)} />);
@@ -244,6 +244,49 @@ export function resolveAlertResourceMetricEnableValue(
return undefined;
}
/**
* A global default that may stay unset because another setting decides it,
* such as TrueNAS disk temperature following Disk temperature by type. Unset,
* the cell is empty with `label` as its placeholder and `title` explaining
* where the value comes from. `off` says the other setting is itself off, so
* the unset cell reads Off and switching it on has to stage `enableValue`.
*/
export interface AlertResourceGlobalDefaultFallback {
label: string;
title: string;
off?: boolean;
enableValue?: number;
}
export interface AlertResourceGlobalDefaultCell {
/** Unset and decided by the fallback. */
follows: boolean;
isOff: boolean;
/** Value staged when the cell is switched back on from Off. */
enableValue: number | undefined;
}
/**
* State of one Global Defaults metric cell. An unset default is Off, as the
* alert engine reads it, unless the metric has a fallback; then it reads as
* the fallback does. Switching it on from Off returns it to unset when the
* fallback is on, and stages an explicit value when the fallback is off,
* since unset would still be off.
*/
export function resolveAlertResourceGlobalDefaultCell(
metric: string,
value: number | undefined,
fallback: AlertResourceGlobalDefaultFallback | undefined,
): AlertResourceGlobalDefaultCell {
const follows = value === undefined && fallback !== undefined;
const explicitEnableValue = fallback?.enableValue ?? getAlertResourceEnabledDefault(metric);
return {
follows,
isOff: follows ? Boolean(fallback?.off) : isAlertResourceMetricOff(value ?? 0),
enableValue: fallback && !fallback.off ? undefined : explicitEnableValue,
};
}
export function getAlertResourceMetricDelayOverride(
metricDelaySeconds: Record<string, number> | undefined,
metric: string,
@@ -830,16 +830,39 @@ describe('buildTrueNASVMSections additional branches', () => {
// =========================================================================
describe('buildTrueNASDiskSections additional branches', () => {
it('sets Temperature tone to warning at 55 degrees', () => {
expect(findRow(diskRes({ temperature: 55 }), 'Temperature')?.tone).toBe('warning');
// The drawer judges heat by the disk type's alert trigger, through the
// resolver the alerts store provides; these are the factory triggers.
const factoryTriggers: Record<string, number> = { nvme: 70, sas: 65, sata: 55 };
const resolveThresholds = (diskType: string) => {
const critical = factoryTriggers[diskType.toLowerCase()] ?? 55;
return { warning: critical - 5, critical };
};
const temperatureTone = (disk: Partial<ResourcePhysicalDiskMeta>) =>
buildTrueNASDetailSections(diskRes(disk), resolveThresholds)
.flatMap((section) => section.rows)
.find((row) => row.label === 'Temperature')?.tone;
it('sets Temperature tone to warning from the disk type trigger', () => {
expect(temperatureTone({ diskType: 'sata', temperature: 55 })).toBe('warning');
expect(temperatureTone({ diskType: 'sas', temperature: 65 })).toBe('warning');
expect(temperatureTone({ diskType: 'nvme', temperature: 70 })).toBe('warning');
});
it('sets Temperature tone to warning above 55 degrees', () => {
expect(findRow(diskRes({ temperature: 60 }), 'Temperature')?.tone).toBe('warning');
it('keeps a reading under its type trigger at default tone', () => {
expect(temperatureTone({ diskType: 'nvme', temperature: 60 })).toBe('default');
expect(temperatureTone({ diskType: 'sas', temperature: 63 })).toBe('default');
expect(temperatureTone({ diskType: 'sata', temperature: 40 })).toBe('default');
});
it('sets Temperature tone to default below 55 degrees', () => {
expect(findRow(diskRes({ temperature: 40 }), 'Temperature')?.tone).toBe('default');
it('judges no heat without thresholds', () => {
expect(findRow(diskRes({ diskType: 'sata', temperature: 80 }), 'Temperature')?.tone).toBe(
'default',
);
expect(
buildTrueNASDetailSections(diskRes({ diskType: 'sata', temperature: 80 }), () => null)
.flatMap((section) => section.rows)
.find((row) => row.label === 'Temperature')?.tone,
).toBe('default');
});
it('formats Wearout percentage for a valid value', () => {
@@ -381,8 +381,9 @@ describe('resourceDetailDrawerTrueNASModel', () => {
},
},
});
const sataTrigger = () => ({ warning: 50, critical: 55 });
const temperatureRow = (resource: Resource) =>
buildTrueNASDetailSections(resource)
buildTrueNASDetailSections(resource, sataTrigger)
.find((section) => section.label === 'Health')
?.rows.find((row) => row.label === 'Temperature');
@@ -20,7 +20,11 @@ import {
type DetailSection,
type DetailValueTone,
} from '@/components/shared/detailSectionModel';
import { getPhysicalDiskTemperaturePresentation } from '@/features/storageBackups/diskTemperaturePresentation';
import {
getPhysicalDiskTemperaturePresentation,
isPhysicalDiskRunningHot,
} from '@/features/storageBackups/diskTemperaturePresentation';
import type { TrueNASDiskTemperatureThresholdResolver } from '@/features/truenas/truenasPageModel';
import { hasImpairedResourceSource } from '@/utils/resourceSourceHealth';
export type ResourceDetailDrawerTrueNASRowTone = DetailValueTone;
@@ -399,9 +403,13 @@ const diskTemperature = (disk: ResourcePhysicalDiskMeta) =>
});
// A retained reading, such as a silent host agent's on a merged disk row, is
// not current: it takes no heat tone and its title says why.
// not current: it takes no heat tone and its title says why. A current reading
// is hot from its disk type's alert trigger, the threshold the TrueNAS storage
// table and the disk's temperature alert judge it by. Without a resolver, heat
// is not judged.
const diskTemperatureRow = (
disk: ResourcePhysicalDiskMeta,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASRow | null => {
const reading = diskTemperature(disk);
if (!reading) return null;
@@ -411,9 +419,12 @@ const diskTemperatureRow = (
tone: 'muted',
});
}
return row('Temperature', reading.label, {
tone: (disk.temperature ?? 0) >= 55 ? 'warning' : 'default',
});
const thresholds = resolveDiskTemperatureThresholds?.(disk.diskType ?? '') ?? null;
const hot = isPhysicalDiskRunningHot(
{ temperature: disk.temperature ?? 0, collection: disk.collection },
thresholds,
);
return row('Temperature', reading.label, { tone: hot ? 'warning' : 'default' });
};
const currentDiskTemperatureLabel = (disk: ResourcePhysicalDiskMeta): string | null => {
@@ -424,6 +435,7 @@ const currentDiskTemperatureLabel = (disk: ResourcePhysicalDiskMeta): string | n
const buildTrueNASDiskSections = (
resource: Resource,
disk: ResourcePhysicalDiskMeta,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASSection[] => {
const riskReasons = (disk.risk?.reasons ?? [])
.map((reason) => asString(reason.summary))
@@ -439,7 +451,7 @@ const buildTrueNASDiskSections = (
const healthRows = compactRows([
row('Health', normalizeDelimitedLabel(disk.health), { tone: diskStateTone(disk) }),
diskTemperatureRow(disk),
diskTemperatureRow(disk, resolveDiskTemperatureThresholds),
row(
'Wearout',
disk.wearout === undefined || disk.wearout < 0 ? null : formatPercent(disk.wearout),
@@ -746,6 +758,7 @@ const buildTrueNASShareSections = (
export const buildTrueNASDetailSections = (
resource: Resource,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASSection[] => {
if (resource.truenas?.share) return buildTrueNASShareSections(resource.truenas.share);
if (resource.truenas?.vm) return buildTrueNASVMSections(resource, resource.truenas.vm);
@@ -754,7 +767,11 @@ export const buildTrueNASDetailSections = (
return buildTrueNASStorageSections(resource, resource.storage);
}
if (isTrueNASScopedResource(resource) && resource.physicalDisk) {
return buildTrueNASDiskSections(resource, resource.physicalDisk);
return buildTrueNASDiskSections(
resource,
resource.physicalDisk,
resolveDiskTemperatureThresholds,
);
}
if (isTrueNASScopedResource(resource) && resource.truenas) {
return buildTrueNASSystemSections(resource, resource.truenas);
@@ -23,6 +23,7 @@ import {
} from '@/utils/resourceIdentity';
import { isPulseAgentPlatformResource } from '@/utils/agentResources';
import { areSystemSettingsLoaded, shouldHideDockerUpdateActions } from '@/stores/systemSettings';
import { useAlertsActivation } from '@/stores/alertsActivation';
import {
getResourcePolicyBadges,
getResourcePolicyGovernedSummary,
@@ -381,7 +382,10 @@ export const useResourceDetailDrawerDerivedState = (
const vmwareDetailsSummary = createMemo(() =>
buildVMwareDetailsSummary(resource().type, vmwareData()),
);
const trueNASDetailSections = createMemo(() => buildTrueNASDetailSections(resource()));
const { getDiskTemperatureThresholds } = useAlertsActivation();
const trueNASDetailSections = createMemo(() =>
buildTrueNASDetailSections(resource(), getDiskTemperatureThresholds),
);
const hasTrueNASDetails = createMemo(() => trueNASDetailSections().length > 0);
const trueNASDetailsSummary = createMemo(() => buildTrueNASDetailsSummary(resource()));
const kubernetesDetailSections = createMemo(() => buildKubernetesDetailSections(resource()));
@@ -440,6 +440,54 @@ describe('GuestDrawer', () => {
}
});
it("judges a linked agent's disk metric in History by that agent, not the Proxmox reason", async () => {
chartsApiMocks.getMetricsHistory.mockResolvedValue({
resourceType: 'vm',
resourceId: 'inst1:node1:100',
range: '24h',
start: 1,
end: 3,
metrics: {},
source: 'store',
});
// Proxmox has no guest filesystems, so the agent's filesystems and disk
// metric fill in while Proxmox still reports its own agent-not-running.
const [guest, setGuest] = createSignal(
makeGuest({
disk: { total: 100, used: 40, free: 60, usage: 40 },
disks: [{ total: 100, used: 40, free: 60, usage: 40, mountpoint: '/' }],
disksFromAgent: true,
diskStatusReason: 'agent-not-running',
}),
);
render(() => <GuestDrawer guest={guest()} onClose={vi.fn()} />);
fireEvent.click(screen.getByRole('tab', { name: 'History' }));
await waitFor(() =>
expect(screen.getAllByText('No stored history in this range')).toHaveLength(3),
);
const history = screen.getAllByTestId('guest-history-group-chart')[0];
expect(history.querySelector('[data-history-current="disk"]')).toHaveTextContent(
'Disk40.0%current',
);
expect(history.querySelector('[data-history-deferred="disk"]')).toBeNull();
setGuest({ ...guest(), agentStale: true });
expect(history.querySelector('[data-history-current="disk"]')).toBeNull();
expect(history.querySelector('[data-history-last-known="disk"]')).toHaveTextContent(
'Disk40.0%last known',
);
expect(history.querySelector('[data-history-deferred="disk"]')).toHaveTextContent(
'The Pulse Agent in this guest stopped reporting.',
);
setGuest({ ...guest(), status: 'stopped' });
expect(history.querySelector('[data-history-last-known="disk"]')).toBeNull();
expect(history.querySelector('[data-history-current="disk"]')).toBeNull();
expect(history.querySelector('[data-history-deferred="disk"]')).toHaveTextContent(
'Guest filesystem stats unavailable while the VM is stopped.',
);
});
it('uses the shared discovery loading fallback instead of a drawer-local spinner row', () => {
expect(guestDrawerSource).toContain('DiscoveryLoadingFallback');
expect(guestDrawerSource).not.toContain(
@@ -15,7 +15,7 @@ import type { DiscoveryIdentifiedSummary } from '@/utils/discoveryPresentation';
import { formatBytes } from '@/utils/format';
import type { MetricDisplayThresholds } from '@/utils/metricThresholds';
import {
getWorkloadGuestDiskStatusMessage,
getWorkloadGuestDiskRead,
getWorkloadsGuestProtectionPresentation,
} from '@/utils/workloadGuestPresentation';
@@ -152,44 +152,19 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {
props.guest.vmware?.datacenterName ||
props.guest.vmware?.clusterName,
);
// Rows the linked Pulse agent supplied follow that agent's freshness, not
// the Proxmox reason for its own guest reads. A stopped guest has no
// current filesystems from either source, with or without the Proxmox
// vm-stopped reason, which it gives only for a guest with allocated disk.
const vmDiskReason = isGuestDrawerVM(props.guest) ? props.guest.diskStatusReason : undefined;
const agentDisks = props.guest.disksFromAgent === true;
const agentDisksStopped =
agentDisks && (props.guest.status === 'stopped' || vmDiskReason === 'vm-stopped');
const diskReason = agentDisks ? undefined : vmDiskReason;
const diskReadState = agentDisks
? agentDisksStopped
// Rows the linked Pulse agent supplied follow that agent's freshness, the
// same rule the workload table's disk cell and History read.
const diskRead = getWorkloadGuestDiskRead(props.guest, isGuestDrawerVM(props.guest));
const diskReadState =
!props.guest.disksFromAgent && props.guest.telemetryAvailability?.disk === false
? 'unavailable'
: agentLastKnownReason()
? 'last-known'
: 'current'
: props.guest.telemetryAvailability?.disk === false
? 'unavailable'
: diskReason
? diskReason.startsWith('prev-')
? 'last-known'
: 'unavailable'
: 'current';
: diskRead.state;
const diskRows = (props.guest.disks ?? []).map((disk, index) =>
buildWorkloadsDiskPresentation(disk, index, props.diskThresholds, diskReadState),
);
const diskStatusMessage = agentDisks
? agentDisksStopped
? isGuestDrawerVM(props.guest)
? getWorkloadGuestDiskStatusMessage('vm-stopped')
: 'Filesystem usage is unavailable.'
: agentLastKnownReason()
? 'Using last known disk stats. The Pulse Agent in this guest stopped reporting.'
: null
: diskReason
? getWorkloadGuestDiskStatusMessage(diskReason)
: diskReadState === 'unavailable'
? 'Filesystem usage is unavailable.'
: null;
const diskStatusMessage =
diskRead.message ??
(diskReadState === 'unavailable' ? 'Filesystem usage is unavailable.' : null);
return compactDetailSections([
discovery
@@ -54,10 +54,6 @@ export { GUEST_COLUMNS, VIEW_MODE_COLUMNS } from './guestRowModel';
export type { GuestRowProps, WorkloadIOEmphasis } from './guestRowModel';
import { getGuestColumnStyle } from './guestRowModel';
// Disk-read reasons that point at something broken the user can fix, as
// opposed to a guest that simply has no agent or is not running.
const DISK_READ_ACTIONABLE_REASONS = new Set(['permission-denied', 'agent-error']);
export function GuestRow(props: GuestRowProps) {
const [rowActionProps] = splitProps(props, ['onClick']);
const selectionGuard = createRowTextSelectionGuard();
@@ -82,6 +78,7 @@ export function GuestRow(props: GuestRowProps) {
discoveryReadinessPresentation,
diskRead,
diskThresholds,
diskUsageRead,
diskWrite,
displayId,
dockerHostId,
@@ -286,7 +283,9 @@ export function GuestRow(props: GuestRowProps) {
};
const getDiskStatusTooltip = () => {
if (!isVM(props.guest)) return 'Disk stats unavailable';
const message = diskUsageRead().message;
if (message) return message;
if (!isVM(props.guest) || props.guest.disksFromAgent) return 'Disk stats unavailable';
const vm = props.guest as VM;
return getWorkloadGuestDiskStatusMessage(vm.diskStatusReason);
@@ -294,10 +293,9 @@ export function GuestRow(props: GuestRowProps) {
// A native title or bar tooltip is not a visible freshness cue on touch,
// and disappears from the chart's accessible value when the history lens opens.
const diskReadStatus = createMemo(() => {
const guest = props.guest;
if (!isVM(guest) || !guest.diskStatusReason) return undefined;
const reason = guest.diskStatusReason;
const retained = reason.startsWith('prev-');
const read = diskUsageRead();
if (read.state === 'current') return undefined;
const retained = read.state === 'last-known';
return {
label: retained
? usesCompactTableLayout()
@@ -306,11 +304,11 @@ export function GuestRow(props: GuestRowProps) {
: usesCompactTableLayout()
? 'N/A'
: 'Unavailable',
message: getDiskStatusTooltip(),
message: read.message,
valueLabelContext: retained ? ('last known' as const) : ('current' as const),
// Most reasons are a setup gap (no guest agent, VM stopped) rather than a
// fault, so they stay muted; only a broken read the user can fix is amber.
needsAction: DISK_READ_ACTIONABLE_REASONS.has(reason.replace(/^prev-/, '')),
needsAction: read.needsAction,
};
});
const diskValueLabelContext = (): MetricMiniSparklineValueLabelContext =>
@@ -632,11 +630,7 @@ export function GuestRow(props: GuestRowProps) {
aggregateDisk={props.guest.disk}
anomaly={diskAnomaly()}
thresholds={diskThresholds()}
statusMessage={
isVM(props.guest) && props.guest.diskStatusReason
? getDiskStatusTooltip()
: undefined
}
statusMessage={diskUsageRead().message ?? undefined}
/>
}
>
@@ -1076,6 +1076,61 @@ describe('GuestRow', () => {
);
expect(container.querySelector('[data-workload-disk-read-status]')).toBeNull();
});
it("follows a linked agent's filesystems, not the Proxmox read reason", () => {
// Proxmox has no guest filesystems, so the agent's fill in while
// Proxmox still reports its own agent-not-running.
const agentGuest = (overrides: Partial<WorkloadGuest> = {}) =>
makeGuest({
diskStatusReason: 'agent-not-running',
disksFromAgent: true,
disks: [makeDisk({ mountpoint: '/' })],
...overrides,
});
const [guest, setGuest] = createSignal(agentGuest());
const { container } = render(() => (
<table>
<tbody>
<GuestRow guest={guest()} visibleColumnIds={['name', 'disk']} />
</tbody>
</table>
));
expect(screen.getByTestId('disk-bar')).not.toHaveAttribute('data-status-message');
expect(container.querySelector('[data-workload-disk-read-status]')).toBeNull();
const lastKnown =
'Using last known disk stats. The Pulse Agent in this guest stopped reporting.';
setGuest(agentGuest({ agentStale: true }));
expect(screen.getByTestId('disk-bar')).toHaveAttribute('data-status-message', lastKnown);
let notice = container.querySelector('[data-workload-disk-read-status]');
expect(notice?.querySelector('[aria-hidden="true"]')).toHaveTextContent('Last known');
expect(notice).toHaveAttribute('title', lastKnown);
expect(notice?.className).toContain('text-muted');
setGuest(agentGuest({ agentStale: true, status: 'stopped', diskStatusReason: 'vm-stopped' }));
expect(screen.queryByTestId('disk-bar')).toBeNull();
notice = container.querySelector('[data-workload-disk-read-status]');
expect(notice?.querySelector('[aria-hidden="true"]')).toHaveTextContent('Unavailable');
expect(notice).toHaveAttribute(
'title',
'Guest filesystem stats unavailable while the VM is stopped.',
);
});
it("does not announce a silent linked agent's sparkline reading as current", () => {
renderGuestRow({
guest: makeGuest({
diskStatusReason: 'agent-not-running',
disksFromAgent: true,
agentStale: true,
}),
visibleColumnIds: ['name', 'disk'],
metricDisplayMode: 'sparklines',
});
expect(
screen.getByRole('img', { name: 'test-vm disk usage history, last known 50%' }),
).toBeInTheDocument();
});
});
describe('lock label', () => {
@@ -13,6 +13,7 @@ import {
} from '@/features/alerts/metricAlertPresentation';
import { formatBytes, formatPercent, getBackupInfo, type BackupThresholds } from '@/utils/format';
import {
getWorkloadGuestDiskRead,
getWorkloadGuestDiskStatusMessage,
getWorkloadsGuestBackupStatusPresentation,
} from '@/utils/workloadGuestPresentation';
@@ -153,12 +154,13 @@ export const getGuestDrawerDeferredMetrics = (
guest: Guest,
): Record<string, GuestDrawerHistoryDeferredMetric> => {
const metrics: Record<string, GuestDrawerHistoryDeferredMetric> = {};
if (isGuestDrawerVM(guest) && guest.diskStatusReason) {
// History judges the aggregate disk metric by the same rule as the
// filesystem rows, so a linked agent's reading follows that agent.
const disk = getWorkloadGuestDiskRead(guest, isGuestDrawerVM(guest));
if (disk.state !== 'current') {
metrics.disk = {
lastKnownValue: guest.diskStatusReason.startsWith('prev-')
? getGuestDrawerDiskUsage(guest)
: undefined,
message: getWorkloadGuestDiskStatusMessage(guest.diskStatusReason),
lastKnownValue: disk.state === 'last-known' ? getGuestDrawerDiskUsage(guest) : undefined,
message: disk.message,
};
}
const memory = getGuestDrawerMemoryReading(guest);
@@ -188,7 +190,9 @@ export const getGuestDrawerCurrentMetrics = (guest: Guest): Record<string, numbe
// A paused filesystem read is not current, even if the snapshot still
// carries a numeric summary. Retained evidence has its own labelled path.
const diskUsage =
isGuestDrawerVM(guest) && guest.diskStatusReason ? undefined : getGuestDrawerDiskUsage(guest);
getWorkloadGuestDiskRead(guest, isGuestDrawerVM(guest)).state === 'current'
? getGuestDrawerDiskUsage(guest)
: undefined;
const finite = (value: number | undefined): number | undefined =>
typeof value === 'number' && Number.isFinite(value) ? value : undefined;
return {
@@ -9,6 +9,7 @@ import { buildMetricKey } from '@/utils/metricsKeys';
import { getGuestHealthIndicator, isGuestRunning } from '@/utils/status';
import { formatBytes } from '@/utils/format';
import { getContainerRuntimeBadgeForRuntime } from '@/utils/resourceBadgePresentation';
import { getWorkloadGuestDiskRead } from '@/utils/workloadGuestPresentation';
import {
getCanonicalWorkloadId,
getWorkloadPlatformScopes,
@@ -225,21 +226,21 @@ export function useGuestRowState(props: GuestRowProps) {
return ((disk.used ?? 0) / total) * 100;
});
// The cell's filesystem bars and aggregate follow the drawer's rule: a
// linked agent's filesystems follow that agent, Proxmox's its read reason.
const diskUsageRead = createMemo(() =>
getWorkloadGuestDiskRead(props.guest, workloadType() === 'vm'),
);
const hasDiskUsage = createMemo(() => {
const guest = props.guest;
if (!guest.disk || guest.telemetryAvailability?.disk === false) return false;
const total = guest.disk.total ?? 0;
if (!Number.isFinite(total) || total <= 0) return false;
// A generic canonical merge can retain the previous numeric metric while
// the provider explicitly reports no usable guest reading. Only prev-
// reasons allow that number to remain visible as last-known evidence.
if (
workloadType() === 'vm' &&
'diskStatusReason' in guest &&
guest.diskStatusReason &&
!guest.diskStatusReason.startsWith('prev-')
)
return false;
// its source reports no usable guest reading. Only a last known reading
// may keep that number visible.
if (diskUsageRead().state === 'unavailable') return false;
const percent = diskPercent();
return Number.isFinite(percent) && percent >= 0;
});
@@ -315,6 +316,7 @@ export function useGuestRowState(props: GuestRowProps) {
discoveryReadinessPresentation,
diskRead,
diskThresholds,
diskUsageRead,
diskWrite,
displayId,
dockerHostId,
@@ -5,9 +5,12 @@ import { TableCell, TableRow, type TableRowProps } from './Table';
export const INLINE_DETAIL_TABLE_CELL_CLASS = 'p-0 border-b border-border bg-surface-alt';
// overflow-x-clip keeps over-wide content from painting outside the row's
// border below the lg breakpoint (issue #1622) without creating a scroll
// container the way overflow-hidden would.
// container the way overflow-hidden would. Below lg the sticky content is as
// wide as the table's visible area: 100cqi resolves against the table scroll
// shell (a size container), so a guess at the page chrome cannot leave an
// empty strip beside the drawer or push it past the visible edge.
export const INLINE_DETAIL_TABLE_CONTENT_CLASS =
'sticky left-0 min-w-0 max-w-[calc(100vw-3.5rem)] whitespace-normal overflow-x-clip px-2 py-3 sm:px-4 sm:py-4 lg:static lg:max-w-none lg:overflow-x-visible';
'sticky left-0 min-w-0 max-w-[100cqi] whitespace-normal overflow-x-clip px-2 py-3 sm:px-4 sm:py-4 lg:static lg:max-w-none lg:overflow-x-visible';
export interface InlineDetailTableRowProps extends TableRowProps {
cellId?: string;
@@ -8757,7 +8757,10 @@ describe('shared primitive guardrails', () => {
expect(summaryRowActionButtonSource).toContain('h-9 w-9');
expect(searchFieldSource).toContain('min-h-11');
expect(filterToolbarSource).toContain('min-h-11');
expect(inlineDetailTableRowSource).toContain('max-w-[calc(100vw-3.5rem)]');
// Phone and tablet drawers fill the table's visible width (the scroll
// shell is a size container), not a viewport guess at the page chrome.
expect(inlineDetailTableRowSource).toContain('max-w-[100cqi]');
expect(inlineDetailTableRowSource).not.toContain('100vw-3.5rem');
});
it('keeps shared settings panels dense without forking their desktop framing', () => {
@@ -39,7 +39,7 @@ describe('InlineDetailTableRow', () => {
expect(detail.parentElement).toHaveClass('left-0');
expect(detail.parentElement).toHaveClass('min-w-0');
expect(detail.parentElement).toHaveClass('whitespace-normal');
expect(detail.parentElement).toHaveClass('max-w-[calc(100vw-3.5rem)]');
expect(detail.parentElement).toHaveClass('max-w-[100cqi]');
expect(detail.parentElement).toHaveClass('lg:static');
expect(detail.parentElement).toHaveClass('lg:max-w-none');
});
@@ -56,7 +56,8 @@ describe('readAlertsConfigurationSnapshot — absent optional sections', () => {
expect(snapshot.trueNASDefaults.cpu).toBe(80);
expect(snapshot.trueNASDefaults.usage).toBe(85);
expect(snapshot.trueNASDefaults.diskRead).toBe(-1);
expect(snapshot.trueNASDiskDefaults.temperature).toBe(55);
// Unset: each TrueNAS disk follows Disk temperature by type.
expect(snapshot.trueNASDiskDefaults.temperature).toBeUndefined();
expect(snapshot.vmwareDefaults.cpu).toBe(80);
expect(snapshot.vmwareDefaults.usage).toBe(85);
});
@@ -232,7 +233,8 @@ describe('readAlertsConfigurationSnapshot — factory fallback (undefined-trigge
expect(snapshot.trueNASDefaults.diskWrite).toBe(-1);
expect(snapshot.trueNASDefaults.networkIn).toBe(-1);
expect(snapshot.trueNASDefaults.networkOut).toBe(-1);
expect(snapshot.trueNASDiskDefaults.temperature).toBe(55);
// No usable trigger leaves TrueNAS disks on Disk temperature by type.
expect(snapshot.trueNASDiskDefaults.temperature).toBeUndefined();
});
it('falls back to factory defaults for vmwareDefaults and agentDefaults', () => {
@@ -1526,3 +1528,48 @@ describe('dockerDefaults.updateAlertDelayHours', () => {
expect(alertConfig?.dockerDefaults?.updateAlertDelayHours).toBe(24);
});
});
describe('TrueNAS disk temperature round trip', () => {
const buildTrueNASDiskPayload = (temperature: number | undefined) => {
const snapshot = createDefaultAlertsConfigurationSnapshot();
snapshot.trueNASDiskDefaults = { temperature };
return buildAlertsConfigurationPayload({
snapshot,
rawOverridesConfig: {},
alertsActivationState: null,
alertsActivationConfig: null,
}).alertConfig;
};
it('omits an unset TrueNAS disk temperature so disks follow Disk temperature by type', () => {
const alertConfig = buildTrueNASDiskPayload(undefined);
expect(alertConfig?.truenasDiskDefaults).toEqual({});
expect(alertConfig?.truenasDiskTemperatureByType).toBe(true);
});
it('writes Off as a zero trigger, which the backend keeps as off', () => {
expect(buildTrueNASDiskPayload(-1)?.truenasDiskDefaults).toEqual({
temperature: { trigger: 0, clear: 0 },
});
});
it('writes a TrueNAS-wide value, 55 included, as a trigger', () => {
expect(buildTrueNASDiskPayload(55)?.truenasDiskDefaults).toEqual({
temperature: { trigger: 55, clear: 50 },
});
});
it('reads an absent temperature as unset and a zero trigger as off', () => {
const base = { overrides: {} } as unknown as AlertConfig;
expect(
readAlertsConfigurationSnapshot({ ...base, truenasDiskDefaults: {} }).trueNASDiskDefaults
.temperature,
).toBeUndefined();
expect(
readAlertsConfigurationSnapshot({
...base,
truenasDiskDefaults: { temperature: { trigger: 0, clear: 0 } },
}).trueNASDiskDefaults.temperature,
).toBe(0);
});
});
@@ -185,6 +185,15 @@ const createHysteresisThreshold = (trigger: number | undefined, clearMargin = 5)
};
};
// Unset leaves TrueNAS disks on Disk temperature by type, so it is omitted
// rather than written as a trigger. Off is written as 0: the backend reads a
// negative trigger as unset.
const buildTrueNASDiskDefaultsConfig = (temperature: number | undefined) => {
if (temperature === undefined) return {};
if (temperature <= 0) return { temperature: { trigger: 0, clear: 0 } };
return { temperature: createHysteresisThreshold(temperature) };
};
const normalizeGap = (value: unknown, fallback: number) => {
const numeric = Number(value);
if (!Number.isFinite(numeric)) {
@@ -413,10 +422,10 @@ export function readAlertsConfigurationSnapshot(config: AlertConfig): AlertsConf
}
if (config.truenasDiskDefaults) {
// Unset means each TrueNAS disk follows Disk temperature by type.
const temperature = config.truenasDiskDefaults.temperature;
snapshot.trueNASDiskDefaults = {
temperature:
getTriggerValue(config.truenasDiskDefaults.temperature) ??
FACTORY_TRUENAS_DISK_DEFAULTS.temperature,
temperature: temperature === undefined ? undefined : getTriggerValue(temperature),
};
}
@@ -847,9 +856,8 @@ export function buildAlertsConfigurationPayload({
networkIn: createHysteresisThreshold(snapshot.trueNASDefaults.networkIn),
networkOut: createHysteresisThreshold(snapshot.trueNASDefaults.networkOut),
},
truenasDiskDefaults: {
temperature: createHysteresisThreshold(snapshot.trueNASDiskDefaults.temperature),
},
truenasDiskDefaults: buildTrueNASDiskDefaultsConfig(snapshot.trueNASDiskDefaults.temperature),
truenasDiskTemperatureByType: true,
vmwareDefaults: {
cpu: createHysteresisThreshold(snapshot.vmwareDefaults.cpu),
memory: createHysteresisThreshold(snapshot.vmwareDefaults.memory),
@@ -0,0 +1,44 @@
import { describe, expect, it } from 'vitest';
import {
getDiskTemperatureByTypeItems,
resolveDiskTemperatureTriggerForType,
resolveTrueNASDiskTemperatureDefault,
} from '../trueNASDiskTemperature';
const factoryPolicy = {
agentDiskTemperature: 55,
diskTempByType: { nvme: 70, sas: 65, sata: 55 },
};
describe('TrueNAS disk temperature in the thresholds editor', () => {
it('gives each disk its type trigger, and unknown types the agent default', () => {
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'NVMe')).toBe(70);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'sas')).toBe(65);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'sata')).toBe(55);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'hdd')).toBe(55);
expect(
resolveDiskTemperatureTriggerForType({ ...factoryPolicy, agentDiskTemperature: 60 }, ''),
).toBe(60);
});
it('switches every type off with the agent Disk Temp default', () => {
const off = { ...factoryPolicy, agentDiskTemperature: -1 };
expect(resolveDiskTemperatureTriggerForType(off, 'nvme')).toBe(0);
expect(getDiskTemperatureByTypeItems(off)).toEqual([]);
});
it('lets a TrueNAS-wide value, Off included, replace the type trigger', () => {
expect(resolveTrueNASDiskTemperatureDefault(undefined, factoryPolicy, 'nvme')).toBe(70);
expect(resolveTrueNASDiskTemperatureDefault(62, factoryPolicy, 'nvme')).toBe(62);
expect(resolveTrueNASDiskTemperatureDefault(-1, factoryPolicy, 'nvme')).toBe(-1);
});
it('names the per-type triggers', () => {
expect(
getDiskTemperatureByTypeItems({
...factoryPolicy,
diskTempByType: { nvme: 75, sas: 65, sata: 55 },
}),
).toEqual(['NVMe 75°C', 'SAS 65°C', 'SATA 55°C']);
});
});
@@ -9,6 +9,7 @@ import type { Resource } from '@/types/resource';
import type { Resource as TableResource } from '../tableTypes';
import type { Override } from '../types';
import type { ThresholdsDataInputs } from '../thresholdsResourceModel';
import { resolveTrueNASDiskTemperatureDefault } from '../trueNASDiskTemperature';
import {
createOverridesMap,
findOverrideByCandidates,
@@ -307,7 +308,18 @@ export function useThresholdsPlatformData(inputs: ThresholdsDataInputs) {
resource,
type: 'truenasDisk',
resourceType: 'TrueNAS Disk',
defaults: trueNASDiskDefaults(),
// Each disk inherits the trigger saving would apply to it: the
// TrueNAS-wide value, else its type's disk temperature trigger.
defaults: {
temperature: resolveTrueNASDiskTemperatureDefault(
trueNASDiskDefaults().temperature,
{
agentDiskTemperature: props.agentDefaults?.diskTemperature,
diskTempByType: props.diskTempByType ?? {},
},
resource.physicalDisk?.diskType,
),
},
overridesMap,
node: trueNASSystemName(resource),
instance: readString(resource.physicalDisk?.diskType) || 'TrueNAS',
@@ -0,0 +1,51 @@
import { formatMetricValue } from './helpers';
import { FACTORY_AGENT_DEFAULTS } from '@/utils/alertThresholdDefaults';
// The thresholds editor's view of the TrueNAS disk temperature trigger. It
// mirrors the backend (trueNASDiskTemperatureDefaultNoLock): a TrueNAS-wide
// value the user saved applies to every TrueNAS disk; unset, each disk follows
// Disk temperature by type, which the agent Disk Temp default switches off for
// every type when it is off. It reads the unsaved editor state, so a row shows
// what saving would apply.
export const DISK_TEMPERATURE_TYPE_FIELDS: readonly { key: string; label: string }[] = [
{ key: 'nvme', label: 'NVMe' },
{ key: 'sas', label: 'SAS' },
{ key: 'sata', label: 'SATA' },
];
export interface DiskTemperaturePolicyState {
/** The agent Disk Temp default trigger; zero or less is off. */
agentDiskTemperature: number | undefined;
diskTempByType: Record<string, number>;
}
/** The disk temperature policy's trigger for a disk type; zero or less is off. */
export function resolveDiskTemperatureTriggerForType(
policy: DiskTemperaturePolicyState,
diskType: string | null | undefined,
): number {
const base = policy.agentDiskTemperature ?? FACTORY_AGENT_DEFAULTS.diskTemperature;
if (base <= 0) return 0;
const normalizedType = (diskType ?? '').trim().toLowerCase();
const byType = normalizedType ? policy.diskTempByType[normalizedType] : undefined;
return typeof byType === 'number' && byType > 0 ? byType : base;
}
/** The trigger a TrueNAS disk without its own override inherits. */
export function resolveTrueNASDiskTemperatureDefault(
trueNASDiskTemperature: number | undefined,
policy: DiskTemperaturePolicyState,
diskType: string | null | undefined,
): number {
if (trueNASDiskTemperature !== undefined) return trueNASDiskTemperature;
return resolveDiskTemperatureTriggerForType(policy, diskType);
}
/** "NVMe 70°C", one per disk type, or none when the policy is off. */
export function getDiskTemperatureByTypeItems(policy: DiskTemperaturePolicyState): string[] {
return DISK_TEMPERATURE_TYPE_FIELDS.flatMap((field) => {
const trigger = resolveDiskTemperatureTriggerForType(policy, field.key);
return trigger > 0 ? [`${field.label} ${formatMetricValue('temperature', trigger)}`] : [];
});
}
@@ -63,7 +63,7 @@ import {
import type { EvidenceEnvelope } from '@/types/operationalTrust';
import type { ActionDetailResponse } from '@/types/actionAudit';
import { getAlertResourceIncidentAcknowledgedByLabel } from '@/utils/alertIncidentPresentation';
import { formatRelativeTime } from '@/utils/format';
import { formatAbsoluteTime, formatRelativeTime, formatTimeUntil } from '@/utils/format';
import { useRelativeTimeNow } from '@/utils/relativeTimeClock';
import { copyToClipboard } from '@/utils/clipboard';
import type { PatrolAutonomyLevel } from '@/api/patrol';
@@ -152,14 +152,31 @@ function getAttentionSuppressionEndPresentation(reviewed: boolean) {
durationLabel: 'End suppression after',
actionLabel: 'End suppression',
successLabel: 'Suppression ended. It stays reviewed',
countdownLabel: (remaining: string) => `Suppression ends ${remaining}. It stays reviewed.`,
}
: {
durationLabel: 'Return it to active attention after',
actionLabel: 'Return to active attention',
successLabel: 'Returned to decision inbox',
countdownLabel: (remaining: string) => `Returns to active attention ${remaining}.`,
};
}
// The suppression line counts down to the expiry in the same words as the
// early-end button, and says so once the expiry has passed.
export function getAttentionSuppressionExpiryLabel(
expiresAt: string,
reviewed: boolean,
now: number,
): string {
const expiry = Date.parse(expiresAt);
if (!Number.isFinite(expiry)) return '';
if (expiry <= now) return 'Suppression has ended.';
return getAttentionSuppressionEndPresentation(reviewed).countdownLabel(
formatTimeUntil(expiry, { compact: true, now }),
);
}
export function PatrolAttentionWorkbench(
props: {
autonomyLevel?: PatrolAutonomyLevel;
@@ -1284,8 +1301,9 @@ function AttentionLifecycleControls(props: {
onSuppress: (itemId: string, reason: string, expiresAt: string) => Promise<void>;
onUnsuppress: (itemId: string, reviewed: boolean) => Promise<void>;
}) {
// The acknowledgement time never changes while the detail stays open, so its
// age reads the shared clock.
// The acknowledgement and suppression expiry times never change while the
// detail stays open, so the acknowledgement age and the countdown to the
// expiry read the shared clock.
const now = useRelativeTimeNow();
const [showSuppression, setShowSuppression] = createSignal(false);
const [reason, setReason] = createSignal('');
@@ -1328,10 +1346,13 @@ function AttentionLifecycleControls(props: {
</p>
<Show when={suppression().expiresAt}>
{(expiresAt) => (
<p>
Returns to active attention {formatRelativeTime(expiresAt(), { compact: true })}
.
</p>
<Show when={getAttentionSuppressionExpiryLabel(expiresAt(), reviewed(), now())}>
{(label) => (
<p title={`Suppressed until ${formatAbsoluteTime(Date.parse(expiresAt()))}`}>
{label()}
</p>
)}
</Show>
)}
</Show>
</div>
@@ -23,7 +23,6 @@ import { getPatrolTriggerStatusSummary } from '@/utils/patrolRunPresentation';
import { getPatrolSetupAction } from '@/utils/patrolRuntimeActions';
import { getPatrolRecencyPresentation } from '@/utils/patrolSummaryPresentation';
import { PATROL_CONTROL_ANCHOR, PATROL_OPERATIONS_LOOP_ANCHOR } from '@/routing/resourceLinks';
import type { PatrolConfigurationFailureInput } from './patrolInvestigationContextModel';
import { getPatrolAutonomyAvailabilityPresentation } from './patrolAutonomyAvailability';
import { PATROL_AUTONOMY_POLICY_PRESENTATION } from './patrolControlPresentation';
import { PATROL_AUTONOMY_EXPERIENCE } from './patrolHomePresentation';
@@ -35,25 +34,6 @@ import { PatrolAutopilotAcknowledgementDialog } from './PatrolAutopilotAcknowled
export { PATROL_AUTONOMY_POLICY_PRESENTATION } from './patrolControlPresentation';
const isNonEmptyConfigurationDetail = (value?: string | null): value is string =>
Boolean(value?.trim());
export function getPatrolConfigurationFailureInlineDetails(
failure: PatrolConfigurationFailureInput,
): string[] {
const readiness = failure.readiness ?? null;
const codeAndCause = [failure.code, readiness?.cause || failure.blockedCause]
.filter(isNonEmptyConfigurationDetail)
.join(' · ');
return [
codeAndCause || undefined,
readiness?.summary ? `Setup: ${readiness.summary}` : undefined,
readiness?.provider ? `Provider: ${readiness.provider}` : undefined,
readiness?.model ? `Model: ${readiness.model}` : undefined,
].filter(isNonEmptyConfigurationDetail);
}
export function getPatrolAutopilotExpiry(expiresAt?: string | null): Date | null {
if (!expiresAt?.trim()) return null;
const expiry = new Date(expiresAt);
@@ -91,6 +91,7 @@ vi.mock('@/features/actions/ActionReviewDialog', async () => {
});
import {
getAttentionSuppressionExpiryLabel,
getDistinctPatrolImpact,
getPatrolDecisionDisplayTitle,
PatrolAttentionWorkbench,
@@ -519,6 +520,74 @@ describe('PatrolAttentionWorkbench', () => {
}
});
it('counts down to when a suppressed issue returns to active attention', async () => {
vi.useFakeTimers({
toFake: ['Date', 'setInterval', 'clearInterval'],
now: Date.parse('2026-07-19T08:10:00Z'),
});
const suppressed = item({ state: 'suppressed' });
const suppressedDetail = detail(suppressed);
// A 24-hour suppression set just now.
suppressedDetail.operationalRecord.suppression = {
at: '2026-07-19T08:10:00Z',
by: 'operator',
reason: 'Planned storage maintenance',
expiresAt: '2026-07-20T08:10:00Z',
};
const handled = summary({ suppressedCount: 1 });
apiMocks.getList.mockImplementation((scope: string) =>
Promise.resolve(listResponse(scope === 'all' ? [suppressed] : [], handled)),
);
apiMocks.getDetail.mockResolvedValue(suppressedDetail);
try {
renderWorkbench();
fireEvent.click(await screen.findByRole('button', { name: 'Reviewed and suppressed (1)' }));
fireEvent.click(
await screen.findByRole('button', { name: 'Open Database VM · Disk pressure' }),
);
const detailRegion = await screen.findByRole('complementary', {
name: 'Database VM · Disk pressure',
});
const expiry = await within(detailRegion).findByText(/Returns to active attention/);
expect(expiry).toHaveTextContent('Returns to active attention in 1d.');
expect(expiry).toHaveAttribute('title', expect.stringMatching(/^Suppressed until /));
// The detail is not re-read: its suppression never changes and only the
// clock moves.
vi.advanceTimersByTime(4 * 60 * 60 * 1000);
expect(expiry).toHaveTextContent('Returns to active attention in 20h.');
vi.setSystemTime(Date.parse('2026-07-20T07:30:00Z'));
vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);
expect(expiry).toHaveTextContent('Returns to active attention in 40m.');
vi.setSystemTime(Date.parse('2026-07-20T08:11:00Z'));
vi.advanceTimersByTime(RELATIVE_TIME_TICK_MS);
expect(expiry).toHaveTextContent('Suppression has ended.');
expect(detailRegion).not.toHaveTextContent('just now');
expect(apiMocks.getDetail).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
}
});
it('says a reviewed issue stays reviewed when its suppression ends', () => {
const now = Date.parse('2026-07-19T08:10:00Z');
// Expiry restores Acknowledged for a reviewed alert, which is not active
// attention, so only an unreviewed issue "returns to active attention".
expect(getAttentionSuppressionExpiryLabel('2026-07-20T08:10:00Z', true, now)).toBe(
'Suppression ends in 1d. It stays reviewed.',
);
expect(getAttentionSuppressionExpiryLabel('2026-07-19T11:10:00Z', false, now)).toBe(
'Returns to active attention in 3h.',
);
expect(getAttentionSuppressionExpiryLabel('2026-07-19T08:00:00Z', true, now)).toBe(
'Suppression has ended.',
);
expect(getAttentionSuppressionExpiryLabel('not a date', false, now)).toBe('');
});
it('keeps the detail header and queue controls pinned while a long decision scrolls', async () => {
const active = item();
apiMocks.getList.mockResolvedValue(
@@ -5,7 +5,6 @@ import { describe, expect, it } from 'vitest';
import {
PATROL_AUTONOMY_POLICY_PRESENTATION,
getPatrolAutopilotExpiry,
getPatrolConfigurationFailureInlineDetails,
} from '../PatrolIntelligenceHeader';
import {
getPatrolAlertAnalysisAvailabilityPresentation,
@@ -36,38 +35,6 @@ describe('PatrolIntelligenceHeader', () => {
);
});
it('keeps Patrol mode readiness context visible inline', () => {
expect(
getPatrolConfigurationFailureInlineDetails({
message: 'Patrol mode could not be saved.',
code: 'patrol_readiness_not_ready',
readiness: {
status: 'not_ready',
cause: 'model_unsupported_tools',
summary:
'The selected Patrol model is a reasoning-only model family that commonly does not emit tool calls.',
provider: 'openrouter',
model: 'openrouter:deepseek/deepseek-r1',
},
}),
).toEqual([
'patrol_readiness_not_ready · model_unsupported_tools',
'Setup: The selected Patrol model is a reasoning-only model family that commonly does not emit tool calls.',
'Provider: openrouter',
'Model: openrouter:deepseek/deepseek-r1',
]);
});
it('falls back to the blocked cause when readiness cause is absent', () => {
expect(
getPatrolConfigurationFailureInlineDetails({
message: 'Patrol mode could not be saved.',
code: 'patrol_autonomy_pro_required',
blockedCause: 'license_required',
}),
).toEqual(['patrol_autonomy_pro_required · license_required']);
});
it('keeps trigger runtime status in the header chrome after setup', () => {
expect(headerSource).toContain('getPatrolTriggerStatusSummary');
expect(headerSource).toContain('state.patrolStatus()?.trigger_status');
@@ -870,7 +870,6 @@ export function usePatrolIntelligenceState() {
aiIntelligenceStore.loadIntelligenceSummary(),
aiIntelligenceStore.loadFindings(),
aiIntelligenceStore.loadCircuitBreakerStatus(),
aiIntelligenceStore.loadCorrelations(),
]);
}
@@ -916,24 +915,6 @@ export function usePatrolIntelligenceState() {
window.history.replaceState(window.history.state, '', nextPath);
}
const summaryStats = () => {
const patrolFindings = allPatrolFindings();
const activeFindings = patrolFindings.filter((finding) => finding.status === 'active');
return {
criticalFindings: activeFindings.filter((finding) => finding.severity === 'critical').length,
warningFindings: activeFindings.filter((finding) => finding.severity === 'warning').length,
totalActive: activeFindings.length,
fixedCount: patrolFindings.filter(
(finding) =>
finding.investigationOutcome === 'fix_verified' ||
finding.investigationOutcome === 'fix_executed' ||
finding.investigationOutcome === 'resolved',
).length,
hasAnyPatrolFindings: patrolFindings.length > 0,
};
};
const activePatrolFindings = () =>
allPatrolFindings().filter((finding) => finding.status === 'active');
const shouldShowPatrolSetupOnly = createMemo(() => {
@@ -1134,7 +1115,6 @@ export function usePatrolIntelligenceState() {
showBlockedBanner,
showReadinessBanner,
shouldShowPatrolSetupOnly,
summaryStats,
triggerPatrolDisabledReason,
};
}
@@ -1,6 +1,6 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/testing-library';
import { Suspense, createSignal, type JSX } from 'solid-js';
import { Suspense, type JSX } from 'solid-js';
import { resetCreateNonSuspendingQueryCacheForTest } from '@/hooks/createNonSuspendingQuery';
import {
PATROL_CONTROL_ANCHOR,
@@ -17,6 +17,7 @@ import patrolIntelligenceHeaderSource from '@/features/patrol/PatrolIntelligence
import patrolIntelligenceSurfaceSource from '@/features/patrol/PatrolIntelligenceSurface.tsx?raw';
import patrolIntelligenceStateSource from '@/features/patrol/usePatrolIntelligenceState.ts?raw';
import { AGENT_PATROL_CONTROL_STATUS_PATH } from '@/api/agentCapabilities';
import { aiIntelligenceStore } from '@/stores/aiIntelligence';
import { AIIntelligence } from '../AIIntelligence';
@@ -85,29 +86,9 @@ const { findingsPanelState, runHistoryState, intelligenceState } = vi.hoisted(()
incidents_tracked: number;
};
} | null,
correlations: null as {
correlations: Array<{
source_id: string;
source_name: string;
source_type: string;
target_id: string;
target_name: string;
target_type: string;
event_pattern: string;
occurrences: number;
avg_delay: number | string;
confidence: number;
last_seen: string;
description?: string;
}>;
count: number;
} | null,
},
}));
const getCorrelationsMock = vi.fn();
const [correlationsState, setCorrelationsState] =
createSignal<(typeof intelligenceState)['correlations']>(null);
const getPatrolStatusMock = vi.fn();
const getPatrolAutonomySettingsMock = vi.fn();
const updatePatrolAutonomySettingsMock = vi.fn();
@@ -215,7 +196,6 @@ vi.mock('@/api/patrol', () => ({
vi.mock('@/api/ai', () => ({
AIAPI: {
getCorrelations: (...args: unknown[]) => getCorrelationsMock(...args),
getModels: (...args: unknown[]) => apiFetchJSONMock('/api/ai/models', ...args),
getSettings: (...args: unknown[]) => apiFetchJSONMock('/api/settings/ai', ...args),
updateSettings: (...args: unknown[]) => apiFetchJSONMock('/api/settings/ai/update', ...args),
@@ -276,19 +256,12 @@ vi.mock('@/stores/aiIntelligence', () => {
loadIntelligenceSummary: vi.fn().mockResolvedValue(undefined),
loadCircuitBreakerStatus: vi.fn().mockResolvedValue(undefined),
loadPendingApprovals: vi.fn().mockResolvedValue(undefined),
loadCorrelations: vi.fn().mockImplementation(async () => {
const response = await getCorrelationsMock();
intelligenceState.correlations = response;
setCorrelationsState(response);
return response;
}),
loadDashboardData: vi.fn().mockImplementation(async () => {
await Promise.all([
store.loadFindings(),
store.loadIntelligenceSummary(),
store.loadCircuitBreakerStatus(),
store.loadPendingApprovals(),
store.loadCorrelations(),
]);
}),
get findings() {
@@ -306,9 +279,6 @@ vi.mock('@/stores/aiIntelligence', () => {
get patrolPendingApprovals() {
return [];
},
get correlations() {
return correlationsState();
},
};
return {
@@ -663,9 +633,6 @@ describe('AIIntelligence entitlement gating', () => {
intelligenceState.findings = [];
intelligenceState.circuitBreakerStatus = null;
intelligenceState.summary = null;
intelligenceState.correlations = null;
setCorrelationsState(null);
getCorrelationsMock.mockReset();
getPatrolStatusMock.mockResolvedValue(defaultPatrolStatus());
getPatrolAutonomySettingsMock.mockResolvedValue(defaultPatrolAutonomySettings());
@@ -728,10 +695,6 @@ describe('AIIntelligence entitlement gating', () => {
presentationPolicyHidesCommercialSurfacesMock.mockReturnValue(false);
recordWorkflowPromptActivityMock.mockResolvedValue(undefined);
window.history.replaceState({}, '', '/patrol');
getCorrelationsMock.mockResolvedValue({
correlations: [],
count: 0,
});
});
it('keeps the Patrol page heading accessible name singular when the logo is present', async () => {
@@ -957,15 +920,23 @@ describe('AIIntelligence entitlement gating', () => {
expect(screen.getAllByRole('button', { name: /Check now/i }).length).toBeGreaterThan(0);
});
getCorrelationsMock.mockImplementation(() => new Promise(() => {}));
// Stall the Patrol refresh that follows an accepted start: the run button
// must leave Starting without waiting on it.
const loadIntelligenceSummary = vi.mocked(aiIntelligenceStore.loadIntelligenceSummary);
loadIntelligenceSummary.mockClear();
loadIntelligenceSummary.mockImplementation(() => new Promise(() => {}));
try {
fireEvent.click(screen.getAllByRole('button', { name: /Check now/i })[0]);
fireEvent.click(screen.getAllByRole('button', { name: /Check now/i })[0]);
await waitFor(() => {
expect(triggerPatrolRunMock).toHaveBeenCalled();
expect(screen.getAllByRole('button', { name: /Running/i }).length).toBeGreaterThan(0);
});
expect(screen.queryByRole('button', { name: /Starting/i })).not.toBeInTheDocument();
await waitFor(() => {
expect(triggerPatrolRunMock).toHaveBeenCalled();
expect(screen.getAllByRole('button', { name: /Running/i }).length).toBeGreaterThan(0);
expect(loadIntelligenceSummary).toHaveBeenCalled();
});
expect(screen.queryByRole('button', { name: /Starting/i })).not.toBeInTheDocument();
} finally {
loadIntelligenceSummary.mockResolvedValue(null);
}
});
it('surfaces backend readiness rejection when a stale manual run request reaches the server', async () => {
@@ -1066,40 +1037,6 @@ describe('AIIntelligence entitlement gating', () => {
incidents_tracked: 0,
},
};
getCorrelationsMock.mockResolvedValue({
correlations: [
{
source_id: 'storage-2',
source_name: 'Storage 2',
source_type: 'storage',
target_id: 'vm-200',
target_name: 'VM 200',
target_type: 'vm',
event_pattern: 'disk_full -> restart',
occurrences: 2,
avg_delay: '1m30s',
confidence: 0.95,
last_seen: '2026-03-01T00:05:00Z',
description: 'Disk pressure often precedes restarts',
},
{
source_id: 'storage-1',
source_name: 'Storage 1',
source_type: 'storage',
target_id: 'vm-100',
target_name: 'VM 100',
target_type: 'vm',
event_pattern: 'cpu_high -> restart',
occurrences: 1,
avg_delay: '3m',
confidence: 0.5,
last_seen: '2026-03-01T00:03:00Z',
description: 'Lower-confidence backup pattern',
},
],
count: 2,
});
render(() => <AIIntelligence />);
await waitFor(() => {
@@ -1126,9 +1063,6 @@ describe('AIIntelligence entitlement gating', () => {
expect(
screen.queryByText('What Patrol was allowed to inspect or act on.'),
).not.toBeInTheDocument();
expect(screen.queryByText('Storage 2')).not.toBeInTheDocument();
expect(screen.queryByText('VM 200')).not.toBeInTheDocument();
expect(screen.queryByText('Disk Full → Restart')).not.toBeInTheDocument();
await openPatrolActivityMode();
fireEvent.click(screen.getByRole('button', { name: 'History' }));
@@ -11,7 +11,6 @@ vi.mock('@/api/ai', () => ({
approvePendingApproval: vi.fn(),
denyPendingApproval: vi.fn(),
getIntelligenceSummary: vi.fn(),
getCorrelations: vi.fn(),
},
}));
@@ -6,7 +6,7 @@ vi.mock('@/api/ai', () => ({
getRemediationPlans: vi.fn(),
getPendingApprovals: vi.fn(),
getIntelligenceSummary: vi.fn(),
getCorrelations: vi.fn(),
getCircuitBreakerStatus: vi.fn(),
},
}));
@@ -593,35 +593,6 @@ describe('aiIntelligenceStore', () => {
});
});
it('loads the canonical correlations response', async () => {
vi.mocked(AIAPI.getCorrelations).mockResolvedValueOnce({
correlations: [
{
source_id: 'storage-1',
source_name: 'Storage 1',
source_type: 'storage',
target_id: 'vm-42',
target_name: 'VM 42',
target_type: 'vm',
event_pattern: 'disk_full -> restart',
occurrences: 2,
avg_delay: '1m30s',
confidence: 0.875,
last_seen: '2026-03-01T00:00:00Z',
description: 'Disk pressure often precedes restarts',
},
],
count: 1,
});
await aiIntelligenceStore.loadCorrelations();
expect(aiIntelligenceStore.correlations).toMatchObject({
count: 1,
});
expect(aiIntelligenceStore.correlations?.correlations).toHaveLength(1);
});
it('loads the canonical AI dashboard bundle', async () => {
vi.mocked(AIAPI.getIntelligenceSummary).mockResolvedValueOnce({
timestamp: '2026-03-01T00:00:00Z',
@@ -651,17 +622,27 @@ describe('aiIntelligenceStore', () => {
incidents_tracked: 0,
},
});
vi.mocked(AIAPI.getCorrelations).mockResolvedValueOnce({
correlations: [],
count: 0,
});
vi.mocked(AIAPI.getCircuitBreakerStatus).mockResolvedValueOnce(null as never);
await aiIntelligenceStore.loadDashboardData();
expect(AIAPI.getIntelligenceSummary).toHaveBeenCalledTimes(1);
expect(AIAPI.getCorrelations).toHaveBeenCalledTimes(1);
expect(aiIntelligenceStore.intelligenceSummary?.findings_count.total).toBe(0);
expect(aiIntelligenceStore.correlations?.count).toBe(0);
// These are every AIAPI read the bundle makes (Patrol findings come from
// the Patrol API module). It does not load the global learned-correlation
// list: no page renders it, and the resource drawer reads per-resource
// correlations from resource intelligence instead.
const aiReads = Object.entries(AIAPI)
.filter(([, read]) => vi.isMockFunction(read) && read.mock.calls.length > 0)
.map(([name]) => name)
.sort();
expect(aiReads).toEqual([
'getCircuitBreakerStatus',
'getIntelligenceSummary',
'getPendingApprovals',
'getUnifiedFindings',
]);
expect(getPatrolFindings).toHaveBeenCalledTimes(1);
});
it('treats queued fixes without a live approval as findings needing attention', async () => {
+1 -27
View File
@@ -41,7 +41,7 @@ import {
import { getApprovalExpiryTime, isLivePendingApproval } from '@/utils/approvalState';
import { sortPendingApprovalsByUrgency } from '@/utils/approvalRiskPresentation';
import { logger } from '@/utils/logger';
import type { CorrelationsResponse, IntelligenceSummary } from '@/types/aiIntelligence';
import type { IntelligenceSummary } from '@/types/aiIntelligence';
import { normalizeIntelligenceSummary } from './aiIntelligenceSummaryModel';
import { presentationPolicyIsDemoMode } from './sessionPresentationPolicy';
import { eventBus } from './events';
@@ -465,12 +465,6 @@ const [intelligenceSummary, setIntelligenceSummary] = createSignal<IntelligenceS
null,
);
// ============================================
// Learned Correlations
// ============================================
const [correlations, setCorrelations] = createSignal<CorrelationsResponse | null>(null);
// ============================================
// Org Switch Reset
// ============================================
@@ -490,7 +484,6 @@ eventBus.on('org_switched', () => {
setApprovalsError(null);
setCircuitBreakerStatus(null);
setIntelligenceSummary(null);
setCorrelations(null);
});
// ============================================
@@ -809,30 +802,12 @@ export const aiIntelligenceStore = {
},
circuitBreakerStatusSignal: circuitBreakerStatus,
// Learned Correlations
get correlations() {
return correlations();
},
correlationsSignal: correlations,
// Canonical Intelligence Summary
get intelligenceSummary() {
return intelligenceSummary();
},
intelligenceSummarySignal: intelligenceSummary,
async loadCorrelations(resourceId?: string) {
try {
const response = await AIAPI.getCorrelations(resourceId);
setCorrelations(response);
return response;
} catch (e) {
logger.error('Failed to load correlations:', e);
setCorrelations(null);
return null;
}
},
async loadDashboardData() {
await Promise.all([
this.loadIntelligenceSummary(),
@@ -840,7 +815,6 @@ export const aiIntelligenceStore = {
this.loadPatrolFindings(),
this.loadCircuitBreakerStatus(),
this.loadPendingApprovals(),
this.loadCorrelations(),
]);
},
+3
View File
@@ -147,6 +147,9 @@ export interface AlertConfig {
kubernetesDefaults?: AlertThresholds;
truenasDefaults?: AlertThresholds;
truenasDiskDefaults?: AlertThresholds;
// Set on every config written since an unset truenasDiskDefaults
// temperature means each TrueNAS disk follows Disk temperature by type.
truenasDiskTemperatureByType?: boolean;
vmwareDefaults?: AlertThresholds;
snapshotDefaults?: SnapshotAlertConfig;
backupDefaults?: BackupAlertConfig;
@@ -96,7 +96,8 @@ describe('alertThresholdDefaults', () => {
describe('FACTORY_TRUENAS_DISK_DEFAULTS', () => {
it('only carries a disk temperature trigger', () => {
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: 55 });
// Unset: each TrueNAS disk follows Disk temperature by type.
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: undefined });
expect(Object.keys(FACTORY_TRUENAS_DISK_DEFAULTS)).toEqual(['temperature']);
});
});
@@ -175,6 +175,17 @@ describe('docsLinks', () => {
expect(handler).toContain('agentID != "" && len(*compatibility.ObservationLocationIDs) == 1');
});
it('documents alert configuration saves as replacing only the keys they send', () => {
const api = readFileSync(path.join(repoRoot, 'docs', 'API.md'), 'utf8');
expect(api).toContain('`PUT /api/alerts/config` — replaces only the top-level keys the body');
expect(api).toContain('a key left out keeps its stored value');
const handler = readFileSync(
path.join(repoRoot, 'internal', 'api', 'alerting', 'alerts.go'),
'utf8',
);
expect(handler).toContain('GetAlertManager().ApplyConfigUpdate(body)');
});
it('separates server removal from persistent-data erasure', () => {
const installation = readFileSync(path.join(repoRoot, 'docs', 'INSTALL.md'), 'utf8');
const removal = installation.split('## 🗑️ Uninstall')[1];
@@ -13,6 +13,7 @@ import {
formatUptime,
formatAbsoluteTime,
formatRelativeTime,
formatTimeUntil,
getBackupInfo,
} from '@/utils/format';
@@ -263,6 +264,67 @@ describe('formatRelativeTime', () => {
});
});
describe('formatTimeUntil', () => {
const now = Date.parse('2024-03-15T12:00:00Z');
const minute = 60 * 1000;
const hour = 60 * minute;
const day = 24 * hour;
it('returns empty text for falsy or unparseable input', () => {
expect(formatTimeUntil(undefined, { now })).toBe('');
expect(formatTimeUntil(0, { now, emptyText: '-' })).toBe('-');
expect(formatTimeUntil('not a date', { now, emptyText: '-' })).toBe('-');
});
it('counts down to a future time instead of reading it as just now', () => {
expect(formatTimeUntil(now + 30 * 1000, { now })).toBe('in under a minute');
expect(formatTimeUntil(now + 5 * minute, { now, compact: true })).toBe('in 5m');
expect(formatTimeUntil(now + 3 * hour, { now, compact: true })).toBe('in 3h');
expect(formatTimeUntil(now + 6 * day, { now, compact: true })).toBe('in 6d');
expect(formatTimeUntil(now + 45 * day, { now, compact: true })).toBe('in 45d');
});
it('spells units out in the long form', () => {
expect(formatTimeUntil(now + minute, { now })).toBe('in 1 min');
expect(formatTimeUntil(now + 5 * minute, { now })).toBe('in 5 mins');
expect(formatTimeUntil(now + hour, { now })).toBe('in 1 hour');
expect(formatTimeUntil(now + 5 * hour, { now })).toBe('in 5 hours');
expect(formatTimeUntil(now + day, { now })).toBe('in 1 day');
expect(formatTimeUntil(now + 7 * day, { now })).toBe('in 7 days');
expect(formatTimeUntil(now + 60 * day, { now })).toBe('in 2 months');
expect(formatTimeUntil(now + 365 * day, { now })).toBe('in 1 year');
});
it('rounds to the nearest unit, so a duration just set reads as chosen', () => {
// A 24-hour, 1-hour and 7-day suppression set a few seconds ago.
expect(formatTimeUntil(now + day - 5000, { now, compact: true })).toBe('in 1d');
expect(formatTimeUntil(now + hour - 5000, { now, compact: true })).toBe('in 1h');
expect(formatTimeUntil(now + 7 * day - 5000, { now, compact: true })).toBe('in 7d');
expect(formatTimeUntil(now + 2 * hour + 10 * minute, { now, compact: true })).toBe('in 2h');
expect(formatTimeUntil(now + 2 * hour + 50 * minute, { now, compact: true })).toBe('in 3h');
// Months round from the remaining time itself, not from rounded days.
expect(formatTimeUntil(now + 44 * day + 12 * hour, { now })).toBe('in 1 month');
expect(formatTimeUntil(now + 45 * day + hour, { now })).toBe('in 2 months');
});
it('reads as due once the time has arrived or passed', () => {
expect(formatTimeUntil(now, { now })).toBe('now');
expect(formatTimeUntil(now - hour, { now, compact: true })).toBe('now');
expect(formatTimeUntil(now - hour, { now, dueText: 'overdue' })).toBe('overdue');
});
it('accepts ISO strings and Date objects and defaults now to the current time', () => {
vi.useFakeTimers();
vi.setSystemTime(now);
try {
expect(formatTimeUntil('2024-03-15T15:00:00Z', { compact: true })).toBe('in 3h');
expect(formatTimeUntil(new Date(now + 10 * minute), { compact: true })).toBe('in 10m');
} finally {
vi.useRealTimers();
}
});
});
describe('getBackupInfo', () => {
beforeEach(() => {
vi.useFakeTimers();
@@ -478,7 +478,6 @@ import systemLogsPresentationSource from '@/utils/systemLogsPresentation.ts?raw'
import patrolEmptyStatePresentationSource from '@/utils/patrolEmptyStatePresentation.ts?raw';
import approvalRiskPresentationSource from '@/utils/approvalRiskPresentation.ts?raw';
import patrolRunPresentationSource from '@/utils/patrolRunPresentation.ts?raw';
import patrolSummaryPresentationSource from '@/utils/patrolSummaryPresentation.ts?raw';
import aiCostDashboardSource from '@/components/AI/AICostDashboard.tsx?raw';
import modelSelectorSource from '@/components/AI/Chat/ModelSelector.tsx?raw';
import aiModelPickerSource from '@/components/shared/AIModelPicker.tsx?raw';
@@ -3570,8 +3569,6 @@ describe('frontend resource type boundaries', () => {
expect(patrolIntelligenceSurfaceSource).not.toContain(
'inline-flex items-center rounded-full border px-2.5 py-1 text-[11px] font-semibold uppercase tracking-[0.16em]',
);
expect(patrolSummaryPresentationSource).toContain('badgeVariant:');
expect(patrolSummaryPresentationSource).not.toContain('badgeClass:');
expect(patrolIntelligenceWorkspaceSource).toContain('MetadataBadge');
// Work-group badge tones resolve in the patrolControlPresentation adapter
// (getPatrolWorkspaceWorkGroups), never inline in the workspace component.
@@ -3651,18 +3648,6 @@ describe('frontend resource type boundaries', () => {
'historicalRegressionCount={state.historicalRegressionCount()}',
);
expect(patrolIntelligenceBannersSource).not.toContain('trackUpgradeClicked');
expect(patrolIntelligenceSurfaceSource).not.toContain(
"summaryStats().criticalFindings > 0\n ? 'bg-red-50 dark:bg-red-900/25 border-red-200 dark:border-red-800'",
);
expect(patrolIntelligenceSurfaceSource).not.toContain(
"summaryStats().warningFindings > 0\n ? 'bg-amber-50 dark:bg-amber-900/25 border-amber-200 dark:border-amber-800'",
);
expect(patrolIntelligenceSurfaceSource).not.toContain(
"summaryStats().fixedCount > 0\n ? 'bg-green-50 dark:bg-green-900/25 border-green-200 dark:border-green-800'",
);
expect(patrolSummaryPresentationSource).toContain(
'export function getPatrolSummaryPresentation',
);
expect(patrolIntelligenceSurfaceSource).not.toContain(
"(patrolStatus()?.quickstart_credits_remaining ?? 0) > 0\n ? 'bg-blue-50 dark:bg-blue-950/25 border-blue-200 dark:border-blue-800 text-blue-700 dark:text-blue-300'",
);
@@ -507,7 +507,7 @@ describe('metricThresholds', () => {
usage: 85,
temperature: 80,
});
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: 55 });
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: undefined });
expect(FACTORY_VMWARE_DEFAULTS).toEqual({
cpu: 80,
memory: 85,
@@ -1,47 +1,13 @@
import { describe, expect, it } from 'vitest';
import type { PatrolRunRecord } from '@/api/patrol';
import type { IntelligenceHealthScore } from '@/types/aiIntelligence';
import {
getPatrolAssessmentPresentation,
getPatrolAssessmentShellPresentation,
getPatrolCompactAssessmentLabel,
getPatrolRecencyPresentation,
getPatrolSummaryMetricState,
getPatrolVerificationPresentation,
} from '@/utils/patrolSummaryPresentation';
// ---------------------------------------------------------------------------
// Fixture builders — minimal, typed objects that satisfy the module's
// internal PatrolAssessmentFinding / PatrolRunRecord / IntelligenceHealthScore
// shapes without needing the non-exported PatrolAssessmentFinding alias.
// ---------------------------------------------------------------------------
type AssessmentFinding = NonNullable<
Parameters<typeof getPatrolCompactAssessmentLabel>[0]['activeFindings']
>[number];
function makeFinding(overrides: Partial<AssessmentFinding> = {}): AssessmentFinding {
return {
resourceId: 'vm-100',
resourceName: 'web-1',
title: 'Disk nearly full',
severity: 'warning',
status: 'active',
...overrides,
};
}
function makeRuntimeFinding(overrides: Partial<AssessmentFinding> = {}): AssessmentFinding {
return makeFinding({
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider connection issue',
...overrides,
});
}
// Minimal typed PatrolRunRecord fixtures.
function makeRun(overrides: Partial<PatrolRunRecord> = {}): PatrolRunRecord {
return {
id: 'run-1',
@@ -73,26 +39,6 @@ function makeRun(overrides: Partial<PatrolRunRecord> = {}): PatrolRunRecord {
};
}
function makeHealth(overrides: Partial<IntelligenceHealthScore> = {}): IntelligenceHealthScore {
return {
score: 90,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
...overrides,
};
}
function coverageFactor() {
return {
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
};
}
function successfulFullRun(resourcesChecked = 50): PatrolRunRecord {
return makeRun({
type: 'patrol',
@@ -102,668 +48,6 @@ function successfulFullRun(resourcesChecked = 50): PatrolRunRecord {
});
}
// ===========================================================================
// getPatrolAssessmentShellPresentation — covers all SemanticTone map entries
// and the fallback for an unknown tone.
// ===========================================================================
describe('getPatrolAssessmentShellPresentation', () => {
it('maps the success tone to the emerald shell', () => {
expect(getPatrolAssessmentShellPresentation('success')).toEqual({
headerClass: 'bg-emerald-50/60 dark:bg-emerald-950/30',
badgeVariant: 'success',
iconClass: 'text-emerald-600 dark:text-emerald-300',
iconContainerClass:
'border-emerald-200 bg-emerald-50 dark:border-emerald-800 dark:bg-emerald-950/40',
});
});
it('maps the error tone to the red shell with a danger badge', () => {
expect(getPatrolAssessmentShellPresentation('error')).toEqual({
headerClass: 'bg-red-50/70 dark:bg-red-950/30',
badgeVariant: 'danger',
iconClass: 'text-red-600 dark:text-red-300',
iconContainerClass: 'border-red-200 bg-red-50 dark:border-red-800 dark:bg-red-950/40',
});
});
it('maps an explicit info tone to the blue shell', () => {
expect(getPatrolAssessmentShellPresentation('info')).toEqual({
headerClass: 'bg-blue-50/70 dark:bg-blue-950/30',
badgeVariant: 'info',
iconClass: 'text-blue-600 dark:text-blue-300',
iconContainerClass: 'border-blue-200 bg-blue-50 dark:border-blue-800 dark:bg-blue-950/40',
});
});
it('falls back to the info shell for an unrecognised tone string', () => {
expect(getPatrolAssessmentShellPresentation('critical' as never)).toEqual(
getPatrolAssessmentShellPresentation('info'),
);
});
});
// ===========================================================================
// getPatrolSummaryMetricState — covers classifyActiveFindings severity
// branches and default fallbacks.
// ===========================================================================
describe('getPatrolSummaryMetricState', () => {
it('flags primarySeverity as critical when an infrastructure critical finding is active', () => {
const state = getPatrolSummaryMetricState({
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1', title: 'Host down' }),
],
});
expect(state.primarySeverity).toBe('critical');
expect(state.primaryValue).toBe(1);
expect(state.criticalValue).toBe(1);
expect(state.secondarySeverity).toBe('warning');
});
it('flags secondarySeverity as critical when the runtime issue is critical', () => {
const state = getPatrolSummaryMetricState({
activeFindings: [makeRuntimeFinding({ severity: 'critical' })],
});
expect(state.secondarySeverity).toBe('critical');
expect(state.secondaryValue).toBe(1);
expect(state.criticalValue).toBe(1);
expect(state.primaryLabel).toBe('Infrastructure findings');
});
it('defaults fixedValue to 0 when fixedCount is omitted', () => {
expect(getPatrolSummaryMetricState({}).fixedValue).toBe(0);
});
it('ignores findings whose status is not active', () => {
const state = getPatrolSummaryMetricState({
activeFindings: [
makeFinding({ status: 'resolved', severity: 'critical' }),
makeRuntimeFinding({ status: 'dismissed', severity: 'critical' }),
],
});
expect(state.primaryValue).toBe(0);
expect(state.secondaryValue).toBe(0);
expect(state.criticalValue).toBe(0);
});
it('combines mixed infrastructure and runtime severities into the critical total', () => {
const state = getPatrolSummaryMetricState({
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1' }),
makeRuntimeFinding({ severity: 'critical' }),
makeFinding({ severity: 'warning', resourceId: 'vm-2' }),
],
});
expect(state.criticalValue).toBe(2);
expect(state.primarySeverity).toBe('critical');
expect(state.secondarySeverity).toBe('critical');
});
});
// ===========================================================================
// getPatrolCompactAssessmentLabel — covers formatIssueLabel singular/plural,
// totalActive fallback, health-score suppression by absence.
// ===========================================================================
describe('getPatrolCompactAssessmentLabel', () => {
it('lists both critical and warning infrastructure findings when both are active', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1', title: 'Host down' }),
makeFinding({ severity: 'warning', resourceId: 'vm-2', title: 'Disk full' }),
],
}),
).toBe('1 critical issue · 1 warning issue');
});
it('pluralises infrastructure critical issues', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1' }),
makeFinding({ severity: 'critical', resourceId: 'vm-2' }),
],
}),
).toBe('2 critical issues');
});
it('pluralises Patrol runtime issues when more than one is active', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
activeFindings: [
makeRuntimeFinding({ severity: 'warning', title: 'Pulse Patrol: Issue A' }),
makeRuntimeFinding({ severity: 'warning', title: 'Pulse Patrol: Issue B' }),
],
}),
).toBe('2 Patrol runtime issues');
});
it('falls back to the totalActive count when classification yields no parts', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
totalActive: 3,
activeFindings: [makeFinding({ status: 'resolved', severity: 'warning' })],
}),
).toBe('3 active issues');
});
it('omits the health-score segment when overallHealth is absent', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'No active issues',
}),
).toBe('No active issues');
});
it('rounds the health score to the nearest whole number', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'No active issues',
overallHealth: makeHealth({ score: 72.4 }),
}),
).toBe('No active issues · health score 72/100');
});
it('pluralises historical regressions', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'No active issues',
historicalRegressionCount: 3,
}),
).toBe('No active issues · 3 past regressions');
});
it('combines findings, regressions, and health score into one compact label', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
overallHealth: makeHealth({ score: 60, grade: 'C' }),
activeFindings: [makeFinding({ severity: 'critical', resourceId: 'vm-1' })],
historicalRegressionCount: 1,
}),
).toBe('1 critical issue · 1 past regression · health score 60/100');
});
});
// ===========================================================================
// getPatrolAssessmentPresentation — runtime-state branches, critical-finding
// branches, coverage-gap health tones (getHealthSummaryTone), and
// getCoverageDescription fallback.
// ===========================================================================
describe('getPatrolAssessmentPresentation — runtime state', () => {
it('maps the blocked state to the paused runtime presentation', () => {
expect(getPatrolAssessmentPresentation({ runtimeState: 'blocked' })).toEqual({
title: 'Patrol paused',
description: 'Patrol cannot check infrastructure until the blocking condition is cleared.',
eyebrow: 'Patrol paused',
compactLabel: 'Patrol paused',
tone: 'warning',
});
});
it('maps the disabled state to the disabled runtime presentation', () => {
expect(getPatrolAssessmentPresentation({ runtimeState: 'disabled' })).toEqual({
title: 'Patrol disabled',
description: 'Enable Patrol to resume checks.',
eyebrow: 'Patrol disabled',
compactLabel: 'Patrol disabled',
tone: 'info',
});
});
it('maps the unavailable state to the unavailable runtime presentation', () => {
expect(getPatrolAssessmentPresentation({ runtimeState: 'unavailable' })).toEqual({
title: 'Patrol unavailable',
description: 'Patrol is not ready yet. Check Provider & Models and runtime availability.',
eyebrow: 'Patrol unavailable',
compactLabel: 'Patrol unavailable',
tone: 'error',
});
});
it('passes the blocked reason through to the assessment description', () => {
expect(
getPatrolAssessmentPresentation({
runtimeState: 'blocked',
blockedReason: ' provider offline ',
}).description,
).toBe('provider offline');
});
});
describe('getPatrolAssessmentPresentation — critical findings', () => {
it('reports critical issues detected for infrastructure critical findings', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 1,
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1', title: 'Host down' }),
],
}),
).toEqual({
title: 'Critical issues detected',
description:
'Patrol surfaced 1 active critical finding in your infrastructure. Review the active findings for more detail.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'error',
});
});
it('reports a critical Patrol runtime issue when only a runtime finding is critical', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 1,
activeFindings: [makeRuntimeFinding({ severity: 'critical' })],
}),
).toEqual({
title: 'Critical Patrol runtime issue',
description:
'Patrol is currently blocked by a critical runtime issue: Provider connection issue. Review the Patrol runtime issue for more detail.',
eyebrow: 'Status',
compactLabel: 'Patrol runtime issue',
tone: 'error',
});
});
});
describe('getPatrolAssessmentPresentation — coverage gap health tones (getHealthSummaryTone)', () => {
it('uses the error tone when health grade is D', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 45,
grade: 'D',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
}),
).toMatchObject({ title: 'Coverage incomplete', tone: 'error' });
});
it('uses the error tone when health grade is F', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 25,
grade: 'F',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
}),
).toMatchObject({ title: 'Coverage incomplete', tone: 'error' });
});
it('uses the warning tone when health grade is C with a coverage gap', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 70,
grade: 'C',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
}),
).toMatchObject({ title: 'Coverage incomplete', tone: 'warning' });
});
});
describe('getPatrolAssessmentPresentation — getCoverageDescription fallback', () => {
it('falls back to the default message when the prediction is whitespace-only', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 50,
grade: 'D',
factors: [coverageFactor()],
prediction: ' ',
}),
}).description,
).toBe('Patrol has not finished a current check.');
});
});
describe('getPatrolAssessmentPresentation — health requires attention', () => {
it('uses a usable prediction as the description for non-A health', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 80,
grade: 'B',
prediction: 'Some warnings need review.',
}),
}),
).toEqual({
title: 'Health requires attention',
description: 'Some warnings need review.',
eyebrow: 'Status',
compactLabel: 'Health requires attention',
tone: 'warning',
});
});
it('suppresses a coverage-gap prediction after a verified full run', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 80,
grade: 'B',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete. Run Patrol to check everything.',
}),
runs: [successfulFullRun()],
}),
).toEqual({
title: 'Health requires attention',
description: 'Patrol still needs attention.',
eyebrow: 'Status',
compactLabel: 'Health requires attention',
tone: 'warning',
});
});
it('falls back when the health prediction is empty', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({ score: 80, grade: 'B', prediction: '' }),
}).description,
).toBe('Patrol still needs attention.');
});
it('keeps a non-coverage-gap prediction even after a verified full run', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 80,
grade: 'B',
factors: [coverageFactor()],
prediction: 'A few warnings were detected.',
}),
runs: [successfulFullRun()],
}).description,
).toBe('A few warnings were detected.');
});
});
describe('getPatrolAssessmentPresentation — default success', () => {
it('returns the default success shell when no health is provided', () => {
expect(getPatrolAssessmentPresentation({})).toEqual({
title: 'No active issues detected',
description: 'Infrastructure is healthy with no significant issues detected.',
eyebrow: 'Status',
compactLabel: 'No active issues',
tone: 'success',
});
});
it('falls back to the default description when grade-A health has an empty prediction', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({ grade: 'A', prediction: '' }),
}).description,
).toBe('Infrastructure is healthy with no significant issues detected.');
});
});
// ===========================================================================
// getFindingAssessmentDescription (private) — exercised through
// getPatrolAssessmentPresentation. Covers single-runtime-issue branches,
// joinAssessmentParts with 1/2/3+ parts, formatRuntimeIssueCount,
// getRuntimeFindingSummaryLabel, shouldUsePrediction,
// predictionReadsAsAllClear, predictionReadsAsCoverageGap,
// criticalFindings/warningFindings fallbacks.
// ===========================================================================
describe('getFindingAssessmentDescription — single runtime issue', () => {
it('uses the critical-severity runtime summary for a single critical runtime issue', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 1,
activeFindings: [makeRuntimeFinding({ severity: 'critical' })],
}).description,
).toContain('Patrol is currently blocked by a critical runtime issue');
});
it('appends the coverage-gap caveat for a single critical runtime issue', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 1,
overallHealth: makeHealth({
score: 50,
grade: 'D',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
activeFindings: [makeRuntimeFinding({ severity: 'critical' })],
}).description,
).toBe(
'Patrol is currently blocked by a critical runtime issue: Provider connection issue. Recent coverage is incomplete. Run Patrol to check everything.',
);
});
it('uses a usable prediction for a single runtime issue', () => {
expect(
getPatrolAssessmentPresentation({
warningFindings: 1,
overallHealth: makeHealth({
score: 85,
grade: 'B',
prediction: 'A runtime hiccup was observed.',
}),
activeFindings: [makeRuntimeFinding({ severity: 'warning' })],
}).description,
).toBe('A runtime hiccup was observed.');
});
it('falls back to "a Patrol runtime issue" when the runtime finding has no title', () => {
expect(
getPatrolAssessmentPresentation({
warningFindings: 1,
activeFindings: [makeRuntimeFinding({ title: '' })],
}).description,
).toBe(
'Patrol has an active runtime issue: a Patrol runtime issue. Review the Patrol runtime issue for more detail.',
);
});
it('keeps an unnormalised runtime title that has no Pulse Patrol prefix', () => {
expect(
getPatrolAssessmentPresentation({
warningFindings: 1,
activeFindings: [makeRuntimeFinding({ title: 'Some custom runtime error' })],
}).description,
).toContain('Some custom runtime error');
});
});
describe('getFindingAssessmentDescription — multi-finding joinAssessmentParts', () => {
it('joins two summary parts with "and"', () => {
expect(
getPatrolAssessmentPresentation({
warningFindings: 2,
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1', title: 'Host down' }),
makeFinding({ severity: 'warning', resourceId: 'vm-2', title: 'Disk full' }),
],
}).description,
).toBe(
'Patrol surfaced 1 active critical finding in your infrastructure and 1 active warning finding in your infrastructure. Review the active findings for more detail.',
);
});
it('joins four summary parts with an Oxford comma', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 2,
activeFindings: [
makeFinding({ severity: 'critical', resourceId: 'vm-1', title: 'Host down' }),
makeFinding({ severity: 'warning', resourceId: 'vm-2', title: 'Disk full' }),
makeRuntimeFinding({ severity: 'critical' }),
makeRuntimeFinding({
severity: 'warning',
title: 'Pulse Patrol: Slow response',
}),
],
}).description,
).toBe(
'Patrol surfaced 1 active critical finding in your infrastructure, 1 active warning finding in your infrastructure, 1 active critical Patrol runtime issue, and 1 active Patrol runtime issue. Review the active findings for more detail.',
);
});
it('uses the criticalFindings fallback when no active findings classify', () => {
expect(
getPatrolAssessmentPresentation({
criticalFindings: 3,
activeFindings: [makeFinding({ status: 'resolved', severity: 'critical' })],
}).description,
).toBe(
'Patrol surfaced 3 active critical findings. Review the active findings for more detail.',
);
});
it('uses the warningFindings fallback when no active findings classify and no criticals exist', () => {
expect(
getPatrolAssessmentPresentation({
warningFindings: 2,
}).description,
).toBe(
'Patrol surfaced 2 active warning findings. Review the active findings for more detail.',
);
});
});
describe('predictionReadsAsAllClear (via getFindingAssessmentDescription)', () => {
it.each([
['healthy with no significant issue'],
['no significant issues detected'],
['no active issues'],
['no issues detected'],
['all clear'],
['ALL CLEAR'], // case-insensitive
])('suppresses the all-clear prediction "%s" for a single runtime issue', (prediction) => {
const description = getPatrolAssessmentPresentation({
warningFindings: 1,
overallHealth: makeHealth({ score: 95, grade: 'A', prediction }),
activeFindings: [makeRuntimeFinding({ severity: 'warning' })],
}).description;
expect(description).not.toBe(prediction);
expect(description).toContain('Patrol has an active runtime issue');
});
it('returns false for an empty prediction so the runtime summary is used', () => {
const description = getPatrolAssessmentPresentation({
warningFindings: 1,
overallHealth: makeHealth({ score: 95, grade: 'A', prediction: '' }),
activeFindings: [makeRuntimeFinding({ severity: 'warning' })],
}).description;
expect(description).toContain('Review the Patrol runtime issue for more detail.');
});
});
describe('predictionReadsAsCoverageGap (via getPatrolAssessmentPresentation)', () => {
it.each([
['coverage is incomplete'],
['coverage incomplete'],
['not fully verified'],
['limited to scoped runs'],
['limited to targeted'],
['runs encountered errors'],
['ended with errors'],
['current issue list'],
['current full issue list'],
['current health may be incomplete'],
['summary may be incomplete'],
])('suppresses the coverage-gap prediction "%s" after a verified full run', (prediction) => {
const description = getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 80,
grade: 'B',
factors: [coverageFactor()],
prediction,
}),
runs: [successfulFullRun()],
}).description;
expect(description).toBe('Patrol still needs attention.');
});
});
// ===========================================================================
// hasSuccessfulFullCoverageRun (private) — exercised through
// getPatrolAssessmentPresentation's coverage-gap branch.
// ===========================================================================
describe('hasSuccessfulFullCoverageRun (via coverage-gap branch)', () => {
it('treats a full run with errors as not successful so the coverage gap stays', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 70,
grade: 'C',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
runs: [
makeRun({
type: 'patrol',
resources_checked: 50,
error_count: 3,
status: 'error',
}),
],
}),
).toMatchObject({ title: 'Coverage incomplete' });
});
it('treats a full run with zero resources as not successful so the coverage gap stays', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 70,
grade: 'C',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
runs: [makeRun({ type: 'patrol', resources_checked: 0, error_count: 0 })],
}),
).toMatchObject({ title: 'Coverage incomplete' });
});
it('treats a scoped-only run as not a full coverage run', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 70,
grade: 'C',
factors: [coverageFactor()],
prediction: 'Coverage is incomplete.',
}),
runs: [makeRun({ type: 'scoped', resources_checked: 5, error_count: 0 })],
}),
).toMatchObject({ title: 'Coverage incomplete' });
});
it('clears the coverage gap after a successful full run', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: makeHealth({
score: 85,
grade: 'B',
factors: [coverageFactor()],
prediction: 'A few warnings were detected.',
}),
runs: [successfulFullRun(60)],
}),
).toMatchObject({ title: 'Health requires attention' });
});
});
// ===========================================================================
// getPatrolVerificationPresentation — runtime-state branches, full-run-with-
// errors, zero-resource full run, limited-run variants (verification/scoped/
@@ -1033,7 +317,7 @@ describe('getVerificationActivityMixLabel (via getPatrolVerificationPresentation
// ===========================================================================
// getPatrolRecencyPresentation — timestamp fallback logic branches,
// formatRecencyResourcesCheckedLabel singular verified case.
// formatRecencyResourcesCheckedLabel singular case.
// ===========================================================================
describe('getPatrolRecencyPresentation — timestamp fallback logic', () => {
@@ -1,568 +1,10 @@
import { describe, expect, it } from 'vitest';
import {
getPatrolAssessmentPresentation,
getPatrolAssessmentAction,
getPatrolAssessmentShellPresentation,
getPatrolCompactAssessmentLabel,
getPatrolRecencyPresentation,
getPatrolScoreChipLabel,
getPatrolSummaryMetricState,
getPatrolVerificationPresentation,
getPatrolNoIssuesPresentation,
getPatrolSummaryPresentation,
PATROL_NO_ISSUES_LABEL,
} from '@/utils/patrolSummaryPresentation';
describe('getPatrolSummaryPresentation', () => {
it('returns active critical presentation', () => {
expect(getPatrolSummaryPresentation('critical', true)).toEqual({
iconClass: 'text-red-500 dark:text-red-400',
iconContainerClass: 'bg-red-50 dark:bg-red-900/25 border-red-200 dark:border-red-800',
valueClass: 'text-red-600 dark:text-red-400',
});
});
it('returns quiet presentation when the metric is zero', () => {
expect(getPatrolSummaryPresentation('warning', false)).toEqual({
iconClass: 'text-muted',
iconContainerClass: 'bg-surface border-border',
valueClass: 'text-muted',
});
});
it('exports canonical patrol empty-state copy', () => {
expect(PATROL_NO_ISSUES_LABEL).toBe('No active issues');
});
it('promotes coverage gaps into the primary patrol assessment state', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 70,
grade: 'C',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction: 'Patrol coverage is incomplete.',
},
}),
).toEqual({
title: 'Coverage incomplete',
description: 'Patrol coverage is incomplete.',
eyebrow: 'Status',
compactLabel: 'Coverage incomplete',
tone: 'warning',
});
});
it('reports issues detected when active warning findings exist', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 84,
grade: 'B',
trend: 'stable',
factors: [],
prediction: 'Warnings require review.',
},
warningFindings: 2,
}),
).toEqual({
title: 'Issues detected',
description: 'Warnings require review.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'warning',
});
});
it('surfaces an active running Patrol instead of a stale all-clear summary', () => {
expect(
getPatrolAssessmentPresentation({
runtimeState: 'running',
overallHealth: {
score: 100,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
}),
).toEqual({
title: 'Patrol running',
description: 'Patrol is checking your infrastructure now.',
eyebrow: 'Status',
compactLabel: 'Patrol is checking now',
tone: 'info',
});
});
it('does not reuse an all-clear health prediction while active infrastructure findings exist', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 95,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
warningFindings: 1,
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'vm-100',
resourceName: 'web-1',
title: 'Backup age drift',
},
] as never,
}),
).toEqual({
title: 'Issues detected',
description:
'Patrol surfaced 1 active warning finding in your infrastructure. Review the active findings for more detail.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'warning',
});
});
it('does not reuse an all-clear health prediction while a Patrol runtime issue is active', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 95,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
warningFindings: 1,
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider billing or quota issue',
},
] as never,
}),
).toEqual({
title: 'Patrol runtime issue',
description:
'Patrol has an active runtime issue: Provider billing or quota issue. Review the Patrol runtime issue for more detail.',
eyebrow: 'Status',
compactLabel: 'Patrol runtime issue',
tone: 'warning',
});
});
it('combines active findings with the coverage caveat when both are present', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 60,
grade: 'C',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction:
'Recent Patrol runs encountered errors, so the current health summary may be incomplete.',
},
warningFindings: 1,
}),
).toEqual({
title: 'Issues detected',
description:
'Patrol surfaced 1 active warning finding. Recent coverage is incomplete. Run Patrol to check everything.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'warning',
});
});
it('drops stale coverage caveats after a successful full patrol verified resources', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 85,
grade: 'B',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction:
'Recent Patrol activity only covered targeted checks and ended with errors. Run Patrol to check everything.',
},
warningFindings: 1,
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'backup-1',
resourceName: 'delly',
title: 'Backup failed',
},
] as never,
runs: [
{
completed_at: '2026-03-12T10:00:00Z',
type: 'patrol',
resources_checked: 58,
error_count: 0,
status: 'issues_found',
},
] as never,
}),
).toEqual({
title: 'Issues detected',
description:
'Patrol surfaced 1 active warning finding in your infrastructure. Review the active findings for more detail.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'warning',
});
});
it('keeps Patrol runtime issues distinct from infrastructure warning findings', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 65,
grade: 'C',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction:
'Recent Patrol activity only covered targeted checks and ended with errors. Run Patrol to check everything.',
},
warningFindings: 2,
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider connection issue',
},
{
status: 'active',
severity: 'warning',
resourceId: 'backup-1',
resourceName: 'delly',
title: 'Backup failed',
},
] as never,
}),
).toEqual({
title: 'Issues detected',
description:
'Patrol surfaced 1 active warning finding in your infrastructure and 1 active Patrol runtime issue. Recent coverage is incomplete. Run Patrol to check everything.',
eyebrow: 'Status',
compactLabel: 'Issues detected',
tone: 'warning',
});
});
it('classifies patrol-owned service failures as runtime issues instead of infrastructure issues', () => {
expect(
getPatrolAssessmentPresentation({
overallHealth: {
score: 60,
grade: 'C',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction:
'Recent Patrol activity only covered targeted checks and ended with errors. Run Patrol to check everything.',
},
warningFindings: 1,
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider billing or quota issue',
},
] as never,
}),
).toEqual({
title: 'Patrol runtime issue',
description:
'Patrol has an active runtime issue: Provider billing or quota issue. Recent coverage is incomplete. Run Patrol to check everything.',
eyebrow: 'Status',
compactLabel: 'Patrol runtime issue',
tone: 'warning',
});
});
it('uses assessment labeling for coverage or runtime-limited score states', () => {
expect(
getPatrolScoreChipLabel({
overallHealth: {
score: 60,
grade: 'C',
trend: 'stable',
factors: [
{
name: 'Patrol coverage incomplete',
impact: -0.35,
description: 'Patrol coverage is incomplete.',
category: 'coverage',
},
],
prediction: 'Patrol coverage is incomplete.',
},
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider billing or quota issue',
},
] as never,
}),
).toBe('Assessment');
});
it('offers a direct Patrol provider settings action for Patrol runtime issues', () => {
expect(
getPatrolAssessmentAction({
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider billing or quota issue',
},
] as never,
}),
).toEqual({
label: 'Check Patrol model',
href: '/settings/pulse-intelligence/patrol',
});
});
it('uses the shared neutral shell with a warning accent for degraded patrol summaries', () => {
expect(getPatrolAssessmentShellPresentation('warning')).toEqual({
headerClass: 'bg-amber-50/70 dark:bg-amber-950/30',
badgeVariant: 'warning',
iconClass: 'text-amber-600 dark:text-amber-300',
iconContainerClass: 'border-amber-200 bg-amber-50 dark:border-amber-800 dark:bg-amber-950/40',
});
});
it('defaults patrol shell styling to the informational accent', () => {
expect(getPatrolAssessmentShellPresentation()).toEqual({
headerClass: 'bg-blue-50/70 dark:bg-blue-950/30',
badgeVariant: 'info',
iconClass: 'text-blue-600 dark:text-blue-300',
iconContainerClass: 'border-blue-200 bg-blue-50 dark:border-blue-800 dark:bg-blue-950/40',
});
});
it('does not offer an AI settings action for infrastructure findings', () => {
expect(
getPatrolAssessmentAction({
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'vm-100',
resourceName: 'web-1',
title: 'Disk nearly full',
},
] as never,
}),
).toBeUndefined();
});
it('splits infrastructure findings from patrol runtime issues in supporting metrics', () => {
expect(
getPatrolSummaryMetricState({
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider billing or quota issue',
},
] as never,
fixedCount: 0,
}),
).toEqual({
primaryLabel: 'Infrastructure findings',
primaryValue: 0,
primarySeverity: 'warning',
secondaryLabel: 'Runtime issues',
secondaryValue: 1,
secondarySeverity: 'warning',
criticalLabel: 'Critical',
criticalValue: 0,
fixedLabel: 'Fixed',
fixedValue: 0,
});
});
it('keeps active findings and warnings labels when only infrastructure findings are active', () => {
expect(
getPatrolSummaryMetricState({
activeFindings: [
{
status: 'active',
severity: 'warning',
resourceId: 'vm-101',
resourceName: 'db-01',
title: 'Disk nearly full',
},
] as never,
fixedCount: 2,
}),
).toEqual({
primaryLabel: 'Active findings',
primaryValue: 1,
primarySeverity: 'warning',
secondaryLabel: 'Warnings',
secondaryValue: 1,
secondarySeverity: 'warning',
criticalLabel: 'Critical',
criticalValue: 0,
fixedLabel: 'Fixed',
fixedValue: 2,
});
});
it('keeps health labeling for verified healthy states', () => {
expect(
getPatrolScoreChipLabel({
overallHealth: {
score: 100,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
activeFindings: [],
}),
).toBe('Health');
});
it('names compact assessment evidence instead of exposing bare counters', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'No active issues',
overallHealth: {
score: 95,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
activeFindings: [],
historicalRegressionCount: 1,
}),
).toBe('No active issues · 1 past regression · health score 95/100');
});
it('uses issue nouns for compact active finding summaries', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Issues detected',
overallHealth: {
score: 84,
grade: 'B',
trend: 'stable',
factors: [],
prediction: 'Warnings require review.',
},
activeFindings: [
{
status: 'active',
severity: 'critical',
resourceId: 'vm-100',
resourceName: 'web-1',
title: 'High CPU usage',
},
{
status: 'active',
severity: 'warning',
resourceId: 'ai-service',
resourceName: 'Pulse Patrol Service',
title: 'Pulse Patrol: Provider connection issue',
},
] as never,
}),
).toBe('1 critical issue · 1 Patrol runtime issue · health score 84/100');
});
it('can suppress stale health scores while Patrol is still running', () => {
expect(
getPatrolCompactAssessmentLabel({
assessmentLabel: 'Patrol is checking now',
overallHealth: {
score: 65,
grade: 'C',
trend: 'stable',
factors: [],
prediction: 'Previous assessment needs attention.',
},
includeHealthScore: false,
}),
).toBe('Patrol is checking now');
});
it('keeps no-issues copy only for fully healthy patrol states', () => {
expect(
getPatrolNoIssuesPresentation({
overallHealth: {
score: 100,
grade: 'A',
trend: 'stable',
factors: [],
prediction: 'Infrastructure is healthy with no significant issues detected.',
},
}),
).toEqual({
label: 'No active issues',
tone: 'success',
});
});
describe('Patrol recency and verification presentation', () => {
it('reports a recent successful Patrol check', () => {
expect(
getPatrolVerificationPresentation({
@@ -935,7 +377,7 @@ describe('getPatrolSummaryPresentation', () => {
it('omits resourcesChecked when the most recent completed run reports zero coverage', () => {
// A run that completed without checking any resources (e.g. an early
// failure or a no-op trigger fire) should not surface a "verified 0
// failure or a no-op trigger fire) should not surface a "checked 0
// resources" line on the page header — that reads as an alarm signal
// when it's just a degenerate run. The presentation must omit the
// field instead of returning resourcesChecked: 0 so render code's
@@ -1706,6 +1706,82 @@ describe('fast merge path for metrics-only delta patches', () => {
}
});
it("lets a linked agent's filesystems and freshness replace withdrawn ones", () => {
const raw = createPveGuestRaw();
raw.sources = ['proxmox-pve', 'agent'] as Resource['sources'];
raw.platformData = { ...raw.platformData, sources: ['proxmox-pve', 'agent'] };
raw.proxmox = {
...raw.proxmox,
runtimeStatus: 'running',
guestAgentStatus: 'available',
disks: [{ mountpoint: '/', total: 2048, used: 512 }],
};
raw.agent = {
agentId: 'agent-in-vm',
stale: true,
disks: [{ mountpoint: '/', total: 4096, used: 1024 }],
} as Resource['agent'];
raw.platformData = { ...raw.platformData, agent: raw.agent };
const previous = seedDisplayRows([raw]);
// The guest agent stopped, so Proxmox withdraws its filesystems, and the
// linked Pulse agent reports again with an explicit stale=false.
const incoming = structuredClone(raw);
incoming.proxmox = {
vmid: 100,
nodeName: 'pve-node-1',
runtimeStatus: 'running',
guestAgentStatus: 'expected-unreachable',
diskStatusReason: 'agent-not-running',
};
incoming.agent = { ...raw.agent, stale: false } as Resource['agent'];
incoming.platformData = { ...incoming.platformData, agent: incoming.agent };
const changed = new Set([raw.id]);
const full = mergeCanonicalResourceSnapshot([incoming], previous);
const delta = mergeCanonicalResourceDeltaSnapshot([incoming], previous, changed);
const fast = mergeCanonicalResourceDeltaSnapshot(
[incoming],
previous,
changed,
new Map([[raw.id, ['proxmox', 'agent']]]),
);
for (const [merged] of [full, delta, fast]) {
expect(merged.proxmox?.disks).toBeUndefined();
expect(merged.proxmox?.diskStatusReason).toBe('agent-not-running');
expect(merged.agent?.stale).toBe(false);
expect(merged.agent?.disks).toEqual(raw.agent?.disks);
}
// A partial facet without a native outcome keeps the prior filesystems.
const partial = { ...raw, proxmox: { vmid: 100, uptime: 1001 } } as Resource;
const [kept] = mergeCanonicalResourceSnapshot([partial], previous);
expect(kept.proxmox?.disks).toEqual(raw.proxmox?.disks);
// A native agent report (it always carries stale) without disks has
// withdrawn them; a legacy facet without stale keeps the prior list.
const withdrawn = structuredClone(raw);
withdrawn.agent = { agentId: 'agent-in-vm', stale: false } as Resource['agent'];
withdrawn.platformData = { ...withdrawn.platformData, agent: withdrawn.agent };
for (const [merged] of [
mergeCanonicalResourceSnapshot([withdrawn], previous),
mergeCanonicalResourceDeltaSnapshot([withdrawn], previous, changed),
mergeCanonicalResourceDeltaSnapshot(
[withdrawn],
previous,
changed,
new Map([[raw.id, ['agent']]]),
),
]) {
expect(merged.agent?.disks).toBeUndefined();
expect(merged.agent?.stale).toBe(false);
// The compatibility mirror that host threshold rows read drops them too.
const mirror = (merged.platformData as { agent?: { disks?: unknown } } | undefined)?.agent;
expect(mirror?.disks).toBeUndefined();
}
const legacy = { ...raw, agent: { agentId: 'agent-in-vm' } } as Resource;
const [legacyKept] = mergeCanonicalResourceSnapshot([legacy], previous);
expect(legacyKept.agent?.disks).toEqual(raw.agent?.disks);
});
it('keeps explicit lock and deferral evidence on a new outcome, including false expected', () => {
const raw = createPveGuestRaw();
raw.proxmox = {
@@ -4,6 +4,7 @@ import {
getWorkloadsGuestBackupStatusPresentation,
getWorkloadsGuestBackupTooltip,
getWorkloadsGuestProtectionPresentation,
getWorkloadGuestDiskRead,
getWorkloadGuestDiskStatusMessage,
getWorkloadsGuestNetworkEmptyState,
} from '@/utils/workloadGuestPresentation';
@@ -104,4 +105,63 @@ describe('workloadGuestPresentation', () => {
'Using last known disk stats. No filesystems found. VM may be booting or using a Live ISO.',
);
});
it("judges a linked agent's filesystems by that agent, not the Proxmox read reason", () => {
const agent = { diskStatusReason: 'agent-not-running', disksFromAgent: true };
expect(getWorkloadGuestDiskRead({ ...agent, status: 'running' }, true)).toEqual({
state: 'current',
message: null,
needsAction: false,
});
expect(
getWorkloadGuestDiskRead({ ...agent, status: 'running', agentStale: true }, true),
).toEqual({
state: 'last-known',
message: 'Using last known disk stats. The Pulse Agent in this guest stopped reporting.',
needsAction: false,
});
// A stopped guest has no current filesystems from either source, with or
// without the Proxmox vm-stopped reason.
for (const guest of [
{ ...agent, status: 'stopped', agentStale: true },
{ ...agent, status: 'running', diskStatusReason: 'vm-stopped' },
]) {
expect(getWorkloadGuestDiskRead(guest, true)).toEqual({
state: 'unavailable',
message: 'Guest filesystem stats unavailable while the VM is stopped.',
needsAction: false,
});
}
expect(getWorkloadGuestDiskRead({ disksFromAgent: true, status: 'stopped' }, false)).toEqual({
state: 'unavailable',
message: 'Filesystem usage is unavailable.',
needsAction: false,
});
});
it('keeps Proxmox read reasons for its own guest filesystems', () => {
expect(getWorkloadGuestDiskRead({ status: 'running' }, true).state).toBe('current');
expect(getWorkloadGuestDiskRead({ diskStatusReason: 'prev-vm-locked' }, true)).toEqual({
state: 'last-known',
message: getWorkloadGuestDiskStatusMessage('prev-vm-locked'),
needsAction: false,
});
expect(getWorkloadGuestDiskRead({ diskStatusReason: 'agent-not-running' }, true)).toEqual({
state: 'unavailable',
message: getWorkloadGuestDiskStatusMessage('agent-not-running'),
needsAction: false,
});
expect(
getWorkloadGuestDiskRead({ diskStatusReason: 'prev-agent-error' }, true).needsAction,
).toBe(true);
expect(
getWorkloadGuestDiskRead({ diskStatusReason: 'permission-denied' }, true).needsAction,
).toBe(true);
// Only a VM carries a Proxmox guest read reason.
expect(getWorkloadGuestDiskRead({ diskStatusReason: 'prev-vm-locked' }, false).state).toBe(
'current',
);
// An agent's staleness says nothing about Proxmox's own filesystems.
expect(getWorkloadGuestDiskRead({ agentStale: true }, true).state).toBe('current');
});
});
@@ -5,7 +5,7 @@ import type { MetadataBadgeTone } from '@/components/shared/MetadataBadge';
import { isLivePendingApproval } from '@/utils/approvalState';
import { getPatrolProviderSettingsAction } from '@/utils/patrolRuntimeActions';
import { formatIdentifierLabel, titleCaseDelimitedLabel } from '@/utils/textPresentation';
import { formatBytes } from '@/utils/format';
import { formatAbsoluteTime, formatBytes, formatTimeUntil } from '@/utils/format';
const DEFAULT_BADGE_CLASSES = 'border-border bg-surface-alt text-muted';
const DEFAULT_LOOP_STATE_CLASSES = 'border-border bg-surface-alt text-muted';
@@ -275,6 +275,11 @@ export interface FindingRecencyPresentation {
timestamp: string;
}
export interface FindingCountdownPresentation {
label: string;
title: string;
}
export interface PatrolFindingClassification {
kind: 'runtime' | 'infrastructure';
label: string;
@@ -913,6 +918,49 @@ export const getFindingRecencyPresentation = (
};
};
// A will_fix_later reminder and the end of a snooze are future times, so the
// row counts down to them ("Reminding in 6d") from the `now` the panel passes
// from the shared relative-time clock, and keeps the exact time on hover. A
// reminder reads overdue when `overdue` says the panel's overdue list holds it,
// the same list the Overdue commitments chip and filter count, so the three
// agree; read between the deadline and the clock tick that adds it there, it
// says "Reminder due". It stays overdue until the hourly reminder sweep brings
// the finding back, whether or not it still trips. A passed snooze has ended,
// so neither keeps counting.
export const getFindingReminderPresentation = (
finding: Pick<UnifiedFinding, 'dismissedReason' | 'remindAt'>,
now: number,
overdue: boolean,
): FindingCountdownPresentation | null => {
if (finding.dismissedReason !== 'will_fix_later' || !finding.remindAt) return null;
const due = Date.parse(finding.remindAt);
if (!Number.isFinite(due)) return null;
const at = formatAbsoluteTime(due);
if (overdue) {
return {
label: 'Reminder overdue',
title: `Reminder was due ${at}. Pulse brings this finding back on its next reminder check.`,
};
}
return {
label:
due <= now ? 'Reminder due' : `Reminding ${formatTimeUntil(due, { compact: true, now })}`,
title: `Reminder due ${at}. Pulse brings this finding back on its first reminder check after that.`,
};
};
export const getFindingSnoozePresentation = (
finding: Pick<UnifiedFinding, 'status' | 'snoozedUntil'>,
now: number,
): FindingCountdownPresentation | null => {
if (finding.status !== 'snoozed' || !finding.snoozedUntil) return null;
const until = Date.parse(finding.snoozedUntil);
if (!Number.isFinite(until)) return null;
const title = `Snoozed until ${formatAbsoluteTime(until)}`;
if (until <= now) return { label: 'snooze ended', title };
return { label: `snoozed, returns ${formatTimeUntil(until, { compact: true, now })}`, title };
};
export const getInvestigationStatusBadgeClasses = (status: InvestigationStatus): string =>
INVESTIGATION_STATUS_CLASSES[status] || DEFAULT_BADGE_CLASSES;
@@ -44,8 +44,12 @@ export const FACTORY_TRUENAS_DEFAULTS = {
networkOut: -1,
};
export const FACTORY_TRUENAS_DISK_DEFAULTS = {
temperature: 55,
// TrueNAS disk temperature has no flat factory value. Unset, each TrueNAS disk
// follows Disk temperature by type, as the backend resolves it
// (trueNASDiskTemperatureDefaultNoLock); a saved value applies to every
// TrueNAS disk.
export const FACTORY_TRUENAS_DISK_DEFAULTS: Record<string, number | undefined> = {
temperature: undefined,
};
export const FACTORY_VMWARE_DEFAULTS = {
+53 -12
View File
@@ -169,9 +169,18 @@ export function formatAbsoluteTime(timestamp: number | undefined): string {
return `${day} ${month} ${hours}:${minutes}`;
}
function relativeTimeInputMs(timestamp: number | string | Date): number {
if (typeof timestamp === 'number') return timestamp;
if (typeof timestamp === 'string') return new Date(timestamp).getTime();
return timestamp.getTime();
}
/**
* Format a timestamp as a human-readable relative time string.
* ALL relative time formatting MUST use this function.
* Format a timestamp as a human-readable relative age ("5m ago").
* ALL relative ages MUST use this function. It is past-only: a time ahead of
* `now` is clock skew on something already observed and reads as "just now".
* A time meant to be in the future (an expiry, a reminder, a schedule) uses
* formatTimeUntil instead.
*
* @param timestamp - Unix ms number, ISO date string, Date object, or undefined
* @param options.compact - Use short format: "5m ago" instead of "5 mins ago"
@@ -185,19 +194,51 @@ export function formatRelativeTime(
): string {
if (!timestamp) return options?.emptyText ?? '';
let ms: number;
if (typeof timestamp === 'number') {
ms = timestamp;
} else if (typeof timestamp === 'string') {
ms = new Date(timestamp).getTime();
} else {
ms = timestamp.getTime();
}
const diffMs = (options?.now ?? Date.now()) - ms;
const diffMs = (options?.now ?? Date.now()) - relativeTimeInputMs(timestamp);
return formatTimeDiff(diffMs, options?.compact);
}
/**
* Format a future timestamp as a countdown: "in 3h" (compact) or "in 3 hours".
* ALL relative future times (expiries, reminders, schedules) MUST use this
* function, because formatRelativeTime reads any future time as "just now".
*
* Unlike an age, a countdown rounds to the nearest unit, so a 24-hour
* suppression set a moment ago reads "in 1d" rather than "in 23h".
*
* @param timestamp - Unix ms number, ISO date string, Date object, or undefined
* @param options.compact - Use short format: "in 5m" instead of "in 5 mins"
* @param options.emptyText - Text for falsy or unparseable input (default: '')
* @param options.dueText - Text once the time has arrived or passed (default: 'now')
* @param options.now - The time to count down from (default: Date.now()). A
* countdown that must keep moving passes the shared relative-time clock.
*/
export function formatTimeUntil(
timestamp: number | string | Date | undefined,
options?: { compact?: boolean; emptyText?: string; dueText?: string; now?: number },
): string {
if (!timestamp) return options?.emptyText ?? '';
const remainingMs = relativeTimeInputMs(timestamp) - (options?.now ?? Date.now());
if (!Number.isFinite(remainingMs)) return options?.emptyText ?? '';
if (remainingMs <= 0) return options?.dueText ?? 'now';
if (remainingMs < 60_000) return 'in under a minute';
const compact = options?.compact;
const inUnits = (value: number, short: string, singular: string, plural: string) =>
compact ? `in ${value}${short}` : `in ${value} ${value === 1 ? singular : plural}`;
const minutes = Math.round(remainingMs / 60_000);
if (minutes < 60) return inUnits(minutes, 'm', 'min', 'mins');
const hours = Math.round(remainingMs / 3_600_000);
if (hours < 24) return inUnits(hours, 'h', 'hour', 'hours');
const days = Math.round(remainingMs / 86_400_000);
if (compact || days < 30) return inUnits(days, 'd', 'day', 'days');
const months = Math.round(remainingMs / (30 * 86_400_000));
if (months < 12) return inUnits(months, '', 'month', 'months');
return inUnits(Math.round(remainingMs / (365 * 86_400_000)), '', 'year', 'years');
}
/**
* Formats a time difference in milliseconds to a human-readable string.
* Internal helper used by formatRelativeTime and formatBackupAge.
@@ -1,64 +1,10 @@
import type { PatrolRunRecord, PatrolRuntimeState } from '@/api/patrol';
import type { UnifiedFinding } from '@/stores/aiIntelligence';
import type { IntelligenceHealthScore } from '@/types/aiIntelligence';
import {
isPatrolRuntimeFinding,
normalizePatrolRuntimeFindingLabel,
} from '@/utils/aiFindingPresentation';
import {
formatPatrolActivityBreakdown,
getPatrolActivityBreakdown,
} from '@/utils/patrolRunPresentation';
import { getPatrolProviderSettingsAction } from '@/utils/patrolRuntimeActions';
import type { SemanticTone } from '@/utils/semanticTonePresentation';
import { getPatrolRuntimePresentation } from '@/utils/patrolRuntimePresentation';
import type { StatusIndicatorVariant } from '@/utils/status';
export type PatrolSummaryTone = 'critical' | 'warning' | 'success';
export interface PatrolSummaryPresentation {
iconClass: string;
iconContainerClass: string;
valueClass: string;
}
export interface PatrolNoIssuesPresentation {
label: string;
tone: SemanticTone;
}
export interface PatrolAssessmentPresentation {
title: string;
description: string;
eyebrow: string;
compactLabel: string;
tone: SemanticTone;
}
export interface PatrolAssessmentShellPresentation {
headerClass: string;
badgeVariant: StatusIndicatorVariant;
iconClass: string;
iconContainerClass: string;
}
export interface PatrolAssessmentAction {
label: string;
href: string;
}
export interface PatrolSummaryMetricState {
primaryLabel: string;
primaryValue: number;
primarySeverity: 'critical' | 'warning';
secondaryLabel: string;
secondaryValue: number;
secondarySeverity: 'critical' | 'warning';
criticalLabel: string;
criticalValue: number;
fixedLabel: string;
fixedValue: number;
}
export interface PatrolVerificationPresentation {
title: string;
@@ -73,386 +19,13 @@ export interface PatrolRecencyPresentation {
label: string;
timestamp?: string;
// resourcesChecked is the raw coverage signal for the most recent
// completed run. resourcesCheckedLabel is the operator-facing phrase; it
// says "verified" only for successful full checks and uses "checked" for
// limited or errored activity.
// completed run. resourcesCheckedLabel is the operator-facing phrase:
// "checked N resources" for any completed run with a positive count,
// whatever its type or error state. It never says "verified".
resourcesChecked?: number;
resourcesCheckedLabel?: string;
}
type PatrolAssessmentFinding = Pick<
UnifiedFinding,
'resourceId' | 'resourceName' | 'title' | 'severity' | 'status'
>;
export interface PatrolCompactAssessmentLabelInput {
assessmentLabel: string;
overallHealth?: IntelligenceHealthScore;
activeFindings?: PatrolAssessmentFinding[];
totalActive?: number;
historicalRegressionCount?: number;
includeHealthScore?: boolean;
}
export const PATROL_NO_ISSUES_LABEL = 'No active issues';
const PATROL_STATUS_EYEBROW = 'Status';
const QUIET_ICON_CONTAINER = 'bg-surface border-border';
const QUIET_ICON = 'text-muted';
const QUIET_VALUE = 'text-muted';
const ACTIVE_PRESENTATION: Record<PatrolSummaryTone, PatrolSummaryPresentation> = {
critical: {
iconClass: 'text-red-500 dark:text-red-400',
iconContainerClass: 'bg-red-50 dark:bg-red-900/25 border-red-200 dark:border-red-800',
valueClass: 'text-red-600 dark:text-red-400',
},
success: {
iconClass: 'text-green-500 dark:text-green-400',
iconContainerClass: 'bg-green-50 dark:bg-green-900/25 border-green-200 dark:border-green-800',
valueClass: 'text-green-600 dark:text-green-400',
},
warning: {
iconClass: 'text-amber-500 dark:text-amber-400',
iconContainerClass: 'bg-amber-50 dark:bg-amber-900/25 border-amber-200 dark:border-amber-800',
valueClass: 'text-amber-600 dark:text-amber-400',
},
};
const ASSESSMENT_SHELL_PRESENTATION: Record<SemanticTone, PatrolAssessmentShellPresentation> = {
success: {
headerClass: 'bg-emerald-50/60 dark:bg-emerald-950/30',
badgeVariant: 'success',
iconClass: 'text-emerald-600 dark:text-emerald-300',
iconContainerClass:
'border-emerald-200 bg-emerald-50 dark:border-emerald-800 dark:bg-emerald-950/40',
},
warning: {
headerClass: 'bg-amber-50/70 dark:bg-amber-950/30',
badgeVariant: 'warning',
iconClass: 'text-amber-600 dark:text-amber-300',
iconContainerClass: 'border-amber-200 bg-amber-50 dark:border-amber-800 dark:bg-amber-950/40',
},
error: {
headerClass: 'bg-red-50/70 dark:bg-red-950/30',
badgeVariant: 'danger',
iconClass: 'text-red-600 dark:text-red-300',
iconContainerClass: 'border-red-200 bg-red-50 dark:border-red-800 dark:bg-red-950/40',
},
info: {
headerClass: 'bg-blue-50/70 dark:bg-blue-950/30',
badgeVariant: 'info',
iconClass: 'text-blue-600 dark:text-blue-300',
iconContainerClass: 'border-blue-200 bg-blue-50 dark:border-blue-800 dark:bg-blue-950/40',
},
};
export function getPatrolSummaryPresentation(
tone: PatrolSummaryTone,
active: boolean,
): PatrolSummaryPresentation {
if (!active) {
return {
iconClass: QUIET_ICON,
iconContainerClass: QUIET_ICON_CONTAINER,
valueClass: QUIET_VALUE,
};
}
return ACTIVE_PRESENTATION[tone];
}
export function getPatrolAssessmentShellPresentation(
tone: SemanticTone = 'info',
): PatrolAssessmentShellPresentation {
return ASSESSMENT_SHELL_PRESENTATION[tone] || ASSESSMENT_SHELL_PRESENTATION.info;
}
function getHealthSummaryTone(overallHealth: IntelligenceHealthScore): SemanticTone {
return overallHealth.grade === 'D' || overallHealth.grade === 'F' ? 'error' : 'warning';
}
function getCoverageDescription(overallHealth: IntelligenceHealthScore | undefined): string {
return overallHealth?.prediction?.trim() || 'Patrol has not finished a current check.';
}
function formatFindingCount(count: number, severity: 'critical' | 'warning'): string {
return `${count} active ${severity} finding${count === 1 ? '' : 's'}`;
}
function formatRuntimeIssueCount(count: number, severity: 'critical' | 'warning'): string {
const severityLabel = severity === 'critical' ? 'critical ' : '';
return `${count} active ${severityLabel}Patrol runtime issue${count === 1 ? '' : 's'}`;
}
function getRuntimeFindingSummaryLabel(finding: PatrolAssessmentFinding | undefined): string {
const title = String(finding?.title || '').trim();
if (!title) {
return 'a Patrol runtime issue';
}
return normalizePatrolRuntimeFindingLabel(title) || title;
}
function classifyActiveFindings(activeFindings: PatrolAssessmentFinding[] | undefined) {
const runtimeCritical = (activeFindings ?? []).filter(
(finding) =>
finding.status === 'active' &&
finding.severity === 'critical' &&
isPatrolRuntimeFinding(finding),
).length;
const runtimeWarning = (activeFindings ?? []).filter(
(finding) =>
finding.status === 'active' &&
finding.severity === 'warning' &&
isPatrolRuntimeFinding(finding),
).length;
const infrastructureCritical = (activeFindings ?? []).filter(
(finding) =>
finding.status === 'active' &&
finding.severity === 'critical' &&
!isPatrolRuntimeFinding(finding),
).length;
const infrastructureWarning = (activeFindings ?? []).filter(
(finding) =>
finding.status === 'active' &&
finding.severity === 'warning' &&
!isPatrolRuntimeFinding(finding),
).length;
return {
runtimeCritical,
runtimeWarning,
infrastructureCritical,
infrastructureWarning,
runtimeTotal: runtimeCritical + runtimeWarning,
infrastructureTotal: infrastructureCritical + infrastructureWarning,
};
}
export function getPatrolSummaryMetricState(args: {
activeFindings?: PatrolAssessmentFinding[];
fixedCount?: number;
}): PatrolSummaryMetricState {
const classified = classifyActiveFindings(args.activeFindings);
const hasRuntimeIssues = classified.runtimeTotal > 0;
return {
primaryLabel: hasRuntimeIssues ? 'Infrastructure findings' : 'Active findings',
primaryValue: classified.infrastructureTotal,
primarySeverity: classified.infrastructureCritical > 0 ? 'critical' : 'warning',
secondaryLabel: hasRuntimeIssues ? 'Runtime issues' : 'Warnings',
secondaryValue: hasRuntimeIssues ? classified.runtimeTotal : classified.infrastructureWarning,
secondarySeverity: classified.runtimeCritical > 0 ? 'critical' : 'warning',
criticalLabel: 'Critical',
criticalValue: classified.infrastructureCritical + classified.runtimeCritical,
fixedLabel: 'Fixed',
fixedValue: args.fixedCount ?? 0,
};
}
export function getPatrolScoreChipLabel(args: {
overallHealth?: IntelligenceHealthScore;
activeFindings?: PatrolAssessmentFinding[];
}): string {
const classified = classifyActiveFindings(args.activeFindings);
const hasCoverageGap = Boolean(
args.overallHealth?.factors.some((factor) => factor.category === 'coverage'),
);
if (hasCoverageGap || (classified.infrastructureTotal === 0 && classified.runtimeTotal > 0)) {
return 'Assessment';
}
return 'Health';
}
function formatIssueLabel(count: number, label: string): string {
return `${count} ${label} issue${count === 1 ? '' : 's'}`;
}
export function getPatrolCompactAssessmentLabel(input: PatrolCompactAssessmentLabelInput): string {
const classified = classifyActiveFindings(input.activeFindings);
const parts: string[] = [];
if (classified.infrastructureCritical > 0) {
parts.push(formatIssueLabel(classified.infrastructureCritical, 'critical'));
}
if (classified.infrastructureWarning > 0) {
parts.push(formatIssueLabel(classified.infrastructureWarning, 'warning'));
}
if (classified.runtimeTotal > 0) {
parts.push(
`${classified.runtimeTotal} Patrol runtime issue${classified.runtimeTotal === 1 ? '' : 's'}`,
);
}
if (parts.length === 0 && (input.totalActive ?? 0) > 0) {
parts.push(formatIssueLabel(input.totalActive ?? 0, 'active'));
}
if (parts.length === 0) {
parts.push(input.assessmentLabel);
}
const historicalRegressionCount = input.historicalRegressionCount ?? 0;
if (historicalRegressionCount > 0) {
parts.push(
`${historicalRegressionCount} past regression${historicalRegressionCount === 1 ? '' : 's'}`,
);
}
if (input.includeHealthScore !== false && input.overallHealth) {
parts.push(`health score ${Math.round(input.overallHealth.score)}/100`);
}
return parts.join(' · ');
}
export function getPatrolAssessmentAction(args: {
activeFindings?: PatrolAssessmentFinding[];
}): PatrolAssessmentAction | undefined {
const classified = classifyActiveFindings(args.activeFindings);
if (classified.infrastructureTotal === 0 && classified.runtimeTotal > 0) {
return getPatrolProviderSettingsAction();
}
return undefined;
}
function joinAssessmentParts(parts: string[]): string {
if (parts.length <= 1) return parts[0] ?? '';
if (parts.length === 2) return `${parts[0]} and ${parts[1]}`;
return `${parts.slice(0, -1).join(', ')}, and ${parts.at(-1)}`;
}
function predictionReadsAsAllClear(prediction: string | undefined): boolean {
const normalized = String(prediction || '')
.trim()
.toLowerCase();
if (!normalized) return false;
return (
normalized.includes('healthy with no significant issue') ||
normalized.includes('no significant issues detected') ||
normalized.includes('no active issues') ||
normalized.includes('no issues detected') ||
normalized.includes('all clear')
);
}
function predictionReadsAsCoverageGap(prediction: string | undefined): boolean {
const normalized = String(prediction || '')
.trim()
.toLowerCase();
if (!normalized) return false;
return (
normalized.includes('coverage is incomplete') ||
normalized.includes('coverage incomplete') ||
normalized.includes('not fully verified') ||
normalized.includes('limited to scoped runs') ||
normalized.includes('limited to targeted') ||
normalized.includes('runs encountered errors') ||
normalized.includes('ended with errors') ||
normalized.includes('current issue list') ||
normalized.includes('current full issue list') ||
normalized.includes('current health may be incomplete') ||
normalized.includes('summary may be incomplete')
);
}
function hasSuccessfulFullCoverageRun(runs: PatrolRunRecord[] | undefined): boolean {
const recentFullRun = (runs ?? [])
.filter((run) => isCompletedPatrolRun(run))
.find((run) => isFullPatrolRun(run));
return Boolean(
recentFullRun && !hasRunErrors(recentFullRun) && (recentFullRun.resources_checked || 0) > 0,
);
}
function getFindingAssessmentDescription(args: {
criticalFindings?: number;
warningFindings?: number;
overallHealth?: IntelligenceHealthScore;
activeFindings?: PatrolAssessmentFinding[];
runs?: PatrolRunRecord[];
}): string {
const criticalFindings = args.criticalFindings ?? 0;
const warningFindings = args.warningFindings ?? 0;
const hasVerifiedFullCoverage = hasSuccessfulFullCoverageRun(args.runs);
const hasCoverageGapFactor = Boolean(
args.overallHealth?.factors.some((factor) => factor.category === 'coverage'),
);
const hasCoverageGap = hasCoverageGapFactor && !hasVerifiedFullCoverage;
const classified = classifyActiveFindings(args.activeFindings);
const activeRuntimeFindings = (args.activeFindings ?? []).filter(
(finding) => finding.status === 'active' && isPatrolRuntimeFinding(finding),
);
const shouldUsePrediction = (prediction: string | undefined): prediction is string => {
if (!prediction) return false;
if (predictionReadsAsAllClear(prediction)) return false;
return !(hasVerifiedFullCoverage && predictionReadsAsCoverageGap(prediction));
};
if (classified.infrastructureTotal === 0 && classified.runtimeTotal === 1) {
const runtimeFinding = activeRuntimeFindings[0];
const runtimeLabel = getRuntimeFindingSummaryLabel(runtimeFinding);
const runtimeSummary =
runtimeFinding?.severity === 'critical'
? `Patrol is currently blocked by a critical runtime issue: ${runtimeLabel}.`
: `Patrol has an active runtime issue: ${runtimeLabel}.`;
if (hasCoverageGap) {
return `${runtimeSummary} Recent coverage is incomplete. Run Patrol to check everything.`;
}
const prediction = args.overallHealth?.prediction?.trim();
if (shouldUsePrediction(prediction)) {
return prediction;
}
return `${runtimeSummary} Review the Patrol runtime issue for more detail.`;
}
const findingSummaryParts: string[] = [];
if (classified.infrastructureCritical > 0) {
findingSummaryParts.push(
`${formatFindingCount(classified.infrastructureCritical, 'critical')} in your infrastructure`,
);
}
if (classified.infrastructureWarning > 0) {
findingSummaryParts.push(
`${formatFindingCount(classified.infrastructureWarning, 'warning')} in your infrastructure`,
);
}
if (classified.runtimeCritical > 0) {
findingSummaryParts.push(formatRuntimeIssueCount(classified.runtimeCritical, 'critical'));
}
if (classified.runtimeWarning > 0) {
findingSummaryParts.push(formatRuntimeIssueCount(classified.runtimeWarning, 'warning'));
}
const findingSummary =
findingSummaryParts.length > 0
? `Patrol surfaced ${joinAssessmentParts(findingSummaryParts)}.`
: criticalFindings > 0
? `Patrol surfaced ${formatFindingCount(criticalFindings, 'critical')}.`
: `Patrol surfaced ${formatFindingCount(warningFindings, 'warning')}.`;
if (hasCoverageGap) {
return `${findingSummary} Recent coverage is incomplete. Run Patrol to check everything.`;
}
const prediction = args.overallHealth?.prediction?.trim();
if (shouldUsePrediction(prediction)) {
return prediction;
}
return `${findingSummary} Review the active findings for more detail.`;
}
function normalizeRunType(type: string | undefined): string {
return String(type || '')
.trim()
@@ -517,121 +90,6 @@ function formatRecencyResourcesCheckedLabel(run: PatrolRunRecord): string | unde
return `checked ${resourcesChecked} resource${resourcesChecked === 1 ? '' : 's'}`;
}
export function getPatrolAssessmentPresentation(args: {
overallHealth?: IntelligenceHealthScore;
runtimeState?: PatrolRuntimeState;
blockedReason?: string;
criticalFindings?: number;
warningFindings?: number;
activeFindings?: PatrolAssessmentFinding[];
runs?: PatrolRunRecord[];
}): PatrolAssessmentPresentation {
const classified = classifyActiveFindings(args.activeFindings);
const hasVerifiedFullCoverage = hasSuccessfulFullCoverageRun(args.runs);
if (args.runtimeState === 'running') {
return {
title: 'Patrol running',
description: 'Patrol is checking your infrastructure now.',
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Patrol is checking now',
tone: 'info',
};
}
if (
args.runtimeState === 'blocked' ||
args.runtimeState === 'disabled' ||
args.runtimeState === 'unavailable'
) {
const runtime = getPatrolRuntimePresentation(args.runtimeState, args.blockedReason);
return {
title: runtime.label,
description: runtime.description,
eyebrow: runtime.title,
compactLabel: runtime.label,
tone: runtime.tone,
};
}
if ((args.criticalFindings ?? 0) > 0) {
if (classified.infrastructureTotal === 0 && classified.runtimeTotal > 0) {
return {
title: 'Critical Patrol runtime issue',
description: getFindingAssessmentDescription(args),
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Patrol runtime issue',
tone: 'error',
};
}
return {
title: 'Critical issues detected',
description: getFindingAssessmentDescription(args),
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Issues detected',
tone: 'error',
};
}
if ((args.warningFindings ?? 0) > 0) {
if (classified.infrastructureTotal === 0 && classified.runtimeTotal > 0) {
return {
title: 'Patrol runtime issue',
description: getFindingAssessmentDescription(args),
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Patrol runtime issue',
tone: 'warning',
};
}
return {
title: 'Issues detected',
description: getFindingAssessmentDescription(args),
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Issues detected',
tone: 'warning',
};
}
if (
args.overallHealth?.factors.some((factor) => factor.category === 'coverage') &&
!hasVerifiedFullCoverage
) {
return {
title: 'Coverage incomplete',
description: getCoverageDescription(args.overallHealth),
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Coverage incomplete',
tone: getHealthSummaryTone(args.overallHealth),
};
}
if (args.overallHealth && args.overallHealth.grade !== 'A') {
const prediction = args.overallHealth.prediction?.trim();
return {
title: 'Health requires attention',
description:
prediction && !(hasVerifiedFullCoverage && predictionReadsAsCoverageGap(prediction))
? prediction
: 'Patrol still needs attention.',
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: 'Health requires attention',
tone: getHealthSummaryTone(args.overallHealth),
};
}
return {
title: 'No active issues detected',
description:
args.overallHealth?.prediction?.trim() ||
'Infrastructure is healthy with no significant issues detected.',
eyebrow: PATROL_STATUS_EYEBROW,
compactLabel: PATROL_NO_ISSUES_LABEL,
tone: 'success',
};
}
export function getPatrolVerificationPresentation(args: {
runs?: PatrolRunRecord[];
runtimeState?: PatrolRuntimeState;
@@ -780,15 +238,3 @@ export function getPatrolRecencyPresentation(args: {
label: 'Last activity',
};
}
export function getPatrolNoIssuesPresentation(args: {
overallHealth?: IntelligenceHealthScore;
runtimeState?: PatrolRuntimeState;
blockedReason?: string;
}): PatrolNoIssuesPresentation {
const assessment = getPatrolAssessmentPresentation(args);
return {
label: assessment.compactLabel,
tone: assessment.tone,
};
}
@@ -124,8 +124,9 @@ const mergeRecord = <T extends JsonRecord>(incoming?: T, existing?: T): T | unde
};
// Native VM facets carry a runtime state and an explicit guest-agent outcome.
// Their optional read reason, lock and false expected flag are omitempty on
// wire. A new outcome supersedes those fields, not unrelated richer metadata.
// Their optional read reason, lock, false expected flag and guest filesystems
// are omitempty on wire. A new outcome supersedes those fields, not unrelated
// richer metadata; withdrawn filesystems must not hide a linked agent's.
// Partial/legacy facets without that evidence must retain the prior facts.
const mergeProxmoxFacet = <T extends JsonRecord>(incoming?: T, existing?: T): T | undefined => {
const merged = mergeRecord(incoming, existing);
@@ -142,12 +143,30 @@ const mergeProxmoxFacet = <T extends JsonRecord>(incoming?: T, existing?: T): T
)
return merged;
const next = { ...merged };
for (const key of ['diskStatusReason', 'guestAgentExpected', 'lock']) {
for (const key of ['diskStatusReason', 'guestAgentExpected', 'lock', 'disks']) {
if (!Object.prototype.hasOwnProperty.call(incoming, key)) delete next[key];
}
return next;
};
// Native agent facets always carry `stale`; facets synthesized from legacy
// platform data never do. Their filesystem list is omitempty on wire, so a
// native report without `disks` has withdrawn them, while a partial or legacy
// facet keeps the prior list.
const mergeAgentFacet = <T extends JsonRecord>(incoming?: T, existing?: T): T | undefined => {
const merged = mergeRecord(incoming, existing);
if (
!incoming ||
!merged ||
typeof incoming.stale !== 'boolean' ||
Object.prototype.hasOwnProperty.call(incoming, 'disks')
)
return merged;
const next = { ...merged };
delete next.disks;
return next;
};
const mergePlatformData = (
incomingValue: Resource['platformData'],
existingValue: Resource['platformData'],
@@ -180,7 +199,9 @@ const mergePlatformData = (
const nested =
key === 'proxmox'
? mergeProxmoxFacet(asRecord(incoming[key]), asRecord(existing[key]))
: mergeRecord(asRecord(incoming[key]), asRecord(existing[key]));
: key === 'agent'
? mergeAgentFacet(asRecord(incoming[key]), asRecord(existing[key]))
: mergeRecord(asRecord(incoming[key]), asRecord(existing[key]));
if (key === 'docker') {
if (hasDockerFacetEvidence(nested)) {
merged[key] = nested;
@@ -895,7 +916,9 @@ const mergeCanonicalSourceFacet = <T extends JsonRecord>(
shouldKeepSourceFacet(incomingSources, ...sourceCandidates)
? sourceCandidates.includes('proxmox-pve')
? mergeProxmoxFacet(incomingFacet, existingFacet)
: mergeRecord(incomingFacet, existingFacet)
: sourceCandidates.includes('agent')
? mergeAgentFacet(incomingFacet, existingFacet)
: mergeRecord(incomingFacet, existingFacet)
: incomingFacet;
// A missing field alone can be a partial snapshot. Explicit unavailable raw
@@ -146,3 +146,61 @@ export function getWorkloadGuestDiskStatusMessage(reason?: string): string {
return carriedForward ? `Using last known disk stats. ${message}` : message;
}
export interface WorkloadGuestDiskReadSource {
status?: string;
/** Proxmox's reason for its own guest filesystem read. */
diskStatusReason?: string;
/** `disks` came from the linked Pulse agent because Proxmox reported none. */
disksFromAgent?: boolean;
agentStale?: boolean;
}
export type WorkloadGuestDiskRead =
| { state: 'current'; message: null; needsAction: false }
| {
state: 'last-known' | 'unavailable';
/** Why the reading is not current. */
message: string;
/** A broken read the user can fix, rather than a setup gap. */
needsAction: boolean;
};
// Disk-read reasons that point at something broken the user can fix.
const DISK_READ_ACTIONABLE_REASONS = new Set(['permission-denied', 'agent-error']);
// One freshness rule for a guest's filesystem rows and aggregate disk metric,
// shared by the workload table cell, the drawer and its History. Proxmox
// records no guest disk usage without guest filesystems, so when the linked
// Pulse agent's filesystems fill in, the registry keeps the agent's disk
// metric too and the reading follows that agent, not the Proxmox reason.
export function getWorkloadGuestDiskRead(
guest: WorkloadGuestDiskReadSource,
isVM: boolean,
): WorkloadGuestDiskRead {
const reason = isVM ? guest.diskStatusReason : undefined;
if (guest.disksFromAgent) {
if (guest.status === 'stopped' || reason === 'vm-stopped') {
return {
state: 'unavailable',
message: isVM
? getWorkloadGuestDiskStatusMessage('vm-stopped')
: 'Filesystem usage is unavailable.',
needsAction: false,
};
}
return guest.agentStale
? {
state: 'last-known',
message: 'Using last known disk stats. The Pulse Agent in this guest stopped reporting.',
needsAction: false,
}
: { state: 'current', message: null, needsAction: false };
}
if (!reason) return { state: 'current', message: null, needsAction: false };
return {
state: reason.startsWith('prev-') ? 'last-known' : 'unavailable',
message: getWorkloadGuestDiskStatusMessage(reason),
needsAction: DISK_READ_ACTIONABLE_REASONS.has(reason.replace(/^prev-/, '')),
};
}
+23 -4
View File
@@ -19,6 +19,8 @@ package alerts
// 4. the per-resource override, through the identity-aware lookup for
// the kind (guest stable keys, storage aliases, canonical registry
// translation)
// 5. a guest's pulse-relaxed tag, which lifts its CPU/memory/disk
// triggers to a 95/92/95 floor
// alertPolicyQuery identifies one resource asking for its effective
// alert policy.
@@ -28,13 +30,20 @@ type alertPolicyQuery struct {
TypeKey string
ResourceID string
// Guest carries the live guest model when the query is for a guest
// kind: filter-scoped custom rules and the clustered override
// identity need it. Nil is valid — filter rules then never match and
// override lookup falls back to the raw resource ID.
// kind: filter-scoped custom rules, the clustered override identity and
// the pulse-relaxed tag need it. Nil is valid — filter rules then never
// match, override lookup falls back to the raw resource ID, and no tag
// relaxes the thresholds.
Guest any
// StorageAliases are the storage resource's alias IDs, honored by the
// storage override lookup.
StorageAliases []string
// DiskType is a physical disk's type ("nvme", "sas", "sata"), which
// picks its disk temperature threshold. DiskTypeUnknown marks a disk
// whose type was never recorded, such as an alert persisted before it
// carried one; it is judged against the lowest per-type trigger.
DiskType string
DiskTypeUnknown bool
}
// EffectiveAlertPolicy is the resolved policy for one resource.
@@ -47,7 +56,8 @@ type EffectiveAlertPolicy struct {
// connectivity opt-outs are Thresholds.DisableConnectivity.
OfflineDisabled bool
// Thresholds is the folded threshold block: type defaults, then
// custom rules, then the per-resource override.
// custom rules, then the per-resource override, then a guest's
// pulse-relaxed floor.
Thresholds ThresholdConfig
}
@@ -116,10 +126,19 @@ func (m *Manager) effectiveAlertPolicyNoLock(q alertPolicyQuery) EffectiveAlertP
policy.AllDisabled, policy.OfflineDisabled = m.alertPolicyTypeSwitchesNoLock(q.TypeKey)
thresholds := m.defaultThresholdsForResourceType(q.TypeKey)
if q.TypeKey == "truenas-disk" {
thresholds.Temperature = m.trueNASDiskTemperatureDefaultNoLock(q.DiskType, q.DiskTypeUnknown)
}
switch {
case isGuestThresholdResourceType(q.TypeKey):
thresholds = m.customRuleThresholdsNoLock(thresholds, q.Guest)
thresholds = m.resolveGuestThresholdOverride(thresholds, q.Guest, q.ResourceID)
// Polling and config-save reevaluation both resolve through here,
// so a relaxed guest's alert is judged by one threshold. Applied
// only by CheckGuest, a save re-judged it by the unrelaxed one.
if guest, ok := extractGuestSnapshot(q.Guest); ok && parsePulseTags(guest.Tags).Relaxed {
thresholds = applyRelaxedGuestThresholds(thresholds)
}
case q.TypeKey == "storage":
thresholds = m.resolveStorageThresholdOverride(thresholds, q.ResourceID, q.StorageAliases)
default:
+492 -1
View File
@@ -784,6 +784,137 @@ func TestPulseRelaxedThresholdsIncreaseCpuTrigger(t *testing.T) {
}
}
// pulse-relaxed lifted an Off trigger to its floor like any low trigger, so a
// relaxed guest with disk alerts off raised one at 96%. A config save then
// re-judged it by the Off threshold and resolved it, and the next poll raised
// it again. Relaxing a guest must not raise an alert its config turned off.
func TestPulseRelaxedKeepsOffGuestThresholdOff(t *testing.T) {
m := newTestManager(t)
m.mu.Lock()
m.config.Enabled = true
m.config.TimeThresholds = map[string]int{}
m.config.GuestDefaults = ThresholdConfig{
CPU: &HysteresisThreshold{Trigger: 80, Clear: 75},
Memory: &HysteresisThreshold{Trigger: 85, Clear: 80},
Disk: &HysteresisThreshold{Trigger: 0, Clear: 0},
}
m.mu.Unlock()
vm := models.VM{
ID: "inst/qemu/105",
Name: "relaxed-vm",
Node: "node1",
Instance: "inst",
Status: "running",
Memory: models.Memory{Usage: 40},
Disk: models.Disk{Usage: 96},
Tags: []string{"pulse-relaxed"},
}
m.CheckGuest(vm, "inst")
if testHasActiveAlert(t, m, canonicalMetricStateID(vm.ID, "disk")) {
t.Fatalf("expected no disk alert on a relaxed guest whose disk threshold is off")
}
}
// A config save re-judges a relaxed guest's alert by the thresholds CheckGuest
// raised it under. Reevaluation rebuilt the guest from alert metadata without
// its tags, so an unset memory threshold, which pulse-relaxed sets to 92, read
// as no threshold there: the save resolved the alert with a recovery
// notification and the next poll raised it again. The tags are read back in
// the shape a restart restores them.
func TestConfigSaveJudgesRelaxedGuestAlertByRelaxedThresholds(t *testing.T) {
m := newTestManager(t)
m.mu.Lock()
m.config.Enabled = true
m.config.TimeThresholds = map[string]int{}
m.config.GuestDefaults = ThresholdConfig{}
m.mu.Unlock()
vm := models.VM{
ID: "inst/qemu/106",
Name: "relaxed-vm",
Node: "node1",
Instance: "inst",
Status: "running",
Memory: models.Memory{Usage: 95},
Tags: []string{"pulse-relaxed"},
}
memoryAlertID := canonicalMetricStateID(vm.ID, "memory")
m.CheckGuest(vm, "inst")
alert := testRequireActiveAlert(t, m, memoryAlertID)
if alert.Threshold != 92 {
t.Fatalf("memory alert threshold = %v, want the relaxed 92", alert.Threshold)
}
m.mu.Lock()
raw, err := json.Marshal(alert.Metadata)
if err != nil {
m.mu.Unlock()
t.Fatalf("marshal metadata: %v", err)
}
var restored map[string]interface{}
if err := json.Unmarshal(raw, &restored); err != nil {
m.mu.Unlock()
t.Fatalf("unmarshal metadata: %v", err)
}
alert.Metadata = restored
m.mu.Unlock()
if _, ok := restored["tags"].([]interface{}); !ok {
t.Fatalf("restored tags = %T, want []interface{}", restored["tags"])
}
m.UpdateConfig(m.GetConfig())
if !testHasActiveAlert(t, m, memoryAlertID) {
t.Fatalf("expected a config save to keep a relaxed guest's memory alert over its relaxed trigger")
}
}
// pulse-relaxed lifts a guest's disk threshold, but a filesystem's own
// override still decides that filesystem, while polling and on a config save.
// Judged by the guest's relaxed floor on save, a filesystem alert over its
// lower override resolved and the next poll raised it again.
func TestConfigSaveKeepsRelaxedGuestFilesystemOverride(t *testing.T) {
m := newTestManager(t)
guestID := BuildGuestKey("pve1", "node1", 108)
m.mu.Lock()
m.config.Enabled = true
m.config.TimeThresholds = map[string]int{}
m.config.GuestDefaults = ThresholdConfig{
Disk: &HysteresisThreshold{Trigger: 90, Clear: 85},
}
m.config.Overrides = map[string]ThresholdConfig{
"guest-disk:guest:pve1:108/disk:boot-dev-vda1": {
Disk: &HysteresisThreshold{Trigger: 85, Clear: 80},
},
}
m.mu.Unlock()
vm := models.VM{
ID: guestID,
VMID: 108,
Name: "relaxed-fs",
Node: "node1",
Instance: "pve1",
Status: "running",
Disks: []models.Disk{
{Mountpoint: "/boot", Device: "/dev/vda1", Usage: 92, Total: 100, Used: 92, Free: 8},
},
Tags: []string{"pulse-relaxed"},
}
bootAlertID := canonicalMetricStateID(guestID+"-disk-boot-dev-vda1", "disk")
m.CheckGuest(vm, "pve1")
if !testHasActiveAlert(t, m, bootAlertID) {
t.Fatalf("expected a /boot alert over its 85%% override on a relaxed guest")
}
m.UpdateConfig(m.GetConfig())
if !testHasActiveAlert(t, m, bootAlertID) {
t.Fatalf("expected a config save to keep a relaxed guest's /boot alert over its own 85%% override")
}
}
func TestClearAlertMarksResolutionAndReturnsStatus(t *testing.T) {
m := newTestManager(t)
m.ClearActiveAlerts()
@@ -2756,6 +2887,101 @@ func TestCheckBackupsIgnoresVMIDs(t *testing.T) {
}
}
// Host and guest evaluation used to fill in the Clear of a disk override's
// threshold after dropping m.mu, writing through the live config's pointer
// while GetConfig cloned it under the read lock, and Docker evaluation read
// m.config.DockerDefaults with no lock while UpdateConfig replaced m.config.
// Evaluation now copies what it needs under m.mu. Run with -race.
func TestAlertEvaluationLeavesTheLiveConfigToTheLock(t *testing.T) {
originalLogLevel := zerolog.GlobalLevel()
zerolog.SetGlobalLevel(zerolog.Disabled)
t.Cleanup(func() {
zerolog.SetGlobalLevel(originalLogLevel)
})
m := newTestManager(t)
host := models.Host{
ID: "eval-lock",
Hostname: "eval-lock",
Status: "online",
Disks: []models.Disk{{Mountpoint: "/", Device: "/dev/sda1", Usage: 50, Total: 100, Used: 50, Free: 50}},
}
hostDiskOverrideID, _ := hostDiskResourceID(host, host.Disks[0])
vm := models.VM{
ID: BuildGuestKey("pve1", "node1", 101),
VMID: 101,
Name: "eval-lock",
Node: "node1",
Instance: "pve1",
Status: "running",
Disks: []models.Disk{{Mountpoint: "/boot", Device: "/dev/vda1", Usage: 50, Total: 100, Used: 50, Free: 50}},
}
guestDiskOverrideID := "guest-disk:guest:pve1:101/disk:boot-dev-vda1"
dockerHost := models.DockerHost{
ID: "eval-lock-docker",
Hostname: "eval-lock-docker",
DisplayName: "eval-lock-docker",
Containers: []models.DockerContainer{{
ID: "abcdef123456",
Name: "web",
State: "running",
CPUPercent: 10,
MemoryPercent: 10,
MemoryUsage: 100 << 20,
MemoryLimit: 1 << 30,
}},
}
// A disk override saved with only a trigger leaves Clear for evaluation
// to default.
withTriggerOnlyDiskOverrides := func(cfg AlertConfig) AlertConfig {
cfg.Enabled = true
cfg.ActivationState = ActivationActive
cfg.Overrides = map[string]ThresholdConfig{
hostDiskOverrideID: {Disk: &HysteresisThreshold{Trigger: 90}},
guestDiskOverrideID: {Disk: &HysteresisThreshold{Trigger: 90}},
}
return cfg
}
m.UpdateConfig(withTriggerOnlyDiskOverrides(m.GetConfig()))
m.CheckHost(host)
m.CheckGuest(vm, "pve1")
live := m.GetConfig()
for _, id := range []string{hostDiskOverrideID, guestDiskOverrideID} {
if got := live.Overrides[id].Disk.Clear; got != 0 {
t.Fatalf("evaluating %s wrote clear %v into the live override", id, got)
}
}
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
for i := 0; i < 100; i++ {
// Resetting the overrides gives evaluation a Clear to fill again.
m.UpdateConfig(withTriggerOnlyDiskOverrides(m.GetConfig()))
m.UpdateConfig(m.GetConfig())
}
}()
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
_ = m.GetConfig()
}
}()
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
m.CheckHost(host)
m.CheckGuest(vm, "pve1")
m.CheckDockerHost(dockerHost)
}
}()
wg.Wait()
}
func TestCheckDockerHostIgnoresContainersByPrefix(t *testing.T) {
m := newTestManager(t)
@@ -6254,6 +6480,22 @@ func TestApplyRelaxedGuestThresholds(t *testing.T) {
}
})
t.Run("off thresholds stay off", func(t *testing.T) {
cfg := ThresholdConfig{
CPU: &HysteresisThreshold{Trigger: 0, Clear: 0},
Memory: &HysteresisThreshold{Trigger: -1, Clear: -1},
Disk: &HysteresisThreshold{Trigger: 0, Clear: 0},
}
result := applyRelaxedGuestThresholds(cfg)
for name, th := range map[string]*HysteresisThreshold{"CPU": result.CPU, "Memory": result.Memory, "Disk": result.Disk} {
if th == nil || th.Trigger > 0 {
t.Errorf("%s = %+v, want it to stay off", name, th)
}
}
})
t.Run("original config unchanged", func(t *testing.T) {
// t.Parallel()
original := ThresholdConfig{
@@ -12300,6 +12542,65 @@ func TestCheckEscalations(t *testing.T) {
})
}
// A config save re-judges open TrueNAS disk temperature alerts against the
// threshold the next evaluation applies to each disk's type.
func TestTrueNASDiskTemperatureAlertsReevaluatePerDiskType(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
configureUnifiedEvalManager(t, m, cfg)
nvme := trueNASTemperatureDisk("nvme0n1", "nvme", 72)
sata := trueNASTemperatureDisk("sda", "sata", 57)
checkTrueNASTemperatureDisk(t, m, nvme)
checkTrueNASTemperatureDisk(t, m, sata)
for _, disk := range []unifiedresources.Resource{nvme, sata} {
if _, firing := trueNASDiskTemperatureAlert(t, m, disk); !firing {
t.Fatalf("%s did not fire before the save: %v", disk.ID, alertKeys(m))
}
}
// Raising only the NVMe trigger resolves the NVMe alert and leaves the
// SATA alert, still over its own trigger, open.
m.mu.Lock()
m.config.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 75, Clear: 70}
m.reevaluateActiveAlertsLocked()
m.mu.Unlock()
if _, firing := trueNASDiskTemperatureAlert(t, m, nvme); firing {
t.Fatalf("NVMe alert at 72C stayed open under a 75C NVMe trigger")
}
if _, firing := trueNASDiskTemperatureAlert(t, m, sata); !firing {
t.Fatalf("SATA alert at 57C resolved under its unchanged 55C trigger")
}
}
// An alert raised before it recorded its disk type is held to the lowest
// per-type trigger, so a save never resolves an alert the next evaluation
// raises again.
func TestTrueNASDiskTemperatureAlertWithoutDiskTypeHeldAtLowestTrigger(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 70, Clear: 65}
cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 60, Clear: 55}}
configureUnifiedEvalManager(t, m, cfg)
nvme := trueNASTemperatureDisk("nvme0n1", "nvme", 66)
checkTrueNASTemperatureDisk(t, m, nvme)
alert, firing := trueNASDiskTemperatureAlert(t, m, nvme)
if !firing {
t.Fatalf("NVMe at 66C did not fire under a 60C NVMe trigger: %v", alertKeys(m))
}
m.mu.Lock()
delete(alert.Metadata, "diskType")
m.reevaluateActiveAlertsLocked()
m.mu.Unlock()
if _, firing := trueNASDiskTemperatureAlert(t, m, nvme); !firing {
t.Fatalf("an alert without a disk type was resolved by the 70C agent default although its NVMe trigger is 60C")
}
}
func TestCleanupAlertsForNodes(t *testing.T) {
// t.Parallel()
@@ -22190,14 +22491,20 @@ func TestStaleCleanupKeepsThresholdAlertStillBeingEvaluated(t *testing.T) {
// Resource history owns each PVE disk alert row by the hardware identity the
// alert records, and reads by the path reference find those rows by alert
// identifier. Both must match what CheckDiskHealth produces.
// identifier. Both must match what CheckDiskHealth and
// CheckProxmoxDiskTemperature produce.
func TestCheckDiskHealthAlertsCarryHistoryOwnershipIdentity(t *testing.T) {
m := newTestManager(t)
m.ClearActiveAlerts()
m.mu.Lock()
m.config.TimeThresholds = map[string]int{}
m.config.MetricTimeThresholds = nil
m.mu.Unlock()
disk := proxmox.Disk{DevPath: "/dev/sdb", Model: "Crucial MX500", Serial: "2117E59AB123", WWN: "0x5002538f12345678",
Type: "ssd", Health: "FAILED", Wearout: 3}
for i := 0; i < 3; i++ {
m.CheckDiskHealth("lab", "pve1", disk)
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: disk, Celsius: 80})
}
ref := unifiedresources.ProxmoxPhysicalDiskAlertResourceID("lab", "pve1", disk.DevPath)
@@ -22218,3 +22525,187 @@ func TestCheckDiskHealthAlertsCarryHistoryOwnershipIdentity(t *testing.T) {
}
}
}
func proxmoxTemperatureDisk(devPath, diskType string) proxmox.Disk {
return proxmox.Disk{DevPath: devPath, Model: "Test Disk", Serial: "SER-" + devPath, Type: diskType, Health: "PASSED"}
}
func proxmoxDiskTemperatureAlert(t *testing.T, m *Manager, devPath string) (*Alert, bool) {
t.Helper()
ref := unifiedresources.ProxmoxPhysicalDiskAlertResourceID("lab", "pve1", devPath)
return testLookupActiveAlert(t, m, canonicalMetricStateID(ref, proxmoxDiskTemperatureMetric))
}
func proxmoxDiskTemperatureResolution(m *Manager, devPath string) (*models.AlertResolution, bool) {
ref := unifiedresources.ProxmoxPhysicalDiskAlertResourceID("lab", "pve1", devPath)
id := canonicalMetricStateID(ref, proxmoxDiskTemperatureMetric)
for _, resolved := range m.GetRecentlyResolved() {
if resolved.ID == id {
return resolved.Resolution, true
}
}
return nil, false
}
// TestProxmoxDiskTemperatureAlertsFollowDiskTemperaturePolicy pins Proxmox disk
// temperature alerts to the disk temperature policy that judges an agent's
// disks, TrueNAS disks and the Physical Disks verdict.
func TestProxmoxDiskTemperatureAlertsFollowDiskTemperaturePolicy(t *testing.T) {
m := newTestManager(t)
configureUnifiedEvalManager(t, m, unifiedEvalBaseConfig())
check := func(devPath, diskType string, celsius int) {
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: proxmoxTemperatureDisk(devPath, diskType), Celsius: celsius})
}
check("/dev/nvme0n1", "nvme", 63)
check("/dev/nvme1n1", "nvme", 72)
check("/dev/sda", "sata", 56)
check("/dev/sdb", "sas", 63)
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/nvme0n1"); firing {
t.Fatalf("NVMe at 63C fired under its 70C trigger: %v", alertKeys(m))
}
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/sdb"); firing {
t.Fatalf("SAS at 63C fired under its 65C trigger: %v", alertKeys(m))
}
for devPath, trigger := range map[string]float64{"/dev/nvme1n1": 70, "/dev/sda": 55} {
alert, firing := proxmoxDiskTemperatureAlert(t, m, devPath)
if !firing {
t.Fatalf("%s did not fire: %v", devPath, alertKeys(m))
}
if alert.Threshold != trigger {
t.Fatalf("%s trigger = %v, want %v", devPath, alert.Threshold, trigger)
}
if alert.Type != proxmoxDiskTemperatureMetric || alert.Metadata["resourceType"] != proxmoxDiskResourceType {
t.Fatalf("%s alert type %q, resourceType %v", devPath, alert.Type, alert.Metadata["resourceType"])
}
if alert.Metadata["diskType"] == nil || alert.Metadata[unifiedresources.MetadataDiskSerial] != "SER-"+devPath {
t.Fatalf("%s metadata = %v", devPath, alert.Metadata)
}
}
if alert, _ := proxmoxDiskTemperatureAlert(t, m, "/dev/sda"); alert.Message != "Disk temperature at 56.0°C" {
t.Fatalf("message = %q", alert.Message)
}
// A raised NVMe trigger moves NVMe disks with it.
m.mu.Lock()
m.config.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 75, Clear: 70}
m.mu.Unlock()
check("/dev/nvme2n1", "nvme", 72)
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/nvme2n1"); firing {
t.Fatalf("NVMe at 72C fired under a 75C trigger")
}
}
// A disk the linked agent reports is the agent's: its open alert closes as
// moved to the agent, and the disk never alerts twice. An excluded disk, or a
// policy the agent Disk Temp default switched off, closes it as a plain clear.
// A disk with no current reading keeps its alert.
func TestProxmoxDiskTemperatureAlertOwnershipAndHolds(t *testing.T) {
m := newTestManager(t)
configureUnifiedEvalManager(t, m, unifiedEvalBaseConfig())
disk := proxmoxTemperatureDisk("/dev/sda", "sata")
read := func(reading ProxmoxDiskTemperatureReading) {
reading.Disk = disk
m.CheckProxmoxDiskTemperature("lab", "pve1", reading)
}
firing := func() bool {
_, ok := proxmoxDiskTemperatureAlert(t, m, "/dev/sda")
return ok
}
read(ProxmoxDiskTemperatureReading{Celsius: 60})
if !firing() {
t.Fatalf("SATA at 60C did not fire: %v", alertKeys(m))
}
read(ProxmoxDiskTemperatureReading{Celsius: 0})
if !firing() {
t.Fatalf("a poll without a current reading resolved the alert")
}
read(ProxmoxDiskTemperatureReading{Celsius: 60, AgentOwned: true})
if firing() {
t.Fatalf("an agent-owned disk kept its Proxmox temperature alert")
}
if resolution, ok := proxmoxDiskTemperatureResolution(m, "/dev/sda"); !ok || resolution == nil || resolution.Reason != string(AlertResolutionMovedToAgent) {
t.Fatalf("handover resolution = %+v (resolved %v), want moved_to_agent", resolution, ok)
}
read(ProxmoxDiskTemperatureReading{Celsius: 80, AgentOwned: true})
if firing() {
t.Fatalf("an agent-owned disk raised a Proxmox temperature alert")
}
read(ProxmoxDiskTemperatureReading{Celsius: 60})
if !firing() {
t.Fatalf("the disk did not alert again once the agent stopped listing it")
}
read(ProxmoxDiskTemperatureReading{Celsius: 60, Excluded: true})
if firing() {
t.Fatalf("an excluded disk kept its temperature alert")
}
read(ProxmoxDiskTemperatureReading{Celsius: 60})
m.mu.Lock()
m.config.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 0, Clear: 0}
m.mu.Unlock()
read(ProxmoxDiskTemperatureReading{Celsius: 60})
if firing() {
t.Fatalf("a switched-off agent Disk Temp default kept the Proxmox disk alert")
}
}
// A Proxmox disk temperature warning waits out the factory stability window for
// a noisy gauge, as agent and TrueNAS disk temperature warnings do; a critical
// reading bypasses it.
func TestProxmoxDiskTemperatureWarningWaitsOutStabilityWindow(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
m.UpdateConfig(cfg)
if delay := m.GetConfig().TimeThresholds[proxmoxDiskResourceType]; delay != 5 {
t.Fatalf("proxmox-disk delay = %d, want the 5s factory default", delay)
}
m.mu.RLock()
grace := m.getTimeThreshold("lab:pve1:disk:dev-sda", proxmoxDiskResourceType, proxmoxDiskTemperatureMetric)
m.mu.RUnlock()
if grace != defaultNoisyGaugeStabilitySeconds {
t.Fatalf("Proxmox disk temperature delay = %ds, want the %ds stability window", grace, defaultNoisyGaugeStabilitySeconds)
}
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: proxmoxTemperatureDisk("/dev/sda", "sata"), Celsius: 57})
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/sda"); firing {
t.Fatalf("a 57C SATA warning fired before its stability window")
}
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: proxmoxTemperatureDisk("/dev/sdb", "sata"), Celsius: 70})
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/sdb"); !firing {
t.Fatalf("a 70C SATA critical reading waited out the stability window")
}
}
// A config save re-judges an open Proxmox disk temperature alert against its
// disk type's trigger, as the next poll does; one without a recorded disk type
// is held to the lowest per-type trigger.
func TestProxmoxDiskTemperatureAlertsReevaluatePerDiskType(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 70, Clear: 65}
configureUnifiedEvalManager(t, m, cfg)
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: proxmoxTemperatureDisk("/dev/nvme0n1", "nvme"), Celsius: 72})
m.CheckProxmoxDiskTemperature("lab", "pve1", ProxmoxDiskTemperatureReading{Disk: proxmoxTemperatureDisk("/dev/sda", "sata"), Celsius: 57})
sata, ok := proxmoxDiskTemperatureAlert(t, m, "/dev/sda")
if !ok {
t.Fatalf("SATA at 57C did not fire: %v", alertKeys(m))
}
m.mu.Lock()
m.config.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 75, Clear: 70}
delete(sata.Metadata, "diskType")
m.reevaluateActiveAlertsLocked()
m.mu.Unlock()
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/nvme0n1"); firing {
t.Fatalf("NVMe alert at 72C survived a save raising its trigger to 75C")
}
if _, firing := proxmoxDiskTemperatureAlert(t, m, "/dev/sda"); !firing {
t.Fatalf("a SATA alert without a disk type was resolved by the 70C agent default although its trigger is 55C")
}
}
@@ -794,7 +794,6 @@ func TestBranchCovNormalizeTrueNASDefaults(t *testing.T) {
{"DiskWrite", t1.DiskWrite, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"NetworkIn", t1.NetworkIn, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"NetworkOut", t1.NetworkOut, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"Disk.Temperature", cfg.TrueNASDiskDefaults.Temperature, alertconfig.HysteresisThreshold{Trigger: 55, Clear: 50}},
} {
if f.got == nil {
t.Fatalf("%s is nil, want non-nil", f.name)
@@ -803,6 +802,11 @@ func TestBranchCovNormalizeTrueNASDefaults(t *testing.T) {
t.Fatalf("%s = %+v, want %+v", f.name, *f.got, f.want)
}
}
// TrueNAS disks follow the disk temperature policy until the user
// saves a TrueNAS-wide value.
if cfg.TrueNASDiskDefaults.Temperature != nil {
t.Fatalf("Disk.Temperature = %+v, want unset", *cfg.TrueNASDiskDefaults.Temperature)
}
})
t.Run("derivation and operator-enabled IO metric via CPU/DiskRead", func(t *testing.T) {
+164
View File
@@ -0,0 +1,164 @@
package config
import (
"maps"
"slices"
)
// Clone returns a deep copy of the configuration that shares no maps, slices
// or pointers with c, so the copy can be changed, normalized or encoded
// without touching c. Nil and empty maps and slices keep their distinction.
// Custom rule filter values are copied as the JSON values the API and
// alerts.json decode them to; other Go values stored there stay shared.
//
// The alert manager hands out its live configuration only through Clone;
// callers persist, encode and edit what it returns outside the manager lock.
func (c AlertConfig) Clone() AlertConfig {
clone := c
clone.ActivationTime = clonePointer(c.ActivationTime)
clone.GuestDefaults = c.GuestDefaults.Clone()
clone.NodeDefaults = c.NodeDefaults.Clone()
clone.AgentDefaults = c.AgentDefaults.Clone()
clone.DiskFillByType = maps.Clone(c.DiskFillByType)
clone.DiskTempByType = maps.Clone(c.DiskTempByType)
clone.DockerIgnoredContainerPrefixes = slices.Clone(c.DockerIgnoredContainerPrefixes)
clone.IgnoredGuestPrefixes = slices.Clone(c.IgnoredGuestPrefixes)
clone.GuestTagWhitelist = slices.Clone(c.GuestTagWhitelist)
clone.GuestTagBlacklist = slices.Clone(c.GuestTagBlacklist)
clone.PBSDefaults = c.PBSDefaults.Clone()
clone.KubernetesDefaults = c.KubernetesDefaults.Clone()
clone.TrueNASDefaults = c.TrueNASDefaults.Clone()
clone.TrueNASDiskDefaults = c.TrueNASDiskDefaults.Clone()
clone.VMwareDefaults = c.VMwareDefaults.Clone()
clone.BackupDefaults = c.BackupDefaults.clone()
if c.Overrides != nil {
clone.Overrides = make(map[string]ThresholdConfig, len(c.Overrides))
for id, override := range c.Overrides {
clone.Overrides[id] = override.Clone()
}
}
if c.CustomRules != nil {
clone.CustomRules = make([]CustomAlertRule, len(c.CustomRules))
for i, rule := range c.CustomRules {
clone.CustomRules[i] = rule.clone()
}
}
clone.Schedule.QuietHours.Days = maps.Clone(c.Schedule.QuietHours.Days)
if c.Schedule.Escalation.Levels != nil {
clone.Schedule.Escalation.Levels = make([]EscalationLevel, len(c.Schedule.Escalation.Levels))
for i, level := range c.Schedule.Escalation.Levels {
level.DestinationIDs = slices.Clone(level.DestinationIDs)
clone.Schedule.Escalation.Levels[i] = level
}
}
clone.TimeThresholds = maps.Clone(c.TimeThresholds)
clone.MetricTimeThresholds = cloneNestedIntMap(c.MetricTimeThresholds)
clone.MetricEvaluationWindows = cloneNestedIntMap(c.MetricEvaluationWindows)
return clone
}
// Clone returns a deep copy of the thresholds that shares no pointers with t.
func (t ThresholdConfig) Clone() ThresholdConfig {
clone := t
clone.CPU = clonePointer(t.CPU)
clone.Memory = clonePointer(t.Memory)
clone.Disk = clonePointer(t.Disk)
clone.DiskRead = clonePointer(t.DiskRead)
clone.DiskWrite = clonePointer(t.DiskWrite)
clone.NetworkIn = clonePointer(t.NetworkIn)
clone.NetworkOut = clonePointer(t.NetworkOut)
clone.Usage = clonePointer(t.Usage)
clone.Temperature = clonePointer(t.Temperature)
clone.DiskTemperature = clonePointer(t.DiskTemperature)
clone.SMARTHealthFailure = clonePointer(t.SMARTHealthFailure)
clone.SMARTReallocated = clonePointer(t.SMARTReallocated)
clone.SMARTPending = clonePointer(t.SMARTPending)
clone.SMARTUncorrectable = clonePointer(t.SMARTUncorrectable)
clone.SMARTMediaErrors = clonePointer(t.SMARTMediaErrors)
clone.SMARTCRCErrorDelta = clonePointer(t.SMARTCRCErrorDelta)
clone.SMARTLifeWarning = clonePointer(t.SMARTLifeWarning)
clone.SMARTLifeCritical = clonePointer(t.SMARTLifeCritical)
clone.SMARTSpareWarning = clonePointer(t.SMARTSpareWarning)
clone.SMARTSpareCritical = clonePointer(t.SMARTSpareCritical)
if t.Backup != nil {
backup := t.Backup.clone()
clone.Backup = &backup
}
clone.Snapshot = clonePointer(t.Snapshot)
clone.Note = clonePointer(t.Note)
return clone
}
func (b BackupAlertConfig) clone() BackupAlertConfig {
clone := b
clone.AlertOrphaned = clonePointer(b.AlertOrphaned)
clone.IgnoreVMIDs = slices.Clone(b.IgnoreVMIDs)
return clone
}
func (r CustomAlertRule) clone() CustomAlertRule {
clone := r
if r.FilterConditions.Filters != nil {
clone.FilterConditions.Filters = make([]FilterCondition, len(r.FilterConditions.Filters))
for i, filter := range r.FilterConditions.Filters {
filter.Value = cloneJSONValue(filter.Value)
clone.FilterConditions.Filters[i] = filter
}
}
clone.Thresholds = r.Thresholds.Clone()
if r.Notifications.Email != nil {
email := *r.Notifications.Email
email.Recipients = slices.Clone(email.Recipients)
clone.Notifications.Email = &email
}
clone.Notifications.Webhook = clonePointer(r.Notifications.Webhook)
return clone
}
// clonePointer copies the value p points at. Only use it for types that hold
// no maps, slices or pointers of their own.
func clonePointer[T any](p *T) *T {
if p == nil {
return nil
}
clone := *p
return &clone
}
func cloneNestedIntMap(input map[string]map[string]int) map[string]map[string]int {
if input == nil {
return nil
}
clone := make(map[string]map[string]int, len(input))
for key, inner := range input {
clone[key] = maps.Clone(inner)
}
return clone
}
// cloneJSONValue copies a filter value as encoding/json decodes one: arrays
// and objects are copied, scalars are values already.
func cloneJSONValue(value interface{}) interface{} {
switch typed := value.(type) {
case []interface{}:
if typed == nil {
return typed
}
clone := make([]interface{}, len(typed))
for i, item := range typed {
clone[i] = cloneJSONValue(item)
}
return clone
case map[string]interface{}:
if typed == nil {
return typed
}
clone := make(map[string]interface{}, len(typed))
for key, item := range typed {
clone[key] = cloneJSONValue(item)
}
return clone
default:
return value
}
}
+133
View File
@@ -0,0 +1,133 @@
package config
import (
"reflect"
"testing"
"time"
)
var timeType = reflect.TypeOf(time.Time{})
// Every field is filled, so a map, slice or pointer added to the config later
// fails here until Clone copies it.
func TestAlertConfigCloneSharesNothingWithTheOriginal(t *testing.T) {
var original AlertConfig
fillForClone(reflect.ValueOf(&original).Elem())
clone := original.Clone()
if !reflect.DeepEqual(clone, original) {
t.Fatalf("clone differs from the original:\n got %+v\nwant %+v", clone, original)
}
assertNothingShared(t, "AlertConfig", reflect.ValueOf(original), reflect.ValueOf(clone))
}
func TestAlertConfigCloneKeepsNilAndEmptyApart(t *testing.T) {
original := AlertConfig{
Overrides: map[string]ThresholdConfig{},
CustomRules: []CustomAlertRule{{FilterConditions: FilterStack{Filters: []FilterCondition{{Value: []interface{}(nil)}}}}},
}
clone := original.Clone()
if !reflect.DeepEqual(clone, original) {
t.Fatalf("clone differs from the original:\n got %#v\nwant %#v", clone, original)
}
if clone.Overrides == nil {
t.Fatal("clone turned an empty overrides map into nil")
}
if clone.TimeThresholds != nil {
t.Fatal("clone turned a nil delay map into an empty one")
}
}
func fillForClone(value reflect.Value) {
switch value.Kind() {
case reflect.Bool:
value.SetBool(true)
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
value.SetInt(7)
case reflect.Float32, reflect.Float64:
value.SetFloat(7.5)
case reflect.String:
value.SetString("filled")
case reflect.Pointer:
value.Set(reflect.New(value.Type().Elem()))
fillForClone(value.Elem())
case reflect.Map:
value.Set(reflect.MakeMap(value.Type()))
key := reflect.New(value.Type().Key()).Elem()
fillForClone(key)
item := reflect.New(value.Type().Elem()).Elem()
fillForClone(item)
value.SetMapIndex(key, item)
case reflect.Slice:
value.Set(reflect.MakeSlice(value.Type(), 1, 1))
fillForClone(value.Index(0))
case reflect.Interface:
// Filter values arrive from encoding/json.
value.Set(reflect.ValueOf(map[string]interface{}{"filled": []interface{}{"filled", 7.5}}))
case reflect.Struct:
if value.Type() == timeType {
value.Set(reflect.ValueOf(time.Date(2026, 10, 7, 12, 0, 0, 0, time.UTC)))
return
}
for i := 0; i < value.NumField(); i++ {
if value.Field(i).CanSet() {
fillForClone(value.Field(i))
}
}
}
}
func assertNothingShared(t *testing.T, path string, original, clone reflect.Value) {
t.Helper()
switch original.Kind() {
case reflect.Pointer:
if original.IsNil() {
return
}
if original.Pointer() == clone.Pointer() {
t.Errorf("%s: clone shares the pointer", path)
return
}
assertNothingShared(t, path, original.Elem(), clone.Elem())
case reflect.Map:
if original.IsNil() {
return
}
if original.Pointer() == clone.Pointer() {
t.Errorf("%s: clone shares the map", path)
return
}
iter := original.MapRange()
for iter.Next() {
assertNothingShared(t, path+"[key]", iter.Value(), clone.MapIndex(iter.Key()))
}
case reflect.Slice:
if original.Len() == 0 {
return
}
if original.Pointer() == clone.Pointer() {
t.Errorf("%s: clone shares the slice", path)
return
}
for i := 0; i < original.Len(); i++ {
assertNothingShared(t, path+"[i]", original.Index(i), clone.Index(i))
}
case reflect.Interface:
if original.IsNil() {
return
}
assertNothingShared(t, path, original.Elem(), clone.Elem())
case reflect.Struct:
if original.Type() == timeType {
return
}
for i := 0; i < original.NumField(); i++ {
if original.Type().Field(i).IsExported() {
assertNothingShared(t, path+"."+original.Type().Field(i).Name, original.Field(i), clone.Field(i))
}
}
}
}
+31 -1
View File
@@ -388,9 +388,35 @@ func NormalizeTrueNASDefaults(config *AlertConfig) {
config.TrueNASDefaults.NetworkIn = normalizeThresholdPointer(config.TrueNASDefaults.NetworkIn, 0, 0, "truenas.networkIn")
config.TrueNASDefaults.NetworkOut = normalizeThresholdPointer(config.TrueNASDefaults.NetworkOut, 0, 0, "truenas.networkOut")
config.TrueNASDiskDefaults.Temperature = normalizeThresholdPointer(config.TrueNASDiskDefaults.Temperature, 55, 50, "truenas.disk.temperature")
config.TrueNASDiskDefaults.Temperature = normalizeTrueNASDiskTemperature(config.TrueNASDiskDefaults.Temperature, config.TrueNASDiskTemperatureByType)
config.TrueNASDiskTemperatureByType = true
}
// normalizeTrueNASDiskTemperature keeps the TrueNAS Disks temperature default
// unset unless the user chose one. Unset, each TrueNAS disk follows the disk
// temperature policy for its type. Before that, the factory default was a
// flat 55/50 and the thresholds page wrote it back on every save, so in a
// config written before (byType false) a stored 55/50 marks a value nobody
// chose and is migrated to unset. Any other value is the user's TrueNAS-wide
// choice and is kept, 0 meaning off. Once byType is set, 55 is a choice too.
func normalizeTrueNASDiskTemperature(current *HysteresisThreshold, byType bool) *HysteresisThreshold {
if current == nil || current.Trigger < 0 {
return nil
}
if !byType && current.Trigger == legacyTrueNASDiskTemperatureTrigger &&
(current.Clear == legacyTrueNASDiskTemperatureClear || current.Clear <= 0) {
return nil
}
return normalizeThresholdPointer(current, 0, 0, "truenas.disk.temperature")
}
// The flat TrueNAS disk temperature factory default before TrueNAS disks
// followed the disk temperature policy.
const (
legacyTrueNASDiskTemperatureTrigger = 55
legacyTrueNASDiskTemperatureClear = 50
)
func NormalizeVMwareDefaults(config *AlertConfig) {
config.VMwareDefaults.CPU = normalizeThresholdPointer(config.VMwareDefaults.CPU, 80, 75, "vmware.cpu")
config.VMwareDefaults.Memory = normalizeThresholdPointer(config.VMwareDefaults.Memory, 85, 80, "vmware.memory")
@@ -554,6 +580,10 @@ func NormalizeTimeThresholds(config *AlertConfig) {
ensureDelay("truenas-pool")
ensureDelay("truenas-dataset")
ensureDelay("truenas-disk")
// Proxmox disk temperature alerts: the factory stability window for a
// noisy gauge applies to this type's unchanged default, as it does to
// agent and TrueNAS disk temperatures.
ensureDelay("proxmox-disk")
ensureDelay("vmware-host")
ensureDelay("vmware-vm")
ensureDelay("vmware-datastore")
@@ -0,0 +1,58 @@
package config
import "testing"
// TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues pins the saved
// config migration that lets TrueNAS disks follow the disk temperature policy.
// In a config written before it, the old flat factory default was written
// back on every save, so it is not a choice; anything else the user saved is
// kept.
func TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues(t *testing.T) {
tests := []struct {
name string
stored *HysteresisThreshold
want *HysteresisThreshold
}{
{name: "unset stays unset", stored: nil, want: nil},
{name: "old factory default is migrated to unset", stored: &HysteresisThreshold{Trigger: 55, Clear: 50}, want: nil},
{name: "old factory trigger without a clear is migrated to unset", stored: &HysteresisThreshold{Trigger: 55}, want: nil},
{name: "negative trigger is unset", stored: &HysteresisThreshold{Trigger: -1}, want: nil},
{name: "off is kept", stored: &HysteresisThreshold{Trigger: 0, Clear: 10}, want: &HysteresisThreshold{Trigger: 0, Clear: 0}},
{name: "custom value is kept", stored: &HysteresisThreshold{Trigger: 62, Clear: 58}, want: &HysteresisThreshold{Trigger: 62, Clear: 58}},
{name: "custom value gets a clear", stored: &HysteresisThreshold{Trigger: 62}, want: &HysteresisThreshold{Trigger: 62, Clear: 57}},
{name: "55 with a custom clear is a choice", stored: &HysteresisThreshold{Trigger: 55, Clear: 45}, want: &HysteresisThreshold{Trigger: 55, Clear: 45}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := &AlertConfig{TrueNASDiskDefaults: ThresholdConfig{Temperature: tt.stored}}
NormalizeTrueNASDefaults(cfg)
if !cfg.TrueNASDiskTemperatureByType {
t.Fatalf("normalized config is not marked as following disk type")
}
got := cfg.TrueNASDiskDefaults.Temperature
switch {
case tt.want == nil && got != nil:
t.Fatalf("Temperature = %+v, want unset", *got)
case tt.want != nil && got == nil:
t.Fatalf("Temperature unset, want %+v", *tt.want)
case tt.want != nil && *got != *tt.want:
t.Fatalf("Temperature = %+v, want %+v", *got, *tt.want)
}
})
}
}
// Once a config follows disk type, 55 is a value the user typed, and a save
// or restart must keep it.
func TestNormalizeTrueNASDiskTemperatureKeepsChosen55(t *testing.T) {
cfg := &AlertConfig{
TrueNASDiskDefaults: ThresholdConfig{Temperature: &HysteresisThreshold{Trigger: 55, Clear: 50}},
TrueNASDiskTemperatureByType: true,
}
NormalizeTrueNASDefaults(cfg)
NormalizeTrueNASDefaults(cfg)
got := cfg.TrueNASDiskDefaults.Temperature
if got == nil || *got != (HysteresisThreshold{Trigger: 55, Clear: 50}) {
t.Fatalf("Temperature = %+v, want the chosen 55/50 kept", got)
}
}
+7
View File
@@ -388,6 +388,13 @@ type AlertConfig struct {
FlappingWindowSeconds int `json:"flappingWindowSeconds"` // Time window for counting state changes
FlappingThreshold int `json:"flappingThreshold"` // Number of state changes to trigger flapping
FlappingCooldownMinutes int `json:"flappingCooldownMinutes"` // Cooldown period after flapping detected
// TrueNASDiskTemperatureByType marks a config written since an unset
// TrueNASDiskDefaults.Temperature means each TrueNAS disk follows the
// disk temperature policy for its type. An older config carries the
// former flat factory value, which nobody chose, and normalization drops
// it once.
TrueNASDiskTemperatureByType bool `json:"truenasDiskTemperatureByType,omitempty"`
}
// UnmarshalJSON accepts canonical v6 alert config keys.
+90
View File
@@ -0,0 +1,90 @@
package config
import (
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
)
var errAlertConfigUpdateNotObject = errors.New("alert configuration update must be a JSON object")
// ApplyAlertConfigUpdate decodes a client's alert configuration update onto
// the stored configuration. A top-level key the client sent replaces that
// setting whole; a key it left out keeps the stored value.
//
// Clients only send what they know about. The settings page has no control
// for flapping detection, alert TTL cleanup or custom rules, and an older
// browser bundle or API script predates any field added since. Decoding the
// body into a zero AlertConfig used to turn every one of those settings off
// on each save.
//
// The body is decoded on its own, so the keys it carries mean exactly what
// encoding/json makes of them. Each setting it left out is copied from a JSON
// round trip of stored, so the result shares no maps or slices with it.
func ApplyAlertConfigUpdate(stored AlertConfig, update []byte) (AlertConfig, error) {
var sent map[string]json.RawMessage
if err := json.Unmarshal(update, &sent); err != nil {
return AlertConfig{}, err
}
if sent == nil {
return AlertConfig{}, errAlertConfigUpdateNotObject
}
var result AlertConfig
if err := json.Unmarshal(update, &result); err != nil {
return AlertConfig{}, err
}
kept, err := CloneAlertConfig(stored)
if err != nil {
return AlertConfig{}, err
}
resultValue := reflect.ValueOf(&result).Elem()
keptValue := reflect.ValueOf(kept)
for i := 0; i < resultValue.NumField(); i++ {
if !alertConfigKeySent(sent, alertConfigJSONKey(resultValue.Type().Field(i))) {
resultValue.Field(i).Set(keptValue.Field(i))
}
}
return result, nil
}
// CloneAlertConfig returns a copy of config that shares no maps, slices or
// pointers with it, by the same JSON round trip config takes through
// alerts.json.
func CloneAlertConfig(config AlertConfig) (AlertConfig, error) {
data, err := json.Marshal(config)
if err != nil {
return AlertConfig{}, fmt.Errorf("encode alert config: %w", err)
}
var clone AlertConfig
if err := json.Unmarshal(data, &clone); err != nil {
return AlertConfig{}, fmt.Errorf("decode alert config: %w", err)
}
return clone, nil
}
// alertConfigJSONKey returns the key encoding/json reads a field from.
func alertConfigJSONKey(field reflect.StructField) string {
name, _, _ := strings.Cut(field.Tag.Get("json"), ",")
if name == "" {
return field.Name
}
return name
}
// alertConfigKeySent matches keys the way encoding/json matches them to
// fields: exactly, or else case-insensitively.
func alertConfigKeySent(sent map[string]json.RawMessage, key string) bool {
if _, ok := sent[key]; ok {
return true
}
for sentKey := range sent {
if strings.EqualFold(sentKey, key) {
return true
}
}
return false
}
+257
View File
@@ -0,0 +1,257 @@
package config
import (
"encoding/json"
"reflect"
"strings"
"testing"
"time"
)
func storedAlertConfigForUpdate() AlertConfig {
activated := time.Date(2026, 10, 1, 9, 30, 0, 0, time.UTC)
note := "keep"
return AlertConfig{
IdentitySchemaVersion: 2,
Enabled: true,
ActivationState: ActivationActive,
ActivationTime: &activated,
ObservationWindowHours: 24,
GuestDefaults: ThresholdConfig{CPU: &HysteresisThreshold{Trigger: 80, Clear: 75}},
StorageDefault: HysteresisThreshold{Trigger: 85, Clear: 80},
DiskTempByType: map[string]HysteresisThreshold{"nvme": {Trigger: 70, Clear: 65}},
IgnoredGuestPrefixes: []string{"test-"},
Overrides: map[string]ThresholdConfig{"node:pve1": {Disabled: true, Note: &note}},
CustomRules: []CustomAlertRule{{ID: "rule-1", Name: "Busy web", Enabled: true}},
Schedule: ScheduleConfig{Cooldown: 5, MaxAlertsHour: 10, InitialNotify: "all"},
MinimumDelta: 3,
SuppressionWindow: 10,
HysteresisMargin: 4,
TimeThresholds: map[string]int{"guest": 5},
MaxAlertAgeDays: 7,
MaxAcknowledgedAgeDays: 1,
AutoAcknowledgeAfterHours: 24,
FlappingEnabled: true,
FlappingWindowSeconds: 600,
FlappingThreshold: 4,
FlappingCooldownMinutes: 20,
}
}
func TestApplyAlertConfigUpdateKeepsSettingsTheClientDidNotSend(t *testing.T) {
stored := storedAlertConfigForUpdate()
// The settings page sends thresholds and schedule but has no control for
// flapping detection, alert TTL cleanup or custom rules.
got, err := ApplyAlertConfigUpdate(stored, []byte(`{
"enabled": true,
"guestDefaults": {"cpu": {"trigger": 90, "clear": 85}},
"overrides": {},
"flapping": {"enabled": true, "threshold": 5}
}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
if got.GuestDefaults.CPU == nil || got.GuestDefaults.CPU.Trigger != 90 {
t.Fatalf("guest CPU default = %+v, want the sent trigger 90", got.GuestDefaults.CPU)
}
if len(got.Overrides) != 0 {
t.Fatalf("overrides = %v, want the sent empty map to replace the stored overrides", got.Overrides)
}
if !got.FlappingEnabled || got.FlappingWindowSeconds != 600 || got.FlappingThreshold != 4 || got.FlappingCooldownMinutes != 20 {
t.Fatalf("flapping = %v/%d/%d/%d, want the stored true/600/4/20",
got.FlappingEnabled, got.FlappingWindowSeconds, got.FlappingThreshold, got.FlappingCooldownMinutes)
}
if got.MaxAlertAgeDays != 7 || got.MaxAcknowledgedAgeDays != 1 || got.AutoAcknowledgeAfterHours != 24 {
t.Fatalf("TTL cleanup = %d/%d/%d, want the stored 7/1/24",
got.MaxAlertAgeDays, got.MaxAcknowledgedAgeDays, got.AutoAcknowledgeAfterHours)
}
if !reflect.DeepEqual(got.CustomRules, stored.CustomRules) {
t.Fatalf("custom rules = %+v, want the stored rules", got.CustomRules)
}
if got.MinimumDelta != 3 || got.SuppressionWindow != 10 || got.HysteresisMargin != 4 {
t.Fatalf("general settings = %v/%d/%v, want the stored 3/10/4",
got.MinimumDelta, got.SuppressionWindow, got.HysteresisMargin)
}
}
func TestApplyAlertConfigUpdateEmptyObjectChangesNothing(t *testing.T) {
stored := storedAlertConfigForUpdate()
got, err := ApplyAlertConfigUpdate(stored, []byte(`{}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate({}): %v", err)
}
if !reflect.DeepEqual(got, stored) {
t.Fatalf("ApplyAlertConfigUpdate({}) changed the config:\n got %+v\nwant %+v", got, stored)
}
}
func TestApplyAlertConfigUpdateHonorsExplicitOffAndClear(t *testing.T) {
stored := storedAlertConfigForUpdate()
got, err := ApplyAlertConfigUpdate(stored, []byte(`{
"flappingEnabled": false,
"maxAlertAgeDays": 0,
"maxAcknowledgedAgeDays": 0,
"autoAcknowledgeAfterHours": 0,
"customRules": null,
"ignoredGuestPrefixes": []
}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
if got.FlappingEnabled || got.MaxAlertAgeDays != 0 || got.MaxAcknowledgedAgeDays != 0 || got.AutoAcknowledgeAfterHours != 0 {
t.Fatalf("explicit off was not applied: flapping=%v ttl=%d/%d/%d",
got.FlappingEnabled, got.MaxAlertAgeDays, got.MaxAcknowledgedAgeDays, got.AutoAcknowledgeAfterHours)
}
if got.CustomRules != nil {
t.Fatalf("custom rules = %+v, want null to clear them", got.CustomRules)
}
if len(got.IgnoredGuestPrefixes) != 0 {
t.Fatalf("ignored guest prefixes = %v, want the sent empty list", got.IgnoredGuestPrefixes)
}
if !got.Enabled || got.ActivationState != ActivationActive {
t.Fatalf("unsent enabled/activation = %v/%q, want the stored true/active", got.Enabled, got.ActivationState)
}
}
// The merge is top-level only: a sent object replaces the stored one whole,
// so a sent truenasDiskDefaults without a temperature must not inherit the
// stored temperature.
func TestApplyAlertConfigUpdateReplacesSentObjectsWhole(t *testing.T) {
stored := storedAlertConfigForUpdate()
stored.TrueNASDiskDefaults = ThresholdConfig{Temperature: &HysteresisThreshold{Trigger: 62, Clear: 57}}
got, err := ApplyAlertConfigUpdate(stored, []byte(`{"truenasDiskDefaults": {}}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
if got.TrueNASDiskDefaults.Temperature != nil {
t.Fatalf("TrueNAS disk temperature = %+v, want the sent empty object to clear it", got.TrueNASDiskDefaults.Temperature)
}
}
// encoding/json matches keys to fields case-insensitively, so a key in another
// casing still names the field and must not leave the stored value in place.
func TestApplyAlertConfigUpdateMatchesKeysLikeTheDecoder(t *testing.T) {
stored := storedAlertConfigForUpdate()
got, err := ApplyAlertConfigUpdate(stored, []byte(`{"FlappingEnabled": false}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
if got.FlappingEnabled {
t.Fatal("FlappingEnabled sent as false stayed true")
}
}
// Sent keys mean what decoding the body alone makes of them, duplicates and
// all.
func TestApplyAlertConfigUpdateDecodesSentKeysLikeTheBodyAlone(t *testing.T) {
stored := storedAlertConfigForUpdate()
for _, test := range []struct {
body string
field func(AlertConfig) any
}{
{`{"enabled": false, "enabled": null}`, func(c AlertConfig) any { return c.Enabled }},
{`{"flappingEnabled": false, "FlappingEnabled": true}`, func(c AlertConfig) any { return c.FlappingEnabled }},
{
`{"guestDefaults": {"cpu": {"trigger": 90, "clear": 85}}, "guestDefaults": {"memory": {"trigger": 70, "clear": 65}}}`,
func(c AlertConfig) any { return c.GuestDefaults },
},
} {
var alone AlertConfig
if err := json.Unmarshal([]byte(test.body), &alone); err != nil {
t.Fatalf("decode %s alone: %v", test.body, err)
}
got, err := ApplyAlertConfigUpdate(stored, []byte(test.body))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate(%s): %v", test.body, err)
}
if !reflect.DeepEqual(test.field(got), test.field(alone)) {
t.Fatalf("ApplyAlertConfigUpdate(%s) = %+v, decoding the body alone gives %+v",
test.body, test.field(got), test.field(alone))
}
}
if _, err := ApplyAlertConfigUpdate(stored, []byte(`{"enabled": "yes", "enabled": true}`)); err == nil {
t.Fatal("a type error in a duplicated key succeeded, want the decoder's error")
}
}
// The merge copies unsent settings field by field under their JSON keys, so
// an embedded, unexported or untagged field would be skipped or matched
// under the wrong key.
func TestAlertConfigFieldsHaveExplicitJSONKeys(t *testing.T) {
fields := reflect.TypeOf(AlertConfig{})
keys := make([]string, fields.NumField())
for i := 0; i < fields.NumField(); i++ {
field := fields.Field(i)
keys[i], _, _ = strings.Cut(field.Tag.Get("json"), ",")
if field.Anonymous || !field.IsExported() || keys[i] == "" || keys[i] == "-" {
t.Errorf("AlertConfig.%s needs to be an exported, named field with an explicit json key", field.Name)
}
// Keys that fold equal would both count as sent for either key.
for j := 0; j < i; j++ {
if strings.EqualFold(keys[i], keys[j]) {
t.Errorf("AlertConfig.%s and AlertConfig.%s have json keys that fold equal",
fields.Field(j).Name, field.Name)
}
}
}
}
func TestApplyAlertConfigUpdateSharesNoStateWithStored(t *testing.T) {
stored := storedAlertConfigForUpdate()
got, err := ApplyAlertConfigUpdate(stored, []byte(`{"enabled": true}`))
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
got.TimeThresholds["guest"] = 99
got.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 1, Clear: 0}
got.IgnoredGuestPrefixes[0] = "changed-"
*got.ActivationTime = time.Time{}
if stored.TimeThresholds["guest"] != 5 || stored.DiskTempByType["nvme"].Trigger != 70 ||
stored.IgnoredGuestPrefixes[0] != "test-" || stored.ActivationTime.IsZero() {
t.Fatal("editing the merged config changed the stored config it was merged from")
}
}
func TestApplyAlertConfigUpdateRejectsMalformedBodies(t *testing.T) {
stored := storedAlertConfigForUpdate()
for _, body := range []string{`null`, `[]`, `"config"`, `{"flappingEnabled": "yes"}`, `{`} {
if _, err := ApplyAlertConfigUpdate(stored, []byte(body)); err == nil {
t.Fatalf("ApplyAlertConfigUpdate(%s) succeeded, want a decode error", body)
}
}
}
// A config marshaled by a Go client still replaces every field it writes.
// Empty omitempty fields are left out of that body, so they keep their
// stored values like any other unsent key.
func TestApplyAlertConfigUpdateFullBodyReplacesWrittenFields(t *testing.T) {
stored := storedAlertConfigForUpdate()
replacement := AlertConfig{Enabled: true, Overrides: map[string]ThresholdConfig{}, TimeThresholds: map[string]int{}}
body, err := json.Marshal(replacement)
if err != nil {
t.Fatalf("marshal replacement: %v", err)
}
got, err := ApplyAlertConfigUpdate(stored, body)
if err != nil {
t.Fatalf("ApplyAlertConfigUpdate: %v", err)
}
if got.FlappingEnabled || got.MaxAlertAgeDays != 0 || got.MinimumDelta != 0 || len(got.Overrides) != 0 {
t.Fatalf("full body left written fields at their stored values: %+v", got)
}
if !reflect.DeepEqual(got.CustomRules, stored.CustomRules) {
t.Fatalf("custom rules = %+v, want the stored rules for the omitted key", got.CustomRules)
}
}
+62 -4
View File
@@ -1,6 +1,8 @@
package alerts
import (
"errors"
"fmt"
"strings"
"time"
@@ -13,7 +15,36 @@ import (
func (m *Manager) UpdateConfig(config AlertConfig) {
m.mu.Lock()
defer m.mu.Unlock()
m.updateConfigLocked(config)
}
// ErrConfigSnapshot reports an update that was applied but could not be copied
// for the caller.
var ErrConfigSnapshot = errors.New("snapshot applied alert configuration")
// ApplyConfigUpdate applies a client's JSON update to the stored
// configuration: keys the update carries replace their settings and the rest
// keep their stored values (see alertconfig.ApplyAlertConfigUpdate). Reading,
// merging and applying under one lock keeps two partial updates from
// reverting each other's settings. It returns a snapshot of the applied
// config that the caller owns, so persisting it cannot write into the live
// config's maps.
func (m *Manager) ApplyConfigUpdate(update []byte) (AlertConfig, error) {
m.mu.Lock()
defer m.mu.Unlock()
config, err := alertconfig.ApplyAlertConfigUpdate(m.config, update)
if err != nil {
return AlertConfig{}, err
}
m.updateConfigLocked(config)
snapshot, err := alertconfig.CloneAlertConfig(m.config)
if err != nil {
return AlertConfig{}, fmt.Errorf("%w: %w", ErrConfigSnapshot, err)
}
return snapshot, nil
}
func (m *Manager) updateConfigLocked(config AlertConfig) {
// Clients and rollback-era config writers may not know about the additive
// identity schema marker. Never lower a version already held by this
// process; an older binary can still omit it on disk and the migration will
@@ -409,7 +440,17 @@ func (m *Manager) reevaluateActiveAlertsLocked() {
alertsToResolve = append(alertsToResolve, alertID)
continue
}
thresholds := m.resolveResourceThresholds(primaryResourceType, resourceID)
query := alertPolicyQuery{TypeKey: primaryResourceType, ResourceID: resourceID}
if primaryResourceType == "truenas-disk" {
// Judge the alert against its disk type's threshold, as the
// next evaluation will. An alert raised before it recorded a
// disk type is held to the lowest per-type trigger, so a save
// never resolves an alert its type would raise again.
diskType, known := alert.Metadata["diskType"].(string)
query.DiskType = diskType
query.DiskTypeUnknown = !known
}
thresholds := m.effectiveAlertPolicyNoLock(query).Thresholds
if thresholds.Disabled {
alertsToResolve = append(alertsToResolve, alertID)
continue
@@ -417,6 +458,19 @@ func (m *Manager) reevaluateActiveAlertsLocked() {
threshold = getThresholdForMetric(thresholds, metricType)
}
// A Proxmox disk temperature alert is judged by the disk temperature
// policy for its disk type, as CheckProxmoxDiskTemperature judges it.
// The node and guest policies below have no disk temperature, so a
// save would otherwise resolve an alert the next poll raises again.
if primaryResourceType == proxmoxDiskResourceType && metricType == proxmoxDiskTemperatureMetric {
diskType, known := alert.Metadata["diskType"].(string)
threshold = m.proxmoxDiskTemperatureThresholdNoLock(diskType, !known)
if threshold == nil || threshold.Trigger <= 0 {
alertsToResolve = append(alertsToResolve, alertID)
continue
}
}
isAgentResource := alertResourceTypeKeysContain(resourceTypeKeys, "agent")
if !handledModernPlatformType && isAgentResource {
if allDisabled, _ := m.alertPolicyTypeSwitchesNoLock("agent"); allDisabled {
@@ -545,7 +599,7 @@ func (m *Manager) reevaluateActiveAlertsLocked() {
continue
}
guestThresholds := m.getGuestThresholds(guestSnapshotFromAlert(alert, resourceID), resourceID)
guestThresholds := m.resolveGuestAlertThresholdsNoLock(alert, resourceID)
if guestThresholds.Disabled {
alertsToResolve = append(alertsToResolve, alertID)
continue
@@ -634,11 +688,15 @@ func (m *Manager) reevaluateActiveAlertsLocked() {
}
}
// GetConfig returns the current alert configuration.
// GetConfig returns a copy of the alert configuration that the caller owns.
// Callers encode, persist and edit it outside m.mu, and UpdateConfig
// normalizes the maps it is handed in place, so a copy that shared the live
// config's maps let a save or a re-apply write them while evaluation read
// them, which Go aborts as a concurrent map write.
func (m *Manager) GetConfig() AlertConfig {
m.mu.RLock()
defer m.mu.RUnlock()
return m.config
return m.config.Clone()
}
func cloneThreshold(threshold *HysteresisThreshold) *HysteresisThreshold {
+76
View File
@@ -205,3 +205,79 @@ func TestUpdateConfigNormalizesExactEscalationRoutingAndRepeatPolicy(t *testing.
}
}
}
// The thresholds page sends no flapping, alert TTL or custom-rule keys. A save
// through ApplyConfigUpdate must leave those settings as stored, also after
// the merged config is normalized in place.
func TestUpdateConfigKeepsSettingsAThresholdsSaveDidNotSend(t *testing.T) {
m := newTestManager(t)
stored := m.GetConfig()
stored.FlappingEnabled = true
stored.FlappingWindowSeconds = 600
stored.MaxAlertAgeDays = 7
stored.MaxAcknowledgedAgeDays = 1
stored.AutoAcknowledgeAfterHours = 24
stored.CustomRules = []CustomAlertRule{{ID: "rule-1", Name: "Busy web", Enabled: true}}
m.UpdateConfig(stored)
snapshot, err := m.ApplyConfigUpdate([]byte(`{
"enabled": true,
"nodeDefaults": {"cpu": {"trigger": 82, "clear": 77}},
"overrides": {}
}`))
if err != nil {
t.Fatalf("ApplyConfigUpdate: %v", err)
}
snapshot.TimeThresholds["guest"] = 999
if m.GetConfig().TimeThresholds["guest"] == 999 {
t.Fatal("the returned snapshot shares its maps with the live config")
}
got := m.GetConfig()
if got.NodeDefaults.CPU == nil || got.NodeDefaults.CPU.Trigger != 82 {
t.Fatalf("node CPU default = %+v, want the saved trigger 82", got.NodeDefaults.CPU)
}
if !got.FlappingEnabled || got.FlappingWindowSeconds != 600 {
t.Fatalf("flapping = %v/%d after a thresholds save, want true/600", got.FlappingEnabled, got.FlappingWindowSeconds)
}
if got.MaxAlertAgeDays != 7 || got.MaxAcknowledgedAgeDays != 1 || got.AutoAcknowledgeAfterHours != 24 {
t.Fatalf("TTL cleanup = %d/%d/%d after a thresholds save, want 7/1/24",
got.MaxAlertAgeDays, got.MaxAcknowledgedAgeDays, got.AutoAcknowledgeAfterHours)
}
if len(got.CustomRules) != 1 || got.CustomRules[0].ID != "rule-1" {
t.Fatalf("custom rules = %+v after a thresholds save, want the stored rule", got.CustomRules)
}
}
// Each partial update reads, merges and applies under the manager lock, so
// two saves that touch different settings cannot revert each other.
func TestApplyConfigUpdateKeepsConcurrentPartialUpdates(t *testing.T) {
m := newTestManager(t)
for round := 0; round < 50; round++ {
base := m.GetConfig()
base.FlappingEnabled = false
base.MaxAlertAgeDays = 0
m.UpdateConfig(base)
bodies := []string{`{"flappingEnabled": true}`, `{"maxAlertAgeDays": 9}`}
done := make(chan struct{}, len(bodies))
for _, body := range bodies {
go func(body string) {
defer func() { done <- struct{}{} }()
if _, err := m.ApplyConfigUpdate([]byte(body)); err != nil {
t.Errorf("ApplyConfigUpdate(%s): %v", body, err)
}
}(body)
}
for range bodies {
<-done
}
got := m.GetConfig()
if !got.FlappingEnabled || got.MaxAlertAgeDays != 9 {
t.Fatalf("round %d: flapping=%v maxAlertAgeDays=%d, want both concurrent updates kept",
round, got.FlappingEnabled, got.MaxAlertAgeDays)
}
}
}
+6 -3
View File
@@ -106,9 +106,11 @@ func defaultAlertConfig() AlertConfig {
NetworkIn: &HysteresisThreshold{Trigger: 0, Clear: 0},
NetworkOut: &HysteresisThreshold{Trigger: 0, Clear: 0},
},
TrueNASDiskDefaults: ThresholdConfig{
Temperature: &HysteresisThreshold{Trigger: 55, Clear: 50},
},
// TrueNAS disk temperature is unset: each disk follows the disk
// temperature policy for its type (DiskTempByType) until the user
// saves a TrueNAS-wide value.
TrueNASDiskDefaults: ThresholdConfig{},
TrueNASDiskTemperatureByType: true,
VMwareDefaults: ThresholdConfig{
CPU: &HysteresisThreshold{Trigger: 80, Clear: 75},
Memory: &HysteresisThreshold{Trigger: 85, Clear: 80},
@@ -148,6 +150,7 @@ func defaultAlertConfig() AlertConfig {
"truenas-pool": 5,
"truenas-dataset": 5,
"truenas-disk": 5,
"proxmox-disk": 5,
"vmware-host": 5,
"vmware-vm": 5,
"vmware-datastore": 5,
+16 -13
View File
@@ -352,8 +352,19 @@ func (m *Manager) evaluateDockerContainer(host models.DockerHost, container mode
resourceType := "app-container"
containerTags := dockerLabelTags(container.Labels)
// Resolve the metric thresholds while the lock is held: the Docker defaults
// are fields of m.config, which UpdateConfig replaces, and the override
// still points into it. applyThresholdOverride clones what it keeps.
m.mu.RLock()
overrideConfig, hasOverride := m.lookupDockerContainerOverrideNoLock(host.ID, container.Name, resourceID)
thresholds := ThresholdConfig{
CPU: cloneThreshold(&m.config.DockerDefaults.CPU),
Memory: cloneThreshold(&m.config.DockerDefaults.Memory),
Disk: cloneThreshold(&m.config.DockerDefaults.Disk),
}
if hasOverride {
thresholds = m.applyThresholdOverride(thresholds, overrideConfig)
}
m.mu.RUnlock()
if hasOverride && overrideConfig.Disabled {
// Alerts disabled via override; clear any existing alerts and skip evaluation.
@@ -377,16 +388,6 @@ func (m *Manager) evaluateDockerContainer(host models.DockerHost, container mode
} else {
m.clearDockerContainerStateAlert(resourceID)
// Use Docker-specific defaults for containers
thresholds := ThresholdConfig{
CPU: &m.config.DockerDefaults.CPU,
Memory: &m.config.DockerDefaults.Memory,
Disk: &m.config.DockerDefaults.Disk,
}
if hasOverride {
thresholds = m.applyThresholdOverride(thresholds, overrideConfig)
}
if thresholds.CPU != nil {
cpuCapacityPercent := models.DockerContainerCPUCapacityPercent(container, host.CPUs)
cpuMetadata := map[string]interface{}{
@@ -998,6 +999,8 @@ func (m *Manager) checkDockerContainerRestartLoop(host models.DockerHost, contai
alertID := fmt.Sprintf("docker-container-restart-loop-%s", resourceID)
now := time.Now()
m.mu.Lock()
// Get config values with defaults
restartThreshold := m.config.DockerDefaults.RestartCount
if restartThreshold == 0 {
@@ -1008,8 +1011,6 @@ func (m *Manager) checkDockerContainerRestartLoop(host models.DockerHost, contai
timeWindow = 300 // Default: 5 minutes (300 seconds)
}
m.mu.Lock()
record, exists := m.dockerRestartTracking[resourceID]
if !exists {
record = &dockerRestartRecord{
@@ -1164,11 +1165,13 @@ func (m *Manager) checkDockerContainerMemoryLimit(host models.DockerHost, contai
alertID := fmt.Sprintf("docker-container-memory-limit-%s", resourceID)
// Get config values with defaults
m.mu.RLock()
warnThreshold := float64(m.config.DockerDefaults.MemoryWarnPct)
criticalThreshold := float64(m.config.DockerDefaults.MemoryCriticalPct)
m.mu.RUnlock()
if warnThreshold == 0 {
warnThreshold = 90.0 // Default: 90%
}
criticalThreshold := float64(m.config.DockerDefaults.MemoryCriticalPct)
if criticalThreshold == 0 {
criticalThreshold = 95.0 // Default: 95%
}
+53 -21
View File
@@ -141,8 +141,7 @@ func (m *Manager) CheckGuest(guest any, instanceName string) {
return
}
settings := policy.TagSettings
monitorOnly := settings.MonitorOnly
monitorOnly := policy.TagSettings.MonitorOnly
if monitorOnly || m.guestHasMonitorOnlyAlerts(guestID) {
log.Debug().
Str("guest", name).
@@ -215,19 +214,12 @@ func (m *Manager) CheckGuest(guest any, instanceName string) {
// If guest is running, clear any powered-off alert
m.clearGuestPoweredOffAlert(guestID, name)
// Get thresholds (check custom rules, then overrides, then defaults)
// Get thresholds (defaults, custom rules, overrides, then the
// pulse-relaxed floor)
m.mu.RLock()
thresholds := m.getGuestThresholds(guest, guestID)
m.mu.RUnlock()
if settings.Relaxed {
thresholds = applyRelaxedGuestThresholds(thresholds)
log.Info().
Str("guest", name).
Float64("trigger", thresholds.CPU.Trigger).
Msg("Applied relaxed thresholds for pulse-relaxed tag")
}
// If alerts are disabled for this guest, clear any existing alerts and return
if thresholds.Disabled {
if alertsCleared := m.clearGuestMetricAlerts(guestID); alertsCleared > 0 {
@@ -327,18 +319,12 @@ func (m *Manager) CheckGuest(guest any, instanceName string) {
seenDiskResources[perDiskResourceID] = struct{}{}
message := fmt.Sprintf("%s disk (%s) at %.1f%%", guestType, label, disk.Usage)
effectiveDiskThreshold := thresholds.Disk
m.mu.RLock()
diskOverride, hasDiskOverride := lookupGuestDiskOverride(m.config.Overrides, guest, guestID, keySource)
effectiveDiskThreshold, diskDisabled := m.guestDiskUsageThresholdNoLock(guest, guestID, keySource, thresholds.Disk)
m.mu.RUnlock()
if hasDiskOverride {
if diskOverride.Disabled {
m.clearAlert(canonicalMetricStateID(perDiskResourceID, "disk"))
continue
}
if diskOverride.Disk != nil {
effectiveDiskThreshold = ensureHysteresisThreshold(diskOverride.Disk)
}
if diskDisabled {
m.clearAlert(canonicalMetricStateID(perDiskResourceID, "disk"))
continue
}
if snapshot.DiskUnavailable {
if effectiveDiskThreshold == nil || effectiveDiskThreshold.Trigger <= 0 {
@@ -423,6 +409,45 @@ func guestDiskIdentity(disk models.Disk, idx int) (label, keySource, sanitizedKe
return label, keySource, sanitizedKey
}
// guestDiskUsageThresholdNoLock resolves the usage threshold one guest
// filesystem is judged by: the filesystem's own override, else the guest's disk
// threshold. disabled reports an override that switches the filesystem's
// alerts off, and then the threshold is nil. CheckGuest and config-save
// reevaluation both resolve through it. An override threshold is returned as
// an owned copy, so normalizing it never writes into the live config. Callers
// must hold m.mu.
func (m *Manager) guestDiskUsageThresholdNoLock(guest any, guestID, diskKey string, guestDisk *HysteresisThreshold) (threshold *HysteresisThreshold, disabled bool) {
if override, ok := lookupGuestDiskOverride(m.config.Overrides, guest, guestID, diskKey); ok {
if override.Disabled {
return nil, true
}
if override.Disk != nil {
return ensureHysteresisThreshold(cloneThreshold(override.Disk)), false
}
}
return guestDisk, false
}
// resolveGuestAlertThresholdsNoLock resolves thresholds for a persisted guest
// alert. A filesystem alert ("<guestID>-disk-<key>") resolves its guest's
// thresholds by the guest ID, and its Disk by the filesystem's own override, as
// CheckGuest does. Resolved by the filesystem's resource ID, which names no
// guest, it missed both overrides, and a config save resolved an alert the next
// poll raised again. Callers must hold m.mu.
func (m *Manager) resolveGuestAlertThresholdsNoLock(alert *Alert, resourceID string) ThresholdConfig {
resourceID = strings.TrimSpace(resourceID)
if identity, ok := parseGuestAlertIdentity(resourceID); ok {
if diskKey, isFilesystem := strings.CutPrefix(identity.resourceSuffix, "-disk-"); isFilesystem {
guestID := strings.TrimSuffix(resourceID, identity.resourceSuffix)
guest := guestSnapshotFromAlert(alert, guestID)
thresholds := m.getGuestThresholds(guest, guestID)
thresholds.Disk, _ = m.guestDiskUsageThresholdNoLock(guest, guestID, diskKey, thresholds.Disk)
return thresholds
}
}
return m.getGuestThresholds(guestSnapshotFromAlert(alert, resourceID), resourceID)
}
func guestIORateMetrics(snapshot guestSnapshot) (diskRead, diskWrite, networkIn, networkOut *UnifiedResourceMetric) {
validity := snapshot.IORateValidity.EffectiveForRates(
snapshot.DiskRead,
@@ -655,6 +680,10 @@ func parsePulseTags(tags []string) pulseTagSettings {
return settings
}
// applyRelaxedGuestThresholds lifts a pulse-relaxed guest's CPU, memory and
// disk triggers to a 95/92/95 floor. An unset threshold gets the floor; one
// switched Off (trigger <= 0) stays Off, since relaxing a guest must never
// raise an alert its config turned off.
func applyRelaxedGuestThresholds(cfg ThresholdConfig) ThresholdConfig {
relaxed := cloneThresholdConfig(cfg)
@@ -663,6 +692,9 @@ func applyRelaxedGuestThresholds(cfg ThresholdConfig) ThresholdConfig {
*th = &HysteresisThreshold{Trigger: minTrigger, Clear: minTrigger - 5}
return
}
if (*th).Trigger <= 0 {
return
}
ensureHysteresisThreshold(*th)
if (*th).Trigger < minTrigger {
(*th).Trigger = minTrigger
+3
View File
@@ -251,6 +251,9 @@ func guestSnapshotFromAlert(alert *Alert, resourceID string) guestSnapshot {
if snapshot.VMID <= 0 {
snapshot.VMID = metadataIntValue(alert.Metadata["guestVmid"])
}
// Tags carry pulse-relaxed, so a config save re-judges the alert by the
// thresholds CheckGuest raised it under.
snapshot.Tags, _ = metadataStringsValue(alert.Metadata, "tags")
return snapshot.normalizeCollections()
}
+45 -31
View File
@@ -172,9 +172,50 @@ func (m *Manager) resolveHostAlertThresholdsNoLock(alert *Alert, resourceID stri
thresholds.DiskTemperature = m.lowestHostDiskTemperatureThresholdNoLock(thresholds.DiskTemperature)
}
}
// A filesystem alert is judged by the threshold CheckHost evaluates that
// filesystem against, so a config save does not resolve an alert its
// override or per-type fill threshold still holds. A disabled filesystem
// resolves with no threshold, as CheckHost clears it.
if alert != nil && alert.Type == "disk" && isHostFilesystemResourceID(resourceID) {
thresholds.Disk, _ = m.hostDiskUsageThresholdNoLock(strings.TrimSpace(resourceID), metadataStringValue(alert.Metadata, "device"), thresholds.Disk)
}
return thresholds
}
// isHostFilesystemResourceID reports whether resourceID names one agent
// filesystem ("agent:<host>/disk:<label>") rather than the host itself.
func isHostFilesystemResourceID(resourceID string) bool {
_, child, ok := strings.Cut(stripHostResourcePrefix(resourceID), "/")
return ok && strings.HasPrefix(child, "disk:")
}
// hostDiskUsageThresholdNoLock resolves the usage threshold one agent
// filesystem is judged by: the filesystem's own override, then the fill
// threshold for its hardware type while the host's disk alerting is on, then
// the host's disk threshold. disabled reports an override that switches the
// filesystem's alerts off, and then the threshold is nil. CheckHost and
// config-save reevaluation both resolve through it. An override or per-type
// threshold is returned as an owned copy, so normalizing it never writes into
// the live config. Callers must hold m.mu.
func (m *Manager) hostDiskUsageThresholdNoLock(diskResourceID, device string, hostDisk *HysteresisThreshold) (threshold *HysteresisThreshold, disabled bool) {
if override, ok := m.config.Overrides[diskResourceID]; ok {
if override.Disabled {
return nil, true
}
if override.Disk != nil {
return ensureHysteresisThreshold(cloneThreshold(override.Disk)), false
}
}
if hostDisk != nil && hostDisk.Trigger > 0 {
if hwType := inferDiskHardwareType(device); hwType != "" {
if byType, ok := m.config.DiskFillByType[hwType]; ok {
return &byType, false
}
}
}
return hostDisk, false
}
func sanitizeHostComponent(value string) string {
value = strings.TrimSpace(strings.ToLower(value))
if value == "" {
@@ -450,39 +491,12 @@ func (m *Manager) CheckHost(host models.Host) {
diskResourceID, diskName := hostDiskResourceID(host, disk)
seenDisks[diskResourceID] = struct{}{}
// Check for disk-specific override
m.mu.RLock()
diskOverride, hasDiskOverride := m.config.Overrides[diskResourceID]
effectiveDiskThreshold, diskDisabled := m.hostDiskUsageThresholdNoLock(diskResourceID, disk.Device, thresholds.Disk)
m.mu.RUnlock()
// Determine the effective disk threshold
var effectiveDiskThreshold *HysteresisThreshold
if hasDiskOverride {
// If disk is disabled via override, skip alerting
if diskOverride.Disabled {
m.clearAlert(canonicalMetricStateID(diskResourceID, "disk"))
continue
}
// Use disk-specific threshold if set
if diskOverride.Disk != nil {
effectiveDiskThreshold = ensureHysteresisThreshold(diskOverride.Disk)
}
}
// Per-type override: consult DiskFillByType if hardware type is inferable
// from the device path and no disk-specific override applied above.
if effectiveDiskThreshold == nil && thresholds.Disk != nil && thresholds.Disk.Trigger > 0 {
if hwType := inferDiskHardwareType(disk.Device); hwType != "" {
m.mu.RLock()
if th, ok := m.config.DiskFillByType[hwType]; ok {
t := th
effectiveDiskThreshold = &t
}
m.mu.RUnlock()
}
}
// Fall back to host-level threshold
if effectiveDiskThreshold == nil {
effectiveDiskThreshold = thresholds.Disk
if diskDisabled {
m.clearAlert(canonicalMetricStateID(diskResourceID, "disk"))
continue
}
// Skip if no threshold configured (nil)
-10
View File
@@ -86,16 +86,6 @@ func getThresholdForMetric(config ThresholdConfig, metricType string) *Hysteresi
}
}
// getThresholdForMetricFromConfig returns the threshold for a specific metric type from a ThresholdConfig
// ensuring hysteresis is properly set.
func getThresholdForMetricFromConfig(config ThresholdConfig, metricType string) *HysteresisThreshold {
th := getThresholdForMetric(config, metricType)
if th == nil {
return nil
}
return ensureHysteresisThreshold(th)
}
// getTimeThreshold determines the delay to apply for a metric/resource combination.
func (m *Manager) getTimeThreshold(resourceID string, resourceType, metricType string) int {
effective := m.resolveEffectiveIntentPolicyNoLock(resourceID, resourceType, MetricAlertIntentSignal(metricType))
+110
View File
@@ -0,0 +1,110 @@
package alerts
import (
"fmt"
"strings"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
"github.com/rs/zerolog/log"
)
// proxmoxDiskTemperatureMetric is the metric of a Proxmox physical disk's
// temperature alert, the same metric CheckHost raises for an agent's disks.
const proxmoxDiskTemperatureMetric = "diskTemperature"
// proxmoxDiskResourceType is the resource type PVE disk alerts carry in their
// spec and metadata (see CheckDiskHealth).
const proxmoxDiskResourceType = "proxmox-disk"
// ProxmoxDiskTemperatureReading is one Proxmox physical disk's temperature as
// the PVE disk poller judged it.
type ProxmoxDiskTemperatureReading struct {
Disk proxmox.Disk
// Celsius is the reading collected this poll, or 0 when there is none:
// no sensor reports the disk, it is in standby, or normalization kept a
// last-known value it did not collect now.
Celsius int
// AgentOwned says a linked Pulse agent lists the disk in its SMART
// report, so the agent's CheckHost owns its temperature alert.
AgentOwned bool
// Excluded says the linked agent's --disk-exclude patterns match the
// device, which excludes it from disk alerts.
Excluded bool
}
// CheckProxmoxDiskTemperature raises a Proxmox physical disk's temperature
// alert under the disk temperature policy (DiskTemperatureThreshold): the
// same per-type thresholds that judge an agent's disks, TrueNAS disks and the
// Physical Disks verdict. Without it a disk that only the node's sensors
// report could read Running Hot and never alert.
//
// A disk a linked agent lists is the agent's: its open alert here closes as
// moved to the agent rather than recovered, so the disk never alerts twice.
// An excluded disk, or a policy switched off by the agent Disk Temp default,
// closes its alert. A disk with no current reading keeps its alert as it is,
// since no reading shows it cooled.
func (m *Manager) CheckProxmoxDiskTemperature(instance, node string, reading ProxmoxDiskTemperatureReading) {
if m == nil {
return
}
disk := reading.Disk
resourceID := proxmoxDiskCanonicalResourceID(instance, node, disk.DevPath)
resourceName := fmt.Sprintf("%s (%s)", disk.Model, disk.DevPath)
diskType := strings.ToLower(strings.TrimSpace(disk.Type))
m.mu.RLock()
enabled := m.config.Enabled
threshold := m.proxmoxDiskTemperatureThresholdNoLock(diskType, false)
m.mu.RUnlock()
if !enabled {
return
}
if reading.AgentOwned || reading.Excluded || threshold == nil || threshold.Trigger <= 0 {
spec, err := buildCanonicalMetricSpec(resourceID, resourceName, unifiedresources.ResourceType(proxmoxDiskResourceType), proxmoxDiskTemperatureMetric, nil)
if err != nil {
return
}
var resolution *AlertResolution
if reading.AgentOwned && !reading.Excluded {
resolution = &AlertResolution{Reason: AlertResolutionMovedToAgent}
}
m.releaseCanonicalMetricAlert(spec, resourceName, node, instance, proxmoxDiskResourceType, 0, resolution)
return
}
if reading.Celsius <= 0 {
return
}
spec, err := buildCanonicalMetricSpec(resourceID, resourceName, unifiedresources.ResourceType(proxmoxDiskResourceType), proxmoxDiskTemperatureMetric, threshold)
if err != nil {
log.Warn().
Err(err).
Str("resourceID", resourceID).
Str("node", node).
Str("disk", disk.DevPath).
Msg("Skipping invalid canonical Proxmox disk temperature metric spec")
return
}
metadata := proxmoxDiskAlertMetadata(disk)
metadata["metric"] = proxmoxDiskTemperatureMetric
// A config save re-judges the open alert against its disk type.
metadata["diskType"] = diskType
value := float64(reading.Celsius)
m.checkMetricWithCanonicalSpec(spec, resourceName, node, instance, proxmoxDiskResourceType, value, threshold, &metricOptions{
Metadata: metadata,
Message: fmt.Sprintf("Disk temperature at %s", formatMetricValue(value, "°C")),
})
}
// proxmoxDiskTemperatureThresholdNoLock is the disk temperature policy for a
// Proxmox disk of diskType, or the lowest per-type trigger for an alert whose
// disk type was never recorded, so a config save never resolves an alert the
// next poll raises again. Callers must hold m.mu.
func (m *Manager) proxmoxDiskTemperatureThresholdNoLock(diskType string, typeUnknown bool) *HysteresisThreshold {
if typeUnknown {
return cloneThreshold(m.lowestHostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature))
}
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
}
+18
View File
@@ -196,6 +196,24 @@ func metadataStringValue(metadata map[string]interface{}, key string) string {
}
}
// metadataStringsValue reads a string list from alert metadata. A live alert
// holds a []string; one reloaded from disk holds a []interface{}.
func metadataStringsValue(metadata map[string]interface{}, key string) ([]string, bool) {
switch values := metadata[key].(type) {
case []string:
return append([]string(nil), values...), true
case []interface{}:
result := make([]string, 0, len(values))
for _, value := range values {
if s, ok := value.(string); ok {
result = append(result, s)
}
}
return result, true
}
return nil, false
}
func metadataBoolValue(metadata map[string]interface{}, key string) bool {
if metadata == nil {
return false
@@ -1,6 +1,7 @@
package alerts
import (
"encoding/json"
"fmt"
"runtime"
"sync"
@@ -8,6 +9,7 @@ import (
"time"
alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rs/zerolog"
)
@@ -304,3 +306,80 @@ func waitForResolvedConcurrencyGroup(t *testing.T, wg *sync.WaitGroup) {
t.Fatal("resolved alert concurrency paths deadlocked")
}
}
// GetConfig snapshots are encoded, persisted and edited outside m.mu, and
// UpdateConfig normalizes the maps it is handed in place. While a snapshot
// shared the live config's maps, re-applying one rewrote the override map
// that the GET /api/alerts/config handler was encoding from another snapshot,
// and Go aborted the process with a concurrent map write. Run with -race.
func TestConfigSnapshotsShareNothingWithTheLiveConfig(t *testing.T) {
originalLogLevel := zerolog.GlobalLevel()
zerolog.SetGlobalLevel(zerolog.Disabled)
t.Cleanup(func() {
zerolog.SetGlobalLevel(originalLogLevel)
})
m := NewManagerWithDataDir(t.TempDir())
t.Cleanup(m.Stop)
seed := m.GetConfig()
seed.Enabled = true
seed.ActivationState = ActivationActive
seed.Overrides = make(map[string]ThresholdConfig)
for i := 0; i < 64; i++ {
seed.Overrides[fmt.Sprintf("agent:snapshot-%d", i)] = ThresholdConfig{
CPU: &HysteresisThreshold{Trigger: 95, Clear: 90},
}
}
m.UpdateConfig(seed)
snapshot := m.GetConfig()
snapshot.Overrides["agent:snapshot-0"].CPU.Trigger = 1
delete(snapshot.Overrides, "agent:snapshot-1")
snapshot.TimeThresholds["guest"] = 99
snapshot.AgentDefaults.CPU.Trigger = 1
live := m.GetConfig()
if got := live.Overrides["agent:snapshot-0"].CPU.Trigger; got != 95 {
t.Fatalf("editing a snapshot changed the live override trigger to %v", got)
}
if _, ok := live.Overrides["agent:snapshot-1"]; !ok {
t.Fatal("deleting from a snapshot removed the live override")
}
if live.TimeThresholds["guest"] == 99 {
t.Fatal("editing a snapshot changed the live guest delay")
}
if live.AgentDefaults.CPU.Trigger == 1 {
t.Fatal("editing a snapshot changed the live agent CPU trigger")
}
var wg sync.WaitGroup
wg.Add(3)
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
m.UpdateConfig(m.GetConfig())
}
}()
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
if _, err := json.Marshal(m.GetConfig()); err != nil {
t.Errorf("encode snapshot: %v", err)
return
}
}
}()
go func() {
defer wg.Done()
for i := 0; i < 200; i++ {
id := fmt.Sprintf("snapshot-%d", i%64)
m.CheckHost(models.Host{ID: id, Hostname: id, Status: "online", CPUUsage: 50})
}
}()
wg.Wait()
if got := len(m.GetConfig().Overrides); got != 64 {
t.Fatalf("live overrides = %d after re-applying snapshots, want 64", got)
}
}
+2 -11
View File
@@ -76,17 +76,8 @@ func storagePolicyAliases(alert *Alert) []string {
if alert == nil {
return nil
}
switch aliases := alert.Metadata[storagePolicyAliasesKey].(type) {
case []string:
return append([]string(nil), aliases...)
case []interface{}:
result := make([]string, 0, len(aliases))
for _, value := range aliases {
if alias, ok := value.(string); ok {
result = append(result, alias)
}
}
return result
if aliases, ok := metadataStringsValue(alert.Metadata, storagePolicyAliasesKey); ok {
return aliases
}
// Older PBS snapshots predate alias metadata. The poller records the
// instance and datastore name separately: only reconstruct an alias when
@@ -1350,6 +1350,123 @@ func TestGuestFilesystemEvidenceSuppressesDuplicateAggregateAlert(t *testing.T)
}
}
// A config save re-judges every active alert. A filesystem alert must be judged
// by the threshold its evaluator used for that filesystem, not the host or guest
// default: saving any unrelated setting used to resolve an alert a per-disk
// threshold of 70 raised at 80 under a default of 90, notifying recovery, and
// the next poll raised it again as a new alert.
func TestConfigSaveJudgesFilesystemAlertsByTheirOwnThreshold(t *testing.T) {
perDisk := func() *HysteresisThreshold { return &HysteresisThreshold{Trigger: 70, Clear: 65} }
abovePerDisk := func() *HysteresisThreshold { return &HysteresisThreshold{Trigger: 85, Clear: 80} }
agentHost := func(device string) models.Host {
return models.Host{
ID: "host-fs", Hostname: "host-fs", Platform: "linux", Status: "online",
IntervalSeconds: 30, LastSeen: time.Now(),
Disks: []models.Disk{{Mountpoint: "/data", Device: device, Usage: 80, Total: 100, Used: 80, Free: 20}},
}
}
guestID := BuildGuestKey("pve1", "node1", 101)
guestVM := models.VM{
ID: guestID, VMID: 101, Name: "db", Node: "node1", Instance: "pve1", Status: "running",
Disks: []models.Disk{{Mountpoint: "/data", Device: "/dev/vdb1", Usage: 80, Total: 100, Used: 80, Free: 20}},
}
guestAlertID := canonicalMetricStateID(guestID+"-disk-data-dev-vdb1", "disk")
type diskCase struct {
name string
alertID string
configure func(cfg *AlertConfig)
evaluate func(m *Manager)
// raise lifts the threshold that judges this filesystem above its reading.
raise func(cfg *AlertConfig)
}
agentDiskID, _ := hostDiskResourceID(agentHost("/dev/sdb1"), agentHost("/dev/sdb1").Disks[0])
guestDiskOverrideID := guestDiskOverrideKey(guestVM, guestID, "/data-/dev/vdb1")
guestOverrideID := guestOverridePrimaryKey(guestVM, guestID)
cases := []diskCase{
{
name: "agent filesystem override",
alertID: canonicalMetricStateID(agentDiskID, "disk"),
configure: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{agentDiskID: {Disk: perDisk()}}
},
evaluate: func(m *Manager) { m.CheckHost(agentHost("/dev/sdb1")) },
raise: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{agentDiskID: {Disk: abovePerDisk()}}
},
},
{
name: "agent disk fill by hardware type",
alertID: canonicalMetricStateID(agentDiskID, "disk"),
configure: func(cfg *AlertConfig) {
cfg.DiskFillByType["nvme"] = *perDisk()
},
evaluate: func(m *Manager) { m.CheckHost(agentHost("/dev/nvme0n1p1")) },
raise: func(cfg *AlertConfig) {
cfg.DiskFillByType["nvme"] = *abovePerDisk()
},
},
{
name: "guest filesystem override",
alertID: guestAlertID,
configure: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{guestDiskOverrideID: {Disk: perDisk()}}
},
evaluate: func(m *Manager) { m.CheckGuest(guestVM, "pve1") },
raise: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{guestDiskOverrideID: {Disk: abovePerDisk()}}
},
},
{
name: "guest override on a filesystem alert",
alertID: guestAlertID,
configure: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{guestOverrideID: {Disk: perDisk()}}
},
evaluate: func(m *Manager) { m.CheckGuest(guestVM, "pve1") },
raise: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{guestOverrideID: {Disk: abovePerDisk()}}
},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
m := newTestManager(t)
cfg := m.GetConfig()
cfg.Enabled = true
// UpdateConfig fills missing delays with 5s; zero fires at once.
cfg.TimeThresholds = map[string]int{"agent": 0, "guest": 0}
cfg.AgentDefaults.Disk = &HysteresisThreshold{Trigger: 90, Clear: 85}
cfg.GuestDefaults.Disk = &HysteresisThreshold{Trigger: 90, Clear: 85}
tc.configure(&cfg)
m.UpdateConfig(cfg)
tc.evaluate(m)
startedAt := testRequireActiveAlert(t, m, tc.alertID).StartTime
m.UpdateConfig(m.GetConfig())
if !testHasActiveAlert(t, m, tc.alertID) {
t.Fatalf("an unchanged config save resolved %q, which its own threshold of 70 still holds at 80", tc.alertID)
}
tc.evaluate(m)
if got := testRequireActiveAlert(t, m, tc.alertID).StartTime; !got.Equal(startedAt) {
t.Fatalf("alert %q was raised again after the save: started %v, want %v", tc.alertID, got, startedAt)
}
cfg = m.GetConfig()
tc.raise(&cfg)
m.UpdateConfig(cfg)
if testHasActiveAlert(t, m, tc.alertID) {
t.Fatalf("a save lifting %q's own trigger above its reading kept the alert", tc.alertID)
}
})
}
}
// Regression: checkMetric stores canonical-identity alerts under the
// canonical state key, so hysteresis resolution must not remove only the
// unregistered legacy "<resourceID>-<metric>" ID.

Some files were not shown because too many files have changed in this diff Show More