diff --git a/docs/API.md b/docs/API.md index 296d4cc116..002d29a1e2 100644 --- a/docs/API.md +++ b/docs/API.md @@ -836,9 +836,12 @@ delivery for all resources. - `GET /api/admin/reports/catalog` (admin, `settings:read`) - Returns the canonical reporting catalog for the settings surface, including locked-state teaser copy, enabled-surface guidance copy, performance-report options, canonical single-report filename subject, canonical fallback filename date style, and the nested VM inventory export definition. - Metadata route: readable without the `advanced_reporting` feature so locked admin surfaces can render the same reporting definition before upsell. -- `GET /api/admin/reports/generate` (admin, `settings:read`) - - Query params: `format` (pdf/csv, default `pdf`), `resourceType`, `resourceId`, `metricType` (optional), `start`/`end` (RFC3339, optional; defaults to last 24h), `title` (optional) +- `POST /api/admin/reports/generate` (admin, `settings:read`) + - Body fields: `format` (pdf/csv, default `pdf`), `resourceType`, `resourceId`, `metricType` (optional), `start`/`end` (RFC3339, optional; defaults to last 24h), `title` (optional) - If `title` is omitted, the backend applies the canonical default title for that resource report. + - When Pulse Assistant is configured, the configured AI provider narrates a PDF's executive summary; each narration counts against the AI budget and appears in AI usage. If the call fails, times out or the budget is spent, the PDF falls back to the deterministic summary. CSV output is never narrated. The settings UI uses this method. +- `GET /api/admin/reports/generate` (admin, `settings:read`) + - Takes the same fields as query params and returns the same report, except that it never calls an AI provider: a PDF carries the deterministic summary and a note saying AI narration needs `POST`. Other methods return `405` with `Allow: GET, POST`. - `POST /api/admin/reports/generate-multi` (admin, `settings:read`) - Body fields: `resources` (1-50 entries of `{resourceType,resourceId}`), `format`, `metricType` (optional), `start`/`end` (RFC3339, optional; defaults to last 24h), `title` (optional) - If `title` is omitted, the backend applies the canonical default fleet report title. @@ -851,8 +854,8 @@ Validation and limits: - Malformed `start`/`end` values, `end` not strictly after `start`, or report windows over 366 days return `400 invalid_time_range`. - `metricType` must match `[a-zA-Z0-9._:-]+` and be <= 64 chars, otherwise `400 invalid_metric_type`. - `title` must be <= 256 chars, otherwise `400 invalid_title`. -- Multi-report body max size is 1MB; oversized payloads return `400 body_too_large`. -- Multi-report bodies reject trailing payload and unknown JSON fields with `400 invalid_body`. +- Report body max size is 1MB; oversized payloads return `400 body_too_large`. +- Report bodies reject trailing payload and unknown JSON fields with `400 invalid_body`. - VM inventory export only accepts `csv`. Common reporting error codes: @@ -891,7 +894,9 @@ For an existing incident whose notifications should be paused without clearing it, use the snooze and unsnooze endpoints below. - `GET /api/alerts/config` -- `PUT /api/alerts/config` +- `PUT /api/alerts/config` — replaces only the top-level keys the body + carries; a key left out keeps its stored value, and the merged config is + normalized as before. The body must be a JSON object. - `GET /api/alerts/deadman/config` — returns only whether an external watchdog is configured; `pingUrl` is `***REDACTED***` when present and never returns the credential-bearing URL diff --git a/docs/MSP.md b/docs/MSP.md index 7561f520c6..9302b66a1d 100644 --- a/docs/MSP.md +++ b/docs/MSP.md @@ -260,10 +260,12 @@ Each client runtime (or organization) generates its own reports, scoped to that client's resources: - **UI**: Settings → Data & Reports. -- **API**: `GET /api/admin/reports/generate` (single resource) and +- **API**: `POST /api/admin/reports/generate` (single resource) and `POST /api/admin/reports/generate-multi` (up to 50 resources per report), returning PDF or CSV. In shared-process mode, scope with `X-Pulse-Org-ID` - or an org-bound token. + or an org-bound token. `GET /api/admin/reports/generate` with query + parameters still returns a single-resource report, but never adds the + Pulse Assistant narrative. - **Schedules**: `GET`/`POST /api/admin/reports/schedules`, `PUT`/`DELETE /api/admin/reports/schedules/{id}`, and `POST /api/admin/reports/schedules/{id}/run`. Schedules can target explicit diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md index a2ee20031e..21997c6afe 100644 --- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md +++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md @@ -248,6 +248,18 @@ Without comparable identity (a SAS, SCSI or USB replacement with no WWN on one side) the row still matches. Report admission and the SMART wire contract are unchanged; the regression is `TestHostAgentSMARTRowForSwappedOutDiskDoesNotLendItsIdentity`. +A PVE disk that a still-reporting linked Agent's SMART row matches is marked +`AgentSMARTReported` (internal poll evidence on `models.PhysicalDisk`, never +serialized); disks the poller builds from that report when the Proxmox disk +query fails go through the same merge, and an Agent whose lease lapsed owns +none. The Agent then owns the disk's temperature alert +through `CheckHost`, and the PVE disk temperature check closes its own as +moved to the Agent, so one disk never alerts twice. Report admission, the SMART +wire contract and Agent authority are unchanged; +`TestMergeHostAgentSMARTIntoDisksMarksDisksTheAgentReports` in +`internal/monitoring/physical_disk_roundtrip_test.go` and +`TestPhysicalDiskAgentSMARTReportedStaysInternal` in +`internal/models/deepcopy_test.go` pin the marker. Assistant historical metric wiring uses the current monitor's retained store and registry metrics coordinates. Historical reads do not alter enrollment, @@ -3678,6 +3690,14 @@ Agent` secondary handoff against the live setup wizard instead of relying ## Current State +### Alert settings saves do not touch agent lifecycle + +`PUT /api/alerts/config` in `internal/api/alerting/alerts.go` now keeps stored +values for the top-level keys a client leaves out. Agent alert defaults +(`agentDefaults`) follow the same rule as every other key; enrollment, +reporting leases, commands and agent removal are unchanged. +`internal/api/alerting/alerts_test.go` pins the handler. + ### VM guest execution admission (backup precaution) `AgentRegisterPayload.guest_exec_guard_version` is optional; version 1 on a @@ -4712,6 +4732,12 @@ agent-lifecycle operations. If those routes are called directly, the API must fail them as unsupported rather than presenting file diffs or reverts as agent command rollback, enrollment repair, update rollback, or fleet-control authority. +Assistant session mutations (`/abort`, `/summarize`, `/fork`, `/undo`, +`/redo`, `/steer` under `/api/ai/sessions/{id}`) accept only `POST`; any other +method that reaches those handlers gets `405` before touching session state. +That method guard +is AI-runtime transport hardening in `internal/api/ai_handler.go`; it adds no +agent command, enrollment, update, or fleet-control path. That same shared dependency now also assumes hosted cloud handoff authorizes tenant org access before browser lifecycle continues. Lifecycle-adjacent opens into hosted workspaces may depend on `internal/api/cloud_handoff_handlers.go`, @@ -7639,6 +7665,16 @@ bounded and classified temperature nodes, and leaves the host report intact when the helper is absent or incompatible. It does not treat Windows ACPI thermal zones as hardware sensor evidence, accept remote provider locations, or grant Pulse hardware-control authority. +### Single-resource report AI narration requires POST + +`internal/api/metrics_reporting_handlers.go`, `internal/api/router.go` and +`internal/api/ai_handlers.go` changed only how `/api/admin/reports/generate` +reads its fields (a `POST` JSON body or `GET` query parameters), which of those +may use the tenant's AI narrator (`POST` only), and how a `GET` finds Patrol +findings (`ExistingAIService`, which never constructs a tenant AI service). No +agent registration, enrolment, install, update, removal or report ingest path +changed. + ### Agent profile adoption is reported as a count only The usage telemetry snapshot reads `LoadAgentProfiles` and reports @@ -8807,6 +8843,25 @@ in `internal/monitoring/monitor_host_agents_test.go` `internal/monitoring/issue1595_collection_trust_test.go` (`TestAgentDiskHistoryFollowsTheSerialItsUnraidRowReports`). +### Unraid temperatures chart agent disk history + +`internal/monitoring/monitor_agents.go` changed only so the SMART history +writer charts the temperature a host agent's disk shows. A SMART row without +its own reading shows the one the host's Unraid inventory reports for the disk, +when that inventory row describes the disk (`HostSMARTDiskTemperature`), and a +disk only the Unraid inventory reports, +such as a member the agent's SMART collection skips under `--disk-exclude`, +shows that row's reading (`HostUnraidDiskTemperature`). Both are now written +as `smart_temp` under the key the disk's metrics target reads +(`HostSMARTDiskMetricID`, `HostUnraidDiskMetricID`); a reading the disk does +not show as collected, a spun-down disk's or an expired host's, is not. Agent +registration, enrolment, install, update, removal, report identity and disk +source IDs are unchanged. Focused proof lives in +`internal/monitoring/monitor_host_agents_test.go` +(`TestApplyHostReportChartsUnraidTemperatureOfDiskWithoutSMART`) and +`internal/monitoring/issue1595_collection_trust_test.go` +(`TestAgentDiskChartsTheUnraidTemperatureItShows`). + ### Windows braced MachineGuid does not abort agent startup The Windows unified agent resolves host information through gopsutil's combined @@ -8935,3 +8990,11 @@ like any other write instead of always reaching its handler. Agent command websocket handshakes to `/api/agent/ws` are HTTP/1.1 `GET` requests, the only transport its gorilla upgrader serves, and still connect on a demo instance. No agent registration, enrolment, install, update or removal path changed. + +### Demo mode hides the pprof routes + +`internal/api/demo_mode_operations.go` changed only so the demo guard on a +`DEMO_MODE` instance answers `/debug/pprof` and every path below it with `404` +for every method. No agent route lives under `/debug/`, so agent registration, +enrolment, install, update, removal, report ingest and `/api/agent/ws` +handshakes are unchanged on a demo instance. diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.md index 9ab54a9d35..bc4f13bd3f 100644 --- a/docs/release-control/v6/internal/subsystems/ai-runtime.md +++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md @@ -128,6 +128,25 @@ on component mount. Closing clears the registered input so later keyboard commands cannot target a detached composer. A handoff must leave Escape and keyboard input in Assistant, not the underlying alert search. +### Process-wide memory ID sequences + +`internal/ai/memory` mints incident, incident-event, remediation-record and +context-memory IDs from package-level sequences that every store in the process +shares. Multi-tenant monitoring gives each tenant monitor its own +`IncidentStore`, and their alert checks call `RecordAlertFired` from separate +goroutines, while a store's mutex covers only its own state. Each sequence is +therefore an `atomic.Int64` minted through `nextSequencedID` in +`internal/ai/memory/ids.go`, never a plain package counter. The ID shape stays +`-`, but `n` no longer wraps at 1000. The old +`counter%1000` suffix repeated an ID whenever one generator minted more than a +thousand in a wall-clock second, and incident notes resolve an incident by ID +while timeline merges deduplicate events by ID. Within one process IDs are now +unique by construction. Across restarts only the timestamp separates them, so a +repeated local second (DST fall-back or a backward clock step) can still meet a +restarted sequence's suffix. Persisted IDs and the stored format are unchanged. +`internal/ai/memory/ids_test.go` mints from every generator concurrently and +records alerts into two tenant stores concurrently; run it under `-race`. + ### Canonical incident-history queries Incident context reads `IncidentStore.QueryIncidents`, whose page contains @@ -3556,6 +3575,12 @@ query...`, and `Reading storage...` before streamed tool arguments are answer (nothing streaming, no error block, no pending approval or question, and a user prompt available to re-send) because the undo boundary only operates on the last durable turn. + Session mutations stay POST-only at the handler: abort, summarize, fork, + undo, redo, and steer refuse every other method with `405` and + `Allow: POST` before resolving the Assistant service. A `GET` or `HEAD` + that the demo-mode read-only guard or the CSRF check admits as a read + therefore cannot cancel a running turn, compact or rewind a transcript, + or persist a fork. The empty Assistant drawer may surface recent non-empty sessions as direct resume actions using the backend session list already owned by the drawer; it must not create a parallel recent-chat store or product-authored prompt @@ -7510,6 +7535,18 @@ prompt budget. Reporting is therefore an additive consumer of AI runtime, not a new ownership boundary, and the narrator/findings surfaces inherit the same governance the rest of the canonical AI runtime already enforces. +Report narration is caller-requested spend, so on the single-resource HTTP +route only `POST /api/admin/reports/generate` reaches it (fleet reports, from +`POST .../generate-multi` or a report schedule, use the fleet narrator). The +reporting narrator resolver goes through `AISettingsHandler.GetAIService`, +which constructs a tenant service on first use and can then list provider +models or start background discovery. A GET single-resource report never calls +that resolver: +it narrates deterministically and reads Patrol findings through +`AISettingsHandler.ExistingAIService`, which returns only a service that is +already running and never constructs one +(`TestAISettingsHandler_ExistingAIServiceNeverConstructs`, and through the +router's real wiring `TestContract_SingleReportGETDoesNotConstructTenantAIService`). The same canonical AI runtime now also owns the fleet-level report narrative through `report_fleet_narrator.go`. `Service` implements diff --git a/docs/release-control/v6/internal/subsystems/alerts.md b/docs/release-control/v6/internal/subsystems/alerts.md index bc2a9294f4..513b44c27d 100644 --- a/docs/release-control/v6/internal/subsystems/alerts.md +++ b/docs/release-control/v6/internal/subsystems/alerts.md @@ -1114,6 +1114,43 @@ history and affirmative recovery. This is source-fixture lifecycle proof, not an installed registry/destination result or a diagnosis of a reporter's daily resolve/reopen cycle. +### A settings save keeps the alert settings it did not send + +`PUT /api/alerts/config` applies the body to the stored configuration through +`Manager.ApplyConfigUpdate` and `ApplyAlertConfigUpdate` in +`internal/alerts/config/update.go`: a top-level key the client sent replaces +that setting whole, a key it left out keeps the stored value, and the result +is normalized as any update is. The thresholds page has no control for +flapping detection (`flappingEnabled` and its window, threshold and cooldown), +alert TTL cleanup (`maxAlertAgeDays`, `maxAcknowledgedAgeDays`, +`autoAcknowledgeAfterHours`) or `customRules`, so it never sends them. The +handler used to decode the body into a zero `AlertConfig`, so every thresholds +save turned flapping detection and TTL cleanup off, reset the flapping tuning +to its defaults, dropped custom rules, and persisted that. The page's +`flapping`, `aggregation` and `ioNormalization` payload objects are not +`AlertConfig` fields and never enabled anything. A stored config keeps what it +holds, including values an earlier save zeroed; nothing rewrites them. + +The body is decoded on its own, so sent keys mean exactly what the decoder +makes of them, duplicates and case-folded keys included. Unsent fields are +copied from a JSON round trip of the stored config, so the result shares no +maps or slices with it. `ApplyConfigUpdate` reads, merges and applies under +the manager lock, so two concurrent partial saves cannot revert each other in +memory, and returns a JSON-cloned snapshot so that persistence, which +normalizes the config it is handed in place, never writes into the live +config's maps. + +`internal/alerts/config/update_test.go` covers unsent, explicit-off, +case-folded, duplicate-key, malformed and full-body updates, a sent object +replacing its stored value whole (a sent `truenasDiskDefaults: {}` does not +inherit the stored temperature) and the no-sharing rule. +`TestUpdateConfigKeepsSettingsAThresholdsSaveDidNotSend` and +`TestApplyConfigUpdateKeepsConcurrentPartialUpdates` in +`config_validation_test.go` run saves through the manager, and +`TestUpdateAlertConfig_KeepsStoredValuesForUnsentKeys` in +`internal/api/alerting/alerts_test.go` drives the handler against a real +manager. + ### Confirmed empty storage is recovery evidence Static storage capacity evaluation must admit a zero usage observation when @@ -1473,7 +1510,8 @@ That same guest-threshold owner also governs guest-derived lifecycle and posture alerts. Snapshot age, backup age, powered-off state, and configuration-change reevaluation must all construct a canonical lightweight guest snapshot and route threshold resolution through the shared -guest-defaults → filter-driven custom rules → guest-override chain. +guest-defaults → filter-driven custom rules → guest-override → `pulse-relaxed` +chain. That canonical guest context must preserve the live guest name and tags for snapshot and backup posture evaluation. Ignored prefixes, `pulse-no-alerts`, configured ignored tags, and required-tag filtering must resolve through the @@ -1483,6 +1521,27 @@ operator's suppression policy. Passing `nil` guest context or resolving only overrides/defaults is forbidden because it silently bypasses custom guest rules and makes guest lifecycle alerting diverge from running-guest metric truth. +The `pulse-relaxed` step lives in `effectiveAlertPolicyNoLock`, not in +`CheckGuest`, so polling and config-save reevaluation judge a relaxed guest's +alert by one threshold. Reevaluation rebuilds the guest from alert metadata, +so `guestSnapshotFromAlert` reads back the `tags` the evaluator recorded +(`[]string` live, `[]interface{}` after a restart). Without them a save judged +the alert by unrelaxed thresholds, resolved one raised under an unset +threshold's relaxed floor with a recovery notification, and the next poll +raised it again. Those tags must stay current: `metricOptionsWithTags` drops +the key from an open alert once its resource has no tags, or a removed +`pulse-relaxed` would keep judging the alert by the relaxed trigger. The tag +lifts CPU, memory and disk triggers to a 95/92/95 floor and gives an unset +threshold that floor, but an Off threshold (trigger <= 0) stays Off: relaxing +a guest must never raise an alert its config turned off. A filesystem's own +override still replaces the relaxed guest disk threshold for that filesystem, +on save as while polling. Proof: +`TestConfigSaveJudgesRelaxedGuestAlertByRelaxedThresholds`, +`TestConfigSaveKeepsRelaxedGuestFilesystemOverride`, +`TestPulseRelaxedKeepsOffGuestThresholdOff` and the Off case of +`TestApplyRelaxedGuestThresholds` in `internal/alerts/alerts_test.go`, and +`TestCheckGuestDropsRemovedTagsFromOpenAlerts` in +`internal/alerts/unified_eval_test.go`. That same guest-alert owner also has to retire per-disk guest alerts when the guest stops, disk alerting is disabled, or the reported disk set changes. Canonical guest disk identity is only valid while the guest still exposes that @@ -1851,14 +1910,84 @@ A disk is hot from its trigger. Alerts and Patrol findings stay open until the reading falls to the clear value, or under the trigger when there is no band below it. The Physical Disks verdict and the TrueNAS Health cell judge only the current reading against the trigger. PDF reports colour disk temperatures by -the same thresholds. Judges that still differ: TrueNAS disk temperature alerts -use `TrueNASDiskDefaults.Temperature` (a flat 55/50), and the TrueNAS disk -drawer tones the reading from a fixed 55C. +the same thresholds. `TestDiskTemperatureThresholdMatchesCheckHostPolicy` in `internal/alerts/threshold_resolution_shared_test.go` pins per-type resolution, a raised NVMe trigger that `CheckHost` also honours, the copy, the disabled default and the nil manager. +TrueNAS disk temperature alerts follow the policy too: +`trueNASDiskTemperatureDefaultNoLock` +(`internal/alerts/truenas_disk_temperature.go`) is the type-default tier of +`effectiveAlertPolicyNoLock` for `truenas-disk`, so an unset +`TrueNASDiskDefaults.Temperature` resolves per disk type (the +`alertPolicyQuery.DiskType` the unified input carries from +`PhysicalDisk.DiskType`). A TrueNAS-wide value the user saved under TrueNAS +Disks replaces it for every TrueNAS disk, as a host Disk Temp override does for +an agent's disks, 0 meaning off, and a per-disk override beats both. The alert +records `diskType` so a config save re-judges it per type; one without it is +held to the lowest per-type trigger. A retained reading +(`diskinventory.TemperatureCollected` false) is no evidence and raises nothing. +The factory value is unset. `normalizeTrueNASDiskTemperature` drops a stored +55/50 only from a config without `TrueNASDiskTemperatureByType`, because the +thresholds page wrote that flat factory value back on every save; the page +writes the marker, so a 55 typed later is kept. Every config `GetConfig` +returns carries the marker, so a client that reads and writes back keeps it. +Without the marker (an old saved file, or a page loaded before the upgrade) +55/50 is read as that unchosen factory value; a client building a config from +scratch with an explicit TrueNAS-wide 55 has to send the marker. The TrueNAS +Disks Global Defaults cell reads `By type` while unset +(`globalDefaultFallbacks` on `ResourceTable`, +`resolveAlertResourceGlobalDefaultCell`), or Off when the agent Disk Temp +default switches the policy off, and switching it on then stages an explicit +TrueNAS-wide 55 because unset would stay off. Each disk row inherits its type's +trigger from the unsaved editor state (`resolveTrueNASDiskTemperatureDefault`). +The TrueNAS disk drawer tones a current reading from the same per-type trigger +(`getDiskTemperatureThresholds` passed into `buildTrueNASDetailSections`), so it +no longer judges heat from a fixed 55C. Judges that still differ: the TrueNAS +storage table and drawer do not apply a TrueNAS-wide value or per-disk +override. +`TestTrueNASDiskTemperatureAlertsFollowDiskTemperaturePolicy` and +`TestTrueNASDiskTemperatureAlertIgnoresRetainedReading` in +`internal/alerts/unified_eval_test.go` pin the tiers and retained readings; +`TestTrueNASDiskTemperatureAlertsReevaluatePerDiskType` and +`TestTrueNASDiskTemperatureAlertWithoutDiskTypeHeldAtLowestTrigger` in +`internal/alerts/alerts_test.go` pin re-judging on a config save; +`TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues` pins the +saved-config migration. + +Proxmox physical disks alert on temperature under the same policy. +`CheckProxmoxDiskTemperature` (`internal/alerts/proxmox_disk_temperature.go`) +raises a `diskTemperature` metric alert under the disk's PVE alert reference +(`ProxmoxPhysicalDiskAlertResourceID`, resource type `proxmox-disk`), judged by +`proxmoxDiskTemperatureThresholdNoLock`: the disk type's `DiskTempByType` entry +under the agent Disk Temp default, which switches it off for every type. It +carries the disk's serial, WWN and `diskType` like the health and wearout +alerts, so `ProxmoxPhysicalDiskAlertIdentifiers` lists its +`::metric-threshold:diskTemperature` identifier for history +ownership, and `reevaluateActiveAlertsLocked` re-judges it per disk type (the +lowest per-type trigger when the type was never recorded) instead of falling +through to guest thresholds that have no disk temperature. A disk a +still-reporting linked agent lists in its SMART report is the agent's: its +alert closes with the `moved_to_agent` resolution and `CheckHost` raises the +agent's own. The handover happens on the next disk poll, so for that one +interval an agent newly linked to a disk with an open PVE alert can hold its +own alert beside it. A lapsed agent owns nothing, so the node's own readings +keep alerting while its agent is silent. An +excluded disk, or a policy switched off, closes the alert; a poll without a +current reading holds it. Like the health and wearout alerts, a disk that +leaves the inventory keeps its alert until the 24-hour stale sweep. The +`proxmox-disk` alert delay defaults to the factory 5 seconds, so a warning +waits out the same 300-second noisy-gauge stability window agent and TrueNAS +disk temperature warnings do, and a critical reading bypasses it. +`TestProxmoxDiskTemperatureAlertsFollowDiskTemperaturePolicy`, +`TestProxmoxDiskTemperatureAlertOwnershipAndHolds`, +`TestProxmoxDiskTemperatureWarningWaitsOutStabilityWindow`, +`TestProxmoxDiskTemperatureAlertsReevaluatePerDiskType` and +`TestCheckDiskHealthAlertsCarryHistoryOwnershipIdentity` in +`internal/alerts/alerts_test.go` pin the policy, ownership, holds, re-judging +and identifiers. + ### Agent disk temperature alerts clear when their disk leaves the report `CheckHost` also clears a disk temperature alert, and drops its pending @@ -1905,6 +2034,61 @@ alert. `shouldPreserveAlertOutsideNodeCleanup` now keeps any alert carrying `internal/alerts/alerts_test.go` pins both signals, and its sibling pins that a guest alert on a removed Proxmox node is still removed. +### Agent disk temperature alerts judge the reading the disk shows + +`CheckHost` judges each agent disk on the temperature, collection state and +disk type the disk shows, read by `unifiedresources.HostDiskTemperatureReadings` +from the same adapters the registry builds the disks with. A SMART row shows its +own reading or, when the agent's probe returned none, the one the host's Unraid +inventory reports for that disk (`HostSMARTDiskTemperature`), and keeps its +`agent:/disk_temp:` resource. An Unraid inventory row whose disk +key no SMART row carries, a disk only the inventory lists, shows that row's +reading (`HostUnraidDiskTemperature`) and its own transport as disk type, under +its kernel block device token, the resource a plain Linux SMART row for the +same device uses. A SMART row and an Unraid row with one key are one registry +disk even under different device labels, such as a controller member +(`0 [megaraid,0]`) and its Unraid device (`sda`), and alert once, under the +SMART row's resource. When smartctl and Unraid name +one device with different serials the registry shows two disks on it, and the +device's one alert judges the reading that stands highest against its own disk +type's threshold (`judgedHostDiskTemperature`): over its trigger first, then +inside its recovery band, then the larger margin over the trigger, with a tie +keeping the reading listed first. +Before this `CheckHost` read only the SMART row's own field, while the Physical +Disks Temp cell, the Running Hot verdict ("at or above its alert threshold") +and Patrol judged the Unraid reading against the same trigger, so such a disk +could read as at its alert threshold with no alert. Unraid's own warning and +critical disk temperature notifications do not make this redundant: Pulse +already alerts on the smartctl readings and array state of the same hosts, +which Unraid also notifies on, and a user who routes alerts through Pulse sets +these thresholds here. + +Only a reading collected now fires or resolves +(`HostDiskTemperatureReading.Collected`: no standby row, and +`diskinventory.TemperatureCollected`), so a spun-down disk holds the alert +like a listed disk without a temperature. A disk only the +inventory lists whose device matches the agent's `--disk-exclude` patterns +(`fsfilters.MatchesDeviceExclude`) raises nothing, and an alert it raised before +the exclusion resolves on the next report (`clearHostDiskTemperatureResources`), +whatever the SMART list holds, unless another listed reading uses the same +resource: `docs/UNIFIED_AGENT.md` documents exclusion as +removing the disk from monitoring, and linked Proxmox hosts already judge an +excluded disk healthy. A disk only the inventory lists is also judged in a +report whose SMART list is empty, because the disk shows the inventory's +reading in that report. The absence cleanup above still runs only on a +non-empty SMART list, and such a report neither counts toward nor restarts a +disk's three, so a disk that leaves the inventory while every report's SMART +list stays empty keeps its alert until the 24-hour stale-alert sweep. +`TestCheckHostJudgesTheUnraidTemperatureADiskShows` and +`TestCheckHostHoldsUnraidDiskTemperatureAlertUntilAReadingOrDeparture` in +`internal/alerts/host_unraid_lifecycle_test.go` pin the fallback, the +inventory-only disk, the empty SMART list, two disks on one device (also of +different types), a controller member under another device label, standby, +exclusion, the disk type +and host override thresholds, the holds, the handover to a SMART reading, +departure, and the exclusion clear with and without SMART rows and beside a +listed SMART row. `TestJudgedHostDiskTemperatureOutranks` pins the ranking. + ### Configured flapping thresholds remain reachable Every accepted positive `FlappingThreshold`, including values above ten, must @@ -3180,6 +3364,76 @@ ownership is `internal/alerts/alert_identity_migration_test.go`, `internal/monitoring/monitor_alert_override_migration_test.go`, and `frontend-modern/src/features/alerts/__tests__/alertsConfigurationModel.snapshot.test.ts`. +### Alert config snapshots are owned copies + +`Manager.GetConfig()` returns `AlertConfig.Clone()`, a typed deep copy that +shares no map, slice or pointer with the live config (custom rule filter +values are copied as decoded JSON values). Callers encode, persist +and edit that snapshot outside `m.mu`: the GET config handler, alert +activation, the identity migration, backup evaluation and diagnostics. It used +to be a shallow copy, so `SaveAlertConfig` normalizing it in place, or +`UpdateConfig` re-normalizing the overrides of a re-applied snapshot, wrote the +live maps while evaluation or another encoder read them, and Go aborted Pulse +with a concurrent map write. `SaveAlertConfig` also clones before normalizing, +because a config just handed to `UpdateConfig` is the live config. Persistence +keeps an explicit `0` type-level delay as no delay on save and load, matching +`NormalizeTimeThresholds`. It used to store and load it as the 5-second factory +delay, which the noisy-gauge rule stretches to 300 seconds for memory and +temperature, and the shared map write was what made the running manager agree. +Pulse saves never wrote a `0`, so only a newly sent or hand-edited `0` changes +meaning. `internal/alerts/config/clone_test.go` +fills every field and fails when a new reference field is not copied. +Regression ownership is `internal/alerts/resolved_lock_discipline_test.go` and +`internal/config/persistence_alert_ownership_test.go`. + +### Alert evaluation owns the thresholds it normalizes + +Host, guest and Docker container evaluation copy what they read from +`m.config` while holding `m.mu`, and only then fill in defaults. +`CheckHost` and `CheckGuest` clone a per-disk +override's `Disk` threshold before `ensureHysteresisThreshold` sets a missing +`Clear` to the trigger minus 5. They used to normalize the live override's +pointer after unlocking, so a `GetConfig()` clone under the read lock raced +the write, and the saved override changed to whatever the last poll filled in. +The live override now keeps the clear it was saved with, and the config API +returns that value. Docker container evaluation resolves its thresholds from +cloned `DockerDefaults` plus the override under the same lock, and reads the +restart-loop and memory-limit settings under the lock. It used to point into +`m.config.DockerDefaults` with no lock while `UpdateConfig` replaced +`m.config`. The test-only `getThresholdForMetricFromConfig`, which normalized +in place, is gone. `internal/alerts/alerts_test.go` +`TestAlertEvaluationLeavesTheLiveConfigToTheLock` runs host, guest and Docker +evaluation against `GetConfig()` and `UpdateConfig(GetConfig())` under +`-race`. + +### Config-save reevaluation judges a filesystem alert by its own threshold + +`UpdateConfig` re-judges every active alert, and a filesystem usage alert's +threshold resolves through the same per-filesystem helper its evaluator uses. +`hostDiskUsageThresholdNoLock` resolves an agent filesystem +(`agent:/disk: