mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 04:33:17 +00:00
Judge disk heat by the alert disk temperature thresholds
Pulse decided whether a physical disk was too hot in three places with three thresholds. Disk risk added temperature_high at a flat 60/70C for every disk type and drove the disk status and the Physical Disks Health verdict. Patrol flagged anything over 55C and the AI chat context listed disks from 50C. Alerts and the Temp column used the per-type thresholds users tune in Alerts (NVMe 70, SAS 65, SATA 55 by default). An NVMe at 63C read Running Hot beside a green Temp cell. A SATA disk at 56C read Healthy beside a red one. A user who raised the NVMe trigger to 75 still got a hot verdict at 70. The alert disk temperature policy now decides disk heat. alerts.Manager.DiskTemperatureThreshold exposes the per-type resolution CheckHost already applies. - Disk risk no longer judges temperature. Every registry that rebuilds risk lacks the alert configuration, so no rule there could honour it. A hot disk keeps an online status. - The Health verdict reads Running Hot when a current reading reaches the disk type's trigger. It uses the thresholds object that colours the Temp cell, so the two cannot disagree. The amber band below the trigger stays a cell colour, not a verdict. - The TrueNAS storage table lost the heat it used to get from risk. It now judges heat the same way: an Attention row with the reason, and a warning dot for phones. - Patrol triage, its disk issue filter, finding recovery and the chat context's attention list judge the collected reading against the same trigger, including scoped runs and a provider set before Patrol starts. - A switched-off agent Disk Temp default resolves to no thresholds, which the displays that read them now treat as off. Per-host Disk Temp overrides do not reach these surfaces yet. TrueNAS disk alerts, the TrueNAS disk drawer and PDF reports keep their own fixed temperatures for now.
This commit is contained in:
rcourtman
@@ -306,6 +306,22 @@ retained value as "last known 41C (disk is in standby)". Proofs:
|
||||
`TestPatrolPhysicalDiskRowsJudgeHeatOnCollectedTemperature` in
|
||||
`internal/ai/patrol_ai_more_test.go`.
|
||||
|
||||
They judge that collected reading against the alert disk temperature policy,
|
||||
not a fixed line. `ThresholdProvider.GetDiskTemperatureThreshold(diskType)`
|
||||
(the alert manager's per-type trigger and clear) resolves each disk's limits
|
||||
once in `patrolPhysicalDiskRows`, and the run state carries the provider
|
||||
(`patrolRuntimeState.thresholdProvider`). With no provider the factory alert
|
||||
configuration applies. A disk is hot from its trigger: a triage warning, a disk
|
||||
issue and an AI chat "needing attention" entry, as its temperature alert and
|
||||
Physical Disks Running Hot verdict are. A disk-high finding recovers at or
|
||||
below the clear value (under the trigger when there is no band below it), as
|
||||
the alert does. Scoped runs keep the provider, and a
|
||||
provider set before Patrol starts is handed to it. Below the trigger Patrol flags nothing, so an NVMe at 63C stays
|
||||
quiet and a SATA disk at 56C is flagged. Proofs:
|
||||
`internal/ai/patrol_disk_temperature_test.go` and
|
||||
`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` in
|
||||
`internal/ai/alert_threshold_adapter_test.go`.
|
||||
|
||||
Retained summaries disclose that point and bucket timestamps describe returned
|
||||
history inside the requested window. Their spacing does not measure collection
|
||||
uptime or explain missing history. Collector lifecycle and retention settings
|
||||
|
||||
@@ -1682,6 +1682,35 @@ an alert the next evaluation raises again.
|
||||
`TestConfigSaveKeepsAlertTheEvaluatorKeepsWithoutRecoveryBand` in
|
||||
`internal/alerts/host_unraid_lifecycle_test.go` pin these paths.
|
||||
|
||||
### The disk temperature policy decides disk heat
|
||||
|
||||
The policy `CheckHost` applies to a SMART disk (the agent Disk Temp default,
|
||||
refined by the disk type's `DiskTempByType` entry, and switched off for every
|
||||
type when that default is off) is the canonical answer to "is this disk too
|
||||
hot". `diskTemperatureThresholdForType` in `internal/alerts/host.go` holds it,
|
||||
and `Manager.DiskTemperatureThreshold(diskType)` exposes it, without host
|
||||
overrides, to Patrol through `AlertThresholdAdapter.GetDiskTemperatureThreshold`.
|
||||
`DefaultDiskTemperatureThreshold` serves callers with no manager. Disk risk
|
||||
(`internal/storagehealth`) judges no temperature. The Physical Disks Temp cell
|
||||
and Health verdict mirror the per-type triggers and a switched-off default in
|
||||
`resolveDiskTemperatureDisplayThresholds`; host overrides do not reach that
|
||||
page yet. That resolver returns null when the agent Disk Temp default is off.
|
||||
The displays that read it treat null as off: no threshold colour
|
||||
(`getTemperatureTextClass` with `diskTemperature`) and never hot. A
|
||||
`DiskTempByType` entry cannot be switched off on its own, because
|
||||
normalization restores a non-positive entry to its default.
|
||||
A disk is hot from its trigger. Alerts and Patrol findings stay open until the
|
||||
reading falls to the clear value, or under the trigger when there is no band
|
||||
below it. The Physical Disks verdict and the TrueNAS Health cell judge only the
|
||||
current reading against the trigger. Judges that still differ: TrueNAS disk
|
||||
temperature alerts use `TrueNASDiskDefaults.Temperature` (a flat 55/50), the
|
||||
TrueNAS disk drawer tones the reading from a fixed 55C, and PDF reports colour
|
||||
disk temperatures at a fixed 50/60C.
|
||||
`TestDiskTemperatureThresholdMatchesCheckHostPolicy` in
|
||||
`internal/alerts/threshold_resolution_shared_test.go` pins per-type resolution, a raised
|
||||
NVMe trigger that `CheckHost` also honours, the copy, the disabled default and
|
||||
the nil manager.
|
||||
|
||||
### Agent disk temperature alerts clear when their disk leaves the report
|
||||
|
||||
`CheckHost` also clears a disk temperature alert, and drops its pending
|
||||
|
||||
@@ -414,8 +414,9 @@ available. Stable disk History catalog/organisation/access ownership is unchange
|
||||
|
||||
The physical-disk verdict words and their phone forms are `Needs Attention`
|
||||
(`Attention`), `Running Hot` (`Hot`) and `Replace Now` (`Replace`), all from
|
||||
`getPhysicalDiskHealthCompactLabel` in `diskPresentation.ts`. A hot disk keeps
|
||||
the same label at either tier; its tone carries the severity.
|
||||
`getPhysicalDiskHealthCompactLabel` in `diskPresentation.ts`. `Running Hot` is
|
||||
red and starts at the disk's alert trigger, from the same thresholds object
|
||||
that colours its Temp cell (`getPhysicalDiskTemperatureThresholds`).
|
||||
|
||||
Controller expansions carry the supplied kind-specific targets, absolute times,
|
||||
duration and cluster/namespace when the phone row omits those columns. Proxmox
|
||||
|
||||
@@ -4501,17 +4501,18 @@ emit structured storage topology such as Unraid per-disk state, the shared
|
||||
assessment layer must derive canonical risk and alert severity from that
|
||||
richer disk topology instead of letting coarser aggregate counters override it
|
||||
and flap the operator-facing storage alert surface.
|
||||
That shared assessment judges only a disk temperature that was collected now.
|
||||
`storagehealth.CollectedTemperature` passes a reading through when its
|
||||
`collection.temperature` state is `available`, or when the source predates
|
||||
collection state, and otherwise drops it before `AssessPhysicalDisk` and
|
||||
`AssessHostSMARTDisk` build their sample. A last known value retained for a
|
||||
disk in standby or a host agent past its reporting lease therefore raises no
|
||||
`temperature_high` reason and cannot turn the disk's verdict or status into
|
||||
a warning, while the value itself stays on the resource as last-known
|
||||
history. Proof: `TestDiskAssessmentsIgnoreRetainedTemperature` and
|
||||
`TestCollectedTemperatureKeepsOnlyCurrentReadings` in
|
||||
`internal/storagehealth/risk_test.go`.
|
||||
That shared assessment never judges disk temperature. Heat is a metric owned
|
||||
by the alert disk temperature policy (`alerts.Manager.DiskTemperatureThreshold`:
|
||||
the per-type `diskTempByType` trigger, else the agent Disk Temp default), which
|
||||
users tune per disk type. Disk risk is rebuilt by registries that cannot see
|
||||
that configuration, so a heat rule here would contradict it: a flat 60/70C rule
|
||||
once called a healthy 63C NVMe hot and a 56C SATA disk fine. No disk type,
|
||||
temperature or collection state reaches `storagehealth.Sample`, the TrueNAS
|
||||
provider and the Unraid inventory assessment included, and a hot disk keeps an
|
||||
`online` status. Proof: `TestDiskAssessmentsNeverJudgeTemperature` in
|
||||
`internal/storagehealth/risk_test.go` and
|
||||
`TestRecordsLeaveDiskHeatToTheAlertPolicy` in
|
||||
`internal/truenas/provider_test.go`.
|
||||
That same monitoring-owned storage polling boundary also owns cluster-shared
|
||||
Proxmox storage status coherence. `internal/monitoring/monitor_polling_storage.go`
|
||||
must merge shared storage observations across nodes into one cluster-scoped
|
||||
|
||||
@@ -6759,23 +6759,33 @@ not live appliance collection or recovery success.
|
||||
|
||||
### Physical-disk heat is not a replacement verdict
|
||||
|
||||
The Physical Disks Health verdict names the action it asks for. Risk whose only
|
||||
reason is `temperature_high` reads `Running Hot` (compact `Hot`), red at the
|
||||
critical tier and amber at warning, with the temperature reason as its summary.
|
||||
`Replace Now` stays reserved for FAILED health, any other critical reason, and
|
||||
a critical level that no listed reason explains. `Needs Attention` keeps the
|
||||
remaining warning evidence. When heat and other evidence both apply, the more
|
||||
severe class names the verdict and failure evidence wins a tie, whatever order
|
||||
merged reasons arrive in. Sorting follows the verdict (replace, critical heat,
|
||||
attention, warning heat), and the health filter places a hot disk at its
|
||||
temperature tier, so `Needs attention` still lists it. This matches alerting,
|
||||
which raises disk heat as a Temperature metric and never as disk health.
|
||||
Thresholds are unchanged: the tier still comes from the server's disk risk
|
||||
(flat 60/70C), not the per-type alert thresholds that colour the Temp column.
|
||||
Reconciling those is separate work. `frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts`
|
||||
and `frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx` cover
|
||||
the split, merged reason order, the unexplained critical level, sorting and
|
||||
filtering.
|
||||
The Physical Disks Health verdict names the action it asks for. A disk whose
|
||||
current temperature has reached its alert trigger reads `Running Hot` (compact
|
||||
`Hot`), red, with the reading and the trigger as its summary. `Replace Now`
|
||||
stays reserved for FAILED health, any critical risk reason, and a critical
|
||||
level that no listed reason explains. `Needs Attention` keeps the remaining
|
||||
warning evidence. Heat outranks warning evidence and failure evidence outranks
|
||||
heat. Sorting follows the verdict (replace, heat, attention), and the health
|
||||
filter places a hot disk with the critical disks, as its Temp cell reads red.
|
||||
|
||||
Heat has one owner: the alert disk temperature policy (the per-type triggers in
|
||||
Alerts > Thresholds, `diskTempByType`, else the agent Disk Temp default).
|
||||
Disk risk never carries heat, because every registry that rebuilds it lacks the
|
||||
alert configuration. `extractPhysicalDiskPresentationData` takes the alerts
|
||||
store's `getDiskTemperatureThresholds` resolver, so the Temp cell colour and the
|
||||
verdict read one thresholds object. The band below the trigger is a Temp cell
|
||||
colour only, and a retained (not current) reading is never heat. The verdict
|
||||
judges the current reading, with no memory of an alert. A disk temperature
|
||||
alert held between its clear value and trigger shows an amber Temp cell, not
|
||||
`Running Hot`. A switched-off agent Disk Temp default resolves to null, which
|
||||
leaves the reading uncoloured and never hot, as alerting is off for it.
|
||||
Per-host Disk Temp overrides do not yet reach Physical Disks.
|
||||
`frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts`
|
||||
pins cell-and-verdict agreement for every type from 40C to 80C under factory and
|
||||
raised triggers, plus retained readings, failure evidence, sorting and
|
||||
filtering. `frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx`
|
||||
covers NVMe 63C, SATA 56C and a raised NVMe trigger loaded through the alerts
|
||||
store.
|
||||
|
||||
### Pool-to-physical-disk ownership in Storage details
|
||||
|
||||
|
||||
@@ -1111,6 +1111,15 @@ The first reason renders in the desktop Health column on the row's single line,
|
||||
coloured by bucket, with a count for the rest and the full list in the title
|
||||
and screen-reader text. Phones keep the status dot and reach the reason through
|
||||
the row drawer. Health sorts by severity rank, not label text.
|
||||
Disk risk carries no heat, so the table judges it itself. A disk whose current
|
||||
reading has reached its type's alert disk temperature trigger is an Attention
|
||||
row with the reason "Disk temperature is 57°C, at or above its 55°C alert
|
||||
threshold.". The trigger comes from the alerts store's
|
||||
`getDiskTemperatureThresholds`, which the table passes to the row build,
|
||||
filter, counts, sort and issue. `isPhysicalDiskRunningHot` in
|
||||
`features/storageBackups/diskTemperaturePresentation.ts` makes the same call
|
||||
for the Physical Disks verdict. The status dot keeps the source's own state.
|
||||
`truenasPageModel.test.ts` and `TrueNASStorageTopologyTable.test.tsx` pin it.
|
||||
The vSphere Datastores and Networks tables follow the same exception-first
|
||||
Health column. `getVmwareDatastoreIssue` and `getVmwareNetworkIssue` return
|
||||
nothing for a green row; otherwise the reasons are, in order, an impaired
|
||||
@@ -3564,8 +3573,8 @@ and otherwise `available`. The value is kept as last-known context; it is
|
||||
collected again only when a reporting host sends a positive reading for a disk
|
||||
that is no longer spun down. The Unraid-native disk row carries that state, and
|
||||
so does a SMART row that falls back to the inventory reading because it has
|
||||
none of its own. The adapter's risk assessment of an Unraid row no longer counts
|
||||
a withdrawn inventory temperature as heat.
|
||||
none of its own. The adapter's risk assessment of an Unraid row judges no
|
||||
temperature at all; heat belongs to the alert disk temperature policy.
|
||||
|
||||
Cross-source correlation compares normalized serial and WWN values across
|
||||
fields without truncation, allowing a PVE bare-hex array-volume serial to join
|
||||
@@ -4111,13 +4120,13 @@ incidents during cross-source merges. A provider alert such as TrueNAS
|
||||
`truenas_smart` is not presentation-only context; it must become a canonical
|
||||
`physicalDisk.risk.reasons` entry so hybrid agent/API disk resources keep one
|
||||
shared disk-health truth after deduplication.
|
||||
That shared risk contract judges only a collected temperature. Both the
|
||||
adapter assessment and the registry's recompute from merged metadata
|
||||
(`physicalDiskAssessmentFromMeta`) pass the temperature through
|
||||
`storagehealth.CollectedTemperature`, so a retained `unavailable`,
|
||||
`unsupported` or `missing` temperature stays visible as last-known evidence
|
||||
but adds no `temperature_high` reason and leaves the disk `online` rather
|
||||
than `warning`. Proof: `TestPhysicalDiskRiskIgnoresRetainedTemperature` in
|
||||
That shared risk contract never judges temperature, collected or retained.
|
||||
Heat belongs to the alert disk temperature policy, which no registry can see,
|
||||
so neither the adapter assessment nor the registry's recompute from merged
|
||||
metadata (`physicalDiskAssessmentFromMeta`) passes a temperature into
|
||||
`storagehealth`. A 72C disk keeps its reading on the resource, adds no
|
||||
`temperature_high` reason and stays `online`. Proof:
|
||||
`TestPhysicalDiskRiskNeverJudgesTemperature` in
|
||||
`internal/unifiedresources/registry_test.go`.
|
||||
That same canonical disk contract now also owns recent aggregate temperature
|
||||
history. When a provider such as TrueNAS can supply `disk.temperature_agg`
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
{
|
||||
"version": 1,
|
||||
"base_sha": "1b6a9b45d7a85ae2add1ff4213ef80f6b35f7309",
|
||||
"verified_at": "2026-10-06T22:49:34Z",
|
||||
"result": "passed",
|
||||
"changed_paths": [
|
||||
"frontend-modern/src/components/Storage/DiskList.tsx",
|
||||
"frontend-modern/src/components/Storage/useDiskDetailModel.ts",
|
||||
"frontend-modern/src/components/Storage/useDiskListModel.ts",
|
||||
"frontend-modern/src/components/Workloads/GuestPhysicalDisks.tsx",
|
||||
"frontend-modern/src/features/storageBackups/diskDetailPresentation.ts",
|
||||
"frontend-modern/src/features/storageBackups/diskPresentation.ts",
|
||||
"frontend-modern/src/features/storageBackups/diskTemperaturePresentation.ts",
|
||||
"frontend-modern/src/features/truenas/TrueNASStorageTopologyTable.tsx",
|
||||
"frontend-modern/src/features/truenas/truenasPageModel.ts",
|
||||
"frontend-modern/src/utils/metricThresholds.ts",
|
||||
"frontend-modern/src/utils/temperature.ts"
|
||||
],
|
||||
"content_sha256": {
|
||||
"frontend-modern/src/components/Storage/DiskList.tsx": "9fbcdb4659cfdf443f521c9b15969f485876cbdde8d7503110e12325b1899538",
|
||||
"frontend-modern/src/components/Storage/useDiskDetailModel.ts": "51dea57998a0c0d88f4a8d6410ff16dc0559674cfecf456b61706bac145321af",
|
||||
"frontend-modern/src/components/Storage/useDiskListModel.ts": "778db34c9def3a921e4bb6f9a36a79dcaa804962e9fb5bc3c03d908c585d8503",
|
||||
"frontend-modern/src/components/Workloads/GuestPhysicalDisks.tsx": "ef649564bd14366aa71121e9c09ac4309bcd299fd1d388ceff5d5969671ae471",
|
||||
"frontend-modern/src/features/storageBackups/diskDetailPresentation.ts": "dac5b69aa5c603b719946fc3262d810a5a9bdc86469f33c3097542516f73d5cd",
|
||||
"frontend-modern/src/features/storageBackups/diskPresentation.ts": "4210a0e7d61113f774462571700582d2f32cfe4474f04048e03d728272bd2247",
|
||||
"frontend-modern/src/features/storageBackups/diskTemperaturePresentation.ts": "f983ea771d8f21ab018ca38a802aa7fa9795860c0ff010adcb7507cf9c47678c",
|
||||
"frontend-modern/src/features/truenas/TrueNASStorageTopologyTable.tsx": "168af9011efa41fc545dfb198f6eff5e9d5dbb003a0cd330c1238a820979c6b9",
|
||||
"frontend-modern/src/features/truenas/truenasPageModel.ts": "36497cd8a519e3de9564eefb4b1a245fac6b0478c9f9a44a262c24228fc33dc9",
|
||||
"frontend-modern/src/utils/metricThresholds.ts": "227994b089456c54489d64c90d3a3111421ec0c1b13adddc2744ab2d3749cfc2",
|
||||
"frontend-modern/src/utils/temperature.ts": "a67d4ec80af39b6057ed155dc8ce5ce4c7e697345f7b9fadb798dcbd31605f9f"
|
||||
},
|
||||
"routes": ["/proxmox/storage?tab=disks&q=West%20Production", "/truenas/storage"],
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900
|
||||
},
|
||||
{
|
||||
"width": 390,
|
||||
"height": 844
|
||||
}
|
||||
],
|
||||
"states": [
|
||||
"Private mock stack built from this exact staged content (embedded UI and backend from one tree). A local-only generator overlay, never committed, pinned Proxmox disks from a control file: Samsung 990 PRO NVMe 63C, Crucial MX500 SATA 56C, WD Black SN850X NVMe 72C, Seagate IronWolf SATA 52C, plus TrueNAS mock disk sda (SATA) at 57C. The API returned no physicalDisk.risk and status online for every pinned disk",
|
||||
"Same-session control on a second private stack built from the base without this change (main plus #2623): NVMe 63C read 'Running Hot' (amber) beside a green 63C cell with risk warning [temperature_high] and status warning; SATA 56C read 'Healthy' beside a red 56C cell; with the NVMe trigger raised to 75 the NVMe 72C still read 'Running Hot' from the flat 70C rule beside an amber cell",
|
||||
"After, factory thresholds, 1440: Crucial MX500 56C 'Running Hot' red, title 'Disk temperature is 56\u00b0C, at or above its 55\u00b0C alert threshold.', red 56C cell, sorted first; WD Black 72C 'Running Hot' red with its 70C threshold; Samsung 990 PRO 63C 'Healthy' with a green cell; Crucial MX500 51C 'Healthy' with an amber cell (the band below the trigger stays colour only)",
|
||||
"After, factory thresholds, 390: same rows with compact 'Hot' in red beside red cells, 'Healthy' beside the green 63C NVMe; no horizontal overflow (scrollWidth 390)",
|
||||
"After, NVMe trigger raised to 75/70 through PUT /api/alerts/config: WD Black NVMe 72C is no longer Running Hot (its mock SMART health is UNKNOWN so it reads 'Unknown') and its cell is amber; SATA 56C still Running Hot",
|
||||
"After, agent Disk Temp default switched off (trigger 0) through PUT /api/alerts/config: no disk reads Running Hot and every Temp cell is uncoloured (text-base-content); TrueNAS sda reads healthy with an Online dot",
|
||||
"After, disk drawer for the 56C SATA disk: header 'Running Hot Disk temperature is 56\u00b0C, at or above its 55\u00b0C alert threshold.' and the Temperature card reads 56\u00b0C",
|
||||
"After, TrueNAS Storage at 1440: sda (57C) is an Attention row with Health 'Disk temperature is 57\u00b0C, at or above its 55\u00b0C alert threshold.'; the Attention status filter lists it; other disks healthy and unchanged",
|
||||
"After, TrueNAS Storage at 390: the Health column is phone-hidden, and sda's status dot reads Warning (amber) while cool disks stay Online (green)"
|
||||
],
|
||||
"interactions": [
|
||||
"Loaded Physical Disks filtered to West Production at 1440x900 and 390x844 and read each row's health word, title, colour and Temp cell class on both the before and after stacks",
|
||||
"Changed diskTempByType.nvme to 75/70, then agentDefaults.diskTemperature to 0, through GET/PUT /api/alerts/config and reloaded Physical Disks and TrueNAS Storage after each, then restored the factory config",
|
||||
"Opened the inline drawer of the 56C SATA disk by clicking its row",
|
||||
"Loaded TrueNAS Storage at 1440 and 390, clicked the Attention status filter, and read sda's Health cell, title and status dot"
|
||||
]
|
||||
}
|
||||
@@ -13,7 +13,6 @@ import {
|
||||
TableHeader,
|
||||
TableRow,
|
||||
} from '@/components/shared/Table';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import { formatBytes } from '@/utils/format';
|
||||
import { getTemperatureTextClass } from '@/utils/temperature';
|
||||
import {
|
||||
@@ -77,6 +76,7 @@ import {
|
||||
getPhysicalDiskEmptyStatePresentation,
|
||||
getPhysicalDiskHealthCompactLabel,
|
||||
getPhysicalDiskHealthStatus,
|
||||
getPhysicalDiskTemperatureThresholds,
|
||||
getPhysicalDiskHealthSummary,
|
||||
getPhysicalDiskHostLabel,
|
||||
getPhysicalDiskLifeLabel,
|
||||
@@ -132,7 +132,6 @@ export const DiskList: Component<DiskListProps> = (props) => {
|
||||
};
|
||||
const columnStyle = (columnId: PhysicalDiskTableColumnId) =>
|
||||
getPhysicalDiskColumnWidthStyle(layoutMode(), columnId);
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
const model = useDiskListModel({
|
||||
disks: () => props.disks,
|
||||
nodes: () => props.nodes,
|
||||
@@ -523,7 +522,7 @@ export const DiskList: Component<DiskListProps> = (props) => {
|
||||
reading().current
|
||||
? getTemperatureTextClass(
|
||||
data().temperature,
|
||||
getDiskTemperatureThresholds(data().type),
|
||||
getPhysicalDiskTemperatureThresholds(data()),
|
||||
'diskTemperature',
|
||||
)
|
||||
: PHYSICAL_DISK_TEMPERATURE_LAST_KNOWN_CLASS
|
||||
|
||||
@@ -2,8 +2,12 @@ import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/te
|
||||
import { createSignal } from 'solid-js';
|
||||
import { createStore, reconcile } from 'solid-js/store';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { AlertsAPI } from '@/api/alerts';
|
||||
import type { AlertConfig } from '@/types/alerts';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { DiskList } from '@/components/Storage/DiskList';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import { eventBus } from '@/stores/events';
|
||||
|
||||
vi.mock('@/components/Storage/DiskDetail', () => ({
|
||||
DiskDetail: (props: { disk: Resource }) => <div data-testid="disk-detail">{props.disk.id}</div>,
|
||||
@@ -348,19 +352,9 @@ describe('DiskList', () => {
|
||||
disks: [
|
||||
buildDisk('sda', 'pve3', {
|
||||
model: 'Crucial MX500 2TB',
|
||||
diskType: 'ssd',
|
||||
diskType: 'sata',
|
||||
wearout: 95,
|
||||
temperature: 72,
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{
|
||||
code: 'temperature_high',
|
||||
severity: 'critical',
|
||||
summary: 'Disk temperature is 72C',
|
||||
},
|
||||
],
|
||||
},
|
||||
temperature: 56,
|
||||
}),
|
||||
buildDisk('sdb', 'pve3', {
|
||||
temperature: 72,
|
||||
@@ -368,11 +362,6 @@ describe('DiskList', () => {
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{
|
||||
code: 'temperature_high',
|
||||
severity: 'critical',
|
||||
summary: 'Disk temperature is 72C',
|
||||
},
|
||||
{
|
||||
code: 'pending_sectors',
|
||||
severity: 'critical',
|
||||
@@ -381,6 +370,11 @@ describe('DiskList', () => {
|
||||
],
|
||||
},
|
||||
}),
|
||||
buildDisk('nvme0n1', 'pve3', {
|
||||
model: 'Samsung 990 PRO',
|
||||
diskType: 'nvme',
|
||||
temperature: 63,
|
||||
}),
|
||||
],
|
||||
nodes: [buildNode('node-pve3', 'pve3')],
|
||||
selectedNode: null,
|
||||
@@ -389,29 +383,34 @@ describe('DiskList', () => {
|
||||
|
||||
const rows = Array.from(view.container.querySelectorAll('[data-row-id]'));
|
||||
// The disk that needs replacing sorts above the one that needs cooling.
|
||||
expect(rows.map((row) => row.getAttribute('data-row-id'))).toEqual(['sdb', 'sda']);
|
||||
expect(rows.map((row) => row.getAttribute('data-row-id'))).toEqual(['sdb', 'sda', 'nvme0n1']);
|
||||
const hotRow = within(rows[1] as HTMLElement);
|
||||
expect(hotRow.queryByText('Replace Now')).not.toBeInTheDocument();
|
||||
expect(hotRow.getByText('Running Hot')).toHaveClass(
|
||||
'platform-table-label-full',
|
||||
'text-red-700',
|
||||
);
|
||||
expect(hotRow.getByText('Running Hot')).toHaveAttribute('title', 'Disk temperature is 72C');
|
||||
expect(hotRow.getByText('Running Hot')).toHaveAttribute(
|
||||
'title',
|
||||
'Disk temperature is 56°C, at or above its 55°C alert threshold.',
|
||||
);
|
||||
expect(hotRow.getByText('Hot')).toHaveClass('platform-table-label-compact', 'text-red-700');
|
||||
// The Temp cell beside it reads red too: the same SATA alert trigger.
|
||||
expect(hotRow.getByText('56°C')).toHaveClass('text-red-600');
|
||||
const failingRow = within(rows[0] as HTMLElement);
|
||||
expect(failingRow.getByText('Replace Now')).toHaveAttribute(
|
||||
'title',
|
||||
'Pending sectors detected (2)',
|
||||
);
|
||||
// An NVMe at 63C is under its 70C trigger and its 65C warning colour.
|
||||
const nvmeRow = within(rows[2] as HTMLElement);
|
||||
expect(nvmeRow.getByText('Healthy')).toBeInTheDocument();
|
||||
expect(nvmeRow.getByText('63°C')).toHaveClass('text-green-600');
|
||||
});
|
||||
|
||||
it('follows a disk from warm to critically hot and back to healthy', async () => {
|
||||
const heat = (severity: 'warning' | 'critical', celsius: number) => ({
|
||||
level: severity,
|
||||
reasons: [{ code: 'temperature_high', severity, summary: `Disk temperature is ${celsius}C` }],
|
||||
});
|
||||
it('follows a disk from warm to hot and back to healthy', async () => {
|
||||
const [disks, setDisks] = createStore({
|
||||
items: [buildDisk('sda', 'pve3', { temperature: 63, risk: heat('warning', 63) })],
|
||||
items: [buildDisk('sda', 'pve3', { temperature: 52 })],
|
||||
});
|
||||
const [healthFilter, setHealthFilter] = createSignal<'all' | 'critical'>('all');
|
||||
const view = render(() => (
|
||||
@@ -426,21 +425,15 @@ describe('DiskList', () => {
|
||||
/>
|
||||
));
|
||||
const row = () => view.container.querySelector('[data-row-id="sda"]') as HTMLElement | null;
|
||||
expect(within(row()!).getByText('Running Hot')).toHaveClass('text-amber-700');
|
||||
expect(within(row()!).getByText('Hot')).toHaveClass('text-amber-700');
|
||||
// Warm: the Temp cell warns in amber, the verdict stays Healthy.
|
||||
expect(within(row()!).getByText('52°C')).toHaveClass('text-amber-600');
|
||||
expect(within(row()!).getByText('Healthy')).toBeInTheDocument();
|
||||
setHealthFilter('critical');
|
||||
await waitFor(() => expect(row()).toBeNull());
|
||||
|
||||
setDisks(
|
||||
'items',
|
||||
reconcile([buildDisk('sda', 'pve3', { temperature: 72, risk: heat('critical', 72) })]),
|
||||
);
|
||||
setDisks('items', reconcile([buildDisk('sda', 'pve3', { temperature: 56 })]));
|
||||
await waitFor(() => expect(row()).not.toBeNull());
|
||||
expect(within(row()!).getByText('Running Hot')).toHaveClass('text-red-700');
|
||||
expect(within(row()!).getByText('Running Hot')).toHaveAttribute(
|
||||
'title',
|
||||
'Disk temperature is 72C',
|
||||
);
|
||||
|
||||
setHealthFilter('all');
|
||||
setDisks('items', reconcile([buildDisk('sda', 'pve3', { temperature: 41 })]));
|
||||
@@ -448,6 +441,40 @@ describe('DiskList', () => {
|
||||
expect(within(row()!).queryByText('Running Hot')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('judges heat by the NVMe trigger the user raised in Alerts', async () => {
|
||||
const getConfig = vi.spyOn(AlertsAPI, 'getConfig').mockResolvedValue({
|
||||
enabled: true,
|
||||
activationState: 'active',
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: {
|
||||
nvme: { trigger: 75, clear: 70 },
|
||||
sas: { trigger: 65, clear: 60 },
|
||||
sata: { trigger: 55, clear: 50 },
|
||||
},
|
||||
} as unknown as AlertConfig);
|
||||
try {
|
||||
const view = renderDiskList({
|
||||
disks: [buildDisk('nvme0n1', 'pve3', { diskType: 'nvme', temperature: 72 })],
|
||||
nodes: [],
|
||||
selectedNode: null,
|
||||
searchTerm: '',
|
||||
});
|
||||
const row = () =>
|
||||
within(view.container.querySelector('[data-row-id="nvme0n1"]') as HTMLElement);
|
||||
// Factory thresholds until the user's configuration loads: 72C passes
|
||||
// the 70C NVMe trigger.
|
||||
expect(row().getByText('Running Hot')).toBeInTheDocument();
|
||||
expect(row().getByText('72°C')).toHaveClass('text-red-600');
|
||||
|
||||
await useAlertsActivation().refreshConfig();
|
||||
await waitFor(() => expect(row().getByText('Healthy')).toBeInTheDocument());
|
||||
expect(row().getByText('72°C')).toHaveClass('text-amber-600');
|
||||
} finally {
|
||||
getConfig.mockRestore();
|
||||
eventBus.emit('org_switched', 'default');
|
||||
}
|
||||
});
|
||||
|
||||
it('renders SSD life and falls back to the Proxmox usage string for Belongs', () => {
|
||||
renderDiskList({
|
||||
disks: [
|
||||
|
||||
@@ -3,6 +3,7 @@ import type { HistoryTimeRange } from '@/api/charts';
|
||||
import {
|
||||
getPhysicalDiskCollectionMessages,
|
||||
extractPhysicalDiskPresentationData,
|
||||
getPhysicalDiskTemperatureThresholds,
|
||||
type PhysicalDiskPresentationData,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import {
|
||||
@@ -23,11 +24,11 @@ export const useDiskDetailModel = (options: UseDiskDetailModelOptions) => {
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
|
||||
const diskData = createMemo<PhysicalDiskPresentationData>(() =>
|
||||
extractPhysicalDiskPresentationData(options.disk()),
|
||||
extractPhysicalDiskPresentationData(options.disk(), getDiskTemperatureThresholds),
|
||||
);
|
||||
const historyResourceId = createMemo(() => resolvePhysicalDiskHistoryResourceId(options.disk()));
|
||||
const attributeCards = createMemo(() =>
|
||||
getDiskDetailAttributeCards(diskData(), getDiskTemperatureThresholds(diskData().type)),
|
||||
getDiskDetailAttributeCards(diskData(), getPhysicalDiskTemperatureThresholds(diskData())),
|
||||
);
|
||||
const historyCharts = createMemo(() => getDiskDetailHistoryCharts(diskData()));
|
||||
const healthPresentation = createMemo(() => getDiskDetailHealthPresentation(diskData()));
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
filterAndSortPhysicalDisks,
|
||||
type PhysicalDiskPresentationData,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import { matchesPhysicalDiskNode } from './diskResourceUtils';
|
||||
|
||||
type UseDiskListModelOptions = {
|
||||
@@ -26,13 +27,19 @@ type UseDiskListModelOptions = {
|
||||
|
||||
export const useDiskListModel = (options: UseDiskListModelOptions) => {
|
||||
const hasPVENodes = createMemo(() => options.nodes().length > 0);
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
|
||||
const diskDataById = createMemo(() => buildPhysicalDiskPresentationDataMap(options.disks()));
|
||||
// Each disk's heat is judged by the user's alert disk temperature thresholds,
|
||||
// so the presentation data recomputes when the alert configuration loads.
|
||||
const diskDataById = createMemo(() =>
|
||||
buildPhysicalDiskPresentationDataMap(options.disks(), getDiskTemperatureThresholds),
|
||||
);
|
||||
const roleFilterOptions = createMemo(() => buildPhysicalDiskRoleFilterOptions(options.disks()));
|
||||
const groupFilterOptions = createMemo(() => buildPhysicalDiskGroupFilterOptions(options.disks()));
|
||||
|
||||
const getDiskData = (disk: Resource): PhysicalDiskPresentationData =>
|
||||
diskDataById().get(disk.id) ?? extractPhysicalDiskPresentationData(disk);
|
||||
diskDataById().get(disk.id) ??
|
||||
extractPhysicalDiskPresentationData(disk, getDiskTemperatureThresholds);
|
||||
|
||||
const selectedNodeResource = createMemo(
|
||||
() => options.nodes().find((node) => node.id === options.selectedNode()) ?? null,
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
PHYSICAL_DISK_TEMPERATURE_LAST_KNOWN_CLASS,
|
||||
getPhysicalDiskTemperaturePresentation,
|
||||
} from '@/features/storageBackups/diskTemperaturePresentation';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { formatBytes } from '@/utils/format';
|
||||
|
||||
@@ -20,7 +21,8 @@ const LazyDiskDetail = lazy(() =>
|
||||
|
||||
const GuestPhysicalDiskRow: Component<{ disk: Resource }> = (props) => {
|
||||
const [expanded, setExpanded] = createSignal(false);
|
||||
const data = () => extractPhysicalDiskPresentationData(props.disk);
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
const data = () => extractPhysicalDiskPresentationData(props.disk, getDiskTemperatureThresholds);
|
||||
const health = () => getPhysicalDiskHealthStatus(data());
|
||||
const temperature = () => getPhysicalDiskTemperaturePresentation(data());
|
||||
const label = () => data().model || data().devPath || props.disk.name;
|
||||
|
||||
@@ -69,6 +69,8 @@ describe('diskDetailPresentation', () => {
|
||||
expect(getLinkedDiskTemperatureTextClass(65)).toBe('text-red-500');
|
||||
expect(getLinkedDiskTemperatureTextClass(52)).toBe('text-yellow-500');
|
||||
expect(getLinkedDiskTemperatureTextClass(45)).toBe('text-muted');
|
||||
// Disk temperature alerting off for the disk: no reading is warm or hot.
|
||||
expect(getLinkedDiskTemperatureTextClass(80, null)).toBe('text-muted');
|
||||
// Explicit thresholds (e.g. resolved for an NVMe disk) shift the colors.
|
||||
expect(getLinkedDiskTemperatureTextClass(55, { warning: 65, critical: 70 })).toBe('text-muted');
|
||||
expect(getLinkedDiskTemperatureTextClass(67, { warning: 65, critical: 70 })).toBe(
|
||||
|
||||
@@ -40,6 +40,7 @@ import {
|
||||
getPhysicalDiskSourceBadgePresentation,
|
||||
getPhysicalDiskTableLayoutModeForContainer,
|
||||
getPhysicalDiskTemperaturePresentation,
|
||||
getPhysicalDiskTemperatureThresholds,
|
||||
hasUnraidPhysicalDiskFaultSignal,
|
||||
hasPhysicalDiskSmartWarning,
|
||||
isPhysicalDiskWearoutReported,
|
||||
@@ -51,6 +52,11 @@ import {
|
||||
type PhysicalDiskPresentationData,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import * as diskTemperaturePresentation from '@/features/storageBackups/diskTemperaturePresentation';
|
||||
import {
|
||||
resolveDiskTemperatureDisplayThresholds,
|
||||
type MetricDisplayThresholds,
|
||||
} from '@/utils/metricThresholds';
|
||||
import { getTemperatureTextClass } from '@/utils/temperature';
|
||||
|
||||
function makeDiskData(
|
||||
overrides: Partial<PhysicalDiskPresentationData> = {},
|
||||
@@ -293,131 +299,188 @@ describe('diskPresentation', () => {
|
||||
expect(getPhysicalDiskHealthCompactLabel('Unknown')).toBe('Unknown');
|
||||
});
|
||||
|
||||
describe('heat versus replacement evidence', () => {
|
||||
const buildRiskDisk = (
|
||||
id: string,
|
||||
risk: { level: string; reasons: { code: string; severity: string; summary: string }[] },
|
||||
overrides: Record<string, unknown> = {},
|
||||
): Resource =>
|
||||
describe('heat judged by the alert disk temperature policy', () => {
|
||||
// Disk risk never carries heat. The verdict judges a current reading
|
||||
// against the same per-type alert thresholds the Temp cell colours by.
|
||||
const buildDisk = (id: string, overrides: Record<string, unknown> = {}): Resource =>
|
||||
({
|
||||
id,
|
||||
name: id,
|
||||
type: 'physical_disk',
|
||||
status: 'warning',
|
||||
status: 'online',
|
||||
physicalDisk: {
|
||||
devPath: `/dev/${id}`,
|
||||
model: 'Crucial MX500 2TB',
|
||||
diskType: 'sata',
|
||||
health: 'PASSED',
|
||||
wearout: 95,
|
||||
temperature: 72,
|
||||
risk,
|
||||
temperature: 40,
|
||||
collection: { temperature: { state: 'available', source: 'smartctl' } },
|
||||
...overrides,
|
||||
},
|
||||
identity: { hostname: 'pve3' },
|
||||
canonicalIdentity: { hostname: 'pve3' },
|
||||
platformType: 'proxmox-pve',
|
||||
}) as unknown as Resource;
|
||||
const hot = (severity: string, celsius = 72) => ({
|
||||
code: 'temperature_high',
|
||||
severity,
|
||||
summary: `Disk temperature is ${celsius}C`,
|
||||
});
|
||||
const verdictOf = (
|
||||
disk: Resource,
|
||||
resolve?: (diskType: string) => MetricDisplayThresholds | null,
|
||||
) => getPhysicalDiskHealthStatus(extractPhysicalDiskPresentationData(disk, resolve));
|
||||
// A user who raised the NVMe trigger to 75C in Alerts; warning colours
|
||||
// start at its 70C clear value.
|
||||
const raisedNVMe = (diskType: string) =>
|
||||
diskType === 'nvme'
|
||||
? { warning: 70, critical: 75 }
|
||||
: resolveDiskTemperatureDisplayThresholds(null, diskType);
|
||||
|
||||
it('calls a disk that is only hot Running Hot at the tier of its reading', () => {
|
||||
const criticalDisk = buildRiskDisk('sda', { level: 'critical', reasons: [hot('critical')] });
|
||||
const criticalData = extractPhysicalDiskPresentationData(criticalDisk);
|
||||
expect(criticalData.riskReasonDetails).toEqual([hot('critical')]);
|
||||
expect(getPhysicalDiskHealthStatus(criticalData)).toEqual({
|
||||
it('calls a disk Running Hot from its type alert trigger, not a flat line', () => {
|
||||
expect(verdictOf(buildDisk('nvme0n1', { diskType: 'nvme', temperature: 63 })).label).toBe(
|
||||
'Healthy',
|
||||
);
|
||||
expect(verdictOf(buildDisk('nvme0n1', { diskType: 'nvme', temperature: 72 }))).toEqual({
|
||||
label: 'Running Hot',
|
||||
summary: 'Disk temperature is 72C',
|
||||
summary: 'Disk temperature is 72°C, at or above its 70°C alert threshold.',
|
||||
tone: 'text-red-700 dark:text-red-300',
|
||||
});
|
||||
expect(getPhysicalDiskNormalizedHealth(criticalDisk, criticalData)).toBe('critical');
|
||||
|
||||
const warmDisk = buildRiskDisk(
|
||||
'sdb',
|
||||
{ level: 'warning', reasons: [hot('warning', 63)] },
|
||||
{ temperature: 63 },
|
||||
);
|
||||
const warmData = extractPhysicalDiskPresentationData(warmDisk);
|
||||
expect(getPhysicalDiskHealthStatus(warmData)).toEqual({
|
||||
expect(verdictOf(buildDisk('sda', { temperature: 56 }))).toEqual({
|
||||
label: 'Running Hot',
|
||||
summary: 'Disk temperature is 63C',
|
||||
tone: 'text-amber-700 dark:text-amber-300',
|
||||
summary: 'Disk temperature is 56°C, at or above its 55°C alert threshold.',
|
||||
tone: 'text-red-700 dark:text-red-300',
|
||||
});
|
||||
expect(getPhysicalDiskNormalizedHealth(warmDisk, warmData)).toBe('warning');
|
||||
// The amber band below the trigger is a Temp cell colour, not a verdict.
|
||||
expect(verdictOf(buildDisk('sdb', { temperature: 52 })).label).toBe('Healthy');
|
||||
|
||||
const hotDisk = buildDisk('sdc', { temperature: 56 });
|
||||
expect(
|
||||
getPhysicalDiskNormalizedHealth(hotDisk, extractPhysicalDiskPresentationData(hotDisk)),
|
||||
).toBe('critical');
|
||||
});
|
||||
|
||||
it('keeps Replace Now for failure evidence even when heat is listed first', () => {
|
||||
// Merged risks keep insertion order, so heat can precede the reason
|
||||
// that actually calls for a new disk.
|
||||
const disk = buildRiskDisk('sda', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
hot('critical'),
|
||||
{
|
||||
code: 'pending_sectors',
|
||||
severity: 'critical',
|
||||
summary: 'Pending sectors detected (2)',
|
||||
},
|
||||
],
|
||||
it('follows a raised trigger from the alert configuration', () => {
|
||||
const nvme72 = buildDisk('nvme0n1', { diskType: 'nvme', temperature: 72 });
|
||||
expect(verdictOf(nvme72, raisedNVMe).label).toBe('Healthy');
|
||||
expect(
|
||||
verdictOf(buildDisk('nvme1n1', { diskType: 'nvme', temperature: 76 }), raisedNVMe),
|
||||
).toEqual({
|
||||
label: 'Running Hot',
|
||||
summary: 'Disk temperature is 76°C, at or above its 75°C alert threshold.',
|
||||
tone: 'text-red-700 dark:text-red-300',
|
||||
});
|
||||
const data = extractPhysicalDiskPresentationData(disk);
|
||||
expect(getPhysicalDiskHealthStatus(data)).toEqual({
|
||||
expect(verdictOf(buildDisk('sda', { temperature: 56 }), raisedNVMe).label).toBe(
|
||||
'Running Hot',
|
||||
);
|
||||
expect(extractPhysicalDiskPresentationData(nvme72, raisedNVMe).temperatureThresholds).toEqual(
|
||||
{ warning: 70, critical: 75 },
|
||||
);
|
||||
});
|
||||
|
||||
it('reads red in the Temp cell exactly when the verdict says Running Hot', () => {
|
||||
const red = getTemperatureTextClass(99, { warning: 1, critical: 2 }, 'diskTemperature');
|
||||
// A user who switched disk temperature alerting off resolves to null.
|
||||
const alertingOff = () => null;
|
||||
for (const resolve of [undefined, raisedNVMe, alertingOff]) {
|
||||
for (const diskType of ['nvme', 'sata', 'sas', 'hdd', '']) {
|
||||
for (let celsius = 40; celsius <= 80; celsius += 1) {
|
||||
const data = extractPhysicalDiskPresentationData(
|
||||
buildDisk('sda', { diskType, temperature: celsius }),
|
||||
resolve,
|
||||
);
|
||||
const cellIsRed =
|
||||
getTemperatureTextClass(
|
||||
celsius,
|
||||
getPhysicalDiskTemperatureThresholds(data),
|
||||
'diskTemperature',
|
||||
) === red;
|
||||
expect([
|
||||
diskType,
|
||||
celsius,
|
||||
getPhysicalDiskHealthStatus(data).label === 'Running Hot',
|
||||
]).toEqual([diskType, celsius, cellIsRed]);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('calls no disk hot when its temperature alerting is off', () => {
|
||||
const off = () => null;
|
||||
const data = extractPhysicalDiskPresentationData(buildDisk('sda', { temperature: 80 }), off);
|
||||
expect(data.temperatureThresholds).toBeNull();
|
||||
expect(getPhysicalDiskHealthStatus(data).label).toBe('Healthy');
|
||||
expect(
|
||||
getTemperatureTextClass(80, getPhysicalDiskTemperatureThresholds(data), 'diskTemperature'),
|
||||
).toBe('text-base-content');
|
||||
});
|
||||
|
||||
it('never calls a retained reading hot', () => {
|
||||
const silent = buildDisk('sda', {
|
||||
temperature: 80,
|
||||
collection: {
|
||||
temperature: {
|
||||
state: 'unavailable',
|
||||
source: 'host_agent',
|
||||
reason: 'host agent stopped reporting',
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(verdictOf(silent).label).toBe('Healthy');
|
||||
// A source that predates collection state is current.
|
||||
expect(verdictOf(buildDisk('sdb', { temperature: 80, collection: undefined })).label).toBe(
|
||||
'Running Hot',
|
||||
);
|
||||
});
|
||||
|
||||
it('keeps Replace Now for failure evidence on a hot disk', () => {
|
||||
const pending = verdictOf(
|
||||
buildDisk('sda', {
|
||||
temperature: 72,
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{
|
||||
code: 'pending_sectors',
|
||||
severity: 'critical',
|
||||
summary: 'Pending sectors detected (2)',
|
||||
},
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(pending).toEqual({
|
||||
label: 'Replace Now',
|
||||
summary: 'Pending sectors detected (2)',
|
||||
tone: 'text-red-700 dark:text-red-300',
|
||||
});
|
||||
|
||||
const failed = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk(
|
||||
'sdb',
|
||||
{
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
hot('critical'),
|
||||
{
|
||||
code: 'health_status',
|
||||
severity: 'critical',
|
||||
summary: 'Disk reports health status FAILED',
|
||||
},
|
||||
],
|
||||
},
|
||||
{ health: 'FAILED' },
|
||||
),
|
||||
expect(verdictOf(buildDisk('sdb', { temperature: 72, health: 'FAILED' })).label).toBe(
|
||||
'Replace Now',
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(failed).label).toBe('Replace Now');
|
||||
expect(getPhysicalDiskHealthStatus(failed).summary).toBe('Disk reports health status FAILED');
|
||||
});
|
||||
|
||||
it('lets the more severe of heat and wear evidence name the verdict', () => {
|
||||
const hotAndWorn = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sda', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
hot('critical'),
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(hotAndWorn).label).toBe('Running Hot');
|
||||
expect(getPhysicalDiskHealthStatus(hotAndWorn).summary).toBe('Disk temperature is 72C');
|
||||
|
||||
const warmAndWorn = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk(
|
||||
'sdb',
|
||||
{
|
||||
it('lets a hot disk outrank warning-level wear and SMART evidence', () => {
|
||||
const wornAndHot = verdictOf(
|
||||
buildDisk('sda', {
|
||||
temperature: 72,
|
||||
risk: {
|
||||
level: 'warning',
|
||||
reasons: [
|
||||
hot('warning', 63),
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
},
|
||||
{ temperature: 63 },
|
||||
),
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(warmAndWorn)).toEqual({
|
||||
expect(wornAndHot.label).toBe('Running Hot');
|
||||
|
||||
const wornAndWarm = verdictOf(
|
||||
buildDisk('sdb', {
|
||||
temperature: 52,
|
||||
risk: {
|
||||
level: 'warning',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(wornAndWarm).toEqual({
|
||||
label: 'Needs Attention',
|
||||
summary: 'SSD life remaining is 8%',
|
||||
tone: 'text-amber-700 dark:text-amber-300',
|
||||
@@ -426,7 +489,15 @@ describe('diskPresentation', () => {
|
||||
|
||||
it('keeps a critical level that no listed reason explains as Replace Now', () => {
|
||||
const data = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sda', { level: 'critical', reasons: [hot('warning', 63)] }),
|
||||
buildDisk('sda', {
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'crc_errors', severity: 'monitor', summary: 'UDMA CRC errors detected (2)' },
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(data)).toEqual({
|
||||
label: 'Replace Now',
|
||||
@@ -435,42 +506,19 @@ describe('diskPresentation', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('decides the verdict from codes and severities even without display text', () => {
|
||||
const pendingWithoutText = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sda', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
hot('critical'),
|
||||
{ code: 'pending_sectors', severity: 'critical', summary: '' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
expect(pendingWithoutText.riskReasons).toEqual(['Disk temperature is 72C']);
|
||||
expect(getPhysicalDiskHealthStatus(pendingWithoutText)).toEqual({
|
||||
label: 'Replace Now',
|
||||
summary: 'Disk health has degraded to a critical state.',
|
||||
tone: 'text-red-700 dark:text-red-300',
|
||||
});
|
||||
|
||||
const heatWithoutText = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sdb', {
|
||||
level: 'critical',
|
||||
reasons: [{ code: 'temperature_high', severity: 'critical', summary: '' }],
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(heatWithoutText).label).toBe('Running Hot');
|
||||
});
|
||||
|
||||
it('does not let a weaker reason stand in for an unexplained critical level', () => {
|
||||
it('decides the verdict from severities even without display text', () => {
|
||||
const data = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sda', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'crc_errors', severity: 'monitor', summary: 'UDMA CRC errors detected (2)' },
|
||||
hot('warning', 63),
|
||||
],
|
||||
buildDisk('sda', {
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
{ code: 'pending_sectors', severity: 'critical', summary: '' },
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(data.riskReasons).toEqual(['SSD life remaining is 8%']);
|
||||
expect(getPhysicalDiskHealthStatus(data)).toEqual({
|
||||
label: 'Replace Now',
|
||||
summary: 'Disk health has degraded to a critical state.',
|
||||
@@ -479,90 +527,61 @@ describe('diskPresentation', () => {
|
||||
});
|
||||
|
||||
it('reads severities case-insensitively and in any order within a class', () => {
|
||||
const shouting = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sda', {
|
||||
level: 'CRITICAL',
|
||||
reasons: [{ ...hot('critical'), severity: ' Critical ' }],
|
||||
const shouting = verdictOf(
|
||||
buildDisk('sda', {
|
||||
risk: {
|
||||
level: 'CRITICAL',
|
||||
reasons: [
|
||||
{
|
||||
code: 'media_errors',
|
||||
severity: ' Critical ',
|
||||
summary: 'Media errors detected (4)',
|
||||
},
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(shouting).label).toBe('Running Hot');
|
||||
expect(getPhysicalDiskHealthStatus(shouting).tone).toBe('text-red-700 dark:text-red-300');
|
||||
expect(shouting.label).toBe('Replace Now');
|
||||
expect(shouting.summary).toBe('Media errors detected (4)');
|
||||
|
||||
const unsorted = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk('sdb', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
{
|
||||
code: 'pending_sectors',
|
||||
severity: 'critical',
|
||||
summary: 'Pending sectors detected (2)',
|
||||
},
|
||||
],
|
||||
const unsorted = verdictOf(
|
||||
buildDisk('sdb', {
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
{
|
||||
code: 'pending_sectors',
|
||||
severity: 'critical',
|
||||
summary: 'Pending sectors detected (2)',
|
||||
},
|
||||
],
|
||||
},
|
||||
}),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(unsorted).summary).toBe('Pending sectors detected (2)');
|
||||
});
|
||||
|
||||
it('lets SMART counters and low life outrank warning heat', () => {
|
||||
const warmAndLow = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk(
|
||||
'sda',
|
||||
{ level: 'warning', reasons: [hot('warning', 63)] },
|
||||
{ diskType: 'ssd', wearout: 8, temperature: 63 },
|
||||
),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(warmAndLow)).toEqual({
|
||||
label: 'Needs Attention',
|
||||
summary: 'SSD life is running low.',
|
||||
tone: 'text-amber-700 dark:text-amber-300',
|
||||
});
|
||||
|
||||
const warmAndReallocating = extractPhysicalDiskPresentationData(
|
||||
buildRiskDisk(
|
||||
'sdb',
|
||||
{ level: 'warning', reasons: [hot('warning', 63)] },
|
||||
{ temperature: 63, smart: { reallocatedSectors: 3 } },
|
||||
),
|
||||
);
|
||||
expect(getPhysicalDiskHealthStatus(warmAndReallocating).label).toBe('Needs Attention');
|
||||
});
|
||||
|
||||
it('sorts a FAILED disk with no risk payload ahead of a critically hot one', () => {
|
||||
const failedDisk = buildRiskDisk(
|
||||
'sdb',
|
||||
{ level: 'healthy', reasons: [] },
|
||||
{ risk: undefined, health: 'FAILED', temperature: 40 },
|
||||
);
|
||||
const hotDisk = buildRiskDisk('sda', { level: 'critical', reasons: [hot('critical')] });
|
||||
const failedData = extractPhysicalDiskPresentationData(failedDisk);
|
||||
const hotData = extractPhysicalDiskPresentationData(hotDisk);
|
||||
expect(getPhysicalDiskHealthStatus(failedData).label).toBe('Replace Now');
|
||||
expect(
|
||||
comparePhysicalDiskPresentation(failedDisk, failedData, hotDisk, hotData),
|
||||
).toBeLessThan(0);
|
||||
expect(unsorted.summary).toBe('Pending sectors detected (2)');
|
||||
});
|
||||
|
||||
it('sorts and filters hot disks by the action their verdict asks for', () => {
|
||||
const warm = buildRiskDisk('sdc', { level: 'warning', reasons: [hot('warning', 63)] });
|
||||
const worn = buildRiskDisk('sdd', {
|
||||
level: 'warning',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
const warm = buildDisk('sdc', { temperature: 52 });
|
||||
const worn = buildDisk('sdd', {
|
||||
risk: {
|
||||
level: 'warning',
|
||||
reasons: [
|
||||
{ code: 'wearout_low', severity: 'warning', summary: 'SSD life remaining is 8%' },
|
||||
],
|
||||
},
|
||||
});
|
||||
const hotDisk = buildRiskDisk('sda', { level: 'critical', reasons: [hot('critical')] });
|
||||
const failing = buildRiskDisk('sdb', {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'media_errors', severity: 'critical', summary: 'Media errors detected (4)' },
|
||||
],
|
||||
const hotDisk = buildDisk('sda', { temperature: 72 });
|
||||
const failing = buildDisk('sdb', {
|
||||
risk: {
|
||||
level: 'critical',
|
||||
reasons: [
|
||||
{ code: 'media_errors', severity: 'critical', summary: 'Media errors detected (4)' },
|
||||
],
|
||||
},
|
||||
});
|
||||
const healthy = buildRiskDisk(
|
||||
'sde',
|
||||
{ level: 'healthy', reasons: [] },
|
||||
{ risk: undefined, temperature: 38 },
|
||||
);
|
||||
const healthy = buildDisk('sde', { temperature: 38 });
|
||||
const disks = [healthy, warm, worn, hotDisk, failing];
|
||||
const dataMap = buildPhysicalDiskPresentationDataMap(disks);
|
||||
const sorted = (healthFilter: 'all' | 'attention' | 'critical' | 'warning') =>
|
||||
@@ -576,9 +595,9 @@ describe('diskPresentation', () => {
|
||||
|
||||
// Device order alone would put each hot disk first.
|
||||
expect(sorted('all')).toEqual(['sdb', 'sda', 'sdd', 'sdc', 'sde']);
|
||||
expect(sorted('attention')).toEqual(['sdb', 'sda', 'sdd', 'sdc']);
|
||||
expect(sorted('attention')).toEqual(['sdb', 'sda', 'sdd']);
|
||||
expect(sorted('critical')).toEqual(['sdb', 'sda']);
|
||||
expect(sorted('warning')).toEqual(['sdd', 'sdc']);
|
||||
expect(sorted('warning')).toEqual(['sdd']);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -25,7 +25,8 @@ export function getLinkedDiskTemperatureTextClass(
|
||||
tempCelsius: number,
|
||||
thresholds?: MetricDisplayThresholds | null,
|
||||
): string {
|
||||
if (!Number.isFinite(tempCelsius) || tempCelsius <= 0) {
|
||||
// Null thresholds mean disk temperature alerting is off for this disk.
|
||||
if (!Number.isFinite(tempCelsius) || tempCelsius <= 0 || thresholds === null) {
|
||||
return 'text-muted';
|
||||
}
|
||||
const severity = getMetricSeverity(tempCelsius, 'diskTemperature', thresholds);
|
||||
@@ -137,7 +138,9 @@ export function getDiskDetailAttributeCards(
|
||||
cards.push({
|
||||
label: 'Temperature',
|
||||
value: temperature.label,
|
||||
ok: getMetricSeverity(disk.temperature, 'diskTemperature', diskTempThresholds) !== 'critical',
|
||||
ok:
|
||||
diskTempThresholds === null ||
|
||||
getMetricSeverity(disk.temperature, 'diskTemperature', diskTempThresholds) !== 'critical',
|
||||
});
|
||||
} else if (temperature) {
|
||||
// A retained reading is neither healthy nor hot now. The collection
|
||||
|
||||
@@ -14,7 +14,15 @@ import {
|
||||
import { getAllFilterOptionLabel } from '@/components/shared/filterOptionPresentation';
|
||||
import { getPhysicalDiskNodeIdentity } from '@/components/Storage/diskResourceUtils';
|
||||
import { getInfrastructureSettingsLocationLabel } from '@/utils/infrastructureSettingsPresentation';
|
||||
import {
|
||||
resolveDiskTemperatureDisplayThresholds,
|
||||
type MetricDisplayThresholds,
|
||||
} from '@/utils/metricThresholds';
|
||||
import { normalizeStorageSourceKey, storageSourceMatchesFilter } from '@/utils/storageSources';
|
||||
import {
|
||||
getPhysicalDiskHeatSummary,
|
||||
isPhysicalDiskRunningHot,
|
||||
} from './diskTemperaturePresentation';
|
||||
import type { NormalizedHealth, StorageHealthFilter } from './models';
|
||||
import { matchesStorageNodeTerms, parseStorageSearchQuery } from './storageSearchQuery';
|
||||
|
||||
@@ -67,6 +75,12 @@ export interface PhysicalDiskPresentationData {
|
||||
controller?: string;
|
||||
target?: string;
|
||||
temperature: number;
|
||||
/**
|
||||
* The alert disk temperature thresholds for this disk's type. The Temp cell
|
||||
* colours the reading by them and the Health verdict judges heat by them.
|
||||
* Absent means the factory thresholds for the type.
|
||||
*/
|
||||
temperatureThresholds?: MetricDisplayThresholds | null;
|
||||
rpm: number;
|
||||
used: string;
|
||||
collection?: PhysicalDiskCollectionStatus;
|
||||
@@ -395,13 +409,35 @@ export function getPhysicalDiskHostLabel(
|
||||
return (disk.node || resource.parentName || '').trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the alert disk temperature thresholds for a disk type. Callers with
|
||||
* the user's alert configuration pass `getDiskTemperatureThresholds` from the
|
||||
* alerts activation store; the default is the factory configuration.
|
||||
*/
|
||||
export type PhysicalDiskTemperatureThresholdResolver = (
|
||||
diskType: string,
|
||||
) => MetricDisplayThresholds | null;
|
||||
|
||||
const resolveFactoryDiskTemperatureThresholds: PhysicalDiskTemperatureThresholdResolver = (
|
||||
diskType,
|
||||
) => resolveDiskTemperatureDisplayThresholds(null, diskType);
|
||||
|
||||
/** The thresholds the Temp cell colours this disk's reading by. */
|
||||
export const getPhysicalDiskTemperatureThresholds = (
|
||||
disk: Pick<PhysicalDiskPresentationData, 'temperatureThresholds' | 'type'>,
|
||||
): MetricDisplayThresholds | null =>
|
||||
disk.temperatureThresholds !== undefined
|
||||
? disk.temperatureThresholds
|
||||
: resolveFactoryDiskTemperatureThresholds(disk.type);
|
||||
|
||||
export function extractPhysicalDiskPresentationData(
|
||||
resource: Resource,
|
||||
resolveTemperatureThresholds: PhysicalDiskTemperatureThresholdResolver = resolveFactoryDiskTemperatureThresholds,
|
||||
): PhysicalDiskPresentationData {
|
||||
const pd = resource.physicalDisk || ((resource.platformData as any)?.physicalDisk ?? {});
|
||||
const diskNode = getPhysicalDiskNodeIdentity(resource);
|
||||
// A reason's code and severity decide the verdict even when it carries no
|
||||
// display text, so only the summary list drops empty summaries.
|
||||
// A reason's severity decides the verdict even when it carries no display
|
||||
// text, so only the summary list drops empty summaries.
|
||||
const riskReasonDetails: ResourceStorageRiskReason[] = Array.isArray(pd.risk?.reasons)
|
||||
? pd.risk.reasons
|
||||
.filter((reason: unknown) => reason !== null && typeof reason === 'object')
|
||||
@@ -431,6 +467,7 @@ export function extractPhysicalDiskPresentationData(
|
||||
health: pd.health || 'UNKNOWN',
|
||||
wearout: pd.wearout ?? -1,
|
||||
temperature: pd.temperature ?? 0,
|
||||
temperatureThresholds: resolveTemperatureThresholds(pd.diskType || ''),
|
||||
rpm: pd.rpm ?? 0,
|
||||
used: pd.used || '',
|
||||
storageRole: pd.storageRole,
|
||||
@@ -503,23 +540,24 @@ export function getPhysicalDiskCollectionMessages(disk: PhysicalDiskPresentation
|
||||
|
||||
export function buildPhysicalDiskPresentationDataMap(
|
||||
disks: Resource[],
|
||||
resolveTemperatureThresholds?: PhysicalDiskTemperatureThresholdResolver,
|
||||
): Map<string, PhysicalDiskPresentationData> {
|
||||
const map = new Map<string, PhysicalDiskPresentationData>();
|
||||
for (const disk of disks || []) {
|
||||
map.set(disk.id, extractPhysicalDiskPresentationData(disk));
|
||||
map.set(disk.id, extractPhysicalDiskPresentationData(disk, resolveTemperatureThresholds));
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
// Disks sort in the order their verdicts ask for action: replacement first,
|
||||
// then a disk running critically hot, then other warnings, then a warm disk.
|
||||
// then a disk running hot, then other warnings.
|
||||
const getPhysicalDiskVerdictPriority = (disk: PhysicalDiskPresentationData): number => {
|
||||
const verdict = getPhysicalDiskHealthVerdict(disk);
|
||||
switch (verdict.kind) {
|
||||
case 'replace':
|
||||
return 400;
|
||||
case 'hot':
|
||||
return verdict.critical ? 300 : 100;
|
||||
return 300;
|
||||
case 'attention':
|
||||
return 200;
|
||||
default:
|
||||
@@ -677,11 +715,6 @@ export function hasPhysicalDiskSmartWarning(disk: PhysicalDiskPresentationData):
|
||||
);
|
||||
}
|
||||
|
||||
// Heat is the one disk risk whose fix is cooling, not a replacement disk.
|
||||
// Alerts raise it as a temperature metric rather than as disk health, so the
|
||||
// verdict names it apart from the evidence that does mean replacing the disk.
|
||||
const PHYSICAL_DISK_TEMPERATURE_RISK_CODE = 'temperature_high';
|
||||
|
||||
const PHYSICAL_DISK_RISK_SEVERITY_RANK: Record<string, number> = {
|
||||
monitor: 1,
|
||||
warning: 2,
|
||||
@@ -696,20 +729,14 @@ interface PhysicalDiskRiskEvidence {
|
||||
summary?: string;
|
||||
}
|
||||
|
||||
// Splits the disk's risk into heat and everything else, keeping the most
|
||||
// severe reason of each. A risk level that none of its reasons explains, or
|
||||
// one sent without reason codes, stays with the disk-health evidence.
|
||||
function splitPhysicalDiskRisk(disk: PhysicalDiskPresentationData): {
|
||||
health: PhysicalDiskRiskEvidence;
|
||||
heat: PhysicalDiskRiskEvidence;
|
||||
} {
|
||||
const health: PhysicalDiskRiskEvidence = { rank: 0 };
|
||||
const heat: PhysicalDiskRiskEvidence = { rank: 0 };
|
||||
// Keeps the disk's most severe risk reason. A risk level that none of its
|
||||
// reasons explains, or one sent without reasons, still sets the rank.
|
||||
function getPhysicalDiskRiskEvidence(disk: PhysicalDiskPresentationData): PhysicalDiskRiskEvidence {
|
||||
const evidence: PhysicalDiskRiskEvidence = { rank: 0 };
|
||||
const details =
|
||||
disk.riskReasonDetails ??
|
||||
disk.riskReasons.map((summary) => ({ code: '', severity: disk.riskLevel || '', summary }));
|
||||
for (const reason of details) {
|
||||
const evidence = reason.code === PHYSICAL_DISK_TEMPERATURE_RISK_CODE ? heat : health;
|
||||
const rank = getPhysicalDiskRiskSeverityRank(reason.severity);
|
||||
if (rank > evidence.rank) {
|
||||
evidence.rank = rank;
|
||||
@@ -719,25 +746,25 @@ function splitPhysicalDiskRisk(disk: PhysicalDiskPresentationData): {
|
||||
}
|
||||
}
|
||||
const levelRank = getPhysicalDiskRiskSeverityRank(disk.riskLevel);
|
||||
if (levelRank > Math.max(health.rank, heat.rank)) {
|
||||
if (levelRank > evidence.rank) {
|
||||
// A reason with a known, lower severity does not explain this level, so
|
||||
// its text must not stand in as the verdict's reason.
|
||||
if (health.rank > 0) health.summary = undefined;
|
||||
health.rank = levelRank;
|
||||
if (evidence.rank > 0) evidence.summary = undefined;
|
||||
evidence.rank = levelRank;
|
||||
}
|
||||
return { health, heat };
|
||||
return evidence;
|
||||
}
|
||||
|
||||
type PhysicalDiskHealthVerdict =
|
||||
| { kind: 'replace'; summary: string }
|
||||
| { kind: 'hot'; critical: boolean; summary: string }
|
||||
| { kind: 'hot'; summary: string }
|
||||
| { kind: 'attention'; summary: string }
|
||||
| { kind: 'clear' };
|
||||
|
||||
function getPhysicalDiskHealthVerdict(
|
||||
disk: PhysicalDiskPresentationData,
|
||||
): PhysicalDiskHealthVerdict {
|
||||
const { health, heat } = splitPhysicalDiskRisk(disk);
|
||||
const health = getPhysicalDiskRiskEvidence(disk);
|
||||
const critical = PHYSICAL_DISK_RISK_SEVERITY_RANK.critical;
|
||||
const warning = PHYSICAL_DISK_RISK_SEVERITY_RANK.warning;
|
||||
const lowLife = isPhysicalDiskWearoutReported(disk) && disk.wearout < 10;
|
||||
@@ -748,8 +775,12 @@ function getPhysicalDiskHealthVerdict(
|
||||
summary: health.summary || 'Disk health has degraded to a critical state.',
|
||||
};
|
||||
}
|
||||
if (heat.rank >= critical) {
|
||||
return { kind: 'hot', critical: true, summary: heat.summary || 'Disk temperature is high.' };
|
||||
const temperatureThresholds = getPhysicalDiskTemperatureThresholds(disk);
|
||||
if (isPhysicalDiskRunningHot(disk, temperatureThresholds)) {
|
||||
return {
|
||||
kind: 'hot',
|
||||
summary: getPhysicalDiskHeatSummary(disk.temperature, temperatureThresholds),
|
||||
};
|
||||
}
|
||||
if (health.rank >= warning || hasPhysicalDiskSmartWarning(disk) || lowLife) {
|
||||
return {
|
||||
@@ -759,9 +790,6 @@ function getPhysicalDiskHealthVerdict(
|
||||
(lowLife ? 'SSD life is running low.' : 'SMART counters indicate elevated risk.'),
|
||||
};
|
||||
}
|
||||
if (heat.rank >= warning) {
|
||||
return { kind: 'hot', critical: false, summary: heat.summary || 'Disk temperature is high.' };
|
||||
}
|
||||
return { kind: 'clear' };
|
||||
}
|
||||
|
||||
@@ -778,11 +806,7 @@ export function getPhysicalDiskHealthStatus(
|
||||
}
|
||||
|
||||
if (verdict.kind === 'hot') {
|
||||
return {
|
||||
label: 'Running Hot',
|
||||
summary: verdict.summary,
|
||||
tone: verdict.critical ? PHYSICAL_DISK_CRITICAL_TONE : PHYSICAL_DISK_WARNING_TONE,
|
||||
};
|
||||
return { label: 'Running Hot', summary: verdict.summary, tone: PHYSICAL_DISK_CRITICAL_TONE };
|
||||
}
|
||||
|
||||
if (verdict.kind === 'attention') {
|
||||
@@ -814,8 +838,9 @@ export function getPhysicalDiskNormalizedHealth(
|
||||
if (clearLabel === 'Online') return 'healthy';
|
||||
if (resource.status === 'offline') return 'offline';
|
||||
if (verdict.kind === 'replace') return 'critical';
|
||||
// A disk running hot sits in the health filter at its temperature tier.
|
||||
if (verdict.kind === 'hot') return verdict.critical ? 'critical' : 'warning';
|
||||
// A disk running hot sits in the health filter with the critical disks, as
|
||||
// its Temp cell reads red.
|
||||
if (verdict.kind === 'hot') return 'critical';
|
||||
if (verdict.kind === 'attention') return 'warning';
|
||||
return clearLabel === 'Healthy' ? 'healthy' : 'unknown';
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { PhysicalDiskCollectionStatus, PhysicalDiskFieldStatus } from '@/types/resource';
|
||||
import type { MetricDisplayThresholds } from '@/utils/metricThresholds';
|
||||
import { formatTemperature } from '@/utils/temperature';
|
||||
|
||||
// The canonical decision on whether a disk temperature is a current reading.
|
||||
@@ -51,3 +52,29 @@ export function getPhysicalDiskTemperaturePresentation(disk: {
|
||||
: getPhysicalDiskLastKnownTemperatureTitle(disk.collection?.temperature),
|
||||
};
|
||||
}
|
||||
|
||||
// Disk heat is judged by the alert disk temperature policy, through the
|
||||
// thresholds the Temp column colours a reading by: a disk runs hot from its
|
||||
// type's alert trigger (`critical`), the reading its disk temperature alert
|
||||
// fires at. The band just below the trigger is a colour, not heat. A retained
|
||||
// reading is not current, and null thresholds mean disk temperature alerting
|
||||
// is off for the disk, so neither is ever hot. Disk risk never carries heat.
|
||||
export function isPhysicalDiskRunningHot(
|
||||
disk: { temperature: number; collection?: PhysicalDiskCollectionStatus | null },
|
||||
thresholds: MetricDisplayThresholds | null,
|
||||
): boolean {
|
||||
if (!thresholds || !(thresholds.critical > 0)) return false;
|
||||
if (!Number.isFinite(disk.temperature) || disk.temperature <= 0) return false;
|
||||
if (!isPhysicalDiskTemperatureCurrent(disk.collection)) return false;
|
||||
return disk.temperature >= thresholds.critical;
|
||||
}
|
||||
|
||||
export function getPhysicalDiskHeatSummary(
|
||||
temperature: number,
|
||||
thresholds: MetricDisplayThresholds | null,
|
||||
): string {
|
||||
const reading = formatTemperature(temperature);
|
||||
return thresholds && thresholds.critical > 0
|
||||
? `Disk temperature is ${reading}, at or above its ${formatTemperature(thresholds.critical)} alert threshold.`
|
||||
: `Disk temperature is ${reading}.`;
|
||||
}
|
||||
|
||||
@@ -37,13 +37,15 @@ import {
|
||||
createPlatformResourceLabelResolver,
|
||||
getPlatformResourceDetailRowClass,
|
||||
} from '@/features/platformPage/PlatformResourceDetailTableRow';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import {
|
||||
buildTrueNASStorageTopologyRows,
|
||||
filterTrueNASStorageTopologyRows,
|
||||
filterTrueNASStorageTopologyRowsByKind,
|
||||
getTrueNASResourceDisplayStatus,
|
||||
getTrueNASStorageDotStatus,
|
||||
getTrueNASStorageIssue,
|
||||
type TrueNASDiskTemperatureThresholdResolver,
|
||||
type TrueNASStorageIssue,
|
||||
type TrueNASStorageStatusFilter,
|
||||
type TrueNASStorageKindFilter,
|
||||
@@ -232,6 +234,7 @@ type TrueNASStorageSortKey = (typeof TRUENAS_STORAGE_SORT_KEYS)[number];
|
||||
const getTrueNASStorageSortValue = (
|
||||
row: TrueNASStorageTopologyRow,
|
||||
key: TrueNASStorageSortKey,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): PlatformTableSortValue => {
|
||||
switch (key) {
|
||||
case 'resource':
|
||||
@@ -255,7 +258,9 @@ const getTrueNASStorageSortValue = (
|
||||
return typeof temperature === 'number' && Number.isFinite(temperature) ? temperature : null;
|
||||
}
|
||||
case 'health':
|
||||
return HEALTH_SORT_RANK[getTrueNASStorageIssue(row.resource)?.status ?? 'healthy'];
|
||||
return HEALTH_SORT_RANK[
|
||||
getTrueNASStorageIssue(row.resource, resolveDiskTemperatureThresholds)?.status ?? 'healthy'
|
||||
];
|
||||
default:
|
||||
key satisfies never;
|
||||
return null;
|
||||
@@ -271,6 +276,7 @@ const getTrueNASStorageSortValue = (
|
||||
const sortTrueNASStorageTopologyRows = (
|
||||
rows: readonly TrueNASStorageTopologyRow[],
|
||||
sort: PlatformTableSortState<TrueNASStorageSortKey>,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): readonly TrueNASStorageTopologyRow[] => {
|
||||
if (!sort.sortKey()) return rows;
|
||||
const present = new Set(rows.map((row) => row.id));
|
||||
@@ -288,7 +294,9 @@ const sortTrueNASStorageTopologyRows = (
|
||||
}
|
||||
const ordered: TrueNASStorageTopologyRow[] = [];
|
||||
const emit = (siblings: TrueNASStorageTopologyRow[]) => {
|
||||
for (const row of sort.sortRows(siblings, getTrueNASStorageSortValue)) {
|
||||
for (const row of sort.sortRows(siblings, (sibling, key) =>
|
||||
getTrueNASStorageSortValue(sibling, key, resolveDiskTemperatureThresholds),
|
||||
)) {
|
||||
ordered.push(row);
|
||||
const children = childrenByParent.get(row.id);
|
||||
if (children) emit(children);
|
||||
@@ -308,7 +316,9 @@ export const getTrueNASStorageTopologyIndentClass = (depth: number): string => {
|
||||
const ResourceCell: Component<{ row: TrueNASStorageTopologyRow; detailToggle?: JSX.Element }> = (
|
||||
props,
|
||||
) => {
|
||||
const displayStatus = () => getTrueNASResourceDisplayStatus(props.row.resource);
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
const displayStatus = () =>
|
||||
getTrueNASStorageDotStatus(props.row.resource, getDiskTemperatureThresholds);
|
||||
const indicator = () => getSimpleStatusIndicator(displayStatus());
|
||||
const name = () => resourceName(props.row.resource);
|
||||
return (
|
||||
@@ -342,7 +352,11 @@ export const TrueNASStorageTopologyTable: Component<{
|
||||
kindFilter?: TrueNASStorageKindFilter;
|
||||
onKindFilterChange?: (value: TrueNASStorageKindFilter) => void;
|
||||
}> = (props) => {
|
||||
const rows = createMemo(() => buildTrueNASStorageTopologyRows(props.resources));
|
||||
// Disk heat follows the user's alert disk temperature thresholds.
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
const rows = createMemo(() =>
|
||||
buildTrueNASStorageTopologyRows(props.resources, getDiskTemperatureThresholds),
|
||||
);
|
||||
const [internalKindFilter, setInternalKindFilter] = createSignal<TrueNASStorageKindFilter>('all');
|
||||
const kindFilter = () => props.kindFilter ?? internalKindFilter();
|
||||
const setKindFilter = (value: TrueNASStorageKindFilter) => {
|
||||
@@ -356,7 +370,8 @@ export const TrueNASStorageTopologyTable: Component<{
|
||||
const tableState = createPlatformTableFilterState({
|
||||
resources: scopedRows,
|
||||
initialStatus: 'all' as TrueNASStorageStatusFilter,
|
||||
filter: filterTrueNASStorageTopologyRows,
|
||||
filter: (rows, search, status) =>
|
||||
filterTrueNASStorageTopologyRows(rows, search, status, getDiskTemperatureThresholds),
|
||||
});
|
||||
const kindFilters = createMemo<FilterDef[]>(() => [
|
||||
{
|
||||
@@ -373,6 +388,7 @@ export const TrueNASStorageTopologyTable: Component<{
|
||||
filterTrueNASStorageTopologyRowsByKind(rows(), option.value),
|
||||
tableState.search(),
|
||||
tableState.status(),
|
||||
getDiskTemperatureThresholds,
|
||||
).length,
|
||||
})),
|
||||
value: kindFilter,
|
||||
@@ -392,7 +408,9 @@ export const TrueNASStorageTopologyTable: Component<{
|
||||
sortKeys: TRUENAS_STORAGE_SORT_KEYS,
|
||||
descendingFirst: ['usage', 'disks', 'temp'],
|
||||
});
|
||||
const sortedRows = createMemo(() => sortTrueNASStorageTopologyRows(tableState.filtered(), sort));
|
||||
const sortedRows = createMemo(() =>
|
||||
sortTrueNASStorageTopologyRows(tableState.filtered(), sort, getDiskTemperatureThresholds),
|
||||
);
|
||||
|
||||
return (
|
||||
<Show
|
||||
@@ -499,7 +517,9 @@ export const TrueNASStorageTopologyTable: Component<{
|
||||
<PlatformWindowedRows items={sortedRows} estimatedRowHeight={32}>
|
||||
{(row) => {
|
||||
const resource = () => row.resource;
|
||||
const issue = createMemo(() => getTrueNASStorageIssue(resource()));
|
||||
const issue = createMemo(() =>
|
||||
getTrueNASStorageIssue(resource(), getDiskTemperatureThresholds),
|
||||
);
|
||||
const detailRowId = () => drawer.detailRowId(resource());
|
||||
const isExpanded = () => drawer.isExpanded(resource());
|
||||
return (
|
||||
|
||||
+105
-2
@@ -1,5 +1,10 @@
|
||||
import { cleanup, fireEvent, render, screen } from '@solidjs/testing-library';
|
||||
import { afterEach, describe, expect, it } from 'vitest';
|
||||
import { cleanup, fireEvent, render, screen, waitFor } from '@solidjs/testing-library';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { AlertsAPI } from '@/api/alerts';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import { eventBus } from '@/stores/events';
|
||||
import type { AlertConfig } from '@/types/alerts';
|
||||
|
||||
import {
|
||||
TrueNASStorageTopologyTable,
|
||||
@@ -215,4 +220,102 @@ describe('TrueNASStorageTopologyTable', () => {
|
||||
expect(healthyRow?.querySelector('[data-truenas-storage-health]')).toBeNull();
|
||||
expect(healthyRow).not.toHaveTextContent('Healthy');
|
||||
});
|
||||
|
||||
it('flags a disk running hot by its type alert trigger', () => {
|
||||
// Disk risk carries no heat, so the table judges it with the alerts
|
||||
// store's per-type thresholds (factory here: SATA 55C, NVMe 70C).
|
||||
const disk = (id: string, diskType: string, temperature: number) =>
|
||||
makeStorageResource({
|
||||
id,
|
||||
type: 'physical_disk',
|
||||
name: id,
|
||||
storage: undefined,
|
||||
physicalDisk: { devPath: `/dev/${id}`, serial: `serial-${id}`, diskType, temperature },
|
||||
});
|
||||
const resources = [disk('sda', 'sata', 56), disk('nvme0n1', 'nvme', 63)];
|
||||
const { container } = render(() => (
|
||||
<TrueNASStorageTopologyTable
|
||||
resources={resources}
|
||||
scope={resources}
|
||||
emptyIcon={<span />}
|
||||
emptyTitle="No storage"
|
||||
emptyDescription="No storage"
|
||||
showToolbar={false}
|
||||
/>
|
||||
));
|
||||
|
||||
const hotRow = container.querySelector('[data-truenas-storage-resource="sda"]');
|
||||
expect(hotRow?.querySelector('[data-truenas-storage-health="attention"]')).toHaveAttribute(
|
||||
'title',
|
||||
'Disk temperature is 56°C, at or above its 55°C alert threshold.',
|
||||
);
|
||||
const warmRow = container.querySelector('[data-truenas-storage-resource="nvme0n1"]');
|
||||
expect(warmRow?.querySelector('[data-truenas-storage-health]')).toBeNull();
|
||||
// Phones hide the Health cell, so the status dot carries the heat too.
|
||||
expect(hotRow?.querySelector('[title="Warning"]')).not.toBeNull();
|
||||
expect(warmRow?.querySelector('[title="Online"]')).not.toBeNull();
|
||||
// The hot disk sorts ahead of its cooler sibling.
|
||||
expect(
|
||||
[...container.querySelectorAll('[data-truenas-storage-resource]')].map((row) =>
|
||||
row.getAttribute('data-truenas-storage-resource'),
|
||||
),
|
||||
).toEqual(['sda', 'nvme0n1']);
|
||||
});
|
||||
|
||||
it('follows a disk temperature trigger the user raised in Alerts', async () => {
|
||||
const getConfig = vi.spyOn(AlertsAPI, 'getConfig').mockResolvedValue({
|
||||
enabled: true,
|
||||
activationState: 'active',
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: {
|
||||
nvme: { trigger: 70, clear: 65 },
|
||||
sas: { trigger: 65, clear: 60 },
|
||||
sata: { trigger: 60, clear: 55 },
|
||||
},
|
||||
} as unknown as AlertConfig);
|
||||
try {
|
||||
const resources = [
|
||||
makeStorageResource({
|
||||
id: 'sda',
|
||||
type: 'physical_disk',
|
||||
name: 'sda',
|
||||
storage: undefined,
|
||||
physicalDisk: {
|
||||
devPath: '/dev/sda',
|
||||
serial: 'serial-sda',
|
||||
diskType: 'sata',
|
||||
temperature: 56,
|
||||
},
|
||||
}),
|
||||
];
|
||||
const { container } = render(() => (
|
||||
<TrueNASStorageTopologyTable
|
||||
resources={resources}
|
||||
scope={resources}
|
||||
emptyIcon={<span />}
|
||||
emptyTitle="No storage"
|
||||
emptyDescription="No storage"
|
||||
/>
|
||||
));
|
||||
const health = () =>
|
||||
container.querySelector(
|
||||
'[data-truenas-storage-resource="sda"] [data-truenas-storage-health]',
|
||||
);
|
||||
expect(health()).toHaveAttribute('data-truenas-storage-health', 'attention');
|
||||
expect(screen.getByRole('button', { name: /Attention/ })).toHaveTextContent('1');
|
||||
// With the Attention filter on, the storage-type counts judge heat too.
|
||||
await fireEvent.click(screen.getByRole('button', { name: /Attention/ }));
|
||||
expect(screen.getByRole('button', { name: 'Physical disks, 1' })).toBeInTheDocument();
|
||||
|
||||
await useAlertsActivation().refreshConfig();
|
||||
await waitFor(() =>
|
||||
expect(screen.getByRole('button', { name: 'Physical disks, 0' })).toBeInTheDocument(),
|
||||
);
|
||||
// The status filter counts follow the same thresholds.
|
||||
expect(screen.getByRole('button', { name: /Attention/ })).toHaveTextContent('0');
|
||||
} finally {
|
||||
getConfig.mockRestore();
|
||||
eventBus.emit('org_switched', 'default');
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { RecoveryPoint } from '@/types/recovery';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { resolveDiskTemperatureDisplayThresholds } from '@/utils/metricThresholds';
|
||||
import {
|
||||
TRUENAS_TAB_SPECS,
|
||||
buildTrueNASPageModel,
|
||||
@@ -630,6 +631,72 @@ describe('truenasPageModel', () => {
|
||||
expect(mapTrueNASStorageStatus(healthyByState)).toBe('healthy');
|
||||
});
|
||||
|
||||
it('flags a disk whose current reading reaches its alert trigger', () => {
|
||||
// Disk risk carries no heat, so a hot disk stays `online`; the page judges
|
||||
// it by the alert disk temperature thresholds, as Physical Disks does.
|
||||
const resolveFactory = (diskType: string) =>
|
||||
resolveDiskTemperatureDisplayThresholds(null, diskType);
|
||||
const hotDisk = makeResource({
|
||||
id: 'disk-hot',
|
||||
type: 'physical_disk',
|
||||
name: 'sda',
|
||||
status: 'online',
|
||||
physicalDisk: {
|
||||
devPath: '/dev/sda',
|
||||
serial: 'serial-hot',
|
||||
diskType: 'sata',
|
||||
health: 'UNKNOWN',
|
||||
temperature: 56,
|
||||
},
|
||||
});
|
||||
expect(mapTrueNASStorageStatus(hotDisk, resolveFactory)).toBe('attention');
|
||||
expect(getTrueNASStorageIssue(hotDisk, resolveFactory)).toEqual({
|
||||
status: 'attention',
|
||||
label: 'Attention',
|
||||
reasons: ['Disk temperature is 56°C, at or above its 55°C alert threshold.'],
|
||||
});
|
||||
// A cool sibling that sorts first by name: heat moves the hot disk ahead
|
||||
// of it, and the status filter splits them.
|
||||
const coolDisk = makeResource({
|
||||
...hotDisk,
|
||||
id: 'disk-cool',
|
||||
name: 'ada0',
|
||||
physicalDisk: { ...hotDisk.physicalDisk!, serial: 'serial-cool', temperature: 40 },
|
||||
});
|
||||
const rows = buildTrueNASStorageTopologyRows([coolDisk, hotDisk], resolveFactory);
|
||||
expect(rows.map((row) => row.id)).toEqual(['disk:disk-hot', 'disk:disk-cool']);
|
||||
expect(buildTrueNASStorageTopologyRows([coolDisk, hotDisk]).map((row) => row.id)).toEqual([
|
||||
'disk:disk-cool',
|
||||
'disk:disk-hot',
|
||||
]);
|
||||
expect(
|
||||
filterTrueNASStorageTopologyRows(rows, '', 'attention', resolveFactory).map((row) => row.id),
|
||||
).toEqual(['disk:disk-hot']);
|
||||
expect(
|
||||
filterTrueNASStorageTopologyRows(rows, '', 'healthy', resolveFactory).map((row) => row.id),
|
||||
).toEqual(['disk:disk-cool']);
|
||||
|
||||
// An NVMe at 63C is under its 70C trigger.
|
||||
const warmNVMe = makeResource({
|
||||
...hotDisk,
|
||||
id: 'disk-nvme',
|
||||
physicalDisk: { ...hotDisk.physicalDisk!, diskType: 'nvme', temperature: 63 },
|
||||
});
|
||||
expect(mapTrueNASStorageStatus(warmNVMe, resolveFactory)).toBe('healthy');
|
||||
// A trigger the user raised, alerting switched off, and a retained reading
|
||||
// are never heat.
|
||||
expect(mapTrueNASStorageStatus(hotDisk, () => ({ warning: 55, critical: 60 }))).toBe('healthy');
|
||||
expect(mapTrueNASStorageStatus(hotDisk, () => null)).toBe('healthy');
|
||||
const retained = makeResource({
|
||||
...hotDisk,
|
||||
physicalDisk: {
|
||||
...hotDisk.physicalDisk!,
|
||||
collection: { temperature: { state: 'unavailable', source: 'truenas', reason: 'stale' } },
|
||||
},
|
||||
});
|
||||
expect(mapTrueNASStorageStatus(retained, resolveFactory)).toBe('healthy');
|
||||
});
|
||||
|
||||
it('renders expected receive-side replication readonly posture as healthy', () => {
|
||||
const replicatedDataset = makeResource({
|
||||
id: 'dataset-replica',
|
||||
|
||||
@@ -2,6 +2,11 @@ import { resolveResourcePlatformType } from '@/utils/sourcePlatforms';
|
||||
import { asTrimmedString } from '@/utils/stringUtils';
|
||||
import { hasImpairedResourceSource } from '@/utils/resourceSourceHealth';
|
||||
import { getTrueNASDatasetStateSummary } from '@/utils/truenasDatasetState';
|
||||
import type { MetricDisplayThresholds } from '@/utils/metricThresholds';
|
||||
import {
|
||||
getPhysicalDiskHeatSummary,
|
||||
isPhysicalDiskRunningHot,
|
||||
} from '@/features/storageBackups/diskTemperaturePresentation';
|
||||
import type {
|
||||
Resource,
|
||||
ResourceIncident,
|
||||
@@ -402,7 +407,10 @@ export function buildTrueNASStorageChildCounts(
|
||||
|
||||
export function buildTrueNASStorageTopologyRows(
|
||||
resources: Resource[],
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): TrueNASStorageTopologyRow[] {
|
||||
const compareStorageResources = (left: Resource, right: Resource): number =>
|
||||
compareStorageResourcesByStatus(left, right, resolveDiskTemperatureThresholds);
|
||||
const storageResources = resources.filter(
|
||||
(resource) =>
|
||||
isTrueNASPlatform(resource) &&
|
||||
@@ -556,8 +564,14 @@ function filterTrueNASStorageTopologyRow(
|
||||
row: TrueNASStorageTopologyRow,
|
||||
search: string,
|
||||
status: TrueNASStorageStatusFilter,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): boolean {
|
||||
if (status !== 'all' && mapTrueNASStorageStatus(row.resource) !== status) return false;
|
||||
if (
|
||||
status !== 'all' &&
|
||||
mapTrueNASStorageStatus(row.resource, resolveDiskTemperatureThresholds) !== status
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
if (!search.trim()) return true;
|
||||
if (row.kind.includes(search.trim().toLowerCase())) return true;
|
||||
return matchesTrueNASStorageSearch(row.resource, search);
|
||||
@@ -567,9 +581,14 @@ export function filterTrueNASStorageTopologyRows(
|
||||
rows: TrueNASStorageTopologyRow[],
|
||||
search: string,
|
||||
status: TrueNASStorageStatusFilter,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): TrueNASStorageTopologyRow[] {
|
||||
const directMatches = new Set(
|
||||
rows.filter((row) => filterTrueNASStorageTopologyRow(row, search, status)).map((row) => row.id),
|
||||
rows
|
||||
.filter((row) =>
|
||||
filterTrueNASStorageTopologyRow(row, search, status, resolveDiskTemperatureThresholds),
|
||||
)
|
||||
.map((row) => row.id),
|
||||
);
|
||||
if (directMatches.size === rows.length) return rows;
|
||||
|
||||
@@ -628,8 +647,11 @@ const titleize = (value: string): string =>
|
||||
.map((part) => part.charAt(0).toUpperCase() + part.slice(1).toLowerCase())
|
||||
.join(' ');
|
||||
|
||||
const storageStatusRank = (resource: Resource): number => {
|
||||
switch (mapTrueNASStorageStatus(resource)) {
|
||||
const storageStatusRank = (
|
||||
resource: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): number => {
|
||||
switch (mapTrueNASStorageStatus(resource, resolveDiskTemperatureThresholds)) {
|
||||
case 'attention':
|
||||
return 0;
|
||||
case 'offline':
|
||||
@@ -641,8 +663,14 @@ const storageStatusRank = (resource: Resource): number => {
|
||||
}
|
||||
};
|
||||
|
||||
const compareStorageResources = (left: Resource, right: Resource): number => {
|
||||
const rankDelta = storageStatusRank(left) - storageStatusRank(right);
|
||||
const compareStorageResourcesByStatus = (
|
||||
left: Resource,
|
||||
right: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): number => {
|
||||
const rankDelta =
|
||||
storageStatusRank(left, resolveDiskTemperatureThresholds) -
|
||||
storageStatusRank(right, resolveDiskTemperatureThresholds);
|
||||
if (rankDelta !== 0) return rankDelta;
|
||||
const nameDelta = resourceDisplayName(left).localeCompare(resourceDisplayName(right));
|
||||
if (nameDelta !== 0) return nameDelta;
|
||||
@@ -699,8 +727,45 @@ export function mapTrueNASIncidentSeverity(
|
||||
return 'info';
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves a disk type's alert disk temperature thresholds: the alerts store's
|
||||
* `getDiskTemperatureThresholds`. Without one, disk heat is not judged.
|
||||
*/
|
||||
export type TrueNASDiskTemperatureThresholdResolver = (
|
||||
diskType: string,
|
||||
) => MetricDisplayThresholds | null;
|
||||
|
||||
// Disk risk carries no heat, so a disk whose current reading has reached its
|
||||
// type's alert trigger is judged here, by the same policy as the Physical
|
||||
// Disks Running Hot verdict. Returns the heat reason, or null.
|
||||
function getTrueNASDiskHeatSummary(
|
||||
resource: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): string | null {
|
||||
const disk = resource.physicalDisk;
|
||||
if (!resolveDiskTemperatureThresholds || resource.type !== 'physical_disk' || !disk) return null;
|
||||
const temperature = disk.temperature ?? 0;
|
||||
const thresholds = resolveDiskTemperatureThresholds(disk.diskType || '');
|
||||
return isPhysicalDiskRunningHot({ temperature, collection: disk.collection }, thresholds)
|
||||
? getPhysicalDiskHeatSummary(temperature, thresholds)
|
||||
: null;
|
||||
}
|
||||
|
||||
// A hot disk reports `online` (disk risk carries no heat), so its status dot
|
||||
// takes the warning tone its Health cell gives it. On phones the dot is the
|
||||
// row's only health signal.
|
||||
export function getTrueNASStorageDotStatus(
|
||||
resource: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): string {
|
||||
const status = getTrueNASResourceDisplayStatus(resource);
|
||||
if (normalize(status) !== 'online') return status;
|
||||
return getTrueNASDiskHeatSummary(resource, resolveDiskTemperatureThresholds) ? 'warning' : status;
|
||||
}
|
||||
|
||||
export function mapTrueNASStorageStatus(
|
||||
resource: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): Exclude<TrueNASStorageStatusFilter, 'all'> | 'unknown' {
|
||||
const status = normalize(getTrueNASResourceDisplayStatus(resource));
|
||||
const zfsState = normalize(resource.storage?.zfsPoolState);
|
||||
@@ -719,6 +784,7 @@ export function mapTrueNASStorageStatus(
|
||||
if (diskHealth && !['passed', 'healthy', 'ok', 'unknown', 'unavailable'].includes(diskHealth)) {
|
||||
return 'attention';
|
||||
}
|
||||
if (getTrueNASDiskHeatSummary(resource, resolveDiskTemperatureThresholds)) return 'attention';
|
||||
if (['online', 'running', 'healthy'].includes(status)) return 'healthy';
|
||||
return 'unknown';
|
||||
}
|
||||
@@ -753,8 +819,11 @@ const reasonKey = (text: string): string =>
|
||||
// the failing member or test, so it leads. Pulse's derived risk summaries
|
||||
// restate the same pool and disk state, so they only stand in when TrueNAS
|
||||
// raised no alert; the row drawer still lists every one of them.
|
||||
export function getTrueNASStorageIssue(resource: Resource): TrueNASStorageIssue | null {
|
||||
const status = mapTrueNASStorageStatus(resource);
|
||||
export function getTrueNASStorageIssue(
|
||||
resource: Resource,
|
||||
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
|
||||
): TrueNASStorageIssue | null {
|
||||
const status = mapTrueNASStorageStatus(resource, resolveDiskTemperatureThresholds);
|
||||
if (status === 'healthy') return null;
|
||||
|
||||
const reasons: string[] = [];
|
||||
@@ -775,6 +844,7 @@ export function getTrueNASStorageIssue(resource: Resource): TrueNASStorageIssue
|
||||
if (reasons.length === leading) {
|
||||
for (const reason of resource.storage?.risk?.reasons ?? []) push(reason.summary);
|
||||
for (const reason of resource.physicalDisk?.risk?.reasons ?? []) push(reason.summary);
|
||||
push(getTrueNASDiskHeatSummary(resource, resolveDiskTemperatureThresholds));
|
||||
}
|
||||
// A read-only dataset raises no incident, so its state tag is the only
|
||||
// reason the provider gives. The drawer reads the same shared mapping.
|
||||
|
||||
@@ -374,6 +374,29 @@ describe('metricThresholds', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('switches every disk type off with the agent disk temperature default', () => {
|
||||
const config = {
|
||||
enabled: true,
|
||||
guestDefaults: {},
|
||||
nodeDefaults: {},
|
||||
agentDefaults: { diskTemperature: { trigger: 0, clear: 0 } },
|
||||
diskTempByType: {
|
||||
nvme: { trigger: 70, clear: 65 },
|
||||
sata: { trigger: 55, clear: 50 },
|
||||
},
|
||||
storageDefault: { trigger: 85, clear: 80 },
|
||||
overrides: { 'host-1': { diskTemperature: { trigger: 80, clear: 75 } } },
|
||||
} as AlertConfig;
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'nvme')).toBeNull();
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'sata')).toBeNull();
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, '')).toBeNull();
|
||||
// An explicit host override still applies, as CheckHost applies it.
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'nvme', 'host-1')).toEqual({
|
||||
warning: 75,
|
||||
critical: 80,
|
||||
});
|
||||
});
|
||||
|
||||
it('falls back to seeded per-type disk temperature defaults without config', () => {
|
||||
expect(resolveDiskTemperatureDisplayThresholds(null, 'nvme')).toEqual({
|
||||
warning: 65,
|
||||
|
||||
@@ -130,6 +130,12 @@ describe('temperature', () => {
|
||||
expect(getTemperatureTextClass(80)).toContain('text-red-600');
|
||||
});
|
||||
|
||||
it('colours no disk reading whose disk temperature alerting is off', () => {
|
||||
expect(getTemperatureTextClass(80, null, 'diskTemperature')).toBe('text-base-content');
|
||||
// Node temperatures keep the factory colours without configured thresholds.
|
||||
expect(getTemperatureTextClass(80, null, 'temperature')).toContain('text-red-600');
|
||||
});
|
||||
|
||||
it('honors configured warning and critical thresholds', () => {
|
||||
const thresholds = { warning: 80, critical: 85 };
|
||||
|
||||
|
||||
@@ -269,7 +269,9 @@ export const resolveMetricDisplayThresholds = (
|
||||
* Resolve display thresholds for a physical disk's SMART temperature.
|
||||
* Mirrors the backend precedence: an explicit diskTemperature override on the
|
||||
* host (or inherited linked resource) wins, then the per-type map
|
||||
* (diskTempByType: nvme/sas/sata), then the global agent default.
|
||||
* (diskTempByType: nvme/sas/sata), then the global agent default. A
|
||||
* switched-off agent default switches every type off, as disk temperature
|
||||
* alerts do. Null means disk temperature alerting is off for this disk.
|
||||
*/
|
||||
export const resolveDiskTemperatureDisplayThresholds = (
|
||||
config: AlertConfig | null,
|
||||
@@ -285,6 +287,11 @@ export const resolveDiskTemperatureDisplayThresholds = (
|
||||
return resolveThreshold(overrideValue, FACTORY_AGENT_DEFAULTS.diskTemperature, margin);
|
||||
}
|
||||
|
||||
const baseValue = getBaseThresholdValue(config?.agentDefaults, 'diskTemperature');
|
||||
if (baseValue !== undefined && resolveThreshold(baseValue, undefined, margin) === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const byTypeFallback = normalizedType ? FACTORY_DISK_TEMP_BY_TYPE[normalizedType] : undefined;
|
||||
if (normalizedType) {
|
||||
const byType = config?.diskTempByType?.[normalizedType];
|
||||
@@ -293,7 +300,6 @@ export const resolveDiskTemperatureDisplayThresholds = (
|
||||
}
|
||||
}
|
||||
|
||||
const baseValue = getBaseThresholdValue(config?.agentDefaults, 'diskTemperature');
|
||||
return resolveThreshold(
|
||||
baseValue,
|
||||
byTypeFallback ?? FACTORY_AGENT_DEFAULTS.diskTemperature,
|
||||
|
||||
@@ -123,6 +123,11 @@ export const getTemperatureTextClass = (
|
||||
if (celsius === null || celsius === undefined || !Number.isFinite(celsius)) {
|
||||
return 'text-muted';
|
||||
}
|
||||
// Null disk thresholds mean disk temperature alerting is off for that disk,
|
||||
// so no reading of it is warm or hot.
|
||||
if (metric === 'diskTemperature' && thresholds === null) {
|
||||
return 'text-base-content';
|
||||
}
|
||||
const readingSeverity = getMetricSeverity(
|
||||
celsius,
|
||||
metric,
|
||||
|
||||
@@ -68,6 +68,22 @@ func (a *AlertThresholdAdapter) GetGuestDiskThreshold() float64 {
|
||||
}, 90)
|
||||
}
|
||||
|
||||
// GetDiskTemperatureThreshold returns the disk temperature alert trigger and
|
||||
// clear values for a disk type, resolved exactly as disk temperature alerts
|
||||
// resolve them (alerts.Manager.DiskTemperatureThreshold).
|
||||
func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(diskType string) (float64, float64) {
|
||||
var threshold *alerts.HysteresisThreshold
|
||||
if a.manager != nil {
|
||||
threshold = a.manager.DiskTemperatureThreshold(diskType)
|
||||
} else {
|
||||
threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
|
||||
}
|
||||
if threshold == nil || threshold.Trigger <= 0 {
|
||||
return 0, 0
|
||||
}
|
||||
return threshold.Trigger, threshold.Clear
|
||||
}
|
||||
|
||||
// GetStorageThreshold returns the usage alert trigger threshold for storage (0-100%)
|
||||
func (a *AlertThresholdAdapter) GetStorageThreshold() float64 {
|
||||
return a.getThreshold(func(cfg alerts.AlertConfig) (float64, bool) {
|
||||
|
||||
@@ -82,3 +82,28 @@ func TestAlertThresholdAdapter_Fallbacks(t *testing.T) {
|
||||
t.Fatalf("GetStorageThreshold default = %v", a.GetStorageThreshold())
|
||||
}
|
||||
}
|
||||
|
||||
func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {
|
||||
if trigger, clear := NewAlertThresholdAdapter(nil).GetDiskTemperatureThreshold("nvme"); trigger != 70 || clear != 65 {
|
||||
t.Fatalf("no manager: nvme = %v/%v, want the factory 70/65", trigger, clear)
|
||||
}
|
||||
|
||||
mgr := alerts.NewManager()
|
||||
cfg := mgr.GetConfig()
|
||||
cfg.DiskTempByType["nvme"] = alerts.HysteresisThreshold{Trigger: 75, Clear: 70}
|
||||
mgr.UpdateConfig(cfg)
|
||||
a := NewAlertThresholdAdapter(mgr)
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("nvme"); trigger != 75 || clear != 70 {
|
||||
t.Fatalf("raised nvme = %v/%v, want 75/70", trigger, clear)
|
||||
}
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("sata"); trigger != 55 || clear != 50 {
|
||||
t.Fatalf("sata = %v/%v, want 55/50", trigger, clear)
|
||||
}
|
||||
|
||||
cfg = mgr.GetConfig()
|
||||
cfg.AgentDefaults.DiskTemperature = &alerts.HysteresisThreshold{Trigger: 0, Clear: 0}
|
||||
mgr.UpdateConfig(cfg)
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("nvme"); trigger != 0 || clear != 0 {
|
||||
t.Fatalf("disk temperature alerting off: nvme = %v/%v, want 0/0", trigger, clear)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/agentexec"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/providers"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
unifiedresources "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
)
|
||||
|
||||
@@ -51,6 +52,9 @@ type mockThresholdProvider struct {
|
||||
guestMem float64
|
||||
guestDisk float64
|
||||
storage float64
|
||||
// diskTemperature holds trigger/clear by disk type; a type it does not
|
||||
// list resolves to the factory disk temperature policy.
|
||||
diskTemperature map[string][2]float64
|
||||
}
|
||||
|
||||
func (m *mockThresholdProvider) GetNodeCPUThreshold() float64 { return m.nodeCPU }
|
||||
@@ -59,6 +63,15 @@ func (m *mockThresholdProvider) GetGuestCPUThreshold() float64 { return 0 }
|
||||
func (m *mockThresholdProvider) GetGuestMemoryThreshold() float64 { return m.guestMem }
|
||||
func (m *mockThresholdProvider) GetGuestDiskThreshold() float64 { return m.guestDisk }
|
||||
func (m *mockThresholdProvider) GetStorageThreshold() float64 { return m.storage }
|
||||
func (m *mockThresholdProvider) GetDiskTemperatureThreshold(diskType string) (float64, float64) {
|
||||
if limits, ok := m.diskTemperature[diskType]; ok {
|
||||
return limits[0], limits[1]
|
||||
}
|
||||
if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
|
||||
return threshold.Trigger, threshold.Clear
|
||||
}
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
type mockUnifiedResourceProvider struct {
|
||||
UnifiedResourceProvider
|
||||
|
||||
@@ -66,6 +66,10 @@ type ThresholdProvider interface {
|
||||
GetGuestDiskThreshold() float64
|
||||
// GetStorageThreshold returns the usage alert trigger threshold for storage (0-100%)
|
||||
GetStorageThreshold() float64
|
||||
// GetDiskTemperatureThreshold returns the disk temperature alert trigger
|
||||
// and clear values (°C) for a disk type. A zero trigger means disk
|
||||
// temperature alerting is off.
|
||||
GetDiskTemperatureThreshold(diskType string) (trigger, clear float64)
|
||||
}
|
||||
|
||||
// AlertResolver provides the ability to review and resolve alerts
|
||||
|
||||
@@ -1930,8 +1930,9 @@ type patrolPhysicalDiskRow struct {
|
||||
wearout int
|
||||
// temperature.Collected is the only temperature Patrol may judge as
|
||||
// heat; a retained last-known value is shown as context.
|
||||
temperature tools.DiskTemperature
|
||||
smartEvidence []string
|
||||
temperature tools.DiskTemperature
|
||||
temperatureLimits diskTemperatureLimits
|
||||
smartEvidence []string
|
||||
}
|
||||
|
||||
type patrolPrecomputeNodeSource struct {
|
||||
@@ -2464,7 +2465,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
health = "UNKNOWN"
|
||||
}
|
||||
|
||||
rows = append(rows, patrolPhysicalDiskRow{
|
||||
row := patrolPhysicalDiskRow{
|
||||
id: r.ID,
|
||||
name: name,
|
||||
node: strings.TrimSpace(r.ParentName),
|
||||
@@ -2477,7 +2478,9 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
wearout: r.PhysicalDisk.Wearout,
|
||||
temperature: tools.SplitDiskTemperature(r.PhysicalDisk.Temperature, r.PhysicalDisk.Collection),
|
||||
smartEvidence: unifiedPhysicalDiskSMARTIssueParts(r.PhysicalDisk.SMART),
|
||||
})
|
||||
}
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(row.diskType)
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows
|
||||
}
|
||||
@@ -2505,7 +2508,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
health = "UNKNOWN"
|
||||
}
|
||||
|
||||
rows = append(rows, patrolPhysicalDiskRow{
|
||||
row := patrolPhysicalDiskRow{
|
||||
id: d.ID,
|
||||
name: name,
|
||||
node: strings.TrimSpace(d.Node),
|
||||
@@ -2518,7 +2521,9 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
wearout: d.Wearout,
|
||||
temperature: tools.SplitDiskTemperature(d.Temperature, d.Collection),
|
||||
smartEvidence: modelPhysicalDiskSMARTIssueParts(d.SmartAttributes),
|
||||
})
|
||||
}
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(row.diskType)
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows
|
||||
}
|
||||
@@ -2534,7 +2539,7 @@ func patrolPhysicalDiskHealthIssue(row patrolPhysicalDiskRow) bool {
|
||||
(row.wearout == 0 &&
|
||||
(strings.EqualFold(row.diskType, "ssd") || strings.EqualFold(row.diskType, "nvme")))) &&
|
||||
row.wearout < 20) ||
|
||||
row.temperature.Collected > 55 ||
|
||||
row.temperatureLimits.hot(row.temperature.Collected) ||
|
||||
len(row.smartEvidence) > 0
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
package ai
|
||||
|
||||
import "github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
|
||||
// diskTemperatureLimits is the alert disk temperature policy for one disk
|
||||
// type, the same policy disk temperature alerts and the Physical Disks Health
|
||||
// verdict judge heat by. A disk is hot at the trigger and stays hot until it
|
||||
// cools to the clear value. A zero trigger means disk temperature alerting is
|
||||
// off, so no reading counts as heat.
|
||||
type diskTemperatureLimits struct {
|
||||
trigger float64
|
||||
clear float64
|
||||
}
|
||||
|
||||
// diskTemperatureLimitsFor resolves the policy from the user's alert
|
||||
// configuration, or from the factory alert configuration when Patrol has no
|
||||
// threshold provider.
|
||||
func diskTemperatureLimitsFor(provider ThresholdProvider, diskType string) diskTemperatureLimits {
|
||||
var trigger, clear float64
|
||||
if provider != nil {
|
||||
trigger, clear = provider.GetDiskTemperatureThreshold(diskType)
|
||||
} else if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
|
||||
trigger, clear = threshold.Trigger, threshold.Clear
|
||||
}
|
||||
if trigger <= 0 {
|
||||
return diskTemperatureLimits{}
|
||||
}
|
||||
if clear <= 0 || clear > trigger {
|
||||
clear = trigger
|
||||
}
|
||||
return diskTemperatureLimits{trigger: trigger, clear: clear}
|
||||
}
|
||||
|
||||
// hot reports a reading at or above the alert trigger.
|
||||
func (l diskTemperatureLimits) hot(temperature int) bool {
|
||||
return l.trigger > 0 && temperature > 0 && float64(temperature) >= l.trigger
|
||||
}
|
||||
|
||||
// cooled reports a reading at or below the clear value, where a disk
|
||||
// temperature alert recovers. With no band below the trigger, a reading at
|
||||
// the trigger still fires, so only a reading under it has cooled.
|
||||
func (l diskTemperatureLimits) cooled(temperature int) bool {
|
||||
if l.trigger <= 0 || temperature <= 0 {
|
||||
return true
|
||||
}
|
||||
if l.clear >= l.trigger {
|
||||
return float64(temperature) < l.trigger
|
||||
}
|
||||
return float64(temperature) <= l.clear
|
||||
}
|
||||
@@ -0,0 +1,228 @@
|
||||
package ai
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
)
|
||||
|
||||
// raisedNVMeThresholds is a user who raised the NVMe disk temperature trigger
|
||||
// to 75C (clear 70) in Alerts and left every other type at its default.
|
||||
func raisedNVMeThresholds() *mockThresholdProvider {
|
||||
return &mockThresholdProvider{diskTemperature: map[string][2]float64{"nvme": {75, 70}}}
|
||||
}
|
||||
|
||||
func TestDiskTemperatureLimitsFollowTheAlertPolicy(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
provider ThresholdProvider
|
||||
diskType string
|
||||
want diskTemperatureLimits
|
||||
}{
|
||||
{name: "factory nvme", diskType: "nvme", want: diskTemperatureLimits{trigger: 70, clear: 65}},
|
||||
{name: "factory sata", diskType: "sata", want: diskTemperatureLimits{trigger: 55, clear: 50}},
|
||||
{name: "factory untyped", diskType: "", want: diskTemperatureLimits{trigger: 55, clear: 50}},
|
||||
{name: "raised nvme", provider: raisedNVMeThresholds(), diskType: "nvme", want: diskTemperatureLimits{trigger: 75, clear: 70}},
|
||||
{name: "raised nvme leaves sata", provider: raisedNVMeThresholds(), diskType: "sata", want: diskTemperatureLimits{trigger: 55, clear: 50}},
|
||||
{name: "alerting off", provider: &mockThresholdProvider{diskTemperature: map[string][2]float64{"sata": {0, 0}}}, diskType: "sata", want: diskTemperatureLimits{}},
|
||||
{name: "clear above trigger", provider: &mockThresholdProvider{diskTemperature: map[string][2]float64{"sas": {60, 64}}}, diskType: "sas", want: diskTemperatureLimits{trigger: 60, clear: 60}},
|
||||
} {
|
||||
if got := diskTemperatureLimitsFor(tc.provider, tc.diskType); got != tc.want {
|
||||
t.Errorf("%s: limits = %+v, want %+v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
|
||||
off := diskTemperatureLimits{}
|
||||
if off.hot(95) || !off.cooled(95) {
|
||||
t.Fatal("with disk temperature alerting off no reading is heat")
|
||||
}
|
||||
}
|
||||
|
||||
func diskTemperatureTriageState(provider ThresholdProvider) patrolRuntimeState {
|
||||
state := newPatrolRuntimeState(models.StateSnapshot{PhysicalDisks: []models.PhysicalDisk{
|
||||
{ID: "nvme-63", DevPath: "/dev/nvme0n1", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 63},
|
||||
{ID: "nvme-66", DevPath: "/dev/nvme1n1", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 66},
|
||||
{ID: "nvme-72", DevPath: "/dev/nvme2n1", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 72},
|
||||
{ID: "sata-52", DevPath: "/dev/sda", Type: "sata", Health: "PASSED", Wearout: -1, Temperature: 52},
|
||||
{ID: "sata-56", DevPath: "/dev/sdb", Type: "sata", Health: "PASSED", Wearout: -1, Temperature: 56},
|
||||
}})
|
||||
state.thresholdProvider = provider
|
||||
return state
|
||||
}
|
||||
|
||||
// Patrol judges disk heat by the alert disk temperature policy: a warning from
|
||||
// the per-type trigger and nothing below it, the band under the trigger being
|
||||
// a Temp cell colour rather than a finding. An NVMe at 63C is not hot; a SATA
|
||||
// disk at 56C is.
|
||||
func TestTriageDiskTemperatureFollowsTheAlertPolicy(t *testing.T) {
|
||||
temperatureFlag := func(flags []TriageFlag, id string) *TriageFlag {
|
||||
return triageFindFlag(flags, func(f TriageFlag) bool {
|
||||
return f.ResourceID == id && strings.Contains(f.Reason, "Disk temperature")
|
||||
})
|
||||
}
|
||||
expect := func(flags []TriageFlag, id, severity string, threshold float64) {
|
||||
t.Helper()
|
||||
flag := temperatureFlag(flags, id)
|
||||
if severity == "" {
|
||||
if flag != nil {
|
||||
t.Errorf("%s: unexpected temperature flag %+v", id, *flag)
|
||||
}
|
||||
return
|
||||
}
|
||||
if flag == nil || flag.Severity != severity || flag.Threshold != threshold {
|
||||
t.Errorf("%s: temperature flag = %+v, want %s at threshold %.0f", id, flag, severity, threshold)
|
||||
}
|
||||
}
|
||||
|
||||
flags := triageDiskHealthChecksState(diskTemperatureTriageState(nil), nil)
|
||||
expect(flags, "nvme-63", "", 0)
|
||||
expect(flags, "nvme-66", "", 0)
|
||||
expect(flags, "nvme-72", "warning", 70)
|
||||
expect(flags, "sata-52", "", 0)
|
||||
expect(flags, "sata-56", "warning", 55)
|
||||
if flag := temperatureFlag(flags, "sata-56"); flag == nil || flag.Reason != "Disk temperature 56°C (alert threshold: 55°C)" {
|
||||
t.Errorf("sata-56 reason = %+v, want the reading and the alert trigger", flag)
|
||||
}
|
||||
|
||||
flags = triageDiskHealthChecksState(diskTemperatureTriageState(raisedNVMeThresholds()), nil)
|
||||
expect(flags, "nvme-72", "", 0)
|
||||
expect(flags, "sata-56", "warning", 55)
|
||||
|
||||
off := &mockThresholdProvider{diskTemperature: map[string][2]float64{"nvme": {0, 0}, "sata": {0, 0}}}
|
||||
if flags := triageDiskHealthChecksState(diskTemperatureTriageState(off), nil); len(flags) != 0 {
|
||||
t.Fatalf("disk temperature alerting off should raise no heat flags, got %+v", flags)
|
||||
}
|
||||
}
|
||||
|
||||
// The seed's disk issue list and finding verification share the policy: a
|
||||
// disk is an issue from the trigger and recovers below the clear value.
|
||||
func TestPatrolDiskHeatIssueAndRecoveryFollowTheAlertPolicy(t *testing.T) {
|
||||
issues := map[string]bool{}
|
||||
for _, row := range patrolPhysicalDiskRows(diskTemperatureTriageState(nil), nil) {
|
||||
issues[row.id] = patrolPhysicalDiskHealthIssue(row)
|
||||
}
|
||||
for id, want := range map[string]bool{"nvme-63": false, "nvme-66": false, "nvme-72": true, "sata-52": false, "sata-56": true} {
|
||||
if issues[id] != want {
|
||||
t.Errorf("%s: health issue = %v, want %v", id, issues[id], want)
|
||||
}
|
||||
}
|
||||
for _, row := range patrolPhysicalDiskRows(diskTemperatureTriageState(raisedNVMeThresholds()), nil) {
|
||||
if row.id == "nvme-72" && patrolPhysicalDiskHealthIssue(row) {
|
||||
t.Error("nvme-72 under a raised 75C trigger should not be an issue")
|
||||
}
|
||||
}
|
||||
|
||||
state := diskTemperatureTriageState(nil)
|
||||
for id, want := range map[string]bool{"nvme-63": true, "nvme-66": false, "nvme-72": false, "sata-52": false} {
|
||||
recovered, err := verifyMetricRecoveredState(state, PatrolThresholds{}, "disk-high", id, "physical_disk")
|
||||
if err != nil {
|
||||
t.Fatalf("%s: verification error %v", id, err)
|
||||
}
|
||||
if recovered != want {
|
||||
t.Errorf("%s: recovered = %v, want %v", id, recovered, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceContextListsDisksHotByTheAlertPolicy(t *testing.T) {
|
||||
registry := unifiedresources.NewRegistry(nil)
|
||||
now := time.Now()
|
||||
registry.IngestSnapshot(models.StateSnapshot{PhysicalDisks: []models.PhysicalDisk{
|
||||
{ID: "pve-nvme", Node: "pve", Instance: "lab", DevPath: "/dev/nvme0n1", Model: "Samsung 980", Serial: "N63", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 63, LastChecked: now},
|
||||
{ID: "pve-nvme-72", Node: "pve", Instance: "lab", DevPath: "/dev/nvme1n1", Model: "Samsung 990", Serial: "N72", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 72, LastChecked: now},
|
||||
{ID: "pve-sata", Node: "pve", Instance: "lab", DevPath: "/dev/sda", Model: "WDC WD80EFAX", Serial: "S56", Type: "sata", Health: "PASSED", Wearout: -1, Temperature: 56, LastChecked: now},
|
||||
}})
|
||||
|
||||
s := &Service{}
|
||||
s.SetUnifiedResourceProvider(unifiedresources.NewUnifiedAIAdapter(registry))
|
||||
attention := func() string {
|
||||
got := s.buildUnifiedResourceContext()
|
||||
_, section, found := strings.Cut(got, "Physical Disks Needing Attention")
|
||||
if !found {
|
||||
return ""
|
||||
}
|
||||
section, _, _ = strings.Cut(section, "\n\n")
|
||||
return section
|
||||
}
|
||||
|
||||
// Disk names are redacted in this context, so the readings identify them.
|
||||
section := attention()
|
||||
if !strings.Contains(section, "Temp: 56C") || !strings.Contains(section, "Temp: 72C") || strings.Contains(section, "Temp: 63C") {
|
||||
t.Fatalf("factory policy: attention section = %q, want the SATA 56C and NVMe 72C disks only", section)
|
||||
}
|
||||
|
||||
s.SetPatrolThresholdProvider(raisedNVMeThresholds())
|
||||
section = attention()
|
||||
if !strings.Contains(section, "Temp: 56C") || strings.Contains(section, "Temp: 72C") || strings.Contains(section, "Temp: 63C") {
|
||||
t.Fatalf("raised NVMe trigger: attention section = %q, want only the SATA 56C disk", section)
|
||||
}
|
||||
}
|
||||
|
||||
// A Patrol run judges disk heat by the thresholds the user configured, through
|
||||
// both ways a run builds its state.
|
||||
func TestPatrolRunStateCarriesTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
snapshot := models.StateSnapshot{PhysicalDisks: []models.PhysicalDisk{
|
||||
{ID: "nvme-72", DevPath: "/dev/nvme0n1", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 72},
|
||||
}}
|
||||
ps := NewPatrolService(nil, &mockStateProvider{state: snapshot})
|
||||
if got := ps.currentPatrolRuntimeState().diskTemperatureLimits("nvme"); got.trigger != 70 {
|
||||
t.Fatalf("no provider: nvme trigger = %v, want the factory 70", got.trigger)
|
||||
}
|
||||
|
||||
ps.SetThresholdProvider(raisedNVMeThresholds())
|
||||
for name, state := range map[string]patrolRuntimeState{
|
||||
"current": ps.currentPatrolRuntimeState(),
|
||||
"snapshot": ps.patrolRuntimeStateForSnapshot(snapshot),
|
||||
} {
|
||||
if got := state.diskTemperatureLimits("nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
t.Errorf("%s state: nvme limits = %+v, want the configured 75/70", name, got)
|
||||
}
|
||||
if flags := triageDiskHealthChecksState(state, nil); len(flags) != 0 {
|
||||
t.Errorf("%s state: nvme at 72C under a 75C trigger flagged %+v", name, flags)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A scoped Patrol run filters its state; the filter must keep the configured
|
||||
// thresholds, or a scoped run would judge heat by the factory triggers.
|
||||
func TestScopedPatrolStateKeepsTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
state := diskTemperatureTriageState(raisedNVMeThresholds())
|
||||
scoped := filterPatrolStateByScopeState(state, PatrolScope{ResourceIDs: []string{"nvme-72"}})
|
||||
if got := scoped.diskTemperatureLimits("nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
t.Fatalf("scoped state nvme limits = %+v, want the configured 75/70", got)
|
||||
}
|
||||
if flags := triageDiskHealthChecksState(scoped, nil); len(flags) != 0 {
|
||||
t.Fatalf("scoped run flagged nvme at 72C under a 75C trigger: %+v", flags)
|
||||
}
|
||||
}
|
||||
|
||||
// The alert threshold provider can arrive before Patrol exists; Patrol must
|
||||
// still start with it.
|
||||
func TestPatrolStartsWithAThresholdProviderSetBeforeIt(t *testing.T) {
|
||||
s := NewService(nil, nil)
|
||||
s.SetPatrolThresholdProvider(raisedNVMeThresholds())
|
||||
s.SetStateProvider(&mockStateProvider{})
|
||||
patrol := s.GetPatrolService()
|
||||
if patrol == nil {
|
||||
t.Fatal("expected Patrol to start once a state provider exists")
|
||||
}
|
||||
if got := patrol.currentPatrolRuntimeState().diskTemperatureLimits("nvme"); got.trigger != 75 {
|
||||
t.Fatalf("Patrol nvme trigger = %v, want the provider's 75", got.trigger)
|
||||
}
|
||||
}
|
||||
|
||||
// A heat finding recovers at the clear value, where its alert recovers.
|
||||
func TestDiskHeatRecoversAtTheClearValue(t *testing.T) {
|
||||
limits := diskTemperatureLimitsFor(nil, "nvme")
|
||||
if !limits.cooled(65) || limits.cooled(66) {
|
||||
t.Fatalf("nvme limits %+v: want 65C cooled and 66C still warm", limits)
|
||||
}
|
||||
// With no band below the trigger, a reading at the trigger is still hot.
|
||||
noBand := diskTemperatureLimitsFor(&mockThresholdProvider{diskTemperature: map[string][2]float64{"sas": {70, 70}}}, "sas")
|
||||
if !noBand.hot(70) || noBand.cooled(70) || !noBand.cooled(69) {
|
||||
t.Fatalf("no-band limits %+v: want 70C hot and not cooled, 69C cooled", noBand)
|
||||
}
|
||||
}
|
||||
@@ -2291,7 +2291,9 @@ func verifyMetricRecoveredState(snap patrolRuntimeState, thresholds PatrolThresh
|
||||
if disk.wearout >= 0 && disk.wearout < 20 {
|
||||
return false, nil
|
||||
}
|
||||
if disk.temperature > 55 {
|
||||
// A hot disk recovers once it cools to the alert clear
|
||||
// value, as its disk temperature alert does.
|
||||
if !disk.temperatureLimits.cooled(disk.temperature) {
|
||||
return false, nil
|
||||
}
|
||||
return true, nil
|
||||
@@ -2315,9 +2317,10 @@ func verifyMetricRecoveredState(snap patrolRuntimeState, thresholds PatrolThresh
|
||||
}
|
||||
|
||||
type patrolPhysicalDiskVerification struct {
|
||||
health string
|
||||
wearout int
|
||||
temperature int
|
||||
health string
|
||||
wearout int
|
||||
temperature int
|
||||
temperatureLimits diskTemperatureLimits
|
||||
}
|
||||
|
||||
type patrolPhysicalDiskVisitor func(identifiers []string, verification patrolPhysicalDiskVerification) bool
|
||||
@@ -2461,9 +2464,10 @@ func patrolVisitPhysicalDiskVerification(snap patrolRuntimeState, visit patrolPh
|
||||
rows := patrolPhysicalDiskRows(snap, nil)
|
||||
for _, disk := range rows {
|
||||
if !visit([]string{disk.id, disk.name, disk.devPath, disk.model}, patrolPhysicalDiskVerification{
|
||||
health: strings.TrimSpace(disk.health),
|
||||
wearout: disk.wearout,
|
||||
temperature: disk.temperature.Collected,
|
||||
health: strings.TrimSpace(disk.health),
|
||||
wearout: disk.wearout,
|
||||
temperature: disk.temperature.Collected,
|
||||
temperatureLimits: disk.temperatureLimits,
|
||||
}) {
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -1464,6 +1464,7 @@ func newPatrolScopedFilterState(snap patrolRuntimeState) patrolScopedFilterState
|
||||
filtered: patrolRuntimeState{
|
||||
readState: snap.readState,
|
||||
unifiedResourceProvider: snap.unifiedResourceProvider,
|
||||
thresholdProvider: snap.thresholdProvider,
|
||||
},
|
||||
includedResourceIDs: make(map[string]bool),
|
||||
includedGuestVMIDs: make(map[int]bool),
|
||||
|
||||
@@ -89,6 +89,15 @@ type patrolRuntimeState struct {
|
||||
ConnectionHealth map[string]bool
|
||||
ActiveAlerts []models.Alert
|
||||
RecentlyResolved []models.ResolvedAlert
|
||||
// thresholdProvider carries the user's alert thresholds into the run. Nil
|
||||
// means the factory alert configuration.
|
||||
thresholdProvider ThresholdProvider
|
||||
}
|
||||
|
||||
// diskTemperatureLimits resolves the alert disk temperature policy for one
|
||||
// disk type.
|
||||
func (s patrolRuntimeState) diskTemperatureLimits(diskType string) diskTemperatureLimits {
|
||||
return diskTemperatureLimitsFor(s.thresholdProvider, diskType)
|
||||
}
|
||||
|
||||
func newPatrolRuntimeState(snapshot models.StateSnapshot) patrolRuntimeState {
|
||||
@@ -171,11 +180,16 @@ func (p *PatrolService) currentPatrolRuntimeState() patrolRuntimeState {
|
||||
stateProvider := p.stateProvider
|
||||
readState := p.readState
|
||||
unifiedResourceProvider := p.unifiedResourceProvider
|
||||
thresholdProvider := p.thresholdProvider
|
||||
p.mu.RUnlock()
|
||||
var state patrolRuntimeState
|
||||
if stateProvider == nil {
|
||||
return emptyPatrolRuntimeState(readState, unifiedResourceProvider)
|
||||
state = emptyPatrolRuntimeState(readState, unifiedResourceProvider)
|
||||
} else {
|
||||
state = newPatrolRuntimeStateWithProviders(stateProvider.ReadSnapshot(), readState, unifiedResourceProvider)
|
||||
}
|
||||
return newPatrolRuntimeStateWithProviders(stateProvider.ReadSnapshot(), readState, unifiedResourceProvider)
|
||||
state.thresholdProvider = thresholdProvider
|
||||
return state
|
||||
}
|
||||
|
||||
func (p *PatrolService) hasPatrolRuntimeInputs() bool {
|
||||
@@ -194,8 +208,11 @@ func (p *PatrolService) patrolRuntimeStateForSnapshot(snapshot models.StateSnaps
|
||||
p.mu.RLock()
|
||||
readState := p.readState
|
||||
unifiedResourceProvider := p.unifiedResourceProvider
|
||||
thresholdProvider := p.thresholdProvider
|
||||
p.mu.RUnlock()
|
||||
return newPatrolRuntimeStateWithProviders(snapshot, readState, unifiedResourceProvider)
|
||||
state := newPatrolRuntimeStateWithProviders(snapshot, readState, unifiedResourceProvider)
|
||||
state.thresholdProvider = thresholdProvider
|
||||
return state
|
||||
}
|
||||
|
||||
func patrolVisitRuntimeResources(s patrolRuntimeState, visit func(patrolRuntimeResourceRecord) bool) {
|
||||
|
||||
@@ -20,6 +20,9 @@ func (m MockThresholdProvider) GetNodeMemoryThreshold() float64 { return m.Node
|
||||
func (m MockThresholdProvider) GetGuestMemoryThreshold() float64 { return m.GuestMem }
|
||||
func (m MockThresholdProvider) GetGuestDiskThreshold() float64 { return m.GuestDisk }
|
||||
func (m MockThresholdProvider) GetStorageThreshold() float64 { return m.Storage }
|
||||
func (m MockThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
func TestCalculatePatrolThresholds_Default(t *testing.T) {
|
||||
// Test default behavior (exact mode)
|
||||
|
||||
@@ -345,13 +345,16 @@ func triageDiskHealthChecksState(snap patrolRuntimeState, scopedSet map[string]b
|
||||
flags := make([]TriageFlag, 0)
|
||||
|
||||
for _, disk := range patrolPhysicalDiskRows(snap, scopedSet) {
|
||||
flags = append(flags, triagePhysicalDiskFlags(disk.id, disk.name, disk.health, disk.wearout, disk.temperature.Collected)...)
|
||||
flags = append(flags, triagePhysicalDiskFlags(disk.id, disk.name, disk.health, disk.wearout, disk.temperature.Collected, disk.temperatureLimits)...)
|
||||
}
|
||||
|
||||
return flags
|
||||
}
|
||||
|
||||
func triagePhysicalDiskFlags(resourceID, resourceName, health string, wearout, temperature int) []TriageFlag {
|
||||
// triagePhysicalDiskFlags judges heat by the alert disk temperature policy: a
|
||||
// warning from the alert trigger, the reading the disk's temperature alert and
|
||||
// its Running Hot verdict start at.
|
||||
func triagePhysicalDiskFlags(resourceID, resourceName, health string, wearout, temperature int, temperatureLimits diskTemperatureLimits) []TriageFlag {
|
||||
flags := make([]TriageFlag, 0, 3)
|
||||
if health != "" && !strings.EqualFold(health, "PASSED") && !strings.EqualFold(health, "UNKNOWN") && !strings.EqualFold(health, "OK") {
|
||||
flags = append(flags, TriageFlag{
|
||||
@@ -377,17 +380,17 @@ func triagePhysicalDiskFlags(resourceID, resourceName, health string, wearout, t
|
||||
Threshold: 20,
|
||||
})
|
||||
}
|
||||
if temperature > 55 {
|
||||
if temperatureLimits.hot(temperature) {
|
||||
flags = append(flags, TriageFlag{
|
||||
ResourceID: resourceID,
|
||||
ResourceName: resourceName,
|
||||
ResourceType: "physical_disk",
|
||||
Category: "health",
|
||||
Severity: "warning",
|
||||
Reason: fmt.Sprintf("Disk temperature %d°C", temperature),
|
||||
Reason: fmt.Sprintf("Disk temperature %d°C (alert threshold: %.0f°C)", temperature, temperatureLimits.trigger),
|
||||
Metric: "disk",
|
||||
Value: float64(temperature),
|
||||
Threshold: 55,
|
||||
Threshold: temperatureLimits.trigger,
|
||||
})
|
||||
}
|
||||
return flags
|
||||
|
||||
@@ -52,6 +52,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
|
||||
urp := s.unifiedResourceProvider
|
||||
ap := s.alertProvider
|
||||
agentServer := s.agentServer
|
||||
thresholdProvider := s.thresholdProvider
|
||||
s.mu.RUnlock()
|
||||
|
||||
if urp != nil {
|
||||
@@ -410,13 +411,14 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
|
||||
attention := make([]unifiedresources.Resource, 0)
|
||||
for _, disk := range physicalDisks {
|
||||
health := ""
|
||||
temperature := 0
|
||||
hot := false
|
||||
if disk.PhysicalDisk != nil {
|
||||
health = strings.ToUpper(strings.TrimSpace(disk.PhysicalDisk.Health))
|
||||
// A retained last-known reading is not current heat.
|
||||
temperature = tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected
|
||||
collected := tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected
|
||||
hot = diskTemperatureLimitsFor(thresholdProvider, disk.PhysicalDisk.DiskType).hot(collected)
|
||||
}
|
||||
if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || temperature >= 50 {
|
||||
if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || hot {
|
||||
attention = append(attention, disk)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -261,6 +261,7 @@ type Service struct {
|
||||
patrolAutopilotPolicy func() unifiedresources.PatrolAutopilotServerPolicy
|
||||
patrolObjectiveStore *PatrolObjectiveStore
|
||||
patrolService *PatrolService // Background AI monitoring service
|
||||
thresholdProvider ThresholdProvider // User-configured alert thresholds; nil means factory defaults
|
||||
metadataProvider MetadataProvider // Enables AI to update resource URLs
|
||||
incidentStore *memory.IncidentStore // Alert-scoped investigation memory; not the canonical durable resource history
|
||||
chatService ChatServiceProvider // Chat service for investigation orchestrator
|
||||
@@ -584,6 +585,9 @@ func (s *Service) initPatrolServiceLocked() {
|
||||
|
||||
s.patrolService = NewPatrolService(s, s.stateProvider)
|
||||
s.patrolService.SetObjectiveStore(s.patrolObjectiveStore)
|
||||
if s.thresholdProvider != nil {
|
||||
s.patrolService.SetThresholdProvider(s.thresholdProvider)
|
||||
}
|
||||
if s.knowledgeStore != nil {
|
||||
s.patrolService.SetKnowledgeStore(s.knowledgeStore)
|
||||
}
|
||||
@@ -831,9 +835,10 @@ func (s *Service) ClearCostHistory() error {
|
||||
// SetPatrolThresholdProvider sets the threshold provider for patrol
|
||||
// This should be called with an AlertThresholdAdapter to connect patrol to user-configured thresholds
|
||||
func (s *Service) SetPatrolThresholdProvider(provider ThresholdProvider) {
|
||||
s.mu.RLock()
|
||||
s.mu.Lock()
|
||||
s.thresholdProvider = provider
|
||||
patrol := s.patrolService
|
||||
s.mu.RUnlock()
|
||||
s.mu.Unlock()
|
||||
|
||||
if patrol != nil {
|
||||
patrol.SetThresholdProvider(provider)
|
||||
|
||||
+33
-6
@@ -990,21 +990,50 @@ func hostDiskTemperatureResourceID(hostID, device string) string {
|
||||
}
|
||||
|
||||
// hostDiskTemperatureThresholdNoLock returns the threshold for one SMART disk
|
||||
// given the host's resolved disk temperature threshold. An enabled threshold
|
||||
// gives way to the disk type's DiskTempByType entry unless an explicit host or
|
||||
// linked-resource override set it. Callers must hold m.mu.
|
||||
// given the host's resolved disk temperature threshold. Callers must hold m.mu.
|
||||
func (m *Manager) hostDiskTemperatureThresholdNoLock(hostThreshold *HysteresisThreshold, overridden bool, diskType string) *HysteresisThreshold {
|
||||
return diskTemperatureThresholdForType(m.config.DiskTempByType, hostThreshold, overridden, diskType)
|
||||
}
|
||||
|
||||
// diskTemperatureThresholdForType is the disk temperature policy. An enabled
|
||||
// host threshold gives way to the disk type's DiskTempByType entry unless an
|
||||
// explicit host or linked-resource override set it; a disabled one switches
|
||||
// disk temperature alerting off for every type.
|
||||
func diskTemperatureThresholdForType(byType map[string]HysteresisThreshold, hostThreshold *HysteresisThreshold, overridden bool, diskType string) *HysteresisThreshold {
|
||||
if hostThreshold == nil || hostThreshold.Trigger <= 0 || overridden {
|
||||
return hostThreshold
|
||||
}
|
||||
if diskType = strings.ToLower(strings.TrimSpace(diskType)); diskType != "" {
|
||||
if th, ok := m.config.DiskTempByType[diskType]; ok {
|
||||
if th, ok := byType[diskType]; ok {
|
||||
return &th
|
||||
}
|
||||
}
|
||||
return hostThreshold
|
||||
}
|
||||
|
||||
// DiskTemperatureThreshold returns the threshold that judges a physical disk's
|
||||
// temperature when no host override applies: the disk type's DiskTempByType
|
||||
// entry, else the agent default. This is the canonical disk heat policy. Disk
|
||||
// temperature alerts, Patrol and the Physical Disks Health verdict all judge
|
||||
// heat against it; disk risk does not. The disk is hot at the trigger, and the
|
||||
// clear value is where it stops being hot. A nil threshold or a non-positive
|
||||
// trigger means disk temperature alerting is off.
|
||||
func (m *Manager) DiskTemperatureThreshold(diskType string) *HysteresisThreshold {
|
||||
if m == nil {
|
||||
return DefaultDiskTemperatureThreshold(diskType)
|
||||
}
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
|
||||
}
|
||||
|
||||
// DefaultDiskTemperatureThreshold is DiskTemperatureThreshold under the
|
||||
// factory alert configuration, for callers with no alert manager.
|
||||
func DefaultDiskTemperatureThreshold(diskType string) *HysteresisThreshold {
|
||||
config := defaultAlertConfig()
|
||||
return cloneThreshold(diskTemperatureThresholdForType(config.DiskTempByType, config.AgentDefaults.DiskTemperature, false, diskType))
|
||||
}
|
||||
|
||||
// lowestHostDiskTemperatureThresholdNoLock returns the lowest enabled
|
||||
// threshold any SMART disk of the host can be evaluated against. An alert
|
||||
// whose disk type is unknown is judged against it, so a config save never
|
||||
@@ -1547,8 +1576,6 @@ func splitSMARTAlertReasons(reasons []storagehealth.Reason) ([]storagehealth.Rea
|
||||
switch reason.Code {
|
||||
case "wearout_low", "nvme_available_spare_low", "nvme_percentage_used_high":
|
||||
wearReasons = append(wearReasons, reason)
|
||||
case "temperature_high":
|
||||
continue
|
||||
default:
|
||||
healthReasons = append(healthReasons, reason)
|
||||
}
|
||||
|
||||
@@ -1387,3 +1387,61 @@ func TestCheckMetricResolveRemovesCanonicallyKeyedAlert(t *testing.T) {
|
||||
t.Fatal("expected a recently-resolved entry for the cleared alert")
|
||||
}
|
||||
}
|
||||
|
||||
// DiskTemperatureThreshold is the policy Patrol and the Physical Disks Health
|
||||
// verdict judge disk heat by, so it must resolve every disk type exactly as
|
||||
// CheckHost does: the per-type trigger the user set, else the agent default,
|
||||
// and nothing at all once the agent default is switched off.
|
||||
func TestDiskTemperatureThresholdMatchesCheckHostPolicy(t *testing.T) {
|
||||
m := configureDiskTempTypeHostManager(t)
|
||||
|
||||
for diskType, want := range map[string]HysteresisThreshold{
|
||||
"nvme": {Trigger: 70, Clear: 65},
|
||||
" NVMe": {Trigger: 70, Clear: 65},
|
||||
"sas": {Trigger: 65, Clear: 60},
|
||||
"sata": {Trigger: 55, Clear: 50},
|
||||
"hdd": {Trigger: 55, Clear: 50},
|
||||
"": {Trigger: 55, Clear: 50},
|
||||
} {
|
||||
if got := m.DiskTemperatureThreshold(diskType); got == nil || *got != want {
|
||||
t.Errorf("DiskTemperatureThreshold(%q) = %+v, want %+v", diskType, got, want)
|
||||
}
|
||||
if got := DefaultDiskTemperatureThreshold(diskType); got == nil || *got != want {
|
||||
t.Errorf("DefaultDiskTemperatureThreshold(%q) = %+v, want the factory %+v", diskType, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A user who raised the NVMe trigger to 75 gets no alert at 72C, and the
|
||||
// policy says the same disk is not hot.
|
||||
m.mu.Lock()
|
||||
m.config.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 75, Clear: 70}
|
||||
m.mu.Unlock()
|
||||
if got := m.DiskTemperatureThreshold("nvme"); got == nil || got.Trigger != 75 || got.Clear != 70 {
|
||||
t.Fatalf("raised nvme threshold = %+v, want 75/70", got)
|
||||
}
|
||||
host := hostWithSMARTDiskTemp("host-temp-raised-nvme", "nvme", 72)
|
||||
m.CheckHost(host)
|
||||
if _, exists := testLookupActiveAlert(t, m, hostDiskTempAlertID(host)); exists {
|
||||
t.Fatalf("expected no alert for nvme at 72C under a raised 75C trigger, active: %v", alertKeys(m))
|
||||
}
|
||||
|
||||
// The returned threshold is a copy.
|
||||
m.DiskTemperatureThreshold("nvme").Trigger = 1
|
||||
if got := m.DiskTemperatureThreshold("nvme"); got.Trigger != 75 {
|
||||
t.Fatalf("caller mutation reached the config: %+v", got)
|
||||
}
|
||||
|
||||
m.mu.Lock()
|
||||
m.config.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 0, Clear: 0}
|
||||
m.mu.Unlock()
|
||||
for _, diskType := range []string{"nvme", "sata", "hdd"} {
|
||||
if got := m.DiskTemperatureThreshold(diskType); got == nil || got.Trigger > 0 {
|
||||
t.Errorf("DiskTemperatureThreshold(%q) with disk temperature alerting off = %+v, want a disabled threshold", diskType, got)
|
||||
}
|
||||
}
|
||||
|
||||
var nilManager *Manager
|
||||
if got := nilManager.DiskTemperatureThreshold("nvme"); got == nil || got.Trigger != 70 {
|
||||
t.Fatalf("nil manager threshold = %+v, want the factory nvme 70", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -67,6 +67,9 @@ func (concurrencyThresholdProvider) GetGuestMemoryThreshold() float64 {
|
||||
}
|
||||
func (concurrencyThresholdProvider) GetGuestDiskThreshold() float64 { return 95 }
|
||||
func (concurrencyThresholdProvider) GetStorageThreshold() float64 { return 92 }
|
||||
func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
return 70, 65
|
||||
}
|
||||
|
||||
type concurrencyMetricsHistoryProvider struct{}
|
||||
|
||||
|
||||
@@ -154,6 +154,9 @@ func (stubThresholdProvider) GetGuestMemoryThreshold() float64 {
|
||||
}
|
||||
func (stubThresholdProvider) GetGuestDiskThreshold() float64 { return 95 }
|
||||
func (stubThresholdProvider) GetStorageThreshold() float64 { return 92 }
|
||||
func (stubThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
return 70, 65
|
||||
}
|
||||
|
||||
type stubMetricsHistoryProvider struct{}
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
|
||||
)
|
||||
|
||||
type RiskLevel string
|
||||
@@ -31,7 +30,12 @@ type Assessment struct {
|
||||
|
||||
// SMARTThresholds controls the discrete health evidence that becomes a disk
|
||||
// risk. Counter values at zero disable that rule; the endurance percentages
|
||||
// use the same convention. Temperature remains a separate metric threshold.
|
||||
// use the same convention.
|
||||
//
|
||||
// Disk risk never judges temperature. Heat is a metric judged by the alert
|
||||
// disk temperature policy (alerts.Manager.DiskTemperatureThreshold), which
|
||||
// users tune per disk type and per host. Risk is rebuilt by registries that
|
||||
// cannot see that configuration, so a heat rule here would disagree with it.
|
||||
type SMARTThresholds struct {
|
||||
HealthFailure bool
|
||||
ReallocatedSectors int64
|
||||
@@ -61,7 +65,6 @@ func DefaultSMARTThresholds() SMARTThresholds {
|
||||
type Sample struct {
|
||||
Model string
|
||||
Health string
|
||||
Temperature int
|
||||
Wearout int
|
||||
WearoutKnown bool
|
||||
PowerOnHours int64
|
||||
@@ -77,29 +80,10 @@ type Sample struct {
|
||||
UnsafeShutdowns int64
|
||||
}
|
||||
|
||||
// CollectedTemperature returns a disk temperature only when its collection
|
||||
// state marks it as observed now. Normalization may keep the last known value
|
||||
// when the current observation is not available, such as a disk in standby or
|
||||
// a host agent past its reporting lease. That retained value is history, not
|
||||
// evidence that the disk is hot now, so risk must not judge it. A source that
|
||||
// predates collection state counts as collected.
|
||||
func CollectedTemperature(temperature int, collection *diskinventory.CollectionStatus) int {
|
||||
if collection == nil {
|
||||
return temperature
|
||||
}
|
||||
switch collection.Temperature.State {
|
||||
case "", diskinventory.FieldAvailable:
|
||||
return temperature
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
func AssessPhysicalDisk(disk models.PhysicalDisk) Assessment {
|
||||
sample := Sample{
|
||||
Model: disk.Model,
|
||||
Health: disk.Health,
|
||||
Temperature: CollectedTemperature(disk.Temperature, disk.Collection),
|
||||
Wearout: disk.Wearout,
|
||||
WearoutKnown: WearoutReported(disk.Wearout, disk.Type),
|
||||
}
|
||||
@@ -113,10 +97,9 @@ func AssessHostSMARTDisk(disk models.HostDiskSMART) Assessment {
|
||||
|
||||
func AssessHostSMARTDiskWithThresholds(disk models.HostDiskSMART, thresholds SMARTThresholds) Assessment {
|
||||
sample := Sample{
|
||||
Model: disk.Model,
|
||||
Health: disk.Health,
|
||||
Temperature: CollectedTemperature(disk.Temperature, disk.Collection),
|
||||
Wearout: -1,
|
||||
Model: disk.Model,
|
||||
Health: disk.Health,
|
||||
Wearout: -1,
|
||||
}
|
||||
applySMARTAttributes(&sample, disk.Attributes)
|
||||
return AssessSampleWithThresholds(sample, thresholds)
|
||||
@@ -216,11 +199,6 @@ func AssessSampleWithThresholds(sample Sample, thresholds SMARTThresholds) Asses
|
||||
} else if percentageWarning {
|
||||
addReason("nvme_percentage_used_high", RiskWarning, fmt.Sprintf("NVMe endurance used is %d%%", sample.PercentageUsed))
|
||||
}
|
||||
if sample.Temperature >= 70 {
|
||||
addReason("temperature_high", RiskCritical, fmt.Sprintf("Disk temperature is %dC", sample.Temperature))
|
||||
} else if sample.Temperature >= 60 {
|
||||
addReason("temperature_high", RiskWarning, fmt.Sprintf("Disk temperature is %dC", sample.Temperature))
|
||||
}
|
||||
if thresholds.ReallocatedSectors > 0 && sample.ReallocatedSectors >= thresholds.ReallocatedSectors {
|
||||
addReason("reallocated_sectors", RiskWarning, fmt.Sprintf("Reallocated sectors detected (%d)", sample.ReallocatedSectors))
|
||||
}
|
||||
|
||||
@@ -33,7 +33,7 @@ func TestAssessPhysicalDisk_Branches(t *testing.T) {
|
||||
}{
|
||||
{
|
||||
// nil SmartAttributes arm: every *int64 pointer is skipped; only
|
||||
// Model/Health/Temperature/Wearout are sourced from the disk.
|
||||
// Model/Health/Wearout are sourced from the disk.
|
||||
name: "healthy disk with nil smart attributes stays healthy",
|
||||
disk: models.PhysicalDisk{Model: "Crucial MX500", Health: "PASSED", Temperature: 35, Wearout: 80},
|
||||
wantLevel: RiskHealthy,
|
||||
@@ -165,18 +165,10 @@ func TestAssessPhysicalDisk_Branches(t *testing.T) {
|
||||
wantReasons: []string{"nvme_percentage_used_high"},
|
||||
},
|
||||
{
|
||||
// Temperature >= 70 -> critical.
|
||||
name: "temperature critical",
|
||||
disk: models.PhysicalDisk{Health: "PASSED", Temperature: 72},
|
||||
wantLevel: RiskCritical,
|
||||
wantReasons: []string{"temperature_high"},
|
||||
},
|
||||
{
|
||||
// Temperature 60..69 -> warning.
|
||||
name: "temperature warning",
|
||||
disk: models.PhysicalDisk{Health: "PASSED", Temperature: 63},
|
||||
wantLevel: RiskWarning,
|
||||
wantReasons: []string{"temperature_high"},
|
||||
// Heat is the alert disk temperature policy's to judge, never risk's.
|
||||
name: "hot disk stays healthy",
|
||||
disk: models.PhysicalDisk{Health: "PASSED", Temperature: 72},
|
||||
wantLevel: RiskHealthy,
|
||||
},
|
||||
{
|
||||
// Reallocated sectors > 0 via SMART -> warning.
|
||||
@@ -247,20 +239,20 @@ func TestAssessPhysicalDisk_NilSmartAttributesDoesNotPanic(t *testing.T) {
|
||||
// is preserved through the SMART-attribute mapping.
|
||||
func TestAssessPhysicalDisk_MultipleReasonsTakeHighest(t *testing.T) {
|
||||
got := AssessPhysicalDisk(models.PhysicalDisk{
|
||||
Health: "PASSED",
|
||||
Temperature: 63, // warning
|
||||
Health: "PASSED",
|
||||
SmartAttributes: &models.SMARTAttributes{
|
||||
PendingSectors: int64Ptr(1), // critical
|
||||
UDMACRCErrors: int64Ptr(2), // monitor
|
||||
ReallocatedSectors: int64Ptr(3), // warning
|
||||
PendingSectors: int64Ptr(1), // critical
|
||||
UDMACRCErrors: int64Ptr(2), // monitor
|
||||
},
|
||||
})
|
||||
if got.Level != RiskCritical {
|
||||
t.Fatalf("expected critical (highest), got %q; reasons=%+v", got.Level, got.Reasons)
|
||||
}
|
||||
if len(got.Reasons) != 3 {
|
||||
t.Fatalf("expected 3 reasons (temperature_high, pending_sectors, crc_errors), got %d: %+v", len(got.Reasons), got.Reasons)
|
||||
t.Fatalf("expected 3 reasons (reallocated_sectors, pending_sectors, crc_errors), got %d: %+v", len(got.Reasons), got.Reasons)
|
||||
}
|
||||
for _, code := range []string{"pending_sectors", "temperature_high", "crc_errors"} {
|
||||
for _, code := range []string{"pending_sectors", "reallocated_sectors", "crc_errors"} {
|
||||
if _, ok := reasonSeverity(got, code); !ok {
|
||||
t.Errorf("expected reason %q present, got %+v", code, got.Reasons)
|
||||
}
|
||||
@@ -282,7 +274,7 @@ func TestAssessHostSMARTDisk_Branches(t *testing.T) {
|
||||
wantReasons []string
|
||||
}{
|
||||
{
|
||||
// nil Attributes arm: only Model/Health/Temperature are sourced;
|
||||
// nil Attributes arm: only Model/Health are sourced;
|
||||
// Wearout stays -1 so the wearout branch is skipped.
|
||||
name: "nil attributes stays healthy",
|
||||
disk: models.HostDiskSMART{Model: "WD Blue", Health: "PASSED", Temperature: 35},
|
||||
@@ -402,18 +394,10 @@ func TestAssessHostSMARTDisk_Branches(t *testing.T) {
|
||||
wantReasons: []string{"nvme_available_spare_low"},
|
||||
},
|
||||
{
|
||||
// Temperature >= 70 -> critical.
|
||||
name: "temperature critical",
|
||||
disk: models.HostDiskSMART{Health: "PASSED", Temperature: 75},
|
||||
wantLevel: RiskCritical,
|
||||
wantReasons: []string{"temperature_high"},
|
||||
},
|
||||
{
|
||||
// Temperature 60..69 -> warning.
|
||||
name: "temperature warning",
|
||||
disk: models.HostDiskSMART{Health: "PASSED", Temperature: 60},
|
||||
wantLevel: RiskWarning,
|
||||
wantReasons: []string{"temperature_high"},
|
||||
// Heat is the alert disk temperature policy's to judge, never risk's.
|
||||
name: "hot disk stays healthy",
|
||||
disk: models.HostDiskSMART{Health: "PASSED", Temperature: 75},
|
||||
wantLevel: RiskHealthy,
|
||||
},
|
||||
{
|
||||
// Reallocated sectors > 0 via host SMART -> warning.
|
||||
|
||||
@@ -11,9 +11,8 @@ import (
|
||||
|
||||
func TestAssessSample_HealthyDisk(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 35,
|
||||
Wearout: 80,
|
||||
Health: "PASSED",
|
||||
Wearout: 80,
|
||||
})
|
||||
if assessment.Level != RiskHealthy {
|
||||
t.Errorf("expected healthy, got %s", assessment.Level)
|
||||
@@ -308,38 +307,6 @@ func TestAssessSample_NVMePercentageUsedWarning(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssessSample_TemperatureCritical(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 72, // >=70 critical
|
||||
})
|
||||
if assessment.Level != RiskCritical {
|
||||
t.Errorf("expected critical for temp=72, got %s", assessment.Level)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssessSample_TemperatureWarning(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 63, // >=60, <70 warning
|
||||
})
|
||||
if assessment.Level != RiskWarning {
|
||||
t.Errorf("expected warning for temp=63, got %s", assessment.Level)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssessSample_TemperatureHealthy(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 45,
|
||||
})
|
||||
for _, r := range assessment.Reasons {
|
||||
if r.Code == "temperature_high" {
|
||||
t.Error("temp=45 should not trigger temperature_high")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssessSample_ReallocatedSectors(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
@@ -371,10 +338,10 @@ func TestAssessSample_CRCErrors(t *testing.T) {
|
||||
|
||||
func TestAssessSample_MultipleIssuesTakesHighest(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 63, // warning
|
||||
PendingSectors: 1, // critical
|
||||
UDMACRCErrors: 10, // monitor
|
||||
Health: "PASSED",
|
||||
ReallocatedSectors: 1, // warning
|
||||
PendingSectors: 1, // critical
|
||||
UDMACRCErrors: 10, // monitor
|
||||
})
|
||||
if assessment.Level != RiskCritical {
|
||||
t.Errorf("expected critical (highest), got %s", assessment.Level)
|
||||
@@ -387,10 +354,11 @@ func TestAssessSample_MultipleIssuesTakesHighest(t *testing.T) {
|
||||
func TestAssessSample_ReasonsSortedBySeverityDescending(t *testing.T) {
|
||||
assessment := AssessSample(Sample{
|
||||
Health: "PASSED",
|
||||
Temperature: 63, // warning
|
||||
PendingSectors: 1, // critical
|
||||
UDMACRCErrors: 10, // monitor
|
||||
ReallocatedSectors: 1, // warning
|
||||
WearoutKnown: true,
|
||||
Wearout: 8, // warning
|
||||
})
|
||||
for i := 1; i < len(assessment.Reasons); i++ {
|
||||
prevRank := severityRank(assessment.Reasons[i-1].Severity)
|
||||
@@ -540,58 +508,20 @@ func TestAssessPhysicalDisk_RotationalZeroIsNotRisk(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCollectedTemperatureKeepsOnlyCurrentReadings(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
collection *diskinventory.CollectionStatus
|
||||
want int
|
||||
}{
|
||||
{name: "no collection state", collection: nil, want: 72},
|
||||
{name: "empty state", collection: &diskinventory.CollectionStatus{}, want: 72},
|
||||
{name: "available", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Available("smartctl")}, want: 72},
|
||||
{name: "unavailable", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("smartctl", "disk is in standby")}, want: 0},
|
||||
{name: "unsupported", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unsupported("proxmox_disks", "")}, want: 0},
|
||||
{name: "missing", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Missing("smartctl", "")}, want: 0},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := CollectedTemperature(72, tc.collection); got != tc.want {
|
||||
t.Errorf("%s: CollectedTemperature = %d, want %d", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiskAssessmentsIgnoreRetainedTemperature(t *testing.T) {
|
||||
retained := &diskinventory.CollectionStatus{
|
||||
Temperature: diskinventory.Unavailable("host_agent", "host agent stopped reporting"),
|
||||
}
|
||||
// Heat belongs to the alert disk temperature policy, which users tune per disk
|
||||
// type and per host. Disk risk cannot see that configuration, so it must not
|
||||
// judge temperature at all, however hot and however current the reading.
|
||||
func TestDiskAssessmentsNeverJudgeTemperature(t *testing.T) {
|
||||
collected := &diskinventory.CollectionStatus{Temperature: diskinventory.Available("host_agent")}
|
||||
|
||||
hasTemperatureReason := func(assessment Assessment) bool {
|
||||
for _, reason := range assessment.Reasons {
|
||||
if reason.Code == "temperature_high" {
|
||||
return true
|
||||
}
|
||||
for _, temperature := range []int{56, 63, 72, 95} {
|
||||
disk := models.PhysicalDisk{Model: "WDC WD80EFAX", Type: "sata", Health: "PASSED", Wearout: -1, Temperature: temperature, Collection: collected}
|
||||
if got := AssessPhysicalDisk(disk); got.Level != RiskHealthy || len(got.Reasons) != 0 {
|
||||
t.Errorf("%dC physical disk = %+v, want healthy with no reasons", temperature, got)
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
disk := models.PhysicalDisk{Model: "WDC WD80EFAX", Type: "sata", Health: "PASSED", Wearout: -1, Temperature: 72}
|
||||
disk.Collection = collected
|
||||
if got := AssessPhysicalDisk(disk); got.Level != RiskCritical || !hasTemperatureReason(got) {
|
||||
t.Fatalf("collected 72C physical disk = %+v, want critical temperature_high", got)
|
||||
}
|
||||
disk.Collection = retained
|
||||
if got := AssessPhysicalDisk(disk); got.Level != RiskHealthy || hasTemperatureReason(got) {
|
||||
t.Fatalf("retained 72C physical disk = %+v, want healthy", got)
|
||||
}
|
||||
|
||||
smart := models.HostDiskSMART{Device: "/dev/sda", Model: "WDC WD80EFAX", Type: "sata", Health: "PASSED", Temperature: 72}
|
||||
smart.Collection = collected
|
||||
if got := AssessHostSMARTDisk(smart); got.Level != RiskCritical || !hasTemperatureReason(got) {
|
||||
t.Fatalf("collected 72C host SMART disk = %+v, want critical temperature_high", got)
|
||||
}
|
||||
smart.Collection = retained
|
||||
if got := AssessHostSMARTDisk(smart); got.Level != RiskHealthy || hasTemperatureReason(got) {
|
||||
t.Fatalf("retained 72C host SMART disk = %+v, want healthy", got)
|
||||
smart := models.HostDiskSMART{Device: "/dev/nvme0n1", Model: "Samsung 990 PRO", Type: "nvme", Health: "PASSED", Temperature: temperature, Collection: collected}
|
||||
if got := AssessHostSMARTDisk(smart); got.Level != RiskHealthy || len(got.Reasons) != 0 {
|
||||
t.Errorf("%dC host SMART disk = %+v, want healthy with no reasons", temperature, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1953,10 +1953,9 @@ func zfsPoolRecommendation(assessment storagehealth.Assessment) string {
|
||||
|
||||
func assessDisk(disk Disk) storagehealth.Assessment {
|
||||
sampleAssessment := storagehealth.AssessSample(storagehealth.Sample{
|
||||
Model: strings.TrimSpace(disk.Model),
|
||||
Health: healthForAssessment(disk),
|
||||
Temperature: disk.Temperature,
|
||||
Wearout: -1,
|
||||
Model: strings.TrimSpace(disk.Model),
|
||||
Health: healthForAssessment(disk),
|
||||
Wearout: -1,
|
||||
})
|
||||
|
||||
stateUpper := normalizedDiskStatus(disk)
|
||||
|
||||
@@ -745,8 +745,9 @@ func TestRecordsIncludeDiskResourcesWithCorrectParentChain(t *testing.T) {
|
||||
foundSmartReason = true
|
||||
}
|
||||
}
|
||||
if !foundTemperatureReason {
|
||||
t.Fatalf("expected sdc physical-disk risk to include temperature_high, got %+v", sdc.Resource.PhysicalDisk.Risk.Reasons)
|
||||
// Heat is the alert disk temperature policy's to judge, not disk risk's.
|
||||
if foundTemperatureReason {
|
||||
t.Fatalf("expected sdc physical-disk risk to leave its 63C temperature to the alert policy, got %+v", sdc.Resource.PhysicalDisk.Risk.Reasons)
|
||||
}
|
||||
if !foundSmartReason {
|
||||
t.Fatalf("expected sdc physical-disk risk to include truenas_smart, got %+v", sdc.Resource.PhysicalDisk.Risk.Reasons)
|
||||
@@ -931,7 +932,9 @@ func TestRecordsIncludeTrueNASVMsAsCanonicalWorkloads(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordsElevateOnlineDiskWhenTemperatureCritical(t *testing.T) {
|
||||
// A hot disk stays online with no disk risk. Its temperature is judged by the
|
||||
// alert disk temperature policy, which users tune per disk type.
|
||||
func TestRecordsLeaveDiskHeatToTheAlertPolicy(t *testing.T) {
|
||||
previous := IsFeatureEnabled()
|
||||
SetFeatureEnabled(true)
|
||||
t.Cleanup(func() {
|
||||
@@ -967,14 +970,14 @@ func TestRecordsElevateOnlineDiskWhenTemperatureCritical(t *testing.T) {
|
||||
if diskRecord == nil {
|
||||
t.Fatal("expected physical disk record")
|
||||
}
|
||||
if diskRecord.Resource.Status != unifiedresources.StatusWarning {
|
||||
t.Fatalf("expected hot disk status warning, got %s", diskRecord.Resource.Status)
|
||||
if diskRecord.Resource.Status != unifiedresources.StatusOnline {
|
||||
t.Fatalf("expected hot disk status online, got %s", diskRecord.Resource.Status)
|
||||
}
|
||||
if diskRecord.Resource.PhysicalDisk == nil || diskRecord.Resource.PhysicalDisk.Risk == nil {
|
||||
t.Fatalf("expected hot disk physical risk, got %+v", diskRecord.Resource.PhysicalDisk)
|
||||
if diskRecord.Resource.PhysicalDisk == nil || diskRecord.Resource.PhysicalDisk.Temperature != 72 {
|
||||
t.Fatalf("expected the hot disk to keep its 72C reading, got %+v", diskRecord.Resource.PhysicalDisk)
|
||||
}
|
||||
if diskRecord.Resource.PhysicalDisk.Risk.Level != storagehealth.RiskCritical {
|
||||
t.Fatalf("expected hot disk critical risk, got %+v", diskRecord.Resource.PhysicalDisk.Risk)
|
||||
if diskRecord.Resource.PhysicalDisk.Risk != nil {
|
||||
t.Fatalf("expected no disk risk from temperature, got %+v", diskRecord.Resource.PhysicalDisk.Risk)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1051,7 +1051,8 @@ func TestMetricPercent(t *testing.T) {
|
||||
// derives their state from the host's reporting lease and the disk's spin
|
||||
// state. The native row and a SMART row that borrows the inventory reading
|
||||
// both carry it, and a SMART row with its own reading keeps the agent's state.
|
||||
// A withdrawn reading never counts as heat in the disk's risk.
|
||||
// Disk risk never judges heat in any state: the alert disk temperature policy
|
||||
// owns that.
|
||||
func TestUnraidDiskTemperatureFollowsTheHostLease(t *testing.T) {
|
||||
smartWithout := models.HostDiskSMART{Device: "/dev/sdc", Serial: "UNRAID-1", Collection: &diskinventory.CollectionStatus{
|
||||
Serial: diskinventory.Available("smartctl"),
|
||||
@@ -1088,7 +1089,6 @@ func TestUnraidDiskTemperatureFollowsTheHostLease(t *testing.T) {
|
||||
Temperature: 72, SpunDown: tc.spunDown}
|
||||
host := models.Host{ID: "agent-tower", Hostname: "tower", Status: tc.status,
|
||||
Unraid: &models.HostUnraidStorage{Disks: []models.HostUnraidDisk{inventory}}}
|
||||
collected := tc.borrowed.State == diskinventory.FieldAvailable
|
||||
|
||||
native, _ := resourceFromHostUnraidPhysicalDisk(host, inventory)
|
||||
got := native.PhysicalDisk
|
||||
@@ -1099,8 +1099,8 @@ func TestUnraidDiskTemperatureFollowsTheHostLease(t *testing.T) {
|
||||
if got.Temperature != 72 || nativeStatus != tc.native {
|
||||
t.Fatalf("%s: Unraid row temperature=%d state=%+v, want 72 with %+v", tc.name, got.Temperature, nativeStatus, tc.native)
|
||||
}
|
||||
if hotRisk(got.Risk) != collected {
|
||||
t.Fatalf("%s: Unraid row temperature risk = %v, want %v: %+v", tc.name, hotRisk(got.Risk), collected, got.Risk)
|
||||
if hotRisk(got.Risk) {
|
||||
t.Fatalf("%s: Unraid row risk judged its 72C temperature: %+v", tc.name, got.Risk)
|
||||
}
|
||||
|
||||
borrowed, _ := resourceFromHostSMARTDisk(host, smartWithout)
|
||||
@@ -1109,8 +1109,8 @@ func TestUnraidDiskTemperatureFollowsTheHostLease(t *testing.T) {
|
||||
got.Collection.Serial != diskinventory.Available("smartctl") {
|
||||
t.Fatalf("%s: SMART row borrowing the inventory reading: temperature=%d collection=%+v", tc.name, got.Temperature, got.Collection)
|
||||
}
|
||||
if hotRisk(got.Risk) != collected {
|
||||
t.Fatalf("%s: borrowed temperature risk = %v, want %v: %+v", tc.name, hotRisk(got.Risk), collected, got.Risk)
|
||||
if hotRisk(got.Risk) {
|
||||
t.Fatalf("%s: SMART row risk judged the borrowed 72C temperature: %+v", tc.name, got.Risk)
|
||||
}
|
||||
if smartWithout.Collection.Temperature.State != diskinventory.FieldUnsupported {
|
||||
t.Fatal("the adapter mutated the SMART row's own collection state")
|
||||
|
||||
@@ -804,7 +804,7 @@ func resourceFromHostUnraidPhysicalDisk(host models.Host, disk models.HostUnraid
|
||||
model := strings.TrimSpace(disk.Model)
|
||||
name := firstNonEmpty(model, disk.Name, disk.Device, host.Hostname)
|
||||
health := unraidPhysicalDiskHealth(disk)
|
||||
assessment := assessUnraidPhysicalDisk(host, disk)
|
||||
assessment := assessUnraidPhysicalDisk(disk)
|
||||
sizeBytes := unraidDiskSizeBytes(host, disk)
|
||||
var collection *diskinventory.CollectionStatus
|
||||
if status := unraidDiskTemperatureStatus(host, disk); status.State != "" {
|
||||
@@ -939,7 +939,7 @@ func resourceFromHostSMARTDisk(host models.Host, disk models.HostDiskSMART) (Res
|
||||
}
|
||||
assessment := storagehealth.AssessHostSMARTDisk(disk)
|
||||
if unraidDisk != nil {
|
||||
assessment = storagehealth.SummarizeAssessments(assessment, assessUnraidPhysicalDisk(host, *unraidDisk))
|
||||
assessment = storagehealth.SummarizeAssessments(assessment, assessUnraidPhysicalDisk(*unraidDisk))
|
||||
}
|
||||
wearout := physicalDiskWearoutFromSMARTAttributes(disk.Attributes)
|
||||
|
||||
@@ -1324,17 +1324,11 @@ func unraidPhysicalDiskHealth(disk models.HostUnraidDisk) string {
|
||||
}
|
||||
}
|
||||
|
||||
func assessUnraidPhysicalDisk(host models.Host, disk models.HostUnraidDisk) storagehealth.Assessment {
|
||||
// A retained inventory temperature is not current evidence of heat.
|
||||
temperature := 0
|
||||
if status := unraidDiskTemperatureStatus(host, disk); status.State == diskinventory.FieldAvailable {
|
||||
temperature = disk.Temperature
|
||||
}
|
||||
func assessUnraidPhysicalDisk(disk models.HostUnraidDisk) storagehealth.Assessment {
|
||||
assessment := storagehealth.AssessSample(storagehealth.Sample{
|
||||
Model: disk.Model,
|
||||
Health: unraidPhysicalDiskHealth(disk),
|
||||
Temperature: temperature,
|
||||
Wearout: -1,
|
||||
Model: disk.Model,
|
||||
Health: unraidPhysicalDiskHealth(disk),
|
||||
Wearout: -1,
|
||||
})
|
||||
addReason := func(code string, severity storagehealth.RiskLevel, summary string) {
|
||||
if strings.TrimSpace(summary) == "" {
|
||||
|
||||
@@ -82,7 +82,6 @@ func physicalDiskAssessmentFromMeta(meta *PhysicalDiskMeta) storagehealth.Assess
|
||||
sample := storagehealth.Sample{
|
||||
Model: meta.Model,
|
||||
Health: meta.Health,
|
||||
Temperature: storagehealth.CollectedTemperature(meta.Temperature, meta.Collection),
|
||||
Wearout: meta.Wearout,
|
||||
WearoutKnown: storagehealth.WearoutReported(meta.Wearout, meta.DiskType),
|
||||
}
|
||||
|
||||
@@ -41,7 +41,7 @@ func TestPhysicalDiskRiskFromAssessment_ConvertsReasons(t *testing.T) {
|
||||
Level: storagehealth.RiskCritical,
|
||||
Reasons: []storagehealth.Reason{
|
||||
{Code: "pending_sectors", Severity: storagehealth.RiskCritical, Summary: "Pending sectors detected (5)"},
|
||||
{Code: "temperature_high", Severity: storagehealth.RiskWarning, Summary: "Disk temperature is 65C"},
|
||||
{Code: "reallocated_sectors", Severity: storagehealth.RiskWarning, Summary: "Reallocated sectors detected (4)"},
|
||||
},
|
||||
}
|
||||
result := PhysicalDiskRiskFromAssessment(assessment)
|
||||
@@ -66,7 +66,7 @@ func TestPhysicalDiskRiskFromAssessmentAndIncidents_IncludesDiskHealthIncidents(
|
||||
assessment := storagehealth.Assessment{
|
||||
Level: storagehealth.RiskWarning,
|
||||
Reasons: []storagehealth.Reason{
|
||||
{Code: "temperature_high", Severity: storagehealth.RiskWarning, Summary: "Disk temperature is 63C"},
|
||||
{Code: "reallocated_sectors", Severity: storagehealth.RiskWarning, Summary: "Reallocated sectors detected (2)"},
|
||||
},
|
||||
}
|
||||
incidents := []ResourceIncident{
|
||||
@@ -137,14 +137,15 @@ func TestPhysicalDiskAssessmentFromMeta_PendingSectors(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestPhysicalDiskAssessmentFromMeta_HighTemperature(t *testing.T) {
|
||||
// Heat is the alert disk temperature policy's to judge, never disk risk's.
|
||||
func TestPhysicalDiskAssessmentFromMeta_HighTemperatureIsNotRisk(t *testing.T) {
|
||||
meta := &PhysicalDiskMeta{
|
||||
Health: "PASSED",
|
||||
Temperature: 72,
|
||||
}
|
||||
assessment := physicalDiskAssessmentFromMeta(meta)
|
||||
if assessment.Level != storagehealth.RiskCritical {
|
||||
t.Errorf("72°C should produce critical assessment, got %s", assessment.Level)
|
||||
if assessment.Level != storagehealth.RiskHealthy || len(assessment.Reasons) != 0 {
|
||||
t.Errorf("72°C should leave the assessment healthy, got %+v", assessment)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -7035,10 +7035,11 @@ func TestRegistryProjectionSnapshotDetachesAndTracksMappings(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A retained temperature is last-known history, not evidence that the disk is
|
||||
// hot now. The risk verdict (and the warning status it drives) must judge only
|
||||
// a temperature whose collection state says it was observed now.
|
||||
func TestPhysicalDiskRiskIgnoresRetainedTemperature(t *testing.T) {
|
||||
// Disk risk (and the warning status it drives) never judges temperature,
|
||||
// collected or retained. Heat belongs to the alert disk temperature policy,
|
||||
// which users tune per disk type and per host and which no registry can see.
|
||||
// The reading itself stays on the resource for that policy to judge.
|
||||
func TestPhysicalDiskRiskNeverJudgesTemperature(t *testing.T) {
|
||||
// A fresh check time keeps the registry's source-staleness sweep out of
|
||||
// the status this test reads.
|
||||
now := time.Now()
|
||||
@@ -7047,15 +7048,12 @@ func TestPhysicalDiskRiskIgnoresRetainedTemperature(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
collection *diskinventory.CollectionStatus
|
||||
wantHot bool
|
||||
}{
|
||||
{name: "collected now", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Available("proxmox_node_smart")}, wantHot: true},
|
||||
{name: "source predates collection state", collection: nil, wantHot: true},
|
||||
{name: "state left empty", collection: &diskinventory.CollectionStatus{}, wantHot: true},
|
||||
{name: "retained after the agent went silent", collection: &diskinventory.CollectionStatus{Temperature: retained}, wantHot: false},
|
||||
{name: "retained while in standby", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("smartctl", "disk is in standby")}, wantHot: false},
|
||||
{name: "retained after the source stopped exposing it", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unsupported("proxmox_disks", "Proxmox disk inventory does not expose temperature")}, wantHot: false},
|
||||
{name: "retained while unexpectedly missing", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Missing("smartctl", "")}, wantHot: false},
|
||||
{name: "collected now", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Available("proxmox_node_smart")}},
|
||||
{name: "source predates collection state", collection: nil},
|
||||
{name: "state left empty", collection: &diskinventory.CollectionStatus{}},
|
||||
{name: "retained after the agent went silent", collection: &diskinventory.CollectionStatus{Temperature: retained}},
|
||||
{name: "retained while in standby", collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("smartctl", "disk is in standby")}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
@@ -7089,100 +7087,60 @@ func TestPhysicalDiskRiskIgnoresRetainedTemperature(t *testing.T) {
|
||||
if disk.PhysicalDisk.Temperature != 72 {
|
||||
t.Fatalf("temperature = %d, want the reading kept at 72", disk.PhysicalDisk.Temperature)
|
||||
}
|
||||
hot := false
|
||||
if disk.PhysicalDisk.Risk != nil {
|
||||
for _, reason := range disk.PhysicalDisk.Risk.Reasons {
|
||||
hot = hot || reason.Code == "temperature_high"
|
||||
}
|
||||
}
|
||||
if hot != tc.wantHot {
|
||||
t.Fatalf("temperature_high = %v, want %v (risk %+v)", hot, tc.wantHot, disk.PhysicalDisk.Risk)
|
||||
}
|
||||
if tc.wantHot && disk.Status != StatusWarning {
|
||||
t.Fatalf("status = %q, want warning for a collected 72C", disk.Status)
|
||||
}
|
||||
if !tc.wantHot && (disk.PhysicalDisk.Risk != nil || disk.Status != StatusOnline) {
|
||||
t.Fatalf("retained 72C: status = %q risk = %+v, want online with no risk", disk.Status, disk.PhysicalDisk.Risk)
|
||||
if disk.PhysicalDisk.Risk != nil || disk.Status != StatusOnline {
|
||||
t.Fatalf("72C disk: status = %q risk = %+v, want online with no risk", disk.Status, disk.PhysicalDisk.Risk)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// A host agent's SMART row merged with the Proxmox inventory row for the
|
||||
// same disk: the agent's retained reading keeps its unavailable state
|
||||
// through the registry merge and recompute, so it stays out of risk.
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
state diskinventory.FieldStatus
|
||||
wantHot bool
|
||||
}{
|
||||
{name: "agent reading collected", state: diskinventory.Available("host_agent"), wantHot: true},
|
||||
{name: "agent reading retained", state: retained, wantHot: false},
|
||||
} {
|
||||
t.Run("merged "+tc.name, func(t *testing.T) {
|
||||
adapter := NewMonitorAdapter(NewRegistry(nil))
|
||||
adapter.PopulateFromSnapshot(models.StateSnapshot{
|
||||
Hosts: []models.Host{{
|
||||
ID: "host-pve",
|
||||
Hostname: "pve",
|
||||
Status: "online",
|
||||
LastSeen: now,
|
||||
Sensors: models.HostSensorSummary{SMART: []models.HostDiskSMART{{
|
||||
Device: "/dev/sda",
|
||||
Model: "WDC WD80EFAX",
|
||||
Serial: "SERIAL-SDA",
|
||||
Type: "sata",
|
||||
Temperature: 72,
|
||||
Health: "PASSED",
|
||||
Collection: &diskinventory.CollectionStatus{Temperature: tc.state},
|
||||
}}},
|
||||
}},
|
||||
PhysicalDisks: []models.PhysicalDisk{{
|
||||
ID: "homelab-pve--dev-sda",
|
||||
Node: "pve",
|
||||
Instance: "homelab",
|
||||
DevPath: "/dev/sda",
|
||||
Model: "WDC WD80EFAX",
|
||||
Serial: "SERIAL-SDA",
|
||||
Type: "sata",
|
||||
Health: "PASSED",
|
||||
Wearout: -1,
|
||||
Collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unsupported("proxmox_disks", "Proxmox disk inventory does not expose temperature")},
|
||||
LastChecked: now,
|
||||
}},
|
||||
})
|
||||
var disks []Resource
|
||||
for _, resource := range adapter.GetAll() {
|
||||
if resource.Type == ResourceTypePhysicalDisk {
|
||||
disks = append(disks, resource)
|
||||
}
|
||||
}
|
||||
if len(disks) != 1 || disks[0].PhysicalDisk == nil {
|
||||
t.Fatalf("physical disks = %d, want the agent and Proxmox rows merged into one", len(disks))
|
||||
}
|
||||
disk := disks[0]
|
||||
if disk.PhysicalDisk.Temperature != 72 {
|
||||
t.Fatalf("merged temperature = %d, want 72", disk.PhysicalDisk.Temperature)
|
||||
}
|
||||
hot := false
|
||||
if disk.PhysicalDisk.Risk != nil {
|
||||
for _, reason := range disk.PhysicalDisk.Risk.Reasons {
|
||||
hot = hot || reason.Code == "temperature_high"
|
||||
}
|
||||
}
|
||||
if hot != tc.wantHot {
|
||||
t.Fatalf("merged temperature_high = %v, want %v (collection %+v, risk %+v)", hot, tc.wantHot, disk.PhysicalDisk.Collection, disk.PhysicalDisk.Risk)
|
||||
}
|
||||
})
|
||||
// same disk goes through the registry merge and its risk recompute.
|
||||
adapter := NewMonitorAdapter(NewRegistry(nil))
|
||||
adapter.PopulateFromSnapshot(models.StateSnapshot{
|
||||
Hosts: []models.Host{{
|
||||
ID: "host-pve",
|
||||
Hostname: "pve",
|
||||
Status: "online",
|
||||
LastSeen: now,
|
||||
Sensors: models.HostSensorSummary{SMART: []models.HostDiskSMART{{
|
||||
Device: "/dev/sda",
|
||||
Model: "WDC WD80EFAX",
|
||||
Serial: "SERIAL-SDA",
|
||||
Type: "sata",
|
||||
Temperature: 72,
|
||||
Health: "PASSED",
|
||||
Collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Available("host_agent")},
|
||||
}}},
|
||||
}},
|
||||
PhysicalDisks: []models.PhysicalDisk{{
|
||||
ID: "homelab-pve--dev-sda",
|
||||
Node: "pve",
|
||||
Instance: "homelab",
|
||||
DevPath: "/dev/sda",
|
||||
Model: "WDC WD80EFAX",
|
||||
Serial: "SERIAL-SDA",
|
||||
Type: "sata",
|
||||
Health: "PASSED",
|
||||
Wearout: -1,
|
||||
Collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Unsupported("proxmox_disks", "Proxmox disk inventory does not expose temperature")},
|
||||
LastChecked: now,
|
||||
}},
|
||||
})
|
||||
var merged []Resource
|
||||
for _, resource := range adapter.GetAll() {
|
||||
if resource.Type == ResourceTypePhysicalDisk {
|
||||
merged = append(merged, resource)
|
||||
}
|
||||
}
|
||||
if len(merged) != 1 || merged[0].PhysicalDisk == nil {
|
||||
t.Fatalf("physical disks = %d, want the agent and Proxmox rows merged into one", len(merged))
|
||||
}
|
||||
if disk := merged[0].PhysicalDisk; disk.Temperature != 72 || disk.Risk != nil {
|
||||
t.Fatalf("merged disk temperature = %d risk = %+v, want 72 with no risk", disk.Temperature, disk.Risk)
|
||||
}
|
||||
|
||||
// The registry recomputes risk from merged metadata after a source merge;
|
||||
// that path follows the same rule.
|
||||
meta := &PhysicalDiskMeta{Health: "PASSED", Temperature: 72, Collection: &diskinventory.CollectionStatus{Temperature: retained}}
|
||||
meta := &PhysicalDiskMeta{Health: "PASSED", Temperature: 72, Collection: &diskinventory.CollectionStatus{Temperature: diskinventory.Available("host_agent")}}
|
||||
if risk := physicalDiskRiskFromMeta(meta, nil); risk != nil {
|
||||
t.Fatalf("meta recompute risk = %+v, want none for a retained temperature", risk)
|
||||
}
|
||||
meta.Collection = &diskinventory.CollectionStatus{Temperature: diskinventory.Available("host_agent")}
|
||||
if risk := physicalDiskRiskFromMeta(meta, nil); risk == nil || risk.Level != storagehealth.RiskCritical {
|
||||
t.Fatalf("meta recompute risk = %+v, want critical for a collected 72C", risk)
|
||||
t.Fatalf("meta recompute risk = %+v, want none for a collected 72C", risk)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,7 +35,7 @@ func TestStorageRiskFromAssessment_CriticalConvertsReasons(t *testing.T) {
|
||||
Level: storagehealth.RiskCritical,
|
||||
Reasons: []storagehealth.Reason{
|
||||
{Code: "pending_sectors", Severity: storagehealth.RiskCritical, Summary: "Bad sectors"},
|
||||
{Code: "temperature_high", Severity: storagehealth.RiskWarning, Summary: "Hot disk"},
|
||||
{Code: "reallocated_sectors", Severity: storagehealth.RiskWarning, Summary: "Reallocated sectors"},
|
||||
},
|
||||
})
|
||||
if got == nil {
|
||||
@@ -47,7 +47,7 @@ func TestStorageRiskFromAssessment_CriticalConvertsReasons(t *testing.T) {
|
||||
if len(got.Reasons) != 2 {
|
||||
t.Fatalf("expected 2 reasons, got %d", len(got.Reasons))
|
||||
}
|
||||
if got.Reasons[0].Code != "pending_sectors" || got.Reasons[1].Code != "temperature_high" {
|
||||
if got.Reasons[0].Code != "pending_sectors" || got.Reasons[1].Code != "reallocated_sectors" {
|
||||
t.Error("reasons should preserve order and codes")
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user