Judge TrueNAS disk temperature alerts by disk type

Every disk heat judge follows the per-type disk temperature policy
(NVMe 70C, SAS 65C, SATA 55C, under the agent Disk Temp default) except
TrueNAS disk temperature alerts, which used a flat 55/50 TrueNAS Disks
default. A TrueNAS NVMe at 60C raised a warning while the TrueNAS storage
table, Physical Disks and Patrol called it fine, the false NVMe alert
#1388 removed for agent disks.

TrueNAS disks now resolve their trigger per disk type unless the user
saved a TrueNAS-wide value, which still applies to every TrueNAS disk; a
per-disk override beats both. Saved configs keep a chosen value: a stored
55/50 is dropped only from configs written before the new
truenasDiskTemperatureByType marker, because the thresholds page wrote
that factory value back on every save. Alerts record their disk type so
a config save re-judges them per type, and a retained reading no longer
raises one.

The TrueNAS Disks Global Defaults cell reads "By type" while unset (Off
when the agent default switches the policy off), each disk row shows its
type's trigger, and Off is written as trigger 0. The shared alert table
now reads its props reactively, so the Custom badge and reset control
follow a config that loads after the table mounts. The TrueNAS disk
drawer tones a current reading by the same per-type trigger instead of
a fixed 55C.
This commit is contained in:
rcourtman
2026-10-07 15:11:44 +01:00
parent bdf0f76b7b
commit 8e1a564d7e
35 changed files with 1051 additions and 67 deletions
@@ -1830,14 +1830,52 @@ A disk is hot from its trigger. Alerts and Patrol findings stay open until the
reading falls to the clear value, or under the trigger when there is no band
below it. The Physical Disks verdict and the TrueNAS Health cell judge only the
current reading against the trigger. PDF reports colour disk temperatures by
the same thresholds. Judges that still differ: TrueNAS disk temperature alerts
use `TrueNASDiskDefaults.Temperature` (a flat 55/50), and the TrueNAS disk
drawer tones the reading from a fixed 55C.
the same thresholds.
`TestDiskTemperatureThresholdMatchesCheckHostPolicy` in
`internal/alerts/threshold_resolution_shared_test.go` pins per-type resolution, a raised
NVMe trigger that `CheckHost` also honours, the copy, the disabled default and
the nil manager.
TrueNAS disk temperature alerts follow the policy too:
`trueNASDiskTemperatureDefaultNoLock`
(`internal/alerts/truenas_disk_temperature.go`) is the type-default tier of
`effectiveAlertPolicyNoLock` for `truenas-disk`, so an unset
`TrueNASDiskDefaults.Temperature` resolves per disk type (the
`alertPolicyQuery.DiskType` the unified input carries from
`PhysicalDisk.DiskType`). A TrueNAS-wide value the user saved under TrueNAS
Disks replaces it for every TrueNAS disk, as a host Disk Temp override does for
an agent's disks, 0 meaning off, and a per-disk override beats both. The alert
records `diskType` so a config save re-judges it per type; one without it is
held to the lowest per-type trigger. A retained reading
(`diskinventory.TemperatureCollected` false) is no evidence and raises nothing.
The factory value is unset. `normalizeTrueNASDiskTemperature` drops a stored
55/50 only from a config without `TrueNASDiskTemperatureByType`, because the
thresholds page wrote that flat factory value back on every save; the page
writes the marker, so a 55 typed later is kept. Every config `GetConfig`
returns carries the marker, so a client that reads and writes back keeps it.
Without the marker (an old saved file, or a page loaded before the upgrade)
55/50 is read as that unchosen factory value; a client building a config from
scratch with an explicit TrueNAS-wide 55 has to send the marker. The TrueNAS
Disks Global Defaults cell reads `By type` while unset
(`globalDefaultFallbacks` on `ResourceTable`,
`resolveAlertResourceGlobalDefaultCell`), or Off when the agent Disk Temp
default switches the policy off, and switching it on then stages an explicit
TrueNAS-wide 55 because unset would stay off. Each disk row inherits its type's
trigger from the unsaved editor state (`resolveTrueNASDiskTemperatureDefault`).
The TrueNAS disk drawer tones a current reading from the same per-type trigger
(`getDiskTemperatureThresholds` passed into `buildTrueNASDetailSections`), so it
no longer judges heat from a fixed 55C. Judges that still differ: the TrueNAS
storage table and drawer do not apply a TrueNAS-wide value or per-disk
override.
`TestTrueNASDiskTemperatureAlertsFollowDiskTemperaturePolicy` and
`TestTrueNASDiskTemperatureAlertIgnoresRetainedReading` in
`internal/alerts/unified_eval_test.go` pin the tiers and retained readings;
`TestTrueNASDiskTemperatureAlertsReevaluatePerDiskType` and
`TestTrueNASDiskTemperatureAlertWithoutDiskTypeHeldAtLowestTrigger` in
`internal/alerts/alerts_test.go` pin re-judging on a config save;
`TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues` pins the
saved-config migration.
### Agent disk temperature alerts clear when their disk leaves the report
`CheckHost` also clears a disk temperature alert, and drops its pending
@@ -2116,6 +2116,18 @@ the same record; the table renders defaults once, in both desktop table and
narrow card layouts, and per-section metric columns must resolve to metric
keys the shared column normalizer produces so the defaults editor reads and
writes the same record keys the section persists.
A global default that another setting decides while it is unset passes a
`globalDefaultFallbacks` entry for that metric (label, title, whether the other
setting is itself off, and the value switching on stages when it is). The
TrueNAS Disks temperature uses it to read `By type`, following Disk
temperature by type, with `features/alerts/thresholds/trueNASDiskTemperature.ts`
giving each disk row its type's trigger from the unsaved editor state.
`resolveAlertResourceGlobalDefaultCell` decides the cell for both layouts:
unset follows the fallback, switching on from Off returns to unset unless the
fallback is off, and an unset default with no fallback stays Off as the engine
reads it. `ResourceTable.tsx` hands its props to the shared table state through
getters, so the Custom badge and reset control follow defaults that load after
the table mounts.
Platform sub-routes that add native provider inventory must stay on the shared
platform page and table primitives. The vSphere Networks surface routes through
`/vmware/networks`, the shared platform tab model, the command palette
@@ -6102,6 +6102,13 @@ presentation. `frontend-modern/src/components/Infrastructure/`
placement, signal, and snapshot context through the canonical resource drawer
and debug/source sections rather than introducing a VMware-only detail route,
drawer tab, or provider-local investigation shell.
The TrueNAS physical-disk drawer judges a current temperature reading by the
disk temperature policy, like the TrueNAS storage table:
`useResourceDetailDrawerDerivedState.ts` passes the alerts store's
`getDiskTemperatureThresholds` into `buildTrueNASDetailSections`, and the
Temperature row takes a warning tone only when `isPhysicalDiskRunningHot`
says the reading reached its disk type's alert trigger. A retained reading
keeps its muted last-known row, and without a resolver heat is not judged.
That same infrastructure consumer boundary also owns source selection
continuity. Settings infrastructure panels and platform/runtime pages must
keep canonical sources such as `truenas` and `availability` present in their
@@ -0,0 +1,69 @@
{
"version": 1,
"base_sha": "bdf0f76b7bf61d550b1cb8574497eef75b354acb",
"verified_at": "2026-10-07T14:11:27Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/components/Alerts/AlertResourceTableDesktop.tsx",
"frontend-modern/src/components/Alerts/AlertResourceTableMobile.tsx",
"frontend-modern/src/components/Alerts/ResourceTable.tsx",
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx",
"frontend-modern/src/components/Alerts/ThresholdsTableTrueNASTab.tsx",
"frontend-modern/src/components/Alerts/alertResourceTableModel.ts",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerTrueNASModel.ts",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts",
"frontend-modern/src/features/alerts/alertsConfigurationModel.ts",
"frontend-modern/src/features/alerts/thresholds/hooks/useThresholdsPlatformData.ts",
"frontend-modern/src/features/alerts/thresholds/trueNASDiskTemperature.ts",
"frontend-modern/src/types/alerts.ts",
"frontend-modern/src/utils/alertThresholdDefaults.ts"
],
"content_sha256": {
"frontend-modern/src/components/Alerts/AlertResourceTableDesktop.tsx": "4d09dd97adc2868889e256455c207b6b9089ea311713878f0d6ff00efb6b7123",
"frontend-modern/src/components/Alerts/AlertResourceTableMobile.tsx": "ceaf136da9361a5b1756350626a833da4dfebcb4f211c7269d7d89d37fb7e085",
"frontend-modern/src/components/Alerts/ResourceTable.tsx": "7e80a6c789b3069fcadcb86e0a6689e5319e7b0ac5d5bfa1a596520daec2d7e1",
"frontend-modern/src/components/Alerts/ThresholdsTableAgentsResourcesSection.tsx": "fc4e9f4d05c0bf5799acbbe64c37b901e104869ff120819342432563789ad0e9",
"frontend-modern/src/components/Alerts/ThresholdsTableTrueNASTab.tsx": "a22b86c28b65ec18ba81171fb1206a24107a0d97db19d7dd488bdfd0a689a956",
"frontend-modern/src/components/Alerts/alertResourceTableModel.ts": "5881aeaf7ee575589997bf67fb713a01784ce7ca16ecb5a5eaea77f36010837a",
"frontend-modern/src/components/Infrastructure/resourceDetailDrawerTrueNASModel.ts": "b60ba59688d6ea0cabfb2b413b8adcaa85c8d41524c1d1f0df9bdb9e6d85733c",
"frontend-modern/src/components/Infrastructure/useResourceDetailDrawerDerivedState.ts": "63b07de3f354f8328298348a8892ad73fde2f11ec6768e9c0f6fc37eb4c43516",
"frontend-modern/src/features/alerts/alertsConfigurationModel.ts": "0e116e65c992a72e21d700835b0cb0db7173be9ec9c4980382958506d4e75976",
"frontend-modern/src/features/alerts/thresholds/hooks/useThresholdsPlatformData.ts": "8525b0b012198ff87e29063908a9907b7fa1080cd84187202b1cd3cf198a3b46",
"frontend-modern/src/features/alerts/thresholds/trueNASDiskTemperature.ts": "80433d42dd4658a1449c0f0c378d7fa52cd6dfab117d626195099768e750352f",
"frontend-modern/src/types/alerts.ts": "d072175ab98e3785eae643d655be4c2d54cac0eddc8b55803a2841a312062e5a",
"frontend-modern/src/utils/alertThresholdDefaults.ts": "077e291d5c224063a373a5e9f5555d9b470f585801b0cfdd9dc1acacf233af28"
},
"routes": [
"/alerts/thresholds/truenas",
"/truenas/storage"
],
"viewports": [
{
"width": 1366,
"height": 900
},
{
"width": 375,
"height": 812
}
],
"states": [
"Private mock stack (8 nodes, mock TrueNAS truenas-main) with a local-only generator overlay pinning TrueNAS disk temperatures: nvme0n1 60C, nvme1n1 72C, sda 57C (SATA), sdc 63C (SAS); after = this branch, before = main logic in the same tree",
"Thresholds > TrueNAS > Disks with no saved TrueNAS-wide value: header reads 'Defaults: NVMe 70\u00b0C \u00b7 SAS 65\u00b0C \u00b7 SATA 55\u00b0C', note 'Unless set here, each disk alerts at Disk temperature by type, set under Agents.', Global Defaults Temp cell empty with placeholder 'By type' (fits after widening to w-20) and On, disk rows nvme0n1/nvme1n1 70\u00b0C, sda/sdb 55\u00b0C, sdc-sdf 65\u00b0C",
"Same section at 375px: Global Defaults card shows TEMP \u00b0C 'By type' with On, disk cards show their type triggers, no horizontal overflow",
"Agent Disk Temp default switched off through the API: header 'Defaults: all metric alerts off', Global Defaults cell Off with title 'Off because the agent Disk Temp default under Agents is off...', disk rows Off",
"Saved TrueNAS-wide 62: header 'Defaults: Temp 62\u00b0C', every disk row 62\u00b0C, Custom badge and reset control after reload; saved 55 survives save and reload as 55 (stored with truenasDiskTemperatureByType)",
"Alerts with factory per-type thresholds: after stack raises nvme1n1 72 (trigger 70) and sda 57 (trigger 55) with diskType metadata, none for nvme0n1 60 or sdc 63; before (main) raised all four at a flat 55",
"TrueNAS storage table agrees: Attention 'Disk temperature is 72\u00b0C, at or above its 70\u00b0C alert threshold.' on nvme1n1 and '57\u00b0C ... 55\u00b0C' on sda, none on nvme0n1",
"TrueNAS disk drawer (Platform details > Show TrueNAS > Health): nvme0n1 Temperature 60\u00b0C in default text colour (main toned it amber from the fixed 55C); sda 57\u00b0C amber rgb(252,211,77) at 1366 and at 375 with no overflow"
],
"interactions": [
"Typed 62 into the TrueNAS Disks Global Defaults Temp cell, saved, reloaded: API truenasDiskDefaults {trigger 62, clear 57}, cell shows 62 with Custom",
"Triple-clicked and typed 55, saved, reloaded: API keeps {55, 50}, cell shows 55 with Custom and the reset control",
"Clicked Reset to factory defaults: cell returns to 'By type'; saved: API truenasDiskDefaults {}",
"On the 375px card, clicked On to switch the default Off, saved: API stores {trigger 0, clear 0} and the after stack's TrueNAS disk alerts resolve; clicked again: returns to 'By type', saved: {}",
"With the agent Disk Temp default off, clicked the Off badge: stages an explicit 55 (Custom, rows 55\u00b0C); discarded",
"Loaded an old saved config (flat 55/50, no marker) with the new binary: GET returns truenasDiskDefaults {} and truenasDiskTemperatureByType true",
"Expanded nvme0n1 and sda rows on /truenas/storage, opened Platform details and Show TrueNAS, read the Temperature row value and computed colour at 1366 and 375"
]
}
@@ -43,12 +43,11 @@ import {
ALERT_RESOURCE_METRIC_OFF_VALUE,
getAlertResourceColumnKind,
getAlertResourceColumnHeaderTooltip,
getAlertResourceEnabledDefault,
getAlertResourceMetricBounds,
getAlertResourceMetricDelayOverride,
getAlertResourceMetricStep,
isAlertResourceMetricOff,
normalizeAlertResourceMetricKey,
resolveAlertResourceGlobalDefaultCell,
} from './alertResourceTableModel';
import type { OfflineState, Resource, ResourceTableProps } from './ResourceTable';
@@ -210,7 +209,10 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
{(column) => (
<TableHead
class={`${getPlatformTableHeadClassForKind(getAlertResourceColumnKind(column))} whitespace-normal wrap-break-word`}
title={getAlertResourceColumnHeaderTooltip(column)}
title={
props.table.columnTooltips?.[column] ??
getAlertResourceColumnHeaderTooltip(column)
}
>
{column}
</TableHead>
@@ -281,10 +283,14 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
{(column) => {
const metric = normalizeAlertResourceMetricKey(column);
const bounds = getAlertResourceMetricBounds(metric);
const value = () => props.table.globalDefaults?.[metric] ?? 0;
const value = () => props.table.globalDefaults?.[metric];
const fallback = () => props.table.globalDefaultFallbacks?.[metric];
// An unset default and a stored 0 are both disabled to the
// alert engine, so neither may render as On.
const isOff = () => isAlertResourceMetricOff(value());
// alert engine, so neither may render as On, unless the
// metric's unset default follows another setting.
const cell = () =>
resolveAlertResourceGlobalDefaultCell(metric, value(), fallback());
const isOff = () => cell().isOff;
return (
<TableCell
@@ -297,8 +303,12 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
min={bounds.min}
max={bounds.max}
step={getAlertResourceMetricStep(metric)}
value={isOff() ? '' : value()}
placeholder={getAlertResourceTableMetricPlaceholder(isOff())}
value={isOff() || cell().follows ? '' : (value() ?? '')}
placeholder={
cell().follows && !isOff()
? fallback()?.label
: getAlertResourceTableMetricPlaceholder(isOff())
}
disabled={isOff()}
onInput={(e) => {
const nextValue = parseFloat(e.currentTarget.value);
@@ -318,12 +328,16 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
}));
props.table.setHasUnsavedChanges?.(true);
}}
class={`w-16 px-2 py-0.5 text-sm text-center border rounded ${
class={`${cell().follows && !isOff() ? 'w-20' : 'w-16'} px-2 py-0.5 text-sm text-center border rounded ${
isOff()
? 'border-border bg-surface-alt text-muted italic placeholder: dark:placeholder: placeholder:opacity-60 pointer-events-none'
: 'border-border text-base-content focus:border-blue-500 focus:ring-1 focus:ring-blue-500'
}`}
title={getAlertResourceTableMetricInputTitle(isOff())}
title={
cell().follows
? fallback()?.title
: getAlertResourceTableMetricInputTitle(isOff())
}
/>
<Show when={isOff()}>
<button
@@ -332,7 +346,7 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
onClick={() => {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: getAlertResourceEnabledDefault(metric),
[metric]: cell().enableValue,
}));
props.table.setHasUnsavedChanges?.(true);
}}
@@ -348,7 +362,7 @@ export function AlertResourceTableDesktop(props: AlertResourceTableDesktopProps)
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: isOff()
? getAlertResourceEnabledDefault(metric)
? cell().enableValue
: ALERT_RESOURCE_METRIC_OFF_VALUE,
}));
props.table.setHasUnsavedChanges?.(true);
@@ -26,7 +26,6 @@ import {
ALERT_RESOURCE_METRIC_OFF_VALUE,
alertResourceSupportsMetric,
buildAlertResourceEditPayload,
getAlertResourceEnabledDefault,
getAlertResourceLabel,
getAlertResourceMetricBounds,
getAlertResourceMetricDisplayValue,
@@ -34,6 +33,7 @@ import {
isAlertResourceMetricOff,
isAlertResourceMetricOverridden,
normalizeAlertResourceMetricKey,
resolveAlertResourceGlobalDefaultCell,
resolveAlertResourceMetricEnableValue,
} from './alertResourceTableModel';
import type { Resource } from '@/features/alerts/thresholds/tableTypes';
@@ -212,10 +212,14 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
{(column) => {
const metric = normalizeAlertResourceMetricKey(column);
const bounds = getAlertResourceMetricBounds(metric);
const value = () => props.table.globalDefaults?.[metric] ?? 0;
const value = () => props.table.globalDefaults?.[metric];
const fallback = () => props.table.globalDefaultFallbacks?.[metric];
// An unset default and a stored 0 are both disabled to the alert
// engine, so neither may render as On.
const isOff = () => isAlertResourceMetricOff(value());
// engine, so neither may render as On, unless the metric's unset
// default follows another setting.
const cell = () =>
resolveAlertResourceGlobalDefaultCell(metric, value(), fallback());
const isOff = () => cell().isOff;
return (
<div class="p-2 bg-surface rounded-sm border border-border-subtle flex flex-col gap-1">
@@ -227,8 +231,13 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
min={bounds.min}
max={bounds.max}
step={getAlertResourceMetricStep(metric)}
value={isOff() ? '' : value()}
placeholder={getAlertResourceTableMetricPlaceholder(isOff())}
value={isOff() || cell().follows ? '' : (value() ?? '')}
placeholder={
cell().follows && !isOff()
? fallback()?.label
: getAlertResourceTableMetricPlaceholder(isOff())
}
title={cell().follows ? fallback()?.title : undefined}
disabled={isOff()}
class={`min-h-11 w-full rounded-sm border p-1 text-center text-sm sm:min-h-0 ${isOff() ? 'bg-surface-hover' : ' border-border'}`}
onInput={(e) => {
@@ -257,7 +266,7 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
onClick={() => {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: getAlertResourceEnabledDefault(metric),
[metric]: cell().enableValue,
}));
props.table.setHasUnsavedChanges?.(true);
}}
@@ -271,7 +280,7 @@ export function AlertResourceTableMobile(props: AlertResourceTableMobileProps) {
props.table.setGlobalDefaults?.((prev) => ({
...prev,
[metric]: isOff()
? getAlertResourceEnabledDefault(metric)
? cell().enableValue
: ALERT_RESOURCE_METRIC_OFF_VALUE,
}));
props.table.setHasUnsavedChanges?.(true);
@@ -1,4 +1,5 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { createSignal } from 'solid-js';
import { cleanup, fireEvent, render, screen } from '@solidjs/testing-library';
import { ALERT_BULK_EDIT_CLEAR_LABEL } from '@/utils/alertBulkEditPresentation';
import resourceTableSource from '@/components/Alerts/ResourceTable.tsx?raw';
@@ -1007,6 +1008,144 @@ describe('ResourceTable', () => {
expect(updater({ cpu: 0 })).toEqual({ cpu: 80 });
});
it.each([
['desktop', false],
['mobile', true],
])('lets an unset %s global default follow its fallback', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: undefined },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
// Unset reads On and names where the value comes from.
expect(screen.getByTestId('status-badge')).toHaveTextContent('On');
const input = screen.getByPlaceholderText('By type') as HTMLInputElement;
expect(input.value).toBe('');
expect(input.disabled).toBe(false);
expect(input.title).toBe('Follows Disk temperature by type');
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: undefined })).toEqual({ temperature: -1 });
});
it.each([
['desktop', false],
['mobile', true],
])('switches an Off %s fallback default back to following', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: -1 },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByTestId('status-badge')).toHaveTextContent('Off');
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: -1 })).toEqual({ temperature: undefined });
});
it.each([
['desktop', false],
['mobile', true],
])('reads an unset %s default Off when its fallback is off', (_layout, mobile) => {
mockIsMobile.mockReturnValue(mobile);
const setGlobalDefaults = vi.fn();
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: undefined },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Off by type', off: true, enableValue: 55 },
},
setGlobalDefaults,
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByTestId('status-badge')).toHaveTextContent('Off');
expect(screen.queryByPlaceholderText('By type')).toBeNull();
// Unset would still be off, so switching on stages an explicit value.
fireEvent.click(screen.getByTestId('status-badge'));
const updater = setGlobalDefaults.mock.calls[0][0] as (
prev: Record<string, number | undefined>,
) => Record<string, number | undefined>;
expect(updater({ temperature: undefined })).toEqual({ temperature: 55 });
});
it('shows a set fallback global default as its number', () => {
const props = makeProps({
resources: [],
columns: ['Temp °C'],
globalDefaults: { temperature: 62 },
globalDefaultFallbacks: {
temperature: { label: 'By type', title: 'Follows Disk temperature by type' },
},
setGlobalDefaults: vi.fn(),
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} />);
expect(screen.getByDisplayValue('62')).toBeInTheDocument();
expect(screen.queryByPlaceholderText('By type')).toBeNull();
});
it('follows global defaults that arrive after the table mounts', () => {
const [globalDefaults, setGlobalDefaults] = createSignal<Record<string, number | undefined>>({
temperature: undefined,
});
const props = makeProps({
resources: [],
columns: ['Temp °C'],
factoryDefaults: { temperature: undefined },
onResetDefaults: vi.fn(),
setGlobalDefaults: vi.fn(),
setHasUnsavedChanges: vi.fn(),
});
render(() => <ResourceTable {...props} globalDefaults={globalDefaults()} />);
expect(screen.queryByText('Custom')).toBeNull();
// The saved config loads after mount.
setGlobalDefaults({ temperature: 62 });
expect(screen.getByText('Custom')).toBeInTheDocument();
});
it('uses a table column tooltip over the generic one', () => {
const props = makeProps({
resources: [],
columns: ['Temp °C'],
columnTooltips: { 'Temp °C': 'Disk temperature that raises an alert.' },
});
render(() => <ResourceTable {...props} />);
expect(screen.getByText('Temp °C', { selector: 'th' })).toHaveAttribute(
'title',
'Disk temperature that raises an alert.',
);
});
it('stages an explicit threshold from the mobile row editor', () => {
mockIsMobile.mockReturnValue(true);
const setEditingThresholds = vi.fn();
@@ -11,6 +11,7 @@ import { ActionIconButton } from '@/components/shared/Button';
import { useObservedElementWidth } from '@/hooks/useObservedElementWidth';
import { useAlertResourceTableState } from './useAlertResourceTableState';
import type { GroupHeaderMeta, Resource } from '@/features/alerts/thresholds/tableTypes';
import type { AlertResourceGlobalDefaultFallback } from './alertResourceTableModel';
import { AlertResourceTableDesktop } from './AlertResourceTableDesktop';
import { AlertResourceTableMobile } from './AlertResourceTableMobile';
@@ -49,6 +50,10 @@ export interface ResourceTableProps {
formatMetricValue: (metric: string, value: number | undefined) => string;
hasActiveAlert: (resourceId: string, metric: string) => boolean;
globalDefaults?: Record<string, number | undefined>;
/** Metrics whose unset global default follows another setting. */
globalDefaultFallbacks?: Record<string, AlertResourceGlobalDefaultFallback>;
/** Header tooltips that replace the generic one for a column label. */
columnTooltips?: Record<string, string>;
setGlobalDefaults?: (
value:
| Record<string, number | undefined>
@@ -97,10 +102,20 @@ export function ResourceTable(props: ResourceTableProps) {
someSelected,
clearSelectedIds,
} = useAlertResourceTableState({
resources: props.resources,
groupedResources: props.groupedResources,
globalDefaults: props.globalDefaults,
factoryDefaults: props.factoryDefaults,
// Getters keep the state reactive: saved defaults arrive after mount, and
// the Custom badge and reset control must follow them.
get resources() {
return props.resources;
},
get groupedResources() {
return props.groupedResources;
},
get globalDefaults() {
return props.globalDefaults;
},
get factoryDefaults() {
return props.factoryDefaults;
},
});
const totalColumnCount = createMemo(
() =>
@@ -82,9 +82,9 @@ export function ThresholdsTableAgentsResourcesSection(props: ThresholdsTableSect
<Card padding="md" tone="card" class="mt-4">
<h3 class="text-sm font-semibold text-base-content">Disk temperature by type</h3>
<p class="mt-1 text-xs text-muted">
Alert trigger in °C for each disk type. Warning colors start 5°C below the trigger.
Setting a Disk Temp override on a host above replaces these for all of that host's
disks.
Alert trigger in °C for each disk type, for agent and TrueNAS disks. Warning colors
start 5°C below the trigger. A Disk Temp override on a host above replaces these for
all of that host's disks, and a TrueNAS Disks value replaces them for TrueNAS disks.
</p>
<div class="mt-4 grid gap-4 sm:grid-cols-3">
<For each={DISK_TEMP_TYPE_FIELDS}>
@@ -8,6 +8,9 @@ import { CollapsibleSection } from './Thresholds/sections/CollapsibleSection';
import { formatMetricValue } from '@/features/alerts/thresholds/helpers';
import type { ThresholdsTableSectionProps } from '@/features/alerts/thresholds/thresholdsTableSectionProps';
import type { Resource } from '@/features/alerts/thresholds/tableTypes';
import { getDiskTemperatureByTypeItems } from '@/features/alerts/thresholds/trueNASDiskTemperature';
import { FACTORY_AGENT_DEFAULTS } from '@/utils/alertThresholdDefaults';
import type { AlertResourceGlobalDefaultFallback } from './alertResourceTableModel';
import { getAlertThresholdsDefaultsSummary } from '@/utils/alertThresholdsSectionPresentation';
const TRUENAS_SYSTEM_COLUMNS = [
@@ -22,6 +25,7 @@ const TRUENAS_SYSTEM_COLUMNS = [
];
const TRUENAS_STORAGE_COLUMNS = ['Usage %'];
const TRUENAS_DISK_COLUMNS = ['Temp °C'];
const TRUENAS_DISK_COLUMN_TOOLTIPS = { 'Temp °C': 'Disk temperature that raises an alert.' };
function TrueNASResourceSection(
props: ThresholdsTableSectionProps & {
@@ -39,6 +43,10 @@ function TrueNASResourceSection(
| ((prev: Record<string, number | undefined>) => Record<string, number | undefined>),
) => void;
onResetDefaults?: () => void;
defaultsSummary?: string;
globalDefaultFallbacks?: Record<string, AlertResourceGlobalDefaultFallback>;
columnTooltips?: Record<string, string>;
note?: string;
},
) {
const { state, tableProps } = props;
@@ -48,7 +56,9 @@ function TrueNASResourceSection(
<CollapsibleSection
id={props.id}
title={props.title}
defaultsSummary={getAlertThresholdsDefaultsSummary(props.columns, props.defaults)}
defaultsSummary={
props.defaultsSummary ?? getAlertThresholdsDefaultsSummary(props.columns, props.defaults)
}
resourceCount={props.resources().length}
collapsed={state.isCollapsed(props.id)}
onToggle={() => state.toggleSection(props.id)}
@@ -57,6 +67,9 @@ function TrueNASResourceSection(
emptyMessage="No TrueNAS alert targets match the current filters."
>
<div ref={state.registerSection(props.id)} class="scroll-mt-24">
<Show when={props.note}>
<p class="mb-3 text-xs text-muted">{props.note}</p>
</Show>
<ResourceTable
title=""
onConfigureResourceIntent={tableProps.onConfigureResourceIntent}
@@ -79,6 +92,8 @@ function TrueNASResourceSection(
formatMetricValue={formatMetricValue}
hasActiveAlert={state.hasActiveAlert}
globalDefaults={props.defaults}
globalDefaultFallbacks={props.globalDefaultFallbacks}
columnTooltips={props.columnTooltips}
setGlobalDefaults={props.setGlobalDefaults}
setHasUnsavedChanges={tableProps.setHasUnsavedChanges}
globalDisableFlag={tableProps.disableAllTrueNAS}
@@ -104,6 +119,38 @@ export function ThresholdsTableTrueNASTab(props: ThresholdsTableSectionProps) {
const trueNASDefaults = () => props.tableProps.trueNASDefaults ?? {};
const trueNASStorageDefaults = () => ({ usage: trueNASDefaults().usage ?? 85 });
const trueNASDiskDefaults = () => props.tableProps.trueNASDiskDefaults ?? {};
const diskTemperatureByType = () =>
getDiskTemperatureByTypeItems({
agentDiskTemperature: props.tableProps.agentDefaults.diskTemperature,
diskTempByType: props.tableProps.diskTempByType,
});
// Unset, TrueNAS disks follow Disk temperature by type, so the header names
// the per-type triggers rather than one number.
const trueNASDiskDefaultsSummary = () => {
if (trueNASDiskDefaults().temperature !== undefined) return undefined;
return getAlertThresholdsDefaultsSummary(
TRUENAS_DISK_COLUMNS,
diskTemperatureByType().length > 0 ? {} : { temperature: 0 },
{ ruleItems: diskTemperatureByType() },
);
};
// When the agent Disk Temp default is off, Disk temperature by type is off
// too, so the unset cell reads Off and switching it on stages a TrueNAS-wide
// value at the factory disk temperature trigger.
const trueNASDiskTemperatureFallback = (): Record<string, AlertResourceGlobalDefaultFallback> => {
const items = diskTemperatureByType();
return {
temperature: {
label: 'By type',
title:
items.length > 0
? `Follows Disk temperature by type under Agents: ${items.join(', ')}. Enter a value to use one temperature for every TrueNAS disk.`
: 'Off because the agent Disk Temp default under Agents is off. Turn it on here to alert on TrueNAS disks anyway.',
off: items.length === 0,
enableValue: FACTORY_AGENT_DEFAULTS.diskTemperature,
},
};
};
return (
<>
@@ -175,6 +222,10 @@ export function ThresholdsTableTrueNASTab(props: ThresholdsTableSectionProps) {
icon={<HardDrive class="w-5 h-5" />}
typeKey="truenas-disk"
defaults={trueNASDiskDefaults()}
defaultsSummary={trueNASDiskDefaultsSummary()}
globalDefaultFallbacks={trueNASDiskTemperatureFallback()}
columnTooltips={TRUENAS_DISK_COLUMN_TOOLTIPS}
note="Unless set here, each disk alerts at Disk temperature by type, set under Agents."
factoryDefaults={props.tableProps.factoryTrueNASDiskDefaults}
setGlobalDefaults={props.tableProps.setTrueNASDiskDefaults}
onResetDefaults={props.tableProps.resetTrueNASDiskDefaults}
@@ -230,6 +230,49 @@ export function resolveAlertResourceMetricEnableValue(
return undefined;
}
/**
* A global default that may stay unset because another setting decides it,
* such as TrueNAS disk temperature following Disk temperature by type. Unset,
* the cell is empty with `label` as its placeholder and `title` explaining
* where the value comes from. `off` says the other setting is itself off, so
* the unset cell reads Off and switching it on has to stage `enableValue`.
*/
export interface AlertResourceGlobalDefaultFallback {
label: string;
title: string;
off?: boolean;
enableValue?: number;
}
export interface AlertResourceGlobalDefaultCell {
/** Unset and decided by the fallback. */
follows: boolean;
isOff: boolean;
/** Value staged when the cell is switched back on from Off. */
enableValue: number | undefined;
}
/**
* State of one Global Defaults metric cell. An unset default is Off, as the
* alert engine reads it, unless the metric has a fallback; then it reads as
* the fallback does. Switching it on from Off returns it to unset when the
* fallback is on, and stages an explicit value when the fallback is off,
* since unset would still be off.
*/
export function resolveAlertResourceGlobalDefaultCell(
metric: string,
value: number | undefined,
fallback: AlertResourceGlobalDefaultFallback | undefined,
): AlertResourceGlobalDefaultCell {
const follows = value === undefined && fallback !== undefined;
const explicitEnableValue = fallback?.enableValue ?? getAlertResourceEnabledDefault(metric);
return {
follows,
isOff: follows ? Boolean(fallback?.off) : isAlertResourceMetricOff(value ?? 0),
enableValue: fallback && !fallback.off ? undefined : explicitEnableValue,
};
}
export function getAlertResourceMetricDelayOverride(
metricDelaySeconds: Record<string, number> | undefined,
metric: string,
@@ -830,16 +830,39 @@ describe('buildTrueNASVMSections additional branches', () => {
// =========================================================================
describe('buildTrueNASDiskSections additional branches', () => {
it('sets Temperature tone to warning at 55 degrees', () => {
expect(findRow(diskRes({ temperature: 55 }), 'Temperature')?.tone).toBe('warning');
// The drawer judges heat by the disk type's alert trigger, through the
// resolver the alerts store provides; these are the factory triggers.
const factoryTriggers: Record<string, number> = { nvme: 70, sas: 65, sata: 55 };
const resolveThresholds = (diskType: string) => {
const critical = factoryTriggers[diskType.toLowerCase()] ?? 55;
return { warning: critical - 5, critical };
};
const temperatureTone = (disk: Partial<ResourcePhysicalDiskMeta>) =>
buildTrueNASDetailSections(diskRes(disk), resolveThresholds)
.flatMap((section) => section.rows)
.find((row) => row.label === 'Temperature')?.tone;
it('sets Temperature tone to warning from the disk type trigger', () => {
expect(temperatureTone({ diskType: 'sata', temperature: 55 })).toBe('warning');
expect(temperatureTone({ diskType: 'sas', temperature: 65 })).toBe('warning');
expect(temperatureTone({ diskType: 'nvme', temperature: 70 })).toBe('warning');
});
it('sets Temperature tone to warning above 55 degrees', () => {
expect(findRow(diskRes({ temperature: 60 }), 'Temperature')?.tone).toBe('warning');
it('keeps a reading under its type trigger at default tone', () => {
expect(temperatureTone({ diskType: 'nvme', temperature: 60 })).toBe('default');
expect(temperatureTone({ diskType: 'sas', temperature: 63 })).toBe('default');
expect(temperatureTone({ diskType: 'sata', temperature: 40 })).toBe('default');
});
it('sets Temperature tone to default below 55 degrees', () => {
expect(findRow(diskRes({ temperature: 40 }), 'Temperature')?.tone).toBe('default');
it('judges no heat without thresholds', () => {
expect(findRow(diskRes({ diskType: 'sata', temperature: 80 }), 'Temperature')?.tone).toBe(
'default',
);
expect(
buildTrueNASDetailSections(diskRes({ diskType: 'sata', temperature: 80 }), () => null)
.flatMap((section) => section.rows)
.find((row) => row.label === 'Temperature')?.tone,
).toBe('default');
});
it('formats Wearout percentage for a valid value', () => {
@@ -381,8 +381,9 @@ describe('resourceDetailDrawerTrueNASModel', () => {
},
},
});
const sataTrigger = () => ({ warning: 50, critical: 55 });
const temperatureRow = (resource: Resource) =>
buildTrueNASDetailSections(resource)
buildTrueNASDetailSections(resource, sataTrigger)
.find((section) => section.label === 'Health')
?.rows.find((row) => row.label === 'Temperature');
@@ -20,7 +20,11 @@ import {
type DetailSection,
type DetailValueTone,
} from '@/components/shared/detailSectionModel';
import { getPhysicalDiskTemperaturePresentation } from '@/features/storageBackups/diskTemperaturePresentation';
import {
getPhysicalDiskTemperaturePresentation,
isPhysicalDiskRunningHot,
} from '@/features/storageBackups/diskTemperaturePresentation';
import type { TrueNASDiskTemperatureThresholdResolver } from '@/features/truenas/truenasPageModel';
import { hasImpairedResourceSource } from '@/utils/resourceSourceHealth';
export type ResourceDetailDrawerTrueNASRowTone = DetailValueTone;
@@ -399,9 +403,13 @@ const diskTemperature = (disk: ResourcePhysicalDiskMeta) =>
});
// A retained reading, such as a silent host agent's on a merged disk row, is
// not current: it takes no heat tone and its title says why.
// not current: it takes no heat tone and its title says why. A current reading
// is hot from its disk type's alert trigger, the threshold the TrueNAS storage
// table and the disk's temperature alert judge it by. Without a resolver, heat
// is not judged.
const diskTemperatureRow = (
disk: ResourcePhysicalDiskMeta,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASRow | null => {
const reading = diskTemperature(disk);
if (!reading) return null;
@@ -411,9 +419,12 @@ const diskTemperatureRow = (
tone: 'muted',
});
}
return row('Temperature', reading.label, {
tone: (disk.temperature ?? 0) >= 55 ? 'warning' : 'default',
});
const thresholds = resolveDiskTemperatureThresholds?.(disk.diskType ?? '') ?? null;
const hot = isPhysicalDiskRunningHot(
{ temperature: disk.temperature ?? 0, collection: disk.collection },
thresholds,
);
return row('Temperature', reading.label, { tone: hot ? 'warning' : 'default' });
};
const currentDiskTemperatureLabel = (disk: ResourcePhysicalDiskMeta): string | null => {
@@ -424,6 +435,7 @@ const currentDiskTemperatureLabel = (disk: ResourcePhysicalDiskMeta): string | n
const buildTrueNASDiskSections = (
resource: Resource,
disk: ResourcePhysicalDiskMeta,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASSection[] => {
const riskReasons = (disk.risk?.reasons ?? [])
.map((reason) => asString(reason.summary))
@@ -439,7 +451,7 @@ const buildTrueNASDiskSections = (
const healthRows = compactRows([
row('Health', normalizeDelimitedLabel(disk.health), { tone: diskStateTone(disk) }),
diskTemperatureRow(disk),
diskTemperatureRow(disk, resolveDiskTemperatureThresholds),
row(
'Wearout',
disk.wearout === undefined || disk.wearout < 0 ? null : formatPercent(disk.wearout),
@@ -746,6 +758,7 @@ const buildTrueNASShareSections = (
export const buildTrueNASDetailSections = (
resource: Resource,
resolveDiskTemperatureThresholds?: TrueNASDiskTemperatureThresholdResolver,
): ResourceDetailDrawerTrueNASSection[] => {
if (resource.truenas?.share) return buildTrueNASShareSections(resource.truenas.share);
if (resource.truenas?.vm) return buildTrueNASVMSections(resource, resource.truenas.vm);
@@ -754,7 +767,11 @@ export const buildTrueNASDetailSections = (
return buildTrueNASStorageSections(resource, resource.storage);
}
if (isTrueNASScopedResource(resource) && resource.physicalDisk) {
return buildTrueNASDiskSections(resource, resource.physicalDisk);
return buildTrueNASDiskSections(
resource,
resource.physicalDisk,
resolveDiskTemperatureThresholds,
);
}
if (isTrueNASScopedResource(resource) && resource.truenas) {
return buildTrueNASSystemSections(resource, resource.truenas);
@@ -23,6 +23,7 @@ import {
} from '@/utils/resourceIdentity';
import { isPulseAgentPlatformResource } from '@/utils/agentResources';
import { areSystemSettingsLoaded, shouldHideDockerUpdateActions } from '@/stores/systemSettings';
import { useAlertsActivation } from '@/stores/alertsActivation';
import {
getResourcePolicyBadges,
getResourcePolicyGovernedSummary,
@@ -381,7 +382,10 @@ export const useResourceDetailDrawerDerivedState = (
const vmwareDetailsSummary = createMemo(() =>
buildVMwareDetailsSummary(resource().type, vmwareData()),
);
const trueNASDetailSections = createMemo(() => buildTrueNASDetailSections(resource()));
const { getDiskTemperatureThresholds } = useAlertsActivation();
const trueNASDetailSections = createMemo(() =>
buildTrueNASDetailSections(resource(), getDiskTemperatureThresholds),
);
const hasTrueNASDetails = createMemo(() => trueNASDetailSections().length > 0);
const trueNASDetailsSummary = createMemo(() => buildTrueNASDetailsSummary(resource()));
const kubernetesDetailSections = createMemo(() => buildKubernetesDetailSections(resource()));
@@ -56,7 +56,8 @@ describe('readAlertsConfigurationSnapshot — absent optional sections', () => {
expect(snapshot.trueNASDefaults.cpu).toBe(80);
expect(snapshot.trueNASDefaults.usage).toBe(85);
expect(snapshot.trueNASDefaults.diskRead).toBe(-1);
expect(snapshot.trueNASDiskDefaults.temperature).toBe(55);
// Unset: each TrueNAS disk follows Disk temperature by type.
expect(snapshot.trueNASDiskDefaults.temperature).toBeUndefined();
expect(snapshot.vmwareDefaults.cpu).toBe(80);
expect(snapshot.vmwareDefaults.usage).toBe(85);
});
@@ -232,7 +233,8 @@ describe('readAlertsConfigurationSnapshot — factory fallback (undefined-trigge
expect(snapshot.trueNASDefaults.diskWrite).toBe(-1);
expect(snapshot.trueNASDefaults.networkIn).toBe(-1);
expect(snapshot.trueNASDefaults.networkOut).toBe(-1);
expect(snapshot.trueNASDiskDefaults.temperature).toBe(55);
// No usable trigger leaves TrueNAS disks on Disk temperature by type.
expect(snapshot.trueNASDiskDefaults.temperature).toBeUndefined();
});
it('falls back to factory defaults for vmwareDefaults and agentDefaults', () => {
@@ -1526,3 +1528,48 @@ describe('dockerDefaults.updateAlertDelayHours', () => {
expect(alertConfig?.dockerDefaults?.updateAlertDelayHours).toBe(24);
});
});
describe('TrueNAS disk temperature round trip', () => {
const buildTrueNASDiskPayload = (temperature: number | undefined) => {
const snapshot = createDefaultAlertsConfigurationSnapshot();
snapshot.trueNASDiskDefaults = { temperature };
return buildAlertsConfigurationPayload({
snapshot,
rawOverridesConfig: {},
alertsActivationState: null,
alertsActivationConfig: null,
}).alertConfig;
};
it('omits an unset TrueNAS disk temperature so disks follow Disk temperature by type', () => {
const alertConfig = buildTrueNASDiskPayload(undefined);
expect(alertConfig?.truenasDiskDefaults).toEqual({});
expect(alertConfig?.truenasDiskTemperatureByType).toBe(true);
});
it('writes Off as a zero trigger, which the backend keeps as off', () => {
expect(buildTrueNASDiskPayload(-1)?.truenasDiskDefaults).toEqual({
temperature: { trigger: 0, clear: 0 },
});
});
it('writes a TrueNAS-wide value, 55 included, as a trigger', () => {
expect(buildTrueNASDiskPayload(55)?.truenasDiskDefaults).toEqual({
temperature: { trigger: 55, clear: 50 },
});
});
it('reads an absent temperature as unset and a zero trigger as off', () => {
const base = { overrides: {} } as unknown as AlertConfig;
expect(
readAlertsConfigurationSnapshot({ ...base, truenasDiskDefaults: {} }).trueNASDiskDefaults
.temperature,
).toBeUndefined();
expect(
readAlertsConfigurationSnapshot({
...base,
truenasDiskDefaults: { temperature: { trigger: 0, clear: 0 } },
}).trueNASDiskDefaults.temperature,
).toBe(0);
});
});
@@ -182,6 +182,15 @@ const createHysteresisThreshold = (trigger: number | undefined, clearMargin = 5)
};
};
// Unset leaves TrueNAS disks on Disk temperature by type, so it is omitted
// rather than written as a trigger. Off is written as 0: the backend reads a
// negative trigger as unset.
const buildTrueNASDiskDefaultsConfig = (temperature: number | undefined) => {
if (temperature === undefined) return {};
if (temperature <= 0) return { temperature: { trigger: 0, clear: 0 } };
return { temperature: createHysteresisThreshold(temperature) };
};
const normalizeGap = (value: unknown, fallback: number) => {
const numeric = Number(value);
if (!Number.isFinite(numeric)) {
@@ -410,10 +419,10 @@ export function readAlertsConfigurationSnapshot(config: AlertConfig): AlertsConf
}
if (config.truenasDiskDefaults) {
// Unset means each TrueNAS disk follows Disk temperature by type.
const temperature = config.truenasDiskDefaults.temperature;
snapshot.trueNASDiskDefaults = {
temperature:
getTriggerValue(config.truenasDiskDefaults.temperature) ??
FACTORY_TRUENAS_DISK_DEFAULTS.temperature,
temperature: temperature === undefined ? undefined : getTriggerValue(temperature),
};
}
@@ -844,9 +853,8 @@ export function buildAlertsConfigurationPayload({
networkIn: createHysteresisThreshold(snapshot.trueNASDefaults.networkIn),
networkOut: createHysteresisThreshold(snapshot.trueNASDefaults.networkOut),
},
truenasDiskDefaults: {
temperature: createHysteresisThreshold(snapshot.trueNASDiskDefaults.temperature),
},
truenasDiskDefaults: buildTrueNASDiskDefaultsConfig(snapshot.trueNASDiskDefaults.temperature),
truenasDiskTemperatureByType: true,
vmwareDefaults: {
cpu: createHysteresisThreshold(snapshot.vmwareDefaults.cpu),
memory: createHysteresisThreshold(snapshot.vmwareDefaults.memory),
@@ -0,0 +1,44 @@
import { describe, expect, it } from 'vitest';
import {
getDiskTemperatureByTypeItems,
resolveDiskTemperatureTriggerForType,
resolveTrueNASDiskTemperatureDefault,
} from '../trueNASDiskTemperature';
const factoryPolicy = {
agentDiskTemperature: 55,
diskTempByType: { nvme: 70, sas: 65, sata: 55 },
};
describe('TrueNAS disk temperature in the thresholds editor', () => {
it('gives each disk its type trigger, and unknown types the agent default', () => {
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'NVMe')).toBe(70);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'sas')).toBe(65);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'sata')).toBe(55);
expect(resolveDiskTemperatureTriggerForType(factoryPolicy, 'hdd')).toBe(55);
expect(
resolveDiskTemperatureTriggerForType({ ...factoryPolicy, agentDiskTemperature: 60 }, ''),
).toBe(60);
});
it('switches every type off with the agent Disk Temp default', () => {
const off = { ...factoryPolicy, agentDiskTemperature: -1 };
expect(resolveDiskTemperatureTriggerForType(off, 'nvme')).toBe(0);
expect(getDiskTemperatureByTypeItems(off)).toEqual([]);
});
it('lets a TrueNAS-wide value, Off included, replace the type trigger', () => {
expect(resolveTrueNASDiskTemperatureDefault(undefined, factoryPolicy, 'nvme')).toBe(70);
expect(resolveTrueNASDiskTemperatureDefault(62, factoryPolicy, 'nvme')).toBe(62);
expect(resolveTrueNASDiskTemperatureDefault(-1, factoryPolicy, 'nvme')).toBe(-1);
});
it('names the per-type triggers', () => {
expect(
getDiskTemperatureByTypeItems({
...factoryPolicy,
diskTempByType: { nvme: 75, sas: 65, sata: 55 },
}),
).toEqual(['NVMe 75°C', 'SAS 65°C', 'SATA 55°C']);
});
});
@@ -9,6 +9,7 @@ import type { Resource } from '@/types/resource';
import type { Resource as TableResource } from '../tableTypes';
import type { Override } from '../types';
import type { ThresholdsDataInputs } from '../thresholdsResourceModel';
import { resolveTrueNASDiskTemperatureDefault } from '../trueNASDiskTemperature';
import {
createOverridesMap,
findOverrideByCandidates,
@@ -307,7 +308,18 @@ export function useThresholdsPlatformData(inputs: ThresholdsDataInputs) {
resource,
type: 'truenasDisk',
resourceType: 'TrueNAS Disk',
defaults: trueNASDiskDefaults(),
// Each disk inherits the trigger saving would apply to it: the
// TrueNAS-wide value, else its type's disk temperature trigger.
defaults: {
temperature: resolveTrueNASDiskTemperatureDefault(
trueNASDiskDefaults().temperature,
{
agentDiskTemperature: props.agentDefaults?.diskTemperature,
diskTempByType: props.diskTempByType ?? {},
},
resource.physicalDisk?.diskType,
),
},
overridesMap,
node: trueNASSystemName(resource),
instance: readString(resource.physicalDisk?.diskType) || 'TrueNAS',
@@ -0,0 +1,51 @@
import { formatMetricValue } from './helpers';
import { FACTORY_AGENT_DEFAULTS } from '@/utils/alertThresholdDefaults';
// The thresholds editor's view of the TrueNAS disk temperature trigger. It
// mirrors the backend (trueNASDiskTemperatureDefaultNoLock): a TrueNAS-wide
// value the user saved applies to every TrueNAS disk; unset, each disk follows
// Disk temperature by type, which the agent Disk Temp default switches off for
// every type when it is off. It reads the unsaved editor state, so a row shows
// what saving would apply.
export const DISK_TEMPERATURE_TYPE_FIELDS: readonly { key: string; label: string }[] = [
{ key: 'nvme', label: 'NVMe' },
{ key: 'sas', label: 'SAS' },
{ key: 'sata', label: 'SATA' },
];
export interface DiskTemperaturePolicyState {
/** The agent Disk Temp default trigger; zero or less is off. */
agentDiskTemperature: number | undefined;
diskTempByType: Record<string, number>;
}
/** The disk temperature policy's trigger for a disk type; zero or less is off. */
export function resolveDiskTemperatureTriggerForType(
policy: DiskTemperaturePolicyState,
diskType: string | null | undefined,
): number {
const base = policy.agentDiskTemperature ?? FACTORY_AGENT_DEFAULTS.diskTemperature;
if (base <= 0) return 0;
const normalizedType = (diskType ?? '').trim().toLowerCase();
const byType = normalizedType ? policy.diskTempByType[normalizedType] : undefined;
return typeof byType === 'number' && byType > 0 ? byType : base;
}
/** The trigger a TrueNAS disk without its own override inherits. */
export function resolveTrueNASDiskTemperatureDefault(
trueNASDiskTemperature: number | undefined,
policy: DiskTemperaturePolicyState,
diskType: string | null | undefined,
): number {
if (trueNASDiskTemperature !== undefined) return trueNASDiskTemperature;
return resolveDiskTemperatureTriggerForType(policy, diskType);
}
/** "NVMe 70°C", one per disk type, or none when the policy is off. */
export function getDiskTemperatureByTypeItems(policy: DiskTemperaturePolicyState): string[] {
return DISK_TEMPERATURE_TYPE_FIELDS.flatMap((field) => {
const trigger = resolveDiskTemperatureTriggerForType(policy, field.key);
return trigger > 0 ? [`${field.label} ${formatMetricValue('temperature', trigger)}`] : [];
});
}
+3
View File
@@ -147,6 +147,9 @@ export interface AlertConfig {
kubernetesDefaults?: AlertThresholds;
truenasDefaults?: AlertThresholds;
truenasDiskDefaults?: AlertThresholds;
// Set on every config written since an unset truenasDiskDefaults
// temperature means each TrueNAS disk follows Disk temperature by type.
truenasDiskTemperatureByType?: boolean;
vmwareDefaults?: AlertThresholds;
snapshotDefaults?: SnapshotAlertConfig;
backupDefaults?: BackupAlertConfig;
@@ -96,7 +96,8 @@ describe('alertThresholdDefaults', () => {
describe('FACTORY_TRUENAS_DISK_DEFAULTS', () => {
it('only carries a disk temperature trigger', () => {
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: 55 });
// Unset: each TrueNAS disk follows Disk temperature by type.
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: undefined });
expect(Object.keys(FACTORY_TRUENAS_DISK_DEFAULTS)).toEqual(['temperature']);
});
});
@@ -507,7 +507,7 @@ describe('metricThresholds', () => {
usage: 85,
temperature: 80,
});
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: 55 });
expect(FACTORY_TRUENAS_DISK_DEFAULTS).toEqual({ temperature: undefined });
expect(FACTORY_VMWARE_DEFAULTS).toEqual({
cpu: 80,
memory: 85,
@@ -44,8 +44,12 @@ export const FACTORY_TRUENAS_DEFAULTS = {
networkOut: -1,
};
export const FACTORY_TRUENAS_DISK_DEFAULTS = {
temperature: 55,
// TrueNAS disk temperature has no flat factory value. Unset, each TrueNAS disk
// follows Disk temperature by type, as the backend resolves it
// (trueNASDiskTemperatureDefaultNoLock); a saved value applies to every
// TrueNAS disk.
export const FACTORY_TRUENAS_DISK_DEFAULTS: Record<string, number | undefined> = {
temperature: undefined,
};
export const FACTORY_VMWARE_DEFAULTS = {
+9
View File
@@ -35,6 +35,12 @@ type alertPolicyQuery struct {
// StorageAliases are the storage resource's alias IDs, honored by the
// storage override lookup.
StorageAliases []string
// DiskType is a physical disk's type ("nvme", "sas", "sata"), which
// picks its disk temperature threshold. DiskTypeUnknown marks a disk
// whose type was never recorded, such as an alert persisted before it
// carried one; it is judged against the lowest per-type trigger.
DiskType string
DiskTypeUnknown bool
}
// EffectiveAlertPolicy is the resolved policy for one resource.
@@ -116,6 +122,9 @@ func (m *Manager) effectiveAlertPolicyNoLock(q alertPolicyQuery) EffectiveAlertP
policy.AllDisabled, policy.OfflineDisabled = m.alertPolicyTypeSwitchesNoLock(q.TypeKey)
thresholds := m.defaultThresholdsForResourceType(q.TypeKey)
if q.TypeKey == "truenas-disk" {
thresholds.Temperature = m.trueNASDiskTemperatureDefaultNoLock(q.DiskType, q.DiskTypeUnknown)
}
switch {
case isGuestThresholdResourceType(q.TypeKey):
thresholds = m.customRuleThresholdsNoLock(thresholds, q.Guest)
+59
View File
@@ -12628,6 +12628,65 @@ func TestCleanupAlertsForNodes(t *testing.T) {
})
}
// A config save re-judges open TrueNAS disk temperature alerts against the
// threshold the next evaluation applies to each disk's type.
func TestTrueNASDiskTemperatureAlertsReevaluatePerDiskType(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
configureUnifiedEvalManager(t, m, cfg)
nvme := trueNASTemperatureDisk("nvme0n1", "nvme", 72)
sata := trueNASTemperatureDisk("sda", "sata", 57)
checkTrueNASTemperatureDisk(t, m, nvme)
checkTrueNASTemperatureDisk(t, m, sata)
for _, disk := range []unifiedresources.Resource{nvme, sata} {
if _, firing := trueNASDiskTemperatureAlert(t, m, disk); !firing {
t.Fatalf("%s did not fire before the save: %v", disk.ID, alertKeys(m))
}
}
// Raising only the NVMe trigger resolves the NVMe alert and leaves the
// SATA alert, still over its own trigger, open.
m.mu.Lock()
m.config.DiskTempByType["nvme"] = HysteresisThreshold{Trigger: 75, Clear: 70}
m.reevaluateActiveAlertsLocked()
m.mu.Unlock()
if _, firing := trueNASDiskTemperatureAlert(t, m, nvme); firing {
t.Fatalf("NVMe alert at 72C stayed open under a 75C NVMe trigger")
}
if _, firing := trueNASDiskTemperatureAlert(t, m, sata); !firing {
t.Fatalf("SATA alert at 57C resolved under its unchanged 55C trigger")
}
}
// An alert raised before it recorded its disk type is held to the lowest
// per-type trigger, so a save never resolves an alert the next evaluation
// raises again.
func TestTrueNASDiskTemperatureAlertWithoutDiskTypeHeldAtLowestTrigger(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 70, Clear: 65}
cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 60, Clear: 55}}
configureUnifiedEvalManager(t, m, cfg)
nvme := trueNASTemperatureDisk("nvme0n1", "nvme", 66)
checkTrueNASTemperatureDisk(t, m, nvme)
alert, firing := trueNASDiskTemperatureAlert(t, m, nvme)
if !firing {
t.Fatalf("NVMe at 66C did not fire under a 60C NVMe trigger: %v", alertKeys(m))
}
m.mu.Lock()
delete(alert.Metadata, "diskType")
m.reevaluateActiveAlertsLocked()
m.mu.Unlock()
if _, firing := trueNASDiskTemperatureAlert(t, m, nvme); !firing {
t.Fatalf("an alert without a disk type was resolved by the 70C agent default although its NVMe trigger is 60C")
}
}
func TestCheckZFSPoolHealth(t *testing.T) {
// t.Parallel()
poolStateAlertID := buildCanonicalStateID("local-zfs/zfs-pool:rpool", "local-zfs/zfs-pool:rpool-state")
@@ -802,7 +802,6 @@ func TestBranchCovNormalizeTrueNASDefaults(t *testing.T) {
{"DiskWrite", t1.DiskWrite, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"NetworkIn", t1.NetworkIn, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"NetworkOut", t1.NetworkOut, alertconfig.HysteresisThreshold{Trigger: 0, Clear: 0}},
{"Disk.Temperature", cfg.TrueNASDiskDefaults.Temperature, alertconfig.HysteresisThreshold{Trigger: 55, Clear: 50}},
} {
if f.got == nil {
t.Fatalf("%s is nil, want non-nil", f.name)
@@ -811,6 +810,11 @@ func TestBranchCovNormalizeTrueNASDefaults(t *testing.T) {
t.Fatalf("%s = %+v, want %+v", f.name, *f.got, f.want)
}
}
// TrueNAS disks follow the disk temperature policy until the user
// saves a TrueNAS-wide value.
if cfg.TrueNASDiskDefaults.Temperature != nil {
t.Fatalf("Disk.Temperature = %+v, want unset", *cfg.TrueNASDiskDefaults.Temperature)
}
})
t.Run("derivation and operator-enabled IO metric via CPU/DiskRead", func(t *testing.T) {
+27 -1
View File
@@ -442,9 +442,35 @@ func NormalizeTrueNASDefaults(config *AlertConfig) {
config.TrueNASDefaults.NetworkIn = normalizeThresholdPointer(config.TrueNASDefaults.NetworkIn, 0, 0, "truenas.networkIn")
config.TrueNASDefaults.NetworkOut = normalizeThresholdPointer(config.TrueNASDefaults.NetworkOut, 0, 0, "truenas.networkOut")
config.TrueNASDiskDefaults.Temperature = normalizeThresholdPointer(config.TrueNASDiskDefaults.Temperature, 55, 50, "truenas.disk.temperature")
config.TrueNASDiskDefaults.Temperature = normalizeTrueNASDiskTemperature(config.TrueNASDiskDefaults.Temperature, config.TrueNASDiskTemperatureByType)
config.TrueNASDiskTemperatureByType = true
}
// normalizeTrueNASDiskTemperature keeps the TrueNAS Disks temperature default
// unset unless the user chose one. Unset, each TrueNAS disk follows the disk
// temperature policy for its type. Before that, the factory default was a
// flat 55/50 and the thresholds page wrote it back on every save, so in a
// config written before (byType false) a stored 55/50 marks a value nobody
// chose and is migrated to unset. Any other value is the user's TrueNAS-wide
// choice and is kept, 0 meaning off. Once byType is set, 55 is a choice too.
func normalizeTrueNASDiskTemperature(current *HysteresisThreshold, byType bool) *HysteresisThreshold {
if current == nil || current.Trigger < 0 {
return nil
}
if !byType && current.Trigger == legacyTrueNASDiskTemperatureTrigger &&
(current.Clear == legacyTrueNASDiskTemperatureClear || current.Clear <= 0) {
return nil
}
return normalizeThresholdPointer(current, 0, 0, "truenas.disk.temperature")
}
// The flat TrueNAS disk temperature factory default before TrueNAS disks
// followed the disk temperature policy.
const (
legacyTrueNASDiskTemperatureTrigger = 55
legacyTrueNASDiskTemperatureClear = 50
)
func NormalizeVMwareDefaults(config *AlertConfig) {
config.VMwareDefaults.CPU = normalizeThresholdPointer(config.VMwareDefaults.CPU, 80, 75, "vmware.cpu")
config.VMwareDefaults.Memory = normalizeThresholdPointer(config.VMwareDefaults.Memory, 85, 80, "vmware.memory")
@@ -0,0 +1,58 @@
package config
import "testing"
// TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues pins the saved
// config migration that lets TrueNAS disks follow the disk temperature policy.
// In a config written before it, the old flat factory default was written
// back on every save, so it is not a choice; anything else the user saved is
// kept.
func TestNormalizeTrueNASDiskTemperatureKeepsOnlyChosenValues(t *testing.T) {
tests := []struct {
name string
stored *HysteresisThreshold
want *HysteresisThreshold
}{
{name: "unset stays unset", stored: nil, want: nil},
{name: "old factory default is migrated to unset", stored: &HysteresisThreshold{Trigger: 55, Clear: 50}, want: nil},
{name: "old factory trigger without a clear is migrated to unset", stored: &HysteresisThreshold{Trigger: 55}, want: nil},
{name: "negative trigger is unset", stored: &HysteresisThreshold{Trigger: -1}, want: nil},
{name: "off is kept", stored: &HysteresisThreshold{Trigger: 0, Clear: 10}, want: &HysteresisThreshold{Trigger: 0, Clear: 0}},
{name: "custom value is kept", stored: &HysteresisThreshold{Trigger: 62, Clear: 58}, want: &HysteresisThreshold{Trigger: 62, Clear: 58}},
{name: "custom value gets a clear", stored: &HysteresisThreshold{Trigger: 62}, want: &HysteresisThreshold{Trigger: 62, Clear: 57}},
{name: "55 with a custom clear is a choice", stored: &HysteresisThreshold{Trigger: 55, Clear: 45}, want: &HysteresisThreshold{Trigger: 55, Clear: 45}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cfg := &AlertConfig{TrueNASDiskDefaults: ThresholdConfig{Temperature: tt.stored}}
NormalizeTrueNASDefaults(cfg)
if !cfg.TrueNASDiskTemperatureByType {
t.Fatalf("normalized config is not marked as following disk type")
}
got := cfg.TrueNASDiskDefaults.Temperature
switch {
case tt.want == nil && got != nil:
t.Fatalf("Temperature = %+v, want unset", *got)
case tt.want != nil && got == nil:
t.Fatalf("Temperature unset, want %+v", *tt.want)
case tt.want != nil && *got != *tt.want:
t.Fatalf("Temperature = %+v, want %+v", *got, *tt.want)
}
})
}
}
// Once a config follows disk type, 55 is a value the user typed, and a save
// or restart must keep it.
func TestNormalizeTrueNASDiskTemperatureKeepsChosen55(t *testing.T) {
cfg := &AlertConfig{
TrueNASDiskDefaults: ThresholdConfig{Temperature: &HysteresisThreshold{Trigger: 55, Clear: 50}},
TrueNASDiskTemperatureByType: true,
}
NormalizeTrueNASDefaults(cfg)
NormalizeTrueNASDefaults(cfg)
got := cfg.TrueNASDiskDefaults.Temperature
if got == nil || *got != (HysteresisThreshold{Trigger: 55, Clear: 50}) {
t.Fatalf("Temperature = %+v, want the chosen 55/50 kept", got)
}
}
+7
View File
@@ -388,6 +388,13 @@ type AlertConfig struct {
FlappingWindowSeconds int `json:"flappingWindowSeconds"` // Time window for counting state changes
FlappingThreshold int `json:"flappingThreshold"` // Number of state changes to trigger flapping
FlappingCooldownMinutes int `json:"flappingCooldownMinutes"` // Cooldown period after flapping detected
// TrueNASDiskTemperatureByType marks a config written since an unset
// TrueNASDiskDefaults.Temperature means each TrueNAS disk follows the
// disk temperature policy for its type. An older config carries the
// former flat factory value, which nobody chose, and normalization drops
// it once.
TrueNASDiskTemperatureByType bool `json:"truenasDiskTemperatureByType,omitempty"`
}
// UnmarshalJSON accepts canonical v6 alert config keys.
+11 -1
View File
@@ -409,7 +409,17 @@ func (m *Manager) reevaluateActiveAlertsLocked() {
alertsToResolve = append(alertsToResolve, alertID)
continue
}
thresholds := m.resolveResourceThresholds(primaryResourceType, resourceID)
query := alertPolicyQuery{TypeKey: primaryResourceType, ResourceID: resourceID}
if primaryResourceType == "truenas-disk" {
// Judge the alert against its disk type's threshold, as the
// next evaluation will. An alert raised before it recorded a
// disk type is held to the lowest per-type trigger, so a save
// never resolves an alert its type would raise again.
diskType, known := alert.Metadata["diskType"].(string)
query.DiskType = diskType
query.DiskTypeUnknown = !known
}
thresholds := m.effectiveAlertPolicyNoLock(query).Thresholds
if thresholds.Disabled {
alertsToResolve = append(alertsToResolve, alertID)
continue
+5 -3
View File
@@ -106,9 +106,11 @@ func defaultAlertConfig() AlertConfig {
NetworkIn: &HysteresisThreshold{Trigger: 0, Clear: 0},
NetworkOut: &HysteresisThreshold{Trigger: 0, Clear: 0},
},
TrueNASDiskDefaults: ThresholdConfig{
Temperature: &HysteresisThreshold{Trigger: 55, Clear: 50},
},
// TrueNAS disk temperature is unset: each disk follows the disk
// temperature policy for its type (DiskTempByType) until the user
// saves a TrueNAS-wide value.
TrueNASDiskDefaults: ThresholdConfig{},
TrueNASDiskTemperatureByType: true,
VMwareDefaults: ThresholdConfig{
CPU: &HysteresisThreshold{Trigger: 80, Clear: 75},
Memory: &HysteresisThreshold{Trigger: 85, Clear: 80},
@@ -0,0 +1,18 @@
package alerts
// trueNASDiskTemperatureDefaultNoLock returns the default tier of a TrueNAS
// disk's temperature threshold. A TrueNAS-wide value the user saved under
// TrueNAS Disks applies to every TrueNAS disk, as a host Disk Temp override
// does for an agent's disks. Without one the disk follows the disk
// temperature policy for its type, the threshold DiskTemperatureThreshold
// returns, so its alert agrees with the TrueNAS storage table and Patrol. A
// per-disk override still beats both. Callers must hold m.mu.
func (m *Manager) trueNASDiskTemperatureDefaultNoLock(diskType string, typeUnknown bool) *HysteresisThreshold {
if explicit := m.config.TrueNASDiskDefaults.Temperature; explicit != nil {
return cloneThreshold(explicit)
}
if typeUnknown {
return cloneThreshold(m.lowestHostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature))
}
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
}
+16 -2
View File
@@ -6,6 +6,7 @@ import (
alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
"github.com/rs/zerolog/log"
)
@@ -57,6 +58,9 @@ type UnifiedResourceInput struct {
Temperature *UnifiedResourceMetric
StorageAliases []string // Durable policy lookup identities for storage alerts.
// DiskType is a physical disk's type ("nvme", "sas", "sata"). It picks
// the disk temperature threshold of a TrueNAS disk.
DiskType string
}
type unifiedMetricCandidate struct {
@@ -297,6 +301,11 @@ func (m *Manager) evaluateUnifiedMetrics(input *UnifiedResourceInput, thresholds
if platformType != "" {
metadata[alertPlatformTypeKey] = platformType
}
if input.Type == "truenas-disk" {
// A config save re-judges the open temperature alert against
// the threshold for this disk type.
metadata["diskType"] = input.DiskType
}
merged.Metadata = metadata
opts = &merged
}
@@ -551,10 +560,15 @@ func UnifiedResourceInputFromResource(resource unifiedresources.Resource) (*Unif
}
if resource.Temperature != nil {
input.Temperature = &UnifiedResourceMetric{Value: *resource.Temperature, Percent: *resource.Temperature}
} else if resource.PhysicalDisk != nil && resource.PhysicalDisk.Temperature > 0 {
} else if resource.PhysicalDisk != nil &&
diskinventory.TemperatureCollected(resource.PhysicalDisk.Temperature, resource.PhysicalDisk.Collection) {
// A retained last-known reading is not current evidence of heat.
value := float64(resource.PhysicalDisk.Temperature)
input.Temperature = &UnifiedResourceMetric{Value: value, Percent: value}
}
if resource.PhysicalDisk != nil {
input.DiskType = strings.ToLower(strings.TrimSpace(resource.PhysicalDisk.DiskType))
}
return input, true
}
@@ -798,7 +812,7 @@ func (m *Manager) CheckUnifiedResourceWithCapacityTrend(input *UnifiedResourceIn
return
}
thresholds := m.resolveResourceThresholds(input.Type, input.ID)
thresholds := m.effectiveAlertPolicyNoLock(alertPolicyQuery{TypeKey: input.Type, ResourceID: input.ID, DiskType: input.DiskType}).Thresholds
m.mu.RUnlock()
if thresholds.Disabled {
+165
View File
@@ -10,6 +10,7 @@ import (
alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
)
func unifiedEvalBaseConfig() AlertConfig {
@@ -1778,3 +1779,167 @@ func TestBuildMetricAlertStatusCarriesWindowAndMonotonicRecovery(t *testing.T) {
t.Fatalf("status without clear level = %+v", noClear)
}
}
func trueNASTemperatureDisk(device, diskType string, celsius int) unifiedresources.Resource {
return unifiedresources.Resource{
ID: "physical-disk:truenas-main/" + device,
Type: unifiedresources.ResourceTypePhysicalDisk,
Name: device,
ParentName: "truenas-main",
Sources: []unifiedresources.DataSource{unifiedresources.SourceTrueNAS},
PhysicalDisk: &unifiedresources.PhysicalDiskMeta{
DevPath: "/dev/" + device,
DiskType: diskType,
Temperature: celsius,
},
}
}
func trueNASDiskTemperatureAlertID(resource unifiedresources.Resource) string {
return canonicalMetricStateID(resource.ID, "temperature")
}
func checkTrueNASTemperatureDisk(t *testing.T, m *Manager, resource unifiedresources.Resource) {
t.Helper()
input, ok := UnifiedResourceInputFromResource(resource)
if !ok {
t.Fatalf("TrueNAS disk %s did not become alert input", resource.ID)
}
m.CheckUnifiedResource(input)
}
func trueNASDiskTemperatureAlert(t *testing.T, m *Manager, resource unifiedresources.Resource) (*Alert, bool) {
t.Helper()
return testLookupActiveAlert(t, m, trueNASDiskTemperatureAlertID(resource))
}
// TestTrueNASDiskTemperatureAlertsFollowDiskTemperaturePolicy pins TrueNAS
// disk temperature alerts to the disk temperature policy that the TrueNAS
// storage table, Physical Disks and Patrol judge heat by: per disk type,
// unless the user saved a TrueNAS-wide value, with a per-disk override first.
func TestTrueNASDiskTemperatureAlertsFollowDiskTemperaturePolicy(t *testing.T) {
nvme63 := trueNASTemperatureDisk("nvme0n1", "nvme", 63)
nvme72 := trueNASTemperatureDisk("nvme1n1", "nvme", 72)
sata56 := trueNASTemperatureDisk("sda", "sata", 56)
sata60 := trueNASTemperatureDisk("sdb", "sata", 60)
tests := []struct {
name string
mutate func(*AlertConfig)
disks []unifiedresources.Resource
firing []unifiedresources.Resource
trigger map[string]float64
}{
{
name: "factory thresholds judge each disk by its type",
disks: []unifiedresources.Resource{nvme63, nvme72, sata56},
firing: []unifiedresources.Resource{nvme72, sata56},
trigger: map[string]float64{nvme72.ID: 70, sata56.ID: 55},
},
{
name: "a raised NVMe trigger moves NVMe alerts with it",
mutate: func(cfg *AlertConfig) {
cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 75, Clear: 70}}
},
disks: []unifiedresources.Resource{nvme72, sata56},
firing: []unifiedresources.Resource{sata56},
},
{
name: "a saved TrueNAS-wide value applies to every TrueNAS disk",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
},
disks: []unifiedresources.Resource{nvme63, sata60},
firing: []unifiedresources.Resource{nvme63},
trigger: map[string]float64{nvme63.ID: 62},
},
{
name: "a per-disk override beats the TrueNAS-wide value",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
cfg.Overrides = map[string]ThresholdConfig{
sata56.ID: {Temperature: &HysteresisThreshold{Trigger: 50, Clear: 45}},
}
},
disks: []unifiedresources.Resource{sata56},
firing: []unifiedresources.Resource{sata56},
trigger: map[string]float64{sata56.ID: 50},
},
{
name: "a switched-off agent Disk Temp default switches TrueNAS disks off",
mutate: func(cfg *AlertConfig) {
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 0, Clear: 0}
},
disks: []unifiedresources.Resource{nvme72, sata60},
},
{
name: "a saved TrueNAS-wide value still applies when the agent default is off",
mutate: func(cfg *AlertConfig) {
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 0, Clear: 0}
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 58, Clear: 53}
},
disks: []unifiedresources.Resource{sata60},
firing: []unifiedresources.Resource{sata60},
},
{
name: "a saved TrueNAS-wide off switches TrueNAS disks off",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 0, Clear: 0}
},
disks: []unifiedresources.Resource{nvme72, sata60},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
if tt.mutate != nil {
tt.mutate(&cfg)
}
configureUnifiedEvalManager(t, m, cfg)
for _, disk := range tt.disks {
checkTrueNASTemperatureDisk(t, m, disk)
}
want := make(map[string]bool, len(tt.firing))
for _, disk := range tt.firing {
want[disk.ID] = true
}
for _, disk := range tt.disks {
alert, firing := trueNASDiskTemperatureAlert(t, m, disk)
if firing != want[disk.ID] {
t.Fatalf("%s firing = %v, want %v (active: %v)", disk.ID, firing, want[disk.ID], alertKeys(m))
}
if !firing {
continue
}
if got := alert.Metadata["diskType"]; got != disk.PhysicalDisk.DiskType {
t.Fatalf("%s diskType metadata = %v, want %q", disk.ID, got, disk.PhysicalDisk.DiskType)
}
if trigger, ok := tt.trigger[disk.ID]; ok && alert.Threshold != trigger {
t.Fatalf("%s threshold = %v, want %v", disk.ID, alert.Threshold, trigger)
}
}
})
}
}
// A last-known reading the source no longer collects is not evidence of heat.
func TestTrueNASDiskTemperatureAlertIgnoresRetainedReading(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
configureUnifiedEvalManager(t, m, cfg)
disk := trueNASTemperatureDisk("sda", "sata", 80)
disk.PhysicalDisk.Collection = &diskinventory.CollectionStatus{
Temperature: diskinventory.Unavailable("truenas", "disk is in standby"),
}
checkTrueNASTemperatureDisk(t, m, disk)
if _, firing := trueNASDiskTemperatureAlert(t, m, disk); firing {
t.Fatalf("retained 80C reading raised an alert: %v", alertKeys(m))
}
}