mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Judge disk heat under the reporting host's Disk Temp override
Disk temperature alerts evaluate each SMART disk under the thresholds of the host agent that reports it. An explicit Disk Temp override on that host, or one inherited from its linked node or guest, replaces the per-type trigger, and a host whose alerts are switched off raises none. Since #2636 the Physical Disks verdict and Patrol judged heat by disk type alone. A host with an 80C override still read Running Hot at the 70C NVMe trigger, and a host with its alerts off still read hot. - alerts.Manager.HostDiskTemperatureThreshold resolves the policy for a host exactly as CheckHost does, from the same helpers. DiskTemperatureThreshold is its hostless case. - Patrol and the AI chat context find each disk's host by walking its parent to the agent machine (through the pool for Unraid disks) and ask the threshold adapter for that host's limits. - Physical Disks, the disk detail, the pool drawer's linked disks and the guest drawer pass the reporting machine's override keys to the thresholds resolver, in the order CheckHost reads them: agent ID, canonical ID, then the merged node or guest. The resolver treats an override or agent default that switches alerts off as no heat. Two cases still fall back to the per-type trigger in the browser while alerts and Patrol apply the override: on the Storage page, a disk whose agent was manually linked into a VM, and an override an agent inherits from a guest it is linked to without a merge.
This commit is contained in:
rcourtman
@@ -307,10 +307,17 @@ retained value as "last known 41C (disk is in standby)". Proofs:
|
||||
`internal/ai/patrol_ai_more_test.go`.
|
||||
|
||||
They judge that collected reading against the alert disk temperature policy,
|
||||
not a fixed line. `ThresholdProvider.GetDiskTemperatureThreshold(diskType)`
|
||||
(the alert manager's per-type trigger and clear) resolves each disk's limits
|
||||
once in `patrolPhysicalDiskRows`, and the run state carries the provider
|
||||
(`patrolRuntimeState.thresholdProvider`). With no provider the factory alert
|
||||
not a fixed line. `ThresholdProvider.GetDiskTemperatureThreshold(host, diskType)`
|
||||
(the alert manager's `HostDiskTemperatureThreshold` trigger and clear) resolves
|
||||
each disk's limits once in `patrolPhysicalDiskRows` and in the AI chat context,
|
||||
for the host agent that reports the disk. `physicalDiskTemperatureHost` finds it
|
||||
by walking the disk's parent to the agent machine, through the pool for an
|
||||
Unraid disk, so that agent's Disk Temp override (or one inherited from its
|
||||
linked node or guest) replaces the per-type trigger, and an override or agent
|
||||
default that switches the host's alerts off leaves no heat to judge. A disk no
|
||||
agent reports gets the per-type policy. The global alerts switch and the agent
|
||||
alert-type switch stop `CheckHost` without changing this policy, as before.
|
||||
The run state carries the provider (`patrolRuntimeState.thresholdProvider`). With no provider the factory alert
|
||||
configuration applies. A disk is hot from its trigger: a triage warning, a disk
|
||||
issue and an AI chat "needing attention" entry, as its temperature alert and
|
||||
Physical Disks Running Hot verdict are. A disk-high finding recovers at or
|
||||
@@ -319,8 +326,10 @@ the alert does. With no current reading, a disk last seen hot leaves that
|
||||
verification unknown rather than recovered. Scoped runs keep the provider, and a
|
||||
provider set before Patrol starts is handed to it. Below the trigger Patrol flags nothing, so an NVMe at 63C stays
|
||||
quiet and a SATA disk at 56C is flagged. Proofs:
|
||||
`internal/ai/patrol_disk_temperature_test.go` and
|
||||
`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` in
|
||||
`internal/ai/patrol_disk_temperature_test.go` (a real registry with one
|
||||
overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`),
|
||||
`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` and
|
||||
`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` in
|
||||
`internal/ai/alert_threshold_adapter_test.go`.
|
||||
|
||||
Retained summaries disclose that point and bucket timestamps describe returned
|
||||
|
||||
@@ -1773,14 +1773,24 @@ an alert the next evaluation raises again.
|
||||
The policy `CheckHost` applies to a SMART disk (the agent Disk Temp default,
|
||||
refined by the disk type's `DiskTempByType` entry, and switched off for every
|
||||
type when that default is off) is the canonical answer to "is this disk too
|
||||
hot". `diskTemperatureThresholdForType` in `internal/alerts/host.go` holds it,
|
||||
and `Manager.DiskTemperatureThreshold(diskType)` exposes it, without host
|
||||
overrides, to Patrol through `AlertThresholdAdapter.GetDiskTemperatureThreshold`.
|
||||
hot". `diskTemperatureThresholdForType` in `internal/alerts/host.go` holds it.
|
||||
`Manager.HostDiskTemperatureThreshold(host, diskType)` resolves it for the host
|
||||
agent that reports a disk exactly as `CheckHost` does: an explicit Disk Temp
|
||||
override on that host, or one inherited from its linked node or guest, replaces
|
||||
the per-type entry, the first override in that chain decides even when it sets
|
||||
no Disk Temp, and an override that switches the host's alerts off leaves no
|
||||
threshold. `TestHostDiskTemperatureThresholdMatchesCheckHostOverrides` pins each
|
||||
case against what `CheckHost` fires. `Manager.DiskTemperatureThreshold(diskType)`
|
||||
is the hostless case. Patrol reads the policy through
|
||||
`AlertThresholdAdapter.GetDiskTemperatureThreshold`.
|
||||
`DefaultDiskTemperatureThreshold` serves callers with no manager. Disk risk
|
||||
(`internal/storagehealth`) judges no temperature. The Physical Disks Temp cell
|
||||
and Health verdict mirror the per-type triggers and a switched-off default in
|
||||
`resolveDiskTemperatureDisplayThresholds`; host overrides do not reach that
|
||||
page yet. That resolver returns null when the agent Disk Temp default is off.
|
||||
and Health verdict mirror the policy in `resolveDiskTemperatureDisplayThresholds`
|
||||
under the override keys of the machine whose agent reports the disk, in the
|
||||
order `CheckHost` reads them. That resolver returns null when the agent Disk
|
||||
Temp default is off or an override or agent default switches the host's alerts
|
||||
off. The global alerts switch and the agent alert-type switch stop `CheckHost`
|
||||
without changing the policy.
|
||||
The displays that read it treat null as off: no threshold colour
|
||||
(`getTemperatureTextClass` with `diskTemperature`) and never hot. A
|
||||
`DiskTempByType` entry cannot be switched off on its own, because
|
||||
|
||||
@@ -429,7 +429,9 @@ The physical-disk verdict words and their phone forms are `Needs Attention`
|
||||
(`Attention`), `Running Hot` (`Hot`) and `Replace Now` (`Replace`), all from
|
||||
`getPhysicalDiskHealthCompactLabel` in `diskPresentation.ts`. `Running Hot` is
|
||||
red and starts at the disk's alert trigger, from the same thresholds object
|
||||
that colours its Temp cell (`getPhysicalDiskTemperatureThresholds`).
|
||||
that colours its Temp cell (`getPhysicalDiskTemperatureThresholds`). Callers
|
||||
pass the override keys of the machine that reports the disk, so that machine's
|
||||
Disk Temp override sets those thresholds.
|
||||
|
||||
Controller expansions carry the supplied kind-specific targets, absolute times,
|
||||
duration and cluster/namespace when the phone row omits those columns. Proxmox
|
||||
|
||||
@@ -3458,6 +3458,10 @@ whole-estate physical-disk work to Workloads. The guest's existing agent RAID
|
||||
snapshot remains local to the opened drawer; an empty or failed child query
|
||||
must not imply that host disks belong to the guest. Focused hook and drawer
|
||||
tests pin the query and no-data/error presentations.
|
||||
The drawer judges those disks' heat under the guest agent's alert overrides:
|
||||
its `agentId`, which `useWorkloads` copies from `resource.agent.agentId` onto
|
||||
the workload row, then the guest's own override keys. That reads the alert
|
||||
configuration already loaded and adds no query, poll or per-row work.
|
||||
|
||||
### Configuration transfer authorization stays off persistence hot paths
|
||||
|
||||
|
||||
@@ -6781,13 +6781,30 @@ Alerts > Thresholds, `diskTempByType`, else the agent Disk Temp default).
|
||||
Disk risk never carries heat, because every registry that rebuilds it lacks the
|
||||
alert configuration. `extractPhysicalDiskPresentationData` takes the alerts
|
||||
store's `getDiskTemperatureThresholds` resolver, so the Temp cell colour and the
|
||||
verdict read one thresholds object. The band below the trigger is a Temp cell
|
||||
verdict read one thresholds object. It resolves them under the alert override
|
||||
keys of the machine whose agent reports the disk, as disk temperature alerts
|
||||
are evaluated under that host. `getPhysicalDiskAlertResourceIds`
|
||||
(`components/Storage/physicalDiskAlertOwner.ts`) follows the disk's parent to
|
||||
the agent machine, through the pool for an Unraid array or cache disk, and
|
||||
reads that machine's keys in the order `CheckHost` reads them
|
||||
(`getAgentMachineAlertOverrideKeys`: the agent ID, the machine's canonical ID,
|
||||
then the node or guest the agent is merged with).
|
||||
A Disk Temp override set on the machine therefore replaces the per-type trigger
|
||||
for every disk it reports, on Physical Disks, the disk detail and the pool
|
||||
drawer's linked disks. A guest drawer passes the guest agent's ID, then the
|
||||
guest's own keys. An override that switches the machine's alerts off leaves its
|
||||
disks uncoloured and never hot. The band below the trigger is a Temp cell
|
||||
colour only, and a retained (not current) reading is never heat. The verdict
|
||||
judges the current reading, with no memory of an alert. A disk temperature
|
||||
alert held between its clear value and trigger shows an amber Temp cell, not
|
||||
`Running Hot`. A switched-off agent Disk Temp default resolves to null, which
|
||||
leaves the reading uncoloured and never hot, as alerting is off for it.
|
||||
Per-host Disk Temp overrides do not yet reach Physical Disks. PDF performance
|
||||
Two cases stay on the per-type trigger in the browser while alerts and Patrol
|
||||
apply the override: on the Storage page, a disk whose agent an operator
|
||||
manually linked into a VM (the page loads agents and pools, not guests; the
|
||||
guest drawer applies it), and an override an agent inherits from a guest it is
|
||||
linked to without a merge, because agent link hints are not published to the
|
||||
browser. PDF performance
|
||||
reports colour a disk reading by the same per-type thresholds, which
|
||||
`reporting.DiskInfo` carries.
|
||||
`frontend-modern/src/features/storageBackups/__tests__/diskPresentation.test.ts`
|
||||
@@ -6795,7 +6812,10 @@ pins cell-and-verdict agreement for every type from 40C to 80C under factory and
|
||||
raised triggers, plus retained readings, failure evidence, sorting and
|
||||
filtering. `frontend-modern/src/components/Storage/__tests__/DiskList.test.tsx`
|
||||
covers NVMe 63C, SATA 56C and a raised NVMe trigger loaded through the alerts
|
||||
store.
|
||||
store, and NVMe disks at 72C on three machines judged by an 80C override, the
|
||||
per-type trigger and switched-off alerts.
|
||||
`frontend-modern/src/components/Storage/__tests__/physicalDiskAlertOwner.test.ts`
|
||||
pins the owner walk and the key order.
|
||||
|
||||
### Pool-to-physical-disk ownership in Storage details
|
||||
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
{
|
||||
"version": 1,
|
||||
"base_sha": "38ce73c24a7fdf9f3523fbf7f9f9a23ac2be7276",
|
||||
"verified_at": "2026-10-07T10:53:35Z",
|
||||
"result": "passed",
|
||||
"changed_paths": [
|
||||
"frontend-modern/src/components/Storage/DiskDetail.tsx",
|
||||
"frontend-modern/src/components/Storage/DiskList.tsx",
|
||||
"frontend-modern/src/components/Storage/Storage.tsx",
|
||||
"frontend-modern/src/components/Storage/StorageContentCard.tsx",
|
||||
"frontend-modern/src/components/Storage/StoragePoolDetail.tsx",
|
||||
"frontend-modern/src/components/Storage/StoragePoolRow.tsx",
|
||||
"frontend-modern/src/components/Storage/StoragePoolsTable.tsx",
|
||||
"frontend-modern/src/components/Storage/physicalDiskAlertOwner.ts",
|
||||
"frontend-modern/src/components/Storage/useDiskDetailModel.ts",
|
||||
"frontend-modern/src/components/Storage/useDiskListModel.ts",
|
||||
"frontend-modern/src/components/Storage/useStoragePageModel.ts",
|
||||
"frontend-modern/src/components/Storage/useStoragePageResources.ts",
|
||||
"frontend-modern/src/components/Storage/useStoragePoolDetailModel.ts",
|
||||
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx",
|
||||
"frontend-modern/src/components/Workloads/GuestPhysicalDisks.tsx",
|
||||
"frontend-modern/src/features/storageBackups/diskPresentation.ts",
|
||||
"frontend-modern/src/features/storageBackups/storagePoolDetailPresentation.ts",
|
||||
"frontend-modern/src/hooks/useWorkloads.ts",
|
||||
"frontend-modern/src/types/workloads.ts",
|
||||
"frontend-modern/src/utils/metricThresholds.ts"
|
||||
],
|
||||
"content_sha256": {
|
||||
"frontend-modern/src/components/Storage/DiskDetail.tsx": "f6617192db42acb3da59297ef86e03e2d01dafc16cb5744ad87f84ffd37f6773",
|
||||
"frontend-modern/src/components/Storage/DiskList.tsx": "0a457320ac9aea99618f9c1be12cd85a59d0536341436b225eeaeadec09896f3",
|
||||
"frontend-modern/src/components/Storage/Storage.tsx": "c1ecf5d710c427f0acc8230728e15648376628b43131f8504f12e0ff841dccff",
|
||||
"frontend-modern/src/components/Storage/StorageContentCard.tsx": "faa6896330d7ea74b6886fc3d449fe8bab3b6275f00bf15e7c9d8bb24a1b2d8e",
|
||||
"frontend-modern/src/components/Storage/StoragePoolDetail.tsx": "0a6401690187f4b3fa886ece2862edb34cf99122e175a1c4806384cdc8e4fed4",
|
||||
"frontend-modern/src/components/Storage/StoragePoolRow.tsx": "ce177b691b800852f35ec66267150e6259d3bf9cc65b3492bbe7055879dade7c",
|
||||
"frontend-modern/src/components/Storage/StoragePoolsTable.tsx": "ad1bc1967c8e4561a943ec12629f3c64ccb0423ebe6551d4a8ccd2c6afe72872",
|
||||
"frontend-modern/src/components/Storage/physicalDiskAlertOwner.ts": "279aa8cd48705a31e91579bdbcf4964495301bb7478a4fa0c4549959596dd9fa",
|
||||
"frontend-modern/src/components/Storage/useDiskDetailModel.ts": "b47926cc53cca725a2bca4300742b5ad211103140b809c86140648affa6d5b36",
|
||||
"frontend-modern/src/components/Storage/useDiskListModel.ts": "84aa1e3c298f614a0e2e7da012ea0cc4f7f8993d9fe686d67c7865cd84298de5",
|
||||
"frontend-modern/src/components/Storage/useStoragePageModel.ts": "d274908f0a7fa531921f226936bab93166676904dc8544717f7fa2172ddb37d3",
|
||||
"frontend-modern/src/components/Storage/useStoragePageResources.ts": "adf35ccbe0fcae8d7559177419be0e9086720b6e039632b9c1ea829f7f7bd6c1",
|
||||
"frontend-modern/src/components/Storage/useStoragePoolDetailModel.ts": "7654fbb97eaf9d1ed20b7ef41c73a0061f83f29cb989ded5e3e9b4b65c0e23ef",
|
||||
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx": "9a35f11b805b2c067afd9aeff08a2cc017caa345a72f1512a8d1bf0fc98b5615",
|
||||
"frontend-modern/src/components/Workloads/GuestPhysicalDisks.tsx": "74f6b660e4d91bffad9c1c8e7580a7085036bff68e15daaf7709b4c3f615d5e3",
|
||||
"frontend-modern/src/features/storageBackups/diskPresentation.ts": "ce492c2348a7e2a690b2613d0e8c0284b6de37dd5b0c0e63cd32c22bdfe150db",
|
||||
"frontend-modern/src/features/storageBackups/storagePoolDetailPresentation.ts": "75765a1f902eb717aacc995143d750f0fa5957e6217442995566d00592f9effa",
|
||||
"frontend-modern/src/hooks/useWorkloads.ts": "f25d948645c4de8984ff53242f0426c42ee24f2e5f916b4906c80faf50772be1",
|
||||
"frontend-modern/src/types/workloads.ts": "013a5de46caf8c2d968fd1459383f0fcf0fb09efbc924b46c9aa781683d6c5e9",
|
||||
"frontend-modern/src/utils/metricThresholds.ts": "cbe2299b0a2e3b917d7fa8c0c8ccabb2f7a0862708736636c74908f969e5c9ea"
|
||||
},
|
||||
"routes": [
|
||||
"/proxmox/storage?tab=disks&q=West%20Production"
|
||||
],
|
||||
"viewports": [
|
||||
{
|
||||
"width": 1440,
|
||||
"height": 900
|
||||
},
|
||||
{
|
||||
"width": 375,
|
||||
"height": 812
|
||||
}
|
||||
],
|
||||
"states": [
|
||||
"Private mock stack (8 nodes, each with a linked Pulse agent): Vite dev server on this exact staged frontend tree, backend built from the same tree with a local-only generator overlay (never committed) pinning West Production A/B/C NVMe disks at 72C and a pve3 SATA disk at 41C. Each disk's parent is its node's merged agent machine (agent IDs host-node-mock-cluster-1-pve1/2/3)",
|
||||
"Overrides set through GET/PUT /api/alerts/config: host-node-mock-cluster-1-pve1 diskTemperature 80/75, host-node-mock-cluster-1-pve3 disabled; pve2 none. diskTempByType nvme 70/65",
|
||||
"Same-session control with the changed runtime files swapped to origin/main under the running Vite (then restored, full diff cmp-identical): all three NVMe disks read 'Running Hot' with red 72C cells, including pve1 under its 80C override (detail: 'Disk temperature is 72\u00b0C, at or above its 70\u00b0C alert threshold.') and pve3 with alerts off",
|
||||
"After, 1440: pve1 NVMe 72C 'Healthy' with a green cell (rgb(22,163,74)); pve2 NVMe 72C 'Running Hot' with a red cell (rgb(220,38,38)); pve3 NVMe 72C uncoloured (text-base-content) and not hot (its mock SMART health is UNKNOWN, so the verdict reads 'Unknown'), and pve3's other disks lose their threshold colour too; no horizontal overflow (scrollWidth 1440)",
|
||||
"After, 375: same verdicts in compact form (pve1 'Healthy' green, pve2 'Hot' red, pve3 uncoloured); scrollWidth 375",
|
||||
"After, inline disk detail for pve1's NVMe: health 'Healthy' and Temperature 72\u00b0C, agreeing with its row",
|
||||
"After, override moved to pve2's linked Proxmox node ID (mock-cluster-1-pve2, diskTemperature 80/75): pve2's NVMe reads 'Healthy' with a green cell, as CheckHost inherits a linked-node override; config restored afterwards",
|
||||
"Not reachable on the mock estate and covered by unit tests instead: pool-drawer linked disks (storagePoolDetailPresentation/useStoragePoolDetailModel tests), the guest drawer for an agent merged into a VM (GuestPhysicalDisks.test.tsx), and the owner walk through an Unraid pool (physicalDiskAlertOwner.test.ts)"
|
||||
],
|
||||
"interactions": [
|
||||
"Loaded Physical Disks filtered to West Production at 1440x900 and 375x812 and read each pinned row's verdict, Temp cell class and computed colour, before (main frontend) and after",
|
||||
"Clicked pve1's NVMe row to open its inline disk detail and read its health and temperature",
|
||||
"Changed alert overrides through GET/PUT /api/alerts/config (host-keyed, then node-keyed) and reloaded Physical Disks after each",
|
||||
"Measured document scrollWidth at both widths"
|
||||
]
|
||||
}
|
||||
@@ -46,6 +46,8 @@ import { useDiskDetailModel } from './useDiskDetailModel';
|
||||
interface DiskDetailProps {
|
||||
disk: Resource;
|
||||
nodes: Resource[];
|
||||
/** Alert override keys of the machine whose agent reports the disk. */
|
||||
alertResourceIds?: string[];
|
||||
}
|
||||
|
||||
type DiskDetailTab = 'overview' | 'history';
|
||||
@@ -65,6 +67,7 @@ export const DiskDetail: Component<DiskDetailProps> = (props) => {
|
||||
liveIOAvailable,
|
||||
} = useDiskDetailModel({
|
||||
disk: () => props.disk,
|
||||
alertResourceIds: () => props.alertResourceIds,
|
||||
});
|
||||
const rangeOptions = createMemo(() =>
|
||||
getUnlockedHistoryRangeOptions(DISK_DETAIL_HISTORY_RANGE_OPTIONS, maxHistoryDays()),
|
||||
|
||||
@@ -86,6 +86,7 @@ import {
|
||||
getPhysicalDiskTableLayoutModeForContainer,
|
||||
getPhysicalDiskTemperaturePresentation,
|
||||
isPhysicalDiskColumnVisible,
|
||||
type PhysicalDiskAlertResourceIdResolver,
|
||||
type PhysicalDiskTableColumnId,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import { useObservedElementWidth } from '@/hooks/useObservedElementWidth';
|
||||
@@ -98,6 +99,8 @@ import { useDiskListModel } from './useDiskListModel';
|
||||
|
||||
interface DiskListProps {
|
||||
disks: Resource[];
|
||||
/** Alert override keys of the machine reporting each disk; absent means none. */
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
nodes: Resource[];
|
||||
selectedNode: string | null;
|
||||
sourceFilter?: string;
|
||||
@@ -134,6 +137,7 @@ export const DiskList: Component<DiskListProps> = (props) => {
|
||||
getPhysicalDiskColumnWidthStyle(layoutMode(), columnId);
|
||||
const model = useDiskListModel({
|
||||
disks: () => props.disks,
|
||||
getDiskAlertResourceIds: (disk) => props.getDiskAlertResourceIds?.(disk) ?? [],
|
||||
nodes: () => props.nodes,
|
||||
selectedNode: () => props.selectedNode,
|
||||
sourceFilter: () => props.sourceFilter ?? 'all',
|
||||
@@ -569,7 +573,11 @@ export const DiskList: Component<DiskListProps> = (props) => {
|
||||
colSpan={9}
|
||||
class={PHYSICAL_DISK_DETAIL_ROW_CELL_CLASS}
|
||||
>
|
||||
<DiskDetail disk={disk} nodes={props.nodes} />
|
||||
<DiskDetail
|
||||
disk={disk}
|
||||
nodes={props.nodes}
|
||||
alertResourceIds={props.getDiskAlertResourceIds?.(disk)}
|
||||
/>
|
||||
</TableCell>
|
||||
</TableRow>
|
||||
</Show>
|
||||
|
||||
@@ -57,6 +57,7 @@ const Storage: Component<StorageProps> = (props) => {
|
||||
focusedSummaryStorageGroupId,
|
||||
hoveredSummaryStorageGroupScope,
|
||||
physicalDisks,
|
||||
getDiskAlertResourceIds,
|
||||
nodes,
|
||||
groupedRecords,
|
||||
expandedGroups,
|
||||
@@ -139,6 +140,7 @@ const Storage: Component<StorageProps> = (props) => {
|
||||
<StorageContentCard
|
||||
view={view}
|
||||
physicalDisks={physicalDisks}
|
||||
getDiskAlertResourceIds={getDiskAlertResourceIds}
|
||||
nodes={nodes}
|
||||
sourceFilter={sourceFilter}
|
||||
healthFilter={healthFilter}
|
||||
|
||||
@@ -7,6 +7,7 @@ import { STORAGE_CONTENT_CARD_BODY_CLASS } from '@/features/storageBackups/stora
|
||||
import type { StorageCapacityDeltaPresentation } from '@/features/storageBackups/storageCapacityDeltaPresentation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import type { StorageHealthFilter } from '@/features/storageBackups/models';
|
||||
import type { PhysicalDiskAlertResourceIdResolver } from '@/features/storageBackups/diskPresentation';
|
||||
import type { StorageGroupKey, StorageGroupedRecords } from './useStorageModel';
|
||||
import type { StorageAlertRowState } from '@/features/storageBackups/storageAlertState';
|
||||
import type { StorageView } from './storagePageState';
|
||||
@@ -16,6 +17,7 @@ import type { SummarySeriesGroupScope } from '@/components/shared/summaryCardInt
|
||||
type StorageContentCardProps = {
|
||||
view: () => StorageView;
|
||||
physicalDisks: () => Resource[];
|
||||
getDiskAlertResourceIds: PhysicalDiskAlertResourceIdResolver;
|
||||
nodes: () => Resource[];
|
||||
sourceFilter: () => string;
|
||||
healthFilter: () => StorageHealthFilter;
|
||||
@@ -75,6 +77,7 @@ export const StorageContentCard: Component<StorageContentCardProps> = (props) =>
|
||||
<div class={STORAGE_CONTENT_CARD_BODY_CLASS}>
|
||||
<DiskList
|
||||
disks={props.physicalDisks()}
|
||||
getDiskAlertResourceIds={props.getDiskAlertResourceIds}
|
||||
nodes={props.nodes()}
|
||||
sourceFilter={props.sourceFilter()}
|
||||
healthFilter={props.healthFilter()}
|
||||
@@ -100,6 +103,7 @@ export const StorageContentCard: Component<StorageContentCardProps> = (props) =>
|
||||
storageGrowthBySeriesId={props.storageGrowthBySeriesId()}
|
||||
storageGrowthColumnLabel={props.storageGrowthColumnLabel()}
|
||||
physicalDisks={props.physicalDisks()}
|
||||
getDiskAlertResourceIds={props.getDiskAlertResourceIds}
|
||||
nodeOnlineByLabel={props.nodeOnlineByLabel()}
|
||||
highlightedRecordId={props.highlightedRecordId()}
|
||||
getRecordAlertState={props.getRecordAlertState}
|
||||
|
||||
@@ -39,6 +39,7 @@ import {
|
||||
STORAGE_DETAIL_SECTION_TITLE_SPACED_CLASS,
|
||||
} from '@/features/storageBackups/detailPresentation';
|
||||
import type { StorageRecord } from '@/features/storageBackups/models';
|
||||
import type { PhysicalDiskAlertResourceIdResolver } from '@/features/storageBackups/diskPresentation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { formatTemperature } from '@/utils/temperature';
|
||||
import { StorageDetailKeyValueRow } from './StorageDetailKeyValueRow';
|
||||
@@ -47,6 +48,7 @@ import { useStoragePoolDetailModel } from './useStoragePoolDetailModel';
|
||||
interface StoragePoolDetailProps {
|
||||
record: StorageRecord;
|
||||
physicalDisks: Resource[];
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
summarySeriesId: string;
|
||||
controlsId?: string;
|
||||
}
|
||||
@@ -67,6 +69,7 @@ export const StoragePoolDetail: Component<StoragePoolDetailProps> = (props) => {
|
||||
} = useStoragePoolDetailModel({
|
||||
record: () => props.record,
|
||||
physicalDisks: () => props.physicalDisks,
|
||||
getDiskAlertResourceIds: props.getDiskAlertResourceIds,
|
||||
});
|
||||
const rangeOptions = createMemo(() =>
|
||||
getUnlockedHistoryRangeOptions(STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS, maxHistoryDays()),
|
||||
@@ -285,7 +288,10 @@ export const StoragePoolDetail: Component<StoragePoolDetailProps> = (props) => {
|
||||
? PHYSICAL_DISK_TEMPERATURE_LAST_KNOWN_CLASS
|
||||
: getLinkedDiskTemperatureTextClass(
|
||||
disk.temperature,
|
||||
getDiskTemperatureThresholds(disk.diskType),
|
||||
getDiskTemperatureThresholds(
|
||||
disk.diskType,
|
||||
disk.alertResourceIds,
|
||||
),
|
||||
)
|
||||
}`}
|
||||
title={disk.temperatureLastKnownTitle}
|
||||
|
||||
@@ -24,6 +24,7 @@ import {
|
||||
STORAGE_POOL_ROW_USAGE_WRAP_CLASS,
|
||||
} from '@/features/storageBackups/storagePoolRowPresentation';
|
||||
import type { StorageCapacityDeltaPresentation } from '@/features/storageBackups/storageCapacityDeltaPresentation';
|
||||
import type { PhysicalDiskAlertResourceIdResolver } from '@/features/storageBackups/diskPresentation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import {
|
||||
getStoragePoolCellPaddingClass,
|
||||
@@ -57,6 +58,7 @@ interface StoragePoolRowProps {
|
||||
alertHeadlineCompact?: string | null;
|
||||
alertHeadlineClass?: string;
|
||||
physicalDisks: Resource[];
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
alertDataAttrs: {
|
||||
'data-row-id': string;
|
||||
'data-alert-state': string;
|
||||
@@ -247,6 +249,7 @@ export const StoragePoolRow: Component<StoragePoolRowProps> = (props) => {
|
||||
<StoragePoolDetail
|
||||
record={props.record}
|
||||
physicalDisks={props.physicalDisks}
|
||||
getDiskAlertResourceIds={props.getDiskAlertResourceIds}
|
||||
summarySeriesId={props.summarySeriesId}
|
||||
controlsId={detailControlsId()}
|
||||
/>
|
||||
|
||||
@@ -25,6 +25,7 @@ import { useObservedElementWidth } from '@/hooks/useObservedElementWidth';
|
||||
import { resolveStorageRecordMetricResourceId } from '@/features/storageBackups/storageMetricsIdentity';
|
||||
import type { StorageCapacityDeltaPresentation } from '@/features/storageBackups/storageCapacityDeltaPresentation';
|
||||
import type { StorageAlertRowState } from '@/features/storageBackups/storageAlertState';
|
||||
import type { PhysicalDiskAlertResourceIdResolver } from '@/features/storageBackups/diskPresentation';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { StorageGroupRow } from './StorageGroupRow';
|
||||
import { StoragePoolRow } from './StoragePoolRow';
|
||||
@@ -45,6 +46,7 @@ type StoragePoolsTableProps = {
|
||||
storageGrowthBySeriesId: Map<string, StorageCapacityDeltaPresentation>;
|
||||
storageGrowthColumnLabel: string;
|
||||
physicalDisks: Resource[];
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
nodeOnlineByLabel: Map<string, boolean>;
|
||||
highlightedRecordId: string | null;
|
||||
getRecordAlertState: (recordId: string) => StorageAlertRowState;
|
||||
@@ -195,6 +197,7 @@ export const StoragePoolsTable: Component<StoragePoolsTableProps> = (props) => {
|
||||
alertHeadlineCompact={rowModel().alertHeadlineCompact}
|
||||
alertHeadlineClass={rowModel().alertHeadlineClass}
|
||||
physicalDisks={props.physicalDisks}
|
||||
getDiskAlertResourceIds={props.getDiskAlertResourceIds}
|
||||
alertDataAttrs={rowModel().alertDataAttrs}
|
||||
/>
|
||||
);
|
||||
|
||||
@@ -69,12 +69,14 @@ describe('DiskList', () => {
|
||||
nodes: Resource[];
|
||||
selectedNode: string | null;
|
||||
searchTerm: string;
|
||||
getDiskAlertResourceIds?: (disk: Resource) => string[];
|
||||
}) =>
|
||||
render(() => {
|
||||
const [selectedDiskId, setSelectedDiskId] = createSignal<string | null>(null);
|
||||
return (
|
||||
<DiskList
|
||||
disks={props.disks}
|
||||
getDiskAlertResourceIds={props.getDiskAlertResourceIds}
|
||||
nodes={props.nodes}
|
||||
selectedNode={props.selectedNode}
|
||||
searchTerm={props.searchTerm}
|
||||
@@ -475,6 +477,48 @@ describe('DiskList', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('judges each disk by the Disk Temp override of the machine that reports it', async () => {
|
||||
const getConfig = vi.spyOn(AlertsAPI, 'getConfig').mockResolvedValue({
|
||||
enabled: true,
|
||||
activationState: 'active',
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: { nvme: { trigger: 70, clear: 65 } },
|
||||
overrides: {
|
||||
'host-pve3': { diskTemperature: { trigger: 80, clear: 75 } },
|
||||
'host-pve5': { disabled: true },
|
||||
},
|
||||
} as unknown as AlertConfig);
|
||||
try {
|
||||
const view = renderDiskList({
|
||||
disks: [
|
||||
buildDisk('nvme-pve3', 'pve3', { diskType: 'nvme', temperature: 72 }),
|
||||
buildDisk('nvme-pve4', 'pve4', { diskType: 'nvme', temperature: 72 }),
|
||||
buildDisk('nvme-pve5', 'pve5', { diskType: 'nvme', temperature: 72 }),
|
||||
],
|
||||
nodes: [],
|
||||
selectedNode: null,
|
||||
searchTerm: '',
|
||||
getDiskAlertResourceIds: (disk) => [`host-${disk.parentId?.replace('node-', '')}`],
|
||||
});
|
||||
const row = (id: string) =>
|
||||
within(view.container.querySelector(`[data-row-id="${id}"]`) as HTMLElement);
|
||||
|
||||
await useAlertsActivation().refreshConfig();
|
||||
// pve3's 80C override puts 72C under its warning band: healthy and green.
|
||||
await waitFor(() => expect(row('nvme-pve3').getByText('Healthy')).toBeInTheDocument());
|
||||
expect(row('nvme-pve3').getByText('72°C')).toHaveClass('text-green-600');
|
||||
// pve4 has no override, so the 70C NVMe trigger judges its disk.
|
||||
expect(row('nvme-pve4').getByText('Running Hot')).toBeInTheDocument();
|
||||
expect(row('nvme-pve4').getByText('72°C')).toHaveClass('text-red-600');
|
||||
// pve5's alerts are switched off, so its disk is never judged hot.
|
||||
expect(row('nvme-pve5').queryByText('Running Hot')).not.toBeInTheDocument();
|
||||
expect(row('nvme-pve5').getByText('Healthy')).toBeInTheDocument();
|
||||
} finally {
|
||||
getConfig.mockRestore();
|
||||
eventBus.emit('org_switched', 'default');
|
||||
}
|
||||
});
|
||||
|
||||
it('renders SSD life and falls back to the Proxmox usage string for Belongs', () => {
|
||||
renderDiskList({
|
||||
disks: [
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import {
|
||||
getAgentMachineAlertOverrideKeys,
|
||||
getPhysicalDiskAlertResourceIds,
|
||||
indexPhysicalDiskOwners,
|
||||
} from '@/components/Storage/physicalDiskAlertOwner';
|
||||
|
||||
const resource = (fields: Record<string, unknown>): Resource =>
|
||||
({ name: String(fields.id), status: 'online', lastSeen: 0, ...fields }) as unknown as Resource;
|
||||
|
||||
const agentMachine = resource({
|
||||
id: 'agent-tower',
|
||||
type: 'agent',
|
||||
agent: { agentId: 'host-tower' },
|
||||
proxmox: { sourceId: 'lab-tower', nodeName: 'tower' },
|
||||
});
|
||||
const nodeWithoutAgent = resource({
|
||||
id: 'node-bare',
|
||||
type: 'agent',
|
||||
proxmox: { sourceId: 'lab-bare', nodeName: 'bare' },
|
||||
});
|
||||
const unraidArray = resource({ id: 'storage-array', type: 'storage', parentId: 'agent-tower' });
|
||||
const disk = (parentId?: string) =>
|
||||
resource({ id: `disk-${parentId}`, type: 'physical_disk', parentId });
|
||||
|
||||
describe('physical disk alert owner', () => {
|
||||
const owners = indexPhysicalDiskOwners([
|
||||
agentMachine,
|
||||
nodeWithoutAgent,
|
||||
unraidArray,
|
||||
disk('agent-tower'),
|
||||
]);
|
||||
|
||||
it('indexes only agent machines and storage pools', () => {
|
||||
expect([...owners.keys()].sort()).toEqual(['agent-tower', 'storage-array']);
|
||||
});
|
||||
|
||||
it('reads the reporting machine override keys in the order CheckHost reads them', () => {
|
||||
// Agent ID, the canonical ID it resolves to, then the merged node.
|
||||
const expected = ['host-tower', 'agent-tower', 'lab-tower'];
|
||||
expect(getAgentMachineAlertOverrideKeys(agentMachine)).toEqual(expected);
|
||||
expect(getPhysicalDiskAlertResourceIds(disk('agent-tower'), owners)).toEqual(expected);
|
||||
// An Unraid array or cache disk hangs off its pool, which hangs off the machine.
|
||||
expect(getPhysicalDiskAlertResourceIds(disk('storage-array'), owners)).toEqual(expected);
|
||||
});
|
||||
|
||||
it('follows an agent merged into a guest to the guest override keys', () => {
|
||||
const guest = resource({
|
||||
id: 'vm-unraid',
|
||||
type: 'vm',
|
||||
agent: { agentId: 'host-unraid' },
|
||||
proxmox: { instance: 'lab', node: 'tower', nodeName: 'tower', vmid: 105 },
|
||||
});
|
||||
const keys = getAgentMachineAlertOverrideKeys(guest);
|
||||
expect(keys.slice(0, 2)).toEqual(['host-unraid', 'vm-unraid']);
|
||||
expect(keys).toContain('lab:tower:105');
|
||||
});
|
||||
|
||||
it('leaves disks no loaded agent reports on the per-type thresholds', () => {
|
||||
expect(getPhysicalDiskAlertResourceIds(disk('node-bare'), owners)).toEqual([]);
|
||||
expect(getPhysicalDiskAlertResourceIds(disk('vm-not-loaded'), owners)).toEqual([]);
|
||||
expect(getPhysicalDiskAlertResourceIds(disk(undefined), owners)).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -94,6 +94,7 @@ describe('useStoragePoolDetailModel', () => {
|
||||
devPath: '/dev/sda',
|
||||
model: 'Disk A',
|
||||
diskType: '',
|
||||
alertResourceIds: [],
|
||||
temperature: 44,
|
||||
hasIssue: false,
|
||||
errorCount: 0,
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
import { guestOverrideIdCandidates } from '@/features/alerts/guestOverrideIdentity';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { getExplicitAgentIdFromResource } from '@/utils/agentResources';
|
||||
|
||||
/**
|
||||
* Indexes the resources a physical disk can reach the machine that reports it
|
||||
* through: machines running a Pulse agent, and the storage pools disks are
|
||||
* parented to.
|
||||
*/
|
||||
export const indexPhysicalDiskOwners = (resources: readonly Resource[]): Map<string, Resource> => {
|
||||
const owners = new Map<string, Resource>();
|
||||
for (const resource of resources) {
|
||||
if (resource.type === 'storage' || getExplicitAgentIdFromResource(resource)) {
|
||||
owners.set(resource.id, resource);
|
||||
}
|
||||
}
|
||||
return owners;
|
||||
};
|
||||
|
||||
const uniqueKeys = (...keys: Array<string | undefined>): string[] => [
|
||||
...new Set(keys.map((key) => key?.trim() ?? '').filter((key) => key.length > 0)),
|
||||
];
|
||||
|
||||
/**
|
||||
* The alert override keys of a machine running a Pulse agent, in the order
|
||||
* CheckHost reads them for that agent: its own ID, the machine's canonical
|
||||
* resource ID (which the backend resolves the agent ID to), then the Proxmox
|
||||
* node or guest the agent is merged with. The first key with an override
|
||||
* decides.
|
||||
*/
|
||||
export const getAgentMachineAlertOverrideKeys = (machine: Resource): string[] => {
|
||||
const linkedGuestKeys =
|
||||
machine.type === 'vm' || machine.type === 'system-container'
|
||||
? guestOverrideIdCandidates(machine)
|
||||
: [];
|
||||
return uniqueKeys(
|
||||
getExplicitAgentIdFromResource(machine),
|
||||
machine.id,
|
||||
machine.type === 'agent' ? machine.proxmox?.sourceId : undefined,
|
||||
...linkedGuestKeys,
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
* The alert override keys that judge a physical disk's heat: those of the
|
||||
* machine whose Pulse agent reports the disk, which its disk temperature
|
||||
* alerts are evaluated under. That is the agent machine the disk is parented
|
||||
* to or, for an Unraid array or cache disk parented to its pool, the pool's
|
||||
* machine. A Disk Temp override set on that machine therefore replaces the
|
||||
* per-type thresholds for every disk it reports. Empty when no agent reports
|
||||
* the disk or its machine is not loaded, which leaves the per-type thresholds.
|
||||
*/
|
||||
export const getPhysicalDiskAlertResourceIds = (
|
||||
disk: Resource,
|
||||
owners: ReadonlyMap<string, Resource>,
|
||||
): string[] => {
|
||||
let parentId = disk.parentId?.trim();
|
||||
for (let hops = 0; parentId && hops < 2; hops += 1) {
|
||||
const parent = owners.get(parentId);
|
||||
if (!parent) break;
|
||||
if (getExplicitAgentIdFromResource(parent)) return getAgentMachineAlertOverrideKeys(parent);
|
||||
if (parent.type !== 'storage') break;
|
||||
parentId = parent.parentId?.trim();
|
||||
}
|
||||
return [];
|
||||
};
|
||||
@@ -17,6 +17,8 @@ import { resolvePhysicalDiskHistoryResourceId } from './diskResourceUtils';
|
||||
|
||||
type UseDiskDetailModelOptions = {
|
||||
disk: Accessor<Resource>;
|
||||
/** Alert override keys of the machine whose agent reports the disk. */
|
||||
alertResourceIds?: Accessor<string[] | undefined>;
|
||||
};
|
||||
|
||||
export const useDiskDetailModel = (options: UseDiskDetailModelOptions) => {
|
||||
@@ -24,7 +26,11 @@ export const useDiskDetailModel = (options: UseDiskDetailModelOptions) => {
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
|
||||
const diskData = createMemo<PhysicalDiskPresentationData>(() =>
|
||||
extractPhysicalDiskPresentationData(options.disk(), getDiskTemperatureThresholds),
|
||||
extractPhysicalDiskPresentationData(
|
||||
options.disk(),
|
||||
getDiskTemperatureThresholds,
|
||||
options.alertResourceIds?.(),
|
||||
),
|
||||
);
|
||||
const historyResourceId = createMemo(() => resolvePhysicalDiskHistoryResourceId(options.disk()));
|
||||
const attributeCards = createMemo(() =>
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
buildPhysicalDiskRoleFilterOptions,
|
||||
extractPhysicalDiskPresentationData,
|
||||
filterAndSortPhysicalDisks,
|
||||
type PhysicalDiskAlertResourceIdResolver,
|
||||
type PhysicalDiskPresentationData,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
@@ -14,6 +15,7 @@ import { matchesPhysicalDiskNode } from './diskResourceUtils';
|
||||
|
||||
type UseDiskListModelOptions = {
|
||||
disks: () => Resource[];
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
nodes: () => Resource[];
|
||||
selectedNode: () => string | null;
|
||||
sourceFilter?: () => string;
|
||||
@@ -30,16 +32,25 @@ export const useDiskListModel = (options: UseDiskListModelOptions) => {
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
|
||||
// Each disk's heat is judged by the user's alert disk temperature thresholds,
|
||||
// so the presentation data recomputes when the alert configuration loads.
|
||||
// under the Disk Temp override of the machine that reports it, so the
|
||||
// presentation data recomputes when the alert configuration loads.
|
||||
const diskDataById = createMemo(() =>
|
||||
buildPhysicalDiskPresentationDataMap(options.disks(), getDiskTemperatureThresholds),
|
||||
buildPhysicalDiskPresentationDataMap(
|
||||
options.disks(),
|
||||
getDiskTemperatureThresholds,
|
||||
options.getDiskAlertResourceIds,
|
||||
),
|
||||
);
|
||||
const roleFilterOptions = createMemo(() => buildPhysicalDiskRoleFilterOptions(options.disks()));
|
||||
const groupFilterOptions = createMemo(() => buildPhysicalDiskGroupFilterOptions(options.disks()));
|
||||
|
||||
const getDiskData = (disk: Resource): PhysicalDiskPresentationData =>
|
||||
diskDataById().get(disk.id) ??
|
||||
extractPhysicalDiskPresentationData(disk, getDiskTemperatureThresholds);
|
||||
extractPhysicalDiskPresentationData(
|
||||
disk,
|
||||
getDiskTemperatureThresholds,
|
||||
options.getDiskAlertResourceIds?.(disk),
|
||||
);
|
||||
|
||||
const selectedNodeResource = createMemo(
|
||||
() => options.nodes().find((node) => node.id === options.selectedNode()) ?? null,
|
||||
|
||||
@@ -66,6 +66,7 @@ export const useStoragePageModel = (options: UseStoragePageModelOptions = {}) =>
|
||||
storageResources,
|
||||
nodes,
|
||||
physicalDisks,
|
||||
getDiskAlertResourceIds,
|
||||
cephResources,
|
||||
alertsEnabled,
|
||||
} = useStoragePageResources({ resourceSource: options.resourceSource });
|
||||
@@ -471,6 +472,7 @@ export const useStoragePageModel = (options: UseStoragePageModelOptions = {}) =>
|
||||
initialDataReceived: surfaceInitialDataReceived,
|
||||
nodeOptions,
|
||||
physicalDisks,
|
||||
getDiskAlertResourceIds,
|
||||
nodes,
|
||||
groupedRecords,
|
||||
expandedGroups,
|
||||
|
||||
@@ -3,6 +3,7 @@ import { useWebSocket } from '@/contexts/appRuntime';
|
||||
import { useUnifiedResources } from '@/hooks/useUnifiedResources';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import type { Resource, ResourceType } from '@/types/resource';
|
||||
import { getPhysicalDiskAlertResourceIds, indexPhysicalDiskOwners } from './physicalDiskAlertOwner';
|
||||
|
||||
const STORAGE_PAGE_RESOURCES_QUERY = 'type=agent,pbs,storage,physical_disk,ceph';
|
||||
|
||||
@@ -42,6 +43,11 @@ export const useStoragePageResources = (options: UseStoragePageResourcesOptions
|
||||
...resourceSource,
|
||||
resources: storageResourceRows,
|
||||
};
|
||||
// A disk's heat is judged under the alert overrides of the machine whose
|
||||
// agent reports it; this page loads those machines and pools with the disks.
|
||||
const physicalDiskOwners = createMemo(() => indexPhysicalDiskOwners(resourceSource.resources()));
|
||||
const getDiskAlertResourceIds = (disk: Resource): string[] =>
|
||||
getPhysicalDiskAlertResourceIds(disk, physicalDiskOwners());
|
||||
const alertsEnabled = alertsActivation.detectionEnabled;
|
||||
|
||||
return {
|
||||
@@ -53,6 +59,7 @@ export const useStoragePageResources = (options: UseStoragePageResourcesOptions
|
||||
reconnect,
|
||||
nodes,
|
||||
physicalDisks,
|
||||
getDiskAlertResourceIds,
|
||||
cephResources,
|
||||
storageResources,
|
||||
alertsEnabled,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Accessor, createMemo, createSignal } from 'solid-js';
|
||||
import type { HistoryTimeRange } from '@/api/charts';
|
||||
import type { StorageRecord } from '@/features/storageBackups/models';
|
||||
import type { PhysicalDiskAlertResourceIdResolver } from '@/features/storageBackups/diskPresentation';
|
||||
import {
|
||||
buildStoragePoolDetailConfigRows,
|
||||
buildStoragePoolDetailTopologyRows,
|
||||
@@ -13,6 +14,7 @@ import type { Resource } from '@/types/resource';
|
||||
type UseStoragePoolDetailModelOptions = {
|
||||
record: Accessor<StorageRecord>;
|
||||
physicalDisks: Accessor<Resource[]>;
|
||||
getDiskAlertResourceIds?: PhysicalDiskAlertResourceIdResolver;
|
||||
};
|
||||
|
||||
export const useStoragePoolDetailModel = (options: UseStoragePoolDetailModelOptions) => {
|
||||
@@ -22,7 +24,11 @@ export const useStoragePoolDetailModel = (options: UseStoragePoolDetailModelOpti
|
||||
const configRows = createMemo(() => buildStoragePoolDetailConfigRows(options.record()));
|
||||
const zfsSummary = createMemo(() => buildStoragePoolDetailZfsSummary(options.record()));
|
||||
const linkedDisks = createMemo(() =>
|
||||
getStoragePoolLinkedDisks(options.record(), options.physicalDisks()),
|
||||
getStoragePoolLinkedDisks(
|
||||
options.record(),
|
||||
options.physicalDisks(),
|
||||
options.getDiskAlertResourceIds,
|
||||
),
|
||||
);
|
||||
const topologyRows = createMemo(() =>
|
||||
buildStoragePoolDetailTopologyRows(options.record(), linkedDisks()),
|
||||
|
||||
@@ -29,6 +29,7 @@ import {
|
||||
} from './guestDrawerModel';
|
||||
import type { NestedWorkloadContext } from './nestedWorkloadContext';
|
||||
import { WORKLOAD_ACTION_AGENT_LABEL } from './workloadAgentReadiness';
|
||||
import { getWorkloadAlertResourceIdCandidates } from './workloadTopology';
|
||||
import { RaidCard } from '@/components/shared/cards/RaidCard';
|
||||
|
||||
import type { GuestDrawerProps } from './guestDrawerModel';
|
||||
@@ -326,7 +327,14 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {
|
||||
}
|
||||
>
|
||||
<Suspense fallback={null}>
|
||||
<GuestPhysicalDisks parentId={props.guest.canonicalResourceId ?? props.guest.id} />
|
||||
<GuestPhysicalDisks
|
||||
parentId={props.guest.canonicalResourceId ?? props.guest.id}
|
||||
alertResourceIds={[
|
||||
...(props.guest.agentId ? [props.guest.agentId] : []),
|
||||
...(props.guest.canonicalResourceId ? [props.guest.canonicalResourceId] : []),
|
||||
...getWorkloadAlertResourceIdCandidates(props.guest),
|
||||
]}
|
||||
/>
|
||||
</Suspense>
|
||||
</Show>
|
||||
<Show when={props.guest.agentRaid?.length}>
|
||||
|
||||
@@ -19,10 +19,15 @@ const LazyDiskDetail = lazy(() =>
|
||||
import('@/components/Storage/DiskDetail').then(({ DiskDetail }) => ({ default: DiskDetail })),
|
||||
);
|
||||
|
||||
const GuestPhysicalDiskRow: Component<{ disk: Resource }> = (props) => {
|
||||
const GuestPhysicalDiskRow: Component<{ disk: Resource; alertResourceIds: string[] }> = (props) => {
|
||||
const [expanded, setExpanded] = createSignal(false);
|
||||
const { getDiskTemperatureThresholds } = useAlertsActivation();
|
||||
const data = () => extractPhysicalDiskPresentationData(props.disk, getDiskTemperatureThresholds);
|
||||
const data = () =>
|
||||
extractPhysicalDiskPresentationData(
|
||||
props.disk,
|
||||
getDiskTemperatureThresholds,
|
||||
props.alertResourceIds,
|
||||
);
|
||||
const health = () => getPhysicalDiskHealthStatus(data());
|
||||
const temperature = () => getPhysicalDiskTemperaturePresentation(data());
|
||||
const label = () => data().model || data().devPath || props.disk.name;
|
||||
@@ -68,7 +73,11 @@ const GuestPhysicalDiskRow: Component<{ disk: Resource }> = (props) => {
|
||||
</p>
|
||||
}
|
||||
>
|
||||
<LazyDiskDetail disk={props.disk} nodes={[]} />
|
||||
<LazyDiskDetail
|
||||
disk={props.disk}
|
||||
nodes={[]}
|
||||
alertResourceIds={props.alertResourceIds}
|
||||
/>
|
||||
</Suspense>
|
||||
</div>
|
||||
</Show>
|
||||
@@ -76,7 +85,15 @@ const GuestPhysicalDiskRow: Component<{ disk: Resource }> = (props) => {
|
||||
);
|
||||
};
|
||||
|
||||
export const GuestPhysicalDisks: Component<{ parentId: string }> = (props) => {
|
||||
export const GuestPhysicalDisks: Component<{
|
||||
parentId: string;
|
||||
/**
|
||||
* Alert override keys judging these disks' heat, in the order the backend
|
||||
* reads them: the guest's own agent reports them, so its Disk Temp override
|
||||
* applies, then the guest's canonical ID and override keys.
|
||||
*/
|
||||
alertResourceIds?: string[];
|
||||
}> = (props) => {
|
||||
// The Proxmox Overview snapshot deliberately excludes physical disks. Query
|
||||
// only the open guest's direct children rather than hydrating every disk in
|
||||
// a large estate (or mistaking the VMID-based table key for the resource ID).
|
||||
@@ -104,7 +121,11 @@ export const GuestPhysicalDisks: Component<{ parentId: string }> = (props) => {
|
||||
Physical Disks & SMART ({disks().length})
|
||||
</h3>
|
||||
<div class="space-y-2">
|
||||
<For each={disks()}>{(disk) => <GuestPhysicalDiskRow disk={disk} />}</For>
|
||||
<For each={disks()}>
|
||||
{(disk) => (
|
||||
<GuestPhysicalDiskRow disk={disk} alertResourceIds={props.alertResourceIds ?? []} />
|
||||
)}
|
||||
</For>
|
||||
</div>
|
||||
</InfoCardFrame>
|
||||
</Show>
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { cleanup, fireEvent, render, screen, within } from '@solidjs/testing-library';
|
||||
import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/testing-library';
|
||||
import { createSignal } from 'solid-js';
|
||||
import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { AlertsAPI } from '@/api/alerts';
|
||||
import { useAlertsActivation } from '@/stores/alertsActivation';
|
||||
import { eventBus } from '@/stores/events';
|
||||
import type { AlertConfig } from '@/types/alerts';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import { GuestPhysicalDisks } from '../GuestPhysicalDisks';
|
||||
|
||||
@@ -83,6 +87,38 @@ describe('GuestPhysicalDisks', () => {
|
||||
expect(within(card).getAllByTestId('guest-physical-disk')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("judges the guest's disks by its own agent's Disk Temp override", async () => {
|
||||
const getConfig = vi.spyOn(AlertsAPI, 'getConfig').mockResolvedValue({
|
||||
enabled: true,
|
||||
activationState: 'active',
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: { sata: { trigger: 55, clear: 50 } },
|
||||
overrides: { 'host-guest-101': { diskTemperature: { trigger: 65, clear: 60 } } },
|
||||
} as unknown as AlertConfig);
|
||||
try {
|
||||
queryState.resources = [
|
||||
{ ...disk, physicalDisk: { ...disk.physicalDisk!, temperature: 58 } } as Resource,
|
||||
];
|
||||
await useAlertsActivation().refreshConfig();
|
||||
const guestCard = render(() => (
|
||||
<GuestPhysicalDisks parentId="vm-resource-101" alertResourceIds={['host-guest-101']} />
|
||||
));
|
||||
// 58C passes the 55C SATA trigger but not the agent's 65C override.
|
||||
await waitFor(() =>
|
||||
expect(within(guestCard.container).getByText('Healthy')).toBeInTheDocument(),
|
||||
);
|
||||
guestCard.unmount();
|
||||
|
||||
const unmatched = render(() => (
|
||||
<GuestPhysicalDisks parentId="vm-resource-101" alertResourceIds={['host-other']} />
|
||||
));
|
||||
expect(within(unmatched.container).getByText('Running Hot')).toBeInTheDocument();
|
||||
} finally {
|
||||
getConfig.mockRestore();
|
||||
eventBus.emit('org_switched', 'default');
|
||||
}
|
||||
});
|
||||
|
||||
it('does not add an empty storage card to an uninstrumented guest', () => {
|
||||
render(() => <GuestPhysicalDisks parentId="vm-resource-102" />);
|
||||
expect(screen.queryByTestId('guest-physical-disks')).toBeNull();
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import type { AlertConfig } from '@/types/alerts';
|
||||
import type { Resource } from '@/types/resource';
|
||||
import {
|
||||
buildPhysicalDiskPresentationDataMap,
|
||||
@@ -401,6 +402,31 @@ describe('diskPresentation', () => {
|
||||
}
|
||||
});
|
||||
|
||||
it('judges a disk under the Disk Temp override of the machine that reports it', () => {
|
||||
const config = {
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: { nvme: { trigger: 70, clear: 65 } },
|
||||
overrides: { 'host-raised': { diskTemperature: { trigger: 80, clear: 75 } } },
|
||||
} as unknown as AlertConfig;
|
||||
const resolve = (diskType: string, alertResourceIds?: string[]) =>
|
||||
resolveDiskTemperatureDisplayThresholds(config, diskType, alertResourceIds);
|
||||
const raisedHostDisk = buildDisk('nvme0n1', { diskType: 'nvme', temperature: 72 });
|
||||
const plainHostDisk = buildDisk('nvme1n1', { diskType: 'nvme', temperature: 72 });
|
||||
const dataMap = buildPhysicalDiskPresentationDataMap(
|
||||
[raisedHostDisk, plainHostDisk],
|
||||
resolve,
|
||||
(disk) => (disk === raisedHostDisk ? ['host-raised'] : ['host-plain']),
|
||||
);
|
||||
|
||||
expect(dataMap.get('nvme0n1')!.temperatureThresholds).toEqual({ warning: 75, critical: 80 });
|
||||
expect(getPhysicalDiskHealthStatus(dataMap.get('nvme0n1')!).label).toBe('Healthy');
|
||||
expect(getPhysicalDiskHealthStatus(dataMap.get('nvme1n1')!).label).toBe('Running Hot');
|
||||
expect(
|
||||
extractPhysicalDiskPresentationData(raisedHostDisk, resolve, ['host-raised'])
|
||||
.temperatureThresholds,
|
||||
).toEqual({ warning: 75, critical: 80 });
|
||||
});
|
||||
|
||||
it('calls no disk hot when its temperature alerting is off', () => {
|
||||
const off = () => null;
|
||||
const data = extractPhysicalDiskPresentationData(buildDisk('sda', { temperature: 80 }), off);
|
||||
|
||||
+6
-1
@@ -231,6 +231,7 @@ describe('storagePoolDetailPresentation', () => {
|
||||
state: 'online',
|
||||
sizeLabel: '1000 B',
|
||||
diskType: '',
|
||||
alertResourceIds: [],
|
||||
temperature: 44,
|
||||
hasIssue: true,
|
||||
spunDown: false,
|
||||
@@ -313,9 +314,13 @@ describe('storagePoolDetailPresentation', () => {
|
||||
},
|
||||
] as Resource[];
|
||||
|
||||
const linkedDisks = getStoragePoolLinkedDisks(record, disks);
|
||||
const linkedDisks = getStoragePoolLinkedDisks(record, disks, (disk) => [`owner-of-${disk.id}`]);
|
||||
|
||||
expect(linkedDisks.map((disk) => disk.role)).toEqual(['parity', 'data']);
|
||||
// Each linked disk carries the override keys its Temp colour is judged under.
|
||||
expect(linkedDisks.map((disk) => disk.alertResourceIds)).toEqual(
|
||||
linkedDisks.map((disk) => [`owner-of-${disk.id}`]),
|
||||
);
|
||||
expect(linkedDisks[0].temperatureLastKnownTitle).toBeUndefined();
|
||||
expect(linkedDisks[1]).toEqual(
|
||||
expect.objectContaining({
|
||||
|
||||
@@ -76,9 +76,10 @@ export interface PhysicalDiskPresentationData {
|
||||
target?: string;
|
||||
temperature: number;
|
||||
/**
|
||||
* The alert disk temperature thresholds for this disk's type. The Temp cell
|
||||
* colours the reading by them and the Health verdict judges heat by them.
|
||||
* Absent means the factory thresholds for the type.
|
||||
* The alert disk temperature thresholds for this disk's type, or the Disk
|
||||
* Temp override of the machine whose agent reports it. The Temp cell colours
|
||||
* the reading by them and the Health verdict judges heat by them. Absent
|
||||
* means the factory thresholds for the type.
|
||||
*/
|
||||
temperatureThresholds?: MetricDisplayThresholds | null;
|
||||
rpm: number;
|
||||
@@ -410,14 +411,23 @@ export function getPhysicalDiskHostLabel(
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the alert disk temperature thresholds for a disk type. Callers with
|
||||
* the user's alert configuration pass `getDiskTemperatureThresholds` from the
|
||||
* alerts activation store; the default is the factory configuration.
|
||||
* Resolves the alert disk temperature thresholds for a disk type, under the
|
||||
* alert override keys of the machine whose agent reports the disk. Callers
|
||||
* with the user's alert configuration pass `getDiskTemperatureThresholds` from
|
||||
* the alerts activation store; the default is the factory configuration.
|
||||
*/
|
||||
export type PhysicalDiskTemperatureThresholdResolver = (
|
||||
diskType: string,
|
||||
alertResourceIds?: string[],
|
||||
) => MetricDisplayThresholds | null;
|
||||
|
||||
/**
|
||||
* Returns the alert override keys of the machine whose agent reports a disk,
|
||||
* so that machine's Disk Temp override judges the disk's heat. Empty when no
|
||||
* agent reports it.
|
||||
*/
|
||||
export type PhysicalDiskAlertResourceIdResolver = (disk: Resource) => string[];
|
||||
|
||||
const resolveFactoryDiskTemperatureThresholds: PhysicalDiskTemperatureThresholdResolver = (
|
||||
diskType,
|
||||
) => resolveDiskTemperatureDisplayThresholds(null, diskType);
|
||||
@@ -433,6 +443,7 @@ export const getPhysicalDiskTemperatureThresholds = (
|
||||
export function extractPhysicalDiskPresentationData(
|
||||
resource: Resource,
|
||||
resolveTemperatureThresholds: PhysicalDiskTemperatureThresholdResolver = resolveFactoryDiskTemperatureThresholds,
|
||||
alertResourceIds: string[] = [],
|
||||
): PhysicalDiskPresentationData {
|
||||
const pd = resource.physicalDisk || ((resource.platformData as any)?.physicalDisk ?? {});
|
||||
const diskNode = getPhysicalDiskNodeIdentity(resource);
|
||||
@@ -467,7 +478,7 @@ export function extractPhysicalDiskPresentationData(
|
||||
health: pd.health || 'UNKNOWN',
|
||||
wearout: pd.wearout ?? -1,
|
||||
temperature: pd.temperature ?? 0,
|
||||
temperatureThresholds: resolveTemperatureThresholds(pd.diskType || ''),
|
||||
temperatureThresholds: resolveTemperatureThresholds(pd.diskType || '', alertResourceIds),
|
||||
rpm: pd.rpm ?? 0,
|
||||
used: pd.used || '',
|
||||
storageRole: pd.storageRole,
|
||||
@@ -541,10 +552,18 @@ export function getPhysicalDiskCollectionMessages(disk: PhysicalDiskPresentation
|
||||
export function buildPhysicalDiskPresentationDataMap(
|
||||
disks: Resource[],
|
||||
resolveTemperatureThresholds?: PhysicalDiskTemperatureThresholdResolver,
|
||||
getAlertResourceIds?: PhysicalDiskAlertResourceIdResolver,
|
||||
): Map<string, PhysicalDiskPresentationData> {
|
||||
const map = new Map<string, PhysicalDiskPresentationData>();
|
||||
for (const disk of disks || []) {
|
||||
map.set(disk.id, extractPhysicalDiskPresentationData(disk, resolveTemperatureThresholds));
|
||||
map.set(
|
||||
disk.id,
|
||||
extractPhysicalDiskPresentationData(
|
||||
disk,
|
||||
resolveTemperatureThresholds,
|
||||
getAlertResourceIds?.(disk),
|
||||
),
|
||||
);
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
import type { HistoryTimeRange, ResourceType as HistoryChartResourceType } from '@/api/charts';
|
||||
import { getPhysicalDiskTemperaturePresentation } from '@/features/storageBackups/diskPresentation';
|
||||
import {
|
||||
getPhysicalDiskTemperaturePresentation,
|
||||
type PhysicalDiskAlertResourceIdResolver,
|
||||
} from '@/features/storageBackups/diskPresentation';
|
||||
import type { StorageRecord } from '@/features/storageBackups/models';
|
||||
import {
|
||||
getStorageRecordContent,
|
||||
@@ -27,6 +30,8 @@ export type StoragePoolDetailLinkedDisk = {
|
||||
state: string;
|
||||
sizeLabel: string;
|
||||
diskType: string;
|
||||
/** Alert override keys of the machine whose agent reports the disk. */
|
||||
alertResourceIds?: string[];
|
||||
temperature: number;
|
||||
/** Set only when the temperature is a retained, not current, reading. */
|
||||
temperatureLastKnownTitle?: string;
|
||||
@@ -402,6 +407,7 @@ const normalizeDevicePath = (value: string): string => value.trim().replace(/^\/
|
||||
export function getStoragePoolLinkedDisks(
|
||||
record: StorageRecord,
|
||||
physicalDisks: Resource[],
|
||||
getAlertResourceIds?: PhysicalDiskAlertResourceIdResolver,
|
||||
): StoragePoolDetailLinkedDisk[] {
|
||||
const pool = getStorageRecordZfsPool(record);
|
||||
const unraidStorageGroup = getUnraidStorageGroup(record).toLowerCase();
|
||||
@@ -432,6 +438,7 @@ export function getStoragePoolLinkedDisks(
|
||||
state: readDiskState(disk),
|
||||
sizeLabel: readDiskSizeLabel(disk),
|
||||
diskType: readDiskType(disk),
|
||||
alertResourceIds: getAlertResourceIds?.(disk) ?? [],
|
||||
temperature: readDiskTemperature(disk),
|
||||
temperatureLastKnownTitle: readDiskTemperatureLastKnownTitle(disk),
|
||||
hasIssue: readDiskHasIssue(disk),
|
||||
|
||||
@@ -99,11 +99,12 @@ describe('useWorkloads', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('preserves linked-agent RAID arrays from the VM resource for guest detail', async () => {
|
||||
it('preserves linked-agent RAID arrays and agent ID from the VM resource for guest detail', async () => {
|
||||
const [snapshot] = createSignal([
|
||||
{
|
||||
...sampleResource,
|
||||
agent: {
|
||||
agentId: 'host-guest-agent',
|
||||
agentVersion: '6.4.5',
|
||||
raid: [
|
||||
{
|
||||
@@ -130,6 +131,8 @@ describe('useWorkloads', () => {
|
||||
try {
|
||||
await flushAsync();
|
||||
expect(result!.workloads()[0]?.canonicalResourceId).toBe(sampleResource.id);
|
||||
// The guest drawer judges the agent's disks under this ID's overrides.
|
||||
expect(result!.workloads()[0]?.agentId).toBe('host-guest-agent');
|
||||
expect(result!.workloads()[0]?.agentRaid).toMatchObject([
|
||||
{ device: '/dev/md0', level: 'raid1', state: 'clean' },
|
||||
]);
|
||||
|
||||
@@ -137,6 +137,7 @@ type APIResource = {
|
||||
vcpus?: number;
|
||||
};
|
||||
agent?: {
|
||||
agentId?: string;
|
||||
hostname?: string;
|
||||
uptimeSeconds?: number;
|
||||
osName?: string;
|
||||
@@ -608,6 +609,7 @@ const mapResourceToWorkload = (resource: APIResource): WorkloadGuest | null => {
|
||||
osVersion: resource.agent?.osVersion ?? resource.proxmox?.osVersion,
|
||||
agentVersion: resource.agent?.agentVersion,
|
||||
agentKind: resource.agent?.agentVersion ? 'pulse' : undefined,
|
||||
agentId: resource.agent?.agentId,
|
||||
agentRaid: resource.agent?.raid,
|
||||
networkInterfaces: mapNetworkInterfaces(resource.agent?.networkInterfaces),
|
||||
networkIn: finiteMetricNumber(resource.metrics?.netIn?.value) ?? 0,
|
||||
|
||||
@@ -56,6 +56,8 @@ export type WorkloadGuest = (VM | Container) & {
|
||||
containerRuntime?: string;
|
||||
/** Identifies the producer of agentVersion instead of inferring it from VM type. */
|
||||
agentKind?: 'pulse' | 'qemu-guest';
|
||||
/** ID of the Pulse agent running in this guest, the key of its alert overrides. */
|
||||
agentId?: string;
|
||||
/** RAID arrays reported by the linked Pulse agent, not Proxmox virtual disks. */
|
||||
agentRaid?: HostRAIDArray[];
|
||||
updateStatus?: DockerContainerUpdateStatus;
|
||||
|
||||
@@ -397,6 +397,50 @@ describe('metricThresholds', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it('judges a disk by the alert overrides of the machine that reports it', () => {
|
||||
const config = {
|
||||
enabled: true,
|
||||
guestDefaults: {},
|
||||
nodeDefaults: {},
|
||||
agentDefaults: { diskTemperature: { trigger: 55, clear: 50 } },
|
||||
diskTempByType: { nvme: { trigger: 70, clear: 65 } },
|
||||
storageDefault: { trigger: 85, clear: 80 },
|
||||
overrides: {
|
||||
'host-raised': { diskTemperature: { trigger: 80, clear: 75 } },
|
||||
'host-memory-only': { memory: { trigger: 95, clear: 90 } },
|
||||
'host-off': { disabled: true },
|
||||
},
|
||||
} as AlertConfig;
|
||||
// The machine's first matching key decides, as CheckHost's chain does.
|
||||
expect(
|
||||
resolveDiskTemperatureDisplayThresholds(config, 'nvme', [
|
||||
'agent:host-raised',
|
||||
'host-raised',
|
||||
]),
|
||||
).toEqual({ warning: 75, critical: 80 });
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'nvme', ['host-memory-only'])).toEqual(
|
||||
{
|
||||
warning: 65,
|
||||
critical: 70,
|
||||
},
|
||||
);
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'nvme', ['host-plain'])).toEqual({
|
||||
warning: 65,
|
||||
critical: 70,
|
||||
});
|
||||
// A host whose alerts are switched off raises no disk temperature alert,
|
||||
// so none of its disks is judged hot.
|
||||
expect(resolveDiskTemperatureDisplayThresholds(config, 'nvme', ['host-off'])).toBeNull();
|
||||
// An agent default that switches every host's alerts off does the same.
|
||||
const agentsOff = {
|
||||
...config,
|
||||
agentDefaults: { ...config.agentDefaults, disabled: true },
|
||||
} as AlertConfig;
|
||||
expect(
|
||||
resolveDiskTemperatureDisplayThresholds(agentsOff, 'nvme', ['host-raised']),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it('falls back to seeded per-type disk temperature defaults without config', () => {
|
||||
expect(resolveDiskTemperatureDisplayThresholds(null, 'nvme')).toEqual({
|
||||
warning: 65,
|
||||
|
||||
@@ -271,7 +271,10 @@ export const resolveMetricDisplayThresholds = (
|
||||
* host (or inherited linked resource) wins, then the per-type map
|
||||
* (diskTempByType: nvme/sas/sata), then the global agent default. A
|
||||
* switched-off agent default switches every type off, as disk temperature
|
||||
* alerts do. Null means disk temperature alerting is off for this disk.
|
||||
* alerts do, and so does an override (or agent default) that switches the
|
||||
* host's alerts off. Null means disk temperature alerting is off for this
|
||||
* disk. `resourceIds` are the alert override keys of the machine whose agent
|
||||
* reports the disk, in the order the backend reads them.
|
||||
*/
|
||||
export const resolveDiskTemperatureDisplayThresholds = (
|
||||
config: AlertConfig | null,
|
||||
@@ -282,6 +285,9 @@ export const resolveDiskTemperatureDisplayThresholds = (
|
||||
const normalizedType = (diskType ?? '').trim().toLowerCase();
|
||||
|
||||
const override = findOverride(config?.overrides, resourceIds);
|
||||
if (override?.disabled || config?.agentDefaults?.disabled === true) {
|
||||
return null;
|
||||
}
|
||||
const overrideValue = getOverrideValue(override, 'diskTemperature');
|
||||
if (overrideValue !== undefined) {
|
||||
return resolveThreshold(overrideValue, FACTORY_AGENT_DEFAULTS.diskTemperature, margin);
|
||||
|
||||
@@ -69,12 +69,13 @@ func (a *AlertThresholdAdapter) GetGuestDiskThreshold() float64 {
|
||||
}
|
||||
|
||||
// GetDiskTemperatureThreshold returns the disk temperature alert trigger and
|
||||
// clear values for a disk type, resolved exactly as disk temperature alerts
|
||||
// resolve them (alerts.Manager.DiskTemperatureThreshold).
|
||||
func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(diskType string) (float64, float64) {
|
||||
// clear values for a disk of a type that a host agent reports, resolved
|
||||
// exactly as disk temperature alerts resolve them
|
||||
// (alerts.Manager.HostDiskTemperatureThreshold), host overrides included.
|
||||
func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(host alerts.DiskTemperatureHost, diskType string) (float64, float64) {
|
||||
var threshold *alerts.HysteresisThreshold
|
||||
if a.manager != nil {
|
||||
threshold = a.manager.DiskTemperatureThreshold(diskType)
|
||||
threshold = a.manager.HostDiskTemperatureThreshold(host, diskType)
|
||||
} else {
|
||||
threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
|
||||
}
|
||||
|
||||
@@ -84,7 +84,7 @@ func TestAlertThresholdAdapter_Fallbacks(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {
|
||||
if trigger, clear := NewAlertThresholdAdapter(nil).GetDiskTemperatureThreshold("nvme"); trigger != 70 || clear != 65 {
|
||||
if trigger, clear := NewAlertThresholdAdapter(nil).GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, "nvme"); trigger != 70 || clear != 65 {
|
||||
t.Fatalf("no manager: nvme = %v/%v, want the factory 70/65", trigger, clear)
|
||||
}
|
||||
|
||||
@@ -93,17 +93,42 @@ func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {
|
||||
cfg.DiskTempByType["nvme"] = alerts.HysteresisThreshold{Trigger: 75, Clear: 70}
|
||||
mgr.UpdateConfig(cfg)
|
||||
a := NewAlertThresholdAdapter(mgr)
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("nvme"); trigger != 75 || clear != 70 {
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, "nvme"); trigger != 75 || clear != 70 {
|
||||
t.Fatalf("raised nvme = %v/%v, want 75/70", trigger, clear)
|
||||
}
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("sata"); trigger != 55 || clear != 50 {
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, "sata"); trigger != 55 || clear != 50 {
|
||||
t.Fatalf("sata = %v/%v, want 55/50", trigger, clear)
|
||||
}
|
||||
|
||||
cfg = mgr.GetConfig()
|
||||
cfg.AgentDefaults.DiskTemperature = &alerts.HysteresisThreshold{Trigger: 0, Clear: 0}
|
||||
mgr.UpdateConfig(cfg)
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold("nvme"); trigger != 0 || clear != 0 {
|
||||
if trigger, clear := a.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, "nvme"); trigger != 0 || clear != 0 {
|
||||
t.Fatalf("disk temperature alerting off: nvme = %v/%v, want 0/0", trigger, clear)
|
||||
}
|
||||
}
|
||||
|
||||
// A host's Disk Temp override reaches Patrol for that host's disks only, and a
|
||||
// host whose alerts are switched off has no disk heat policy at all.
|
||||
func TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides(t *testing.T) {
|
||||
mgr := alerts.NewManager()
|
||||
cfg := mgr.GetConfig()
|
||||
cfg.Overrides = map[string]alerts.ThresholdConfig{
|
||||
"host-raised": {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 80, Clear: 75}},
|
||||
"host-off": {Disabled: true},
|
||||
}
|
||||
mgr.UpdateConfig(cfg)
|
||||
a := NewAlertThresholdAdapter(mgr)
|
||||
|
||||
for host, want := range map[string][2]float64{
|
||||
"host-raised": {80, 75},
|
||||
"host-plain": {70, 65},
|
||||
"host-off": {0, 0},
|
||||
"": {70, 65},
|
||||
} {
|
||||
trigger, clear := a.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{ID: host}, "nvme")
|
||||
if trigger != want[0] || clear != want[1] {
|
||||
t.Errorf("host %q nvme = %v/%v, want %v/%v", host, trigger, clear, want[0], want[1])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,7 +63,7 @@ func (m *mockThresholdProvider) GetGuestCPUThreshold() float64 { return 0 }
|
||||
func (m *mockThresholdProvider) GetGuestMemoryThreshold() float64 { return m.guestMem }
|
||||
func (m *mockThresholdProvider) GetGuestDiskThreshold() float64 { return m.guestDisk }
|
||||
func (m *mockThresholdProvider) GetStorageThreshold() float64 { return m.storage }
|
||||
func (m *mockThresholdProvider) GetDiskTemperatureThreshold(diskType string) (float64, float64) {
|
||||
func (m *mockThresholdProvider) GetDiskTemperatureThreshold(_ alerts.DiskTemperatureHost, diskType string) (float64, float64) {
|
||||
if limits, ok := m.diskTemperature[diskType]; ok {
|
||||
return limits[0], limits[1]
|
||||
}
|
||||
|
||||
@@ -67,9 +67,10 @@ type ThresholdProvider interface {
|
||||
// GetStorageThreshold returns the usage alert trigger threshold for storage (0-100%)
|
||||
GetStorageThreshold() float64
|
||||
// GetDiskTemperatureThreshold returns the disk temperature alert trigger
|
||||
// and clear values (°C) for a disk type. A zero trigger means disk
|
||||
// temperature alerting is off.
|
||||
GetDiskTemperatureThreshold(diskType string) (trigger, clear float64)
|
||||
// and clear values (°C) for a disk of the given type that the given host
|
||||
// agent reports. A host with no ID means no agent reports the disk. A zero
|
||||
// trigger means disk temperature alerting is off for it.
|
||||
GetDiskTemperatureThreshold(host alerts.DiskTemperatureHost, diskType string) (trigger, clear float64)
|
||||
}
|
||||
|
||||
// AlertResolver provides the ability to review and resolve alerts
|
||||
|
||||
@@ -24,6 +24,7 @@ import (
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/memory"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/providers"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/tools"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/servicediscovery"
|
||||
@@ -2447,6 +2448,10 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
if urp != nil {
|
||||
diskResources := urp.GetByType(unifiedresources.ResourceTypePhysicalDisk)
|
||||
rows := make([]patrolPhysicalDiskRow, 0, len(diskResources))
|
||||
var owners map[string]unifiedresources.Resource
|
||||
if len(diskResources) > 0 {
|
||||
owners = physicalDiskOwnerIndex(urp)
|
||||
}
|
||||
for _, r := range diskResources {
|
||||
if !seedIsInScope(scopedSet, r.ID) || r.PhysicalDisk == nil {
|
||||
continue
|
||||
@@ -2479,7 +2484,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
temperature: tools.SplitDiskTemperature(r.PhysicalDisk.Temperature, r.PhysicalDisk.Collection),
|
||||
smartEvidence: unifiedPhysicalDiskSMARTIssueParts(r.PhysicalDisk.SMART),
|
||||
}
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(row.diskType)
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(physicalDiskTemperatureHost(r, owners), row.diskType)
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows
|
||||
@@ -2522,7 +2527,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
|
||||
temperature: tools.SplitDiskTemperature(d.Temperature, d.Collection),
|
||||
smartEvidence: modelPhysicalDiskSMARTIssueParts(d.SmartAttributes),
|
||||
}
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(row.diskType)
|
||||
row.temperatureLimits = snap.diskTemperatureLimits(alerts.DiskTemperatureHost{}, row.diskType)
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
package ai
|
||||
|
||||
import "github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
import (
|
||||
"strings"
|
||||
|
||||
// diskTemperatureLimits is the alert disk temperature policy for one disk
|
||||
// type, the same policy disk temperature alerts and the Physical Disks Health
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
)
|
||||
|
||||
// diskTemperatureLimits is the alert disk temperature policy for one disk,
|
||||
// the same policy disk temperature alerts and the Physical Disks Health
|
||||
// verdict judge heat by. A disk is hot at the trigger and stays hot until it
|
||||
// cools to the clear value. A zero trigger means disk temperature alerting is
|
||||
// off, so no reading counts as heat.
|
||||
@@ -12,13 +17,14 @@ type diskTemperatureLimits struct {
|
||||
clear float64
|
||||
}
|
||||
|
||||
// diskTemperatureLimitsFor resolves the policy from the user's alert
|
||||
// configuration, or from the factory alert configuration when Patrol has no
|
||||
// threshold provider.
|
||||
func diskTemperatureLimitsFor(provider ThresholdProvider, diskType string) diskTemperatureLimits {
|
||||
// diskTemperatureLimitsFor resolves the policy for a disk of the given type
|
||||
// that the given host agent reports from the user's alert configuration, so
|
||||
// that host's Disk Temp override applies, or from the factory alert
|
||||
// configuration when Patrol has no threshold provider.
|
||||
func diskTemperatureLimitsFor(provider ThresholdProvider, host alerts.DiskTemperatureHost, diskType string) diskTemperatureLimits {
|
||||
var trigger, clear float64
|
||||
if provider != nil {
|
||||
trigger, clear = provider.GetDiskTemperatureThreshold(diskType)
|
||||
trigger, clear = provider.GetDiskTemperatureThreshold(host, diskType)
|
||||
} else if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
|
||||
trigger, clear = threshold.Trigger, threshold.Clear
|
||||
}
|
||||
@@ -48,3 +54,44 @@ func (l diskTemperatureLimits) cooled(temperature int) bool {
|
||||
}
|
||||
return float64(temperature) <= l.clear
|
||||
}
|
||||
|
||||
// physicalDiskOwnerIndex indexes, by resource ID, the resources a physical disk
|
||||
// can reach its reporting host agent through: agent-bearing machines and the
|
||||
// storage pools disks hang off.
|
||||
func physicalDiskOwnerIndex(urp UnifiedResourceProvider) map[string]unifiedresources.Resource {
|
||||
index := make(map[string]unifiedresources.Resource)
|
||||
for _, resource := range urp.GetAll() {
|
||||
if resource.Agent != nil || unifiedresources.CanonicalResourceType(resource.Type) == unifiedresources.ResourceTypeStorage {
|
||||
index[resource.ID] = resource
|
||||
}
|
||||
}
|
||||
return index
|
||||
}
|
||||
|
||||
// physicalDiskTemperatureHost returns the host agent that reports a physical
|
||||
// disk, the host its disk temperature alerts are evaluated under: the agent on
|
||||
// the machine the disk is parented to or, for a disk parented to a storage
|
||||
// pool (Unraid array and cache disks), the agent on the pool's machine. A disk
|
||||
// no agent reports gets the hostless policy.
|
||||
func physicalDiskTemperatureHost(disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) alerts.DiskTemperatureHost {
|
||||
parentID := disk.ParentID
|
||||
for hops := 0; parentID != nil && hops < 2; hops++ {
|
||||
parent, ok := owners[strings.TrimSpace(*parentID)]
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
if parent.Agent != nil && strings.TrimSpace(parent.Agent.AgentID) != "" {
|
||||
return alerts.DiskTemperatureHost{
|
||||
ID: strings.TrimSpace(parent.Agent.AgentID),
|
||||
LinkedNodeID: parent.Agent.LinkedNodeID,
|
||||
LinkedVMID: parent.Agent.LinkedVMID,
|
||||
LinkedContainerID: parent.Agent.LinkedContainerID,
|
||||
}
|
||||
}
|
||||
if unifiedresources.CanonicalResourceType(parent.Type) != unifiedresources.ResourceTypeStorage {
|
||||
break
|
||||
}
|
||||
parentID = parent.ParentID
|
||||
}
|
||||
return alerts.DiskTemperatureHost{}
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
|
||||
"github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
|
||||
@@ -33,7 +34,7 @@ func TestDiskTemperatureLimitsFollowTheAlertPolicy(t *testing.T) {
|
||||
{name: "alerting off", provider: &mockThresholdProvider{diskTemperature: map[string][2]float64{"sata": {0, 0}}}, diskType: "sata", want: diskTemperatureLimits{}},
|
||||
{name: "clear above trigger", provider: &mockThresholdProvider{diskTemperature: map[string][2]float64{"sas": {60, 64}}}, diskType: "sas", want: diskTemperatureLimits{trigger: 60, clear: 60}},
|
||||
} {
|
||||
if got := diskTemperatureLimitsFor(tc.provider, tc.diskType); got != tc.want {
|
||||
if got := diskTemperatureLimitsFor(tc.provider, alerts.DiskTemperatureHost{}, tc.diskType); got != tc.want {
|
||||
t.Errorf("%s: limits = %+v, want %+v", tc.name, got, tc.want)
|
||||
}
|
||||
}
|
||||
@@ -171,7 +172,7 @@ func TestPatrolRunStateCarriesTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
{ID: "nvme-72", DevPath: "/dev/nvme0n1", Type: "nvme", Health: "PASSED", Wearout: 90, Temperature: 72},
|
||||
}}
|
||||
ps := NewPatrolService(nil, &mockStateProvider{state: snapshot})
|
||||
if got := ps.currentPatrolRuntimeState().diskTemperatureLimits("nvme"); got.trigger != 70 {
|
||||
if got := ps.currentPatrolRuntimeState().diskTemperatureLimits(alerts.DiskTemperatureHost{}, "nvme"); got.trigger != 70 {
|
||||
t.Fatalf("no provider: nvme trigger = %v, want the factory 70", got.trigger)
|
||||
}
|
||||
|
||||
@@ -180,7 +181,7 @@ func TestPatrolRunStateCarriesTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
"current": ps.currentPatrolRuntimeState(),
|
||||
"snapshot": ps.patrolRuntimeStateForSnapshot(snapshot),
|
||||
} {
|
||||
if got := state.diskTemperatureLimits("nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
if got := state.diskTemperatureLimits(alerts.DiskTemperatureHost{}, "nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
t.Errorf("%s state: nvme limits = %+v, want the configured 75/70", name, got)
|
||||
}
|
||||
if flags := triageDiskHealthChecksState(state, nil); len(flags) != 0 {
|
||||
@@ -194,7 +195,7 @@ func TestPatrolRunStateCarriesTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
func TestScopedPatrolStateKeepsTheConfiguredDiskTemperaturePolicy(t *testing.T) {
|
||||
state := diskTemperatureTriageState(raisedNVMeThresholds())
|
||||
scoped := filterPatrolStateByScopeState(state, PatrolScope{ResourceIDs: []string{"nvme-72"}})
|
||||
if got := scoped.diskTemperatureLimits("nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
if got := scoped.diskTemperatureLimits(alerts.DiskTemperatureHost{}, "nvme"); got.trigger != 75 || got.clear != 70 {
|
||||
t.Fatalf("scoped state nvme limits = %+v, want the configured 75/70", got)
|
||||
}
|
||||
if flags := triageDiskHealthChecksState(scoped, nil); len(flags) != 0 {
|
||||
@@ -212,19 +213,19 @@ func TestPatrolStartsWithAThresholdProviderSetBeforeIt(t *testing.T) {
|
||||
if patrol == nil {
|
||||
t.Fatal("expected Patrol to start once a state provider exists")
|
||||
}
|
||||
if got := patrol.currentPatrolRuntimeState().diskTemperatureLimits("nvme"); got.trigger != 75 {
|
||||
if got := patrol.currentPatrolRuntimeState().diskTemperatureLimits(alerts.DiskTemperatureHost{}, "nvme"); got.trigger != 75 {
|
||||
t.Fatalf("Patrol nvme trigger = %v, want the provider's 75", got.trigger)
|
||||
}
|
||||
}
|
||||
|
||||
// A heat finding recovers at the clear value, where its alert recovers.
|
||||
func TestDiskHeatRecoversAtTheClearValue(t *testing.T) {
|
||||
limits := diskTemperatureLimitsFor(nil, "nvme")
|
||||
limits := diskTemperatureLimitsFor(nil, alerts.DiskTemperatureHost{}, "nvme")
|
||||
if !limits.cooled(65) || limits.cooled(66) {
|
||||
t.Fatalf("nvme limits %+v: want 65C cooled and 66C still warm", limits)
|
||||
}
|
||||
// With no band below the trigger, a reading at the trigger is still hot.
|
||||
noBand := diskTemperatureLimitsFor(&mockThresholdProvider{diskTemperature: map[string][2]float64{"sas": {70, 70}}}, "sas")
|
||||
noBand := diskTemperatureLimitsFor(&mockThresholdProvider{diskTemperature: map[string][2]float64{"sas": {70, 70}}}, alerts.DiskTemperatureHost{}, "sas")
|
||||
if !noBand.hot(70) || noBand.cooled(70) || !noBand.cooled(69) {
|
||||
t.Fatalf("no-band limits %+v: want 70C hot and not cooled, 69C cooled", noBand)
|
||||
}
|
||||
@@ -247,3 +248,98 @@ func TestDiskHeatRecoveryNeedsACurrentReading(t *testing.T) {
|
||||
t.Fatalf("retained 40C: recovered = %v, err = %v, want recovered", recovered, err)
|
||||
}
|
||||
}
|
||||
|
||||
// overriddenAgentDisksRegistry is two host agents, each reporting one NVMe
|
||||
// disk hotter than the NVMe trigger, with a Disk Temp override of 80C (clear
|
||||
// 75) set on agent-cool only. Alerts judge agent-cool's disk by the override.
|
||||
func overriddenAgentDisksRegistry(t *testing.T) (*unifiedresources.ResourceRegistry, *AlertThresholdAdapter) {
|
||||
t.Helper()
|
||||
now := time.Now()
|
||||
agent := func(id string, temperature int) models.Host {
|
||||
return models.Host{
|
||||
ID: id, Hostname: id, DisplayName: id, Status: "online", LastSeen: now, IntervalSeconds: 30,
|
||||
Sensors: models.HostSensorSummary{SMART: []models.HostDiskSMART{{
|
||||
Device: "nvme0n1", Model: "Samsung 990", Serial: id + "-serial", Type: "nvme",
|
||||
Health: "PASSED", Temperature: temperature,
|
||||
}}},
|
||||
}
|
||||
}
|
||||
registry := unifiedresources.NewRegistry(nil)
|
||||
registry.IngestSnapshot(models.StateSnapshot{Hosts: []models.Host{agent("agent-cool", 76), agent("agent-plain", 72)}})
|
||||
|
||||
mgr := alerts.NewManager()
|
||||
cfg := mgr.GetConfig()
|
||||
cfg.Overrides = map[string]alerts.ThresholdConfig{
|
||||
"agent-cool": {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 80, Clear: 75}},
|
||||
}
|
||||
mgr.UpdateConfig(cfg)
|
||||
return registry, NewAlertThresholdAdapter(mgr)
|
||||
}
|
||||
|
||||
// A host agent's Disk Temp override replaces the per-type trigger for every
|
||||
// disk that agent reports, in Patrol exactly as in its alerts: agent-cool's
|
||||
// NVMe at 76C is under its 80C override, agent-plain's at 72C is over the
|
||||
// NVMe trigger of 70C.
|
||||
func TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride(t *testing.T) {
|
||||
registry, provider := overriddenAgentDisksRegistry(t)
|
||||
state := newPatrolRuntimeStateWithProviders(models.StateSnapshot{}, nil, unifiedresources.NewUnifiedAIAdapter(registry))
|
||||
state.thresholdProvider = provider
|
||||
|
||||
rows := patrolPhysicalDiskRows(state, nil)
|
||||
if len(rows) != 2 {
|
||||
t.Fatalf("rows = %+v, want one disk per agent", rows)
|
||||
}
|
||||
for _, row := range rows {
|
||||
want, wantIssue := diskTemperatureLimits{trigger: 70, clear: 65}, true
|
||||
if row.temperature.Collected == 76 {
|
||||
want, wantIssue = diskTemperatureLimits{trigger: 80, clear: 75}, false
|
||||
}
|
||||
if row.temperatureLimits != want {
|
||||
t.Errorf("disk at %dC: limits = %+v, want %+v", row.temperature.Collected, row.temperatureLimits, want)
|
||||
}
|
||||
if got := patrolPhysicalDiskHealthIssue(row); got != wantIssue {
|
||||
t.Errorf("disk at %dC: health issue = %v, want %v", row.temperature.Collected, got, wantIssue)
|
||||
}
|
||||
}
|
||||
|
||||
s := &Service{}
|
||||
s.SetUnifiedResourceProvider(unifiedresources.NewUnifiedAIAdapter(registry))
|
||||
s.SetPatrolThresholdProvider(provider)
|
||||
_, section, _ := strings.Cut(s.buildUnifiedResourceContext(), "Physical Disks Needing Attention")
|
||||
section, _, _ = strings.Cut(section, "\n\n")
|
||||
if !strings.Contains(section, "Temp: 72C") || strings.Contains(section, "Temp: 76C") {
|
||||
t.Fatalf("attention section = %q, want agent-plain's 72C disk only", section)
|
||||
}
|
||||
}
|
||||
|
||||
// A disk reaches its reporting agent through its parent: the agent's machine
|
||||
// directly, or the storage pool an Unraid array or cache disk hangs off. A
|
||||
// disk on a machine with no agent gets the hostless policy.
|
||||
func TestPhysicalDiskTemperatureHostFollowsTheParentChain(t *testing.T) {
|
||||
ref := func(id string) *string { return &id }
|
||||
owners := map[string]unifiedresources.Resource{
|
||||
"agent-1": {ID: "agent-1", Type: unifiedresources.ResourceTypeAgent, Agent: &unifiedresources.AgentData{
|
||||
AgentID: "host-1", LinkedNodeID: "lab-pve1",
|
||||
}},
|
||||
"vm-1": {ID: "vm-1", Type: unifiedresources.ResourceTypeVM, Agent: &unifiedresources.AgentData{
|
||||
AgentID: "host-vm", LinkedVMID: "lab-pve1-101",
|
||||
}},
|
||||
"array-1": {ID: "array-1", Type: unifiedresources.ResourceTypeStorage, ParentID: ref("agent-1")},
|
||||
"node-1": {ID: "node-1", Type: unifiedresources.ResourceTypeAgent},
|
||||
}
|
||||
for parent, want := range map[string]alerts.DiskTemperatureHost{
|
||||
"agent-1": {ID: "host-1", LinkedNodeID: "lab-pve1"},
|
||||
"vm-1": {ID: "host-vm", LinkedVMID: "lab-pve1-101"},
|
||||
"array-1": {ID: "host-1", LinkedNodeID: "lab-pve1"},
|
||||
"node-1": {},
|
||||
"missing": {},
|
||||
} {
|
||||
disk := unifiedresources.Resource{ID: "disk-under-" + parent, Type: unifiedresources.ResourceTypePhysicalDisk, ParentID: ref(parent)}
|
||||
if got := physicalDiskTemperatureHost(disk, owners); got != want {
|
||||
t.Errorf("disk under %s: host = %+v, want %+v", parent, got, want)
|
||||
}
|
||||
}
|
||||
if got := physicalDiskTemperatureHost(unifiedresources.Resource{ID: "orphan"}, owners); got != (alerts.DiskTemperatureHost{}) {
|
||||
t.Errorf("parentless disk: host = %+v, want hostless", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -100,9 +100,9 @@ type patrolRuntimeState struct {
|
||||
}
|
||||
|
||||
// diskTemperatureLimits resolves the alert disk temperature policy for one
|
||||
// disk type.
|
||||
func (s patrolRuntimeState) diskTemperatureLimits(diskType string) diskTemperatureLimits {
|
||||
return diskTemperatureLimitsFor(s.thresholdProvider, diskType)
|
||||
// disk of the given type that the given host agent reports.
|
||||
func (s patrolRuntimeState) diskTemperatureLimits(host alerts.DiskTemperatureHost, diskType string) diskTemperatureLimits {
|
||||
return diskTemperatureLimitsFor(s.thresholdProvider, host, diskType)
|
||||
}
|
||||
|
||||
func newPatrolRuntimeState(snapshot models.StateSnapshot) patrolRuntimeState {
|
||||
|
||||
@@ -3,6 +3,7 @@ package ai
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
@@ -20,7 +21,7 @@ func (m MockThresholdProvider) GetNodeMemoryThreshold() float64 { return m.Node
|
||||
func (m MockThresholdProvider) GetGuestMemoryThreshold() float64 { return m.GuestMem }
|
||||
func (m MockThresholdProvider) GetGuestDiskThreshold() float64 { return m.GuestDisk }
|
||||
func (m MockThresholdProvider) GetStorageThreshold() float64 { return m.Storage }
|
||||
func (m MockThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
func (m MockThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
|
||||
return 0, 0
|
||||
}
|
||||
|
||||
|
||||
@@ -409,6 +409,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
|
||||
|
||||
if len(physicalDisks) > 0 {
|
||||
attention := make([]unifiedresources.Resource, 0)
|
||||
owners := physicalDiskOwnerIndex(urp)
|
||||
for _, disk := range physicalDisks {
|
||||
health := ""
|
||||
hot := false
|
||||
@@ -416,7 +417,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
|
||||
health = strings.ToUpper(strings.TrimSpace(disk.PhysicalDisk.Health))
|
||||
// A retained last-known reading is not current heat.
|
||||
collected := tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected
|
||||
hot = diskTemperatureLimitsFor(thresholdProvider, disk.PhysicalDisk.DiskType).hot(collected)
|
||||
hot = diskTemperatureLimitsFor(thresholdProvider, physicalDiskTemperatureHost(disk, owners), disk.PhysicalDisk.DiskType).hot(collected)
|
||||
}
|
||||
if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || hot {
|
||||
attention = append(attention, disk)
|
||||
|
||||
+29
-2
@@ -1017,14 +1017,41 @@ func diskTemperatureThresholdForType(byType map[string]HysteresisThreshold, host
|
||||
// temperature alerts, Patrol and the Physical Disks Health verdict all judge
|
||||
// heat against it; disk risk does not. The disk is hot at the trigger, and the
|
||||
// clear value is where it stops being hot. A nil threshold or a non-positive
|
||||
// trigger means disk temperature alerting is off.
|
||||
// trigger means disk temperature alerting is off. A disk a host agent reports
|
||||
// is judged by HostDiskTemperatureThreshold instead.
|
||||
func (m *Manager) DiskTemperatureThreshold(diskType string) *HysteresisThreshold {
|
||||
return m.HostDiskTemperatureThreshold(DiskTemperatureHost{}, diskType)
|
||||
}
|
||||
|
||||
// DiskTemperatureHost is the host agent whose report carries a physical disk,
|
||||
// with the node and guests that agent is linked to. CheckHost evaluates the
|
||||
// disk's temperature under that host's thresholds.
|
||||
type DiskTemperatureHost struct {
|
||||
ID string
|
||||
LinkedNodeID string
|
||||
LinkedVMID string
|
||||
LinkedContainerID string
|
||||
}
|
||||
|
||||
// HostDiskTemperatureThreshold is the disk heat policy for a disk the given
|
||||
// host agent reports, resolved as CheckHost resolves it. An explicit disk
|
||||
// temperature override on the host, or one inherited from its linked node or
|
||||
// guest, replaces the disk type's entry. An override that switches the host's
|
||||
// alerts off switches its disk heat off too, so the result is nil. A host with
|
||||
// no ID resolves as DiskTemperatureThreshold.
|
||||
func (m *Manager) HostDiskTemperatureThreshold(host DiskTemperatureHost, diskType string) *HysteresisThreshold {
|
||||
if m == nil {
|
||||
return DefaultDiskTemperatureThreshold(diskType)
|
||||
}
|
||||
m.mu.RLock()
|
||||
defer m.mu.RUnlock()
|
||||
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
|
||||
thresholds := m.resolveHostThresholdsNoLock(host.ID, host.LinkedNodeID, host.LinkedVMID, host.LinkedContainerID)
|
||||
if thresholds.Disabled {
|
||||
return nil
|
||||
}
|
||||
override, exists := m.hostThresholdOverrideNoLock(host.ID, host.LinkedNodeID, host.LinkedVMID, host.LinkedContainerID)
|
||||
overridden := exists && override.DiskTemperature != nil
|
||||
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(thresholds.DiskTemperature, overridden, diskType))
|
||||
}
|
||||
|
||||
// DefaultDiskTemperatureThreshold is DiskTemperatureThreshold under the
|
||||
|
||||
@@ -1445,3 +1445,116 @@ func TestDiskTemperatureThresholdMatchesCheckHostPolicy(t *testing.T) {
|
||||
t.Fatalf("nil manager threshold = %+v, want the factory nvme 70", got)
|
||||
}
|
||||
}
|
||||
|
||||
// HostDiskTemperatureThreshold is the disk heat policy for a disk a host agent
|
||||
// reports, so it must agree with CheckHost on every host-level override: an
|
||||
// explicit Disk Temp override on the host, or inherited from its linked node,
|
||||
// replaces the per-type entry; the first override in the chain decides even
|
||||
// when it sets no Disk Temp; and a host whose alerts are off judges no heat.
|
||||
func TestHostDiskTemperatureThresholdMatchesCheckHostOverrides(t *testing.T) {
|
||||
const hostID = "host-temp-policy"
|
||||
const nodeID = "pve1-node1"
|
||||
perType := &HysteresisThreshold{Trigger: 70, Clear: 65}
|
||||
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
overrides map[string]ThresholdConfig
|
||||
linkNode bool
|
||||
want *HysteresisThreshold
|
||||
off bool
|
||||
}{
|
||||
{name: "no override", want: perType},
|
||||
{
|
||||
name: "host disk temp override",
|
||||
overrides: map[string]ThresholdConfig{hostID: {DiskTemperature: &HysteresisThreshold{Trigger: 80, Clear: 75}}},
|
||||
want: &HysteresisThreshold{Trigger: 80, Clear: 75},
|
||||
},
|
||||
{
|
||||
name: "host override without disk temp",
|
||||
overrides: map[string]ThresholdConfig{hostID: {Memory: &HysteresisThreshold{Trigger: 90, Clear: 85}}},
|
||||
want: perType,
|
||||
},
|
||||
{
|
||||
name: "linked node disk temp override",
|
||||
overrides: map[string]ThresholdConfig{nodeID: {DiskTemperature: &HysteresisThreshold{Trigger: 78, Clear: 72}}},
|
||||
linkNode: true,
|
||||
want: &HysteresisThreshold{Trigger: 78, Clear: 72},
|
||||
},
|
||||
{
|
||||
name: "host override shadows linked node",
|
||||
overrides: map[string]ThresholdConfig{
|
||||
hostID: {Memory: &HysteresisThreshold{Trigger: 90, Clear: 85}},
|
||||
nodeID: {DiskTemperature: &HysteresisThreshold{Trigger: 78, Clear: 72}},
|
||||
},
|
||||
linkNode: true,
|
||||
want: perType,
|
||||
},
|
||||
{
|
||||
name: "host disk temp switched off",
|
||||
overrides: map[string]ThresholdConfig{hostID: {DiskTemperature: &HysteresisThreshold{}}},
|
||||
off: true,
|
||||
},
|
||||
{
|
||||
name: "host alerts disabled",
|
||||
overrides: map[string]ThresholdConfig{hostID: {Disabled: true}},
|
||||
},
|
||||
{
|
||||
name: "linked node alerts disabled",
|
||||
overrides: map[string]ThresholdConfig{nodeID: {Disabled: true}},
|
||||
linkNode: true,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
m := configureDiskTempTypeHostManager(t)
|
||||
m.mu.Lock()
|
||||
for id, override := range tc.overrides {
|
||||
m.config.Overrides[id] = override
|
||||
}
|
||||
m.mu.Unlock()
|
||||
|
||||
owner := DiskTemperatureHost{ID: hostID}
|
||||
if tc.linkNode {
|
||||
owner.LinkedNodeID = nodeID
|
||||
}
|
||||
got := m.HostDiskTemperatureThreshold(owner, "nvme")
|
||||
switch {
|
||||
case tc.off:
|
||||
if got == nil || got.Trigger > 0 {
|
||||
t.Fatalf("HostDiskTemperatureThreshold = %+v, want a switched-off threshold", got)
|
||||
}
|
||||
case (got == nil) != (tc.want == nil) || (got != nil && *got != *tc.want):
|
||||
t.Fatalf("HostDiskTemperatureThreshold = %+v, want %+v", got, tc.want)
|
||||
}
|
||||
|
||||
check := func(temperature int) bool {
|
||||
host := hostWithSMARTDiskTemp(hostID, "nvme", temperature)
|
||||
host.LinkedNodeID = owner.LinkedNodeID
|
||||
m.CheckHost(host)
|
||||
_, exists := testLookupActiveAlert(t, m, hostDiskTempAlertID(host))
|
||||
return exists
|
||||
}
|
||||
if got == nil || got.Trigger <= 0 {
|
||||
if check(99) {
|
||||
t.Fatalf("CheckHost alerted at 99C on a host the policy judges no heat for, active: %v", alertKeys(m))
|
||||
}
|
||||
return
|
||||
}
|
||||
trigger := int(got.Trigger)
|
||||
if check(trigger - 1) {
|
||||
t.Fatalf("CheckHost alerted at %dC, under the policy trigger %v", trigger-1, got.Trigger)
|
||||
}
|
||||
if !check(trigger) {
|
||||
t.Fatalf("CheckHost stayed quiet at the policy trigger %dC, active: %v", trigger, alertKeys(m))
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
m := configureDiskTempTypeHostManager(t)
|
||||
if got, want := m.HostDiskTemperatureThreshold(DiskTemperatureHost{}, "sas"), m.DiskTemperatureThreshold("sas"); got == nil || want == nil || *got != *want {
|
||||
t.Fatalf("hostless policy = %+v, want DiskTemperatureThreshold %+v", got, want)
|
||||
}
|
||||
var nilManager *Manager
|
||||
if got := nilManager.HostDiskTemperatureThreshold(DiskTemperatureHost{ID: hostID}, "nvme"); got == nil || got.Trigger != 70 {
|
||||
t.Fatalf("nil manager host policy = %+v, want the factory nvme 70", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
|
||||
@@ -67,7 +68,7 @@ func (concurrencyThresholdProvider) GetGuestMemoryThreshold() float64 {
|
||||
}
|
||||
func (concurrencyThresholdProvider) GetGuestDiskThreshold() float64 { return 95 }
|
||||
func (concurrencyThresholdProvider) GetStorageThreshold() float64 { return 92 }
|
||||
func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
|
||||
return 70, 65
|
||||
}
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/cost"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/ai/memory"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/config"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/models"
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/monitoring"
|
||||
@@ -154,7 +155,7 @@ func (stubThresholdProvider) GetGuestMemoryThreshold() float64 {
|
||||
}
|
||||
func (stubThresholdProvider) GetGuestDiskThreshold() float64 { return 95 }
|
||||
func (stubThresholdProvider) GetStorageThreshold() float64 { return 92 }
|
||||
func (stubThresholdProvider) GetDiskTemperatureThreshold(string) (float64, float64) {
|
||||
func (stubThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
|
||||
return 70, 65
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user