storage(pools): carry the full ZFS pool payload to the storage surface

The unified-resource adapter flattened models.ZFSPool to four scalars
(state + R/W/C totals), so scan status and the per-device report never
reached the frontend. The storage detail's ZFS card, the usage bar's
scrub/resilver overlay and ZFS tooltip, and ZFS-device-to-disk matching
were all dead code on the unified path: a degraded pool rendered as a
one-word State cell with no way to see which device failed or that a
resilver was running. Found by the v5-parity audit of the pools table
(same pipeline disease as #1471 on the disks table).

- Add StorageMeta.ZFSPool and populate it in resourceFromStorage
  (normalized copy, not aliased).
- Serialize it in monitorStoragePlatformData so the /api/state path
  carries it alongside the existing flat scalars.
- Map it through normalizeStorageMeta and the storage record adapter
  (meta first, flat platformData fallback).
- Render the per-device report (name, vdev type, state, R/W/C errors,
  message) in the detail's ZFS Pool card - the v6 home for what v5
  showed via the health-map dots and the degraded-pool warning row.
- Pin the end-to-end path in code_standards_test and update the
  unified-resources, monitoring, storage-recovery, and
  frontend-primitives contracts.

Verified live in mock mode: degraded pool now shows the resilver scan,
pool error totals, per-device states, and the bar pulse overlay.
This commit is contained in:
rcourtman
2026-06-11 16:48:57 +01:00
parent cbb3b42ce0
commit b53e1a3e0e
17 changed files with 248 additions and 10 deletions
@@ -1475,7 +1475,11 @@ not a replacement status card, CTA band, or page-local nested card.
must distinguish inventory datastores from backup repositories so VMware must distinguish inventory datastores from backup repositories so VMware
rows on the shared storage route stay canonical to the admitted phase-1 floor instead of rows on the shared storage route stay canonical to the admitted phase-1 floor instead of
reviving backup-target, protected-target, or recovery-local semantics on a reviving backup-target, protected-target, or recovery-local semantics on a
shared page. shared page. Those presenters must also source ZFS pool health from the
canonical `details.zfsPool` payload (meta-first `storage.zfsPool`, flat
`platformData.zfsPool` fallback) when building pool detail and bar
summaries, rather than re-deriving device-level health from risk-reason
strings or presenting flattened pool-state scalars as the full report.
30. Keep infrastructure settings-shell API alternatives on the shared shell 30. Keep infrastructure settings-shell API alternatives on the shared shell
contract. `frontend-modern/src/components/Settings/InfrastructureWorkspace.tsx`, contract. `frontend-modern/src/components/Settings/InfrastructureWorkspace.tsx`,
`frontend-modern/src/components/Settings/settingsHeaderMeta.ts`, and `frontend-modern/src/components/Settings/settingsHeaderMeta.ts`, and
@@ -431,7 +431,12 @@ state remains a separate `unraid_sync_active` reason. Realtime resource
broadcasts must preserve canonical identity, discovery target, metrics target, broadcasts must preserve canonical identity, discovery target, metrics target,
incident rollups, and raw `agent`/`storage` facet payloads so frontend incident rollups, and raw `agent`/`storage` facet payloads so frontend
infrastructure surfaces can explain degraded/warning rows without falling back infrastructure surfaces can explain degraded/warning rows without falling back
to generic status labels. That realtime broadcast contract also owns source and to generic status labels. Storage platform-data payloads built by
`monitorStoragePlatformData` must carry the full `zfsPool` report (scan
activity and per-device states/errors/messages) alongside the flattened
`zfsPoolState`/error scalars whenever canonical `StorageMeta.ZFSPool` is
present, so `/api/state` and websocket consumers can render device-level ZFS
health in parity with the unified-resources read path. That realtime broadcast contract also owns source and
platform identity for storage resources: `internal/monitoring/monitor.go` must platform identity for storage resources: `internal/monitoring/monitor.go` must
carry the canonical `Resource.Sources` array onto `ResourceFrontend` and carry the canonical `Resource.Sources` array onto `ResourceFrontend` and
`platformData.sources`, and must derive storage `platformType` from the owning `platformData.sources`, and must derive storage `platformType` from the owning
@@ -1502,6 +1502,18 @@ chart entitlement sequence. They must expose `14d` between `7d` and `30d` and
pass the selected range through to `HistoryChart` unchanged, rather than pass the selected range through to `HistoryChart` unchanged, rather than
inventing storage-local range catalogs, paid-tier labels, or alternate inventing storage-local range catalogs, paid-tier labels, or alternate
metrics-history gating. metrics-history gating.
The storage pool detail's ZFS Pool card is the canonical home for
device-level ZFS health.
`frontend-modern/src/features/storageBackups/storagePoolDetailPresentation.ts`
builds the pool summary (state, scan activity, pool error totals) and the
per-device report (name, vdev type, state, R/W/C error counts, message) from
the record's `details.zfsPool`, which
`frontend-modern/src/features/storageBackups/storageAdapters.ts` and
`resourceStorageMapping.ts` resolve meta-first from canonical
`storage.zfsPool` with flat `platformData.zfsPool` fallback.
`frontend-modern/src/components/Storage/StoragePoolDetail.tsx` renders that
report inside the row expansion so degraded pools name the failing device and
running scrub/resilver without re-promoting per-device noise into table rows.
Shared chart transport that storage and recovery coexist with must also stay Shared chart transport that storage and recovery coexist with must also stay
on rendered-metric budgets. When `internal/api/router.go` batches workload on rendered-metric budgets. When `internal/api/router.go` batches workload
history for adjacent overview or shared summary cards, it may parallelize the history for adjacent overview or shared summary cards, it may parallelize the
@@ -826,7 +826,13 @@ AI-only summary payloads, or page-local heuristics.
`frontend-modern/src/hooks/useUnifiedResources.ts` must carry the `frontend-modern/src/hooks/useUnifiedResources.ts` must carry the
provider-reported storage `pool` metadata alongside path and ZFS health so provider-reported storage `pool` metadata alongside path and ZFS health so
storage consumers do not have to recover backing-pool identity from names storage consumers do not have to recover backing-pool identity from names
or path heuristics. or path heuristics. ZFS health means the full pool report:
`StorageMeta.ZFSPool` carries the provider's pool object (scan activity
plus per-device states, error counters, and messages) to consumers
alongside the flattened `zfsPoolState`/`zfs*Errors` scalars, so storage
surfaces can show which device failed and whether a scrub/resilver is
running instead of reducing pool health to a one-word state
(`TestCanonicalStorageMetadataCarriesFullZFSPoolReport` pins the path).
17. Keep compatible unified-resource consumers on one shared snapshot truth. 17. Keep compatible unified-resource consumers on one shared snapshot truth.
`frontend-modern/src/hooks/useUnifiedResources.ts` may seed narrower `frontend-modern/src/hooks/useUnifiedResources.ts` may seed narrower
type-filtered consumers from a fresh `all-resources` snapshot when the type-filtered consumers from a fresh `all-resources` snapshot when the
@@ -13,6 +13,7 @@ import {
} from '@/features/storageBackups/diskDetailPresentation'; } from '@/features/storageBackups/diskDetailPresentation';
import { import {
STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS, STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS,
getZfsDeviceStateTextClass,
getZfsErrorTextClass, getZfsErrorTextClass,
getZfsScanTextClass, getZfsScanTextClass,
} from '@/features/storageBackups/storagePoolDetailPresentation'; } from '@/features/storageBackups/storagePoolDetailPresentation';
@@ -151,6 +152,36 @@ export const StoragePoolDetail: Component<StoragePoolDetailProps> = (props) => {
{zfsSummary()!.errorSummary} {zfsSummary()!.errorSummary}
</div> </div>
</Show> </Show>
<Show when={zfsSummary()!.devices.length > 0}>
<div class={`${STORAGE_DETAIL_FULL_WIDTH_ROW_CLASS} space-y-0.5 pt-1`}>
<For each={zfsSummary()!.devices}>
{(device) => (
<div class={STORAGE_DETAIL_LINKED_DISK_ROW_CLASS}>
<span
class={STORAGE_DETAIL_LINKED_DISK_PATH_CLASS}
title={device.name}
>
{device.name}
</span>
<Show when={device.type}>
<span class={STORAGE_DETAIL_MUTED_TEXT_CLASS}>{device.type}</span>
</Show>
<span class={getZfsDeviceStateTextClass(device.state)}>
{device.state}
</span>
<Show when={device.errorSummary}>
<span class={getZfsErrorTextClass()}>{device.errorSummary}</span>
</Show>
<Show when={device.message}>
<span class={`${STORAGE_DETAIL_MUTED_TEXT_CLASS} italic`}>
{device.message}
</span>
</Show>
</div>
)}
</For>
</div>
</Show>
</div> </div>
</div> </div>
</Show> </Show>
@@ -558,4 +558,50 @@ describe('storageAdapters', () => {
}); });
expect(records[0].refs?.resourceId).toBe('pool:tank'); expect(records[0].refs?.resourceId).toBe('pool:tank');
}); });
it('carries the full ZFS pool payload from storage meta into record details', () => {
const zfsPool = {
name: 'rpool',
state: 'DEGRADED',
status: 'Degraded',
scan: 'resilver in progress',
readErrors: 0,
writeErrors: 0,
checksumErrors: 2,
devices: [
{
name: 'sda2',
type: 'disk',
state: 'ONLINE',
readErrors: 0,
writeErrors: 0,
checksumErrors: 0,
},
{
name: 'sdb2',
type: 'disk',
state: 'FAULTED',
readErrors: 3,
writeErrors: 0,
checksumErrors: 0,
},
],
};
const state = baseState();
const records = buildStorageRecords({
state,
resources: [
makeResourceStorage({
storage: {
type: 'zfspool',
isZfs: true,
zfsPool,
} as Resource['storage'],
}),
],
});
expect(records).toHaveLength(1);
expect(records[0].details?.zfsPool).toEqual(zfsPool);
});
}); });
@@ -37,7 +37,17 @@ const buildRecord = (overrides: Partial<StorageRecord> = {}): StorageRecord => (
readErrors: 1, readErrors: 1,
writeErrors: 2, writeErrors: 2,
checksumErrors: 3, checksumErrors: 3,
devices: [{ name: 'sda' }], devices: [
{ name: 'sda', type: 'disk', state: 'ONLINE' },
{
name: 'sdb',
type: 'disk',
state: 'FAULTED',
readErrors: 3,
checksumErrors: 1,
message: 'too many errors',
},
],
}, },
}, },
...overrides, ...overrides,
@@ -114,6 +124,16 @@ describe('storagePoolDetailPresentation', () => {
state: 'ONLINE', state: 'ONLINE',
scan: 'scrub repaired 0B', scan: 'scrub repaired 0B',
errorSummary: 'Errors: R:1 W:2 C:3', errorSummary: 'Errors: R:1 W:2 C:3',
devices: [
{ name: 'sda', type: 'disk', state: 'ONLINE', errorSummary: '', message: '' },
{
name: 'sdb',
type: 'disk',
state: 'FAULTED',
errorSummary: '3R/0W/1C errors',
message: 'too many errors',
},
],
}); });
expect(getStoragePoolLinkedDisks(record, [buildDisk()])).toEqual([ expect(getStoragePoolLinkedDisks(record, [buildDisk()])).toEqual([
{ {
@@ -1,4 +1,5 @@
import type { Resource } from '@/types/resource'; import type { Resource } from '@/types/resource';
import type { ZFSPool } from '@/types/api';
import type { StorageCapability, StorageCategory } from './models'; import type { StorageCapability, StorageCategory } from './models';
export type ResourceStorageMeta = { export type ResourceStorageMeta = {
@@ -19,6 +20,7 @@ export type ResourceStorageMeta = {
numMissing?: number; numMissing?: number;
isCeph?: boolean; isCeph?: boolean;
isZfs?: boolean; isZfs?: boolean;
zfsPool?: ZFSPool;
}; };
type ResourceWithStorageMeta = Resource & { type ResourceWithStorageMeta = Resource & {
@@ -65,6 +67,10 @@ const normalizeStorageMeta = (value: unknown): ResourceStorageMeta | null => {
numMissing: typeof candidate.numMissing === 'number' ? candidate.numMissing : undefined, numMissing: typeof candidate.numMissing === 'number' ? candidate.numMissing : undefined,
isCeph: typeof candidate.isCeph === 'boolean' ? candidate.isCeph : undefined, isCeph: typeof candidate.isCeph === 'boolean' ? candidate.isCeph : undefined,
isZfs: typeof candidate.isZfs === 'boolean' ? candidate.isZfs : undefined, isZfs: typeof candidate.isZfs === 'boolean' ? candidate.isZfs : undefined,
zfsPool:
candidate.zfsPool && typeof candidate.zfsPool === 'object'
? (candidate.zfsPool as ResourceStorageMeta['zfsPool'])
: undefined,
}; };
}; };
@@ -213,7 +213,7 @@ const mapResourceStorageRecord = (resource: Resource, adapterId: string): Storag
protection: storageMeta?.protection, protection: storageMeta?.protection,
isCeph: storageMeta?.isCeph, isCeph: storageMeta?.isCeph,
isZfs: storageMeta?.isZfs, isZfs: storageMeta?.isZfs,
zfsPool: platformData.zfsPool, zfsPool: storageMeta?.zfsPool ?? platformData.zfsPool,
arrayState: storageMeta?.arrayState, arrayState: storageMeta?.arrayState,
syncAction: storageMeta?.syncAction, syncAction: storageMeta?.syncAction,
syncProgress: storageMeta?.syncProgress, syncProgress: storageMeta?.syncProgress,
@@ -42,10 +42,19 @@ export type StoragePoolDetailChartTarget = {
resourceId: string | null; resourceId: string | null;
}; };
export type StoragePoolDetailZfsDevice = {
name: string;
type: string;
state: string;
errorSummary: string;
message: string;
};
export type StoragePoolDetailZfsSummary = { export type StoragePoolDetailZfsSummary = {
state: string; state: string;
scan: string; scan: string;
errorSummary: string | null; errorSummary: string | null;
devices: StoragePoolDetailZfsDevice[];
}; };
export const STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS: readonly { export const STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS: readonly {
@@ -199,6 +208,22 @@ export function buildStoragePoolDetailTopologyRows(
return rows; return rows;
} }
export function getZfsDeviceErrorSummary(
readErrors: number,
writeErrors: number,
checksumErrors: number,
): string {
if (readErrors <= 0 && writeErrors <= 0 && checksumErrors <= 0) return '';
return `${readErrors}R/${writeErrors}W/${checksumErrors}C errors`;
}
export function getZfsDeviceStateTextClass(state: string): string {
const normalized = (state || '').trim().toUpperCase();
if (!normalized || normalized === 'ONLINE') return 'text-muted';
if (normalized === 'DEGRADED') return 'font-medium text-amber-700 dark:text-amber-300';
return 'font-medium text-red-700 dark:text-red-300';
}
export function buildStoragePoolDetailZfsSummary( export function buildStoragePoolDetailZfsSummary(
record: StorageRecord, record: StorageRecord,
): StoragePoolDetailZfsSummary | null { ): StoragePoolDetailZfsSummary | null {
@@ -213,6 +238,17 @@ export function buildStoragePoolDetailZfsSummary(
errorSummary: hasErrors errorSummary: hasErrors
? getZfsErrorSummary(pool.readErrors, pool.writeErrors, pool.checksumErrors) ? getZfsErrorSummary(pool.readErrors, pool.writeErrors, pool.checksumErrors)
: null, : null,
devices: (pool.devices || []).map((device) => ({
name: device.name,
type: device.type || '',
state: device.state || '',
errorSummary: getZfsDeviceErrorSummary(
device.readErrors || 0,
device.writeErrors || 0,
device.checksumErrors || 0,
),
message: (device.message || '').trim(),
})),
}; };
} }
@@ -319,6 +319,7 @@ type APIResource = {
nodes?: string[]; nodes?: string[];
pool?: string; pool?: string;
path?: string; path?: string;
zfsPool?: unknown;
zfsPoolState?: string; zfsPoolState?: string;
zfsReadErrors?: number; zfsReadErrors?: number;
zfsWriteErrors?: number; zfsWriteErrors?: number;
+2
View File
@@ -20,6 +20,7 @@ import type {
PBSPruneJob, PBSPruneJob,
PBSSyncJob, PBSSyncJob,
PBSVerifyJob, PBSVerifyJob,
ZFSPool,
} from '@/types/api'; } from '@/types/api';
import { import {
PLATFORM_TYPE_KEYS as GENERATED_PLATFORM_TYPE_KEYS, PLATFORM_TYPE_KEYS as GENERATED_PLATFORM_TYPE_KEYS,
@@ -385,6 +386,7 @@ export interface ResourceStorageMeta {
nodes?: string[]; nodes?: string[];
pool?: string; pool?: string;
path?: string; path?: string;
zfsPool?: ZFSPool;
zfsPoolState?: string; zfsPoolState?: string;
zfsReadErrors?: number; zfsReadErrors?: number;
zfsWriteErrors?: number; zfsWriteErrors?: number;
+3
View File
@@ -6018,6 +6018,9 @@ func monitorStoragePlatformData(resource unifiedresources.Resource, platformID s
payload["zfsReadErrors"] = storage.ZFSReadErrors payload["zfsReadErrors"] = storage.ZFSReadErrors
payload["zfsWriteErrors"] = storage.ZFSWriteErrors payload["zfsWriteErrors"] = storage.ZFSWriteErrors
payload["zfsChecksumErrors"] = storage.ZFSChecksumErrors payload["zfsChecksumErrors"] = storage.ZFSChecksumErrors
if storage.ZFSPool != nil {
payload["zfsPool"] = storage.ZFSPool
}
return payload return payload
} }
+4
View File
@@ -1610,7 +1610,10 @@ func resourceFromStorage(storage models.Storage) (Resource, ResourceIdentity) {
zfsPoolState := "" zfsPoolState := ""
var zfsReadErrors, zfsWriteErrors, zfsChecksumErrors int64 var zfsReadErrors, zfsWriteErrors, zfsChecksumErrors int64
var zfsPool *models.ZFSPool
if storage.ZFSPool != nil { if storage.ZFSPool != nil {
normalized := storage.ZFSPool.NormalizeCollections()
zfsPool = &normalized
zfsPoolState = strings.TrimSpace(storage.ZFSPool.State) zfsPoolState = strings.TrimSpace(storage.ZFSPool.State)
zfsReadErrors = storage.ZFSPool.ReadErrors zfsReadErrors = storage.ZFSPool.ReadErrors
zfsWriteErrors = storage.ZFSPool.WriteErrors zfsWriteErrors = storage.ZFSPool.WriteErrors
@@ -1641,6 +1644,7 @@ func resourceFromStorage(storage models.Storage) (Resource, ResourceIdentity) {
Nodes: append([]string(nil), storage.Nodes...), Nodes: append([]string(nil), storage.Nodes...),
Pool: storage.Pool, Pool: storage.Pool,
Path: storage.Path, Path: storage.Path,
ZFSPool: zfsPool,
ZFSPoolState: zfsPoolState, ZFSPoolState: zfsPoolState,
ZFSReadErrors: zfsReadErrors, ZFSReadErrors: zfsReadErrors,
ZFSWriteErrors: zfsWriteErrors, ZFSWriteErrors: zfsWriteErrors,
+22 -1
View File
@@ -310,7 +310,12 @@ func TestResourceFromStorageDetectsZFSMetadata(t *testing.T) {
Active: true, Active: true,
ZFSPool: &models.ZFSPool{ ZFSPool: &models.ZFSPool{
Name: "rpool", Name: "rpool",
State: "ONLINE", State: "DEGRADED",
Scan: "resilver in progress since Thu Jun 11 09:00:00 2026",
Devices: []models.ZFSDevice{
{Name: "sda2", Type: "disk", State: "ONLINE"},
{Name: "sdb2", Type: "disk", State: "FAULTED", ReadErrors: 3},
},
}, },
} }
@@ -327,6 +332,22 @@ func TestResourceFromStorageDetectsZFSMetadata(t *testing.T) {
if resource.Storage.Shared { if resource.Storage.Shared {
t.Fatalf("expected shared=false") t.Fatalf("expected shared=false")
} }
pool := resource.Storage.ZFSPool
if pool == nil {
t.Fatal("expected full ZFS pool payload on storage metadata")
}
if pool.Scan != storage.ZFSPool.Scan {
t.Fatalf("zfs scan = %q, want %q", pool.Scan, storage.ZFSPool.Scan)
}
if len(pool.Devices) != 2 {
t.Fatalf("zfs devices = %d, want 2", len(pool.Devices))
}
if pool.Devices[1].State != "FAULTED" || pool.Devices[1].ReadErrors != 3 {
t.Fatalf("zfs device passthrough mismatch: %+v", pool.Devices[1])
}
if pool == storage.ZFSPool {
t.Fatal("expected ZFS pool to be copied, not aliased to the source pointer")
}
} }
func TestResourceFromStorageDerivesZFSTopologyRisk(t *testing.T) { func TestResourceFromStorageDerivesZFSTopologyRisk(t *testing.T) {
@@ -377,6 +377,43 @@ func TestCanonicalStorageMetadataPreservesBackingPoolField(t *testing.T) {
} }
} }
func TestCanonicalStorageMetadataCarriesFullZFSPoolReport(t *testing.T) {
requiredSnippets := map[string][]string{
"types.go": {
"ZFSPool *models.ZFSPool `json:\"zfsPool,omitempty\"`",
},
"adapters.go": {
"normalized := storage.ZFSPool.NormalizeCollections()",
"ZFSPool: zfsPool,",
},
filepath.Join("..", "monitoring", "monitor.go"): {
"payload[\"zfsPool\"] = storage.ZFSPool",
},
filepath.Join("..", "..", "frontend-modern", "src", "types", "resource.ts"): {
"zfsPool?: ZFSPool;",
},
filepath.Join("..", "..", "frontend-modern", "src", "hooks", "useUnifiedResources.ts"): {
"zfsPool?: unknown;",
},
filepath.Join("..", "..", "frontend-modern", "src", "features", "storageBackups", "storageAdapters.ts"): {
"zfsPool: storageMeta?.zfsPool ?? platformData.zfsPool,",
},
}
for path, snippets := range requiredSnippets {
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("failed to read %s: %v", path, err)
}
source := string(data)
for _, snippet := range snippets {
if !strings.Contains(source, snippet) {
t.Fatalf("%s must contain %q", path, snippet)
}
}
}
}
func TestCephPoolsProjectThroughCanonicalStoragePath(t *testing.T) { func TestCephPoolsProjectThroughCanonicalStoragePath(t *testing.T) {
requiredSnippets := map[string][]string{ requiredSnippets := map[string][]string{
filepath.Join("..", "models", "models.go"): { filepath.Join("..", "models", "models.go"): {
+8 -4
View File
@@ -390,10 +390,14 @@ type StorageMeta struct {
Path string `json:"path,omitempty"` // local mount path on the node Path string `json:"path,omitempty"` // local mount path on the node
// ZFS metadata (when IsZFS is true and the source provides details). // ZFS metadata (when IsZFS is true and the source provides details).
ZFSPoolState string `json:"zfsPoolState,omitempty"` // ZFSPool carries the full pool report (scan status, per-device states
ZFSReadErrors int64 `json:"zfsReadErrors,omitempty"` // and error counts); the scalar fields below are the flattened summary
ZFSWriteErrors int64 `json:"zfsWriteErrors,omitempty"` // kept for consumers that only need pool-level state.
ZFSChecksumErrors int64 `json:"zfsChecksumErrors,omitempty"` ZFSPool *models.ZFSPool `json:"zfsPool,omitempty"`
ZFSPoolState string `json:"zfsPoolState,omitempty"`
ZFSReadErrors int64 `json:"zfsReadErrors,omitempty"`
ZFSWriteErrors int64 `json:"zfsWriteErrors,omitempty"`
ZFSChecksumErrors int64 `json:"zfsChecksumErrors,omitempty"`
// Unraid metadata (when Platform is "unraid"). // Unraid metadata (when Platform is "unraid").
ArrayState string `json:"arrayState,omitempty"` ArrayState string `json:"arrayState,omitempty"`