feat(storage): surface Proxmox ZFS datasets

This commit is contained in:
courtmanr@gmail.com
2026-07-30 16:45:13 +01:00
parent c700454b18
commit 218661a396
26 changed files with 787 additions and 18 deletions
+3
View File
@@ -9,12 +9,15 @@ Pulse automatically detects and monitors ZFS pools on your Proxmox nodes.
* **Auto-Detection**: No configuration needed.
* **Health Status**: Tracks `ONLINE`, `DEGRADED`, and `FAULTED` states.
* **Error Tracking**: Monitors read, write, and checksum errors.
* **Dataset Inventory**: With a Unified Agent on the node, expanded pool details list ZFS filesystems and zvols with used, available, referenced, and mountpoint information.
* **Alerts**: Notifies you of degraded pools or failing devices.
## ⚙️ Requirements
The Pulse user needs `Sys.Audit` permission on `/nodes/{node}/disks` (included in the standard Pulse role).
Pool health and device status come from the Proxmox API. Dataset inventory additionally requires a Unified Agent on the Proxmox node with read access to the local `zfs` command. If the command is unavailable, Pulse falls back to the mounted ZFS datasets visible to the agent.
```bash
# Grant permission manually if needed
pveum acl modify /nodes -user pulse-monitor@pve -role PVEAuditor
@@ -5624,3 +5624,14 @@ tenant-local `reportBranding` settings record, but it must not enumerate
agents, resolve install tokens, mutate agent state, or reuse agent lifecycle
authority. Agent setup and reporting remain independent of whether an
operator has configured a custom application name or logo.
### ZFS dataset inventory is read-only host evidence
The Unified Agent host report may attach a bounded `zfsDatasets` collection to
each detected ZFS pool disk. The collector executes only the fixed read-only
`zfs list` projection for pools it already discovered, admits only filesystem
and volume rows beneath those pools, and bounds command output, row counts, and
text fields. Dataset collection does not broaden registration, token,
configuration, update, or command authority. Servers validate and copy the
optional evidence during normal authenticated host-report ingest; agents that
do not send it remain wire-compatible.
@@ -8730,3 +8730,13 @@ invalid or unknown image without discarding a valid display name.
`internal/api/runtime_branding_test.go` and
`internal/api/route_inventory_test.go` pin the entitlement, shape, image, and
route contracts.
### ZFS dataset evidence is an optional nested host-report contract
The existing authenticated host-report payload may include `zfsDatasets` on a
disk row. Each entry exposes only `name`, optional `type` and `mountpoint`, and
non-negative `usedBytes`, `availableBytes`, and `referencedBytes`. The field is
optional and additive: older agents omit it, older stored hosts may have no
dataset collection, and provider ZFS pool responses omit `datasets` when no
linked host evidence exists. Ingest trims text, rejects invalid counters, and
bounds rows and field lengths before the data reaches runtime models.
@@ -5521,3 +5521,12 @@ It must not create a page-local branding cache or render configured values
when `white_label` is unavailable. Focused proofs live in
`BrandingSettingsCard.test.tsx`, `AppLayout.test.tsx`, and
`stores/__tests__/systemSettings.test.ts`.
### ZFS datasets extend the existing storage detail primitive
`StoragePoolDetail` remains the owner of the expandable ZFS pool presentation.
When a pool carries optional `datasets`, its already-expanded detail region
renders the canonical dataset name and formatted used, available, referenced,
and mountpoint values. Empty dataset collections add no new panel, route, or
navigation state. The presentation mapper owns byte formatting and missing
mountpoint fallback so components do not reinterpret provider data.
@@ -2605,3 +2605,14 @@ the same restriction, so both paths now agree.
unrestricted storage still appears on every node, that a globally disabled
unrestricted storage still surfaces as disabled, and that the shared-storage
node list excludes non-member nodes.
### Linked host evidence enriches provider-owned ZFS pools
The host collector supplements mounted filesystem facts with a bounded,
read-only `zfs list` query for filesystems and zvols under already-discovered
pools. Authenticated report ingest validates that optional evidence and stores
it on the canonical host model. During Proxmox storage polling, a node's linked
host dataset evidence is copied onto the matching provider-owned ZFS pool;
provider health, scan, device, and error fields remain authoritative. When the
provider cannot return pool detail, monitoring may synthesize only a minimal
`UNKNOWN` pool so valid dataset evidence is still inspectable.
@@ -4827,3 +4827,10 @@ configuration export/restore therefore carries the display name and bounded
inline logo exactly as it already carries report branding. Runtime reads
through `/api/runtime/branding` are read-only and entitlement-filtered; they do
not mutate, relocate, or synthesize persisted branding state.
ZFS dataset inventory is likewise read-only runtime evidence, not recovery or
storage-configuration state. The agent does not mount, create, snapshot,
destroy, or alter datasets, and the server does not persist a second provider
inventory. Optional dataset rows travel with the current host snapshot and
only enrich the matching Proxmox pool presentation; backup and restore
authority, storage mutation paths, and recovery semantics are unchanged.
@@ -2014,6 +2014,14 @@ through the canonical resource model, but unified-resource consumers must not
reintroduce removed workload aliases or feature-local resource-type shims just
to satisfy one table, drawer, or badge surface.
ZFS dataset inventory remains nested evidence rather than a new unified
resource kind. The canonical host snapshot owns a deep-copied list of reported
pool datasets, and Proxmox storage polling copies that list into the matching
pool detail without changing resource identity, counts, parentage, or source
selection. Deep-copy boundaries must clone both pool and dataset slices so
state snapshots and unified-resource consumers cannot alias mutable ingest
buffers.
### Alert-intent identity and availability evidence projection
The monitor adapter may expose two optional read capabilities to the alert
@@ -182,6 +182,56 @@ export const StoragePoolDetail: Component<StoragePoolDetailProps> = (props) => {
</For>
</div>
</Show>
<Show when={zfsSummary()!.datasets.length > 0}>
<div class={`${STORAGE_DETAIL_FULL_WIDTH_ROW_CLASS} space-y-2 pt-2`}>
<div class="text-xs font-medium text-base-content">
Datasets ({zfsSummary()!.datasets.length})
</div>
<div class="overflow-x-auto rounded border border-border">
<table class="w-full min-w-[34rem] text-left text-xs">
<thead class="bg-surface-alt text-muted">
<tr>
<th class="px-2 py-1.5 font-medium">Dataset</th>
<th class="px-2 py-1.5 font-medium">Used</th>
<th class="px-2 py-1.5 font-medium">Available</th>
<th class="px-2 py-1.5 font-medium">Referenced</th>
<th class="px-2 py-1.5 font-medium">Mountpoint</th>
</tr>
</thead>
<tbody>
<For each={zfsSummary()!.datasets}>
{(dataset) => (
<tr class="border-t border-border">
<td
class="max-w-64 truncate px-2 py-1.5 font-mono"
title={dataset.name}
>
{dataset.name}
</td>
<td class="whitespace-nowrap px-2 py-1.5 font-mono">
{dataset.usedLabel}
</td>
<td class="whitespace-nowrap px-2 py-1.5 font-mono">
{dataset.availableLabel}
</td>
<td class="whitespace-nowrap px-2 py-1.5 font-mono">
{dataset.referencedLabel}
</td>
<td
class="max-w-48 truncate px-2 py-1.5 text-muted"
title={dataset.mountpoint}
>
{dataset.mountpoint ||
(dataset.type === 'volume' ? 'zvol' : '—')}
</td>
</tr>
)}
</For>
</tbody>
</table>
</div>
</div>
</Show>
</div>
</div>
</Show>
@@ -198,4 +198,48 @@ describe('StoragePoolDetail', () => {
expect(screen.getByText('spun down')).toBeInTheDocument();
expect(screen.getByText('16 errors')).toBeInTheDocument();
});
it('renders linked host-agent ZFS datasets inside the expanded pool detail', () => {
render(() => (
<table>
<tbody>
<StoragePoolDetail
record={makeRecord({
details: {
type: 'zfspool',
zfsPool: {
name: 'tank',
state: 'ONLINE',
status: 'Healthy',
scan: 'none',
readErrors: 0,
writeErrors: 0,
checksumErrors: 0,
devices: [],
datasets: [
{
name: 'tank/apps',
type: 'filesystem',
mountpoint: '/tank/apps',
usedBytes: 1_073_741_824,
availableBytes: 2_147_483_648,
referencedBytes: 536_870_912,
},
],
},
},
})}
physicalDisks={[]}
summarySeriesId="tank"
/>
</tbody>
</table>
));
expect(screen.getByText('Datasets (1)')).toBeInTheDocument();
expect(screen.getByText('tank/apps')).toBeInTheDocument();
expect(screen.getByText('1.00 GB')).toBeInTheDocument();
expect(screen.getByText('2.00 GB')).toBeInTheDocument();
expect(screen.getByText('/tank/apps')).toBeInTheDocument();
});
});
@@ -37,6 +37,16 @@ const buildRecord = (overrides: Partial<StorageRecord> = {}): StorageRecord => (
readErrors: 1,
writeErrors: 2,
checksumErrors: 3,
datasets: [
{
name: 'tank/apps',
type: 'filesystem',
mountpoint: '/tank/apps',
usedBytes: 1024,
availableBytes: 2048,
referencedBytes: 512,
},
],
devices: [
{ name: 'sda', type: 'disk', state: 'ONLINE' },
{
@@ -134,6 +144,16 @@ describe('storagePoolDetailPresentation', () => {
message: 'too many errors',
},
],
datasets: [
{
name: 'tank/apps',
type: 'filesystem',
mountpoint: '/tank/apps',
usedLabel: '1.00 KB',
availableLabel: '2.00 KB',
referencedLabel: '512 B',
},
],
});
expect(getStoragePoolLinkedDisks(record, [buildDisk()])).toEqual([
{
@@ -57,6 +57,14 @@ export type StoragePoolDetailZfsSummary = {
scan: string;
errorSummary: string | null;
devices: StoragePoolDetailZfsDevice[];
datasets: Array<{
name: string;
type: string;
mountpoint: string;
usedLabel: string;
availableLabel: string;
referencedLabel: string;
}>;
};
export const STORAGE_POOL_DETAIL_HISTORY_RANGE_OPTIONS: readonly {
@@ -252,6 +260,14 @@ export function buildStoragePoolDetailZfsSummary(
),
message: (device.message || '').trim(),
})),
datasets: (pool.datasets || []).map((dataset) => ({
name: dataset.name,
type: dataset.type || 'filesystem',
mountpoint: dataset.mountpoint || '',
usedLabel: formatBytes(dataset.usedBytes || 0),
availableLabel: formatBytes(dataset.availableBytes || 0),
referencedLabel: formatBytes(dataset.referencedBytes || 0),
})),
};
}
+10
View File
@@ -839,6 +839,16 @@ export interface ZFSPool {
writeErrors: number;
checksumErrors: number;
devices: ZFSDevice[];
datasets?: ZFSDataset[];
}
export interface ZFSDataset {
name: string;
type?: 'filesystem' | 'volume' | string;
mountpoint?: string;
usedBytes?: number;
availableBytes?: number;
referencedBytes?: number;
}
export interface ZFSScan {
+1
View File
@@ -325,6 +325,7 @@ func collectDisks(ctx context.Context, diskExclude []string) []agentshost.Disk {
}
zfsDisks := summarizeZFSPools(ctx, zfsDatasets)
enrichZFSPoolDisksWithDatasets(ctx, zfsDisks, zfsDatasets)
log.Debug().Int("zfsDatasets", len(zfsDatasets)).Int("zfsDisks", len(zfsDisks)).Int("regularDisks", len(disks)).Msg("disk: collection summary")
disks = append(disks, zfsDisks...)
+200 -1
View File
@@ -17,7 +17,13 @@ import (
"github.com/rs/zerolog/log"
)
const maxZpoolCommandOutputSize = 1 << 20 // 1 MiB
const (
maxZpoolCommandOutputSize = 1 << 20 // 1 MiB
maxZFSCommandOutputSize = 4 << 20 // 4 MiB
maxZFSDatasetsPerPool = 128
maxZFSDatasetNameBytes = 512
maxZFSMountpointBytes = 1024
)
var errZpoolCommandOutputTooLarge = errors.New("zpool command output exceeded limit")
@@ -39,9 +45,13 @@ type zfsDatasetUsage struct {
}
var queryZpoolStats = fetchZpoolStats
var queryZFSDatasets = fetchZFSDatasets
var zpoolLookPath = exec.LookPath
var zpoolStat = os.Stat
var zpoolCommandRunner = runZpoolCommand
var zfsLookPath = exec.LookPath
var zfsStat = os.Stat
var zfsCommandRunner = runZFSCommand
type limitedBuffer struct {
buf bytes.Buffer
@@ -205,6 +215,66 @@ func fallbackZFSDisks(bestDatasets map[string]zfsDatasetUsage, mountpoints map[s
return disks
}
func enrichZFSPoolDisksWithDatasets(
ctx context.Context,
disks []agentshost.Disk,
mounted []zfsDatasetUsage,
) {
if len(disks) == 0 {
return
}
pools := make([]string, 0, len(disks))
for _, disk := range disks {
if pool, ok := normalizeZFSPoolName(disk.Device); ok {
pools = append(pools, pool)
}
}
datasets, err := queryZFSDatasets(ctx, pools)
if err != nil {
log.Debug().Err(err).Msg("zfs: dataset inventory unavailable, using mounted datasets")
datasets = mountedZFSDatasets(mounted)
}
for index := range disks {
disks[index].ZFSDatasets = append(
[]agentshost.ZFSDataset(nil),
datasets[disks[index].Device]...,
)
}
}
func mountedZFSDatasets(mounted []zfsDatasetUsage) map[string][]agentshost.ZFSDataset {
result := make(map[string][]agentshost.ZFSDataset)
seen := make(map[string]struct{})
for _, dataset := range mounted {
pool, ok := normalizeZFSPoolName(dataset.Pool)
name := strings.TrimSpace(dataset.Dataset)
mountpoint := strings.TrimSpace(dataset.Mountpoint)
if !ok || name == pool || !strings.HasPrefix(name, pool+"/") ||
len(name) > maxZFSDatasetNameBytes || len(mountpoint) > maxZFSMountpointBytes ||
len(result[pool]) >= maxZFSDatasetsPerPool {
continue
}
if _, duplicate := seen[name]; duplicate {
continue
}
seen[name] = struct{}{}
result[pool] = append(result[pool], agentshost.ZFSDataset{
Name: name,
Type: "filesystem",
Mountpoint: mountpoint,
UsedBytes: int64(dataset.Used),
AvailableBytes: int64(dataset.Free),
})
}
for pool := range result {
sort.Slice(result[pool], func(i, j int) bool {
return result[pool][i].Name < result[pool][j].Name
})
}
return result
}
// commonZpoolPaths lists common locations for the zpool binary.
// TrueNAS SCALE, FreeBSD, and various Linux distributions may install
// zpool in different locations that might not be in the agent's PATH.
@@ -218,6 +288,15 @@ var commonZpoolPaths = []string{
"/usr/bin/zpool", // Some distributions
}
var commonZFSPaths = []string{
"/usr/sbin/zfs",
"/sbin/zfs",
"/usr/local/sbin/zfs",
"/usr/local/bin/zfs",
"/opt/zfs/bin/zfs",
"/usr/bin/zfs",
}
// findZpool returns the path to the zpool binary by preferring known absolute
// locations first, then falling back to PATH lookup.
func findZpool() (string, error) {
@@ -249,6 +328,126 @@ func findZpool() (string, error) {
return path, nil
}
func findZFS() (string, error) {
for _, path := range commonZFSPaths {
if _, err := zfsStat(path); err == nil {
return path, nil
}
}
path, err := zfsLookPath("zfs")
if err != nil {
return "", fmt.Errorf("zfs binary not found in PATH or common locations")
}
path = filepath.Clean(path)
if !filepath.IsAbs(path) {
return "", fmt.Errorf("zfs path is not absolute: %q", path)
}
if _, err := zfsStat(path); err != nil {
return "", fmt.Errorf("zfs path unavailable: %w", err)
}
return path, nil
}
func fetchZFSDatasets(
ctx context.Context,
pools []string,
) (map[string][]agentshost.ZFSDataset, error) {
pools = filterValidZFSPoolNames(pools)
if len(pools) == 0 {
return nil, nil
}
path, err := findZFS()
if err != nil {
return nil, err
}
args := []string{
"list", "-Hp", "-t", "filesystem,volume",
"-o", "name,type,used,available,referenced,mountpoint",
}
args = append(args, pools...)
output, stderr, err := zfsCommandRunner(ctx, path, args...)
if err != nil {
if errors.Is(err, errZpoolCommandOutputTooLarge) {
return nil, fmt.Errorf("zfs list output exceeded %d bytes", maxZFSCommandOutputSize)
}
return nil, fmt.Errorf("zfs list failed (stderr %d bytes): %w", len(stderr), err)
}
return parseZFSList(output, pools)
}
func runZFSCommand(ctx context.Context, name string, args ...string) ([]byte, []byte, error) {
cmd := exec.CommandContext(ctx, name, args...)
stdout := limitedBuffer{maxBytes: maxZFSCommandOutputSize}
stderr := limitedBuffer{maxBytes: maxZFSCommandOutputSize}
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
if stdout.exceeded || stderr.exceeded {
return stdout.Bytes(), stderr.Bytes(), errZpoolCommandOutputTooLarge
}
return stdout.Bytes(), stderr.Bytes(), err
}
func parseZFSList(
output []byte,
allowedPools []string,
) (map[string][]agentshost.ZFSDataset, error) {
allowed := make(map[string]struct{}, len(allowedPools))
for _, pool := range filterValidZFSPoolNames(allowedPools) {
allowed[pool] = struct{}{}
}
result := make(map[string][]agentshost.ZFSDataset)
scanner := bufio.NewScanner(bytes.NewReader(output))
scanner.Buffer(make([]byte, 64*1024), maxZFSCommandOutputSize)
for scanner.Scan() {
fields := strings.Split(scanner.Text(), "\t")
if len(fields) != 6 {
continue
}
name := strings.TrimSpace(fields[0])
pool := zfsPoolFromDevice(name)
if _, ok := allowed[pool]; !ok || name == pool || !strings.HasPrefix(name, pool+"/") ||
len(name) > maxZFSDatasetNameBytes || len(result[pool]) >= maxZFSDatasetsPerPool {
continue
}
datasetType := strings.ToLower(strings.TrimSpace(fields[1]))
if datasetType != "filesystem" && datasetType != "volume" {
continue
}
used, usedErr := strconv.ParseInt(fields[2], 10, 64)
available, availableErr := strconv.ParseInt(fields[3], 10, 64)
referenced, referencedErr := strconv.ParseInt(fields[4], 10, 64)
if usedErr != nil || availableErr != nil || referencedErr != nil ||
used < 0 || available < 0 || referenced < 0 {
continue
}
mountpoint := strings.TrimSpace(fields[5])
if mountpoint == "-" || mountpoint == "none" || mountpoint == "legacy" {
mountpoint = ""
}
if len(mountpoint) > maxZFSMountpointBytes {
continue
}
result[pool] = append(result[pool], agentshost.ZFSDataset{
Name: name,
Type: datasetType,
Mountpoint: mountpoint,
UsedBytes: used,
AvailableBytes: available,
ReferencedBytes: referenced,
})
}
if err := scanner.Err(); err != nil {
return nil, fmt.Errorf("scan zfs list output: %w", err)
}
for pool := range result {
sort.Slice(result[pool], func(i, j int) bool {
return result[pool][i].Name < result[pool][j].Name
})
}
return result, nil
}
func fetchZpoolStats(ctx context.Context, pools []string) (map[string]zpoolStats, error) {
if len(pools) == 0 {
return nil, nil
+76
View File
@@ -2,8 +2,12 @@ package hostmetrics
import (
"context"
"fmt"
"os"
"strings"
"testing"
agentshost "github.com/rcourtman/pulse-go-rewrite/pkg/agents/host"
)
func TestSummarizeZFSPoolsUsesZpoolStats(t *testing.T) {
@@ -545,6 +549,78 @@ func TestParseZpoolList(t *testing.T) {
}
}
func TestParseZFSListPreservesDatasetHierarchyAndUsage(t *testing.T) {
output := []byte(
"rpool\tfilesystem\t100\t900\t75\t/rpool\n" +
"rpool/ROOT\tfilesystem\t200\t800\t150\tlegacy\n" +
"rpool/data/vm-100-disk-0\tvolume\t300\t700\t300\t-\n" +
"other/apps\tfilesystem\t400\t600\t350\t/other/apps\n",
)
got, err := parseZFSList(output, []string{"rpool"})
if err != nil {
t.Fatal(err)
}
if len(got["rpool"]) != 2 {
t.Fatalf("datasets = %#v, want two descendants", got["rpool"])
}
if got["rpool"][0] != (agentshost.ZFSDataset{
Name: "rpool/ROOT",
Type: "filesystem",
UsedBytes: 200,
AvailableBytes: 800,
ReferencedBytes: 150,
}) {
t.Fatalf("filesystem dataset = %#v", got["rpool"][0])
}
if got["rpool"][1].Type != "volume" || got["rpool"][1].Mountpoint != "" {
t.Fatalf("volume dataset = %#v", got["rpool"][1])
}
}
func TestParseZFSListBoundsDatasetInventory(t *testing.T) {
var output strings.Builder
for i := 0; i < maxZFSDatasetsPerPool+2; i++ {
_, _ = fmt.Fprintf(
&output,
"tank/data-%03d\tfilesystem\t1\t2\t1\t/tank/data-%03d\n",
i,
i,
)
}
_, _ = fmt.Fprintf(
&output,
"tank/%s\tfilesystem\t1\t2\t1\t/tank/too-long\n",
strings.Repeat("x", maxZFSDatasetNameBytes),
)
got, err := parseZFSList([]byte(output.String()), []string{"tank"})
if err != nil {
t.Fatal(err)
}
if len(got["tank"]) != maxZFSDatasetsPerPool {
t.Fatalf("dataset count = %d, want %d", len(got["tank"]), maxZFSDatasetsPerPool)
}
}
func TestEnrichZFSPoolDisksWithDatasetsFallsBackToMountedInventory(t *testing.T) {
originalQuery := queryZFSDatasets
t.Cleanup(func() { queryZFSDatasets = originalQuery })
queryZFSDatasets = func(context.Context, []string) (map[string][]agentshost.ZFSDataset, error) {
return nil, context.DeadlineExceeded
}
disks := []agentshost.Disk{{Device: "tank", Type: "zfs"}}
enrichZFSPoolDisksWithDatasets(context.Background(), disks, []zfsDatasetUsage{
{Pool: "tank", Dataset: "tank", Mountpoint: "/tank"},
{Pool: "tank", Dataset: "tank/apps", Mountpoint: "/tank/apps", Used: 25, Free: 75},
})
if len(disks[0].ZFSDatasets) != 1 || disks[0].ZFSDatasets[0].Name != "tank/apps" {
t.Fatalf("fallback datasets = %#v", disks[0].ZFSDatasets)
}
}
func TestUniqueZFSPools(t *testing.T) {
tests := []struct {
name string
+6
View File
@@ -377,6 +377,11 @@ func cloneHost(src Host) Host {
dest := src
dest.LoadAverage = append([]float64(nil), src.LoadAverage...)
dest.Disks = append([]Disk(nil), src.Disks...)
dest.ZFSPools = make([]HostZFSPool, len(src.ZFSPools))
for i := range src.ZFSPools {
dest.ZFSPools[i] = src.ZFSPools[i]
dest.ZFSPools[i].Datasets = append([]ZFSDataset(nil), src.ZFSPools[i].Datasets...)
}
dest.DiskIO = append([]DiskIO(nil), src.DiskIO...)
dest.NetworkInterfaces = cloneHostNetworkInterfaces(src.NetworkInterfaces)
dest.Sensors = cloneHostSensorSummary(src.Sensors)
@@ -1084,6 +1089,7 @@ func cloneZFSPool(src *ZFSPool) *ZFSPool {
}
dest := *src
dest.Devices = append([]ZFSDevice(nil), src.Devices...)
dest.Datasets = append([]ZFSDataset(nil), src.Datasets...)
normalized := dest.NormalizeCollections()
return &normalized
}
+19
View File
@@ -24,6 +24,25 @@ func TestCloneBoolPtr_Value(t *testing.T) {
}
}
func TestCloneHostAndZFSPoolIsolateZFSDatasets(t *testing.T) {
host := Host{ZFSPools: []HostZFSPool{{
Name: "tank",
Datasets: []ZFSDataset{{Name: "tank/apps", UsedBytes: 100}},
}}}
hostClone := cloneHost(host)
hostClone.ZFSPools[0].Datasets[0].Name = "mutated"
if host.ZFSPools[0].Datasets[0].Name != "tank/apps" {
t.Fatal("host clone aliased zfs datasets")
}
pool := &ZFSPool{Name: "tank", Datasets: []ZFSDataset{{Name: "tank/apps"}}}
poolClone := cloneZFSPool(pool)
poolClone.Datasets[0].Name = "mutated"
if pool.Datasets[0].Name != "tank/apps" {
t.Fatal("pool clone aliased zfs datasets")
}
}
func TestCloneDockerContainer_PreservesIndependentOOMEvidence(t *testing.T) {
oomKilled := false
src := DockerContainer{ID: "container-1", OOMKilled: &oomKilled}
+40 -9
View File
@@ -297,6 +297,7 @@ type Host struct {
Memory Memory `json:"memory"`
LoadAverage []float64 `json:"loadAverage,omitempty"`
Disks []Disk `json:"disks,omitempty"`
ZFSPools []HostZFSPool `json:"zfsPools,omitempty"`
DiskIO []DiskIO `json:"diskIO,omitempty"`
NetworkInterfaces []HostNetworkInterface `json:"networkInterfaces,omitempty"`
Sensors HostSensorSummary `json:"sensors,omitempty"`
@@ -407,6 +408,12 @@ func (h Host) NormalizeCollections() Host {
if h.Disks == nil {
h.Disks = []Disk{}
}
if h.ZFSPools == nil {
h.ZFSPools = []HostZFSPool{}
}
for i := range h.ZFSPools {
h.ZFSPools[i] = h.ZFSPools[i].NormalizeCollections()
}
if h.DiskIO == nil {
h.DiskIO = []DiskIO{}
}
@@ -2385,15 +2392,16 @@ func (s Storage) NormalizeCollections() Storage {
// ZFSPool represents a ZFS pool with health and error information
type ZFSPool struct {
Name string `json:"name"`
State string `json:"state"` // ONLINE, DEGRADED, FAULTED, OFFLINE, REMOVED, UNAVAIL
Status string `json:"status"` // Healthy, Degraded, Faulted, etc.
Scan string `json:"scan"` // Current scan status (scrub, resilver, none)
ScanDetails *ZFSScan `json:"scanDetails,omitempty"`
ReadErrors int64 `json:"readErrors"`
WriteErrors int64 `json:"writeErrors"`
ChecksumErrors int64 `json:"checksumErrors"`
Devices []ZFSDevice `json:"devices"`
Name string `json:"name"`
State string `json:"state"` // ONLINE, DEGRADED, FAULTED, OFFLINE, REMOVED, UNAVAIL
Status string `json:"status"` // Healthy, Degraded, Faulted, etc.
Scan string `json:"scan"` // Current scan status (scrub, resilver, none)
ScanDetails *ZFSScan `json:"scanDetails,omitempty"`
ReadErrors int64 `json:"readErrors"`
WriteErrors int64 `json:"writeErrors"`
ChecksumErrors int64 `json:"checksumErrors"`
Devices []ZFSDevice `json:"devices"`
Datasets []ZFSDataset `json:"datasets,omitempty"`
}
func (p ZFSPool) NormalizeCollections() ZFSPool {
@@ -2403,6 +2411,29 @@ func (p ZFSPool) NormalizeCollections() ZFSPool {
return p
}
// ZFSDataset is the read-only dataset capacity projection reported by a linked
// host agent and attached to the provider-owned pool.
type ZFSDataset struct {
Name string `json:"name"`
Type string `json:"type,omitempty"`
Mountpoint string `json:"mountpoint,omitempty"`
UsedBytes int64 `json:"usedBytes,omitempty"`
AvailableBytes int64 `json:"availableBytes,omitempty"`
ReferencedBytes int64 `json:"referencedBytes,omitempty"`
}
type HostZFSPool struct {
Name string `json:"name"`
Datasets []ZFSDataset `json:"datasets"`
}
func (p HostZFSPool) NormalizeCollections() HostZFSPool {
if p.Datasets == nil {
p.Datasets = []ZFSDataset{}
}
return p
}
// ZFSScan is the provider-native scrub or resilver observation for a pool.
// The legacy Scan string remains for compact consumers, while this shape keeps
// progress, terminal errors, and lifecycle timestamps machine-readable.
+57
View File
@@ -28,6 +28,12 @@ const hostContinuityRetention = 72 * time.Hour
const maxRetiredHostReportStreams = 8
const (
maxHostZFSDatasetsPerPool = 128
maxHostZFSDatasetNameBytes = 512
maxHostZFSMountpointBytes = 1024
)
func normalizeAgentMemory(total, used, free, cache int64, usage float64, swapTotal, swapUsed int64) models.Memory {
if total <= 0 {
return models.Memory{}
@@ -2571,6 +2577,7 @@ func (m *Monitor) ApplyHostReport(report agentshost.Report, tokenRecord *config.
)
disks := make([]models.Disk, 0, len(report.Disks))
hostZFSPools := hostZFSPoolsFromAgentDisks(report.Disks)
for _, disk := range report.Disks {
// Filter virtual/system filesystems and read-only filesystems to avoid cluttering
// the UI with tmpfs, devtmpfs, /dev, /run, /sys, docker overlay mounts, snap mounts,
@@ -2748,6 +2755,7 @@ func (m *Monitor) ApplyHostReport(report agentshost.Report, tokenRecord *config.
LoadAverage: append([]float64(nil), report.Host.LoadAverage...),
Memory: memory,
Disks: disks,
ZFSPools: hostZFSPools,
DiskIO: diskIO,
NetworkInterfaces: network,
Sensors: models.HostSensorSummary{
@@ -2792,6 +2800,9 @@ func (m *Monitor) ApplyHostReport(report agentshost.Report, tokenRecord *config.
if len(host.Disks) == 0 {
host.Disks = nil
}
if len(host.ZFSPools) == 0 {
host.ZFSPools = nil
}
if len(host.DiskIO) == 0 {
host.DiskIO = nil
}
@@ -3002,6 +3013,52 @@ func (m *Monitor) ApplyHostReport(report agentshost.Report, tokenRecord *config.
return host, nil
}
func hostZFSPoolsFromAgentDisks(disks []agentshost.Disk) []models.HostZFSPool {
pools := make([]models.HostZFSPool, 0)
for _, disk := range disks {
name := strings.TrimSpace(disk.Device)
if name == "" ||
!strings.EqualFold(strings.TrimSpace(disk.Type), "zfs") ||
len(disk.ZFSDatasets) == 0 {
continue
}
datasets := make([]models.ZFSDataset, 0, len(disk.ZFSDatasets))
for _, dataset := range disk.ZFSDatasets {
if len(datasets) >= maxHostZFSDatasetsPerPool {
break
}
datasetName := strings.TrimSpace(dataset.Name)
datasetType := strings.ToLower(strings.TrimSpace(dataset.Type))
if datasetType == "" {
datasetType = "filesystem"
}
mountpoint := strings.TrimSpace(dataset.Mountpoint)
if datasetName == "" ||
!strings.HasPrefix(datasetName, name+"/") ||
(datasetType != "filesystem" && datasetType != "volume") ||
len(datasetName) > maxHostZFSDatasetNameBytes ||
len(mountpoint) > maxHostZFSMountpointBytes ||
dataset.UsedBytes < 0 ||
dataset.AvailableBytes < 0 ||
dataset.ReferencedBytes < 0 {
continue
}
datasets = append(datasets, models.ZFSDataset{
Name: datasetName,
Type: datasetType,
Mountpoint: mountpoint,
UsedBytes: dataset.UsedBytes,
AvailableBytes: dataset.AvailableBytes,
ReferencedBytes: dataset.ReferencedBytes,
})
}
if len(datasets) > 0 {
pools = append(pools, models.HostZFSPool{Name: name, Datasets: datasets})
}
}
return pools
}
func convertHostPackageUpdateStatus(status *agentshost.PackageUpdateStatus, observedAt time.Time) *models.HostPackageUpdateStatus {
if status == nil {
return nil
@@ -21,6 +21,61 @@ import (
"github.com/rcourtman/pulse-go-rewrite/pkg/metrics"
)
func TestHostZFSPoolsFromAgentDisksPreservesDatasetFacts(t *testing.T) {
got := hostZFSPoolsFromAgentDisks([]agentshost.Disk{
{Device: "/", Type: "ext4"},
{
Device: "tank",
Type: "zfs",
ZFSDatasets: []agentshost.ZFSDataset{
{
Name: " tank/apps ",
Type: " filesystem ",
Mountpoint: " /tank/apps ",
UsedBytes: 100,
AvailableBytes: 200,
ReferencedBytes: 50,
},
},
},
})
if len(got) != 1 || got[0].Name != "tank" || len(got[0].Datasets) != 1 {
t.Fatalf("host zfs pools = %#v", got)
}
if got[0].Datasets[0].Name != "tank/apps" ||
got[0].Datasets[0].Mountpoint != "/tank/apps" {
t.Fatalf("dataset was not normalized: %#v", got[0].Datasets[0])
}
}
func TestHostZFSPoolsFromAgentDisksRejectsInvalidAndBoundsRows(t *testing.T) {
datasets := []agentshost.ZFSDataset{
{Name: "tank/negative", Type: "filesystem", UsedBytes: -1},
{Name: "other/wrong-pool", Type: "filesystem"},
{Name: strings.Repeat("x", maxHostZFSDatasetNameBytes+1), Type: "filesystem"},
{Name: "tank/long-mount", Type: "filesystem", Mountpoint: strings.Repeat("x", maxHostZFSMountpointBytes+1)},
}
for i := 0; i < maxHostZFSDatasetsPerPool+2; i++ {
datasets = append(datasets, agentshost.ZFSDataset{
Name: fmt.Sprintf("tank/data-%03d", i),
Type: "filesystem",
})
}
got := hostZFSPoolsFromAgentDisks([]agentshost.Disk{{
Device: "tank",
Type: "zfs",
ZFSDatasets: datasets,
}})
if len(got) != 1 || len(got[0].Datasets) != maxHostZFSDatasetsPerPool {
t.Fatalf("bounded host zfs pools = %#v", got)
}
if got[0].Datasets[0].Name != "tank/data-000" {
t.Fatalf("invalid rows were not rejected: %#v", got[0].Datasets[0])
}
}
func TestMonitoringBroadcastCarriesEveryAvailabilityProjection(t *testing.T) {
input := monitorResourceToConvertInput(unifiedresources.Resource{
ID: "agent-core2026",
@@ -144,6 +144,41 @@ func matchZFSPoolForStorage(storage models.Storage, zfsPoolMap map[string]*model
return nil
}
func mergeLinkedHostZFSDatasets(
pools map[string]*models.ZFSPool,
host *models.Host,
) {
if host == nil {
return
}
for _, reported := range host.ZFSPools {
name := strings.TrimSpace(reported.Name)
if name == "" || len(reported.Datasets) == 0 {
continue
}
pool := pools[name]
if pool == nil {
for candidateName, candidate := range pools {
if strings.EqualFold(strings.TrimSpace(candidateName), name) {
pool = candidate
break
}
}
}
if pool == nil {
pool = &models.ZFSPool{
Name: name,
State: "UNKNOWN",
Status: "Dataset inventory available",
Scan: "none",
Devices: []models.ZFSDevice{},
}
pools[name] = pool
}
pool.Datasets = append([]models.ZFSDataset(nil), reported.Datasets...)
}
}
// pollVMsWithNodes polls VMs from all nodes in parallel using goroutines
// When the instance is part of a cluster, the cluster name is used for guest IDs to prevent duplicates
// when multiple cluster nodes are configured as separate PVE instances.
@@ -321,6 +356,10 @@ func (m *Monitor) pollStorageWithNodes(ctx context.Context, instanceName string,
Msg("Could not get ZFS pool status (may require additional permissions)")
}
}
mergeLinkedHostZFSDatasets(
zfsPoolMap,
m.linkedHostForNode(instanceName, n.Node, n.Node),
)
// Process each storage
for _, storage := range nodeStorage {
+37
View File
@@ -0,0 +1,37 @@
package monitoring
import (
"testing"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
)
func TestMergeLinkedHostZFSDatasetsEnrichesAndSynthesizesPools(t *testing.T) {
pools := map[string]*models.ZFSPool{
"rpool": {Name: "rpool", State: "ONLINE"},
}
host := &models.Host{ZFSPools: []models.HostZFSPool{
{
Name: "rpool",
Datasets: []models.ZFSDataset{
{Name: "rpool/data", Type: "filesystem", UsedBytes: 100},
},
},
{
Name: "tank",
Datasets: []models.ZFSDataset{
{Name: "tank/vm", Type: "volume", UsedBytes: 200},
},
},
}}
mergeLinkedHostZFSDatasets(pools, host)
if len(pools["rpool"].Datasets) != 1 || pools["rpool"].State != "ONLINE" {
t.Fatalf("existing pool not enriched safely: %#v", pools["rpool"])
}
if pools["tank"] == nil || pools["tank"].State != "UNKNOWN" ||
len(pools["tank"].Datasets) != 1 {
t.Fatalf("agent-only pool not synthesized: %#v", pools["tank"])
}
}
+1
View File
@@ -191,6 +191,7 @@ func cloneZFSPool(in *models.ZFSPool) *models.ZFSPool {
}
out := *in
out.Devices = append([]models.ZFSDevice(nil), in.Devices...)
out.Datasets = append([]models.ZFSDataset(nil), in.Datasets...)
if in.ScanDetails != nil {
scan := *in.ScanDetails
scan.StartedAt = cloneTimePtr(in.ScanDetails.StartedAt)
+12
View File
@@ -23,6 +23,18 @@ func TestCloneResourcePtr_Nil(t *testing.T) {
}
}
func TestCloneZFSPoolIsolatesDatasets(t *testing.T) {
input := &models.ZFSPool{
Name: "tank",
Datasets: []models.ZFSDataset{{Name: "tank/apps", UsedBytes: 100}},
}
cloned := cloneZFSPool(input)
cloned.Datasets[0].Name = "mutated"
if input.Datasets[0].Name != "tank/apps" {
t.Fatal("unified resource zfs pool clone aliased datasets")
}
}
func TestCloneResource_MutateOriginalSlice(t *testing.T) {
original := &Resource{
ID: "r-1",
+19 -8
View File
@@ -208,14 +208,25 @@ type MemoryMetric struct {
// Disk represents disk utilisation metrics.
type Disk struct {
Device string `json:"device,omitempty"`
Mountpoint string `json:"mountpoint,omitempty"`
Filesystem string `json:"filesystem,omitempty"`
Type string `json:"type,omitempty"`
TotalBytes int64 `json:"totalBytes,omitempty"`
UsedBytes int64 `json:"usedBytes,omitempty"`
FreeBytes int64 `json:"freeBytes,omitempty"`
Usage float64 `json:"usage,omitempty"`
Device string `json:"device,omitempty"`
Mountpoint string `json:"mountpoint,omitempty"`
Filesystem string `json:"filesystem,omitempty"`
Type string `json:"type,omitempty"`
TotalBytes int64 `json:"totalBytes,omitempty"`
UsedBytes int64 `json:"usedBytes,omitempty"`
FreeBytes int64 `json:"freeBytes,omitempty"`
Usage float64 `json:"usage,omitempty"`
ZFSDatasets []ZFSDataset `json:"zfsDatasets,omitempty"`
}
// ZFSDataset is a bounded, read-only projection of `zfs list` for one pool.
type ZFSDataset struct {
Name string `json:"name"`
Type string `json:"type,omitempty"`
Mountpoint string `json:"mountpoint,omitempty"`
UsedBytes int64 `json:"usedBytes,omitempty"`
AvailableBytes int64 `json:"availableBytes,omitempty"`
ReferencedBytes int64 `json:"referencedBytes,omitempty"`
}
// DiskIO represents disk I/O statistics for a block device.
+26
View File
@@ -170,6 +170,32 @@ func TestMemoryMetric_Fields(t *testing.T) {
}
}
func TestDiskZFSDatasetsJSONRoundTrip(t *testing.T) {
input := Disk{
Device: "tank",
Type: "zfs",
ZFSDatasets: []ZFSDataset{{
Name: "tank/apps",
Type: "filesystem",
Mountpoint: "/tank/apps",
UsedBytes: 100,
AvailableBytes: 200,
ReferencedBytes: 50,
}},
}
data, err := json.Marshal(input)
if err != nil {
t.Fatal(err)
}
var output Disk
if err := json.Unmarshal(data, &output); err != nil {
t.Fatal(err)
}
if len(output.ZFSDatasets) != 1 || output.ZFSDatasets[0].Name != "tank/apps" {
t.Fatalf("zfs datasets = %#v", output.ZFSDatasets)
}
}
func TestDisk_Fields(t *testing.T) {
disk := Disk{
Device: "/dev/sda1",