Match automatic system-mount exclusions at directory boundaries rather than textual prefixes. Local /snapshots, /runtime and /var/lib/docker-data volumes retain capacity and usage through agent collection, both report paths and QGA aggregation; genuine system and remote/container mounts remain filtered. Regression controls include host usage alert onset and recovery.
Contract-Neutral: Restore existing filesystem eligibility without changing schemas, identities, explicit operator patterns or guest-command safety policy.
Change-source: pulse-maintainer
Update gopsutil and x/sys together with the purego version required by Darwin. Keep all other selected module versions unchanged and exercise real Linux CPU, memory, network and mount parsers alongside existing collector regressions.
Contract-Neutral: Dependency and compatibility-test maintenance only; no API, collection policy, platform support, deployment or privacy contract changes.
Change-source: pulse-maintainer
Carry the operator include override in disk reports so server filtering does not discard selected tmpfs mounts again. Keep automatic filtering for unmarked reports and agent exclusion precedence. Cover both ingest paths, collection, forwarding and wire compatibility for #1875.
Change-source: pulse-maintainer
The failed #1875 beta retest leaves runtime environment and mount visibility unresolved. Existing fixtures stub partition enumeration, so cover the real gopsutil parser through filtering and capacity projection for single and combined tmpfs includes, missing includes and exclusion precedence. This proves the expected synthetic path, not the reporter installation or a product fix.
Change-source: pulse-maintainer
A completed probe remained discoverable after its first caller returned,
allowing the next collection to reuse stale filesystem measurements.
Remove it and publish completion within one registry critical section.
Keep in-flight sharing, cancellation and timeout behaviour intact.
The controlled regression fails before this change. Twenty full package
runs and three race runs pass on the worker.
Refs #1928
Equal-cadence CPU regression checks can miss a shared cached percentage because both reporting loops expect the same average. Add unequal host and Docker intervals and replace the Docker collector to guard independent baseline ownership without resetting the host loop. This protects the existing issue #1894 repair; synthetic coverage does not establish installed-agent qualification or reporter resolution.
Contract-Neutral: regression-test coverage only; no runtime behaviour or product surface changes.
Change-source: pulse-maintainer
An already-cancelled collector could still admit a filesystem syscall which ignores cancellation and blocks indefinitely. Reject expired contexts before registering work while retaining shared in-flight probes. The new regression test reproduces one unwanted probe before this fix; focused collector and guarded-probe tests pass ten repeats under the race detector.
Change-source: pulse-maintainer
Shared probes prevent overlapping collectors from stacking blocked statfs calls, but their short-lived sync.Map entries added measured collection overhead. Use a mutex-protected typed map and allocate only for the owning caller, retaining deadlines, result publication and one in-flight syscall per mount.
The paired filtering benchmark drops from 244 to 206 allocations and 21345 to 20129 bytes per operation. Five hostmetrics package repeats pass under the race detector, including new concurrent healthy-mount coverage. This is local mocked-collection evidence, not release or real-mount qualification.
Change-source: pulse-maintainer
Host and Docker collectors can enter the same mount probe before either timeout expires. Register the in-flight call atomically and share its result and deadline so a blocked syscall cannot accumulate duplicate workers. Preserve the guard until the syscall returns even when a waiter is cancelled.
Change-source: pulse-maintainer
Near-synchronous host and Docker reports consumed a shared CPU baseline, measuring collection bursts rather than each module's reporting interval. Retain a collector per host collector and a separate Docker module collector while preserving the package-level convenience API and disk filters.
Add an interleaved-counter regression covering both collection entry points. It fails when routed through the shared baseline and passes with isolated state. All hostmetrics, hostagent and dockeragent tests pass, as do the focused CPU regression tests under the race detector.
Change-source: pulse-maintainer
Operators can explicitly select tmpfs paths that automatic filesystem
filtering would otherwise hide. Multiple such mounts commonly share both the
generic tmpfs device name and capacity, so applying normal storage
deduplication to them silently dropped configured disks from collection.
Limit device-and-capacity deduplication to normally visible storage while
preserving it for NAS bind mounts and subvolumes.
Change-source: issue #1875
gopsutil defaults to PID 1's mount table, which differs from the namespace created for Pulse's hardened systemd service. Enumerate the collector's own mount table so explicitly included tmpfs and bind mounts are discoverable, while preserving deliberate host-proc overrides for container deployments.
Change-source: pulse-maintainer
A hard-mounted network filesystem with an unreachable server blocks
statfs in an uninterruptible kernel wait. The collector issued that
syscall inline for every mount, so one dead NFS mount silently froze
the whole reporting cycle and stretched agent shutdown into the kernel
retry window, and it paid that price for mounts the fstype filter was
going to discard anyway.
Decide type- and mountpoint-based skips before the usage syscall, so
network filesystems are never probed unless explicitly included, and
bound every remaining usage call with a timeout that leaves at most one
in-flight probe per mountpoint. A mount whose call never returned is
skipped on later cycles and re-included when the stalled call answers.
Refs discussion #1747
Contract-Neutral: Discussion #1747: behavioral bugfix in host disk collection; no payload or API shape change, filtered mounts were never reported
99ad8c2c4 routed collectDiskIO through fsfilters.IsVirtualBlockDevice to
stop ZFS zvols being collected. That helper answers "can this device report
SMART", which is the wrong question for I/O accounting: it also matches
vd* and xvd*, so every agent running inside a KVM, Proxmox or Xen VM lost
host disk I/O entirely, since vda is the real disk on those machines. It
also dropped md, nbd, rbd, drbd and pmem, none of which were part of the
report.
Add IsNonPhysicalDiskIODevice for the I/O question and use it here. It
covers only loop, ram, zram, dm- and zd, so the zvol fix for #1671 stands
while everything that was collected before 99ad8c2c4 is collected again.
Tests assert vda and xvda survive the I/O filter while still being excluded
from SMART, so the two predicates cannot be conflated again.
Contract-Neutral: behavioural regression fix, no contract delta
Refs #1671
collectDiskIO hand-rolled a partial exclusion list (loop, ram, dm-) while
the two sibling paths, agent SMART collection and the server-side resource
registry, both use fsfilters.IsVirtualBlockDevice. ZFS zvols were therefore
collected as if they were physical disks: a Proxmox host with ZFS-backed
guest storage exposes one zd<N> device per zvol, so a few hundred guests
produced a few hundred phantom disks in every agent report, driving
sustained metrics and resource_changes growth.
Route the filter through the canonical helper so zd, zram, nbd, rbd, drbd,
md, pmem, vd and xvd are all excluded. md and dm- aggregates additionally
restated the I/O of their own physical members, so counting them alongside
those members double counted host disk I/O.
Refs #1671
collectCPUUsage took a blocking 1-second spot sample once per report
interval, which overstates CPU on mostly-idle guests (reported on a
2-vCPU aarch64 KVM guest) and delayed every Collect() by a second.
Track the previous cumulative cpu.Times sample and report busy/total
across the elapsed interval instead, falling back to the spot sample
only for the first collection and counter resets (reboot, migration).
CPUUsagePercent keeps its meaning in the report schema.
Contract-Neutral: hostmetrics sampling behavior change with no subsystem contract surface delta
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Refs #1510
Refs #1501
Refs #1507
Refs #1442
- persist scoped workloads status filters across platform navigation
- derive host memory pressure from available memory
- reapply system settings after every monitor reload path
- bound PBS backup snapshot polling workers during large backup scans
f62f35e24 restored the v5 used | cache | free memory split for Proxmox
nodes and guests, but standalone host agents still reported a flat
used/free pair, so the Machines page memory bar could not show the
reclaimable segment. Flagged by the Machines page v5 parity audit.
- Host agent reports cacheBytes (gopsutil Available minus Free); the
ZFS ARC adjustment recomputes free so used + cache + free still
covers the total.
- ApplyHostReport maps the field into models.Memory.Cache and clamps
inconsistent or older-agent reports so used + cache never exceeds
total.
- AgentMemoryMeta carries cache onto unified resources so the frontend
agent payload exposes it.
- Mock generic hosts split a third of non-used pages as cache, and the
node-linked host conversion now holds the invariant instead of
stacking the node's cache on top of a recomputed free.
- Contracts: monitoring, unified-resources, and storage-recovery now
document the split (also covering the f62f35e24 node/guest surface,
which landed without contract deltas).
Back-port v5 fix b83f717ce to v6. zfsMountpointScore now gives top
priority (score 0) to a dataset whose mountpoint is exactly '/', even a
child dataset like rpool/ROOT/pve-1, so it outranks a pool-root dataset
mounted elsewhere (e.g. rpool at /mnt/storage). Previously a pool-root
dataset won unconditionally, so the pool summary could reflect the wrong
dataset's usage. Updates scoring tests and adds a regression case.
syscall.SysctlRaw is Darwin-only in Go's standard library; FreeBSD
requires the equivalent from golang.org/x/sys/unix. This fixes the
Docker cross-compilation build failure for the freebsd/amd64 target.
(cherry picked from commit 5fe16c75a075b817f90b7192d8270a7bd6677017)
Linux VM page cache (#1270): QEMU VM memory now falls back to Proxmox
RRD's memavailable metric (which excludes reclaimable page cache) when
the qemu-guest-agent doesn't provide MemInfo.Available. Previously the
fallback was detailedStatus.Mem (total - MemFree), inflating usage to
80%+ on VMs with normal Linux page cache. Mirrors the existing LXC
rrd-memavailable path.
FreeBSD ZFS ARC (#1264, #1051): The host agent now reads
kstat.zfs.misc.arcstats.size via SysctlRaw on FreeBSD and subtracts
the ARC size from reported memory usage. ZFS ARC is reclaimable under
memory pressure (like Linux SReclaimable) but gopsutil counts it as
wired/non-reclaimable, causing false 90%+ memory alerts on TrueNAS
and FreeBSD hosts. Build-tagged so it compiles cleanly on all platforms.
Fixes#1270Fixes#1264Fixes#1051
(cherry picked from commit 94502f83ff9ffc6da28aaadc946a2f7d8b4e9bac)
The previous reconciliation logic (issue #1052) used per-dataset statfs
values for Total and Used. On Proxmox systems, statfs on a mounted
dataset (e.g. rpool/ROOT/pve-1) only reports that dataset's own usage,
completely missing zvols (VM disk images) and other datasets. This caused
storage bars to show ~0% usage (a few GB of OS files) when the pool
actually had terabytes of VM data allocated.
Fix: derive usable pool capacity from the ratio of dataset Free (usable
pool-available from statfs) to zpool Free (raw pool-available from zpool
list). This ratio converts raw zpool Size to usable total, and Used is
computed as Total - Free. This captures all pool consumers including
zvols, handles RAIDZ parity overhead and mirrors uniformly, and produces
correct usage percentages.
Verified with tests for RAIDZ, mirrors, and both with zvols present.
Adds zerolog debug statements throughout the ZFS collection pipeline
(collector.go and zfs.go) to trace partition discovery, dataset
collection, zpool stats fetching, and pool summarization. This will
help diagnose issues like empty storage bars on mirror-vdev pools.
- Add MatchesDiskExclude() to check both device path and mountpoint
- Add MatchesDeviceExclude() for device-only matching
- Update collectDisks to check device in addition to mountpoint
- Update collectDiskIO to respect disk exclusions
- Patterns like /dev/sda, sda, or /mnt/backup all work now
Related to #1142
For RAIDZ pools, zpool ALLOC includes parity overhead, but users expect
to see actual data usage. Now using dataset Used value (from statfs)
when RAIDZ is detected, matching the existing fix for total capacity.
Fixes the second part of #1052 where used capacity was inflated.
For RAIDZ/mirror pools, zpool list SIZE reports raw capacity (sum of
all disks), but users expect usable capacity (accounting for parity).
The dataset stats from statfs give the correct usable capacity.
Now uses dataset Total when it's smaller than zpool Size, indicating
RAIDZ/mirror overhead.
Related to #1052
- Add persistent volume mounts for Go/npm caches (faster rebuilds)
- Add shell config with helpful aliases and custom prompt
- Add comprehensive devcontainer documentation
- Add pre-commit hooks for Go formatting and linting
- Use go-version-file in CI workflows instead of hardcoded versions
- Simplify docker compose commands with --wait flag
- Add gitignore entries for devcontainer auth files
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Synology NAS creates multiple shared folders (e.g., /volume1/docker, /volume1/photos)
that are all mount points on the same underlying BTRFS volume. Each reported the same
16TB total, causing Pulse to show 64TB+ instead of 16TB.
The fix tracks device+total combinations and only counts each unique pair once.
When duplicates are found, the shallowest mountpoint (e.g., /volume1) is preferred.
Added a unit test to verify the deduplication works correctly.
Users can now exclude specific mount points from disk monitoring:
- Via CLI: --disk-exclude /mnt/backup --disk-exclude '/media/*'
- Via env: PULSE_DISK_EXCLUDE=/mnt/backup,*pbs*
Patterns support:
- Exact paths: /mnt/backup
- Prefix patterns: /mnt/ext*
- Contains patterns: *pbs*
This addresses the common case where external disks or
PBS datastores are being monitored but shouldn't be.
Enhanced zpool binary lookup to try common paths when exec.LookPath fails.
This fixes issue #718 where TrueNAS SCALE reports inflated storage because
the agent runs with a restricted PATH that doesn't include /usr/sbin.
Changes:
- Added findZpool() helper that tries common paths like /usr/sbin/zpool,
/sbin/zpool, /usr/local/sbin/zpool for TrueNAS/FreeBSD/Linux systems
- Added commonZpoolPaths variable listing typical zpool locations
- Added tests for the new findZpool function
This ensures zpool list is used for accurate pool-level capacity instead
of falling back to dataset-level summation.
Host agent was including Docker overlay2 mounts from TrueNAS SCALE's
.ix-apps directory in disk totals. These mounts inherit the ZFS pool's
AVAIL space, causing massively inflated storage numbers (e.g., 173 TB
per container overlay instead of actual usage).
Changes:
- Add /mnt/.ix-apps/docker/ to container overlay path exclusions
- Use ShouldSkipFilesystem() in host agent disk collection (was only
using ShouldIgnoreReadOnlyFilesystem() which missed container paths)
- Add test cases for TrueNAS overlay paths
Related to #718
- Refactor collector to support mocking
- Fix ZFS detection to support 'fuse.zfs' and case-insensitivity
- Add regression tests for ZFS dataset deduplication
Squashfs snap mounts on Ubuntu (and similar read-only filesystems like
erofs on Home Assistant OS) always report near-full usage and trigger
false disk alerts. The filter logic existed in Proxmox monitoring but
wasn't applied to host agents.
Changes:
- Extract read-only filesystem filter to shared pkg/fsfilters package
- Apply filter in hostmetrics.collectDisks() for host/docker agents
- Apply filter in monitor.ApplyHostReport() for backward compatibility
- Convert internal/monitoring/fs_filters.go to wrapper functions
This prevents squashfs, erofs, iso9660, cdfs, udf, cramfs, romfs, and
saturated overlay filesystems from generating alerts. Filtering happens
at both collection time (agents) and ingestion time (server) to ensure
older agents don't cause false alerts until they're updated.
pulse-triage[bot]