mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-25 04:59:43 +00:00
rcourtman
fb55d47517
`checkMockAlerts` evaluated guests, agents, nodes, storage, PBS, PMG and Docker against the mock estate but never physical disks. The mock generator deliberately keeps a FAILED health cohort and worn SSDs stable across restarts, so mock and demo stacks showed failing and "Replace Now" disks with no `disk-health` or `disk-wearout` alert, and the disk alert lifecycle could not be exercised without real hardware. ## Change - `checkMockAlerts` now evaluates fixture disks through `checkMockPhysicalDiskAlerts`, applying the physical disk poller's boundary: only disks on online nodes (matched by instance and node name), with the linked agent's `--disk-exclude` patterns forcing a matched device healthy. - The poller's per-disk rule (exclusion forces healthy, else `CheckDiskHealth`) moves into `checkPhysicalDiskAlerts`, shared by `maybePollPhysicalDisksAsync` and the mock pass. The poller's behaviour is unchanged; the source guardrail `TestProxmoxDiskAlertsRunOnMergedDiskState` now pins the helper call. - The poller's host-agent link settle deferral (#1674) is not applied in mock mode: fixture links and exclusions are complete from the first pass. ## Demo-facing effect Measured on private stacks with the public demo's exact estate (`/etc/pulse/.env`: 8 nodes, `DEMO_MODE`), baseline vs patched: the demo estate has 12 Proxmox disks and none FAILED. The only new disk alert is one warning-level `disk-wearout` ("SSD has less than 10% life remaining (7% wearout)" on West Production E), whose resource ID is one of the disk resource's canonical aliases. No `disk-health` alert opens on that estate. The default test estate (50 nodes) raises five `disk-health` and one `disk-wearout` alert. ## Known gaps, left out of scope - Like the other loops in `checkMockAlerts`, the disk loop relies on the pass's single up-front mode check, so a pass in flight when mock mode is switched off can reopen alerts after `ClearActiveAlerts`. That race predates this change and also involves the frontend refresh and backup rollup paths, so it needs a monitor-wide fix rather than a lock around this pass alone. - `CheckDiskHealth` logs at error/warn on every evaluation of a FAILED or worn disk, so in mock mode such a disk logs once per alert tick. - The wearout message calls 7% life remaining "7% wearout". ## Verification - `TestCheckMockAlertsEvaluatesPhysicalDisks` (pinned default estate) and `TestCheckMockPhysicalDiskAlertsFollowsPollerBoundary` (offline node, same node name on another instance, exclusion resolving existing alerts, node departure cleanup). Mutations that drop the pass, ignore offline nodes, ignore exclusions, or key by node name only each fail a test. - Full `go test ./internal/monitoring/` and the mock tests under `-race` pass.