mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 04:33:17 +00:00
rcourtman
7eb32f3181
## Problem
Setting an alert global default to a positive trigger of 5 or less, for
example Machines CPU at 1% in Alerts > Thresholds, stored
`agentDefaults.cpu` as `{trigger: 1, clear: 75}`, a clear far above its
trigger.
The thresholds page sends `clear = max(0, trigger - 5)`, which is 0 for
those triggers. `NormalizeAgentDefaults` derived `trigger - 5`, found it
non-positive, and fell back to the factory clear (75/80/85).
`ValidateHysteresisThresholds` never checked agent CPU, memory or disk,
so the pair survived. PBS CPU/memory, node temperature and
`internal/config/persistence.go` had the same fallback. On the save
path, validation repaired PBS and node temperature, but
`LoadAlertConfig` still turned a stored node temperature clear of 0 into
75; at startup `UpdateConfig` repaired that in memory.
Per-type `DiskFillByType` / `DiskTempByType` entries were worse: any
clear ≤0 reset the whole pair. A SATA disk temperature trigger of 4 from
the thresholds page was stored as 55, and an API-written fill entry of
`{88, 0}` became `{90, 85}`.
## What a user saw
- **Firing and resolving did not change for these pairs.**
`buildCanonicalMetricSpec` drops a clear at or above the trigger, and
the reducer then clears at the trigger. That held both before and after
this change. There was no flapping and no alert that never re-armed.
- `/api/alerts/config` and `alerts.json` served the factory clear above
the trigger.
- Firing agent alerts carried it as `metadata.clearThreshold`: 75 on a
1% CPU alert, and 85 on a disk alert still firing at 81.8%.
- A per-type disk trigger of 1–5 was silently replaced by the type
default.
- Machines metric bars: `resolveThreshold` in `metricThresholds.ts`
clamps the warning level to `min(clear, trigger)`, so a 1–5% agent
trigger had no warning band. Readings below the trigger, including 0,
now show amber instead of green, which matches guest, node, Docker and
Kubernetes bars at the same setting. This one is read from the code; I
did not exercise it in a browser.
## Fix
- PBS, node temperature, agent and per-type disk defaults now use
`normalizeThresholdPointer`, the normalizer Kubernetes, TrueNAS and
vSphere already use. A missing clear sits five points under the trigger,
floored at 0. A clear at or above the trigger is repaired. Zero still
means off for global defaults, and nil or negative still takes the
factory default. For the canonical per-type keys (the types the page
edits), a non-positive trigger still resets to the type default, because
the page cannot represent a per-type Off; other per-type keys are stored
as written, as before.
- `internal/config/persistence.go` drops its duplicate
agent/node-temperature and storage helpers and calls the same normalizer
through a new `NormalizeHysteresisThreshold`. A config saved before this
fix loads repaired.
- Agent CPU/memory/disk/disk temperature are added to
`ValidateHysteresisThresholds`, as every other family already is.
- One deliberate behaviour change for invalid pairs above the margin: an
API-written `{50, 80}` is repaired to `{50, 45}`, so it now has a real
recovery band where it used to clear at the trigger. This matches what
`EnsureValidHysteresis` already does for every other family.
- An alerts contract clause records the rule.
## Verification
- `internal/alerts/config/normalize_low_trigger_clear_test.go` runs
every default family and both per-type maps through the `UpdateConfig`
normalizer chain.
- `TestUpdateConfigKeepsLowTriggerClearBelowTrigger` pins the saved
config and a firing alert's `clearThreshold`.
- `persistence_test.go` checks the written `alerts.json` itself as well
as load repair.
- All new tests fail against the previous normalizers. Coverage cases
that pinned the old fallback (including one labelled "suspected source
bug") are updated.
- `go test ./internal/alerts/... ./internal/config/ ./internal/ai/`
passes.
- Live, on private mock stacks (baseline binary vs this branch): I saved
the payload the thresholds page sends, then read back
`/api/alerts/config`.
- Baseline: agent `{1,75}`, `{3,80}`, `{5,85}`; `diskTempByType.sata`
`{55,50}` for a 4°C trigger; firing agent alerts reported
`clearThreshold` 75/80/85.
- This branch: every pair `{t, 0}`, `diskTempByType.sata` `{4, 0}`, and
alerts reported 1/3/5.
- After a restart, this branch also repaired a hand-written
`alerts.json` holding `{1,75}` and node temperature `{2,75}`.