mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
cfe3a4c3985c2eca498e32247892cf14f93a7735
13061 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rcourtman
|
cfe3a4c398 | Merge origin/main into refactor/drop-summary-chart-routes | ||
|
rcourtman
|
ad96886eab |
Remove unused CorrelationsResponse type (#2710)
CorrelationsResponse described the `/api/ai/intelligence/correlations` payload. #2674 removed its uses in the Patrol investigation context model and that model's tests, and #2677 removed the last ones (`AIAPI.getCorrelations` and the aiIntelligence store's correlation loader). This deletes the now-unreferenced interface from `frontend-modern/src/types/aiIntelligence.ts`. `ResourceCorrelation` stays: `ResourceIntelligence.correlations`, the resource drawer's relationship map (`ResourceCorrelationSummary.tsx`) and `resourceCorrelationPresentation.ts` still use it. Proof: - `git grep CorrelationsResponse` over the whole repository at the base commit finds only the definition. - `npx tsc --noEmit` in frontend-modern (tsconfig includes all of `src`, tests too) passes. - `vitest run src/types src/api src/stores src/features/patrol src/utils`: 7295 passed. The only failures are 6 tests in `agentInstallCommand.test.ts` that need a TTY (`termios.error: Inappropriate ioctl for device`); they fail identically on the unchanged base tree. - `npm run build` before and after emitted the same 187 dist files with identical SHA-256 sums, so the shipped bundle is byte-identical. - Browser receipt `frontend-modern/browser-verification/71a0ce6b24830582.json`: embedded build in a demo-mode mock stack, `/patrol` and the `/kubernetes` deployment drawer at 1280x800 and 390x844, no console or page errors. No contract prose mentions the type, so the commit carries a `Contract-Neutral:` trailer instead of a patrol-intelligence contract note. |
||
|
rcourtman
|
4a0a05858d |
Remove the summary chart routes no client calls
GET /api/charts/workloads-summary has had no first-party caller since the v6 import: ChartsAPI.getWorkloadsSummaryCharts arrived with it and nothing ever called it. GET /api/charts/storage-summary lost its last reader when storageSummaryTrendCache.ts was deleted on 2026-05-26. Both still cost something. The workloads route carried a payload cache, an SLO test in the release latency gate and a benchmark, and mock mode rebuilt the 24-hour storage aggregate on every sampler tick for a cache nothing read. No stable release ever requested either route from the browser: v6.0.0 and later carry no caller, and the storage trend reader shipped only in v6.0.0-rc.1 to rc.5. Neither route is in docs/API.md (a list of common endpoints, not a complete one), and pulse-pro, pulse-mobile and pulse-dev-infra never call them. A monitoring:read token could reach them, so a private script may notice, but nothing known depends on them. pulse-enterprise names the SLO test only as fixture data, and its latency gate passes a test that is absent from the candidate as removed. So the routes go with their clients: handlers, response types, the monitor trend helpers, the mock aggregate and its prewarm, the SLO budget, the benchmark, and the ChartsAPI methods and types. Requests to either path now get 404. buildMockWorkloadMetricHistorySeries goes too, because the workloads-summary handler was its only production caller. A TypeScript reference graph from src/index.tsx finds no production reference to the removed methods or types. The production build loses only the two static methods from the ChartsAPI chunk (326 B raw, 49 B gzip); every other file differs only by chunk hashes and SRI values. deadcode ./cmd/pulse and golangci unused report nothing newly unreachable. On private mock stacks the Proxmox overview in Trends, a guest History drawer and the storage page make the same chart requests before and after at 1440 and 390 px, and neither build requested the removed routes. Contract prose that still described either route as live now records the retirement, and the workload chart guard pins the per-type metric sets the surviving /api/charts/workloads route requests. |
||
|
rcourtman
|
c5416b6800 |
Alert on the Unraid temperature an agent disk shows (#2701)
Stacked on #2698 (which sits on #2694, #2685 and #2679); this PR's own change is the top commit, "Alert on the Unraid temperature an agent disk shows", plus a merge of main for an alerts.md anchor both touched. ## Why A host agent's physical disk can show a temperature that only the host's Unraid inventory reports: - a SMART row whose smartctl probe returned no temperature takes its Unraid row's reading (`HostSMARTDiskTemperature`), and - a disk only the inventory lists, such as a `--disk-exclude` match, is built from its Unraid row alone (`HostUnraidDiskTemperature`). Physical Disks (Temp cell and the `Running Hot` verdict, "at or above its alert threshold") and Patrol judge that reading against the alert disk temperature policy, but `CheckHost` only read the SMART row's own field. A disk could read as at its alert threshold while no alert fired. Unraid's own warning/critical disk temperature notifications do not make this redundant: Pulse already alerts on the smartctl readings and array state of the same hosts, which Unraid also notifies on. ## What changes - `unifiedresources.HostDiskTemperatureReadings(host)` lists the reading each of a host's disks shows, built with the registry's own adapters: one per SMART row the registry ingests, then one per Unraid row whose disk key no SMART row carries (the same rule as the history writer in #2698). - `CheckHost` judges those readings: - SMART rows keep their `agent:<host>/disk_temp:<device>` resource IDs. An inventory-only disk uses its block device token, which is the ID a plain Linux SMART row for that device uses, so a later SMART reading continues the same alert. - Per-type thresholds and host overrides use the disk type the disk shows. - Only a reading collected now fires or resolves, so a spun-down disk holds its alert. - When conflicting serials put two disks on one device, its alert judges the reading that stands highest against its own type's threshold. - An inventory-only disk matching the agent's `--disk-exclude` raises nothing, and an alert it raised before the exclusion resolves on the next report. This matches the documented exclusion from monitoring and how linked Proxmox hosts already treat excluded disks. - Absence cleanup still runs only on a non-empty SMART list. ## Verification - New alert tests in `internal/alerts/host_unraid_lifecycle_test.go` cover: the fallback, an inventory-only disk, an empty SMART list, two disks on one device (also of different types), a controller member under another device label, standby, exclusion, disk-type and host-override thresholds, holds, handover to a SMART reading, departure, and the exclusion clear. - Each fails on the previous `host.go`. - Removing any guard (exclusion skip, exclusion clear, disk type, collected-now, threshold ranking) fails its own test. - `TestHostDiskTemperatureReadingsMatchTheDisksTheRegistryShows` compares every reading with the disk the live registry ingests, including conflicting serials, a virtual device and a MegaRAID member Unraid lists under its block device. - `go test ./internal/alerts/... ./internal/unifiedresources/...` passes on the merged tree, as do the Unraid/SMART/temperature tests in `internal/monitoring`. |
||
|
rcourtman
|
a14584d9f1 |
Drop the summary focus bridge members nothing reads (#2709)
The jump-to-row affordance lost its only consumer when the summary components were deleted, yet the bridge kept computing it on every Workloads and Storage page: a capture-phase window scroll listener and a resize listener re-measured the hovered or focused row on each frame and hover change, walking its ancestors with getComputedStyle. On the 12-node mock storage table that was 128 calls across eight row hovers and 320 across ten scroll steps, now none. Storage's reveal callback kept branches that switched between the pools and disks views for that jump. Focus only ever names a row in the active view, so drop them and the disk-id memo that served them. The chart-hover input went the same way. No summary chart renders, so setChartHoverSync was never called and the resolver's chart branch was always fed null. Remove the signal, its type and the resolver option, and say in the contract how a returning chart re-enters the shared scope instead. Focus reveal, pinned-group reveal, Escape and row emphasis behave as before at desktop and phone widths; the browser receipt records the before/after pass. |
||
|
rcourtman
|
e7cabc72f1 |
Hide pprof routes on demo-mode instances (#2708)
The `/debug/pprof` handlers are gated on `RequireAdmin` plus a `settings:read` scope that session logins skip. A `DEMO_MODE` instance that signs visitors in as the configured admin therefore served every visitor goroutine and heap dumps, the process command line, and CPU profiles and execution traces that load the whole process. Demo mode already hides diagnostics, logs, admin users and manual discovery through the admin-operations policy; pprof was missing from it. ## Change - `internal/api/demo_mode_operations.go`: through the admin-operations policy, the demo guard answers `/debug/pprof` and every path below it with `404` for every method, ahead of route authentication, so a write probe such as the `POST` symbol lookup gets the same `404` instead of the generic demo `403`. Instances outside demo mode keep the routes, and `PULSE_PPROF_DISABLED` still removes them everywhere. - Contract notes in api-contracts (owner), agent-lifecycle and storage-recovery. ## Proof - `TestContract_DemoModeHidesPprofFromDemoAdminSession` signs in as the demo admin through the full router. The same session gets `200` from `/debug/pprof/cmdline` below the demo guard, and `404` for eight pprof paths × `GET`/`HEAD`/`OPTIONS`/`POST` through the served handler. Against the previous policy (`go test -overlay`) `GET` and `HEAD` returned pprof output (`200`, or the root's `301` into the index), `OPTIONS` got the router's preflight `200` and `POST` the generic demo `403`. - `TestPublicDemoAdminOperationsPolicyHidesEveryDebugRoute` parses the router files and fails if a literal route under `/debug/`, with or without a method prefix, escapes the policy for any of seven methods, so a new debug handler there cannot ship reachable on a demo by default. A temporary `GET /debug/vars` registration made it fail as expected. - Path-handling probe (not committed): 210 raw requests (35 request targets × six methods), covering trailing and double slashes, dot segments, `%2F`/`%2e` encodings, absolute-form targets and `CONNECT`, run against a copy of the router's mux dispatch. None reached a pprof handler; paths the mux would clean onto pprof only receive a redirect, and the policy catches the follow-up request. |
||
|
rcourtman
|
6263c580f7 |
Judge a linked agent's guest disks by that agent everywhere (#2712)
Follow-up to #2669, which made the guest drawer's filesystem rows follow a linked Pulse agent's freshness. The workload table's disk cell and the drawer History still judged the same guest by the Proxmox read reason, so one guest disagreed with itself across the three. ## What changes - **One rule for three readers.** `getWorkloadGuestDiskRead` (`workloadGuestPresentation.ts`) now drives the table disk cell, the drawer's filesystem rows and History. When the linked agent's filesystems fill in for a VM that has no Proxmox guest filesystems, the reading is current while the agent reports, last known once it is stale, and unavailable once the guest is stopped. Proxmox's own guest filesystems keep their read reason. - **Where the aggregate disk metric comes from.** On a manually linked VM, Proxmox records no guest disk usage without guest filesystems, so the registry's link merge keeps the agent's disk metric. A registry test pins that. - **Poller carry-forward.** `previousVMFromView` built the previous VM from the merged metric, so the next poll carried the agent's disk forward as a Proxmox `prev-agent-not-running` read. Proxmox priority then froze it over the agent's live value. Only Proxmox's own disk reading is carried now (`VMView.DiskFromLinkedAgent`). - **`agent.stale` is always on the wire.** The browser merges agent facets field by field, so an omitted false kept a resumed agent "stopped reporting" until a reload. This affected Machines rows and node drawers too. - **Withdrawn filesystems.** The websocket merge kept filesystems that a native Proxmox VM facet or a native agent facet (which always carries `stale`) no longer reported, including in the `platformData.agent` mirror that host threshold rows read. ## Verification - New tests, each failing on main: `GuestRow.test.tsx`, `GuestDrawer.test.tsx`, `workloadGuestPresentation.test.ts`, `resourceStateAdapters.test.ts`, `TestPreviousVMFromViewKeepsLinkedAgentDiskOutOfProxmoxCarry`, `TestAgentDataAlwaysSendsStale`, `TestVMViewDiskFromLinkedAgentFollowsSelectedMetricSource`, and `TestResourceRegistry_ManualGuestAgentLinkDiskMetricFollowsProxmoxGuestFilesystems`. The registry test is a characterization test; it fails when the agent outranks Proxmox on guest disk. - Private mock stack at 1440 and 390, with agents linked into VMs through `POST /api/resources/{vm}/link`: - Live agent, no QEMU guest agent: before, "- Unavailable · Guest agent not running", with History deferred; after, the agent's bars, nothing deferred. - Silent agent: before, "- Unavailable"; after, bars plus "Last known" (phone: "Prior"), with History deferred on the agent message. - Stopped guest: Unavailable. - A guest with Proxmox filesystems: unchanged. - Machines row for a standalone agent released from a silent hold, watched without a reload: on main it kept "(stale)"; now it clears. - Bundle: WorkloadsSurface 61.93 kB gzip against a 70.23 kB cap. ## Known residuals - Mock mode's websocket state does not apply manual links to resources ingested as records (`ingestRecords` never runs `applyManualLinks`), so the browser pass ran the Proxmox page on REST data. In production this likely affects links to vSphere or TrueNAS resources; it is tracked separately. - A stale Proxmox source can lose the aggregate to a live agent while its own filesystem rows and `prev-` reason remain. That guest stays on the Proxmox rule and reads its live aggregate as last known, which is the same as main and errs toward caution. A guest with no platform read reason can likewise keep a retained aggregate after the agent withdraws its filesystems, also as on main. Both need aggregate provenance on the wire. |
||
|
rcourtman
|
15322e1573 |
Drop contract prose for the deleted problem-resource surfaces (#2706)
Contract prose still described surfaces that no longer exist: - `frontend-primitives.md` told new route-level copy to extend `problemResourcePresentation` (deleted in |
||
|
rcourtman
|
23bf99e1ba |
Keep alert settings a config save leaves out (#2689)
## Problem
Saving anything on Alerts > Thresholds turned flapping detection and
alert TTL cleanup off. The page's `PUT /api/alerts/config` body never
carries `flappingEnabled`, `maxAlertAgeDays`, `maxAcknowledgedAgeDays`,
`autoAcknowledgeAfterHours` or `customRules` (the page has no control
for any of them). `UpdateAlertConfig` decoded the body into a zero
`AlertConfig`, so every save wrote `false`/`0` for those fields, reset
the flapping tuning to defaults, dropped custom rules, and persisted
that. The `flapping`, `aggregation` and `ioNormalization` objects the
page sends are not `AlertConfig` fields and never enabled anything.
## Fix
- `PUT /api/alerts/config` now replaces only the top-level keys the body
carries. A key left out keeps its stored value. A sent key replaces its
setting whole, so `"overrides": {}` still clears overrides, and the
usual normalization still runs. A body that is not a JSON object
(including `null`) is a 400 and changes nothing.
- `alertconfig.ApplyAlertConfigUpdate` decodes the body on its own, so
sent keys get exact `encoding/json` semantics, including duplicates and
case-folded keys. It copies every unsent top-level field from a JSON
clone of the stored config.
- `Manager.ApplyConfigUpdate` reads, merges and applies under the
manager lock, so concurrent partial saves cannot revert each other. It
returns an owned snapshot, which the handler persists, so
`SaveAlertConfig`'s in-place normalization no longer writes into the
live config's maps. The handler runs one save at a time, so successful
saves reach `alerts.json` in the order they were applied.
- Stored configs are not rewritten; a value an earlier save zeroed stays
as it is.
## Verification
- Unit: `internal/alerts/config/update_test.go`; manager save path and
concurrent partial updates in
`internal/alerts/config_validation_test.go`; handler against a real
manager in `internal/api/alerting/alerts_test.go`. Each new regression
fails against main's code (overlay negative controls). The concurrency
test fails at round 0 against a non-atomic variant.
- `go test ./internal/alerts/... ./internal/api/alerting/` and the
`internal/api` alert endpoint, tenant and scope tests pass; the alert
tests pass under `-race`.
- Private stack (mock mode, Vite on main's frontend):
- On main, a node CPU change saved from the Thresholds page took
flapping and all three TTLs to false/0, in memory and in `alerts.json`.
- With this change, the same UI save kept them (true/7/1/24) and stored
the new threshold.
- The API repro (GET, drop the four keys, PUT) keeps them; explicit
`false`/`0` still applies; `null` and `[]` return 400.
- Five rounds of concurrent partial PUTs kept both changes in memory and
on disk.
## Follow-ups
- The thresholds page still sends the dead
`flapping`/`aggregation`/`ioNormalization` objects and hard-coded
`minimumDelta`/`suppressionWindow`/`hysteresisMargin`. A frontend
cleanup will follow once the open PRs touching
`alertsConfigurationModel.ts` land.
- `SaveAlertConfig` still normalizes caller-owned maps in place for
other callers (`ActivateAlerts`, startup paths), and the GET handler
encodes a shallow copy. That pre-existing race is tracked separately.
|
||
|
rcourtman
|
36d3f8c557 |
Judge pulse-relaxed guests by one threshold on poll and save (#2705)
Stacked on #2700 (which carries #2696 and #2692). Merge order is enforced by the stack: this change must not reach main before #2700's per-filesystem reevaluation (see "Why stacked" below). ## Problem `pulse-relaxed` lifts a guest's CPU, memory and disk triggers to a 95/92/95 floor. Only `CheckGuest` applied it. Config-save reevaluation rebuilt the guest from alert metadata without its tags (`guestSnapshotFromAlert`), so a save judged a relaxed guest's alert by its unrelaxed thresholds: - A relaxed guest with an unset memory threshold alerts at 92 while polling. On save that reads as no threshold, so the save resolved the alert and sent a recovery notification, and the next poll raised it again. - The floor also lifted an Off threshold (trigger 0, or -1 from the editors) like any low trigger, so tagging a guest relaxed turned an Off alert on, with no recovery band (`{95, 0}`). A save then re-judged it Off and resolved it, and the next poll raised it again. ## Change - The relaxed step moves into `effectiveAlertPolicyNoLock`'s guest branch, so `CheckGuest`, config-save reevaluation (including #2700's `resolveGuestAlertThresholdsNoLock`) and posture checks share one resolver. - `guestSnapshotFromAlert` reads the recorded `tags` back, as `[]string` live or `[]interface{}` after a restart, through a new `metadataStringsValue` reader. `storagePolicyAliases` now uses the same reader. - An Off threshold (trigger <= 0) stays Off under `pulse-relaxed`; an unset one still gets the floor, as the existing tests pin. The tag is documented (issue #863) as relaxing thresholds, so it must never make a guest more sensitive than its config. - `metricOptionsWithTags` drops the `tags` key from an open alert once its resource has no tags. Reevaluation now reads that key, so a removed `pulse-relaxed` left behind would judge the alert by the relaxed trigger while polling uses the guest's own, and the alert would flap. Notification tag routing reads the same key, so it also stops routing on removed tags. ## Why stacked A filesystem's own override replaces the relaxed guest disk threshold in `CheckGuest`. On main, reevaluation misses filesystem overrides (fixed by #2700). With this change alone, a save would judge a relaxed guest's filesystem alert by the 95 floor instead of, say, an 85 override, and flap it. `TestConfigSaveKeepsRelaxedGuestFilesystemOverride` fails on main plus this change and passes on this stack. ## Proof New tests in `internal/alerts/alerts_test.go` and `internal/alerts/unified_eval_test.go`. With `go test -overlay` mapping the six runtime files back to #2700's head, these fail: - `TestConfigSaveJudgesRelaxedGuestAlertByRelaxedThresholds` (the save resolved the alert) - `TestPulseRelaxedKeepsOffGuestThresholdOff` (a disk alert was raised with disk Off) - `TestApplyRelaxedGuestThresholds/off_thresholds_stay_off` (Off became `{95, 0}` / `{92, 0}`) - `TestCheckGuestDropsRemovedTagsFromOpenAlerts` (stale tags; with only the metadata assertion relaxed, the save resolves the alert) `go test ./internal/alerts/... ./internal/notifications/` and the monitoring alert tests pass. |
||
|
rcourtman
|
b7049ff778 |
Raise Proxmox disk temperature alerts under the per-type policy (#2680)
**Stacked on #2675** (TrueNAS disk temperature alerts by disk type). Merge that first; this branch contains its commit. ## Problem A Proxmox physical disk that only the node's own sensors report never raised a temperature alert: - `CheckDiskHealth` judges health and wearout only; - unified evaluation types TrueNAS physical disks, not PVE ones. So Physical Disks could read a disk **Running Hot** under the per-type thresholds while alerts stayed silent. Whether a disk alerted depended on whether a Pulse agent ran on the node: agent-covered disks alert through `CheckHost` at the same thresholds. #1347 shows users look for disk temperature thresholds in the Proxmox tabs. ## Change - **`CheckProxmoxDiskTemperature`** (`internal/alerts/proxmox_disk_temperature.go`) raises a `diskTemperature` metric alert under the disk's existing PVE alert reference. It is judged by the disk temperature policy: the disk type's `DiskTempByType` entry under the agent Disk Temp default, which switches it off for every type. The alert carries the disk's serial, WWN and `diskType`, like the health and wearout alerts. - **One alert per disk.** `mergeHostAgentSMARTIntoDisks` marks a disk that a still-reporting linked agent lists in its SMART report (`AgentSMARTReported`, internal, never serialized). That disk is the agent's: `CheckHost` alerts on it, and an open PVE alert closes with `moved_to_agent`. A lapsed agent owns nothing, so the node's own current readings keep alerting while its agent is silent. - **Holds and clears.** Only a reading collected this poll is judged (`diskinventory.TemperatureCollected`), so a retained value holds the alert. An excluded device, or a switched-off policy, closes it. - **Stability.** A new `proxmox-disk` alert delay defaults to 5s, so warnings wait the same 300s noisy-gauge window as agent and TrueNAS disk temperatures. Critical readings bypass it. - **Config save.** `reevaluateActiveAlertsLocked` re-judges the alert per disk type. Without this it would fall through to guest thresholds, which have no disk temperature, and resolve on every save. An alert without a recorded type uses the lowest per-type trigger. - **History.** `ProxmoxPhysicalDiskAlertIdentifiers` lists the new `<ref>::metric-threshold:diskTemperature` identifier, so resource history owns its rows by recorded hardware identity, as #2646 does for health and wearout. - **Copy.** The Agents "Disk temperature by type" card says it covers agent, TrueNAS and Proxmox disks. ## Evidence Private mock stack (8 PVE nodes) with a local-only overlay pinning PVE disk readings: | Disk | Physical Disks | Alert | |---|---|---| | pve1 NVMe 63°C | Healthy | none | | pve2 SATA 56°C | Running Hot | warning, trigger 55, after the 300s window | | pve3 NVMe 72°C | Running Hot | warning, trigger 70, after the 300s window | | pve3 SAS 63°C | Healthy | none | | pve3 SATA 66°C | Running Hot | critical at once | The alert card reads: "Samsung 870 EVO 1TB (/dev/nvme2) · Temperature · Temperature 66°C, above the 55°C alert level · Clears once it drops to 50°C or lower and stays there for 5 minutes." ## Tests - `internal/alerts/alerts_test.go`: - the policy (per type, raised trigger); - ownership with `moved_to_agent`, the hold without a reading, exclusion, the off switch; - the stability window; - re-judging on save, including the lowest-trigger hold; - history identifiers. - `internal/monitoring/physical_disk_roundtrip_test.go`: the marker for reporting agents only (not a lapsed one), the SMART fallback path, and that `checkPhysicalDiskAlerts` passes collected readings, ownership and exclusion. - `internal/unifiedresources/store_test.go`: a temperature alert row is owned by its disk and still found by a read by its reference. - `internal/models/deepcopy_test.go`: the marker survives cloning and stays out of JSON. - `ThresholdsTable.test.tsx`: the card copy. - Ten wiring points were mutation-checked (marker, lease condition, collected reading, ownership, exclusion, re-judge branch, lowest trigger, resolution, history identifier); each removal fails a test. ## Second opinion - Fixed: an offline agent's retained SMART rows kept ownership, so neither side alerted while the agent was silent. Ownership now needs a reporting lease. - Kept, documented in `alerts.md`: on the first poll after an agent is newly linked to a disk with an open PVE alert, both alerts can be open until the next disk poll. This is the same window as the node-to-agent usage alert handover. ## Not in this PR - Like the health and wearout alerts, a disk that leaves the inventory keeps its temperature alert until the 24-hour stale sweep. - Per-host Disk Temp overrides do not reach PVE-only disks, which have no agent host. |
||
|
rcourtman
|
b814ccdd81 |
Count down to future Patrol times instead of reading just now (#2702)
`formatRelativeTime` is past-only: a time ahead of now reads "just now"
(compact) or "0s ago". Three Patrol lines handed it future times, so
each read wrong for its whole duration:
- Attention detail, suppressed issue: "Returns to active attention just
now." (also for an issue marked reviewed, which expiry hands back to
Reviewed, not active attention)
- Findings, will-fix-later: "Reminding just now"
- Findings, snoozed: "snoozed until just now"
Supersedes #2613, rebuilt on main now that #2598 has landed (#2613 was
stacked on two early #2598 commits whose contract wording main has since
replaced).
## Change
- `formatTimeUntil` in `utils/format.ts`, beside `formatRelativeTime`:
counts down ("in 3h", "in 1d"), rounds to the nearest unit (a 24-hour
suppression set a moment ago reads "in 1d", not "in 23h"), and returns
`dueText` (default "now") once the time arrives. `formatRelativeTime`
stays past-only and its doc points future times at the new helper.
- The three lines use it with the shared relative-time clock
(`useRelativeTimeNow`), since none of those times change while the panel
stays open, and keep the exact time on hover.
- Suppression line: "Returns to active attention in 1d." for an open
issue, "Suppression ends in 1d. It stays reviewed." for one marked
reviewed, "Suppression has ended." once due. It takes its words from
`getAttentionSuppressionEndPresentation` (added on main by
|
||
|
rcourtman
|
0b6473affc |
Fill the table's visible width with phone drawers (#2703)
Phone and tablet table drawers capped their sticky content at
`calc(100vw - 3.5rem)`, a guess at the surrounding page chrome. On pages
with less chrome than that, every drawer stopped short of the table and
left an empty strip on the right: 27px at 390px, measured on the
Availability checks, Proxmox node and guest, Docker host and container,
Kubernetes node and Machines drawers. At tablet widths a table wider
than its shell got a drawer wider than the visible area (712px in
695px).
The table scroll shell already declares `container-type: inline-size`
(the responsive column priorities use it), so `InlineDetailTableRow` now
caps the default content shell at `100cqi`, the shell's own visible
width. Measured live: content equals the visible table width at 320,
390, 768 and 1000px, with no right gap; desktop keeps the static
uncapped layout. The availability drawer's hostname now fits instead of
truncating. Callers passing their own `contentClass` are unaffected.
Not changed here: `.table-fixed td { overflow: hidden }` makes the
sticky offset inert, so a sideways-scrolled table still moves its drawer
(same before and after). Long unwrapped identity chips that were already
clipped are clipped less now; wrapping them is a separate follow-up.
Proof: `InlineDetailTableRow.test.tsx` and the shared-primitive
guardrail pin the new cap and reject the old one; the guardrail already
pins the scroll shell's `container-type`. Browser receipt covers five
routes at 320 to 1440px, before and after.
|
||
|
rcourtman
|
3fb267333d |
Require POST for Assistant session mutation routes (#2670)
The session router dispatches `/api/ai/sessions/{id}/abort`,
`summarize`, `fork`, `undo` and `redo` by path alone, and none of those
handlers checked the HTTP method, so a GET or HEAD cancelled a running
turn, compacted or rewound a transcript, or persisted a fork.
- **Demo mode:** `DemoModeMiddleware` admits every GET/HEAD as a read,
so a public-demo session could mutate Assistant sessions despite the
read-only guard.
- **CSRF:** `CheckCSRF` skips GET/HEAD, and session cookies default to
SameSite=Lax, which browsers send on cross-site top-level GET
navigations. A link carrying a session ID could therefore trigger these
mutations for a signed-in user.
## Change
- `requireAssistantSessionMutationMethod` in
`internal/api/ai_handler.go`: abort, summarize, fork, undo, redo and
steer admit only POST and answer anything else with `405` + `Allow:
POST` before resolving the Assistant service. Steer already refused
non-POST but sent no `Allow` header; it now uses the same guard.
- The guard sits in the handlers, matching how chat, steer and the
approval approve/deny handlers already own their method contracts.
- `messages` (GET) and the unsupported `diff`/`revert`/`unrevert` routes
are unchanged. Session rename/delete and the collection route already
dispatch on the method.
- Callers: `frontend-modern/src/api/aiChat.ts` uses POST for all six;
pulse-mobile's abort call and its OTA compatibility probe use POST.
## Proof
- `TestAssistantSessionMutationsRequirePost`: GET/HEAD/PUT/DELETE get
405 + `Allow: POST` for every mutation, with no mutation call and no
`IsRunning` check reaching the service. POST still dispatches.
- `TestAssistantSessionReadSubresourcesKeepTheirMethods`: GET `messages`
returns 200 and GET `diff` returns 501, with no mutation either way.
- `TestDemoModeRouterRejectsAssistantSessionMutationsOverSafeMethods`:
drives `NewRouter(...).Handler()` with `DemoMode` on. GET/HEAD to every
mutation pass the demo guard and get 405, POST stays 403, nothing
mutates, and GET `messages` still reads.
- Negative controls via `go test -overlay`: on main's `ai_handler.go`,
every non-POST case fails and the demo test reports `GET abort status =
200`. With abort's guard moved after the `IsRunning` check, the ordering
assertion fails.
|
||
|
rcourtman
|
1ac600ec2b |
Judge filesystem alerts by their own threshold on config save (#2700)
Stacked on #2696, which is stacked on #2692. The unique commit is the
last one,
|
||
|
rcourtman
|
77d2119792 |
Remove unread Patrol summaryStats and its audit rule (#2690)
Stacked on #2686, which sits on #2682 and #2674; its own change is the top commit, |
||
|
rcourtman
|
1231392e8d |
Judge a linked agent's guest disks by that agent everywhere
PR 2669 made the guest drawer's filesystem rows follow a linked Pulse agent's freshness when its filesystems fill in for a VM that has no Proxmox guest filesystems. The workload table's disk cell and the drawer History still judged the same reading only by the Proxmox read reason. So one guest read as current in the drawer and as "Unavailable: Guest agent not running" in the table and History, and a silent agent's reading could not show as last known there. getWorkloadGuestDiskRead now owns one rule for the table cell, the drawer rows and History: current while the agent reports, last known once it is stale, unavailable once the guest is stopped. Proxmox's own guest filesystems keep their read reason. The rule also covers the aggregate disk metric, because Proxmox records no guest disk usage without guest filesystems, so the manual link merge keeps the agent's metric. Three paths broke that: - The poller built its previous VM from the merged metric, so the next poll carried the agent's disk forward as a Proxmox prev- read, and Proxmox priority then froze it over the agent's live value. Only Proxmox's own disk reading is carried now. - agent.stale was omitempty, and the browser merges agent facets field by field, so a resumed agent stayed "stopped reporting" until a reload. It is always sent now. - The same merge kept filesystems a native Proxmox VM facet or a native agent facet had withdrawn, so the rows did not switch source. |
||
|
rcourtman
|
239cd06933 | Merge origin/main into fix/unraid-disk-temperature-alerts | ||
|
rcourtman
|
aafa4793c3 |
Drop the summary focus bridge members nothing reads
The jump-to-row affordance lost its only consumer when the summary components were deleted, yet the bridge kept computing it on every Workloads and Storage page: a capture-phase window scroll listener and a resize listener re-measured the hovered or focused row on each frame and hover change, walking its ancestors with getComputedStyle. On the 12-node mock storage table that was 128 calls across eight row hovers and 320 across ten scroll steps, now none. Storage's reveal callback kept branches that switched between the pools and disks views for that jump. Focus only ever names a row in the active view, so drop them and the disk-id memo that served them. The chart-hover input went the same way. No summary chart renders, so setChartHoverSync was never called and the resolver's chart branch was always fed null. Remove the signal, its type and the resolver option, and say in the contract how a returning chart re-enters the shared scope instead. Focus reveal, pinned-group reveal, Escape and row emphasis behave as before at desktop and phone widths; the browser receipt records the before/after pass. |
||
|
rcourtman
|
3bef5c3028 |
Remove unused CorrelationsResponse type
CorrelationsResponse described the /api/ai/intelligence/correlations payload. #2674 removed its uses in the Patrol investigation context model and that model's tests, and #2677 removed the last ones, AIAPI.getCorrelations and the aiIntelligence store's correlation loader, because no page renders the global learned-correlation list. Nothing in the repository references the interface any more, tests included. ResourceCorrelation stays: ResourceIntelligence.correlations, the resource drawer's relationship map (ResourceCorrelationSummary.tsx) and resourceCorrelationPresentation.ts all still use it. TypeScript erases the interface, so the production build is byte-identical: npm run build before and after this change emitted the same 187 dist files with identical SHA-256 sums. Contract-Neutral: Deletes an unreferenced TypeScript interface; the compiler erases it, the production bundle is byte-identical, and no wire, store or page contract changes. |
||
|
rcourtman
|
0180b33431 |
Hide pprof routes on demo-mode instances
The /debug/pprof handlers are gated on RequireAdmin plus a settings:read scope that session logins skip. The public demo signs every visitor in as the configured admin, so any visitor could fetch goroutine and heap dumps and the process command line, or start CPU profiles and execution traces that load the whole process. DEMO_MODE already hides diagnostics, logs, admin users and manual discovery through the admin-operations policy; pprof was missing from it. The policy now has the demo guard answer /debug/pprof and every path below it with 404 for every method, ahead of route authentication. Instances outside demo mode keep the routes, and PULSE_PPROF_DISABLED still removes them everywhere. TestContract_DemoModeHidesPprofFromDemoAdminSession signs in as the demo admin through the full router, shows the same session still reads /debug/pprof/cmdline below the demo guard, and gets 404 for eight pprof paths through the served handler. On the previous policy GET and HEAD returned pprof output (200, or the root's 301 into the index) and POST got the generic demo 403. A policy test also fails if a literal route the router files register under /debug/, with or without a method prefix, escapes the policy for any common method. |
||
|
rcourtman
|
48c2d6cdce |
Stop Patrol fetching the correlation list nobody reads (#2677)
The Patrol page fetched `GET /api/ai/intelligence/correlations` on every mount, every 60 s poll, every return to the tab and on Retry, and the store's dashboard bundle fetched it again for FindingsPanel's refresh. Nothing has read the list since the Patrol supporting-context block was removed in 27ba85ddc6; the resource drawer reads per-resource correlations from the resource-intelligence payload. ## Change - `stores/aiIntelligence.ts`: remove the correlation signal, `correlations` getter, `correlationsSignal`, `loadCorrelations()`, its org-switch reset, and its entry in `loadDashboardData()`. - `features/patrol/usePatrolIntelligenceState.ts`: drop the read from the Retry path's background supporting reads. - `api/ai.ts`: remove `AIAPI.getCorrelations` (no other caller). The backend route stays as an API surface behind `ai:execute`. - Tests: the store bundle test now pins every AIAPI read the bundle makes; the manual-run test stalls the post-start refresh through a remaining read so it still proves "Starting" clears without waiting (negative controls: awaiting the refresh, or removing it, both fail the test). Dead correlation route stubs removed from six E2E specs. - Contract prose in api-contracts, patrol-intelligence, unified-resources, ai-runtime and frontend-primitives no longer says the Patrol page or store loads or renders correlations or recent changes. The remaining stale Patrol correlation paragraphs (patrol-intelligence ~L2135/~L2197, ai-runtime ~L6854) are rewritten by #2674; the two PRs split hunks by agreement and merge cleanly together. `CorrelationsResponse` stays for now: `patrolInvestigationContextModel.ts` still imports it until #2674 lands. It goes in a follow-up once both are on main. ## Evidence - Demo-mode stack, headless Chromium at 1440x900 and 390x844: correlation requests go from 1 to 0 per Patrol load, browser reload, 60 s poll and tab hide/show. Every other `/api/ai/*` read per phase is identical; no console errors. Re-run on the final base. - Production build: the route string is gone from the bundle; `check-bundle-size.mjs` passes with no chunk renamed (JS gzip -0.18 kB on the same base). - `tsc --noEmit` passes; vitest for src/stores, src/api, src/features/patrol, src/components/AI and the Patrol page test passes; contract, registry and status audits pass. |
||
|
rcourtman
|
71a723abb1 |
Read controller-member disk history at its written key (#2694)
## Problem A physical disk behind a RAID controller (target such as `megaraid,3`) with no usable serial or WWN had its SMART history written under one key and read under another, so its temperature and SMART charts stayed empty. - Host agent: writes under `HostSMARTDiskMetricID`, e.g. `host-pve:sdd@ctrl0/megaraid,3`. - Proxmox: writes under `PhysicalDiskMetricID`, which returns `ProxmoxPhysicalDiskSourceID`, e.g. `pve-node1--dev-sdx:sdx@/megaraid,0`. - Reader: `BuildMetricsTarget` passed those source IDs, which already name the member, to `PhysicalDiskMetaMetricID`. That function appended the topology again: `host-pve:sdd@ctrl0/megaraid,3:sdd@ctrl0/megaraid,3`. The same happened in a registry rehydrated from persisted resources. ## Fix `PhysicalDiskMetaMetricID` now follows the Proxmox writer. A fallback that already ends in the member's topology is returned unchanged. The topology is appended only to a fallback without it: - a Proxmox source ID from before members were scoped, which `PhysicalDiskMetricID` scopes the same way; - the canonical resource ID that `PhysicalDiskView.MetricResourceID` falls back to. TrueNAS disks carry no controller target and are unaffected. ## Existing history - **SMART:** no SMART writer used the doubled key, so samples already stored become readable without a migration. - **Disk I/O:** a linked node's agent with no SMART reading for a member, such as a standby row, files the device's I/O under the Proxmox disk's metrics target. That target was the doubled key. The I/O now lands beside the member's SMART history. Samples already stored under the doubled key are left to age out: nothing reads them, and they are the block path's counter filed under its one merged member. - Mock seeding keys these disks by `disk.ID`, the single-scoped key, so it already matched. ## Proof - `TestIdentitylessControllerMembersReadTheirWritersHistory` (`internal/monitoring/issue1595_collection_trust_test.go`) drives the real writers: `writeHostSMARTMetrics`, `writeHostPhysicalDiskIOMetrics` and the Proxmox `writeSMARTMetrics`. It then queries the store at `MetricsTarget` for five members in a live registry and in one rehydrated through a JSON round trip. The members are two agent members sharing a path, a lone agent member with I/O, a Proxmox member, and a standby member merged into its linked node's Proxmox row. Before the fix all ten targets read the doubled key. - `TestPhysicalDiskMetaMetricIDScopesAControllerMemberOnce` (`internal/unifiedresources/canonical_ids_types_test.go`) pins each fallback shape against its writer: agent, smartctl-labelled agent, Proxmox, legacy Proxmox, canonical resource ID, and a disk that is no controller member. Before the fix, three cases failed with the doubled key. ## Stacking This branch sits on #2685, which sits on #2679, because both edit `physical_disk_ids.go` and the proof file. It supersedes #2661, which carried the same reader fix but conflicted with this stack and with main. |
||
|
rcourtman
|
843daf7ae3 |
Remove dead Patrol assessment and save-failure handoffs (#2682)
Stacked on #2674 (branch `refactor/drop-patrol-context-summary`); this branch contains that commit. ## What Removes the two Patrol-to-Assistant handoff builders in `frontend-modern/src/features/patrol/patrolInvestigationContextModel.ts` that no production code calls: - `buildPatrolAssessmentAssistantHandoff`: its caller, the summary card's Discuss handler, went in |
||
|
rcourtman
|
2eef42c512 |
Keep Off alert defaults off after saving (#2684)
## Summary
Switching a metric Off in a **Global Defaults** row (Alerts >
Thresholds) did not stick. After Save and a reload the metric was back
On at its factory threshold and kept alerting. This affected every
section whose factory threshold is positive: Machines CPU, memory, disk
and disk temperature, Proxmox node temperature, PBS, Docker, Kubernetes,
TrueNAS, vSphere and storage.
The editors stage `-1` for Off, and the save payload passed it through
as `{trigger: -1}`. Backend config normalization reads a negative
default trigger as unset and restores the factory value. Guest and node
CPU, memory and disk escaped only because nothing normalizes them.
- `buildAlertsConfigurationPayload` now writes every Off default as
`{trigger: 0, clear: 0}`, the value the engine and normalization both
read as off. Per-resource overrides are serialized separately and keep
`-1`, which the override path stores as written. A `-1` already stored
on a guest or node default becomes `0` on the next save.
- The Global Defaults **Custom** badge now treats `-1` and `0` as the
same Off value. Without that, a fresh install showed Custom on
Kubernetes, TrueNAS Systems and vSphere, and VMs & Containers turned
Custom after any save, though nothing had changed. An unset factory
value stays distinct from Off: an unset TrueNAS disk temperature follows
Disk temperature by type.
- A row's custom-threshold marker still compares raw values. It drives
the row's revert control and override removal, so a stored `-1` override
must stay visible against an Off default. A test pins that.
- The alerts contract no longer claims Off is written as `-1`
everywhere.
## Verification
- New payload test: every section's Off default is written as `{0, 0}`
and reads back Off. It fails on `main` with `guestDefaults.cpu: expected
{ trigger: -1 }`.
- New backend test: a `{0, 0}` default stays off through `UpdateConfig`
in every section, and a host at 99% raises no CPU, memory or disk alert
when the agent defaults are off. Both fail when the defaults are written
as `-1`.
- Live on a private mock stack. On `main`, Proxmox CPU and Temp switched
Off saved as `{-1}` and `{80, 75}`, and after reload Temp read On. With
the fix:
- Machines CPU at 1% raised 8 alerts. Switching it Off stored `{0, 0}`
and resolved all 8.
- Proxmox Temp Off resolved the node temperature alert.
- Nothing re-fired over two minutes.
- Values stayed Off after a backend restart and a reload. Machines,
Proxmox, Kubernetes, TrueNAS and vSphere were each checked at desktop or
phone width.
- Custom badge on a fresh install: no Proxmox, Kubernetes, TrueNAS or
vSphere group reads Custom, and after an unrelated save only the group
actually changed does.
|
||
|
rcourtman
|
60527fa8ab |
Drop contract prose for the deleted problem-resource surfaces
frontend-primitives.md still told new route-level copy to extend problemResourcePresentation, deleted in |
||
|
rcourtman
|
e3a93efcee |
Judge pulse-relaxed guests by one threshold on poll and save
CheckGuest lifted a pulse-relaxed guest's CPU, memory and disk triggers
to the 95/92/95 floor itself, but config-save reevaluation rebuilt the
guest from alert metadata without its tags and judged the alert by the
unrelaxed thresholds. A relaxed guest whose memory threshold was unset
got the floor of 92 while polling, read as no threshold on save, and
the save resolved the alert with a recovery notification; the next poll
raised it again.
The floor also lifted an Off threshold (trigger 0, or -1 from the
editors) like any low trigger, so tagging a guest relaxed turned on an
alert its config switched off, with no recovery band at all ({95, 0}).
The tag is documented as relaxing thresholds; it must never make a
guest more sensitive than its config.
- Apply the relaxed step in effectiveAlertPolicyNoLock's guest branch,
so CheckGuest, reevaluation and any later resolver share it.
- Read the recorded tags back in guestSnapshotFromAlert ([]string live,
[]interface{} after a restart) through a shared metadata reader that
storagePolicyAliases now uses too.
- Leave an Off threshold off; an unset one still gets the floor.
- Drop the tags key from an open alert once its resource has no tags.
Reevaluation now reads it, and a removed pulse-relaxed left behind
would judge the alert by the relaxed trigger while polling uses the
guest's own.
This stacks on the per-filesystem reevaluation fix. Without it, a save
judges a filesystem alert by the guest's relaxed floor instead of the
filesystem's own lower override, and flaps it;
TestConfigSaveKeepsRelaxedGuestFilesystemOverride guards that.
The other new tests fail against the base branch's runtime files via
-overlay.
|
||
|
rcourtman
|
dace9295c6 |
Remove exports orphaned by the summary component deletions (#2671)
Stacked on #2662 (its commits are ancestors of this branch, so this PR shows them until #2662 merges). The May cleanup deleted InfrastructureSummary, StorageSummary, the shared summary table and the SummaryMetricCard, InteractiveSparkline, DensityMap and SummaryPanel primitives, but left the helpers only they called. This removes them: - `getMetricsChartKeyCandidatesFromResource` (utils/agentResources.ts) - `getNormalizedIdentityLookupVariants` and `getAgentLikeIdentityAliases` (utils/resourceIdentity.ts) - `getPBSStatusIndicator` (utils/status.ts) - `useSummaryContextualFocusState` (components/shared/contextualFocus.ts), plus `filterSummarySeriesByGroupScope`, `resolveSummaryCardInteractionState` and `SummaryCardInteractionState` that only it used - the deprecated `ChartsAPI.getInfrastructureCharts` delegate; its branch tests now exercise `getInfrastructureSummaryCharts` Proof of deadness: a TypeScript import/export graph from `src/index.tsx` finds no production importer of any removed symbol, and a repo-wide grep finds no remaining reference. The production build differs from its parent only in the shared chunk that carries ChartsAPI, which loses the delegate's static method (99 bytes); after normalising chunk hashes, SRI hashes and minified identifiers every other file is identical. Tests: `src/utils`, `src/components/shared`, `src/api`, `src/features`, `src/hooks`, `src/__tests__`, Workloads, Storage and Infrastructure suites pass (only the known Mac-only `agentInstallCommand` failures). Contracts: frontend-primitives, storage-recovery and performance-and-scalability still described the deleted summary cards, sparkline, density map, summary panel, sticky summary section and Jump to row button as live owners, and credited contextualFocus.ts with the removed hook's series filtering. Those clauses now name the live owners (summaryCardInteraction.ts behind summaryTableFocus.ts, the storage summary cache behind the pool Growth column, TooltipPortal for the mini sparkline and backup activity chart tooltips), become conditional on a summary strip returning, or go. api-contracts and unified-resources record the removed chart client and chart-key helper. Browser pass (embedded production build, demo mock stack, 1440x900 and 390x844): Proxmox node-to-guests reveal, guest row hover emphasis and inline detail reveal, storage Growth column, pool row hover and inline detail, pool-group hover preview, pin and Escape clear. Receipt: `frontend-modern/browser-verification/232e5c23490f8cd4.json`. |
||
|
rcourtman
|
864a088db6 |
Fill the table's visible width with phone drawers
Inline table drawers below the lg breakpoint capped their sticky content at calc(100vw - 3.5rem), a guess at the page chrome. Pages with less chrome than that left an empty strip beside every drawer (27px at 390px on the Availability checks, Proxmox, Docker, Kubernetes and Machines tables), and a table wider than its shell at tablet widths got a drawer wider than the visible area. The table scroll shell is already a size container, so the content now caps at 100cqi: exactly the table's visible width at every width, measured live at 320, 390, 768 and 1000px. Desktop keeps the static uncapped layout. |
||
|
rcourtman
|
e747f2e57c |
Count down to future Patrol times instead of reading just now
formatRelativeTime is past-only: any time ahead of now reads "just now"
(compact) or "0s ago", which is right for clock skew on an observed
time. Patrol handed it three future times, so each read wrong for its
whole duration: a suppressed issue said "Returns to active attention
just now.", a will-fix-later finding said "Reminding just now", and a
snoozed finding said "snoozed until just now".
formatTimeUntil in utils/format.ts is the future counterpart. It counts
down ("in 3h", "in 1d"), rounds to the nearest unit so a 24-hour
suppression set a moment ago reads "in 1d" rather than "in 23h", and
returns "now" (or the caller's dueText) once the time arrives. The three
Patrol lines use it with the shared relative-time clock, because none of
those times change while the panel stays open, and keep the exact time
on hover.
The suppression line names the state expiry restores. An open issue
reads "Returns to active attention in 1d."; one marked reviewed reads
"Suppression ends in 1d. It stays reviewed.", because expiry restores
Acknowledged, which is not active attention; a passed expiry reads
"Suppression has ended.". The countdown takes its words from
getAttentionSuppressionEndPresentation, which already words the
early-end button and the duration field by the same reviewed split, so
all three agree.
A reminder reads "Reminder overdue" once one overdue id list, sampled on
the shared clock, holds it. The Overdue commitments chip and filter
count the same list, so the label, chip and filter agree even for a row
that mounts between ticks (it reads "Reminder due" until the next tick),
and the list only notifies when its ids change, so a tick that moves no
deadline past now does not rebuild open rows. The reminder hover no
longer promises the finding returns only "if it is still tripping": the
hourly reminder sweep brings it back regardless. An ended snooze reads
"snooze ended" until the next findings re-read drops the snoozed status.
The existing countdowns did not fit: CountdownTimer runs its own
one-second interval and has no day unit (a 7-day suppression would read
"167h 59m"), and the replication Next sync helper is private to that
table with sync-specific overdue tones.
An audit of every formatRelativeTime and
formatPlatformTableRelativeTimeValue caller found no other future
timestamps; the autopilot and alert snooze expiries already show an
absolute time.
This rebuilds PR 2613 on main after PR 2598 landed. PR 2613 was stacked
on two early PR 2598 commits whose relative-age contract wording main
has since replaced (every read of the shared clock now returns the wall
clock), so the future-time note sits under main's wording and before the
read-time age exception, and that exception now opens "The relative-age
rule" so it does not read as an exception to the future-time note.
|
||
|
rcourtman
|
d02d6441dd |
Merge origin/main into fix/unraid-disk-temperature-alerts
Main's Proxmox node cleanup section landed at the same alerts.md anchor as the Unraid disk temperature section; both are kept, main's first. |
||
|
rcourtman
|
539eadfab2 |
Judge filesystem alerts by their own threshold on config save
A config save re-judges every active alert. Agent filesystem alerts were re-judged by the host disk threshold, ignoring the filesystem's own override and the per-type DiskFillByType threshold CheckHost uses. Guest filesystem alerts were resolved by their "<guestID>-disk-<key>" resource ID, which matches no guest-disk override and names no guest, so the guest's own override was missed too. A filesystem override of 70 raised an alert at 80 under a default of 90; saving any alert setting resolved it and sent a recovery, and the next poll raised it again as a new alert. CheckHost and CheckGuest now resolve a filesystem's threshold through hostDiskUsageThresholdNoLock and guestDiskUsageThresholdNoLock, and reevaluation calls the same helpers: resolveHostAlertThresholdsNoLock for agent filesystem alerts, and a new resolveGuestAlertThresholdsNoLock that resolves a guest filesystem alert's guest by the ID parseGuestAlertIdentity reads from its resource ID. The helpers return override and per-type thresholds as owned copies, so evaluation still never normalizes the live config. TestConfigSaveJudgesFilesystemAlertsByTheirOwnThreshold covers an agent filesystem override, an NVMe fill threshold, a guest filesystem override and a guest override. With host.go, guest.go and config_runtime.go overlaid by their origin/main or #2696 copies, all four fail: the unchanged save resolves the alert. Stacked on #2696. |
||
|
rcourtman
|
aa7053a07e | Merge remote-tracking branch 'origin/main' into fix/alert-config-keep-unsent-fields | ||
|
rcourtman
|
ac631eadb8 |
Alert on the Unraid temperature an agent disk shows
A host agent's disk can show a temperature only its Unraid inventory reports: a SMART row whose smartctl probe returned none takes its Unraid row's reading, and a disk only the inventory lists is built from that row alone. The Physical Disks Temp cell, the Running Hot verdict and Patrol judge that reading against the alert disk temperature policy, but CheckHost read only the SMART row's own field, so a disk could read as at its alert threshold while no alert fired. unifiedresources.HostDiskTemperatureReadings now lists the reading each of a host's disks shows, built with the registry's own adapters: one per SMART row, then one per Unraid row whose disk key no SMART row carries (the history writer's rule). CheckHost judges those. SMART rows keep their alert resource IDs, an inventory-only disk uses its block device token, and per-type thresholds follow the disk type the disk shows. Only a reading collected now fires or resolves, so spun-down disks hold their alerts. When conflicting serials put two disks on one device, its alert judges the reading that stands highest against its own type's threshold. An inventory-only disk the agent's --disk-exclude patterns match raises nothing, and an alert it raised before the exclusion resolves on the next report, as the documented exclusion from monitoring already suppresses linked Proxmox health and wear alerts. Absence cleanup still runs only on a non-empty SMART list. Unraid's own disk temperature notifications do not make this redundant: Pulse already alerts on the smartctl readings and array state of the same hosts, which Unraid also notifies on. |
||
|
rcourtman
|
55e865ecb8 |
Move the config save regressions to the end of their file
main added TestUpdateConfigKeepsLowTriggerClearBelowTrigger (#2678) at the spot these tests were inserted, with its own import block, so the PR no longer merged. The tests now sit at the end of the file and the concurrency test waits on a channel instead of sync.WaitGroup, leaving the import line as main has it. |
||
|
Richard Courtman
|
7b884bc98a |
Chart the Unraid temperature an agent disk shows
A host agent's disk can show a temperature only its Unraid inventory reports. The adapter takes the Unraid row's reading when a SMART row has none, which happens when the agent's smartctl probe returns no temperature for a disk Unraid reads with its own per-disk SMART settings, and a disk the agent sends no SMART row for, such as a --disk-exclude match, is built from its Unraid row alone. The only agent disk temperature writer took the SMART row's own reading, and the unified sync skips agent disks as natively written, so these temperatures were shown live but never charted. writeHostSMARTMetrics now writes the reading the disk shows (HostSMARTDiskTemperature, the adapter's own rule), and a new pass writes each Unraid row's reading under HostUnraidDiskMetricID, the key its disk's metrics target reads, unless that key is one of the host's SMART rows' keys. An Unraid reading therefore never overwrites a SMART row's own at the same timestamp. Unraid rows sharing a key are one disk in the registry, which shows the latest of their readings, so the last row with a reading decides the sample. TemperatureCollected still keeps spun-down and expired-host readings out of history. Charting the fallback makes its row matching matter, so the fallback and the serial a row without one takes now need an Unraid row that describes the disk: one carrying the row's usable serial, or one naming the row's device path when the row is that whole block device, not a controller member (even the only one reported) or one of several rows on the path. matchUnraidDisk now tries usable serials across the whole inventory before device paths, and a placeholder serial matches nothing, so a row no longer takes another disk's Unraid row through a shared placeholder serial or an earlier path match. A lone member without a serial or WWN now charts under its own topology key, which its metrics target reads as written. |
||
|
rcourtman
|
7eb32f3181 |
Keep a low alert default's clear below its trigger (#2678)
## Problem
Setting an alert global default to a positive trigger of 5 or less, for
example Machines CPU at 1% in Alerts > Thresholds, stored
`agentDefaults.cpu` as `{trigger: 1, clear: 75}`, a clear far above its
trigger.
The thresholds page sends `clear = max(0, trigger - 5)`, which is 0 for
those triggers. `NormalizeAgentDefaults` derived `trigger - 5`, found it
non-positive, and fell back to the factory clear (75/80/85).
`ValidateHysteresisThresholds` never checked agent CPU, memory or disk,
so the pair survived. PBS CPU/memory, node temperature and
`internal/config/persistence.go` had the same fallback. On the save
path, validation repaired PBS and node temperature, but
`LoadAlertConfig` still turned a stored node temperature clear of 0 into
75; at startup `UpdateConfig` repaired that in memory.
Per-type `DiskFillByType` / `DiskTempByType` entries were worse: any
clear ≤0 reset the whole pair. A SATA disk temperature trigger of 4 from
the thresholds page was stored as 55, and an API-written fill entry of
`{88, 0}` became `{90, 85}`.
## What a user saw
- **Firing and resolving did not change for these pairs.**
`buildCanonicalMetricSpec` drops a clear at or above the trigger, and
the reducer then clears at the trigger. That held both before and after
this change. There was no flapping and no alert that never re-armed.
- `/api/alerts/config` and `alerts.json` served the factory clear above
the trigger.
- Firing agent alerts carried it as `metadata.clearThreshold`: 75 on a
1% CPU alert, and 85 on a disk alert still firing at 81.8%.
- A per-type disk trigger of 1–5 was silently replaced by the type
default.
- Machines metric bars: `resolveThreshold` in `metricThresholds.ts`
clamps the warning level to `min(clear, trigger)`, so a 1–5% agent
trigger had no warning band. Readings below the trigger, including 0,
now show amber instead of green, which matches guest, node, Docker and
Kubernetes bars at the same setting. This one is read from the code; I
did not exercise it in a browser.
## Fix
- PBS, node temperature, agent and per-type disk defaults now use
`normalizeThresholdPointer`, the normalizer Kubernetes, TrueNAS and
vSphere already use. A missing clear sits five points under the trigger,
floored at 0. A clear at or above the trigger is repaired. Zero still
means off for global defaults, and nil or negative still takes the
factory default. For the canonical per-type keys (the types the page
edits), a non-positive trigger still resets to the type default, because
the page cannot represent a per-type Off; other per-type keys are stored
as written, as before.
- `internal/config/persistence.go` drops its duplicate
agent/node-temperature and storage helpers and calls the same normalizer
through a new `NormalizeHysteresisThreshold`. A config saved before this
fix loads repaired.
- Agent CPU/memory/disk/disk temperature are added to
`ValidateHysteresisThresholds`, as every other family already is.
- One deliberate behaviour change for invalid pairs above the margin: an
API-written `{50, 80}` is repaired to `{50, 45}`, so it now has a real
recovery band where it used to clear at the trigger. This matches what
`EnsureValidHysteresis` already does for every other family.
- An alerts contract clause records the rule.
## Verification
- `internal/alerts/config/normalize_low_trigger_clear_test.go` runs
every default family and both per-type maps through the `UpdateConfig`
normalizer chain.
- `TestUpdateConfigKeepsLowTriggerClearBelowTrigger` pins the saved
config and a firing alert's `clearThreshold`.
- `persistence_test.go` checks the written `alerts.json` itself as well
as load repair.
- All new tests fail against the previous normalizers. Coverage cases
that pinned the old fallback (including one labelled "suspected source
bug") are updated.
- `go test ./internal/alerts/... ./internal/config/ ./internal/ai/`
passes.
- Live, on private mock stacks (baseline binary vs this branch): I saved
the payload the thresholds page sends, then read back
`/api/alerts/config`.
- Baseline: agent `{1,75}`, `{3,80}`, `{5,85}`; `diskTempByType.sata`
`{55,50}` for a 4°C trigger; firing agent alerts reported
`clearThreshold` 75/80/85.
- This branch: every pair `{t, 0}`, `diskTempByType.sata` `{4, 0}`, and
alerts reported 1/3/5.
- After a restart, this branch also repaired a hand-written
`alerts.json` holding `{1,75}` and node temperature `{2,75}`.
|
||
|
rcourtman
|
71a470129f |
Stop alert evaluation writing live thresholds outside the lock
Host and guest evaluation filled in the Clear of a per-disk override's threshold after dropping m.mu, writing through the live config's pointer. GetConfig clones that pointer under the read lock, which does not exclude the writer, and the saved override silently took whatever the last poll filled in. Docker evaluation pointed into m.config.DockerDefaults, and read the restart-loop and memory-limit settings, with no lock held while UpdateConfig replaced m.config. Clone the override's Disk threshold under m.mu before normalizing it, resolve Docker container thresholds from cloned defaults plus the override under the lock, and read the restart-loop and memory-limit settings under it. Drop getThresholdForMetricFromConfig: nothing has called it since the v6 split, and it normalized in place. TestAlertEvaluationLeavesTheLiveConfigToTheLock runs host, guest and Docker evaluation against GetConfig() and UpdateConfig(GetConfig()) under -race. With host.go, guest.go, docker.go and metric_runtime.go overlaid by their origin/main copies it fails: the live override's clear reads 85, and -race reports every site above. Stacked on #2692, which made GetConfig return an owned copy. |
||
|
rcourtman
|
5c55197900 |
Remove test-only Patrol context summary helpers (#2674)
Removes two exports from
`frontend-modern/src/features/patrol/patrolInvestigationContextModel.ts`
that only tests still called, and corrects the contract prose that
described the removed Patrol supporting-context surface as live.
## Why
`buildPatrolInvestigationContextSummary` (with its input type) and
`selectPatrolSupportingRecentChanges` lost their last production caller
in
|
||
|
rcourtman
|
a543907130 |
Type the Patrol run test's summary mock restore
The manual-run test restored the stalled loadIntelligenceSummary mock with mockResolvedValue(undefined), which the frontend type-check rejects because the store method resolves IntelligenceSummary | null. Restore it with null instead; the test behaviour is unchanged. |
||
|
rcourtman
|
ef45926b12 | Merge commit 'cd4e564fbf' into fix/proxmox-disk-temperature-alerts | ||
|
rcourtman
|
cd4e564fbf | Merge remote-tracking branch 'origin/main' into fix/disk-heat-remaining-judges | ||
|
rcourtman
|
b1fbee1338 |
Move TrueNAS re-judge tests clear of the cleanup tests
#2672 added its node cleanup tests right after TestCleanupAlertsForNodes, where this branch had added its TrueNAS disk temperature re-judge tests, so the branch no longer merged with main. Place the TrueNAS tests just before TestCleanupAlertsForNodes instead; no test changes. |
||
|
rcourtman
|
409c5482ee |
Read controller-member disk history at its written key
A physical disk behind a RAID controller (target such as megaraid,3) with no usable serial or WWN had its SMART history written under one key and read under another, so its temperature and SMART charts stayed empty. The host agent writes under HostSMARTDiskMetricID and Proxmox under PhysicalDiskMetricID, and both keys already scope the member (host-pve:sdd@ctrl0/megaraid,3). BuildMetricsTarget passed that source ID to PhysicalDiskMetaMetricID, which appended the topology again (host-pve:sdd@ctrl0/megaraid,3:sdd@ctrl0/megaraid,3). The reader now does what the Proxmox writer does: a fallback that already names the member is returned unchanged, and only a fallback without the topology gets it appended. That covers a Proxmox source ID from before members were scoped and the canonical resource ID a view falls back to. TrueNAS disks carry no controller target and are unaffected. SMART history needs no migration: no SMART writer used the doubled key, so samples already stored become readable. A linked node's agent with no SMART reading for a member, such as a standby row, files the device's I/O under the Proxmox disk's metrics target, which was the doubled key. That I/O now lands beside the member's SMART history, and samples already stored under the doubled key are left to age out. This carries forward the fix from PR 2661, whose branch conflicts with the disk ID stack it now sits on, and proves it against the real SMART and I/O writers in the live and rehydrated registries. |
||
|
rcourtman
|
ad1e34277d |
Stop alert config saves writing into the live config
SaveAlertConfig normalized the config it was handed in place, and its
callers hand it alerts.Manager.GetConfig(), a shallow copy whose maps
and threshold pointers were the running config's own. Alert activation
and the alert identity migration therefore wrote the live delay map and
threshold pointers outside the manager lock while evaluation read them.
The shallow copy also let UpdateConfig, which normalizes the override
map it is handed in place, rewrite the live overrides while
GET /api/alerts/config encoded another snapshot. A concurrent map read
or iteration plus a write is a fatal Go runtime error, so these paths
could kill Pulse.
AlertConfig.Clone is a typed deep copy. GetConfig returns one, so
callers own what they encode, persist and edit, and SaveAlertConfig
clones before normalizing, because a config just applied with
UpdateConfig is the live config. GetConfig has no per-resource caller
(once per poll cycle, backup check, request or Patrol run); a clone
with 1,000 overrides takes about 0.3 ms. A test fills every field and
fails when a new map, slice or pointer is not copied.
Persistence also stored and loaded an explicit 0 type-level alert delay
("no delay") as the 5-second factory delay, which the noisy-gauge rule
stretches to 300 seconds for memory and temperature, while the manager
keeps 0. The shared map write was what made the running manager agree
with the file; without it, memory would say 0 and disk 5. Persistence
now keeps 0 on save and load like NormalizeTimeThresholds, and only a
missing or negative delay takes the default. Pulse saves never wrote a
0, and the first v6 start re-saves alerts.json, so only a newly sent or
hand-edited 0 changes meaning.
On main, a reproducer taking the activation, migration and GET steps
against live evaluation died with "fatal error: concurrent map
iteration and map write" in 5 of 5 plain runs and reported 16 races
under -race. It passes with this change.
|
||
|
rcourtman
|
22b8b995ca |
Read the shared wall clock on Alerts overview cards (#2673)
The overview hook kept its own minute signal and handed its value to each card as "now". A card that mounted between ticks measured from up to a minute earlier: an alert raised 70 seconds ago read "this minute", a reading already past the 10-minute stale cut-off led the card as the live one, a just-raised alert stayed out of Triggered (24h) until the next tick, and the stale flip and 24h drop landed up to a minute late. The card now reads useRelativeTimeNow for its started age and the live reading's stale cut-off, and the hook reads it for the 24h count, so they measure from the wall clock and move with the shared 30-second tick. The minute interval stays only for the delivery diagnoses refresh, a server read that should not double its rate. |
||
|
rcourtman
|
2d6a2937fc |
Keep platform metric alerts through Proxmox node cleanup (#2672)
`CleanupAlertsForNodes` runs on every Proxmox node poll
(`monitor_pve_storage.go`) and on every mock alert tick. It removes
alerts whose `Node` is not a current Proxmox node or PBS instance, and
keeps the types `shouldPreserveAlertOutsideNodeCleanup` lists.
Kubernetes, TrueNAS and vSphere metric alerts carry a cluster or
platform host as `Node`, and a display label (`TrueNAS Disk`,
`Kubernetes Pod`, `vSphere VM`) in `metadata.resourceType`. The preserve
list only had the bare words `truenas`, `kubernetes`, `k8s` and
`vmware`, so none of these labels matched. In an estate with Proxmox
plus one of those platforms:
- every Proxmox poll deleted these alerts without resolving them;
- their reducer incident stayed firing, so the next unified evaluation
re-created each one and handed it to the alert callback again as a new
alert.
## Change
`shouldPreserveAlertOutsideNodeCleanup` keeps an alert that carries
`metadata.platformType` (set on every unified Kubernetes/TrueNAS/vSphere
metric alert), or whose label maps through `CanonicalAlertResourceType`
to one of those platform types. The label path covers alerts persisted
before `platformType` existed. A guest alert whose Proxmox node left the
estate is still removed.
## Evidence
- Unit probe on main: a TrueNAS disk temperature alert, a Kubernetes pod
CPU alert and a vSphere VM CPU alert were all deleted by
`CleanupAlertsForNodes({"pve1": true})`. Over three
cleanup-then-evaluate cycles the TrueNAS alert went to the alert
callback 3 times; with this change, 0 times and it stays active.
- Private mock stack: TrueNAS disk temperature alerts kept vanishing
between mock alert ticks, while the disks kept their readings in
`/api/resources`. With this change they stay open.
## Tests
`TestCleanupAlertsForNodesKeepsPlatformMetricAlerts` (in
`internal/alerts/alerts_test.go`) raises a TrueNAS disk, a TrueNAS pool,
a Kubernetes pod and a vSphere VM alert, then runs the cleanup three
ways: as raised, with only `platformType` (label unrecognised), and with
only the label (`platformType` removed).
`TestCleanupAlertsForNodesStillRemovesMissingProxmoxNodeAlerts` keeps
the removal of a guest alert on a removed Proxmox node. Each preserve
condition was mutation-checked: removing either one fails the first
test.
|
||
|
rcourtman
|
e30b44ea8f |
Read a silent in-guest agent's guest drawer readings as last known (#2669)
A Pulse agent linked into a VM keeps its last report on the VM resource after it stops reporting: the registry copies the agent's whole facet, `agent.stale` included, onto the guest. The guest drawer never read that flag. A quiet agent's Guest RAID kept its live colours and "Rebuild: 42% · 118M/sec", its network rows and version read as current, and nothing said the agent had gone. ## Change - `useWorkloads` maps `agent.stale` onto the workload as `agentStale`, and sets `disksFromAgent` when Proxmox reported no guest filesystems and the agent's own `disks` filled in. Filesystem rows are the only agent reading with a second source. - When the agent is stale, the guest drawer: - passes `HOST_AGENT_STOPPED_REPORTING_REASON` to `RaidCard` (`lastKnownReason`, #2652); - puts a status line over muted Network rows; - marks the Pulse Agent row "· stopped reporting"; - reads agent-supplied filesystem rows through the existing `last-known` disk read state. - Agent-supplied filesystem rows now follow the agent's freshness instead of the Proxmox guest-read reason. Before, a VM with a **live** linked agent but no qemu-guest-agent showed the agent's filesystems as "Usage unavailable · ?/328 GB" with advice to install qemu-guest-agent. A stopped guest still reads those rows as unavailable, whether its status or the Proxmox `vm-stopped` reason says so. Proxmox's own guest filesystems keep their read state. ## Verification - Private mock stack built from `main` plus this change. Two host agents were held past their reporting lease and two left live, then each was linked into a VM through `POST /api/resources/{vm}/link`. The two silent VMs serve `agent.stale: true` with the agent's raid, disks and interfaces; the two live ones carry no stale flag. - Playwright at 1440 and 390: - silent VMs read as described above; - the live VM with agent-only filesystems shows usage with bars; - the live control is unchanged; - no horizontal overflow. - New tests in `GuestDrawer.test.tsx` cover: - silent agent; - live agent with Proxmox agent-not-running; - Proxmox filesystems under a silent agent; - stopped guest, with and without the reason; - stale flip in both directions on an open drawer. The new tests fail against the old drawer. `useWorkloads.test.ts` asserts source selection with distinct mountpoints. - Frontend suites over Workloads, hooks, utils, shared, platformPage and Infrastructure pass, apart from the Mac-only PTY tests in `agentInstallCommand.test.ts`. - WorkloadsSurface grows by 0.22 kB gzip (64.85 kB against a ~71.9 kB cap), and the chunk set is unchanged. ## Not covered - The workload table's disk cell and the drawer History still read the aggregate disk metric with the Proxmox reason. - The agent's OS, its addresses in the identity rows, and the lazily loaded Physical Disks card's health and wearout carry no stale cue yet. |
||
|
rcourtman
|
6d8dce2cae |
Remove unread Patrol summaryStats and its audit rule
usePatrolIntelligenceState() still returned summaryStats, a closure that
counted critical, warning and active Patrol findings plus fixed outcomes.
Its only reader was PatrolIntelligenceSummary.tsx, deleted in
|
||
|
rcourtman
|
f7ac26b573 |
Keep alert settings a config save leaves out
The Thresholds page builds its PUT /api/alerts/config body from the settings it edits. It has no control for flapping detection, alert TTL cleanup or custom rules, so it never sends flappingEnabled, maxAlertAgeDays, maxAcknowledgedAgeDays, autoAcknowledgeAfterHours or customRules. The handler decoded the body into a zero AlertConfig, so every thresholds save turned flapping detection and TTL cleanup off, reset the flapping tuning, dropped custom rules, and persisted that. The page's flapping, aggregation and ioNormalization objects are not AlertConfig fields and never enabled anything. The handler now hands the body to Manager.ApplyConfigUpdate, which reads the stored config, merges and applies it under the manager lock, so two concurrent partial saves cannot revert each other. The body is decoded on its own, so the keys it carries mean exactly what encoding/json makes of them; every top-level field it left out is copied from a JSON round trip of the stored config, sharing no maps with it. ApplyConfigUpdate returns a cloned snapshot, which the handler persists, because SaveAlertConfig normalizes the config it is handed in place and used to write into the live config's maps. Saves run one at a time so alerts.json ends with the last applied config. A body that is not a JSON object, null included, is a 400. Stored configs are not rewritten, so a value an earlier save zeroed stays as it is. On a private stack, a node CPU change saved from the Thresholds page zeroed all four fields in memory and in alerts.json on main; with this change the same save kept them and stored the new threshold. |