Commit Graph

282 Commits

Author SHA1 Message Date
rcourtman a36bb254d5 Fix storage trend metrics targets 2026-05-14 22:12:16 +01:00
rcourtman 3580a5ed6d Clarify storage topology and recovery guards 2026-05-14 20:51:32 +01:00
rcourtman a6c460daa0 Stabilize infrastructure realtime identity 2026-05-14 11:50:29 +01:00
rcourtman 2be14562ee Preserve infrastructure continuity on first login
Ensure unified resource snapshots include recent standalone host-agent continuity so Infrastructure does not briefly undercount connected systems after login or restart.
2026-05-13 23:36:17 +01:00
rcourtman 5fd05efa83 Add connection-degraded alert for wedged platform connections
A Proxmox host wedged on a ZFS deadlock yesterday took the cluster API poll
with it (context deadline exceeded). The unified connections aggregator
flipped the Connection from active to stale to unreachable, and the
Settings / Infrastructure page rendered the right badges, but no top-nav
alert ever fired because nothing was actively notifying off that derived
state. Patrol's deterministic triage flagged it every minute, but its LLM
investigation stage has been broken since 2026-02-26 so flags never
escalated into user-visible findings. Result: a 3 hour outage I only
noticed because I happened to open Settings.

This wires an active notification off the same connection state the
Settings badges already use:

- internal/alerts/connection.go: new CheckConnection +
  clearConnectionDegradedAlert that fire connection-degraded after three
  consecutive stale or unreachable observations. Severity scales: stale
  warning, unreachable / unauthorized critical. Clear runs through the
  same recovery-confirmation gate as clearNodeOfflineAlert so a single
  flap back to active doesn't silently resolve a real outage. Paused,
  disabled, and non-platform connections are no-ops.

- internal/api/connections_alerts.go: snapshot translator that turns
  api.Connection into the narrow alerts.ConnectionSnapshot view. Keeping
  the snapshot type inside the alerts package preserves the existing
  api -> monitoring import direction; the monitor would have cycled if
  it called back into api directly.

- internal/monitoring: new SetConnectionsSnapshotLister hook + a
  per-tick checkConnectionAlerts call in the main poll loop, alongside
  the existing evaluate*Agents passes.

- internal/api/router.go: register the lister closure on r.monitor so
  the alerts loop sees the same Connection rows the HTTP handler does.

- internal/alerts/specs/types.go: add "connection" to the migration
  bridge list of accepted ResourceTypes, alongside node / docker-host /
  proxmox-disk / etc. The connection concept doesn't have a canonical
  unified resource type yet; this matches the existing pattern for
  alert-keyed resources that aren't first-class canonical.

Test coverage in internal/alerts/connection_test.go covers active never
fires, three stale observations escalate from pending to warning,
unreachable escalates warning to critical, unauthorized fires critical
cold, paused / disabled / agent never fire, recovery confirmation gate,
and a stale flap during recovery resets the gate.
TestResourceAlertSpecValidateAllowsConnectionMigrationBridgeType mirrors
the existing migration-bridge proof tests for the new type.
2026-05-12 00:29:04 +01:00
rcourtman cb772737e7 Fix platform identity source contracts 2026-05-08 14:54:08 +01:00
rcourtman 31d07e74ce Fix Unraid storage health explanations 2026-05-08 12:18:09 +01:00
rcourtman d6ca8b12e6 Add agentless availability targets
Refs #1460
2026-05-06 10:35:34 +01:00
rcourtman 1a3e5ec27d Fix tenant monitor broadcast nil hub panic 2026-05-05 16:25:00 +01:00
rcourtman 81b31e4d3b Remove monitored-system volume caps
Retire runtime/API/UI monitored-system volume enforcement now that infrastructure monitoring is no longer capped.

Keep only legacy metadata scrubbing and purchase-start compatibility for old max_monitored_systems references.

Rename the remaining preview surface to monitored-system impact and make previews explanatory rather than save-blocking.

Update subsystem contracts and RA7 evidence for the caps-retired invariant.
2026-05-05 12:59:59 +01:00
rcourtman db97478566 Reduce metrics rollup write amplification
Refs #1124
2026-05-03 21:43:20 +01:00
rcourtman fe597554c3 Fix backup orphan inventory readiness
Refs #1352
2026-05-01 20:28:12 +01:00
rcourtman 09231a9ef7 Fix Docker agent reconnect token binding
Refs #1447

Use the canonical Docker host identity for token bindings after a matched reconnect, preserve previous/current identity aliases for recreated containers, and keep rejecting token reuse by genuinely different Docker hosts.

Record the RC3 release-gate evidence for the Docker-in-LXC reconnect path.
2026-05-01 20:28:11 +01:00
rcourtman 82ba940524 Merge linked host disks into guest overviews
Fixes #1438
2026-04-30 14:37:43 +01:00
rcourtman af7d727d45 Gate RAID rebuild alerts on mdstat operation
Parse the /proc/mdstat operation keyword for mdadm arrays and propagate it through host reports, models, unified resources, monitoring views, alert metadata, and AI storage summaries.

Treat recovery and reshape as rebuild signals while silencing routine check and resync maintenance, with fallback rebuild detection only when no mdstat operation is available.

Tests cover mdstat operation parsing plus recovery, check, and resync alert behavior.

Fixes #1446
2026-04-30 14:31:14 +01:00
rcourtman 386099aeee Surface ZFS pool membership on physical disks 2026-04-23 20:38:33 +01:00
rcourtman 0a4584535e Preserve agent rows across monitor reloads 2026-04-22 20:48:16 +01:00
rcourtman d573d3a85f Preserve standalone host continuity across restart
Refs #1402
2026-04-15 16:23:42 +01:00
rcourtman b73dab2b64 Port v5 alert lifecycle and linked agent overrides 2026-04-15 12:18:57 +01:00
rcourtman 05fa111ca1 Stabilize backend race tests for v6 RC publish 2026-04-11 22:46:34 +01:00
rcourtman 1e28a03b57 Stabilize rc1 mock mode and metrics history 2026-04-11 16:47:37 +01:00
rcourtman b64782c083 Align mock state resources with canonical contract 2026-04-11 13:40:01 +01:00
rcourtman 4692d88a0d Speed up demo mock chart loading 2026-04-10 14:26:55 +01:00
rcourtman b846d66fd0 Gate release mock fixtures behind demo entitlement 2026-04-10 12:33:57 +01:00
rcourtman 4524baa1b0 Gate grandfather floor capture on settled supplemental inventory 2026-04-08 17:37:55 +01:00
rcourtman 83f4b28b3a Canonicalize mock chart history continuity 2026-04-01 20:41:56 +01:00
rcourtman 2fb76b90af Add guest agent meminfo VM memory fallback 2026-04-01 19:00:48 +01:00
rcourtman 96e6745737 Serve fresh alert snapshots from monitor state reads 2026-04-01 17:39:35 +01:00
rcourtman 77d18aed4d Forward-port Proxmox SMART disk merge semantics 2026-04-01 15:29:11 +01:00
rcourtman bbcb153140 Unify mock chart history timelines 2026-04-01 01:04:06 +01:00
rcourtman 9187ee727f Make fixture graph the only mock runtime API 2026-03-31 15:42:16 +01:00
rcourtman a09f61d214 Modernize platform mock runtime fixtures 2026-03-31 13:36:11 +01:00
rcourtman c511638acc Wire TrueNAS and VMware into mock runtime 2026-03-31 12:53:08 +01:00
rcourtman 3ac9ca2a15 Project VMware activity onto canonical timelines 2026-03-30 21:29:54 +01:00
rcourtman 412c9821fc Implement VMware metrics history floor 2026-03-30 20:13:53 +01:00
rcourtman a71f370731 feat(truenas): add native disk chart fallback 2026-03-29 21:17:51 +01:00
rcourtman aab6b5e69d feat(truenas): sync api-backed disk history 2026-03-29 18:35:47 +01:00
rcourtman 16f976537f feat(truenas): project system telemetry through canonical host history 2026-03-29 17:56:42 +01:00
rcourtman e14ddcf348 feat(truenas): project app stats through canonical workload history 2026-03-29 17:17:57 +01:00
rcourtman 2afb96ee13 fix(release): align api and hostagent rc contracts 2026-03-26 17:08:48 +00:00
rcourtman da20a171dd Project incident timelines from canonical history 2026-03-20 11:42:26 +00:00
rcourtman 778a2577b6 feat: Pulse v6 release 2026-03-18 16:06:30 +00:00
rcourtman 2fe22c3308 fix(backups): prevent template backups from being flagged as orphaned
Proxmox VM/LXC templates are intentionally excluded from the monitored
guest list, but their backup files exist on storage. The orphan-detection
logic was firing for every template backup because the VMID was never
in the guest lookup maps.

Fix: track template VMID→node pairs in State.templateVMIDs (unexported,
not serialised to API/frontend) during the resources poll loop, expose
via StateSnapshot.TemplateVMIDs, and use in both buildGuestLookups() and
the storage backup node-resolution map so orphan detection treats template
backups as valid. Also preserves the template map through the cluster
health grace-period path (zero-resource preservation), the partial-node
grace-period path, and clears it on instance removal.

Closes #1352
2026-03-17 09:04:22 +00:00
rcourtman caff845c1a fix(ui): use Proxmox tag colours from datacenter config
Pulse was generating tag colours from a hash of the tag name instead
of using the colours configured in Proxmox. Now polls /cluster/options
once per PVE instance and merges the tag-style colour map into state,
which the frontend uses as the first-priority colour source for tag
badges. Falls back to the existing special-tag and hash-based colours
when Proxmox hasn't set a custom colour for a tag.
2026-03-15 19:49:46 +00:00
rcourtman d05a00b931 fix(monitoring): smooth transient VM memory fallback spikes 2026-03-10 23:06:17 +00:00
rcourtman afcfb23a30 fix(monitoring): retain intermittent FreeBSD SMART data 2026-03-10 22:52:25 +00:00
rcourtman 7dab977d91 Add split memory bar showing Used | Cache | Free segments (#1302)
Show reclaimable buff/cache as a distinct amber segment between used
(green) and free (gray) in the memory bar. This explains why Pulse's
memory percentage differs from Proxmox: Pulse reports cache-aware
usage (MemAvailable) while Proxmox includes cache as used (Total-Free).

Backend: add Cache field to Memory model, derived from MemInfo
(Available - Free). Only uses MemInfo.Free (not FreeMem fallback) to
avoid inflating cache by the balloon gap on ballooned VMs.

Frontend: StackedMemoryBar renders three segments with tooltip
breakdown. Tooltip Free accounts for balloon limit when active.
Percentage label and alerts remain cache-aware (unchanged).
2026-03-10 10:16:14 +00:00
rcourtman 7a394ed724 Use explicit success flag for disk carry-forward guard (#1319)
Replace the diskUsage <= 0 heuristic with a diskFromAgent bool that is
only set when the guest agent actually returns valid filesystem data.
Prevents carry-forward from firing on a genuine 0% disk reading.
2026-03-09 18:54:27 +00:00
rcourtman 9c279732f7 Skip disk carry-forward when guest agent is explicitly disabled (#1319)
Prevents stale disk data from persisting indefinitely in the efficient
poller when a user disables the guest agent after it had been providing
data.  Matches the fallback poller's agent-disabled exclusion.
2026-03-09 18:37:38 +00:00
rcourtman abbd0df609 Fix disk metric spikes when guest agent intermittently fails (#1319)
Carry forward previous cycle's disk data when the QEMU guest agent
times out or errors, instead of falling back to Proxmox cluster/resources
which always reports 0 for VM disk usage.  Applied to both polling paths
(pollVMsAndContainersEfficient and pollVMsWithNodes) with safety guards
against uint64 underflow and permanent-failure exclusions.
2026-03-09 18:23:15 +00:00