Cloned VMs that keep the same /etc/machine-id collapse into a single
Docker host in Pulse, with each clone's report silently overwriting the
other (#1584). The server now tracks per-identity hostname and machine
ID observations and flags a conflict when a value flaps back to one
already seen inside a 15-minute window, a signature a one-time rename
never produces. The conflict rides the DockerHost model through unified
resources, and the Docker page shows a warning naming the flapping
hostnames with the machine-id remedy. The warning self-clears once one
clone stops reporting for the window.
The alert engine fired powered-off alerts for every stopped VM and
container regardless of whether the guest was configured to autostart.
This meant intentionally-stopped guests (onboot=0) generated alarm
fatigue — in the live lab, 7 of 12 alerts were noise from stopped VMs
that are deliberately off.
Now the alert engine checks the Proxmox onboot setting before firing:
- onboot=true → stopped is unexpected, alert (preserved behavior)
- onboot=false → stopped is expected, suppress
- onboot=nil → unknown, alert (preserved behavior)
For containers, onboot is parsed from the already-fetched container
config in enrichContainerMetadata (no new API call). For VMs, the
config is fetched only for stopped VMs via GetVMConfig (one extra call
per stopped VM, running VMs are unaffected).
Models: add OnBoot *bool to VM and Container.
Alerts: add onboot-aware suppression branch in CheckGuest.
Monitoring: add parseProxmoxOnBoot helper and fetchVMOnBoot.
v5 modeled memory as used | cache | free (Memory.Cache, 'reclaimable
buff/cache') and the memory bar's tooltip carried a 'Shown in Proxmox'
row explaining why Pulse's percentage reads lower than the Proxmox UI's
cache-inclusive number — a recurring support question. The v6 rebuild
deleted the field from the backend model, so the split and the
reconciliation vanished product-wide. Flagged by the Proxmox overview
parity audit.
Backend: re-add Memory.Cache; split it out via a shared
splitReclaimableMemory helper at the node resolver (node status reports
truly-free directly) and the VM builder (when guest meminfo reported
free pages); transport as proxmox.memoryCache on unified resources
alongside swap/balloon; mock mode populates plausible cache for nodes
and VMs.
Frontend: cache prop on StackedMemoryBar with the v5 muted-amber
segment between active and balloon, tooltip rows for reclaimable cache,
truly-free Free (balloon-capped), and the 'Shown in Proxmox'
reconciliation; guest and node memory adapters normalize free to
truly-free at the boundary; guest and node drawers grow a Reclaimable
cache row; the Proxmox nodes table passes cache and node swap through.
Nodes that set up SSH temperature monitoring before v6.0.0-rc.6 have an
authorized_keys entry locked to `sensors -j`, so the pulse-sensors wrapper
never runs and SMART (SATA/SAS) disk temperatures silently never arrive;
only NVMe shows via kernel hwmon (what the #1471 reporters hit). The server
already detected the legacy payload but only logged it.
- models.Temperature gains LegacySensorsFormat, set by parseSensorsJSON
when the payload is raw sensors -j, and preserved across the host-agent
merge; it rides the existing temperatureDetails projection on unified
node resources.
- New PlatformOutdatedSensorSetupNotice on the Proxmox page, data-gated:
collection succeeded AND payload was legacy AND the node has SATA/SAS
disks without temperatures, so nodes whose disk temps arrive via a host
agent stay clean. Copy points at re-running the node setup script, which
replaces the locked key entry and installs the wrapper.
- monitoring.md contract extended with the legacy-format marker behavior.
resourceFromStorage and resourceFromDockerContainer stamped LastSeen with
time.Now() at conversion because their source models carried no poll
timestamp. The registry rebuilds from the retained state snapshot every
cycle, so those resources re-reported a fresh sighting each rebuild even
after their upstream source (PVE instance, docker host agent) stopped
delivering, and their per-source SourceStatus could never go stale via
markStaleLocked. 53faa4e46 fixed this fabrication at the ingest layer but
left these two adapter-level stamps.
- models.Storage gains LastSeen (omitzero), stamped where entries are
built: the PVE storage poll (poll start time, including synthesized
cluster-shared entries; preserved entries for unpolled nodes keep their
old stamp), the PBS datastore conversion (PBS instance sighting), Ceph
pool projection (cluster LastUpdated), and the mock generator (offline
mock nodes get a backdated stamp so the stale path renders).
- resourceFromStorage passes storage.LastSeen through; zero stays zero
("never seen") instead of becoming conversion time.
- resourceFromDockerContainer uses host.LastSeen: containers are delivered
wholesale with each host report, so the host report timestamp is the
container sighting. This matches every other docker sub-resource adapter
(services, tasks, volumes, networks, images already use host.LastSeen).
- ingestStorage routes PBS-poller datastore entries (instance "pbs-<name>",
type pbs) to SourcePBS, parented to the PBS instance. Keying them
SourceProxmox would judge their freshness against the 60s Proxmox stale
threshold while PBS polls every 60s by default, flapping healthy
datastores stale between polls; SourcePBS carries the cadence-matched
120s threshold. PVE-reported pbs-typed storage.cfg backends stay
SourceProxmox. Side effect: syncUnifiedStorageMetrics no longer skips
PBS datastore storage, so those entries gain usage history.
- storageFromReadStateView round-trips LastSeen so the legacy storage API
reports the honest sighting; mock refresh re-stamps available storage on
each simulated poll.
The parent host/node staleness was already honest, so platform pages
reflected outages at the parent level; this makes the per-resource source
freshness honest too.
Tighten v5-to-v6 upgrade safety, release installability, provider MSP mode handling, AI cost accounting, metrics flushing, and frontend guardrails for the v6.0.0 GA candidate.
On a Proxmox node, physical disks collected by the host agent were keyed by
the NVMe controller (e.g. "nvme0 [nvme]") instead of the namespace, reported
sizeBytes 0 (or a stale filesystem-usage value), and flickered as the agent
reading intermittently replaced the authoritative Proxmox disks/list reading.
Root causes:
- smartctl --scan-open reports NVMe disks by their controller char device
(/dev/nvme0), and that scan label became the reported devPath.
- DiskSMART carried no capacity, so the server backfilled size by matching the
SMART device against host filesystem-usage entries, which never match a whole
partitioned/LVM/ZFS disk, leaving size 0.
- The unified-resource merge let the agent's controller label overwrite the
canonical Proxmox /dev/... devPath.
Fixes:
- The agent now reports the canonical block device (an NVMe controller resolves
to its namespace) and the authoritative capacity from /sys/block, with the
smartctl user_capacity / nvme_total_capacity as a cross-platform fallback.
Disks behind multiplexing controllers (megaraid, cciss, areca) keep their
disambiguating label and smartctl-reported size.
- SizeBytes flows through the agent report, host model, and adapter; the
filesystem-usage match is demoted to a legacy fallback.
- The merge keeps a canonical /dev/<device> devPath and never downgrades it to
a scan label, so an un-updated agent can no longer corrupt Proxmox data.
Refs #1483.
Closes the only API-coverage gap from the Docker / Kubernetes IA
maturity review: Roles, ClusterRoles, RoleBindings, and
ClusterRoleBindings now flow from the Kubernetes agent through the
canonical resource registry into the Kubernetes platform-page
Configuration tab.
Agent: pkg/agents/kubernetes/report.go gains four new report struct
types that carry summary counts plus subject-kind sets; individual
subject names and full PolicyRule contents are deliberately omitted
so Pulse stays a "what permissions exist where" surface, not an RBAC
enumeration tool. internal/kubernetesagent/agent.go gains four
collectors that call rbacv1.RoleList/ClusterRoleList/etc. through the
existing runKubernetesCallWithRetry wrapper, matching the
ServiceAccount collector's RBAC-forbidden retry pattern.
Canonical: internal/models mirrors with NormalizeCollections coverage;
convert* funcs in internal/monitoring/kubernetes_agents.go translate
agent report -> model; ResourceTypeK8sRole / K8sClusterRole /
K8sRoleBinding / K8sClusterRoleBinding join the canonical type set;
registry ingest* + adapter resourceFrom* functions emit one Resource
per RBAC object with ruleCount / roleKind / roleName / subjectCount /
subjectKinds / aggregationLabels on the K8s meta; search mapping in
internal/api/resources.go and the privacy allow-list in
internal/api/org_handlers.go pick up the four new type tokens; the
K8s privacy category in unifiedresources/policy_metadata.go classifies
them like the rest of K8s.
Frontend: ResourceType union + ResourceKubernetesMeta carry the new
kinds and RBAC summary fields; KubernetesPageSurface query asks for
them; the page model buckets them into the Configuration group;
KubernetesConfigTable renders Role / ClusterRole rule counts and the
aggregated flag, plus RoleBinding / ClusterRoleBinding role refs and
"N subjects · Kind1, Kind2 +overflow" subject summaries.
Curated demo seeds per-namespace Roles + RoleBindings plus an
aggregated ClusterRole + ClusterRoleBinding for pulse-demo-monitoring
in each cluster so the Configuration tab renders 18 RBAC rows across
the three demo clusters.
Contracts updated for the canonical-shape guard: monitoring,
api-contracts, unified-resources, frontend-primitives,
organization-settings (canonical) plus agent-lifecycle and
storage-recovery (dependent via Extension Points). Verification
proofs extended: kubernetes_registry_test.go, kubernetes_agents_test.go,
agent_inventory_test.go (new TestCollectRBACInventoryReportsSummaryCountsOnly
that pins the subject-name-omission contract), demo_scenarios_test.go,
adapter_coverage_test.go, contract_test.go, org_handlers_test.go,
resourceIdentity.test.ts, reportingResourceTypes.test.ts,
KubernetesConfigTable.test.tsx, and the
subsystem_lookup_test.py line-anchor bumps that the contract edits
shifted (api-contracts 246 -> 253, organization-settings 92 -> 93).
Verified:
- go build ./internal/... ./cmd/... clean
- go test ./internal/unifiedresources/..., ./internal/mock/...,
./internal/kubernetesagent/..., ./internal/api/...,
the K8s subset of ./internal/monitoring/... all clean (three
pre-existing unrelated monitoring failures noted earlier remain
unchanged by this commit)
- npm run type-check, lint:eslint, lint:theme,
lint:canonical-platforms clean
- vitest: 70 K8s frontend tests pass including the new RBAC render
coverage in KubernetesConfigTable.test.tsx
- browser proof on /kubernetes/configuration: 36 config rows
including 18 RBAC rows across three clusters; ClusterRole
"pulse-demo-monitoring" shows "12 rules · Aggregated";
ClusterRoleBinding shows "3 subjects · Group, ServiceAccount +1"
Collect native Kubernetes config, policy, and autoscaling objects.
Project the new resource types through API filters, unified resources, mock fixtures, and Kubernetes tabs.
Keep Secret inventory metadata-only and route k8s-secret policy as restricted local-only.
Parse the /proc/mdstat operation keyword for mdadm arrays and propagate it through host reports, models, unified resources, monitoring views, alert metadata, and AI storage summaries.
Treat recovery and reshape as rebuild signals while silencing routine check and resync maintenance, with fallback rebuild detection only when no mdstat operation is available.
Tests cover mdstat operation parsing plus recovery, check, and resync alert behavior.
Fixes#1446
Proxmox VM/LXC templates are intentionally excluded from the monitored
guest list, but their backup files exist on storage. The orphan-detection
logic was firing for every template backup because the VMID was never
in the guest lookup maps.
Fix: track template VMID→node pairs in State.templateVMIDs (unexported,
not serialised to API/frontend) during the resources poll loop, expose
via StateSnapshot.TemplateVMIDs, and use in both buildGuestLookups() and
the storage backup node-resolution map so orphan detection treats template
backups as valid. Also preserves the template map through the cluster
health grace-period path (zero-resource preservation), the partial-node
grace-period path, and clears it on instance removal.
Closes#1352
Pulse was generating tag colours from a hash of the tag name instead
of using the colours configured in Proxmox. Now polls /cluster/options
once per PVE instance and merges the tag-style colour map into state,
which the frontend uses as the first-priority colour source for tag
badges. Falls back to the existing special-tag and hash-based colours
when Proxmox hasn't set a custom colour for a tag.
Show reclaimable buff/cache as a distinct amber segment between used
(green) and free (gray) in the memory bar. This explains why Pulse's
memory percentage differs from Proxmox: Pulse reports cache-aware
usage (MemAvailable) while Proxmox includes cache as used (Total-Free).
Backend: add Cache field to Memory model, derived from MemInfo
(Available - Free). Only uses MemInfo.Free (not FreeMem fallback) to
avoid inflating cache by the balloon gap on ballooned VMs.
Frontend: StackedMemoryBar renders three segments with tooltip
breakdown. Tooltip Free accounts for balloon limit when active.
Percentage label and alerts remain cache-aware (unchanged).
Host agents removed from the UI would reappear on the next report cycle
because there was no rejection mechanism — unlike Docker agents which
already had resurrection prevention. Mirror the Docker agent pattern:
- Track removed host IDs in a `removedHosts` map with 24hr TTL
- Persist removal records in `State.RemovedHosts` for frontend display
- Reject reports from removed hosts in `ApplyHostReport()`
- Add `AllowHostReenroll()` + API route to clear the block
- Show removed host agents in the Settings UI with "Allow re-enroll"
- Sync removed-agent maps from state on startup for all agent types
- Fix mock integration snapshot missing `RemovedDockerHosts` field
The --disk-exclude agent flag only filtered local metric collection but
had no effect on server-side Proxmox disk health and SSD wearout alerts,
which poll the Proxmox API directly. Users excluding disks (e.g.
--disk-exclude sda) still received alerts for those disks.
Agent now sends its DiskExclude patterns in each report. The server
stores them on the Host model and consults them during Proxmox disk
polling — excluded disks get a synthetic healthy status passed to
CheckDiskHealth so any existing alerts clear immediately.
Also adds FreeBSD pseudo-filesystem types (fdescfs, devfs, linprocfs,
linsysfs) to the virtual FS filter and /var/run/ to special mount
prefixes, fixing false disk-full alerts on FreeBSD for fdescfs mounts.
When two standalone (non-clustered) PVE hosts share the same storage (NFS,
etc.), both instances see the same backup files during polling. Each instance
creates its own StorageBackup entry, causing guests with the same VMID on
different hosts to incorrectly show each other's backups.
Detect shared-storage duplicates by checking if the same volid appears across
multiple instances. When it does AND the VMID is ambiguous (exists on multiple
instances), skip the backup in SyncGuestBackupTimes rather than guessing which
instance owns it. This uses the same ambiguity pattern already applied to PBS
backups.
Fixes#1177
rcourtman