Keep explicit status/current zero availability through decoding and JSON copies so genuine exhaustion cannot become a healthy cache fallback. Connected direct/cluster collector controls cover canonical memory, History and breach/recovery. Complete race validation also exposed a shared-logger guidance fixture race: keep guidance on its caller logger with ordinary fallback, and stop the named scheduler-health fixture monitor. Message, backup precautions, command fences, thresholds and dependencies are unchanged; prior failed proof receipts remain retained.
Change-source: pulse-maintainer
Incorporate the exact observed upstream head additively, retaining reviewed tip 1c1ba27fe6 and every accepted commit. Preserve published source-keyed node carry-over, silent-agent temperature bounds and memory identity. PR2576 remains unchanged while the owned combined-Patrol browser proof is unavailable; composition does not establish validation, publication or installed recovery.
Change-source: pulse-maintainer
The poller reads the previous poll's nodes back from the unified read
state, but previousNodeFromView returned the registry's unified resource
ID and no temperature. Every lookup keyed on the Proxmox source ID
missed: nodeLastOnline in the offline grace policy, the temperature
carry, the CPU low/record tracking (reset to the current reading every
poll) and the node memory carry. When a whole instance poll failed,
markPVEInstanceNodesUnreachable wrote the projected nodes back, so each
failed poll re-ingested the node under a new unified ID. Cluster members
also lost the cluster identity their canonical ID derives from, so they
changed ID for the length of an outage.
The projection now returns the source ID, the stored temperature
details, the cluster name and the linked agent ID, and
previousNodesForInstance restores the config-derived identity stamps
(provider-scoped flag, native aliases, TLS fingerprint) the way
pollPVENode writes them.
Reviving those paths needed four bounds:
- The node memory carry uses the poller's own validated snapshot rather
than the read state, which holds the agent's reading on a node merged
with a host agent, and keeps used + cache + free equal to the total.
- A carried temperature keeps a lapsed host agent's lease. A reading
stamped no later than the agent's last report is not carried, so the
five-minute carry window cannot hold a silent agent's reading in
alerts and history past its lease, which #2582 guarantees.
- A carried reading keeps the node's CPU and GPU temperatures only. Its
SMART and NVMe rows also stamp the node's physical disks, so a disk
that went to standby would be stamped and recorded with its pre-sleep
value.
- Outage preservation and authoritative membership reconciliation keep a
node's temperature only under those rules and drop it when the node
goes offline.
TestPollPVEInstanceMarksStaleNodesOfflineWhenGetNodesReturnsEmpty relied
on the dead nodeLastOnline lookup and now ages that sighting too. The
silent-agent test from #2582 now rebuilds the store as production does
and feeds the polled node back, which is what exercises the carry.
Prevent optional Linux memory and metadata failures from starving supported filesystem usage. Carry shared admission deferral independently of a completed disk sample. Preserve backup locks, uncertain completion, no replay and original observation ages; bind both poll paths and truthful History to connected controls. This is source repair, not a native discussion #2538 cause or installed recovery.
Change-source: pulse-maintainer
Check every QGA read against fresh VM config locks, coordinate configured endpoint aliases and VM migration, discard in-flight backup payloads and cool down uncertain completions without retries. Preserve labelled cache evidence and keep carried samples out of History. Includes real-client monitoring, memory and counter provenance controls; native freeze/thaw acceptance remains separate. Refs #2439.
Change-source: pulse-maintainer
Extend the #1962 linked-agent memory fallback to Proxmox LXC containers (#2148). A correlated online Pulse agent running inside a container now supplies the container's memory metric instead of the cache-inclusive cluster/resources fallback, which can badly under-report shared-memory workloads. The agent sample is only trusted when its total matches the container's configured limit, so an agent without lxcfs cannot leak host memory into the guest row. Threads the previous-guest agent map through the efficient and node-by-node container builders and reads agent-owned memory from merged ContainerView resources.
Change-source: pulse-maintainer
Recorded PVE 8 and PVE 9 guest rrddata responses (fixtures under
pkg/proxmox/testdata/rrd/) prove guest RRD never carries the cache-aware
memused/memavailable columns — they exist only in node RRD — so every
consumer branch reading them was dead code that #1634's listing fallback
(7d7d2b6a3) had already routed around.
Drop the two fields from GuestRRDPoint (now time/maxmem only, matching
the recordings), delete the dead VM RRD memory fallback and its
getVMRRDMetrics/getVMRRDMemory helpers plus the vmRRDMemCache they fed,
remove the pointless per-poll guest RRD fetch from the LXC memory path,
and retire the guest RRD lookups from PVEClientInterface. VMMemoryRaw
loses its never-populated RRD diagnostic fields, and guest reliability
scoring no longer treats the node-only rrd-* sources as trusted guest
evidence. The knownDeadGuestRRDFields allowlist in the fixture
alignment test is gone; a new reflection guard in
code_standards_test.go keeps GuestRRDPoint pinned to recorded columns,
and cleanupRRDCache pruning of the guest-agent meminfo cache gains
direct coverage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Real PVE guest RRD responses carry only the cache-inclusive mem/maxmem
columns; the cache-aware memused/memavailable columns exist only in node
RRD, so the LXC RRD branches can never match a live response. Removing
the cluster-resources fallback in bf67ba920 therefore left every running
LXC reporting unavailable memory, rendered as 0% (#1634).
Running containers now fall back to the listing value under the
low-trust cluster-resources source when cache-aware RRD evidence is
absent; unavailable remains reserved for running containers with no
listing evidence at all.
Fixes#1634
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ac0fb263c changed preserved guests to carry their real runtime status
("running") instead of the stringified aggregate status ("online"). The
three guest metric recording loops gate on Status == "running", so guests
carried forward while their node sits in the grace period started passing
that gate. The carried-forward projection has no counters, so every cycle
wrote CPU, disk and network zeroes into the history and the persistent
store. Sparklines and stored metrics showed a collapse to zero for a guest
Pulse could not see, rather than a gap. Memory escaped only by accident,
because the projection leaves usage unknown and historyMemoryUsage returns
the -1 sentinel.
Freshly built guests are stamped with the cycle's sample time while
preserved guests keep the LastSeen of the cycle that observed them, so that
field already distinguishes the two. All three loops now gate on
guestObservedInCycle: the efficient path in recordGuestMetrics, and the
traditional VM and container paths, which had the same defect inline.
The guard fails open when LastSeen is absent. Dropping a real sample is the
worse error and the harder one to notice, so no evidence means record.
Regression coverage asserts both directions, and both were verified by
breaking them: removing the guard reports the fabricated zero sample, and
inverting it reports the observed guest losing its sample entirely.
golangci-lint run ./... failed on ~190 pre-existing errcheck violations and
5 unformatted files, burying any new regression in noise. Fix all of them:
- Test files that hand-rolled mock-mode set/restore (vmware, truenas, and
friends) now use the canonical setMockModeForTest/testutil.SetMockMode
helper instead of drift copies that ignored SetEnabled errors.
- internal/mock and internal/monitoring tests get package-local
mustSetEnabled/mustSetMockEnabled/mustSetMonitorMockMode helpers that
fail the test on toggle errors.
- pkg/auth/sqlite_manager.go, pkg/metrics/store.go, pkg/server/server.go:
rollbacks in defers use the explicit-discard idiom, migration renames and
rollup commits log failures, the hosted reaper goroutine logs an error
exit, shutdown mock-disable logs failures.
- Remaining test sites check errors with t.Fatalf/t.Errorf or explicitly
discard best-effort calls (restore-chmods, handler-closure unmarshals)
per existing repo style.
- gofmt: internal/api/maintenance_verification.go, internal/ai/demo.go and
three findings test files.
Only dupl findings remain (44 pre-existing production-code duplication
pairs) — those need real refactors, not mechanical fixes.
Full test suites pass for every touched package.
Prefer QEMU guest-agent MemAvailable when Proxmox reports saturated VM memory without guest free fields.
Add regression coverage for the issue #1319 Windows fsinfo volume payload so usable C/E/F volumes remain counted while System Reserved partitions are skipped.
Refs #1319
pulse-triage[bot]