mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-25 21:20:00 +00:00
Read agentless LXC listing memory as real pressure in Patrol (#3021)
Commita5baa7563bstopped Patrol reading cache-inclusive Proxmox memory as guest pressure (issue #2762, a QEMU VM). It also put `cluster-resources` in that cache-inclusive group for every guest type. An LXC without a linked Pulse agent only ever gets `cluster-resources`, so Patrol dropped memory pressure for all agentless containers: - a container at 97% of its limit got no triage flag, forecast or anomaly - the seed said "guest pressure unknown; may include reclaimable cache" - older memory alerts went to a billed model review on every run, and any resolve verdict was thrown away **That premise is wrong for LXC.** On both PVE 9 (`master`) and PVE 8 (`stable-bookworm` / `stable-8`), Proxmox subtracts page cache before reporting a container's memory: - `pve-common` `src/PVE/CGroup.pm` `get_memory_stat`: `$res->{mem} = $mem - $stat->{file};` on cgroup v2 (`memory.current`), and `$res->{mem} = $mem - $stat->{total_cache};` on v1 (`memory.usage_in_bytes`). - `pve-container` `src/PVE/LXC.pm` `vmstatus` stores that value with `$d->{mem} = int($mem->{mem});`. `PVE::LXC::CGroup` inherits `get_memory_stat` from `PVE::CGroup` without overriding it. - `pve-manager` `pvestatd` `update_lxc_status` writes `$d->{mem}` into RRD field 8. `/cluster/resources` (`PVE/API2/Cluster.pm` → `PVE::API2Tools::extract_vm_stats`) returns `$entry->{mem} = ($d->[8] || 0) + 0;`. A QEMU guest's listing `mem` is the host's view of guest RAM, which does include the guest's cache, so it stays pressure-unknown. **Change** - `unifiedresources.QualifyGuestMemoryForKind` now takes the guest kind: other platform, Proxmox VM or Proxmox container. - A current, dated `cluster-resources` reading counts as known pressure for a Proxmox container and stays cache-inclusive and unknown for QEMU. - `status-mem`, `status-freemem` and `derived-total-minus-used` are unchanged for every guest type. - `GuestMemoryEvidenceForResource` takes the kind from the resource type, so the Patrol inventory, the observer runtime and AI query projections agree. - `QualifyGuestMemory(bool)` keeps the conservative QEMU reading for callers that don't know the guest type. - The wrong comments in `calculateLXCMemory` and the LXC monitor tests are corrected. - Everything else ina5baa7563bstill applies: retained, undated, unknown and unavailable readings still cannot establish pressure or recovery. **Tests** - `TestPatrolAgentlessLXCListingMemoryIsPressure` (new) builds an LXC and a QEMU guest at 97%, both on `cluster-resources`, through legacy and canonical read state. The LXC gets a memory triage flag and alert review reads its value; the QEMU guest gets neither. - `TestGuestMemoryClusterResourcesIsTypeAware` (new) covers both guest kinds, retained readings, the other cache-inclusive sources and the resource-type mapping. - The `cluster-cache` fixture in `TestPatrolGuestMemoryEvidence` now expects pressure for the container variant. It previously encoded the wrong behaviour. - Against origin/main's production files (`go test -overlay`), the new and updated Patrol tests fail; with this change they pass. Refs #2762 Supersedes #3011, which failed the documentation recipe test that pinned the old `QualifyGuestMemory(` call in Patrol; this commit also updates that pin to `QualifyGuestMemoryForKind(`.
This commit is contained in:
@@ -178,6 +178,17 @@ parser. These controls do not establish issue #2762's installed input or cause,
|
||||
a cleared finding, or a containing release. Monitoring collection, guest-agent
|
||||
admission, configured alert thresholds, history storage and access are unchanged.
|
||||
|
||||
The `cluster-resources` source is read by guest type. For an LXC, Proxmox's
|
||||
listing `mem` is cgroup usage with page cache subtracted (`PVE::CGroup::get_memory_stat`:
|
||||
`memory.current` minus `memory.stat` `file` on cgroup v2, `usage_in_bytes` minus
|
||||
`total_cache` on v1), measured against the container's own limit, so a current
|
||||
dated reading on an agentless container keeps its pressure meaning for flags,
|
||||
forecasts, anomalies and alert review. A QEMU guest's listing value is the host's
|
||||
view of guest RAM and stays cache-inclusive and pressure-unknown, as do
|
||||
`status-mem`, `status-freemem` and `derived-total-minus-used` for every guest.
|
||||
`TestPatrolAgentlessLXCListingMemoryIsPressure` checks both guest types through
|
||||
legacy and canonical read state.
|
||||
|
||||
Legacy Assistant mode echo and stored-preference compatibility follow the
|
||||
[API settings contract](api-contracts.md#assistant-control_level-is-an-interactive-projection).
|
||||
|
||||
|
||||
@@ -58,6 +58,14 @@ absence and refreshed rows in both views. This shared interpretation changes
|
||||
no registry wire format, metric selection, guest collection, History, correlation,
|
||||
identity or permission. AI query projections own their additional evidence fields.
|
||||
|
||||
`QualifyGuestMemoryForKind` takes the Proxmox guest kind because one source name
|
||||
covers two accountings: an LXC's `cluster-resources` reading already excludes page
|
||||
cache and establishes pressure when current, while a QEMU guest's does not.
|
||||
`GuestMemoryEvidenceForResource` derives the kind from the resource type
|
||||
(system-container versus VM). `QualifyGuestMemory` keeps the conservative QEMU
|
||||
reading for callers that do not know the type. `TestGuestMemoryClusterResourcesIsTypeAware`
|
||||
covers both kinds, retained readings and the other cache-inclusive sources.
|
||||
|
||||
### Selected guest memory observation in typed read state
|
||||
|
||||
VM and system-container views expose a value-copy of the selected memory
|
||||
|
||||
@@ -1980,7 +1980,7 @@ func patrolGuestInventoryRows(snap patrolRuntimeState, scopedSet map[string]bool
|
||||
status: patrolGuestRuntimeStatus(vmv.RuntimeStatus(), vmv.Status()),
|
||||
cpu: vmv.CPUPercent(),
|
||||
mem: vmv.MemoryPercent(),
|
||||
memory: readPatrolGuestMemory(vmv.MemoryPercent(), observation, hasMemory, hasProxmox || vmv.VMID() > 0),
|
||||
memory: readPatrolGuestMemory(vmv.MemoryPercent(), observation, hasMemory, unifiedresources.GuestMemoryKindFor(hasProxmox || vmv.VMID() > 0, false)),
|
||||
disk: vmv.DiskPercent(),
|
||||
vmid: vmv.VMID(),
|
||||
ip: patrolFirstIP(vmv.IPAddresses()),
|
||||
@@ -2003,7 +2003,7 @@ func patrolGuestInventoryRows(snap patrolRuntimeState, scopedSet map[string]bool
|
||||
status: patrolGuestRuntimeStatus(ctv.RuntimeStatus(), ctv.Status()),
|
||||
cpu: ctv.CPUPercent(),
|
||||
mem: ctv.MemoryPercent(),
|
||||
memory: readPatrolGuestMemory(ctv.MemoryPercent(), observation, hasMemory, hasProxmox || ctv.VMID() > 0),
|
||||
memory: readPatrolGuestMemory(ctv.MemoryPercent(), observation, hasMemory, unifiedresources.GuestMemoryKindFor(hasProxmox || ctv.VMID() > 0, true)),
|
||||
disk: ctv.DiskPercent(),
|
||||
vmid: ctv.VMID(),
|
||||
ip: patrolFirstIP(ctv.IPAddresses()),
|
||||
@@ -2028,7 +2028,7 @@ func patrolGuestInventoryRows(snap patrolRuntimeState, scopedSet map[string]bool
|
||||
status: vm.Status,
|
||||
cpu: unifiedresources.ProxmoxGuestCPUPercent(vm.CPU),
|
||||
mem: vm.Memory.Usage,
|
||||
memory: readPatrolGuestMemory(vm.Memory.Usage, vm.Memory.Observation, vm.Memory.HasKnownUsage(), vm.VMID > 0 || vm.Type == "qemu"),
|
||||
memory: readPatrolGuestMemory(vm.Memory.Usage, vm.Memory.Observation, vm.Memory.HasKnownUsage(), unifiedresources.GuestMemoryKindFor(vm.VMID > 0 || vm.Type == "qemu", false)),
|
||||
disk: vm.Disk.Usage,
|
||||
vmid: vm.VMID,
|
||||
ip: patrolFirstIP(vm.IPAddresses),
|
||||
@@ -2049,7 +2049,7 @@ func patrolGuestInventoryRows(snap patrolRuntimeState, scopedSet map[string]bool
|
||||
status: ct.Status,
|
||||
cpu: unifiedresources.ProxmoxGuestCPUPercent(ct.CPU),
|
||||
mem: ct.Memory.Usage,
|
||||
memory: readPatrolGuestMemory(ct.Memory.Usage, ct.Memory.Observation, ct.Memory.HasKnownUsage(), ct.VMID > 0 || ct.Type == "lxc"),
|
||||
memory: readPatrolGuestMemory(ct.Memory.Usage, ct.Memory.Observation, ct.Memory.HasKnownUsage(), unifiedresources.GuestMemoryKindFor(ct.VMID > 0 || ct.Type == "lxc", true)),
|
||||
disk: ct.Disk.Usage,
|
||||
vmid: ct.VMID,
|
||||
ip: patrolFirstIP(ct.IPAddresses),
|
||||
|
||||
@@ -44,8 +44,8 @@ type patrolGuestMemoryReading struct {
|
||||
mayIncludeCache bool
|
||||
}
|
||||
|
||||
func readPatrolGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool) patrolGuestMemoryReading {
|
||||
e := unifiedresources.QualifyGuestMemory(percent, observation, available, proxmoxGuest, time.Now())
|
||||
func readPatrolGuestMemory(percent float64, observation models.MemoryObservation, available bool, kind unifiedresources.GuestMemoryKind) patrolGuestMemoryReading {
|
||||
e := unifiedresources.QualifyGuestMemoryForKind(percent, observation, available, kind, time.Now())
|
||||
return patrolGuestMemoryReading{percent: percent, available: e.Available, pressureKnown: e.PressureKnown,
|
||||
state: e.State, source: e.Source, observedAt: e.ObservedAt, mayIncludeCache: e.MayIncludeCache}
|
||||
}
|
||||
|
||||
@@ -14,26 +14,46 @@ func TestPatrolGuestMemoryReadingBoundaries(t *testing.T) {
|
||||
at := time.Now().UTC().Add(-time.Minute)
|
||||
obs := models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: at}
|
||||
for _, percent := range []float64{math.NaN(), math.Inf(1), math.Inf(-1), -1, 101} {
|
||||
reading := readPatrolGuestMemory(percent, obs, true, true)
|
||||
reading := readPatrolGuestMemory(percent, obs, true, unifiedresources.GuestMemoryProxmoxVM)
|
||||
if reading.pressureKnown || reading.available || !strings.HasPrefix(reading.display(), "N/A") {
|
||||
t.Errorf("invalid percent became live pressure: %+v", reading)
|
||||
}
|
||||
}
|
||||
for _, source := range []string{"available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent"} {
|
||||
obs.Source = source
|
||||
if r := readPatrolGuestMemory(96, obs, true, true); !r.pressureKnown {
|
||||
if r := readPatrolGuestMemory(96, obs, true, unifiedresources.GuestMemoryProxmoxVM); !r.pressureKnown {
|
||||
t.Errorf("current cache-aware positive rejected: %s", source)
|
||||
}
|
||||
}
|
||||
for _, state := range []string{"", "future", "last-known", "unavailable"} {
|
||||
obs.State = state
|
||||
if r := readPatrolGuestMemory(96, obs, true, true); r.pressureKnown {
|
||||
if r := readPatrolGuestMemory(96, obs, true, unifiedresources.GuestMemoryProxmoxVM); r.pressureKnown {
|
||||
t.Errorf("non-current state became current: %s", state)
|
||||
}
|
||||
}
|
||||
obs.State = "current"
|
||||
obs.ObservedAt = time.Unix(0, 0)
|
||||
if r := readPatrolGuestMemory(96, obs, true, true); r.pressureKnown || !r.observedAt.IsZero() {
|
||||
// Proxmox's listing mem means different things per guest type: a QEMU
|
||||
// guest's counts its own page cache, an LXC's has page cache subtracted.
|
||||
obs = models.MemoryObservation{State: "current", Source: "cluster-resources", ObservedAt: at}
|
||||
if r := readPatrolGuestMemory(97, obs, true, unifiedresources.GuestMemoryProxmoxVM); r.pressureKnown || !r.mayIncludeCache {
|
||||
t.Errorf("QEMU cluster-resources memory became pressure: %+v", r)
|
||||
}
|
||||
ct := readPatrolGuestMemory(97, obs, true, unifiedresources.GuestMemoryProxmoxContainer)
|
||||
if !ct.pressureKnown || ct.mayIncludeCache || !strings.Contains(ct.display(), "cache-aware guest usage") {
|
||||
t.Errorf("LXC cluster-resources memory lost its pressure meaning: %+v %s", ct, ct.display())
|
||||
}
|
||||
for _, source := range []string{"status-mem", "status-freemem", "derived-total-minus-used"} {
|
||||
obs.Source = source
|
||||
if r := readPatrolGuestMemory(97, obs, true, unifiedresources.GuestMemoryProxmoxContainer); r.pressureKnown {
|
||||
t.Errorf("container kind widened another cache-inclusive source: %s", source)
|
||||
}
|
||||
}
|
||||
obs.Source = "cluster-resources"
|
||||
obs.State = "last-known"
|
||||
if r := readPatrolGuestMemory(97, obs, true, unifiedresources.GuestMemoryProxmoxContainer); r.pressureKnown {
|
||||
t.Error("retained LXC listing reading became current pressure")
|
||||
}
|
||||
obs = models.MemoryObservation{State: "current", Source: "available-field", ObservedAt: time.Unix(0, 0)}
|
||||
if r := readPatrolGuestMemory(96, obs, true, unifiedresources.GuestMemoryProxmoxVM); r.pressureKnown || !r.observedAt.IsZero() {
|
||||
t.Fatal("invalid observation origin became current")
|
||||
}
|
||||
if status := patrolGuestRuntimeStatus("paused", unifiedresources.StatusOffline); status != "paused" {
|
||||
|
||||
@@ -54,6 +54,13 @@ func TestPatrolGuestMemoryEvidence(t *testing.T) {
|
||||
for _, kind := range []string{"vm", "system-container"} {
|
||||
for _, path := range []string{"legacy", "canonical"} {
|
||||
t.Run(tc.name+"/"+kind+"/"+path, func(t *testing.T) {
|
||||
tc := tc
|
||||
if tc.source == "cluster-resources" && kind == "system-container" {
|
||||
// An agentless LXC's listing mem already excludes page
|
||||
// cache (PVE::CGroup::get_memory_stat), so a current
|
||||
// reading at 96% of its limit is real pressure.
|
||||
tc.label, tc.pressure = "cache-aware guest usage", true
|
||||
}
|
||||
mem := patrolMemoryFixture(tc.percent, tc.state, tc.source, tc.at)
|
||||
mem.UsageUnavailable = tc.unavailable
|
||||
snapshot := models.StateSnapshot{}
|
||||
@@ -230,3 +237,43 @@ func TestIssue2762NamedMemoryFields(t *testing.T) {
|
||||
t.Error("partial free-only payload manufactured available memory")
|
||||
}
|
||||
}
|
||||
|
||||
// An LXC without a linked Pulse agent only has Proxmox's listing reading. That
|
||||
// value is cgroup usage with page cache subtracted, so it keeps its pressure
|
||||
// meaning; a QEMU guest's listing reading counts guest cache and stays unknown.
|
||||
func TestPatrolAgentlessLXCListingMemoryIsPressure(t *testing.T) {
|
||||
now := time.Now().UTC()
|
||||
mem := patrolMemoryFixture(97, "current", "cluster-resources", now.Add(-30*time.Second))
|
||||
snapshot := models.StateSnapshot{
|
||||
VMs: []models.VM{{ID: "vm", Name: "vm", VMID: 100, Type: "qemu", Instance: "pve", Node: "node", Status: "running", Memory: mem}},
|
||||
Containers: []models.Container{{ID: "ct", Name: "ct", VMID: 101, Type: "lxc", Instance: "pve", Node: "node", Status: "running", Memory: mem}},
|
||||
}
|
||||
for _, path := range []string{"legacy", "canonical"} {
|
||||
t.Run(path, func(t *testing.T) {
|
||||
snap := newPatrolRuntimeState(snapshot)
|
||||
if path == "canonical" {
|
||||
registry := unifiedresources.NewRegistry(nil)
|
||||
registry.IngestSnapshot(snapshot)
|
||||
snap = patrolRuntimeState{readState: registry}
|
||||
}
|
||||
flags := triageThresholdChecksState(snap, nil, DefaultPatrolThresholds())
|
||||
if triageFindFlag(flags, func(f TriageFlag) bool { return f.Metric == "memory" && f.ResourceName == "ct" }) == nil {
|
||||
t.Errorf("agentless LXC at 97%% of its limit raised no memory flag: %+v", flags)
|
||||
}
|
||||
if flag := triageFindFlag(flags, func(f TriageFlag) bool { return f.Metric == "memory" && f.ResourceName == "vm" }); flag != nil {
|
||||
t.Errorf("QEMU listing memory became pressure: %+v", flag)
|
||||
}
|
||||
ps := NewPatrolService(nil, nil)
|
||||
if got := ps.getCurrentMetricValueState(AlertInfo{Type: "memory", ResourceID: "ct", ResourceName: "ct", ResourceType: "system-container"}, snap); got != 97 {
|
||||
t.Errorf("LXC memory alert review lost the current reading: %v", got)
|
||||
}
|
||||
if got := ps.getCurrentMetricValueState(AlertInfo{Type: "memory", ResourceID: "vm", ResourceName: "vm", ResourceType: "vm"}, snap); got != -1 {
|
||||
t.Errorf("QEMU memory alert review read listing memory as pressure: %v", got)
|
||||
}
|
||||
gaps := patrolGuestMemoryGaps(patrolGuestInventoryRows(snap, nil, nil))
|
||||
if !strings.Contains(gaps, "vm:") || strings.Contains(gaps, "ct:") {
|
||||
t.Errorf("pressure-unknown rollup misclassified guests: %s", gaps)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,12 +13,13 @@ import (
|
||||
func (m *Monitor) calculateLXCMemory(
|
||||
res proxmox.ClusterResource,
|
||||
) (uint64, uint64, string, VMMemoryRaw) {
|
||||
// The cluster resources API returns mem from cgroup which includes
|
||||
// cache/buffers (inflated). PVE guest RRD offers nothing better: it
|
||||
// carries only the same cache-inclusive mem/maxmem columns — the
|
||||
// cache-aware memused/memavailable columns exist only in node RRD — so
|
||||
// the listing value is the best evidence available for running
|
||||
// containers (issue #1634).
|
||||
// The cluster resources mem for a container is its cgroup usage with page
|
||||
// cache already subtracted: PVE::CGroup::get_memory_stat reports
|
||||
// memory.current - memory.stat file (cgroup v2) or usage_in_bytes -
|
||||
// total_cache (v1), and pvestatd feeds that vmstatus value to the listing.
|
||||
// It is a cache-excluded reading against the container's own limit, unlike
|
||||
// a QEMU guest's listing mem. Guest RRD carries the same mem/maxmem pair,
|
||||
// so the listing value is the evidence for running containers (issue #1634).
|
||||
memTotal := res.MaxMem
|
||||
memUsed := uint64(0)
|
||||
memorySource := "powered-off"
|
||||
|
||||
@@ -549,8 +549,8 @@ type stubPVEClientLXCRRD struct {
|
||||
}
|
||||
|
||||
// GetLXCRRDData tracks lookups of the guest RRD endpoint. It is intentionally
|
||||
// not part of PVEClientInterface anymore: guest rrddata carries only the
|
||||
// cache-inclusive mem/maxmem columns (#1634), so the LXC memory path must not
|
||||
// not part of PVEClientInterface anymore: guest rrddata carries only the same
|
||||
// mem/maxmem columns as the listing (#1634), so the LXC memory path must not
|
||||
// consult it.
|
||||
func (s *stubPVEClientLXCRRD) GetLXCRRDData(ctx context.Context, node string, vmid int, timeframe, cf string, ds []string) ([]proxmox.GuestRRDPoint, error) {
|
||||
s.lxcRRDCalls++
|
||||
@@ -767,9 +767,9 @@ func TestBuildContainerFromClusterResource_AppliesAgentLXCFilesystems(t *testing
|
||||
}
|
||||
|
||||
// Issue #2148: a Proxmox LXC with a correlated, online Pulse agent must report
|
||||
// the agent's own memory sample instead of the cache-inclusive
|
||||
// cluster/resources fallback, which can badly under-report shared-memory
|
||||
// workloads (mirrors the #1962 VM behaviour).
|
||||
// the agent's own memory sample instead of the cluster/resources fallback,
|
||||
// whose page-cache subtraction also removes shared memory and tmpfs and so can
|
||||
// badly under-report shared-memory workloads (mirrors the #1962 VM behaviour).
|
||||
func TestIssue2148LXCPrefersLinkedAgentMemoryOverClusterResources(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -20,10 +20,34 @@ type GuestMemoryEvidence struct {
|
||||
MayIncludeCache bool `json:"may_include_reclaimable_cache,omitempty"`
|
||||
}
|
||||
|
||||
// QualifyGuestMemory shares the existing Patrol interpretation with its query
|
||||
// and observer inputs. State is authored by collection; consumers do not renew
|
||||
// the original observation or invent a separate collection lease.
|
||||
// GuestMemoryKind names the guest whose accounting a Proxmox memory source
|
||||
// describes. The same source name means different things for QEMU and LXC.
|
||||
type GuestMemoryKind uint8
|
||||
|
||||
const (
|
||||
// GuestMemoryOtherPlatform keeps a non-Proxmox platform's selected reading.
|
||||
GuestMemoryOtherPlatform GuestMemoryKind = iota
|
||||
// GuestMemoryProxmoxVM is a QEMU guest, or a Proxmox guest of unknown type.
|
||||
GuestMemoryProxmoxVM
|
||||
// GuestMemoryProxmoxContainer is an LXC (including OCI) container.
|
||||
GuestMemoryProxmoxContainer
|
||||
)
|
||||
|
||||
// QualifyGuestMemory qualifies a reading without knowing the Proxmox guest
|
||||
// type, so it uses the conservative QEMU interpretation. Callers that know the
|
||||
// guest is a container use QualifyGuestMemoryForKind.
|
||||
func QualifyGuestMemory(percent float64, observation models.MemoryObservation, available, proxmoxGuest bool, now time.Time) GuestMemoryEvidence {
|
||||
kind := GuestMemoryOtherPlatform
|
||||
if proxmoxGuest {
|
||||
kind = GuestMemoryProxmoxVM
|
||||
}
|
||||
return QualifyGuestMemoryForKind(percent, observation, available, kind, now)
|
||||
}
|
||||
|
||||
// QualifyGuestMemoryForKind shares the existing Patrol interpretation with its
|
||||
// query and observer inputs. State is authored by collection; consumers do not
|
||||
// renew the original observation or invent a separate collection lease.
|
||||
func QualifyGuestMemoryForKind(percent float64, observation models.MemoryObservation, available bool, kind GuestMemoryKind, now time.Time) GuestMemoryEvidence {
|
||||
e := GuestMemoryEvidence{Available: available && !math.IsNaN(percent) && !math.IsInf(percent, 0) && percent >= 0 && percent <= 100}
|
||||
if !e.Available || observation.State == "unavailable" {
|
||||
e.Available = false
|
||||
@@ -31,7 +55,7 @@ func QualifyGuestMemory(percent float64, observation models.MemoryObservation, a
|
||||
return e
|
||||
}
|
||||
// Other platforms need not implement Proxmox's memory observation contract.
|
||||
if !proxmoxGuest && observation == (models.MemoryObservation{}) {
|
||||
if kind == GuestMemoryOtherPlatform && observation == (models.MemoryObservation{}) {
|
||||
e.PressureKnown = true
|
||||
return e
|
||||
}
|
||||
@@ -49,7 +73,19 @@ func QualifyGuestMemory(percent float64, observation models.MemoryObservation, a
|
||||
case "available-field", "derived-free-buffers-cached", "guest-agent-meminfo", "guest-agent-meminfo-derived", "agent":
|
||||
e.Source = observation.Source
|
||||
e.PressureKnown = e.State == "current"
|
||||
case "status-mem", "status-freemem", "cluster-resources", "derived-total-minus-used":
|
||||
case "cluster-resources":
|
||||
e.Source = observation.Source
|
||||
// An LXC's listing mem is its cgroup usage minus page cache: Proxmox
|
||||
// computes memory.current - memory.stat file (cgroup v2) or
|
||||
// usage_in_bytes - total_cache (v1) in PVE::CGroup::get_memory_stat,
|
||||
// and /cluster/resources reports that vmstatus value. A QEMU listing is
|
||||
// the host's view of guest RAM, which counts the guest's own cache.
|
||||
if kind == GuestMemoryProxmoxContainer {
|
||||
e.PressureKnown = e.State == "current"
|
||||
} else {
|
||||
e.MayIncludeCache = true
|
||||
}
|
||||
case "status-mem", "status-freemem", "derived-total-minus-used":
|
||||
e.Source = observation.Source
|
||||
e.MayIncludeCache = true
|
||||
case "previous-snapshot":
|
||||
@@ -88,9 +124,23 @@ func (e GuestMemoryEvidence) Format(percent float64) string {
|
||||
// raw platform facet or resource LastSeen must never lend it provenance.
|
||||
func GuestMemoryEvidenceForResource(r *Resource, now time.Time) GuestMemoryEvidence {
|
||||
if r == nil || r.Metrics == nil || r.Metrics.Memory == nil {
|
||||
return QualifyGuestMemory(0, models.MemoryObservation{}, false, true, now)
|
||||
return QualifyGuestMemoryForKind(0, models.MemoryObservation{}, false, GuestMemoryProxmoxVM, now)
|
||||
}
|
||||
_, hasProxmox := r.SourceStatus[SourceProxmox]
|
||||
m := r.Metrics.Memory
|
||||
return QualifyGuestMemory(m.Percent, m.Observation, true, r.Proxmox != nil || hasProxmox || m.Source == SourceProxmox, now)
|
||||
kind := GuestMemoryKindFor(r.Proxmox != nil || hasProxmox || m.Source == SourceProxmox, r.Type == ResourceTypeSystemContainer)
|
||||
return QualifyGuestMemoryForKind(m.Percent, m.Observation, true, kind, now)
|
||||
}
|
||||
|
||||
// GuestMemoryKindFor keeps non-Proxmox guests on their own contract and tells
|
||||
// Proxmox containers apart from QEMU guests.
|
||||
func GuestMemoryKindFor(proxmoxGuest, container bool) GuestMemoryKind {
|
||||
switch {
|
||||
case !proxmoxGuest:
|
||||
return GuestMemoryOtherPlatform
|
||||
case container:
|
||||
return GuestMemoryProxmoxContainer
|
||||
default:
|
||||
return GuestMemoryProxmoxVM
|
||||
}
|
||||
}
|
||||
|
||||
@@ -620,3 +620,46 @@ func TestGuestMemoryEvidenceQualification(t *testing.T) {
|
||||
t.Fatal("other platform contract changed")
|
||||
}
|
||||
}
|
||||
|
||||
// Proxmox's listing mem is page-cache-excluded cgroup usage for an LXC
|
||||
// (PVE::CGroup::get_memory_stat) but the host's view of guest RAM for QEMU.
|
||||
func TestGuestMemoryClusterResourcesIsTypeAware(t *testing.T) {
|
||||
now := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
|
||||
at := now.Add(-time.Minute)
|
||||
current := models.MemoryObservation{State: "current", Source: "cluster-resources", ObservedAt: at}
|
||||
ct := QualifyGuestMemoryForKind(97, current, true, GuestMemoryProxmoxContainer, now)
|
||||
if !ct.PressureKnown || ct.MayIncludeCache || ct.Source != "cluster-resources" || !strings.Contains(ct.Format(97), "cache-aware guest usage") {
|
||||
t.Errorf("LXC listing memory lost its pressure meaning: %+v", ct)
|
||||
}
|
||||
if e := QualifyGuestMemoryForKind(97, current, true, GuestMemoryProxmoxVM, now); e.PressureKnown || !e.MayIncludeCache {
|
||||
t.Errorf("QEMU listing memory became pressure: %+v", e)
|
||||
}
|
||||
if e := QualifyGuestMemory(97, current, true, true, now); e.PressureKnown || !e.MayIncludeCache {
|
||||
t.Errorf("type-unknown Proxmox guest dropped the conservative reading: %+v", e)
|
||||
}
|
||||
for _, state := range []string{"last-known", ""} {
|
||||
obs := current
|
||||
obs.State = state
|
||||
if e := QualifyGuestMemoryForKind(97, obs, true, GuestMemoryProxmoxContainer, now); e.PressureKnown {
|
||||
t.Errorf("retained LXC listing reading became current: %s: %+v", state, e)
|
||||
}
|
||||
}
|
||||
for _, source := range []string{"status-mem", "status-freemem", "derived-total-minus-used"} {
|
||||
obs := current
|
||||
obs.Source = source
|
||||
if e := QualifyGuestMemoryForKind(97, obs, true, GuestMemoryProxmoxContainer, now); e.PressureKnown || !e.MayIncludeCache {
|
||||
t.Errorf("container kind widened %s: %+v", source, e)
|
||||
}
|
||||
}
|
||||
// Typed read state picks the kind from the resource type.
|
||||
for _, tc := range []struct {
|
||||
typ ResourceType
|
||||
want bool
|
||||
}{{ResourceTypeSystemContainer, true}, {ResourceTypeVM, false}} {
|
||||
r := &Resource{Type: tc.typ, Proxmox: &ProxmoxData{VMID: 101},
|
||||
Metrics: &ResourceMetrics{Memory: &MetricValue{Percent: 97, Source: SourceProxmox, Observation: current}}}
|
||||
if e := GuestMemoryEvidenceForResource(r, now); e.PressureKnown != tc.want {
|
||||
t.Errorf("%s listing memory pressure=%t, want %t", tc.typ, e.PressureKnown, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -399,7 +399,7 @@ class GuestMemoryInterpretationDocsTest(unittest.TestCase):
|
||||
# Patrol delegates interpretation and display to the shared selected-
|
||||
# sample formatter. Keep the guide bound to that implementation, not
|
||||
# the former location of its warning strings.
|
||||
self.assertIn("unifiedresources.QualifyGuestMemory(", patrol)
|
||||
self.assertIn("unifiedresources.QualifyGuestMemoryForKind(", patrol)
|
||||
self.assertIn("unifiedresources.GuestMemoryEvidence{", patrol)
|
||||
self.assertIn("}).Format(r.percent)", patrol)
|
||||
self.assertIn('"guest pressure unknown"', evidence)
|
||||
|
||||
Reference in New Issue
Block a user