From 2ca102cc210f4d05d1e14f32e40e2ce7cbe005b8 Mon Sep 17 00:00:00 2001 From: "pulse-triage[bot]" <249995291+pulse-triage[bot]@users.noreply.github.com> Date: Sun, 11 Oct 2026 10:56:42 +0100 Subject: [PATCH] Distinguish memory baseline regressions from growth in support guidance The latest memory report is about a higher steady baseline, not a leak. Keep steady regressions actionable, separate unlike process ages and CPU savings from RSS recovery, and preserve existing evidence without another restart, downgrade or profile. Keep the shipped guides, optional intake and regression controls aligned. Contract-Neutral: Clarifies interpretation of existing memory reports and documentation only; no runtime, API, route, collection recipe, label synchronizer or subsystem contract changes. Change-source: pulse-maintainer --- .github/ISSUE_TEMPLATE/bug_report.yml | 2 ++ .github/ISSUE_TEMPLATE/v6_rc_feedback.yml | 2 ++ .github/scripts/issue-version-triage.test.cjs | 11 ++++++++++ docs/FAQ.md | 5 +++++ docs/ISSUE_TRIAGE.md | 11 ++++++++++ docs/TROUBLESHOOTING.md | 19 +++++++++++++++- frontend-modern/public/docs/FAQ.md | 5 +++++ frontend-modern/public/docs/ISSUE_TRIAGE.md | 11 ++++++++++ .../public/docs/TROUBLESHOOTING.md | 19 +++++++++++++++- .../tests/test_memory_troubleshooting_docs.py | 22 +++++++++++++++++++ 10 files changed, 105 insertions(+), 2 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 6bf10731f5..4f7ea3eb3e 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -159,6 +159,8 @@ body: For CPU, memory or disk-write reports about Pulse itself, use existing readings or safe, passive observations. Say whether the reading covers the Pulse process, its container or the whole host, and give its units, measurement window and uptime. Include allocated CPUs or memory limit, fleet size, polling interval and open dashboards where relevant and known. A process-start CPU average is not a recent sampling window; database size is not a write rate. Existing screenshots are useful; write "unavailable" when safe collection is not possible. Do not restart, create load or change polling or retention just to measure. Do not attach raw profiles, heap dumps, databases or full process command lines; summarise only relevant counters after local review. + For Pulse memory reports, distinguish a higher steady baseline from growth during one uninterrupted run. A higher baseline is worth reporting even if usage stays flat. Keep already available versions, uptime, workload and enabled collectors/features with each reading; a lower reading shortly after an upgrade is not proof of recovery compared with a days-old run. Unknown context is valid; do not restart, downgrade or change features to create a comparison. + For VM or LXC memory disagreements, say where each existing reading appears (overview, History, alert, Patrol or another tool) and whether the comparison was measured inside the affected guest or on its hypervisor node. Give original times, units and memory source or sample age only if already known or displayed; use consistent private aliases and write "unknown" for missing details. Available memory, buff/cache and a process's RSS measure different things: RSS alone is not total guest usage, and a cache-inclusive hypervisor footprint does not establish guest pressure. A Proxmox VM API connection may still obtain QEMU guest-agent (QGA) readings without a Pulse agent. Use existing observations only. Do not run commands, Diagnostics or guest-agent probes, install an agent, change memory settings or workloads, or repeat a backup just to answer. diff --git a/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml b/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml index 4a020f03a5..78686eea52 100644 --- a/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml +++ b/.github/ISSUE_TEMPLATE/v6_rc_feedback.yml @@ -201,6 +201,8 @@ body: For CPU, memory or disk-write reports about Pulse itself, use existing readings or safe, passive observations. Say whether the reading covers the Pulse process, its container or the whole host, and give its units, measurement window and uptime. Include allocated CPUs or memory limit, fleet size, polling interval and open dashboards where relevant and known. A process-start CPU average is not a recent sampling window; database size is not a write rate. Existing screenshots are useful; write "unavailable" when safe collection is not possible. Do not restart, create load or change polling or retention just to measure. Do not attach raw profiles, heap dumps, databases or full process command lines; summarise only relevant counters after local review. + For Pulse memory reports, distinguish a higher steady baseline from growth during one uninterrupted run. A higher baseline is worth reporting even if usage stays flat. Keep already available versions, uptime, workload and enabled collectors/features with each reading; a lower reading shortly after an upgrade is not proof of recovery compared with a days-old run. Unknown context is valid; do not restart, downgrade or change features to create a comparison. + For VM or LXC memory disagreements, say where each existing reading appears (overview, History, alert, Patrol or another tool) and whether the comparison was measured inside the affected guest or on its hypervisor node. Give original times, units and memory source or sample age only if already known or displayed; use consistent private aliases and write "unknown" for missing details. Available memory, buff/cache and a process's RSS measure different things: RSS alone is not total guest usage, and a cache-inclusive hypervisor footprint does not establish guest pressure. A Proxmox VM API connection may still obtain QEMU guest-agent (QGA) readings without a Pulse agent. Use existing observations only. Do not run commands, Diagnostics or guest-agent probes, install an agent, change memory settings or workloads, or repeat a backup just to answer. diff --git a/.github/scripts/issue-version-triage.test.cjs b/.github/scripts/issue-version-triage.test.cjs index d0e4a8cab8..bef5eda062 100644 --- a/.github/scripts/issue-version-triage.test.cjs +++ b/.github/scripts/issue-version-triage.test.cjs @@ -627,6 +627,17 @@ for (const [name, field] of [ assert.match(attachment, /summarise only relevant counters after local review/); }); + test(`baseline memory regressions need no continuing growth or restart: ${name}`, () => { + const attachment = evidenceField(); + assert.match(attachment, /higher steady baseline from growth during one uninterrupted run/); + assert.match(attachment, /worth reporting even if usage stays flat/); + assert.match(attachment, /already available versions, uptime, workload and enabled collectors\/features/); + assert.match(attachment, /not proof of recovery compared with a days-old run/); + assert.match(attachment, /Unknown context is valid/); + assert.match(attachment, /do not restart, downgrade or change features to create a comparison/); + assert.doesNotMatch(attachment, /required: true|render:/); + }); + test(`guest memory comparisons retain optional target and sample context: ${name}`, () => { const attachment = evidenceField(); assert.match(attachment, /For CPU, memory or disk-write reports about Pulse itself/); diff --git a/docs/FAQ.md b/docs/FAQ.md index ffb0026b7c..237352d175 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -261,6 +261,11 @@ LXC or Docker chart alone does not establish a leak. Use the [read-only memory checks](TROUBLESHOOTING.md#memory-use-keeps-growing) for your deployment while it is responsive. +A higher steady baseline is worth reporting even without continuing growth. +A lower reading ten minutes into a new process is not a like-for-like comparison +with a week-old run, or proof of recovery. Keep already available readings, +versions, uptime and workload context; do not restart to create a comparison. + Do not shorten retention, slow polling, restart Pulse or drop caches just to lower the reading. Shorter retention removes history, and slower polling can delay monitoring; neither establishes the cause. If the container is near its diff --git a/docs/ISSUE_TRIAGE.md b/docs/ISSUE_TRIAGE.md index 7330851173..7d283f87fe 100644 --- a/docs/ISSUE_TRIAGE.md +++ b/docs/ISSUE_TRIAGE.md @@ -132,6 +132,17 @@ profiles, heap dumps, databases or full process command lines in a public thread; use locally reviewed counter summaries, without restarting, creating load or changing polling or retention just to measure. +For Pulse memory reports, a higher steady baseline and continuing growth are +different problems. A plateau does not dismiss an increased memory requirement; +do not require a leak or a rising series before retaining a baseline regression. +Compare already supplied process scope, units, versions, uptime, workload and +enabled collectors/features. A week-old run and a ten-minute new process are +not comparable steady-state evidence, and a startup CPU saving does not prove +lower RSS. Preserve those limits instead of claiming recovery or asking for +another restart, downgrade, profile or unchanged sample. If the reporter +clarifies that the problem is the baseline, carry that goal to the engineering +owner; growth-rate evidence alone does not answer it. Unknown context is valid. + For VM or LXC memory disagreements, retain where each existing reading appears (overview, History, alert, Patrol or another tool), whether the comparison was measured inside the affected guest or on its hypervisor node, and the original diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 731908474a..1ab6df91f3 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -667,6 +667,22 @@ own cache accounting. Neither is interchangeable with Pulse's resident memory (RSS), a Go heap size, or virtual address space (`VSZ` / `VmSize`). A large container reading alone does not establish a Pulse memory leak. +**A higher steady baseline is also worth reporting; continuing growth is not +required.** First distinguish what your existing readings show: + +| Observation | What it establishes | +| --- | --- | +| Comparable Pulse process RSS settles at a higher level than before with an unchanged workload | A possible baseline regression, even if usage then stays flat. Keep the old and new versions, uptime and measurement scope; a plateau does not dismiss the increased requirement. | +| Comparable samples rise during one uninterrupted process run | Growth during that interval, not by itself a leak or its cause. Keep the same PID/start time, units, workload and collection settings. | +| A new process uses less memory shortly after an upgrade than an older process did after days | Different runs at different ages, not verified recovery or growth within one run. Startup and long-lived usage can differ. | + +For example, **510 MiB after a week** and **420 MiB ten minutes into a new +process** cannot establish that steady-state memory improved. Use already +available readings at comparable uptimes and workloads if you have them; +otherwise keep the mismatch explicit. Do not restart, downgrade or change +retention, polling or enabled features to manufacture a comparison. A startup +CPU improvement is not evidence that the resident-memory baseline fell. + For a responsive **Linux systemd / Proxmox LXC** install, take this small, read-only sample at a normally occurring high point. Run it **inside the Pulse container** for LXC, not on the Proxmox host. Substitute the actual service name @@ -738,7 +754,8 @@ values. If it fails or Pulse stops, retain that fact; do not loosen access controls or keep retrying against an unresponsive installation. Keep the sample time, PID/start time, running version, container memory limit, -fleet size, polling interval and whether dashboards were open. If safe, compare +fleet size, polling interval, enabled collectors/features and whether dashboards +were open. If safe, compare another naturally occurring point with the **same PID and process start time**; record any restart or upgrade as a different run, not evidence that the cause was fixed. For Docker, the [bounded container statistics below](#excessive-cpu-writes-or-database-growth) diff --git a/frontend-modern/public/docs/FAQ.md b/frontend-modern/public/docs/FAQ.md index ffb0026b7c..237352d175 100644 --- a/frontend-modern/public/docs/FAQ.md +++ b/frontend-modern/public/docs/FAQ.md @@ -261,6 +261,11 @@ LXC or Docker chart alone does not establish a leak. Use the [read-only memory checks](TROUBLESHOOTING.md#memory-use-keeps-growing) for your deployment while it is responsive. +A higher steady baseline is worth reporting even without continuing growth. +A lower reading ten minutes into a new process is not a like-for-like comparison +with a week-old run, or proof of recovery. Keep already available readings, +versions, uptime and workload context; do not restart to create a comparison. + Do not shorten retention, slow polling, restart Pulse or drop caches just to lower the reading. Shorter retention removes history, and slower polling can delay monitoring; neither establishes the cause. If the container is near its diff --git a/frontend-modern/public/docs/ISSUE_TRIAGE.md b/frontend-modern/public/docs/ISSUE_TRIAGE.md index 7330851173..7d283f87fe 100644 --- a/frontend-modern/public/docs/ISSUE_TRIAGE.md +++ b/frontend-modern/public/docs/ISSUE_TRIAGE.md @@ -132,6 +132,17 @@ profiles, heap dumps, databases or full process command lines in a public thread; use locally reviewed counter summaries, without restarting, creating load or changing polling or retention just to measure. +For Pulse memory reports, a higher steady baseline and continuing growth are +different problems. A plateau does not dismiss an increased memory requirement; +do not require a leak or a rising series before retaining a baseline regression. +Compare already supplied process scope, units, versions, uptime, workload and +enabled collectors/features. A week-old run and a ten-minute new process are +not comparable steady-state evidence, and a startup CPU saving does not prove +lower RSS. Preserve those limits instead of claiming recovery or asking for +another restart, downgrade, profile or unchanged sample. If the reporter +clarifies that the problem is the baseline, carry that goal to the engineering +owner; growth-rate evidence alone does not answer it. Unknown context is valid. + For VM or LXC memory disagreements, retain where each existing reading appears (overview, History, alert, Patrol or another tool), whether the comparison was measured inside the affected guest or on its hypervisor node, and the original diff --git a/frontend-modern/public/docs/TROUBLESHOOTING.md b/frontend-modern/public/docs/TROUBLESHOOTING.md index 731908474a..1ab6df91f3 100644 --- a/frontend-modern/public/docs/TROUBLESHOOTING.md +++ b/frontend-modern/public/docs/TROUBLESHOOTING.md @@ -667,6 +667,22 @@ own cache accounting. Neither is interchangeable with Pulse's resident memory (RSS), a Go heap size, or virtual address space (`VSZ` / `VmSize`). A large container reading alone does not establish a Pulse memory leak. +**A higher steady baseline is also worth reporting; continuing growth is not +required.** First distinguish what your existing readings show: + +| Observation | What it establishes | +| --- | --- | +| Comparable Pulse process RSS settles at a higher level than before with an unchanged workload | A possible baseline regression, even if usage then stays flat. Keep the old and new versions, uptime and measurement scope; a plateau does not dismiss the increased requirement. | +| Comparable samples rise during one uninterrupted process run | Growth during that interval, not by itself a leak or its cause. Keep the same PID/start time, units, workload and collection settings. | +| A new process uses less memory shortly after an upgrade than an older process did after days | Different runs at different ages, not verified recovery or growth within one run. Startup and long-lived usage can differ. | + +For example, **510 MiB after a week** and **420 MiB ten minutes into a new +process** cannot establish that steady-state memory improved. Use already +available readings at comparable uptimes and workloads if you have them; +otherwise keep the mismatch explicit. Do not restart, downgrade or change +retention, polling or enabled features to manufacture a comparison. A startup +CPU improvement is not evidence that the resident-memory baseline fell. + For a responsive **Linux systemd / Proxmox LXC** install, take this small, read-only sample at a normally occurring high point. Run it **inside the Pulse container** for LXC, not on the Proxmox host. Substitute the actual service name @@ -738,7 +754,8 @@ values. If it fails or Pulse stops, retain that fact; do not loosen access controls or keep retrying against an unresponsive installation. Keep the sample time, PID/start time, running version, container memory limit, -fleet size, polling interval and whether dashboards were open. If safe, compare +fleet size, polling interval, enabled collectors/features and whether dashboards +were open. If safe, compare another naturally occurring point with the **same PID and process start time**; record any restart or upgrade as a different run, not evidence that the cause was fixed. For Docker, the [bounded container statistics below](#excessive-cpu-writes-or-database-growth) diff --git a/scripts/tests/test_memory_troubleshooting_docs.py b/scripts/tests/test_memory_troubleshooting_docs.py index 19e7bca90f..c16287a100 100644 --- a/scripts/tests/test_memory_troubleshooting_docs.py +++ b/scripts/tests/test_memory_troubleshooting_docs.py @@ -321,6 +321,28 @@ class MemoryTroubleshootingDocsTest(unittest.TestCase): r"(?