The typed helper runs with PrivateNetwork=true. pct list, pct df and
lxc-info reach pmxcfs and the LXC monitor through abstract Unix
sockets, which belong to the network namespace, so inside the helper
they always fail. Least-privilege installs using the helper therefore
reported provider_unavailable for proxmox.lxc_filesystems on every
report and never got per-mount guest usage (#2511).
Discover running guests without commands: list /etc/pve/lxc through the
pmxcfs mount, find each guest's init in /sys/fs/cgroup/lxc/<vmid> (NSpid
exactly "<pid> 1", cgroup inside lxc/<vmid>), and name it by hostname or
CT<vmid> as pct list does. Discovery hands the whole node back to pct
whenever a guest's state cannot be established, so a guest it could not
read never disappears from a healthy-looking inventory.
Read usage through the shared filesystem prober instead of a second,
plain statfs path. It now accepts lxc requests: it pins the process,
requires the guest's exact cgroup and init (a nested namespace's PID 1
cannot stand in), skips declared mounts that are not mounted, and keeps
one bounded probe per guest. The helper's RestrictSUIDSGID makes
systemd answer openat2 with ENOSYS, so the prober falls back to a
component walk that refuses every symbolic link rather than resolving
it; Docker and Podman reads gain the same fallback.
Exercised on a disposable PVE 9.2.18 node under the helper unit's
sandbox properties: unprivileged and privileged guests matched pct df,
a stopped guest was omitted, and the prober's Linux tests passed as
root. A second opinion reviewed the design and the first draft; this
version addresses its findings.
Since #2550, three command identity tests fail on Windows x64:
TempDir cleanup cannot delete operation-receipts.db because the agent's
command client still holds it open. Those tests never call run, so
nothing closes the client that New built, and Windows refuses to delete
an open file. Other platforms delete it anyway.
Stop and close the command client in a harness cleanup. It is
registered after the state TempDir, so it runs first, and Close is
idempotent for the tests that do run the agent. Production already
closes the client when Run exits or an identity supersedes it.
A token binds the first agent ID its command channel presents and never
moves. The agent registered that channel before its first report, under
its configured --agent-id, or an ID Pulse had forked for a cloned
machine ID or kept from an earlier enrollment. Pulse meanwhile resolved
its reports to another host ID and acknowledged it, and the agent
persisted that ID. The config gate, which judges the resolved ID, then
turned the operator's commands off, and the next restart under the
persisted ID was refused for good.
Report acknowledgements now name commandAgentId: the resolved host ID,
only when the token's binding already holds it or a registration
presenting it would take the first bind, backfill, legacy migration,
hostname rebind or deploy repair it would get anyway. The credential
must also be one command admission takes, and the host must be held
under the reporting token. Naming grants nothing that registration
would not. An established binding to another ID is never moved from
report contents, because resolution can map a report onto another
host's continuity record by machine ID, so a forged report and a stale
one look the same. The config gate takes the same decision.
The agent keeps its channel staged until a report is acknowledged and
registers under the named identity, or under its own ID when none is
named, which is what an older Pulse or an already-bound token sees.
An unreadable ack keeps it waiting, and a 403 releases it under its own
ID because Pulse can never answer that token. A running client is never
rebuilt, since that would reopen its receipt store under in-flight
operations. A superseded client can no longer take the run slot.
Tokens already bound to a diverged ID stay as they are; re-running the
install command issues a fresh token that binds the resolved identity.
A deployed agent's first run gave the remote config client, the updater, and
the Docker / Podman and Kubernetes modules the bootstrap token, which only the
host module exchanged. Pulse revokes that token on exchange, so those clients
failed with 401 until the service restarted. Enroll once at startup and build
every module from the runtime token and the server-assigned agent ID.
Native Windows and macOS correctly advertise no PVE guest guard, so their real-server fixture must assert refusal before any config or QGA handoff. Keep the existing Linux admission, outcome and resumption assertions and exercise foreign platform labels without changing runtime capability policy.
Contract-Neutral: Test-only platform evidence repair; guest execution and Discovery runtime contracts, capability advertisement and safety gates are unchanged.
Change-source: pulse-maintainer
Project verified guest status and decoded output instead of qm transport success. Reject ambiguous terminal outcomes without replay; retain known failed or truncated results as failures without inventing uncertainty. Admit only successful stdout to Discovery and preserve saved results when attempted probes yield no evidence. Keep backup precautions, source bounds and native acceptance dependencies unchanged.
Change-source: pulse-maintainer
Add source controls for actual guest exit/output projection, ambiguous terminal receipts, failed probe evidence exclusion, and saved manual/background Discovery through explicit recovery. The fixtures never invoke PVE or QGA.
Contract-Neutral: Test-only regression controls exercise existing command and Discovery boundaries without changing production code, payloads, capabilities or budgets.
Change-source: pulse-maintainer
A zero qm CLI exit may return only a handed-off PID or an unverified guest result. Preserve actual CLI output/exit semantics but pause uncertain terminal dictionaries without another QGA command; accept complete canonical termination evidence and keep native thaw acceptance separate.
Change-source: pulse-maintainer
A cancellation before or during postflight config verification must not become an ordinary optional Discovery failure with usable output, especially on the final probe. Preserve fixed safety classification and no-handoff truth on canceled local admission.
Contract-Neutral: Completes the existing cancellation and unavailable-evidence requirements without changing protocol fields, guard version, lock/identity admission, time budgets or command catalogues.
Change-source: pulse-maintainer
The first exact proof established all ten parent no-admission failures and registered-session controls, then stopped on a missing strings import. Complete that import and preserve ordinary hyphenated PVE dictionary keys while keeping duplicate/case-conflicting lock rejection.
Contract-Neutral: Preserves the guest safety protocol and its lock, identity, budget and dispatch requirements; completes compilation and accepts canonical hyphenated config keys.
Change-source: pulse-maintainer
Require local current-node lock evidence and registered guard support before QGA execution. Serialise VM probes, stop on deferral, retain actual in-flight outcomes and saved Discovery evidence, and pause uncertain completion without replay. Config checks do not reserve QGA or prove native thaw; the existing backup monitoring-outage precaution remains.
Change-source: pulse-maintainer
Approvals and trusted Discovery commands currently cross QGA handoff without evidence of a local unlocked guest. Assert explicit no-handoff deferral without contacting a native target.
Change-source: pulse-maintainer
Retain configured RAID members separately from source-native totals through collection, reports and canonical read views. Correct the healthy legacy mdadm tuple from #2369 without subtracting spares from mdstat requirements or suppressing real deficits, failures and recovery warnings. Preserve observed zero-active counts through fallback, and verify collector, wire, ingestion, health and canonical activation/resolution boundaries.
Change-source: pulse-maintainer
Separate PVE/PBS setup and telemetry credentials from shell source and download URLs. Use silent root/sudo input, private-file handoff, complete downloads and the agent preflight; preserve scope, TLS defaults, single-line paste and coherent rolling-upgrade metadata. Reveal tokens through the existing dialog and discard late issuance after close. Pin executable shell, history, TLS/registration and browser contracts without using real credentials.
Change-source: pulse-maintainer
Fall back to bounded, type-checked thermal-zone and hwmon sysfs readings when lm-sensors is absent or empty. Classify Armada thermal JSON as a CPU reading, preserve nonempty sensor output, and cover units, invalid values, and the agent mapping.
Change-source: pulse-maintainer
The agentexec read loop removed a dead session from the agent map but never
closed its done channel, so any dispatch already sent on that session kept
waiting for a result that could never arrive and only returned at its
operation timeout. That's what made the Proxmox cancellation replay test
flaky on the shared CI runner. The restarted action runner dropped its first
session right after registering, the replay had already been sent there,
and the server waited the full 30s while the runner sat reconnected on a
new session.
The read loop now signals the session done on exit, so in-flight typed
dispatches and commands report the disconnect straight away. The
integration test sends the same request again on the runner's next
session when that happens, which is the same durable replay it is there
to prove, still with exactly one mutation. It also keeps the runner's log
and prints it on failure so the cause of any future drop is visible.
The Windows unified agent exits at startup when HKLM\SOFTWARE\Microsoft\Cryptography\MachineGuid is a braced GUID. gopsutil v4.26.6 reads the value through a fixed 74-byte buffer and rejects any length other than 36, so a braced 38-character GUID fails the whole InfoWithContext call and newAgent aborts with 'fetch host info'. Recover the host info on Windows when the host-ID read is the failing field: read MachineGuid directly, strip braces and fill the other fields from gopsutil accessors, while keeping any non-host-ID failure fatal. Canonicalise Windows machine IDs by removing braces so both forms map to one stable agent identity. Includes agent-lifecycle contract section and focused proof in the registry-accepted agent_new_test.go. Windows recovery is cross-compiled with GOOS=windows; runtime execution on Windows is not qualified in this environment.
Change-source: pulse-maintainer
Carry the operator include override in disk reports so server filtering does not discard selected tmpfs mounts again. Keep automatic filtering for unmarked reports and agent exclusion precedence. Cover both ingest paths, collection, forwarding and wire compatibility for #1875.
Change-source: pulse-maintainer
Issue #1895 reports parity alerts when mdNumDisks=0 on a pool-only Unraid system. Array service state alone does not establish that a parity array exists.
Preserve the optional disk count from collection through canonical runtime conversion and suppress only the no-parity warning for an explicit zero. Missing or malformed counts retain legacy behaviour, and disk failure reasons remain active.
Validated focused Unraid tests in hostagent, storagehealth, monitoring, unifiedresources and alerts, including JSON zero preservation and canonical round trip. The new pool-only regression fails against the previous warning condition. Both agent and server need this change; no release or reporter retest is claimed.
Change-source: pulse-maintainer
Near-synchronous host and Docker reports consumed a shared CPU baseline, measuring collection bursts rather than each module's reporting interval. Retain a collector per host collector and a separate Docker module collector while preserving the package-level convenience API and disk filters.
Add an interleaved-counter regression covering both collection entry points. It fails when routed through the shared baseline and passes with isolated state. All hostmetrics, hostagent and dockeragent tests pass, as do the focused CPU regression tests under the race detector.
Change-source: pulse-maintainer
Treat Unraid's fsType=auto value as a placeholder rather than disk assignment evidence at both agent collection and server ingestion boundaries. Preserve real assigned and explicit missing members.
Change-source: pulse-maintainer
Contract-Neutral: Unraid fsType=auto placeholder normalization fixes false missing-slot alerts without changing wire or subsystem contracts
(cherry picked from commit fd843da7dfc51bba29bb5ddcbcdb4480d0e67079)
(cherry picked from commit 7a6456969d)
Since 60d0651a88 every typed request registers a per-connection cancellable
slot that its handler goroutine releases in a deferred cleanup after sending
its result. The server replays a request id when it wants the durable receipt
again, and that replay can reach the reader before the previous handler's
deferred release runs. launchCancellableRequest treated that as a duplicate
and dropped it, so the server waited out the operation's full timeout for a
result the agent already held. The Linux x64 native-verification leg failed
this way on 12 of the last 25 main runs, always on a "replay 1" dispatch of
host update, storage cleanup, or Docker lifecycle.
Give each slot a done channel that closes on release. A replay whose id is
still registered on the same connection now waits for that release and then
runs, answering from the durable receipt. Invalid ids and over-capacity
requests are still dropped. A unit test pins the wait-then-run behaviour and
the agent-lifecycle contract records the replay rule.
The cancellation replay test stopped the first action runner and started a
second one as soon as the client goroutine exited. The server observes the
socket close on its own reader, so under GOMAXPROCS=1 with the race detector
the second startRunner saw the stale session as still connected and the
replay was dispatched to the dead socket, timing out after 30s on the
sharded release preflight while passing on multi-core hosts. Wait until the
server reports the agent disconnected before reconnecting, as the agentexec
server tests already do.
The group-writable executable check wrote its fixture through os.WriteFile,
which filters the mode through the process umask, so under the runner's 022
umask the group-write bit never reached disk and validateTrustedExecutable
correctly trusted the file. Chmod the fixture to the asserted mode.
The Proxmox guest executor tests stamped their after observations one second
past the test-start clock, but actionStartedAt is stamped inside
ExecuteAction after handler setup, so a loaded runner overran the margin and
the independent observation was discarded as pre-action. Widen the
observation offset to a minute.
Both failures broke build-and-test on main (run 33454838531). Reproduced the
hostagent failure on Linux under umask 022 and verified both packages green
after the fix.
Join Proxmox config paths with Linux semantics across native test platforms, add the regression proof, and update the canonical agent lifecycle contract for the new namespace-statfs collector.
Change-source: pulse-maintainer
pct df takes the container config lock and costs over a second per
guest, and every per-container query shared one 10-second budget with
pct list, so hosts with more than a handful of running containers only
ever reported filesystem usage for the first few (signal: killed on
the rest).
Resolve per-mount usage with statfs through /proc/<pid>/root instead:
read the config-declared mounts from /etc/pve/lxc, resolve the init
pid via lxc-info, and drop config mounts whose device identity matches
their parent directory rather than inventing the parent's numbers.
pct df remains as a per-container fallback for least-privilege
installs that cannot traverse /proc/<pid>/root, each container now
carries its own deadline inside a bounded collection window, and
exhausting that window logs how many containers were skipped instead
of failing silently at debug level.
Refs #1477
rcourtman