219 Commits

Author SHA1 Message Date
pulse-triage[bot] c8cead7d5a fix(agents): preserve explicit disk includes at ingestion
Carry the operator include override in disk reports so server filtering does not discard selected tmpfs mounts again. Keep automatic filtering for unmarked reports and agent exclusion precedence. Cover both ingest paths, collection, forwarding and wire compatibility for #1875.

Change-source: pulse-maintainer
2026-09-09 11:35:51 +01:00
pulse-triage[bot] 3334cccfd9 fix(unraid): suppress parity warning for explicit pool-only arrays
Issue #1895 reports parity alerts when mdNumDisks=0 on a pool-only Unraid system. Array service state alone does not establish that a parity array exists.

Preserve the optional disk count from collection through canonical runtime conversion and suppress only the no-parity warning for an explicit zero. Missing or malformed counts retain legacy behaviour, and disk failure reasons remain active.

Validated focused Unraid tests in hostagent, storagehealth, monitoring, unifiedresources and alerts, including JSON zero preservation and canonical round trip. The new pool-only regression fails against the previous warning condition. Both agent and server need this change; no release or reporter retest is claimed.

Change-source: pulse-maintainer
2026-09-04 23:41:22 +01:00
pulse-triage[bot] 0f972f42f2 Isolate host and Docker CPU sampling baselines (#1894)
Near-synchronous host and Docker reports consumed a shared CPU baseline, measuring collection bursts rather than each module's reporting interval. Retain a collector per host collector and a separate Docker module collector while preserving the package-level convenience API and disk filters.

Add an interleaved-counter regression covering both collection entry points. It fails when routed through the shared baseline and passes with isolated state. All hostmetrics, hostagent and dockeragent tests pass, as do the focused CPU regression tests under the race detector.

Change-source: pulse-maintainer
2026-09-04 22:29:32 +01:00
pulse-triage[bot] ace72f8442 Ignore Unraid auto filesystem on empty slots
Treat Unraid's fsType=auto value as a placeholder rather than disk assignment evidence at both agent collection and server ingestion boundaries. Preserve real assigned and explicit missing members.

Change-source: pulse-maintainer
Contract-Neutral: Unraid fsType=auto placeholder normalization fixes false missing-slot alerts without changing wire or subsystem contracts
(cherry picked from commit fd843da7dfc51bba29bb5ddcbcdb4480d0e67079)
(cherry picked from commit 7a6456969d)
2026-09-02 05:40:46 +01:00
rcourtman b1044cd8a4 Let replayed request ids wait for the in-flight handler instead of dropping
Since 60d0651a88 every typed request registers a per-connection cancellable
slot that its handler goroutine releases in a deferred cleanup after sending
its result. The server replays a request id when it wants the durable receipt
again, and that replay can reach the reader before the previous handler's
deferred release runs. launchCancellableRequest treated that as a duplicate
and dropped it, so the server waited out the operation's full timeout for a
result the agent already held. The Linux x64 native-verification leg failed
this way on 12 of the last 25 main runs, always on a "replay 1" dispatch of
host update, storage cleanup, or Docker lifecycle.

Give each slot a done channel that closes on release. A replay whose id is
still registered on the same connection now waits for that release and then
runs, answering from the durable receipt. Invalid ids and over-capacity
requests are still dropped. A unit test pins the wait-then-run behaviour and
the agent-lifecycle contract records the replay rule.
2026-09-01 23:22:55 +01:00
rcourtman e1e4c3e700 Wait for the server to observe runner disconnect before replay reconnect
The cancellation replay test stopped the first action runner and started a
second one as soon as the client goroutine exited. The server observes the
socket close on its own reader, so under GOMAXPROCS=1 with the race detector
the second startRunner saw the stale session as still connected and the
replay was dispatched to the dead socket, timing out after 30s on the
sharded release preflight while passing on multi-core hosts. Wait until the
server reports the agent disconnected before reconnecting, as the agentexec
server tests already do.
2026-09-01 22:51:56 +01:00
rcourtman 60d0651a88 Cancel abandoned typed agent operations 2026-09-01 11:21:26 +01:00
rcourtman 4e22a49f02 Add native PVE action qualification harness 2026-09-01 10:43:35 +01:00
rcourtman 01ff388c7e Preserve native Proxmox guest handoff 2026-09-01 09:31:40 +01:00
rcourtman 2ad094c927 Deflake typed action containment tests
The group-writable executable check wrote its fixture through os.WriteFile,
which filters the mode through the process umask, so under the runner's 022
umask the group-write bit never reached disk and validateTrustedExecutable
correctly trusted the file. Chmod the fixture to the asserted mode.

The Proxmox guest executor tests stamped their after observations one second
past the test-start clock, but actionStartedAt is stamped inside
ExecuteAction after handler setup, so a loaded runner overran the margin and
the independent observation was discarded as pre-action. Widen the
observation offset to a minute.

Both failures broke build-and-test on main (run 33454838531). Reproduced the
hostagent failure on Linux under umask 022 and verified both packages green
after the fix.
2026-09-01 06:15:46 +01:00
rcourtman f9ae289bd5 Contain typed action subprocesses with systemd 2026-09-01 01:26:39 +01:00
rcourtman f967857928 Harden secure agent recovery transports 2026-08-31 23:12:00 +01:00
rcourtman cf021bc9ce Surface typed helper container degradation 2026-08-31 15:53:50 +01:00
rcourtman d19398d82a Surface typed helper degradation in Agent Doctor 2026-08-31 13:06:28 +01:00
Richard Courtman 08f7c5f0d5 Harden secure agent runtime boundaries 2026-08-31 00:06:24 +01:00
Richard Courtman 770733fc92 Make action runner rotation activation-safe 2026-08-30 18:00:33 +01:00
Pulse Test c0b190bfa1 Add application response contracts to availability checks 2026-08-30 17:38:59 +01:00
Pulse Test 712f1a3bfe Add availability history and fleet view 2026-08-30 15:38:34 +01:00
pulse-triage[bot] 95a7191ca9 Enforce native private state permissions 2026-08-30 03:31:00 +01:00
pulse-triage[bot] 184b3cbfa6 Restore Windows native agent verification 2026-08-30 02:05:30 +01:00
Pulse Test d06ffc233d Harden secure agent runtime transitions 2026-08-30 01:41:57 +01:00
Pulse Test 5b2e377843 Expose secure agent runtime posture 2026-08-30 00:38:44 +01:00
Pulse Test d607d5cf46 Separate agent remediation runtime 2026-08-29 23:48:28 +01:00
Pulse Test 6d4ee48000 Add typed agent privilege helper 2026-08-29 22:51:58 +01:00
Pulse Test a966264bb1 Contain agent command authority 2026-08-29 22:12:41 +01:00
pulse-triage[bot] 5570ac1e2e fix(agent): silence unchanged remote config refreshes 2026-08-28 21:56:26 +01:00
pulse-triage[bot] 9e820355b6 Fix Unraid sentinel empty slot reporting 2026-08-28 14:20:43 +01:00
pulse-triage[bot] cb27b8eb24 Fix Unraid empty slot health reporting 2026-08-28 01:23:12 +01:00
rcourtman a9076de15a fix(patrol): require Docker health recovery 2026-08-27 20:43:54 +01:00
rcourtman 6163a2a564 feat(patrol): verify Docker actions independently 2026-08-27 20:14:38 +01:00
pulse-triage[bot] 8fde82b8a2 Keep LXC filesystem paths native-proof
Join Proxmox config paths with Linux semantics across native test platforms, add the regression proof, and update the canonical agent lifecycle contract for the new namespace-statfs collector.

Change-source: pulse-maintainer
2026-08-27 06:51:36 +01:00
rcourtman 8318c74687 fix(hostagent): stop pct df from starving LXC filesystem collection
pct df takes the container config lock and costs over a second per
guest, and every per-container query shared one 10-second budget with
pct list, so hosts with more than a handful of running containers only
ever reported filesystem usage for the first few (signal: killed on
the rest).

Resolve per-mount usage with statfs through /proc/<pid>/root instead:
read the config-declared mounts from /etc/pve/lxc, resolve the init
pid via lxc-info, and drop config mounts whose device identity matches
their parent directory rather than inventing the parent's numbers.
pct df remains as a per-container fallback for least-privilege
installs that cannot traverse /proc/<pid>/root, each container now
carries its own deadline inside a bounded collection window, and
exhausting that window logs how many containers were skipped instead
of failing silently at debug level.

Refs #1477
2026-08-27 06:22:10 +01:00
pulse-triage[bot] a4a26fe842 Preserve availability result delivery order
Change-source: pulse-maintainer
2026-08-26 19:32:17 +01:00
rcourtman 4bca38395a Harden custom sensor test fixture permissions 2026-08-26 19:25:56 +01:00
rcourtman 434e1448ff Restore Docker update preflight through unified agent 2026-08-23 08:35:26 +01:00
rcourtman ebb08e0de2 Make native-agent test fixtures path-portable on Windows
The dockeragent credential-store tests keyed their fake filesystem and
path-bearing env seams on POSIX literals, while dockerConfigCredentials
builds candidate config paths with filepath.Join. On Windows every lookup
therefore missed the fixture and the nine credential-expecting
TestDockerConfigCredentials_* cases failed with a nil credential; the
hostagent PULSE_PCT_PATH override test likewise asserted a POSIX path that
is not absolute on Windows. Map fixture keys and env values through
filepath.FromSlash and anchor the override on t.TempDir().

Taken from PR #1755 (ff96aed8c) so the correction lands on main instead of
a parked agent branch. The unified-agent-native Windows x64 job has been
red on every push run since these tests landed on 2026-08-13; the
intervening green runs were pull_request events exercising that PR's own
corrected fixtures.

Contract-Neutral: test fixtures only, no production runtime change.
2026-08-21 22:36:52 +01:00
rcourtman 4c7b1a2434 Fix Docker-in-LXC probe storm against slow Proxmox hosts
The guest Docker socket probe hung minipc hard enough to need a power
cycle (2026-08-20): ~100 orphaned pct exec children, load 133, sshd and
pveproxy starved. Three bugs chained, each fixed here:

1. Dispatcher re-issued a probe while the previous one was still
   executing. The poll cycle's enrichment context had expired, so
   ExecuteCommand dispatched, returned the context error 50ms later,
   and the next 3s cycle sent the identical command again — unbounded
   concurrency against a host that was slow to begin with. The
   monitoring dispatcher now takes a per-guest in-flight claim before
   dispatching probe or inventory commands (completed probes release
   it; abandoned ones hold it for a 2-minute window), and both dispatch
   paths bail out under a dead context.

2. The host agent never got the July process-leak fix: 45480a5cc
   landed only on pulse/v6-release, so main-line agents killed just the
   direct shell on timeout, orphaning pct exec → lxc-attach children
   and blocking Wait on their inherited pipes (10s timeouts reported as
   300s+ durations). Port it: run each command in its own process
   group, SIGKILL the group on cancel, bound Wait with WaitDelay, and
   treat ErrWaitDelay after a clean exit as success.

3. Server-side abandonment never reached the agent. ExecuteCommand and
   ReadFile now refuse to dispatch under an already-expired context,
   and send a best-effort cancel_command when they stop waiting; the
   agent cancels the in-flight execution (killing its process group)
   and reports "command canceled". Older agents ignore the unknown
   message type.

Also add a per-node circuit breaker: three consecutive command failures
on one node suspend all Docker probe/inventory dispatch to it on the
existing 1m→30m backoff schedule, so a host-level stall (NFS flapping)
stops the probing entirely instead of failing guest by guest.

Regression tests simulate the storm without hardware: a never-returning
executor is not re-issued across poll cycles, an expired context
dispatches nothing and records no failure, abandoned probes hold their
claim, the breaker blocks new guests on a failing node, and the agent
kills the whole process group on timeout and on server-issued cancel.

Contract-Neutral: monitor.go delta is three private struct fields holding Docker probe dispatch state; host-agent deletion/re-enrollment lifecycle untouched — contracts and all other proofs are staged
2026-08-20 23:33:36 +01:00
rcourtman 4686efd8c8 Add a supported least-privilege agent install profile
The unified agent's Linux installer only offered the root profile, and the
docs called non-root unsupported. That default is the most-cited reason
security-conscious evaluators reject Pulse without trying it. install.sh
gains --least-privilege: the service runs as a dedicated nologin
pulse-agent system user with every existing hardening directive, no
LXC-attach ambient capabilities, docker-group membership for socket reads,
and refusal (not silent root fallback) on appliance platforms, non-systemd
init systems, and --enable-commands. Optional --grant-smart and --grant-pct
restore the only two collectors that need elevation through
visudo-validated exact-command sudoers rules and root-owned wrappers the
agent reaches via new absolute-path-only PULSE_SMARTCTL_PATH /
PULSE_PCT_PATH overrides; the pct grant covers pct list and pct df only
and can never widen into pct exec. --update preserves the profile and its
grants by reading the installed unit.

The agent now authors a privilege block in its report (effective root,
service user, active helpers), carried through models into the fleet
doctor as a descriptive field: Agent Doctor shows the profile and its
helpers instead of presenting intentionally absent collectors as a fault,
and a least-privilege agent can never be marked unhealthy on that evidence
alone.
2026-08-20 20:49:44 +01:00
rcourtman 2d801e20f3 test(hostagent): stop the standby tests reading the runner's own disks
TestCollectDeviceSMARTStandby and its observability twin stub smartctl's
execution but not the device-class probe, so collectDeviceSMART fell through
to the *runner's* real /sys/block/sda/queue/rotational.

smartctlArgs only sends the -n standby guard when the disk is not a confirmed
SSD (#1516), and smartctlArgsUseStandbyExitStatus gates the standby reading of
exit status 3 on that guard having been sent. On a Linux host whose own
/dev/sda is non-rotational the guard is dropped, exit 3 stops meaning standby,
and both tests fail deterministically:

    run smartctl for /dev/sda: exit status 3

They pass on macOS only because linuxNonRotationalBlockDevice returns false off
Linux, and on Linux only where /dev/sda is absent or spinning. That makes the
release-qualification suite unrunnable on an ordinary SSD-backed Linux worker.

Pin the probed disk to rotational through the package's existing stubLinuxSysfs
seam so the guard is always in play. Product behaviour is untouched; this only
stops two unit tests depending on the hardware underneath them.
2026-08-19 09:45:44 +01:00
rcourtman ff0c9ea4d8 fix(agent): warn when the server overrides a presented agent ID
Host identity continuity deliberately keeps a known machine on its
enrolled ID, so a custom --agent-id or hand-edited agent-id state file
is silently ignored for an already-known host, and the acknowledged ID
then overwrites the state file within one report cycle. That looked
like corruption from the operator's seat. Warn once per resolved
identity with both IDs and the supported path to a fresh enrollment
(remove the host in Pulse first).

Refs #1739

Contract-Neutral: diagnostic logging only in hostagent report ack path, refs #1739
2026-08-18 06:25:20 +01:00
rcourtman 48fac73ff0 Rework Patrol around outcome-driven autonomous operations 2026-08-14 12:52:51 +01:00
rcourtman e31fc37983 Gate Patrol actions on agent preflight 2026-08-14 01:12:49 +01:00
rcourtman a7646e5f86 Fix Patrol autonomy and refusal reporting 2026-08-13 23:26:08 +01:00
rcourtman a39935182d Parse smartctl 7.5 power_mode object on guarded probes
smartmontools 7.5 emits power_mode as an {ata_value, name} object whenever
the -n guard runs CHECK POWER MODE, which is every rotational-disk probe.
The parser declared the field as a string, so json.Unmarshal failed for the
whole document and healthy spinning disks degraded to the lossy text
fallback, surfacing as no usable SMART data while guard-free SSD probes
kept working. That is the exact rotational-only failure split in the
discussion 1690 debug log (SAS3224 HBA, smartctl 7.5). Decode both shapes
the field has used, key standby on the reported name, and never fail the
document over this field. Also broaden the text fallback standby match to
the EPC names (STANDBY_Y, STANDBY (OS), SLEEP) that the mode-suffixed
match missed.

Refs #1690

Contract-Neutral: smartctl 7.5 power_mode JSON parse fix, internal decoder only, no DiskSMART payload or subsystem contract delta (discussion #1690)
2026-08-13 10:25:28 +01:00
courtmanr@gmail.com 4dac4dd163 Allow agents to include filtered disk mounts 2026-08-11 16:37:37 +01:00
rcourtman d699dc66a0 feat(agent): converge agent self-update within one report cycle
The server now echoes its version on unified-agent report acks, and the
agent nudges its auto-updater the moment an ack carries a newer version.
After a server upgrade, agents converge within one report interval
instead of waiting out the hourly update check, so the "older Pulse
agent" notice self-resolves in seconds once upgraded agents report in.

The hourly loop stays as the retry and backstop path. Nudges dedupe per
server version, refuse downgrades, skip disabled and development-mode
updaters, and never fire from observer destination acks — only the
authoritative server may steer an agent's updater, and a nudged check
re-validates against the server and runs the existing checksum and
self-test pipeline before swapping binaries. Agents deployed before
this change still converge on their old hourly cadence once; every
upgrade after that lands within a report cycle.

Contract deltas recorded in agent-lifecycle and api-contracts, with
boundary notes in security-privacy (no update authority in the echo),
performance-and-scalability (no steady-state work), and
storage-recovery (nothing persisted).
2026-08-07 11:40:46 +01:00
Richard Courtman 99407ee74c Fix QNAP RAID role bitmap parsing
Addresses #1688.
2026-08-06 23:39:15 +01:00
rcourtman 9615b5f2b0 fix(security): close the four open CodeQL findings
Resolves every open code scanning alert on the repository. Dependabot and
secret scanning were already clear.

SMART temperature truncation (alerts 312, 313). parseRawValue returns a
64-bit raw attribute value, but DiskSMART.Temperature is an int, which is
32 bits wide on the 386 and arm release builds Pulse ships. The range check
ran after the narrowing conversion, so a raw value of 4294967316 truncated
to 20 and was published as a plausible 20 degree reading.
validSMARTTemperature64 now gates the conversion.

Provider MSP restore archive names (alert 314). cleanProviderMSPArchiveName
rejected a leading "../" but not a bare "..", which path.Clean produces from
entries such as ".." and "a/../..". pathIsInside caught the escape
downstream, so this was not exploitable, but the sanitizer now rejects it
outright instead of depending on a second gate.

TrueNAS device paths (alert 315). vdev.Device is supplied by the appliance,
concatenated into a path and published verbatim on ZFSDevice.Path, so values
like "//evil.example.com/share" and "/\evil.example.com" passed straight
through. devicePath now drops traversal segments and backslashes and
collapses a leading double slash. The alert's open-redirect framing does not
apply here, there is no redirect sink on this path, but the value is
untrusted input rendered as a path and is worth normalising.

Patrol readiness cache key (alert 311). The key is persisted to
ai_patrol_model_readiness.json and embedded an unkeyed SHA-256 of the Ollama
Basic Auth username and password. That password is chosen by a human, so
anyone holding the evidence file could recover it offline at two SHA-256
operations per guess. The fingerprint is now HMAC-SHA256 keyed with a
32-byte per-install salt stored beside the evidence at mode 600. Credential
rotation still invalidates the cache and the key still survives a restart.

Each fix carries a regression test confirmed to fail against the previous
implementation.

monitoring.md carries the one warranted contract refinement. It already
required SMART temperature selection to accept only plausible readings, and
that rule now states the width at which plausibility is decided.

Contract-Neutral: CodeQL security fixes with no public-contract delta and no payload change. monitoring.md carries the one warranted refinement (SMART plausibility decided at 64-bit width). Residual demands are inapplicable: ai-runtime readiness prose documents interruption semantics, not cache-key derivation, and the credential-invalidation contract is unchanged; cloud-paid and deployment-installability contracts never name archive-entry sanitisation; agent-lifecycle owns smartctl.go but its SMART temperature prose lives in the staged monitoring.md.
2026-08-06 18:11:51 +01:00
rcourtman 198c66c6c4 feat(monitoring): alert on certificate validity
Refs #1673
2026-08-06 09:47:03 +01:00
courtmanr@gmail.com 9b6df327ad feat(proxmox): show all LXC filesystems 2026-07-30 22:42:04 +01:00