# Pulse Unified Agent The unified agent (`pulse-agent`) is the single host-installed Pulse infrastructure agent binary. It combines host, Docker/Podman, Kubernetes, Proxmox-local, and other enabled node-local telemetry modules into one deployment and one service. Install it on standalone hosts and on machines where Pulse needs full node-local telemetry. For API-backed platforms, start with the platform connection first and add the agent only where local telemetry is needed. For Proxmox, install the agent only where you need telemetry that the Proxmox API cannot provide, such as host SMART and temperature data, local ZFS/Ceph/mdadm detail, arbitrary host mount reads, or the full mounted filesystem breakdown for running LXCs. Docker containers inside LXCs can be reported by a Proxmox host agent when the server has explicitly enabled the privacy-bounded LXC inventory mode; Docker/Podman inside VMs still needs a guest-local agent or another explicit guest reporting path. Basic Proxmox inventory and utilization can use a read-only or narrowly scoped Proxmox API token instead. Settings uses that API inventory path as the default for new PVE/PBS setup. See [Agent Security](AGENT_SECURITY.md) for the root-service trade-off, restricted-user expectations, and supply-chain verification guidance. > Note: For agent-based temperature monitoring, use `pulse-agent --enable-proxmox` or SSH-based collection. The legacy sensor proxy has been removed. See `docs/TEMPERATURE_MONITORING.md`. When each Proxmox cluster node runs an agent, add `--disable-cluster-peer-sensors` (or set `PULSE_DISABLE_CLUSTER_PEER_SENSORS=true`) to keep temperature collection local to each node. This preserves local host telemetry and Proxmox integration while preventing duplicate root-SSH sensor polling between cluster peers. ## Quick Start Choose the host profile and create a monitoring token in the UI: **Settings → Infrastructure → Install on a host** Choose a target profile in that screen when you want explicit install flags for Docker, Kubernetes, Proxmox VE, or Proxmox Backup Server. Keep the token out of shell commands, history, URLs and screenshots. Use the private-file installation steps below; the command arguments contain only the file's path. A Pulse agent token is not a Proxmox or PBS API token. Monitoring does not require command execution: leave that option off unless you intend to grant it. See [Agent Security](AGENT_SECURITY.md) before choosing a root host agent instead of an API connection or the opt-in Linux safe profile. For a Proxmox VE cluster, one API connection already provides cluster-wide inventory; the agent is per host. Repeat the private-file installation on each cluster node where you want agent-provided telemetry (temperatures, SMART, host identity), using the intended profile. Each agent registers itself and attaches to its own cluster member. The same installer is also the supported v5-to-v6 agent upgrade path. Run it on the host that already has the v5 `pulse-agent` service to replace the binary and service configuration in place; do not uninstall the old service first unless you are intentionally removing that host from Pulse. ### Moving Pulse to a new address Configuration export/import restores API-token records, not the server-side agent inventory, enrolment state, profiles or assignments. See the [configuration-transfer scope](MIGRATION.md#configuration-transfer) before retiring the old server. Import cannot rewrite the primary Pulse URL on remote machines because agents initiate the connection. Prefer a stable DNS name for the primary URL, but still verify fresh reports and agent admission after a move. After importing the configuration on a Pulse server with a different address, retarget each existing standard Linux agent from that agent machine. Download and inspect the installer from the **new** Pulse address using the HTTPS preparation below; retargeting reuses the saved credential, so do not create another token just for this operation: ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --retarget --url https://pulse.example.com ``` The retarget operation recovers the existing token, agent ID, enabled collectors, and other service options. It does not carry the old endpoint's TLS bypass, custom CA, or certificate fingerprint to the new address. Supply `--cacert` or an independently verified `--server-fingerprint` explicitly when the new agent endpoint requires it. The installer download itself must use a trusted certificate or a separately verified CA file; an agent fingerprint option does not verify that earlier download. The script must come from the new server so it supports retargeting. If retargeting is not supported on that host, use the private-file installation below with the intended profile, rather than uninstalling the existing service first. On Windows, use the PowerShell private-file installation below from the new Pulse server as Administrator, including the desired collector options. Do not expect the configuration import itself to make agent-only machines appear at the new address. An installed agent has one **primary** Pulse URL and token. The primary is the only server allowed to supply remote configuration, commands, enrollment, or updates. The same collection can also be sent to explicitly configured, report-only **observer** instances; see [Observer destinations](#observer-destinations). After an upgrade, check the relevant platform page or **Machines** view once the agent has reported, and confirm the host-local version with `pulse-agent --version` if the UI has not received a fresh report yet. This is the agent installer served by your Pulse server. It is separate from the top-level GitHub `install.sh`, which installs or updates the Pulse server itself. ### Private-file installation: Linux, macOS and NAS Use an administrative shell **on the host being monitored**, not the Pulse server or an unrelated container. Linux/systemd, macOS, Synology and TrueNAS use the same preparation. TrueNAS SCALE uses systemd; CORE uses rc.d. An API connection is usually enough for TrueNAS inventory and usage; an agent is optional for host-local data. 1. In that shell, protect the file before opening the editor. Save only the Pulse agent token, with no quotes or header. This preparation preserves an existing regular token file and refuses symlinked or non-regular credential paths. **Stop if preparation fails**; do not continue to the installer: ```bash ( set -eu umask 077 config_dir="$HOME/.config/pulse" credential_file="$config_dir/agent-token" if [ -L "$HOME/.config" ] || [ -L "$config_dir" ] || [ -L "$credential_file" ]; then printf 'Refusing a symlinked credential path.\n' >&2 exit 1 fi if [ -e "$credential_file" ] && [ ! -f "$credential_file" ]; then printf 'Credential file must be a regular file.\n' >&2 exit 1 fi mkdir -p "$config_dir" chmod 700 "$config_dir" touch "$credential_file" chmod 600 "$credential_file" vi "$credential_file" ) ``` This is a plain token file, **not a shell script or `.env` file**. Do not include `PULSE_TOKEN=`, `export`, comments or surrounding quotes. For an existing authentication failure, use the [token-file format guidance](#token-file-format-is-not-shell-syntax) before changing the saved credential or running another installation. Before a first installation, check the complete value privately against the token just generated, not only its visible ends. Current **Infrastructure → Install on a host** tokens contain exactly **64 lowercase hexadecimal characters** (`0–9`, `a–f`); duplicated characters or an internal space or newline are not part of that value. This describes that generator, not every legacy, manually supplied or bootstrap credential. A matching shape alone does not authenticate a token. Keep the value out of commands, terminal output, screenshots and reports. 2. Replace `https://pulse.example.com` with your Pulse server's HTTPS address in both the download and installation commands. Download to a new private file. This ignores local curl settings, accepts only HTTP **200** over HTTPS, and does not follow redirects or overwrite an existing installer: ```bash ( set -eu umask 077 config_dir="$HOME/.config/pulse" installer_file="$config_dir/agent-install.sh" if [ -L "$HOME/.config" ] || [ -L "$config_dir" ] || [ ! -d "$config_dir" ]; then printf 'Prepare the private token directory first.\n' >&2 exit 1 fi if [ -e "$installer_file" ] || [ -L "$installer_file" ]; then printf 'Refusing to replace an existing installer path.\n' >&2 exit 1 fi chmod 700 "$config_dir" download_file=$(mktemp "$config_dir/agent-download.XXXXXX") curl_exit=0 status=$(curl --disable --fail --silent --show-error --proto '=https' \ --connect-timeout 10 --max-time 60 --output "$download_file" \ --write-out '%{http_code}' https://pulse.example.com/install.sh) || curl_exit=$? printf 'HTTP %s\n' "$status" [ "$curl_exit" -eq 0 ] || exit "$curl_exit" [ "$status" = 200 ] mv -n "$download_file" "$installer_file" [ ! -e "$download_file" ] ) ``` **Stop if the download fails.** A temporary download, redirect or TLS error is not an installer; do not execute its temporary file. Inspect the saved `agent-install.sh` only after a successful download. If that path already exists, review it locally rather than deleting or overwriting it blindly; retargeting needs the installer from the **new** server, not an older saved copy. Preserve any earlier copy separately through normal maintenance before downloading again. Keep `--disable` first in the curl command: local settings could enable tracing, redirects or a TLS bypass. Do not bypass certificate verification or pipe an unchecked response into a privileged shell. For a private CA, add curl's `--cacert` **after `--disable`**, using a separately verified CA file, and supply the installer's `--cacert` option for the agent connection as well. A successful download is not signature verification; inspect and trust the serving Pulse installation before executing its script. Do not substitute GitHub's top-level `install.sh`: that installs the Pulse server, not the agent. 3. Install with the private token file, adding a profile from [Installation Options](#installation-options) when needed: ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" ``` Keep the bootstrap token file private and remove that copy when no longer needed; do not delete the installed agent's runtime credential. Check a fresh report in Pulse and `pulse-agent --version` on the host. A started service or hardware capacity alone does not prove that every requested metric is present. ### Windows (PowerShell, run as Administrator) Use a **new** setup directory under your own Windows profile; if it already exists, choose another name. The following preparation grants access only to your account and SYSTEM. Stop if directory creation or the ACL command fails: ```powershell $setupDir = Join-Path $env:USERPROFILE 'PulseAgentSetup' New-Item -ItemType Directory -Path $setupDir -ErrorAction Stop | Out-Null $userSid = [Security.Principal.WindowsIdentity]::GetCurrent().User.Value icacls.exe $setupDir /inheritance:r /grant:r "*${userSid}:(OI)(CI)F" '*S-1-5-18:(OI)(CI)F' if ($LASTEXITCODE -ne 0) { throw 'Could not protect the setup directory' } $tokenFile = Join-Path $setupDir 'agent-token.txt' New-Item -ItemType File -Path $tokenFile -ErrorAction Stop | Out-Null Start-Process -FilePath notepad.exe -ArgumentList "`"$tokenFile`"" -Wait ``` In Notepad, save only the Pulse agent token, with no quotes or header. Never paste it into a PowerShell command or assign a literal secret to an environment variable. Replace the example HTTPS address in both commands below. Download and inspect the script first; do not use `Invoke-Expression` on a web response: ```powershell $installerFile = Join-Path $setupDir 'install.ps1' Invoke-WebRequest -Uri 'https://pulse.example.com/install.ps1' -OutFile $installerFile -ErrorAction Stop ``` Only after a successful download and inspection, run the saved script. Clear an old `PULSE_TOKEN` environment value so it cannot override the file: ```powershell Remove-Item Env:PULSE_TOKEN -ErrorAction SilentlyContinue & $installerFile -Url 'https://pulse.example.com' -TokenFile $tokenFile ``` Keep the setup directory private and remove the bootstrap copy when no longer needed, not the installed service's token. Verify a fresh report and the installed agent version; the remaining platform trust trade-offs are in [Agent Security](AGENT_SECURITY.md). ## Features - **Host Metrics**: CPU, memory, disk, network I/O, temperatures - **Docker Monitoring**: Container metrics, health checks, Swarm support (when enabled) - **Kubernetes Monitoring**: Cluster, node, pod, and deployment health (when enabled) - **Libvirt/KVM Monitoring**: Read-only VM inventory, state, vCPU, memory, and disk/network rates when the Linux host exposes `virsh` - **XCP-ng Monitoring**: Read-only pool and VM inventory, power state, vCPU, and memory when an XCP-ng control domain exposes `xe` - **External Probes** (Pro): runs availability checks assigned to this agent from the Pulse server and reports the results back — see below - **Auto-Update**: Automatically updates when a new version is released - **Multi-Platform**: Linux, macOS, Windows support The opt-in Linux typed-helper collector profile keeps auto-update enabled without making its root-owned binary writable by the service account. The collector downloads and self-tests the signed artifact in a fixed collector-owned quarantine; the no-network helper revalidates it, promotes it to root-only staging, atomically activates it, and owns identity-bound rollback if restart fails. There is no fallback to direct unprivileged replacement. This transaction is covered by unit, race, release-build, and installer contract tests; live-host restart/health and rollback qualification remains required before the profile becomes the general default. See [Agent Security](AGENT_SECURITY.md#least-privilege-agent-profile). On Linux systemd, the safe monitoring profile and remediation lifecycle are separate install choices. `--least-privilege --enable-privileged-helper` selects the opt-in unprivileged collector and no-network helper. Adding `--enable-action-runner --action-token-file ` installs the root-owned runner with a separately issued, host-bound action credential. `--disable-action-runner` and `--uninstall-action-runner` remove remediation without removing the collector. The runner accepts only the documented typed host, Proxmox guest, and container operations; shell, generic exec, unrestricted file reads, and deploy requests remain forbidden. Agent Doctor shows the reported collector service user, command-authority ceiling, collector credential scope, and typed-helper configuration separately from the tenant- and host-bound action-runner credential and connected-session state. It does not treat a healthy collector as proof that actions are ready. For an eligible Linux systemd host, enrollment returns one credential reveal kept only in the open page's memory. Start the generated `/dev/tty` prompt, copy and paste the secret into that prompt to write the private root-owned token file, and then run the installer command, which carries only `--action-token-file ` in its arguments. Re-enrollment atomically replaces the prior credential for the same tenant and canonical agent ID; a failed persistence write retains the previous credential, while a successful rotation invalidates it. Use `--safe-profile-inspect` to report the current profile and calculated differences without changing the host. `--safe-profile-apply` performs the explicit collector/helper migration and retains a rollback snapshot; `--safe-profile-rollback` restores it. These commands are proven only for Linux systemd and fail closed elsewhere. Appliance, non-systemd, Windows, and macOS installs remain explicit legacy/full-trust profiles until their own runtime and migration boundaries are qualified. Inspection includes unit identity/groups, ambient capabilities, binary ownership/mode, provider flags, helper/command state, independent runner presence, and the expected telemetry/action degradation. Apply preserves the collector token and agent identity, commits only after local health, helper socket, and server registration checks, and restores the exact prior collector/helper profile on failure. It never changes a separately installed runner, and ordinary `--update` never implies a profile migration. The secure runtime separation remains a proposed hardening lane; the product-wide default does not change until representative real Linux hosts pass migration, activation, rollback, provider-parity, and action-session qualification. On Linux, the host module automatically checks for `virsh`. When the agent can open the default libvirt connection read-only, defined domains appear as VM workloads under that host. Collection uses libvirt's bounded bulk statistics interface and does not grant Pulse VM start, stop, console, or configuration authority. If `virsh` is absent, the socket is inaccessible, or the driver does not support the requested statistics, normal host reporting continues without libvirt inventory. Appliance packaging can still differ. In particular, a QNAP installation must make its Container Station libvirt client/socket available to the agent service; the presence of KVM processes alone is not enough to establish a readable libvirt connection. On XCP-ng, the host module automatically checks for the local `xe` CLI. It uses only bounded `pool-list`, `host-list`, and `vm-list` queries: no XAPI credentials or VM lifecycle authority are added. The XCP-ng pool becomes the host's cluster grouping, and pool-wide VMs are de-duplicated by UUID and parented to the resident Pulse host when that node also reports. If several pool nodes run the Unified Agent, their identical pool views coalesce rather than creating duplicate workloads. A failed `xe` query preserves the last successful inventory while normal host metrics continue to report. This local integration covers one XCP-ng pool. Multi-pool deployments that need a central Xen Orchestra connection remain a separate integration surface. ### Windows CPU and motherboard temperatures Windows does not expose dependable built-in CPU or motherboard temperature readings. The Unified Agent can import those readings from [LibreHardwareMonitor](https://github.com/LibreHardwareMonitor/LibreHardwareMonitor), which supplies the required driver-backed hardware access: 1. Run LibreHardwareMonitor as Administrator on the Windows host. 2. Keep its HTTP port at the default `8085`. 3. Select **Options → Remote Web Server → Run**. 4. Do not permit inbound network access to port `8085` in Windows Firewall. Pulse connects only to `http://127.0.0.1:8085/data.json`. LibreHardwareMonitor's web authentication must remain disabled for this loopback-only integration. Pulse uses a fixed local URL, follows no redirects, and accepts only bounded, validated CPU and motherboard Celsius readings. If LibreHardwareMonitor is stopped, unavailable, or returns unsupported data, the rest of the Windows host report continues normally. Native Windows Storage reliability counters remain the source for supported physical-disk temperatures. NVIDIA GPU telemetry continues to come directly from `nvidia-smi`; Pulse deliberately ignores LibreHardwareMonitor GPU and storage nodes to avoid duplicate or ambiguously correlated readings. ## External Probes (Pro) With the Pro `external_probe` entitlement, availability checks configured in Pulse can run from one or more connected agents. In **Settings → Monitoring → Availability checks**, select the agents under **Observation locations** in the check editor. Keep **This Pulse server** selected for a local observation too, or deselect it for agent-only checks. Each location keeps its own evidence; a failure on one path is not a universal outage. To observe a service from outside its local network, deploy the agent on a machine elsewhere — a cloud VM, a Docker host at another location — and assign checks to it. **The agent does not send notifications directly.** The Pulse server must be running and able to reach the notification destination; target failures are evaluated there through your normal alert routes. There is nothing to configure on the agent itself. Assignments arrive through the agent's signed remote configuration, the agent runs each check on its configured interval, and results are delivered with its regular reports. Results waiting for delivery use a bounded **in-memory** queue of up to 200 observations across the agent's checks. Oldest pending observations are dropped when it fills, and an agent restart loses the queue; it is not a complete outage record. If reports stop arriving, the affected location becomes indeterminate ("no recent report from probe agent"). This is missing evidence, not proof the target is down. The missing-report window is the longer of **five minutes or three check intervals**. During normal server evaluation, eligible stale probes raise one `availability_probe_unavailable` warning per disconnected probe, regardless of how many checks it owns. Pulse measures that reporting window from server receipt time rather than the agent's clock, so clock skew cannot create or conceal the disconnect. That warning uses the normal email, webhook, Apprise, and recovery-notification pipeline, subject to alert and connectivity policies. The alert identity belongs to the probe agent, so adding or removing an assigned check does not resolve and reopen it. When the host heartbeat itself is offline, Pulse keeps the existing host-offline alert as the single canonical incident and suppresses the probe-results warning, rather than producing two normal alerts for the same agent failure. If Pulse itself stops or loses outbound connectivity, the external agent cannot deliver Pulse alerts in its place. Configure an [external watchdog](TROUBLESHOOTING.md#no-alert-when-pulse-power-or-internet-goes-down) outside the shared power/network failure, with an independently reachable notification destination. Verify recipient delivery in an authorised test environment, not by interrupting a production site. Existing paired Pulse Mobile/Relay users retain their current push path until **31 March 2027**. Relay is no longer sold. Its instance-disconnect push does not evaluate individual targets while Pulse is offline, and is not a permanent substitute for the watchdog. See [Mobile retirement](RELAY.md). If the external-probe entitlement lapses, checks resume on the Pulse server, not the agent. That changes the observation location and may change the result; see the [configuration guide](CONFIGURATION.md#external-probes-pro). The module appears as `availability` in the agent's module status when at least one check is assigned. Note for ICMP (ping) checks: the probe uses the system `ping` binary. In containers or hardened service units without `CAP_NET_RAW`, ICMP checks fail; prefer TCP or HTTP checks there, or grant the capability. See "ICMP probe privileges" in docs/CONFIGURATION.md. ## Custom metrics The host module can report numeric, boolean, and timestamp metrics produced by local executables or HTTP(S) REST endpoints. This is intended for site-specific signals such as queue depth, UPS load, service status, DNS update age, or a backup timestamp that Pulse cannot collect natively. Create a private YAML file: ```yaml version: 1 sensors: - id: queue_depth name: Queue depth command: /usr/local/libexec/pulse-queue-depth unit: items interval: 1m timeout: 2s warningAbove: 20 criticalAbove: 50 alertOnError: true - id: main_dns_update name: Main DNS update group: Main server subgroup: Domain kind: timestamp url: https://metrics.example.net/dns/main interval: 1m timeout: 2s staleAfter: 10m warningAbove: 3600 criticalAbove: 7200 - id: checkout_online name: Checkout service group: Main server subgroup: Service statuses kind: boolean url: http://monitoring.internal/checkout criticalBelow: 0.5 ``` Then start or restart the agent with `--custom-sensors-file /etc/pulse/custom-sensors.yaml`, or set `PULSE_CUSTOM_SENSORS_FILE` to that absolute path. Each metric configures exactly one source: - `command`: an absolute executable path. It receives no arguments and writes one scalar to standard output. - `url`: an absolute HTTP(S) URL polled with `GET`. Redirects are not followed, non-2xx responses fail, and the response is limited to 4 KiB. REST endpoints can return a plain scalar or a JSON object: ```json {"value": 42.5, "observedAt": "2026-07-30T20:00:00Z"} ``` `value` may be a number, string, or boolean. `observedAt` is optional RFC3339 source time. When `staleAfter` is configured, older source data becomes a stale error and follows `alertOnError`. `kind` defaults to `number`. Boolean metrics accept true/false, 1/0, yes/no, on/off, up/down, and online/offline; Pulse stores true as 1 and false as 0, so `criticalBelow: 0.5` alerts when a service is offline. Timestamp metrics accept RFC3339 or Unix seconds, display time since the event, and apply thresholds to the age in seconds. Optional `group` and `subgroup` values organize labels in the **Custom Metrics** card. The agent evaluates optional `warningAbove`, `criticalAbove`, `warningBelow`, and `criticalBelow` thresholds locally. Pulse displays the typed value and unit under **Custom Metrics** and creates normal warning/critical alerts. A collection failure alerts by default; set `alertOnError: false` to make failures report-only. If a probe fails after a successful reading, the last good value is shown as stale with its original observation time. Configuration is deliberately local-only. The Pulse server and remote agent configuration cannot supply commands, URLs, or arguments. The file is limited to 32 metrics; intervals must be between 10 seconds and 24 hours; timeouts must be between 100 milliseconds and 10 seconds and shorter than the interval. `staleAfter` must be between 10 seconds and 30 days. At most four probes run concurrently, output is bounded, HTTP credentials in URLs are rejected, and each executable is revalidated before use. Standard TLS certificate validation applies to HTTPS endpoints; use network policy to constrain destinations where required. On POSIX systems, the YAML file must be a regular, non-symlink file owned by the agent service user with no group/other permissions (normally mode `0600`). Commands and their immediate parent directories must also be regular, non-symlink, owned by the service user, and not group/other writable. Commands must have an executable bit. For example: ```bash sudo chown root:root /etc/pulse/custom-sensors.yaml /usr/local/libexec/pulse-queue-depth sudo chmod 0600 /etc/pulse/custom-sensors.yaml sudo chmod 0700 /usr/local/libexec/pulse-queue-depth ``` ## Configuration | Flag | Env Var | Description | Default | |------|---------|-------------|---------| | `--url` | `PULSE_URL` | Pulse server URL | `http://localhost:7655` | | `--token` | `PULSE_TOKEN` | API token. Required when enrolling (`--enroll`); otherwise the `token` file in the state directory is used when unset | *(none)* | | `--observers-file` | `PULSE_OBSERVERS_FILE` | Private JSON file defining report-only destinations | *(none)* | | `--custom-sensors-file` | `PULSE_CUSTOM_SENSORS_FILE` | Private YAML file defining command/REST custom metrics and thresholds | *(none)* | | `--token-file` | - | Read API token from file | *(unset)* | | `--interval` | `PULSE_INTERVAL` | Reporting interval | `30s` | | `--enable-host` | `PULSE_ENABLE_HOST` | Enable host metrics | `true` | | `--enable-docker` | `PULSE_ENABLE_DOCKER` | Enable Docker / Podman metrics | `false` (auto-detect if not configured) | | `--docker-runtime` | `PULSE_DOCKER_RUNTIME` | Preferred Docker / Podman runtime: `auto`, `docker`, or `podman`. `docker` only uses an endpoint that reports Docker. `podman` is tried first, but if no Podman socket answers the agent falls back to a Docker endpoint and reports it as `docker` | `auto` | | `--enable-kubernetes` | `PULSE_ENABLE_KUBERNETES` | Enable Kubernetes metrics | `false` (installer auto-detect if not configured) | | `--enable-proxmox` | `PULSE_ENABLE_PROXMOX` | Enable Proxmox integration | `false` | | `--proxmox-type` | `PULSE_PROXMOX_TYPE` | Proxmox type: `pve` or `pbs` | *(auto-detect)* | | `--enable-commands` | `PULSE_ENABLE_COMMANDS` | Enable Pulse command execution: Docker / Podman container actions from the UI (start/stop/restart/update), Patrol actions, and Proxmox LXC Docker inventory (disabled by default) | `false` | | `--disable-commands` | `PULSE_DISABLE_COMMANDS` | **Deprecated** (commands are disabled by default) | - | | `--disk-exclude` | `PULSE_DISK_EXCLUDE` | Device name/path or mount point patterns to exclude from disk and S.M.A.R.T. monitoring (repeatable or CSV) | *(none)* | | `--disk-include` | `PULSE_DISK_INCLUDE` | Device name/path or mount point patterns to include despite automatic filesystem filtering (repeatable or CSV) | *(none)* | | `--kubeconfig` | `PULSE_KUBECONFIG` | Kubeconfig path (optional) | *(auto)* | | `--kube-context` | `PULSE_KUBE_CONTEXT` | Kubeconfig context (optional) | *(auto)* | | `--kube-include-namespace` | `PULSE_KUBE_INCLUDE_NAMESPACES` | Limit namespaces (repeatable or CSV, wildcards supported) | *(all)* | | `--kube-exclude-namespace` | `PULSE_KUBE_EXCLUDE_NAMESPACES` | Exclude namespaces (repeatable or CSV, wildcards supported) | *(none)* | | `--kube-include-all-pods` | `PULSE_KUBE_INCLUDE_ALL_PODS` | Include all non-succeeded pods | `false` | | `--kube-include-all-deployments` | `PULSE_KUBE_INCLUDE_ALL_DEPLOYMENTS` | Include all deployments, not just problems | `false` | | `--kube-max-pods` | `PULSE_KUBE_MAX_PODS` | Max pods per report | `200` | | `--disable-auto-update` | `PULSE_DISABLE_AUTO_UPDATE` | Disable auto-updates | `false` | | `--disable-docker-update-checks` | `PULSE_DISABLE_DOCKER_UPDATE_CHECKS` | Disable Docker image update detection | `false` | | `--disable-registry-credentials` | `PULSE_DISABLE_REGISTRY_CREDENTIALS` | Do not read host Docker credentials (config.json / credential helpers) for registry update checks | `false` | | `--insecure` | `PULSE_INSECURE_SKIP_VERIFY` | Skip TLS verification | `false` | | `--allow-plaintext-http` | `PULSE_AGENT_ALLOW_PLAINTEXT_HTTP` | Allow plain HTTP to a Pulse server at an address that is not local. Local-network Pulse URLs (private IP, single-label, `.local`/`.lan`/`.home`/`.home.arpa`/`.internal`, or resolves to private addresses) already work over HTTP for the primary URL without this flag. The flag extends plain HTTP to any host, including servers numbered from public IP space. Sends the API token in cleartext; only for networks you fully control. Report-only observers still need their own `allowPlaintextHTTP` opt-in | `false` | | `--hostname` | `PULSE_HOSTNAME` | Override hostname | *(OS hostname)* | | `--agent-id` | `PULSE_AGENT_ID` | Unique agent identifier | *(saved ID, then machine ID)* | | `--report-ip` | `PULSE_REPORT_IP` | Override reported IP (multi-NIC) | *(auto)* | | `--disable-ceph` | `PULSE_DISABLE_CEPH` | Disable local Ceph status polling | `false` | | `--tag` | `PULSE_TAGS` | Apply tags (repeatable or CSV) | *(none)* | | `--log-level` | `LOG_LEVEL` | Log verbosity (`debug`, `info`, `warn`, `error`) | `info` | | `--health-addr` | `PULSE_HEALTH_ADDR` | Health/metrics server address | `127.0.0.1:9191` | Use `--health-addr :9191` only when another host must scrape the health/metrics endpoint over the network. Use `--health-addr ""` or `PULSE_HEALTH_ADDR=off` to disable that listener. **Token resolution order**: `--token` → `--token-file` → `PULSE_TOKEN` → the `token` file in the state directory (`/token`; the default state directory is `/var/lib/pulse-agent` on Linux and macOS and `%ProgramData%\Pulse` on Windows). An explicitly selected token file must be readable by the agent service and contain a non-empty bare token, not quotes or a `PULSE_TOKEN=` assignment. Surrounding whitespace is ignored. A missing, unreadable or empty selected file stops startup rather than using an environment or implicit state-file token. The startup error does not print the file path or token. With `--enroll`, a previously stored non-empty `runtime.token` still takes precedence, so removing the bootstrap file does not break ordinary agent restarts. Do not share token contents, service environment contents or token hints when asking for help. ### Agent log level The default `info` level records normal lifecycle and connection messages. Use `warn` to retain warnings and errors while suppressing routine messages; log level does not change collection, reporting, alerts, or notifications. For an agent installed as a Linux systemd service, use a persistent drop-in instead of editing the generated unit (which an agent update may replace): ```bash sudo systemctl edit pulse-agent ``` Add the following content, then save and exit: ```ini [Service] Environment="LOG_LEVEL=warn" ``` Apply and verify the change: ```bash sudo systemctl daemon-reload sudo systemctl restart pulse-agent ``` These are configuration changes, not evidence-collection steps: apply them in a suitable maintenance window, not just to file a report. Inspect the existing journal with the [bounded agent log reader](#collect-agent-logs-safely). Start with the current log level; changing it to `debug` can expose more private details and a restart changes the run being investigated. Do not enable debug logging or restart the agent merely to manufacture a missing diagnostic entry. For a container agent, set `LOG_LEVEL=warn` in the container environment and recreate the container. Pro installations can also manage `log_level` with an [agent configuration profile](CENTRALIZED_MANAGEMENT.md). ## Observer destinations Observer destinations receive the same already-collected host, Docker/Podman, and Kubernetes reports. Collection runs once per interval. Delivery, retries, and persisted host-report buffers are isolated per destination, so an observer outage does not replay or block the primary stream. Observer responses cannot change configuration, execute commands, enroll the agent, or select updates. Create a separate API token on each observer and store every token in its own absolute-path file. On Unix, both the JSON file and token files must be regular, non-symlink files with no group or other permissions (for example mode `0600`). ```json { "version": 1, "observers": [ { "name": "dev", "url": "https://pulse-dev.example.test", "tokenFile": "/etc/pulse-agent/dev-observer.token", "serverFingerprint": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", "provisionProxmox": true } ] } ``` Start or install the service with `--observers-file /etc/pulse-agent/observers.json`. Plaintext remote HTTP is rejected unless that observer explicitly sets `"allowPlaintextHTTP": true`. `insecureSkipVerify` is available per observer but should be replaced with a CA file or certificate fingerprint wherever possible. When Proxmox integration is enabled, each observer gets a distinct destination-scoped PVE/PBS API token and registration-state directory. Pulse must answer the registration check before the agent creates or rotates any Proxmox token; an unavailable destination therefore leaves existing credentials unchanged. Set `"provisionProxmox": false` when an observer should receive only Unified Agent telemetry and no separately registered PVE/PBS source. Per-destination delivery status is exported on the health listener as `pulse_agent_destination_configured` and `pulse_agent_destination_delivery_up`, labelled by module, destination, and role. ### Advanced Flags - `--version`: Print the agent version and exit. - `--self-test`: Perform a self-test and exit (used during auto-update). ## Auto-Detection Auto-detection behavior: - **Host metrics**: Enabled by default. - **Docker/Podman**: Enabled automatically by the agent if Docker/Podman is detected and `PULSE_ENABLE_DOCKER` was not explicitly set. A local `--enable-docker=false` or `PULSE_ENABLE_DOCKER=false` is a hard opt-out and is not re-enabled by auto-detection or remote profile config. - **Kubernetes**: Enabled automatically by the installer when a kubeconfig is detected and none of `--enable-kubernetes`, `--disable-kubernetes` or `--kubeconfig` was passed. The Linux/macOS installer (`install.sh`) does not read `PULSE_ENABLE_KUBERNETES` from the environment; pass `--disable-kubernetes` to opt out at install time. - **Proxmox**: Enabled automatically by the installer when Proxmox is detected. Type auto-detects `pve` vs `pbs` if not specified. To disable auto-detection, explicitly set the relevant flags or env vars, for example: - `--enable-docker=false` or `PULSE_ENABLE_DOCKER=false` - `--enable-kubernetes=false` or `PULSE_ENABLE_KUBERNETES=false` (the Linux/macOS installer honours only the flag at install time) - `--enable-proxmox=false` or `PULSE_ENABLE_PROXMOX=false` for the agent binary and `install.ps1`. `install.sh` ignores the environment variable; pass `--enable-proxmox=false` or `--disable-proxmox` to it instead. ### Inside-Guest Runtime Boundaries Docker/Podman inside a VM or LXC is monitored from inside that guest. Install the Unified Agent in the guest when you want full Docker host, container, service, and task inventory on the Docker page. Pulse does not use a Proxmox node agent to look inside LXCs by default. The node agent does automatically collect filesystem capacity for running LXCs when the local `pct` tool is available. The current collector first uses socket-free cgroup v2 and `/proc` discovery plus bounded mount probes. A root or explicitly `--grant-pct` collector can fall back to `pct list` and `pct df ` (with `lxc-info` where permitted); those commands cannot recover readings inside the isolated typed helper. See [helper-backed LXC readings](AGENT_SECURITY.md#proxmox-lxc-filesystem-readings-with-the-typed-helper) for partial, unavailable and older-helper results. Do not remove `PrivateNetwork` or broaden the agent's privileges to clear a missing reading. It reports only mount keys, volume labels, mount paths, and capacity/usage values; it does not run commands inside a guest or read guest files. Stopped LXCs retain the normal API-derived disk view. The optional Proxmox-side LXC Docker hint is off unless the Pulse server is started with `PULSE_ENABLE_PROXMOX_GUEST_DOCKER_DETECTION=true`. That hint uses `pct exec` only to check whether `/var/run/docker.sock` exists in a running LXC; it does not enumerate containers, images, environment variables, files, or processes. The stronger Proxmox-side LXC Docker inventory path is separately disabled by default. An admin can turn it on with the **Discover Docker in LXC guests** toggle in Settings → System → General, or the server can be started with `PULSE_ENABLE_PROXMOX_GUEST_DOCKER_INVENTORY=true`, which locks the toggle to the environment value. Use either path only when operators are comfortable with Proxmox-side guest probing. ## Installation Options These commands use the protected token file and successfully downloaded, inspected installer from [Private-file installation](#private-file-installation-linux-macos-and-nas). Use the same HTTPS address in preparation and installation. The flags below choose collectors; they do not grant command execution. ### Simple Install (host + Docker auto-detect) ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" ``` ### Proxmox VE Node (explicit profile) ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-proxmox --proxmox-type pve ``` ### Proxmox Backup Server Node (explicit profile) ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-proxmox --proxmox-type pbs ``` ### Force Enable Docker (if auto-detection fails) ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-docker ``` ### Disable Docker (even if detected) ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-docker=false ``` ### Host + Kubernetes Monitoring ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-kubernetes ``` ### Docker Monitoring Only ```bash bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" --enable-host=false --enable-docker ``` ### Exclude Specific Disks from Monitoring ```bash # Exclude whole block devices by name or path pulse-agent --disk-exclude sda --disk-exclude /dev/sdb # Exclude specific mount points pulse-agent --disk-exclude /mnt/backup --disk-exclude /var/run/samba/fd # Exclude using patterns (prefix match) pulse-agent --disk-exclude '/mnt/pbs*' # Matches /mnt/pbs-data, /mnt/pbs-backup, etc. # Exclude using patterns (contains match) pulse-agent --disk-exclude '*pbs*' # Matches any path containing 'pbs' # Via environment variable (comma-separated) PULSE_DISK_EXCLUDE=/dev/sda,*pbs*,/var/run/samba/fd ``` **Pattern types:** - Exact: `/dev/sda`, `sda`, or `/mnt/backup` - matches that device path, device name, or mount point - Prefix: `/dev/nvme*` or `/mnt/ext*` - matches device paths or mount points with that prefix - Contains: `*cache*` or `*pbs*` - matches device paths, device names, or mount points containing that text Exclusions are applied before filesystem usage, disk I/O, and S.M.A.R.T. collection. On linked Proxmox hosts, matching physical-disk health and SSD wear alerts are also suppressed. ### Include an Automatically Filtered Filesystem Pulse normally filters pseudo-filesystems such as `tmpfs` to keep system mounts out of disk monitoring. A specific mount can be opted back in when its capacity matters, such as a log2ram volume mounted at `/var/log`. ```bash pulse-agent --disk-include /var/log # Via environment variable PULSE_DISK_INCLUDE=/var/log ``` Include patterns use the same exact, prefix, and contains matching rules as exclusions. They only override automatic filesystem filtering. An explicit `--disk-exclude` match still wins. ## S.M.A.R.T. Disk Health The agent can report S.M.A.R.T. disk temperatures, health status, identity, and normalized health counters when running in Agent mode. This requires: 1. **smartmontools** installed on the host: ```bash # Debian/Ubuntu apt install smartmontools # RHEL/CentOS yum install smartmontools # Alpine apk add smartmontools ``` 2. The agent must have permission to run `smartctl` (typically requires root) **Notes:** - Disks in standby mode are reported as such (no temperature) to avoid waking them - S.M.A.R.T. data is collected alongside other host metrics and can enrich the Physical Disks view with temperature, stable disk identity, power-on hours, SSD life, pending sectors, media errors, and related counters - If `smartctl` is not available, S.M.A.R.T. monitoring is silently skipped - **Disk exclusions** (`--disk-exclude` / `PULSE_DISK_EXCLUDE`) also apply to S.M.A.R.T. monitoring. Use patterns like `sda`, `/dev/sdb`, `nvme*`, or `*cache*` to exclude specific block devices. ## Auto-Update Eligible v6 agents automatically check the Pulse server for updates every hour. The check is asynchronous: updating the Pulse server changes the target version, but does not prove every agent is online, eligible, or already current. When a new version is available: 1. Agent downloads the new binary from the Pulse server 2. Verifies the checksum 3. Verifies the release signature when trusted update keys are embedded 4. Runs the downloaded binary with `--self-test` 5. Replaces itself atomically (with backup) 6. Restarts with the same configuration Use the manual update path for v5 agents, PVE host agents, agents with auto-update disabled, and agents blocked by authentication, missing connection state, download, trust, or self-test failures. Open an outdated-agent notice or `/settings/infrastructure?agentDoctor=1` to open **Agent Doctor** and copy the command for each reported host. Pulse does not remotely execute those commands. If an already-installed v5 `pulse-agent` follows its legacy automatic updater path instead of the supported manual installer path, the first hop is performed by the v5 updater. That hop verifies TLS by default, the SHA-256 checksum, executable magic, size limits, and atomic replacement, but the newer v6 signature and `--self-test` checks apply only after the agent has landed on v6. Use HTTPS or a trusted local network for that legacy migration. For high-assurance environments, install the v6 `pulse-agent` through the signed installer path instead of relying on a plain-HTTP first hop. To disable auto-updates, set `PULSE_DISABLE_AUTO_UPDATE=true` (or pass `--disable-auto-update`) on the `pulse-agent` process itself. The agent installer (`agent-install.sh`) does not accept `--disable-auto-update`, and setting the variable in the shell that runs the installer does not reach the installed service. For an agent installed as a Linux systemd service, add a drop-in with `sudo systemctl edit pulse-agent`: ```ini [Service] Environment="PULSE_DISABLE_AUTO_UPDATE=true" ``` Then apply it with `sudo systemctl daemon-reload` and `sudo systemctl restart pulse-agent`. ## Remote Configuration (Agent Profiles, Pro/legacy Pro+/Cloud) Pro, legacy Pro+, and Cloud can push centralized settings to agents via Agent Profiles. Behavior: - The agent fetches remote config on startup from `/api/agents/agent/{agent_id}/config`. - Profile settings override local flags/env for supported keys. - With the host module running, current agents refresh about once a minute. Host `interval`, `report_ip` and `disable_ceph` settings can apply live; other profile keys require a successful startup fetch. A refresh is not an all-module live reload, and a saved assignment is not applied-state proof. - Removing an assignment does not undo settings already applied to a running process, revoke its token or stop monitoring. Keep incident recovery separate from profile rollout; do not restart or re-enrol solely to diagnose a reading. - Command execution (`commandsEnabled`) is controlled per agent from the Infrastructure agent controls and can change live. - Remote config responses can be signed with `PULSE_AGENT_CONFIG_SIGNING_KEY` (base64 Ed25519 private key). - To require signed payloads, set `PULSE_AGENT_CONFIG_SIGNATURE_REQUIRED=true` on Pulse and agents. - If you use a custom signing key, set `PULSE_AGENT_CONFIG_PUBLIC_KEYS` on agents to trust the matching public key. See [Centralized Agent Management](CENTRALIZED_MANAGEMENT.md) for supported keys and profile setup. ## Uninstall Uninstall is a **destructive removal**, not a repair for missing readings, authentication errors or duplicate agents. It interrupts agent monitoring and can remove the saved identity, connection/runtime credentials and local agent logs, as well as binaries, services, helper/runner state and platform boot entries. Keep any needed evidence with the [bounded private log reader](#collect-agent-logs-safely) before removal; never put credential files or full service configuration in a report. Arrange independent monitoring where needed. ### Remove the whole agent Use the installer from the **same trusted Pulse instance** on the affected agent host. Download and inspect it using the preparation above; do not use GitHub's top-level `install.sh`, which removes the Pulse server instead. Uninstallation needs no new token: it uses saved connection and credential state where available. Do not reinstall or re-enrol to supply missing state. For Unix, the installer requires root. On a sudo host, run the privately prepared script by its existing path (the example assumes it was prepared in your current account): ```bash sudo bash "$HOME/.config/pulse/agent-install.sh" --uninstall ``` On an appliance without sudo, use its authorised administrative shell and the actual inspected script path. An explicit `--state-dir` authorises removal of that directory; do not add it with a guessed path to force cleanup. On Linux systemd, full removal also tears down the separately installed action runner and typed helper. If only remediation should stop, use the runner-only path below instead. Unix removal with a known Pulse URL and an existing collector credential requires authenticated server confirmation before local collector teardown. **Do not attempt removal if a collector credential remains but the saved Pulse URL is missing.** The current Unix installer refuses before stopping services or deleting local state. An older served or prepared script may not have this refusal; do not rerun uninstall to test it. Keep the saved identity, credentials and connection files; do not erase them to force a local-only removal. Recover the original connection state or provide `--url` for the **same trusted Pulse instance** to the inspected installer, without a token on the command line. A different server or a new token cannot confirm removal of the original record. Retry only after restoring that connection; keep certificate verification enabled. Credential-free Unix state can still be removed locally, but local cleanup is not proof that Pulse removed a server record or revoked every credential. For Windows, use the [privately prepared PowerShell installer](#windows-powershell-run-as-administrator) in an Administrator session, not the Unix script. Use the same prepared session, or set `$installerFile` to the already inspected private script path: ```powershell & $installerFile -Uninstall $true ``` Windows removes the service first and attempts server unregistration before local binary/state cleanup. A failed server notification can still be followed by local removal; its completion message does not prove server-side revocation. ### Remove only the separate Linux action runner For an opted-in Linux systemd runner, use the same inspected Unix installer: ```bash sudo bash "$HOME/.config/pulse/agent-install.sh" --uninstall-action-runner ``` This standalone option leaves the collector installed and running. Do not combine it with `--uninstall`, `--update` or `--enable-action-runner`. It stops and disables the runner before revoking its separate credential. If revocation cannot be confirmed, removal fails and retains the runner's local recovery material; **stopped is not revoked**. Do not delete that material or bypass TLS to turn the failure into a success. This option is not a replacement for removing legacy collector command authority; see [Agent Security](AGENT_SECURITY.md). ### Check the actual result Keep the exit status and relevant redacted error. A failure can leave a partial result: full Unix removal can have removed the collector's server record before a later runner-revocation failure, and Windows may have removed the service before cleanup fails. Inspect the affected host's service manager and the same-instance **Settings → Infrastructure** record before deciding on another action. Do not infer complete removal from a disappearing row or installer message, or assume that all saved state was erased. Retained-state warnings can reflect intentional path protections; do not force removal with `rm -rf`, widen authority or blindly repeat the operation. Keep remaining credential and identity material private for safe recovery. ## Migration Notes Use the unified installer (`install.sh`) for all new and existing deployments. ## Health Checks & Metrics The agent exposes HTTP endpoints for health checks and Prometheus metrics on `127.0.0.1:9191` by default (loopback only; see `--health-addr` in the configuration table). ### Endpoints | Endpoint | Description | |----------|-------------| | `/healthz` | Liveness probe - returns 200 if agent is running | | `/readyz` | Readiness probe - returns 200 when agents are initialized | | `/metrics` | Prometheus metrics | ### Prometheus Metrics | Metric | Type | Description | |--------|------|-------------| | `pulse_agent_info` | Gauge | Agent info with version, host_enabled, docker_enabled labels | | `pulse_agent_up` | Gauge | 1 when running, 0 when shutting down | ### Kubernetes Probes The default listener binds to `127.0.0.1`, which kubelet probes cannot reach because they connect to the pod IP. Set `PULSE_HEALTH_ADDR=:9191` (or `--health-addr :9191`) on the agent container before using these probes. That exposes `/healthz`, `/readyz` and `/metrics` on all pod interfaces, so restrict access with a NetworkPolicy. ```yaml livenessProbe: httpGet: path: /healthz port: 9191 initialDelaySeconds: 5 periodSeconds: 10 readinessProbe: httpGet: path: /readyz port: 9191 initialDelaySeconds: 5 periodSeconds: 5 ``` ### Disable Health Server Set `--health-addr=""` or `PULSE_HEALTH_ADDR=off` to disable the health/metrics server. Set `--health-addr :9191` when network Prometheus scraping is intentional. ## Troubleshooting ### Copied agent command fails before installation The copied Unix command prepares a download and, for installation, runs a `--preflight-only` check before entering a private token prompt. Passing preflight is **not installation** or accepted reporting. Read the original error to identify which step stopped; do not run the command again to find out. - **`stty: command not found`, followed by a terminal-required message:** the command's environment cannot run the tool used to prepare the private prompt. A real terminal can still produce this error. It does not establish that the token is invalid, that the target platform cannot run an agent, or that an existing agent was removed. - **A terminal-required or interrupted-input message without that error:** token entry did not complete safely. Keep it separate from an installer's later result; do not assume a missing tool from the terminal message alone. - **`Safe-profile migration is unsupported on this platform`:** this concerns the Linux safe-profile migration path. `--safe-profile-inspect` is a standalone **read-only** action, not removal; `--uninstall` is a separate, destructive action. The inspection can produce this error on an unsupported platform such as QNAP. The error alone does not prove that removal ran. Do not combine those flags or request a broader-privilege fallback. **Stop rather than bypass the private prompt.** Do not enable token echo, put a secret in command arguments or an environment assignment, weaken TLS, generate a replacement token or delete installed identity, connection files or logs. For a planned first installation where the prompt is unavailable, the [private-file installation](#private-file-installation-linux-macos-and-nas) uses a private editor and `--token-file`, with the same download trust and collector choices. It is not an instruction to repeat a failed recovery attempt. For an existing unavailable or outdated agent, retain that original failure separately and use [reporting checks](#agent-lookup-returns-http-401-or-403) or [update guidance](#auto-update). If removal was intentional, read its [state-preservation and result checks](#uninstall) before another action. A failed new command does not establish the cause of the earlier outage or automatic-update failure. When reporting a command failure, share only the original failure stage and manually redacted error. If the action is consequential, name only the flag already used (`--safe-profile-inspect` or `--uninstall`), or say **unknown**. Do not post the full copied command, a token, its file contents or service configuration, and do not repeat installation or removal to complete a report. ### Collect agent logs safely Prefer existing observations from the original failure. On a **Linux systemd agent host**, use this bounded Bash reader with an account already authorised to read that journal. Run it on the affected agent host, not the Pulse server or a different Proxmox node; use the actual service name and original time window if they differ from the example. This reads logs only: it does not contact Pulse, a container runtime or a guest agent. ```bash ( umask 077 for tool in timeout bash head mktemp wc rm cat; do command -v "$tool" >/dev/null 2>&1 || { printf 'Agent log read unavailable: required tool %s is missing; no unbounded fallback.\n' "$tool" >&2 exit 1 } done if ! agent_log_file=$(mktemp "${TMPDIR:-/tmp}/pulse-log.XXXXXX" 2>/dev/null); then printf 'Agent log read unavailable: private temporary file could not be created.\n' >&2 exit 1 fi trap 'rm -- "$agent_log_file"' EXIT if timeout --signal=KILL 8s bash -o pipefail > "$agent_log_file" 2>/dev/null <<'PULSE_LOG_READ' journalctl -u pulse-agent.service --since '15 minutes ago' --lines 200 --no-pager --output=cat 2>&1 | head -c 262145 PULSE_LOG_READ then if ! agent_log_bytes=$(wc -c < "$agent_log_file" 2>/dev/null); then printf 'Agent log read unavailable: excerpt size could not be checked.\n' >&2 exit 1 fi if [ "$agent_log_bytes" -gt 262144 ]; then printf 'Agent log read unavailable: excerpt exceeds 256 KiB; no partial excerpt shown.\n' >&2 exit 1 fi cat -- "$agent_log_file" else agent_log_status=$? printf 'Agent log read unavailable (exit %s); no partial excerpt shown.\n' "$agent_log_status" >&2 exit "$agent_log_status" fi ) ``` This requires **GNU `timeout`** and the listed local utilities, reads at most 200 records from the default 15-minute window, and limits the excerpt to **256 KiB**: a record limit alone does not bound bytes. The eight-second hard deadline kills only the read-only collection group, including the reader and byte limiter, not the agent. A failed, oversized or timed-out read is unavailable, not an empty search; partial output is withheld. Stop if a utility, service journal or existing access is unavailable; do not follow the journal indefinitely, remove the deadline or broaden permissions to collect it. A successful empty read is inconclusive: the event may be outside the window or absent at the configured log level. Do not restart, re-enrol or enable debug logging just to obtain an entry. Both streams are buffered in an owner-only temporary file, capped at 256 KiB plus one overflow-detection byte and removed when the subshell exits. If the shell is forcibly interrupted, a private `pulse-log.*` file may remain in its temporary directory; remove that exact file locally, not saved agent state. The excerpt is **not sanitised**. Review it locally before searching for the relevant error or sharing a manually redacted timestamp and failure stage. Tokens, cookies, URLs, private names or addresses can appear in errors; do not post the full excerpt, service environment, connection files or saved identity. A log message does not prove fresh readings, successful updates or delivery. For a container agent or a non-systemd host, use the existing deployment's log source instead; this journal recipe does not apply. Keep its read bounded and private, and report evidence as unavailable if safe collection is not possible. QNAP and Unraid log locations are described in the [space troubleshooting section](#installer-fails-with-not-enough-free-disk-space). ### Agent lookup returns HTTP 401 or 403 At `/api/agents/agent/lookup`, Pulse authenticates the request **before** searching for the host. **HTTP 401** is an authentication rejection; **HTTP 403** with `missing_scope` means the authenticated token lacks `agent:report`. An authenticated lookup can return **HTTP 404** when no matching host exists. A 401 alone does not identify a mistyped, expired or revoked token, the wrong Pulse instance, or an upstream authentication layer. The silent token prompt only keeps the secret out of the copied command and shell history; it does not change reporting permissions. The normal **Settings → Infrastructure → Install on a host** flow issues a monitoring token. It does not use `--enroll`; the separate cluster deployment flow exchanges a bootstrap token for a runtime token. Do not substitute a bootstrap token or assume an enrolment-specific correction applies to a normal host installation. Use the [bounded agent log reader](#collect-agent-logs-safely) on the affected agent host, or inspect an already retained excerpt privately. Distinguish: - `remote_config_client` reporting a lookup failure: this concerns configuration retrieval and does not establish whether ordinary host reports were accepted. - **Pulse rejected this agent's API token** from host reporting: keep its HTTP status and redacted error, separately from the lookup failure. If both lookup and host reporting return 401, this is not a lookup-only failure. An older host log's restore/upgrade explanation is generic, not an established cause for that installation. If you already tried reinstalling, do not repeat it to collect evidence; check the intended token record below first. A 401 alone does not require a replacement credential, and a 403 is not a reason to disable host monitoring or enable command execution. Ordinary reporting can recover with the same credential after the endpoint's access is restored. A running local service or successful-looking installer output does not prove that Pulse accepted a fresh report. No host-report error in the selected log window is inconclusive, not proof of successful reporting. Do not restart, re-enrol, delete identity or enable debug logging to manufacture that evidence. In your signed-in administrator session on the **same Pulse instance** the agent targets, open **Settings → API Access** in the Security group. Check whether the intended token is listed, its **Created** time, **Last used** value and **Scopes**, without revealing or copying its secret. A missing record and a listed record with `Never` are different observations; neither proves that the service is presenting the same token. Last-used metadata is not a token-use audit and does not identify which consumer authenticated. These observations do not prove that the running service is using that record. **If the listed record has only Docker reporting:** `docker:report` permits Docker/Podman telemetry, not the host module's lookup or OS, CPU and disk reports. A normal host needs **Agent reporting** (`agent:report`) and **Agent config fetch** (`agent:config:read`). Command execution and administrator scope are not required. Once authentication succeeds, a missing reporting scope returns 403 with `missing_scope`; Docker-only scope does not explain an earlier 401 or prove which credential the service presented. The **Infrastructure → Install on a host → Generate token** flow includes both host scopes automatically. **API Access → Create token** allows a manually selected scope set. If the intended host record is Docker-only, retain which screen generated it before choosing a repair; do not generate another token, broaden its permissions, reinstall or probe the agent just to distinguish the two flows. Inventory presence and `Never` alone establish neither successful credential hand-off nor accepted reporting. **If the intended record is absent:** separate server registration from the agent's credential hand-off before another installation. On that same instance, open **Settings → Infrastructure → Install on a host**, give one new install token a distinct name and choose **Generate token**. Leave command execution disabled for a monitoring-only host. Before running any install command, reopen **Settings → API Access** and refresh the page: does that named record appear and remain? Generating the record alone does not change the Pi or any other agent's saved credential. If the new record is missing or disappears, stop there; do not generate more tokens, rerun the installer or restart Pulse to test it. Report only whether generation showed an error and whether the named record appeared after the page refresh, with any error manually redacted. If it remains, that establishes current registration, not persistence across a server restart, successful credential hand-off or accepted host reports. Retain the existing installation while those remaining distinctions are investigated. A listed record with `Never` is not, by itself, a reason to mint a replacement. #### Token-file format is not shell syntax If the intended record exists with the host scopes but lookup and reporting still return 401, keep credential selection separate from file contents. `--token-file` names a file; it does not establish which credential reached Pulse. An environment or explicit-token override is a separate possibility, not a diagnosis from `Never`. The agent trims whitespace at the beginning and end of a token file. It does **not** interpret shell syntax, strip quotes or extract a value from an assignment. These file forms therefore mean different things: | File form | What the agent reads | | --- | --- | | Bare token, possibly followed by a newline | The token value; this does not prove it matches the intended record or Pulse instance. | | Token surrounded by single or double quotes | The quotes remain part of the credential. | | Assignment prefix such as `PULSE_TOKEN=` or `export` | The prefix remains part of the credential; this is not an environment-file parser. | | Duplicated or missing characters, or whitespace inside a bare value | These change the credential; trimming the file's ends does not repair them. | | Unknown | The format and cause remain unknown; no new installation is needed to complete a report. | Quotes or a prefix can cause rejection even when the intended record has the right scopes. A rejected credential can leave that record showing `Never`; this does not identify the cause of a particular installation's failure. **Bare does not mean intact.** For a token from the current **Infrastructure → Install on a host** generator, privately check the 64-character lowercase hexadecimal shape described in [private-file installation](#private-file-installation-linux-macos-and-nas). A matching token hint confirms only the visible ends, not the whole token. The inventory's **Hint** shows only the first six and last four characters: duplicated or changed middle characters can leave both ends matching. Pulse authenticates the whole token: duplicated or missing characters in the middle can still cause rejection. Finding such a difference does not establish which step introduced the error. Even an intact shape and matching hint do not prove full equality, the intended Pulse instance or accepted reporting. Legacy token formats remain supported; do not replace a token merely because its length or format differs from a newly generated one. Do not apply this shape check to other token families or rotate a working credential because it looks different. Use how the file was saved or an existing private observation. If you cannot safely tell, say **unknown**. If asked, report only **bare**, **quoted**, **prefixed** or **unknown**, not the value, length, hint, screenshot or file. Do not print, source, upload or edit the installed token file, restart the service, generate a replacement or reinstall merely to classify it. A bare value still leaves credential selection, server record and instance matching to investigate; it is not proof of accepted reporting. If asked about the inventory, report only whether the record exists and the relevant metadata; do not share the token or its hint. Keep service environments, token files and connection files private; do not paste a credential into a command, URL or issue. Do not grant administrator scope, change Proxmox permissions or disable authentication/TLS to test a guess. If reporting this problem, say whether the existing error is lookup-only or also affects host reporting, and share only the failure stage, HTTP status and manually redacted error line. Keep the installation and saved credentials unchanged until the failing stage is understood; a 401 is not itself a reason to reinstall or rotate every token. ### pfSense service disabled after a major upgrade Pulse installs two service files on pfSense: the FreeBSD rc.d service at `/usr/local/etc/rc.d/pulse-agent` and an executable `/usr/local/etc/rc.d/pulse_agent.sh` boot wrapper. pfSense only runs custom scripts from that directory at boot when their filename ends in `.sh`. If both files still exist after a pfSense upgrade, run the following as root to restore the enable flag, permissions and running service: ```sh sysrc pulse_agent_enable=YES chmod 755 /usr/local/etc/rc.d/pulse-agent /usr/local/etc/rc.d/pulse_agent.sh service pulse-agent start service pulse-agent status ``` If either file is missing, copy the current command from **Settings → Infrastructure → Install on a host** and run it again on the same firewall. Do not uninstall the existing agent first. The installer repairs the rc.d service, enable flag and pfSense boot wrapper in place and reuses saved connection and agent identity state when it is still present. ### Installer Fails With "Not enough free disk space" The installer checks space for both the staged download and installed binary before downloading. If both paths share a filesystem, it requires room for both copies there; moving staging cannot fix a full installation filesystem. Keep the paths and available/required sizes from the error, rather than assuming that every NAS installs to `/usr/local/bin`. On **QNAP QTS/QuTS hero**, the current installer selects a writable data volume and installs under `/.pulse-agent`. When `TMPDIR` is unset, it also tries to stage under that directory, avoiding the small RAM-backed root. An existing `TMPDIR` overrides this staging default. If no writable data volume is available, stop and check the volume's mount, health and access locally; do not force installation onto the root filesystem or change shared-folder permissions. Only when the error identifies staging as the constrained path, choose an existing writable data volume with enough space. This example uses `/share/CACHEDEV1_DATA`; replace it with the actual mounted volume. It creates a new private staging directory and runs the already downloaded and inspected installer from the private-file preparation above, keeping the same token file and collector options: ```bash ( set -eu umask 077 data_volume=/share/CACHEDEV1_DATA if [ ! -d "$data_volume" ] || [ ! -w "$data_volume" ]; then printf 'Selected data volume is unavailable or not writable; stop.\n' >&2 exit 1 fi staging_dir="$(mktemp -d "$data_volume/pulse-agent-stage.XXXXXX")" # Remove only this new directory, and only if the installer left it empty. trap 'rmdir "$staging_dir" 2>/dev/null || true' EXIT TMPDIR="$staging_dir" bash "$HOME/.config/pulse/agent-install.sh" \ --url https://pulse.example.com \ --token-file "$HOME/.config/pulse/agent-token" ) ``` Add the same collector options used for this agent, if any. `TMPDIR` affects this invocation only; it does not move the installed binary, saved identity or logs. If preparation or the installer fails, stop rather than loop, uninstall or delete state. Unexpected files left in the staging directory are preserved for local inspection, not recursively removed. On QNAP the agent's rotating log is written to the data volume (`/.pulse-agent/logs/pulse-agent.log`); on Unraid it is written to `/var/log/pulse-agent/pulse-agent.log` with size-capped rotation. If an older `/var/log/pulse-agent.log` is large, check whether it is still growing and which installed agent writes it. Preserve a bounded redacted excerpt and, when needed, a private copy on a healthy data volume before planned log maintenance. Do not delete or truncate a live log, remove `.pulse-agent` or re-enrol an agent as a diagnostic fix. A newer installer does not establish that high CPU or an existing space problem is resolved; verify fresh reporting, free space and log growth after any planned repair. Keep log contents, connection files and tokens private. ### Agent Not Updating - Inspect existing errors with the [bounded agent log reader](#collect-agent-logs-safely); a missing update entry is not proof that an update succeeded. - Verify network connectivity to Pulse server - Ensure auto-update is not disabled - Confirm the agent can authenticate and that its saved connection state still identifies the Pulse URL and token. - Open **Agent Doctor** from an outdated-agent notice or `/settings/infrastructure?agentDoctor=1` and use the command for that reported host. Do not substitute the public GitHub server installer. - Administrators can query the read-only Agent Fleet Doctor endpoint, `GET /api/agents/diagnostics`, for liveness, version, profile, telemetry, and identity evidence. The endpoint reports repair handoffs but does not run them. ### Duplicate Agents Cloning an installed agent can copy its saved identity and credentials. Different hostnames, MAC addresses or IPs do not make those agents distinct in Pulse. Identity selection is: 1. An explicit `--agent-id` argument (which overrides `PULSE_AGENT_ID`). 2. `PULSE_AGENT_ID` when no explicit argument is supplied. 3. The saved `agent-id` file, when no override is set. 4. The machine-derived ID, with hostname as a fallback, when no saved ID exists. First compare the affected hosts in **Agent Doctor**. Inspect the identity file locally: the default Linux path is `/var/lib/pulse-agent/agent-id`; `--state-dir` changes that directory and `--agent-id-file` or `PULSE_AGENT_ID_FILE` can select another file. Do not upload `connection.env`, token files or whole service and environment dumps; older installations can contain credentials there. **Do not delete `/etc/machine-id`, agent state or Pulse history to repair a duplicate.** A saved agent ID survives an OS machine-ID change, and changing the OS identity affects more than Pulse. Prepare OS identities when provisioning a clone, using the distribution's instructions, not during live Pulse diagnosis. For a confirmed duplicate, keep the original host unchanged and choose one stable, unique override for the clone. Persist it in the configuration that actually starts that agent, rather than launching a second agent process: - **Systemd, without an existing `--agent-id` argument**: use `sudo systemctl edit pulse-agent.service` and add the following drop-in, replacing the example ID with the clone's unique ID: ```ini [Service] Environment="PULSE_AGENT_ID=vm-clone-02" ``` Then restart only that agent with `sudo systemctl restart pulse-agent.service`. If its managed launch command already supplies `--agent-id`, change that existing override instead: the argument wins over the environment variable. - **Docker Compose**: set `PULSE_AGENT_ID` for the clone's agent service, keeping its existing volumes, collector options and token-file configuration. An existing `--agent-id` in `command` takes precedence; update it instead if set. Keep the override across restarts and upgrades. After restart, verify that both hosts report fresh, distinct IDs in Agent Doctor and that each row's metrics belong to the correct host. Changing an ID does not split or recover historical data already combined under the old ID. If authentication or reporting fails, keep the bounded, redacted error for diagnosis; do not delete tokens, re-enrol blindly or remove the original host's record. ### Permission Denied (Docker) Check the account that actually runs the agent, not your interactive `$USER`. For a systemd installation, these read-only checks show the service account and the default socket's ownership; they do not print service environments or tokens: ```bash systemctl show pulse-agent.service --property=User --property=Group --property=SupplementaryGroups ls -l /var/run/docker.sock ``` A blank systemd `User` means the service runs as root. Adding your login user to the `docker` group does not change a different agent service account. Access to a rootful Docker socket, including membership of its `docker` group, is root-equivalent; do not make the socket world-writable or expose the daemon over unauthenticated TCP to clear an error. Use the [supported collector profile](AGENT_SECURITY.md#safe-profile-support-and-qualification-matrix) for that service account. The least-privilege profile intentionally limits a rootful runtime to typed-helper summary inventory; full collection needs a collector-owned rootless socket or an explicitly chosen full-telemetry profile. Keep that distinction when diagnosing missing statistics, Swarm data or actions: summary-only collection is not a socket-permission defect. ### Docker visible in one LXC but missing in another Proxmox-side discovery uses the **owning node's** connected command-capable agent, not the agent on whichever node hosts Pulse. Check **Discover Docker in LXC guests** in Settings → System → General and any configured `PULSE_PROXMOX_GUEST_DOCKER_INVENTORY_VMIDS` allowlist. Collection skips stopped guests and guests already linked to an online guest-local agent. In the latter case, check that guest agent's Docker monitoring instead. If the owning node says **Remote control blocked**, first check its [command channel](#commands-enabled-but-remote-control-blocked). A successful root `pct exec` check does not establish that Pulse can reach that node's agent; do not repeat the guest probe when those results are already known. The host-side path requires the guest's `docker` executable and `/var/run/docker.sock` in the root `pct exec` context. A working Docker CLI in a user's login session does not prove that this context can reach the daemon; rootless or non-default sockets may need guest-local agent monitoring. Different Debian or Docker versions alone do not establish the cause. If the Proxmox node and guest are healthy, run this **once** on the owning node as root, replacing `123` with the affected LXC's VMID: ```bash timeout --kill-after=2s 15s pct exec 123 -- sh -c ' command -v docker || exit if test -S /var/run/docker.sock; then printf "default_socket=present\n" else printf "default_socket=absent\n" fi docker version --format "server={{.Server.Version}}" || exit docker ps -a --format "{{.State}}" ' ``` This checks guest entry, the default socket, daemon access and container listing without printing container names, images, environment values or credentials. `default_socket=absent` explains why the host-side probe skips that guest even if another Docker context works. A CLI/daemon error is different from a successful empty list. If these checks pass but Pulse still omits the guest, keep the bounded error from the existing collection logs and whether the working and missing guests share the same PVE node; do not enable debug logging for the whole server just to collect this distinction. Exit `124` means the check timed out; stop rather than loop or restart the host. Review output before posting and redact any private endpoint in an error. Do not change LXC privilege, `keyctl`, socket permissions or Docker versions merely to test a guess. ### Commands enabled but remote control blocked **Remote control blocked** means the agent reports commands enabled, but Pulse has no admitted command channel connected for it. Agent Doctor's `commands command-capable · credential grants exec` line describes the reported local command ceiling and credential scope, not a connected session. A fresh monitoring report or **Automatic updates ready** also does not prove that this separate WebSocket channel is working. Changing Proxmox API permissions cannot repair a Pulse command-channel connection. Before treating this as a connection failure, read the existing **Privilege** and **Action runner** lines. `root` is the collector's OS user; `commands legacy` is a compatibility profile, not proof that commands are enabled. A `monitoring credential` cannot authenticate the legacy command channel. `Credential not issued` or `Runner not connected` concerns the separate typed runner, not collector health; an absent runner alone does not exclude an authorised legacy command connection. Use the [workload-update authority guidance](DOCKER.md#command-agent-is-not-connected) to distinguish these cases without widening a token or converting the collector. Keep an intentionally monitoring-only installation unchanged. Choosing an execution-capable installation is a separate trust decision, not a connection test; remember whether that choice was made, or say it is unknown. For a systemd agent, inspect the recent journal **locally on the affected node** with the [bounded agent log reader](#collect-agent-logs-safely) only when an authorised command connection was intended and its failure stage is still unknown. Do not collect more logs just to confirm the known monitoring scope. Look for **Connected and registered with Pulse command server**, or **WebSocket connection failed repeatedly, reconnecting** and its error. A `dial websocket` error is a connection/handshake failure; `registration failed` means registration was attempted but not accepted. Scope alone does not prove that the credential's host/agent binding was admitted. No matching entry is inconclusive: initial retries may be debug-only; do not restart the host or enable server-wide debug logging to manufacture an error. If reporting the problem, share only the failure stage and redacted error reason (or say no command-channel entry is visible), not the full journal or service configuration. Omit tokens, cookies, URLs, hostnames, addresses and agent/token IDs. Keep saved identity and credentials intact while distinguishing connection failures from admission failures; do not delete state, loosen TLS verification or broaden permissions to force a connection. Guest-local Docker monitoring with commands disabled remains an alternative to host-side discovery. ### Check Status ```bash # Linux systemctl status pulse-agent # macOS launchctl list | grep pulse ``` ### Docker Swarm Not Detected If your Docker Swarm cluster isn't being detected: 1. **Check runtime detection**: Pulse disables Swarm for Podman. Look locally for "Podman runtime detected" in the existing excerpt from the [bounded agent log reader](#collect-agent-logs-safely). An empty search does not establish that Docker was detected or that Swarm collection succeeded. 2. **Force Docker runtime**: If auto-detection is incorrect: ```bash --docker-runtime docker # Or set environment variable PULSE_DOCKER_RUNTIME=docker ``` 3. **Check Docker info**: Verify Swarm is active on the host: ```bash docker info | grep -i swarm # Should show "Swarm: active" ``` 4. **Check the service account and collector profile**: follow [Permission Denied (Docker)](#permission-denied-docker). A rootful typed-helper summary does not include Swarm inventory; granting broader socket access is a security decision, not a routine permission repair. 5. **Keep the original failure evidence**: retain the failure stage and a reviewed, redacted error, or say that no relevant entry is available. Do not enable debug logging, restart the agent or change its runtime merely to obtain a log entry. Use the existing observations before deciding that any configuration repair is needed. ### PVE Backups Not Showing (Recovery) Pulse's **server collects PVE backup inventory through the saved Proxmox API connection**, including storage and storage-content requests. A working host agent or successful `--enable-proxmox` setup does not prove that this separate API collection is current. Guest-agent disk readings are not backup inventory. 1. **Check existing evidence in Proxmox first.** In the native PVE interface, identify the existing archive's node, storage, guest type/ID and time. Do not run another backup or restore to create evidence. A missing Pulse row is not proof that the archive is absent, and an OK task does not prove guest thaw. 2. **Compare the same source in Pulse.** Open **Proxmox → Backups → By date** and check the selected connection and filters against that archive. Coverage posture is a different reading: expand its explanation rather than treating an unprotected badge as a missing backup. Direct PBS collection and backups seen through PVE are distinct sources; see [PBS data sources](PBS.md#data-source-indicator) and [Recovery](RECOVERY.md#missing-or-inconsistent-evidence). 3. **Inspect ordinary collection, not a new probe.** In **Settings → Infrastructure**, check the affected saved PVE connection and existing status. Keep connection/agent liveness separate from backup collection freshness; use existing collection timestamps where available, and retain the original redacted error and time. The server checks online nodes and queryable storage configured for backup content. A failed, unavailable or partial read is not an empty inventory; a successful connection test is not proof of sustained collection. 4. **Repair access only when the evidence supports it.** Ask the Proxmox administrator to compare the actual rejected endpoint and installed PVE version with the configured service user and token's effective storage permissions. With privilege separation, both user and token ACLs must allow the request, including the relevant scope and inheritance. Audit-only inventory access does not establish access to every backup-content request. Do not apply blanket storage-administrator ACLs merely because a table is empty, disable privilege separation or use an administrator token as a test. See [Proxmox permission checks](TROUBLESHOOTING.md#check-permissions-proxmox). **Do not delete the monitored node, remove registration state or rerun setup as a diagnostic.** Setup can rotate an existing API token and change permissions; it is not a harmless refresh. Preserve the connection, credentials, registration state and backup/history data. Make any evidenced configuration repair through normal maintenance outside backups, then observe ordinary polling without manual guest-agent probes, restarts or repeated **Test Connection** requests. For freeze-enabled backups, follow the [backup safety precaution](VM_DISK_MONITORING.md#backup-safety). It is not an incident recovery procedure or proof of a repaired thaw failure. If reporting a continuing mismatch, retain the running server version, source, original time, relevant missing privilege or redacted error and the differing native/Pulse observations. Keep token secrets, full ACL listings and private infrastructure details out of public reports; follow [Getting Help](TROUBLESHOOTING.md#-getting-help).