Files
pulse/docs/METRICS_HISTORY.md
T
pulse-triage[bot] 04429cb42c Keep setup and metrics guidance credential-safe
Replace token arguments, an ignored API_TOKENS bootstrap and plaintext DaemonSet credentials with scoped UI-created tokens, private files and a pre-created Secret. Route hosted installation to the existing private-file flow and retain HTTP errors and encoded resource identities in metrics queries. Exercise copied recipes, failure stops and actual curl requests with synthetic credentials.

Contract-Neutral: Documentation-only private-file setup and metrics access guidance; no runtime, API, entitlement or deployment contract delta.
Change-source: pulse-maintainer
2026-10-01 13:26:10 +01:00

5.9 KiB

Metrics History (Persistent)

Pulse persists metrics history to disk so trend views and sparklines survive restarts.

Storage Location

Metrics history is stored in a SQLite database named metrics.db under the Pulse data directory:

  • systemd/LXC installs: typically /etc/pulse/metrics.db
  • Docker/Kubernetes installs: typically /data/metrics.db

Retention Model (Tiered)

Pulse keeps multiple resolutions of the same data, which allows longer history without storing raw samples forever:

  • Raw (high-resolution, short window)
  • Minute aggregates
  • Hourly aggregates
  • Daily aggregates

Default retention values (subject to change) are:

  • Raw: 2 hours
  • Minute: 24 hours
  • Hourly: 7 days
  • Daily: 90 days
  • Rollups: every 15 minutes by default, bounded so rollups still run well before raw samples expire

Advanced: Retention Tuning

Tiered retention is stored in system.json in the Pulse data directory:

  • systemd/LXC installs: typically /etc/pulse/system.json
  • Docker/Kubernetes installs: typically /data/system.json

Keys:

{
  "metricsRetentionRawHours": 2,
  "metricsRetentionMinuteHours": 24,
  "metricsRetentionHourlyDays": 7,
  "metricsRetentionDailyDays": 90
}

After changing these values, restart Pulse.

Advanced: Disk Write Tuning

Pulse keeps metrics history on disk by default. SSD-sensitive installs can move only the metrics SQLite database without moving secrets or general config:

PULSE_METRICS_DB_PATH=/dev/shm/pulse/metrics.db

Use a dedicated directory owned by the account running Pulse. Do not place the database directly in a shared parent such as /tmp/metrics.db: Pulse secures the database directory to mode 0700 and will reject a directory owned by another user. On systemd installs, PrivateTmp=true also means a path below /tmp is private to the service and will not appear in the host's /tmp.

The default systemd sandbox permits /dev/shm/pulse and Pulse can create that subdirectory itself. A dedicated mount elsewhere, such as /mnt/ramdisk, also needs an explicit writable-path grant:

sudo install -d -o pulse -g pulse -m 0700 /mnt/ramdisk/pulse
sudo systemctl edit pulse

Add this drop-in (replace pulse with the installed unit name if different):

[Service]
ReadWritePaths=/mnt/ramdisk/pulse

Then set PULSE_METRICS_DB_PATH=/mnt/ramdisk/pulse/metrics.db in /etc/pulse/.env and restart Pulse. ProtectSystem=strict deliberately keeps paths outside the unit's writable allowlist read-only even when Unix ownership would otherwise permit writes.

For Docker, mount a tmpfs at the selected directory and keep /data on a persistent volume:

services:
  pulse:
    environment:
      PULSE_METRICS_DB_PATH: /metrics-tmpfs/metrics.db
    tmpfs:
      - /metrics-tmpfs:size=512m,uid=1000,gid=1000,mode=0700

Using tmpfs makes metrics history ephemeral across restarts. It should not be used for /data, because /data also contains config, encrypted credentials, tokens, and other state that must remain durable.

The aggregation cadence can also be lengthened when an install prefers fewer, larger rollup writes over more frequent smaller writes:

PULSE_METRICS_ROLLUP_INTERVAL=30m

Values below 5 minutes are ignored. Values longer than half of the raw-retention window are capped by the metrics store so raw samples are still rolled up before retention pruning can remove them.

API Access

Pulse exposes the persistent metrics store via:

  • GET /api/metrics-store/stats
  • GET /api/metrics-store/history

These endpoints require authentication with the monitoring:read scope.

History Query Parameters

GET /api/metrics-store/history supports:

  • resourceType (required): node, vm, container, storage, dockerHost, dockerContainer
  • resourceId (required): resource identifier (for guests use instance:node:vmid)
  • metric (optional): cpu, memory, disk, etc. Omit to return all metrics for the resource.
  • range (optional): 1h, 6h, 12h, 24h, 1d, 7d, 30d, 90d (default 24h; duration strings also accepted)
  • maxPoints (optional): Downsample to a target number of points

For a one-off read, open /api/metrics-store/stats in your signed-in Pulse browser; do not extract its session cookie. For curl, prepare the private header file described in API authentication, using a token with only monitoring:read. The example requires curl 7.76 or later and runs on the Pulse host:

curl --disable --fail-with-body --connect-timeout 10 --max-time 30 \
  --header "@$HOME/.config/pulse/api-header" --get \
  --data-urlencode 'resourceType=vm' \
  --data-urlencode 'resourceId=pve1:node1:100' \
  --data-urlencode 'range=7d' --data-urlencode 'metric=cpu' \
  http://127.0.0.1:7655/api/metrics-store/history

Replace the example resource ID with the ID reported by your instance. For a remote request, use your Pulse HTTPS address and keep certificate verification enabled. Keep --disable first to ignore local curl defaults that could enable credential-bearing traces. HTTP errors, including 401, 402 and 403, return a non-zero exit; do not treat the error body as history. Never put a token in a command, URL or report, and share only the relevant redacted error rather than the whole infrastructure response.

License: Requests beyond Community's 7d floor require the paid long_term_metrics entitlement. Relay unlocks 14d, Pro and legacy Pro+ unlock 90d, and requests beyond the active tier's limit return 402 Payment Required. Aliases: guest (VM/LXC) and docker (Docker container) are accepted, but persistent store data uses the canonical types above.

Troubleshooting

  • No sparklines / empty history: confirm the instance can write to the data directory and that metrics.db exists.
  • Large disk usage: reduce polling frequency first. If you need tighter retention, adjust the tiered retention settings in system.json (advanced) and restart Pulse.