Removed-agent records kept no platform identity, so the fleet doctor
emitted an empty platform for removed rows and the Agent Doctor
uninstall handoff could never tighten to a single platform command.
Host and Docker removal now capture the agent's last-known reported
platform onto the removed record, and the fleet diagnostic resolves the
retained value through the same strict normalization as live subjects,
so /api/agents/diagnostics reports a platform for removed agents and
the handoff shipped in 4fc86ec18 scopes to one command automatically.
A retained value that does not normalize to a known platform still
yields an empty diagnostic platform rather than a guess, and removed
Kubernetes clusters retain nothing because the cluster report never
carries a platform. The field is additive and optional, so removed
lists serialized before it existed load unchanged. Monitoring contract
updated; proofs staged in agent_fleet_doctor_test.go and
monitor_host_agents_test.go.
Wiping the agent state dir regenerates the agent ID and a fresh install
command mints a fresh token, so Docker host identity resolution correctly
refuses to adopt the old record (a foreign token must never take over a
live host) and creates a new one. Nothing ever reaped the old record, so
it sat in state forever with a stale agent version, stale containers and
stale image digests. That is the source of the outdated-agent banner that
contradicts the update dialog in #1586, the doubled container rows in
#1564, and plausibly the digest-mismatch preflight refusal there too.
Reap a matching record only when it stopped reporting before the
superseding token was minted, the same explicit-reenroll-intent rule the
removal block uses (#1581). A record that is still reporting keeps
advancing LastSeen and is never touched, and supersession sets no
resurrection block.
Refs #1586#1564
PULSE_ALLOW_CONTRACT_NEUTRAL_COMMIT: docker host state bug fix; live Codex agent WIP holds the contract docs
Cloned VMs that keep the same /etc/machine-id collapse into a single
Docker host in Pulse, with each clone's report silently overwriting the
other (#1584). The server now tracks per-identity hostname and machine
ID observations and flags a conflict when a value flaps back to one
already seen inside a 15-minute window, a signature a one-time rename
never produces. The conflict rides the DockerHost model through unified
resources, and the Docker page shows a warning naming the flapping
hostnames with the machine-id remedy. The warning self-clears once one
clone stops reporting for the window.
A full monitor reload tears the state down and rebuilds it empty, so
agent-reported host rows vanish until each agent's next report lands.
Config fetches in that window 404ed with a perfectly valid token, which
showed up as rare correlated agent_config_fetch failures across
unrelated agents (#1570).
When the live snapshot has no match, resolve the host from the
persisted continuity store with the same semantics as the live path. A
report-scoped token resolves only its bound host and manage-scoped
tokens resolve by host ID. Deliberately removed hosts stay 404 because
removal deletes their continuity entry.
Deleting a host writes a machine-id-keyed removal block that rejected
every future report with HTTP 400, and the error pointed at an Allow
reconnect control that is not wired into the UI, leaving the machine
permanently unable to enroll without changing its machine-id (#1581).
Three holes made the block effectively immortal:
- The 24h TTL sweep only iterated the in-memory removal maps, which
reset on every restart, so persisted blocks never expired. Sweep the
persisted entries by their own RemovedAt for host agents, Docker
hosts, and Kubernetes clusters.
- AllowHostAgentReenroll (and the Docker and Kubernetes equivalents)
bailed out when the ID was missing from the in-memory map, so even
the API escape hatch stopped clearing persisted blocks after a
restart. Check and clear the persisted store independently.
- A report presenting an API token created after the removal is
explicit re-add intent (the user generated a fresh install command),
so clear the block and accept it. A still-running old agent keeps
presenting its pre-removal token and stays blocked.
Also reword the rejection to describe the two working recovery paths
instead of the unwired Settings control.
Stamp LastSeen and rate-tracker samples with server receipt time instead
of the agent-reported timestamp, matching the host agent fix. A drifted
agent clock could otherwise mark a reporting Docker host or Kubernetes
cluster permanently stale and loop offline/recovery alerts.
Refs #1519
writeHostPhysicalDiskIOMetrics gated ALL per-disk I/O metrics on
host.Sensors.SMART being non-empty. When the agent's SMART collection
fails (smartctl not installed, LXC container can't see /sys/block),
DiskIO data from gopsutil is valid but silently discarded.
Fix: remove the SMART requirement. When no SMART match is found for a
DiskIO entry, try matching against Proxmox API physical disks by device
name (via LinkedNodeID), then fall back to hostID:device as the metric
resource ID. This matches the fallback scheme already used by
HostSMARTDiskSourceID and PhysicalDiskMetaMetricID.
Refs #1487
f62f35e24 restored the v5 used | cache | free memory split for Proxmox
nodes and guests, but standalone host agents still reported a flat
used/free pair, so the Machines page memory bar could not show the
reclaimable segment. Flagged by the Machines page v5 parity audit.
- Host agent reports cacheBytes (gopsutil Available minus Free); the
ZFS ARC adjustment recomputes free so used + cache + free still
covers the total.
- ApplyHostReport maps the field into models.Memory.Cache and clamps
inconsistent or older-agent reports so used + cache never exceeds
total.
- AgentMemoryMeta carries cache onto unified resources so the frontend
agent payload exposes it.
- Mock generic hosts split a third of non-used pages as cache, and the
node-linked host conversion now holds the invariant instead of
stacking the node's cache on top of a recomputed free.
- Contracts: monitoring, unified-resources, and storage-recovery now
document the split (also covering the f62f35e24 node/guest surface,
which landed without contract deltas).
Fixes#1341
Forward-port of the release/5.1 fix (ce1607694).
When a Pulse host-agent reports Ceph data, ApplyHostReport upserts the
cluster into state but only the Proxmox-API polling path ran
checkCephPoolStorage. Users with agent-reported Ceph (instance prefix
"agent:hostname") saved per-pool overrides under agent-prefixed IDs
that the alert manager never evaluated. The threshold appeared to
save (and showed Custom in the UI), but the polling cycle was
checking a different storage ID, so the alert silently stayed
dormant. Run checkCephPoolStorage right after the agent upsert so
the override key actually drives evaluation.
Keep desired config fingerprints as response metadata derived from the signed command and settings payload.
Use merged agent profile settings when building remote config fingerprints.
Refs #1447
Use the canonical Docker host identity for token bindings after a matched reconnect, preserve previous/current identity aliases for recreated containers, and keep rejecting token reuse by genuinely different Docker hosts.
Record the RC3 release-gate evidence for the Docker-in-LXC reconnect path.
Parse the /proc/mdstat operation keyword for mdadm arrays and propagate it through host reports, models, unified resources, monitoring views, alert metadata, and AI storage summaries.
Treat recovery and reshape as rebuild signals while silencing routine check and resync maintenance, with fallback rebuild detection only when no mdstat operation is available.
Tests cover mdstat operation parsing plus recovery, check, and resync alert behavior.
Fixes#1446
rcourtman