Keep agent-reported disks out of cluster member state (#2543)

An agent on a Proxmox cluster node reports SMART data for the node's
disks, and the registry stamps each disk with its node's Proxmox
identity so it stays discoverable in the Proxmox workspace. The
`/api/connections` member projection accepted any resource carrying that
identity, so each disk merged into its node's member row. Its health
became the node's state and its model name became a host alias.

A disk in warning (60C or more, wear, errors) made an actively reporting
node show Stale; a disk with unknown health made it show Pending. Since
6.4.5 merges members by freshness, the row flips whenever the agent
report and the Proxmox poll swap places, several times a minute on a
customer's three-node 6.4.5 cluster. Standalone nodes are unaffected
because only cluster nodes carry the cluster-member flag.

Only node resources now compose cluster members. The disk keeps its
ownership facet, risk and storage evidence.

`TestContract_ConnectionSystemMembersIgnoreOwnedPhysicalDisks` runs a
cluster node, its linked agent and a 63C NVMe disk through the real
registry. On current main it reproduces the customer's payload exactly
(aliases include `wd_black sn7100 4tb`, state `stale`); with the fix the
member stays active.

Refs #2062, which reports the same Pending/Stale symptom on a 3-node
cluster and was expected to be fixed by 54c3cdc061 in 6.4.5.
This commit is contained in:
rcourtman
2026-10-05 23:14:31 +01:00
committed by GitHub
5 changed files with 100 additions and 2 deletions
@@ -3011,7 +3011,10 @@ agent inventory, registration state, or command-channel readiness.
agent `Host.LastSeen`-backed rows on that ledger, but must not
reinterpret derived `state` (active/paused/unauthorized/unreachable/
stale/pending) as install authority or treat the probe response as
enrollment state. Metadata, link-local, multicast, and unspecified
enrollment state. A cluster member's derived state on that ledger is node
liveness only: physical disks an agent reports on the node never compose
into the member, so a disk health warning cannot read as the agent having
stopped reporting. Metadata, link-local, multicast, and unspecified
probe destinations must fail closed before any outbound dial, and
lifecycle surfaces must surface that canonical rejection instead of
retrying through lane-local probe helpers. Ledger writes still flow
@@ -3732,7 +3732,12 @@ a new API state machine, queue contract, or verification-accounting field.
payload must also carry the backend-authored cluster member collection
with node identity, endpoint, node-local status, and any linked agent
connection id so the frontend can render child node composition without
reverse-engineering it from standalone agent rows. Those member records,
reverse-engineering it from standalone agent rows. Only node records
compose that member collection: a physical disk an agent reports on a
member carries the node's Proxmox identity so it stays discoverable in the
Proxmox workspace, but it never becomes a member and never lends its
health, timestamp, or model name to the node's state or aliases, so a disk
warning cannot render an actively reporting node as Stale. Those member records,
plus any primary or attached connection row that represents the same host,
must also carry canonical host aliases when the backend knows them, so
discovery and settings surfaces can reconcile hostname-only and IP-only
@@ -6609,6 +6609,14 @@ The change is navigation presentation only. It adds no storage, backup,
snapshot or recovery surface, and it changes no route, authority or recovery
state. The Recovery settings item and its capability gate are untouched.
### Connection member composition ignores physical disks
`internal/api/connections_grouping.go` now composes Proxmox cluster members on
the `/api/connections` systems payload from node records only, so an
agent-reported physical disk that carries its node's identity no longer turns
the node row Stale. The disk's own risk, storage evidence and recovery state
are unchanged, and this adds no backup, restore, retention or recovery surface.
### Pulse Mobile pairing copy only
`internal/api/onboarding_handlers.go` changed only the human-readable pairing
+6
View File
@@ -254,6 +254,12 @@ func connectionSystemMemberFromResource(
if resource.Proxmox == nil || !resource.Proxmox.IsClusterMember {
return ConnectionSystemMember{}, "", false
}
// Disks an agent reports on a cluster node carry the node's Proxmox
// identity so they stay discoverable in the Proxmox workspace. Only the
// node itself is a member; a disk's health is not the node's liveness.
if unified.CanonicalResourceType(resource.Type) != unified.ResourceTypeAgent {
return ConnectionSystemMember{}, "", false
}
nodeName := strings.TrimSpace(resource.Proxmox.NodeName)
if nodeName == "" {
return ConnectionSystemMember{}, "", false
+76
View File
@@ -16608,6 +16608,82 @@ func TestContract_PBSHostAgentComposesIntoOwningSystem(t *testing.T) {
}
}
// An agent on a Proxmox cluster node reports SMART data for the node's disks,
// and the registry stamps each disk with the owning node's identity so it
// stays discoverable in the Proxmox workspace. A disk is not a cluster member:
// projecting it as one folded its health into the node row, so a warm NVMe
// drive rendered an actively reporting node as Stale whenever the agent's
// report was newer than the last Proxmox poll.
func TestContract_ConnectionSystemMembersIgnoreOwnedPhysicalDisks(t *testing.T) {
cfg := &config.Config{DataPath: t.TempDir()}
monitor, err := monitoring.New(cfg)
if err != nil {
t.Fatalf("monitoring.New: %v", err)
}
t.Cleanup(func() { monitor.Stop() })
polledAt := time.Now().UTC().Add(-10 * time.Second)
reportedAt := polledAt.Add(6 * time.Second)
adapter := unifiedresources.NewMonitorAdapter(nil)
adapter.PopulateFromSnapshot(models.StateSnapshot{
Nodes: []models.Node{{
ID: "nexus-aurora",
NodeIdentity: "nexus-aurora",
Name: "aurora",
Instance: "nexus",
ClusterName: "Nexus",
IsClusterMember: true,
Host: "https://192.0.2.5:8006",
Type: "node",
Status: "online",
ConnectionHealth: "healthy",
LastSeen: polledAt,
LinkedAgentID: "host-aurora",
}},
Hosts: []models.Host{{
ID: "host-aurora",
Hostname: "aurora",
Status: "online",
LastSeen: reportedAt,
LinkedNodeID: "nexus-aurora",
Sensors: models.HostSensorSummary{
SMART: []models.HostDiskSMART{{
Device: "/dev/nvme0n1",
Model: "WD_BLACK SN7100 4TB",
Serial: "SN7100-AURORA",
Type: "nvme",
Temperature: 63,
Health: "PASSED",
}},
},
}},
})
setTestUnexportedField(t, monitor, "resourceStore", monitoring.ResourceStoreInterface(adapter))
disks := adapter.GetByType(unifiedresources.ResourceTypePhysicalDisk)
if len(disks) != 1 || disks[0].Status != unifiedresources.StatusWarning ||
disks[0].Proxmox == nil || !disks[0].Proxmox.IsClusterMember {
t.Fatalf("precondition: want one warning disk owned by a cluster node, got %+v", disks)
}
systems := buildConnectionSystems([]Connection{
{ID: "pve:nexus", Type: ConnectionTypePVE, Name: "nexus", Address: "https://192.0.2.5:8006", State: ConnectionStateActive, Enabled: true},
{ID: "agent:host-aurora", Type: ConnectionTypeAgent, Name: "aurora", Address: "aurora", State: ConnectionStateActive, Enabled: true, LastSeen: &reportedAt},
}, monitor)
if len(systems) != 1 || len(systems[0].Members) != 1 {
t.Fatalf("want one cluster system with one member, got %+v", systems)
}
member := systems[0].Members[0]
if member.State != ConnectionStateActive {
t.Fatalf("member state = %q, want %q: disk health leaked into node liveness (%+v)", member.State, ConnectionStateActive, member)
}
for _, alias := range member.HostAliases {
if alias == "wd_black sn7100 4tb" {
t.Fatalf("member host aliases include the disk model: %+v", member.HostAliases)
}
}
}
func TestContract_AgentConnectionPayloadIncludesVersionFields(t *testing.T) {
conn := Connection{
ID: "agent:host-1",