gsadmin/pulse
1
0
Fork 0
mirror of https://github.com/rcourtman/Pulse.git synced 2026-10-24 04:33:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

Let an operator split override a Proxmox node's agent link (#2741)

Browse Source 
## Problem

The resource drawer's **Split merged resource** (`POST
/api/resources/{id}/report-merge`), and `POST
/api/resources/{id}/unlink`, on a Proxmox node merged with its
pulse-agent recorded exclusions and answered "Resource split applied",
but the pair stayed merged on every rebuild, in the monitor and in the
resources API.

`ResourceRegistry.ingestRecord` joined the pair through the node↔agent
link in `resolveLinkedResource` (both sides declared, or a node-side
link the agent's hostname corroborates) before `findMatch`, and that
path never read `rr.exclusions`. The linked node had also already taken
the agent's identity in `resourceFromProxmoxNode`, so it minted the
agent's machine-derived canonical ID before the agent was ingested.

## Decision

The operator's split wins. The node↔agent link is Pulse's own inference
from endpoint addresses and hostnames, the evidence cloned machines and
reused short hostnames defeat, and report-merge exists precisely for
wrong merges. A later `POST /api/resources/{id}/link` between the split
rows joins them again.

## Change

- `nodeAgentSplit` (`proxmox_node_links.go`) decides the pair from the
operator's decisions read across the IDs each side can hold: the store
records them against the merged ID and the `SourceTargets` candidates,
while a split node holds its own ID. The latest decision across those
sets wins (registries now keep each exclusion's `CreatedAt`). An
exclusion naming only the agent's candidate against the agent's
machine-derived ID does not count, because report-merge records that
same pair when it splits another source (a Docker host) off the agent.
- Every path that joined the pair applies it: the snapshot ingest no
longer lends a split node the agent's identity (`splitProxmoxNodeLink`);
identity-pin completion refuses the agent's machine key from a
joined-era pin; `resolveLinkedResource` refuses the declared and the
one-sided link; `applyManualLinks` skips an older link between split
rows; and the presentation filter keeps split rows apart when no single
exclusion names their IDs.
- A split node names the agent only where the monitor's node record
does, so guest discovery, deploy and service discovery are not handed
the rejected agent.
- A relink of the split rows is recorded as a folded ID, so pin
succession keeps it instead of re-keying it onto the joined ID.

Builds on #2725 (now merged). This PR's own changes are 4a97d2225f, plus
two repairs after `main` moved: 5b5026fafd records a relinked row as a
`ManualLinkFold` (#2721 replaced `linkMergedIDs`), and e4519ec5bc passes
thresholds to `CoalesceForPresentation` (#2745). The rest of the branch
is merges of `main`.

## Verification

- `TestOperatorSplitOverridesProxmoxNodeAgentLink`
(`registry_merge_policy_test.go`): seven link shapes (declared both
ways, node-side only, agent-side only, unclustered and clustered, an
agent hostname in another form, an agent reporting no machine key) × six
requests (report-merge, unlink naming the node's candidate, unlink
naming the split rows, an exclusion between the two candidates, an
exclusion naming only the agent's candidate, an unrelated exclusion).
Each step runs three rebuilds of a SQLite-backed `MonitorAdapter`,
checked in the monitor listing, the broadcast coalesce, and the
resources API's registry seeded from the listing and from the snapshot.
The test also covers agent-first ingests with and without the host, a
relink (plus an agent-first rebuild that persists its pins), and the
same request again. The unrelated exclusion and the agent-only exclusion
under a machine-derived ID are the negative controls: those pairs stay
merged.
- `TestOperatorSplitFromOneAgentKeepsAnotherAgentsPin`: a node split
from a second agent still completes its own linked agent's boot-window
pin. `TestOperatorSplitHoldsWhenTheAgentStopsReportingItsMachineID`: a
split holds when the agent comes back without its machine ID.
- 15 mutation controls, one per rule. Each makes these tests fail.
- Full `./internal/unifiedresources/...`,
`./internal/api/resourceapi/...` and `./internal/monitoring/...` pass,
as does `./internal/api/` filtered to resource, link, unlink, merge and
broadcast tests. The targeted tests also pass on the pushed head, which
includes current `main` (#2719).

## Not covered (contract "What the split does not reach")

- The browser's realtime coalesce (`mergeCanonicalResourceSnapshot`)
receives no exclusions and still joins a Proxmox-only row and an
agent-only row that share a hostname. A split pair, from this change or
from an identity-match split, therefore still renders as one row in the
UI.
- The monitor's own node record keeps its link. The SMART fallback,
disk-exclude patterns, guest discovery, deploy and service discovery
still follow the agent.
- `resolveLinkedPhysicalDisk` still joins disks inside one host without
reading exclusions.
- The agents API's node link does not replace a resource split.
- Exclusions keyed by cluster- or hostname-derived IDs carry no source.
This commit is contained in:
rcourtman
2026-10-08 10:59:13 +01:00
committed by GitHub
parent 55b6e5145e 8464939716
commit 7606b72a8b
7 changed files with 862 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/release-control/v6/internal/subsystems/unified-resources.md
+121 -2
View File
@@ -7185,8 +7185,127 @@ folded into the agent by ID, a path no exclusion check sees, so the pair
stayed merged with the link row gone. A pin whose canonical ID the operator
excluded from the ID the record takes without the pin (its `chooseNewID` or
source-specific ID) now completes nothing. This covers pins already written
that way, not only new ones. Merges through an agent's own declared node link
(`resolveLinkedResource`) still ignore exclusions.
that way, not only new ones. A Proxmox node's link to an agent honours the
split as well; the next paragraphs cover it.
An operator's split overrides a Proxmox node's link to a pulse-agent. That
link is normally Pulse's own inference rather than a resource-store
decision: monitoring links a node to the one agent whose endpoint address or
hostname matches it (the agents API can also set it by hand), and the
registry joined the pair in `resolveLinkedResource` (a link both sides
declare, or a node-side link the agent's hostname corroborates) before
`findMatch` ran. The linked node had already taken the agent's identity in
`resourceFromProxmoxNode`, and with an agent reporting a machine key it
minted the agent's machine-derived canonical ID before the agent was
ingested. Neither path read exclusions, so the drawer's Split merged
resource (report-merge) on a node and its agent answered "Resource split
applied" and left the pair merged on every rebuild, in the monitor and in
the resources API. Cloned machines and reused short hostnames are where that
link is wrong, and report-merge exists for wrong merges, so the split wins.
`nodeAgentSplit` (`proxmox_node_links.go`) decides the pair: the operator
split the node off when an exclusion names one of the node's own IDs (its
source-specific ID, or the ID its identity derives without a machine key,
which a Proxmox node never reports itself) against one of the agent's (its
source-specific ID, the ID its identity derives, the ID it holds, or the ID
the joined pair derives). Unlink and report-merge name the merged ID and the
`SourceTargets` candidate IDs, and the split node can hold an ID it did not
hold while joined, so the decision is read across those sets, not one exact
pair. Where the pair merged under the agent's machine-derived ID, an
exclusion naming only the agent's candidate against that ID does not count:
report-merge records the same pair when it splits another source, such as a
Docker host, off the agent. Where it merged under the node's own ID (an agent
reporting no machine key), that exclusion splits the agent off the node.
`splitProxmoxNodeLink` keeps a split node from taking the agent's identity in
`ingestSnapshot` and remembers the agent for identity-pin completion, which
refuses the agent's machine key from a pin written while they were joined
when the node is split from the agent its own link or an agent row holding
the pin's ID names, or from the remembered agent where the pin can be that
agent's (no machine key or DMI UUID both report differs), so also while the
agent is missing from the snapshot or stops reporting its machine ID;
`resolveLinkedResource` refuses the link from either side. The registry adds
no `LinkedAgentID` to the node: guests, agent deployment and service
discovery act on a node's linked agent. The node then holds the ID its own
identity derives, with only its hostname and endpoint address, which
identity matching scores at 0.80 at most, under the 0.85 auto-merge floor.
The agent holds the ID its own identity derives once pins complete it (its
machine-derived ID when it has a machine key).
The latest decision across the two sets wins, so a link between them that
postdates the newest exclusion joins the pair again even where it names
another pair than the exclusions do, as a relink of the two rows a
report-merge left does (a registry now keeps each exclusion's `CreatedAt`);
a tie keeps the pair apart. Once a relink rejoins them, the declared link
folds one of the rows it names into the other's ID, and `applyManualLinks`
records that as a `ManualLinkFold` on the joined row (it recognises the
node's own IDs from the row's Proxmox facet, and the agent's source-specific
ID), so pin succession treats the folded ID as observed and keeps the link
instead of re-keying it onto the joined ID and handing the decision back to
the older exclusions, and report-merge of the joined row names that pair. The paths
that join rows after ingest apply the same decision: `applyManualLinks`
skips a link between a node row and an agent row split after it, and the
presentation filter (`ListForPresentation` and the broadcast) keeps such
rows apart when no single exclusion names their IDs; an exclusion naming
the two rows exactly keeps them apart there even against a newer link
across the sets, which only a listing older than the registry's rebuild can
present as two rows. It shares the
presentation limit the broadcast paragraph below names: a third same-host
row (another agent reporting no machine ID, say) that merges with one side
first carries the pair into one row. `TestOperatorSplitOverridesProxmoxNodeAgentLink`
pins report-merge, unlink naming the node's candidate or the split rows, an
exclusion between the two candidates and one naming only the agent's
candidate, for seven link shapes (declared both ways, node-side only and
agent-side only, unclustered and clustered, an agent hostname in a different
form, and an agent reporting no machine key), through three rebuilds of a
SQLite-backed monitor adapter in the monitor listing, the broadcast coalesce
and the resources API's registry seeded from the listing or the snapshot,
after agent-first ingests (with and without the host in the snapshot) taken
while the joined-era pins still stand; then a relink of the rows (also
followed by an agent-first rebuild that persists its pins) and the same
request again. An exclusion naming the merged ID
against an unrelated ID leaves the pair merged, and the split node names the
agent only where its monitor record does.
`TestOperatorSplitFromOneAgentKeepsAnotherAgentsPin` keeps a node's own
agent's pin while another agent it was split from declares it, and
`TestOperatorSplitHoldsWhenTheAgentStopsReportingItsMachineID` keeps a split
whose agent comes back without its machine ID.
What the split does not reach: the agents API's node link
(`/api/agents/agent/link`) records a monitor-side link intent, not a store
decision, so it does not override a resource split; rejoin through
`POST /api/resources/{id}/link`. The monitor's own node record keeps its
link where it has one, so what reads it still follows the agent: the
agent's SMART inventory as the node's disk fallback when the Proxmox disk
query fails, the agent's disk-exclude patterns on the node's disks, and the
node's linked agent that guest discovery, agent deployment and service
discovery act on.
Physical disks inside one host still join through
`resolveLinkedPhysicalDisk` without reading exclusions; a node split puts the
two sides' disks under different parents, so they no longer meet there.
Cluster- and hostname-derived IDs carry no source, so an exclusion recorded
against another resource that derives the node's own ID (a Docker Swarm host
reporting no machine key, named like the node in a swarm named like its
cluster) also splits that node from its agent, and one splitting the node's
candidate off its own ID while another source was merged there (a Docker
host linked into it) also splits it from a later agent reporting no machine
key, which joins under that ID. A rebuild that sees the agent in neither the
snapshot nor the registry, with no node link naming it, cannot tie an
exclusion naming the agent only by its candidate ID to the agent's key on a
joined-era pin, so until the agent reports again the node takes that key
and the agent's ID. The other way round, a node split from an agent that
reports no machine key and declares it refuses any joined-era pin while its
own linked agent is missing, keeping its own ID until that agent reports. An exclusion keyed by the
agent's machine-derived ID stops naming the agent if the agent's effective
canonical key changes (it stops reporting a machine ID and no pin restores
it), as every exclusion keyed by a canonical ID does. The browser's realtime
coalesce (`mergeCanonicalResourceSnapshot` in `resourceStateAdapters.ts`,
used by the websocket store and `useUnifiedResources`) receives no
exclusions and joins a Proxmox-only row and an agent-only row that share a
hostname unless their machine IDs or DMI UUIDs disagree, their Proxmox
facets name different clusters or provider scopes, or several Proxmox nodes
share the hostname and the node does not name the agent, so the UI still
renders such a split pair, from this change or from an identity-match split,
as one row.
The resources API seeds its registry from the monitor's listing, which
already has the link applied, so it cannot split the pair itself: an unlink
internal/unifiedresources/canonical_id_pins.go
+2 -1
View File
@@ -265,7 +265,8 @@ func (rr *ResourceRegistry) completeIdentityFromPins(source DataSource, sourceID
// The exclusion names the IDs the record holds without the pin.
if len(rr.exclusions) > 0 &&
(rr.isExcluded(pin.CanonicalID, rr.chooseNewID(resource.Type, identity, source, sourceID)) ||
rr.isExcluded(pin.CanonicalID, rr.sourceSpecificID(resource.Type, source, sourceID))) {
rr.isExcluded(pin.CanonicalID, rr.sourceSpecificID(resource.Type, source, sourceID)) ||
rr.proxmoxNodePinOfSplitAgentLocked(source, sourceID, resource, identity, pin)) {
return identity
}
if strings.TrimSpace(identity.MachineID) == "" {
internal/unifiedresources/presentation_exclusions.go
+15 -7
View File
@@ -4,17 +4,23 @@ import "time"
// presentationExclusionFilter reports whether the presentation host coalesce
// must keep two resources apart because the operator split the pair (unlink
// or report-merge). It reads a copy of the registry's exclusions, so callers
// coalesce without holding the registry lock; nil means no pair is excluded.
// or report-merge): an exclusion names the two rows, or they are a Proxmox
// node and an agent split across the IDs they can hold (nodeAgentSplit). It
// reads a copy of the registry's decisions, so callers coalesce without
// holding the registry lock; nil means no pair is excluded.
func (rr *ResourceRegistry) presentationExclusionFilter() func(left, right Resource) bool {
rr.mu.RLock()
if len(rr.exclusions) == 0 {
rr.mu.RUnlock()
return nil
}
exclusions := make(map[string]struct{}, len(rr.exclusions))
for key := range rr.exclusions {
exclusions[key] = struct{}{}
// The link index is built once per registry and never written again.
decisions := operatorPairDecisions{
exclusions: make(map[string]time.Time, len(rr.exclusions)),
linksByID: rr.linksByID,
}
for key, at := range rr.exclusions {
decisions.exclusions[key] = at
}
rr.mu.RUnlock()
@@ -24,8 +30,10 @@ func (rr *ResourceRegistry) presentationExclusionFilter() func(left, right Resou
if leftID == "" || rightID == "" {
return false
}
_, ok := exclusions[exclusionKey(leftID, rightID)]
return ok
if _, ok := decisions.exclusions[exclusionKey(leftID, rightID)]; ok {
return true
}
return rr.nodeAgentRowsSplit(decisions, &left, &right)
}
}
internal/unifiedresources/proxmox_node_links.go
+292
View File
@@ -2,11 +2,303 @@ package unifiedresources
import (
"net"
"slices"
"strings"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
)
// A Proxmox node's link to a pulse-agent is Pulse's own inference, not an
// operator decision: monitoring links a node to the one agent whose endpoint
// address or hostname matches it, and inferLinkedHostsForProxmoxNodes and
// resolveLinkedResource corroborate that link. Cloned machines and reused
// short hostnames make it wrong at times, which is what the resources API's
// unlink and report-merge are for, so an operator's split overrides the link
// as it overrides identity matching in findMatch. A later POST
// /api/resources/{id}/link between the two rows joins them again.
// nodeAgentLinkSide is one side of a Proxmox node<->agent pair: the record's
// source ID, the identity it derives IDs from and the resource ID it holds in
// the registry, if any.
type nodeAgentLinkSide struct {
sourceID string
identity ResourceIdentity
heldID string
}
// operatorPairDecisions is the operator's manual decisions a node<->agent
// split is read from: a registry's own, or the copy the presentation filter
// holds outside the registry lock.
type operatorPairDecisions struct {
exclusions map[string]time.Time
linksByID map[string][]ResourceLink
}
func (rr *ResourceRegistry) operatorPairDecisionsLocked() operatorPairDecisions {
return operatorPairDecisions{exclusions: rr.exclusions, linksByID: rr.linksByID}
}
// indexLinksByID lists each manual link under both of its IDs.
func indexLinksByID(links []ResourceLink) map[string][]ResourceLink {
if len(links) == 0 {
return nil
}
index := make(map[string][]ResourceLink, 2*len(links))
for _, link := range links {
a, b := CanonicalResourceID(link.ResourceA), CanonicalResourceID(link.ResourceB)
index[a] = append(index[a], link)
if b != a {
index[b] = append(index[b], link)
}
}
return index
}
// nodeAgentSplit reports whether the operator split the node off a
// node<->agent pair: an exclusion names one of the node's own IDs against one
// of the agent's IDs, and no link between those IDs postdates it (a tie keeps
// the pair apart, as it does for one pair in effectiveManualPairDecisions).
//
// Unlink and report-merge name the merged resource's ID and the source-specific
// candidate IDs its SourceTargets list, and a split moves the node onto an ID
// it did not hold while joined, so decisions are read across sets rather
// than one exact pair. The node's own IDs are its source-specific ID and the
// ID its identity derives without a machine key. The agent's are its
// source-specific ID, the ID its identity derives, the ID it holds, and the
// ID the joined pair derives (the agent's machine-derived ID once a node
// takes its identity). An exclusion naming only the agent's candidate against
// the joined ID does not count: report-merge records that same pair when it
// splits another source, such as a Docker host, off the agent's
// machine-derived ID.
//
// A link between the two sets that postdates the newest exclusion wins: a
// relink names the rows a split left, a different pair than the exclusions
// report-merge recorded against the merged ID, which the store's
// one-decision-per-pair rule cannot match.
func (rr *ResourceRegistry) nodeAgentSplit(decisions operatorPairDecisions, node, agent nodeAgentLinkSide) bool {
if len(decisions.exclusions) == 0 {
return false
}
nodeIDs, agentIDs := rr.nodeAgentPairIDs(node, agent)
var splitAt time.Time
split := false
for _, nodeID := range nodeIDs {
for _, agentID := range agentIDs {
if nodeID == agentID {
continue
}
if at, ok := decisions.exclusions[exclusionKey(nodeID, agentID)]; ok && (!split || at.After(splitAt)) {
split, splitAt = true, at
}
}
}
if !split {
return false
}
for _, nodeID := range nodeIDs {
for _, link := range decisions.linksByID[nodeID] {
other := CanonicalResourceID(link.ResourceB)
if other == nodeID {
other = CanonicalResourceID(link.ResourceA)
}
if other != nodeID && slices.Contains(agentIDs, other) && link.CreatedAt.After(splitAt) {
return false
}
}
}
return true
}
// nodeAgentPairIDs returns the node's own IDs and the agent's IDs a split is
// read across. A Proxmox node reports no machine ID or DMI UUID of its own:
// either one on its identity came from a linked agent or an identity pin, so
// the node's own ID is derived without them. The two IDs are then the same
// whether or not the node took the agent's identity in this ingest.
func (rr *ResourceRegistry) nodeAgentPairIDs(node, agent nodeAgentLinkSide) (nodeIDs, agentIDs []string) {
nodeIDs = rr.proxmoxNodeOwnIDs(node.sourceID, node.identity)
if agent.sourceID != "" {
agentIDs = append(agentIDs, rr.sourceSpecificID(ResourceTypeAgent, SourceAgent, agent.sourceID))
}
if agent.sourceID != "" || agent.identity.MachineID != "" || agent.identity.DMIUUID != "" {
agentIDs = append(agentIDs, rr.chooseNewID(ResourceTypeAgent, agent.identity, SourceAgent, agent.sourceID))
}
if held := CanonicalResourceID(agent.heldID); held != "" {
agentIDs = append(agentIDs, held)
}
// The joined ID counts as the agent's even where it is also one of the
// node's (an agent reporting no machine key joins under the node's own
// ID): an exclusion splitting the node's candidate off it then splits
// the node from the agent, and the equal pair is skipped.
nodeIdentity := node.identity
nodeIdentity.MachineID, nodeIdentity.DMIUUID = "", ""
agentIDs = append(agentIDs, rr.chooseNewID(ResourceTypeAgent, mergeIdentity(nodeIdentity, agent.identity), SourceProxmox, node.sourceID))
return nodeIDs, agentIDs
}
// proxmoxNodeOwnIDs returns the IDs a Proxmox node record holds without any
// agent: its source-specific ID and the ID its identity derives with the
// machine keys it cannot report itself removed.
func (rr *ResourceRegistry) proxmoxNodeOwnIDs(sourceID string, identity ResourceIdentity) []string {
identity.MachineID, identity.DMIUUID = "", ""
return []string{
rr.sourceSpecificID(ResourceTypeAgent, SourceProxmox, sourceID),
rr.chooseNewID(ResourceTypeAgent, identity, SourceProxmox, sourceID),
}
}
// proxmoxNodeAgentSplitLocked applies nodeAgentSplit to the registry's own
// decisions. Callers hold rr.mu.
func (rr *ResourceRegistry) proxmoxNodeAgentSplitLocked(node, agent nodeAgentLinkSide) bool {
return rr.nodeAgentSplit(rr.operatorPairDecisionsLocked(), node, agent)
}
// proxmoxNodePinOfSplitAgentLocked reports whether an identity pin a Proxmox
// node record would complete its machine keys from belongs to an agent the
// operator split the node from. A node reports no machine key of its own, so
// a pin's is an agent's, written while the pair was joined, and completing
// it would mint that agent's machine-derived ID and fold the node into the
// agent by ID or by identity. The pin's canonical ID and keys stand for the
// agent the node's link or an agent row holding the pin's ID names, so this
// holds while the agent is missing from the snapshot. The agent this
// snapshot's split dropped (rr.splitAgents) can be another agent than the
// pin's, so it refuses only a pin that can be its own.
func (rr *ResourceRegistry) proxmoxNodePinOfSplitAgentLocked(source DataSource, sourceID string, resource Resource, identity ResourceIdentity, pin ResourceIdentityPin) bool {
if source != SourceProxmox || resource.Proxmox == nil || len(rr.exclusions) == 0 {
return false
}
node := nodeAgentLinkSide{sourceID: sourceID, identity: identity}
pinAgent := nodeAgentLinkSide{
sourceID: normalizeSourceID(resource.Proxmox.LinkedAgentID),
identity: ResourceIdentity{MachineID: pin.MachineID, DMIUUID: pin.DMIUUID},
heldID: pin.CanonicalID,
}
if rr.proxmoxNodeAgentSplitLocked(node, pinAgent) {
return true
}
if held := agentOnlyRow(rr.resources[CanonicalResourceID(pin.CanonicalID)]); held != nil {
if heldSourceID := normalizeSourceID(held.Agent.AgentID); heldSourceID != pinAgent.sourceID {
pinAgent.sourceID = heldSourceID
if rr.proxmoxNodeAgentSplitLocked(node, pinAgent) {
return true
}
}
}
split, ok := rr.splitAgents[normalizeSourceID(sourceID)]
return ok && pinBelongsToAgent(pin, split.identity) && rr.proxmoxNodeAgentSplitLocked(node, split)
}
// pinBelongsToAgent reports whether an identity pin can be the agent's: only
// a key both report that differs shows the pin is another agent's. An agent
// that stops reporting its machine ID keeps its split that way, while another
// agent with its own key does not claim the pin.
func pinBelongsToAgent(pin ResourceIdentityPin, agent ResourceIdentity) bool {
if pinKey, agentKey := strings.TrimSpace(pin.MachineID), strings.TrimSpace(agent.MachineID); pinKey != "" && agentKey != "" {
return strings.EqualFold(pinKey, agentKey)
}
if pinKey, agentKey := strings.TrimSpace(pin.DMIUUID), strings.TrimSpace(agent.DMIUUID); pinKey != "" && agentKey != "" {
return strings.EqualFold(pinKey, agentKey)
}
return true
}
// joinedPairSideLocked reports whether id is an ID one side of a joined
// Proxmox node+agent row holds on its own, and that side's source: the
// node's own IDs (derived from the row's Proxmox facet, not its merged
// identity, whose first hostname can be the agent's) or the agent's
// source-specific ID, which an agent reporting no machine key holds when
// split.
func (rr *ResourceRegistry) joinedPairSideLocked(joined *Resource, id string) (DataSource, bool) {
if joined == nil || joined.Proxmox == nil || joined.Agent == nil || CanonicalResourceType(joined.Type) != ResourceTypeAgent {
return "", false
}
id = CanonicalResourceID(id)
if agentID := normalizeSourceID(joined.Agent.AgentID); agentID != "" && id == rr.sourceSpecificID(ResourceTypeAgent, SourceAgent, agentID) {
return SourceAgent, true
}
sourceID := normalizeSourceID(joined.Proxmox.SourceID)
if sourceID == "" {
return "", false
}
_, identity := resourceFromProxmoxNode(models.Node{
ID: sourceID,
Name: joined.Proxmox.NodeName,
NativeNameAliases: joined.Proxmox.NodeAliases,
Instance: joined.Proxmox.Instance,
ClusterName: joined.Proxmox.ClusterName,
}, nil)
return SourceProxmox, slices.Contains(rr.proxmoxNodeOwnIDs(sourceID, identity), id)
}
// nodeAgentRowsSplit reports whether two listed rows are a Proxmox node and
// an agent the operator split, for the paths that join rows after ingest:
// manual links (an older link between the rows a split left loses to a newer
// report-merge) and the presentation host coalesce (the rows can hold IDs no
// single exclusion names, a clustered node's cluster-derived ID for one).
func (rr *ResourceRegistry) nodeAgentRowsSplit(decisions operatorPairDecisions, left, right *Resource) bool {
node, agent := proxmoxNodeRow(left), agentOnlyRow(right)
if node == nil || agent == nil {
node, agent = proxmoxNodeRow(right), agentOnlyRow(left)
}
if node == nil || agent == nil {
return false
}
return rr.nodeAgentSplit(decisions,
nodeAgentLinkSide{sourceID: normalizeSourceID(node.Proxmox.SourceID), identity: node.Identity},
nodeAgentLinkSide{sourceID: normalizeSourceID(agent.Agent.AgentID), identity: agent.Identity, heldID: agent.ID},
)
}
func proxmoxNodeRow(resource *Resource) *Resource {
if resource == nil || CanonicalResourceType(resource.Type) != ResourceTypeAgent ||
resource.Proxmox == nil || resource.Agent != nil || strings.TrimSpace(resource.Proxmox.SourceID) == "" {
return nil
}
return resource
}
func agentOnlyRow(resource *Resource) *Resource {
if resource == nil || CanonicalResourceType(resource.Type) != ResourceTypeAgent ||
resource.Agent == nil || resource.Proxmox != nil || strings.TrimSpace(resource.Agent.AgentID) == "" {
return nil
}
return resource
}
// splitProxmoxNodeLink returns the node record and the linked host it takes
// identity from, dropping the host for a node the operator split from that
// host's agent. A linked node takes the agent's identity, and with it the
// agent's machine-derived canonical ID, before the agent is ingested, so
// refusing only the later merge would leave the node holding the agent's ID
// and machine key, and identity matching would join the agent to it again.
// Without the agent's identity the node keeps its own ID, hostname and
// endpoint address, which identity matching never merges on. rr.splitAgents
// remembers the agent for the pin guard, which reads the split from it where
// the node record itself does not name that agent; it is rewritten for every
// node on every snapshot, and nothing else reads it.
func (rr *ResourceRegistry) splitProxmoxNodeLink(node models.Node, host *models.Host) (models.Node, *models.Host) {
nodeID := normalizeSourceID(node.ID)
rr.mu.Lock()
defer rr.mu.Unlock()
delete(rr.splitAgents, nodeID)
if host == nil || len(rr.exclusions) == 0 {
return node, host
}
_, nodeIdentity := resourceFromProxmoxNode(node, nil)
_, hostIdentity := resourceFromHost(*host)
if !rr.proxmoxNodeAgentSplitLocked(
nodeAgentLinkSide{sourceID: nodeID, identity: nodeIdentity},
nodeAgentLinkSide{sourceID: normalizeSourceID(host.ID), identity: hostIdentity},
) {
return node, host
}
if rr.splitAgents == nil {
rr.splitAgents = make(map[string]nodeAgentLinkSide)
}
rr.splitAgents[nodeID] = nodeAgentLinkSide{sourceID: normalizeSourceID(host.ID), identity: hostIdentity}
return node, nil
}
func inferLinkedHostsForProxmoxNodes(nodes []models.Node, hostByID map[string]*models.Host) map[string]*models.Host {
if len(nodes) == 0 || len(hostByID) == 0 {
return nil
internal/unifiedresources/registry.go
+53 -9
View File
@@ -93,7 +93,9 @@ type ResourceRegistry struct {
matcher *IdentityMatcher
store ResourceStore
links []ResourceLink
exclusions map[string]struct{}
exclusions map[string]time.Time // exclusionKey -> when the operator last split the pair
linksByID map[string][]ResourceLink
splitAgents map[string]nodeAgentLinkSide // node source ID -> the linked agent the operator split it from
identityPins *identityPinIndex
pbsBackups []models.PBSBackup
// linkFoldIndex maps each canonical ID a manual link folded into another
@@ -233,7 +235,7 @@ func NewRegistryWithStaleThresholds(store ResourceStore, thresholds map[DataSour
bySource: make(map[DataSource]map[string]string),
matcher: NewIdentityMatcher(),
store: store,
exclusions: make(map[string]struct{}),
exclusions: make(map[string]time.Time),
canonicalMetadataDirty: true,
staleThresholds: cloneStaleThresholds(thresholds),
}
@@ -268,9 +270,12 @@ func (rr *ResourceRegistry) loadOverrides() {
// An unlink or link landing between the two reads would load both
// decisions for its pair (manual_link_decisions.go).
rr.links, exclusions = effectiveManualPairDecisions(rr.links, exclusions)
rr.linksByID = indexLinksByID(rr.links)
for _, exclusion := range exclusions {
key := exclusionKey(exclusion.ResourceA, exclusion.ResourceB)
rr.exclusions[key] = struct{}{}
if at, ok := rr.exclusions[key]; !ok || exclusion.CreatedAt.After(at) {
rr.exclusions[key] = exclusion.CreatedAt
}
}
} else {
log.Printf("unifiedresources: failed to load manual exclusions from store: %v", err)
@@ -325,7 +330,7 @@ func (rr *ResourceRegistry) ingestSnapshot(snapshot models.StateSnapshot, thresh
}
}
}
rr.ingestProxmoxNode(node, inferredLinkedHostByNodeID[strings.TrimSpace(node.ID)])
rr.ingestProxmoxNode(rr.splitProxmoxNodeLink(node, inferredLinkedHostByNodeID[strings.TrimSpace(node.ID)]))
}
for _, host := range snapshot.Hosts {
rr.ingestHost(host)
@@ -3455,12 +3460,17 @@ func (rr *ResourceRegistry) resolveLinkedResource(source DataSource, sourceID st
return ""
}
linkedNodeID := strings.TrimSpace(existing.Agent.LinkedNodeID)
if linkedNodeID == sourceID {
return id
if linkedNodeID != sourceID &&
(linkedNodeID != "" || !identitiesShareHostname(existing.Identity, resource.Identity)) {
return ""
}
if linkedNodeID == "" && identitiesShareHostname(existing.Identity, resource.Identity) {
return id
if rr.proxmoxNodeAgentSplitLocked(
nodeAgentLinkSide{sourceID: sourceID, identity: resource.Identity},
nodeAgentLinkSide{sourceID: strings.TrimSpace(resource.Proxmox.LinkedAgentID), identity: existing.Identity, heldID: id},
) {
return ""
}
return id
}
}
case SourceAgent:
@@ -3474,6 +3484,12 @@ func (rr *ResourceRegistry) resolveLinkedResource(source DataSource, sourceID st
if linkedHostID == "" || linkedHostID != sourceID {
return ""
}
if rr.proxmoxNodeAgentSplitLocked(
nodeAgentLinkSide{sourceID: strings.TrimSpace(resource.Agent.LinkedNodeID), identity: existing.Identity, heldID: id},
nodeAgentLinkSide{sourceID: sourceID, identity: resource.Identity},
) {
return ""
}
return id
}
}
@@ -3945,7 +3961,8 @@ func (rr *ResourceRegistry) findCorroboratedOneSidedProxmoxLink(
}
matchID := ""
for _, resourceID := range rr.bySource[SourceProxmox] {
var matchNodeIDs []string
for nodeID, resourceID := range rr.bySource[SourceProxmox] {
existing := rr.resources[resourceID]
if existing == nil || existing.Proxmox == nil {
continue
@@ -3960,6 +3977,15 @@ func (rr *ResourceRegistry) findCorroboratedOneSidedProxmoxLink(
return ""
}
matchID = resourceID
matchNodeIDs = append(matchNodeIDs, nodeID)
}
for _, nodeID := range matchNodeIDs {
if rr.proxmoxNodeAgentSplitLocked(
nodeAgentLinkSide{sourceID: nodeID, identity: rr.resources[matchID].Identity, heldID: matchID},
nodeAgentLinkSide{sourceID: hostAgentID, identity: identity},
) {
return ""
}
}
return matchID
@@ -4988,6 +5014,18 @@ func (rr *ResourceRegistry) applyManualLinks(thresholds map[DataSource]time.Dura
otherID = link.ResourceA
}
other := rr.resources[otherID]
if other == nil && otherID != primaryID {
if source, ok := rr.joinedPairSideLocked(primary, otherID); ok {
// A relink of a split node and its agent names the two rows,
// and the declared link folds one of them into the other's
// ID once the relink rejoins them. Record the fold so pin
// succession keeps the link rather than re-keying it onto
// the joined ID, which would let the split decide again,
// and so report-merge of the joined row names this pair.
recordManualLinkFold(primary, primaryID, &Resource{Sources: []DataSource{source}}, otherID)
rr.indexLinkFoldsLocked(primary)
}
}
if other == nil || otherID == primaryID {
continue
}
@@ -4997,6 +5035,12 @@ func (rr *ResourceRegistry) applyManualLinks(thresholds map[DataSource]time.Dura
if isAvailabilityOwnedResource(*primary) || isAvailabilityOwnedResource(*other) {
continue
}
// A link between a Proxmox node and an agent loses to a newer split
// of the two read across their IDs: report-merge cannot delete a
// link naming the rows an earlier split left (nodeAgentSplit).
if rr.nodeAgentRowsSplit(rr.operatorPairDecisionsLocked(), primary, other) {
continue
}
// A manual link records operator intent to unify identities, but its
// direction must not change the semantic resource shape. In particular,
internal/unifiedresources/registry_merge_policy_test.go
+378
View File
@@ -145,6 +145,384 @@ func onlyResourceOfType(t *testing.T, rr *ResourceRegistry, resourceType Resourc
return resources[0]
}
// An operator who splits a Proxmox node from the pulse-agent Pulse linked it
// to (the drawer's report-merge, or unlink naming the node's side) must see
// the split on every monitor rebuild and in the resources API. The
// node<->agent link merged the pair in resolveLinkedResource before
// findMatch, and the linked node had already taken the agent's machine
// identity, so the pair stayed merged although the request answered 200.
// Each step is judged over three rebuilds of a store-backed adapter (pins
// persist on one rebuild and steer the next) and in every view: the
// monitor's listing, the broadcast coalesce, and the resources API's
// registry seeded from that listing or from the snapshot.
func TestOperatorSplitOverridesProxmoxNodeAgentLink(t *testing.T) {
now := time.Now().UTC()
for _, link := range []struct {
name string
nodeLinked bool
agentLinked bool
cluster string
agentHostname string
machineID string
}{
{"declared-both-ways", true, true, "", "pve1", "0123456789abcdef"},
// A node-side link the agent does not declare joins only when the
// two share a hostname (findCorroboratedOneSidedProxmoxLink).
{"node-side-only", true, false, "", "pve1", "0123456789abcdef"},
{"agent-side-only", false, true, "", "pve1", "0123456789abcdef"},
// A clustered node's own ID is cluster-derived, not one the
// candidate exclusions name, and its identity pin is found by
// cluster and hostname, carrying the agent's machine key.
{"clustered-node", true, true, "lab", "pve1", "0123456789abcdef"},
{"clustered-agent-side-only", false, true, "lab", "pve1", "0123456789abcdef"},
// The joined row's first hostname is the agent's when the agent
// is ingested first.
{"clustered-agent-fqdn", true, true, "lab", "pve1.example", "0123456789abcdef"},
// An agent reporting no machine key joins under the node's own ID.
{"clustered-machine-keyless-agent", true, true, "lab", "pve1", ""},
} {
for _, request := range []struct {
name string
// exclusions returns the pairs the request records.
exclusions func(ids splitRequestIDs) [][2]string
split func(ids splitRequestIDs) bool
}{
{"report-merge", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.merged, ids.nodeCandidate}, {ids.merged, ids.agentCandidate}}
}, splitAlways},
{"unlink-naming-node", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.merged, ids.nodeCandidate}}
}, splitAlways},
// The ID the node row holds once split (cluster-derived for a
// clustered node), as an unlink of the two rows records it.
{"unlink-naming-node-row", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.nodeOwn, ids.agentOwn}}
}, func(ids splitRequestIDs) bool { return ids.nodeOwn != ids.agentOwn }},
// Neither row's ID: a relink of the rows replaces no exclusion
// row, so the newer decision across the pair's IDs must decide.
{"unlink-naming-both-candidates", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.agentCandidate, ids.nodeCandidate}}
}, splitAlways},
// The agent's candidate against the agent's machine-derived ID,
// under which the pair merged, is also what report-merge records
// when it splits another source (a Docker host, say) off the
// agent, so it does not split the node from the agent. Merged
// under the node's own ID, it splits the agent off the node.
{"exclusion-naming-only-the-agent", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.merged, ids.agentCandidate}}
}, func(ids splitRequestIDs) bool { return ids.merged == ids.nodeOwn }},
{"unrelated-exclusion", func(ids splitRequestIDs) [][2]string {
return [][2]string{{ids.merged, "agent-00000000deadbeef"}}
}, func(splitRequestIDs) bool { return false }},
} {
t.Run(link.name+"/"+request.name, func(t *testing.T) {
store, err := NewSQLiteResourceStore(t.TempDir(), "default")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = store.Close() })
node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: link.cluster, Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now}
host := models.Host{
ID: "host-pve1",
Hostname: link.agentHostname,
MachineID: link.machineID,
Status: "online",
LastSeen: now,
// The node's endpoint address and a MAC: what a correct
// link shares, and enough for identity matching to merge
// the pair again if the node kept the agent's identity.
NetworkInterfaces: []models.HostNetworkInterface{{Name: "vmbr0", MAC: "aa:bb:cc:dd:ee:01", Addresses: []string{"10.0.0.5/24"}}},
}
if link.nodeLinked {
node.LinkedAgentID = host.ID
}
if link.agentLinked {
host.LinkedNodeID = node.ID
}
snapshot := models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{host}}
// The IDs each side holds on its own, with nothing to link to.
alone := NewRegistry(nil)
alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}})
nodeAlone := hostRowsByFacet(t, alone.List()).node
alone = NewRegistry(nil)
alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Hosts: []models.Host{host}})
agentAlone := hostRowsByFacet(t, alone.List()).agent
adapter := NewMonitorAdapter(NewRegistry(store))
adapter.PopulateFromSnapshot(snapshot)
merged := hostRowsByFacet(t, adapter.GetAll())
if merged.joined == "" || nodeAlone == "" || agentAlone == "" {
t.Fatalf("fixture did not link the node and the agent: %+v (alone: node %q, agent %q)", merged, nodeAlone, agentAlone)
}
reportMerge := func(mergedID string) (nodeCandidate, agentCandidate string) {
t.Helper()
for _, target := range adapter.currentRegistry().SourceTargets(mergedID) {
switch target.Source {
case SourceProxmox:
nodeCandidate = target.CandidateID
case SourceAgent:
agentCandidate = target.CandidateID
}
}
if nodeCandidate == "" || agentCandidate == "" || nodeCandidate == mergedID || agentCandidate == mergedID {
t.Fatalf("merged %s lists candidates node=%q agent=%q", mergedID, nodeCandidate, agentCandidate)
}
return nodeCandidate, agentCandidate
}
nodeCandidate, agentCandidate := reportMerge(merged.joined)
ids := splitRequestIDs{merged: merged.joined, nodeCandidate: nodeCandidate, agentCandidate: agentCandidate, nodeOwn: nodeAlone, agentOwn: agentAlone}
for _, pair := range request.exclusions(ids) {
if err := store.AddExclusion(ResourceExclusion{ResourceA: pair[0], ResourceB: pair[1], CreatedAt: time.Now().UTC()}); err != nil {
t.Fatal(err)
}
}
// Split, each side returns to the ID it holds alone (for an
// agent reporting a machine key, the ID the pair merged
// under); otherwise the pair stays merged under its ID.
split := request.split(ids)
want := nodeAgentRows{joined: merged.joined}
if split {
want = nodeAgentRows{node: nodeAlone, agent: agentAlone}
}
assertViews := func(step string, rebuilds int, want nodeAgentRows) {
t.Helper()
for rebuild := 1; rebuild <= rebuilds; rebuild++ {
adapter.PopulateFromSnapshot(snapshot)
listed := adapter.GetAll()
broadcast, ok := adapter.CoalesceForPresentation(listed, nil)
if !ok {
t.Fatal("store-backed adapter did not coalesce with its exclusions")
}
rest := NewRegistry(store)
rest.IngestResources(listed)
restFromSnapshot := NewRegistry(store)
restFromSnapshot.IngestSnapshot(snapshot)
for view, resources := range map[string][]Resource{
"monitor": listed,
"broadcast": broadcast,
"rest": rest.List(),
"rest presentation": rest.ListForPresentation(),
"rest from snapshot": restFromSnapshot.List(),
"rest presentation snapshot": restFromSnapshot.ListForPresentation(),
} {
if got := hostRowsByFacet(t, resources); got != want {
t.Fatalf("%s rebuild %d %s: got %+v, want %+v", step, rebuild, view, got, want)
}
}
}
}
// Snapshots ingest nodes first. A registry that meets the agent
// first, which already holds its machine-derived ID, splits the
// same way, also when the snapshot then lacks the host: before
// any split rebuild rewrites the pins, the node's pin from the
// joined era alone carries the agent's machine key.
if split {
withoutHost := snapshot
withoutHost.Hosts = nil
for name, nodes := range map[string]models.StateSnapshot{"with host": snapshot, "without host": withoutHost} {
agentFirst := NewRegistry(store)
agentFirst.IngestRecords(SourceAgent, []IngestRecord{HostIngestRecord(host)})
agentFirst.IngestSnapshot(nodes)
for view, resources := range map[string][]Resource{"list": agentFirst.List(), "presentation": agentFirst.ListForPresentation()} {
if got := hostRowsByFacet(t, resources); got != want {
t.Fatalf("agent ingested first, snapshot %s, %s: got %+v, want %+v", name, view, got, want)
}
}
}
}
assertViews("after the request", 3, want)
if !split {
return
}
// The split node names the agent only where the monitor's own
// node record does: guest actions, deploy and service
// discovery act on a node's linked agent.
for _, resource := range adapter.GetAll() {
if resource.ID == want.node && resource.Proxmox.LinkedAgentID != node.LinkedAgentID {
t.Fatalf("split node links agent %q, its record links %q", resource.Proxmox.LinkedAgentID, node.LinkedAgentID)
}
}
// Linking the rows the split left joins the pair again, although
// report-merge recorded exclusions against other IDs.
if err := store.AddLink(ResourceLink{ResourceA: want.agent, ResourceB: want.node, PrimaryID: want.agent, CreatedAt: time.Now().UTC()}); err != nil {
t.Fatal(err)
}
assertViews("after relink", 2, nodeAgentRows{joined: merged.joined})
// The joined row records the relinked row it took in, so a
// report-merge of it names the relink's own pair.
folded := want.node
if folded == merged.joined {
folded = want.agent
}
recorded := false
for _, fold := range adapter.currentRegistry().ManualLinkFolds(merged.joined) {
recorded = recorded || (fold.HolderID == merged.joined && fold.FoldedID == folded)
}
if !recorded {
t.Fatalf("joined %s records no fold of relinked %s: %+v", merged.joined, folded, adapter.currentRegistry().ManualLinkFolds(merged.joined))
}
// A rebuild that meets the agent first orders the joined row's
// hostnames the agent's way; persisting its pins must not
// re-key the relink onto the joined ID either.
reordered := NewRegistry(store)
reordered.IngestRecords(SourceAgent, []IngestRecord{HostIngestRecord(host)})
reordered.IngestSnapshot(snapshot)
reordered.PersistIdentityPins()
assertViews("after an agent-first rebuild persisted its pins", 2, nodeAgentRows{joined: merged.joined})
// The same request after that relink splits them again. Where
// it names a pair other than the relinked rows, the store keeps
// the older link, which must lose to the newer split.
nodeCandidate, agentCandidate = reportMerge(merged.joined)
ids = splitRequestIDs{merged: merged.joined, nodeCandidate: nodeCandidate, agentCandidate: agentCandidate, nodeOwn: nodeAlone, agentOwn: agentAlone}
for _, pair := range request.exclusions(ids) {
if err := store.AddExclusion(ResourceExclusion{ResourceA: pair[0], ResourceB: pair[1], CreatedAt: time.Now().UTC()}); err != nil {
t.Fatal(err)
}
}
assertViews("after splitting again", 2, want)
})
}
}
}
// A node split from one agent keeps another agent's pin. The node links
// agent A, whose joined-era pin carries A's machine key for the node's
// cluster and hostname; while A is missing from a rebuild, agent B declares
// the node, and the operator has split the node from B. The pin is A's, so
// the node still completes A's key and keeps the ID it holds with A, as any
// boot window before A checks in does.
func TestOperatorSplitFromOneAgentKeepsAnotherAgentsPin(t *testing.T) {
now := time.Now().UTC()
store, err := NewSQLiteResourceStore(t.TempDir(), "default")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = store.Close() })
node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: "lab", Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now, LinkedAgentID: "host-a"}
agentA := models.Host{ID: "host-a", Hostname: "pve1", MachineID: "aaaaaaaaaaaaaaaa", LinkedNodeID: node.ID, Status: "online", LastSeen: now}
agentB := models.Host{ID: "host-b", Hostname: "pve1", MachineID: "bbbbbbbbbbbbbbbb", LinkedNodeID: node.ID, Status: "online", LastSeen: now}
adapter := NewMonitorAdapter(NewRegistry(store))
adapter.PopulateFromSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agentA, agentB}})
var joinedWithA string
for _, resource := range adapter.GetAll() {
if resource.Proxmox != nil && resource.Agent != nil && resource.Agent.AgentID == agentA.ID {
joinedWithA = resource.ID
}
}
if joinedWithA == "" {
t.Fatalf("fixture did not join the node with agent A: %+v", adapter.GetAll())
}
splitFromB := ResourceExclusion{
ResourceA: SourceSpecificID(ResourceTypeAgent, SourceProxmox, node.ID),
ResourceB: SourceSpecificID(ResourceTypeAgent, SourceAgent, agentB.ID),
CreatedAt: time.Now().UTC(),
}
if err := store.AddExclusion(splitFromB); err != nil {
t.Fatal(err)
}
withoutA := models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agentB}}
rebuilt := NewRegistry(store)
rebuilt.IngestSnapshot(withoutA)
for _, resource := range rebuilt.List() {
if resource.Proxmox == nil {
continue
}
if resource.ID != joinedWithA {
t.Fatalf("node took %s while agent A was missing, want %s from A's pin", resource.ID, joinedWithA)
}
if resource.Agent != nil && resource.Agent.AgentID == agentB.ID {
t.Fatalf("node joined agent B, which the operator split it from")
}
}
}
// An agent that stops reporting its machine ID stays split from the node.
// The joined-era pin carries the agent's old machine key for the clustered
// node's cluster and hostname, and the node record names no agent (only the
// agent declares the node), so nothing but the split drop ties the pin to
// the agent: its keyless report must not count as another agent's.
func TestOperatorSplitHoldsWhenTheAgentStopsReportingItsMachineID(t *testing.T) {
now := time.Now().UTC()
store, err := NewSQLiteResourceStore(t.TempDir(), "default")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = store.Close() })
node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: "lab", Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now}
agent := models.Host{ID: "host-pve1", Hostname: "pve1", MachineID: "0123456789abcdef", LinkedNodeID: node.ID, Status: "online", LastSeen: now}
adapter := NewMonitorAdapter(NewRegistry(store))
adapter.PopulateFromSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agent}})
merged := hostRowsByFacet(t, adapter.GetAll())
if merged.joined == "" {
t.Fatalf("fixture did not join the node and the agent: %+v", merged)
}
alone := NewRegistry(nil)
alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}})
nodeAlone := hostRowsByFacet(t, alone.List()).node
if err := store.AddExclusion(ResourceExclusion{
ResourceA: SourceSpecificID(ResourceTypeAgent, SourceProxmox, node.ID),
ResourceB: SourceSpecificID(ResourceTypeAgent, SourceAgent, agent.ID),
CreatedAt: time.Now().UTC(),
}); err != nil {
t.Fatal(err)
}
agent.MachineID = ""
rebuilt := NewRegistry(store)
rebuilt.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agent}})
got := hostRowsByFacet(t, rebuilt.List())
if got.joined != "" || got.node != nodeAlone || got.agent == "" {
t.Fatalf("keyless agent rebuild: got %+v, want the node apart on %s", got, nodeAlone)
}
}
// splitRequestIDs are the IDs an unlink or report-merge of a joined node and
// agent can name: the merged row, its SourceTargets candidates, and the IDs
// the two rows hold once split.
type splitRequestIDs struct {
merged, nodeCandidate, agentCandidate, nodeOwn, agentOwn string
}
func splitAlways(splitRequestIDs) bool { return true }
type nodeAgentRows struct {
node string // a row with the Proxmox facet only
agent string // a row with the agent facet only
joined string // a row with both
}
func hostRowsByFacet(t *testing.T, resources []Resource) nodeAgentRows {
t.Helper()
var rows nodeAgentRows
set := func(slot *string, id string) {
if *slot != "" {
t.Fatalf("two host rows of one kind: %s and %s", *slot, id)
}
*slot = id
}
for _, resource := range resources {
if resource.Type != ResourceTypeAgent {
continue
}
switch {
case resource.Proxmox != nil && resource.Agent != nil:
set(&rows.joined, resource.ID)
case resource.Proxmox != nil:
set(&rows.node, resource.ID)
case resource.Agent != nil:
set(&rows.agent, resource.ID)
}
}
return rows
}
// A host agent past its reporting lease is offline. Its source sighting is
// stale too, and the stale pass used to rank that above offline, so the
// machine reached the frontend as a warning with its last report rendered
internal/unifiedresources/registry_views_branchcov0722pm_test.go
+1 -1
View File
@@ -203,7 +203,7 @@ func TestBranchcov0722PM(t *testing.T) {
}
// Register the exclusion between the two IDs and re-run; the
// closure's lookup must now return true and block the merge.
rr.exclusions[exclusionKey("agent-proxmox-pve1", "agent-runtime-pve1")] = struct{}{}
rr.exclusions[exclusionKey("agent-proxmox-pve1", "agent-runtime-pve1")] = time.Now()
got := rr.ListForPresentation()
if len(got) != 2 {
t.Fatalf("expected exclusion to keep both host views split, got %d: %+v", len(got), got)