diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.md index 9c5dc08450..ac2cdfa09f 100644 --- a/docs/release-control/v6/internal/subsystems/unified-resources.md +++ b/docs/release-control/v6/internal/subsystems/unified-resources.md @@ -7185,8 +7185,127 @@ folded into the agent by ID, a path no exclusion check sees, so the pair stayed merged with the link row gone. A pin whose canonical ID the operator excluded from the ID the record takes without the pin (its `chooseNewID` or source-specific ID) now completes nothing. This covers pins already written -that way, not only new ones. Merges through an agent's own declared node link -(`resolveLinkedResource`) still ignore exclusions. +that way, not only new ones. A Proxmox node's link to an agent honours the +split as well; the next paragraphs cover it. + +An operator's split overrides a Proxmox node's link to a pulse-agent. That +link is normally Pulse's own inference rather than a resource-store +decision: monitoring links a node to the one agent whose endpoint address or +hostname matches it (the agents API can also set it by hand), and the +registry joined the pair in `resolveLinkedResource` (a link both sides +declare, or a node-side link the agent's hostname corroborates) before +`findMatch` ran. The linked node had already taken the agent's identity in +`resourceFromProxmoxNode`, and with an agent reporting a machine key it +minted the agent's machine-derived canonical ID before the agent was +ingested. Neither path read exclusions, so the drawer's Split merged +resource (report-merge) on a node and its agent answered "Resource split +applied" and left the pair merged on every rebuild, in the monitor and in +the resources API. Cloned machines and reused short hostnames are where that +link is wrong, and report-merge exists for wrong merges, so the split wins. + +`nodeAgentSplit` (`proxmox_node_links.go`) decides the pair: the operator +split the node off when an exclusion names one of the node's own IDs (its +source-specific ID, or the ID its identity derives without a machine key, +which a Proxmox node never reports itself) against one of the agent's (its +source-specific ID, the ID its identity derives, the ID it holds, or the ID +the joined pair derives). Unlink and report-merge name the merged ID and the +`SourceTargets` candidate IDs, and the split node can hold an ID it did not +hold while joined, so the decision is read across those sets, not one exact +pair. Where the pair merged under the agent's machine-derived ID, an +exclusion naming only the agent's candidate against that ID does not count: +report-merge records the same pair when it splits another source, such as a +Docker host, off the agent. Where it merged under the node's own ID (an agent +reporting no machine key), that exclusion splits the agent off the node. +`splitProxmoxNodeLink` keeps a split node from taking the agent's identity in +`ingestSnapshot` and remembers the agent for identity-pin completion, which +refuses the agent's machine key from a pin written while they were joined +when the node is split from the agent its own link or an agent row holding +the pin's ID names, or from the remembered agent where the pin can be that +agent's (no machine key or DMI UUID both report differs), so also while the +agent is missing from the snapshot or stops reporting its machine ID; +`resolveLinkedResource` refuses the link from either side. The registry adds +no `LinkedAgentID` to the node: guests, agent deployment and service +discovery act on a node's linked agent. The node then holds the ID its own +identity derives, with only its hostname and endpoint address, which +identity matching scores at 0.80 at most, under the 0.85 auto-merge floor. +The agent holds the ID its own identity derives once pins complete it (its +machine-derived ID when it has a machine key). + +The latest decision across the two sets wins, so a link between them that +postdates the newest exclusion joins the pair again even where it names +another pair than the exclusions do, as a relink of the two rows a +report-merge left does (a registry now keeps each exclusion's `CreatedAt`); +a tie keeps the pair apart. Once a relink rejoins them, the declared link +folds one of the rows it names into the other's ID, and `applyManualLinks` +records that as a `ManualLinkFold` on the joined row (it recognises the +node's own IDs from the row's Proxmox facet, and the agent's source-specific +ID), so pin succession treats the folded ID as observed and keeps the link +instead of re-keying it onto the joined ID and handing the decision back to +the older exclusions, and report-merge of the joined row names that pair. The paths +that join rows after ingest apply the same decision: `applyManualLinks` +skips a link between a node row and an agent row split after it, and the +presentation filter (`ListForPresentation` and the broadcast) keeps such +rows apart when no single exclusion names their IDs; an exclusion naming +the two rows exactly keeps them apart there even against a newer link +across the sets, which only a listing older than the registry's rebuild can +present as two rows. It shares the +presentation limit the broadcast paragraph below names: a third same-host +row (another agent reporting no machine ID, say) that merges with one side +first carries the pair into one row. `TestOperatorSplitOverridesProxmoxNodeAgentLink` +pins report-merge, unlink naming the node's candidate or the split rows, an +exclusion between the two candidates and one naming only the agent's +candidate, for seven link shapes (declared both ways, node-side only and +agent-side only, unclustered and clustered, an agent hostname in a different +form, and an agent reporting no machine key), through three rebuilds of a +SQLite-backed monitor adapter in the monitor listing, the broadcast coalesce +and the resources API's registry seeded from the listing or the snapshot, +after agent-first ingests (with and without the host in the snapshot) taken +while the joined-era pins still stand; then a relink of the rows (also +followed by an agent-first rebuild that persists its pins) and the same +request again. An exclusion naming the merged ID +against an unrelated ID leaves the pair merged, and the split node names the +agent only where its monitor record does. +`TestOperatorSplitFromOneAgentKeepsAnotherAgentsPin` keeps a node's own +agent's pin while another agent it was split from declares it, and +`TestOperatorSplitHoldsWhenTheAgentStopsReportingItsMachineID` keeps a split +whose agent comes back without its machine ID. + +What the split does not reach: the agents API's node link +(`/api/agents/agent/link`) records a monitor-side link intent, not a store +decision, so it does not override a resource split; rejoin through +`POST /api/resources/{id}/link`. The monitor's own node record keeps its +link where it has one, so what reads it still follows the agent: the +agent's SMART inventory as the node's disk fallback when the Proxmox disk +query fails, the agent's disk-exclude patterns on the node's disks, and the +node's linked agent that guest discovery, agent deployment and service +discovery act on. +Physical disks inside one host still join through +`resolveLinkedPhysicalDisk` without reading exclusions; a node split puts the +two sides' disks under different parents, so they no longer meet there. +Cluster- and hostname-derived IDs carry no source, so an exclusion recorded +against another resource that derives the node's own ID (a Docker Swarm host +reporting no machine key, named like the node in a swarm named like its +cluster) also splits that node from its agent, and one splitting the node's +candidate off its own ID while another source was merged there (a Docker +host linked into it) also splits it from a later agent reporting no machine +key, which joins under that ID. A rebuild that sees the agent in neither the +snapshot nor the registry, with no node link naming it, cannot tie an +exclusion naming the agent only by its candidate ID to the agent's key on a +joined-era pin, so until the agent reports again the node takes that key +and the agent's ID. The other way round, a node split from an agent that +reports no machine key and declares it refuses any joined-era pin while its +own linked agent is missing, keeping its own ID until that agent reports. An exclusion keyed by the +agent's machine-derived ID stops naming the agent if the agent's effective +canonical key changes (it stops reporting a machine ID and no pin restores +it), as every exclusion keyed by a canonical ID does. The browser's realtime +coalesce (`mergeCanonicalResourceSnapshot` in `resourceStateAdapters.ts`, +used by the websocket store and `useUnifiedResources`) receives no +exclusions and joins a Proxmox-only row and an agent-only row that share a +hostname unless their machine IDs or DMI UUIDs disagree, their Proxmox +facets name different clusters or provider scopes, or several Proxmox nodes +share the hostname and the node does not name the agent, so the UI still +renders such a split pair, from this change or from an identity-match split, +as one row. The resources API seeds its registry from the monitor's listing, which already has the link applied, so it cannot split the pair itself: an unlink diff --git a/internal/unifiedresources/canonical_id_pins.go b/internal/unifiedresources/canonical_id_pins.go index 847380b1bd..c48dffd3dc 100644 --- a/internal/unifiedresources/canonical_id_pins.go +++ b/internal/unifiedresources/canonical_id_pins.go @@ -265,7 +265,8 @@ func (rr *ResourceRegistry) completeIdentityFromPins(source DataSource, sourceID // The exclusion names the IDs the record holds without the pin. if len(rr.exclusions) > 0 && (rr.isExcluded(pin.CanonicalID, rr.chooseNewID(resource.Type, identity, source, sourceID)) || - rr.isExcluded(pin.CanonicalID, rr.sourceSpecificID(resource.Type, source, sourceID))) { + rr.isExcluded(pin.CanonicalID, rr.sourceSpecificID(resource.Type, source, sourceID)) || + rr.proxmoxNodePinOfSplitAgentLocked(source, sourceID, resource, identity, pin)) { return identity } if strings.TrimSpace(identity.MachineID) == "" { diff --git a/internal/unifiedresources/presentation_exclusions.go b/internal/unifiedresources/presentation_exclusions.go index b00f0dd4c2..dde1f8325b 100644 --- a/internal/unifiedresources/presentation_exclusions.go +++ b/internal/unifiedresources/presentation_exclusions.go @@ -4,17 +4,23 @@ import "time" // presentationExclusionFilter reports whether the presentation host coalesce // must keep two resources apart because the operator split the pair (unlink -// or report-merge). It reads a copy of the registry's exclusions, so callers -// coalesce without holding the registry lock; nil means no pair is excluded. +// or report-merge): an exclusion names the two rows, or they are a Proxmox +// node and an agent split across the IDs they can hold (nodeAgentSplit). It +// reads a copy of the registry's decisions, so callers coalesce without +// holding the registry lock; nil means no pair is excluded. func (rr *ResourceRegistry) presentationExclusionFilter() func(left, right Resource) bool { rr.mu.RLock() if len(rr.exclusions) == 0 { rr.mu.RUnlock() return nil } - exclusions := make(map[string]struct{}, len(rr.exclusions)) - for key := range rr.exclusions { - exclusions[key] = struct{}{} + // The link index is built once per registry and never written again. + decisions := operatorPairDecisions{ + exclusions: make(map[string]time.Time, len(rr.exclusions)), + linksByID: rr.linksByID, + } + for key, at := range rr.exclusions { + decisions.exclusions[key] = at } rr.mu.RUnlock() @@ -24,8 +30,10 @@ func (rr *ResourceRegistry) presentationExclusionFilter() func(left, right Resou if leftID == "" || rightID == "" { return false } - _, ok := exclusions[exclusionKey(leftID, rightID)] - return ok + if _, ok := decisions.exclusions[exclusionKey(leftID, rightID)]; ok { + return true + } + return rr.nodeAgentRowsSplit(decisions, &left, &right) } } diff --git a/internal/unifiedresources/proxmox_node_links.go b/internal/unifiedresources/proxmox_node_links.go index e38e0d2de2..72cb6bbe72 100644 --- a/internal/unifiedresources/proxmox_node_links.go +++ b/internal/unifiedresources/proxmox_node_links.go @@ -2,11 +2,303 @@ package unifiedresources import ( "net" + "slices" "strings" + "time" "github.com/rcourtman/pulse-go-rewrite/internal/models" ) +// A Proxmox node's link to a pulse-agent is Pulse's own inference, not an +// operator decision: monitoring links a node to the one agent whose endpoint +// address or hostname matches it, and inferLinkedHostsForProxmoxNodes and +// resolveLinkedResource corroborate that link. Cloned machines and reused +// short hostnames make it wrong at times, which is what the resources API's +// unlink and report-merge are for, so an operator's split overrides the link +// as it overrides identity matching in findMatch. A later POST +// /api/resources/{id}/link between the two rows joins them again. + +// nodeAgentLinkSide is one side of a Proxmox node<->agent pair: the record's +// source ID, the identity it derives IDs from and the resource ID it holds in +// the registry, if any. +type nodeAgentLinkSide struct { + sourceID string + identity ResourceIdentity + heldID string +} + +// operatorPairDecisions is the operator's manual decisions a node<->agent +// split is read from: a registry's own, or the copy the presentation filter +// holds outside the registry lock. +type operatorPairDecisions struct { + exclusions map[string]time.Time + linksByID map[string][]ResourceLink +} + +func (rr *ResourceRegistry) operatorPairDecisionsLocked() operatorPairDecisions { + return operatorPairDecisions{exclusions: rr.exclusions, linksByID: rr.linksByID} +} + +// indexLinksByID lists each manual link under both of its IDs. +func indexLinksByID(links []ResourceLink) map[string][]ResourceLink { + if len(links) == 0 { + return nil + } + index := make(map[string][]ResourceLink, 2*len(links)) + for _, link := range links { + a, b := CanonicalResourceID(link.ResourceA), CanonicalResourceID(link.ResourceB) + index[a] = append(index[a], link) + if b != a { + index[b] = append(index[b], link) + } + } + return index +} + +// nodeAgentSplit reports whether the operator split the node off a +// node<->agent pair: an exclusion names one of the node's own IDs against one +// of the agent's IDs, and no link between those IDs postdates it (a tie keeps +// the pair apart, as it does for one pair in effectiveManualPairDecisions). +// +// Unlink and report-merge name the merged resource's ID and the source-specific +// candidate IDs its SourceTargets list, and a split moves the node onto an ID +// it did not hold while joined, so decisions are read across sets rather +// than one exact pair. The node's own IDs are its source-specific ID and the +// ID its identity derives without a machine key. The agent's are its +// source-specific ID, the ID its identity derives, the ID it holds, and the +// ID the joined pair derives (the agent's machine-derived ID once a node +// takes its identity). An exclusion naming only the agent's candidate against +// the joined ID does not count: report-merge records that same pair when it +// splits another source, such as a Docker host, off the agent's +// machine-derived ID. +// +// A link between the two sets that postdates the newest exclusion wins: a +// relink names the rows a split left, a different pair than the exclusions +// report-merge recorded against the merged ID, which the store's +// one-decision-per-pair rule cannot match. +func (rr *ResourceRegistry) nodeAgentSplit(decisions operatorPairDecisions, node, agent nodeAgentLinkSide) bool { + if len(decisions.exclusions) == 0 { + return false + } + nodeIDs, agentIDs := rr.nodeAgentPairIDs(node, agent) + var splitAt time.Time + split := false + for _, nodeID := range nodeIDs { + for _, agentID := range agentIDs { + if nodeID == agentID { + continue + } + if at, ok := decisions.exclusions[exclusionKey(nodeID, agentID)]; ok && (!split || at.After(splitAt)) { + split, splitAt = true, at + } + } + } + if !split { + return false + } + for _, nodeID := range nodeIDs { + for _, link := range decisions.linksByID[nodeID] { + other := CanonicalResourceID(link.ResourceB) + if other == nodeID { + other = CanonicalResourceID(link.ResourceA) + } + if other != nodeID && slices.Contains(agentIDs, other) && link.CreatedAt.After(splitAt) { + return false + } + } + } + return true +} + +// nodeAgentPairIDs returns the node's own IDs and the agent's IDs a split is +// read across. A Proxmox node reports no machine ID or DMI UUID of its own: +// either one on its identity came from a linked agent or an identity pin, so +// the node's own ID is derived without them. The two IDs are then the same +// whether or not the node took the agent's identity in this ingest. +func (rr *ResourceRegistry) nodeAgentPairIDs(node, agent nodeAgentLinkSide) (nodeIDs, agentIDs []string) { + nodeIDs = rr.proxmoxNodeOwnIDs(node.sourceID, node.identity) + if agent.sourceID != "" { + agentIDs = append(agentIDs, rr.sourceSpecificID(ResourceTypeAgent, SourceAgent, agent.sourceID)) + } + if agent.sourceID != "" || agent.identity.MachineID != "" || agent.identity.DMIUUID != "" { + agentIDs = append(agentIDs, rr.chooseNewID(ResourceTypeAgent, agent.identity, SourceAgent, agent.sourceID)) + } + if held := CanonicalResourceID(agent.heldID); held != "" { + agentIDs = append(agentIDs, held) + } + // The joined ID counts as the agent's even where it is also one of the + // node's (an agent reporting no machine key joins under the node's own + // ID): an exclusion splitting the node's candidate off it then splits + // the node from the agent, and the equal pair is skipped. + nodeIdentity := node.identity + nodeIdentity.MachineID, nodeIdentity.DMIUUID = "", "" + agentIDs = append(agentIDs, rr.chooseNewID(ResourceTypeAgent, mergeIdentity(nodeIdentity, agent.identity), SourceProxmox, node.sourceID)) + return nodeIDs, agentIDs +} + +// proxmoxNodeOwnIDs returns the IDs a Proxmox node record holds without any +// agent: its source-specific ID and the ID its identity derives with the +// machine keys it cannot report itself removed. +func (rr *ResourceRegistry) proxmoxNodeOwnIDs(sourceID string, identity ResourceIdentity) []string { + identity.MachineID, identity.DMIUUID = "", "" + return []string{ + rr.sourceSpecificID(ResourceTypeAgent, SourceProxmox, sourceID), + rr.chooseNewID(ResourceTypeAgent, identity, SourceProxmox, sourceID), + } +} + +// proxmoxNodeAgentSplitLocked applies nodeAgentSplit to the registry's own +// decisions. Callers hold rr.mu. +func (rr *ResourceRegistry) proxmoxNodeAgentSplitLocked(node, agent nodeAgentLinkSide) bool { + return rr.nodeAgentSplit(rr.operatorPairDecisionsLocked(), node, agent) +} + +// proxmoxNodePinOfSplitAgentLocked reports whether an identity pin a Proxmox +// node record would complete its machine keys from belongs to an agent the +// operator split the node from. A node reports no machine key of its own, so +// a pin's is an agent's, written while the pair was joined, and completing +// it would mint that agent's machine-derived ID and fold the node into the +// agent by ID or by identity. The pin's canonical ID and keys stand for the +// agent the node's link or an agent row holding the pin's ID names, so this +// holds while the agent is missing from the snapshot. The agent this +// snapshot's split dropped (rr.splitAgents) can be another agent than the +// pin's, so it refuses only a pin that can be its own. +func (rr *ResourceRegistry) proxmoxNodePinOfSplitAgentLocked(source DataSource, sourceID string, resource Resource, identity ResourceIdentity, pin ResourceIdentityPin) bool { + if source != SourceProxmox || resource.Proxmox == nil || len(rr.exclusions) == 0 { + return false + } + node := nodeAgentLinkSide{sourceID: sourceID, identity: identity} + pinAgent := nodeAgentLinkSide{ + sourceID: normalizeSourceID(resource.Proxmox.LinkedAgentID), + identity: ResourceIdentity{MachineID: pin.MachineID, DMIUUID: pin.DMIUUID}, + heldID: pin.CanonicalID, + } + if rr.proxmoxNodeAgentSplitLocked(node, pinAgent) { + return true + } + if held := agentOnlyRow(rr.resources[CanonicalResourceID(pin.CanonicalID)]); held != nil { + if heldSourceID := normalizeSourceID(held.Agent.AgentID); heldSourceID != pinAgent.sourceID { + pinAgent.sourceID = heldSourceID + if rr.proxmoxNodeAgentSplitLocked(node, pinAgent) { + return true + } + } + } + split, ok := rr.splitAgents[normalizeSourceID(sourceID)] + return ok && pinBelongsToAgent(pin, split.identity) && rr.proxmoxNodeAgentSplitLocked(node, split) +} + +// pinBelongsToAgent reports whether an identity pin can be the agent's: only +// a key both report that differs shows the pin is another agent's. An agent +// that stops reporting its machine ID keeps its split that way, while another +// agent with its own key does not claim the pin. +func pinBelongsToAgent(pin ResourceIdentityPin, agent ResourceIdentity) bool { + if pinKey, agentKey := strings.TrimSpace(pin.MachineID), strings.TrimSpace(agent.MachineID); pinKey != "" && agentKey != "" { + return strings.EqualFold(pinKey, agentKey) + } + if pinKey, agentKey := strings.TrimSpace(pin.DMIUUID), strings.TrimSpace(agent.DMIUUID); pinKey != "" && agentKey != "" { + return strings.EqualFold(pinKey, agentKey) + } + return true +} + +// joinedPairSideLocked reports whether id is an ID one side of a joined +// Proxmox node+agent row holds on its own, and that side's source: the +// node's own IDs (derived from the row's Proxmox facet, not its merged +// identity, whose first hostname can be the agent's) or the agent's +// source-specific ID, which an agent reporting no machine key holds when +// split. +func (rr *ResourceRegistry) joinedPairSideLocked(joined *Resource, id string) (DataSource, bool) { + if joined == nil || joined.Proxmox == nil || joined.Agent == nil || CanonicalResourceType(joined.Type) != ResourceTypeAgent { + return "", false + } + id = CanonicalResourceID(id) + if agentID := normalizeSourceID(joined.Agent.AgentID); agentID != "" && id == rr.sourceSpecificID(ResourceTypeAgent, SourceAgent, agentID) { + return SourceAgent, true + } + sourceID := normalizeSourceID(joined.Proxmox.SourceID) + if sourceID == "" { + return "", false + } + _, identity := resourceFromProxmoxNode(models.Node{ + ID: sourceID, + Name: joined.Proxmox.NodeName, + NativeNameAliases: joined.Proxmox.NodeAliases, + Instance: joined.Proxmox.Instance, + ClusterName: joined.Proxmox.ClusterName, + }, nil) + return SourceProxmox, slices.Contains(rr.proxmoxNodeOwnIDs(sourceID, identity), id) +} + +// nodeAgentRowsSplit reports whether two listed rows are a Proxmox node and +// an agent the operator split, for the paths that join rows after ingest: +// manual links (an older link between the rows a split left loses to a newer +// report-merge) and the presentation host coalesce (the rows can hold IDs no +// single exclusion names, a clustered node's cluster-derived ID for one). +func (rr *ResourceRegistry) nodeAgentRowsSplit(decisions operatorPairDecisions, left, right *Resource) bool { + node, agent := proxmoxNodeRow(left), agentOnlyRow(right) + if node == nil || agent == nil { + node, agent = proxmoxNodeRow(right), agentOnlyRow(left) + } + if node == nil || agent == nil { + return false + } + return rr.nodeAgentSplit(decisions, + nodeAgentLinkSide{sourceID: normalizeSourceID(node.Proxmox.SourceID), identity: node.Identity}, + nodeAgentLinkSide{sourceID: normalizeSourceID(agent.Agent.AgentID), identity: agent.Identity, heldID: agent.ID}, + ) +} + +func proxmoxNodeRow(resource *Resource) *Resource { + if resource == nil || CanonicalResourceType(resource.Type) != ResourceTypeAgent || + resource.Proxmox == nil || resource.Agent != nil || strings.TrimSpace(resource.Proxmox.SourceID) == "" { + return nil + } + return resource +} + +func agentOnlyRow(resource *Resource) *Resource { + if resource == nil || CanonicalResourceType(resource.Type) != ResourceTypeAgent || + resource.Agent == nil || resource.Proxmox != nil || strings.TrimSpace(resource.Agent.AgentID) == "" { + return nil + } + return resource +} + +// splitProxmoxNodeLink returns the node record and the linked host it takes +// identity from, dropping the host for a node the operator split from that +// host's agent. A linked node takes the agent's identity, and with it the +// agent's machine-derived canonical ID, before the agent is ingested, so +// refusing only the later merge would leave the node holding the agent's ID +// and machine key, and identity matching would join the agent to it again. +// Without the agent's identity the node keeps its own ID, hostname and +// endpoint address, which identity matching never merges on. rr.splitAgents +// remembers the agent for the pin guard, which reads the split from it where +// the node record itself does not name that agent; it is rewritten for every +// node on every snapshot, and nothing else reads it. +func (rr *ResourceRegistry) splitProxmoxNodeLink(node models.Node, host *models.Host) (models.Node, *models.Host) { + nodeID := normalizeSourceID(node.ID) + rr.mu.Lock() + defer rr.mu.Unlock() + delete(rr.splitAgents, nodeID) + if host == nil || len(rr.exclusions) == 0 { + return node, host + } + _, nodeIdentity := resourceFromProxmoxNode(node, nil) + _, hostIdentity := resourceFromHost(*host) + if !rr.proxmoxNodeAgentSplitLocked( + nodeAgentLinkSide{sourceID: nodeID, identity: nodeIdentity}, + nodeAgentLinkSide{sourceID: normalizeSourceID(host.ID), identity: hostIdentity}, + ) { + return node, host + } + if rr.splitAgents == nil { + rr.splitAgents = make(map[string]nodeAgentLinkSide) + } + rr.splitAgents[nodeID] = nodeAgentLinkSide{sourceID: normalizeSourceID(host.ID), identity: hostIdentity} + return node, nil +} + func inferLinkedHostsForProxmoxNodes(nodes []models.Node, hostByID map[string]*models.Host) map[string]*models.Host { if len(nodes) == 0 || len(hostByID) == 0 { return nil diff --git a/internal/unifiedresources/registry.go b/internal/unifiedresources/registry.go index ce631cbcc6..3b7306d01d 100644 --- a/internal/unifiedresources/registry.go +++ b/internal/unifiedresources/registry.go @@ -93,7 +93,9 @@ type ResourceRegistry struct { matcher *IdentityMatcher store ResourceStore links []ResourceLink - exclusions map[string]struct{} + exclusions map[string]time.Time // exclusionKey -> when the operator last split the pair + linksByID map[string][]ResourceLink + splitAgents map[string]nodeAgentLinkSide // node source ID -> the linked agent the operator split it from identityPins *identityPinIndex pbsBackups []models.PBSBackup // linkFoldIndex maps each canonical ID a manual link folded into another @@ -233,7 +235,7 @@ func NewRegistryWithStaleThresholds(store ResourceStore, thresholds map[DataSour bySource: make(map[DataSource]map[string]string), matcher: NewIdentityMatcher(), store: store, - exclusions: make(map[string]struct{}), + exclusions: make(map[string]time.Time), canonicalMetadataDirty: true, staleThresholds: cloneStaleThresholds(thresholds), } @@ -268,9 +270,12 @@ func (rr *ResourceRegistry) loadOverrides() { // An unlink or link landing between the two reads would load both // decisions for its pair (manual_link_decisions.go). rr.links, exclusions = effectiveManualPairDecisions(rr.links, exclusions) + rr.linksByID = indexLinksByID(rr.links) for _, exclusion := range exclusions { key := exclusionKey(exclusion.ResourceA, exclusion.ResourceB) - rr.exclusions[key] = struct{}{} + if at, ok := rr.exclusions[key]; !ok || exclusion.CreatedAt.After(at) { + rr.exclusions[key] = exclusion.CreatedAt + } } } else { log.Printf("unifiedresources: failed to load manual exclusions from store: %v", err) @@ -325,7 +330,7 @@ func (rr *ResourceRegistry) ingestSnapshot(snapshot models.StateSnapshot, thresh } } } - rr.ingestProxmoxNode(node, inferredLinkedHostByNodeID[strings.TrimSpace(node.ID)]) + rr.ingestProxmoxNode(rr.splitProxmoxNodeLink(node, inferredLinkedHostByNodeID[strings.TrimSpace(node.ID)])) } for _, host := range snapshot.Hosts { rr.ingestHost(host) @@ -3455,12 +3460,17 @@ func (rr *ResourceRegistry) resolveLinkedResource(source DataSource, sourceID st return "" } linkedNodeID := strings.TrimSpace(existing.Agent.LinkedNodeID) - if linkedNodeID == sourceID { - return id + if linkedNodeID != sourceID && + (linkedNodeID != "" || !identitiesShareHostname(existing.Identity, resource.Identity)) { + return "" } - if linkedNodeID == "" && identitiesShareHostname(existing.Identity, resource.Identity) { - return id + if rr.proxmoxNodeAgentSplitLocked( + nodeAgentLinkSide{sourceID: sourceID, identity: resource.Identity}, + nodeAgentLinkSide{sourceID: strings.TrimSpace(resource.Proxmox.LinkedAgentID), identity: existing.Identity, heldID: id}, + ) { + return "" } + return id } } case SourceAgent: @@ -3474,6 +3484,12 @@ func (rr *ResourceRegistry) resolveLinkedResource(source DataSource, sourceID st if linkedHostID == "" || linkedHostID != sourceID { return "" } + if rr.proxmoxNodeAgentSplitLocked( + nodeAgentLinkSide{sourceID: strings.TrimSpace(resource.Agent.LinkedNodeID), identity: existing.Identity, heldID: id}, + nodeAgentLinkSide{sourceID: sourceID, identity: resource.Identity}, + ) { + return "" + } return id } } @@ -3945,7 +3961,8 @@ func (rr *ResourceRegistry) findCorroboratedOneSidedProxmoxLink( } matchID := "" - for _, resourceID := range rr.bySource[SourceProxmox] { + var matchNodeIDs []string + for nodeID, resourceID := range rr.bySource[SourceProxmox] { existing := rr.resources[resourceID] if existing == nil || existing.Proxmox == nil { continue @@ -3960,6 +3977,15 @@ func (rr *ResourceRegistry) findCorroboratedOneSidedProxmoxLink( return "" } matchID = resourceID + matchNodeIDs = append(matchNodeIDs, nodeID) + } + for _, nodeID := range matchNodeIDs { + if rr.proxmoxNodeAgentSplitLocked( + nodeAgentLinkSide{sourceID: nodeID, identity: rr.resources[matchID].Identity, heldID: matchID}, + nodeAgentLinkSide{sourceID: hostAgentID, identity: identity}, + ) { + return "" + } } return matchID @@ -4988,6 +5014,18 @@ func (rr *ResourceRegistry) applyManualLinks(thresholds map[DataSource]time.Dura otherID = link.ResourceA } other := rr.resources[otherID] + if other == nil && otherID != primaryID { + if source, ok := rr.joinedPairSideLocked(primary, otherID); ok { + // A relink of a split node and its agent names the two rows, + // and the declared link folds one of them into the other's + // ID once the relink rejoins them. Record the fold so pin + // succession keeps the link rather than re-keying it onto + // the joined ID, which would let the split decide again, + // and so report-merge of the joined row names this pair. + recordManualLinkFold(primary, primaryID, &Resource{Sources: []DataSource{source}}, otherID) + rr.indexLinkFoldsLocked(primary) + } + } if other == nil || otherID == primaryID { continue } @@ -4997,6 +5035,12 @@ func (rr *ResourceRegistry) applyManualLinks(thresholds map[DataSource]time.Dura if isAvailabilityOwnedResource(*primary) || isAvailabilityOwnedResource(*other) { continue } + // A link between a Proxmox node and an agent loses to a newer split + // of the two read across their IDs: report-merge cannot delete a + // link naming the rows an earlier split left (nodeAgentSplit). + if rr.nodeAgentRowsSplit(rr.operatorPairDecisionsLocked(), primary, other) { + continue + } // A manual link records operator intent to unify identities, but its // direction must not change the semantic resource shape. In particular, diff --git a/internal/unifiedresources/registry_merge_policy_test.go b/internal/unifiedresources/registry_merge_policy_test.go index cb4d9a4f08..65ebb8288d 100644 --- a/internal/unifiedresources/registry_merge_policy_test.go +++ b/internal/unifiedresources/registry_merge_policy_test.go @@ -145,6 +145,384 @@ func onlyResourceOfType(t *testing.T, rr *ResourceRegistry, resourceType Resourc return resources[0] } +// An operator who splits a Proxmox node from the pulse-agent Pulse linked it +// to (the drawer's report-merge, or unlink naming the node's side) must see +// the split on every monitor rebuild and in the resources API. The +// node<->agent link merged the pair in resolveLinkedResource before +// findMatch, and the linked node had already taken the agent's machine +// identity, so the pair stayed merged although the request answered 200. +// Each step is judged over three rebuilds of a store-backed adapter (pins +// persist on one rebuild and steer the next) and in every view: the +// monitor's listing, the broadcast coalesce, and the resources API's +// registry seeded from that listing or from the snapshot. +func TestOperatorSplitOverridesProxmoxNodeAgentLink(t *testing.T) { + now := time.Now().UTC() + for _, link := range []struct { + name string + nodeLinked bool + agentLinked bool + cluster string + agentHostname string + machineID string + }{ + {"declared-both-ways", true, true, "", "pve1", "0123456789abcdef"}, + // A node-side link the agent does not declare joins only when the + // two share a hostname (findCorroboratedOneSidedProxmoxLink). + {"node-side-only", true, false, "", "pve1", "0123456789abcdef"}, + {"agent-side-only", false, true, "", "pve1", "0123456789abcdef"}, + // A clustered node's own ID is cluster-derived, not one the + // candidate exclusions name, and its identity pin is found by + // cluster and hostname, carrying the agent's machine key. + {"clustered-node", true, true, "lab", "pve1", "0123456789abcdef"}, + {"clustered-agent-side-only", false, true, "lab", "pve1", "0123456789abcdef"}, + // The joined row's first hostname is the agent's when the agent + // is ingested first. + {"clustered-agent-fqdn", true, true, "lab", "pve1.example", "0123456789abcdef"}, + // An agent reporting no machine key joins under the node's own ID. + {"clustered-machine-keyless-agent", true, true, "lab", "pve1", ""}, + } { + for _, request := range []struct { + name string + // exclusions returns the pairs the request records. + exclusions func(ids splitRequestIDs) [][2]string + split func(ids splitRequestIDs) bool + }{ + {"report-merge", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.merged, ids.nodeCandidate}, {ids.merged, ids.agentCandidate}} + }, splitAlways}, + {"unlink-naming-node", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.merged, ids.nodeCandidate}} + }, splitAlways}, + // The ID the node row holds once split (cluster-derived for a + // clustered node), as an unlink of the two rows records it. + {"unlink-naming-node-row", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.nodeOwn, ids.agentOwn}} + }, func(ids splitRequestIDs) bool { return ids.nodeOwn != ids.agentOwn }}, + // Neither row's ID: a relink of the rows replaces no exclusion + // row, so the newer decision across the pair's IDs must decide. + {"unlink-naming-both-candidates", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.agentCandidate, ids.nodeCandidate}} + }, splitAlways}, + // The agent's candidate against the agent's machine-derived ID, + // under which the pair merged, is also what report-merge records + // when it splits another source (a Docker host, say) off the + // agent, so it does not split the node from the agent. Merged + // under the node's own ID, it splits the agent off the node. + {"exclusion-naming-only-the-agent", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.merged, ids.agentCandidate}} + }, func(ids splitRequestIDs) bool { return ids.merged == ids.nodeOwn }}, + {"unrelated-exclusion", func(ids splitRequestIDs) [][2]string { + return [][2]string{{ids.merged, "agent-00000000deadbeef"}} + }, func(splitRequestIDs) bool { return false }}, + } { + t.Run(link.name+"/"+request.name, func(t *testing.T) { + store, err := NewSQLiteResourceStore(t.TempDir(), "default") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = store.Close() }) + + node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: link.cluster, Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now} + host := models.Host{ + ID: "host-pve1", + Hostname: link.agentHostname, + MachineID: link.machineID, + Status: "online", + LastSeen: now, + // The node's endpoint address and a MAC: what a correct + // link shares, and enough for identity matching to merge + // the pair again if the node kept the agent's identity. + NetworkInterfaces: []models.HostNetworkInterface{{Name: "vmbr0", MAC: "aa:bb:cc:dd:ee:01", Addresses: []string{"10.0.0.5/24"}}}, + } + if link.nodeLinked { + node.LinkedAgentID = host.ID + } + if link.agentLinked { + host.LinkedNodeID = node.ID + } + snapshot := models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{host}} + // The IDs each side holds on its own, with nothing to link to. + alone := NewRegistry(nil) + alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}}) + nodeAlone := hostRowsByFacet(t, alone.List()).node + alone = NewRegistry(nil) + alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Hosts: []models.Host{host}}) + agentAlone := hostRowsByFacet(t, alone.List()).agent + adapter := NewMonitorAdapter(NewRegistry(store)) + + adapter.PopulateFromSnapshot(snapshot) + merged := hostRowsByFacet(t, adapter.GetAll()) + if merged.joined == "" || nodeAlone == "" || agentAlone == "" { + t.Fatalf("fixture did not link the node and the agent: %+v (alone: node %q, agent %q)", merged, nodeAlone, agentAlone) + } + reportMerge := func(mergedID string) (nodeCandidate, agentCandidate string) { + t.Helper() + for _, target := range adapter.currentRegistry().SourceTargets(mergedID) { + switch target.Source { + case SourceProxmox: + nodeCandidate = target.CandidateID + case SourceAgent: + agentCandidate = target.CandidateID + } + } + if nodeCandidate == "" || agentCandidate == "" || nodeCandidate == mergedID || agentCandidate == mergedID { + t.Fatalf("merged %s lists candidates node=%q agent=%q", mergedID, nodeCandidate, agentCandidate) + } + return nodeCandidate, agentCandidate + } + nodeCandidate, agentCandidate := reportMerge(merged.joined) + ids := splitRequestIDs{merged: merged.joined, nodeCandidate: nodeCandidate, agentCandidate: agentCandidate, nodeOwn: nodeAlone, agentOwn: agentAlone} + for _, pair := range request.exclusions(ids) { + if err := store.AddExclusion(ResourceExclusion{ResourceA: pair[0], ResourceB: pair[1], CreatedAt: time.Now().UTC()}); err != nil { + t.Fatal(err) + } + } + + // Split, each side returns to the ID it holds alone (for an + // agent reporting a machine key, the ID the pair merged + // under); otherwise the pair stays merged under its ID. + split := request.split(ids) + want := nodeAgentRows{joined: merged.joined} + if split { + want = nodeAgentRows{node: nodeAlone, agent: agentAlone} + } + assertViews := func(step string, rebuilds int, want nodeAgentRows) { + t.Helper() + for rebuild := 1; rebuild <= rebuilds; rebuild++ { + adapter.PopulateFromSnapshot(snapshot) + listed := adapter.GetAll() + broadcast, ok := adapter.CoalesceForPresentation(listed, nil) + if !ok { + t.Fatal("store-backed adapter did not coalesce with its exclusions") + } + rest := NewRegistry(store) + rest.IngestResources(listed) + restFromSnapshot := NewRegistry(store) + restFromSnapshot.IngestSnapshot(snapshot) + for view, resources := range map[string][]Resource{ + "monitor": listed, + "broadcast": broadcast, + "rest": rest.List(), + "rest presentation": rest.ListForPresentation(), + "rest from snapshot": restFromSnapshot.List(), + "rest presentation snapshot": restFromSnapshot.ListForPresentation(), + } { + if got := hostRowsByFacet(t, resources); got != want { + t.Fatalf("%s rebuild %d %s: got %+v, want %+v", step, rebuild, view, got, want) + } + } + } + } + // Snapshots ingest nodes first. A registry that meets the agent + // first, which already holds its machine-derived ID, splits the + // same way, also when the snapshot then lacks the host: before + // any split rebuild rewrites the pins, the node's pin from the + // joined era alone carries the agent's machine key. + if split { + withoutHost := snapshot + withoutHost.Hosts = nil + for name, nodes := range map[string]models.StateSnapshot{"with host": snapshot, "without host": withoutHost} { + agentFirst := NewRegistry(store) + agentFirst.IngestRecords(SourceAgent, []IngestRecord{HostIngestRecord(host)}) + agentFirst.IngestSnapshot(nodes) + for view, resources := range map[string][]Resource{"list": agentFirst.List(), "presentation": agentFirst.ListForPresentation()} { + if got := hostRowsByFacet(t, resources); got != want { + t.Fatalf("agent ingested first, snapshot %s, %s: got %+v, want %+v", name, view, got, want) + } + } + } + } + assertViews("after the request", 3, want) + if !split { + return + } + // The split node names the agent only where the monitor's own + // node record does: guest actions, deploy and service + // discovery act on a node's linked agent. + for _, resource := range adapter.GetAll() { + if resource.ID == want.node && resource.Proxmox.LinkedAgentID != node.LinkedAgentID { + t.Fatalf("split node links agent %q, its record links %q", resource.Proxmox.LinkedAgentID, node.LinkedAgentID) + } + } + + // Linking the rows the split left joins the pair again, although + // report-merge recorded exclusions against other IDs. + if err := store.AddLink(ResourceLink{ResourceA: want.agent, ResourceB: want.node, PrimaryID: want.agent, CreatedAt: time.Now().UTC()}); err != nil { + t.Fatal(err) + } + assertViews("after relink", 2, nodeAgentRows{joined: merged.joined}) + // The joined row records the relinked row it took in, so a + // report-merge of it names the relink's own pair. + folded := want.node + if folded == merged.joined { + folded = want.agent + } + recorded := false + for _, fold := range adapter.currentRegistry().ManualLinkFolds(merged.joined) { + recorded = recorded || (fold.HolderID == merged.joined && fold.FoldedID == folded) + } + if !recorded { + t.Fatalf("joined %s records no fold of relinked %s: %+v", merged.joined, folded, adapter.currentRegistry().ManualLinkFolds(merged.joined)) + } + // A rebuild that meets the agent first orders the joined row's + // hostnames the agent's way; persisting its pins must not + // re-key the relink onto the joined ID either. + reordered := NewRegistry(store) + reordered.IngestRecords(SourceAgent, []IngestRecord{HostIngestRecord(host)}) + reordered.IngestSnapshot(snapshot) + reordered.PersistIdentityPins() + assertViews("after an agent-first rebuild persisted its pins", 2, nodeAgentRows{joined: merged.joined}) + + // The same request after that relink splits them again. Where + // it names a pair other than the relinked rows, the store keeps + // the older link, which must lose to the newer split. + nodeCandidate, agentCandidate = reportMerge(merged.joined) + ids = splitRequestIDs{merged: merged.joined, nodeCandidate: nodeCandidate, agentCandidate: agentCandidate, nodeOwn: nodeAlone, agentOwn: agentAlone} + for _, pair := range request.exclusions(ids) { + if err := store.AddExclusion(ResourceExclusion{ResourceA: pair[0], ResourceB: pair[1], CreatedAt: time.Now().UTC()}); err != nil { + t.Fatal(err) + } + } + assertViews("after splitting again", 2, want) + }) + } + } +} + +// A node split from one agent keeps another agent's pin. The node links +// agent A, whose joined-era pin carries A's machine key for the node's +// cluster and hostname; while A is missing from a rebuild, agent B declares +// the node, and the operator has split the node from B. The pin is A's, so +// the node still completes A's key and keeps the ID it holds with A, as any +// boot window before A checks in does. +func TestOperatorSplitFromOneAgentKeepsAnotherAgentsPin(t *testing.T) { + now := time.Now().UTC() + store, err := NewSQLiteResourceStore(t.TempDir(), "default") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = store.Close() }) + node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: "lab", Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now, LinkedAgentID: "host-a"} + agentA := models.Host{ID: "host-a", Hostname: "pve1", MachineID: "aaaaaaaaaaaaaaaa", LinkedNodeID: node.ID, Status: "online", LastSeen: now} + agentB := models.Host{ID: "host-b", Hostname: "pve1", MachineID: "bbbbbbbbbbbbbbbb", LinkedNodeID: node.ID, Status: "online", LastSeen: now} + + adapter := NewMonitorAdapter(NewRegistry(store)) + adapter.PopulateFromSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agentA, agentB}}) + var joinedWithA string + for _, resource := range adapter.GetAll() { + if resource.Proxmox != nil && resource.Agent != nil && resource.Agent.AgentID == agentA.ID { + joinedWithA = resource.ID + } + } + if joinedWithA == "" { + t.Fatalf("fixture did not join the node with agent A: %+v", adapter.GetAll()) + } + splitFromB := ResourceExclusion{ + ResourceA: SourceSpecificID(ResourceTypeAgent, SourceProxmox, node.ID), + ResourceB: SourceSpecificID(ResourceTypeAgent, SourceAgent, agentB.ID), + CreatedAt: time.Now().UTC(), + } + if err := store.AddExclusion(splitFromB); err != nil { + t.Fatal(err) + } + + withoutA := models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agentB}} + rebuilt := NewRegistry(store) + rebuilt.IngestSnapshot(withoutA) + for _, resource := range rebuilt.List() { + if resource.Proxmox == nil { + continue + } + if resource.ID != joinedWithA { + t.Fatalf("node took %s while agent A was missing, want %s from A's pin", resource.ID, joinedWithA) + } + if resource.Agent != nil && resource.Agent.AgentID == agentB.ID { + t.Fatalf("node joined agent B, which the operator split it from") + } + } +} + +// An agent that stops reporting its machine ID stays split from the node. +// The joined-era pin carries the agent's old machine key for the clustered +// node's cluster and hostname, and the node record names no agent (only the +// agent declares the node), so nothing but the split drop ties the pin to +// the agent: its keyless report must not count as another agent's. +func TestOperatorSplitHoldsWhenTheAgentStopsReportingItsMachineID(t *testing.T) { + now := time.Now().UTC() + store, err := NewSQLiteResourceStore(t.TempDir(), "default") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = store.Close() }) + node := models.Node{ID: "lab-pve1", Name: "pve1", Instance: "lab", ClusterName: "lab", Host: "https://10.0.0.5:8006", Status: "online", LastSeen: now} + agent := models.Host{ID: "host-pve1", Hostname: "pve1", MachineID: "0123456789abcdef", LinkedNodeID: node.ID, Status: "online", LastSeen: now} + + adapter := NewMonitorAdapter(NewRegistry(store)) + adapter.PopulateFromSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agent}}) + merged := hostRowsByFacet(t, adapter.GetAll()) + if merged.joined == "" { + t.Fatalf("fixture did not join the node and the agent: %+v", merged) + } + alone := NewRegistry(nil) + alone.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}}) + nodeAlone := hostRowsByFacet(t, alone.List()).node + if err := store.AddExclusion(ResourceExclusion{ + ResourceA: SourceSpecificID(ResourceTypeAgent, SourceProxmox, node.ID), + ResourceB: SourceSpecificID(ResourceTypeAgent, SourceAgent, agent.ID), + CreatedAt: time.Now().UTC(), + }); err != nil { + t.Fatal(err) + } + + agent.MachineID = "" + rebuilt := NewRegistry(store) + rebuilt.IngestSnapshot(models.StateSnapshot{LastUpdate: now, Nodes: []models.Node{node}, Hosts: []models.Host{agent}}) + got := hostRowsByFacet(t, rebuilt.List()) + if got.joined != "" || got.node != nodeAlone || got.agent == "" { + t.Fatalf("keyless agent rebuild: got %+v, want the node apart on %s", got, nodeAlone) + } +} + +// splitRequestIDs are the IDs an unlink or report-merge of a joined node and +// agent can name: the merged row, its SourceTargets candidates, and the IDs +// the two rows hold once split. +type splitRequestIDs struct { + merged, nodeCandidate, agentCandidate, nodeOwn, agentOwn string +} + +func splitAlways(splitRequestIDs) bool { return true } + +type nodeAgentRows struct { + node string // a row with the Proxmox facet only + agent string // a row with the agent facet only + joined string // a row with both +} + +func hostRowsByFacet(t *testing.T, resources []Resource) nodeAgentRows { + t.Helper() + var rows nodeAgentRows + set := func(slot *string, id string) { + if *slot != "" { + t.Fatalf("two host rows of one kind: %s and %s", *slot, id) + } + *slot = id + } + for _, resource := range resources { + if resource.Type != ResourceTypeAgent { + continue + } + switch { + case resource.Proxmox != nil && resource.Agent != nil: + set(&rows.joined, resource.ID) + case resource.Proxmox != nil: + set(&rows.node, resource.ID) + case resource.Agent != nil: + set(&rows.agent, resource.ID) + } + } + return rows +} + // A host agent past its reporting lease is offline. Its source sighting is // stale too, and the stale pass used to rank that above offline, so the // machine reached the frontend as a warning with its last report rendered diff --git a/internal/unifiedresources/registry_views_branchcov0722pm_test.go b/internal/unifiedresources/registry_views_branchcov0722pm_test.go index e54b00e62b..2f874e37e9 100644 --- a/internal/unifiedresources/registry_views_branchcov0722pm_test.go +++ b/internal/unifiedresources/registry_views_branchcov0722pm_test.go @@ -203,7 +203,7 @@ func TestBranchcov0722PM(t *testing.T) { } // Register the exclusion between the two IDs and re-run; the // closure's lookup must now return true and block the merge. - rr.exclusions[exclusionKey("agent-proxmox-pve1", "agent-runtime-pve1")] = struct{}{} + rr.exclusions[exclusionKey("agent-proxmox-pve1", "agent-runtime-pve1")] = time.Now() got := rr.ListForPresentation() if len(got) != 2 { t.Fatalf("expected exclusion to keep both host views split, got %d: %+v", len(got), got)