Compose retained disk policy and attribution work with current main

Preserve exact retained Core ancestry while retaining the accepted Assistant target, webhook confidentiality, log-viewer and installer repairs. Qualify the composed source against the original seven-package, inventory-control and real embed/API/Patrol recipe; source acceptance remains separate from native recovery.

Change-source: pulse-maintainer
This commit is contained in:
pulse-triage[bot]
2026-10-09 11:42:30 +01:00
21 changed files with 808 additions and 37 deletions
@@ -9259,6 +9259,18 @@ carrying a prefixed attribute or namespace declaration named like a SAML
attribute is refused. No agent
registration, enrolment, install, update or removal path changed.
### Mock-mode read views fold linked fixture agents
`internal/monitoring/monitor.go` changed only so the mock-mode unified view
applies the operator's manual links (unified-resources contract, "Mock-mode
unified view applies operator links"). A fixture agent the operator linked
into a guest now folds into that guest in mock read views, as a linked agent
already does in live mode, so host snapshots built from them
(`HostsSnapshot`) stop listing it while the guest carries its facet. Agent
registration, enrolment, report admission, removal, continuity and the
fixture agent set itself are unchanged, and the view writes nothing to the
resource store.
### Demo write guard ignores websocket upgrade headers
`internal/api/demo_middleware.go` changed only so a `POST`, `PUT`, `PATCH` or
@@ -9321,3 +9333,14 @@ machine setting. Cleanup restores only the captured prior setting, even when
owned server teardown fails; an unconfirmed stop fails and retains the handle
rather than reporting clean teardown. Real Windows execution and full terminal
cleanup evidence still have to be returned by the existing native CI job.
### Reconciled node-agent relink fold bookkeeping
A declared node-agent join can consume one split row before its newer
operator relink is applied. The relink records that side's ID and source in
the current holder-bound `ManualLinkFold` index, so identity-pin succession
keeps the operator decision and the resources API seed retains the same fold
for report-merge. No synthetic telemetry, identity key, enrollment or service
link is added. `TestOperatorSplitOverridesProxmoxNodeAgentLink` checks that
the monitor and its re-ingested API seed retain the fold across all split
shapes, then survive pin persistence and a repeated split.
@@ -422,6 +422,17 @@ linked node or guest) replaces the per-type trigger, and an override or agent
default that switches the host's alerts off leaves no heat to judge. A disk no
agent reports gets the per-type policy. The global alerts switch and the agent
alert-type switch stop `CheckHost` without changing this policy, as before.
A TrueNAS disk (`alerts.IsTrueNASDiskResource`, the disks the unified evaluator
alerts on as `truenas-disk`) skips that walk, because its TrueNAS system carries
a synthetic `Agent.AgentID` that would pass for a host agent:
`physicalDiskTemperatureLimits` resolves it through
`ThresholdProvider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType)`
(the alert manager's `TrueNASDiskTemperatureThreshold`, the `truenas-disk`
tiers of `effectiveAlertPolicyNoLock`), so the disk's own override, then the
TrueNAS-wide TrueNAS Disks value, then the per-type policy decide, as for its
temperature alert, and a disabled override or TrueNAS Disks default leaves no
heat. A host override under the system's synthetic ID no longer reaches its
disks, and Disable all TrueNAS silences the alert without changing this policy.
The run state carries the provider (`patrolRuntimeState.thresholdProvider`). With no provider the factory alert
configuration applies. A disk is hot from its trigger: a triage warning, a disk
issue and an AI chat "needing attention" entry, as its temperature alert and
@@ -432,9 +443,13 @@ verification unknown rather than recovered. Scoped runs keep the provider, and a
provider set before Patrol starts is handed to it. Below the trigger Patrol flags nothing, so an NVMe at 63C stays
quiet and a SATA disk at 56C is flagged. Proofs:
`internal/ai/patrol_disk_temperature_test.go` (a real registry with one
overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`),
`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy` and
`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` in
overridden agent in `TestPatrolJudgesAgentDisksByTheAgentDiskTemperatureOverride`,
and TrueNAS fixture disks under a per-disk override, a TrueNAS-wide value and a
host override on the system ID, one with a retained reading, in
`TestPatrolJudgesTrueNASDisksByTheirAlertTiers`),
`TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy`,
`TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides` and
`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` in
`internal/ai/alert_threshold_adapter_test.go`.
Retained summaries disclose that point and bucket timestamps describe returned
@@ -4475,3 +4475,27 @@ resubmission and concurrent response encoding/update under the race detector.
Existing snapshot, persistence, partial-update and threshold tests remain
required. This is source ownership acceptance, not installed notification or
guest-agent recovery evidence.
### TrueNAS disk heat policy for Patrol and Assistant
`Manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)` resolves the
`truenas-disk` tiers of `effectiveAlertPolicyNoLock`: the disk's canonical or
identity-resolved override, then TrueNAS Disks defaults, then disk-type policy.
A disabled disk/default or non-positive trigger leaves no heat policy.
Global/platform alert switches silence alert evaluation without changing this
policy, matching existing agent-disk semantics. Callers receive a copy; config
normalisation, saved Off/custom/inherited values and alert severity are unchanged.
`IsTrueNASDiskResource` uses the evaluator's classification, including merged
agent/TrueNAS disks. Patrol/Assistant do not walk those disks to a synthetic
TrueNAS system agent and accidentally apply an unrelated host override.
Retained temperature readings remain unjudged, not affirmative recovery.
Verification: `TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts`
and `TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` in
`internal/alerts/threshold_resolution_shared_test.go` cover tier, identity,
Off/disabled, untyped, clone and nil-manager compatibility.
`TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers` and
`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` cover adapter, Patrol triage,
verification and Assistant context while preserving agent-host overrides.
Frontend table/drawer tiers remain a separate Web-owned change; this backend
repair does not establish browser or native appliance acceptance.
@@ -6502,3 +6502,21 @@ and REST row counts with and without the split, and
`TestBroadcastPresentationCoalescePrefersTheListingReadState` pins the
listing read state over an older store generation and the store's exclusions
for a mock-style view.
### Reconciled node-agent relink fold bookkeeping
A declared node-agent join can consume one split row before its newer
operator relink is applied. The relink records that side's ID and source in
the current holder-bound `ManualLinkFold` index, so identity-pin succession
keeps the operator decision and the resources API seed retains the same fold
for report-merge. No synthetic telemetry, identity key, enrollment or service
link is added. `TestOperatorSplitOverridesProxmoxNodeAgentLink` checks that
the monitor and its re-ingested API seed retain the fold across all split
shapes, then survive pin persistence and a repeated split.
Mock-link lifecycle validation drives the owning ingest/rebuild boundary
after unlink and relink: read hydration intentionally reuses a registry
generation younger than two seconds. The decision takes effect in the next
loaded generation, not retroactively in the listing already served. The
fixture seed keeps default thresholds while the configured live adapter
retains its four-minute Proxmox threshold.
@@ -6950,11 +6950,11 @@ of durable state through it.
as before. With links each monitor builds its own linked snapshot whenever
the fixture data or links change (concurrent cache misses can each build
one).
- Exclusions are not carried, as before: the fixture graph makes its own
merges before the view seeds it, so fixtures that identity matching merged
stay merged ("Unlink replaces the pair's operator link"). Presentation
coalescing still honours them, because the broadcast takes the resource
store adapter's exclusions for every store-less view.
- Exclusions are not carried by the fixture registry, as before. The
broadcast's presentation coalescing consults the monitor adapter's
exclusions, while the resources API's `ListForPresentation` consults its
own store-backed registry's exclusions. Neither can undo an automatic
identity merge already made inside the store-free fixture graph.
- An agent linked into a guest leaves the agent listings (`Hosts()` lists
agent-type rows only) and the guest carries its facet, as in live mode.
@@ -7362,9 +7362,11 @@ agent's SMART inventory as the node's disk fallback when the Proxmox disk
query fails, the agent's disk-exclude patterns on the node's disks, and the
node's linked agent that guest discovery, agent deployment and service
discovery act on.
Physical disks inside one host still join through
`resolveLinkedPhysicalDisk` without reading exclusions; a node split puts the
two sides' disks under different parents, so they no longer meet there.
Physical disks inside one host join through `resolveLinkedPhysicalDisk`,
which now honours their operator exclusions ("Operator split separates disks
inside a linked host" above, upstream #2748). A node split also puts the two
sides' disks under different parents, so they no longer meet there. The node
split and the disk split remain independent operator decisions.
Cluster- and hostname-derived IDs carry no source, so an exclusion recorded
against another resource that derives the node's own ID (a Docker Swarm host
reporting no machine key, named like the node in a swarm named like its
@@ -7395,11 +7397,12 @@ already has the link applied, so it cannot split the pair itself: an unlink
shows on REST, `/api/state` and the websocket from the monitor's next
registry rebuild, the same generation in which a new link reaches the
broadcast. REST applies a new link at once, because its own registry applies
the store's links on top of the seed. The mock-mode view applies the links
the resource store's current generation loaded ("Mock-mode unified view
applies operator links"), so mock mode follows the same timing. Mock fixtures
that identity matching merged stay merged on every surface: the fixture graph
is built without the store's exclusions.
the store's links on top of the seed. The mock-mode view now applies the
current monitor generation's operator links too ("Mock-mode unified view
applies operator links" above), so a mock unlink likewise needs that
monitor rebuild to remove the fold from the seed. Mock fixtures that identity
matching merged stay merged on every surface: the fixture graph is built
without the store's exclusions.
The websocket broadcast coalesced host views with
`CoalescePresentationHostResources`, which honours no exclusions, while the
@@ -7508,3 +7511,27 @@ system into a VM across repeated record ingests, and
`TestManualLinkFoldRepeatedPairKeepsEarlierSources` refolds a side with fewer
sources than it first brought. `TestResourceAPIReportMergeExcludesRegistryLinkFolds`
keeps the handler on the registry's fold record.
### Reconciled node-agent relink fold bookkeeping
A declared node-agent join can consume one split row before its newer
operator relink is applied. The relink records that side's ID and source in
the current holder-bound `ManualLinkFold` index, so identity-pin succession
keeps the operator decision and the resources API seed retains the same fold
for report-merge. No synthetic telemetry, identity key, enrollment or service
link is added. `TestOperatorSplitOverridesProxmoxNodeAgentLink` checks that
the monitor and its re-ingested API seed retain the fold across all split
shapes, then survive pin persistence and a repeated split.
### TrueNAS disk policy consumers use the disk's alert identity
The AI/Patrol physical-disk consumers classify TrueNAS disks through
`alerts.IsTrueNASDiskResource`, the same classifier as unified alert evaluation.
They resolve temperature policy under the disk's canonical resource ID, not its
parent system's synthetic agent ID. Merged agent/TrueNAS disks retain that
classification; ordinary agent disks still follow their reporting host's policy.
Registry identity, source precedence and retained-reading semantics are unchanged.
`TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator` and
`TestPatrolJudgesTrueNASDisksByTheirAlertTiers` exercise the unified registry
projection and its consumer boundaries. Table/drawer frontend acceptance remains
with Web, not established by these backend tests.
+20
View File
@@ -79,6 +79,26 @@ func (a *AlertThresholdAdapter) GetDiskTemperatureThreshold(host alerts.DiskTemp
} else {
threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
}
return diskTemperatureTriggerAndClear(threshold)
}
// GetTrueNASDiskTemperatureThreshold returns the temperature alert trigger and
// clear values for one TrueNAS disk, resolved exactly as its temperature alert
// resolves them (alerts.Manager.TrueNASDiskTemperatureThreshold): the disk's
// own override, then the TrueNAS-wide value, then the per-type policy.
func (a *AlertThresholdAdapter) GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (float64, float64) {
var threshold *alerts.HysteresisThreshold
if a.manager != nil {
threshold = a.manager.TrueNASDiskTemperatureThreshold(resourceID, diskType)
} else {
threshold = alerts.DefaultDiskTemperatureThreshold(diskType)
}
return diskTemperatureTriggerAndClear(threshold)
}
// diskTemperatureTriggerAndClear unpacks a disk temperature threshold, with a
// nil or non-positive trigger meaning off (0, 0).
func diskTemperatureTriggerAndClear(threshold *alerts.HysteresisThreshold) (float64, float64) {
if threshold == nil || threshold.Trigger <= 0 {
return 0, 0
}
@@ -108,6 +108,39 @@ func TestAlertThresholdAdapter_DiskTemperatureFollowsAlertPolicy(t *testing.T) {
}
}
// A TrueNAS disk reaches Patrol through its own alert tiers: its override,
// then the TrueNAS-wide value, then the per-type policy.
func TestAlertThresholdAdapter_TrueNASDiskTemperatureFollowsItsAlertTiers(t *testing.T) {
if trigger, clear := NewAlertThresholdAdapter(nil).GetTrueNASDiskTemperatureThreshold("disk-any", "nvme"); trigger != 70 || clear != 65 {
t.Fatalf("no manager: nvme = %v/%v, want the factory 70/65", trigger, clear)
}
mgr := alerts.NewManager()
t.Cleanup(mgr.Stop)
a := NewAlertThresholdAdapter(mgr)
if trigger, clear := a.GetTrueNASDiskTemperatureThreshold("disk-plain", "sata"); trigger != 55 || clear != 50 {
t.Fatalf("no TrueNAS-wide value: sata = %v/%v, want the per-type 55/50", trigger, clear)
}
cfg := mgr.GetConfig()
cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}
cfg.Overrides = map[string]alerts.ThresholdConfig{
"disk-raised": {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},
"disk-off": {Disabled: true},
}
mgr.UpdateConfig(cfg)
for id, want := range map[string][2]float64{
"disk-raised": {75, 70},
"disk-plain": {62, 57},
"disk-off": {0, 0},
} {
trigger, clear := a.GetTrueNASDiskTemperatureThreshold(id, "nvme")
if trigger != want[0] || clear != want[1] {
t.Errorf("%s nvme = %v/%v, want %v/%v", id, trigger, clear, want[0], want[1])
}
}
}
// A host's Disk Temp override reaches Patrol for that host's disks only, and a
// host whose alerts are switched off has no disk heat policy at all.
func TestAlertThresholdAdapter_DiskTemperatureHonoursHostOverrides(t *testing.T) {
+6
View File
@@ -73,6 +73,12 @@ func (m *mockThresholdProvider) GetDiskTemperatureThreshold(_ alerts.DiskTempera
return 0, 0
}
// GetTrueNASDiskTemperatureThreshold judges a TrueNAS disk by the same
// per-type table, as a TrueNAS disk with no override or TrueNAS-wide value is.
func (m *mockThresholdProvider) GetTrueNASDiskTemperatureThreshold(_ string, diskType string) (float64, float64) {
return m.GetDiskTemperatureThreshold(alerts.DiskTemperatureHost{}, diskType)
}
type mockUnifiedResourceProvider struct {
UnifiedResourceProvider
getAllFunc func() []unifiedresources.Resource
+6
View File
@@ -71,6 +71,12 @@ type ThresholdProvider interface {
// agent reports. A host with no ID means no agent reports the disk. A zero
// trigger means disk temperature alerting is off for it.
GetDiskTemperatureThreshold(host alerts.DiskTemperatureHost, diskType string) (trigger, clear float64)
// GetTrueNASDiskTemperatureThreshold returns the disk temperature alert
// trigger and clear values (°C) for one TrueNAS disk, given its resource
// ID and type: the disk's own override, then the TrueNAS-wide value, then
// the per-type policy, as its temperature alert resolves them. A zero
// trigger means disk temperature alerting is off for it.
GetTrueNASDiskTemperatureThreshold(resourceID, diskType string) (trigger, clear float64)
}
// AlertResolver provides the ability to review and resolve alerts
+1 -1
View File
@@ -2491,7 +2491,7 @@ func patrolPhysicalDiskRows(snap patrolRuntimeState, scopedSet map[string]bool)
temperature: tools.SplitDiskTemperature(r.PhysicalDisk.Temperature, r.PhysicalDisk.Collection),
smartEvidence: unifiedPhysicalDiskSMARTIssueParts(r.PhysicalDisk.SMART),
}
row.temperatureLimits = snap.diskTemperatureLimits(physicalDiskTemperatureHost(r, owners), row.diskType)
row.temperatureLimits = physicalDiskTemperatureLimits(snap.thresholdProvider, r, owners)
rows = append(rows, row)
}
return rows
+47 -5
View File
@@ -22,12 +22,53 @@ type diskTemperatureLimits struct {
// that host's Disk Temp override applies, or from the factory alert
// configuration when Patrol has no threshold provider.
func diskTemperatureLimitsFor(provider ThresholdProvider, host alerts.DiskTemperatureHost, diskType string) diskTemperatureLimits {
var trigger, clear float64
if provider != nil {
trigger, clear = provider.GetDiskTemperatureThreshold(host, diskType)
} else if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
trigger, clear = threshold.Trigger, threshold.Clear
return newDiskTemperatureLimits(provider.GetDiskTemperatureThreshold(host, diskType))
}
return factoryDiskTemperatureLimits(diskType)
}
// trueNASDiskTemperatureLimitsFor resolves the policy for one TrueNAS disk the
// way its temperature alert does, from the user's alert configuration, so the
// disk's own override and the TrueNAS-wide value apply, or from the factory
// alert configuration when Patrol has no threshold provider.
func trueNASDiskTemperatureLimitsFor(provider ThresholdProvider, resourceID, diskType string) diskTemperatureLimits {
if provider != nil {
return newDiskTemperatureLimits(provider.GetTrueNASDiskTemperatureThreshold(resourceID, diskType))
}
return factoryDiskTemperatureLimits(diskType)
}
// physicalDiskTemperatureLimits resolves the policy for one physical disk
// resource under the alert that judges its heat. A TrueNAS disk has its own
// temperature alert, judged by the disk's override, the TrueNAS-wide value and
// the per-type policy; its TrueNAS system carries a synthetic agent ID, so the
// host walk would wrongly treat that system as a host agent. Any other disk is
// judged as a SMART disk of the host agent that reports it.
func physicalDiskTemperatureLimits(provider ThresholdProvider, disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) diskTemperatureLimits {
diskType := ""
if disk.PhysicalDisk != nil {
diskType = disk.PhysicalDisk.DiskType
}
if alerts.IsTrueNASDiskResource(disk) {
return trueNASDiskTemperatureLimitsFor(provider, disk.ID, diskType)
}
return diskTemperatureLimitsFor(provider, physicalDiskTemperatureHost(disk, owners), diskType)
}
// factoryDiskTemperatureLimits is the policy for a disk of the given type
// under the factory alert configuration.
func factoryDiskTemperatureLimits(diskType string) diskTemperatureLimits {
if threshold := alerts.DefaultDiskTemperatureThreshold(diskType); threshold != nil {
return newDiskTemperatureLimits(threshold.Trigger, threshold.Clear)
}
return diskTemperatureLimits{}
}
// newDiskTemperatureLimits builds limits from an alert trigger and clear
// value. A non-positive trigger is off; a clear value missing or above the
// trigger leaves no band below it.
func newDiskTemperatureLimits(trigger, clear float64) diskTemperatureLimits {
if trigger <= 0 {
return diskTemperatureLimits{}
}
@@ -72,7 +113,8 @@ func physicalDiskOwnerIndex(urp UnifiedResourceProvider) map[string]unifiedresou
// disk, the host its disk temperature alerts are evaluated under: the agent on
// the machine the disk is parented to or, for a disk parented to a storage
// pool (Unraid array and cache disks), the agent on the pool's machine. A disk
// no agent reports gets the hostless policy.
// no agent reports gets the hostless policy. TrueNAS disks do not come here;
// physicalDiskTemperatureLimits judges them by their own alert.
func physicalDiskTemperatureHost(disk unifiedresources.Resource, owners map[string]unifiedresources.Resource) alerts.DiskTemperatureHost {
parentID := disk.ParentID
for hops := 0; parentID != nil && hops < 2; hops++ {
+145
View File
@@ -8,6 +8,7 @@ import (
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/truenas"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/aicontracts"
"github.com/rcourtman/pulse-go-rewrite/pkg/diskinventory"
@@ -343,3 +344,147 @@ func TestPhysicalDiskTemperatureHostFollowsTheParentChain(t *testing.T) {
t.Errorf("parentless disk: host = %+v, want hostless", got)
}
}
// trueNASHeatRegistry is a TrueNAS system whose pool holds four disks beside
// the two overridden host agents of overriddenAgentDisksRegistry: nvme0n1 at
// 72C, nvme1n1 at 63C, SATA sda at 60C and SATA sdb, whose 80C reading is
// retained from before standby. The user saved a TrueNAS-wide 62C (clear 57),
// raised nvme0n1's own trigger to 75C (clear 70), and left a 50C Disk Temp
// override under the TrueNAS system's synthetic agent ID. It returns the disk
// resources by name.
func trueNASHeatRegistry(t *testing.T) (*unifiedresources.ResourceRegistry, *AlertThresholdAdapter, map[string]unifiedresources.Resource) {
t.Helper()
registry, _ := overriddenAgentDisksRegistry(t)
now := time.Now()
disk := func(name, transport string, temperature int) truenas.Disk {
return truenas.Disk{
ID: "disk-" + name, Name: name, Pool: "tank", Status: "ONLINE", Model: "Model " + name,
Serial: "SERIAL-" + strings.ToUpper(name), Temperature: temperature, Transport: transport,
}
}
records := truenas.FixtureRecords(truenas.FixtureSnapshot{
CollectedAt: now,
System: truenas.SystemInfo{Hostname: "truenas-main", Healthy: true, CollectedAt: now},
Pools: []truenas.Pool{{ID: "pool-tank", Name: "tank", Status: "ONLINE"}},
Disks: []truenas.Disk{
disk("nvme0n1", "nvme", 72), disk("nvme1n1", "nvme", 63),
disk("sda", "sata", 60), disk("sdb", "sata", 80),
},
})
for i := range records {
if meta := records[i].Resource.PhysicalDisk; meta != nil && records[i].Resource.Name == "sdb" {
meta.Collection = &diskinventory.CollectionStatus{Temperature: diskinventory.Unavailable("truenas", "disk is in standby")}
}
}
registry.IngestRecords(unifiedresources.SourceTrueNAS, records)
adapter := unifiedresources.NewUnifiedAIAdapter(registry)
disks := make(map[string]unifiedresources.Resource)
for _, r := range adapter.GetByType(unifiedresources.ResourceTypePhysicalDisk) {
if alerts.IsTrueNASDiskResource(r) {
disks[r.Name] = r
}
}
systemAgentID := ""
for _, r := range adapter.GetAll() {
if r.Agent != nil && r.Agent.Platform == "truenas" {
systemAgentID = r.Agent.AgentID
}
}
if len(disks) != 4 || systemAgentID == "" {
t.Fatalf("TrueNAS disks = %v, system agent ID = %q; want four disks under one system", disks, systemAgentID)
}
mgr := alerts.NewManager()
t.Cleanup(mgr.Stop)
cfg := mgr.GetConfig()
cfg.TrueNASDiskDefaults.Temperature = &alerts.HysteresisThreshold{Trigger: 62, Clear: 57}
cfg.Overrides = map[string]alerts.ThresholdConfig{
"agent-cool": {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 80, Clear: 75}},
systemAgentID: {DiskTemperature: &alerts.HysteresisThreshold{Trigger: 50, Clear: 45}},
disks["nvme0n1"].ID: {Temperature: &alerts.HysteresisThreshold{Trigger: 75, Clear: 70}},
}
mgr.UpdateConfig(cfg)
return registry, NewAlertThresholdAdapter(mgr), disks
}
// Patrol judges a TrueNAS disk by its own temperature alert's tiers, not as a
// disk of the TrueNAS system's synthetic host agent: nvme0n1's override keeps
// it quiet at 72C, the TrueNAS-wide 62C flags nvme1n1 at 63C (under the NVMe
// trigger of 70C) and spares SATA sda at 60C (over the SATA trigger of 55C),
// and the 50C host override under the system ID reaches none of them. A
// retained reading stays unjudged. Agent disks keep their host's override.
func TestPatrolJudgesTrueNASDisksByTheirAlertTiers(t *testing.T) {
registry, provider, disks := trueNASHeatRegistry(t)
state := newPatrolRuntimeStateWithProviders(models.StateSnapshot{}, nil, unifiedresources.NewUnifiedAIAdapter(registry))
state.thresholdProvider = provider
type want struct {
limits diskTemperatureLimits
issue bool
}
expected := map[string]want{
disks["nvme0n1"].ID: {limits: diskTemperatureLimits{trigger: 75, clear: 70}},
disks["nvme1n1"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}, issue: true},
disks["sda"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},
disks["sdb"].ID: {limits: diskTemperatureLimits{trigger: 62, clear: 57}},
}
rows := patrolPhysicalDiskRows(state, nil)
if len(rows) != 6 {
t.Fatalf("rows = %+v, want four TrueNAS disks and two agent disks", rows)
}
for _, row := range rows {
w, trueNAS := expected[row.id]
if !trueNAS {
// overriddenAgentDisksRegistry: agent-cool's 76C NVMe under its 80C
// override, agent-plain's 72C NVMe over the NVMe trigger.
w = want{limits: diskTemperatureLimits{trigger: 70, clear: 65}, issue: true}
if row.temperature.Collected == 76 {
w = want{limits: diskTemperatureLimits{trigger: 80, clear: 75}}
}
}
if row.temperatureLimits != w.limits {
t.Errorf("disk %s at %dC: limits = %+v, want %+v", row.id, row.temperature.Collected, row.temperatureLimits, w.limits)
}
if got := patrolPhysicalDiskHealthIssue(row); got != w.issue {
t.Errorf("disk %s at %dC: health issue = %v, want %v", row.id, row.temperature.Collected, got, w.issue)
}
}
flags := triageDiskHealthChecksState(state, nil)
for _, name := range []string{"nvme0n1", "sda", "sdb"} {
if flag := triageFindFlag(flags, func(f TriageFlag) bool {
return f.ResourceID == disks[name].ID && strings.Contains(f.Reason, "Disk temperature")
}); flag != nil {
t.Errorf("%s: unexpected temperature flag %+v", name, *flag)
}
}
if flag := triageFindFlag(flags, func(f TriageFlag) bool {
return f.ResourceID == disks["nvme1n1"].ID && strings.Contains(f.Reason, "Disk temperature")
}); flag == nil || flag.Threshold != 62 {
t.Errorf("nvme1n1: temperature flag = %+v, want one at the TrueNAS-wide 62C", flag)
}
if _, err := verifyMetricRecoveredState(state, PatrolThresholds{}, "disk-high", disks["sdb"].ID, "physical_disk"); !errors.Is(err, aicontracts.ErrVerificationUnknown) {
t.Errorf("retained 80C: err = %v, want ErrVerificationUnknown", err)
}
s := &Service{}
s.SetUnifiedResourceProvider(unifiedresources.NewUnifiedAIAdapter(registry))
s.SetPatrolThresholdProvider(provider)
_, section, _ := strings.Cut(s.buildUnifiedResourceContext(), "Physical Disks Needing Attention")
section, _, _ = strings.Cut(section, "\n\n")
for _, reading := range []string{"Temp: 63C", "Temp: 72C"} {
if !strings.Contains(section, reading) {
t.Errorf("attention section = %q, want the disk at %s", section, reading)
}
}
for _, reading := range []string{"Temp: 60C", "Temp: 76C", "Temp: 80C"} {
if strings.Contains(section, reading) {
t.Errorf("attention section = %q, lists a disk at %s", section, reading)
}
}
if strings.Count(section, "Temp: 72C") != 1 {
t.Errorf("attention section = %q, want only agent-plain's 72C disk, not TrueNAS nvme0n1 under its 75C override", section)
}
}
+3
View File
@@ -24,6 +24,9 @@ func (m MockThresholdProvider) GetStorageThreshold() float64 { return m.Stor
func (m MockThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 0, 0
}
func (m MockThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
return 0, 0
}
func TestCalculatePatrolThresholds_Default(t *testing.T) {
// Test default behavior (exact mode)
+1 -1
View File
@@ -417,7 +417,7 @@ func (s *Service) buildUnifiedResourceContextForModel(destinationModel string) s
health = strings.ToUpper(strings.TrimSpace(disk.PhysicalDisk.Health))
// A retained last-known reading is not current heat.
collected := tools.SplitDiskTemperature(disk.PhysicalDisk.Temperature, disk.PhysicalDisk.Collection).Collected
hot = diskTemperatureLimitsFor(thresholdProvider, physicalDiskTemperatureHost(disk, owners), disk.PhysicalDisk.DiskType).hot(collected)
hot = physicalDiskTemperatureLimits(thresholdProvider, disk, owners).hot(collected)
}
if disk.Status != unifiedresources.StatusOnline || (health != "" && health != "PASSED" && health != "UNKNOWN") || hot {
attention = append(attention, disk)
@@ -12,6 +12,7 @@ import (
alertspecs "github.com/rcourtman/pulse-go-rewrite/internal/alerts/specs"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/recovery"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
)
func boolPtr(v bool) *bool {
@@ -2135,3 +2136,239 @@ func TestHostDiskTemperatureThresholdMatchesCheckHostOverrides(t *testing.T) {
t.Fatalf("nil manager host policy = %+v, want the factory nvme 70", got)
}
}
// TrueNASDiskTemperatureThreshold is the disk heat policy for a TrueNAS disk,
// so it must agree with the disk's temperature alert on every tier: the
// disk's own override, then the TrueNAS-wide value, then the per-type policy.
// A host override under the TrueNAS system's synthetic agent ID is not one of
// them. Disable all TrueNAS silences the alert without changing the policy,
// as the agent switches do for agent disks.
func TestTrueNASDiskTemperatureThresholdMatchesTrueNASDiskAlerts(t *testing.T) {
disk := trueNASTemperatureDisk("nvme0n1", "nvme", 0)
const canonicalDiskID = "physical-disk:canonical-nvme0n1"
for _, tc := range []struct {
name string
mutate func(*AlertConfig)
setup func(*Manager)
want *HysteresisThreshold
off bool
silenced bool
}{
{name: "per-type policy", want: &HysteresisThreshold{Trigger: 70, Clear: 65}},
{
name: "raised per-type trigger",
mutate: func(cfg *AlertConfig) {
cfg.DiskTempByType = map[string]HysteresisThreshold{"nvme": {Trigger: 75, Clear: 70}}
},
want: &HysteresisThreshold{Trigger: 75, Clear: 70},
},
{
name: "TrueNAS-wide value",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
},
want: &HysteresisThreshold{Trigger: 62, Clear: 57},
},
{
name: "per-disk override beats the TrueNAS-wide value",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{Trigger: 62, Clear: 57}
cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75, Clear: 70}}}
},
want: &HysteresisThreshold{Trigger: 75, Clear: 70},
},
{
name: "per-disk override without a clear value",
mutate: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{Trigger: 75}}}
},
want: &HysteresisThreshold{Trigger: 75, Clear: 70},
},
{
name: "per-disk override under the canonical identity",
mutate: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{canonicalDiskID: {Temperature: &HysteresisThreshold{Trigger: 78, Clear: 73}}}
},
setup: func(m *Manager) {
m.SetResourceIntentIdentityResolver(func(resourceID string) (string, bool) {
return canonicalDiskID, resourceID == disk.ID
})
},
want: &HysteresisThreshold{Trigger: 78, Clear: 73},
},
{
name: "host override under the TrueNAS system ID does not reach its disks",
mutate: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{"truenas-main": {DiskTemperature: &HysteresisThreshold{Trigger: 50, Clear: 45}}}
},
want: &HysteresisThreshold{Trigger: 70, Clear: 65},
},
{
name: "per-disk override switches the disk off",
mutate: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Temperature: &HysteresisThreshold{}}}
},
off: true,
},
{
name: "per-disk override disables the disk's alerts",
mutate: func(cfg *AlertConfig) {
cfg.Overrides = map[string]ThresholdConfig{disk.ID: {Disabled: true}}
},
},
{
name: "TrueNAS-wide off",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Temperature = &HysteresisThreshold{}
},
off: true,
},
{
name: "TrueNAS Disks defaults disabled",
mutate: func(cfg *AlertConfig) {
cfg.TrueNASDiskDefaults.Disabled = true
},
},
{
name: "agent Disk Temp default off",
mutate: func(cfg *AlertConfig) {
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{}
},
off: true,
},
{
name: "Disable all TrueNAS keeps the policy",
mutate: func(cfg *AlertConfig) {
cfg.DisableAllTrueNAS = true
},
want: &HysteresisThreshold{Trigger: 70, Clear: 65},
silenced: true,
},
} {
t.Run(tc.name, func(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
if tc.mutate != nil {
tc.mutate(&cfg)
}
configureUnifiedEvalManager(t, m, cfg)
if tc.setup != nil {
tc.setup(m)
}
got := m.TrueNASDiskTemperatureThreshold(" "+disk.ID+" ", " NVMe")
switch {
case tc.off:
if got == nil || got.Trigger > 0 {
t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want a switched-off threshold", got)
}
case (got == nil) != (tc.want == nil) || (got != nil && *got != *tc.want):
t.Fatalf("TrueNASDiskTemperatureThreshold = %+v, want %+v", got, tc.want)
}
var alert *Alert
fires := func(temperature int) bool {
reading := disk
meta := *disk.PhysicalDisk
meta.Temperature = temperature
reading.PhysicalDisk = &meta
checkTrueNASTemperatureDisk(t, m, reading)
var firing bool
alert, firing = trueNASDiskTemperatureAlert(t, m, reading)
return firing
}
if tc.silenced {
if fires(99) {
t.Fatalf("alert fired at 99C with TrueNAS alerts disabled, active: %v", alertKeys(m))
}
return
}
if got == nil || got.Trigger <= 0 {
if fires(99) {
t.Fatalf("alert fired at 99C on a disk the policy judges no heat for, active: %v", alertKeys(m))
}
return
}
trigger := int(got.Trigger)
if fires(trigger - 1) {
t.Fatalf("alert fired at %dC, under the policy trigger %v", trigger-1, got.Trigger)
}
if !fires(trigger) {
t.Fatalf("alert stayed quiet at the policy trigger %dC, active: %v", trigger, alertKeys(m))
}
// The alert recovers at the clear value Patrol reads.
if alert.Threshold != got.Trigger || alert.Metadata["clearThreshold"] != got.Clear {
t.Fatalf("alert trigger/clear = %v/%v, want the policy %v/%v", alert.Threshold, alert.Metadata["clearThreshold"], got.Trigger, got.Clear)
}
})
}
// A disk whose type TrueNAS never reported follows the agent Disk Temp
// default, as its alert does.
t.Run("untyped disk", func(t *testing.T) {
m := newTestManager(t)
cfg := unifiedEvalBaseConfig()
cfg.TrueNASDiskDefaults = ThresholdConfig{}
cfg.AgentDefaults.DiskTemperature = &HysteresisThreshold{Trigger: 58, Clear: 53}
configureUnifiedEvalManager(t, m, cfg)
untypedDisk := trueNASTemperatureDisk("sdx", "", 58)
if got := m.TrueNASDiskTemperatureThreshold(untypedDisk.ID, ""); got == nil || got.Trigger != 58 || got.Clear != 53 {
t.Fatalf("untyped disk = %+v, want the agent default 58/53", got)
}
checkTrueNASTemperatureDisk(t, m, untypedDisk)
if _, firing := trueNASDiskTemperatureAlert(t, m, untypedDisk); !firing {
t.Fatalf("untyped disk at the agent default 58C raised no alert, active: %v", alertKeys(m))
}
})
m := configureDiskTempTypeHostManager(t)
m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme").Trigger = 1
if got := m.TrueNASDiskTemperatureThreshold(disk.ID, "nvme"); got == nil || got.Trigger != 70 {
t.Fatalf("caller mutation reached the config: %+v", got)
}
var nilManager *Manager
if got := nilManager.TrueNASDiskTemperatureThreshold(disk.ID, "sata"); got == nil || got.Trigger != 55 {
t.Fatalf("nil manager TrueNAS disk policy = %+v, want the factory sata 55", got)
}
}
// IsTrueNASDiskResource names the disks the unified evaluator judges as
// TrueNAS disks, the ones TrueNASDiskTemperatureThreshold resolves.
func TestIsTrueNASDiskResourceMatchesTheUnifiedEvaluator(t *testing.T) {
agentDisk := unifiedresources.Resource{
ID: "physical-disk:host-1/nvme0n1", Type: unifiedresources.ResourceTypePhysicalDisk,
Sources: []unifiedresources.DataSource{unifiedresources.SourceAgent},
PhysicalDisk: &unifiedresources.PhysicalDiskMeta{DiskType: "nvme"},
}
trueNASFacetDisk := agentDisk
trueNASFacetDisk.TrueNAS = &unifiedresources.TrueNASData{}
mergedDisk := agentDisk
mergedDisk.Sources = []unifiedresources.DataSource{unifiedresources.SourceAgent, unifiedresources.SourceTrueNAS}
noMetaDisk := trueNASTemperatureDisk("sdy", "sata", 40)
noMetaDisk.PhysicalDisk = nil
trueNASSystem := unifiedresources.Resource{
ID: "agent:truenas-main", Type: unifiedresources.ResourceTypeAgent,
Sources: []unifiedresources.DataSource{unifiedresources.SourceTrueNAS},
}
for name, tc := range map[string]struct {
resource unifiedresources.Resource
want bool
}{
"TrueNAS-sourced disk": {resource: trueNASTemperatureDisk("sda", "sata", 40), want: true},
"TrueNAS facet disk": {resource: trueNASFacetDisk, want: true},
"agent and TrueNAS": {resource: mergedDisk, want: true},
"no disk metadata": {resource: noMetaDisk, want: true},
"agent disk": {resource: agentDisk},
"TrueNAS system": {resource: trueNASSystem},
} {
if got := IsTrueNASDiskResource(tc.resource); got != tc.want {
t.Errorf("%s: IsTrueNASDiskResource = %v, want %v", name, got, tc.want)
}
input, ok := UnifiedResourceInputFromResource(tc.resource)
if evaluated := ok && input.Type == "truenas-disk"; evaluated != tc.want {
t.Errorf("%s: unified evaluator judges it as a TrueNAS disk = %v, want %v", name, evaluated, tc.want)
}
}
}
@@ -1,5 +1,11 @@
package alerts
import (
"strings"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
)
// trueNASDiskTemperatureDefaultNoLock returns the default tier of a TrueNAS
// disk's temperature threshold. A TrueNAS-wide value the user saved under
// TrueNAS Disks applies to every TrueNAS disk, as a host Disk Temp override
@@ -16,3 +22,39 @@ func (m *Manager) trueNASDiskTemperatureDefaultNoLock(diskType string, typeUnkno
}
return cloneThreshold(m.hostDiskTemperatureThresholdNoLock(m.config.AgentDefaults.DiskTemperature, false, diskType))
}
// IsTrueNASDiskResource reports whether the unified evaluator judges a
// resource as a TrueNAS disk (truenas-disk), whose temperature alert resolves
// its threshold through TrueNASDiskTemperatureThreshold rather than through
// the host agent that reports it.
func IsTrueNASDiskResource(resource unifiedresources.Resource) bool {
typeKey, ok := unifiedAlertResourceType(resource)
return ok && typeKey == "truenas-disk"
}
// TrueNASDiskTemperatureThreshold is the disk heat policy for one TrueNAS
// disk, resolved by the same effectiveAlertPolicyNoLock tiers its temperature
// alert uses: an override stored under the disk's resource ID, then the
// TrueNAS-wide TrueNAS Disks value, then the per-type policy. An override or
// TrueNAS Disks default that switches the disk's alerts off leaves no
// threshold (nil); a non-positive trigger is off too. Like
// HostDiskTemperatureThreshold it ignores the global and per-platform alert
// switches, which silence the alert without changing the policy. A nil
// manager resolves the factory configuration, where no TrueNAS-wide value is
// set.
func (m *Manager) TrueNASDiskTemperatureThreshold(resourceID, diskType string) *HysteresisThreshold {
if m == nil {
return DefaultDiskTemperatureThreshold(diskType)
}
m.mu.RLock()
defer m.mu.RUnlock()
thresholds := m.effectiveAlertPolicyNoLock(alertPolicyQuery{
TypeKey: "truenas-disk",
ResourceID: unifiedresources.CanonicalResourceID(resourceID),
DiskType: strings.ToLower(strings.TrimSpace(diskType)),
}).Thresholds
if thresholds.Disabled {
return nil
}
return cloneThreshold(thresholds.Temperature)
}
@@ -71,6 +71,9 @@ func (concurrencyThresholdProvider) GetStorageThreshold() float64 { return 92
func (concurrencyThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 70, 65
}
func (concurrencyThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
return 70, 65
}
type concurrencyMetricsHistoryProvider struct{}
+3
View File
@@ -162,6 +162,9 @@ func (stubThresholdProvider) GetStorageThreshold() float64 { return 92 }
func (stubThresholdProvider) GetDiskTemperatureThreshold(alerts.DiskTemperatureHost, string) (float64, float64) {
return 70, 65
}
func (stubThresholdProvider) GetTrueNASDiskTemperatureThreshold(string, string) (float64, float64) {
return 70, 65
}
type stubMetricsHistoryProvider struct{}
@@ -1,6 +1,7 @@
package monitoring
import (
"reflect"
"testing"
"time"
@@ -27,6 +28,8 @@ func TestCrossInstallationIdentitySurvivesStandaloneAddition(t *testing.T) {
{"classified member-hostname coincidence", "https://pmx1:8006", true, false},
{"unclassified FQDN connection and distinct guest domains", "https://pmx1.remote.example:8006", false, true},
{"classified FQDN connection and distinct guest domains", "https://pmx1.remote.example:8006", true, true},
{"unclassified FQDN connection and identical short guest names", "https://pmx1.remote.example:8006", false, false},
{"classified FQDN connection and identical short guest names", "https://pmx1.remote.example:8006", true, false},
} {
name, classified := tc.name, tc.classified
clusterName := ""
@@ -80,6 +83,45 @@ func TestCrossInstallationIdentitySurvivesStandaloneAddition(t *testing.T) {
t.Fatal("initial Docker report absent from read state")
}
homeCanonicalID := adapter.VMs()[0].ID()
initialHosts := m.state.GetSnapshot().Hosts
if len(initialHosts) != 1 || initialHosts[0].LinkedVMID != homeGuestID {
t.Fatalf("initial automatic guest link = %+v, want %s", initialHosts, homeGuestID)
}
initialDocker := m.state.GetSnapshot().DockerHosts
initialDockerHostID := adapter.DockerHosts()[0].ID()
initialWorkloadID := adapter.DockerContainers()[0].ID()
assertUnchangedDocker := func(stage string) {
t.Helper()
// No new Docker report has arrived: a report after link clearing
// could restore deleted inventory and hide the actual regression.
if got := m.state.GetSnapshot().DockerHosts; !reflect.DeepEqual(got, initialDocker) {
t.Fatalf("%s changed the independent Docker snapshot: got %+v, want %+v", stage, got, initialDocker)
}
if hosts, workloads := adapter.DockerHosts(), adapter.DockerContainers(); len(hosts) != 1 || len(workloads) != 1 ||
hosts[0].ID() != initialDockerHostID || workloads[0].ID() != initialWorkloadID {
t.Fatalf("%s lost Docker source identities: hosts=%+v workloads=%+v", stage, hosts, workloads)
}
listed, _, thresholds := adapter.GetAllWithMetricsTargetsAndStaleThresholds()
broadcast, ok := adapter.CoalesceForPresentation(listed, thresholds)
if !ok {
t.Fatal("store-backed Docker presentation was not available")
}
for surface, resources := range map[string][]unifiedresources.Resource{"API list": listed, "broadcast": broadcast} {
var machineID string
var workloads []unifiedresources.Resource
for _, resource := range resources {
if resource.Type == unifiedresources.ResourceTypeAgent && resource.Docker != nil && resource.Docker.HostSourceID == initialDocker[0].ID {
machineID = resource.ID
}
if resource.Type == unifiedresources.ResourceTypeAppContainer && resource.Docker != nil && resource.Docker.ContainerID == "home-workload" {
workloads = append(workloads, resource)
}
}
if machineID == "" || len(workloads) != 1 || workloads[0].ID != initialWorkloadID || workloads[0].ParentID == nil || *workloads[0].ParentID != machineID {
t.Fatalf("%s %s detached Docker workload from its reporting machine: machine=%s workloads=%+v", stage, surface, machineID, workloads)
}
}
}
standalone := config.PVEInstance{Name: "Hetzner pmx1", Host: tc.host}
m.config.PVEInstances = append(m.config.PVEInstances, standalone)
@@ -100,9 +142,19 @@ func TestCrossInstallationIdentitySurvivesStandaloneAddition(t *testing.T) {
// not discard the agent, its Docker inventory, or the original VM's data.
hostReport.Timestamp = now.Add(time.Second)
dockerReport.Timestamp = now.Add(time.Second)
assertUnchangedDocker("provider addition without agent reports")
if _, err := m.ApplyHostReport(hostReport, token); err != nil {
t.Fatal(err)
}
hosts := m.state.GetSnapshot().Hosts
wantLink := ""
if tc.fqdnGuests {
wantLink = homeGuestID
}
if len(hosts) != 1 || hosts[0].LinkedVMID != wantLink || hosts[0].LinkedNodeID != "" || hosts[0].LinkedContainerID != "" {
t.Fatalf("post-addition guest link = %+v, want %q with no node/container guess", hosts, wantLink)
}
assertUnchangedDocker("host-only report after automatic guest-link reconciliation")
if _, err := m.ApplyDockerReport(dockerReport, token); err != nil {
t.Fatal(err)
}
@@ -5925,6 +5925,71 @@ func TestMockUnifiedViewAppliesOperatorManualLinks(t *testing.T) {
if len(after) != len(before) {
t.Fatalf("mock view wrote %d change rows into the durable store", len(after)-len(before))
}
// Reconcile the mock link proposal with the current latest-decision
// rule: a fresh monitor must restore the link, and unlink/relink must
// invalidate the seed cache without a fixture tick. Mock seeds retain
// their own default thresholds, not the live adapter's configured ones.
restarted := &Monitor{
state: models.NewState(),
config: &config.Config{PVEPollingInterval: 2 * time.Minute},
resourceStore: unifiedresources.NewMonitorAdapter(unifiedresources.NewRegistry(store)),
alertManager: alerts.NewManager(),
}
t.Cleanup(restarted.alertManager.Stop)
adapter := restarted.resourceStore.(*unifiedresources.MonitorAdapter)
assertMockLink := func(wantLinked bool) time.Time {
t.Helper()
frontend := restarted.BuildBroadcastFrontendState()
var vmFound, agentFound, vmHasAgent bool
for _, resource := range frontend.Resources {
if resource.ID == vmID {
vmFound, vmHasAgent = true, len(resource.Agent) > 0
}
if resource.ID == agentID {
agentFound = true
}
}
if !vmFound || vmHasAgent != wantLinked || agentFound == wantLinked {
t.Fatalf("mock link=%t: vm=%t vmAgent=%t standaloneAgent=%t", wantLinked, vmFound, vmHasAgent, agentFound)
}
seed, freshness, thresholds := restarted.UnifiedResourceSnapshotWithStaleThresholds()
if len(thresholds) != 0 {
t.Fatalf("mock seed borrowed live thresholds: %v", thresholds)
}
if got := adapter.StaleThresholds()[unifiedresources.SourceProxmox]; got != 4*time.Minute {
t.Fatalf("live adapter threshold = %v, want configured four minutes", got)
}
rest := unifiedresources.NewRegistryWithStaleThresholds(store, thresholds)
rest.IngestResources(seed)
_, hasStandaloneAgent := rest.Get(agentID)
if hasStandaloneAgent == wantLinked {
t.Fatalf("REST seed link=%t: standalone agent=%t", wantLinked, hasStandaloneAgent)
}
if got := mock.FixtureDataVersion(); got != version {
t.Fatalf("fixture version moved from %d to %d during operator lifecycle", version, got)
}
return freshness
}
restartFreshness := assertMockLink(true)
if err := store.AddExclusion(unifiedresources.ResourceExclusion{ResourceA: vmID, ResourceB: agentID}); err != nil {
t.Fatalf("unlink: %v", err)
}
// Reads deliberately reuse a generation younger than two seconds.
// Apply the owning ingest/rebuild boundary instead of waiting for that
// throttle or claiming a recorded decision changes a previous listing.
restarted.updateResourceStore(restarted.currentStateWithScope())
unlinkFreshness := assertMockLink(false)
if !unlinkFreshness.After(restartFreshness) {
t.Fatal("unlink must advance the mock seed's freshness")
}
if err := store.AddLink(unifiedresources.ResourceLink{ResourceA: agentID, ResourceB: vmID, PrimaryID: vmID}); err != nil {
t.Fatalf("relink: %v", err)
}
restarted.updateResourceStore(restarted.currentStateWithScope())
if relinkFreshness := assertMockLink(true); !relinkFreshness.After(unlinkFreshness) {
t.Fatal("relink must advance the mock seed's freshness")
}
}
// TestMockModeDiscardsRealHostReports pins the push-side of the mock clean
@@ -290,8 +290,8 @@ func TestOperatorSplitOverridesProxmoxNodeAgentLink(t *testing.T) {
t.Helper()
for rebuild := 1; rebuild <= rebuilds; rebuild++ {
adapter.PopulateFromSnapshot(snapshot)
listed := adapter.GetAll()
broadcast, ok := adapter.CoalesceForPresentation(listed, nil)
listed, _, thresholds := adapter.GetAllWithMetricsTargetsAndStaleThresholds()
broadcast, ok := adapter.CoalesceForPresentation(listed, thresholds)
if !ok {
t.Fatal("store-backed adapter did not coalesce with its exclusions")
}
@@ -351,18 +351,25 @@ func TestOperatorSplitOverridesProxmoxNodeAgentLink(t *testing.T) {
t.Fatal(err)
}
assertViews("after relink", 2, nodeAgentRows{joined: merged.joined})
// The joined row records the relinked row it took in, so a
// report-merge of it names the relink's own pair.
folded := want.node
if folded == merged.joined {
folded = want.agent
}
recorded := false
for _, fold := range adapter.currentRegistry().ManualLinkFolds(merged.joined) {
recorded = recorded || (fold.HolderID == merged.joined && fold.FoldedID == folded)
}
if !recorded {
t.Fatalf("joined %s records no fold of relinked %s: %+v", merged.joined, folded, adapter.currentRegistry().ManualLinkFolds(merged.joined))
// The inferred join can consume the node row before manual
// links run. Its fold must still reach current succession and
// the API seed, rather than an obsolete registry-wide ID set.
seeded := NewRegistry(store)
seeded.IngestResources(adapter.GetAll())
for name, registry := range map[string]*ResourceRegistry{"monitor": adapter.currentRegistry(), "API seed": seeded} {
found := false
for _, fold := range registry.ManualLinkFolds(merged.joined) {
nodeFold := fold.HolderID == want.agent && fold.FoldedID == want.node && slices.Contains(fold.Sources, SourceProxmox)
agentFold := fold.HolderID == want.node && fold.FoldedID == want.agent && slices.Contains(fold.Sources, SourceAgent)
// A machine-keyless agent rejoins under the node's ID,
// so the same pair is folded in the opposite direction.
if nodeFold || agentFold {
found = true
}
}
if !found {
t.Fatalf("%s lost the relink's pair fold: %+v", name, registry.ManualLinkFolds(merged.joined))
}
}
// A rebuild that meets the agent first orders the joined row's
// hostnames the agent's way; persisting its pins must not