mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Judge source merges by the ingest's stale thresholds (#2717)
Stacked on #2715 (its commit is the parent of this one); either can land first. ## Problem `mergeInto` passed nil thresholds to `mergeMetrics`, so every source merge into an existing row (snapshot and record ingest alike) judged metric freshness by the registry defaults, while the stale pass and manual links used the monitor's configured thresholds. Two visible results: - A Proxmox node polled every two minutes (four-minute threshold) merged with its auto-linked agent showed the silent agent's CPU and memory, while the stale pass called the node's poll current. - A supplemental refresh (`MonitorAdapter.PopulateSupplementalRecords`, and the read-state overlay) of a linked vSphere VM kept the in-guest agent's memory over vSphere's current reading, where a full rebuild from the same records shows vSphere's. ## Change - Snapshot and record ingest hold the caller's thresholds on the registry while they run (the same per-pass pattern as the guest loop's agent index), and `mergeInto` passes them to `mergeMetrics`. The snapshot helpers that reach `mergeInto` keep their signatures. - The stale pass and the metric merge share one threshold rule, `sourceStaleThreshold`, so the merge also honours a sighting's expected update interval (slow Proxmox disk inventory), as the stale pass already did. - Outside mock mode the monitor configures only Proxmox, PBS and PMG, so those are the production sources this changes; mock mode also configures TrueNAS, vSphere and availability. ## Proof - `TestMonitorAdapterSourceMergesUseConfiguredStaleThresholds`: rebuild, supplemental refresh and overlay of a linked vSphere VM, plus a node with an auto-linked agent, through `NewMonitorAdapterWithStaleThresholds`. - `TestSourceMergesJudgeFreshnessByTheIngestStaleThresholds`: snapshot node+agent merge with and without thresholds. - `TestMetricMergeFreshnessMatchesTheStalePass`: the merge and the stale pass agree for default, configured and expected-interval sightings. - Negative control with #2715's `registry.go`: the supplemental, overlay, node+agent, snapshot and expected-interval cases fail; the rebuild case and the defaults case pass on both. - `go test ./internal/unifiedresources/... ./internal/monitoring/... ./internal/api/resourceapi/... ./internal/mock/...` on pulse-dev.
This commit is contained in:
@@ -2699,7 +2699,18 @@ truthfulness, not native thaw, containing-release or workload acceptance.
|
||||
ingestion so canonical host IDs stay stable across restarts (see the
|
||||
unified-resources contract's durable identity-pin obligation). Rebuild
|
||||
paths added to the adapter must keep that persistence step; ephemeral
|
||||
snapshot-bridge adapters stay read-only.
|
||||
snapshot-bridge adapters stay read-only. The rebuild, the live
|
||||
supplemental refresh and the read-state overlay ingest records with the
|
||||
adapter's configured stale thresholds, because record ingest joins
|
||||
operator links and the freshness gate of every metric merge, a link's or
|
||||
a source's into an existing row, reads them (unified-resources contract,
|
||||
"Operator links reach record-ingested resources"). Regression
|
||||
coverage:
|
||||
`TestMonitorAdapterJoinsLinkedRecordsWithConfiguredStaleThresholds` and
|
||||
`TestMonitorAdapterSourceMergesUseConfiguredStaleThresholds` in
|
||||
`internal/unifiedresources/monitor_adapter_read_state_test.go` and
|
||||
`TestManualLinkToSupplementalGuestHoldsWithAndWithoutContinuity` in
|
||||
`internal/monitoring/issue1913_host_continuity_test.go`.
|
||||
|
||||
11. The TrueNAS provider projects pools with `Storage.Topology` fixed to
|
||||
`pool` and the ZFS data vdev layout in `Storage.VDevLayout`. The
|
||||
|
||||
@@ -1609,7 +1609,12 @@ Source freshness is a separate unified-resource status input, not a fixed
|
||||
global health timer. Snapshot rebuilds, resource seeding, supplemental record
|
||||
overlays, and cloned read-state overlays must preserve the monitoring-provided
|
||||
stale thresholds for Proxmox, PBS, and PMG sources so resources do not flap to
|
||||
warning/degraded between successful configured poll cycles.
|
||||
warning/degraded between successful configured poll cycles. A source merge
|
||||
into an existing row judges metric freshness by the ingest's thresholds and
|
||||
the stale pass's own threshold rule, so the merge's freshness gate never
|
||||
calls a source current that the stale pass marks stale, or the reverse. The
|
||||
gate decides only where no stronger rule does: a hypervisor-managed guest
|
||||
keeps its platform CPU even while that source is stale.
|
||||
|
||||
Service-discovery readiness is a unified-resource payload contract, not a
|
||||
drawer-local decoration. Resource list/detail payloads that expose a
|
||||
@@ -2813,7 +2818,9 @@ application resource-provider or WebSocket lifecycle.
|
||||
ID and drops the superseded pin row (`ApplyCanonicalIDSuccessions`
|
||||
in `internal/unifiedresources/canonical_id_succession.go`).
|
||||
Successions never fire while the old ID still belongs to a live
|
||||
resource, never follow a contradicting machine key (a reinstalled
|
||||
resource or to one a manual link folded into its primary (see
|
||||
"Operator links reach record-ingested resources"), never follow a
|
||||
contradicting machine key (a reinstalled
|
||||
machine mints fresh, it does not absorb the old host's operator
|
||||
state), and never rewrite change-journal rows. Known limitation:
|
||||
machine-keyless hosts whose history already merged under a collapsed
|
||||
@@ -2894,8 +2901,9 @@ application resource-provider or WebSocket lifecycle.
|
||||
`ResourceRegistry.IngestRecords` applies the same
|
||||
`ApplyCanonicalIDSuccessions` semantics as pin-driven successions
|
||||
(operator state and action audits re-key, the superseded pin row
|
||||
drops, never while the old ID belongs to a live resource, journal
|
||||
rows never rewritten). Regression coverage:
|
||||
drops, never while the old ID belongs to a live resource or one a
|
||||
manual link folded into its primary, journal rows never rewritten).
|
||||
Regression coverage:
|
||||
`TestRegistryIngestRecordsKeepsSameHostnameSystemsDistinct` /
|
||||
`TestIngestRecordsSucceedLegacyHostnameScopedCanonicalIDs` /
|
||||
`TestIngestRecordsDoNotCompleteTrueNASIdentityFromPins` in
|
||||
@@ -6596,6 +6604,75 @@ full heading. This shared drawer rule applies across resource types. The 390px
|
||||
browser qualification checks heading fit and keyboard expansion; no data model,
|
||||
authorisation, action dispatch or resource admission contract changes.
|
||||
|
||||
### Operator links reach record-ingested resources
|
||||
|
||||
Every registry ingest path applies the operator's manual links: snapshot
|
||||
ingest, resource ingest and record ingest (`IngestRecords`,
|
||||
`IngestRecordsWithStaleThresholds` and the read-state overlay's record pass).
|
||||
Either side of a link can arrive only as a supplemental record, such as a
|
||||
vSphere or TrueNAS VM with a Pulse agent inside, or an agent on a TrueNAS host,
|
||||
and the monitor's rebuild ingests those records after the snapshot. While only
|
||||
snapshot and resource ingest applied links, the rebuilt registry behind the
|
||||
websocket broadcast and `/api/state` kept both rows. The resources API, which
|
||||
seeds through resource ingest, showed them merged, and so did the published
|
||||
inventory whenever an unrelated absent machine made continuity hydration
|
||||
clone the registry. Mock mode builds its unified view from fixtures without
|
||||
the link store, so links do not reach the mock broadcast; that boundary is
|
||||
separate.
|
||||
|
||||
- The monitor adapter passes its configured stale thresholds to record
|
||||
ingest, so a link joined there judges each side's metrics the way a
|
||||
snapshot-time link does. Ordinary source merges into an existing row use
|
||||
the same thresholds on every ingest path: snapshot and record ingest hold
|
||||
the caller's thresholds while they run, and `mergeInto` judges each
|
||||
metric's freshness with them through `sourceStaleThreshold`, the rule the
|
||||
stale pass reads. Before this, those merges used the defaults, so a
|
||||
Proxmox node polled every two minutes showed a silent auto-linked agent's
|
||||
readings while the stale pass called the node's poll current, and a
|
||||
supplemental refresh of a linked vSphere VM kept the agent's memory over
|
||||
vSphere's current reading. Outside mock mode the monitor configures only
|
||||
Proxmox, PBS and PMG; mock mode also configures TrueNAS, vSphere and
|
||||
availability. Regression coverage:
|
||||
`TestMonitorAdapterSourceMergesUseConfiguredStaleThresholds` (rebuild,
|
||||
supplemental refresh, overlay, node with auto-linked agent),
|
||||
`TestSourceMergesJudgeFreshnessByTheIngestStaleThresholds` and
|
||||
`TestMetricMergeFreshnessMatchesTheStalePass`.
|
||||
- Continuity records are the one record ingest that never joins links. A
|
||||
saved enrollment only fills an absent machine, so it stays its own row
|
||||
instead of lending its linked guest an offline verdict and an old agent
|
||||
payload.
|
||||
- A link merge keeps the TrueNAS and vSphere payloads the primary lacks, so
|
||||
an agent chosen as primary over a TrueNAS system still carries the
|
||||
system's facet, and the system's pools re-parent to the merged row.
|
||||
- The registry remembers each resource a manual link folded into its
|
||||
primary. A folded resource is still observed, so neither pin-driven nor
|
||||
record-declared succession may succeed its ID. An agent-type primary
|
||||
carries the folded resource's machine keys and is pinned with them, and
|
||||
succeeding the folded ID re-keyed the link onto the primary itself, which
|
||||
split the pair on the next rebuild. That already broke snapshot links whose
|
||||
primary is a machine-keyless agent-type row, such as a Proxmox node.
|
||||
- Fold history is registry-local: resource seeding and overlay clones do not
|
||||
carry it. A reference to the folded resource's canonical ID, such as an
|
||||
availability check configured against an agent later linked into its
|
||||
guest, does not resolve to the primary, so the check loses its target in
|
||||
the broadcast as it already did in the resources API.
|
||||
|
||||
`registry_merge_policy_test.go` pins the record links in both directions
|
||||
through two rebuilds and the live supplemental refresh, and checks that a
|
||||
registry seeded from the monitor's listing through resource ingest, as the
|
||||
resources API seeds, lists the same rows
|
||||
(`TestManualLinksJoinRecordIngestedResourcesInMonitorRebuild`). The same file
|
||||
pins the node case (`TestManualLinkKeepsItsEndpointsThroughIdentityPinPersistence`),
|
||||
record-declared succession (`TestRecordDeclaredSuccessionSparesALinkFoldedResource`)
|
||||
and both payloads (`TestManualLinkKeepsProviderPayloadsThePrimaryLacks`).
|
||||
`monitor_adapter_read_state_test.go`
|
||||
(`TestMonitorAdapterJoinsLinkedRecordsWithConfiguredStaleThresholds`) pins the
|
||||
thresholds on the rebuild, supplemental and overlay paths, and
|
||||
`internal/monitoring/issue1913_host_continuity_test.go`
|
||||
(`TestManualLinkToSupplementalGuestHoldsWithAndWithoutContinuity`) pins the
|
||||
published inventory with no continuity, an unrelated absent machine and the
|
||||
linked agent's own saved enrollment.
|
||||
|
||||
### Provider link network corroboration
|
||||
|
||||
An existing reciprocal host/provider link remains authoritative, including a
|
||||
|
||||
Reference in New Issue
Block a user