mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-24 04:33:17 +00:00
Merge remote-tracking branch 'origin/main' into fix/disk-heat-remaining-judges
This commit is contained in:
@@ -765,6 +765,15 @@ not fenced: a live report already past its mock-mode check when mock mode is
|
||||
switched on can still be evaluated after the clear, and its alerts can persist
|
||||
until mock mode is left.
|
||||
|
||||
Every tenant monitor registers its own fixture agents, and their node links,
|
||||
in its own alert manager, so the toggle forgets the fixture agents of every
|
||||
running monitor, not only those of the monitor it was called through. A link
|
||||
left behind would keep owning its fixture node's usage alerts after mock mode
|
||||
ends. `TestLeavingMockModeReleasesFixtureAgentNodeLinksOnEveryRunningMonitor`
|
||||
in `internal/monitoring/monitor_host_agent_removal_lifecycle_test.go` switches
|
||||
through the default monitor and fails if the other tenant's node stays
|
||||
suppressed.
|
||||
|
||||
Physical-disk evidence collected by a host agent must survive projection back
|
||||
into monitoring's models. Absent evidence has to carry its declared sentinel
|
||||
rather than a zero value that reads as a real measurement: an absent
|
||||
@@ -8901,3 +8910,12 @@ agent from its last report instead of from the PVE poll, and mark it stale at
|
||||
their own heartbeat cutoff (`fleethealth.AgentStaleThreshold`, which is
|
||||
separate from the monitoring reporting lease). Agent registration, enrolment,
|
||||
install, update, removal and report identity are unchanged.
|
||||
|
||||
### Demo write guard ignores websocket upgrade headers
|
||||
|
||||
`internal/api/demo_middleware.go` changed only so a `POST`, `PUT`, `PATCH` or
|
||||
`DELETE` carrying `Upgrade: websocket` is judged by the demo read-only guard
|
||||
like any other write instead of always reaching its handler. Agent command
|
||||
websocket handshakes to `/api/agent/ws` are HTTP/1.1 `GET` requests, the only
|
||||
transport its gorilla upgrader serves, and still connect on a demo instance.
|
||||
No agent registration, enrolment, install, update or removal path changed.
|
||||
|
||||
@@ -1905,6 +1905,23 @@ well as its alerts.
|
||||
`internal/alerts/host_unraid_lifecycle_test.go` pins the clear, the holds, the
|
||||
restarted count and the pending-run and count cleanup.
|
||||
|
||||
### Proxmox node cleanup keeps platform metric alerts
|
||||
|
||||
`CleanupAlertsForNodes` runs on every Proxmox node poll (and every mock alert
|
||||
tick) and removes alerts whose `Node` is not a current Proxmox node or PBS
|
||||
instance. Kubernetes, TrueNAS and vSphere metric alerts carry a cluster or
|
||||
platform host in `Node` and a display label such as `TrueNAS Disk` or
|
||||
`Kubernetes Pod` in `metadata.resourceType`, which the preserve list's
|
||||
platform words never matched. The cleanup used to delete them silently on
|
||||
each poll while their reducer incident stayed firing, so the next evaluation
|
||||
re-created each one and handed it to the alert callback again as a new
|
||||
alert. `shouldPreserveAlertOutsideNodeCleanup` now keeps any alert carrying
|
||||
`metadata.platformType`, or whose label maps through
|
||||
`CanonicalAlertResourceType` to a Kubernetes, TrueNAS or vSphere type.
|
||||
`TestCleanupAlertsForNodesKeepsPlatformMetricAlerts` in
|
||||
`internal/alerts/alerts_test.go` pins both signals, and its
|
||||
sibling pins that a guest alert on a removed Proxmox node is still removed.
|
||||
|
||||
### Configured flapping thresholds remain reachable
|
||||
|
||||
Every accepted positive `FlappingThreshold`, including values above ten, must
|
||||
@@ -2783,6 +2800,22 @@ derived alert read-model and Last 24 Hours stat refresh for
|
||||
shared acknowledgement owner instead of keeping its own alert mutation fork.
|
||||
Future overview action behavior should extend that shared acknowledgement hook
|
||||
instead of putting acknowledge mutations back into render shells.
|
||||
Overview ages and bands read the shared clock. `AlertOverviewAlertCard.tsx`
|
||||
passes its own `useRelativeTimeNow` reading as `now` to
|
||||
`formatAlertOverviewStartedAgo` and `getMetricAlertPresentation`, and the Last
|
||||
24 Hours count in `useAlertOverviewState.ts` reads the same clock, so all three
|
||||
measure from the wall clock. The hook no longer exposes a minute `tick`: read
|
||||
as "now", that tick measured a card mounted between ticks from up to a minute
|
||||
earlier, so an alert raised 70 seconds ago read "this minute", an alert
|
||||
raised since the last tick stayed out of the 24h count until the next one, a
|
||||
reading already past the 10-minute stale cut-off led the card as live, and the
|
||||
stale flip and 24h drop landed up to a minute late. A mounted card now turns stale,
|
||||
and an alert leaves the 24h count, within one 30-second tick with no data
|
||||
change. The delivery diagnoses refresh is a server read and keeps its own
|
||||
minute interval rather than following the 30-second clock.
|
||||
`AlertOverviewAlertCard.clock.test.tsx` pins the mid-tick mount, the stale flip
|
||||
and the 24h drop under fake timers, and `useAlertOverviewState.test.tsx` pins
|
||||
the diagnoses cadence.
|
||||
Render-heavy alert overview ownership now routes through
|
||||
`frontend-modern/src/features/alerts/AlertOverviewStatsCards.tsx`,
|
||||
`frontend-modern/src/features/alerts/AlertOverviewActiveAlertsSection.tsx`,
|
||||
|
||||
@@ -7651,6 +7651,17 @@ read-only demo account. That same hidden read-side boundary includes `GET` and
|
||||
`HEAD` reads for `/api/admin/users` and manual discovery at `/api/discover`;
|
||||
public demo mode may block writes generically, but it must not reveal that
|
||||
admin-user inventory or manual-discovery read routes exist.
|
||||
That generic write block keys on the request method alone. The demo
|
||||
middleware admits `GET`, `HEAD` and `OPTIONS` plus the login, logout, AI
|
||||
execute and OIDC login/callback allowlist; HTTP/1.1 websocket handshakes for
|
||||
`/ws` and `/api/agent/ws` are `GET` requests and connect through that read
|
||||
branch, so an `Upgrade: websocket` or `Connection: Upgrade` header never
|
||||
carries a `POST`, `PUT`, `PATCH` or `DELETE` past the guard. A demo deployment that signs
|
||||
visitors in as the configured admin, as the public demo does, has only this
|
||||
guard between that session and admin writes such as `/api/system/mock-mode`.
|
||||
`TestContract_DemoModeUpgradeHeadersDoNotExemptWrites` in
|
||||
`internal/api/contract_test.go` signs in as that admin through the full router
|
||||
and pins both halves.
|
||||
That shared infrastructure install boundary now also preserves copied shell
|
||||
command payload continuity: any privilege-escalation wrapper applied at
|
||||
`frontend-modern/src/components/Settings/InfrastructureInstallerSection.tsx`
|
||||
|
||||
@@ -6678,6 +6678,11 @@ icon/content layout, and action-link chrome. The
|
||||
outdated-sensor notices, and the
|
||||
`platform-inline-notice-local-amber-shell` pattern guard blocks future
|
||||
`platformPage` files from reintroducing page-local amber notice shells.
|
||||
A notice's render predicate reads what its page hydrates on every tab the
|
||||
notice renders on. `collectOutdatedSensorSetupNodes` lists the PVE nodes that
|
||||
carry the registry's `proxmox.sensorSetupOutdated` verdict and takes no disk
|
||||
rows, because only the Proxmox Storage tab loads them; a predicate over them
|
||||
left the notice silent on Overview.
|
||||
|
||||
Alert incident-event filter containers, labels, and chips must now route
|
||||
through the shared presentation helpers in
|
||||
@@ -7189,6 +7194,12 @@ shell stays in `frontend-modern/src/features/alerts/OverviewTab.tsx`, while
|
||||
alert stats, filtered ordering, and single/bulk acknowledge runtime behavior.
|
||||
Future overview control flow should extend that hook rather than restoring
|
||||
action timers or acknowledge mutations to the tab shell.
|
||||
The overview follows the shared-clock rule for relative ages: the alert card
|
||||
reads `useRelativeTimeNow` for its started age and the live reading's stale
|
||||
cut-off, and the hook reads it for the Last 24 Hours count. A hook-local
|
||||
minute signal used as "now" is a table-local interval under that rule; the
|
||||
hook keeps a minute interval only to refresh delivery diagnoses, a server read
|
||||
whose cadence is not tied to the age clock.
|
||||
Render-heavy overview ownership now lives in
|
||||
`frontend-modern/src/features/alerts/AlertOverviewStatsCards.tsx`,
|
||||
`frontend-modern/src/features/alerts/AlertOverviewActiveAlertsSection.tsx`,
|
||||
@@ -7911,10 +7922,12 @@ renders above the chart.
|
||||
TrueNAS disk rows follow it too: the drawer's Temperature row reads
|
||||
"(last known)" with the reason as title and no heat tone, and the one-line
|
||||
summary leaves a retained reading out because it has no room for the reason.
|
||||
The Proxmox outdated sensor setup notice (`features/platformPage/sensorSetup.ts`)
|
||||
counts a SATA or SAS disk as having a temperature only when the reading is
|
||||
current, so a retained one cannot hide the notice for a disk Pulse cannot read
|
||||
now.
|
||||
The Proxmox outdated sensor setup notice reads the registry's per-node
|
||||
`proxmox.sensorSetupOutdated` verdict, which counts a SATA or SAS disk as having
|
||||
a temperature only when the reading was collected now
|
||||
(`diskinventory.TemperatureCollected` in
|
||||
`internal/unifiedresources/proxmox_sensor_setup.go`), so a retained one cannot
|
||||
hide the notice for a disk Pulse cannot read now.
|
||||
|
||||
The focused browser proofs are
|
||||
`frontend-modern/src/features/patrol/__tests__/patrolRunAcceptance.test.ts`,
|
||||
|
||||
@@ -1899,11 +1899,33 @@ a live report or poll already past its mock-mode check when mock mode is
|
||||
switched on can still evaluate after that clear, and its alerts can persist
|
||||
until mock mode is left.
|
||||
|
||||
The fence and the clear are per monitor while the mode is process-wide. A
|
||||
tenant monitor other than the one `SetMockMode` was called on neither drains
|
||||
nor clears, so fixture alerts it raised outlive the switch as they did before
|
||||
the fence; switching mock mode with several tenant monitors running is not
|
||||
covered.
|
||||
The mode is process-wide while each monitor keeps its own fence, alert manager,
|
||||
fixture agents and state, and the server runs one monitor per organization, so
|
||||
`SetMockMode` switches every running monitor, whichever tenant's monitor it is
|
||||
called on (the mock-mode API calls the requesting tenant's, the demo-fixture
|
||||
licence sync the default organization's). `Start` joins a process-wide set of
|
||||
running monitors under `mockModeSwitchMu`, and leaves it when its loop returns.
|
||||
A switch flips the mode once, then ends each monitor's epoch, waits for its
|
||||
admitted calls, clears its alerts, forgets its fixture agents when leaving mock
|
||||
mode and resets its state, and only then restarts the mode-dependent runtime of
|
||||
each monitor in the set. `Start` sets its runtime context before it joins and
|
||||
chooses the mock metrics sampler or discovery while it joins, under the same
|
||||
lock, so no switch lands between reading the mode and starting the runtime for
|
||||
it, and a switch that reaches a monitor not yet joined leaves the runtime to
|
||||
its `Start`. A monitor that missed switches between `New` and `Start` first
|
||||
leaves what it holds: `New` records the mode and the number of switches made so
|
||||
far, and `Start` compares both, so even a round trip that brought the flag back
|
||||
drops the fixture alerts a read path raised meanwhile, as well as the live
|
||||
alerts `New` restored outside mock mode. A monitor that is not running is
|
||||
switched only when the switch is made through it; a call for the mode the flag
|
||||
already holds still moves any reached monitor whose alerts belong to the other
|
||||
mode, and leaves the rest alone.
|
||||
|
||||
Release builds reach several tenant monitors in mock mode only on an instance
|
||||
running `DemoMode` with the `demo_fixtures` entitlement (release
|
||||
`ValidateEnablement` refuses mock fixtures otherwise) and with more than one
|
||||
organization's monitor running; the licence sync switches only for the
|
||||
default organization.
|
||||
|
||||
The proofs park a real evaluation at a fixed point, switch modes and release
|
||||
it. `TestLeavingMockModeRefusesTheRestOfAnInFlightPass` holds a mock pass at its
|
||||
@@ -1933,7 +1955,25 @@ previous-state carry returns fixture nodes or guests. All eight are in
|
||||
`TestStoreRefreshTakesItsMockModeScopeBeforeReadingState` in
|
||||
`internal/monitoring/canonical_guardrails_test.go` rejects a call that reads
|
||||
state in an argument ahead of `m.mockModeFence.begin()` and pins the
|
||||
end-epoch-then-clear order in `SetMockMode`.
|
||||
end-epoch-then-clear order in `endMockModeEpoch`, which `SetMockMode` runs for
|
||||
every monitor it reaches.
|
||||
|
||||
`TestLeavingMockModeClearsEveryRunningTenantMonitor` and
|
||||
`TestEnteringMockModeClearsEveryRunningTenantMonitor` start the default and one
|
||||
other tenant monitor through `MultiTenantMonitor`, wait until both have joined,
|
||||
raise alerts in both, switch through one of them and check both. Against a
|
||||
switch that reached only its own monitor, the default monitor kept its fixture
|
||||
alerts (69 to 104 across runs) and all 69 fixture agents after leaving mock
|
||||
mode through the other tenant, and that tenant kept its live alert after mock
|
||||
mode was entered through the default monitor. `TestMonitorStartedAfterAMockModeSwitchLeavesTheModeItWasBuiltIn`
|
||||
fails if `Start` joins without leaving the mode its monitor was built in, and
|
||||
`TestSetMockModeAlignsAMonitorWhoseModeTheFlagAlreadyHolds` fails if a call for
|
||||
the flag's current mode returns without moving such a monitor, and
|
||||
`TestMockModeSwitchLeavesAnUnjoinedMonitorsRuntimeToItsStart` fails if a switch
|
||||
starts the runtime of a monitor whose `Start` has not joined, and
|
||||
`TestMonitorStartedAfterAMockModeRoundTripDropsWhatItRaisedMeanwhile` fails if
|
||||
`Start` compares only the mode. All six are in
|
||||
`internal/monitoring/monitor_mock_alerts_test.go`.
|
||||
|
||||
The disk evaluation runs on the mock alert tick, not the poller's disk
|
||||
interval (five minutes by default, `PhysicalDiskPollingMinutes` per instance).
|
||||
@@ -3175,7 +3215,20 @@ Monitoring retains one current status per target and location, using server
|
||||
receipt time for remote freshness while keeping the agent-authored observation
|
||||
time as evidence metadata. A stale or disconnected agent path derives as
|
||||
indeterminate at read time; slow or fast agent clocks cannot manufacture or
|
||||
conceal a disconnect. Aggregation is conservative: all reachable paths are
|
||||
conceal a disconnect. A local path beside other locations that has not run
|
||||
within the same window derives as indeterminate too ("no recent local check").
|
||||
The target takes its freshness from its newest location, so otherwise a
|
||||
failing remote report kept an old local pass counted as reachable and the
|
||||
check read available with current evidence. For the same reason an available
|
||||
check's evidence lasts only as long as its newest reachable path, not a
|
||||
fresher path that failed. When the external-probe entitlement lapses, a
|
||||
single remote result stands in for the local location until the local poller
|
||||
runs, with the earlier of its agent check time and the server's receipt time
|
||||
as its check time, so a fast agent clock cannot keep it current. An aggregate
|
||||
over several locations never stands in: a target without a retained local
|
||||
observation reads as never checked, and not connected, until the local poller
|
||||
runs.
|
||||
Aggregation is conservative: all reachable paths are
|
||||
healthy, mixed reachable and failed/unknown paths are degraded, all current
|
||||
paths unreachable are unavailable, and no reachable path with incomplete or
|
||||
indeterminate coverage is unknown. Only aggregate unavailability advances the
|
||||
|
||||
@@ -2720,6 +2720,24 @@ must remain distinguishable without hover. Workloads supplies this row metadata,
|
||||
not local table CSS or a duplicate renderer; non-filesystem rows keep the compact
|
||||
default. Verification covers the opt-in at this composition boundary and actual
|
||||
long-path wrapping in the shared renderer's browser proof.
|
||||
A guest's linked Pulse Agent can stop reporting while Proxmox keeps polling the
|
||||
VM. `useWorkloads` copies `resource.agent.stale` onto the row as `agentStale`,
|
||||
and sets `disksFromAgent` when Proxmox reported no guest filesystems and the
|
||||
agent's own `disks` filled in. When that agent is stale, `GuestDrawerOverview.tsx`
|
||||
reads the Guest RAID card as last known through the shared `RaidCard`
|
||||
`lastKnownReason` prop with `HOST_AGENT_STOPPED_REPORTING_REASON`, puts a status
|
||||
line over muted Network rows, reads agent-supplied Filesystems rows through the
|
||||
existing `last-known` disk read state, and marks the Pulse Agent row as stopped
|
||||
reporting. Agent-supplied filesystem rows follow the agent's freshness rather
|
||||
than the Proxmox `diskStatusReason`, which describes Proxmox's own guest reads,
|
||||
so a live agent's rows keep their usage and bars. Those rows read as
|
||||
unavailable once the guest is stopped, whether its status or the Proxmox
|
||||
`vm-stopped` reason says so, and Proxmox's own guest filesystems keep their
|
||||
Proxmox read state. The two flags ride the existing row mapping and
|
||||
add no query or poll; `GuestDrawer.test.tsx` and `useWorkloads.test.ts` pin
|
||||
both sources. The workload table's disk cell and the drawer History still read
|
||||
the aggregate disk metric and the Proxmox reason, and the agent's OS, addresses
|
||||
and physical disk health carry no stale cue yet.
|
||||
Guest, node, and Docker-host drawer headers follow the same frontend-primitives dependency
|
||||
boundary for collapse: Workloads owns which inline row is selected and the
|
||||
close handler, while
|
||||
|
||||
@@ -944,6 +944,15 @@ Patrol coverage, schedule, finding, and approval state belongs on Patrol-owned
|
||||
surfaces or explicit Patrol affordances; Proxmox overview must not treat it as
|
||||
backup coverage, restore readiness, PBS verification/protection proof, or a
|
||||
replacement for the Proxmox Backups tab and workload Backup column.
|
||||
The outdated sensor setup notice above the Proxmox tabs is temperature
|
||||
guidance, not storage or recovery evidence. It reads the registry's per-node
|
||||
`proxmox.sensorSetupOutdated` verdict from the node rows the active tab already
|
||||
loads (Overview, Storage, Replication and Backups load nodes; Ceph and Mail do
|
||||
not, so it does not render there), so the Overview query must not add
|
||||
`physical_disk` rows to feed it; the derived `ProxmoxData` flag opens no storage
|
||||
or recovery path.
|
||||
`ProxmoxPageSurface.contract.test.tsx` pins the notice on Overview with no disk
|
||||
rows and an Overview query without `physical_disk`.
|
||||
The Proxmox inline estate totals are likewise an adjacent frontend-primitives
|
||||
and unified-resource projection, not backup or recovery evidence.
|
||||
`ProxmoxPageSurface` must flow the already-loaded workload inventory into the
|
||||
@@ -6886,3 +6895,14 @@ migration or recovery authority is added or moved.
|
||||
The performance report and reporting runtime snapshot handlers apply the same
|
||||
test before tabulating a disk temperature. That changes only which held value a
|
||||
report shows, not any storage or recovery path.
|
||||
|
||||
### Demo write guard ignores websocket upgrade headers
|
||||
|
||||
`internal/api/demo_middleware.go` no longer exempts every request carrying
|
||||
`Upgrade: websocket` from the demo read-only guard. On a `DEMO_MODE` instance a
|
||||
`POST`, `PUT`, `PATCH` or `DELETE` with websocket handshake headers is now
|
||||
judged like any other write: outside the login, logout, AI execute and OIDC
|
||||
allowlist it gets the generic demo `403`, or `404` on a hidden route, and that
|
||||
includes config import at `/api/config/import`. HTTP/1.1 websocket handshakes
|
||||
are `GET` requests and still connect. No storage, retention, backup, migration
|
||||
or recovery path is added or moved.
|
||||
|
||||
@@ -2987,6 +2987,36 @@ or force every provider type back into the active route payload. Docker,
|
||||
Kubernetes, TrueNAS, and VMware surface contract tests pin their route query
|
||||
maps, and the shared hook test pins facet normalization.
|
||||
|
||||
### Proxmox node verdicts over disks arrive on the node
|
||||
|
||||
The Proxmox page hydrates one query per workflow, and only its Storage tab
|
||||
loads `physical_disk` rows, so a node verdict that depends on the node's disks
|
||||
is derived in the registry and carried on the node, never rebuilt in the page
|
||||
from disk rows. `ProxmoxData.SensorSetupOutdated` (`proxmox.sensorSetupOutdated`
|
||||
on REST and websocket rows) is that verdict for the outdated sensor setup
|
||||
notice. `refreshProxmoxSensorSetupLocked`
|
||||
(`internal/unifiedresources/proxmox_sensor_setup.go`) runs after
|
||||
`buildChildCounts` on every ingest path and gives every Proxmox node an
|
||||
explicit true or false (other resources omit it), because browser facet merges
|
||||
read an omitted field as a partial snapshot and would keep an earlier true. It
|
||||
is true when the node's temperature payload is available and in the legacy
|
||||
`sensors -j` format (`models.Temperature.LegacySensorsFormat`) and a disk whose
|
||||
canonical parent is the node has no reading collected now
|
||||
(`diskinventory.TemperatureCollected`) and a type whose temperature a PVE node
|
||||
gets only through SMART: `sata`, `sas`, or the `hdd` and `ssd` form factors
|
||||
Proxmox's own inventory reports. Proxmox also types a non-rotational USB device
|
||||
`ssd`, so a USB device without SMART counts. A retained reading does not count
|
||||
as current, including one kept while a host agent reports the disk in standby;
|
||||
a reading a linked host agent collects now clears the disk. NVMe disks never
|
||||
set it, because kernel hwmon reports them even on a legacy setup, and a host
|
||||
agent the registry has not linked to the node contributes no disks. The
|
||||
verdict reflects the registry's node and disk state at each ingest.
|
||||
`TestProxmoxNodeSensorSetupOutdatedFromItsDisks` and
|
||||
`TestProxmoxNodeSensorSetupOutdatedFollowsWhereTheReadingArrives` in
|
||||
`internal/unifiedresources/registry_test.go` pin the rule, the linked-agent
|
||||
case, the explicit false on the wire and the presented JSON;
|
||||
`resourceStateAdapters.test.ts` pins that a false clears a merged true.
|
||||
|
||||
### Canonical REST facets preserve realtime workload evidence
|
||||
|
||||
The frontend REST projection retains the complete source-authored Proxmox
|
||||
@@ -3431,6 +3461,20 @@ It owns probe status, incidents, history, evidence, and the outgoing `checks`
|
||||
relationship. The matched resource carries every correlated check in the canonical
|
||||
`availabilityChecks` facet, keyed by saved target id, while `availability`
|
||||
remains an additive singular compatibility summary selected from that set.
|
||||
The summary is the worst check: a confirmed outage first, then any other
|
||||
observed failure, then an unchecked, ambiguous or unresolved check, then a
|
||||
passing one, ties going to the first saved target id. A confirmed outage is an
|
||||
enabled check whose observed failures reach its failure threshold, with an
|
||||
unavailable aggregate when one is reported and an outcome that is not
|
||||
indeterminate. That is the gate the availability poller applies before it
|
||||
raises `availability_unreachable`, so a probe agent that stops reporting keeps
|
||||
its old failure count but confirms nothing. `availabilityOutageConfirmed` in
|
||||
`internal/unifiedresources/availability.go` holds that rule for both the
|
||||
ranking and `EvaluateResourceHealth`, which reads the summary. A confirmed
|
||||
outage on any attached check therefore makes the resource's health critical
|
||||
even when a check that sorts earlier has failed below its threshold, with the
|
||||
`availability_failed` reason unless a critical alert explains the verdict
|
||||
first.
|
||||
Adding a second explicit or unambiguously correlated check must retain both
|
||||
source-owned endpoint rows, project both facets onto the same resource, and
|
||||
emit one `checks` relationship per target from the check to that resource.
|
||||
@@ -6191,9 +6235,20 @@ pass and manual links both apply that function:
|
||||
the best of equal ones. A quiet source drops out of that decision.
|
||||
- Once every source is quiet, an `offline` verdict survives and any other
|
||||
verdict reads `warning`. The best of those wins.
|
||||
- A current facet sighting carries no verdict (the PBS host-agent association,
|
||||
an availability check projected onto its target). It counts as `online` only
|
||||
when no current source has a verdict, and reads `warning` once quiet.
|
||||
- A current facet sighting carries no verdict (the PBS host-agent association).
|
||||
It counts as `online` only when no current source has a verdict, and reads
|
||||
`warning` once quiet.
|
||||
- Availability checks projected onto a monitored resource rank like that facet,
|
||||
but their shared sighting is not their verdict. `availabilityChecksProveOnline`
|
||||
judges each check at the pass's `now` by its own evidence window
|
||||
(`Evidence.ValidUntil`: two poll intervals for a single local check, the
|
||||
probe report window for a remote or multi-location one). An enabled,
|
||||
passing check with current evidence proves the resource answers and reads
|
||||
`online` once one of the resource's own sources has gone quiet and none that
|
||||
is current has a verdict. Every other check abstains, current or quiet: a
|
||||
failing check proves only that one port or service does not answer, and a
|
||||
pass whose evidence lapsed proves nothing now. A check's own row keeps the ordinary
|
||||
rules.
|
||||
|
||||
The stale pass used to read delivery as the verdict. A Proxmox node the cluster
|
||||
reported offline on a live poll came back `online` once its linked agent fell
|
||||
@@ -6217,24 +6272,52 @@ online stays `online` when its linked agent stops reporting. A live agent keeps
|
||||
a node `online` when the Proxmox poll reports it offline, which the Proxmox
|
||||
nodes table reads with `connectionHealth: error` as a stale provider. When a
|
||||
guest's or container's only source goes quiet, a running one stays `warning`
|
||||
and a stopped one stays `offline`.
|
||||
and a stopped one stays `offline`. In each quiet case a projected availability
|
||||
check with current evidence lifts the resource to `online`, as described
|
||||
below.
|
||||
|
||||
The sighting still reads `stale`, so health keeps its `telemetry_stale` reason
|
||||
and monitored-system reasons are unchanged. In-memory clones keep the verdicts,
|
||||
including a facet's missing one. A serialized copy carries none, so
|
||||
`IngestResources` gives each sighting of a copy that lost them all the stored
|
||||
resource status; a merged row's separate source verdicts do not survive that
|
||||
round trip. A manual link that joins two resources reported by one source keeps
|
||||
round trip, and a copy whose status a passing check supplied gives that status
|
||||
to every source. A manual link that joins two resources reported by one source keeps
|
||||
the fresher sighting. `registry_merge_policy_test.go` pins the node,
|
||||
poller-expired, guest, round-trip, copy, agent, Docker, Kubernetes,
|
||||
mixed-source and manual-link cases, and the aggregation table.
|
||||
|
||||
Availability checks still count by delivery. With every source that has a
|
||||
verdict quiet, a current check keeps its target `online` even when the check
|
||||
fails, as it did before. Several checks share one sighting whose freshness
|
||||
comes from whichever check was projected last, so an availability verdict
|
||||
needs each check's own freshness, including local checks that miss their
|
||||
cadence, and has to survive manual links. That is left to a follow-up.
|
||||
Availability checks used to count by delivery too. With every source that
|
||||
has a verdict quiet, a current failing check kept a node `preserveOrExpireNodes`
|
||||
expired `online`, and a quiet one lifted it to `warning`. Several checks share
|
||||
one sighting whose freshness comes from whichever check was projected last, so
|
||||
a failing check that just ran could also vouch for another check's old pass.
|
||||
Each check is now judged by its own evidence. A failing probe no longer holds
|
||||
an expired node `online`, and it never makes a guest read stopped to the
|
||||
consumers that count `offline` as stopped (the workloads summary, the Proxmox
|
||||
page counts). The failure stays on the check's own row, which raises an
|
||||
outage incident once the check confirms one (past its failure threshold, from
|
||||
every location). A check observed from several locations relies on the poller
|
||||
ageing out a location that stopped running and lasting an available check's
|
||||
evidence only as long as its newest reachable path (monitoring.md), so one
|
||||
location's fresh failure cannot lend its freshness to another location's old
|
||||
pass.
|
||||
|
||||
The verdict is read from the checks on the resource when status is
|
||||
aggregated. Projecting, re-running or retargeting a check also re-applies the
|
||||
stale pass's rule to every target it touches whose status that pass owns,
|
||||
because a check is not a delivery from the target's own sources and the
|
||||
resources API replays checks after its stale pass. The pass owns a status only
|
||||
through the target's own sightings: the checks' sighting going quiet neither
|
||||
hands it the target nor, once removed, takes the target away, so a target
|
||||
whose own sources have not gone quiet keeps the status they gave it. A check
|
||||
retargeted away therefore takes its verdict with it at once. A
|
||||
manual link keeps only the primary's own checks, so the merged resource is
|
||||
judged by the checks it shows and never by the linked resource's fresher
|
||||
sighting. `registry_test.go` pins the expired-node and quiet-guest cases, both
|
||||
projection orders, re-judging on check changes, ownership through the
|
||||
target's own sightings and the manual link; the aggregation table pins the
|
||||
check rules.
|
||||
|
||||
### Canonical object drawer hierarchy
|
||||
|
||||
@@ -6441,9 +6524,52 @@ conflict): the agent's row may be the slot's previous occupant, retained by a
|
||||
silent agent, and the join would hand the replacement that disk's serial,
|
||||
readings and canonical resource. A hardware identity match still joins on its
|
||||
own, and without a WWN on either side the path fallback stands. The seeded
|
||||
source mapping reused after a restart is a separate path this does not cover.
|
||||
`TestRegistrySASPathJoinRefusesContradictingWWN` covers the stale row and both
|
||||
same-disk shapes; monitoring's SMART merge applies the matching guard first.
|
||||
source mapping a rehydrated registry reuses applies the same refusal (next
|
||||
section). `TestRegistrySASPathJoinRefusesContradictingWWN` covers the stale
|
||||
row and both same-disk shapes; monitoring's SMART merge applies the matching
|
||||
guard first.
|
||||
|
||||
### Seeded slot mapping refuses a replaced disk
|
||||
|
||||
`ingestRecord` honours a source key's existing mapping, such as one
|
||||
`IngestResources` seeds from persisted unified resources, before any
|
||||
identity matching. Proxmox keys a disk by its slot
|
||||
(`ProxmoxPhysicalDiskSourceID`), so a disk swapped into the slot arrived
|
||||
under the previous disk's key, and `mergeInto` kept that disk's canonical ID
|
||||
and, where an agent report had been merged in, its serial, WWN, temperature
|
||||
and failed health. A Proxmox disk observation now refuses the mapping when
|
||||
the two carry hardware identities naming different disks
|
||||
(`proxmoxDiskSlotHoldsAnotherDisk`: `HardwareIdentityConflict` after
|
||||
`HardwareIdentityMatch`). Only fields reported on both sides count, so a
|
||||
missing serial or WWN, including Proxmox's literal `unknown`, is no evidence.
|
||||
Differing serials are no evidence when either side is SAS, because Proxmox
|
||||
may report the SAS address there (#1595), or when the mapped disk was merged
|
||||
with an agent report, whose serial `mergeInto` may have kept (collection
|
||||
status does not reliably say which serial it holds), unless Proxmox read the
|
||||
drive's own serial too, by the rule monitoring's
|
||||
`hostAgentSMARTSerialComparable` applies: NVMe on both sides, or an agent
|
||||
SATA disk under Proxmox vendor `ATA` that Proxmox does not type `usb`, not a
|
||||
SCSI designator or a USB bridge's serial. A refused key falls through to the linked-disk join,
|
||||
`findMatch` and `chooseNewID` like an unmapped one, except that none of them
|
||||
may merge it into the refused disk: an ID that lands on it is replaced with
|
||||
the key's source-specific ID. Agreement still wins, as it does in identity
|
||||
matching, so drives that share a serial they do not own, such as a USB
|
||||
bridge's, read as one disk: a second swap in the same enclosure joins the
|
||||
first replacement, which reported the same bridge serial. That belongs to
|
||||
identity matching, not this guard. The refused disk keeps its Proxmox
|
||||
facet and is not updated, so until the registry is rebuilt both disks answer
|
||||
to the slot's PVE alert reference, and canonical reference resolution treats
|
||||
that reference as ambiguous rather than handing it to either. A later
|
||||
rehydration may seed the slot key to the old disk again, and the refusal
|
||||
repeats. No current caller ingests Proxmox disk records over a seeded
|
||||
mapping: the monitor rebuilds a fresh registry each poll, and the resource
|
||||
API, host-continuity and Patrol registries seeded from unified resources do
|
||||
not replay Proxmox disk records. This closes the registry contract, not a
|
||||
reported symptom. `TestRegistrySeededSlotMappingRefusesReplacedDisk` covers
|
||||
SATA (with and without a new WWN), SAS, USB-enclosure and Proxmox-only
|
||||
replacements, plus eight same-disk controls (including a SCSI designator
|
||||
from an agent without collection status and a USB bridge serial under
|
||||
Proxmox vendor `ATA`), each through two rehydrations.
|
||||
|
||||
### Drawer tab selection survives a transient snapshot change (#1723)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user