Merge reviewed occurrence-qualified alert recovery

Retain exact Core candidate d0d621a7c2 and source proof; preserve current main frontend content.

Change-source: pulse-maintainer
This commit is contained in:
pulse-triage[bot]
2026-10-03 02:13:17 +01:00
21 changed files with 729 additions and 78 deletions
@@ -8433,3 +8433,14 @@ update commands and credential authority unchanged.
`internal/api/onboarding_handlers.go` changed only the human-readable pairing
readiness messages so they name Settings > Pulse Mobile. No agent
registration, lifecycle or install path changed.
### Monitor recovery callback ownership
The monitor's external alert wiring uses a captured resolved-occurrence
snapshot for notification cancellation. Re-enrolled or refiring resources keep
their new occurrence's queued notification even when an older recovery callback
arrives late under the same ID. Agent registration, deletion/tombstones,
authorisation and continuity evidence are unchanged. The connected callback
wiring control in `monitor_notification_startup_test.go` verifies the shared
monitor boundary against real alert evaluation and local HTTP delivery; it does
not claim host re-enrolment or installed-agent acceptance.
@@ -3203,3 +3203,36 @@ upgrade prompt, and never load or save relay push settings. Licensed instances
keep the minimum-severity control and the link to Pulse Mobile settings, with
the 31 March 2027 retirement date in the copy. Webhook, email, Apprise and
dead-man destinations are unaffected.
### Occurrence-qualified asynchronous recovery delivery
Every resolution funnel passes the actual `ResolvedAlert` to the callback bus.
`SetResolvedAlertCallback` captures a deep-cloned alert plus its resolution time
before asynchronous execution, rather than reconstructing an occurrence from a
reusable ID later. Legacy ID-only subscribers retain their public IDs; a panic
in one recovery consumer does not prevent the other consumers from running.
The live monitor uses this snapshot for AI recovery and notification delivery.
`CancelResolvedAlert` removes only matching ID/start-time grouping entries and
cooldown state. `CancelByAlertOccurrence` applies the same match to persistent
pending, sending, failed and dead-letter firing rows, preserving newer
occurrences, unrelated grouped members and their operational links. Missing
start-time identity is a no-op, never a request for identifier-wide cancellation.
The existing explicit ID-wide cancellation API remains available. Delivery
gates, pending-only suppression counts, destination receipts, attempt history,
quiet hours and best-effort treatment of already-started sends are unchanged.
`operational_contract_test.go` verifies snapshot isolation across a
blocked callback and changed ID-indexed history, legacy identity and panic
isolation. `queue_occurrence_cancellation_test.go` verifies nanosecond-distinct
recurrences, grouping/cooldown preservation, missing identity and restart
readback for all four cancellable row states without deleting failed audits or
new-occurrence destination receipts. The monitor's connected tests use real PBS
evaluation, normal callbacks, the persistent queue and a local HTTP destination;
these receipts are not native provider or installed-release acceptance.
Pending cancellation is not proof that every destination missed the firing.
The monitor delegates recovery eligibility to persisted occurrence/destination
receipts even when an old retry remains or restart has lost RAM cooldown state.
The partial-destination monitor control establishes a real local HTTP 200/503
split and preserves the newer firing and the old recipient's recovery.
@@ -4634,3 +4634,33 @@ proof lives in `internal/truenas/provider_test.go`
(`TestMaxTrueNASSystemTemperaturePrefersCoresOverSynthesisedAggregate`). This
is source-level proof, not reporter acceptance or appliance confirmation
(#2122).
### Recovery callbacks retain the resolved occurrence
`wireExternalAlertCallbacks` binds the occurrence-qualified recovery callback,
not the legacy ID-only callback. The monitor uses its immutable alert/start/end
snapshot for AI recovery and for exact notification cancellation, so a delayed
recovery cannot remove a newer firing admitted under the same reusable ID.
AI receives a separate clone. An ID-only websocket removal is omitted when the
manager already exposes that ID as active; the regular state stream continues
to own current alert truth. The legacy local ID adapter resolves a snapshot once
and fails closed when no resolved record exists; it is not the live delivery
binding. No browser payload, retention or notification routing contract changes.
`monitor_alert_handling_test.go` and
`monitor_notification_startup_test.go` covers a delayed ID-adapter cancellation
with queue restart/readback, and a real PBS breach/recovery/new breach held at
the callback boundary. Both require the replacement firing's exact ID/start to
reach a loopback HTTP destination with one sent row and one cancelled old row.
No clock, reducer map or configured cooldown is altered. This establishes local
source-connected delivery, not the outstanding natural flapping cooldown check,
a native appliance result or containing stable availability.
Recovery eligibility is destination-receipt-owned, not the cancellation helper's
pending-work count or the RAM cooldown marker. After partial delivery/restart,
a cancelled retry at one destination cannot suppress a recovery to another
that actually accepted the same occurrence. The connected
`TestMonitorDelayedPartialResolutionKeepsDestinationRecovery` retains a real
HTTP 200/503 split, reopens the persistent queue, and requires the old accepted
destination's recovery plus the new firing, with no recovery to the unannounced
destination. Quiet hours and disabled recovery controls still apply.
@@ -1054,3 +1054,27 @@ regression establish frontend control preservation, not provider delivery.
Formatting-only follow-up retains this warning-level contract. The production-component
browser matrix was rerun after formatting at desktop and narrow widths, including
all/critical/warning save/reload, cancel, and warning webhook creation.
### Resolution cancellation is occurrence-qualified
The monitor's delayed recovery path calls `CancelResolvedAlert` with the
captured ID/start-time snapshot. It removes only that occurrence's pending
in-memory grouping members and cooldown record. Persistent
`CancelByAlertOccurrence` shares the existing per-ID delivery gates and
cancellation machinery, but matches both ID and start time: pending, sending,
failed and dead-letter firing rows are cancelled or rewritten without removing
a recurring incident or unrelated members/operational links in a mixed row.
Missing occurrence identity is a no-op. The explicit identifier-wide API, audit
history, destination receipts, retry budget, recovery rows and best-effort
in-flight semantics remain unchanged.
`queue_occurrence_cancellation_test.go` verifies all four row states after a
fresh database reopen, nanosecond-distinct occurrences, operational-link and
failed-attempt preservation, missing-identity safety, grouping/cooldown retention
and the unchanged ID-wide API. Monitor connected controls additionally require
HTTP acceptance of the surviving occurrence through the autonomous queue.
The live monitor does not interpret pending-only cancellation counts as
recipient acceptance. Per-occurrence/per-destination receipts remain the sole
recovery admission proof, including partial delivery and lost RAM markers after
restart; an unannounced destination receives no recovery.