mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-23 20:25:56 +00:00
Revalidate queued alert delivery against current quiet hours
Keep continuous and late replay held without spending a provider attempt. Split mixed batches atomically so eligible alerts retain admitted destinations, occurrence links and retry budgets, and bind saved monitor policy before activating persisted work. Add local receipt, cancellation, rollback, reconstruction and race regression coverage with the owning contracts and verification routes. Change-source: pulse-maintainer
This commit is contained in:
@@ -794,6 +794,14 @@ installer download and the agent's subsequent Pulse TLS connection.
|
||||
|
||||
## Shared Boundaries
|
||||
|
||||
The shared monitor constructor also composes notification bootstrap: it loads
|
||||
saved alert policy and destinations, binds the alert owner's immutable
|
||||
quiet-hours policy provider, then activates persistent queue delivery.
|
||||
This ordering is notifications/alerts-owned and must not change agent enrollment
|
||||
or transport. `internal/monitoring/monitor_notification_startup_test.go` checks
|
||||
the actual constructor and reconstruction, including holding due persisted
|
||||
notifications under a saved continuous schedule without provider attempts.
|
||||
|
||||
- Local administrator setup synchronises the router authorizer before returning a browser session, so API Access remains available to manage agent credentials. This changes neither agent token scopes nor command-policy intent, and must not grant an unrelated identity access to credential management.
|
||||
|
||||
### Notification recovery reload ownership
|
||||
|
||||
@@ -1988,14 +1988,25 @@ quiet-hours policy; nonexistent clock minutes do not shift the configured
|
||||
start or end. Initial queued replay uses the first real minute outside the
|
||||
current non-full-day window, including an unselected day reached at midnight.
|
||||
This does not reinterpret the day toggles as the previous evening's ownership.
|
||||
Full-day windows retain the existing daily replay boundary; this clock repair
|
||||
is not current-policy revalidation of already queued work or external receipt.
|
||||
Full-day windows retain the existing daily revalidation boundary; it is not an
|
||||
invented open minute or permission to send during a continuous schedule.
|
||||
`TestQuietHoursDaylightSavingClockAndReplay` in
|
||||
`internal/alerts/quiet_hours_test.go` checks London, New York and Lord Howe
|
||||
(one-hour and half-hour changes), inclusive boundaries and exact UTC replay.
|
||||
`TestQuietHoursOvernightSelectedCalendarDays` checks overnight day selection.
|
||||
Quiet-hours location fallback is read-only; configuration updates own the
|
||||
location cache, including when public suppression helpers run concurrently.
|
||||
`Manager.QuietHoursNotificationPolicy` exposes an immutable read-only snapshot
|
||||
to the notification owner before it takes queue or per-alert delivery locks.
|
||||
Initial admission and queued replay share civil-clock/category semantics,
|
||||
including repeated/skipped minutes and inclusive ends. The queue revalidates
|
||||
every due attempt against current policy rather than trusting old replay
|
||||
metadata. Policy snapshots neither mutate alert lifecycle/acknowledgement nor
|
||||
select destinations; cancellation and actual delivery remain queue-owned.
|
||||
`TestQuietHoursDeliverySnapshotIsCurrentIndependentAndReadOnly` and
|
||||
`TestQuietHoursDeliverySnapshotConcurrentConfigUpdates` in
|
||||
`internal/alerts/quiet_hours_test.go` cover snapshot isolation and concurrent
|
||||
configuration reads. This establishes source scheduling, not external receipt.
|
||||
Quiet-hours suppression also applies to alert delivery lifecycle, not only the
|
||||
initial raised notification. Resolved notifications must not fan out when the
|
||||
alert was never notified or was already acknowledged, and monitoring-driven
|
||||
|
||||
@@ -1799,6 +1799,19 @@ service-history reads plus denial/recovery without fabricated samples.
|
||||
|
||||
## Current State
|
||||
|
||||
### Saved quiet-hours policy before queue activation
|
||||
|
||||
`Monitor.New` binds `alerts.Manager.QuietHoursNotificationPolicy` to the
|
||||
notification manager after saved policy/destinations load and before
|
||||
`StartQueueProcessing`. The notification owner revalidates due stored work and
|
||||
owns all partitioning, delivery and cancellation; monitoring neither duplicates
|
||||
the schedule nor changes destination selection or operational lifecycle.
|
||||
`TestNewRevalidatesPersistedQuietHoursBeforeDelivery` in
|
||||
`internal/monitoring/monitor_notification_startup_test.go` reconstructs the
|
||||
actual monitor twice with due persisted work under a full-day saved schedule,
|
||||
checking pending state and no local delivery/audit. This is source bootstrap
|
||||
proof, not installed restart or provider acceptance.
|
||||
|
||||
TrueNAS REST alert arguments may be object, scalar, array, null or absent.
|
||||
Non-object arguments must not abort snapshot collection or discard alerts.
|
||||
Only typed object fields supply disk identity and SMART counters; neither scalar
|
||||
|
||||
@@ -73,6 +73,7 @@ or displayed a notification.
|
||||
10. `internal/notifications/delivery_health.go`
|
||||
11. `internal/notifications/deadman_config.go`
|
||||
12. `internal/notifications/failure_class.go`
|
||||
13. `internal/notifications/quiet_hours_queue.go`
|
||||
|
||||
## Shared Boundaries
|
||||
|
||||
@@ -141,6 +142,44 @@ stable opaque routing identities and must not expose credentials.
|
||||
|
||||
## Current State
|
||||
|
||||
### Current-policy quiet-hours replay
|
||||
|
||||
The persistent queue checks every due provider attempt against an immutable
|
||||
snapshot from `alerts.Manager.QuietHoursNotificationPolicy`, installed by
|
||||
`NotificationManager.SetQuietHoursPolicyProvider` before saved work is activated
|
||||
in the monitor constructor. The snapshot is obtained before queue/database or
|
||||
per-alert delivery locks; its evaluator does not call back into the alert owner.
|
||||
Stored replay timestamps are wake conditions, not evidence that the current
|
||||
schedule permits delivery. Full-day schedules recheck at the retained daily
|
||||
boundary, and late replay into another quiet interval remains held.
|
||||
|
||||
`quiet_hours_queue.go` reloads the pending row after acquiring delivery gates,
|
||||
preserving cancellations that rewrote a grouped row after worker discovery.
|
||||
An all-held batch moves only its wake time. A mixed batch atomically retains the
|
||||
held original and creates one ready child with the admitted destination bytes,
|
||||
occurrence/transition links, creation time and remaining attempt budget. Child
|
||||
IDs bind the parent and exact ready payload; held work does not gain an attempt,
|
||||
audit entry, firing receipt or false success. Prior attempt audits remain on the
|
||||
original batch, while actual child attempts carry their own linked receipts.
|
||||
Already-due recovery groups cannot reopen their grouping window while held.
|
||||
Partition persistence or ambiguous linkage failure leaves the original intact
|
||||
and makes no provider attempt. Due-time refresh prevents stale worker snapshots
|
||||
from repeatedly postponing or claiming a now-held row. Actual retries retain
|
||||
the existing claim/cancellation gates and provider enabled checks.
|
||||
|
||||
This does not accelerate a future stored deadline after a schedule edit; the
|
||||
current rule is applied at the next admitted wake. It does not change initial
|
||||
target selection, mutable recovery tag admission, acknowledgement semantics,
|
||||
operator retry budgets or at-least-once delivery after an interrupted send.
|
||||
|
||||
`quiet_hours_queue_test.go` proves continuous/late/DST replay holds, mixed
|
||||
immediate/held delivery, exact admitted destination and operational linkage,
|
||||
budget retention, transaction rollback, durable reopen, cancellation and stale
|
||||
concurrent snapshots, grouped recoveries and all six provider/event families.
|
||||
`internal/monitoring/monitor_notification_startup_test.go` exercises the actual
|
||||
constructor with due persisted work across two startups. These are local source
|
||||
and HTTP-receipt proofs, not installed provider or natural-day acceptance.
|
||||
|
||||
### Persistent resolved-alert grouping
|
||||
|
||||
`SendResolvedAlert` applies the configured grouping window to recoveries as
|
||||
|
||||
@@ -1623,6 +1623,7 @@
|
||||
"internal/monitoring/docker_metric_presence_test.go",
|
||||
"internal/monitoring/monitor_host_agent_removal_lifecycle_test.go",
|
||||
"internal/monitoring/monitor_host_agents_test.go",
|
||||
"internal/monitoring/monitor_notification_startup_test.go",
|
||||
"internal/monitoring/physical_disk_roundtrip_test.go",
|
||||
"scripts/installtests/agent_state_dir_lifecycle_test.go",
|
||||
"scripts/installtests/install_ps1_test.go",
|
||||
@@ -6565,6 +6566,7 @@
|
||||
"internal/monitoring/monitor_docker_test.go",
|
||||
"internal/monitoring/monitor_host_agent_removal_lifecycle_test.go",
|
||||
"internal/monitoring/monitor_host_agents_test.go",
|
||||
"internal/monitoring/monitor_notification_startup_test.go",
|
||||
"internal/monitoring/monitor_mock_alerts_test.go",
|
||||
"internal/monitoring/monitor_pve_cluster_refresh_test.go",
|
||||
"internal/monitoring/monitor_pve_guest_lxc_test.go",
|
||||
|
||||
Reference in New Issue
Block a user