mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-10-14 08:10:26 +00:00
Add powered-off alert tolerance
This commit is contained in:
@@ -396,6 +396,26 @@ Pulse uses a powerful alerting engine with hysteresis (separate trigger/clear th
|
||||
|
||||
**Managed via UI**: Alerts → Thresholds
|
||||
|
||||
### VM and container powered-off tolerance
|
||||
|
||||
Open **Alerts → Thresholds → Alert intent & grace** to configure how long a
|
||||
Proxmox VM or LXC container may remain stopped before Pulse raises its
|
||||
powered-off alert.
|
||||
|
||||
- Leave the VM/container default blank to inherit the existing policy. An
|
||||
installation with no applicable policy keeps the legacy two-poll behavior.
|
||||
- Set it to `0` to alert on the first authoritative stopped observation.
|
||||
- Set a positive number of seconds (for example, `300`) to tolerate a short
|
||||
stop regardless of polling cadence. Per-resource values override the
|
||||
VM/container default; a blank per-resource value inherits it.
|
||||
- **Extend offline grace during Proxmox backups** applies only while Pulse has
|
||||
fresh matching evidence of an active backup. The maximum deferral is a hard
|
||||
cap, so a stale or stuck backup cannot hide a sustained outage.
|
||||
|
||||
This tolerance delays activation; it does not disable powered-off monitoring.
|
||||
Use the existing guest offline-alert toggle when a guest should never produce
|
||||
powered-off alerts.
|
||||
|
||||
<details>
|
||||
<summary><strong>Manual Configuration (JSON)</strong></summary>
|
||||
|
||||
|
||||
@@ -5066,6 +5066,11 @@ Neither an expected-transient decision nor an indeterminate UDP outcome may be
|
||||
translated into an agent action. Any later customer-infrastructure mutation
|
||||
still requires the canonical Actions planner, approval, executor, receipt,
|
||||
audit, and verification boundaries.
|
||||
Powered-off tolerance is evaluated from the Pulse server's monotonic elapsed
|
||||
time and the existing VM/LXC observation stream. It does not accept agent time,
|
||||
poll count, a new report field, or remote configuration authority, and a guest
|
||||
resource-type default cannot change node or host-agent connectivity
|
||||
confirmations.
|
||||
|
||||
### Server-side host deletion and re-enrollment authority
|
||||
|
||||
|
||||
@@ -1283,6 +1283,27 @@ same resolver but restores pending state before returning, so it is read-only.
|
||||
Invalid documents, persistence failures, or revision conflicts must leave the
|
||||
prior in-memory and durable policy active.
|
||||
|
||||
Resource-type inheritance is field-wise from general to specific before the
|
||||
canonical-resource rule: for example `guest` supplies a VM/LXC default and
|
||||
`vm` or `system-container` may replace individual fields. A guest policy must
|
||||
not leak into node or agent connectivity. Omitted grace preserves inherited or
|
||||
factory behavior, explicit zero is a deliberate no-wait policy, and a positive
|
||||
powered-off grace is duration authority rather than a poll-count threshold.
|
||||
The no-policy powered-state path retains the legacy two-confirmation contract
|
||||
for migration compatibility; an applicable powered-state duration activates
|
||||
on the first stopped observation at or after elapsed eligibility.
|
||||
|
||||
Runtime eligibility uses server receipt time plus process-monotonic elapsed
|
||||
progress, never client/provider timestamps or poll counts. Duplicate reports
|
||||
advance no time, delayed polls count actual elapsed time, and wall-clock
|
||||
forward/backward changes do not change the decision. Pending state persists
|
||||
accumulated elapsed progress. Restart re-baselines the monotonic clock without
|
||||
counting unobserved process downtime, while legacy timestamp-only pending files
|
||||
are imported once into the elapsed representation. Recovery, explicit guest
|
||||
suppression, disabling offline alerts, and tracking cleanup remove both
|
||||
durable pending state and its transient monotonic baseline before a later
|
||||
outage can start.
|
||||
|
||||
Operator maintenance and intentionally-offline state are read only through the
|
||||
canonical unified-resource identity. Backup-aware offline deferral consumes
|
||||
fresh, matching, active task evidence, applies the configured post-backup grace,
|
||||
@@ -1291,6 +1312,9 @@ finished, or mismatched backup evidence cannot suppress an outage. This policy
|
||||
changes alert activation only: notification delivery, recovery assurance, and
|
||||
customer-infrastructure mutation retain their existing owners.
|
||||
|
||||
`internal/alerts/intent_policy_test.go` proves precedence, normalization,
|
||||
operator and backup contexts, preview immutability, restart continuity, and
|
||||
first-match lifecycle identity.
|
||||
`internal/alerts/intent_policy_test.go` and
|
||||
`internal/alerts/powered_off_tolerance_test.go` prove field-wise type
|
||||
precedence, normalization, exact zero/positive duration boundaries, VM/LXC
|
||||
coverage, duplicate and delayed reports, wall-clock changes, suppression
|
||||
reset, migration/restart continuity, backup hard-cap behavior, preview
|
||||
immutability, and first-match lifecycle identity.
|
||||
|
||||
@@ -8247,7 +8247,14 @@ default, resource-type, and canonical-resource rules. A stale revision returns
|
||||
`409 Conflict`; invalid policy or preview input returns `400`; unavailable
|
||||
runtime or persistence ownership returns `503`. A save failure restores the
|
||||
previous in-memory document and returns `500`. Preview is bounded and read-only
|
||||
and cannot advance pending grace state.
|
||||
and cannot advance pending grace state. Both write and preview transports
|
||||
accept exactly one JSON object, reject unknown fields, and validate the signal
|
||||
against the same stable policy vocabulary as persistence. JSON coercion,
|
||||
trailing objects, unsupported signals, fractional durations, negative
|
||||
durations, and values above the 30-day bound fail with `400` rather than being
|
||||
silently rounded or ignored. Tenant selection remains request-context owned;
|
||||
policy reads, preview, and writes never fall back to another organization's
|
||||
manager once an organization context is present.
|
||||
|
||||
Availability targets add `udp` plus `response_required` and
|
||||
`open_or_filtered` modes. Request and expected-response payloads remain
|
||||
@@ -8257,7 +8264,7 @@ clients. A silent open-or-filtered result is `indeterminate`, not successful
|
||||
reachability and not a transport failure that may be promoted to an outage.
|
||||
|
||||
`internal/api/alerts_endpoints_test.go` proves scopes, revision conflicts,
|
||||
rollback, and preview behavior.
|
||||
rollback, exact decoding, invalid-signal rejection, and preview behavior.
|
||||
`frontend-modern/src/api/__tests__/alertIntentPolicies.test.ts` and the
|
||||
availability target API and settings tests prove the canonical browser routes
|
||||
and additive UDP wire fields.
|
||||
|
||||
@@ -5263,6 +5263,16 @@ server-owned document after success. Preview renders clear,
|
||||
expected-transient, pending-grace, and would-activate as distinct states and
|
||||
never presents preview as a write.
|
||||
|
||||
The powered-off default is presented and persisted as the `guest` resource-type
|
||||
rule so VM and LXC resources inherit it without changing node or agent
|
||||
connectivity. Blank means inherit, `0` explicitly means no wait, and the UI
|
||||
states both meanings next to the control. Duration fields accept only base-10
|
||||
whole seconds from zero through 30 days; an enabled backup hard cap must be
|
||||
positive. Invalid, fractional, negative, or oversized values remain local,
|
||||
show an actionable error, and issue no API write. Disabling backup extension is
|
||||
an explicit `enabled: false` rule and remains separate from disabling a guest's
|
||||
powered-off alerts.
|
||||
|
||||
Availability controls expose UDP mode, request payload, and optional expected
|
||||
response only where valid for the selected protocol. Unified-resource
|
||||
presentation keeps `indeterminate` visibly distinct from reachable and
|
||||
@@ -5273,6 +5283,7 @@ does not infer detector, operator-intent, or recovery truth.
|
||||
The focused proofs are
|
||||
`frontend-modern/src/features/alerts/__tests__/AlertIntentPolicyPanel.test.tsx`,
|
||||
`frontend-modern/src/features/alerts/__tests__/ThresholdsTab.test.tsx`,
|
||||
`tests/integration/tests/85-powered-off-tolerance.spec.ts`,
|
||||
`frontend-modern/src/components/Settings/ConnectionEditor/__tests__/AvailabilityTargetSlot.test.tsx`,
|
||||
and
|
||||
`frontend-modern/src/utils/__tests__/availabilityProbePresentation.test.ts`.
|
||||
|
||||
@@ -2402,6 +2402,11 @@ evidence fails closed for alert deferral, while PBS and provider recovery
|
||||
evidence continues through its independent mapper and posture rules. An alert
|
||||
grace period or hard-cap expiry cannot be interpreted as backup success or
|
||||
failure and grants no restore or infrastructure mutation authority.
|
||||
The alert runtime measures its base powered-off tolerance and backup hard cap
|
||||
from server-owned monotonic elapsed time. Backup task timestamps may qualify
|
||||
fresh matching context but cannot advance, reset, or extend that clock beyond
|
||||
the cap. A restart conservatively resumes persisted alert progress without
|
||||
claiming process downtime as backup coverage or recovery evidence.
|
||||
|
||||
### Physical-disk collection truth
|
||||
|
||||
|
||||
Reference in New Issue
Block a user