mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-09-10 02:25:56 +00:00
feat(alerts): add initial delivery routing
This commit is contained in:
@@ -43,6 +43,11 @@ command errors retain bounded stderr for compatibility diagnosis.
|
||||
websocket payloads. Carrying plural availability facets through that serializer
|
||||
is an adjacent monitoring/API projection and does not change agent enrollment,
|
||||
report admission, removal, update, profile, or command authority.
|
||||
That shared monitor constructor may also copy the persisted alert schedule's
|
||||
initial notification target into the notification manager. This adjacent
|
||||
alerts/notifications wiring grants no agent enrollment, reporting, removal,
|
||||
profile, update, probe-assignment, or command authority and must not mutate
|
||||
host-agent state.
|
||||
The JSON-excluded Proxmox VM/LXC I/O-rate validity fields carried by
|
||||
`internal/models/models.go` are likewise monitoring-owned sidecar evidence.
|
||||
They distinguish a valid idle interval from an unknown rate for history,
|
||||
|
||||
@@ -17,6 +17,13 @@
|
||||
|
||||
Own alert identity, alert specs, evaluation, persistence semantics, and
|
||||
operator-facing alert routing behavior for live runtime alerts.
|
||||
Alert schedule delivery routing is one persisted, normalized contract.
|
||||
`schedule.initialNotify` accepts `all`, `email`, `webhook`, or `apprise`;
|
||||
missing, legacy, and unknown values preserve the backward-compatible `all`
|
||||
target. The selected initial target owns firing, grouped, and recovery delivery,
|
||||
while every escalation level retains its own independently normalized target.
|
||||
Saving the alert configuration must update the live notification manager as
|
||||
well as persistence so delivery does not differ before and after restart.
|
||||
Docker and Podman container CPU thresholds evaluate host-capacity-normalized
|
||||
CPU percent, not Docker's runtime-native per-core percent. Alert metadata may
|
||||
carry the raw per-core value and reporting host CPU count for evidence, but the
|
||||
|
||||
@@ -23,6 +23,12 @@
|
||||
Own canonical runtime payload shapes between backend and frontend, including
|
||||
the trust boundary that keeps customer-safe support diagnostics and normal
|
||||
product API routes free of maintainer commercial analytics.
|
||||
`GET /api/alerts/config` and `PUT /api/alerts/config` carry
|
||||
`schedule.initialNotify` as the canonical initial notification-routing field.
|
||||
The accepted values are `all`, `email`, `webhook`, and `apprise`; the backend
|
||||
normalizes aliases and invalid values before responding, persisting, and
|
||||
updating the live notification runtime. Escalation levels use the same value
|
||||
vocabulary but remain independent of the initial target.
|
||||
Physical-disk payloads preserve optional SMART counter presence, including
|
||||
explicit zero values, and expose provider vendor metadata without converting
|
||||
missing data into health. Unified-resource clients may request bounded server
|
||||
|
||||
@@ -22,6 +22,11 @@
|
||||
|
||||
Own reusable frontend primitives and canonical page-shell patterns so feature
|
||||
work extends shared components instead of creating new local variants.
|
||||
The alert schedule's initial-delivery selector composes `SettingsPanel` and
|
||||
`FormSelect`, uses the shared alert-configuration presentation vocabulary, and
|
||||
exposes the same email, webhook, Apprise, and all-destination labels used by
|
||||
escalation. It must not introduce a page-local select shell or a second
|
||||
destination-label map.
|
||||
|
||||
Platform-owned workload controls extend the shared `WorkloadsFilter` view
|
||||
options rather than creating page-local toolbar shells. The Proxmox overview
|
||||
|
||||
@@ -19,6 +19,11 @@
|
||||
|
||||
Own polling, typed collection, runtime state assembly, and canonical monitoring
|
||||
truth for live infrastructure data.
|
||||
Monitor construction also applies the persisted alert schedule's normalized
|
||||
initial-delivery target to the tenant notification manager. This is runtime
|
||||
wiring only: monitoring does not choose destinations or own notification
|
||||
policy, and live API saves must apply the same setting without requiring a
|
||||
monitor restart.
|
||||
Monitoring also owns the distinction between Proxmox VM power state and QEMU
|
||||
guest-agent reachability: fresh or never-healthy VMs with an enabled but
|
||||
unavailable guest agent stay `not-running`, while only VMs with recent healthy
|
||||
|
||||
@@ -17,6 +17,12 @@
|
||||
|
||||
Own notification delivery transport, provider configuration, queueing, and
|
||||
notification-management API surfaces.
|
||||
The alert schedule selects firing, grouped, and matching recovery delivery
|
||||
through one normalized target (`all`, `email`, `webhook`, or `apprise`).
|
||||
Escalation delivery remains independently targetable per level, allowing an
|
||||
Apprise/ntfy first notification to escalate through email, or the reverse.
|
||||
Unknown and absent persisted targets preserve historical all-destination
|
||||
behavior, and destination tag filters still apply after target selection.
|
||||
|
||||
## Canonical Files
|
||||
|
||||
|
||||
@@ -35,6 +35,10 @@ availability-check identity by replaying authoritative supplemental records.
|
||||
That availability composition remains owned by API contracts and unified
|
||||
resources; it does not make availability rows recovery points, storage health,
|
||||
backup evidence, or restore authority.
|
||||
The shared alerts API may persist and apply `schedule.initialNotify` for email,
|
||||
webhook, or Apprise delivery. That notification routing is not storage-health,
|
||||
backup, recovery-point, restore, or protection evidence; storage/recovery
|
||||
surfaces must not infer product state from the selected destination.
|
||||
The shared Proxmox config discovery path likewise persists powered-off cluster
|
||||
members and separates membership from member API reachability. Storage and
|
||||
recovery consumers may use the resulting provider-scoped node/instance
|
||||
|
||||
@@ -1599,6 +1599,7 @@
|
||||
{ "path": "src/components/shared/FilterBar/AddFilterMenu.tsx" },
|
||||
{ "path": "src/components/shared/FilterToolbar.tsx" },
|
||||
{ "path": "src/features/alerts/AlertAppriseDestinationsSection.tsx" },
|
||||
{ "path": "src/features/alerts/AlertDeliveryRoutingSection.tsx" },
|
||||
{ "path": "src/features/alerts/AlertEscalationSection.tsx" },
|
||||
{ "path": "src/features/alerts/AlertQuietHoursSection.tsx" }
|
||||
],
|
||||
|
||||
@@ -653,6 +653,7 @@ describe('shared primitive guardrails', () => {
|
||||
'src/components/shared/FilterBar/AddFilterMenu.tsx',
|
||||
'src/components/shared/FilterToolbar.tsx',
|
||||
'src/features/alerts/AlertAppriseDestinationsSection.tsx',
|
||||
'src/features/alerts/AlertDeliveryRoutingSection.tsx',
|
||||
'src/features/alerts/AlertEscalationSection.tsx',
|
||||
'src/features/alerts/AlertQuietHoursSection.tsx',
|
||||
]);
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import { createUniqueId } from 'solid-js';
|
||||
|
||||
import { formHelpText } from '@/components/shared/Form';
|
||||
import { FormSelect } from '@/components/shared/FormSelect';
|
||||
import { SettingsPanel } from '@/components/shared/SettingsPanel';
|
||||
import {
|
||||
ALERT_CONFIG_DELIVERY_DESCRIPTION,
|
||||
ALERT_CONFIG_DELIVERY_HELP,
|
||||
ALERT_CONFIG_DELIVERY_TARGET_LABEL,
|
||||
ALERT_CONFIG_DELIVERY_TITLE,
|
||||
getAlertConfigEscalationNotifyLabel,
|
||||
} from '@/utils/alertConfigPresentation';
|
||||
|
||||
import type { NotificationDeliveryTarget } from './types';
|
||||
|
||||
interface AlertDeliveryRoutingSectionProps {
|
||||
initialNotify: NotificationDeliveryTarget;
|
||||
setInitialNotifyTarget: (value: NotificationDeliveryTarget) => void;
|
||||
}
|
||||
|
||||
export function AlertDeliveryRoutingSection(props: AlertDeliveryRoutingSectionProps) {
|
||||
const fieldId = `alert-initial-delivery-${createUniqueId()}`;
|
||||
|
||||
return (
|
||||
<SettingsPanel
|
||||
title={ALERT_CONFIG_DELIVERY_TITLE}
|
||||
description={ALERT_CONFIG_DELIVERY_DESCRIPTION}
|
||||
>
|
||||
<div class="space-y-3">
|
||||
<FormSelect
|
||||
id={fieldId}
|
||||
label={ALERT_CONFIG_DELIVERY_TARGET_LABEL}
|
||||
value={props.initialNotify}
|
||||
onChange={(event) =>
|
||||
props.setInitialNotifyTarget(event.currentTarget.value as NotificationDeliveryTarget)
|
||||
}
|
||||
>
|
||||
<option value="all">{getAlertConfigEscalationNotifyLabel('all')}</option>
|
||||
<option value="email">{getAlertConfigEscalationNotifyLabel('email')}</option>
|
||||
<option value="webhook">{getAlertConfigEscalationNotifyLabel('webhook')}</option>
|
||||
<option value="apprise">{getAlertConfigEscalationNotifyLabel('apprise')}</option>
|
||||
</FormSelect>
|
||||
<p class={formHelpText}>{ALERT_CONFIG_DELIVERY_HELP}</p>
|
||||
</div>
|
||||
</SettingsPanel>
|
||||
);
|
||||
}
|
||||
@@ -102,6 +102,9 @@ export function AlertEscalationSection(props: AlertEscalationSectionProps) {
|
||||
<option value="webhook">
|
||||
{getAlertConfigEscalationNotifyLabel('webhook')}
|
||||
</option>
|
||||
<option value="apprise">
|
||||
{getAlertConfigEscalationNotifyLabel('apprise')}
|
||||
</option>
|
||||
<option value="all">{getAlertConfigEscalationNotifyLabel('all')}</option>
|
||||
</FormSelect>
|
||||
</div>
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { cleanup, render, screen } from '@solidjs/testing-library';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { AlertCooldownSection } from './AlertCooldownSection';
|
||||
import { AlertDeliveryRoutingSection } from './AlertDeliveryRoutingSection';
|
||||
import { AlertEscalationSection } from './AlertEscalationSection';
|
||||
import { AlertGroupingSection } from './AlertGroupingSection';
|
||||
import { AlertQuietHoursSection } from './AlertQuietHoursSection';
|
||||
@@ -138,9 +139,24 @@ describe('Alert schedule sections', () => {
|
||||
);
|
||||
expect(screen.getByRole('spinbutton', { name: 'After' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('combobox', { name: 'Notify' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('option', { name: 'Apprise' })).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: 'Remove escalation level' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('associates the initial delivery selector and exposes every destination type', () => {
|
||||
render(() => (
|
||||
<AlertDeliveryRoutingSection initialNotify="webhook" setInitialNotifyTarget={vi.fn()} />
|
||||
));
|
||||
|
||||
expect(screen.getByRole('combobox', { name: 'Send initial alerts to' })).toHaveValue('webhook');
|
||||
expect(screen.getAllByRole('option').map((option) => option.textContent)).toEqual([
|
||||
'All channels',
|
||||
'Email',
|
||||
'Webhooks',
|
||||
'Apprise',
|
||||
]);
|
||||
});
|
||||
|
||||
it('associates the visible recovery heading with its status toggle', () => {
|
||||
render(() => (
|
||||
<AlertRecoverySection notifyOnResolve={false} setNotifyOnResolveEnabled={vi.fn()} />
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
ALERT_CONFIG_SUMMARY_TITLE,
|
||||
getAlertConfigSummaryAllDisabled,
|
||||
getAlertConfigSummaryCooldown,
|
||||
getAlertConfigSummaryDelivery,
|
||||
getAlertConfigSummaryEscalation,
|
||||
getAlertConfigSummaryGrouping,
|
||||
getAlertConfigSummaryQuietHours,
|
||||
@@ -13,7 +14,13 @@ import {
|
||||
getAlertConfigSummarySuppressing,
|
||||
} from '@/utils/alertConfigPresentation';
|
||||
|
||||
import type { CooldownConfig, EscalationConfig, GroupingConfig, QuietHoursConfig } from './types';
|
||||
import type {
|
||||
CooldownConfig,
|
||||
EscalationConfig,
|
||||
GroupingConfig,
|
||||
NotificationDeliveryTarget,
|
||||
QuietHoursConfig,
|
||||
} from './types';
|
||||
|
||||
interface QuietSuppressOption {
|
||||
key: keyof QuietHoursConfig['suppress'];
|
||||
@@ -25,6 +32,7 @@ interface AlertScheduleSummarySectionProps {
|
||||
quietHours: QuietHoursConfig;
|
||||
cooldown: CooldownConfig;
|
||||
grouping: GroupingConfig;
|
||||
initialNotify: NotificationDeliveryTarget;
|
||||
notifyOnResolve: boolean;
|
||||
escalation: EscalationConfig;
|
||||
quietHourSuppressOptions: QuietSuppressOption[];
|
||||
@@ -77,6 +85,7 @@ export function AlertScheduleSummarySection(props: AlertScheduleSummarySectionPr
|
||||
)}
|
||||
</p>
|
||||
</Show>
|
||||
<p>{getAlertConfigSummaryDelivery(props.initialNotify)}</p>
|
||||
<Show when={props.notifyOnResolve}>
|
||||
<p>{getAlertConfigSummaryRecoveryEnabled()}</p>
|
||||
</Show>
|
||||
|
||||
@@ -232,6 +232,8 @@ export function AlertsConfigurationSurface(props: AlertsConfigurationSurfaceProp
|
||||
setCooldown={state.setScheduleCooldown}
|
||||
grouping={state.scheduleGrouping}
|
||||
setGrouping={state.setScheduleGrouping}
|
||||
initialNotify={state.initialNotify}
|
||||
setInitialNotify={state.setInitialNotify}
|
||||
notifyOnResolve={state.notifyOnResolve}
|
||||
setNotifyOnResolve={state.setNotifyOnResolve}
|
||||
escalation={state.scheduleEscalation}
|
||||
|
||||
@@ -25,6 +25,7 @@ describe('alertsConfigurationModel', () => {
|
||||
expect(snapshot.timeThresholds['vmware-host']).toBe(5);
|
||||
expect(snapshot.vmwareDefaults.usage).toBe(85);
|
||||
expect(snapshot.scheduleCooldown.enabled).toBe(true);
|
||||
expect(snapshot.initialNotify).toBe('all');
|
||||
expect(snapshot.backupDefaults.ignoreVMIDs).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -69,6 +70,11 @@ describe('alertsConfigurationModel', () => {
|
||||
},
|
||||
cooldown: 45,
|
||||
maxAlertsHour: 7,
|
||||
initialNotify: 'apprise',
|
||||
escalation: {
|
||||
enabled: true,
|
||||
levels: [{ after: 30, notify: 'APPRISE' }],
|
||||
},
|
||||
},
|
||||
} as AlertConfig;
|
||||
|
||||
@@ -96,6 +102,8 @@ describe('alertsConfigurationModel', () => {
|
||||
saturday: false,
|
||||
});
|
||||
expect(snapshot.scheduleCooldown.maxAlerts).toBe(7);
|
||||
expect(snapshot.initialNotify).toBe('apprise');
|
||||
expect(snapshot.scheduleEscalation.levels).toEqual([{ after: 30, notify: 'apprise' }]);
|
||||
});
|
||||
|
||||
it('builds the canonical save payload from the runtime snapshot', () => {
|
||||
@@ -117,6 +125,7 @@ describe('alertsConfigurationModel', () => {
|
||||
byNode: true,
|
||||
byGuest: false,
|
||||
};
|
||||
snapshot.initialNotify = 'email';
|
||||
snapshot.backupDefaults = {
|
||||
enabled: true,
|
||||
warningDays: 30,
|
||||
@@ -147,6 +156,7 @@ describe('alertsConfigurationModel', () => {
|
||||
expect(result.alertConfig?.guestTagWhitelist).toEqual(['prod']);
|
||||
expect(result.alertConfig?.metricTimeThresholds).toEqual({ guest: { cpu: 17 } });
|
||||
expect(result.alertConfig?.schedule?.cooldown).toBe(47);
|
||||
expect(result.alertConfig?.schedule?.initialNotify).toBe('email');
|
||||
expect(result.alertConfig?.schedule?.maxAlertsHour).toBe(10);
|
||||
expect(result.alertConfig?.schedule?.grouping).toEqual({
|
||||
enabled: true,
|
||||
|
||||
@@ -17,6 +17,9 @@ describe('useAlertScheduleState', () => {
|
||||
const [quietHours, setQuietHours] = createSignal(createDefaultQuietHours());
|
||||
const [cooldown, setCooldown] = createSignal(createDefaultCooldown());
|
||||
const [grouping, setGrouping] = createSignal(createDefaultGrouping());
|
||||
const [initialNotify, setInitialNotify] = createSignal<'email' | 'webhook' | 'apprise' | 'all'>(
|
||||
'all',
|
||||
);
|
||||
const [notifyOnResolve, setNotifyOnResolve] = createSignal(false);
|
||||
const [escalation, setEscalation] = createSignal(createDefaultEscalation());
|
||||
|
||||
@@ -29,6 +32,8 @@ describe('useAlertScheduleState', () => {
|
||||
setCooldown,
|
||||
grouping,
|
||||
setGrouping,
|
||||
initialNotify,
|
||||
setInitialNotify,
|
||||
notifyOnResolve,
|
||||
setNotifyOnResolve,
|
||||
escalation,
|
||||
@@ -46,6 +51,7 @@ describe('useAlertScheduleState', () => {
|
||||
result.setGroupingEnabled(true);
|
||||
result.setGroupingWindow('8');
|
||||
result.setGroupingByNode(true);
|
||||
result.setInitialNotifyTarget('apprise');
|
||||
result.setNotifyOnResolveEnabled(true);
|
||||
result.setEscalationEnabled(true);
|
||||
result.addEscalationLevel();
|
||||
@@ -62,6 +68,7 @@ describe('useAlertScheduleState', () => {
|
||||
expect(result.weekendsOnly()).toBe(false);
|
||||
expect(cooldown()).toMatchObject({ enabled: true, minutes: 45, maxAlerts: 5 });
|
||||
expect(grouping()).toMatchObject({ enabled: true, window: 8, byNode: true });
|
||||
expect(initialNotify()).toBe('apprise');
|
||||
expect(notifyOnResolve()).toBe(true);
|
||||
expect(escalation()).toMatchObject({
|
||||
enabled: true,
|
||||
@@ -76,6 +83,7 @@ describe('useAlertScheduleState', () => {
|
||||
expect(quietHours()).toEqual(createDefaultQuietHours());
|
||||
expect(cooldown()).toEqual(createDefaultCooldown());
|
||||
expect(grouping()).toEqual(createDefaultGrouping());
|
||||
expect(initialNotify()).toBe('all');
|
||||
expect(notifyOnResolve()).toBe(createDefaultResolveNotifications());
|
||||
expect(escalation()).toEqual(createDefaultEscalation());
|
||||
});
|
||||
|
||||
@@ -40,6 +40,7 @@ import type {
|
||||
EscalationConfig,
|
||||
EscalationNotifyTarget,
|
||||
GroupingConfig,
|
||||
NotificationDeliveryTarget,
|
||||
QuietHoursConfig,
|
||||
} from './types';
|
||||
import { GROUPING_WINDOW_DEFAULT_SECONDS, clampCooldownMinutes } from './types';
|
||||
@@ -67,6 +68,7 @@ export interface AlertsConfigurationSnapshot {
|
||||
scheduleCooldown: CooldownConfig;
|
||||
scheduleGrouping: GroupingConfig;
|
||||
scheduleEscalation: EscalationConfig;
|
||||
initialNotify: NotificationDeliveryTarget;
|
||||
notifyOnResolve: boolean;
|
||||
guestDefaults: Record<string, number | undefined>;
|
||||
guestDisableConnectivity: boolean;
|
||||
@@ -218,12 +220,29 @@ const normalizeWarningCriticalPair = (
|
||||
const normalizeStringList = (values: string[] | undefined): string[] =>
|
||||
(values ?? []).map((value) => value.trim()).filter((value) => value.length > 0);
|
||||
|
||||
const normalizeNotificationDeliveryTarget = (
|
||||
value: string | undefined,
|
||||
): NotificationDeliveryTarget => {
|
||||
switch (value?.trim().toLowerCase()) {
|
||||
case 'email':
|
||||
return 'email';
|
||||
case 'webhook':
|
||||
case 'webhooks':
|
||||
return 'webhook';
|
||||
case 'apprise':
|
||||
return 'apprise';
|
||||
default:
|
||||
return 'all';
|
||||
}
|
||||
};
|
||||
|
||||
export function createDefaultAlertsConfigurationSnapshot(): AlertsConfigurationSnapshot {
|
||||
return {
|
||||
scheduleQuietHours: createDefaultQuietHours(),
|
||||
scheduleCooldown: createDefaultCooldown(),
|
||||
scheduleGrouping: createDefaultGrouping(),
|
||||
scheduleEscalation: createDefaultEscalation(),
|
||||
initialNotify: 'all',
|
||||
notifyOnResolve: createDefaultResolveNotifications(),
|
||||
guestDefaults: { ...FACTORY_GUEST_DEFAULTS },
|
||||
guestDisableConnectivity: false,
|
||||
@@ -578,6 +597,8 @@ export function readAlertsConfigurationSnapshot(config: AlertConfig): AlertsConf
|
||||
snapshot.disableAllDockerHostsOffline = config.disableAllDockerHostsOffline ?? false;
|
||||
|
||||
if (config.schedule) {
|
||||
snapshot.initialNotify = normalizeNotificationDeliveryTarget(config.schedule.initialNotify);
|
||||
|
||||
if (config.schedule.quietHours) {
|
||||
const quietHours = config.schedule.quietHours;
|
||||
const days = Array.isArray(quietHours.days)
|
||||
@@ -642,7 +663,7 @@ export function readAlertsConfigurationSnapshot(config: AlertConfig): AlertsConf
|
||||
enabled: Boolean(config.schedule.escalation.enabled),
|
||||
levels: (config.schedule.escalation.levels || []).map((level) => ({
|
||||
after: typeof level.after === 'number' ? level.after : 15,
|
||||
notify: (level.notify as EscalationNotifyTarget) || 'all',
|
||||
notify: normalizeNotificationDeliveryTarget(level.notify) as EscalationNotifyTarget,
|
||||
})),
|
||||
};
|
||||
}
|
||||
@@ -827,6 +848,7 @@ export function buildAlertsConfigurationPayload({
|
||||
days: cloneDays(snapshot.scheduleQuietHours.days),
|
||||
},
|
||||
cooldown: normalizedCooldownMinutes,
|
||||
initialNotify: snapshot.initialNotify,
|
||||
notifyOnResolve: snapshot.notifyOnResolve,
|
||||
maxAlertsHour: normalizedMaxAlertsHour,
|
||||
escalation: {
|
||||
|
||||
@@ -8,11 +8,18 @@ import {
|
||||
import { AlertCooldownSection } from '../AlertCooldownSection';
|
||||
import { AlertEscalationSection } from '../AlertEscalationSection';
|
||||
import { AlertGroupingSection } from '../AlertGroupingSection';
|
||||
import { AlertDeliveryRoutingSection } from '../AlertDeliveryRoutingSection';
|
||||
import { AlertQuietHoursSection } from '../AlertQuietHoursSection';
|
||||
import { AlertRecoverySection } from '../AlertRecoverySection';
|
||||
import { AlertScheduleSummarySection } from '../AlertScheduleSummarySection';
|
||||
import { useAlertScheduleState } from '../useAlertScheduleState';
|
||||
import type { CooldownConfig, EscalationConfig, GroupingConfig, QuietHoursConfig } from '../types';
|
||||
import type {
|
||||
CooldownConfig,
|
||||
EscalationConfig,
|
||||
GroupingConfig,
|
||||
NotificationDeliveryTarget,
|
||||
QuietHoursConfig,
|
||||
} from '../types';
|
||||
|
||||
export interface ScheduleTabProps {
|
||||
setHasUnsavedChanges: (value: boolean) => void;
|
||||
@@ -22,6 +29,8 @@ export interface ScheduleTabProps {
|
||||
setCooldown: (value: CooldownConfig) => void;
|
||||
grouping: () => GroupingConfig;
|
||||
setGrouping: (value: GroupingConfig) => void;
|
||||
initialNotify: () => NotificationDeliveryTarget;
|
||||
setInitialNotify: (value: NotificationDeliveryTarget) => void;
|
||||
notifyOnResolve: () => boolean;
|
||||
setNotifyOnResolve: (value: boolean) => void;
|
||||
escalation: () => EscalationConfig;
|
||||
@@ -95,6 +104,11 @@ export function ScheduleTab(props: ScheduleTabProps) {
|
||||
setGroupingByGuest={scheduleState.setGroupingByGuest}
|
||||
/>
|
||||
|
||||
<AlertDeliveryRoutingSection
|
||||
initialNotify={props.initialNotify()}
|
||||
setInitialNotifyTarget={scheduleState.setInitialNotifyTarget}
|
||||
/>
|
||||
|
||||
<AlertRecoverySection
|
||||
notifyOnResolve={props.notifyOnResolve()}
|
||||
setNotifyOnResolveEnabled={scheduleState.setNotifyOnResolveEnabled}
|
||||
@@ -113,6 +127,7 @@ export function ScheduleTab(props: ScheduleTabProps) {
|
||||
quietHours={props.quietHours()}
|
||||
cooldown={props.cooldown()}
|
||||
grouping={props.grouping()}
|
||||
initialNotify={props.initialNotify()}
|
||||
notifyOnResolve={props.notifyOnResolve()}
|
||||
escalation={props.escalation()}
|
||||
quietHourSuppressOptions={quietHourSuppressOptions}
|
||||
|
||||
@@ -193,7 +193,8 @@ export interface GroupingConfig {
|
||||
byGuest?: boolean;
|
||||
}
|
||||
|
||||
export type EscalationNotifyTarget = 'email' | 'webhook' | 'all';
|
||||
export type NotificationDeliveryTarget = 'email' | 'webhook' | 'apprise' | 'all';
|
||||
export type EscalationNotifyTarget = NotificationDeliveryTarget;
|
||||
|
||||
export interface EscalationLevel {
|
||||
after: number;
|
||||
|
||||
@@ -14,6 +14,7 @@ import type {
|
||||
EscalationLevel,
|
||||
EscalationNotifyTarget,
|
||||
GroupingConfig,
|
||||
NotificationDeliveryTarget,
|
||||
QuietHoursConfig,
|
||||
} from './types';
|
||||
import { fallbackCooldownMinutes } from './types';
|
||||
@@ -26,6 +27,8 @@ export interface UseAlertScheduleStateProps {
|
||||
setCooldown: (value: CooldownConfig) => void;
|
||||
grouping: () => GroupingConfig;
|
||||
setGrouping: (value: GroupingConfig) => void;
|
||||
initialNotify: () => NotificationDeliveryTarget;
|
||||
setInitialNotify: (value: NotificationDeliveryTarget) => void;
|
||||
notifyOnResolve: () => boolean;
|
||||
setNotifyOnResolve: (value: boolean) => void;
|
||||
escalation: () => EscalationConfig;
|
||||
@@ -126,6 +129,7 @@ export function useAlertScheduleState(props: UseAlertScheduleStateProps) {
|
||||
props.setQuietHours(createDefaultQuietHours());
|
||||
props.setCooldown(createDefaultCooldown());
|
||||
props.setGrouping(createDefaultGrouping());
|
||||
props.setInitialNotify('all');
|
||||
props.setNotifyOnResolve(createDefaultResolveNotifications());
|
||||
props.setEscalation(createDefaultEscalation());
|
||||
markUnsaved();
|
||||
@@ -283,6 +287,11 @@ export function useAlertScheduleState(props: UseAlertScheduleStateProps) {
|
||||
markUnsaved();
|
||||
};
|
||||
|
||||
const setInitialNotifyTarget = (target: NotificationDeliveryTarget) => {
|
||||
props.setInitialNotify(target);
|
||||
markUnsaved();
|
||||
};
|
||||
|
||||
const setEscalationEnabled = (enabled: boolean) => {
|
||||
props.setEscalation({
|
||||
...props.escalation(),
|
||||
@@ -360,6 +369,7 @@ export function useAlertScheduleState(props: UseAlertScheduleStateProps) {
|
||||
setGroupingWindow,
|
||||
setGroupingByNode,
|
||||
setGroupingByGuest,
|
||||
setInitialNotifyTarget,
|
||||
setNotifyOnResolveEnabled,
|
||||
setEscalationEnabled,
|
||||
setEscalationAfter,
|
||||
|
||||
@@ -21,7 +21,13 @@ import {
|
||||
FACTORY_VMWARE_DEFAULTS,
|
||||
type AlertsConfigurationSnapshot,
|
||||
} from './alertsConfigurationModel';
|
||||
import type { CooldownConfig, EscalationConfig, GroupingConfig, QuietHoursConfig } from './types';
|
||||
import type {
|
||||
CooldownConfig,
|
||||
EscalationConfig,
|
||||
GroupingConfig,
|
||||
NotificationDeliveryTarget,
|
||||
QuietHoursConfig,
|
||||
} from './types';
|
||||
|
||||
interface UseAlertsConfigurationSnapshotStateProps {
|
||||
setHasUnsavedChanges: (value: boolean) => void;
|
||||
@@ -43,6 +49,9 @@ export function useAlertsConfigurationSnapshotState(
|
||||
const [scheduleEscalation, setScheduleEscalation] = createSignal<EscalationConfig>(
|
||||
defaultSnapshot.scheduleEscalation,
|
||||
);
|
||||
const [initialNotify, setInitialNotify] = createSignal<NotificationDeliveryTarget>(
|
||||
defaultSnapshot.initialNotify,
|
||||
);
|
||||
const [notifyOnResolve, setNotifyOnResolve] = createSignal<boolean>(
|
||||
defaultSnapshot.notifyOnResolve,
|
||||
);
|
||||
@@ -165,6 +174,7 @@ export function useAlertsConfigurationSnapshotState(
|
||||
enabled: snapshot.scheduleEscalation.enabled,
|
||||
levels: snapshot.scheduleEscalation.levels.map((level) => ({ ...level })),
|
||||
});
|
||||
setInitialNotify(snapshot.initialNotify);
|
||||
setNotifyOnResolve(snapshot.notifyOnResolve);
|
||||
setGuestDefaults({ ...snapshot.guestDefaults });
|
||||
setGuestDisableConnectivity(snapshot.guestDisableConnectivity);
|
||||
@@ -226,6 +236,7 @@ export function useAlertsConfigurationSnapshotState(
|
||||
enabled: scheduleEscalation().enabled,
|
||||
levels: scheduleEscalation().levels.map((level) => ({ ...level })),
|
||||
},
|
||||
initialNotify: initialNotify(),
|
||||
notifyOnResolve: notifyOnResolve(),
|
||||
guestDefaults: { ...guestDefaults() },
|
||||
guestDisableConnectivity: guestDisableConnectivity(),
|
||||
@@ -344,6 +355,8 @@ export function useAlertsConfigurationSnapshotState(
|
||||
setScheduleGrouping,
|
||||
scheduleEscalation,
|
||||
setScheduleEscalation,
|
||||
initialNotify,
|
||||
setInitialNotify,
|
||||
notifyOnResolve,
|
||||
setNotifyOnResolve,
|
||||
guestDefaults,
|
||||
|
||||
@@ -217,6 +217,7 @@ export interface AlertConfig {
|
||||
};
|
||||
cooldown?: number;
|
||||
maxAlertsHour?: number;
|
||||
initialNotify?: string;
|
||||
notifyOnResolve?: boolean;
|
||||
grouping?: {
|
||||
enabled: boolean;
|
||||
|
||||
@@ -8,8 +8,13 @@ import {
|
||||
ALERT_CONFIG_COOLDOWN_PERIOD_LABEL,
|
||||
ALERT_CONFIG_COOLDOWN_PERIOD_SUFFIX,
|
||||
ALERT_CONFIG_COOLDOWN_TITLE,
|
||||
ALERT_CONFIG_DELIVERY_DESCRIPTION,
|
||||
ALERT_CONFIG_DELIVERY_HELP,
|
||||
ALERT_CONFIG_DELIVERY_TARGET_LABEL,
|
||||
ALERT_CONFIG_DELIVERY_TITLE,
|
||||
ALERT_CONFIG_ESCALATION_DESCRIPTION,
|
||||
ALERT_CONFIG_ESCALATION_NOTIFY_ALL,
|
||||
ALERT_CONFIG_ESCALATION_NOTIFY_APPRISE,
|
||||
ALERT_CONFIG_ESCALATION_NOTIFY_EMAIL,
|
||||
ALERT_CONFIG_ESCALATION_NOTIFY_WEBHOOKS,
|
||||
ALERT_CONFIG_ESCALATION_TITLE,
|
||||
@@ -60,6 +65,7 @@ import {
|
||||
getAlertConfigSaveChangesLabel,
|
||||
getAlertConfigSaveSuccess,
|
||||
getAlertConfigSummaryCooldown,
|
||||
getAlertConfigSummaryDelivery,
|
||||
getAlertConfigSummaryEscalation,
|
||||
getAlertConfigSummaryGrouping,
|
||||
getAlertConfigSummaryAllDisabled,
|
||||
@@ -76,7 +82,7 @@ describe('alertConfigPresentation', () => {
|
||||
expect(ALERT_CONFIG_SAVE_CHANGES).toBe('Save Changes');
|
||||
expect(ALERT_CONFIG_RESET_DEFAULTS).toBe('Reset to defaults');
|
||||
expect(ALERT_CONFIG_RESET_DEFAULTS_TITLE).toBe(
|
||||
'Restore quiet hours, cooldown, grouping, and escalation settings to their defaults',
|
||||
'Restore quiet hours, cooldown, grouping, delivery, and escalation settings to their defaults',
|
||||
);
|
||||
expect(ALERT_CONFIG_SCHEDULING_TITLE).toBe('Alert scheduling');
|
||||
expect(ALERT_CONFIG_SCHEDULING_DESCRIPTION).toBe('Configure when and how alerts are delivered');
|
||||
@@ -131,11 +137,20 @@ describe('alertConfigPresentation', () => {
|
||||
expect(ALERT_CONFIG_ESCALATION_DESCRIPTION).toBe(
|
||||
'Notify additional contacts for persistent issues.',
|
||||
);
|
||||
expect(ALERT_CONFIG_DELIVERY_TITLE).toBe('Initial delivery');
|
||||
expect(ALERT_CONFIG_DELIVERY_DESCRIPTION).toBe(
|
||||
'Choose where firing and recovery notifications are sent.',
|
||||
);
|
||||
expect(ALERT_CONFIG_DELIVERY_TARGET_LABEL).toBe('Send initial alerts to');
|
||||
expect(ALERT_CONFIG_DELIVERY_HELP).toBe(
|
||||
'Escalation levels can use a different destination for persistent alerts.',
|
||||
);
|
||||
expect(ALERT_CONFIG_RECOVERY_HELP).toBe(
|
||||
'Sends on the same channels as live alerts to confirm when a condition clears.',
|
||||
);
|
||||
expect(ALERT_CONFIG_ESCALATION_NOTIFY_EMAIL).toBe('Email');
|
||||
expect(ALERT_CONFIG_ESCALATION_NOTIFY_WEBHOOKS).toBe('Webhooks');
|
||||
expect(ALERT_CONFIG_ESCALATION_NOTIFY_APPRISE).toBe('Apprise');
|
||||
expect(ALERT_CONFIG_ESCALATION_NOTIFY_ALL).toBe('All channels');
|
||||
expect(ALERT_CONFIG_SUMMARY_TITLE).toBe('Configuration summary');
|
||||
expect(ALERT_CONFIG_SUMMARY_DESCRIPTION).toBe('Preview of the active schedule settings.');
|
||||
@@ -154,7 +169,7 @@ describe('alertConfigPresentation', () => {
|
||||
expect(getAlertConfigSaveChangesLabel()).toBe('Save Changes');
|
||||
expect(getAlertConfigResetDefaultsLabel()).toBe('Reset to defaults');
|
||||
expect(getAlertConfigResetDefaultsTitle()).toBe(
|
||||
'Restore quiet hours, cooldown, grouping, and escalation settings to their defaults',
|
||||
'Restore quiet hours, cooldown, grouping, delivery, and escalation settings to their defaults',
|
||||
);
|
||||
expect(getAlertConfigDiscardedSuccess()).toBe('Changes discarded');
|
||||
expect(getAlertConfigReloadFailure()).toBe('Failed to reload configuration');
|
||||
@@ -185,7 +200,11 @@ describe('alertConfigPresentation', () => {
|
||||
expect(getAlertConfigEscalationHelp()).toBe('Define escalation levels for unresolved alerts:');
|
||||
expect(getAlertConfigEscalationNotifyLabel('email')).toBe('Email');
|
||||
expect(getAlertConfigEscalationNotifyLabel('webhook')).toBe('Webhooks');
|
||||
expect(getAlertConfigEscalationNotifyLabel('apprise')).toBe('Apprise');
|
||||
expect(getAlertConfigEscalationNotifyLabel('all')).toBe('All channels');
|
||||
expect(getAlertConfigSummaryDelivery('apprise')).toBe(
|
||||
'• Initial and recovery notifications use Apprise',
|
||||
);
|
||||
expect(getAlertConfigSummaryRecoveryEnabled()).toBe(ALERT_CONFIG_SUMMARY_RECOVERY);
|
||||
expect(getAlertConfigSummaryEscalation(2)).toBe('• 2 escalation levels configured');
|
||||
expect(getAlertConfigSummaryAllDisabled()).toBe(
|
||||
|
||||
@@ -4,7 +4,7 @@ export const ALERT_CONFIG_UNSAVED_CHANGES = 'You have unsaved changes';
|
||||
export const ALERT_CONFIG_SAVE_CHANGES = 'Save Changes';
|
||||
export const ALERT_CONFIG_RESET_DEFAULTS = 'Reset to defaults';
|
||||
export const ALERT_CONFIG_RESET_DEFAULTS_TITLE =
|
||||
'Restore quiet hours, cooldown, grouping, and escalation settings to their defaults';
|
||||
'Restore quiet hours, cooldown, grouping, delivery, and escalation settings to their defaults';
|
||||
export const ALERT_CONFIG_LEAVE_CONFIRMATION =
|
||||
'You have unsaved changes that will be lost. Discard changes and leave?';
|
||||
|
||||
@@ -55,6 +55,12 @@ export const ALERT_CONFIG_RECOVERY_DESCRIPTION =
|
||||
export const ALERT_CONFIG_ESCALATION_TITLE = 'Alert escalation';
|
||||
export const ALERT_CONFIG_ESCALATION_DESCRIPTION =
|
||||
'Notify additional contacts for persistent issues.';
|
||||
export const ALERT_CONFIG_DELIVERY_TITLE = 'Initial delivery';
|
||||
export const ALERT_CONFIG_DELIVERY_DESCRIPTION =
|
||||
'Choose where firing and recovery notifications are sent.';
|
||||
export const ALERT_CONFIG_DELIVERY_TARGET_LABEL = 'Send initial alerts to';
|
||||
export const ALERT_CONFIG_DELIVERY_HELP =
|
||||
'Escalation levels can use a different destination for persistent alerts.';
|
||||
export const ALERT_CONFIG_SUMMARY_QUIET_HOURS_PREFIX = '• Quiet hours active from';
|
||||
export const ALERT_CONFIG_SUMMARY_SUPPRESSING_PREFIX = '• Suppressing';
|
||||
export const ALERT_CONFIG_SUMMARY_SUPPRESSING_SUFFIX = 'during quiet hours';
|
||||
@@ -69,6 +75,7 @@ export const ALERT_CONFIG_ESCALATION_NOTIFY_LABEL = 'Notify';
|
||||
export const ALERT_CONFIG_ESCALATION_MINUTES_SUFFIX = 'min';
|
||||
export const ALERT_CONFIG_ESCALATION_NOTIFY_EMAIL = 'Email';
|
||||
export const ALERT_CONFIG_ESCALATION_NOTIFY_WEBHOOKS = 'Webhooks';
|
||||
export const ALERT_CONFIG_ESCALATION_NOTIFY_APPRISE = 'Apprise';
|
||||
export const ALERT_CONFIG_ESCALATION_NOTIFY_ALL = getAllFilterOptionLabel('channels');
|
||||
export const ALERT_CONFIG_ESCALATION_REMOVE_TITLE = 'Remove escalation level';
|
||||
export const ALERT_CONFIG_ESCALATION_ADD_LABEL = 'Add Escalation Level';
|
||||
@@ -151,17 +158,23 @@ export function getAlertConfigEscalationHelp() {
|
||||
return ALERT_CONFIG_ESCALATION_HELP;
|
||||
}
|
||||
|
||||
export function getAlertConfigEscalationNotifyLabel(type: 'email' | 'webhook' | 'all') {
|
||||
export function getAlertConfigEscalationNotifyLabel(type: 'email' | 'webhook' | 'apprise' | 'all') {
|
||||
switch (type) {
|
||||
case 'email':
|
||||
return ALERT_CONFIG_ESCALATION_NOTIFY_EMAIL;
|
||||
case 'webhook':
|
||||
return ALERT_CONFIG_ESCALATION_NOTIFY_WEBHOOKS;
|
||||
case 'apprise':
|
||||
return ALERT_CONFIG_ESCALATION_NOTIFY_APPRISE;
|
||||
default:
|
||||
return ALERT_CONFIG_ESCALATION_NOTIFY_ALL;
|
||||
}
|
||||
}
|
||||
|
||||
export function getAlertConfigSummaryDelivery(target: 'email' | 'webhook' | 'apprise' | 'all') {
|
||||
return `• Initial and recovery notifications use ${getAlertConfigEscalationNotifyLabel(target)}`;
|
||||
}
|
||||
|
||||
export function getAlertConfigSummaryAllDisabled() {
|
||||
return ALERT_CONFIG_SUMMARY_ALL_DISABLED;
|
||||
}
|
||||
|
||||
@@ -19316,6 +19316,30 @@ func TestDefaultAlertConfigUsesIndependentBackupAlertOrphanedPointer(t *testing.
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpdateConfigNormalizesNotificationDeliveryTargets(t *testing.T) {
|
||||
manager := NewManager()
|
||||
t.Cleanup(manager.Stop)
|
||||
|
||||
config := manager.GetConfig()
|
||||
config.Schedule.InitialNotify = " APPRISE "
|
||||
config.Schedule.Escalation.Levels = []EscalationLevel{
|
||||
{After: 15, Notify: "WEBHOOKS"},
|
||||
{After: 30, Notify: "unsupported"},
|
||||
}
|
||||
manager.UpdateConfig(config)
|
||||
|
||||
updated := manager.GetConfig()
|
||||
if updated.Schedule.InitialNotify != "apprise" {
|
||||
t.Fatalf("initial notify = %q, want apprise", updated.Schedule.InitialNotify)
|
||||
}
|
||||
if updated.Schedule.Escalation.Levels[0].Notify != "webhook" {
|
||||
t.Fatalf("first escalation target = %q, want webhook", updated.Schedule.Escalation.Levels[0].Notify)
|
||||
}
|
||||
if updated.Schedule.Escalation.Levels[1].Notify != "all" {
|
||||
t.Fatalf("invalid escalation target = %q, want all", updated.Schedule.Escalation.Levels[1].Notify)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiagnoseAlertDeliveryReady(t *testing.T) {
|
||||
manager, now := newDeliveryDiagnosisManager(t)
|
||||
alert := addDeliveryDiagnosisAlert(manager, &Alert{
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
package config
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestNormalizeNotificationDeliveryTarget(t *testing.T) {
|
||||
tests := map[string]string{
|
||||
"": "all",
|
||||
"all": "all",
|
||||
"EMAIL": "email",
|
||||
" webhook ": "webhook",
|
||||
"webhooks": "webhook",
|
||||
"Apprise": "apprise",
|
||||
"unknown": "all",
|
||||
}
|
||||
|
||||
for input, expected := range tests {
|
||||
t.Run(input, func(t *testing.T) {
|
||||
if actual := NormalizeNotificationDeliveryTarget(input); actual != expected {
|
||||
t.Fatalf("NormalizeNotificationDeliveryTarget(%q) = %q, want %q", input, actual, expected)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -78,7 +78,7 @@ type QuietHoursSuppression struct {
|
||||
// EscalationLevel represents an escalation rule
|
||||
type EscalationLevel struct {
|
||||
After int `json:"after"` // minutes after initial alert
|
||||
Notify string `json:"notify"` // "email", "webhook", or "all"
|
||||
Notify string `json:"notify"` // "email", "webhook", "apprise", or "all"
|
||||
}
|
||||
|
||||
// EscalationConfig represents alert escalation configuration
|
||||
@@ -100,11 +100,27 @@ type ScheduleConfig struct {
|
||||
QuietHours QuietHours `json:"quietHours"`
|
||||
Cooldown int `json:"cooldown"` // minutes
|
||||
MaxAlertsHour int `json:"maxAlertsHour"` // max alerts per hour per resource
|
||||
InitialNotify string `json:"initialNotify"` // "email", "webhook", "apprise", or "all"
|
||||
NotifyOnResolve bool `json:"notifyOnResolve"` // Send notification when alert clears
|
||||
Escalation EscalationConfig `json:"escalation"`
|
||||
Grouping GroupingConfig `json:"grouping"`
|
||||
}
|
||||
|
||||
// NormalizeNotificationDeliveryTarget keeps schedule delivery routing backward
|
||||
// compatible while rejecting unknown values from persisted or API config.
|
||||
func NormalizeNotificationDeliveryTarget(target string) string {
|
||||
switch strings.ToLower(strings.TrimSpace(target)) {
|
||||
case "email":
|
||||
return "email"
|
||||
case "webhook", "webhooks":
|
||||
return "webhook"
|
||||
case "apprise":
|
||||
return "apprise"
|
||||
default:
|
||||
return "all"
|
||||
}
|
||||
}
|
||||
|
||||
// FilterCondition represents a single filter condition
|
||||
type FilterCondition struct {
|
||||
Type string `json:"type"` // "metric", "text", or "raw"
|
||||
|
||||
@@ -42,6 +42,12 @@ func (m *Manager) UpdateConfig(config AlertConfig) {
|
||||
config.GuestDefaults.PoweredOffSeverity = alertconfig.NormalizePoweredOffSeverity(config.GuestDefaults.PoweredOffSeverity)
|
||||
config.NodeDefaults.PoweredOffSeverity = alertconfig.NormalizePoweredOffSeverity(config.NodeDefaults.PoweredOffSeverity)
|
||||
config.DockerIgnoredContainerPrefixes = alertconfig.NormalizeDockerIgnoredPrefixes(config.DockerIgnoredContainerPrefixes)
|
||||
config.Schedule.InitialNotify = alertconfig.NormalizeNotificationDeliveryTarget(config.Schedule.InitialNotify)
|
||||
for index := range config.Schedule.Escalation.Levels {
|
||||
config.Schedule.Escalation.Levels[index].Notify = alertconfig.NormalizeNotificationDeliveryTarget(
|
||||
config.Schedule.Escalation.Levels[index].Notify,
|
||||
)
|
||||
}
|
||||
|
||||
// Migration logic for activation state (backward compatibility)
|
||||
m.migrateActivationState(&config)
|
||||
|
||||
@@ -163,6 +163,7 @@ func defaultAlertConfig() AlertConfig {
|
||||
},
|
||||
Cooldown: 5, // ON - 5 minutes prevents spam
|
||||
MaxAlertsHour: 10, // ON - 10 alerts/hour prevents flooding
|
||||
InitialNotify: "all",
|
||||
NotifyOnResolve: true,
|
||||
Escalation: EscalationConfig{
|
||||
Enabled: false, // OFF - requires user configuration
|
||||
|
||||
@@ -216,6 +216,7 @@ func (h *AlertHandlers) UpdateAlertConfig(w http.ResponseWriter, r *http.Request
|
||||
updatedConfig.Schedule.Grouping.ByNode,
|
||||
updatedConfig.Schedule.Grouping.ByGuest,
|
||||
)
|
||||
notificationMgr.SetInitialNotifyTarget(updatedConfig.Schedule.InitialNotify)
|
||||
notificationMgr.SetNotifyOnResolve(updatedConfig.Schedule.NotifyOnResolve)
|
||||
|
||||
// Save to persistent storage. Failure here used to be swallowed (logged
|
||||
|
||||
@@ -34,7 +34,8 @@ func TestAlertsEndpoints(t *testing.T) {
|
||||
t.Run("UpdateAlertConfig", func(t *testing.T) {
|
||||
newConfig := alerts.AlertConfig{
|
||||
Schedule: alerts.ScheduleConfig{
|
||||
Cooldown: 300,
|
||||
Cooldown: 300,
|
||||
InitialNotify: "apprise",
|
||||
},
|
||||
}
|
||||
body, _ := json.Marshal(newConfig)
|
||||
@@ -70,6 +71,12 @@ func TestAlertsEndpoints(t *testing.T) {
|
||||
if updatedConfig.Schedule.Cooldown != 300 {
|
||||
t.Errorf("expected cooldown 300, got %d", updatedConfig.Schedule.Cooldown)
|
||||
}
|
||||
if updatedConfig.Schedule.InitialNotify != "apprise" {
|
||||
t.Errorf("expected initial notify apprise, got %q", updatedConfig.Schedule.InitialNotify)
|
||||
}
|
||||
if got := srv.monitor.GetNotificationManager().GetInitialNotifyTarget(); got != "apprise" {
|
||||
t.Errorf("expected live initial target apprise, got %q", got)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("AlertIntentPolicies", func(t *testing.T) {
|
||||
|
||||
@@ -95,6 +95,7 @@ type alertSchedulePresence struct {
|
||||
type alertScheduleFieldPresence struct {
|
||||
Cooldown *int `json:"cooldown"`
|
||||
MaxAlertsHour *int `json:"maxAlertsHour"`
|
||||
InitialNotify *string `json:"initialNotify"`
|
||||
NotifyOnResolve *bool `json:"notifyOnResolve"`
|
||||
Grouping *alertScheduleGroupingPresence `json:"grouping"`
|
||||
}
|
||||
@@ -979,6 +980,7 @@ func (c *ConfigPersistence) LoadAlertConfig() (*alerts.AlertConfig, error) {
|
||||
Schedule: alerts.ScheduleConfig{
|
||||
Cooldown: 5,
|
||||
MaxAlertsHour: 10,
|
||||
InitialNotify: "all",
|
||||
NotifyOnResolve: true,
|
||||
Grouping: alerts.GroupingConfig{
|
||||
Enabled: true,
|
||||
@@ -1038,6 +1040,7 @@ func (c *ConfigPersistence) LoadAlertConfig() (*alerts.AlertConfig, error) {
|
||||
defaultSchedule := alerts.ScheduleConfig{
|
||||
Cooldown: 5,
|
||||
MaxAlertsHour: 10,
|
||||
InitialNotify: "all",
|
||||
NotifyOnResolve: true,
|
||||
Grouping: alerts.GroupingConfig{
|
||||
Enabled: true,
|
||||
@@ -1056,6 +1059,9 @@ func (c *ConfigPersistence) LoadAlertConfig() (*alerts.AlertConfig, error) {
|
||||
if sched == nil || sched.MaxAlertsHour == nil {
|
||||
config.Schedule.MaxAlertsHour = defaultSchedule.MaxAlertsHour
|
||||
}
|
||||
if sched == nil || sched.InitialNotify == nil {
|
||||
config.Schedule.InitialNotify = defaultSchedule.InitialNotify
|
||||
}
|
||||
if sched == nil || sched.NotifyOnResolve == nil {
|
||||
config.Schedule.NotifyOnResolve = defaultSchedule.NotifyOnResolve
|
||||
}
|
||||
|
||||
@@ -28,6 +28,7 @@ func TestLoadAlertConfig_Normalization(t *testing.T) {
|
||||
assert.True(t, cfg.Enabled)
|
||||
assert.Equal(t, 5, cfg.Schedule.Cooldown)
|
||||
assert.Equal(t, 10, cfg.Schedule.MaxAlertsHour)
|
||||
assert.Equal(t, "all", cfg.Schedule.InitialNotify)
|
||||
assert.True(t, cfg.Schedule.NotifyOnResolve)
|
||||
assert.True(t, cfg.Schedule.Grouping.Enabled)
|
||||
assert.Equal(t, 30, cfg.Schedule.Grouping.Window)
|
||||
@@ -92,6 +93,26 @@ func TestLoadAlertConfig_Normalization(t *testing.T) {
|
||||
assert.Equal(t, 5.0, cfg.HysteresisMargin)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Schedule missing initialNotify defaults to all",
|
||||
input: map[string]interface{}{
|
||||
"schedule": map[string]interface{}{},
|
||||
},
|
||||
verify: func(t *testing.T, cfg *alerts.AlertConfig) {
|
||||
assert.Equal(t, "all", cfg.Schedule.InitialNotify)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Schedule explicit initialNotify is preserved",
|
||||
input: map[string]interface{}{
|
||||
"schedule": map[string]interface{}{
|
||||
"initialNotify": "apprise",
|
||||
},
|
||||
},
|
||||
verify: func(t *testing.T, cfg *alerts.AlertConfig) {
|
||||
assert.Equal(t, "apprise", cfg.Schedule.InitialNotify)
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Schedule missing defaults notifyOnResolve to true",
|
||||
input: map[string]interface{}{
|
||||
|
||||
@@ -1734,6 +1734,7 @@ func New(cfg *config.Config) (*Monitor, error) {
|
||||
alertConfig.Schedule.Grouping.ByNode,
|
||||
alertConfig.Schedule.Grouping.ByGuest,
|
||||
)
|
||||
m.notificationMgr.SetInitialNotifyTarget(alertConfig.Schedule.InitialNotify)
|
||||
m.notificationMgr.SetNotifyOnResolve(alertConfig.Schedule.NotifyOnResolve)
|
||||
} else {
|
||||
log.Warn().Err(err).Msg("failed to load alert configuration")
|
||||
|
||||
@@ -3051,6 +3051,40 @@ func TestApplyHostReportSkipsMetricsAndSMARTWritesInMockMode(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestMonitorStartupAppliesPersistedInitialNotificationTarget(t *testing.T) {
|
||||
previous := mock.IsMockEnabled()
|
||||
mustSetMockEnabled(t, false)
|
||||
t.Cleanup(func() { mustSetMockEnabled(t, previous) })
|
||||
|
||||
dataDir := t.TempDir()
|
||||
persistence := config.NewConfigPersistence(dataDir)
|
||||
alertManager := alerts.NewManagerWithDataDir(dataDir)
|
||||
alertConfig := alertManager.GetConfig()
|
||||
alertConfig.Enabled = true
|
||||
alertConfig.ActivationState = alerts.ActivationActive
|
||||
alertConfig.Schedule.InitialNotify = "email"
|
||||
alertManager.UpdateConfig(alertConfig)
|
||||
if err := persistence.SaveAlertConfig(alertManager.GetConfig()); err != nil {
|
||||
alertManager.Stop()
|
||||
t.Fatalf("save alert config: %v", err)
|
||||
}
|
||||
alertManager.Stop()
|
||||
|
||||
monitor, err := New(&config.Config{
|
||||
ConfigPath: dataDir,
|
||||
DataPath: dataDir,
|
||||
EnvOverrides: make(map[string]bool),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("create monitor: %v", err)
|
||||
}
|
||||
t.Cleanup(monitor.Stop)
|
||||
|
||||
if actual := monitor.GetNotificationManager().GetInitialNotifyTarget(); actual != "email" {
|
||||
t.Fatalf("startup initial notification target = %q, want email", actual)
|
||||
}
|
||||
}
|
||||
|
||||
func waitForStoredDiskMetric(t *testing.T, store *metrics.Store, resourceID, metric string) []metrics.MetricPoint {
|
||||
t.Helper()
|
||||
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
package notifications
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
|
||||
)
|
||||
|
||||
func TestInitialNotifyTargetNormalization(t *testing.T) {
|
||||
manager := NewNotificationManagerWithDataDir("", t.TempDir())
|
||||
defer manager.Stop()
|
||||
|
||||
if actual := manager.GetInitialNotifyTarget(); actual != "all" {
|
||||
t.Fatalf("default initial target = %q, want all", actual)
|
||||
}
|
||||
|
||||
tests := map[string]string{
|
||||
"EMAIL": "email",
|
||||
" webhooks": "webhook",
|
||||
"apprise": "apprise",
|
||||
"invalid": "all",
|
||||
}
|
||||
for input, expected := range tests {
|
||||
manager.SetInitialNotifyTarget(input)
|
||||
if actual := manager.GetInitialNotifyTarget(); actual != expected {
|
||||
t.Fatalf("initial target after %q = %q, want %q", input, actual, expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNotificationDeliveryJobsRespectTarget(t *testing.T) {
|
||||
email := EmailConfig{
|
||||
Enabled: true,
|
||||
SMTPHost: "smtp.example.test",
|
||||
SMTPPort: 587,
|
||||
From: "pulse@example.test",
|
||||
To: []string{"ops@example.test"},
|
||||
}
|
||||
webhooks := []WebhookConfig{
|
||||
{ID: "hook-1", Name: "First", URL: "https://hooks.example.test/one", Enabled: true},
|
||||
{ID: "hook-2", Name: "Second", URL: "https://hooks.example.test/two", Enabled: true},
|
||||
}
|
||||
apprise := AppriseConfig{Enabled: true, Targets: []string{"ntfy://example.test/pulse"}}
|
||||
alertList := []*alerts.Alert{{
|
||||
ID: "node-1-cpu",
|
||||
ResourceName: "node-1",
|
||||
StartTime: time.Now(),
|
||||
}}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
target notificationDeliveryTarget
|
||||
expected []string
|
||||
}{
|
||||
{name: "all", target: notificationDeliveryTargetAll, expected: []string{"apprise", "email", "webhook", "webhook"}},
|
||||
{name: "email", target: notificationDeliveryTargetEmail, expected: []string{"email"}},
|
||||
{name: "webhook", target: notificationDeliveryTargetWebhook, expected: []string{"webhook", "webhook"}},
|
||||
{name: "apprise", target: notificationDeliveryTargetApprise, expected: []string{"apprise"}},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
jobs := buildNotificationDeliveryJobsForTarget(
|
||||
email,
|
||||
webhooks,
|
||||
apprise,
|
||||
alertList,
|
||||
eventAlert,
|
||||
time.Time{},
|
||||
test.target,
|
||||
)
|
||||
actual := make([]string, 0, len(jobs))
|
||||
for _, job := range jobs {
|
||||
actual = append(actual, job.Type)
|
||||
}
|
||||
sort.Strings(actual)
|
||||
sort.Strings(test.expected)
|
||||
if len(actual) != len(test.expected) {
|
||||
t.Fatalf("job types = %#v, want %#v", actual, test.expected)
|
||||
}
|
||||
for index := range actual {
|
||||
if actual[index] != test.expected[index] {
|
||||
t.Fatalf("job types = %#v, want %#v", actual, test.expected)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGroupedAlertsUseInitialNotifyTarget(t *testing.T) {
|
||||
queue, err := NewNotificationQueue(t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatalf("create queue: %v", err)
|
||||
}
|
||||
defer queue.Stop()
|
||||
|
||||
manager := &NotificationManager{
|
||||
enabled: true,
|
||||
initialTarget: notificationDeliveryTargetApprise,
|
||||
emailConfig: EmailConfig{
|
||||
Enabled: true,
|
||||
SMTPHost: "smtp.example.test",
|
||||
SMTPPort: 587,
|
||||
From: "pulse@example.test",
|
||||
To: []string{"ops@example.test"},
|
||||
},
|
||||
webhooks: []WebhookConfig{{
|
||||
ID: "hook-1", Name: "Webhook", URL: "https://hooks.example.test/pulse", Enabled: true,
|
||||
}},
|
||||
appriseConfig: AppriseConfig{
|
||||
Enabled: true,
|
||||
Targets: []string{"ntfy://example.test/pulse"},
|
||||
},
|
||||
lastNotified: make(map[string]notificationRecord),
|
||||
deliveryReceipts: make(map[string]struct{}),
|
||||
queue: queue,
|
||||
}
|
||||
manager.pendingAlerts = []*alerts.Alert{{
|
||||
ID: "node-1-cpu",
|
||||
ResourceName: "node-1",
|
||||
StartTime: time.Now(),
|
||||
}}
|
||||
|
||||
manager.sendGroupedAlerts()
|
||||
|
||||
rows, err := queue.db.Query(`SELECT type FROM notification_queue ORDER BY created_at`)
|
||||
if err != nil {
|
||||
t.Fatalf("query queued types: %v", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var queuedTypes []string
|
||||
for rows.Next() {
|
||||
var queuedType string
|
||||
if err := rows.Scan(&queuedType); err != nil {
|
||||
t.Fatalf("scan queued type: %v", err)
|
||||
}
|
||||
queuedTypes = append(queuedTypes, queuedType)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
t.Fatalf("iterate queued types: %v", err)
|
||||
}
|
||||
if len(queuedTypes) != 1 || queuedTypes[0] != "apprise" {
|
||||
t.Fatalf("queued types = %#v, want only apprise", queuedTypes)
|
||||
}
|
||||
}
|
||||
@@ -236,6 +236,7 @@ type NotificationManager struct {
|
||||
appriseConfig AppriseConfig
|
||||
enabled bool
|
||||
cooldown time.Duration
|
||||
initialTarget notificationDeliveryTarget
|
||||
notifyOnResolve bool
|
||||
lastNotified map[string]notificationRecord
|
||||
deliveryReceipts map[string]struct{}
|
||||
@@ -711,6 +712,7 @@ func NewNotificationManagerWithDataDir(publicURL string, dataDir string) *Notifi
|
||||
nm := &NotificationManager{
|
||||
enabled: true,
|
||||
cooldown: 5 * time.Minute,
|
||||
initialTarget: notificationDeliveryTargetAll,
|
||||
notifyOnResolve: true,
|
||||
lastNotified: make(map[string]notificationRecord),
|
||||
deliveryReceipts: make(map[string]struct{}),
|
||||
@@ -883,6 +885,28 @@ func (n *NotificationManager) SetNotifyOnResolve(enabled bool) {
|
||||
}
|
||||
}
|
||||
|
||||
// SetInitialNotifyTarget selects destinations for firing and grouped
|
||||
// notifications. Matching recovery delivery follows firing delivery receipts,
|
||||
// while escalation levels retain their own per-level routing.
|
||||
func (n *NotificationManager) SetInitialNotifyTarget(target string) {
|
||||
normalized := normalizeNotificationDeliveryTarget(target)
|
||||
n.mu.Lock()
|
||||
was := n.initialTarget
|
||||
n.initialTarget = normalized
|
||||
n.mu.Unlock()
|
||||
|
||||
if was != normalized {
|
||||
log.Info().Str("target", string(normalized)).Msg("updated initial alert notification target")
|
||||
}
|
||||
}
|
||||
|
||||
// GetInitialNotifyTarget returns the normalized initial delivery target.
|
||||
func (n *NotificationManager) GetInitialNotifyTarget() string {
|
||||
n.mu.RLock()
|
||||
defer n.mu.RUnlock()
|
||||
return string(n.initialTarget)
|
||||
}
|
||||
|
||||
// GetNotifyOnResolve returns whether resolved alerts trigger notifications.
|
||||
func (n *NotificationManager) GetNotifyOnResolve() bool {
|
||||
n.mu.RLock()
|
||||
@@ -1019,7 +1043,10 @@ func (n *NotificationManager) IsEnabled() bool {
|
||||
|
||||
// SendAlert sends notifications for a newly fired or explicitly re-notified alert.
|
||||
func (n *NotificationManager) SendAlert(alert *alerts.Alert) {
|
||||
n.sendAlert(alert, alertSendOptions{target: notificationDeliveryTargetAll})
|
||||
n.mu.RLock()
|
||||
target := n.initialTarget
|
||||
n.mu.RUnlock()
|
||||
n.sendAlert(alert, alertSendOptions{target: target})
|
||||
}
|
||||
|
||||
// SendEscalatedAlert sends a scheduled escalation notification. Escalation
|
||||
@@ -1436,10 +1463,19 @@ func (n *NotificationManager) sendGroupedAlerts() {
|
||||
emailConfig := copyEmailConfig(n.emailConfig)
|
||||
webhooks := copyWebhookConfigs(n.webhooks)
|
||||
appriseConfig := copyAppriseConfig(n.appriseConfig)
|
||||
initialTarget := n.initialTarget
|
||||
queue := n.queue
|
||||
n.mu.Unlock()
|
||||
|
||||
jobs := buildNotificationDeliveryJobs(emailConfig, webhooks, appriseConfig, alertsToSend, eventAlert, time.Time{})
|
||||
jobs := buildNotificationDeliveryJobsForTarget(
|
||||
emailConfig,
|
||||
webhooks,
|
||||
appriseConfig,
|
||||
alertsToSend,
|
||||
eventAlert,
|
||||
time.Time{},
|
||||
initialTarget,
|
||||
)
|
||||
if len(jobs) == 0 {
|
||||
// Preserve cooldown semantics when notifications are globally enabled
|
||||
// but no destination is configured. Delivery receipts remain empty, so
|
||||
|
||||
Reference in New Issue
Block a user