mirror of
https://github.com/rcourtman/Pulse.git
synced 2026-09-10 02:25:56 +00:00
Make alert-triggered Patrol investigate the specific breach
Previously an alert that triggered Patrol ran a broad health check that explicitly ignored the threshold breach. Now an alert carries its real payload (metric type, value, threshold, identifier, level, message) into the patrol scope, and the alert_fired run is framed around root-causing that specific breach instead of a general assessment. Three coordinated changes: - Carry the alert payload into PatrolScope.AlertContext through the alert bridge (PatrolTriggerEvent), so the patrol prompt sees the breach specifics rather than just an alert-type string. - Frame alert_fired patrol runs around the breach: replace the "ignore threshold breaches" instruction with a root-cause directive targeting the alert's metric and threshold. - Add per-rule control via AIConfig.AlertTriggersInvestigation: a master enable, a minimum-severity floor (patrol_alert_trigger_min_severity, default critical-only), and an optional alert-type allowlist (patrol_alert_trigger_types). The router's bridge callback consults the policy and drops non-qualifying alert_fired events before queuing a scoped patrol. A config-panel selector persists the severity floor. Adds config, handler, and frontend proof tests, and updates the affected subsystem contracts.
This commit is contained in:
@@ -1102,6 +1102,14 @@ the same lock-against-remediation flag that the action broker
|
||||
enforces downstream — no possible drift between "what Patrol
|
||||
proposes" and "what the broker accepts."
|
||||
|
||||
The same router wiring owns the alert-bridge patrol-trigger callback. It now
|
||||
receives the full alert payload as a struct and consults the operator's
|
||||
per-rule trigger policy before queuing a scoped patrol: an `alert_fired` event
|
||||
that fails `AIConfig.AlertTriggersInvestigation` (below the minimum-severity
|
||||
floor or outside the alert-type allowlist) is logged and dropped without
|
||||
entering the trigger manager, so alert-driven investigation lifecycle stays
|
||||
bounded to the alerts the operator opted into.
|
||||
|
||||
`/api/agent/events` is the SSE stream agents subscribe to for
|
||||
real-time notifications: `finding.created` when a new finding is
|
||||
raised, `approval.pending` when a remediation request enters
|
||||
|
||||
@@ -698,6 +698,21 @@ default estate surface. The user-facing Machines label is an app-shell
|
||||
presentation label for the existing `standalone` route/id and must not create a
|
||||
separate AI handoff or prompt namespace.
|
||||
|
||||
Alert-triggered scoped patrols now investigate the specific breach rather than
|
||||
running a broad health check. The alert bridge (`internal/ai/unified/bridge.go`,
|
||||
`internal/ai/unified/setup.go`) carries the firing alert's real payload — type,
|
||||
level, value, threshold, resource identifier, and message — into
|
||||
`PatrolScope.AlertContext`, and `internal/ai/patrol_ai.go` /
|
||||
`internal/ai/patrol_triggers.go` frame the `alert_fired` run around that breach
|
||||
instead of suppressing threshold context. Whether an alert triggers a patrol at
|
||||
all is the operator's per-rule policy: `AIConfig.AlertTriggersInvestigation`
|
||||
(`internal/config/ai.go`) enforces the master enable, a minimum-severity floor
|
||||
(`patrol_alert_trigger_min_severity`, default critical-only), and an optional
|
||||
alert-type allowlist (`patrol_alert_trigger_types`, empty = all types). The
|
||||
router-side bridge wiring consults that policy and skips queuing the scoped
|
||||
patrol when the alert does not qualify; an unknown alert level is treated as
|
||||
critical so it is never silently dropped.
|
||||
|
||||
The route-backed Proxmox platform tab is app-shell navigation only. Adding the
|
||||
tab through `frontend-modern/src/App.tsx` and
|
||||
`frontend-modern/src/AppLayout.tsx` must not fork Assistant or Patrol shell
|
||||
|
||||
@@ -1645,6 +1645,14 @@ Proxmox host rows with workload I/O columns. These fields are a read-model
|
||||
extension over existing resource telemetry; they must remain optional until the
|
||||
backend transport contract explicitly guarantees them for every node source,
|
||||
and consumers must tolerate absence without inventing a second API shape.
|
||||
The AI settings contract (`internal/api/ai_handlers.go`) now carries the
|
||||
per-rule alert-trigger policy for scoped patrols. The response always projects
|
||||
`patrol_alert_trigger_min_severity` (`warning` | `critical`, normalized from the
|
||||
critical-only default) and a non-nil `patrol_alert_trigger_types` allowlist
|
||||
(empty = all types). The update request accepts both as optional pointer fields;
|
||||
the handler rejects any `patrol_alert_trigger_min_severity` other than `warning`
|
||||
or `critical` with `400`, and lowercases, trims, drops blanks, and de-duplicates
|
||||
the types allowlist before persistence so the stored shape is canonical.
|
||||
The shared metrics-history API also treats `metric=temperature` as a canonical
|
||||
agent/node chart metric for Proxmox node drawers. `resourceType=agent` may serve
|
||||
the current host-agent CPU package temperature as a live fallback when persisted
|
||||
|
||||
@@ -1447,6 +1447,12 @@ provider-specific controls from `aiSettingsModel.ts` `extraFields`, including
|
||||
Ollama `keep_alive`, so Assistant and Patrol keep one settings shape across
|
||||
labeling, help affordances, helper copy, and persistence binding.
|
||||
|
||||
The Patrol alert-trigger severity selector under
|
||||
`frontend-modern/src/features/patrol/` is built on the shared `FormSelect`
|
||||
primitive (label-for/id wiring, `selectBaseClass` styling hook) rather than a
|
||||
hand-rolled `<select>`, so its labeling and disabled-state affordances stay
|
||||
consistent with the rest of the AI settings surface.
|
||||
|
||||
Kubernetes RBAC inventory (Roles, ClusterRoles, RoleBindings,
|
||||
ClusterRoleBindings) is part of the existing Kubernetes platform-page
|
||||
Configuration tab, not a new sidebar entry or top-level route, and the
|
||||
|
||||
@@ -453,6 +453,17 @@ surface for Patrol intelligence. This contract now owns that orchestration and
|
||||
presentation boundary while leaving shared transport and payload-shape
|
||||
ownership in the governed AI runtime and API contract surfaces.
|
||||
|
||||
The Patrol configuration panel (`PatrolIntelligenceHeader.tsx`,
|
||||
`usePatrolIntelligenceState.ts`) exposes the per-rule alert-trigger policy
|
||||
directly under the Alert-Triggered Patrols toggle. A minimum-severity selector
|
||||
("Investigate alerts at or above": Critical only / Warning and critical) renders
|
||||
only while alert triggers are enabled and persists through
|
||||
`AIAPI.updateSettings({ patrol_alert_trigger_min_severity })` with optimistic
|
||||
state and revert-on-error, mirroring the existing trigger-toggle handlers. The
|
||||
selector reads `patrol_alert_trigger_min_severity` from the settings response,
|
||||
defaulting to critical-only, and must keep using the shared AI settings shape
|
||||
rather than forking a patrol-local form.
|
||||
|
||||
The route file `frontend-modern/src/pages/AIIntelligence.tsx` is now also a
|
||||
thin shell that delegates to the feature-owned
|
||||
`frontend-modern/src/features/patrol/PatrolIntelligenceSurface.tsx`, so Patrol
|
||||
|
||||
@@ -619,6 +619,14 @@ the canonical Workloads hot-path budget is preserved. Standalone
|
||||
WorkloadsSurface callers (no override props) keep the original
|
||||
persistent-signal-backed behavior.
|
||||
|
||||
The alert-bridge patrol-trigger callback wired in `internal/api/router.go` now
|
||||
short-circuits before queuing a scoped patrol when a firing alert does not meet
|
||||
the operator's per-rule trigger policy (minimum-severity floor plus optional
|
||||
alert-type allowlist, critical-only by default). This bounds alert-driven
|
||||
patrol fan-out: in a noisy-warning estate the default policy keeps the LLM-backed
|
||||
investigation path from being invoked once per warning, so the patrol queue and
|
||||
provider spend stay proportional to the alerts the operator actually opted into.
|
||||
|
||||
The embedded WorkloadsSurface exposes a `compactGroupHeaders` prop on
|
||||
`frontend-modern/src/components/Workloads/useWorkloadsState.ts` that
|
||||
platform pages owning their own hosts table (Proxmox overview today) set
|
||||
|
||||
@@ -199,6 +199,12 @@ controls as normal product settings.
|
||||
This subsystem now gives `L14` an explicit governed home for privacy guidance
|
||||
and telemetry disclosures instead of leaving those trust surfaces as lane-level
|
||||
evidence with no subsystem ownership.
|
||||
The per-rule patrol alert-trigger policy is operator-authored input validated at
|
||||
the API boundary before it reaches persisted AI config: the settings handler
|
||||
(`internal/api/ai_handlers.go`) rejects any minimum-severity value other than
|
||||
`warning` or `critical` and canonicalizes the alert-type allowlist (lowercase,
|
||||
trim, drop blanks, de-duplicate) so untrusted request bodies cannot widen the
|
||||
alert-driven investigation surface beyond the validated shape.
|
||||
That same governed home now also owns the single customer-facing "usage data"
|
||||
vocabulary for anonymous outbound telemetry. Local commercial activation and
|
||||
license-recovery runtime records must stay out of ordinary Settings, support
|
||||
|
||||
@@ -1061,6 +1061,13 @@ is shared inventory context for storage/recovery handoffs only; Pod phase,
|
||||
container readiness, owner, image, and restart fields do not become protection
|
||||
state or recovery-local workload taxonomy.
|
||||
|
||||
The alert payload the router-wired alert bridge (`internal/api/router.go`,
|
||||
`internal/api/ai_handlers.go`) now carries into a scoped patrol — metric type,
|
||||
value, threshold, resource identifier, level, and message — is read-only
|
||||
investigation context for that single run. It must not be persisted as
|
||||
protection state, recovery-local workload taxonomy, or a backup/recovery
|
||||
artifact; storage and recovery state stays owned by their canonical surfaces.
|
||||
|
||||
The Storage and Recovery cross-jump builders
|
||||
(`buildStorageHrefForResource`, `buildRecoveryHrefForResource`) were deleted
|
||||
from `frontend-modern/src/routing/resourceLinks.ts` on 2026-05-16 alongside
|
||||
|
||||
@@ -46,6 +46,7 @@ export function PatrolIntelligenceHeader(props: { state: PatrolIntelligenceState
|
||||
const fieldIds = {
|
||||
alertTriggeredAnalysis: `${fieldIdPrefix}-alert-triggered-analysis`,
|
||||
patrolAlertTriggers: `${fieldIdPrefix}-alert-triggered-patrols`,
|
||||
patrolAlertTriggerMinSeverity: `${fieldIdPrefix}-alert-trigger-min-severity`,
|
||||
patrolAnomalyTriggers: `${fieldIdPrefix}-anomaly-triggered-patrols`,
|
||||
autonomousCriticalRemediation: `${fieldIdPrefix}-autonomous-critical-remediation`,
|
||||
};
|
||||
@@ -396,6 +397,28 @@ export function PatrolIntelligenceHeader(props: { state: PatrolIntelligenceState
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Show when={state.patrolAlertTriggers()}>
|
||||
<div class="pl-3 border-l-2 border-border ml-0.5">
|
||||
<FormSelect
|
||||
label="Investigate alerts at or above"
|
||||
labelClass="text-[11px] font-semibold uppercase tracking-wider text-muted"
|
||||
fieldClass="space-y-1"
|
||||
value={state.patrolAlertTriggerMinSeverity()}
|
||||
onChange={(e) =>
|
||||
state.handlePatrolAlertTriggerMinSeverityChange(
|
||||
e.currentTarget.value === 'warning' ? 'warning' : 'critical',
|
||||
)
|
||||
}
|
||||
disabled={state.isUpdatingSettings() || !state.patrolEnabledLocal()}
|
||||
id={fieldIds.patrolAlertTriggerMinSeverity}
|
||||
selectBaseClass="w-full text-sm bg-base border border-border rounded-md py-1.5 pl-3 pr-8 text-base-content focus:ring-2 focus:ring-blue-500 focus:border-blue-500 disabled:opacity-50"
|
||||
>
|
||||
<option value="critical">Critical only</option>
|
||||
<option value="warning">Warning and critical</option>
|
||||
</FormSelect>
|
||||
</div>
|
||||
</Show>
|
||||
|
||||
<div class="flex items-start justify-between gap-3">
|
||||
<div class="flex-1">
|
||||
<span
|
||||
|
||||
@@ -168,6 +168,9 @@ export function usePatrolIntelligenceState() {
|
||||
const [isTriggeringPatrol, setIsTriggeringPatrol] = createSignal(false);
|
||||
const [alertTriggeredAnalysis, setAlertTriggeredAnalysis] = createSignal<boolean>(false);
|
||||
const [patrolAlertTriggers, setPatrolAlertTriggers] = createSignal<boolean>(true);
|
||||
const [patrolAlertTriggerMinSeverity, setPatrolAlertTriggerMinSeverity] = createSignal<
|
||||
'warning' | 'critical'
|
||||
>('critical');
|
||||
const [patrolAnomalyTriggers, setPatrolAnomalyTriggers] = createSignal<boolean>(true);
|
||||
const [selectedRun, setSelectedRun] = createSignal<PatrolRunRecord | null>(null);
|
||||
const [patrolModelSelectElement, setPatrolModelSelectElement] = createSignal<HTMLSelectElement>();
|
||||
@@ -287,6 +290,9 @@ export function usePatrolIntelligenceState() {
|
||||
setAlertTriggeredAnalysis(!alertAnalysisLocked() && data?.alert_triggered_analysis !== false);
|
||||
const legacyEventTriggersEnabled = data?.patrol_event_triggers_enabled !== false;
|
||||
setPatrolAlertTriggers(data?.patrol_alert_triggers_enabled ?? legacyEventTriggersEnabled);
|
||||
setPatrolAlertTriggerMinSeverity(
|
||||
data?.patrol_alert_trigger_min_severity === 'warning' ? 'warning' : 'critical',
|
||||
);
|
||||
setPatrolAnomalyTriggers(data?.patrol_anomaly_triggers_enabled ?? legacyEventTriggersEnabled);
|
||||
};
|
||||
|
||||
@@ -474,6 +480,32 @@ export function usePatrolIntelligenceState() {
|
||||
}
|
||||
}
|
||||
|
||||
async function handlePatrolAlertTriggerMinSeverityChange(severity: 'warning' | 'critical') {
|
||||
if (isUpdatingSettings()) return;
|
||||
setIsUpdatingSettings(true);
|
||||
setAdvancedSettingsError(null);
|
||||
const previous = patrolAlertTriggerMinSeverity();
|
||||
setPatrolAlertTriggerMinSeverity(severity);
|
||||
try {
|
||||
const updated = await AIAPI.updateSettings({
|
||||
patrol_alert_trigger_min_severity: severity,
|
||||
});
|
||||
syncAIRuntimeSettings(updated);
|
||||
surfaceSavedPatrolReadinessIssue(
|
||||
updated,
|
||||
'Patrol trigger setting was saved, but Patrol is not ready to run.',
|
||||
);
|
||||
} catch (err) {
|
||||
console.error('Failed to update alert-trigger severity threshold:', err);
|
||||
setPatrolAlertTriggerMinSeverity(previous);
|
||||
notificationStore.error(
|
||||
patrolErrorMessage(err, 'Failed to update alert-trigger severity threshold'),
|
||||
);
|
||||
} finally {
|
||||
setIsUpdatingSettings(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handlePatrolAnomalyTriggersChange(enabled: boolean) {
|
||||
if (isUpdatingSettings()) return;
|
||||
setIsUpdatingSettings(true);
|
||||
@@ -949,6 +981,7 @@ export function usePatrolIntelligenceState() {
|
||||
handleIntervalChange,
|
||||
handleModelChange,
|
||||
handlePatrolAlertTriggersChange,
|
||||
handlePatrolAlertTriggerMinSeverityChange,
|
||||
handlePatrolAnomalyTriggersChange,
|
||||
handleRunPatrol,
|
||||
handleTogglePatrol,
|
||||
@@ -967,6 +1000,7 @@ export function usePatrolIntelligenceState() {
|
||||
openAdvancedSettingsErrorInAssistant,
|
||||
patrolEnabledLocal,
|
||||
patrolAlertTriggers,
|
||||
patrolAlertTriggerMinSeverity,
|
||||
patrolAnomalyTriggers,
|
||||
patrolInterval,
|
||||
patrolModel,
|
||||
|
||||
@@ -809,6 +809,59 @@ describe('AIIntelligence entitlement gating', () => {
|
||||
expect(screen.queryByRole('link', { name: 'Upgrade' })).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('persists the alert-trigger severity floor from the Patrol configuration panel', async () => {
|
||||
hasFeatureMock.mockReturnValue(true);
|
||||
licenseStatusMock.mockReturnValue({ subscription_state: 'active' });
|
||||
getPatrolStatusMock.mockResolvedValue(defaultPatrolStatus({ license_required: false }));
|
||||
apiFetchJSONMock.mockImplementation(async (path: string) => {
|
||||
if (path === '/api/settings/ai') {
|
||||
return {
|
||||
...defaultAISettings,
|
||||
patrol_alert_triggers_enabled: true,
|
||||
patrol_alert_trigger_min_severity: 'critical',
|
||||
};
|
||||
}
|
||||
if (path === '/api/settings/ai/update') {
|
||||
return {
|
||||
...defaultAISettings,
|
||||
patrol_alert_triggers_enabled: true,
|
||||
patrol_alert_trigger_min_severity: 'warning',
|
||||
};
|
||||
}
|
||||
return {};
|
||||
});
|
||||
|
||||
render(() => <AIIntelligence />);
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole('heading', { name: 'Patrol' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Configure Patrol' }));
|
||||
|
||||
const select = (await screen.findByLabelText(
|
||||
'Investigate alerts at or above',
|
||||
)) as HTMLSelectElement;
|
||||
expect(select.value).toBe('critical');
|
||||
expect(Array.from(select.options).map((option) => option.value)).toEqual([
|
||||
'critical',
|
||||
'warning',
|
||||
]);
|
||||
|
||||
fireEvent.change(select, { target: { value: 'warning' } });
|
||||
|
||||
await waitFor(() => {
|
||||
expect(apiFetchJSONMock).toHaveBeenCalledWith('/api/settings/ai/update', {
|
||||
patrol_alert_trigger_min_severity: 'warning',
|
||||
});
|
||||
});
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
(screen.getByLabelText('Investigate alerts at or above') as HTMLSelectElement).value,
|
||||
).toBe('warning');
|
||||
});
|
||||
});
|
||||
|
||||
it('renders the canonical intelligence summary card with recent changes', async () => {
|
||||
hasFeatureMock.mockReturnValue(true);
|
||||
licenseStatusMock.mockReturnValue({ subscription_state: 'active' });
|
||||
|
||||
@@ -50,6 +50,8 @@ export interface AISettings {
|
||||
patrol_event_triggers_enabled?: boolean; // legacy aggregate toggle, true if any scoped Patrol trigger source is enabled
|
||||
patrol_alert_triggers_enabled?: boolean; // true if alert-driven scoped Patrol triggers are enabled
|
||||
patrol_anomaly_triggers_enabled?: boolean; // true if anomaly-driven scoped Patrol triggers are enabled
|
||||
patrol_alert_trigger_min_severity?: 'warning' | 'critical'; // minimum alert level that triggers a scoped investigation
|
||||
patrol_alert_trigger_types?: string[]; // optional allowlist of alert types (empty = all types)
|
||||
patrol_auto_fix?: boolean; // true if Patrol can remediate without approval
|
||||
// Multi-provider configuration
|
||||
anthropic_configured: boolean; // true if Anthropic API key or OAuth is set
|
||||
@@ -117,6 +119,8 @@ export interface AISettingsUpdateRequest {
|
||||
patrol_event_triggers_enabled?: boolean; // legacy aggregate toggle, applies to both scoped Patrol trigger sources
|
||||
patrol_alert_triggers_enabled?: boolean; // true if alert-driven scoped Patrol triggers are enabled
|
||||
patrol_anomaly_triggers_enabled?: boolean; // true if anomaly-driven scoped Patrol triggers are enabled
|
||||
patrol_alert_trigger_min_severity?: 'warning' | 'critical'; // minimum alert level that triggers a scoped investigation
|
||||
patrol_alert_trigger_types?: string[]; // optional allowlist of alert types (empty = all types)
|
||||
patrol_auto_fix?: boolean; // true if Patrol can remediate without approval
|
||||
// Multi-provider credentials
|
||||
anthropic_api_key?: string; // Set Anthropic API key
|
||||
|
||||
@@ -3289,7 +3289,19 @@ func (p *PatrolService) seedFindingsAndContextState(scope *PatrolScope, snap pat
|
||||
sb.WriteString(fmt.Sprintf("- Guest Memory warning: %.0f%%\n", thresholds.GuestMemWarning))
|
||||
sb.WriteString(fmt.Sprintf("- Guest Disk warning: %.0f%%, critical: %.0f%%\n", thresholds.GuestDiskWarn, thresholds.GuestDiskCrit))
|
||||
sb.WriteString(fmt.Sprintf("- Storage warning: %.0f%%, critical: %.0f%%\n", thresholds.StorageWarning, thresholds.StorageCritical))
|
||||
sb.WriteString("Note: The real-time alerting system monitors these thresholds continuously. Do NOT report findings for threshold breaches — focus on trends, capacity planning, and issues alerts cannot detect.\n\n")
|
||||
if scope != nil && scope.Reason == TriggerReasonAlertFired && scope.AlertContext != nil {
|
||||
ac := scope.AlertContext
|
||||
level := ac.Level
|
||||
if level == "" {
|
||||
level = "threshold"
|
||||
}
|
||||
sb.WriteString(fmt.Sprintf("Note: A live %s alert just fired on the scoped resource: %s = %.1f (threshold %.1f). "+
|
||||
"Investigate the root cause of THIS breach specifically: what changed, whether it is transient or sustained, the blast radius on dependent workloads, and the concrete remediation. "+
|
||||
"Reporting a finding for this breach is expected. It is the reason this patrol was triggered.\n\n",
|
||||
level, ac.AlertType, ac.Value, ac.Threshold))
|
||||
} else {
|
||||
sb.WriteString("Note: The real-time alerting system monitors these thresholds continuously. Do NOT report findings for threshold breaches. Focus on trends, capacity planning, and issues alerts cannot detect.\n\n")
|
||||
}
|
||||
|
||||
scopedResources := patrolRuntimeKnownResources(snap)
|
||||
stateHasScopedResources := len(scopedResources) > 0
|
||||
|
||||
@@ -57,6 +57,10 @@ type PatrolScope struct {
|
||||
Priority int
|
||||
// AlertIdentifier is the canonical ID of the alert that triggered this patrol (if applicable)
|
||||
AlertIdentifier string
|
||||
// AlertContext carries the firing alert's specifics (metric, value, threshold)
|
||||
// so an alert-triggered patrol can investigate the actual breach instead of
|
||||
// re-checking the resource generically. Nil for non-alert triggers.
|
||||
AlertContext *PatrolAlertContext
|
||||
// FindingID is the ID of the finding that triggered this patrol (if applicable)
|
||||
FindingID string
|
||||
// NoStream skips streaming updates (phase, content, subscriber notifications).
|
||||
@@ -68,6 +72,16 @@ type PatrolScope struct {
|
||||
RetryAfter time.Time
|
||||
}
|
||||
|
||||
// PatrolAlertContext describes the alert that triggered a scoped patrol, so the
|
||||
// patrol prompt can focus the investigation on the specific breach.
|
||||
type PatrolAlertContext struct {
|
||||
AlertType string // cpu, memory, disk, etc.
|
||||
Level string // warning | critical
|
||||
Value float64 // observed metric value at fire time
|
||||
Threshold float64 // threshold that was crossed
|
||||
Message string // human-readable alert message
|
||||
}
|
||||
|
||||
// PatrolDepth controls how thorough a patrol run should be
|
||||
type PatrolDepth int
|
||||
|
||||
|
||||
@@ -42,8 +42,23 @@ type AlertProvider interface {
|
||||
SetResolvedCallback(cb func(alertID string))
|
||||
}
|
||||
|
||||
// PatrolTriggerEvent carries the firing (or resolving) alert's specifics so a
|
||||
// scoped patrol can investigate the actual issue (the metric, the value, the
|
||||
// threshold that was crossed) instead of re-checking the resource generically.
|
||||
type PatrolTriggerEvent struct {
|
||||
ResourceID string
|
||||
ResourceType string
|
||||
Reason string // "alert_fired" | "alert_cleared"
|
||||
AlertType string // cpu, memory, disk, etc.
|
||||
AlertIdentifier string
|
||||
AlertLevel string // warning | critical
|
||||
Value float64
|
||||
Threshold float64
|
||||
Message string
|
||||
}
|
||||
|
||||
// PatrolTriggerFunc is called to trigger a mini-patrol for a resource
|
||||
type PatrolTriggerFunc func(resourceID, resourceType, reason, alertType string)
|
||||
type PatrolTriggerFunc func(event PatrolTriggerEvent)
|
||||
|
||||
// AIEnhancementFunc is called to request AI enhancement of a finding
|
||||
type AIEnhancementFunc func(findingID string) error
|
||||
@@ -200,7 +215,17 @@ func (b *AlertBridge) handleNewAlert(alert AlertAdapter) {
|
||||
|
||||
// Trigger mini-patrol for the resource
|
||||
if triggerPatrol && patrolFn != nil {
|
||||
go patrolFn(finding.ResourceID, finding.ResourceType, "alert_fired", finding.AlertType)
|
||||
go patrolFn(PatrolTriggerEvent{
|
||||
ResourceID: finding.ResourceID,
|
||||
ResourceType: finding.ResourceType,
|
||||
Reason: "alert_fired",
|
||||
AlertType: finding.AlertType,
|
||||
AlertIdentifier: alert.GetAlertIdentifier(),
|
||||
AlertLevel: alert.GetAlertLevel(),
|
||||
Value: alert.GetValue(),
|
||||
Threshold: alert.GetThreshold(),
|
||||
Message: alert.GetMessage(),
|
||||
})
|
||||
}
|
||||
|
||||
// Schedule AI enhancement
|
||||
@@ -236,7 +261,16 @@ func (b *AlertBridge) handleAlertResolved(alertID string) {
|
||||
|
||||
// Trigger verification patrol
|
||||
if triggerPatrol && patrolFn != nil && finding != nil {
|
||||
go patrolFn(finding.ResourceID, finding.ResourceType, "alert_cleared", finding.AlertType)
|
||||
go patrolFn(PatrolTriggerEvent{
|
||||
ResourceID: finding.ResourceID,
|
||||
ResourceType: finding.ResourceType,
|
||||
Reason: "alert_cleared",
|
||||
AlertType: finding.AlertType,
|
||||
AlertIdentifier: finding.AlertIdentifier,
|
||||
Value: finding.Value,
|
||||
Threshold: finding.Threshold,
|
||||
Message: finding.Description,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -76,8 +76,8 @@ func TestAlertBridge_HandleNewAlertAndEnhance(t *testing.T) {
|
||||
bridge.running = true
|
||||
|
||||
patrolCh := make(chan string, 1)
|
||||
bridge.SetPatrolTrigger(func(resourceID, resourceType, reason, alertType string) {
|
||||
patrolCh <- reason
|
||||
bridge.SetPatrolTrigger(func(event PatrolTriggerEvent) {
|
||||
patrolCh <- event.Reason
|
||||
})
|
||||
|
||||
enhanceCh := make(chan string, 1)
|
||||
@@ -124,8 +124,8 @@ func TestAlertBridge_HandleAlertResolved(t *testing.T) {
|
||||
bridge.running = true
|
||||
|
||||
patrolCh := make(chan string, 1)
|
||||
bridge.SetPatrolTrigger(func(resourceID, resourceType, reason, alertType string) {
|
||||
patrolCh <- reason
|
||||
bridge.SetPatrolTrigger(func(event PatrolTriggerEvent) {
|
||||
patrolCh <- event.Reason
|
||||
})
|
||||
|
||||
alert := &SimpleAlertAdapter{
|
||||
|
||||
@@ -42,7 +42,7 @@ type SetupResult struct {
|
||||
// result, err := unified.Setup(unified.SetupConfig{
|
||||
// DataDir: dataPath,
|
||||
// AlertManager: alertManager,
|
||||
// PatrolTriggerFunc: func(resourceID, resourceType, reason, alertType string) { patrol.Trigger(resourceID, reason) },
|
||||
// PatrolTriggerFunc: func(event unified.PatrolTriggerEvent) { patrol.Trigger(event.ResourceID, event.Reason) },
|
||||
// AutoEnhance: true,
|
||||
// TriggerPatrolOnAlert: true,
|
||||
// })
|
||||
|
||||
@@ -32,7 +32,7 @@ func TestQuickSetup(t *testing.T) {
|
||||
|
||||
func TestSetupWithPatrol(t *testing.T) {
|
||||
manager := alerts.NewManager()
|
||||
result, err := SetupWithPatrol(manager, t.TempDir(), func(resourceID, resourceType, reason, alertType string) {})
|
||||
result, err := SetupWithPatrol(manager, t.TempDir(), func(event PatrolTriggerEvent) {})
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
|
||||
+76
-37
@@ -2267,15 +2267,18 @@ type AISettingsResponse struct {
|
||||
AuthMethod string `json:"auth_method"` // "api_key" or "oauth"
|
||||
OAuthConnected bool `json:"oauth_connected"` // true if OAuth tokens are configured
|
||||
// Patrol settings for token efficiency
|
||||
PatrolIntervalMinutes int `json:"patrol_interval_minutes"` // Patrol interval in minutes (0 = disabled)
|
||||
PatrolEnabled bool `json:"patrol_enabled"` // true if patrol is enabled
|
||||
PatrolAutoFix bool `json:"patrol_auto_fix"` // true if patrol can auto-fix issues
|
||||
AlertTriggeredAnalysis bool `json:"alert_triggered_analysis"` // true if AI analyzes when alerts fire
|
||||
PatrolEventTriggersEnabled bool `json:"patrol_event_triggers_enabled"` // Legacy aggregate flag; true when any scoped Patrol trigger source is enabled
|
||||
PatrolAlertTriggersEnabled bool `json:"patrol_alert_triggers_enabled"` // true if alert-driven scoped Patrol triggers are enabled
|
||||
PatrolAnomalyTriggersEnabled bool `json:"patrol_anomaly_triggers_enabled"` // true if anomaly-driven scoped Patrol triggers are enabled
|
||||
UseProactiveThresholds bool `json:"use_proactive_thresholds"` // true if patrol warns before thresholds (false = use exact thresholds)
|
||||
AvailableModels []providers.ModelInfo `json:"available_models"` // List of models for current provider
|
||||
PatrolIntervalMinutes int `json:"patrol_interval_minutes"` // Patrol interval in minutes (0 = disabled)
|
||||
PatrolEnabled bool `json:"patrol_enabled"` // true if patrol is enabled
|
||||
PatrolAutoFix bool `json:"patrol_auto_fix"` // true if patrol can auto-fix issues
|
||||
AlertTriggeredAnalysis bool `json:"alert_triggered_analysis"` // true if AI analyzes when alerts fire
|
||||
PatrolEventTriggersEnabled bool `json:"patrol_event_triggers_enabled"` // Legacy aggregate flag; true when any scoped Patrol trigger source is enabled
|
||||
PatrolAlertTriggersEnabled bool `json:"patrol_alert_triggers_enabled"` // true if alert-driven scoped Patrol triggers are enabled
|
||||
PatrolAnomalyTriggersEnabled bool `json:"patrol_anomaly_triggers_enabled"` // true if anomaly-driven scoped Patrol triggers are enabled
|
||||
// Per-rule policy for alert-driven scoped Patrol triggers.
|
||||
PatrolAlertTriggerMinSeverity string `json:"patrol_alert_trigger_min_severity"` // "warning" | "critical"; minimum alert level that warrants investigation
|
||||
PatrolAlertTriggerTypes []string `json:"patrol_alert_trigger_types"` // optional allowlist of alert types (empty = all types)
|
||||
UseProactiveThresholds bool `json:"use_proactive_thresholds"` // true if patrol warns before thresholds (false = use exact thresholds)
|
||||
AvailableModels []providers.ModelInfo `json:"available_models"` // List of models for current provider
|
||||
// Multi-provider credentials - shows which providers are configured
|
||||
AnthropicConfigured bool `json:"anthropic_configured"` // true if Anthropic API key or OAuth is set
|
||||
OpenAIConfigured bool `json:"openai_configured"` // true if OpenAI API key is set
|
||||
@@ -2338,6 +2341,9 @@ func (r AISettingsResponse) NormalizeCollections() AISettingsResponse {
|
||||
if r.ProtectedGuests == nil {
|
||||
r.ProtectedGuests = []string{}
|
||||
}
|
||||
if r.PatrolAlertTriggerTypes == nil {
|
||||
r.PatrolAlertTriggerTypes = []string{}
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
@@ -2358,7 +2364,10 @@ type AISettingsUpdateRequest struct {
|
||||
PatrolEventTriggersEnabled *bool `json:"patrol_event_triggers_enabled,omitempty"` // Legacy aggregate update; applies to both scoped Patrol trigger sources
|
||||
PatrolAlertTriggersEnabled *bool `json:"patrol_alert_triggers_enabled,omitempty"` // true if alert-driven scoped Patrol triggers are enabled
|
||||
PatrolAnomalyTriggersEnabled *bool `json:"patrol_anomaly_triggers_enabled,omitempty"` // true if anomaly-driven scoped Patrol triggers are enabled
|
||||
UseProactiveThresholds *bool `json:"use_proactive_thresholds,omitempty"` // true if patrol warns before thresholds (default: false = exact thresholds)
|
||||
// Per-rule policy for alert-driven scoped Patrol triggers.
|
||||
PatrolAlertTriggerMinSeverity *string `json:"patrol_alert_trigger_min_severity,omitempty"` // "warning" | "critical"
|
||||
PatrolAlertTriggerTypes *[]string `json:"patrol_alert_trigger_types,omitempty"` // allowlist of alert types (empty slice = all types)
|
||||
UseProactiveThresholds *bool `json:"use_proactive_thresholds,omitempty"` // true if patrol warns before thresholds (default: false = exact thresholds)
|
||||
// Multi-provider credentials
|
||||
AnthropicAPIKey *string `json:"anthropic_api_key,omitempty"` // Set Anthropic API key
|
||||
OpenAIAPIKey *string `json:"openai_api_key,omitempty"` // Set OpenAI API key
|
||||
@@ -2486,15 +2495,17 @@ func (h *AISettingsHandler) HandleGetAISettings(w http.ResponseWriter, r *http.R
|
||||
AuthMethod: authMethod,
|
||||
OAuthConnected: settings.OAuthAccessToken != "",
|
||||
// Patrol settings
|
||||
PatrolIntervalMinutes: settings.PatrolIntervalMinutes,
|
||||
PatrolEnabled: settings.PatrolEnabled,
|
||||
PatrolAutoFix: settings.PatrolAutoFix && hasAutoFixFeature,
|
||||
AlertTriggeredAnalysis: settings.AlertTriggeredAnalysis && hasAlertAnalysisFeature,
|
||||
PatrolEventTriggersEnabled: triggerSettings.AlertTriggersEnabled || triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggersEnabled: triggerSettings.AlertTriggersEnabled,
|
||||
PatrolAnomalyTriggersEnabled: triggerSettings.AnomalyTriggersEnabled,
|
||||
UseProactiveThresholds: settings.UseProactiveThresholds,
|
||||
AvailableModels: nil, // Now populated via /api/ai/models endpoint
|
||||
PatrolIntervalMinutes: settings.PatrolIntervalMinutes,
|
||||
PatrolEnabled: settings.PatrolEnabled,
|
||||
PatrolAutoFix: settings.PatrolAutoFix && hasAutoFixFeature,
|
||||
AlertTriggeredAnalysis: settings.AlertTriggeredAnalysis && hasAlertAnalysisFeature,
|
||||
PatrolEventTriggersEnabled: triggerSettings.AlertTriggersEnabled || triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggersEnabled: triggerSettings.AlertTriggersEnabled,
|
||||
PatrolAnomalyTriggersEnabled: triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggerMinSeverity: settings.GetPatrolAlertTriggerMinSeverity(),
|
||||
PatrolAlertTriggerTypes: settings.PatrolAlertTriggerTypes,
|
||||
UseProactiveThresholds: settings.UseProactiveThresholds,
|
||||
AvailableModels: nil, // Now populated via /api/ai/models endpoint
|
||||
// Multi-provider configuration
|
||||
AnthropicConfigured: settings.HasProvider(config.AIProviderAnthropic),
|
||||
OpenAIConfigured: settings.HasProvider(config.AIProviderOpenAI),
|
||||
@@ -2760,6 +2771,32 @@ func (h *AISettingsHandler) HandleUpdateAISettings(w http.ResponseWriter, r *htt
|
||||
)
|
||||
}
|
||||
|
||||
// Handle alert-trigger investigation policy (minimum severity + type allowlist)
|
||||
if req.PatrolAlertTriggerMinSeverity != nil {
|
||||
sev := strings.ToLower(strings.TrimSpace(*req.PatrolAlertTriggerMinSeverity))
|
||||
if sev != config.AlertTriggerSeverityWarning && sev != config.AlertTriggerSeverityCritical {
|
||||
http.Error(w, "patrol_alert_trigger_min_severity must be 'warning' or 'critical'", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
settings.PatrolAlertTriggerMinSeverity = sev
|
||||
}
|
||||
if req.PatrolAlertTriggerTypes != nil {
|
||||
cleaned := make([]string, 0, len(*req.PatrolAlertTriggerTypes))
|
||||
seen := make(map[string]struct{}, len(*req.PatrolAlertTriggerTypes))
|
||||
for _, t := range *req.PatrolAlertTriggerTypes {
|
||||
t = strings.ToLower(strings.TrimSpace(t))
|
||||
if t == "" {
|
||||
continue
|
||||
}
|
||||
if _, dup := seen[t]; dup {
|
||||
continue
|
||||
}
|
||||
seen[t] = struct{}{}
|
||||
cleaned = append(cleaned, t)
|
||||
}
|
||||
settings.PatrolAlertTriggerTypes = cleaned
|
||||
}
|
||||
|
||||
// Handle request timeout (for slow hardware)
|
||||
if req.RequestTimeoutSeconds != nil {
|
||||
if *req.RequestTimeoutSeconds < 0 {
|
||||
@@ -2902,24 +2939,26 @@ func (h *AISettingsHandler) HandleUpdateAISettings(w http.ResponseWriter, r *htt
|
||||
|
||||
// Return updated settings
|
||||
response := AISettingsResponse{
|
||||
Enabled: settings.Enabled,
|
||||
Model: settings.GetModel(),
|
||||
ChatModel: config.NormalizeQuickstartModelString(settings.ChatModel),
|
||||
PatrolModel: config.NormalizeQuickstartModelString(settings.PatrolModel),
|
||||
AutoFixModel: config.NormalizeQuickstartModelString(settings.AutoFixModel),
|
||||
Configured: settings.IsConfigured(),
|
||||
CustomContext: settings.CustomContext,
|
||||
AuthMethod: authMethod,
|
||||
OAuthConnected: settings.OAuthAccessToken != "",
|
||||
PatrolIntervalMinutes: settings.PatrolIntervalMinutes,
|
||||
PatrolEnabled: settings.PatrolEnabled,
|
||||
PatrolAutoFix: settings.PatrolAutoFix && hasAutoFixFeature,
|
||||
AlertTriggeredAnalysis: settings.AlertTriggeredAnalysis && hasAlertAnalysisFeature,
|
||||
PatrolEventTriggersEnabled: triggerSettings.AlertTriggersEnabled || triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggersEnabled: triggerSettings.AlertTriggersEnabled,
|
||||
PatrolAnomalyTriggersEnabled: triggerSettings.AnomalyTriggersEnabled,
|
||||
UseProactiveThresholds: settings.UseProactiveThresholds,
|
||||
AvailableModels: nil, // Now populated via /api/ai/models endpoint
|
||||
Enabled: settings.Enabled,
|
||||
Model: settings.GetModel(),
|
||||
ChatModel: config.NormalizeQuickstartModelString(settings.ChatModel),
|
||||
PatrolModel: config.NormalizeQuickstartModelString(settings.PatrolModel),
|
||||
AutoFixModel: config.NormalizeQuickstartModelString(settings.AutoFixModel),
|
||||
Configured: settings.IsConfigured(),
|
||||
CustomContext: settings.CustomContext,
|
||||
AuthMethod: authMethod,
|
||||
OAuthConnected: settings.OAuthAccessToken != "",
|
||||
PatrolIntervalMinutes: settings.PatrolIntervalMinutes,
|
||||
PatrolEnabled: settings.PatrolEnabled,
|
||||
PatrolAutoFix: settings.PatrolAutoFix && hasAutoFixFeature,
|
||||
AlertTriggeredAnalysis: settings.AlertTriggeredAnalysis && hasAlertAnalysisFeature,
|
||||
PatrolEventTriggersEnabled: triggerSettings.AlertTriggersEnabled || triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggersEnabled: triggerSettings.AlertTriggersEnabled,
|
||||
PatrolAnomalyTriggersEnabled: triggerSettings.AnomalyTriggersEnabled,
|
||||
PatrolAlertTriggerMinSeverity: settings.GetPatrolAlertTriggerMinSeverity(),
|
||||
PatrolAlertTriggerTypes: settings.PatrolAlertTriggerTypes,
|
||||
UseProactiveThresholds: settings.UseProactiveThresholds,
|
||||
AvailableModels: nil, // Now populated via /api/ai/models endpoint
|
||||
// Multi-provider configuration
|
||||
AnthropicConfigured: settings.HasProvider(config.AIProviderAnthropic),
|
||||
OpenAIConfigured: settings.HasProvider(config.AIProviderOpenAI),
|
||||
|
||||
@@ -592,6 +592,57 @@ func TestAISettingsHandler_UpdateSettingsRejectsInvalidOllamaKeepAlive(t *testin
|
||||
require.Contains(t, rec.Body.String(), "ollama_keep_alive")
|
||||
}
|
||||
|
||||
func TestAISettingsHandler_UpdateSettings_PatrolAlertTriggerPolicy(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
cfg := &config.Config{DataPath: tmp}
|
||||
persistence := config.NewConfigPersistence(tmp)
|
||||
handler := newTestAISettingsHandler(cfg, persistence, nil)
|
||||
|
||||
body, err := json.Marshal(AISettingsUpdateRequest{
|
||||
PatrolAlertTriggerMinSeverity: ptr("warning"),
|
||||
PatrolAlertTriggerTypes: ptr([]string{"CPU", " cpu ", "memory", ""}),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
req := newLoopbackRequest(http.MethodPut, "/api/settings/ai/update", bytes.NewReader(body))
|
||||
rec := httptest.NewRecorder()
|
||||
handler.HandleUpdateAISettings(rec, req)
|
||||
|
||||
require.Equal(t, http.StatusOK, rec.Code, rec.Body.String())
|
||||
var resp AISettingsResponse
|
||||
require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &resp))
|
||||
require.Equal(t, "warning", resp.PatrolAlertTriggerMinSeverity)
|
||||
require.Equal(t, []string{"cpu", "memory"}, resp.PatrolAlertTriggerTypes)
|
||||
|
||||
saved, err := persistence.LoadAIConfig()
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "warning", saved.PatrolAlertTriggerMinSeverity)
|
||||
require.Equal(t, []string{"cpu", "memory"}, saved.PatrolAlertTriggerTypes)
|
||||
}
|
||||
|
||||
func TestAISettingsHandler_UpdateSettingsRejectsInvalidPatrolAlertTriggerMinSeverity(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
cfg := &config.Config{DataPath: tmp}
|
||||
persistence := config.NewConfigPersistence(tmp)
|
||||
handler := newTestAISettingsHandler(cfg, persistence, nil)
|
||||
|
||||
body, err := json.Marshal(AISettingsUpdateRequest{
|
||||
PatrolAlertTriggerMinSeverity: ptr("emergency"),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
req := newLoopbackRequest(http.MethodPut, "/api/settings/ai/update", bytes.NewReader(body))
|
||||
rec := httptest.NewRecorder()
|
||||
handler.HandleUpdateAISettings(rec, req)
|
||||
|
||||
require.Equal(t, http.StatusBadRequest, rec.Code, rec.Body.String())
|
||||
require.Contains(t, rec.Body.String(), "patrol_alert_trigger_min_severity")
|
||||
}
|
||||
|
||||
func TestAISettingsHandler_GetAIService_TenantPatrolUsesCanonicalProviders(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
mtp := config.NewMultiTenantPersistence(tmp)
|
||||
|
||||
@@ -1255,6 +1255,8 @@ func TestContract_AISettingsUpdateProviderResolutionJSONSnapshot(t *testing.T) {
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
@@ -1397,6 +1399,8 @@ func TestContract_AISettingsBYOKOverrideDoesNotExposeQuickstartInventoryJSONSnap
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
@@ -3400,6 +3404,8 @@ func TestContract_HostedAISettingsDoesNotAutoBootstrapQuickstartJSONSnapshot(t *
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
@@ -3461,6 +3467,8 @@ func TestContract_AISettingsRetiredQuickstartAliasJSONSnapshot(t *testing.T) {
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
@@ -3526,6 +3534,8 @@ func TestContract_AISettingsOllamaAuthJSONSnapshot(t *testing.T) {
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
@@ -4022,6 +4032,8 @@ func TestContract_HostedTenantAISettingsDoesNotAutoBootstrapQuickstartJSONSnapsh
|
||||
"patrol_event_triggers_enabled":true,
|
||||
"patrol_alert_triggers_enabled":true,
|
||||
"patrol_anomaly_triggers_enabled":true,
|
||||
"patrol_alert_trigger_min_severity":"critical",
|
||||
"patrol_alert_trigger_types":[],
|
||||
"use_proactive_thresholds":false,
|
||||
"available_models":[],
|
||||
"anthropic_configured":false,
|
||||
|
||||
+38
-17
@@ -2206,45 +2206,66 @@ func (r *Router) initializeAIIntelligenceServices(ctx context.Context, orgID, da
|
||||
Str("org_id", orgID).
|
||||
Msg("Pulse dev background AI disabled; alert bridge patrol trigger not wired")
|
||||
} else if patrol != nil {
|
||||
alertBridge.SetPatrolTrigger(func(resourceID, resourceType, reason, alertType string) {
|
||||
alertBridge.SetPatrolTrigger(func(event unified.PatrolTriggerEvent) {
|
||||
scope := ai.PatrolScope{
|
||||
ResourceIDs: []string{resourceID},
|
||||
ResourceTypes: []string{resourceType},
|
||||
Depth: ai.PatrolDepthQuick,
|
||||
Context: "Alert bridge: " + reason,
|
||||
Priority: 50,
|
||||
ResourceIDs: []string{event.ResourceID},
|
||||
ResourceTypes: []string{event.ResourceType},
|
||||
Depth: ai.PatrolDepthQuick,
|
||||
Context: "Alert bridge: " + event.Reason,
|
||||
Priority: 50,
|
||||
AlertIdentifier: event.AlertIdentifier,
|
||||
}
|
||||
switch reason {
|
||||
if event.AlertType != "" {
|
||||
scope.AlertContext = &ai.PatrolAlertContext{
|
||||
AlertType: event.AlertType,
|
||||
Level: event.AlertLevel,
|
||||
Value: event.Value,
|
||||
Threshold: event.Threshold,
|
||||
Message: event.Message,
|
||||
}
|
||||
}
|
||||
switch event.Reason {
|
||||
case "alert_fired":
|
||||
// Per-rule policy: only investigate alerts the operator opted
|
||||
// in (minimum severity + optional alert-type allowlist).
|
||||
if aiCfg := aiService.GetAIConfig(); aiCfg != nil &&
|
||||
!aiCfg.AlertTriggersInvestigation(event.AlertType, event.AlertLevel) {
|
||||
log.Debug().
|
||||
Str("resource_id", event.ResourceID).
|
||||
Str("alert_type", event.AlertType).
|
||||
Str("level", event.AlertLevel).
|
||||
Msg("Alert bridge: alert-triggered patrol skipped by trigger policy")
|
||||
return
|
||||
}
|
||||
scope.Reason = ai.TriggerReasonAlertFired
|
||||
scope.Priority = 80
|
||||
if alertType != "" {
|
||||
scope.Context = "Alert: " + alertType
|
||||
if event.AlertType != "" {
|
||||
scope.Context = fmt.Sprintf("Alert: %s = %.1f (threshold %.1f)", event.AlertType, event.Value, event.Threshold)
|
||||
}
|
||||
case "alert_cleared":
|
||||
scope.Reason = ai.TriggerReasonAlertCleared
|
||||
scope.Priority = 40
|
||||
if alertType != "" {
|
||||
scope.Context = "Alert cleared: " + alertType
|
||||
if event.AlertType != "" {
|
||||
scope.Context = "Alert cleared: " + event.AlertType
|
||||
}
|
||||
default:
|
||||
scope.Reason = ai.TriggerReasonManual
|
||||
}
|
||||
|
||||
log.Debug().
|
||||
Str("resource_id", resourceID).
|
||||
Str("reason", reason).
|
||||
Str("resource_id", event.ResourceID).
|
||||
Str("reason", event.Reason).
|
||||
Msg("Alert bridge: Triggering mini-patrol")
|
||||
if triggerManager := r.aiSettingsHandler.GetTriggerManagerForOrg(orgID); triggerManager != nil {
|
||||
if triggerManager.TriggerPatrol(scope) {
|
||||
log.Debug().
|
||||
Str("resource_id", resourceID).
|
||||
Str("reason", reason).
|
||||
Str("resource_id", event.ResourceID).
|
||||
Str("reason", event.Reason).
|
||||
Msg("Alert bridge: Queued patrol via trigger manager")
|
||||
} else {
|
||||
log.Warn().
|
||||
Str("resource_id", resourceID).
|
||||
Str("reason", reason).
|
||||
Str("resource_id", event.ResourceID).
|
||||
Str("reason", event.Reason).
|
||||
Msg("Alert bridge: Patrol trigger rejected by trigger manager")
|
||||
}
|
||||
return
|
||||
|
||||
@@ -73,6 +73,14 @@ type AIConfig struct {
|
||||
PatrolAlertTriggersEnabled bool `json:"patrol_alert_triggers_enabled"`
|
||||
PatrolAnomalyTriggersEnabled bool `json:"patrol_anomaly_triggers_enabled"`
|
||||
|
||||
// Fine-grained control over which firing alerts trigger a scoped patrol.
|
||||
// PatrolAlertTriggerMinSeverity is the minimum alert level that warrants an
|
||||
// investigation ("warning" accepts warning+critical; "critical" accepts only
|
||||
// critical; empty = "critical"). PatrolAlertTriggerTypes optionally restricts
|
||||
// triggering to specific alert types (cpu, memory, disk, ...); empty = all types.
|
||||
PatrolAlertTriggerMinSeverity string `json:"patrol_alert_trigger_min_severity,omitempty"`
|
||||
PatrolAlertTriggerTypes []string `json:"patrol_alert_trigger_types,omitempty"`
|
||||
|
||||
// Request timeout - how long to wait for AI responses (default: 300s / 5 min)
|
||||
// Increase this for slow hardware running local models (e.g., Ollama on low-power devices)
|
||||
RequestTimeoutSeconds int `json:"request_timeout_seconds,omitempty"`
|
||||
@@ -210,9 +218,82 @@ func NewDefaultAIConfig() *AIConfig {
|
||||
PatrolEventTriggersEnabled: true,
|
||||
PatrolAlertTriggersEnabled: true,
|
||||
PatrolAnomalyTriggersEnabled: true,
|
||||
// Default to critical-only so alert-triggered investigations stay
|
||||
// token-conservative out of the box. Operators can opt warnings in.
|
||||
PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical,
|
||||
}
|
||||
}
|
||||
|
||||
// Alert-trigger minimum-severity sentinels.
|
||||
const (
|
||||
AlertTriggerSeverityWarning = "warning"
|
||||
AlertTriggerSeverityCritical = "critical"
|
||||
)
|
||||
|
||||
// GetPatrolAlertTriggerMinSeverity returns the configured minimum alert level
|
||||
// that warrants a scoped investigation patrol, normalizing the empty default to
|
||||
// critical-only.
|
||||
func (c *AIConfig) GetPatrolAlertTriggerMinSeverity() string {
|
||||
if c == nil {
|
||||
return AlertTriggerSeverityCritical
|
||||
}
|
||||
min := strings.ToLower(strings.TrimSpace(c.PatrolAlertTriggerMinSeverity))
|
||||
switch min {
|
||||
case AlertTriggerSeverityWarning, AlertTriggerSeverityCritical:
|
||||
return min
|
||||
default:
|
||||
return AlertTriggerSeverityCritical
|
||||
}
|
||||
}
|
||||
|
||||
// AlertTriggersInvestigation reports whether a firing alert of the given type
|
||||
// and level should trigger a scoped investigation patrol, per the operator's
|
||||
// alert-trigger policy. It enforces the master enable, the minimum-severity
|
||||
// floor, and the optional alert-type allowlist.
|
||||
func (c *AIConfig) AlertTriggersInvestigation(alertType, level string) bool {
|
||||
if c == nil || !c.PatrolAlertTriggersEnabled {
|
||||
return false
|
||||
}
|
||||
if !alertLevelMeetsMinimum(level, c.PatrolAlertTriggerMinSeverity) {
|
||||
return false
|
||||
}
|
||||
if len(c.PatrolAlertTriggerTypes) > 0 && !alertTypeAllowed(alertType, c.PatrolAlertTriggerTypes) {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// alertLevelMeetsMinimum ranks warning < critical. An empty minimum defaults to
|
||||
// critical-only. An unknown alert level is treated as critical so it is never
|
||||
// silently dropped.
|
||||
func alertLevelMeetsMinimum(level, minimum string) bool {
|
||||
rank := func(s string) int {
|
||||
switch strings.ToLower(strings.TrimSpace(s)) {
|
||||
case AlertTriggerSeverityWarning:
|
||||
return 1
|
||||
case AlertTriggerSeverityCritical:
|
||||
return 2
|
||||
default:
|
||||
return 2
|
||||
}
|
||||
}
|
||||
min := strings.ToLower(strings.TrimSpace(minimum))
|
||||
if min == "" {
|
||||
min = AlertTriggerSeverityCritical
|
||||
}
|
||||
return rank(level) >= rank(min)
|
||||
}
|
||||
|
||||
func alertTypeAllowed(alertType string, allowed []string) bool {
|
||||
at := strings.ToLower(strings.TrimSpace(alertType))
|
||||
for _, a := range allowed {
|
||||
if strings.ToLower(strings.TrimSpace(a)) == at {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// NormalizeOllamaKeepAlive validates the value Pulse sends as Ollama's
|
||||
// keep_alive request option. Empty is intentional and means "omit keep_alive"
|
||||
// so the Ollama server default applies.
|
||||
|
||||
@@ -949,6 +949,115 @@ func TestAIConfig_PatrolEventTriggerSettings(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestAIConfig_GetPatrolAlertTriggerMinSeverity(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
cfg *AIConfig
|
||||
want string
|
||||
}{
|
||||
{name: "nil receiver defaults to critical", cfg: nil, want: AlertTriggerSeverityCritical},
|
||||
{name: "empty defaults to critical", cfg: &AIConfig{}, want: AlertTriggerSeverityCritical},
|
||||
{name: "warning preserved", cfg: &AIConfig{PatrolAlertTriggerMinSeverity: AlertTriggerSeverityWarning}, want: AlertTriggerSeverityWarning},
|
||||
{name: "critical preserved", cfg: &AIConfig{PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical}, want: AlertTriggerSeverityCritical},
|
||||
{name: "mixed case normalized", cfg: &AIConfig{PatrolAlertTriggerMinSeverity: " Warning "}, want: AlertTriggerSeverityWarning},
|
||||
{name: "unknown defaults to critical", cfg: &AIConfig{PatrolAlertTriggerMinSeverity: "bogus"}, want: AlertTriggerSeverityCritical},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := tt.cfg.GetPatrolAlertTriggerMinSeverity(); got != tt.want {
|
||||
t.Fatalf("GetPatrolAlertTriggerMinSeverity() = %q, want %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIConfig_AlertTriggersInvestigation(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
cfg *AIConfig
|
||||
alertType string
|
||||
level string
|
||||
wantTrigger bool
|
||||
}{
|
||||
{
|
||||
name: "nil receiver never triggers",
|
||||
cfg: nil,
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: false,
|
||||
},
|
||||
{
|
||||
name: "master toggle off never triggers",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: false, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityWarning},
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: false,
|
||||
},
|
||||
{
|
||||
name: "critical floor rejects warning",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical},
|
||||
level: AlertTriggerSeverityWarning,
|
||||
wantTrigger: false,
|
||||
},
|
||||
{
|
||||
name: "critical floor accepts critical",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical},
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: true,
|
||||
},
|
||||
{
|
||||
name: "empty floor defaults to critical-only",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true},
|
||||
level: AlertTriggerSeverityWarning,
|
||||
wantTrigger: false,
|
||||
},
|
||||
{
|
||||
name: "warning floor accepts warning",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityWarning},
|
||||
level: AlertTriggerSeverityWarning,
|
||||
wantTrigger: true,
|
||||
},
|
||||
{
|
||||
name: "warning floor accepts critical",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityWarning},
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: true,
|
||||
},
|
||||
{
|
||||
name: "unknown level treated as critical",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical},
|
||||
level: "",
|
||||
wantTrigger: true,
|
||||
},
|
||||
{
|
||||
name: "allowlist admits matching type",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical, PatrolAlertTriggerTypes: []string{"cpu"}},
|
||||
alertType: "CPU",
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: true,
|
||||
},
|
||||
{
|
||||
name: "allowlist rejects unlisted type",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical, PatrolAlertTriggerTypes: []string{"cpu"}},
|
||||
alertType: "memory",
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: false,
|
||||
},
|
||||
{
|
||||
name: "empty allowlist admits any type",
|
||||
cfg: &AIConfig{PatrolAlertTriggersEnabled: true, PatrolAlertTriggerMinSeverity: AlertTriggerSeverityCritical, PatrolAlertTriggerTypes: []string{}},
|
||||
alertType: "disk",
|
||||
level: AlertTriggerSeverityCritical,
|
||||
wantTrigger: true,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := tt.cfg.AlertTriggersInvestigation(tt.alertType, tt.level); got != tt.wantTrigger {
|
||||
t.Fatalf("AlertTriggersInvestigation(%q, %q) = %v, want %v", tt.alertType, tt.level, got, tt.wantTrigger)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAIConfig_GetRequestTimeout(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
|
||||
Reference in New Issue
Block a user