diff --git a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md index b51b2d57dc..8fa47c7091 100644 --- a/docs/release-control/v6/internal/subsystems/agent-lifecycle.md +++ b/docs/release-control/v6/internal/subsystems/agent-lifecycle.md @@ -1262,6 +1262,10 @@ update, profile rollout, command reachability, or fleet-control authority. lifecycle presentation evidence. It keeps mock active-alert and history payloads aligned with the canonical alert contract, but it cannot enroll, identify, link, command, remove, or otherwise grant authority to an agent. + The optional `models.Alert.Resolution` field is the same kind of + evidence: it says a resolved node alert now belongs to a linked agent's + own alert and names that agent as display text, but the link itself stays + the agent lifecycle's, and the field cannot create, change or authorize it. 27. `internal/monitoring/monitor.go` shared with `monitoring`: monitor construction owns both monitoring runtime initialization and fail-closed agent lifecycle journal hydration before report admission. 28. `internal/monitoring/monitor_agents.go` shared with `monitoring`: server-side Unified Agent report, removal, token binding, tombstone expiry, and re-enrollment semantics are jointly owned by agent lifecycle authority and monitoring ingest. 29. `pkg/agents/host/report.go` shared with `monitoring`: the Unified Agent host report is both an agent lifecycle authored-state contract and a monitoring ingest contract for host maintenance posture. diff --git a/docs/release-control/v6/internal/subsystems/ai-runtime.md b/docs/release-control/v6/internal/subsystems/ai-runtime.md index 1f82e68833..56b2a47c66 100644 --- a/docs/release-control/v6/internal/subsystems/ai-runtime.md +++ b/docs/release-control/v6/internal/subsystems/ai-runtime.md @@ -4638,7 +4638,22 @@ summaries became the alarm name still carry " has VMware alarm ()", so the title strips that wording to the same alarm name. Patrol's seed context lists active and recently resolved alerts by resource: each line names the alert's resource unless the message already does, because -provider incidents such as vCenter alarms say only what was flagged. The projection also collapses open/resolved churn: +provider incidents such as vCenter alarms say only what was flagged. +A recently resolved alert that closed without recovering, such as a node metric +handed to its Pulse agent, must not read as a recovery to Patrol, the +assistant or the alert tool: Patrol's seed writes `closed . ` in place of `resolved `, the +assistant's alert context writes `closed ago) `, the +`pulse_alerts` `resolved` action returns the summary as `resolution` with +`successor_resource_id`, and incident memory titles the close with the summary +(`alertResolvedEventSummary`, or the `alert_resolution` resource-change +metadata when it projects from the canonical timeline). +`TestSeedHealthAndAlerts_HandoverIsNotARecovery`, +`TestService_buildAlertContext_HandoverIsNotARecovery`, +`TestExecuteListResolvedAlertsReportsHandover` and +`TestIncidentStore_HandoverCloseIsNotARecovery` pin it. Performance reports +(`pkg/reporting`, the report narrator) do not carry the resolution yet: a +report generated within the five-minute recently-resolved window after a +handover still lists that alert as resolved. The projection also collapses open/resolved churn: when a record's timeline holds at least four open-to-resolved or resolved-to-open transitions inside the last 24 hours, the item carries a `flapping` summary (transition count, window, first and latest transition) and diff --git a/docs/release-control/v6/internal/subsystems/alerts.md b/docs/release-control/v6/internal/subsystems/alerts.md index b4ebad46e8..706608a8d8 100644 --- a/docs/release-control/v6/internal/subsystems/alerts.md +++ b/docs/release-control/v6/internal/subsystems/alerts.md @@ -793,12 +793,26 @@ its message) and the Alert JSON then carry it. Resolved consumers still receive the close, because integrations close their incident by alert ID. The synthesized closing evidence carries the resolution reason and summary instead of `legacy_alert_recovery_projection`, and the resolve transition carries the -summary as its `reason`. The cause stays `recovery_evidence`, the only cause -operational trust lets enter `resolved`. A nil resolution leaves the -disabled-threshold path and every recovery unchanged. +summary as its `reason`. Its cause is `ownership_transferred`, an +`operationaltrust.TransitionCause` that may only enter `resolved` and only with +evidence (`resolvedTransitionCause` in `operational_contract.go`), so Patrol's +timeline never labels a handover "Recovery evidence" and alert-quality +telemetry does not count a handover during a snooze as resolved while snoozed. +A nil resolution leaves the disabled-threshold path and every recovery on +`recovery_evidence`. `TestReleasedMetricAlertCarriesResolutionInsteadOfRecovery` and `TestResolvedHistoryRowKeepsResolutionWithoutEventLog` in `internal/alerts/operational_contract_test.go` pin this. +The in-app readers carry it too. `GetRecentlyResolved` projects it into +`models.Alert.Resolution` (reason, successor and the `Summary()` text) for the +state snapshot, the websocket and the assistant and Patrol prompts, pinned by +`TestRecentlyResolvedCarriesHandoverResolution` in `history_test.go`. The +alerts history lists such a row as `moved to agent` instead of `resolved`, even +when it was acknowledged, with the account in the badge title and on the phone +card (`buildAlertHistoryItems`, `getAlertResolutionDetail`). The resolved +lifecycle change written to the resource timeline carries the summary as its +reason and the code as `alert_resolution` metadata, so the row's incident +timeline says the alert moved instead of "Alert resolved: ". ## Extension Points @@ -2728,7 +2742,8 @@ legacy alert timestamp. Suppression is bounded and reasoned, leaves the default active queue, and remains inspectable. Expiry or explicit unsuppression returns the record to its detector-owned state; it never resolves it. Only fresh sufficient recovery -evidence may enter resolving, and only detector recovery may resolve. +evidence may enter resolving, and only detector recovery, or a handover +recorded as `ownership_transferred` with its closing evidence, may resolve. Per-alert snooze is the customer-facing bounded suppression contract, not a page-local timer. `SnoozeAlert` writes the same canonical operational record, diff --git a/docs/release-control/v6/internal/subsystems/api-contracts.md b/docs/release-control/v6/internal/subsystems/api-contracts.md index e410966406..8454240e42 100644 --- a/docs/release-control/v6/internal/subsystems/api-contracts.md +++ b/docs/release-control/v6/internal/subsystems/api-contracts.md @@ -661,6 +661,13 @@ transitions. Older payloads without these optional fields remain readable; frontend consumers use `frontend-modern/src/types/operationalTrust.ts` as the single TypeScript projection rather than recreating lifecycle or evidence enums locally. +`TransitionCause` includes `ownership_transferred` for a close whose condition +another record now owns, such as a node metric handed to its Pulse agent; it +may only enter `resolved`. Alert payloads (history, state snapshot and +websocket) may also carry an optional `resolution` (`reason`, +`successorResourceId`, `successorName`, and on state payloads the alert +engine's `summary`), typed once as `AlertResolution` in +`frontend-modern/src/types/api.ts`. Its absence means an ordinary recovery. ## Shared Boundaries diff --git a/docs/release-control/v6/internal/subsystems/frontend-primitives.md b/docs/release-control/v6/internal/subsystems/frontend-primitives.md index aa974bf928..eb3b352729 100644 --- a/docs/release-control/v6/internal/subsystems/frontend-primitives.md +++ b/docs/release-control/v6/internal/subsystems/frontend-primitives.md @@ -7088,6 +7088,13 @@ should extend the model, and section rendering should extend those owners rather than rebuilding any of those concerns in the tab shell. The resource-resolution handoff into the resource-incident panel now belongs to the history state rather than the tab shell. +The history model also owns the close outcome: an alert that closed with a +typed `resolution` is status `moved` (badge `moved to agent`, its account in +`HistoryItem.closeDetail`) rather than `resolved` or `acknowledged`. The table +row carries the account in the badge title and the phone card renders it under +the message, while `getAlertHistoryStatusPresentation` and +`getAlertResolutionDetail` in `utils/alertIncidentPresentation.ts` own the +wording. `frontend-modern/src/features/alerts/useAlertHistoryState.ts` re-exposes the `getResource` resolver it is already given, and `frontend-modern/src/features/alerts/AlertResourceIncidentsPanel.tsx` reads it diff --git a/docs/release-control/v6/internal/subsystems/monitoring.md b/docs/release-control/v6/internal/subsystems/monitoring.md index 9ae9a5d83a..b89afe555b 100644 --- a/docs/release-control/v6/internal/subsystems/monitoring.md +++ b/docs/release-control/v6/internal/subsystems/monitoring.md @@ -3587,6 +3587,15 @@ truth. Monitoring APIs that still serve `StateSnapshot` must project instead of trusting the cached snapshot fields, so externally served alert counts and recently resolved incidents do not lag behind acknowledgement, resolve, or clear operations between explicit sync points. +A recently resolved alert that closed without recovering keeps that on the +projection: `models.Alert.Resolution` (reason, successor resource ID and name, +and the alert engine's summary) is set only for such a close, `cloneAlert` +copies it into every snapshot, and `recordAlertTimelineChange` passes it into +the `alert_resolved` resource change so the incident timeline says the alert +moved. `TestAlertResolutionWireContract` in +`internal/models/metrics_types_test.go` and +`TestMonitor_HandleAlertLifecycle_HandoverCloseIsNotARecovery` in +`internal/monitoring/monitor_alert_handling_test.go` pin it. The container entrypoint in `docker-entrypoint.sh` now also lives under this boundary. Hosted or managed tenant bootstrap changes must preserve safe startup when immutable read-only mounts are layered into `/etc/pulse`; the entrypoint diff --git a/docs/release-control/v6/internal/subsystems/notifications.md b/docs/release-control/v6/internal/subsystems/notifications.md index bb8bba2518..68bb4dcd5c 100644 --- a/docs/release-control/v6/internal/subsystems/notifications.md +++ b/docs/release-control/v6/internal/subsystems/notifications.md @@ -1237,6 +1237,12 @@ payload's `alerts` array. Built-in template titles still say Resolved, which is accurate for the closed alert, and a custom template that hard-codes recovery text keeps it. Issue #1682 is the precedent: a recipient called "is now healthy" a false factual claim when an alert closed without recovering. +The resolve transition such a close links to has cause `ownership_transferred` +in `internal/operationaltrust/contracts.go`, not `recovery_evidence`. A +notification link records the transition ID and lifecycle state, never the +cause, so delivery, receipts and recovery admission treat it exactly like a +recovery's resolve transition. `TestOwnershipTransferOnlyClosesWithEvidence` +pins that the cause may only enter `resolved` and only with evidence. Verification: `resolved_handover_test.go` drives the firing, its receipt and the queued close through the normal entry points for ntfy, and renders Discord, diff --git a/docs/release-control/v6/internal/subsystems/patrol-intelligence.md b/docs/release-control/v6/internal/subsystems/patrol-intelligence.md index 383727f107..251bfc8252 100644 --- a/docs/release-control/v6/internal/subsystems/patrol-intelligence.md +++ b/docs/release-control/v6/internal/subsystems/patrol-intelligence.md @@ -1380,6 +1380,14 @@ to traverse the list again. At widths where the detail replaces the queue's reading position, its visible close affordance is an explicit `Back to list` control; wide layouts retain the compact close icon. Both controls clear the deep link and restore focus to the originating queue row. +A resolved item whose latest close is an `ownership_transferred` transition did +not recover: its condition moved to another record, such as the linked Pulse +agent's own alert. The detail's state badge reads `Moved` in the info tone +instead of the green `Resolved`, and the transition's reason (where the +condition went, and that it is not a recovery) sits directly under the summary +rather than only inside Evidence and history (`getAttentionHandoverClose` in +`patrolHomePresentation.ts`). The timeline names that cause `Ownership +transferred`, never `Recovery evidence`. The active Patrol queue now uses compact severity-accented rows for prioritisation and a single focused review panel for the selected issue. The diff --git a/docs/release-control/v6/internal/subsystems/unified-resources.md b/docs/release-control/v6/internal/subsystems/unified-resources.md index b91304e4d5..d501a9b416 100644 --- a/docs/release-control/v6/internal/subsystems/unified-resources.md +++ b/docs/release-control/v6/internal/subsystems/unified-resources.md @@ -4171,6 +4171,14 @@ such as `alert_fired`, `alert_acknowledged`, `alert_unacknowledged`, AI-local annotations. Snooze and resume projections must preserve the actor and exact suppression expiry when present; they pause delivery and escalation without acknowledging, resolving, or replacing the underlying incident. +An `alert_resolved` change for a close that was not a recovery carries the +alert engine's summary as its reason and the reason code as `alert_resolution` +metadata (`AlertTimelineChange.ResolutionReason` and `ResolutionSummary`), so +incident projection titles the close with where the alert went instead of +"Alert resolved: ". The resource drawer still labels the change +by kind (`Alert resolved`) and prefixes its headline with that label; only the +reason after it says the alert moved. Ordinary recoveries omit the key, and +every other lifecycle kind ignores the fields. Alert-scoped incident memory may still project those events for one investigation thread, but the durable source of truth for resource-affecting alert lifecycle and diff --git a/frontend-modern/browser-verification/8b6df6398a29e2e3.json b/frontend-modern/browser-verification/8b6df6398a29e2e3.json new file mode 100644 index 0000000000..e79614e26f --- /dev/null +++ b/frontend-modern/browser-verification/8b6df6398a29e2e3.json @@ -0,0 +1,54 @@ +{ + "version": 1, + "base_sha": "982a6f6148cf9d39883e4443191b4d40b59cfd7c", + "verified_at": "2026-10-06T15:03:00Z", + "result": "passed", + "changed_paths": [ + "frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx", + "frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx", + "frontend-modern/src/features/alerts/alertHistoryModel.ts", + "frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx", + "frontend-modern/src/features/patrol/patrolHomePresentation.ts", + "frontend-modern/src/types/api.ts", + "frontend-modern/src/types/operationalTrust.ts", + "frontend-modern/src/utils/alertIncidentPresentation.ts" + ], + "content_sha256": { + "frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx": "94bb73163dd86c1f045a0b5559b42486198ab824ef2abccfc3c8c1dcc0ff2784", + "frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx": "aacc7232596a47d64c7e6ec3b29dfaf51be6e1691d9be906b30bfce49f4daf6f", + "frontend-modern/src/features/alerts/alertHistoryModel.ts": "30548565d29ca16fe1c1192687c881defc064ba1b267fdcd40fc28922a51fed4", + "frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx": "b3c87b3ead9bde22a091b2c080b9eaca18face9ba153085bb1a1dbb9b6ae9f54", + "frontend-modern/src/features/patrol/patrolHomePresentation.ts": "04338cdce5b54bc041267b325a6985a9a8f84ef9e5a8dba8040c7d9fc5ab1811", + "frontend-modern/src/types/api.ts": "17a948b6ba78780ef2b95c1f397d2d9a04eebd2726c568a8316e25e7c8f9e6b3", + "frontend-modern/src/types/operationalTrust.ts": "9a50a869b1f6ef2885f30a354fb70b36b64fbb1a3f00ba4476d592389f9ba954", + "frontend-modern/src/utils/alertIncidentPresentation.ts": "f77c42b48f22bddcccb43a3da43b3771bb03d25c6d23b7efa200a34dae1da0e9" + }, + "routes": [ + "/alerts/history", + "/patrol?attention=pve1::metric-threshold:memory" + ], + "viewports": [ + { + "width": 1280, + "height": 900 + }, + { + "width": 375, + "height": 812 + } + ], + "states": [ + "Private non-mock stack (backend :7691, Vite :5291) on a data dir whose alert history was written by the real alerts manager: a pve1 node memory alert closed through releaseCanonicalMetricAlert with a moved_to_agent resolution (successor pve1 (Host Agent)) and a pve1 CPU alert closed as an ordinary release; the handover record's resolve transition carries cause ownership_transferred", + "Alerts history at 1280: the CPU row's status badge reads resolved, the memory row's reads moved to agent in blue, and its title is 'Moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.'", + "Memory row Timeline expanded: the incident timeline lists 'Node memory at 95.0%' then 'Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.' where it said 'Alert resolved' before; the incident header still says resolved (incident status vocabulary, unchanged)", + "Alerts history at 375: the memory card shows the moved to agent badge and the account as a muted line under the breach message; the CPU card is unchanged", + "Patrol deep link to the moved record at 1280 and 375: the detail badge reads Moved (info tone) instead of green Resolved, the handover sentence sits directly under 'Node memory at 95.0%', and Evidence and history shows 'Open to Resolved, Ownership transferred' with the same reason; no Recovery evidence label anywhere; no horizontal scroll at 375 and no console errors" + ], + "interactions": [ + "Loaded /alerts/history at 1280x900 and read both rows' status badges and titles", + "Clicked the memory row's Timeline button and read the expanded incident timeline", + "Reloaded /alerts/history at 375x812 and read both history cards", + "Opened /patrol?attention=pve1::metric-threshold:memory at 1280x900 and 375x812, opened Evidence and history, and read the badge, summary, handover line and timeline", + "Checked document width against the viewport at 375 and read the console for errors" + ] +} diff --git a/frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx b/frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx index 0b4df0f6a7..0cffed81e0 100644 --- a/frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx +++ b/frontend-modern/src/features/alerts/AlertHistoryMobileList.tsx @@ -229,6 +229,9 @@ export function AlertHistoryMobileList(props: AlertHistoryMobileListProps) {

{alert.description}

+ + {(detail) =>

{detail()}

} +
diff --git a/frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx b/frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx index 9187526f50..13d1b3b797 100644 --- a/frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx +++ b/frontend-modern/src/features/alerts/AlertHistoryTableAlertRow.tsx @@ -74,7 +74,7 @@ export function AlertHistoryTableAlertRow(props: AlertHistoryTableAlertRowProps) - + {historyStatusPresentation().label} diff --git a/frontend-modern/src/features/alerts/alertHistoryModel.ts b/frontend-modern/src/features/alerts/alertHistoryModel.ts index bd2d89af25..097a560da5 100644 --- a/frontend-modern/src/features/alerts/alertHistoryModel.ts +++ b/frontend-modern/src/features/alerts/alertHistoryModel.ts @@ -1,5 +1,9 @@ import type { Alert } from '@/types/api'; import type { Resource } from '@/types/resource'; +import { + ALERT_HISTORY_MOVED_STATUS, + getAlertResolutionDetail, +} from '@/utils/alertIncidentPresentation'; import { isPulseSystemAlert } from '@/utils/alertScope'; import { alertTypeDisplayLabel, unifiedTypeToAlertDisplayType } from './helpers'; @@ -25,6 +29,8 @@ export interface HistoryItem { title: string; rawAlertType?: string; description?: string; + // Set when the alert closed without recovering: what happened instead. + closeDetail?: string; acknowledged?: boolean; systemAlert?: boolean; } @@ -206,11 +212,17 @@ export function buildAlertHistoryItems({ alertHistory.forEach((alert) => { if (activeAlertIds.has(alert.id)) return; const systemScoped = isPulseSystemAlert(alert); + // A close that was not a recovery must not read as "resolved". + const closeDetail = getAlertResolutionDetail(alert.resolution); items.push({ id: alert.id, source: 'alert', - status: alert.acknowledged ? 'acknowledged' : 'resolved', + status: closeDetail + ? ALERT_HISTORY_MOVED_STATUS + : alert.acknowledged + ? 'acknowledged' + : 'resolved', startTime: alert.startTime, endTime: alert.lastSeen, duration: formatAlertHistoryDuration(alert.startTime, alert.lastSeen, now), @@ -231,6 +243,7 @@ export function buildAlertHistoryItems({ title: alertTypeDisplayLabel(alert.type), rawAlertType: alert.type, description: alert.message, + ...(closeDetail ? { closeDetail } : {}), acknowledged: alert.acknowledged, }); }); diff --git a/frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx b/frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx index 76bbcbe03d..72ac52cd43 100644 --- a/frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx +++ b/frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx @@ -68,6 +68,7 @@ import { copyToClipboard } from '@/utils/clipboard'; import type { PatrolAutonomyLevel } from '@/api/patrol'; import { getAttentionFlappingPresentation, + getAttentionHandoverClose, getLinkedPatrolFindings, getPatrolRememberedDecisionPresentation, partitionPatrolAttention, @@ -971,7 +972,10 @@ function AttentionDetail(props: {
- + @@ -984,6 +988,9 @@ function AttentionDetail(props: {

{loaded().item.plainLanguageSummary}

+ + {(handover) =>

{handover()}

} + {(flapping) => (

@@ -1793,10 +1800,17 @@ function severityTone(item: AttentionItem): MetadataBadgeTone { } } -function StateBadge(props: { item: AttentionItem }) { +// movedClose marks a resolved item whose condition moved to another owner: it +// did not recover, so it must not wear the green "Resolved" badge. +function StateBadge(props: { item: AttentionItem; movedClose?: boolean }) { return ( - - {formatLabel(props.item.state)} + + {props.movedClose ? 'Moved' : formatLabel(props.item.state)} ); } diff --git a/frontend-modern/src/features/patrol/__tests__/PatrolAttentionWorkbench.test.tsx b/frontend-modern/src/features/patrol/__tests__/PatrolAttentionWorkbench.test.tsx index 3c5ff2b423..e3a01ebc10 100644 --- a/frontend-modern/src/features/patrol/__tests__/PatrolAttentionWorkbench.test.tsx +++ b/frontend-modern/src/features/patrol/__tests__/PatrolAttentionWorkbench.test.tsx @@ -988,6 +988,56 @@ describe('PatrolAttentionWorkbench', () => { expect(screen.getByText('Show all 11 transitions')).toBeInTheDocument(); }); + it('shows an alert handed to its Pulse agent as moved, not recovered', async () => { + const reason = + 'Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.'; + const moved = item({ + id: 'record-moved', + operationalRecordId: 'record-moved', + subjectResourceId: 'pve1', + subjectResourceName: 'pve1', + subjectResourceType: 'node', + kind: 'memory', + title: 'Memory on pve1', + plainLanguageSummary: 'Memory usage at 95%', + severity: 'warning', + state: 'resolved', + protectionPosture: undefined, + }); + const movedDetail = detail(moved); + movedDetail.timeline = [ + ...movedDetail.timeline, + { + id: 'transition-moved', + operationalRecordId: 'record-moved', + from: 'open', + to: 'resolved', + at: '2026-07-19T07:45:00Z', + cause: 'ownership_transferred', + causeKey: 'memory:pve1', + evidenceIds: ['evidence-handover'], + reason, + }, + ]; + window.history.replaceState({}, '', '/patrol?attention=record-moved'); + apiMocks.getList.mockResolvedValue(listResponse([], summary())); + apiMocks.getDetail.mockResolvedValue(movedDetail); + renderWorkbench(); + + const panel = await screen.findByRole('complementary', { name: 'pve1 · Memory' }); + expect(await within(panel).findByText('Moved')).toBeInTheDocument(); + expect(within(panel).queryByText('Resolved')).not.toBeInTheDocument(); + // The account sits under the summary, outside the collapsed history. + const summaryLine = within(panel).getByText('Memory usage at 95%'); + expect(summaryLine.nextElementSibling).toHaveTextContent(reason); + expect(summaryLine.nextElementSibling?.closest('details')).toBeNull(); + + fireEvent.click(within(panel).getByText('Evidence and history')); + expect(within(panel).getByText('Open to Resolved')).toBeInTheDocument(); + expect(within(panel).getByText(/Ownership transferred/)).toBeInTheDocument(); + expect(within(panel).queryByText(/Recovery evidence/)).not.toBeInTheDocument(); + }); + it('opens the canonical governed action review from an eligible attention item', async () => { const actionOffer = { targetResourceId: 'docker:host-1/container-1', diff --git a/frontend-modern/src/features/patrol/__tests__/patrolHomePresentation.test.ts b/frontend-modern/src/features/patrol/__tests__/patrolHomePresentation.test.ts index 95d4ccdd28..9b07da74a7 100644 --- a/frontend-modern/src/features/patrol/__tests__/patrolHomePresentation.test.ts +++ b/frontend-modern/src/features/patrol/__tests__/patrolHomePresentation.test.ts @@ -1,7 +1,8 @@ import { describe, expect, it } from 'vitest'; import type { PatrolObjective } from '@/api/patrol'; -import type { AttentionItem } from '@/api/patrolAttention'; +import type { AttentionItem, AttentionItemDetail } from '@/api/patrolAttention'; import { + getAttentionHandoverClose, getPatrolAttentionDecisionReason, getPatrolObjectiveProtectionSummary, partitionPatrolAttention, @@ -145,3 +146,62 @@ describe('patrol home presentation', () => { }); }); }); + +describe('getAttentionHandoverClose', () => { + const transition = ( + overrides: Partial, + ): AttentionItemDetail['timeline'][number] => ({ + id: 'transition-1', + operationalRecordId: 'record-1', + from: 'open', + to: 'resolved', + at: '2026-08-14T08:00:00Z', + cause: 'recovery_evidence', + causeKey: 'memory:pve1', + evidenceIds: ['evidence-1'], + ...overrides, + }); + const reason = + 'Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.'; + + it('returns the handover account only for a record closed by an ownership transfer', () => { + const resolved = attentionItem({ state: 'resolved' }); + expect( + getAttentionHandoverClose({ + item: resolved, + timeline: [transition({ cause: 'ownership_transferred', reason })], + }), + ).toBe(reason); + expect( + getAttentionHandoverClose({ + item: resolved, + timeline: [transition({ cause: 'ownership_transferred' })], + }), + ).toBe('This alert moved to another owner when it closed. It did not recover.'); + expect(getAttentionHandoverClose({ item: resolved, timeline: [transition({})] })).toBeNull(); + }); + + it('follows the latest close, so a later recovery or reopen wins', () => { + const handover = transition({ cause: 'ownership_transferred', reason }); + const reopened = transition({ + id: 'transition-2', + from: 'resolved', + to: 'open', + at: '2026-08-14T09:00:00Z', + cause: 'detector_decision', + }); + const recovered = transition({ id: 'transition-3', at: '2026-08-14T10:00:00Z' }); + expect( + getAttentionHandoverClose({ + item: attentionItem({ state: 'resolved' }), + timeline: [handover, reopened, recovered], + }), + ).toBeNull(); + expect( + getAttentionHandoverClose({ + item: attentionItem({ state: 'open' }), + timeline: [handover, reopened], + }), + ).toBeNull(); + }); +}); diff --git a/frontend-modern/src/features/patrol/patrolHomePresentation.ts b/frontend-modern/src/features/patrol/patrolHomePresentation.ts index 78d15e94ce..29ae650713 100644 --- a/frontend-modern/src/features/patrol/patrolHomePresentation.ts +++ b/frontend-modern/src/features/patrol/patrolHomePresentation.ts @@ -1,5 +1,5 @@ import type { PatrolAutonomyLevel, PatrolObjective } from '@/api/patrol'; -import type { AttentionFlapping, AttentionItem } from '@/api/patrolAttention'; +import type { AttentionFlapping, AttentionItem, AttentionItemDetail } from '@/api/patrolAttention'; import type { UnifiedFinding } from '@/stores/aiIntelligence'; export interface PatrolAutonomyExperience { @@ -207,6 +207,26 @@ export function getAttentionFlappingPresentation( return getFlappingPresentation(flapping.transitionCount, flapping.windowHours); } +/** + * A resolved item whose close handed the condition to another record, such as + * a node metric its Pulse agent took over, did not recover. Returns the + * account to show instead of a healthy "Resolved", or null for a recovery. + */ +export function getAttentionHandoverClose( + detail: Pick | null | undefined, +): string | null { + if (!detail || detail.item.state !== 'resolved') return null; + let close: AttentionItemDetail['timeline'][number] | undefined; + for (const transition of detail.timeline ?? []) { + if (transition.to !== 'resolved') continue; + if (!close || Date.parse(transition.at) >= Date.parse(close.at)) close = transition; + } + if (close?.cause !== 'ownership_transferred') return null; + return ( + close.reason?.trim() || 'This alert moved to another owner when it closed. It did not recover.' + ); +} + const normalizeResourceKey = (value: string | undefined): string => (value ?? '').trim().toLowerCase(); diff --git a/frontend-modern/src/pages/__tests__/Alerts.helpers.test.ts b/frontend-modern/src/pages/__tests__/Alerts.helpers.test.ts index 75e996aa76..31016b5670 100644 --- a/frontend-modern/src/pages/__tests__/Alerts.helpers.test.ts +++ b/frontend-modern/src/pages/__tests__/Alerts.helpers.test.ts @@ -100,6 +100,7 @@ import { buildProjectedOverrides, normalizeRawOverridesConfig, } from '@/features/alerts/alertOverridesModel'; +import { buildAlertHistoryItems } from '@/features/alerts/alertHistoryModel'; import { getAlertIncidentAcknowledgedBadgeClass, getAlertIncidentEventFilterActionButtonClass, @@ -1320,3 +1321,59 @@ describe('Unified selector parity', () => { ); }); }); + +describe('alert history close outcome', () => { + it('lists a node alert handed to its Pulse agent as moved, not resolved', () => { + const base = { + type: 'memory', + level: 'warning' as const, + resourceName: 'pve1', + node: 'pve1', + instance: 'pve1', + value: 95, + threshold: 85, + startTime: '2026-10-06T09:00:00Z', + lastSeen: '2026-10-06T11:00:00Z', + acknowledged: true, + }; + const items = buildAlertHistoryItems({ + activeAlerts: {}, + alertHistory: [ + { + ...base, + id: 'pve1-memory', + resourceId: 'pve1', + message: 'Memory usage at 95%', + resolution: { + reason: 'moved_to_agent', + successorResourceId: 'agent-pve1', + successorName: 'pve1 (Host Agent)', + }, + }, + { + ...base, + id: 'pve1-cpu', + resourceId: 'pve1', + message: 'CPU usage at 90%', + acknowledged: false, + }, + ], + getResource: () => undefined, + allResources: [], + now: Date.parse('2026-10-06T12:00:00Z'), + }); + + const moved = items.find((item) => item.id === 'pve1-memory'); + // Acknowledgement does not hide the outcome: the condition went to the agent. + expect(moved?.status).toBe('moved'); + expect(moved?.description).toBe('Memory usage at 95%'); + expect(moved?.closeDetail).toBe( + 'Moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.', + ); + const recovered = items.find((item) => item.id === 'pve1-cpu'); + expect(recovered?.status).toBe('resolved'); + expect(recovered).not.toHaveProperty('closeDetail'); + + expect(alertHistoryTableAlertRowSource).toContain('title={props.alert.closeDetail}'); + }); +}); diff --git a/frontend-modern/src/types/api.ts b/frontend-modern/src/types/api.ts index 5e7da6d6de..f1a16f2483 100644 --- a/frontend-modern/src/types/api.ts +++ b/frontend-modern/src/types/api.ts @@ -1380,6 +1380,18 @@ export interface Alert { latestTransition?: LifecycleTransition; transitions?: LifecycleTransition[]; evidence?: EvidenceEnvelope[]; + // Set only on a closed alert whose condition did not clear, such as a node + // metric handed to its Pulse agent. Absent for an ordinary recovery. + resolution?: AlertResolution; +} + +// Why an alert closed without recovering (alerts.AlertResolution). +export interface AlertResolution { + reason: 'moved_to_agent' | (string & {}); + successorResourceId?: string; + successorName?: string; + // The alert engine's one-line account; carried on state snapshots only. + summary?: string; } export interface ResolvedAlert extends Alert { diff --git a/frontend-modern/src/types/operationalTrust.ts b/frontend-modern/src/types/operationalTrust.ts index feeb98a9d2..67ebc0bb7c 100644 --- a/frontend-modern/src/types/operationalTrust.ts +++ b/frontend-modern/src/types/operationalTrust.ts @@ -97,6 +97,7 @@ export type TransitionCause = | 'suppression' | 'suppression_expired' | 'recovery_evidence' + | 'ownership_transferred' | 'collection_stale' | 'collection_unknown'; diff --git a/frontend-modern/src/utils/__tests__/alertIncidentPresentation.test.ts b/frontend-modern/src/utils/__tests__/alertIncidentPresentation.test.ts index 2e4ad59696..deebbc497c 100644 --- a/frontend-modern/src/utils/__tests__/alertIncidentPresentation.test.ts +++ b/frontend-modern/src/utils/__tests__/alertIncidentPresentation.test.ts @@ -10,8 +10,10 @@ import { ALERT_RESOURCE_INCIDENT_REFRESH_LABEL, ALERT_RESOURCE_INCIDENT_SAVE_NOTE_LABEL, ALERT_RESOURCE_INCIDENT_SAVING_NOTE_LABEL, + ALERT_HISTORY_MOVED_STATUS, getAlertHistoryStatusPresentation, getAlertIncidentLevelBadgeClass, + getAlertResolutionDetail, getAlertIncidentTimelineDialogTitle, getAlertHistoryInvestigationDialogTitle, getAlertResourceIncidentAcknowledgedByLabel, @@ -103,6 +105,29 @@ describe('alertIncidentPresentation', () => { className: 'text-xs px-2 py-0.5 rounded-sm bg-surface-hover text-base-content', rowClassName: '', }); + + expect(getAlertHistoryStatusPresentation(ALERT_HISTORY_MOVED_STATUS)).toEqual({ + label: 'moved to agent', + className: + 'text-xs px-2 py-0.5 rounded-sm bg-blue-100 dark:bg-blue-900/25 text-blue-700 dark:text-blue-300', + rowClassName: '', + }); + }); + + it('says a handed-over alert did not recover, and leaves recoveries alone', () => { + expect(getAlertResolutionDetail(undefined)).toBeUndefined(); + expect( + getAlertResolutionDetail({ reason: 'moved_to_agent', successorName: 'pve1 (Host Agent)' }), + ).toBe( + 'Moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading.', + ); + expect(getAlertResolutionDetail({ reason: 'moved_to_agent' })).toBe( + 'Moved to its Pulse agent. This is not a recovery: check the agent for the current reading.', + ); + // A future reason the UI has no copy for still shows the engine's account. + expect( + getAlertResolutionDetail({ reason: 'replaced', summary: 'Alert replaced by a new check.' }), + ).toBe('Alert replaced by a new check.'); }); it('returns canonical resource incident loading and empty-state copy', () => { diff --git a/frontend-modern/src/utils/alertIncidentPresentation.ts b/frontend-modern/src/utils/alertIncidentPresentation.ts index 5649918d23..af6b6d0721 100644 --- a/frontend-modern/src/utils/alertIncidentPresentation.ts +++ b/frontend-modern/src/utils/alertIncidentPresentation.ts @@ -1,3 +1,4 @@ +import type { AlertResolution } from '@/types/api'; import { getAlertSeverityBadgeClass } from '@/utils/alertSeverityPresentation'; export type AlertIncidentLevel = 'warning' | 'critical' | string | null | undefined; @@ -76,6 +77,24 @@ export function getAlertIncidentLevelBadgeClass(level: AlertIncidentLevel): stri return getAlertSeverityBadgeClass(String(level ?? 'warning')); } +// History status for an alert that closed because its Pulse agent took the +// metric over. The condition did not clear; the agent's own alert owns it. +export const ALERT_HISTORY_MOVED_STATUS = 'moved'; + +/** + * The account of a close that was not a recovery, for history rows and their + * tooltips. Returns undefined for an ordinary recovery so callers keep their + * "resolved" wording. + */ +export function getAlertResolutionDetail(resolution?: AlertResolution | null): string | undefined { + if (!resolution) return undefined; + if (resolution.reason === 'moved_to_agent') { + const successor = resolution.successorName?.trim() || 'its Pulse agent'; + return `Moved to ${successor}. This is not a recovery: check the agent for the current reading.`; + } + return resolution.summary?.trim() || undefined; +} + export function getAlertHistoryStatusPresentation( status?: string | null, ): AlertHistoryStatusPresentation { @@ -99,6 +118,17 @@ export function getAlertHistoryStatusPresentation( }; } + // Closed without recovering: neither a live problem on this row nor a + // healthy outcome, so it gets its own neutral-blue badge. + if (normalized === ALERT_HISTORY_MOVED_STATUS) { + return { + label: 'moved to agent', + className: + 'text-xs px-2 py-0.5 rounded-sm bg-blue-100 dark:bg-blue-900/25 text-blue-700 dark:text-blue-300', + rowClassName: '', + }; + } + return { label: normalized || 'resolved', className: 'text-xs px-2 py-0.5 rounded-sm bg-surface-hover text-base-content', diff --git a/internal/ai/alert_adapter.go b/internal/ai/alert_adapter.go index a8a5cd4d35..f34652fd83 100644 --- a/internal/ai/alert_adapter.go +++ b/internal/ai/alert_adapter.go @@ -2,6 +2,7 @@ package ai import ( "fmt" + "strings" "time" "github.com/rcourtman/pulse-go-rewrite/internal/alerts" @@ -56,6 +57,7 @@ func (a *AlertManagerAdapter) GetRecentlyResolved(minutes int) []ResolvedAlertIn AlertInfo: convertAlertFromModels(&resolved.Alert), ResolvedTime: resolved.ResolvedTime, Duration: formatDuration(resolved.ResolvedTime.Sub(resolved.StartTime)), + Resolution: resolvedAlertResolution(resolved.Resolution), } result = append(result, info) } @@ -98,6 +100,7 @@ func (a *AlertManagerAdapter) GetAlertHistory(resourceID string, limit int) []Re AlertInfo: convertAlertFromModels(&resolved.Alert), ResolvedTime: resolved.ResolvedTime, Duration: formatDuration(resolved.ResolvedTime.Sub(resolved.StartTime)), + Resolution: resolvedAlertResolution(resolved.Resolution), } result = append(result, info) if len(result) >= limit { @@ -245,3 +248,12 @@ func formatDuration(d time.Duration) string { } return fmt.Sprintf("%d days", days) } + +// resolvedAlertResolution is the one-line account of a close that was not a +// recovery, empty for an ordinary resolve. +func resolvedAlertResolution(resolution *models.AlertResolution) string { + if resolution == nil { + return "" + } + return strings.TrimSpace(resolution.Summary) +} diff --git a/internal/ai/alert_provider.go b/internal/ai/alert_provider.go index 6c36e68ece..3b7a3b8852 100644 --- a/internal/ai/alert_provider.go +++ b/internal/ai/alert_provider.go @@ -31,6 +31,9 @@ type ResolvedAlertInfo struct { AlertInfo ResolvedTime time.Time `json:"resolved_time"` Duration string `json:"total_duration"` // How long the alert lasted + // Resolution is set only when the alert closed without recovering, such + // as a node metric handed to its Pulse agent. + Resolution string `json:"resolution,omitempty"` } // AlertProvider provides access to the current alert state @@ -131,6 +134,12 @@ func (s *Service) buildAlertContext() string { } for i := 0; i < limit; i++ { a := recentlyResolved[i] + if resolution := strings.TrimSpace(a.Resolution); resolution != "" { + sections = append(sections, fmt.Sprintf("- **%s** on %s: %s (lasted %s, closed %s ago) %s", + a.Type, a.ResourceName, a.Message, a.Duration, + formatTimeAgo(a.ResolvedTime), resolution)) + continue + } sections = append(sections, fmt.Sprintf("- **%s** on %s: %s (lasted %s, resolved %s ago)", a.Type, a.ResourceName, a.Message, a.Duration, formatTimeAgo(a.ResolvedTime))) diff --git a/internal/ai/alert_provider_test.go b/internal/ai/alert_provider_test.go index c223df3060..26680472d2 100644 --- a/internal/ai/alert_provider_test.go +++ b/internal/ai/alert_provider_test.go @@ -5,6 +5,8 @@ import ( "strings" "testing" "time" + + "github.com/rcourtman/pulse-go-rewrite/internal/models" ) type stubAlertProvider struct { @@ -151,6 +153,61 @@ func TestService_buildAlertContext_NoActiveWithResolved(t *testing.T) { } } +// The assistant reads recently resolved alerts through the adapter, so a node +// alert handed to its Pulse agent must reach the prompt as a handover and not +// as "resolved", which the model would read as the node being healthy. +func TestService_buildAlertContext_HandoverIsNotARecovery(t *testing.T) { + now := time.Now() + summary := "Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading." + adapter := NewAlertManagerAdapter(&stubAlertManager{resolved: []models.ResolvedAlert{ + { + Alert: models.Alert{ + ID: "pve1-memory", + Type: "memory", + Level: "warning", + ResourceID: "pve1", + ResourceName: "pve1", + Message: "Memory usage at 95%", + StartTime: now.Add(-time.Hour), + Resolution: &models.AlertResolution{Reason: "moved_to_agent", Summary: summary}, + }, + ResolvedTime: now.Add(-time.Minute), + }, + { + Alert: models.Alert{ + ID: "pve2-cpu", + Type: "cpu", + Level: "warning", + ResourceID: "pve2", + ResourceName: "pve2", + Message: "CPU usage at 90%", + StartTime: now.Add(-time.Hour), + }, + ResolvedTime: now.Add(-time.Minute), + }, + }}) + + resolved := adapter.GetRecentlyResolved(30) + if len(resolved) != 2 || resolved[0].Resolution != summary || resolved[1].Resolution != "" { + t.Fatalf("GetRecentlyResolved = %+v, want only the handover to carry a resolution", resolved) + } + if history := adapter.GetAlertHistory("pve1", 5); len(history) != 1 || history[0].Resolution != summary { + t.Fatalf("GetAlertHistory = %+v, want the handover resolution", history) + } + + s := &Service{} + s.SetAlertProvider(adapter) + ctx := s.buildAlertContext() + if !strings.Contains(ctx, "- **memory** on pve1: Memory usage at 95% (lasted ") || + !strings.Contains(ctx, ", closed 1 minute ago) "+summary) { + t.Fatalf("expected the handover line, got: %s", ctx) + } + if !strings.Contains(ctx, "- **cpu** on pve2: CPU usage at 90% (lasted ") || + !strings.Contains(ctx, ", resolved 1 minute ago)") { + t.Fatalf("expected the ordinary recovery line, got: %s", ctx) + } +} + func TestService_buildTargetAlertContext(t *testing.T) { s := &Service{} s.SetAlertProvider(&stubAlertProvider{ diff --git a/internal/ai/memory/incidents.go b/internal/ai/memory/incidents.go index eaab0dd572..8729c83cef 100644 --- a/internal/ai/memory/incidents.go +++ b/internal/ai/memory/incidents.go @@ -474,6 +474,17 @@ func (s *IncidentStore) RecordAlertUnacknowledged(alert *alerts.Alert, user stri s.saveAsync() } +// alertResolvedEventSummary is the timeline line for an alert's close: the +// handover account when it closed without recovering, else "Alert resolved". +func alertResolvedEventSummary(alert *alerts.Alert) string { + if alert != nil { + if summary := alert.Resolution.Summary(); summary != "" { + return summary + } + } + return "Alert resolved" +} + // RecordAlertResolved records a resolved event and closes the incident. func (s *IncidentStore) RecordAlertResolved(alert *alerts.Alert, resolvedAt time.Time) { if alert == nil { @@ -507,7 +518,7 @@ func (s *IncidentStore) RecordAlertResolved(alert *alerts.Alert, resolvedAt time shell.OccurrenceClosedAt = cloneTime(resolvedAt) if !s.projectsFromCanonicalLocked() { - s.addEventAtLocked(shell, IncidentEventAlertResolved, resolvedAt, "Alert resolved", map[string]interface{}{ + s.addEventAtLocked(shell, IncidentEventAlertResolved, resolvedAt, alertResolvedEventSummary(alert), map[string]interface{}{ "resolved_at": resolvedAt.Format(time.RFC3339), }) } @@ -563,7 +574,7 @@ func (s *IncidentStore) EnsureAlertOccurrence(alert *alerts.Alert, resolvedAt *t if resolvedAt != nil && !resolvedAt.IsZero() && (shell.OccurrenceRefiredAt == nil || !resolvedAt.Before(*shell.OccurrenceRefiredAt)) { shell.OccurrenceClosedAt = cloneTime(*resolvedAt) if !hasIncidentEventType(shell.Events, IncidentEventAlertResolved) { - s.addEventAtLocked(shell, IncidentEventAlertResolved, *resolvedAt, "Alert resolved", map[string]interface{}{ + s.addEventAtLocked(shell, IncidentEventAlertResolved, *resolvedAt, alertResolvedEventSummary(alert), map[string]interface{}{ "resolved_at": resolvedAt.Format(time.RFC3339), "projection_source": incidentSnapshotSource, }) @@ -1113,6 +1124,13 @@ func incidentEventSummaryFromChange(change unifiedresources.ResourceChange, even case IncidentEventAlertUnsnoozed: return "Alert notifications resumed" case IncidentEventAlertResolved: + // A handover closes the alert without recovering; its change reason + // says so and where the condition went. + if _, ok := stringMetadata(change.Metadata, unifiedresources.MetadataAlertResolution); ok { + if reason := strings.TrimSpace(change.Reason); reason != "" { + return reason + } + } return "Alert resolved" default: if summary := strings.TrimSpace(change.Reason); summary != "" { diff --git a/internal/ai/memory/incidents_test.go b/internal/ai/memory/incidents_test.go index 0d218a9a2a..94d8930593 100644 --- a/internal/ai/memory/incidents_test.go +++ b/internal/ai/memory/incidents_test.go @@ -239,6 +239,39 @@ func TestIncidentStore_EqualTimestampLifecycleKeepsResolvedState(t *testing.T) { } } +// Without a canonical resource timeline the store writes its own close event, +// which must say the alert moved rather than resolved. +func TestIncidentStore_HandoverCloseIsNotARecovery(t *testing.T) { + store := NewIncidentStore(IncidentStoreConfig{MaxIncidents: 10, MaxEventsPerIncident: 10, MaxAgeDays: 30}) + startedAt := time.Now().UTC().Truncate(time.Second).Add(-time.Hour) + movedAt := startedAt.Add(30 * time.Minute) + alert := &alerts.Alert{ + ID: "pve1-memory", + Type: "memory", + ResourceID: "pve1", + ResourceName: "pve1", + Message: "Memory usage at 95%", + StartTime: startedAt, + LastSeen: startedAt, + Resolution: &alerts.AlertResolution{Reason: alerts.AlertResolutionMovedToAgent, SuccessorName: "pve1 (Host Agent)"}, + } + summary := "Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading." + + store.RecordAlertFired(alert) + store.RecordAlertResolved(alert, movedAt) + timeline := store.GetTimelineByAlertAt(alert.ID, startedAt) + if timeline == nil || len(timeline.Events) != 2 || timeline.Events[1].Summary != summary { + t.Fatalf("recorded timeline = %#v, want the handover close", timeline) + } + + snapshot := alert.Clone() + snapshot.ID = "pve1-memory-imported" + repaired := store.EnsureAlertOccurrence(snapshot, &movedAt) + if repaired == nil || len(repaired.Events) != 2 || repaired.Events[1].Summary != summary { + t.Fatalf("repaired timeline = %#v, want the handover close", repaired) + } +} + func TestIncidentStore_ProjectsCanonicalTimelineWhenAttached(t *testing.T) { store := NewIncidentStore(IncidentStoreConfig{ MaxIncidents: 10, diff --git a/internal/ai/patrol_ai.go b/internal/ai/patrol_ai.go index 41e3feacfa..ac6c9072fc 100644 --- a/internal/ai/patrol_ai.go +++ b/internal/ai/patrol_ai.go @@ -3499,6 +3499,12 @@ func (p *PatrolService) seedHealthAndAlertsState(snap patrolRuntimeState, scoped sb.WriteString("# Recently Resolved Alerts\n") for _, r := range alerts { ago := seedFormatTimeAgo(now, r.ResolvedTime) + // A handover closes the alert without the condition clearing, so + // it must not read as the resource having recovered. + if r.Resolution != nil && strings.TrimSpace(r.Resolution.Summary) != "" { + sb.WriteString(fmt.Sprintf("- %s — closed %s. %s\n", seedAlertSubject(r.Alert), ago, strings.TrimSpace(r.Resolution.Summary))) + continue + } sb.WriteString(fmt.Sprintf("- %s — resolved %s\n", seedAlertSubject(r.Alert), ago)) } sb.WriteString("\n") diff --git a/internal/ai/patrol_ai_more_test.go b/internal/ai/patrol_ai_more_test.go index b7f168e86d..7613f10a47 100644 --- a/internal/ai/patrol_ai_more_test.go +++ b/internal/ai/patrol_ai_more_test.go @@ -922,6 +922,42 @@ func TestSeedHealthAndAlerts_NoIssues(t *testing.T) { } } +// Patrol's seed must not tell the model a node recovered when its alert only +// moved to the node's Pulse agent. +func TestSeedHealthAndAlerts_HandoverIsNotARecovery(t *testing.T) { + ps := NewPatrolService(nil, nil) + cfg := DefaultPatrolConfig() + now := time.Now() + summary := "Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading." + state := models.StateSnapshot{ + RecentlyResolved: []models.ResolvedAlert{ + { + Alert: models.Alert{ + ResourceName: "pve1", + Message: "Memory usage at 95%", + Resolution: &models.AlertResolution{Reason: "moved_to_agent", Summary: summary}, + }, + ResolvedTime: now.Add(-3 * time.Minute), + }, + { + Alert: models.Alert{ResourceName: "pve2", Message: "CPU usage at 90%"}, + ResolvedTime: now.Add(-5 * time.Minute), + }, + }, + } + + out := ps.seedHealthAndAlertsState(patrolRuntimeStateForTest(ps, state), nil, cfg, now) + if !strings.Contains(out, "- pve1: Memory usage at 95% — closed 3m ago. "+summary+"\n") { + t.Fatalf("expected the handover line, got: %s", out) + } + if strings.Contains(out, "pve1: Memory usage at 95% — resolved") { + t.Fatalf("handover reported as a recovery: %s", out) + } + if !strings.Contains(out, "- pve2: CPU usage at 90% — resolved 5m ago\n") { + t.Fatalf("expected the ordinary recovery line, got: %s", out) + } +} + func TestSeedHealthAndAlerts_UnknownDiskHealthDoesNotClaimSMARTPassed(t *testing.T) { ps := NewPatrolService(nil, nil) cfg := DefaultPatrolConfig() diff --git a/internal/ai/tools/data_types.go b/internal/ai/tools/data_types.go index db3360534a..39c0043c25 100644 --- a/internal/ai/tools/data_types.go +++ b/internal/ai/tools/data_types.go @@ -2364,4 +2364,9 @@ type ResolvedAlertSummary struct { Threshold float64 `json:"threshold,omitempty"` StartTime time.Time `json:"start_time"` ResolvedTime time.Time `json:"resolved_time"` + // Resolution is set only when the alert closed without recovering, such + // as a node metric handed to its Pulse agent; the successor resource + // owns the condition from then on. + Resolution string `json:"resolution,omitempty"` + SuccessorResourceID string `json:"successor_resource_id,omitempty"` } diff --git a/internal/ai/tools/tools_alerts.go b/internal/ai/tools/tools_alerts.go index 252d0dc7f1..1d94770676 100644 --- a/internal/ai/tools/tools_alerts.go +++ b/internal/ai/tools/tools_alerts.go @@ -19,7 +19,7 @@ func (e *PulseToolExecutor) registerAlertsTools() { Actions: - list: List active threshold alerts (CPU > 80%, disk full, etc.) - findings: List AI patrol findings (detected issues) -- resolved: List recently resolved alerts +- resolved: List recently resolved alerts; one with a resolution closed without recovering - resolve: Mark a finding as resolved - dismiss: Dismiss a finding as not an issue @@ -417,7 +417,7 @@ func (e *PulseToolExecutor) executeListResolvedAlerts(_ context.Context, args ma break } - alerts = append(alerts, ResolvedAlertSummary{ + summary := ResolvedAlertSummary{ ID: alert.ID, Type: alert.Type, Level: alert.Level, @@ -430,7 +430,12 @@ func (e *PulseToolExecutor) executeListResolvedAlerts(_ context.Context, args ma Threshold: alert.Threshold, StartTime: alert.StartTime, ResolvedTime: alert.ResolvedTime, - }) + } + if resolution := alert.Resolution; resolution != nil { + summary.Resolution = resolution.Summary + summary.SuccessorResourceID = resolution.SuccessorResourceID + } + alerts = append(alerts, summary) } if alerts == nil { diff --git a/internal/ai/tools/tools_metrics_alerts_test.go b/internal/ai/tools/tools_metrics_alerts_test.go index 5f5e6aa1bb..ada13ba204 100644 --- a/internal/ai/tools/tools_metrics_alerts_test.go +++ b/internal/ai/tools/tools_metrics_alerts_test.go @@ -215,6 +215,49 @@ func TestExecuteListResolvedAlerts(t *testing.T) { if len(resp.Alerts) != 1 || resp.Alerts[0].ID != "a1" { t.Fatalf("unexpected resolved alerts: %+v", resp) } + if resp.Alerts[0].Resolution != "" || resp.Alerts[0].SuccessorResourceID != "" { + t.Fatalf("a recovery must not carry a resolution: %+v", resp.Alerts[0]) + } +} + +// A node alert handed to its Pulse agent closes without recovering, so the +// assistant must see that, and where the condition went, instead of a plain +// resolve it would read as the node being healthy again. +func TestExecuteListResolvedAlertsReportsHandover(t *testing.T) { + alertProv := &mockAlertProvider{} + executor := NewPulseToolExecutor(ExecutorConfig{}) + executor.alertProvider = alertProv + now := time.Now() + summary := "Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading." + alertProv.On("GetRecentlyResolved", 24*60).Return([]models.ResolvedAlert{{ + Alert: models.Alert{ + ID: "pve1-memory", + Type: "memory", + Level: "warning", + ResourceID: "pve1", + ResourceName: "pve1", + Message: "Memory usage at 95%", + StartTime: now.Add(-time.Hour), + Resolution: &models.AlertResolution{ + Reason: "moved_to_agent", + SuccessorResourceID: "agent-pve1", + SuccessorName: "pve1 (Host Agent)", + Summary: summary, + }, + }, + ResolvedTime: now, + }}) + + result, _ := executor.executeListResolvedAlerts(context.Background(), map[string]interface{}{}) + var resp ResolvedAlertsResponse + if err := json.Unmarshal([]byte(result.Content[0].Text), &resp); err != nil { + t.Fatalf("decode resolved alerts: %v", err) + } + if len(resp.Alerts) != 1 || + resp.Alerts[0].Resolution != summary || + resp.Alerts[0].SuccessorResourceID != "agent-pve1" { + t.Fatalf("resolved alerts = %+v, want the handover and its successor", resp.Alerts) + } } func TestExecuteListAlertsAndFindings(t *testing.T) { diff --git a/internal/alerts/history_test.go b/internal/alerts/history_test.go index 37c066d232..89430dc711 100644 --- a/internal/alerts/history_test.go +++ b/internal/alerts/history_test.go @@ -7,7 +7,9 @@ import ( "testing" "time" + "github.com/rcourtman/pulse-go-rewrite/internal/models" "github.com/rcourtman/pulse-go-rewrite/internal/operationaltrust" + "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources" ) // newTestHistoryManager creates a HistoryManager using a temp directory @@ -1058,3 +1060,62 @@ func TestSaveHistoryWithRetry_RestoresBackupOnFailure(t *testing.T) { _ = hm } + +// The state snapshot, the websocket and the assistant read recently resolved +// alerts through GetRecentlyResolved, so a handover must reach them as one +// and not as a recovery. +func TestRecentlyResolvedCarriesHandoverResolution(t *testing.T) { + manager := newUnifiedEvalParityManager(t) + manager.UpdateConfig(AlertConfig{ + Enabled: true, + NodeDefaults: ThresholdConfig{ + CPU: &HysteresisThreshold{Trigger: 80, Clear: 75}, + Memory: &HysteresisThreshold{Trigger: 85, Clear: 80}, + }, + }) + disableTestTimeThresholds(manager) + node := models.Node{ + ID: "pve-1", + Name: "pve-1", + Instance: "test", + Status: "online", + CPU: 0.95, + Memory: models.Memory{Total: 8 << 30, Used: 7 << 30, Usage: 95}, + } + manager.CheckNode(node) + release := func(metric string, resolution *AlertResolution) { + t.Helper() + spec, err := buildCanonicalMetricSpec(node.ID, node.Name, unifiedresources.ResourceType("node"), metric, nil) + if err != nil { + t.Fatalf("build %s spec: %v", metric, err) + } + manager.releaseCanonicalMetricAlert(spec, node.Name, node.Name, node.Instance, "node", 0, resolution) + } + release("memory", &AlertResolution{ + Reason: AlertResolutionMovedToAgent, + SuccessorResourceID: "agent-pve-1", + SuccessorName: "pve-1 (Host Agent)", + }) + release("cpu", nil) + + byType := map[string]models.ResolvedAlert{} + for _, resolved := range manager.GetRecentlyResolved() { + byType[resolved.Type] = resolved + } + moved, ok := byType["memory"] + if !ok { + t.Fatalf("recently resolved = %+v, want the moved memory alert", byType) + } + want := models.AlertResolution{ + Reason: "moved_to_agent", + SuccessorResourceID: "agent-pve-1", + SuccessorName: "pve-1 (Host Agent)", + Summary: "Alert moved to pve-1 (Host Agent). This is not a recovery: check the agent for the current reading.", + } + if moved.Resolution == nil || *moved.Resolution != want { + t.Fatalf("moved resolution = %+v, want %+v", moved.Resolution, want) + } + if recovered, ok := byType["cpu"]; !ok || recovered.Resolution != nil { + t.Fatalf("recovered cpu alert = %+v, want an ordinary resolve with no resolution", recovered) + } +} diff --git a/internal/alerts/operational_contract.go b/internal/alerts/operational_contract.go index f59fe89a1b..5da5ff5e6e 100644 --- a/internal/alerts/operational_contract.go +++ b/internal/alerts/operational_contract.go @@ -285,6 +285,7 @@ func markOperationalResolved( } alert.Evidence = appendOperationalEvidence(alert.Evidence, recoveryEvidence.Clone()) recoveryEvidenceIDs := []string{recoveryEvidence.ID} + cause := resolvedTransitionCause(alert.Resolution) from := record.State record.State = operationaltrust.OperationalResolved record.StateChangedAt = resolvedAt @@ -295,7 +296,7 @@ func markOperationalResolved( from, operationaltrust.OperationalResolved, resolvedAt, - operationaltrust.TransitionRecoveryEvidence, + cause, record.CauseKey, recoveryEvidenceIDs, ) @@ -308,7 +309,7 @@ func markOperationalResolved( From: from, To: operationaltrust.OperationalResolved, At: resolvedAt, - Cause: operationaltrust.TransitionRecoveryEvidence, + Cause: cause, CauseKey: record.CauseKey, EvidenceIDs: recoveryEvidenceIDs, Reason: alert.Resolution.Summary(), @@ -319,6 +320,16 @@ func markOperationalResolved( ) } +// resolvedTransitionCause names why a record entered resolved. Only a close +// with no typed resolution is a recovery; a handover keeps its own cause so +// timelines and alert-quality counts never report it as one. +func resolvedTransitionCause(resolution *AlertResolution) operationaltrust.TransitionCause { + if resolution != nil && resolution.Reason == AlertResolutionMovedToAgent { + return operationaltrust.TransitionOwnershipTransferred + } + return operationaltrust.TransitionRecoveryEvidence +} + func (m *Manager) newResolvedAlert( alert *Alert, resolvedAt time.Time, diff --git a/internal/alerts/operational_contract_test.go b/internal/alerts/operational_contract_test.go index f483dd11b0..c15a8b48f5 100644 --- a/internal/alerts/operational_contract_test.go +++ b/internal/alerts/operational_contract_test.go @@ -504,6 +504,15 @@ func TestReleasedMetricAlertCarriesResolutionInsteadOfRecovery(t *testing.T) { if err := transition.Validate(); err != nil { t.Fatalf("resolve transition Validate() error = %v", err) } + // Patrol's timeline prints the cause above the reason, so a handover + // must not be labelled as recovery evidence. + if transition.Cause != operationaltrust.TransitionOwnershipTransferred { + t.Fatalf("resolve transition cause = %q, want %q", transition.Cause, operationaltrust.TransitionOwnershipTransferred) + } + if recorded := moved.Alert.Transitions[len(moved.Alert.Transitions)-1]; recorded.ID != transition.ID || + recorded.Cause != operationaltrust.TransitionOwnershipTransferred { + t.Fatalf("recorded timeline transition = %+v, want the ownership transfer", recorded) + } closing := moved.Alert.Evidence[len(moved.Alert.Evidence)-1] if closing.ID != transition.EvidenceIDs[0] || closing.Reason == nil || @@ -520,6 +529,10 @@ func TestReleasedMetricAlertCarriesResolutionInsteadOfRecovery(t *testing.T) { last.Reason.Code != legacyAlertRecoveryEvidenceReason { t.Fatalf("plain release evidence = %+v, want legacy recovery projection", last.Reason) } + if recovered.Alert.LatestTransition == nil || + recovered.Alert.LatestTransition.Cause != operationaltrust.TransitionRecoveryEvidence { + t.Fatalf("plain release transition = %+v, want recovery_evidence", recovered.Alert.LatestTransition) + } var historyResolution *AlertResolution for _, entry := range manager.GetAlertHistory(10) { diff --git a/internal/alerts/read_model.go b/internal/alerts/read_model.go index a30630c5ba..68382bc6ea 100644 --- a/internal/alerts/read_model.go +++ b/internal/alerts/read_model.go @@ -301,7 +301,8 @@ func (m *Manager) GetRecentlyResolved() []models.ResolvedAlert { StartTime: exported.StartTime, Acknowledged: exported.Acknowledged, // exported is a deep clone, so the map is already private. - Metadata: exported.Metadata, + Metadata: exported.Metadata, + Resolution: modelAlertResolution(exported.Resolution), }, ResolvedTime: alert.ResolvedTime, }) @@ -309,6 +310,20 @@ func (m *Manager) GetRecentlyResolved() []models.ResolvedAlert { return resolved } +// modelAlertResolution projects a non-recovery close into the state model, so +// snapshot and assistant readers can tell a handover from a recovery. +func modelAlertResolution(resolution *AlertResolution) *models.AlertResolution { + if resolution == nil { + return nil + } + return &models.AlertResolution{ + Reason: string(resolution.Reason), + SuccessorResourceID: resolution.SuccessorResourceID, + SuccessorName: resolution.SuccessorName, + Summary: resolution.Summary(), + } +} + // GetResolvedAlert returns a copy of a recently resolved alert by ID. func (m *Manager) GetResolvedAlert(alertID string) *ResolvedAlert { // Write lock: getResolvedAlertNoLock backfills the resolvedAlias map on a diff --git a/internal/models/deepcopy.go b/internal/models/deepcopy.go index 15d487f630..184633c596 100644 --- a/internal/models/deepcopy.go +++ b/internal/models/deepcopy.go @@ -1337,6 +1337,10 @@ func cloneAlert(src Alert) Alert { dest.Metadata[k] = v } } + if src.Resolution != nil { + resolution := *src.Resolution + dest.Resolution = &resolution + } return dest } diff --git a/internal/models/metrics_types_test.go b/internal/models/metrics_types_test.go index 86d4df09d0..8c29f1aeb9 100644 --- a/internal/models/metrics_types_test.go +++ b/internal/models/metrics_types_test.go @@ -96,6 +96,43 @@ func TestAlertLastSeenWireContract(t *testing.T) { } } +// A resolved alert that closed without recovering says so on the state and +// websocket payloads; an ordinary recovery carries no resolution at all. +func TestAlertResolutionWireContract(t *testing.T) { + recovered, err := json.Marshal(ResolvedAlert{Alert: Alert{ID: "alert-1"}}) + if err != nil { + t.Fatalf("marshal recovered alert: %v", err) + } + if strings.Contains(string(recovered), "resolution") { + t.Fatalf("payload = %s, a recovery must not carry a resolution", recovered) + } + + moved := ResolvedAlert{Alert: Alert{ID: "alert-1", Resolution: &AlertResolution{ + Reason: "moved_to_agent", + SuccessorResourceID: "agent-pve-1", + SuccessorName: "pve-1 (Host Agent)", + Summary: "Alert moved to pve-1 (Host Agent). This is not a recovery: check the agent for the current reading.", + }}} + payload, err := json.Marshal(moved) + if err != nil { + t.Fatalf("marshal moved alert: %v", err) + } + want := `"resolution":{"reason":"moved_to_agent","successorResourceId":"agent-pve-1","successorName":"pve-1 (Host Agent)","summary":"Alert moved to pve-1 (Host Agent). This is not a recovery: check the agent for the current reading."}` + if !strings.Contains(string(payload), want) { + t.Fatalf("payload = %s, want %s", payload, want) + } + + // The state store keeps its own copy of each resolution. + state := NewState() + state.UpdateRecentlyResolved([]ResolvedAlert{moved}) + moved.Resolution.SuccessorName = "changed" + snapshot := state.GetSnapshot().RecentlyResolved + if len(snapshot) != 1 || snapshot[0].Resolution == nil || + snapshot[0].Resolution.SuccessorName != "pve-1 (Host Agent)" { + t.Fatalf("snapshot = %+v, want the resolution as it was recorded", snapshot) + } +} + func TestDockerHostCollectionModeWireContract(t *testing.T) { payload, err := json.Marshal(DockerHost{ ID: "docker-host-1", diff --git a/internal/models/models.go b/internal/models/models.go index 5f1f2af5b8..71cc2c57fa 100644 --- a/internal/models/models.go +++ b/internal/models/models.go @@ -110,6 +110,20 @@ type Alert struct { // Metadata carries alert-engine annotations (notably resourceType) so the // frontend can classify an alert without re-deriving resource identity. Metadata map[string]interface{} `json:"metadata,omitempty"` + // Resolution is set only on a resolved alert whose close was not a + // recovery, so readers never report its resource as healthy. + Resolution *AlertResolution `json:"resolution,omitempty"` +} + +// AlertResolution is the state projection of the alert engine's +// AlertResolution: why an alert closed without its condition clearing. +type AlertResolution struct { + Reason string `json:"reason"` + SuccessorResourceID string `json:"successorResourceId,omitempty"` + SuccessorName string `json:"successorName,omitempty"` + // Summary is the alert engine's one-line account, such as "Alert moved + // to pve1 (Host Agent). This is not a recovery: ...". + Summary string `json:"summary,omitempty"` } // ResolvedAlert represents a recently resolved alert diff --git a/internal/monitoring/monitor_alert_handling_test.go b/internal/monitoring/monitor_alert_handling_test.go index 498bd35b8d..5159501c59 100644 --- a/internal/monitoring/monitor_alert_handling_test.go +++ b/internal/monitoring/monitor_alert_handling_test.go @@ -238,6 +238,68 @@ func TestMonitor_HandleAlertLifecycle_WritesCanonicalChanges(t *testing.T) { } } +// A node alert handed to its Pulse agent closes without recovering. The +// resource history and the alert's incident timeline (the Alerts history row +// expansion) must say where it went, not "Alert resolved: Memory usage at 95%". +func TestMonitor_HandleAlertLifecycle_HandoverCloseIsNotARecovery(t *testing.T) { + resourceStore := unifiedresources.NewMemoryStore() + incidentStore := memory.NewIncidentStore(memory.IncidentStoreConfig{}) + m := &Monitor{ + incidentStore: incidentStore, + resourceStore: unifiedresources.NewMonitorAdapter(unifiedresources.NewRegistry(resourceStore)), + } + incidentStore.SetResourceTimelineStore(m.resourceStore.(memory.IncidentTimelineStore)) + + startedAt := time.Date(2026, 10, 6, 9, 0, 0, 0, time.UTC) + movedAt := startedAt.Add(2 * time.Hour) + alert := &alerts.Alert{ + ID: "pve1-memory", + Type: "memory", + Level: alerts.AlertLevelWarning, + ResourceID: "pve1", + Message: "Memory usage at 95%", + Value: 95, + Threshold: 85, + StartTime: startedAt, + } + m.handleAlertLifecycleEvent(alerts.LifecycleEvent{Type: eventlog.TypeFired, OccurredAt: startedAt, Alert: alert}) + closed := alert.Clone() + closed.Resolution = &alerts.AlertResolution{ + Reason: alerts.AlertResolutionMovedToAgent, + SuccessorResourceID: "agent-pve1", + SuccessorName: "pve1 (Host Agent)", + } + m.handleAlertLifecycleEvent(alerts.LifecycleEvent{Type: eventlog.TypeResolved, OccurredAt: movedAt, Alert: closed}) + + summary := "Alert moved to pve1 (Host Agent). This is not a recovery: check the agent for the current reading." + changes, err := resourceStore.GetRecentChanges("pve1", time.Time{}, 10) + require.NoError(t, err) + require.Len(t, changes, 2) + require.Equal(t, unifiedresources.ChangeAlertResolved, changes[0].Kind) + require.Equal(t, summary, changes[0].Reason) + require.Equal(t, "moved_to_agent", changes[0].Metadata[unifiedresources.MetadataAlertResolution]) + require.Equal(t, unifiedresources.ChangeAlertFired, changes[1].Kind) + require.NotContains(t, changes[1].Metadata, unifiedresources.MetadataAlertResolution) + + timeline := incidentStore.GetTimelineByAlertAt(alert.ID, startedAt) + require.NotNil(t, timeline) + require.Len(t, timeline.Events, 2) + require.Equal(t, memory.IncidentEventAlertResolved, timeline.Events[1].Type) + require.Equal(t, summary, timeline.Events[1].Summary) + + // An ordinary recovery keeps its existing wording. + recovered := alert.Clone() + recovered.ID = "pve1-cpu" + recovered.Type = "cpu" + recovered.Message = "CPU usage at 90%" + m.handleAlertLifecycleEvent(alerts.LifecycleEvent{Type: eventlog.TypeFired, OccurredAt: startedAt, Alert: recovered}) + m.handleAlertLifecycleEvent(alerts.LifecycleEvent{Type: eventlog.TypeResolved, OccurredAt: movedAt.Add(time.Minute), Alert: recovered}) + recoveredTimeline := incidentStore.GetTimelineByAlertAt(recovered.ID, startedAt) + require.NotNil(t, recoveredTimeline) + require.Len(t, recoveredTimeline.Events, 2) + require.Equal(t, "Alert resolved", recoveredTimeline.Events[1].Summary) +} + func TestPausedDeliveryStillBuildsTimelineThroughRealAlertLifecycle(t *testing.T) { manager := alerts.NewManagerWithDataDir(t.TempDir()) t.Cleanup(manager.Stop) diff --git a/internal/monitoring/monitor_alerts.go b/internal/monitoring/monitor_alerts.go index 46445c3ec7..f7aeb761d8 100644 --- a/internal/monitoring/monitor_alerts.go +++ b/internal/monitoring/monitor_alerts.go @@ -576,7 +576,7 @@ func (m *Monitor) recordAlertTimelineChange(alert *alerts.Alert, kind unifiedres return } - change := unifiedresources.BuildAlertTimelineChange(alert.ResourceID, kind, occurredAt, actor, unifiedresources.AlertTimelineChange{ + timelineChange := unifiedresources.AlertTimelineChange{ AlertIdentifier: alert.ID, AlertStartedAt: alert.StartTime, AlertType: alert.Type, @@ -585,7 +585,12 @@ func (m *Monitor) recordAlertTimelineChange(alert *alerts.Alert, kind unifiedres AlertValue: alert.Value, AlertThreshold: alert.Threshold, AlertMetadata: alert.Metadata, - }) + } + if summary := alert.Resolution.Summary(); summary != "" { + timelineChange.ResolutionReason = string(alert.Resolution.Reason) + timelineChange.ResolutionSummary = summary + } + change := unifiedresources.BuildAlertTimelineChange(alert.ResourceID, kind, occurredAt, actor, timelineChange) if change == nil { return } diff --git a/internal/operationaltrust/contracts.go b/internal/operationaltrust/contracts.go index 19b1a1b20e..303ebf857b 100644 --- a/internal/operationaltrust/contracts.go +++ b/internal/operationaltrust/contracts.go @@ -493,8 +493,12 @@ const ( TransitionSuppression TransitionCause = "suppression" TransitionSuppressionExpired TransitionCause = "suppression_expired" TransitionRecoveryEvidence TransitionCause = "recovery_evidence" - TransitionCollectionStale TransitionCause = "collection_stale" - TransitionCollectionUnknown TransitionCause = "collection_unknown" + // TransitionOwnershipTransferred closes a record whose condition another + // record now owns, such as a node metric a linked Pulse agent took over. + // The condition was not observed to clear, so it is not a recovery. + TransitionOwnershipTransferred TransitionCause = "ownership_transferred" + TransitionCollectionStale TransitionCause = "collection_stale" + TransitionCollectionUnknown TransitionCause = "collection_unknown" ) func (cause TransitionCause) valid() bool { @@ -505,6 +509,7 @@ func (cause TransitionCause) valid() bool { TransitionSuppression, TransitionSuppressionExpired, TransitionRecoveryEvidence, + TransitionOwnershipTransferred, TransitionCollectionStale, TransitionCollectionUnknown: return true @@ -617,6 +622,13 @@ func (transition LifecycleTransition) validateWithoutID() error { if len(canonicalIDs(transition.EvidenceIDs)) == 0 { return errors.New("recovery transition requires evidence") } + case TransitionOwnershipTransferred: + if transition.To != OperationalResolved { + return errors.New("ownership transfer must enter resolved state") + } + if len(canonicalIDs(transition.EvidenceIDs)) == 0 { + return errors.New("ownership transfer requires evidence") + } case TransitionCollectionStale: if transition.To != OperationalStale { return errors.New("collection-stale transition must enter stale state") diff --git a/internal/operationaltrust/contracts_test.go b/internal/operationaltrust/contracts_test.go index 13039d5932..80381c4a26 100644 --- a/internal/operationaltrust/contracts_test.go +++ b/internal/operationaltrust/contracts_test.go @@ -235,6 +235,48 @@ func TestLifecycleTransitionEnforcesOperationalSemantics(t *testing.T) { } } +// A close that hands the condition to another record is not a recovery, so it +// has its own cause. It may only close the record, and only with the evidence +// that recorded the handover. +func TestOwnershipTransferOnlyClosesWithEvidence(t *testing.T) { + at := time.Date(2026, 10, 6, 15, 0, 0, 0, time.UTC) + if _, err := NewTransitionID( + "record-1", + OperationalOpen, + OperationalResolved, + at, + TransitionOwnershipTransferred, + "memory-high", + []string{"evidence-handover"}, + ); err != nil { + t.Fatalf("open-to-resolved ownership transfer error = %v", err) + } + if _, err := NewTransitionID( + "record-1", + OperationalAcknowledged, + OperationalResolved, + at, + TransitionOwnershipTransferred, + "memory-high", + nil, + ); err == nil || !strings.Contains(err.Error(), "requires evidence") { + t.Fatalf("evidenceless ownership transfer error = %v", err) + } + for _, to := range []OperationalState{OperationalResolving, OperationalOpen, OperationalStale} { + if _, err := NewTransitionID( + "record-1", + OperationalAcknowledged, + to, + at, + TransitionOwnershipTransferred, + "memory-high", + []string{"evidence-handover"}, + ); err == nil || !strings.Contains(err.Error(), "must enter resolved") { + t.Fatalf("ownership transfer into %q error = %v", to, err) + } + } +} + func TestOperationalRecordRequiresResolutionEvidence(t *testing.T) { firstObserved := time.Date(2026, 7, 18, 20, 0, 0, 0, time.UTC) resolvedAt := firstObserved.Add(5 * time.Minute) diff --git a/internal/unifiedresources/incident_changes.go b/internal/unifiedresources/incident_changes.go index abde5af5c3..8d39473401 100644 --- a/internal/unifiedresources/incident_changes.go +++ b/internal/unifiedresources/incident_changes.go @@ -16,6 +16,9 @@ const ( MetadataAlertMessage = "alert_message" MetadataAlertValue = "alert_value" MetadataAlertThreshold = "alert_threshold" + // MetadataAlertResolution is the reason code of a resolve that was not a + // recovery, such as "moved_to_agent". Ordinary recoveries omit it. + MetadataAlertResolution = "alert_resolution" MetadataCommand = "command" MetadataSuccess = "success" MetadataOutputExcerpt = "output_excerpt" @@ -38,6 +41,11 @@ type AlertTimelineChange struct { AlertValue float64 AlertThreshold float64 AlertMetadata map[string]any + // ResolutionReason and ResolutionSummary describe a resolve that was not + // a recovery. They are empty for an ordinary recovery and ignored on + // every other lifecycle kind. + ResolutionReason string + ResolutionSummary string } // BuildAlertTimelineChange constructs a canonical resource change for alert @@ -74,6 +82,9 @@ func BuildAlertTimelineChange(resourceID string, kind ChangeKind, occurredAt tim if !alert.AlertStartedAt.IsZero() { change.Metadata[MetadataAlertStartedAt] = alert.AlertStartedAt.UTC().Format(time.RFC3339Nano) } + if reason := strings.TrimSpace(alert.ResolutionReason); kind == ChangeAlertResolved && reason != "" { + change.Metadata[MetadataAlertResolution] = reason + } for key, value := range cloneChangeMetadata(alert.AlertMetadata) { if _, exists := change.Metadata[key]; exists { continue @@ -178,6 +189,11 @@ func alertChangeReason(kind ChangeKind, alert AlertTimelineChange) string { case ChangeAlertUnsnoozed: return "Alert notifications resumed" case ChangeAlertResolved: + // A close that was not a recovery must not read as one: the alert's + // own message describes the condition that is still unresolved. + if summary := strings.TrimSpace(alert.ResolutionSummary); summary != "" { + return summary + } if message != "" { return fmt.Sprintf("Alert resolved: %s", message) }