Implement canonical resource monitoring policy

This commit is contained in:
rcourtman
2026-08-11 11:48:51 +01:00
parent 6b79aa9972
commit a04941b55a
53 changed files with 1402 additions and 97 deletions
+1 -1
View File
@@ -234,7 +234,7 @@ the live set.
**Operator state (per-resource intent):**
- `get_operator_state` (Get operator state, `GET /api/resources/{resourceId}/operator-state`, scope `monitoring:read`, mode `read`, approval `scope_only`): Read the operator-set state for a resource (intentionally offline, never auto-remediate, maintenance window, criticality).
- `get_operator_state` (Get operator state, `GET /api/resources/{resourceId}/operator-state`, scope `monitoring:read`, mode `read`, approval `scope_only`): Read the operator-set monitoring mode, lifecycle state, remediation lock, maintenance window, and criticality for a resource.
- `set_operator_state` (Set operator state, `PUT /api/resources/{resourceId}/operator-state`, scope `monitoring:write`, mode `write`, approval `scope_only`): Replace the operator-set state for a resource. URL canonicalId wins over body; server populates setAt and setBy from the authenticated identity.
- `clear_operator_state` (Clear operator state, `DELETE /api/resources/{resourceId}/operator-state`, scope `monitoring:write`, mode `write`, approval `scope_only`): Remove any operator-set state for a resource. Idempotent — succeeds whether or not an entry was present.
@@ -6149,3 +6149,15 @@ agent tokens, command sessions, and report identity. Deprovisioning a human
principal revokes that principal's browser sessions only and does not delete,
reassign, or synthesize agent lifecycle state. The api-contracts and
security-privacy contracts own the authoritative behavior and proof.
### Resource monitoring policy does not mutate agent lifecycle
The shared `internal/api/` router now reconciles alert attention after a
resource operator-state mutation, but it does not unregister, delete, retoken,
or reassign an agent. Monitoring and lifecycle policy is keyed through the
tenant's canonical resource registry and is separate from enrolled agent
identity. Agent context may report `monitoringMode` and `lifecycleState` so an
operator or external agent can explain current attention behavior, without
changing registration or command authority. Retiring an agent-backed resource
blocks automated remediation through the shared action policy while preserving
its enrollment and history.
@@ -6904,6 +6904,22 @@ to the same policy as the outer address, so `PULSE_AI_ALLOW_LOOPBACK` and
`PULSE_AI_ALLOW_PRIVATE_IPS` relax the embedded check exactly as they relax the
outer one, and a transition address wrapping a permitted public target stays
permitted.
### Patrol consumes canonical monitoring and lifecycle policy
The findings operator-state projection carries canonical `MonitoringMode` and
`LifecycleState` in the same store read as maintenance, criticality, and action
lock state. Active maintenance, expected-offline, muted, and retired policy
produce distinct lifecycle causes in durable finding history. Suppression is
reversible when the canonical cause lifts. Legacy providers that project only
`IntentionallyOffline` retain their established cause and behavior.
Retired lifecycle also projects an effective remediation lock into Patrol
investigation and the shared execution admission paths, so a resource cannot
be retired from attention while remaining eligible for autonomous action.
Canonical enum fields are additive in the agent capability manifest and its
generated documentation; the legacy boolean remains a compatibility edge, not
the AI runtime policy source of truth.
### Patrol run records no longer carry a dead auto-fix count
`PatrolRunRecord.AutoFixCount` was removed from `internal/ai/patrol.go`,
@@ -1633,3 +1633,24 @@ incident from cluster health.
`internal/truenas/provider_pool_health_contract_test.go`, and
`internal/unifiedresources/ceph_pool_health_contract_test.go` prove the
lifecycle and deduplication matrix.
### Per-resource monitoring policy is an alert-runtime invariant
The canonical unified-resource operator state is honored without requiring an
explicit versioned alert-intent rule. Factory behavior enables operator-state
evaluation, and a signal rule cannot weaken a persisted resource monitoring or
lifecycle policy. `expected_offline` suppresses only `state.offline` and
`incident.availability`; `muted`, `retired`, and active maintenance suppress all
signals. Unknown alert families fail safe to the default signal and cannot be
hidden by expected-offline.
Every active-alert writer passes through the same operator-state gate, and
notification delivery consults the same decision before quiet-hours policy.
Operator suppression is never converted into a quiet-hours replay, including
for recovery notifications carrying stale replay metadata.
After a persisted policy mutation, `ReconcileResourceOperatorState` resolves
already-active records for that resource immediately; later detector writes
cannot recreate them while suppression remains active. Existing prefix, tag,
and `pulse-no-alerts` bulk rules remain compatible inputs and do not create a
second per-resource state store. `internal/alerts/intent_policy_test.go` pins
factory operator-state evaluation, writer gating, and active reconciliation.
@@ -9255,3 +9255,22 @@ The route-authorization test proves the exact viewer 200 / ledger 403 pairing,
the scope test proves unauthenticated and wrong-scope refusal, and the frontend
API plus Workloads integration tests prove the surface reads only the runtime
projection.
### Resource operator-state API exposes canonical policy with legacy compatibility
`GET` and `PUT /api/resources/{id}/operator-state` add `monitoringMode` and
`lifecycleState` to the stable resource-state shape. The accepted vocabularies
are `normal|expected_offline|muted` and `active|retired`. The existing
`intentionallyOffline` field remains readable and writable for older clients:
when the enum is absent, true maps to expected-offline; when the enum is
present, it is authoritative and the response boolean is derived from it.
Invalid enum values fail with the existing stable operator-state validation
error.
A successful mutation invokes alert reconciliation only after durable commit,
so the API cannot report a policy that has not landed and existing active
alerts leave attention promptly. Agent resource context, fleet context, and
the canonical capability manifest project both new fields. Retired lifecycle
is also an execution-time remediation lock in the shared action lifecycle.
`internal/api/resources_operator_state_test.go`, agent-context tests, manifest
tests, and the generated `cmd/pulse-mcp/README.md` pin this additive contract.
@@ -5968,3 +5968,15 @@ unresolved status would flash the default tab away from an admin on every load.
`__tests__/adminOnlySettingsNavGates.test.ts` and
`__tests__/useSettingsAccess.test.tsx` pin the complete named-capability,
fail-closed mount, failed-status, deduplicated-load, and General-fallback rules.
### Alert monitoring actions preserve domain ownership
The Alerts overview may offer a compact per-resource Monitoring menu, but the
menu is an adapter over the canonical resource operator-state API. It must
preserve unrelated state on every write, distinguish availability-only
expected-offline from all-attention mute, and state that retirement changes
Pulse monitoring rather than deleting provider inventory. Resource detail and
alert surfaces use the same typed monitoring and lifecycle vocabulary and
provider-ownership presentation helper. They must not create local alert mute,
archive, or removal state. The menu remains keyboard reachable, uses ordinary
shared surface and border tokens, and retains usable controls at phone width.
@@ -2940,3 +2940,14 @@ Known and deliberately unchanged: each changed guest still triggers a full-file
save, so one poll cycle over N changed guests performs N marshals and N atomic
writes that serialize on the store mutex. Coalescing them is a behavioural
change beyond the shutdown defect.
### Monitoring projects canonical resource policy into alert evaluation
The monitoring-owned operator-intent adapter projects `monitoringMode` and
`lifecycleState` together with maintenance timing and the legacy compatibility
boolean. It still resolves source-native references through canonical resource
identity before reading the store and fails open on missing, ambiguous, or
errored identity lookup. Monitoring does not reinterpret provider ownership or
invent lifecycle state; Alerts owns signal suppression and unified resources
owns persistence. `internal/monitoring/monitor_alert_intent_test.go` and the
alerts intent-policy proof pin this adapter boundary.
@@ -2169,3 +2169,20 @@ reads, cancellation, and clean retry.
`frontend-modern/src/features/patrol/__tests__/usePatrolIntelligenceState.test.ts`
and `frontend-modern/src/pages/__tests__/AIIntelligence.test.tsx` prove typed
start-failure presentation and the accepted-run page transition.
### Resource monitoring policy is actionable where noise appears
The resource drawer replaces the overloaded intentionally-offline toggle with
canonical Monitoring and Lifecycle selectors. Normal, expected-offline, muted,
active, and retired copy states their exact Alerts, Patrol, inventory, and
automation effects. Provider-aware retirement copy makes clear that Proxmox,
Kubernetes, vSphere, and container-runtime inventory is not deleted by a Pulse
policy change and that the resource can be restored to active monitoring.
Active alert cards expose the same canonical policy through a compact
Monitoring action. It reads the current resource state, preserves maintenance,
criticality, notes, remediation settings, and capability allowlists, then
writes through `@/api/resourceOperatorState`; no alert-local mute document is
created. `ResourceMonitoringPolicyAction.test.tsx` and the resource operator
state section tests pin provider copy, expected-offline persistence, retirement,
and compatibility preservation.
@@ -2364,3 +2364,12 @@ enrichment. It filters healthy, disabled, non-workload, and coverage-free rows
before serialization, so the response is no larger than the actionable banner
input. Client presentation is a single pass over that bounded list and performs
no per-source fetch or detail lookup.
### Resource policy reads stay outside the Workloads hot path
The Proxmox guest drawer mounts `ResourceOperatorStateSection` only for the
currently expanded workload. Its non-suspending query performs one canonical
operator-state read for that selected resource and does not add table-row,
hover-preview, interval, or websocket work. Saving monitoring or lifecycle
policy performs one mutation and one selected-resource refetch. The Workloads
list, selectors, windowing, and polling budgets remain unchanged.
@@ -2126,3 +2126,12 @@ because `RequireAdmin` intentionally leaves the update route reachable there.
The frontend behavior is pinned by `App.architecture.test.ts`, and
`TestContract_SecurityStatusSystemSettingsReadTracksSettingsReadScope` pins the
capability against both `/api/system/settings` and `/api/updates/status`.
### Resource policy identity resolution remains tenant scoped
Operator-state reads and writes resolve source-native or superseded resource
references through the requesting tenant's canonical registry before touching
the tenant resource store. Runtime reconciliation may visit every live monitor,
but each alert manager resolves policy through its own tenant-scoped store, so a
matching provider ID in another organization cannot import the mutation. The
existing route scopes and authenticated actor attribution remain unchanged.
@@ -5157,3 +5157,13 @@ restore, retention, resource-store, or recovery evidence. Additive RBAC schema
upgrade preserves older identities and fails construction on migration errors;
the security-privacy and api-contracts contracts own the authoritative shape
and proof.
### Resource monitoring lifecycle preserves recovery evidence
Mutating `monitoringMode` or `lifecycleState` changes Pulse attention and
automated remediation admission only. Expected-offline, mute, and retirement do
not delete resource history, backups, snapshots, recovery points, retention
configuration, or provider inventory. A retired resource fails closed for
automated remediation through the shared action lifecycle, while recovery
evidence remains available for operator review and for restoration to active
monitoring.
@@ -4450,3 +4450,24 @@ teardown. `TestCachedResourceStoresHaveATenantReleasePath` in
`internal/unifiedresources/code_standards_test.go` is a source-shape guard,
because the leak is invisible at runtime until a tenant is deleted or a data
directory is torn down.
### Canonical resource monitoring and lifecycle policy
`ResourceOperatorState` owns one durable per-resource monitoring posture and
lifecycle. `monitoringMode` is the closed vocabulary `normal`,
`expected_offline`, or `muted`; `lifecycleState` is `active` or `retired`.
Expected-offline is availability-specific, while muted removes all Alerts and
Patrol attention. Retired resources remain in canonical provider inventory and
retain history, but they also suppress attention and block automated
remediation until restored to active. Provider inventory deletion is not a
side effect of this policy.
The SQLite row persists both enums. Schema migration promotes legacy
`intentionally_offline=1` rows to `monitoring_mode=expected_offline`, and the
legacy boolean remains only a derived wire compatibility projection. New
runtime consumers must use the enum helpers rather than assign a second meaning
to that boolean. Identity succession continues to rekey the same state row.
`internal/unifiedresources/resource_operator_state_test.go` and
`resource_operator_state_policy_test.go` pin normalization, validation,
round-trip persistence, attention suppression, and the retired remediation
lock.
+30 -16
View File
@@ -1,18 +1,29 @@
{
"version": 1,
"base_sha": "bcb172c1274b393f1bdd8d51b4fffb7f0585e4bd",
"verified_at": "2026-08-10T08:55:00Z",
"base_sha": "c45a950a74c11d299b1be6277a8b552e87f9c5d9",
"verified_at": "2026-08-11T10:30:03Z",
"result": "passed",
"changed_paths": [
"frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx",
"frontend-modern/src/features/patrol/patrolControlPresentation.ts"
"frontend-modern/src/api/resourceOperatorState.ts",
"frontend-modern/src/components/Infrastructure/ResourceDetailDrawerOverviewTab.tsx",
"frontend-modern/src/components/Infrastructure/ResourceOperatorStateSection.tsx",
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx",
"frontend-modern/src/features/alerts/AlertOverviewAlertCard.tsx",
"frontend-modern/src/features/alerts/ResourceMonitoringPolicyAction.tsx",
"frontend-modern/src/utils/resourceMonitoringPolicy.ts"
],
"content_sha256": {
"frontend-modern/src/features/patrol/PatrolAttentionWorkbench.tsx": "da3ddd2c2fd5bf94e9642d178c8f60c361b21de3559e6085047e911539301fef",
"frontend-modern/src/features/patrol/patrolControlPresentation.ts": "1f395d06a16be436f5222cd36259a9585aaa2f7a036399fbfd4ee8b3c98bf12e"
"frontend-modern/src/api/resourceOperatorState.ts": "2b27d3dcc3415cc2a57b9bd201d06f04150ede3b29d91816ce3715a065eb8459",
"frontend-modern/src/components/Infrastructure/ResourceDetailDrawerOverviewTab.tsx": "9b757063cbd6c0b97ab0057754d45ad2e2067ad58f4ec0b388c7ae28e51d8239",
"frontend-modern/src/components/Infrastructure/ResourceOperatorStateSection.tsx": "3c5008b4db7027a7a08c8dff796a7546b5b343a522bf7439361467c0031318f3",
"frontend-modern/src/components/Workloads/GuestDrawerOverview.tsx": "f0339dc3c3fdd2e08f2c5fea75a62a0a5ecdfab833970653bf9f74982c4ba891",
"frontend-modern/src/features/alerts/AlertOverviewAlertCard.tsx": "83081abb871d76e5531bf66ddcddb47171c05c86c1d91249a0e9c6ef5dad73dd",
"frontend-modern/src/features/alerts/ResourceMonitoringPolicyAction.tsx": "287ff35169c4082f9cb8cac3e36bcbaaab9e343801c497e89da8602d9af276d2",
"frontend-modern/src/utils/resourceMonitoringPolicy.ts": "ba5133bbdeb57a236845f3ee946f0f0c13518e293810eec1ae7e4fab2ab3e2d5"
},
"routes": [
"/patrol (Needs attention workbench rows and attention detail drawer)"
"/alerts",
"/proxmox/overview"
],
"viewports": [
{
@@ -20,19 +31,22 @@
"height": 800
},
{
"width": 744,
"height": 1024
"width": 390,
"height": 844
}
],
"states": [
"Attention rows (46 mock alerts, complete + unknown-freshness evidence) render resource name and relative time with no evidence label; the 'Evidence incomplete; timing unavailable' / 'Evidence timing unavailable' warning line is gone from every row.",
"Attention detail drawer shows the muted 'Evidence recorded' badge next to the state badge, with per-envelope observations reading 'Pulse Complete Confirmed - observed just now'.",
"Full-page text contains no 'timing unavailable' occurrence in any state exercised at either viewport.",
"Backend half verified via scratch instance API: /api/ai/patrol/attention returned all 44-46 mock rows with evidenceCompleteness=complete (previously provider-incident rows were partial via the legacy shim)."
"Active powered-off Proxmox guest alert with the Monitoring policy menu open and provider ownership copy visible.",
"Normal monitoring restored, powered-off detector alert reactivated, then Expected offline applied from that alert and the alert removed immediately.",
"Proxmox guest drawer with Monitoring and Lifecycle controls showing Expected offline and provider-owned retirement guidance.",
"Retired lifecycle selected with monitoring disabled and the retained-inventory, history, attention, and automation notice visible.",
"Mobile Alerts overview with the policy menu open, all four choices reachable, and main content clientWidth and scrollWidth both 362 pixels."
],
"interactions": [
"Ran a scratch mock-mode instance built from this working tree (backend + built dist) on port 7671 with the dev admin bypass.",
"Opened /patrol at 1280px, read the attention list rows, clicked the vaultwarden memory-limit row, and inspected the detail drawer badges and evidence observations.",
"Reloaded /patrol at 744px, re-checked the row metadata line, re-opened the detail drawer, and screenshotted the drawer showing 'Evidence recorded'."
"Opened the Monitoring menu from a live powered-off alert and confirmed Proxmox remains the inventory owner.",
"Applied Expected offline from the alert, observed the success notification, and confirmed the alert disappeared immediately while the active count decreased.",
"Opened dev-portal-01 in the Proxmox guest drawer, changed Monitoring between Normal and Expected offline, and saved through the canonical resource policy.",
"Changed Lifecycle to Retired, confirmed the retirement notice and disabled monitoring control, then restored Active while preserving Expected offline.",
"Repeated the Alerts policy-menu inspection at 390 by 844 pixels and confirmed there was no horizontal content overflow."
]
}
@@ -72,6 +72,8 @@ describe('resourceOperatorState api', () => {
it('PUTs the canonical body shape and returns the read-after-write record', async () => {
apiFetchJSONMock.mockResolvedValueOnce({
canonicalId: 'vm:101',
monitoringMode: 'expected_offline',
lifecycleState: 'active',
intentionallyOffline: true,
neverAutoRemediate: false,
setAt: '2026-05-09T11:00:00Z',
@@ -79,6 +81,8 @@ describe('resourceOperatorState api', () => {
} satisfies ResourceOperatorState);
const input: ResourceOperatorStateInput = {
monitoringMode: 'expected_offline',
lifecycleState: 'active',
intentionallyOffline: true,
neverAutoRemediate: false,
autoRemediationPolicy: {
@@ -100,7 +104,11 @@ describe('resourceOperatorState api', () => {
// (setAt, setBy) — never echo the input verbatim.
expect(result.setAt).toBe('2026-05-09T11:00:00Z');
expect(result.setBy).toBe('operator:richard');
expect(result.monitoringMode).toBe('expected_offline');
expect(result.lifecycleState).toBe('active');
expect(JSON.parse(apiFetchJSONMock.mock.calls[0][1]?.body as string)).toMatchObject({
monitoringMode: 'expected_offline',
lifecycleState: 'active',
autoRemediationPolicy: {
enabled: true,
capabilityNames: ['restart'],
@@ -8,6 +8,8 @@ import { apiFetchJSON } from '@/utils/apiClient';
* for the suppression / refusal semantics each field drives.
*/
export type ResourceCriticality = 'high' | 'medium' | 'low' | '';
export type ResourceMonitoringMode = 'normal' | 'expected_offline' | 'muted';
export type ResourceLifecycleState = 'active' | 'retired';
export interface AutoRemediationWindow {
timezone: string;
@@ -28,6 +30,8 @@ export interface AutoRemediationPolicy {
export interface ResourceOperatorState {
canonicalId: string;
monitoringMode?: ResourceMonitoringMode;
lifecycleState?: ResourceLifecycleState;
/**
* When true, new findings raised against this resource get
* auto-acknowledged with reason=expected_behavior Patrol stops
@@ -74,6 +78,13 @@ export type ResourceOperatorStateInput = Omit<
'canonicalId' | 'setAt' | 'setBy'
>;
const normalizeResourceOperatorState = (state: ResourceOperatorState): ResourceOperatorState => ({
...state,
monitoringMode:
state.monitoringMode || (state.intentionallyOffline ? 'expected_offline' : 'normal'),
lifecycleState: state.lifecycleState || 'active',
});
/**
* Read the operator-set state for a resource. Resolves to null when
* the server returns 404 (no entry recorded the default no-state
@@ -83,10 +94,11 @@ export async function getResourceOperatorState(
resourceId: string,
): Promise<ResourceOperatorState | null> {
try {
return await apiFetchJSON<ResourceOperatorState>(
const state = await apiFetchJSON<ResourceOperatorState>(
`/api/resources/${encodeURIComponent(resourceId)}/operator-state`,
{ cache: 'no-store' },
);
return normalizeResourceOperatorState(state);
} catch (err) {
// The 404 response shape is `{ error: 'operator_state_not_set', ... }`.
// Translating into null lets the caller treat "no state" as a clean
@@ -113,7 +125,7 @@ export async function setResourceOperatorState(
resourceId: string,
state: ResourceOperatorStateInput,
): Promise<ResourceOperatorState> {
return apiFetchJSON<ResourceOperatorState>(
const persisted = await apiFetchJSON<ResourceOperatorState>(
`/api/resources/${encodeURIComponent(resourceId)}/operator-state`,
{
method: 'PUT',
@@ -121,6 +133,7 @@ export async function setResourceOperatorState(
headers: { 'Content-Type': 'application/json' },
},
);
return normalizeResourceOperatorState(persisted);
}
/**
@@ -712,6 +712,8 @@ export const ResourceDetailDrawerOverviewTab: Component<ResourceDetailDrawerOver
<Show when={resource.id}>
<ResourceOperatorStateSection
resourceId={resource.id}
resourceType={resource.type}
platformType={resource.platformType}
capabilities={resource.capabilities}
/>
</Show>
@@ -3,6 +3,8 @@ import { Toggle } from '@/components/shared/Toggle';
import { notificationStore } from '@/stores/notifications';
import {
type ResourceCriticality,
type ResourceLifecycleState,
type ResourceMonitoringMode,
type ResourceOperatorState,
type ResourceOperatorStateInput,
clearResourceOperatorState,
@@ -12,6 +14,7 @@ import {
import { createNonSuspendingQuery } from '@/hooks/createNonSuspendingQuery';
import { formatRelativeTime } from '@/utils/format';
import type { ResourceCapability } from '@/types/resource';
import { describeResourceInventoryOwnership } from '@/utils/resourceMonitoringPolicy';
/**
* ResourceOperatorStateSection surfaces the operator-set per-resource
@@ -31,6 +34,8 @@ import type { ResourceCapability } from '@/types/resource';
*/
interface ResourceOperatorStateSectionProps {
resourceId: string;
resourceType?: string;
platformType?: string;
capabilities?: ResourceCapability[];
}
@@ -61,7 +66,8 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
// Local edit state, hydrated from the persisted record. Operators can
// toggle either flag and the section dirty-tracks until they hit Save
// or Discard.
const [intentionallyOffline, setIntentionallyOffline] = createSignal(false);
const [monitoringMode, setMonitoringMode] = createSignal<ResourceMonitoringMode>('normal');
const [lifecycleState, setLifecycleState] = createSignal<ResourceLifecycleState>('active');
const [neverAutoRemediate, setNeverAutoRemediate] = createSignal(false);
const [criticality, setCriticality] = createSignal<ResourceCriticality>('');
const [note, setNote] = createSignal('');
@@ -79,6 +85,9 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
(capability) => capability.autoAuthorization && capability.autoAuthorization !== 'never',
),
);
const inventoryOwnership = createMemo(() =>
describeResourceInventoryOwnership(props.resourceType, props.platformType),
);
const minuteToTime = (minute: number): string => {
const normalized = Math.max(0, Math.min(1439, minute));
@@ -112,7 +121,10 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
createEffect(() => {
const current = persisted();
if (current === undefined) return;
setIntentionallyOffline(current?.intentionallyOffline ?? false);
setMonitoringMode(
current?.monitoringMode ?? (current?.intentionallyOffline ? 'expected_offline' : 'normal'),
);
setLifecycleState(current?.lifecycleState ?? 'active');
setNeverAutoRemediate(current?.neverAutoRemediate ?? false);
setCriticality(current?.criticality ?? '');
setNote(current?.note ?? '');
@@ -130,7 +142,9 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
const isDirty = createMemo(() => {
const current = persisted();
const persistedOffline = current?.intentionallyOffline ?? false;
const persistedMonitoringMode =
current?.monitoringMode ?? (current?.intentionallyOffline ? 'expected_offline' : 'normal');
const persistedLifecycleState = current?.lifecycleState ?? 'active';
const persistedLocked = current?.neverAutoRemediate ?? false;
const persistedCriticality = current?.criticality ?? '';
const persistedNote = current?.note ?? '';
@@ -149,7 +163,8 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
persistedAuto.window.endMinute !== timeToMinute(autoWindowEnd())
: Boolean(persistedAuto.window);
return (
intentionallyOffline() !== persistedOffline ||
monitoringMode() !== persistedMonitoringMode ||
lifecycleState() !== persistedLifecycleState ||
neverAutoRemediate() !== persistedLocked ||
criticality() !== persistedCriticality ||
note().trim() !== persistedNote ||
@@ -186,7 +201,9 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
try {
const current = persisted();
const input: ResourceOperatorStateInput = {
intentionallyOffline: intentionallyOffline(),
monitoringMode: monitoringMode(),
lifecycleState: lifecycleState(),
intentionallyOffline: monitoringMode() === 'expected_offline',
neverAutoRemediate: neverAutoRemediate(),
autoRemediationPolicy: {
enabled: autoRemediationEnabled(),
@@ -226,7 +243,10 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
const handleDiscard = () => {
const current = persisted();
setIntentionallyOffline(current?.intentionallyOffline ?? false);
setMonitoringMode(
current?.monitoringMode ?? (current?.intentionallyOffline ? 'expected_offline' : 'normal'),
);
setLifecycleState(current?.lifecycleState ?? 'active');
setNeverAutoRemediate(current?.neverAutoRemediate ?? false);
setCriticality(current?.criticality ?? '');
setNote(current?.note ?? '');
@@ -250,7 +270,8 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
// resolve to null and the section will return to the no-state
// posture.
await query.refetch();
setIntentionallyOffline(false);
setMonitoringMode('normal');
setLifecycleState('active');
setNeverAutoRemediate(false);
setCriticality('');
setNote('');
@@ -378,7 +399,9 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
const input: ResourceOperatorStateInput = {
// Keep the toggle state intact when scheduling — operator
// editing one facet must not lose work on the other.
intentionallyOffline: intentionallyOffline(),
monitoringMode: monitoringMode(),
lifecycleState: lifecycleState(),
intentionallyOffline: monitoringMode() === 'expected_offline',
neverAutoRemediate: neverAutoRemediate(),
autoRemediationPolicy: {
enabled: autoRemediationEnabled(),
@@ -416,7 +439,9 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
setSaving(true);
try {
const input: ResourceOperatorStateInput = {
intentionallyOffline: intentionallyOffline(),
monitoringMode: monitoringMode(),
lifecycleState: lifecycleState(),
intentionallyOffline: monitoringMode() === 'expected_offline',
neverAutoRemediate: neverAutoRemediate(),
autoRemediationPolicy: {
enabled: autoRemediationEnabled(),
@@ -459,8 +484,8 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
<div>
<h3 class="text-sm font-semibold text-base-content">Operator overrides</h3>
<p class="text-xs text-muted">
Tell Pulse how Patrol should treat this resource suppress expected noise, prioritize
its findings, or lock it against automated remediation.
Control Alerts, Patrol, lifecycle, priority, and automated remediation from one
persisted resource policy.
</p>
</div>
<Show when={persisted()?.setBy || persisted()?.setAt}>
@@ -781,21 +806,54 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
</div>
</Show>
<div class="flex items-start justify-between gap-3">
<div class="min-w-0 flex-1">
<label class="text-sm font-medium text-base-content">Intentionally offline</label>
<p class="text-[11px] text-muted mt-0.5 leading-tight">
Suppress findings on this resource. Use when a workload is deprecated, a dev environment
is shut down on purpose, or a host is archived.
</p>
</div>
<Toggle
checked={intentionallyOffline()}
onChange={(e) => setIntentionallyOffline(e.currentTarget.checked)}
disabled={saving()}
/>
<div class="grid grid-cols-1 gap-3 border-t border-border-subtle pt-3 sm:grid-cols-2">
<label class="block">
<span class="block text-sm font-medium text-base-content">Monitoring</span>
<select
value={monitoringMode()}
onChange={(event) =>
setMonitoringMode(event.currentTarget.value as ResourceMonitoringMode)
}
disabled={saving() || lifecycleState() === 'retired'}
class="mt-1 block min-h-11 w-full rounded border border-border bg-surface px-2 py-1.5 text-xs text-base-content focus:outline-none focus:ring-1 focus:ring-blue-400 disabled:opacity-50 sm:min-h-0"
>
<option value="normal">Normal monitoring</option>
<option value="expected_offline">Expected offline</option>
<option value="muted">Mute all attention</option>
</select>
<span class="mt-1 block text-[11px] leading-tight text-muted">
Expected offline hides availability noise only. Mute all stops Alerts and Patrol while
keeping this resource visible.
</span>
</label>
<label class="block">
<span class="block text-sm font-medium text-base-content">Lifecycle</span>
<select
value={lifecycleState()}
onChange={(event) =>
setLifecycleState(event.currentTarget.value as ResourceLifecycleState)
}
disabled={saving()}
class="mt-1 block min-h-11 w-full rounded border border-border bg-surface px-2 py-1.5 text-xs text-base-content focus:outline-none focus:ring-1 focus:ring-blue-400 disabled:opacity-50 sm:min-h-0"
>
<option value="active">Active</option>
<option value="retired">Retired from monitoring</option>
</select>
<span class="mt-1 block text-[11px] leading-tight text-muted">
{inventoryOwnership().retirementDescription}
</span>
</label>
</div>
<Show when={lifecycleState() === 'retired'}>
<div class="rounded border border-border bg-surface-alt px-3 py-2 text-xs text-base-content">
Retired resources remain in {inventoryOwnership().ownerLabel} inventory and keep their
Pulse history. All alert attention and automated remediation are disabled until the
lifecycle returns to Active.
</div>
</Show>
<div class="flex items-start justify-between gap-3 pt-2 border-t border-border-subtle">
<div class="min-w-0 flex-1">
<label class="text-sm font-medium text-red-700 dark:text-red-400">
@@ -810,7 +868,7 @@ export const ResourceOperatorStateSection: Component<ResourceOperatorStateSectio
<Toggle
checked={neverAutoRemediate()}
onChange={(e) => handleNeverAutoRemediateToggle(e.currentTarget.checked)}
disabled={saving()}
disabled={saving() || lifecycleState() === 'retired'}
/>
</div>
@@ -58,7 +58,7 @@ describe('ResourceOperatorStateSection render with capabilityNames: null', () =>
// The section itself must be up (pre-fix, the throw escaped render
// and nothing below the header survived).
expect(screen.getByText('Operator overrides')).toBeTruthy();
expect(screen.getByText('Intentionally offline')).toBeTruthy();
expect(screen.getByText('Monitoring')).toBeTruthy();
expect(screen.getByText('Never auto-remediate')).toBeTruthy();
// "Clear all overrides" only renders for persisted() && !isDirty() —
@@ -17,6 +17,11 @@ const supportDisclosureSource = readFileSync(
'utf-8',
);
const guestOverviewSource = readFileSync(
resolve(__dirname, '..', '..', 'Workloads', 'GuestDrawerOverview.tsx'),
'utf-8',
);
describe('ResourceOperatorStateSection', () => {
it('exposes operator-set controls bound to the canonical API client', () => {
// The section is the operator's window into the per-resource state
@@ -28,7 +33,10 @@ describe('ResourceOperatorStateSection', () => {
expect(sectionSource).toContain('getResourceOperatorState');
expect(sectionSource).toContain('setResourceOperatorState');
expect(sectionSource).toContain('clearResourceOperatorState');
expect(sectionSource).toContain('Intentionally offline');
expect(sectionSource).toContain('Normal monitoring');
expect(sectionSource).toContain('Expected offline');
expect(sectionSource).toContain('Mute all attention');
expect(sectionSource).toContain('Retired from monitoring');
expect(sectionSource).toContain('Never auto-remediate');
expect(sectionSource).toContain('Patrol priority');
expect(sectionSource).toContain('Operator note');
@@ -159,7 +167,11 @@ describe('ResourceOperatorStateSection', () => {
expect(sectionSource).toContain('Cancel window');
// handleClearMaintenanceWindow must preserve toggles by reading
// the current edit-state signals rather than nulling everything.
expect(sectionSource).toContain('intentionallyOffline: intentionallyOffline()');
expect(sectionSource).toContain('monitoringMode: monitoringMode()');
expect(sectionSource).toContain('lifecycleState: lifecycleState()');
expect(sectionSource).toContain(
"intentionallyOffline: monitoringMode() === 'expected_offline'",
);
expect(sectionSource).toContain('neverAutoRemediate: neverAutoRemediate()');
expect(sectionSource).toContain('maintenanceStartAt: undefined,');
expect(sectionSource).toContain('maintenanceEndAt: undefined,');
@@ -225,3 +237,14 @@ describe('ResourceDetailDrawerOverviewTab integration', () => {
expect(sectionSource).toContain('<Show when={eligibleAutoCapabilities().length > 0}>');
});
});
describe('Proxmox guest drawer integration', () => {
it('exposes the same canonical policy with provider ownership context', () => {
expect(guestOverviewSource).toContain(
"from '@/components/Infrastructure/ResourceOperatorStateSection'",
);
expect(guestOverviewSource).toContain('<ResourceOperatorStateSection');
expect(guestOverviewSource).toContain('resourceId={props.guestId}');
expect(guestOverviewSource).toContain('platformType="proxmox"');
});
});
@@ -8,6 +8,7 @@ import { aiChatStore } from '@/stores/aiChat';
import { getCanonicalWorkloadId, getWorkloadMetadataId } from '@/utils/workloads';
import { getDiscoveryProvenanceTitle } from '@/utils/discoveryPresentation';
import guestDrawerSource from './GuestDrawer.tsx?raw';
import guestDrawerOverviewSource from './GuestDrawerOverview.tsx?raw';
// ── Mocks ──────────────────────────────────────────────────────────────
@@ -207,6 +208,12 @@ describe('GuestDrawer', () => {
);
});
it('mounts canonical resource policy only in the selected guest overview', () => {
expect(guestDrawerOverviewSource).toContain('<ResourceOperatorStateSection');
expect(guestDrawerOverviewSource).toContain('resourceId={props.guestId}');
expect(guestDrawerOverviewSource).toContain('platformType="proxmox"');
});
describe('Assistant context actions', () => {
it('opens Pulse Assistant with a canonical Proxmox LXC resource handoff', () => {
aiChatStore.setEnabled(true);
@@ -8,6 +8,7 @@ import { DiscoveryProvenanceMarker } from '@/components/shared/DiscoveryProvenan
import { InfoCardFrame } from '@/components/shared/InfoCardFrame';
import { WebInterfaceUrlField } from '@/components/shared/WebInterfaceUrlField';
import { AvailabilityProbeStatusCards } from '@/components/Infrastructure/AvailabilityProbeStatusCard';
import { ResourceOperatorStateSection } from '@/components/Infrastructure/ResourceOperatorStateSection';
import type { DiscoveryIdentifiedSummary } from '@/utils/discoveryPresentation';
import { formatBytes, formatUptime } from '@/utils/format';
import type { MetricDisplayThresholds } from '@/utils/metricThresholds';
@@ -463,7 +464,12 @@ export function GuestDrawerOverview(props: GuestDrawerOverviewProps) {
</Show>
</div>
<div class="mt-3">
<div class="mt-3 space-y-3">
<ResourceOperatorStateSection
resourceId={props.guestId}
resourceType={isGuestDrawerVM(props.guest) ? 'vm' : 'system-container'}
platformType="proxmox"
/>
<WebInterfaceUrlField
metadataKind="guest"
metadataId={props.webInterfaceMetadataId}
@@ -21,6 +21,7 @@ import { alertTypeDisplayLabel } from './helpers';
import { getCanonicalAlertId } from './identity';
import type { AlertIncidentTimelineState } from './useAlertIncidentTimelineState';
import type { AlertOverviewState } from './useAlertOverviewState';
import { ResourceMonitoringPolicyAction } from './ResourceMonitoringPolicyAction';
interface AlertOverviewAlertCardProps {
alert: Alert;
@@ -52,9 +53,34 @@ export function AlertOverviewAlertCard(props: AlertOverviewAlertCardProps) {
return '/docker/overview';
if (resourceType === 'kubernetes' || resourceType.startsWith('k8s-'))
return '/kubernetes/overview';
if (resourceType.startsWith('vmware-') || props.alert.message?.toLowerCase().includes('vmware'))
return '/vmware/overview';
return '/proxmox/overview';
};
const platformTypeForPolicy = (): string | undefined => {
const metadataPlatform = props.alert.metadata?.platformType;
if (typeof metadataPlatform === 'string' && metadataPlatform.trim()) return metadataPlatform;
const link = resourceLink();
if (link.startsWith('/proxmox')) return 'proxmox';
if (link.startsWith('/docker')) return 'docker';
if (link.startsWith('/kubernetes')) return 'kubernetes';
if (link.startsWith('/vmware')) return 'vmware';
if (link.startsWith('/machines')) return 'agent';
return undefined;
};
const resourceTypeForPolicy = (): string | undefined => {
const metadataType = props.alert.metadata?.resourceType;
if (typeof metadataType === 'string' && metadataType.trim()) return metadataType;
const resourceId = (props.alert.resourceId || '').toLowerCase();
if (resourceId.startsWith('agent:')) return 'agent';
if (resourceId.includes('docker')) return 'app-container';
if (resourceId.includes('k8s') || resourceId.includes('kubernetes')) return 'pod';
if (resourceLink() === '/proxmox/overview') return 'vm';
return undefined;
};
return (
<div id={`alert-${alertKey()}`} class={alertCardPresentation().cardClassName}>
<div class="flex flex-col sm:flex-row sm:items-start">
@@ -155,6 +181,14 @@ export function AlertOverviewAlertCard(props: AlertOverviewAlertCardProps) {
props.timelineState.expandedIncidents().has(alertKey()),
)}
</button>
<Show when={(props.alert.resourceId || '').trim()}>
<ResourceMonitoringPolicyAction
resourceId={props.alert.resourceId}
resourceName={props.alert.resourceName || props.alert.resourceId}
resourceType={resourceTypeForPolicy()}
platformType={platformTypeForPolicy()}
/>
</Show>
<InvestigateAlertButton
alert={props.alert}
resourceType={
@@ -0,0 +1,122 @@
import { For, createSignal } from 'solid-js';
import {
type ResourceLifecycleState,
type ResourceMonitoringMode,
type ResourceOperatorStateInput,
getResourceOperatorState,
setResourceOperatorState,
} from '@/api/resourceOperatorState';
import { notificationStore } from '@/stores/notifications';
import { describeResourceInventoryOwnership } from '@/utils/resourceMonitoringPolicy';
interface ResourceMonitoringPolicyActionProps {
resourceId: string;
resourceName: string;
resourceType?: string;
platformType?: string;
}
interface PolicyChoice {
label: string;
description: string;
monitoringMode?: ResourceMonitoringMode;
lifecycleState?: ResourceLifecycleState;
}
export function ResourceMonitoringPolicyAction(props: ResourceMonitoringPolicyActionProps) {
const [saving, setSaving] = createSignal(false);
let detailsRef: HTMLDetailsElement | undefined;
const ownership = () =>
describeResourceInventoryOwnership(props.resourceType, props.platformType);
const choices = (): PolicyChoice[] => [
{
label: 'Normal monitoring',
description: 'Resume all Alerts and Patrol attention.',
monitoringMode: 'normal',
lifecycleState: 'active',
},
{
label: 'Expected offline',
description: 'Hide availability noise but keep other monitoring active.',
monitoringMode: 'expected_offline',
lifecycleState: 'active',
},
{
label: 'Mute all attention',
description: 'Stop Alerts and Patrol while keeping the resource visible.',
monitoringMode: 'muted',
lifecycleState: 'active',
},
{
label: 'Retire from monitoring',
description: `${ownership().ownerLabel} keeps the inventory record. Pulse stops attention and automation.`,
lifecycleState: 'retired',
},
];
const applyChoice = async (choice: PolicyChoice) => {
if (saving()) return;
setSaving(true);
try {
const current = await getResourceOperatorState(props.resourceId);
const monitoringMode =
choice.monitoringMode ??
current?.monitoringMode ??
(current?.intentionallyOffline ? 'expected_offline' : 'normal');
const lifecycleState = choice.lifecycleState ?? current?.lifecycleState ?? 'active';
const input: ResourceOperatorStateInput = {
monitoringMode,
lifecycleState,
intentionallyOffline: monitoringMode === 'expected_offline',
neverAutoRemediate: current?.neverAutoRemediate ?? false,
autoRemediationPolicy: current?.autoRemediationPolicy ?? {
enabled: false,
capabilityNames: [],
},
maintenanceStartAt: current?.maintenanceStartAt,
maintenanceEndAt: current?.maintenanceEndAt,
maintenanceReason: current?.maintenanceReason,
criticality: current?.criticality ?? '',
note: current?.note,
};
await setResourceOperatorState(props.resourceId, input);
if (detailsRef) detailsRef.open = false;
notificationStore.success(`${choice.label} saved for ${props.resourceName}`);
} catch (error) {
notificationStore.error(
error instanceof Error ? error.message : 'Failed to update resource monitoring',
);
} finally {
setSaving(false);
}
};
return (
<details ref={detailsRef} class="relative" onClick={(event) => event.stopPropagation()}>
<summary class="inline-flex min-h-9 cursor-pointer list-none items-center rounded-md border border-border bg-surface px-2.5 py-1 text-xs font-medium text-base-content transition-colors hover:bg-surface-hover">
{saving() ? 'Saving…' : 'Monitoring'}
</summary>
<div class="absolute right-0 z-30 mt-1 w-72 rounded-md border border-border bg-surface p-1.5 shadow-lg">
<p class="px-2 pb-1.5 text-[11px] text-muted">
This changes Pulse policy only. {ownership().ownerLabel} remains the inventory owner.
</p>
<For each={choices()}>
{(choice) => (
<button
type="button"
disabled={saving()}
class="block w-full rounded px-2 py-2 text-left hover:bg-surface-hover disabled:opacity-50"
onClick={() => void applyChoice(choice)}
>
<span class="block text-xs font-medium text-base-content">{choice.label}</span>
<span class="mt-0.5 block text-[11px] leading-tight text-muted">
{choice.description}
</span>
</button>
)}
</For>
</div>
</details>
);
}
@@ -84,6 +84,7 @@ describe('OverviewTab Last 24 Hours stat', () => {
.closest('tr')
?.querySelector('[data-testid="alert-overview-stat-value"]');
expect(statValue?.textContent).toBe('1');
expect(screen.getAllByText('Monitoring')).toHaveLength(2);
});
it('shows 0 when all alerts are older than 24 hours', () => {
@@ -0,0 +1,89 @@
import { cleanup, fireEvent, render, screen, waitFor } from '@solidjs/testing-library';
import { afterEach, describe, expect, it, vi } from 'vitest';
const operatorStateMock = vi.hoisted(() => ({
get: vi.fn(),
set: vi.fn(),
}));
vi.mock('@/api/resourceOperatorState', async (importOriginal) => {
const actual = await importOriginal<typeof import('@/api/resourceOperatorState')>();
return {
...actual,
getResourceOperatorState: operatorStateMock.get,
setResourceOperatorState: operatorStateMock.set,
};
});
vi.mock('@/stores/notifications', () => ({
notificationStore: { success: vi.fn(), error: vi.fn() },
}));
import { ResourceMonitoringPolicyAction } from '../ResourceMonitoringPolicyAction';
describe('ResourceMonitoringPolicyAction', () => {
afterEach(() => {
cleanup();
vi.clearAllMocks();
});
it('persists expected-offline policy through the canonical resource state API', async () => {
operatorStateMock.get.mockResolvedValue(null);
operatorStateMock.set.mockResolvedValue({});
render(() => (
<ResourceMonitoringPolicyAction
resourceId="vm:101"
resourceName="legacy-lxc"
resourceType="system-container"
/>
));
fireEvent.click(screen.getByText('Monitoring'));
expect(screen.getByText(/Proxmox remains the inventory owner/)).toBeInTheDocument();
fireEvent.click(screen.getByText('Expected offline'));
await waitFor(() => {
expect(operatorStateMock.set).toHaveBeenCalledWith(
'vm:101',
expect.objectContaining({
monitoringMode: 'expected_offline',
lifecycleState: 'active',
intentionallyOffline: true,
}),
);
});
});
it('retires provider inventory without changing its previous monitoring mode', async () => {
operatorStateMock.get.mockResolvedValue({
canonicalId: 'vm:101',
monitoringMode: 'expected_offline',
lifecycleState: 'active',
intentionallyOffline: true,
neverAutoRemediate: false,
autoRemediationPolicy: { enabled: false, capabilityNames: [] },
setAt: '2026-08-11T09:00:00Z',
});
operatorStateMock.set.mockResolvedValue({});
render(() => (
<ResourceMonitoringPolicyAction
resourceId="vm:101"
resourceName="legacy-lxc"
resourceType="system-container"
/>
));
fireEvent.click(screen.getByText('Monitoring'));
fireEvent.click(screen.getByText('Retire from monitoring'));
await waitFor(() => {
expect(operatorStateMock.set).toHaveBeenCalledWith(
'vm:101',
expect.objectContaining({
monitoringMode: 'expected_offline',
lifecycleState: 'retired',
}),
);
});
});
});
@@ -0,0 +1,63 @@
export interface ResourceInventoryOwnership {
ownerLabel: string;
providerOwned: boolean;
retirementDescription: string;
}
export function describeResourceInventoryOwnership(
resourceType?: string,
platformType?: string,
): ResourceInventoryOwnership {
const type = (resourceType || '').toLowerCase();
const platform = (platformType || '').toLowerCase();
if (
platform === 'proxmox' ||
['vm', 'system-container', 'oci-container', 'pbs', 'pmg'].includes(type)
) {
return {
ownerLabel: 'Proxmox',
providerOwned: true,
retirementDescription:
'Proxmox owns this inventory record. Retiring it stops Pulse attention and automation without deleting it from Proxmox. Restore active monitoring here at any time.',
};
}
if (platform === 'kubernetes' || type.startsWith('k8s-') || type === 'pod') {
return {
ownerLabel: 'Kubernetes',
providerOwned: true,
retirementDescription:
'Kubernetes owns this object. Retiring it stops Pulse attention and automation without deleting the object from the cluster.',
};
}
if (platform === 'vmware') {
return {
ownerLabel: 'VMware vSphere',
providerOwned: true,
retirementDescription:
'vSphere owns this inventory record. Retiring it changes Pulse monitoring only and does not delete anything from vCenter.',
};
}
if (platform === 'docker' || type.startsWith('docker-') || type === 'app-container') {
return {
ownerLabel: 'container runtime',
providerOwned: true,
retirementDescription:
'The container runtime owns this inventory record. Retiring it changes Pulse monitoring only and does not remove the runtime object.',
};
}
if (type === 'agent' || platform === 'agent') {
return {
ownerLabel: 'Pulse agent',
providerOwned: false,
retirementDescription:
'Retiring this machine stops Pulse attention and automation while keeping its history. Agent removal remains available from Machines.',
};
}
return {
ownerLabel: 'source system',
providerOwned: true,
retirementDescription:
'The source system owns this inventory record. Retiring it changes Pulse monitoring only and preserves the resource history.',
};
}
+2 -2
View File
@@ -1640,7 +1640,7 @@ func boundedActionAvailabilityText(value string, maxRunes int) string {
}
// validateExecutionPolicy enforces operator-set per-resource policy at the
// dispatch decision point, currently the NeverAutoRemediate lock.
// dispatch decision point, including explicit locks and retired lifecycle.
func validateExecutionPolicy(store Store, record unified.ActionAuditRecord) error {
if store == nil {
return errors.New("action audit store unavailable")
@@ -1653,7 +1653,7 @@ func validateExecutionPolicy(store Store, record unified.ActionAuditRecord) erro
if err != nil || !found {
return err
}
if state.NeverAutoRemediate {
if state.BlocksRemediation() {
return unified.ErrResourceRemediationLocked
}
return nil
+27
View File
@@ -1414,6 +1414,33 @@ func TestExecuteRefusesRemediationLockedResource(t *testing.T) {
}
}
func TestExecuteRefusesRetiredResourceWithoutDispatch(t *testing.T) {
now := time.Now().UTC()
env := newServiceEnv(t, testResource(now, unified.ApprovalNone))
plan, err := env.service.Plan(context.Background(), "default", restartRequest(), testActionActor("requester", "default"))
if err != nil {
t.Fatalf("Plan: %v", err)
}
if err := env.store.SetResourceOperatorState(unified.ResourceOperatorState{
CanonicalID: "vm:42",
LifecycleState: unified.LifecycleStateRetired,
}); err != nil {
t.Fatalf("SetResourceOperatorState: %v", err)
}
failed, err := env.service.Execute(context.Background(), "default", plan.ActionID, testActionActor("agent:test", "default"), "")
if !errors.Is(err, unified.ErrResourceRemediationLocked) {
t.Fatalf("error = %v, want ErrResourceRemediationLocked", err)
}
if env.executor.calls != 0 {
t.Fatalf("executor must not run for retired resources, calls = %d", env.executor.calls)
}
if failed.State != unified.ActionStateFailed {
t.Fatalf("retirement refusal must persist a failed audit, state = %q", failed.State)
}
}
func TestExecuteRefusesDriftedPlan(t *testing.T) {
now := time.Now().UTC()
env := newServiceEnv(t, testResource(now, unified.ApprovalNone))
+18 -2
View File
@@ -115,7 +115,15 @@ func resourceCapabilitiesOutputSchema() json.RawMessage {
func operatorStateOutputSchema() json.RawMessage {
return agentObjectOutputSchema([]string{"canonicalId", "intentionallyOffline", "neverAutoRemediate", "setAt"}, map[string]any{
"canonicalId": stringOption("Canonical Pulse resource id."),
"canonicalId": stringOption("Canonical Pulse resource id."),
"monitoringMode": map[string]any{
"type": "string", "enum": []string{"normal", "expected_offline", "muted"},
"description": "Canonical per-resource monitoring posture.",
},
"lifecycleState": map[string]any{
"type": "string", "enum": []string{"active", "retired"},
"description": "Whether the resource remains operationally active in Pulse.",
},
"intentionallyOffline": booleanOption("Whether this resource is expected to be offline."),
"neverAutoRemediate": booleanOption("Whether automated remediation must be refused for this resource."),
"maintenanceStartAt": dateTimeOption("Maintenance window start time when set."),
@@ -277,6 +285,14 @@ func updateNodeInputSchema() json.RawMessage {
func operatorStateInputSchema() json.RawMessage {
schema := agentObjectInputSchemaMap([]string{ResourceIDArgumentName, "intentionallyOffline", "neverAutoRemediate"}, map[string]any{
ResourceIDArgumentName: stringOption("Canonical resource id from get_fleet_context or get_resource_context. This path value wins over any body id."),
"monitoringMode": map[string]any{
"type": "string", "enum": []string{"normal", "expected_offline", "muted"},
"description": "Canonical monitoring posture. When supplied, this takes precedence over the intentionallyOffline compatibility field.",
},
"lifecycleState": map[string]any{
"type": "string", "enum": []string{"active", "retired"},
"description": "Retired resources stay in provider inventory but leave alert attention and automated remediation.",
},
"intentionallyOffline": map[string]any{
"type": "boolean",
"description": "Whether the resource is expected to be offline. This suppresses offline findings for this resource.",
@@ -917,7 +933,7 @@ var canonicalManifest = Manifest{
{
Name: GetOperatorStateCapabilityName,
Title: "Get operator state",
Description: "Read the operator-set state for a resource (intentionally offline, never auto-remediate, maintenance window, criticality).",
Description: "Read the operator-set monitoring mode, lifecycle state, remediation lock, maintenance window, and criticality for a resource.",
Category: "operator-state",
Method: http.MethodGet,
Path: OperatorStateCapabilityPath,
@@ -134,6 +134,29 @@ func TestCanonicalManifestUsesSharedOperatorStateVocabulary(t *testing.T) {
t.Fatalf("%s path = %q, want %q", name, cap.Path, OperatorStateCapabilityPath)
}
}
setCapability, _ := FindCapability(manifest.Capabilities, SetOperatorStateCapabilityName)
var inputSchema map[string]any
if err := json.Unmarshal(ToolInputSchema(setCapability), &inputSchema); err != nil {
t.Fatal(err)
}
inputProperties, _ := inputSchema["properties"].(map[string]any)
for _, field := range []string{"monitoringMode", "lifecycleState"} {
if _, ok := inputProperties[field]; !ok {
t.Fatalf("set_operator_state input schema missing %q: %v", field, inputProperties)
}
}
getCapability, _ := FindCapability(manifest.Capabilities, GetOperatorStateCapabilityName)
var outputSchema map[string]any
if err := json.Unmarshal(getCapability.OutputSchema, &outputSchema); err != nil {
t.Fatal(err)
}
outputProperties, _ := outputSchema["properties"].(map[string]any)
for _, field := range []string{"monitoringMode", "lifecycleState"} {
if _, ok := outputProperties[field]; !ok {
t.Fatalf("get_operator_state output schema missing %q: %v", field, outputProperties)
}
}
}
func TestCanonicalManifestUsesSharedFindingVocabulary(t *testing.T) {
@@ -144,6 +144,13 @@ func TestMCPSurfaceContractMarkdownFallsBackForLegacyManifest(t *testing.T) {
}
}
func TestMCPToolInventoryDescribesCanonicalResourcePolicy(t *testing.T) {
got := MCPToolCapabilityInventoryMarkdown(CanonicalManifest())
if !strings.Contains(got, "monitoring mode, lifecycle state, remediation lock") {
t.Fatalf("operator-state inventory omits canonical policy vocabulary:\n%s", got)
}
}
func TestMCPToolCapabilityInventoryMarkdownProjectsManifestTools(t *testing.T) {
got := MCPToolCapabilityInventoryMarkdown(Manifest{
SurfaceContract: CanonicalManifest().SurfaceContract,
@@ -48,6 +48,8 @@ type Section struct {
// stays independent of HTTP and storage owners.
type OperatorState struct {
IntentionallyOffline bool
MonitoringMode string
LifecycleState string
NeverAutoRemediate bool
MaintenanceWindowActive bool
MaintenanceStartAt *time.Time
@@ -638,9 +640,14 @@ func formatAgentRelationshipFact(rel unified.ResourceRelationship, policy *unifi
func formatOperatorStateFact(state OperatorState) string {
parts := []string{}
if state.IntentionallyOffline {
if state.MonitoringMode != "" {
parts = append(parts, "monitoring="+state.MonitoringMode)
} else if state.IntentionallyOffline {
parts = append(parts, "intentionally offline")
}
if state.LifecycleState != "" {
parts = append(parts, "lifecycle="+state.LifecycleState)
}
if state.NeverAutoRemediate {
parts = append(parts, "never auto-remediate")
}
@@ -85,7 +85,7 @@ func TestBuildResourceContextSectionsAppliesRedactionTrustAndFreshness(t *testin
sections := BuildResourceContextSections(resource, nil, BuildOptions{
GeneratedAt: now,
OperatorState: &OperatorState{IntentionallyOffline: true, NeverAutoRemediate: true, Criticality: "high", NotePresent: true, SetAt: operatorSetAt},
OperatorState: &OperatorState{MonitoringMode: "muted", LifecycleState: "retired", NeverAutoRemediate: true, Criticality: "high", NotePresent: true, SetAt: operatorSetAt},
ActiveFindingCount: 2,
PendingApprovalCount: 1,
RecentActionCount: 3,
@@ -173,7 +173,7 @@ func TestBuildResourceContextSectionsAppliesRedactionTrustAndFreshness(t *testin
if operatorState.ObservedAt == nil || !operatorState.ObservedAt.Equal(operatorSetAt) {
t.Fatalf("operator state observedAt = %v, want %v", operatorState.ObservedAt, operatorSetAt)
}
for _, want := range []string{"intentionally offline", "never auto-remediate", "criticality=high", "note present"} {
for _, want := range []string{"monitoring=muted", "lifecycle=retired", "never auto-remediate", "criticality=high", "note present"} {
if !strings.Contains(operatorState.Value, want) {
t.Fatalf("operator state value = %q, want %q", operatorState.Value, want)
}
+32 -11
View File
@@ -822,6 +822,8 @@ type ResourceOperatorStateMaintenanceWindow struct {
type ResourceOperatorStateProjection struct {
MaintenanceWindow *ResourceOperatorStateMaintenanceWindow
IntentionallyOffline bool
MonitoringMode string
LifecycleState string
// NeverAutoRemediate is the operator's "do not act on this
// resource" flag. The findings store does not consume it during
// suppression (that's the action broker's concern), but the
@@ -833,6 +835,26 @@ type ResourceOperatorStateProjection struct {
Criticality string
}
func (p ResourceOperatorStateProjection) monitoringSuppressionCause() string {
if strings.EqualFold(strings.TrimSpace(p.LifecycleState), "retired") {
return "resource_retired"
}
switch strings.ToLower(strings.TrimSpace(p.MonitoringMode)) {
case "muted":
return "monitoring_muted"
case "expected_offline":
return "expected_offline"
}
if p.IntentionallyOffline {
return "intentionally_offline"
}
return ""
}
func (p ResourceOperatorStateProjection) suppressesFindings() bool {
return p.MaintenanceWindow != nil || p.monitoringSuppressionCause() != ""
}
// ResourceOperatorStateProvider is the narrow interface the findings
// store consumes for per-resource operator intent. Implementations
// adapt the unified-resources `ResourceOperatorState` shape into the
@@ -1789,8 +1811,7 @@ func (s *FindingsStore) Add(f *Finding) bool {
if !willFixReminderDue && existing.DismissedReason == "expected_behavior" && !existing.Suppressed {
if cause := findOperatorStateDismissCause(existing); cause != "" {
if operatorStateProviderPresent {
currentlySuppressing := operatorProjectionOK &&
(operatorProjection.MaintenanceWindow != nil || operatorProjection.IntentionallyOffline)
currentlySuppressing := operatorProjectionOK && operatorProjection.suppressesFindings()
operatorStateLifted = !currentlySuppressing
} else {
// No provider currently wired but the finding was
@@ -1992,18 +2013,18 @@ func (s *FindingsStore) Add(f *Finding) bool {
"maintenance_end_at": window.EndAt.Format(time.RFC3339),
})
s.syncLoopStateLocked(f)
case projection.IntentionallyOffline:
// Operator has marked this resource as expected-to-be-offline,
// so any finding raised against it is by definition expected
// noise. Auto-dismiss with a clear note so the operator
// understands why future findings stayed quiet without
// expanding each row.
case projection.monitoringSuppressionCause() != "":
cause := projection.monitoringSuppressionCause()
f.DismissedReason = "expected_behavior"
f.AcknowledgedAt = &now
f.UserNote = "Resource is marked intentionally offline by the operator. Clear the flag on the resource detail surface to resume notifications."
s.appendLifecycleLocked(f, "dismissed", "Auto-acknowledged: resource is intentionally offline", f.LoopState, string(FindingLoopStateDismissed), map[string]string{
if cause == "intentionally_offline" {
f.UserNote = "Resource is marked intentionally offline by the operator. Clear the flag on the resource detail surface to resume notifications."
} else {
f.UserNote = "Resource monitoring policy suppresses Patrol attention. Return monitoring and lifecycle to active defaults on the resource detail surface to resume findings."
}
s.appendLifecycleLocked(f, "dismissed", "Auto-acknowledged: resource monitoring policy suppresses attention", f.LoopState, string(FindingLoopStateDismissed), map[string]string{
"reason": "expected_behavior",
"operator_state_cause": "intentionally_offline",
"operator_state_cause": cause,
})
s.syncLoopStateLocked(f)
}
+5
View File
@@ -2246,6 +2246,8 @@ func TestFindingsStore_OperatorStateProjectionFor_ProxiesProvider(t *testing.T)
store := NewFindingsStore()
expected := ResourceOperatorStateProjection{
IntentionallyOffline: true,
MonitoringMode: "muted",
LifecycleState: "retired",
NeverAutoRemediate: true,
Criticality: "high",
}
@@ -2262,6 +2264,9 @@ func TestFindingsStore_OperatorStateProjectionFor_ProxiesProvider(t *testing.T)
if got.IntentionallyOffline != expected.IntentionallyOffline {
t.Errorf("IntentionallyOffline must round-trip; got %v want %v", got.IntentionallyOffline, expected.IntentionallyOffline)
}
if got.MonitoringMode != expected.MonitoringMode || got.LifecycleState != expected.LifecycleState {
t.Errorf("canonical monitoring projection = (%q, %q), want (%q, %q)", got.MonitoringMode, got.LifecycleState, expected.MonitoringMode, expected.LifecycleState)
}
if got.NeverAutoRemediate != expected.NeverAutoRemediate {
t.Errorf("NeverAutoRemediate must round-trip — investigation runtime reads this projection too; got %v want %v", got.NeverAutoRemediate, expected.NeverAutoRemediate)
}
+1 -1
View File
@@ -1106,7 +1106,7 @@ func (e *PulseToolExecutor) isResourceRemediationLocked(resourceID string) (bool
if !found {
return false, nil
}
return state.NeverAutoRemediate, nil
return state.BlocksRemediation(), nil
}
// checkRemediationLockForDispatch applies the operator NeverAutoRemediate
+3
View File
@@ -105,6 +105,9 @@ func (m *Manager) setActiveAlertNoLock(storageKey string, alert *Alert) {
return
}
backfillCanonicalIdentity(alert)
if suppressed, _ := m.operatorSuppressionForAlertNoLock(alert, time.Now().UTC()); suppressed {
return
}
ensureOperationalContract(alert, time.Now())
requestedKey := storageKey
storageKey = activeAlertStorageKey(alert, storageKey)
+39 -12
View File
@@ -11,11 +11,34 @@ var ErrAlertIntentPolicyRevisionConflict = errors.New("alert_intent_policy_revis
type OperatorIntentContext struct {
IntentionallyOffline bool `json:"intentionallyOffline"`
MonitoringMode string `json:"monitoringMode"`
LifecycleState string `json:"lifecycleState"`
MaintenanceStartAt *time.Time `json:"maintenanceStartAt,omitempty"`
MaintenanceEndAt *time.Time `json:"maintenanceEndAt,omitempty"`
MaintenanceReason string `json:"maintenanceReason,omitempty"`
}
func (c OperatorIntentContext) suppressionForSignal(signal string) (bool, string) {
mode := strings.ToLower(strings.TrimSpace(c.MonitoringMode))
if mode == "" && c.IntentionallyOffline {
mode = "expected_offline"
}
lifecycle := strings.ToLower(strings.TrimSpace(c.LifecycleState))
if lifecycle == "retired" {
return true, "operator_retired"
}
if mode == "muted" {
return true, "operator_muted"
}
if mode == "expected_offline" {
switch strings.ToLower(strings.TrimSpace(signal)) {
case string(AlertIntentSignalOffline), string(AlertIntentSignalAvailability):
return true, "operator_expected_offline"
}
}
return false, ""
}
func (c OperatorIntentContext) MaintenanceActiveAt(now time.Time) bool {
return c.MaintenanceStartAt != nil && c.MaintenanceEndAt != nil &&
!now.Before(*c.MaintenanceStartAt) && now.Before(*c.MaintenanceEndAt)
@@ -174,7 +197,10 @@ func (m *Manager) resolveEffectiveIntentPolicyNoLock(resourceID, resourceType, s
resourceID = strings.TrimSpace(resourceID)
resourceType = strings.ToLower(strings.TrimSpace(resourceType))
signal = strings.ToLower(strings.TrimSpace(signal))
effective := EffectiveAlertIntentPolicy{Sources: make(map[string]string)}
effective := EffectiveAlertIntentPolicy{
HonorOperatorState: true,
Sources: make(map[string]string),
}
if strings.HasPrefix(signal, "metric.") {
metric := strings.TrimPrefix(signal, "metric.")
@@ -245,7 +271,7 @@ func (m *Manager) resolveEffectiveIntentPolicyNoLock(resourceID, resourceType, s
func (m *Manager) ResolveEffectiveIntentPolicy(resourceID, resourceType, signal string) EffectiveAlertIntentPolicy {
if m == nil {
return EffectiveAlertIntentPolicy{Sources: map[string]string{"graceSeconds": "factory", "honorOperatorState": "factory"}}
return EffectiveAlertIntentPolicy{HonorOperatorState: true, Sources: map[string]string{"graceSeconds": "factory", "honorOperatorState": "factory"}}
}
m.mu.RLock()
effective := m.resolveEffectiveIntentPolicyNoLock(resourceID, resourceType, signal)
@@ -309,13 +335,6 @@ func (m *Manager) evaluateIntentNoLock(resourceID, resourceType, signal, trackin
decision.Reason = "condition_clear"
return decision
}
if !effective.Explicit {
if m.clearIntentPendingNoLock(trackingKey) {
decision.StateChanged = true
}
decision.ShouldActivate = true
return decision
}
observedAt := m.policyNow().UTC()
tick := m.intentTickNoLock()
state, exists := m.intentPending[trackingKey]
@@ -357,15 +376,22 @@ func (m *Manager) evaluateIntentNoLock(resourceID, resourceType, signal, trackin
}
return decision
}
if signal == string(AlertIntentSignalOffline) && operator.IntentionallyOffline {
if suppressed, reason := operator.suppressionForSignal(signal); suppressed {
m.intentPending[trackingKey] = state
decision.Pending = true
decision.Suppressed = true
decision.Reason = "operator_intentionally_offline"
decision.Reason = reason
return decision
}
}
}
if !effective.Explicit {
if m.clearIntentPendingNoLock(trackingKey) {
decision.StateChanged = true
}
decision.ShouldActivate = true
return decision
}
eligibleElapsed := time.Duration(effective.GraceSeconds) * time.Second
if effective.BackupOffline != nil && effective.BackupOffline.Enabled && signal == string(AlertIntentSignalOffline) {
@@ -521,7 +547,8 @@ func (m *Manager) PreviewIntentPolicy(request AlertIntentPolicyPreviewRequest) (
preview.Contexts = append(preview.Contexts, ctx)
}
if operatorFound {
active := operator.IntentionallyOffline || operator.MaintenanceActiveAt(now)
suppressed, _ := operator.suppressionForSignal(request.Signal)
active := suppressed || operator.MaintenanceActiveAt(now)
ctx := AlertIntentPolicyPreviewContext{Kind: "operator_state", Active: active, Evidence: operator.MaintenanceReason}
if operator.MaintenanceEndAt != nil {
expiresAt := operator.MaintenanceEndAt.UTC()
+84
View File
@@ -194,6 +194,90 @@ func TestAlertIntentPreviewHonorsOperatorStateWithoutMutatingRuntime(t *testing.
}
}
func TestAlertIntentHonorsCanonicalResourcePolicyWithoutExplicitRules(t *testing.T) {
m := newTestManager(t)
m.SetOperatorIntentContextResolver(func(resourceID string, observedAt time.Time) (OperatorIntentContext, bool) {
return OperatorIntentContext{MonitoringMode: "expected_offline", LifecycleState: "active"}, true
})
preview, err := m.PreviewIntentPolicy(AlertIntentPolicyPreviewRequest{
ResourceID: "vm:101",
ResourceType: "vm",
Signal: string(AlertIntentSignalOffline),
ConditionActive: true,
})
if err != nil {
t.Fatalf("PreviewIntentPolicy() error = %v", err)
}
if preview.Status != "expected_transient" || preview.Reason != "operator_expected_offline" {
t.Fatalf("preview = %+v", preview)
}
if preview.Effective.Explicit {
t.Fatal("operator policy regression test must not rely on an explicit intent rule")
}
}
func TestCanonicalResourcePolicyGatesAllAlertWritersAndReconcilesExisting(t *testing.T) {
m := newTestManager(t)
mode := "normal"
m.SetOperatorIntentContextResolver(func(resourceID string, observedAt time.Time) (OperatorIntentContext, bool) {
return OperatorIntentContext{MonitoringMode: mode, LifecycleState: "active"}, true
})
existing := &Alert{ID: "backup-vm-101", ResourceID: "vm:101", Type: "backup-age"}
m.mu.Lock()
m.setActiveAlertNoLock(existing.ID, existing)
m.mu.Unlock()
if got := len(m.GetActiveAlerts()); got != 1 {
t.Fatalf("active alerts before mute = %d, want 1", got)
}
mode = "muted"
if cleared := m.ReconcileResourceOperatorState("vm:101"); cleared != 1 {
t.Fatalf("ReconcileResourceOperatorState() cleared = %d, want 1", cleared)
}
m.mu.Lock()
m.setActiveAlertNoLock(existing.ID, existing)
_, reactivated := m.getActiveAlertNoLock(existing.ID)
m.mu.Unlock()
if reactivated {
t.Fatal("muted resource alert was reactivated through a noncanonical writer")
}
}
func TestCanonicalResourcePolicyCannotEnterQuietHoursReplay(t *testing.T) {
m := newTestManager(t)
m.SetOperatorIntentContextResolver(func(resourceID string, observedAt time.Time) (OperatorIntentContext, bool) {
return OperatorIntentContext{MonitoringMode: "muted", LifecycleState: "active"}, true
})
alert := &Alert{
ID: "offline-vm-101",
ResourceID: "vm:101",
Type: "offline",
Level: AlertLevelWarning,
Metadata: map[string]interface{}{
MetadataQuietHoursSuppressed: true,
MetadataQuietHoursSuppressionReason: "non-critical",
MetadataQuietHoursReplayAt: time.Now().Add(time.Hour).UTC().Format(time.RFC3339),
},
}
if !m.ShouldSuppressNotification(alert) {
t.Fatal("muted resource firing notification must be dropped, not queued for replay")
}
if hasQuietHoursNotificationReplay(alert) {
t.Fatal("operator suppression must clear stale quiet-hours replay metadata")
}
markQuietHoursNotificationReplay(alert, "non-critical", time.Now().Add(time.Hour))
if !m.ShouldSuppressResolvedNotification(alert) {
t.Fatal("muted resource recovery notification must stay suppressed despite replay metadata")
}
if hasQuietHoursNotificationReplay(alert) {
t.Fatal("operator-suppressed recovery must not remain queued for quiet-hours replay")
}
}
func TestLifecycleAlertStartsAtFirstIntentMatch(t *testing.T) {
m := NewManagerWithDataDir(t.TempDir())
t.Cleanup(m.Stop)
+22
View File
@@ -408,6 +408,9 @@ func (m *Manager) shouldSuppressNotification(alert *Alert) (bool, string) {
if alert == nil {
return false, ""
}
if suppressed, reason := m.operatorSuppressionForAlertNoLock(alert, time.Now().UTC()); suppressed {
return true, reason
}
if !m.isInQuietHours() {
return false, ""
@@ -520,6 +523,16 @@ func (m *Manager) ShouldSuppressNotification(alert *Alert) bool {
m.mu.RLock()
defer m.mu.RUnlock()
if suppressed, reason := m.operatorSuppressionForAlertNoLock(alert, time.Now().UTC()); suppressed {
clearQuietHoursNotificationReplay(alert)
log.Debug().
Str("alertID", alert.ID).
Str("type", alert.Type).
Str("operatorPolicy", reason).
Msg("Notification suppressed by resource monitoring policy")
return true
}
suppressed, reason := m.shouldSuppressNotification(alert)
if suppressed {
replayAt := m.quietHoursReplayAt()
@@ -557,6 +570,15 @@ func (m *Manager) ShouldSuppressResolvedNotification(alert *Alert) bool {
Msg("Recovery notification suppressed for acknowledged alert")
return true
}
if suppressed, reason := m.operatorSuppressionForAlertNoLock(alert, time.Now().UTC()); suppressed {
clearQuietHoursNotificationReplay(alert)
log.Debug().
Str("alertID", alert.ID).
Str("type", alert.Type).
Str("operatorPolicy", reason).
Msg("Recovery notification suppressed by resource monitoring policy")
return true
}
quietHoursReplay := hasQuietHoursNotificationReplay(alert)
if alert.LastNotified == nil && !quietHoursReplay {
@@ -0,0 +1,86 @@
package alerts
import (
"strings"
"time"
)
// alertIntentSignalForRecord maps every alert family onto the small canonical
// signal vocabulary consumed by per-resource monitoring policy. Unknown alert
// families deliberately remain the default signal so expected_offline cannot
// hide unrelated health, capacity, backup, or security problems.
func alertIntentSignalForRecord(alert *Alert) string {
if alert == nil {
return string(AlertIntentSignalDefault)
}
if quietHoursCategoryForAlert(alert) == "offline" {
if strings.EqualFold(strings.TrimSpace(alert.Type), "resource-incident") {
return string(AlertIntentSignalAvailability)
}
return string(AlertIntentSignalOffline)
}
return string(AlertIntentSignalDefault)
}
func (m *Manager) operatorSuppressionForAlertNoLock(alert *Alert, now time.Time) (bool, string) {
if m == nil || alert == nil || m.operatorIntentResolver == nil {
return false, ""
}
resourceID := strings.TrimSpace(alert.ResourceID)
if resourceID == "" {
return false, ""
}
operator, found := m.operatorIntentResolver(resourceID, now.UTC())
if !found {
return false, ""
}
if operator.MaintenanceActiveAt(now) {
return true, "operator_maintenance"
}
return operator.suppressionForSignal(alertIntentSignalForRecord(alert))
}
// ReconcileResourceOperatorState immediately resolves active alerts that the
// newly persisted resource policy suppresses. Detector writes are also gated
// in setActiveAlertNoLock, so resolved alerts cannot reappear while the policy
// remains active. The return value is the number of active records cleared.
func (m *Manager) ReconcileResourceOperatorState(resourceID string) int {
resourceID = strings.TrimSpace(resourceID)
if m == nil || resourceID == "" {
return 0
}
now := time.Now().UTC()
m.mu.Lock()
alertIDs := make([]string, 0)
for storageKey, alert := range m.activeAlerts {
if alert == nil {
continue
}
candidate := strings.TrimSpace(alert.ResourceID)
if candidate == "" {
continue
}
matches := candidate == resourceID
if !matches && m.resourceIntentResolver != nil {
if canonicalID, found := m.resourceIntentResolver(candidate); found {
matches = strings.TrimSpace(canonicalID) == resourceID
}
}
if !matches {
continue
}
if suppressed, _ := m.operatorSuppressionForAlertNoLock(alert, now); suppressed {
alertIDs = append(alertIDs, effectiveAlertID(alert, storageKey))
}
}
m.mu.Unlock()
cleared := 0
for _, alertID := range alertIDs {
if m.ClearAlert(alertID) {
cleared++
}
}
return cleared
}
+13 -2
View File
@@ -123,6 +123,8 @@ type AgentFleetResourceSummary struct {
ResourceName string `json:"resourceName"`
Technology string `json:"technology,omitempty"`
IntentionallyOffline bool `json:"intentionallyOffline"`
MonitoringMode string `json:"monitoringMode"`
LifecycleState string `json:"lifecycleState"`
NeverAutoRemediate bool `json:"neverAutoRemediate"`
MaintenanceWindowActive bool `json:"maintenanceWindowActive"`
Findings AgentFleetFindingCounts `json:"findings"`
@@ -212,6 +214,8 @@ type AgentOperationsLoopStatus struct {
// the underlying store type's JSON tags shift.
type AgentResourceOperatorState struct {
IntentionallyOffline bool `json:"intentionallyOffline"`
MonitoringMode string `json:"monitoringMode"`
LifecycleState string `json:"lifecycleState"`
NeverAutoRemediate bool `json:"neverAutoRemediate"`
MaintenanceStartAt *time.Time `json:"maintenanceStartAt,omitempty"`
MaintenanceEndAt *time.Time `json:"maintenanceEndAt,omitempty"`
@@ -644,6 +648,8 @@ func agentResourceOperatorStateForContext(state *AgentResourceOperatorState) *ag
}
return &agentcontext.OperatorState{
IntentionallyOffline: state.IntentionallyOffline,
MonitoringMode: state.MonitoringMode,
LifecycleState: state.LifecycleState,
NeverAutoRemediate: state.NeverAutoRemediate,
MaintenanceWindowActive: state.MaintenanceWindowActive,
MaintenanceStartAt: state.MaintenanceStartAt,
@@ -745,7 +751,9 @@ func (h *AgentContextHandler) HandleFleetContext(w http.ResponseWriter, r *http.
}
if state, found, opErr := store.GetResourceOperatorState(canonical); opErr == nil && found {
summary.IntentionallyOffline = state.IntentionallyOffline
summary.NeverAutoRemediate = state.NeverAutoRemediate
summary.MonitoringMode = string(state.MonitoringMode)
summary.LifecycleState = string(state.LifecycleState)
summary.NeverAutoRemediate = state.BlocksRemediation()
summary.MaintenanceWindowActive = state.IsInMaintenanceAt(now)
}
if h.findingsProvider != nil {
@@ -1322,9 +1330,12 @@ func projectAgentResourceOperatorState(
state unified.ResourceOperatorState,
now time.Time,
) AgentResourceOperatorState {
state = unified.NormalizeResourceOperatorState(state)
return AgentResourceOperatorState{
IntentionallyOffline: state.IntentionallyOffline,
NeverAutoRemediate: state.NeverAutoRemediate,
MonitoringMode: string(state.MonitoringMode),
LifecycleState: string(state.LifecycleState),
NeverAutoRemediate: state.BlocksRemediation(),
MaintenanceStartAt: state.MaintenanceStartAt,
MaintenanceEndAt: state.MaintenanceEndAt,
MaintenanceReason: state.MaintenanceReason,
+7
View File
@@ -42,6 +42,13 @@ type ResourceHandlers struct {
actionExecutionAuthorizer actionlifecycle.ExecutionAuthorizer
actionRefreshPlanner actionlifecycle.RefreshPlanner
discoveryReadiness ResourceDiscoveryReadinessProvider
operatorStateChanged func(orgID, resourceID string)
}
// SetOperatorStateChanged installs the runtime reconciliation hook invoked
// after a canonical operator-state mutation commits successfully.
func (h *ResourceHandlers) SetOperatorStateChanged(callback func(orgID, resourceID string)) {
h.operatorStateChanged = callback
}
// ResourceDiscoveryReadinessProvider projects service-discovery state onto a
+34
View File
@@ -19,6 +19,8 @@ import (
// type's evolution stays decoupled from the wire format.
type resourceOperatorStateAPI struct {
CanonicalID string `json:"canonicalId"`
MonitoringMode string `json:"monitoringMode"`
LifecycleState string `json:"lifecycleState"`
IntentionallyOffline bool `json:"intentionallyOffline"`
NeverAutoRemediate bool `json:"neverAutoRemediate"`
AutoRemediationPolicy unified.AutoRemediationPolicy `json:"autoRemediationPolicy"`
@@ -32,8 +34,11 @@ type resourceOperatorStateAPI struct {
}
func toResourceOperatorStateAPI(state unified.ResourceOperatorState) resourceOperatorStateAPI {
state = unified.NormalizeResourceOperatorState(state)
return resourceOperatorStateAPI{
CanonicalID: state.CanonicalID,
MonitoringMode: string(state.MonitoringMode),
LifecycleState: string(state.LifecycleState),
IntentionallyOffline: state.IntentionallyOffline,
NeverAutoRemediate: state.NeverAutoRemediate,
AutoRemediationPolicy: state.AutoRemediationPolicy,
@@ -73,6 +78,7 @@ func (h *ResourceHandlers) HandleResourceOperatorState(w http.ResponseWriter, r
}
orgID := GetOrgID(r.Context())
resourceID = h.resolveOperatorStateCanonicalID(orgID, resourceID)
store, err := h.getStore(orgID)
if err != nil {
http.Error(w, sanitizeErrorForClient(err, "Internal server error"), http.StatusInternalServerError)
@@ -103,6 +109,8 @@ func (h *ResourceHandlers) HandleResourceOperatorState(w http.ResponseWriter, r
// (operator wrote vm:101 in the URL but vm:102 in the body).
state := unified.ResourceOperatorState{
CanonicalID: resourceID,
MonitoringMode: unified.ResourceMonitoringMode(payload.MonitoringMode),
LifecycleState: unified.ResourceLifecycleState(payload.LifecycleState),
IntentionallyOffline: payload.IntentionallyOffline,
NeverAutoRemediate: payload.NeverAutoRemediate,
AutoRemediationPolicy: payload.AutoRemediationPolicy,
@@ -130,6 +138,9 @@ func (h *ResourceHandlers) HandleResourceOperatorState(w http.ResponseWriter, r
http.Error(w, sanitizeErrorForClient(err, "Internal server error"), http.StatusInternalServerError)
return
}
if h.operatorStateChanged != nil {
h.operatorStateChanged(orgID, resourceID)
}
writeJSON(w, http.StatusOK, toResourceOperatorStateAPI(persisted))
case http.MethodDelete:
@@ -141,6 +152,9 @@ func (h *ResourceHandlers) HandleResourceOperatorState(w http.ResponseWriter, r
http.Error(w, sanitizeErrorForClient(err, "Internal server error"), http.StatusInternalServerError)
return
}
if h.operatorStateChanged != nil {
h.operatorStateChanged(orgID, resourceID)
}
w.WriteHeader(http.StatusNoContent)
default:
@@ -148,6 +162,26 @@ func (h *ResourceHandlers) HandleResourceOperatorState(w http.ResponseWriter, r
}
}
// resolveOperatorStateCanonicalID ensures every UI and agent entry point writes
// one policy record even when it starts from a source-native alert ID, a
// superseded ID, or another canonical identity alias. If the live registry
// cannot resolve the reference, preserving the normalized input retains the
// established API behavior for resources that are not currently inventoried.
func (h *ResourceHandlers) resolveOperatorStateCanonicalID(orgID, resourceID string) string {
resourceID = unified.CanonicalResourceID(resourceID)
if resourceID == "" {
return ""
}
registry, err := h.buildRegistry(orgID)
if err != nil || registry == nil {
return resourceID
}
if _, canonicalID, found := registry.GetByReference(resourceID); found {
return canonicalID
}
return resourceID
}
// extractOperatorStateResourceID pulls the canonical resource ID out of a
// `/api/resources/<id>/operator-state` URL path. Tolerates a trailing
// slash on the URL (defense-in-depth — Go 1.22 ServeMux normally rejects
@@ -11,6 +11,7 @@ import (
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/config"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
unified "github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
)
@@ -130,6 +131,90 @@ func TestHandleResourceOperatorState_PutPersistsAndGetReturns200(t *testing.T) {
}
}
func TestHandleResourceOperatorState_CanonicalPolicyTakesPrecedenceAndReconciles(t *testing.T) {
h := newOperatorStateHandlers(t)
var reconciledOrg, reconciledResource string
h.SetOperatorStateChanged(func(orgID, resourceID string) {
reconciledOrg, reconciledResource = orgID, resourceID
})
body := bytes.NewBufferString(`{
"monitoringMode":"muted",
"lifecycleState":"retired",
"intentionallyOffline":true,
"neverAutoRemediate":false
}`)
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPut, "/api/resources/vm:101/operator-state", body)
h.HandleResourceOperatorState(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d body=%s", rec.Code, rec.Body.String())
}
var got resourceOperatorStateAPI
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if got.MonitoringMode != "muted" || got.LifecycleState != "retired" {
t.Fatalf("canonical policy response = %+v", got)
}
if got.IntentionallyOffline {
t.Fatal("compatibility boolean must derive from canonical monitoring mode")
}
if reconciledOrg != "default" || reconciledResource != "vm:101" {
t.Fatalf("reconciliation callback = (%q, %q)", reconciledOrg, reconciledResource)
}
}
func TestHandleResourceOperatorState_ResolvesSourceAliasToCanonicalResource(t *testing.T) {
h := newOperatorStateHandlers(t)
now := time.Date(2026, 8, 11, 10, 0, 0, 0, time.UTC)
h.SetStateProvider(resourceUnifiedSeedProvider{
snapshot: models.StateSnapshot{LastUpdate: now},
resources: []unified.Resource{{
ID: "system-container-canonical",
Type: unified.ResourceTypeSystemContainer,
Name: "dev-portal-01",
LastSeen: now,
SupersededCanonicalIDs: []string{"mock-cluster-pve3-115"},
}},
})
var reconciledResource string
h.SetOperatorStateChanged(func(_, resourceID string) {
reconciledResource = resourceID
})
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPut, "/api/resources/mock-cluster-pve3-115/operator-state", bytes.NewBufferString(`{
"monitoringMode":"expected_offline",
"lifecycleState":"active"
}`))
h.HandleResourceOperatorState(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("PUT status = %d body=%s", rec.Code, rec.Body.String())
}
var got resourceOperatorStateAPI
if err := json.Unmarshal(rec.Body.Bytes(), &got); err != nil {
t.Fatal(err)
}
if got.CanonicalID != "system-container-canonical" {
t.Fatalf("canonical ID = %q", got.CanonicalID)
}
if reconciledResource != "system-container-canonical" {
t.Fatalf("reconciled resource = %q", reconciledResource)
}
store, err := h.getStore("default")
if err != nil {
t.Fatal(err)
}
if _, found, err := store.GetResourceOperatorState("mock-cluster-pve3-115"); err != nil || found {
t.Fatalf("source alias must not receive a duplicate record: found=%v err=%v", found, err)
}
if state, found, err := store.GetResourceOperatorState("system-container-canonical"); err != nil || !found || state.MonitoringMode != unified.MonitoringModeExpectedOffline {
t.Fatalf("canonical policy = %+v found=%v err=%v", state, found, err)
}
}
func TestResourceOperatorState_RecordsMaintenanceWindowLifecycleChanges(t *testing.T) {
h := newOperatorStateHandlers(t)
store, err := h.getStore("default")
+33
View File
@@ -540,6 +540,34 @@ func (r *Router) setupRoutes() {
r.unifiedAgentHandlers.SetServerVersion(r.serverVersion)
r.kubernetesAgentHandlers.SetRecoveryIngestor(r.recoveryHandlers)
r.resourceHandlers = NewResourceHandlers(r.config)
r.resourceHandlers.SetOperatorStateChanged(func(_ string, resourceID string) {
seen := make(map[*monitoring.Monitor]struct{})
reconcile := func(monitor *monitoring.Monitor) {
if monitor == nil {
return
}
if _, duplicate := seen[monitor]; duplicate {
return
}
seen[monitor] = struct{}{}
if reconciler, ok := any(monitor.GetAlertManager()).(interface {
ReconcileResourceOperatorState(string) int
}); ok {
reconciler.ReconcileResourceOperatorState(resourceID)
monitor.SyncAlertState()
}
}
if r.mtMonitor != nil {
// The store mutation is already tenant scoped. Each live monitor's
// resolver reads its own tenant store, so visiting every live monitor
// cannot apply another tenant's policy. It does avoid selecting a newly
// initialized, empty tenant monitor while alerts still live on the
// legacy/default runtime during startup and development transitions.
r.mtMonitor.ForEachMonitor(reconcile)
}
reconcile(r.monitor)
})
actionOrgChecker := NewAuthorizationChecker(NewMultiTenantOrganizationLoader(r.multiTenant))
actionAuth := actionAuthority{authorizer: r.authorizer, orgChecker: actionOrgChecker}
r.resourceHandlers.SetActionAuthorizers(actionAuth, actionAuth)
@@ -2466,9 +2494,14 @@ func (r *Router) startPatrolForContext(ctx context.Context, orgID string) bool {
}
projection := ai.ResourceOperatorStateProjection{
IntentionallyOffline: state.IntentionallyOffline,
MonitoringMode: string(state.MonitoringMode),
LifecycleState: string(state.LifecycleState),
NeverAutoRemediate: state.NeverAutoRemediate,
Criticality: string(state.Criticality),
}
if state.LifecycleState == unifiedresources.LifecycleStateRetired {
projection.NeverAutoRemediate = true
}
if state.IsInMaintenanceAt(now) {
projection.MaintenanceWindow = &ai.ResourceOperatorStateMaintenanceWindow{
StartAt: *state.MaintenanceStartAt,
@@ -50,6 +50,8 @@ func (m *Monitor) installOperatorIntentResolver(store ResourceStoreInterface) {
}
return alerts.OperatorIntentContext{
IntentionallyOffline: state.IntentionallyOffline,
MonitoringMode: string(state.MonitoringMode),
LifecycleState: string(state.LifecycleState),
MaintenanceStartAt: state.MaintenanceStartAt,
MaintenanceEndAt: state.MaintenanceEndAt,
MaintenanceReason: state.MaintenanceReason,
@@ -4,10 +4,41 @@ import (
"testing"
"time"
"github.com/rcourtman/pulse-go-rewrite/internal/alerts"
"github.com/rcourtman/pulse-go-rewrite/internal/models"
"github.com/rcourtman/pulse-go-rewrite/internal/unifiedresources"
"github.com/rcourtman/pulse-go-rewrite/pkg/proxmox"
)
func TestInstallOperatorIntentResolverProjectsCanonicalResourcePolicy(t *testing.T) {
store := unifiedresources.NewMemoryStore()
registry := unifiedresources.NewRegistry(store)
registry.IngestResources([]unifiedresources.Resource{{
ID: "vm:101", Type: unifiedresources.ResourceTypeVM, Name: "database",
}})
if err := store.SetResourceOperatorState(unifiedresources.ResourceOperatorState{
CanonicalID: "vm:101",
MonitoringMode: unifiedresources.MonitoringModeMuted,
LifecycleState: unifiedresources.LifecycleStateRetired,
}); err != nil {
t.Fatal(err)
}
manager := alerts.NewManagerWithDataDir(t.TempDir())
t.Cleanup(manager.Stop)
monitor := &Monitor{alertManager: manager}
monitor.installOperatorIntentResolver(unifiedresources.NewMonitorAdapter(registry))
preview, err := manager.PreviewIntentPolicy(alerts.AlertIntentPolicyPreviewRequest{
ResourceID: "vm:101", ResourceType: "vm", Signal: string(alerts.AlertIntentSignalOffline), ConditionActive: true,
})
if err != nil {
t.Fatal(err)
}
if preview.Reason != "operator_retired" || preview.Status != "expected_transient" {
t.Fatalf("canonical operator policy preview = %+v", preview)
}
}
func TestResolveBackupIntentContextRequiresFreshActiveMatchingEvidence(t *testing.T) {
now := time.Date(2026, 7, 20, 12, 0, 0, 0, time.UTC)
state := models.NewState()
@@ -14,7 +14,8 @@ import (
// state row. Order is not guaranteed.
func (s *SQLiteResourceStore) ListResourceOperatorStates() ([]ResourceOperatorState, error) {
rows, err := s.db.Query(`
SELECT canonical_id, intentionally_offline, never_auto_remediate,
SELECT canonical_id, monitoring_mode, lifecycle_state,
intentionally_offline, never_auto_remediate,
maintenance_start_at, maintenance_end_at, maintenance_reason,
criticality, note, set_at, set_by
FROM resource_operator_state`)
@@ -27,6 +28,8 @@ func (s *SQLiteResourceStore) ListResourceOperatorStates() ([]ResourceOperatorSt
for rows.Next() {
var (
state ResourceOperatorState
monitoringMode string
lifecycleState string
intentional int
neverRemediate int
startAt, endAt sql.NullTime
@@ -37,6 +40,8 @@ func (s *SQLiteResourceStore) ListResourceOperatorStates() ([]ResourceOperatorSt
)
if err := rows.Scan(
&state.CanonicalID,
&monitoringMode,
&lifecycleState,
&intentional,
&neverRemediate,
&startAt,
@@ -49,6 +54,8 @@ func (s *SQLiteResourceStore) ListResourceOperatorStates() ([]ResourceOperatorSt
); err != nil {
return nil, fmt.Errorf("scan resource operator state row: %w", err)
}
state.MonitoringMode = ResourceMonitoringMode(monitoringMode)
state.LifecycleState = ResourceLifecycleState(lifecycleState)
state.IntentionallyOffline = intentional != 0
state.NeverAutoRemediate = neverRemediate != 0
if startAt.Valid {
@@ -71,7 +78,7 @@ func (s *SQLiteResourceStore) ListResourceOperatorStates() ([]ResourceOperatorSt
if setBy.Valid {
state.SetBy = setBy.String
}
out = append(out, state)
out = append(out, NormalizeResourceOperatorState(state))
}
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("iterate resource operator state rows: %w", err)
@@ -14,6 +14,28 @@ import (
// is the default ("no operator-set criticality"), distinct from "medium".
type ResourceCriticality string
// ResourceMonitoringMode is the canonical per-resource alerting posture. It
// separates expected availability from an explicit all-signal mute so callers
// do not have to overload one boolean with incompatible meanings.
type ResourceMonitoringMode string
const (
MonitoringModeNormal ResourceMonitoringMode = "normal"
MonitoringModeExpectedOffline ResourceMonitoringMode = "expected_offline"
MonitoringModeMuted ResourceMonitoringMode = "muted"
)
// ResourceLifecycleState records whether a provider-owned inventory item is
// still operationally active in Pulse. Retired resources remain discoverable
// because their owning provider may continue to report them, but Pulse removes
// them from alert attention and refuses automated remediation.
type ResourceLifecycleState string
const (
LifecycleStateActive ResourceLifecycleState = "active"
LifecycleStateRetired ResourceLifecycleState = "retired"
)
const (
CriticalityHigh ResourceCriticality = "high"
CriticalityMedium ResourceCriticality = "medium"
@@ -51,6 +73,22 @@ func IsValidCriticality(value string) bool {
return false
}
func IsValidMonitoringMode(value string) bool {
switch ResourceMonitoringMode(value) {
case MonitoringModeNormal, MonitoringModeExpectedOffline, MonitoringModeMuted:
return true
}
return false
}
func IsValidLifecycleState(value string) bool {
switch ResourceLifecycleState(value) {
case LifecycleStateActive, LifecycleStateRetired:
return true
}
return false
}
// ResourceOperatorState captures operator-set per-resource intent that
// modulates Patrol's behavior on findings against this resource. The shape
// is intentionally narrow: every field encodes a specific operator intent
@@ -70,11 +108,21 @@ type ResourceOperatorState struct {
// unified-resources store; the API boundary trims and rejects empty.
CanonicalID string `json:"canonicalId"`
// IntentionallyOffline marks the resource as expected-to-be-offline.
// Findings of the form "resource X is offline" against this resource
// will be auto-acknowledged with reason=intentionally_offline by the
// Patrol findings store. Other finding categories (high CPU, disk
// pressure on a still-mounted volume, etc.) are unaffected.
// MonitoringMode is the canonical alert and Patrol posture. Normal keeps
// all monitoring active, expected_offline suppresses availability noise,
// and muted suppresses all alert and finding attention while preserving
// inventory visibility.
MonitoringMode ResourceMonitoringMode `json:"monitoringMode"`
// LifecycleState is active by default. Retired is an operator-owned
// lifecycle decision for resources that remain in provider inventory but
// should no longer participate in alerting or automated remediation.
LifecycleState ResourceLifecycleState `json:"lifecycleState"`
// IntentionallyOffline is the compatibility projection for clients that
// predate MonitoringMode. NormalizeResourceOperatorState derives it from
// MonitoringMode, and maps legacy true writes to expected_offline when the
// new field is absent. Runtime policy must consume MonitoringMode instead.
IntentionallyOffline bool `json:"intentionallyOffline"`
// NeverAutoRemediate forbids Patrol from dispatching automated fixes
@@ -130,7 +178,9 @@ type ResourceOperatorState struct {
// the persistence layer MAY keep an audit row to track that the operator
// explicitly cleared the state.
func (s ResourceOperatorState) IsEmpty() bool {
return !s.IntentionallyOffline &&
normalized := NormalizeResourceOperatorState(s)
return normalized.MonitoringMode == MonitoringModeNormal &&
normalized.LifecycleState == LifecycleStateActive &&
!s.NeverAutoRemediate &&
!s.AutoRemediationPolicy.Enabled &&
len(s.AutoRemediationPolicy.CapabilityNames) == 0 &&
@@ -142,6 +192,28 @@ func (s ResourceOperatorState) IsEmpty() bool {
strings.TrimSpace(s.Note) == ""
}
// SuppressesAllAttention reports whether every Alerts and Patrol signal is
// outside the resource's current operator-owned monitoring posture.
func (s ResourceOperatorState) SuppressesAllAttention() bool {
s = NormalizeResourceOperatorState(s)
return s.MonitoringMode == MonitoringModeMuted || s.LifecycleState == LifecycleStateRetired
}
// ExpectsOffline reports whether availability loss is expected while other
// monitoring signals remain eligible.
func (s ResourceOperatorState) ExpectsOffline() bool {
s = NormalizeResourceOperatorState(s)
return s.MonitoringMode == MonitoringModeExpectedOffline || s.SuppressesAllAttention()
}
// BlocksRemediation reports whether automated action is incompatible with the
// operator's explicit state. Retirement is a lifecycle lock even when the
// legacy NeverAutoRemediate flag was not separately set.
func (s ResourceOperatorState) BlocksRemediation() bool {
s = NormalizeResourceOperatorState(s)
return s.NeverAutoRemediate || s.LifecycleState == LifecycleStateRetired
}
// IsInMaintenanceAt reports whether `now` falls within the configured
// maintenance window. Returns false when no window is configured, when only
// one of start/end is set (treated as no window), or when end <= start.
@@ -173,12 +245,19 @@ var ErrResourceOperatorStateInvalid = errors.New("resource_operator_state_invali
// is structural only — operator-set meaning (was the maintenance window
// actually intended? is the note correct?) is the operator's call.
func ValidateResourceOperatorState(state ResourceOperatorState) error {
state = NormalizeResourceOperatorState(state)
if strings.TrimSpace(state.CanonicalID) == "" {
return fmt.Errorf("%w: canonical_id is required", ErrResourceOperatorStateInvalid)
}
if !IsValidCriticality(string(state.Criticality)) {
return fmt.Errorf("%w: criticality %q is not one of (high, medium, low, empty)", ErrResourceOperatorStateInvalid, state.Criticality)
}
if !IsValidMonitoringMode(string(state.MonitoringMode)) {
return fmt.Errorf("%w: monitoring_mode %q is not one of (normal, expected_offline, muted)", ErrResourceOperatorStateInvalid, state.MonitoringMode)
}
if !IsValidLifecycleState(string(state.LifecycleState)) {
return fmt.Errorf("%w: lifecycle_state %q is not one of (active, retired)", ErrResourceOperatorStateInvalid, state.LifecycleState)
}
if err := ValidateAutoRemediationPolicy(state.AutoRemediationPolicy); err != nil {
return fmt.Errorf("%w: %v", ErrResourceOperatorStateInvalid, err)
}
@@ -201,6 +280,19 @@ func ValidateResourceOperatorState(state ResourceOperatorState) error {
// NOT validate — call ValidateResourceOperatorState afterward.
func NormalizeResourceOperatorState(state ResourceOperatorState) ResourceOperatorState {
state.CanonicalID = strings.TrimSpace(state.CanonicalID)
state.MonitoringMode = ResourceMonitoringMode(strings.ToLower(strings.TrimSpace(string(state.MonitoringMode))))
if state.MonitoringMode == "" {
if state.IntentionallyOffline {
state.MonitoringMode = MonitoringModeExpectedOffline
} else {
state.MonitoringMode = MonitoringModeNormal
}
}
state.IntentionallyOffline = state.MonitoringMode == MonitoringModeExpectedOffline
state.LifecycleState = ResourceLifecycleState(strings.ToLower(strings.TrimSpace(string(state.LifecycleState))))
if state.LifecycleState == "" {
state.LifecycleState = LifecycleStateActive
}
state.MaintenanceReason = strings.TrimSpace(state.MaintenanceReason)
state.Note = strings.TrimSpace(state.Note)
state.SetBy = strings.TrimSpace(state.SetBy)
@@ -64,7 +64,9 @@ func TestAutoRemediationPolicySQLiteRoundTrip(t *testing.T) {
defer store.Close()
state := ResourceOperatorState{
CanonicalID: "docker:container:web",
CanonicalID: "docker:container:web",
MonitoringMode: MonitoringModeMuted,
LifecycleState: LifecycleStateRetired,
AutoRemediationPolicy: AutoRemediationPolicy{
Enabled: true,
CapabilityNames: []string{"restart"},
@@ -85,6 +87,9 @@ func TestAutoRemediationPolicySQLiteRoundTrip(t *testing.T) {
if !got.AutoRemediationPolicy.Enabled || len(got.AutoRemediationPolicy.CapabilityNames) != 1 || got.AutoRemediationPolicy.Window == nil || got.AutoRemediationPolicy.Window.Timezone != "Europe/London" {
t.Fatalf("round-trip policy = %#v", got.AutoRemediationPolicy)
}
if got.MonitoringMode != MonitoringModeMuted || got.LifecycleState != LifecycleStateRetired {
t.Fatalf("round-trip monitoring policy = %+v", got)
}
}
func TestSQLiteOperatorStateNullPolicyColumnNormalizesOnRead(t *testing.T) {
@@ -149,6 +149,49 @@ func TestValidateResourceOperatorState(t *testing.T) {
})
}
func TestResourceOperatorStateMonitoringAndLifecyclePolicy(t *testing.T) {
t.Run("legacy intentionally offline normalizes to expected offline", func(t *testing.T) {
state := NormalizeResourceOperatorState(ResourceOperatorState{IntentionallyOffline: true})
if state.MonitoringMode != MonitoringModeExpectedOffline || !state.IntentionallyOffline {
t.Fatalf("legacy normalization = %+v", state)
}
if state.LifecycleState != LifecycleStateActive {
t.Fatalf("default lifecycle = %q, want active", state.LifecycleState)
}
})
t.Run("canonical mode owns compatibility projection", func(t *testing.T) {
state := NormalizeResourceOperatorState(ResourceOperatorState{
MonitoringMode: MonitoringModeMuted,
IntentionallyOffline: true,
})
if state.IntentionallyOffline {
t.Fatal("muted mode must not remain projected as intentionally offline")
}
if !state.SuppressesAllAttention() {
t.Fatal("muted mode must suppress all attention")
}
})
t.Run("retired lifecycle suppresses attention and blocks remediation", func(t *testing.T) {
state := ResourceOperatorState{LifecycleState: LifecycleStateRetired}
if !state.SuppressesAllAttention() || !state.BlocksRemediation() {
t.Fatalf("retired policy not enforced: %+v", state)
}
})
t.Run("invalid enum values fail closed", func(t *testing.T) {
for _, state := range []ResourceOperatorState{
{CanonicalID: "vm:101", MonitoringMode: "sometimes"},
{CanonicalID: "vm:101", LifecycleState: "deleted"},
} {
if err := ValidateResourceOperatorState(state); !errors.Is(err, ErrResourceOperatorStateInvalid) {
t.Fatalf("ValidateResourceOperatorState(%+v) error = %v", state, err)
}
}
})
}
func TestNormalizeResourceOperatorState_TrimsAndLowersCriticality(t *testing.T) {
got := NormalizeResourceOperatorState(ResourceOperatorState{
CanonicalID: " vm:101 ",
+37 -7
View File
@@ -540,6 +540,8 @@ func (s *SQLiteResourceStore) initSchema() error {
CREATE TABLE IF NOT EXISTS resource_operator_state (
canonical_id TEXT PRIMARY KEY,
monitoring_mode TEXT NOT NULL DEFAULT 'normal',
lifecycle_state TEXT NOT NULL DEFAULT 'active',
intentionally_offline INTEGER NOT NULL DEFAULT 0,
never_auto_remediate INTEGER NOT NULL DEFAULT 0,
auto_remediation_policy_json TEXT,
@@ -642,10 +644,26 @@ func (s *SQLiteResourceStore) migrateResourceOperatorStateSchema() error {
if err != nil {
return err
}
if _, ok := columns["auto_remediation_policy_json"]; !ok {
if _, err := s.db.Exec("ALTER TABLE resource_operator_state ADD COLUMN auto_remediation_policy_json TEXT"); err != nil {
return fmt.Errorf("add resource_operator_state.auto_remediation_policy_json column: %w", err)
definitions := map[string]string{
"monitoring_mode": "TEXT NOT NULL DEFAULT 'normal'",
"lifecycle_state": "TEXT NOT NULL DEFAULT 'active'",
"auto_remediation_policy_json": "TEXT",
}
for name, definition := range definitions {
if _, ok := columns[name]; ok {
continue
}
if _, err := s.db.Exec("ALTER TABLE resource_operator_state ADD COLUMN " + name + " " + definition); err != nil {
return fmt.Errorf("add resource_operator_state.%s column: %w", name, err)
}
}
// Rows written before monitoring_mode existed used the compatibility
// boolean as their source of truth. Promote those rows once so all future
// reads and writes use the canonical enum.
if _, err := s.db.Exec(`UPDATE resource_operator_state
SET monitoring_mode = 'expected_offline'
WHERE intentionally_offline = 1 AND monitoring_mode = 'normal'`); err != nil {
return fmt.Errorf("migrate resource_operator_state monitoring mode: %w", err)
}
return nil
}
@@ -2925,7 +2943,8 @@ func getResourceOperatorStateSQL(queryer resourceOperatorStateQueryRower, canoni
return ResourceOperatorState{}, false, nil
}
row := queryer.QueryRow(`
SELECT canonical_id, intentionally_offline, never_auto_remediate,
SELECT canonical_id, monitoring_mode, lifecycle_state,
intentionally_offline, never_auto_remediate,
auto_remediation_policy_json,
maintenance_start_at, maintenance_end_at, maintenance_reason,
criticality, note, set_at, set_by
@@ -2943,6 +2962,8 @@ func getResourceOperatorStateSQL(queryer resourceOperatorStateQueryRower, canoni
func scanResourceOperatorState(scanner resourceOperatorStateScanner) (ResourceOperatorState, error) {
var state ResourceOperatorState
var (
monitoringMode string
lifecycleState string
intentional int
neverRemediate int
autoPolicyJSON sql.NullString
@@ -2954,6 +2975,8 @@ func scanResourceOperatorState(scanner resourceOperatorStateScanner) (ResourceOp
)
if err := scanner.Scan(
&state.CanonicalID,
&monitoringMode,
&lifecycleState,
&intentional,
&neverRemediate,
&autoPolicyJSON,
@@ -2967,6 +2990,8 @@ func scanResourceOperatorState(scanner resourceOperatorStateScanner) (ResourceOp
); err != nil {
return ResourceOperatorState{}, err
}
state.MonitoringMode = ResourceMonitoringMode(monitoringMode)
state.LifecycleState = ResourceLifecycleState(lifecycleState)
state.IntentionallyOffline = intentional != 0
state.NeverAutoRemediate = neverRemediate != 0
if autoPolicyJSON.Valid && strings.TrimSpace(autoPolicyJSON.String) != "" {
@@ -2998,7 +3023,7 @@ func scanResourceOperatorState(scanner resourceOperatorStateScanner) (ResourceOp
if setBy.Valid {
state.SetBy = setBy.String
}
return state, nil
return NormalizeResourceOperatorState(state), nil
}
// SetResourceOperatorState upserts the state row. Validates and
@@ -3069,12 +3094,15 @@ func setResourceOperatorStateSQL(execer sqlExecutor, state ResourceOperatorState
}
_, err := execer.Exec(`
INSERT INTO resource_operator_state (
canonical_id, intentionally_offline, never_auto_remediate,
canonical_id, monitoring_mode, lifecycle_state,
intentionally_offline, never_auto_remediate,
auto_remediation_policy_json,
maintenance_start_at, maintenance_end_at, maintenance_reason,
criticality, note, set_at, set_by
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(canonical_id) DO UPDATE SET
monitoring_mode = excluded.monitoring_mode,
lifecycle_state = excluded.lifecycle_state,
intentionally_offline = excluded.intentionally_offline,
never_auto_remediate = excluded.never_auto_remediate,
auto_remediation_policy_json = excluded.auto_remediation_policy_json,
@@ -3086,6 +3114,8 @@ func setResourceOperatorStateSQL(execer sqlExecutor, state ResourceOperatorState
set_at = excluded.set_at,
set_by = excluded.set_by`,
state.CanonicalID,
state.MonitoringMode,
state.LifecycleState,
intentional,
neverRemediate,
autoPolicyJSON,