rcourtman 2f967406df perf(ai): gate and batch Patrol's per-alert auto-resolve model calls ...

Production telemetry (30d) shows Patrol averaging ~9.4 billed AI calls
per run while only ~1-2 come from the analysis passes. The remainder is
the alert auto-resolve review: one QuickAnalysis call per active alert
(>=10min old) on every run, re-asking the model about the same standing
alert every interval even when the trigger condition demonstrably still
holds.

Two structural changes, both resolution-neutral:

- Still-firing gate: skip the model review for alerts whose current
  snapshot still shows the trigger condition (metric at/above threshold,
  offline resource still offline). A correct review could only answer
  KEEP, so the question is not worth a billed call. The gate never
  resolves anything locally - uncertain cases (unknown types, missing
  resources, unmapped metrics) still go to the model, which retains
  sole authority over resolution.
- Batched review: the remaining candidates are reviewed in one model
  call per 20 alerts (numbered verdict lines, unparseable -> KEEP,
  bare-RESOLVE fallback for single-alert batches) instead of one call
  per alert.

Steady-state runs with standing alerts drop from N review calls to 0;
runs where conditions may have cleared pay ceil(K/20) instead of K.
QuickAnalysisRequest gains a TargetType tag (recorded on the usage
event) so cost telemetry can decompose alert-review spend from the
main patrol pass going forward. Contract updated in-commit
(ai-runtime: model-owned resolve direction, cost-gated keep direction,
batched review, fail-safe parsing, alert_autoresolve usage tagging).

2026-07-17 23:23:38 +01:00

..

adapters

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

approval

Fix mobile relay runtime startup

2026-07-01 10:38:57 +01:00

baseline

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

chat

Allow Patrol to record sequential findings

2026-07-16 22:50:11 +01:00

circuit

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

correlation

Stabilize dev runtime and complete PMG read-state migration

2026-05-21 15:15:23 +01:00

cost

Price reviewed OpenRouter qualification routes

2026-07-14 18:24:33 +01:00

eval

Require grounded actionable Patrol findings

2026-07-15 18:44:33 +01:00

forecast

Make Patrol reasoning model-owned

2026-05-15 12:22:52 +01:00

infradiscovery

Remove 198 unreachable Go functions

2026-06-03 12:29:37 +01:00

jsonresponse

Fix Discovery response JSON extraction

2026-05-25 18:02:28 +01:00

knowledge

Harden AI knowledge guest listing paths

2026-03-29 14:18:20 +01:00

learning

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

memory

Dedupe AI runtime guest-family, tool-pipeline, and provider clones

2026-06-10 09:38:52 +01:00

modelboundary

Remove the cloud_context_privacy dial; fix cloud context to a lean posture

2026-06-09 09:43:21 +01:00

modelresolution

Add Go branch-coverage tests for alerts monitoring model-resolution ai and licensing helpers

2026-07-16 15:17:45 +01:00

patterns

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

providers

Bound automatic Patrol inference allowances

2026-07-16 19:55:44 +01:00

proxmox

Cap AI patrol history and correlation reads

2026-04-01 14:47:58 +01:00

qualification

Add deep-copy independence and qualification render coverage tests

2026-07-16 22:56:24 +01:00

safety

Shorten fake Google API key fixtures below secret-scanner shape

2026-07-16 13:40:38 +01:00

tools

feat(ai): surface the Assistant at first AI setup and fix blessed-path readiness

2026-07-17 23:20:44 +01:00

unified

Fix missing impact text and incomplete alert type coverage in unified findings

2026-06-27 14:43:09 +01:00

agent_exec_test_helpers_test.go

Align Patrol prober with agent command policy

2026-05-01 20:28:11 +01:00

ai_core_pure_helpers_branchcov0716_test.go

Add Go branch-coverage tests for alerts monitoring model-resolution ai and licensing helpers

2026-07-16 15:17:45 +01:00

alert_adapter_test.go

refactor(ai): canonicalize truenas host resource types

2026-03-30 13:34:08 +01:00

alert_adapter.go

refactor(ai): canonicalize truenas host resource types

2026-03-30 13:34:08 +01:00

alert_provider_additional_test.go

test: expand backend coverage

2026-01-25 21:08:44 +00:00

alert_provider_test.go

Require approval for backend alert investigation commands

2026-05-07 01:24:57 +01:00

alert_provider.go

Require approval for backend alert investigation commands

2026-05-07 01:24:57 +01:00

alert_threshold_adapter_test.go

fix: Docker container update detection showing false positives

2025-12-29 13:49:04 +00:00

alert_threshold_adapter.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

baseline_adapter_test.go

fix: Docker container update detection showing false positives

2025-12-29 13:49:04 +00:00

baseline_adapter.go

Enhance devcontainer and CI workflows

2026-01-01 22:29:15 +00:00

baseline_exports.go

feat(ai): Wire baseline learning loop into router startup

2025-12-12 11:29:47 +00:00

config.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

correlation_exports.go

feat(ai): Add multi-resource correlation detection (Phase 6)

2025-12-12 14:26:10 +00:00

cost_persistence_test.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

cost_persistence.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

cost_recording_audit_test.go

Fix QuickAnalysis cost recording and audit the pattern

2026-05-10 21:57:39 +01:00

coverage_increase_test.go

Remove 198 unreachable Go functions

2026-06-03 12:29:37 +01:00

demo_test.go

Simulate Patrol end to end in demo/mock mode

2026-07-11 11:25:52 +01:00

demo.go

Rename the demo Patrol snapshot parameter so the SRC-04b guard passes

2026-07-11 12:02:05 +01:00

discovery_adapter_test.go

Bypass approval gate for trusted internal Discovery commands

2026-05-17 21:59:39 +01:00

discovery_adapter.go

Remove 198 unreachable Go functions

2026-06-03 12:29:37 +01:00

exports_test.go

Restore RC backend proof regressions

2026-04-09 20:15:17 +01:00

findings_additional_test.go

test: expand backend coverage

2026-01-25 21:08:44 +00:00

findings_apt_workflows_test.go

Complete durable APT workflow continuity

2026-07-12 05:23:02 +01:00

findings_apt_workflows.go

Restore release-blocking backend contracts

2026-07-13 21:51:33 +01:00

findings_backup_verification_test.go

emit backup_verification_stale finding from VerifyIntent

2026-05-12 22:23:55 +01:00

findings_backup_verification.go

emit backup_verification_stale finding from VerifyIntent

2026-05-12 22:23:55 +01:00

findings_clustering_test.go

feat(backend): implement AI Patrol, Investigation, and system-wide refactors

2026-01-30 19:02:14 +00:00

findings_coverage_test.go

Purge resolved legacy alert-mirror findings on load

2026-05-11 11:40:07 +01:00

findings_dedup_test.go

feat(backend): implement AI Patrol, Investigation, and system-wide refactors

2026-01-30 19:02:14 +00:00

findings_investigation_test.go

feat(backend): implement AI Patrol, Investigation, and system-wide refactors

2026-01-30 19:02:14 +00:00

findings_lifecycle_test.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

findings_marshal_mirror_test.go

Dedupe AI runtime guest-family, tool-pipeline, and provider clones

2026-06-10 09:38:52 +01:00

findings_pdm_alert_bridge_test.go

Clear all errcheck and gofmt violations so make lint gates on real findings

2026-06-09 21:42:21 +01:00

findings_pdm_alert_bridge.go

Add PDM HTTP alert source

2026-05-13 11:35:24 +01:00

findings_pdm_alert_emit_test.go

prove PDM alert bridge emits and resolves through FindingsStore.Add

2026-05-13 05:42:25 +01:00

findings_pdm_http_client_test.go

Add PDM HTTP alert source

2026-05-13 11:35:24 +01:00

findings_pdm_http_client.go

Add PDM HTTP alert source

2026-05-13 11:35:24 +01:00

findings_persistence_test.go

Add resource priority to Patrol queue

2026-06-30 10:37:54 +01:00

findings_persistence.go

Simulate Patrol end to end in demo/mock mode

2026-07-11 11:25:52 +01:00

findings_proxmox_lifecycle_test.go

Bind governed action intent across Patrol and web

2026-07-13 16:05:44 +01:00

findings_proxmox_lifecycle.go

refactor(ai): dedupe guest lifecycle snapshot loops and check the tool-loop helper return

2026-07-17 17:29:22 +01:00

findings_remind_persistence_test.go

Clear all errcheck and gofmt violations so make lint gates on real findings

2026-06-09 21:42:21 +01:00

findings_remind_sweep_test.go

add hourly sweep for overdue will_fix_later commitments

2026-05-13 01:13:38 +01:00

findings_resource_type_test.go

refactor(ai): canonicalize truenas host resource types

2026-03-30 13:34:08 +01:00

findings_storm_emit_test.go

prove storm throttler emits and resolves through FindingsStore.Add

2026-05-13 02:41:45 +01:00

findings_storm_throttler_test.go

add storm throttler substrate in FindingsStore.Add

2026-05-13 02:41:45 +01:00

findings_storm_throttler.go

add storm throttler substrate in FindingsStore.Add

2026-05-13 02:41:45 +01:00

findings_suppression_persistence_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

findings_test.go

Add resource priority to Patrol queue

2026-06-30 10:37:54 +01:00

findings_timeout_test.go

feat(patrol): add timeout handling and shorter retry cooldown

2026-02-01 10:12:23 +00:00

findings_tools_adapter_test.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

findings_tools_adapter.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

findings_update_safety_emit_test.go

Clear all errcheck and gofmt violations so make lint gates on real findings

2026-06-09 21:42:21 +01:00

findings_update_safety_test.go

handle repeated update-safety digest changes

2026-05-13 04:18:27 +01:00

findings_update_safety.go

handle repeated update-safety digest changes

2026-05-13 04:18:27 +01:00

findings.go

Add resource priority to Patrol queue

2026-06-30 10:37:54 +01:00

incident_coordinator_additional_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

incident_coordinator_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

incident_coordinator.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

intelligence_coverage_test.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

intelligence_test.go

Route correlations through intelligence facade

2026-03-19 04:15:26 +00:00

intelligence.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

investigation_records_test.go

Complete Patrol action lifecycle continuity

2026-07-10 21:22:06 +01:00

investigation_records.go

Complete Patrol action lifecycle continuity

2026-07-10 21:22:06 +01:00

kubernetes_analysis_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

kubernetes_analysis.go

Audit unified AI exports

2026-03-18 17:22:24 +00:00

manual_ollama_preflight_test.go

Add a guided Ollama quickstart blessing qwen3:8b for Patrol

2026-07-10 13:02:26 +01:00

memory_exports.go

feat(ai): Add operational memory (Phase 3) - change detection and remediation logging

2025-12-12 13:49:37 +00:00

metadata_provider_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

metadata_provider.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

metrics_history_adapter_test.go

fix: Docker container update detection showing false positives

2025-12-29 13:49:04 +00:00

metrics_history_adapter.go

feat(ai): Add enriched context with historical trends and predictions

2025-12-12 09:45:57 +00:00

mock_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

model_resolution_test.go

Keep Assistant chat model routes explicit

2026-06-08 07:21:38 +01:00

model_resolution.go

Fail closed on unknown remediation lock state for autonomous dispatches

2026-07-10 00:14:01 +01:00

parity_test.go

Fix host metric percent normalization

2026-07-06 10:08:36 +01:00

patrol_ai_additional_test.go

Preserve qualification artifacts for bounded transcripts

2026-07-14 17:31:05 +01:00

patrol_ai_eval_additional_test.go

Harden Patrol multi-finding report guidance

2026-07-16 06:40:17 +01:00

patrol_ai_intelligence_test.go

Complete Patrol autonomous qualification loop

2026-07-14 15:35:48 +01:00

patrol_ai_more_test.go

Treat repeated container restarts as actionable evidence

2026-07-15 20:42:17 +01:00

patrol_ai_test.go

fix(ai): filter DeepSeek DSML internal function-call format from responses

2026-02-01 18:07:41 +00:00

patrol_ai.go

Harden Patrol multi-finding report guidance

2026-07-16 06:40:17 +01:00

patrol_alerts_review_test.go

perf(ai): gate and batch Patrol's per-alert auto-resolve model calls

2026-07-17 23:23:38 +01:00

patrol_assistant_handoff_test.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

patrol_assistant_handoff.go

backend and governance: MCP contract, agent capabilities, API, and release-control

2026-06-23 17:26:15 +01:00

patrol_context_additional_test.go

Clarify Patrol scope identities and finding reads

2026-07-14 19:23:36 +01:00

patrol_coverage_test.go

Gate Patrol readiness across runtime entrypoints

2026-05-07 18:24:47 +01:00

patrol_findings_additional_test.go

Complete Patrol autonomous qualification loop

2026-07-14 15:35:48 +01:00

patrol_findings_forecast_test.go

fix(ai): query capacity forecast history under metrics-target ID not canonical ID

2026-06-27 22:18:09 +01:00

patrol_findings_json_test.go

Add branch-coverage tests for AI patrol, qualification, tools and agentcapability helpers

2026-07-14 22:33:40 +01:00

patrol_findings.go

Isolate Patrol investigation dispatch from Watch runs

2026-07-16 14:06:01 +01:00

patrol_history_memory_regression_test.go

test: add memory regression coverage for AI stores

2026-02-04 19:56:12 +00:00

patrol_history_persistence_test.go

Keep Patrol evidence aligned with live findings

2026-05-08 11:08:53 +01:00

patrol_history_persistence.go

Keep Patrol evidence aligned with live findings

2026-05-08 11:08:53 +01:00

patrol_init_additional_test.go

Publish Patrol investigation limits live

2026-07-15 10:49:16 +01:00

patrol_init.go

Clarify Patrol event trigger runtime state

2026-06-15 10:08:54 +01:00

patrol_integration_test.go

Isolate Patrol investigation dispatch from Watch runs

2026-07-16 14:06:01 +01:00

patrol_intelligence_test.go

Align Patrol prober with agent command policy

2026-05-01 20:28:11 +01:00

patrol_intelligence.go

Dedupe AI runtime guest-family, tool-pipeline, and provider clones

2026-06-10 09:38:52 +01:00

patrol_lifecycle_integration_test.go

fix(patrol): improve service lifecycle, graceful shutdown, and concurrency

2026-02-01 16:27:25 +00:00

patrol_metrics_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

patrol_metrics.go

Remove Assistant and Patrol tool-routing heuristics

2026-05-15 14:51:15 +01:00

patrol_preflight_test.go

Allow bounded subscription-agent preflights

2026-07-16 09:54:34 +01:00

patrol_preflight.go

Allow bounded subscription-agent preflights

2026-07-16 09:54:34 +01:00

patrol_prober_test.go

Align Patrol prober with agent command policy

2026-05-01 20:28:11 +01:00

patrol_prober.go

Align Patrol prober with agent command policy

2026-05-01 20:28:11 +01:00

patrol_readiness_test.go

feat(ai): surface the Assistant at first AI setup and fix blessed-path readiness

2026-07-17 23:20:44 +01:00

patrol_readiness.go

feat(ai): surface the Assistant at first AI setup and fix blessed-path readiness

2026-07-17 23:20:44 +01:00

patrol_reconcile_test.go

Canonicalize assistant provider registry

2026-06-23 19:38:22 +01:00

patrol_resource_state_test.go

refactor(ai): canonicalize truenas host resource types

2026-03-30 13:34:08 +01:00

patrol_run_additional_test.go

perf(ai): gate and batch Patrol's per-alert auto-resolve model calls

2026-07-17 23:23:38 +01:00

patrol_run_recency_test.go

Add branch-coverage tests for AI patrol, qualification, tools and agentcapability helpers

2026-07-14 22:33:40 +01:00

patrol_run_test.go

perf(ai): gate and batch Patrol's per-alert auto-resolve model calls

2026-07-17 23:23:38 +01:00

patrol_run.go

perf(ai): gate and batch Patrol's per-alert auto-resolve model calls

2026-07-17 23:23:38 +01:00

patrol_runtime_failure_coverage_test.go

Add branch-coverage tests for patrol runtime failure classifiers

2026-07-14 23:23:14 +01:00

patrol_runtime_failure_test.go

Preserve Patrol evidence on provider errors

2026-07-14 16:34:19 +01:00

patrol_runtime_failure.go

Finish the Assistant & Patrol settings rename to Pulse Intelligence

2026-07-10 12:04:03 +01:00

patrol_scoped_run_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

patrol_seed_budget_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

patrol_signals_test.go

Improve Patrol SMART disk evidence

2026-06-29 18:17:24 +01:00

patrol_signals_vmid_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

patrol_signals.go

Improve Patrol SMART disk evidence

2026-06-29 18:17:24 +01:00

patrol_state.go

Complete Patrol autonomous qualification loop

2026-07-14 15:35:48 +01:00

patrol_stream_debug.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

patrol_test.go

Auto-clear event/persistent Patrol findings on affirmative deterministic verification

2026-06-09 21:47:14 +01:00

patrol_thresholds_test.go

feat: add AI Intelligence system with investigation and forecasting

2026-01-24 22:41:43 +00:00

patrol_triage_integration_test.go

Clarify Patrol scope identities and finding reads

2026-07-14 19:23:36 +01:00

patrol_triage_test.go

Treat repeated container restarts as actionable evidence

2026-07-15 20:42:17 +01:00

patrol_triage.go

Treat repeated container restarts as actionable evidence

2026-07-15 20:42:17 +01:00

patrol_triggers_flapping_test.go

add patrol scope for flapping postmortem

2026-05-13 00:28:24 +01:00

patrol_triggers_test.go

Clarify Patrol event trigger runtime state

2026-06-15 10:08:54 +01:00

patrol_triggers.go

Complete Patrol autonomous qualification loop

2026-07-14 15:35:48 +01:00

patrol.go

Preserve qualification artifacts for bounded transcripts

2026-07-14 17:31:05 +01:00

pattern_exports.go

feat(ai): Add failure pattern detection for predictive intelligence (Phase 5)

2025-12-12 14:11:28 +00:00

report_findings_test.go

Replace heuristic report narrative with optional AI-generated layer

2026-05-10 19:30:54 +01:00

report_findings.go

Replace heuristic report narrative with optional AI-generated layer

2026-05-10 19:30:54 +01:00

report_fleet_narrator_test.go

Add fleet-level AI narrative for multi-resource reports

2026-05-10 21:23:12 +01:00

report_fleet_narrator.go

Forbid the report narrators from acting as parallel detectors

2026-05-10 22:02:10 +01:00

report_narrator_test.go

Record cost events for AI report narration

2026-05-10 21:08:51 +01:00

report_narrator.go

Forbid the report narrators from acting as parallel detectors

2026-05-10 22:02:10 +01:00

resource_context_policy_model_test.go

Remove the cloud_context_privacy dial; fix cloud context to a lean posture

2026-06-09 09:43:21 +01:00

resource_context_policy_model.go

Remove the cloud_context_privacy dial; fix cloud context to a lean posture

2026-06-09 09:43:21 +01:00

resource_context_test.go

Recalibrate resource sensitivity: ordinary workloads aren't secret

2026-06-08 15:21:51 +01:00

resource_context.go

Remove the cloud_context_privacy dial; fix cloud context to a lean posture

2026-06-09 09:43:21 +01:00

resource_export_test.go

Canonicalize export redaction labels

2026-03-19 04:07:47 +00:00

resource_export.go

Canonicalize export redaction labels

2026-03-19 04:07:47 +00:00

resource_type_legacy_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

resource_type_legacy.go

refactor(ai): canonicalize truenas host resource types

2026-03-30 13:34:08 +01:00

routing_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

routing.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

sanitize_test.go

refactor(ai): Remove over-engineered URL discovery service

2025-12-10 08:35:24 +00:00

service_coverage_imp_test.go

Remove hidden discovery provider fallback

2026-06-08 03:48:33 +01:00

service_extended_test.go

refactor(ai): dedupe guest lifecycle snapshot loops and check the tool-loop helper return

2026-07-17 17:29:22 +01:00

service_remediation_test.go

Remove Assistant pre-model context heuristics

2026-05-15 16:28:17 +01:00

service_test.go

Restore release-blocking backend contracts

2026-07-13 21:51:33 +01:00

service_tool_loop_test.go

fix: bound denied model tool loops

2026-07-12 13:29:21 +01:00

service_tool_loop.go

fix: bound denied model tool loops

2026-07-12 13:29:21 +01:00

service_tools_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

service.go

perf(ai): gate and batch Patrol's per-alert auto-resolve model calls

2026-07-17 23:23:38 +01:00

target_host_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00

test_remediation_engine_test.go

feat: Pulse v6 release

2026-03-18 16:06:30 +00:00