AutopilotMonitor/docs/rules/rule-authoring-surface.md

type, title, description, resource, tags, timestamp

type	title	description	resource	tags	timestamp
Concept	Rule Authoring Surface — MCP Resources, validate_rule, and the Analyze Dry-Run	How customer AIs author gather/analyze rules against the real contract: the baked authoring resources (guide, schemas, guardrails) generated by combine.js, the MCP-local validate_rule pre-flight, and the side-effect-free backend dry-run that traces a draft analyze rule against one session's events.	src/McpServer/autopilot-monitor-mcp/src/tools/rules.ts	rules mcp analyze-rules gather-rules dry-run authoring	2026-08-05

Concept

Customers author rules with an AI connected to the MCP server. Retrieval (search_docs) is deliberately NOT the knowledge source for authoring: chunked top-K retrieval can silently omit an operator, a guardrail prefix, or an enum, and the AI then produces a plausible-but-dead rule. The authoring surface is therefore deterministic: three get_resource entries carry the complete contract, a local validator rejects what production would reject or silently ignore, and a backend dry-run evaluates a draft against real session events with a full trace.

The intended loop: draft → validate_rule → test_analyze_rule against a real session (one that should fire, one that should not) → iterate → create in the portal. The author-rule MCP prompt walks a client through exactly this order.

Schema

Resources (deterministic authoring context)

Resource	Content	Source
rule_authoring_guide	Hand-written complete authoring knowledge: collector types, triggers, condition sources, operators, confidence model incl. the exact factor DSL, {{token}} interpolation order, guardrail matching semantics, pitfalls	src/rule-authoring-guide.ts
rule_schemas	Both JSON Schemas (2020-12) verbatim	generated
rule_guardrails	rules/guardrails.json verbatim	generated

Generation: node rules/scripts/combine.js bakes schemas + guardrails into src/McpServer/autopilot-monitor-mcp/src/rule-authoring.generated.ts (same pattern as the web's utils/guardrails.generated.ts; the tsconfig rootDir and the Docker build's exclusion of rules/schema rule out importing the JSON directly). Drift is closed twice: CI's guardrails-in-sync job re-runs combine.js and fails on any diff, and rule-authoring-drift.test.ts pins generated == repo source plus guide enums == schema enums.

validate_rule (MCP-local)

src/rule-validation.ts, no backend call. Three layers:

1. Schema — ajv (draft 2020-12) against the baked schemas, validating the single-rule $defs entry directly (the top-level oneOf wrapper would double every error).
2. Guardrails — mirrors the AGENT's matching semantics from GatherRuleGuards.cs (see gather-rule-guardrails.md): hive-stripped segment-bounded registry/file prefixes, whitespace-bounded WMI prefixes, EXACT command matching, /-bounded event-log channels, and the code-enforced hard blocks (Security etc. channels, C:\Users, C:\Windows\System32\config).
3. Semantic lint — schema-valid-but-dead rules: unreachable confidenceThreshold, factor conditions outside the three parseable shapes (exists, count >= N, phase_duration > N), unknown event types (warning; gather_* softened to info), unanchored not_regex allow-lists, unresolvable {{token}}s, trigger cross-fields, outputEventType convention/collision checks.

The validator is advisory pre-flight only — the agent re-enforces guardrails on-device and the backend re-validates structure. It must never be stricter than production.

POST /api/rules/analyze/dryrun (backend)

DryRunAnalyzeRuleFunction → RuleEngine.DryRunRuleAsync. Body { sessionId, rule } (draft AnalyzeRule, 1 MB cap). Policy tier TenantAdminOrGlobalReader — a POST by shape but a pure read in effect, so the read-only Global Reader is admitted; platform-scope callers reach any tenant's session via FindSessionTenantIdAsync (same pattern as GetRuleResultsFunction).

RuleEngine.DryRun.cs is a partial-class trace twin of EvaluateRule: it shares the exact evaluators (EvaluatePrecondition / EvaluateCondition / EvaluateConfidenceFactor) but evaluates all conditions instead of breaking on the first failed required one — the full picture is the point of a dry run. Verdicts: fired, skipped_by_precondition, required_condition_not_met, no_conditions_matched, below_confidence_threshold, no_events. Each condition entry carries matched, the production evidence dictionary (or the evaluator's reason string) and matchingEventCount — how many session events even have that eventType, the first thing to check on an unexpected miss.

Invariants, pinned by RuleEngineDryRunTests:

• Parity: verdict fired ⇔ EvaluateRule produces a result, with identical confidence and evidence keys. Factor evaluation stays sequential against the accumulating matchedConditions (a factor can observe earlier factor_* entries) — reordering would change results.
• Side-effect freedom: exactly one strict event read; never a RuleResult write, never TryMarkSessionFailedFromRuleAsync, never a notification. wouldMarkSessionAsFailed is reported, not acted on.

test_analyze_rule (MCP) wraps the endpoint and adds the interpolated explanation/remediation preview via the existing interpolate-rule-template.ts against the returned matchedConditions.

POST /api/rules/gather/test-pattern (backend)

The logparser counterpart to the analyze dry-run, added after a field incident: a customer verified a logparser regex with a PHP tester and the agent's .NET engine matched differently. TestLogPatternFunction evaluates a pattern against pasted sample lines with the AGENT's exact matching semantics:

• Regex construction identical to LogParserCollector: Compiled, 1 s timeout, no IgnoreCase — logparser matching is case-sensitive (unlike analyze-rule regex conditions, which run IgnoreCase).
• format behaves exactly like the rule parameter: cmtrace (default) parses each line via the shared CmTraceLogParser and matches against the parsed MESSAGE only; text matches the raw line. First match per line (Regex.Match), group "0" excluded, unsuccessful groups omitted — the returned groups are exactly the event-data fields the device would emit.
• Mirrors the agent's diagnostics: per-line outcomes (matched | no_match | parse_failed | regex_timeout), the all-lines-parse-fail hint ("set parameter format=text") verbatim, and a case-sensitivity note on zero matches.
• Pure compute on the request body — no tenant data touched. Same policy tier as the dry-run (TenantAdminOrGlobalReader); caps: 200 lines, 8 KB/line, 1 MB body.

Parity is structural, not duplicated: CmTraceLogParser moved from the agent (Monitoring/Enrollment/Ime) to Shared (AutopilotMonitor.Shared.Logging) so agent and backend run the SAME parsing implementation. Timestamp caveat: bias-less CMTrace lines parse with AssumeLocal, which on the backend is the server's timezone — the endpoint therefore reports match/parse results, never timestamps. test_log_pattern (MCP) wraps the endpoint; validate_rule additionally lints logparser parameters (missing/non-compiling pattern, unknown format value) and its nextStep routes logparser drafts to test_log_pattern.

What is deliberately absent

• No gather-rule remote execution — gather rules run as SYSTEM on devices; session dry-run is analyze-only. Logparser authors get the pattern tester above; all gather authors get guardrail pre-flight plus the on-device debug log (gather-rule-debug-log).
• No rule write/deploy via MCP — creating rules stays in the portal.

Citations

• src/Backend/AutopilotMonitor.Functions/Functions/Rules/TestLogPatternFunction.cs — logparser pattern tester
• src/Shared/AutopilotMonitor.Shared/Logging/CmTraceLogParser.cs — shared CMTrace parser (agent + backend)
• src/Backend/AutopilotMonitor.Functions/Services/RuleEngine.DryRun.cs — trace twin + DTOs
• src/Backend/AutopilotMonitor.Functions/Functions/Rules/DryRunAnalyzeRuleFunction.cs — endpoint + draft validation
• src/McpServer/autopilot-monitor-mcp/src/rule-validation.ts — validator
• src/McpServer/autopilot-monitor-mcp/src/tools/rules.ts — validate_rule / test_analyze_rule
• src/McpServer/autopilot-monitor-mcp/src/rule-authoring-guide.ts — guide resource
• rules/scripts/combine.js — MCP module generation