Files
AutopilotMonitor/docs/index.md
T
Oliver Kieselbach afa8750cbb feat(mcp): tool-call quality telemetry — soft errors, result size, zero-hit searches, arg rejections
withToolTelemetry now receives the tool args and inspects the RESULT: handlers
never throw (toolError returns isError:true), so the previous isError flag was
effectively always false. Each MCP_TOOL_LOGGING line now carries soft-error
status, resultChars + overCap (vs. the anthropic/maxResultSizeChars hint),
a size-bounded args summary and the caller scope — the only telemetry that
covers backend-free tools (search_docs, validate_rule, get_resource).

New signals: search_zero_hit on the three search tools (unmet-demand signal)
and tool_call_rejected for JSON-RPC-level Zod/unknown-tool rejections, sniffed
at the /mcp route since the SDK answers those before any handler runs.

Bicep default toolLoggingEnabled=true (desired state; live flag via
az containerapp update). Docs: docs/mcp/tool-telemetry.md with KQL examples.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 19:16:35 +02:00

8.8 KiB
Raw Blame History

okf_version
okf_version
0.1

Autopilot Monitor — Technical Knowledge Bundle

Contributor-facing technical documentation for Autopilot Monitor, organized as an Open Knowledge Format (OKF) knowledge bundle. Customer-facing product documentation lives at https://docs.autopilotmonitor.com (separate repository).

Architecture

  • Architecture Guide - High-level architecture overview and solution layout for contributors.
  • Version Contract - One version shape for agent, backend and MCP: the ETag-CAS counter blob that mints build numbers, and the manifests published only after a deploy is verified live.
  • URL Registry - Every well-known host lives in one registry file per component (Constants.cs / utils/config.ts / src/config.ts), enforced by guard tests; includes the agent-download migration to the download alias and its deploy-order constraint.

Agent

  • V2 Agent - How the on-device agent works: runtime overview, decision engine (reducer, signals, completion arms), logs & persistence (signal log, snapshot, crash recovery), Hello wizard un-skip (why a single "WHfB disabled" read never decides completion), Autopilot ZTD diagnostics (event IDs, diagnostic registry, endpoints, error-code map).
  • Build Counter Blob - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
  • Agent Endpoint Migration - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
  • Gather Rule Debug Log - EnableGatherRuleDebugLog (ConfigVersion 36): the per-tenant local trace file that explains every gather-rule evaluation decision — scope skips, on_change suppression, empty collector results, logparser position/parse-mode outcomes — for rules that produce no timeline events.
  • MDM Reboot Coalescing - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
  • Secure Boot CA-2023 Detection - Why ANALYZE-SEC-001 v3 reads the UEFI db/KEK variables directly instead of trusting the SecureBoot\Servicing registry, and how the one-sided uefiCa2023FirmwareConfirmed marker plus a not_exists precondition give absence-tolerant suppression across old and new agents.
  • ESP Policy-Provider Stall Detection - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.

Rules

  • Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
  • Gather Rule Guardrails - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
  • Rule Authoring Surface - How customer AIs author rules against the real contract: the baked MCP resources (guide/schemas/guardrails via combine.js), the MCP-local validate_rule pre-flight, and the side-effect-free POST rules/analyze/dryrun whose trace twin shares the production evaluators (parity + side-effect-freedom pinned by tests).

Backend

  • Business Timestamps - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
  • OOBE Bootstrap go-Code Flow - The bootstrap short-code lifecycle (portal-created session → irm go.autopilotmonitor.com/CODE | iex at Shift+F10 → GetBootstrapScriptFunction), the always-HTTP-200 text/plain contract, the two-line injection defense (GUID/URL/expiry validator + single-quoted PS literals, golden-pinned against the legacy TS template), and the per-IP rate-limit story behind Front Door.
  • Device Journeys & First-Time-Right - How a device's terminal enrollment attempts are grouped into journeys (trim+casefold device key with junk-serial exclusion, first-success end, 30-day gap, WhiteGlove = one attempt), the DeviceHistories chain with its tombstone-driven deleted-session pruning, the additive daily FTR aggregates, and the banner/fleet/MCP surfaces.
  • Enrollment Time Attribution - How a terminal session's authoritative wall clock (DurationSeconds, WhiteGlove pause excluded) is partitioned into named segments with an explicit unattributed remainder, ESP-blocking apps as positive evidence with "up to" what-if bounds, reboot spans from event gaps, and the breakdown/aggregate tables the sweep maintains.
  • Rule Regression Radar - How analyze-rule hit-rate regressions are detected (7-day window ≥2× the 28-day baseline with disjoint Wilson intervals), which suppressions keep the radar quiet (grace period, edit-in-window, kill switch), how alert episodes live in the ruleregression tracker keyspace with re-arm semantics, and the on-fire dimension correlation with its "correlated, not necessarily causal" wording contract.
  • Tenant Activation & the Once-Only Welcome Mail - How a signup becomes an activated tenant (manual approve or the auto-approve queue), and the write-then-read + conditional-marker arbitration that sends the welcome mail exactly once even though activation and the user-typed notification address race each other.
  • ServerActions & On-Demand Diagnostics - The queue-on-session, deliver-on-ingest action channel, the TenantAdminOrOperator tier with in-function per-type re-gating, the portal's Collect Logs flow (quick-config orders rotate_config before request_diagnostics), and the hosted-prefix sweep in the offboarding cascade.

MCP Server

  • MCP Docs Corpus - How the published customer documentation becomes the search_docs corpus: CRLF-safe heading chunking, build-time embedding as a third precomputed section, and why a docs edit needs an MCP redeploy.
  • MCP Tool Telemetry - The three telemetry layers for judging tool quality: backend App Insights via X-MCP-Tool-Name, the per-user usage table, and the MCP_TOOL_LOGGING stderr lines (soft errors, result size vs. cap, zero-hit searches, Zod rejections).

Web Portal

  • Portal Navigation Prefetch & the SWA Runtime Bottleneck - Why every navigation Link sets prefetch={false}: the App Router's viewport prefetch fans out one RSC request per visible link and the Static Web Apps SSR runtime queues them server-side into a multi-minute tail. Records the follow-up measurement (the Standard plan changed nothing; the prefetch storm was an accidental keep-alive), where browser telemetry actually lives, and why this curl build cannot be trusted for HTTP versions.

Security & Identity

  • MCP OAuth Flow - Who authenticates where when connecting an AI client to the MCP server; two identities, three parties plus Entra ID.

Conventions for this bundle

  • Every concept document is a markdown file with YAML frontmatter; type is mandatory, title, description, resource, tags, and timestamp are recommended.
  • Use standard relative markdown links between documents (e.g. agent/overview.md, ../architecture.md). Do NOT use the spec's /-prefixed bundle-absolute form — GitHub resolves those against the repo root (missing /docs) and navigation breaks.
  • Record notable additions and changes in log.md.
  • index.md and log.md are reserved names — never use them for concept documents.