AutopilotMonitor/docs/log.md

Log

2026-07-21

• Update: Added scripts/eval-docs-search.ts, an offline retrieval evaluation harness (2465 queries in three classes: chunk headings as a regression detector, rare identifiers, and hand-written questions as the reality anchor), and used it to close a third search_docs gap found during the live test. The keyword fallback only fired when semantic search returned fewer than topK; a pasted identifier like WOW6432Node instead returned three chunks at ~0.29 that did not contain the word while the one chunk that did was never considered. It now also fires on a weak semantic head (<0.35), letting exact matches take the front while keeping one semantic result. The threshold is measured, not guessed: 0.35 fixes 335 queries and breaks 0 (identifier recall 66.2% → 81.1%), while 0.40 fixes more but starts demoting hand-written questions. The measurement also resized the problem — identifier queries were only succeeding 66.2% of the time, so the reported case was one of roughly 750.

• Update: Closed the two measured recall gaps in search_docs (mcp/docs-corpus.md). (1) Token windows — 87 of 324 chunks exceeded the model's 256-token context and 13% of all tokens never reached the ranking; a character budget cannot fix this (the corpus ranges 2.43–5.81 chars/token), so long text is now embedded as 250-token windows with 40 overlap and scored by its BEST window (embeddings: number[][], max not mean). Token coverage 87% → 100%; "proxy configuration blocking app downloads" went from absent to 0.559 at rank 1; the ten reference queries kept byte-identical scores. Costs +31% vectors, index 2.63 → 3.12 MB. The rules corpus gained the same treatment (128 rules → 172 vectors). (2) Keyword fallback — when the semantic pass returns fewer than topK, results are topped up via literal lexicalMatch on distinctive query terms, requiring two matching terms for corroboration (one incidental substring match returned a page about LLM providers for a sourdough query). Fuzzy matching was measured and rejected: at a threshold low enough to surface "RealmJoin" it invents 5 hits for "SCEPman", which the corpus never mentions. Also sharpened the tool description with the measured context argument for preferring it over fetching the docs site.

• Creation: Added mcp/docs-corpus.md (new mcp/ section) — the MCP server gains search_docs, a third semantic corpus built from the published customer documentation repository (autopilotmonitor-docs, checked out into the Docker build context by the deploy workflow). Documents the h2/h3 + <details> chunking with breadcrumb prefixes, the two deliberate size exceptions (tables never split, unbroken changelog lists split at line level), the CRLF precondition (. does not match \r in JS — headings silently stop matching, which produced 286 wrong-but-plausible chunks before it was found), the optional docs section in search-index.json and its validation asymmetry, embedding rounding to 6 decimals, and the cross-repo coupling: a docs edit reaches clients only after an MCP redeploy, made checkable via DOCS_COMMIT on /health. Container resources raised to 0.5 vCPU / 1.0 GiB for the cold-start query-embedder load.

2026-07-20

• Update: rules/gather-rule-phase-scoping.md — the gather-rule form now shows the phase-scope and emit-mode controls only where they can change behavior (supportsPhaseScope / supportsEmitMode), nulls hidden-control state out of the payload, and rejects a scope mode selected but left unfilled. Offering all three "when" controls unconditionally allowed combinations that could only suppress a one-shot rule, plus a silent no-op state.

• Update: Extended rules/gather-rule-phase-scoping.md with the phase_exit trigger — the closing bookend to phase_change, for one-shot collection when a phase is LEFT (e.g. "state at the end of Account Setup"). Documents the evaluation-before-phase-advance rule (triggerPhase match and scope gate see the phase being left), firing into Failed, the disjoint exit: dedup key space, and why no ConfigVersion bump is needed (no new field — old agents simply never fire the rule).

• Update: Reworked agent/mdm-reboot-coalescing.md after the first real enrollment (session b2e890c1): 2800 records are now aggregated per burst into ONE neutral Info event (rebootUris[]/uriCount/firstRebootUri; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed system_reboot_detected. Removed the Shell-Core esp_reboot_coalescing event entirely: the "RebootCoalescing" token appears in the routine SubcategoryProcessing_Started bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.

• Creation: Added agent/mdm-reboot-coalescing.md — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new MdmRebootPolicyTracker (DM-Enterprise EventID 2800, per-URI mdm_policy_reboot_required events, cross-restart RecordId watermark), a RebootCoalescing branch in ShellCoreTracker (esp_reboot_coalescing, backfill-emitting by design), and the ANALYZE-ESP-005 advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from system_reboot_detected so RebootCount stays uninflated.

• Creation: Added backend/business-timestamps.md (new backend/ section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the OccurredUtc column, the RowKey decoders and RowKey-range date/retention clauses in BusinessTimestamp, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.

• Creation: Added rules/gather-rule-phase-scoping.md (new rules/ section) — gather rules gain activePhases (run only during listed phases), activeFromPhase (sticky latch from a phase onwards, never via Failed) and emitMode: "on_change" (canonical-hash result dedup with suppressedPolls observability), ConfigVersion 34. Includes the prerequisite backend fix: the custom-rule toggle PUT no longer wipes the stored rule definition (partial-merge in GatherRuleService.UpdateRuleAsync).

2026-07-18

• Creation: Added agent/endpoint-migration.md — config-channel agent re-home (MigrateToApiBaseUrl on GET agent/config, ConfigVersion 33): backend or per-tenant API-URL moves without DNS indirection (impossible due to mTLS/Flex-TLS). Live-fetch-only with cache strip, AgentEndpointMigrationRules allowlist validated on both sides, one hop, kill-switch wins. Built as the lesson from the 2026-07-18 WEU→GWC cutover, which hard-cut all in-flight agents.

2026-07-17

• Creation: Added agent/build-counter-blob.md — agent build numbers now come from a shared public counter blob (buildcounter-v{N}.txt, reserve-before-build with ETag-CAS) so local build.ps1 and CI build-agent.yml builds can never mint the same number. Documents the CI parity fixes (SummaryDialog.exe.config in the ZIP, EXE-vs-manifest version check, hash-oracle update gated on stable cutover) and the Sigstore provenance attestation on CI-built ZIPs.

2026-07-16

• Creation: Added agent/autopilot-ztd-diagnostics.md — reference for Windows' own ZTD diagnostic surfaces (ModernDeployment Autopilot event IDs incl. 807/809/815/908, the Diagnostics\Autopilot registry key, deployment-service endpoints, known-issue error-code map with KB fixes). Backs the agent's ZtdEvidence collector (ztdVerdict on autopilot_profile_missing) and the backend known-issue rules; sources carry a re-check RSS feed.

2026-07-13

• Removal: Moved customer-graph-add-on-permissions.md out of this contributor bundle into the customer documentation repository (reference/optional-graph-permissions.md). It was customer-facing product content and did not belong in the tech-docs bundle; removed the entry from index.md.

• Creation: Added agent/hello-wizard-unskip.md — session 772fe502 fix: the HelloWizardStarted decision signal (prevention veto + un-skip cure for the policy-disabled Hello shortcut) and the HelloTracker confirmation second read against flip-flopping WHfB CSP values.

2026-07-11

• Creation: Added the agent/ section — three concept documents on the V2 agent: agent/overview.md (runtime, collectors, telemetry pipeline), agent/decision-engine.md (DecisionCore reducer, signals, A/B/C completion arms, invariants), agent/logs-and-persistence.md (persisted artifacts, signal log vs spool, recovery/replay flow) plus agent/index.md.

• Creation: Converted docs/ into an OKF v0.1 knowledge bundle — added YAML frontmatter to architecture.md, mcp-oauth-flow.md, and customer-graph-add-on-permissions.md; created index.md and this log.