Compare commits

..

34 Commits

Author SHA1 Message Date
Oliver Kieselbach 3f9576d4a9 fix(agent): enforce the gather-rule allowlist in every collector
LogParserCollector and EventLogCollector called no guard at all, while
File/Json/Xml did — and rules/gather-rules.md already promised customers
that log-parser paths were allowlisted. A tenant admin could author a
logparser rule against C:\Users via POST /api/rules/gather and have up to
1000 lines uploaded as telemetry, bypassing the hard C:\Users block the
same page guarantees; eventlog could read any channel, Security included.
The backend validates no collectorType, target, or parameters, so the
agent is the only place this can be enforced.

Both now guard. logparser cannot check rule.Target directly (GetFullPath
throws on a wildcard), so it checks the directory for wildcard targets and
re-checks each resolved path. Event log channels get an allowlist matched
on a "/" boundary. Hard-blocked channels live in a C# constant rather than
guardrails.json: a parse error drops the allowlists to empty, which is
fail-closed, but would have lifted a block sourced from the same file.

Also fixes an inert timeout in CommandCollector: ReadToEnd() blocks until
the pipe closes, so WaitForExit(30000) was only reached once the process
had already exited and a hung command pinned its worker indefinitely.

Adds the first real CI gate. No workflow ran tests on a pull request — the
deploy workflows build without testing and build-agent.yml covered only
DecisionCore.Tests on manual dispatch — while trust/security-faq.md
already told customers automated tests run in CI. ci.yml now runs all six
suites on every PR and merge, plus a job that fails on generated-guardrail
drift, alongside CodeQL and Dependabot. OfficeBinaryWatcherTests is marked
SerialThreading; its bounded re-probe races the shared ThreadPool and
failed once under full-suite load.

Verified: 3101 backend, 1911 agent V2, 474 decision core, 44 dialog, 572
web, 447 MCP. The timeout fix is verified by runtime, not assertion — a
Start-Sleep 120 command now returns after ~30s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 00:28:04 +02:00
Oliver Kieselbach ee58242d33 fix(mcp): tell callers to query the English-only corpora in English
A German question returns zero results from search_docs, not poor ones: both the
documentation and the rules are English, and all-MiniLM-L6-v2 is an English-only
model, so the query vector lands nowhere near any chunk. For a German-speaking
admin driving an MCP client in their own language, the tool is effectively blind
and gives no hint why.

The caller is itself a language model, so the cheapest correct fix is to say so:
both tool descriptions now instruct it to translate the question to English before
searching and to answer the user in their own language. No infrastructure, no
model change, no ranking impact.

A multilingual embedding model would handle it natively and was considered, but it
is substantially larger — which matters on a scale-to-zero container — and it would
change English retrieval as well, so it would have to be measured against
scripts/eval-docs-search.ts rather than swapped in on the assumption that more
languages cannot hurt.

Verified: 447 tests pass, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:42:56 +02:00
Oliver Kieselbach eca528b9b3 docs(mcp): attribute the cold start — the app is 0.5s of it, the platform is the rest
The boot marks answered the question on the first production cold start, and the
answer overturns the reasoning behind an infrastructure change made earlier today.

  +244ms  node started (startup + module loading)
  +431ms  corpora read from disk          (+187ms)
  +491ms  index read, validated, hydrated  (+60ms for 3.12 MB)
  +504ms  listening
  +677ms  query embedder warm             (background)

Against a 31.5s cold start, the application accounts for half a second. The other
~98% is Container Apps activation: KEDA reacting, pod scheduling, mounting the
236 MB image, starting the container. The container also boots faster than a
development laptop (504ms vs 1273ms), so nothing here is CPU-starved.

Consequences recorded rather than quietly dropped:

- Raising the container to 0.5 vCPU was justified by the embedding model load on
  the cold-start path. Prebaked, that load is 173ms and runs in the background
  where only the first search could wait on it. The justification does not survive
  measurement. The setting stays — it plausibly helps concurrent requests, and at
  scale-to-zero the cost difference is negligible — but the Bicep comment now says
  what is true instead of what was assumed, so nobody re-derives the wrong reason.
- Index size is irrelevant to startup (60ms for 3.12 MB), so the corpus can grow
  freely, and the embedding rounding earns its keep on file size alone.
- The only lever that would actually remove the cold start is minReplicas: 1, at
  roughly 7x the monthly free consumption grant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:34:24 +02:00
Oliver Kieselbach 8305f5c850 feat(mcp): time the boot so the cold start can be attributed
A cold start after scale-to-zero measures 24.5s to first byte against 60ms warm,
and nothing outside the process could say why. Log Analytics stamps every console
line of one ingestion batch with the same time, so the boot sequence collapses to
a single instant however it is queried, and piping stdout locally buffers until
exit. So the server times itself: `[boot +Nms]` marks derived from
process.uptime(), which unlike an in-process timer also covers Node's own startup
and module loading.

Measuring the precomputed path locally already contradicts two assumptions worth
recording:

  +860ms   node started (startup + module loading)
  +1199ms  corpora read from disk         (+339ms)
  +1242ms  index read, validated, hydrated (+43ms for 3.12 MB)
  +1273ms  listening                       (+31ms)
  +1497ms  query embedder warm             (+224ms, background)

The index is 43ms, not seconds — its size is not the cold-start cost, and growing
the corpus is cheap. The prebaked embedder is 224ms and loads in the background,
so only the first search could ever wait on it. That was the stated reason for
raising the container to 0.5 vCPU, which now looks like a weaker justification
than it did; the production marks will settle it. If they show the app listening
after a few seconds, the rest of the 24.5s is platform activation that neither CPU
nor index changes can touch.

Verified: 447 tests pass, marks appear on both boot paths locally. Incidentally
the run also confirmed the corpus hash guard and the staleness signal both work —
the local docs repo has moved to 6333d56 while the deployed image reports d8cb7f4,
so the precomputed index was correctly rejected and recomputed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:14:49 +02:00
Oliver Kieselbach c2c02b929c fix(web): show gather-rule scope/emit controls only where they change behavior
The form offered trigger, phase scope and emit mode unconditionally, which
produced combinations that cannot do anything plus one silently wrong state:

- A scope on a Phase Start / Phase End rule that names a concrete phase can only
  ever suppress the single firing — the phase IS the moment. Same for emit mode:
  a one-shot has no second result to dedup against.
- Selecting "From a phase onwards" (or "Only during specific phases") and not
  filling the detail dropdown serialised to null, i.e. the rule ran everywhere
  while the form still read "From a phase onwards".

supportsPhaseScope / supportsEmitMode now drive visibility; buildScopeFields
nulls hidden-control state so a value from an earlier trigger choice cannot leak
into the payload; validateScopeSelection rejects an unfilled scope mode on both
create and save. Startup is the deliberate exception — a scope there defers the
one-shot instead of suppressing it, so the scope control stays and only emit mode
is hidden. One-shot triggers now state in-form when they collect.

Sub-dropdown hints no longer claim "No selection = all phases" (they turn amber
and name the required action instead).

Tests: app/gather-rules/__tests__/gatherRuleFormLogic.test.ts (15) pins the
applicability matrix, the payload nulling incl. the trigger-switch leak, the
unfilled-scope rejection, and the JSON-mode scopeMode derivation.
vitest 572/572; tsc silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:06:50 +02:00
Oliver Kieselbach 1821af6f05 fix(mcp): fire the search_docs keyword fallback on a weak semantic head
The live test found a third recall gap. The fallback only topped up when semantic
search returned FEWER than topK results — but the more common failure is a full
list of unconvincing ones. Querying "WOW6432Node" returned three chunks scoring
~0.29, none containing the word, while the single chunk that does was never
considered at all.

The fix is a ranking change, so it was measured rather than reasoned about. Adds
scripts/eval-docs-search.ts, which scores strategies over 2465 queries in three
deliberately different classes: every chunk heading (easy by construction, so a
regression detector), rare identifiers (the class this fallback serves), and
hand-written questions (the reality anchor — not derived from the corpus, so it
shares little vocabulary with it). Embeddings are computed once and each strategy
is a pure function over the same candidates, so sweeping a threshold costs no
extra inference.

                        known-item  literal  handwritten MRR  fixed  broken
  append-only (before)       71.9%    66.2%            1.000      -       -
  weak < 0.30                71.9%    72.6%            1.000    142       0
  weak < 0.35                73.7%    81.1%            1.000    335       0
  weak < 0.40                76.3%    88.2%            0.944    502       3
  weak < 0.45                77.2%    93.0%            0.889    613       5

0.35 is strictly dominant — better in every class, worse in none — so that is
what ships. Higher thresholds keep repairing identifier queries but start
demoting hand-written questions, the class that most resembles real use, so the
gain stops being free. When the head is weak, exact matches move to the front
while at least one semantic result is kept, so a weak-but-correct hit is never
pushed out entirely.

The measurement also resized the problem: identifier queries were succeeding only
66.2% of the time, so the reported case was one of roughly 750, not an outlier.
That is the part a single anecdote could not have shown, and the reason the
harness exists before the fix rather than after it.

Verified: 447 tests pass, including new cases for the weak path and for leaving a
confident ranking untouched. On the real corpus "WOW6432Node" now returns the
chunk that contains it at rank 1, while "how do I deploy the agent with Intune"
(0.819) and "what firewall URLs do I need to allow" (0.374) are unchanged and the
negative controls stay empty.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:39:26 +02:00
Oliver Kieselbach 62599d972c fix(mcp): put every doc token back into search_docs ranking
Two measured recall gaps in search_docs, both silent.

1. Truncation. MiniLM's context is 256 tokens; 87 of 324 chunks exceeded it and
13% of all tokens never reached the ranking — the tokenizer drops the tail
without complaint. Concretely, in built-in-rules.md :: Apps (734 tokens) the
strings ANALYZE-APP-013, ANALYZE-OFFICE-001, ANALYZE-CORR-003 and Click-to-Run
all sit past the cutoff, so "proxy configuration blocking app downloads" did not
retrieve that section at all.

Shrinking chunks cannot fix this: the corpus ranges 2.43 to 5.81 characters per
token, so no character budget bounds tokens, and a budget safe for the worst case
(~620 chars) would shred prose and tables alike. So the chunk stays whole and the
embedding is split instead — index() cuts long text into 250-token windows with
40 overlap, a document carries embeddings[][], and search() scores it by its BEST
window. Max rather than mean: averaging would blur a long section's several
topics into one mediocre direction, the same dilution that makes large context
windows worse than chunking.

The tokenizer is a build-time dependency only (vocabulary, not ONNX weights,
~93 ms); production boots from precomputed vectors and never loads it.

2. No general fallback. Proper nouns embed poorly — "RealmJoin" occurs in 3
chunks yet scores under the threshold. Fuzzy matching was built, measured and
rejected: at a threshold low enough to surface RealmJoin (0.28), Fuse also
returns 5 hits for "SCEPman", a word this corpus contains zero times. Literal
containment has neither problem, and every provider already exposes it. So a
short result set is now topped up via lexicalMatch on distinctive query terms,
requiring two matching terms as corroboration — without that rule "how do I bake
sourdough bread" matched a page about LLM providers, because "bake" is a
substring of "baked into the image". Keyword hits are appended and flagged, never
sorted together with cosine scores.

Also sharpens the tool description with the measured reason to prefer this over
fetching the docs site (~1k tokens for topK=3 against ~9k raw / ~600k rendered,
version-matched, and matching on meaning), and replaces the growing tail of
positional docs parameters with a DocsSearchBundle.

Verified: token coverage 87% -> 100%; "Office Click-to-Run install never
finished" 0.318 -> 0.442; "proxy configuration blocking app downloads" absent ->
0.559 at rank 1; the ten reference queries keep byte-identical scores and rank 1
(short chunks are one window, so the same vector as before — no long-chunk bias).
The fallback is not a dead path: across 904 rare corpus tokens queried alone it
contributes hits for 431, while negative controls stay empty. Precompute output
is accepted by the boot validator and a mutated corpus still rejected; index
2.63 -> 3.12 MB for 324 chunks / 424 vectors. The rules corpus gains the same
treatment unplanned: 128 rules now carry 172 vectors. 445 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:17:42 +02:00
Oliver Kieselbach 21b751be64 changed sidebar tenant settings entries ordering 2026-07-21 20:02:28 +02:00
Oliver Kieselbach 63ea2e3cee feat(mcp): search_docs — customer documentation as a third search corpus
The MCP server could answer telemetry questions ("why did this session fail")
but not product ones ("how do I deploy the agent", "where is my data stored").
Those answers already exist in the autopilotmonitor-docs bundle; search_docs
makes them semantically searchable.

Deliberately a separate index from search_knowledge: rules and prose answer
different questions, and sharing one topK would dilute both. Registered for
every role — published documentation carries no tenant data — and skipped
entirely when no corpus was baked in, so a doc-less build advertises no
broken tool.

Corpus handling:
- Chunked at h2/h3 and <details>/<summary>, each chunk prefixed with a
  "Page > Section > Subsection" breadcrumb so a mid-page hit is self-describing.
- Newlines are normalized before any parsing. This is a correctness
  precondition, not cosmetics: the bundle ships CRLF and JS `.` does not match
  \r, so /^(#{2,3})\s+(.*)$/ matched zero headings — silently, degrading every
  page to unlabelled paragraph splits.
- Two size exceptions: tables are never split (a split table leaves `| --- |`
  rows with no header), and unbroken changelog lists split at line level (left
  whole, their tails fall outside the model window and become unsearchable).
- Short sections are kept — a one-line answer is often the best hit, and the
  breadcrumb supplies the context a short body lacks.

Build and delivery:
- Docs are embedded at image build time into an optional third
  PrecomputedSection. Optional so a checkout without the docs repo does not
  invalidate the rules and event-type vectors; the reverse (server has docs,
  file does not) is staleness and is rejected.
- The deploy workflow checks the public docs repo into the build context and
  passes its SHA as DOCS_COMMIT, which /health reports alongside the chunk
  count — editing the docs does not update a running server, so staleness is
  made checkable rather than memorable.
- Embeddings are rounded to 6 decimals on serialize: 55% smaller index (2.63
  vs 5.88 MB) for a ~1e-5 norm error against a 1e-3 tolerance. indexPrecomputed
  now asserts unit norm, since the hydrate path never checked it.

Infra: the Bicep template was two changes behind the live deployment (EU
cutover and ACR hardening, both applied directly in Azure). Reconciled against
the running resources — managed-identity image pull replaces the admin
credentials and the acr-password secret — and infra/ is no longer gitignored so
it stops drifting. Container raised to 0.5 vCPU / 1.0 GiB for the cold-start
query-embedder load. The template does not model the mcp.autopilotmonitor.com
custom domain, so a full deployment would remove it; the header warns about it.

Verified: 428 tests pass; 45 pages produce 324 chunks; sample queries hit the
right page top-1/2; all 8 derived URLs return 200 live; the precompute output
is accepted by the boot validator and a mutated corpus is rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:48:28 +02:00
Oliver Kieselbach 93dcac44a4 fix(web): add Contact to the tenant settings sidebar
The section was registered for routing but not in the sidebar, so it was
reachable only by typing the URL. Sidebar entries come from
globalNavConfig.tsx, which is not derived from tenantNavSections.ts -
both have to be touched for a section to exist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:19:44 +02:00
Oliver Kieselbach 72be8ac89f feat(config): tenant contact address, seeded once from the preview email
Enforcement and service notices need somewhere to go. Until now the only
address on file was the Private Preview notification email, which is
onboarding data in the PreviewWhitelist table and not something a tenant
can maintain.

Adds TenantConfiguration.ContactEmail, owned and edited by the tenant's
own admins under Settings -> Tenant -> Contact:

- Seeded once when a preview notification address is saved, and only
  while ContactEmail is still empty. It is never re-synced, so an edit in
  the portal cannot be overwritten later by the onboarding source.
- Backfilled for tenants onboarded before the field existed, as a
  manual-trigger maintenance task alongside the other backfills. It skips
  any tenant that already has an address, so it converges to a no-op and
  can be dropped once everyone is covered.
- Surfaced read-only in Admin -> Tenants next to the preview address, so
  an operator who needs to warn a tenant does not have to read the table.

Purpose is stated where it is entered and in the Privacy Policy: service
matters only - a technical problem, a security matter, or a change
needing an administrator's attention. Never marketing, never shared.

The Terms already say a warning is sent "where a tenant contact address
is on file"; this is the field that makes that promise meaningful.

Wired through the full tenant-config round-trip (types, context state,
hydrate, save payload, reset, provider) so a save from any other section
cannot clobber it.

Verified: solution builds clean, 3101 backend tests and 557 web tests
pass, tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:00:18 +02:00
Oliver Kieselbach cd570c8b3e feat(web): add IP/licence terms, enforcement ladder, security reporting
- Intellectual Property & Licensing section: the repository licence
  governs the code, not the hosted service, its infrastructure, its data,
  or the name and branding. Customer data stays with the customer.
- Acceptable Use now states the enforcement ladder (warning, reduced rate
  limits, suspension, termination) instead of an unqualified right to
  suspend, and routes security research to GitHub Security Advisories
  with explicit testing boundaries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:38:52 +02:00
Oliver Kieselbach 6dfc564be6 fix(web): align privacy and terms with the DPA-owns-the-register split
Privacy: "Sub-processors and Outbound Data Flows" becomes "External
Services". It still names who receives data, which is a controller's own
duty, but drops the Art. 28 vocabulary and the promise to announce new
sub-processors - that commitment lives in the data processing agreement.
Points at the renamed docs page and notes the DPA is available on
request.

Terms: the MSP clause now says a managing organization is responsible for
having the necessary data protection agreements with its customers,
instead of classifying it as a processor or sub-processor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:25:36 +02:00
Oliver Kieselbach afecd18123 fix(web): patch js-yaml and brace-expansion DoS advisories
Dependabot alerts #63/#64/#65 (all high): brace-expansion exponential-time
expansion of consecutive non-expanding {} groups (GHSA-3jxr-9vmj-r5cp,
both the 1.x and 2.x trees) and js-yaml quadratic CPU consumption via
YAML merge-key chains (GHSA-52cp-r559-cp3m).

All three are transitive dev-only dependencies of the eslint toolchain
(eslint -> @eslint/eslintrc -> js-yaml; eslint/typescript-estree ->
minimatch -> brace-expansion) and resolve within their existing semver
ranges, so only the lockfile changes.

npx tsc --noEmit clean, vitest 557 passed, npm audit reports 0
vulnerabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 15:53:28 +02:00
Oliver Kieselbach 5a294ac4f6 fix(mcp): patch transitive protobufjs/tar/body-parser advisories
Dependabot alert #62 (protobufjs 7.6.4, DoS via infinite loop in .proto
option parsing, GHSA fixed in 7.6.5) plus the tar (critical) and
body-parser (low) advisories npm audit surfaced alongside it. All three
are transitive under @huggingface/transformers and resolve within their
existing semver ranges, so only the lockfile changes.

Lockfile also picks up the pending 1.5.0 -> 1.5.1 package version sync.

npx tsc --noEmit clean, vitest 400 passed / 41 skipped, npm audit
reports 0 vulnerabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 15:50:51 +02:00
Oliver Kieselbach dbf6e50e2d feat(agent,web,shared): phase_exit gather trigger — one-shot collection at a phase's end
Follow-up to gather-rule phase scoping. One-shot collection at a phase START
already existed (phase_change dedups per (rule, phase)) and at enrollment END
(on_event + enrollment_complete); the END of a specific phase had no trigger.

New trigger value "phase_exit": fires once when TriggerPhase is LEFT (empty =
every phase transition). Together with phase_change it forms the two one-shot
bookends of a phase.

Agent (GatherRuleExecutor.OnPhaseChanged):
- Exit rules are evaluated BEFORE _currentPhase advances, so both the
  TriggerPhase match and the phase-scope gate see the phase being LEFT. A rule
  scoped to DeviceSetup therefore fires on its own exit even though the phase
  being entered is out of scope.
- A transition into Failed fires the exit rules of the failing phase — the
  snapshot at a failure boundary is the point of the trigger.
- Exit of Unknown never fires; a repeated identical phase signal is not a
  transition.
- Dedup key "exit:{ruleId}|{phase}" shares _phaseRulesExecuted with the
  phase_change keys but stays in a disjoint key space, so an enter-rule and an
  exit-rule on the same phase cannot consume each other's slot.

Portal: trigger labels are now "Phase Start" / "Phase End"; the canonical phase
dropdown and the create/save payloads cover both via PHASE_TRIGGERS; the card
renders "entering X" / "leaving X".

No ConfigVersion bump: no new field is transported. Old agents do not know the
value and simply never fire the rule (silent no-op).

Tests: GatherRuleExecutorPhaseExitTests (11) — enter-vs-exit, no refire on
re-entry, empty-triggerPhase, exit-of-Unknown, same-phase-signal, into-Failed,
Complete, scope-against-left-phase, out-of-scope slot preservation, key-space
disjointness, on_change interplay. Build 0 Errors; V2.Core.Tests 1884/1884;
Functions.Tests 3099/3099; vitest 557/557; tsc silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:37:48 +02:00
Oliver Kieselbach 062962ae41 docs(claude): dedupe and condense agent directives
CLAUDE.md ships with every request. Removed the example from the
operator-only-infrastructure rule - rules state the rule, not an
instance - and applied the same yardstick to the rest of the file.

Merged what said the same thing twice: "would a staff engineer approve"
and "what would a perfectionist reviewer reject"; Demand Elegance and
Code Quality and Core Principles; Task Management's lessons item and the
Self-Improvement Loop; Task Management's plan steps and Plan Mode
Default. Dropped motivational framing that steers no decision, and a
stray heading level.

Every actual rule survives: the 3+ step plan threshold, stop-and-re-plan,
subagent usage, verify-before-done with a diff against main, autonomous
bug fixing, lessons capture, 500 LOC chunked reads, no absence claims
from one grep, the structural-fix mandate, and the Codex review note.

425 words, down from 614 at the start of the session - and that now
includes the customer-facing claims section, which did not exist then.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:15:14 +02:00
Oliver Kieselbach be08660976 docs(claude): condense customer-facing claims and OKF sections
CLAUDE.md ships with every request, so length is a running cost. Cut the
new claims section from 12 lines to 4 and tightened the OKF section,
dropping the spec URL and restating the rules imperatively. Net effect:
the file is now smaller than before the claims section was added
(654 words vs 856).

No rules were removed - verify-before-writing, the change triggers, the
durable-phrasing rule, and the operator-only-infra caveat all remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:09:48 +02:00
Oliver Kieselbach b032005f5b docs(claude): add customer-facing claims to the release checklist
Terms, Privacy, the About page and the trust docs make statements that
customers and security reviewers rely on, but nothing opened them during
a refactor, so they drifted. The July 2026 sweep found a ".NET 8" backend
that was .NET 10, a "Next.js 18" that does not exist, ETL log collection
that never happened, and "each tenant runs its own isolated instance" for
a shared multi-tenant service.

Records which surfaces exist, which kinds of change must update them in
the same commit, and that claims are verified against the code rather
than carried forward.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:01:23 +02:00
Oliver Kieselbach 0ab568f479 fix(web): correct stale About claims; open Community plan to any tenant
Terms: replace the blanket business-use-only clause. It excluded exactly
the wrong group - a registered association is not a consumer under German
law anyway (only natural persons can be), while the clause did shut out
private individuals running their own lab or personal tenant, who are a
real and intended audience for a free community edition. Community is now
stated as open to anyone with an eligible Entra ID/Intune tenant;
business-only stays where it means something, on the Enterprise contract.

About page - claims verified against the code and corrected:
- "each customer tenant runs its own isolated instance" was wrong and
  contradicted the same page two sections earlier. It is one shared
  multi-tenant service with partition-key isolation and delegated MSP
  access.
- Diagnostics: no ETL tracing and no Windows event log export happens;
  the bundle is agent logs, IME logs, agent state and device info. Upload
  is tenant-configured (off/always/on failure) at the end of enrollment,
  not triggered on demand from the portal - the portal downloads it.
- .NET 8 Isolated -> .NET 10 Isolated on Flex Consumption.
- "Next.js 18" -> Next.js 15 (React 18); 18 was React's version.
- Roles: Admin/Operator -> Admin/Operator/Viewer plus delegated MSP.
- Notifications: Teams only -> Teams, Slack, generic JSON webhooks, with
  start/success/failure and SLA triggers. Telegram stays unlisted; it is
  operator-only alerting, not a tenant feature.
- Agent deployment: Win32 app -> Intune platform script; self-destruct is
  on by default, not optional-sounding.
- Dropped the "tamper-evident" audit claim - records are ordinary table
  entities with no hash chain, signing, or immutability policy.

Confirmed while verifying: no tenant-configurable email channel exists
anywhere in the codebase, so the published privacy statement that Resend
serves only the onboarding mail is accurate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 13:58:07 +02:00
Oliver Kieselbach fced858a7a feat(legal,agent): professionalize Terms/Privacy; wire SendTraceEvents gate
Rework the public Terms of Use and Privacy Policy from their early-days
wording to match how the service actually runs today, and fix a setting
that looked like a privacy control but gated nothing.

Legal positioning:
- glueckkanja AG operates the service and is the counterparty on BOTH
  plans (Community/Enterprise is a flag in one system, one subscription).
  Oliver Kieselbach is named as creator and maintainer of the open-source
  project and the Community edition, acting on behalf of glueckkanja AG
  and explicitly not a separate contracting party.
- Community: free, "AS-IS", no availability or support commitment.
  Enterprise: written agreement incl. DPA, higher limits, MSP delegation.
- Added business-use-only clause, simplified governing law (dropped the
  CISG exclusion - it governs sale of goods, not SaaS), split liability
  per plan, added an MSP delegation section and a rules-contribution
  clause covering the community rule pool.
- Imprint links to the glueckkanja AG imprint, whose scope clause covers
  every service linking to it - no duplicated company data to go stale.
  Reachable from the landing footer and both help menus.

Privacy Policy corrections (previous claims were inaccurate):
- "No personal data is transmitted" - backend telemetry carries the UPN;
  only the browser SDK is cookie- and UPN-free.
- "Not shared with third parties" - qualified for MSP delegation,
  customer-configured notification channels, the customer's own LLM via
  MCP, and agent-side geolocation.
- Removed "data subject rights are not guaranteed"; GDPR rights are not
  optional. Added region, sub-processors, retention and IP handling.

Agent - SendTraceEvents gate:
The setting was plumbed from TenantConfiguration through to
AgentConfiguration but never consumed; Trace severity only hides an event
from the timeline view. The outbound IP was therefore always collected
when geolocation is on (which is the default). Gate is now enforced in
TelemetryEventEmitter.Emit, the single choke point before the transport
enqueue. Dropped before the sequence counter advances so suppression
leaves no gaps. Evaluated per event via Func<bool> so a mid-session
remote-config merge takes effect. Fails open: an unwired or throwing
accessor lets trace events flow, because the gate suppresses diagnostics
and must not silently blind the timeline.

Default stays on and the switch remains operator-only via the config
table - deliberately not surfaced in the UI or documentation.

Verified: solution builds clean, 1873 agent core tests pass,
tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 13:42:03 +02:00
Oliver Kieselbach ee947e2ca9 updated docs links 2026-07-20 20:52:56 +02:00
Oliver Kieselbach 5f08a6ec68 updated dev script 2026-07-20 19:52:54 +02:00
Oliver Kieselbach a97888ebd5 fix(agent): neutral aggregated 2800 observations; reboot claim gated on observed reboot
Rework after the first real enrollment (session b2e890c1) exposed three
defects in a654c730:

- Shell-Core "RebootCoalescing" match was a false positive by construction:
  the token appears in the ROUTINE SubcategoryProcessing_Started bootstrap
  marker ("Starting subcategory DeviceSetup.RebootCoalescing...") on EVERY
  enrollment. esp_reboot_coalescing removed entirely (branch, constant,
  catalog, tests).
- 2800 reboot-required flags fire at every policy sync; only DeviceSetup-time
  flags cause the coalesced reboot. The session's flags fired during
  AccountSetup - no reboot - yet our Warning events claimed "device will
  restart". Raw event is now a neutral Info observation; the reboot CLAIM
  moved to ANALYZE-ESP-005 with a REQUIRED system_reboot_detected >= 1
  condition (no observed reboot -> advisory silent).
- One event per 2800 record was spam: records are now buffered and debounced
  (10 s) into ONE aggregated event per burst (rebootUris[]/uriCount/
  firstRebootUri/recordCount/unparsedCount). Watermark is persisted at FLUSH,
  not claim - a pre-emit reboot kill loses nothing, the post-restart backfill
  re-emits the burst at its historical time.

Verified 2800 description shape recorded in code, tests, and concept doc:
"The following URI has triggered a reboot: (./Device/...)."; extraction
covered by a real-text regression test. Lesson added to tasks/lessons.md
(observation-level events vs outcome-gated claims; substring tokens appear
in framework bookkeeping records).

Tests: Agent.V2.Core 1867/1867, Functions 3099/3099, MCP 400 passed,
solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:41:47 +02:00
Oliver Kieselbach a654c7309e feat(agent): attribute mid-ESP coalesced reboots to MDM policy URIs (second sign-in detection)
Device-assigned MDM policies whose CSP URIs match the OS reboot-required
catalog force a coalesced reboot at the end of ESP DeviceSetup - the user
sees an unexpected restart and a second sign-in before AccountSetup.
Invisible in the Intune console; admins simply lived with it. Now attributed
and actionable (PMPC research).

- MdmRebootPolicyTracker (+host): watches DeviceManagement-Enterprise-
  Diagnostics-Provider/Admin EventID 2800, emits one mdm_policy_reboot_required
  per reboot-forcing URI (Warning, ImmediateUpload; tolerant EventData-first
  URI extraction with regex fallback; fail-soft OMADM RebootRequired probe).
  Cross-restart RecordId watermark (mdm-reboot-watermark.json) - mandatory,
  the coalesced reboot restarts the agent and the backfill re-reads the log.
- ShellCoreTracker: RebootCoalescing branch on the already-watched 62407
  stream -> esp_reboot_coalescing corroboration event; deliberate deviation:
  emits on backfill too (live emit races the reboot kill).
- ANALYZE-ESP-005 advisory (warning): names the first URI via {{rebootUri}},
  remediation = reassign profiles device -> user groups; +20 confidence on
  coalescing corroboration, +10 on >=2 URIs.
- Config: Collectors.MdmRebootPolicyWatcherEnabled (default ON, internal
  steering only - agent-native collector, NOT an IAgentAnalyzer, no admin UI),
  MdmRebootPolicyBackfillLookbackMinutes=60; ConfigVersion 35.
- Event types deliberately distinct from system_reboot_detected so the
  backend RebootCount (exact-string match, 2 sites) stays uninflated.
- Docs: docs/agent/mdm-reboot-coalescing.md (+index/log). Exact 2800/62407
  message texts are unverified until E2E VM validation (tolerant matching).

Tests: Agent.V2.Core 1868/1868 (21 new), Functions 3099/3099, MCP 400 passed
(event-types drift), solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:05:11 +02:00
Oliver Kieselbach 6d1ff8d7f0 fix: code-review follow-ups (watcher dispose race, revoke-enforcement tests, scan guard)
- ConsoleBypassWatcher: re-check _disposed after watcher.Start() — a Dispose()
  racing the background arm could leave an armed WMI subscription no code path
  could ever stop (leaked until process exit).
- SignalRContext: collapse rapid accessRevoked pushes into one restart via an
  in-flight guard — a second start() on a non-Disconnected connection throws
  and burns the retry budget.
- config/all: negative-cache managed ids a rescue scan proved missing (5 min
  TTL) — one stale grant (offboarded tenant) no longer makes every delegated
  config/all call pay a full table scan.
- DelegatedAdmin/TenantGroup functions: extract testable revoke-enforcement
  cores (RevokeCoreAsync, SetEnabledCoreAsync, UnassignCoreAsync,
  DeleteGroupCoreAsync, ValidateGrantRequest) and cover the security flows:
  404-on-missing-row writes no audit row and fires no disconnect; real
  revoke/disable/unassign/group-delete always cuts live streams; grant
  validation is fail-closed (GUID tenantId, recognized roles only).
- dashboard/page.tsx: drop unused next/link import left by the banner removal.

Tests: backend 3099 passed, web 557 passed, agent ConsoleBypassWatcher suite
passed, tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 13:13:26 +02:00
Oliver Kieselbach 5c8b273f37 chore: bump skills submodule (system-Timestamp warning in raw-query skills)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 12:38:04 +02:00
Oliver Kieselbach da958b5398 fix(backend): business timestamps survive storage migrations (OccurredUtc + RowKey decode)
The 2026-07-18 storage migration reset every row's system Timestamp,
exposing that AuditLogs, OpsEvents and Events displayed, filtered and
retention-swept on it (custom "Timestamp" writes were silent no-ops —
reserved system property). All migrated rows read as 2026-07-18.

- New Helpers/BusinessTimestamp: RowKey decoders (audit !revtick_guid,
  ops bare revtick, event yyyyMMddHHmmssfff_seq), RowKey-range OData
  clause builders (tick-exact, with the mandatory RowKey lt '"' guard
  so no lower bound can ever match legacy bare-GUID audit rows), and
  dual DateTimeOffset/DateTime column reads.
- Writes in all three tables now persist OccurredUtc (custom column,
  copied verbatim by future migrations); no-op ["Timestamp"] removed.
- Read mappers resolve OccurredUtc -> RowKey decode -> system Timestamp;
  GetEarliestSessionEventTimestampAsync uses the same resolution so
  StartedAt/Duration reconciliation can no longer absorb the migration
  date on migrated sessions.
- Date-window filters and retention moved off the system Timestamp onto
  RowKey ranges; ops retention loses its full-table scan and the
  post-migration mass-delete cliff; audit retention runs a temporary
  second pass for legacy GUID rows (TODO: remove after 2027-03-01).
- New GlobalAdmin backfill endpoint POST /api/maintenance/
  backfill-occurred-utc (dry-run by default, batched, idempotent MERGE
  of only OccurredUtc, table=audit|ops) — Events rows are deliberately
  never mutated; their read path decodes the RowKey instead.
- Tests: decoders, clause boundary proofs incl. GUID-safety pins,
  mapper precedence, backfill row decisions (Functions.Tests 3078).
- Docs: docs/backend/business-timestamps.md (system Timestamp is never
  authoritative; migration checklist rule).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 11:29:53 +02:00
Oliver Kieselbach e17eb607ca docs: gather-rule phase scoping + emit-on-change concept doc (OKF)
New rules/ section in the knowledge bundle: semantics table, from-phase latch
behavior (never via Failed, sticky through it), on_change x emitOnlyIfExists
composition, restart re-emit, no-phase-signal caveat, old-agent degradation,
and the prerequisite toggle-wipe fix. Index + log updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:14 +02:00
Oliver Kieselbach 71000a5dd2 feat(web): gather-rule scope + emit-mode UI, phase dropdowns, edit tags wipe fix
Step 4: portal round-trip for the new fields.

- types.ts: GatherRule + activePhases/activeFromPhase/emitMode; NewRuleForm +
  scopeMode ("always" | "during" | "from") as UI-only view state; canonical
  GATHER_PHASES list (EnrollmentPhase names Start..Complete with display
  labels) + EMIT_MODES; new rules default to emitMode "on_change".
- withDerivedScopeMode: JSON-mode merges rule-shaped JSON (no scopeMode key)
  over form state — the helper derives scopeMode from the data in all four
  JSON paths (create apply/save, edit apply, card edit save) so pasted scope
  fields survive into the payload instead of being silently dropped.
- GatherRuleFormFields: "Active During" mode select + phase multi-select /
  from-phase dropdown (mode switch clears the other mode's fields), "Emit
  Mode" select with helper text; triggerPhase free-text replaced by the same
  canonical dropdown (placeholder used to suggest invalid tokens like
  "DeviceESP"; legacy stored values remain selectable).
- page.tsx: create/save payloads carry the three fields; startEditing maps
  them back (existing rules load as "always", the on_change default is for
  NEW rules only); handleSaveEdit now also sends tags — same wipe class as
  the toggle bug, the full-replace PUT dropped them on every edit.
- GatherRuleCard: trigger section shows the scope + emit mode, e.g.
  "Interval every 60 seconds - from Account Setup - emit on change".

npx tsc --noEmit silent; vitest 557/557.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:04 +02:00
Oliver Kieselbach a88db34af5 feat(agent): gather-rule phase scoping + on_change emit dedup in GatherRuleExecutor
Step 3: the executor honors the new GatherRule fields.

Phase scope (ActivePhases / ActiveFromPhase), applied to ALL trigger types:
- interval: timers keep running, an out-of-scope tick is a free early-return
  (no start/stop lifecycle races on phase transitions).
- startup: scoped rules not yet in scope at UpdateRules are deferred and run
  exactly once from OnPhaseChanged when their scope activates (same
  _startupRulesExecuted dedup; not part of the WaitForStartupRules latch).
- phase_change: evaluated against the NEW phase; scope gate runs BEFORE the
  (rule, phase) dedup add so an out-of-scope pass does not consume the slot.
- on_event: gated by current phase.
- From-phase latch: phase != Unknown/Failed && ordinal >= threshold; sticky for
  the session incl. through Failed; re-evaluated on config refresh so a rule
  delivered after its threshold passed latches immediately.
- Before the first phase signal, scoped rules are inactive.
- IgnorePhaseScope bypass for --run-gather-rules (no phase context; logged).

EmitMode on_change:
- Canonical hash (keys ordinal-sorted at every level, invariant formatting,
  SHA-256) of the raw collector result BEFORE ruleId/ruleTitle injection.
- First non-empty result always emits; unchanged results increment a per-rule
  suppression counter; the next emitted event carries suppressedPolls +
  suppressedSinceUtc. Empty results (emitOnlyIfExists miss) touch nothing, so
  the combo yields zero noise, one event on appearance, then only on change.
- State survives UpdateRules refreshes (agent restart => at most one re-emit).

Tests: GatherRuleExecutorPhaseScopeTests (scope matrix synchronous via internal
IsRuleInScope; trigger wiring async with positive waits),
GatherRuleExecutorEmitModeTests (state machine via internal ShouldEmitOnChange,
canonical-hash properties, ExecuteRule wiring). V2.Core.Tests 1847/1847.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:50 +02:00
Oliver Kieselbach 21dd35f787 feat(backend,shared): gather-rule phase scoping + emit-mode fields, custom-rule toggle wipe fix
Step 0-2 of the gather-rule anti-spam plan:
- GatherRuleService.UpdateRuleAsync: toggle-style partial PUTs ({enabled,
  isBuiltIn, isCommunity}) previously full-replaced the stored custom rule and
  wiped Title/Target/Trigger/Parameters. Partials now merge Enabled into the
  existing row; full edit payloads preserve the original CreatedAt.
- GatherRule: new opt-in fields ActivePhases (run only during listed phases),
  ActiveFromPhase (sticky latch from a phase onwards), EmitMode (always |
  on_change). Absent fields = legacy behavior; schema updated accordingly.
- TableStorageService.Rules: Store+Map for the three columns (absent column /
  stored-empty -> null); MapToGatherRule internal for roundtrip tests.
- ContentEquivalent covers the new fields, null-tolerant for ActivePhases so
  pre-existing rows never diff against seeds (no reseed churn).
- GatherRulesFunction.ValidateScopeAndEmitMode: canonical EnrollmentPhase name
  list Start..Complete (Unknown/Failed and numeric tokens rejected), mutual
  exclusion of the two scope fields, emitMode whitelist; 400 on violation.
- ConfigVersion 33 -> 34 (33 was taken by MigrateToApiBaseUrl).

Tests: GatherRuleUpdatePartialMergeTests (merge/replace/unknown-rule/built-in
path), GatherRuleScopeFieldsTests (Store<->Map roundtrip incl. legacy rows,
ContentEquivalent matrix, validation accept/reject). Functions.Tests 3029/3029.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:35 +02:00
Oliver Kieselbach 9933c039ed refactor(shared,backend): MigrateToApiBaseUrl nullable, drop null-forgiving operator
null = no migration is the documented contract and the agent already
assigns null when stripping the field from the on-disk cache; annotate
the property as string? so the backend assignment needs no ! suppression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 13:33:57 +02:00
Oliver Kieselbach ef67dec263 fix(backend): system-health URLs post EU-migration + GA endpoint visibility
- Agent Binaries check + ops watcher now probe the canonical download alias
  (download.autopilotmonitor.com) AND the legacy blob keepalive; alias failure
  is unhealthy, legacy failure a warning
- McpServerBaseUrl fallback moved from the stale kindwave WEU ACA FQDN to the
  stable custom domain mcp.autopilotmonitor.com (CNAME to the new GWC instance)
- health/detailed and health/mcp include the probed endpoint URLs in check
  details for Global Admins only (storage table endpoint, WEBSITE_HOSTNAME,
  alias/legacy download URLs, MCP server URL)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:42:40 +02:00
122 changed files with 9911 additions and 497 deletions
+25 -67
View File
@@ -1,79 +1,37 @@
## Workflow Orchestration
# Agent Directives: Mechanical Overrides
# Agent Directives
You are operating within a constrained context window and strict system prompts. To produce production-grade code, you MUST adhere to these overrides:
- **Plan first.** Enter plan mode for any non-trivial task (3+ steps or an architectural decision) and write the spec before building. If something goes sideways, stop and re-plan instead of pushing on.
- **Use subagents liberally** for research, exploration, and parallel analysis — one focus each — to keep the main context clean.
- **Verify before claiming done.** Run the tests, check the logs, show the evidence. Diff against `main` where behaviour could regress.
- **Fix bugs autonomously.** Given a report, a failing test, or an error, resolve it — no hand-holding.
- **Capture corrections.** After any correction from the user, record the pattern and a preventing rule in `tasks/lessons.md`; review it at session start.
- Read files over 500 LOC in chunks. Never treat one grep as proof of absence.
Codex reviews your output once you are done.
## Code Quality
### 1. Plan Mode Default
- Enter plan mode for ANY non-trivial task (3+ steps or architectural decisions)
- If something goes sideways, STOP and re-plan immediately — don't keep pushing
- Use plan mode for specification steps, not just building
- Write detailed specs upfront to reduce ambiguity
### 2. Subagent Strategy
- Use subagents liberally to keep main context window clean
- Offload research, exploration, and parallel analysis to subagents
- For complex problems, throw more compute at it via subagents
- One tack per subagent for focused execution
### 3. Self-Improvement Loop
- After ANY correction from the user: update `tasks/lessons.md` with the pattern
- Write rules for yourself that prevent the same mistake
- Ruthlessly iterate on these lessons until mistake rate drops
- Review lessons at session start for relevant project
### 4. Verification Before Done
- Never mark a task complete without proving it works
- Diff behavior between main and your changes when relevant
- Ask yourself: "Would a staff engineer approve this?"
- Run tests, check logs, demonstrate correctness
### 5. Demand Elegance (Balanced)
- For non-trivial changes: pause and ask "is there a more elegant way?"
- If a fix feels hacky: "Knowing everything I know now, implement the elegant solution"
- Skip this for simple, obvious fixes — don't over-engineer
- Challenge your own work before presenting it
### 6. Autonomous Bug Fixing
- When given a bug report: just fix it. Don't ask for hand-holding
- Point at logs, errors, failing tests — then resolve them
- Zero context switching required from the user
- Go fix failing CI tests without being told how
### Tools Execution
- Read any file over 500 LOC in chunks using offset and limit parameters.
- Do not assume a single grep caught everything.
### Code Quality
- If architecture is flawed, state is duplicated, or patterns are inconsistent - propose and implement structural fixes. Ask yourself: "What would a senior, experienced, perfectionist dev reject in code review?" Fix all of it.
Codex will review your output once you are done!
- Simplicity first: minimal, targeted changes. No temporary fixes — find the root cause.
- Where architecture is flawed, state is duplicated, or patterns are inconsistent, propose and implement the structural fix. Ask what a perfectionist reviewer would reject, and fix that too.
- On non-trivial changes, ask whether there is a more elegant way before presenting. Skip it for simple fixes — don't over-engineer.
## Task Management
1. **Plan First**: Write plan to `tasks/todo.md` with checkable items
2. **Verify Plan**: Check in before starting implementation
3. **Track Progress**: Mark items complete as you go
4. **Explain Changes**: High-level summary at each step
5. **Document Results**: Add review section to `tasks/todo.md`
6. **Capture Lessons**: Update `tasks/lessons.md` after corrections
Keep the plan in `tasks/todo.md` as checkable items, confirm it before implementing, tick items off as you go, and add a review section when done.
## Core Principles
## Customer-Facing Claims
- **Simplicity First**: Make every change as simple as possible. Impact minimal code.
- **No Laziness**: Find root causes. No temporary fixes. Senior developer standards.
- **Minimal Impact**: Changes should only touch what's necessary. Avoid introducing bugs.
`src/Web/autopilot-monitor-web/app/{about,terms,privacy}/page.tsx` and `autopilotmonitor-docs/trust/*` state facts customers and security reviewers rely on. They drift silently.
## Technical Docs — OKF Knowledge Bundle (`docs/`)
- Verify every claim against the code before writing it — never carry one forward because it was already on the page.
- Update them in the same change as: runtime/framework versions, roles, notification providers, agent deployment or lifecycle, diagnostics payload, isolation/delegation model, retention caps, sub-processors, or any default governing what is collected.
- Describe only what customers can actually use; operator-only infrastructure is not a feature.
- Trust pages: durable phrasing over exact figures, and bump their "Last reviewed" date.
`docs/` is an [Open Knowledge Format (OKF) v0.1](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) knowledge bundle for contributor/AI-facing technical documentation (customer docs live in the separate autopilotmonitor-docs repo).
## Technical Docs — OKF Bundle (`docs/`)
- **Consume first**: start at `docs/index.md` for progressive disclosure before opening individual documents.
- **When new durable technical knowledge emerges** (architecture decisions, flows, non-obvious mechanisms), capture it as an OKF concept document in `docs/` — markdown with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp` (recommended).
- **Maintain the bundle**: add every new document to `docs/index.md`, note changes in `docs/log.md` (grouped by ISO date). Use standard RELATIVE markdown links between docs — never the OKF `/`-prefixed bundle-absolute form (GitHub resolves those from the repo root and navigation breaks). `index.md` and `log.md` are reserved filenames.
- Documents are English, structural markdown over prose (`# Schema`, `# Examples`, `# Citations` sections where applicable).
Contributor/AI-facing docs (customer docs live in the `autopilotmonitor-docs` repo). Start at `docs/index.md` before opening individual files.
- New durable knowledge (architecture decisions, flows, non-obvious mechanisms) → an OKF concept doc with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp`.
- Register it in `docs/index.md`, note the change in `docs/log.md` (ISO date). `index.md` and `log.md` are reserved names.
- English, structural markdown (`# Schema`, `# Examples`, `# Citations`). Links between docs are RELATIVE — never `/`-prefixed, that breaks GitHub navigation.
+37
View File
@@ -0,0 +1,37 @@
version: 2
updates:
- package-ecosystem: nuget
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
dotnet:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: npm
directory: /src/Web/autopilot-monitor-web
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
web:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: npm
directory: /src/McpServer/autopilot-monitor-mcp
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
mcp:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: github-actions
directory: /
schedule:
interval: monthly
+116
View File
@@ -0,0 +1,116 @@
name: CI
# Runs the full test suite on every pull request and every push to main.
# The deploy workflows build but do not test — this is the gate that decides
# whether a change is allowed to be correct.
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
dotnet:
name: .NET tests (backend + agent)
# windows-latest is required: the agent test projects target net48.
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-dotnet@v5
with:
dotnet-version: '10.0.x'
- name: Restore
run: dotnet restore AutopilotMonitor.sln
- name: Build
run: dotnet build AutopilotMonitor.sln --configuration Release --no-restore
- name: Backend tests
run: dotnet test src/Backend/AutopilotMonitor.Functions.Tests --configuration Release --no-build --verbosity quiet
- name: Agent V2 Core tests
run: dotnet test src/Agent/AutopilotMonitor.Agent.V2.Core.Tests --configuration Release --no-build --verbosity quiet
- name: Decision Core tests
run: dotnet test src/Agent/AutopilotMonitor.DecisionCore.Tests --configuration Release --no-build --verbosity quiet
- name: Summary Dialog tests
run: dotnet test src/Agent/AutopilotMonitor.SummaryDialog.Tests --configuration Release --no-build --verbosity quiet
web:
name: Web (typecheck + tests)
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/Web/autopilot-monitor-web
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
cache: npm
cache-dependency-path: src/Web/autopilot-monitor-web/package-lock.json
- run: npm ci
- name: Typecheck
run: npx tsc --noEmit
- name: Tests
run: npm test
mcp:
name: MCP server tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/McpServer/autopilot-monitor-mcp
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
cache: npm
cache-dependency-path: src/McpServer/autopilot-monitor-mcp/package-lock.json
- run: npm ci
- name: Typecheck
run: npx tsc --noEmit
- name: Tests
run: npm test
guardrails-in-sync:
name: Generated guardrails match rules/guardrails.json
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
- name: Regenerate
run: node rules/scripts/combine.js
- name: Fail if the generated files drifted
run: |
if ! git diff --quiet; then
echo "::error::Generated files are out of date. Run 'node rules/scripts/combine.js' and commit the result."
git diff --stat
exit 1
fi
+58
View File
@@ -0,0 +1,58 @@
name: CodeQL
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
# Weekly, so advisories published after a merge still surface.
- cron: '17 3 * * 1'
permissions:
contents: read
jobs:
analyze:
name: Analyze ${{ matrix.language }}
runs-on: ${{ matrix.runner }}
permissions:
security-events: write
packages: read
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
# csharp needs the net48 agent projects to build, so Windows.
- language: csharp
runner: windows-latest
build-mode: manual
- language: javascript-typescript
runner: ubuntu-latest
build-mode: none
steps:
- uses: actions/checkout@v5
- uses: actions/setup-dotnet@v5
if: matrix.language == 'csharp'
with:
dotnet-version: '10.0.x'
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- name: Build
if: matrix.build-mode == 'manual'
run: dotnet build AutopilotMonitor.sln --configuration Release
- name: Analyze
uses: github/codeql-action/analyze@v3
with:
category: /language:${{ matrix.language }}
+17
View File
@@ -21,6 +21,22 @@ jobs:
steps:
- uses: actions/checkout@v5
# The published customer documentation lives in its own PUBLIC repo and is
# baked into the image as the search_docs corpus (see the Dockerfile). No
# token is needed. Checking it out here rather than vendoring it keeps the
# docs single-sourced — but it also means EDITING THE DOCS DOES NOT UPDATE A
# RUNNING SERVER; this workflow must be re-run. /health reports the SHA
# captured below so that staleness is checkable.
- name: Check out documentation bundle
uses: actions/checkout@v5
with:
repository: autopilotmonitor/autopilotmonitor-docs
path: mcp-docs
- name: Resolve docs commit
id: docs
run: echo "sha=$(git -C mcp-docs rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Login to Azure (OIDC)
uses: azure/login@v3
with:
@@ -35,6 +51,7 @@ jobs:
run: |
docker build \
-f src/McpServer/autopilot-monitor-mcp/Dockerfile \
--build-arg DOCS_COMMIT=${{ steps.docs.outputs.sha }} \
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:${{ env.IMAGE_TAG }} \
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:latest \
.
+5 -2
View File
@@ -117,8 +117,11 @@ scripts/Deployment/
# Migration tooling (local only, vorerst) — cross-tenant storage/cert migration scripts
scripts/Migration/
# Infra (local only)
infra/
# Infra: tracked. The Bicep templates carry no credentials — every secret is a
# @secure() parameter with an empty default, injected at deploy time, and image
# pull runs on managed identity. Keep it that way: never commit a resolved secret
# value or a *.parameters.json holding one.
infra/*.parameters.json
# M2 — local raw session fixtures before anonymization (plan §4 M2)
tests/fixtures/enrollment-sessions-local/
+1 -1
View File
@@ -34,7 +34,7 @@ For detailed information about the system architecture, components, and data flo
## Documentation
Full documentation is available at **[autopilotmonitor.com/docs](https://www.autopilotmonitor.com/docs)**
Full admin documentation is available at **[docs.autopilotmonitor.com](https://docs.autopilotmonitor.com)**
## License
+1
View File
@@ -7,5 +7,6 @@ How the on-device agent works:
* [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine.
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
Reading order for newcomers: overview → decision engine → persistence.
+100
View File
@@ -0,0 +1,100 @@
---
type: Concept
title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs
description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
tags:
- agent
- esp
- reboot
- policy-attribution
- second-sign-in
timestamp: 2026-07-20T00:00:00+02:00
---
# MDM Reboot Coalescing
Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog
(`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced
reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a
**second sign-in screen** before AccountSetup. Invisible in the Intune console; most
admins live with it, not knowing that reassigning the profiles to user groups removes the
reboot entirely. Source research: Patch My PC,
["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/).
The session already counts reboots (`system_reboot_detected` → `session.rebootCount`);
this feature adds the **cause attribution**.
# Verified signal facts (session b2e890c1, 2026-07-20)
* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`,
one record per URI, verified description shape:
`The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).`
Records arrive in sub-second bursts during policy sync.
* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs
applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the
verified session the flags fired during AccountSetup (after DeviceSetup completed) —
**no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that
claims "reboot incoming" on every 2800 is therefore wrong.
* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the
ROUTINE bootstrap marker on every enrollment:
`Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'`
— the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a
reboot is needed). A substring match on it is a false positive by construction. If a
real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be
added then.
# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`)
Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup
backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then
flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst:
* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as
reboot-required: … (+N more)") — the raw event never claims a reboot.
* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct),
`firstRebootUri` (record-time order, drives rule interpolation), `recordCount`,
`unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill),
fail-soft `omadmRebootRequiredFlag` probe of
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown).
* `Timestamp` = earliest record time (backfilled bursts land at their historical instant);
`ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a
reboot that kills the agent).
* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the
coalesced reboot kills the process before the debounced emit, nothing is marked — the
post-restart backfill re-reads and emits the burst. A flushed burst is never repeated.
URI extraction: EventData values starting with `./` preferred, permissive regex over the
description as fallback (trailing punctuation trimmed — the verified text wraps the URI in
parens and ends with a period). Unparsed records still count into the aggregate.
Config (internal steering only, no admin UI — agent-native collector, NOT an
`IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true),
`Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35.
# Advisory — ANALYZE-ESP-005
The reboot **claim** lives here, and only here. Required conditions (BOTH):
1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation)
2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot**
Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional
confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 /
threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or
exclude the Autopilot device group), or accept + communicate the restart.
# Non-goals / invariants
* Event type **deliberately distinct from `system_reboot_detected`** — the backend
RebootCount matches that string exactly in two places
(`EventIngestProcessor.Classification` + terminal recount); attribution must never
inflate the reboot count.
* No session-level field / SessionInfoCard change in this iteration (follow-up candidate).
* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry
the actually-triggered URIs) are consumed.
# Citations
* [Patch My PC — Avoid the second sign-in screen after Autopilot](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/)
* [Microsoft Learn — Enrollment Status Page tracking](https://learn.microsoft.com/en-us/autopilot/enrollment-status)
+95
View File
@@ -0,0 +1,95 @@
---
type: Concept
title: Business Timestamps vs. the Azure Tables System Timestamp
description: Why the system Timestamp is never authoritative, how the 2026-07-18 migration exposed that, and the OccurredUtc + RowKey-decode compensation (mappers, RowKey-range date filters, retention, backfill).
resource: /src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs
tags:
- backend
- table-storage
- audit-log
- ops-events
- events
- migration
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Azure Table Storage's `Timestamp` is a **system property**: the server ignores any supplied
value on write and stamps the row's *write* time; every row rewrite (storage-account
migration, MERGE) resets it. Three tables nevertheless treated it as the business event
time — writes like `["Timestamp"] = evt.Timestamp` were silent no-ops, unnoticed because
write time ≈ event time in normal operation.
The 2026-07-18 EU storage migration rewrote every row, so all migrated rows in
**AuditLogs**, **OpsEvents**, and **Events** read as "2026-07-18 ~13:58 UTC": displays were
wrong, date-window filters matched the wrong rows, and retention froze (nothing older than
the migration moment) with a mass-delete cliff one retention period later.
# Rule
**The system Timestamp is never authoritative.** Every table that needs an event time
stores it in the custom column `OccurredUtc` (`BusinessTimestamp.OccurredUtcColumn`).
Custom columns are copied verbatim by `scripts/Migration/Copy-TableStorage.ps1`, so future
migrations preserve chronology. Migration checklist: any table whose reads/filters depend
on a timestamp must carry it as a custom column — audit this before every storage move.
# Recovery sources per table
The original time survives migrations inside the RowKey:
| Table | RowKey scheme | Decodable |
|---|---|---|
| AuditLogs | `!{MaxTicks-ticks:D19}_{guid:N}` (since 2026-05-06) | Yes; legacy bare-GUID rows (pre-2026-05) are lost |
| OpsEvents | `{MaxTicks-ticks:D19}` | Yes, all rows |
| Events | `{ts:yyyyMMddHHmmssfff}_{seq:D10}` (sanitized agent time) | Yes, all rows (ms precision) |
# Mechanism
`BusinessTimestamp` (Functions/Helpers) is the single source of truth: RowKey decoders,
the OData clause builders, and `GetUtcDateTime` (datetime columns materialize as
`DateTimeOffset` *or* `DateTime` depending on payload shape — always read via this helper).
1. **Read mappers** resolve `OccurredUtc` → RowKey decode → system Timestamp (last
resort; only audit legacy GUID rows end there). Resolvers:
`TableStorageService.ResolveAuditTimestamp`, `TableOpsEventRepository.ResolveTimestamp`,
`TableStorageService.ResolveEventTimestamp` (also used by
`GetEarliestSessionEventTimestampAsync`, which feeds StartedAt/Duration reconciliation —
a system-Timestamp read there would rewrite migrated sessions' derived fields).
2. **Date-window filters and retention use RowKey ranges**, not property comparisons:
OData excludes rows missing a compared property (an `OccurredUtc` filter would drop
every not-yet-backfilled row), and RowKey ranges are index-backed. Boundary semantics
are tick-exact equivalents of the old `Timestamp ge/le` clauses.
**Trap:** audit legacy GUID RowKeys (hex first char) sort *after* all `!`-rows, so
every lower bound `RowKey ge '!…'` must be paired with `RowKey lt '"'`
(`AuditTimeEncodedUpperBound`) — without the guard, a retention sweep would delete all
legacy rows. Pinned by `BusinessTimestampFilterClauseTests`.
3. **Writes** in all three tables now set `OccurredUtc`; the no-op `["Timestamp"]`
entries were removed.
4. **Backfill** (`POST /api/maintenance/backfill-occurred-utc`, GlobalAdminOnly, dry-run
by default, `table=audit|ops`, batched via `maxRows`/`continuation`): MERGE that adds
only `OccurredUtc` decoded from the RowKey — deterministic and idempotent. **Events
rows are deliberately never backfilled** (read-path decode is sufficient; no bulk
mutation of enrollment evidence). Note: a MERGE resets the row's system Timestamp —
irrelevant once nothing depends on it.
5. **Audit retention** runs two passes until ~2027-01: RowKey-range for `!`-rows plus a
`RowKey ge '0' and Timestamp lt …` pass that ages legacy GUID rows out via their frozen
migration-date system Timestamp (remove after 2027-03-01).
# Consequences
* Audit legacy GUID rows have "unknown date" semantics: excluded from date-filtered
views, shown with the (wrong but honest) 2026-07-18 fallback in unfiltered views.
* Raw endpoints (`/api/raw/*`, `query_table`) intentionally keep showing the raw system
Timestamp — they are storage-state inspectors, not business views.
* Event timestamps for pre-cutover rows resolve to the RowKey's sanitized agent time
(ms precision); `ReceivedAt` (server ingest time, rows since 2026-03-19) and
`OriginalTimestamp` (pre-clamp agent time, clamped rows only) remain separate fields.
# Citations
* `src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs` — decoders + clause builders
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Maintenance.cs` — audit write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/DataAccess/TableStorage/TableOpsEventRepository.cs` — ops write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Sessions.cs` — event writes, `ResolveEventTimestamp`, earliest-event read
* `src/Backend/AutopilotMonitor.Functions/Services/OccurredUtcBackfillService.cs` + `Functions/Admin/BackfillOccurredUtcFunction.cs` — backfill
+14
View File
@@ -18,6 +18,20 @@ https://docs.autopilotmonitor.com (separate repository).
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
# Rules
* [Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
* [Gather Rule Guardrails](rules/gather-rule-guardrails.md) - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
# Backend
* [Business Timestamps](backend/business-timestamps.md) - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
# MCP Server
* [MCP Docs Corpus](mcp/docs-corpus.md) - How the published customer documentation becomes the `search_docs` corpus: CRLF-safe heading chunking, build-time embedding as a third precomputed section, and why a docs edit needs an MCP redeploy.
# Security & Identity
+26
View File
@@ -1,5 +1,31 @@
# Log
## 2026-07-22
* **Creation**: Added `rules/gather-rule-guardrails.md` — the gather-rule allowlists are enforced only on the agent (the backend validates no `collectorType`, `target`, or `parameters`), so a collector that forgets to call its guard *is* the vulnerability. Written because two had: `LogParserCollector` read any path in `rule.Target`, bypassing even the `C:\Users` hard block that every other file collector honours, and `EventLogCollector` read any channel including `Security` — while the customer-facing documentation already promised both were allowlisted. Both now guard, with `logparser` checking the directory for wildcard targets (`Path.GetFullPath` throws on `*`) and re-checking each resolved path. Added an event-log channel allowlist (`eventLogChannels`) plus hard-blocked channels held in C# rather than JSON, so a parse error cannot lift them. Also fixed the command timeout, which was inert: `ReadToEnd()` blocks until the pipe closes, so `WaitForExit(30000)` was only reached after the process had already exited and a hung command pinned its worker forever — now read asynchronously and killed at the timeout.
* **Update**: Added `.github/workflows/ci.yml` — the full suite (backend, agent V2 Core, DecisionCore, SummaryDialog, web, MCP) now runs on every pull request and every push to main, plus a job that fails if the generated guardrail files drift from `rules/guardrails.json`. Previously **no** workflow ran tests on a PR: the deploy workflows build without testing, and `build-agent.yml` tested only `DecisionCore.Tests` on manual dispatch — while `trust/security-faq.md` already told customers that automated tests run in CI. Added CodeQL (C# and TypeScript) and Dependabot (nuget, npm ×2, actions). Marked `OfficeBinaryWatcherTests` `[Collection("SerialThreading")]`: its bounded re-probe races the shared ThreadPool and failed once under full-suite load.
## 2026-07-21
* **Update**: `search_docs` and `search_knowledge` now state in their tool descriptions that queries must be in English. Both corpora and `all-MiniLM-L6-v2` are English-only, and the failure is total rather than gradual — a German question returns zero results, not weak ones. The caller is a language model, so translating the question before the call and answering in the user's language costs nothing; a multilingual embedding model would be the alternative but is much larger and would have to be measured against the eval harness first, since it changes English retrieval too.
* **Update**: Added `scripts/eval-docs-search.ts`, an offline retrieval evaluation harness (2465 queries in three classes: chunk headings as a regression detector, rare identifiers, and hand-written questions as the reality anchor), and used it to close a third `search_docs` gap found during the live test. The keyword fallback only fired when semantic search returned *fewer* than `topK`; a pasted identifier like `WOW6432Node` instead returned three chunks at ~0.29 that did not contain the word while the one chunk that did was never considered. It now also fires on a weak semantic head (<0.35), letting exact matches take the front while keeping one semantic result. The threshold is measured, not guessed: 0.35 fixes 335 queries and breaks 0 (identifier recall 66.2% → 81.1%), while 0.40 fixes more but starts demoting hand-written questions. The measurement also resized the problem — identifier queries were only succeeding 66.2% of the time, so the reported case was one of roughly 750.
* **Update**: Closed the two measured recall gaps in `search_docs` (`mcp/docs-corpus.md`). (1) **Token windows** — 87 of 324 chunks exceeded the model's 256-token context and 13% of all tokens never reached the ranking; a character budget cannot fix this (the corpus ranges 2.43–5.81 chars/token), so long text is now embedded as 250-token windows with 40 overlap and scored by its BEST window (`embeddings: number[][]`, max not mean). Token coverage 87% → 100%; "proxy configuration blocking app downloads" went from absent to 0.559 at rank 1; the ten reference queries kept byte-identical scores. Costs +31% vectors, index 2.63 → 3.12 MB. The rules corpus gained the same treatment (128 rules → 172 vectors). (2) **Keyword fallback** — when the semantic pass returns fewer than `topK`, results are topped up via literal `lexicalMatch` on distinctive query terms, requiring two matching terms for corroboration (one incidental substring match returned a page about LLM providers for a sourdough query). Fuzzy matching was measured and rejected: at a threshold low enough to surface "RealmJoin" it invents 5 hits for "SCEPman", which the corpus never mentions. Also sharpened the tool description with the measured context argument for preferring it over fetching the docs site.
* **Creation**: Added `mcp/docs-corpus.md` (new `mcp/` section) — the MCP server gains `search_docs`, a third semantic corpus built from the published customer documentation repository (`autopilotmonitor-docs`, checked out into the Docker build context by the deploy workflow). Documents the h2/h3 + `<details>` chunking with breadcrumb prefixes, the two deliberate size exceptions (tables never split, unbroken changelog lists split at line level), the CRLF precondition (`.` does not match `\r` in JS — headings silently stop matching, which produced 286 wrong-but-plausible chunks before it was found), the optional `docs` section in `search-index.json` and its validation asymmetry, embedding rounding to 6 decimals, and the cross-repo coupling: a docs edit reaches clients only after an MCP redeploy, made checkable via `DOCS_COMMIT` on `/health`. Container resources raised to 0.5 vCPU / 1.0 GiB for the cold-start query-embedder load.
## 2026-07-20
* **Update**: `rules/gather-rule-phase-scoping.md` — the gather-rule form now shows the phase-scope and emit-mode controls only where they can change behavior (`supportsPhaseScope` / `supportsEmitMode`), nulls hidden-control state out of the payload, and rejects a scope mode selected but left unfilled. Offering all three "when" controls unconditionally allowed combinations that could only suppress a one-shot rule, plus a silent no-op state.
* **Update**: Extended `rules/gather-rule-phase-scoping.md` with the `phase_exit` trigger — the closing bookend to `phase_change`, for one-shot collection when a phase is LEFT (e.g. "state at the end of Account Setup"). Documents the evaluation-before-phase-advance rule (triggerPhase match and scope gate see the phase being left), firing into `Failed`, the disjoint `exit:` dedup key space, and why no ConfigVersion bump is needed (no new field — old agents simply never fire the rule).
* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.
* **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated.
* **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.
* **Creation**: Added `rules/gather-rule-phase-scoping.md` (new `rules/` section) — gather rules gain `activePhases` (run only during listed phases), `activeFromPhase` (sticky latch from a phase onwards, never via Failed) and `emitMode: "on_change"` (canonical-hash result dedup with `suppressedPolls` observability), ConfigVersion 34. Includes the prerequisite backend fix: the custom-rule toggle PUT no longer wipes the stored rule definition (partial-merge in `GatherRuleService.UpdateRuleAsync`).
## 2026-07-18
* **Creation**: Added `agent/endpoint-migration.md` — config-channel agent re-home (`MigrateToApiBaseUrl` on `GET agent/config`, ConfigVersion 33): backend or per-tenant API-URL moves without DNS indirection (impossible due to mTLS/Flex-TLS). Live-fetch-only with cache strip, `AgentEndpointMigrationRules` allowlist validated on both sides, one hop, kill-switch wins. Built as the lesson from the 2026-07-18 WEU→GWC cutover, which hard-cut all in-flight agents.
+305
View File
@@ -0,0 +1,305 @@
---
type: Concept
title: MCP Docs Corpus — search_docs and the Cross-Repo Build Coupling
description: How the published customer documentation becomes a third semantic search corpus in the MCP server — CRLF-safe heading chunking, build-time embedding, the corpus hash guard, and why editing the docs does not update a running server.
resource: /src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts
tags:
- mcp
- search
- embeddings
- documentation
- build
timestamp: 2026-07-21T19:30:00+02:00
---
# Purpose
The MCP server answers two different kinds of question, and they need two
different corpora.
* *"Why did this enrollment fail?"* → analysis rules, gather rules, IME log
patterns. That is `search_knowledge`, backed by `knowledge-base.ts`.
* *"How do I deploy the agent?" / "Where is my data stored?"* → the published
customer documentation. That is `search_docs`, backed by `docs-corpus.ts`.
They are deliberately **separate indexes**. Merged into one, they would compete
for the same `topK` and dilute each other: a rules query would surface prose
pages, and a product question would surface rule definitions.
# Schema
## Corpus source
The corpus is the `autopilotmonitor-docs` repository — a public OKF bundle
published at `docs.autopilotmonitor.com`. It is **not** vendored into this repo.
The deploy workflow checks it out into `mcp-docs/` of the Docker build context;
the Dockerfile copies it to `/app/docs` in both the build and the final stage
(byte-identical, or the boot-time corpus hash rejects the precomputed index).
`SUMMARY.md` and `log.md` are skipped as navigation scaffolding. `index.md` is
kept — it is the documentation map and answers "what documentation exists" — but
gets no `url`, because the bundle deliberately keeps it out of `SUMMARY.md` and
GitBook therefore never renders it.
## Chunking
A whole page cannot be one document: `trust/security-faq.md` is 36 KB, while
MiniLM-L6-v2 truncates at ~256 tokens. An un-chunked page would silently embed
only its first paragraph.
Chunks are cut at **h2/h3 headings**, which in this bundle are also the semantic
boundaries — `security-faq.md` carries 37 h3s, one per question.
`<details><summary>Q</summary>` blocks (used by `troubleshooting/faq.md`) are
treated the same way, with the summary as the section title. Each chunk is
prefixed with a `Page Title › Section › Subsection` breadcrumb so a hit lifted
out of the middle of a page is self-describing to both the embedder and the
reader.
Two deliberate exceptions to the ~1500-character budget:
* **Markdown tables are never split.** A table broken across chunks leaves
`| --- |` rows with no header — unreadable in a result and meaningless to
embed. Tables carry real answers here (the role matrix, the settings
reference), so an over-long table is allowed to exceed the budget.
* **Unbroken lists split at line level.** The changelog pages are one bullet
list per release with no blank lines, so paragraph splitting cannot touch
them; left whole, everything past the model's window would be truncated away
and become unsearchable.
Short sections are **kept**, not filtered. A one-line answer ("Is it free?" →
"Yes, the Community plan is free") is often the most valuable hit, and the
breadcrumb supplies the context the short body lacks. Only structural residue
(under 25 alphanumeric characters) is dropped.
## CRLF is a correctness precondition
The docs repo is authored on Windows and ships CRLF. In JavaScript `.` does not
match `\r`, so a pattern like `/^(#{2,3})\s+(.*)$/` matches **nothing** on a CRLF
file: `(.*)` stops before the `\r` and `$` never reaches end-of-string.
This was observed during development, and the failure mode is what makes it
dangerous: no crash, no error — headings and frontmatter simply stop being
recognized and every page silently degrades to unlabelled paragraph splits. The
first working build produced 286 plausible-looking chunks that were almost all
wrong; after normalizing, `security-faq.md` went from 29 blind splits to 46 real
question sections.
`normalizeNewlines()` runs before any parsing. Do not remove it, and do not
"fix" a future parsing bug here by adding `\r?` to one pattern.
## Token windows: why a chunk can hold several vectors
Chunking at heading boundaries keeps sections coherent, but it does not bound the
result by *tokens* — and it cannot. Measured on this corpus the ratio ranges from
**2.43 to 5.81 characters per token**, a factor of 2.4, so no character budget maps
reliably onto the model's 256-token window. A budget safe for the worst case
(~620 characters) would shred prose and tables alike.
Left alone, 87 of 324 chunks exceeded the window and **13% of all tokens never
reached the ranking** — silently, since the tokenizer truncates without complaint.
The effect was concrete: in `built-in-rules.md :: Apps` (734 tokens), the strings
`ANALYZE-APP-013`, `ANALYZE-OFFICE-001`, `ANALYZE-CORR-003` and `Click-to-Run` all
sit past the cutoff, so *"proxy configuration blocking app downloads"* did not
retrieve that section at all.
So the *chunk* stays whole and the *embedding* is split instead:
`VectorSearchProvider.index()` cuts long text into 250-token windows with 40 tokens
of overlap and embeds each. A document therefore carries `embeddings: number[][]`,
and `search()` scores it by its **best** window.
- **Max, not mean.** Averaging window vectors would blur a long section's several
topics into one mediocre direction — the same dilution that makes large context
windows worse than chunking. Max keeps a document findable by any one thing it
discusses.
- **The tokenizer is a build-time dependency only.** `index()` runs during the
Docker build; production boots from precomputed vectors and never loads it. It is
also only the vocabulary, not the ONNX weights (~93 ms).
- Cost: 324 chunks → 424 vectors (+31%), index 2.63 → 3.12 MB, and 424 instead of
324 dot products per query — still microseconds.
- The rules corpus benefits too, unplanned: 128 rules now carry 172 vectors.
Measured effect: token coverage 87% → **100%**, *"Office Click-to-Run install never
finished"* 0.318 → **0.442**, *"proxy configuration blocking app downloads"* absent
→ **0.559 at rank 1**. The ten reference queries kept byte-identical scores — their
chunks are short, hence one window, hence the same vector as before.
## Keyword fallback
Semantic ranking alone leaves a gap: proper nouns embed poorly. `RealmJoin` occurs
in 3 chunks yet scores under the threshold.
Fuzzy matching is the wrong remedy, and the corpus says so. At a threshold low
enough to surface `RealmJoin` (0.28), Fuse also returns 5 hits for `SCEPman` — a
word this corpus contains **zero** times. Literal containment has neither problem.
So when the semantic pass returns fewer than `topK`, `search_docs` tops the list up
via `lexicalMatch`, the substring scan every provider already exposes:
- Query terms are filtered to distinctive ones (≥4 characters, non-stopword, max 6).
- A hit needs **corroboration**: at least two matching terms, unless the query has
only one. Without this rule, "how do I bake sourdough bread" matched a page about
LLM providers, because "bake" is a substring of "baked into the image".
- Hits are **appended, never merged**: they carry no cosine similarity, so semantic
results keep the head of the list and keyword hits fill the tail, flagged
`matchType: "keyword"` with `keywordFallback` in the response.
### Language
The corpus and `all-MiniLM-L6-v2` are both English-only, and the failure is total
rather than gradual: "wo werden meine Daten gespeichert" returns **zero** results,
not weak ones, because the query vector lands nowhere near any English chunk.
The fix is a sentence in the tool description telling the caller to query in English
and answer in the user's language — the caller is a language model, so translating
the question before the call costs nothing. A multilingual embedding model would
handle it natively but is substantially larger and would need to be measured against
`scripts/eval-docs-search.ts` first, since it changes English retrieval too.
`search_knowledge` carries the same note for the same reason.
### When it fires
Two triggers, and the second one exists because the first was not enough:
1. **Fewer than `topK` semantic hits** — the obvious case.
2. **A weak semantic head** (best score below `WEAK_SEMANTIC_SCORE`, 0.35). Topping
up only a short list misses the more common failure: a pasted identifier like
`WOW6432Node` returns three chunks at ~0.29 that do not contain the word, while
the single chunk that does never gets considered. When nothing semantic is
convincing, exact matches take the front — keeping at least one semantic result,
so a weak-but-correct hit is never pushed out entirely.
The threshold was chosen by measurement, not intuition. `scripts/eval-docs-search.ts`
scores strategies over 2465 queries in three classes — every chunk heading
(known-item, a regression detector), rare identifiers (the class this fallback
serves), and hand-written questions (the reality anchor, not derived from the
corpus):
| strategy | known-item | literal | handwritten MRR | fixed | broken |
| --- | --- | --- | --- | --- | --- |
| append-only (previous) | 71.9% | 66.2% | 1.000 | — | — |
| weak < 0.30 | 71.9% | 72.6% | 1.000 | 142 | 0 |
| **weak < 0.35** | **73.7%** | **81.1%** | **1.000** | **335** | **0** |
| weak < 0.40 | 76.3% | 88.2% | 0.944 | 502 | 3 |
| weak < 0.45 | 77.2% | 93.0% | 0.889 | 613 | 5 |
0.35 is strictly dominant: better in every class, worse in none. Higher thresholds
keep repairing more identifier queries but start demoting hand-written questions —
the class that most resembles real use — so the gain is not free past that point.
The measurement also corrected the size of the problem: identifier queries were
succeeding only 66.2% of the time, so `WOW6432Node` was not an outlier but one of
roughly 750 such failures.
## Build-time embedding
Like the rules corpus, docs are embedded once at Docker build time by
`precompute-embeddings.js <rulesDir> <outFile> [docsDir]`, into a third
`PrecomputedSection` (`docs`) of `search-index.json`. The container never embeds
a corpus at boot — only the incoming query.
The `docs` section is **optional** in the file format, which is what makes a
local checkout without the docs repo work: `validatePrecomputedIndex` skips it
when the booting server loaded zero doc chunks. The asymmetry is intentional —
if the server *has* a docs corpus and the file does *not*, that is staleness and
is rejected like any other mismatch.
Embeddings are rounded to 6 decimals on serialize. A float64 component
serializes to ~20 characters of JSON, so 384 dims cost ~7.8 KB per vector;
rounding roughly halves the index file, which is `JSON.parse`d on every
scale-to-zero cold start. The induced L2 norm error (~1e-5 over 384 dims) sits
two orders of magnitude inside `UNIT_NORM_EPSILON`.
## Registration
`search_docs` is registered for **every role**, unlike the platform tools: this
is public documentation carrying no tenant data, and a tenant user asking "how do
I deploy the agent" has as much business here as a Global Admin. It is skipped
entirely when the corpus is empty, so a doc-less build advertises no broken tool.
# Cold start
`minReplicas: 0` means every idle period ends in a cold start someone waits through.
Measured from outside: **24.5 s** to first byte after `ScaledToZero`, against **60 ms**
warm.
Where those seconds go could not be answered from outside the process. Log Analytics
stamps every console line of one ingestion batch with the same time, so the boot
sequence collapses to a single instant however it is queried. The server therefore
times itself — `[boot +Nms]` marks from `process.uptime()`, which includes Node's own
startup and module loading.
Measured in production on a real cold start (31.5 s to first byte that time):
```
+244ms node started (startup + module loading)
+431ms corpora read from disk (+187ms)
+491ms index read, validated, hydrated (+60ms <- for 3.12 MB)
+504ms listening
+677ms query embedder warm (background)
```
**The application accounts for half a second of it.** The other ~31 s — 98% — is
Container Apps activation: KEDA reacting to the request, scheduling a pod, mounting
the 236 MB image, starting the container. The container also boots *faster* than a
development laptop (504 ms against 1273 ms), so nothing here is CPU-starved.
Three conclusions, and two of them overturn assumptions this file previously stated:
- **Index size is irrelevant to startup.** 3.12 MB hydrates in 60 ms. Growing the
corpus is effectively free, and the embedding rounding earns its keep on file size
alone, not on boot time.
- **Raising the container to 0.5 vCPU was not justified by cold start.** The stated
reason was the embedding model load; prebaked, that is 173 ms and runs in the
background, where only the first *search* could ever wait on it. The extra vCPU may
still help concurrent requests, but the cold-start argument does not survive
measurement.
- **The only real lever is `minReplicas: 1`**, which removes activation entirely at
roughly 7× the monthly free consumption grant. Shrinking the image would shave the
mount, but that is a fraction of platform time, not the bulk of it.
# Cross-repo coupling
**Editing the documentation does not update a running MCP server.** The corpus is
baked into the image, so only re-running the *Deploy MCP Server* workflow picks
up doc changes.
That coupling is made checkable rather than memorable: the build passes the docs
checkout SHA as `--build-arg DOCS_COMMIT`, and `/health` reports
`docs: { commit, chunks, sections }`. Comparing that commit against
`git rev-parse HEAD` in `autopilotmonitor-docs` answers "is the deployed
documentation stale?" directly.
Automatic redeploy on a docs push was considered and rejected: it would need a
PAT secret in the docs repo and would turn every typo fix into a container
deployment.
# Examples
Retrieval measured against the real bundle (323 chunks, all-MiniLM-L6-v2):
```
"where is my data stored" → 0.566 troubleshooting/faq.md :: General › Where is my data stored?
"how do I deploy the agent with Intune" → 0.819 troubleshooting/faq.md :: Setup & Agent › How do I deploy the agent?
"which roles exist and what can they do" → 0.500 trust/security-faq.md :: Identity and Access › What roles exist?
"how do I set up Teams notifications" → 0.592 reference/settings.md :: Tenant › Notifications
"what firewall URLs do I need to allow" → 0.374 reference/network-endpoints.md :: (intro)
```
The 0.25 `minScore` default is tuned to that observed band — short keyword
queries on all-MiniLM score low, and a relevant-but-marginal hit lands around
0.34.
# Citations
* `/src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts` — loader, frontmatter
parser, chunker
* `/src/McpServer/autopilot-monitor-mcp/src/tools/search.ts` — `search_docs`
registration
* `/src/McpServer/autopilot-monitor-mcp/src/precomputed-index.ts` — `docs`
section and its validation asymmetry
* `/src/McpServer/autopilot-monitor-mcp/Dockerfile` — `mcp-docs/` copy,
`DOCS_COMMIT`
* `/.github/workflows/deploy-mcp.yml` — docs checkout and SHA capture
* `/src/McpServer/autopilot-monitor-mcp/scripts/eval-docs-search.ts` — retrieval
evaluation harness; run it before changing anything about ranking
* [MCP OAuth Flow](../mcp-oauth-flow.md) — the other MCP concept document
+81
View File
@@ -0,0 +1,81 @@
---
type: Concept
title: Gather Rule Guardrails
description: >-
How gather-rule collectors are fenced in — the allowlists in rules/guardrails.json,
the hard blocks that survive unrestricted mode, and why every collector must call a
guard rather than trusting rule.Target.
resource: src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather
tags: [gather-rules, security, guardrails, agent]
timestamp: 2026-07-22
---
# Concept
A gather rule is authored by a **tenant administrator** through `POST /api/rules/gather` and executed by the agent as **SYSTEM** on an enrolling device. The backend does not validate `collectorType`, `target`, or `parameters` at all — it only enforces scope/emit-mode fields, a 1 MB body cap, and that the rule lands in the caller's own tenant partition. Everything that decides *what a rule may reach* is therefore enforced **on the agent**, in `GatherRuleGuards`.
This is the security boundary. The portal's `ValidationIndicator` is a convenience: it does not block submit, and the API is reachable directly.
# Schema
Allowlists come from `rules/guardrails.json`, embedded into the agent assembly as
`AutopilotMonitor.Agent.V2.Core.Resources.guardrails.json` (see the `EmbeddedResource`
item in `AutopilotMonitor.Agent.V2.Core.csproj`) and regenerated into the portal's
`utils/guardrails.generated.ts` by `node rules/scripts/combine.js`.
| Guard | Source list | Match rule |
| --- | --- | --- |
| `IsRegistryPathAllowed` | `registryPrefixes` | prefix, boundary `\` or end |
| `IsFilePathAllowed` | `filePrefixes` | `Path.GetFullPath` first, then prefix + boundary |
| `IsWmiQueryAllowed` | `wmiQueryPrefixes` | prefix, boundary whitespace or end |
| `IsCommandAllowed` | `allowedCommands` | **exact** match, trimmed — never prefix |
| `IsEventLogChannelAllowed` | `eventLogChannels` | prefix, boundary `/` or end |
**Hard blocks live in code, not in the JSON**, so a parse error cannot lift them — on a
parse failure the allowlists fall back to empty and everything is blocked (fail-closed),
but the hard blocks must hold regardless of configuration:
* `BlockedUsersPrefix` (`C:\Users`) and `AdditionalHardBlockedPathPrefixes`
(`C:\Windows\System32\config`) — exception only for `%LOGGED_ON_USER_PROFILE%`
under `AppData\Local` / `AppData\Roaming`.
* `HardBlockedCommandPatterns` — download, user/group creation, boot config,
persistence, destructive operations; plus `MaxCommandLength`.
* `HardBlockedEventLogChannels` — `Security` (audit trail of user behaviour) and the
PowerShell channels (script-block logging routinely contains secrets in clear text).
`rules/guardrails.json` mirrors the blocked lists (`blockedFilePrefixes`,
`blockedEventLogChannels`) **for display only**. Enforcement is the C# constant.
# Examples
Every collector must call its guard and return `context.EmitSecurityWarning(...)` on
refusal — that emits the `security_warning` event that makes a block visible in the
timeline instead of failing silently.
```csharp
// FileCollector / JsonCollector / XmlCollector
if (!GatherRuleGuards.IsFilePathAllowed(filePath, context.UnrestrictedMode, userProfilePath))
return context.EmitSecurityWarning(rule, "file", filePath);
```
Two collectors were added to this contract on 2026-07-22 after shipping without it:
* **`LogParserCollector`** read any path in `rule.Target`, bypassing even the `C:\Users`
hard block. It cannot guard `rule.Target` directly — a wildcard filename makes
`Path.GetFullPath` throw — so it guards **the directory** when the filename contains
`*`/`?`, and re-guards **each resolved path** after wildcard expansion (a junction can
surface a file the directory check did not cover).
* **`EventLogCollector`** read any channel, including `Security`.
`ImeLogPathOverride` comes only from the local `--ime-log-path` CLI flag, never from
remote config. The operator using it is already a local admin, so it relaxes the
allowlist exactly the way unrestricted mode does — hard blocks still apply.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleGuards.cs`
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleContext.cs` — `EmitSecurityWarning`
* `src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/Gather/GatherRuleCollectorGuardTests.cs`
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — what the backend does *not* validate
* `rules/guardrails.json`, `rules/scripts/combine.js`
* Customer-facing counterpart: `autopilotmonitor-docs/rules/gather-rules.md` (§ Security guardrails)
+136
View File
@@ -0,0 +1,136 @@
---
type: Concept
title: Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change
description: How gather rules are restricted to enrollment phases (activePhases / activeFromPhase), collect once at a phase boundary (phase_change / phase_exit), and how emitMode "on_change" suppresses repeated identical results.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs
tags:
- agent
- backend
- gather-rules
- config
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Interval gather rules (e.g. a registry poll every 60 s) ran during **all** enrollment phases
and emitted an event on **every** tick — including `exists=false` events while the target key
did not exist yet. A rule needed only from Account Setup onwards produced hundreds of
identical noise events per session and wasted collector runs.
# Schema
Three opt-in fields on `GatherRule` (Shared model, `rules/schema/gather-rule.schema.json`,
delivered to the agent via `GET agent/config`; ConfigVersion 34). Absent fields = legacy
behavior — full back-compat.
| Field | Type | Semantics |
|---|---|---|
| `activePhases` | `string[]` \| null | Rule runs only while the current phase is in the list. Null/empty = unrestricted. |
| `activeFromPhase` | `string` \| null | Sticky latch: activates when the phase first reaches the threshold, then stays active for the rest of the session — including through `Failed`. Null = unrestricted. |
| `emitMode` | `"always"` \| `"on_change"` \| null | `on_change`: poll on the trigger cadence, emit only when the collected result changes. Null/`always` = emit every collection (legacy). |
Canonical phase tokens are the `EnrollmentPhase` enum **names** `Start`(0) …`Complete`(7).
Backend validation (`GatherRulesFunction.ValidateScopeAndEmitMode`) rejects `Unknown`/`Failed`,
numeric tokens, unknown emit modes, and both scope fields set at once (400). The agent
defensively prefers `activePhases` if both arrive anyway.
# One-shot collection at phase boundaries
Scoping answers "while *may* this rule run". The trigger answers "when does it fire". For
one-shot collection the two phase triggers are the bookends of a phase:
| Trigger | Fires |
|---|---|
| `phase_change` | Once when `triggerPhase` is **entered** (dedup key `{ruleId}\|{phase}`). |
| `phase_exit` | Once when `triggerPhase` is **left** (dedup key `exit:{ruleId}\|{phase}`). |
Empty `triggerPhase` means every transition. Both keys live in the same
`_phaseRulesExecuted` set; the `exit:` prefix keeps the two key spaces disjoint, so an
enter-rule and an exit-rule on the same phase never cannibalise each other's slot.
`phase_exit` semantics that are easy to get wrong:
* It is evaluated **before** `_currentPhase` advances, so both the `triggerPhase` match and
the phase-scope gate see the phase being **left**. A rule scoped to `DeviceSetup` therefore
still fires on its own exit, even though the phase being entered is outside that scope.
* A transition into `Failed` **does** fire the exit rules of the failing phase — capturing
state at the failure boundary is the point.
* Exit of `Unknown` never fires (nothing was entered yet), and a repeated identical phase
signal is not a transition.
* An exit rule on `Complete` only fires if something transitions *away* from Complete. For
"at the end of enrollment", use `phase_change` on `Complete` or `on_event` with
`enrollment_complete`.
* Old agents do not know the trigger value and simply never fire the rule (silent no-op) —
which is why `phase_exit` needs no ConfigVersion bump: no new field is transported.
# Scope semantics (agent, `GatherRuleExecutor`)
* Scope applies to **all trigger types**:
* `interval` — timers keep running; an out-of-scope tick is a free early-return (no
start/stop lifecycle races on phase transitions).
* `startup` — a scoped startup rule not yet in scope at `UpdateRules` is **deferred**: it
runs exactly once from `OnPhaseChanged` when its scope activates (same
`_startupRulesExecuted` dedup; it is not part of the `WaitForStartupRules` latch).
* `phase_change` — evaluated against the **new** phase; the scope gate runs before the
per-(rule, phase) dedup so an out-of-scope pass does not consume the execution slot.
* `on_event` — gated by the current phase.
* **Before the first phase signal** of a session (`_currentPhase == Unknown`) scoped rules are
inactive. Sessions that never produce phase signals never run them (documented behavior).
* **From-phase latch**: activates when `phase != Unknown && phase != Failed && (int)phase >=
(int)fromPhase`. `Failed=99` would ordinal-satisfy every threshold, hence the explicit
exclusion — but once latched, a rule stays active through `Failed`. A config refresh that
delivers a from-phase rule after its threshold passed latches it immediately.
* `--run-gather-rules` (diagnostic CLI) sets `IgnorePhaseScope = true` — no phase context
exists there, scoped rules execute unconditionally (log line marks the bypass).
# Emit-on-change semantics
* The collector result dict is canonically hashed (keys ordinal-sorted at every nesting
level, invariant value formatting, SHA-256) **before** `ruleId`/`ruleTitle` injection.
* First in-scope non-empty result always emits — on an absent registry key that first
`exists=false` event *is* the visible "we're polling now" indicator.
* Unchanged results increment a per-rule suppression counter; the next emitted event carries
`suppressedPolls` + `suppressedSinceUtc` in its `Data` so the gap is observable.
* Composition with `emitOnlyIfExists=true` (collector-level): a miss returns an empty dict —
no emit AND no hash update. Result: zero noise while absent, one event on appearance, then
only on change.
* State is in-memory and survives `UpdateRules` config refreshes; an agent restart causes at
most one re-emit per rule (acceptable under the per-enrollment lifecycle).
* Volatile collectors (eventlog timestamps, logparser positions) degrade `on_change` to
`always` — every poll hashes differently. The option stays visible for them since some
outputs are stable; this is documented, not hidden.
# Portal & delivery
* Create form defaults new rules to `emitMode: "on_change"`; existing rules load as
`always`. The scope UI is a three-way mode (always / during phases / from phase) backed by
a canonical `GATHER_PHASES` list — the same list replaced the free-text `triggerPhase`
input, whose placeholder used to suggest invalid tokens.
* **Controls are shown only where they change behavior** (`supportsPhaseScope` /
`supportsEmitMode` in `types.ts`). Offering all three "when" controls unconditionally
produced meaningless combinations — a scope on a `phase_change` rule naming a concrete
phase can only ever suppress the single firing, and `on_change` has no second result to
compare against. Hidden controls are nulled by `buildScopeFields` so state from an earlier
trigger choice cannot leak into the payload, and `validateScopeSelection` rejects a scope
mode that was selected but left unfilled (which previously serialised to null and silently
ran the rule everywhere while the form still read "From a phase onwards").
Startup is the deliberate exception: a scope there defers the one-shot rather than
suppressing it, so the scope control stays while emit mode is hidden.
* The custom-rule toggle PUT (`{enabled, isBuiltIn, isCommunity}`) previously full-replaced
the stored rule and wiped Title/Target/Trigger — `GatherRuleService.UpdateRuleAsync` now
merges toggle-style partials (empty Title/CollectorType/Target) into the existing row and
preserves `CreatedAt` on full edits. This fix is a prerequisite: without it every toggle
would silently clear the new fields.
* Old agents (Newtonsoft) ignore unknown config properties: scoped rules run everywhere,
`emitMode` is ignored — graceful degradation until the fleet rolls forward.
* `ContentEquivalent` treats null and empty `activePhases` as equal, so pre-existing DB rows
(absent columns → null) never diff against seeds — no reseed churn.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs` — scope gates, latch, canonical hash
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — `ValidateScopeAndEmitMode`
* `src/Backend/AutopilotMonitor.Functions/Services/GatherRuleService.cs` — partial-PUT merge, `ContentEquivalent`
* `src/Shared/AutopilotMonitor.Shared/Models/Rules/GatherRule.cs` — field contracts
* Tests: `GatherRuleExecutorPhaseScopeTests`, `GatherRuleExecutorEmitModeTests` (agent); `GatherRuleUpdatePartialMergeTests`, `GatherRuleScopeFieldsTests` (backend)
+268
View File
@@ -0,0 +1,268 @@
// Autopilot-Monitor MCP Server — Azure Container App Infrastructure
//
// Reconciled against the live deployment on 2026-07-21 (rg-autopilotmonitor-prd-gwc,
// Germany West Central). The template had drifted behind two changes made directly
// in Azure: the EU cutover (registry/API renamed) and the ACR hardening (admin user
// disabled, image pull moved to the app's system-assigned identity).
//
// !! BEFORE RE-DEPLOYING THIS TEMPLATE, READ THIS !!
// The live app carries an ingress custom-domain binding for mcp.autopilotmonitor.com
// (SNI-enabled, with a managed certificate). Custom domains are NOT modelled below,
// because the certificate is bound out-of-band. A full template deployment would
// therefore REMOVE that binding and take the public MCP endpoint offline.
// For routine changes (CPU/memory, env vars, image) prefer the surgical CLI form:
// az containerapp update -n autopilotmonitor-mcp -g rg-autopilotmonitor-prd-gwc --cpu 0.5 --memory 1.0Gi
// Treat this file as the documented desired state and as a rebuild-from-scratch
// recipe — not as a routinely-applied deployment.
//
// On a FROM-SCRATCH deploy the AcrPull assignment below is created only after the
// Container App exists (it needs the app's principalId), so the very first revision
// can fail to pull its image. Re-run the deployment, or restart the revision, once
// the role assignment has propagated. This does not affect updates to a live app.
//
// Deploy (two-stage; required to set the OAuth public URL safely):
// 1. First deploy without mcpPublicUrl:
// az deployment group create \
// --resource-group <rg-name> \
// --template-file infra/mcp-server.bicep \
// --parameters apiUrl=https://autopilotmonitor-api.azurewebsites.net \
// entraClientSecret=<secret-value>
// This emits `containerAppUrl` as an output (e.g. https://autopilotmonitor-mcp.<env-suffix>.<region>.azurecontainerapps.io).
//
// 2. Re-deploy with that URL pinned, so the OAuth proxy never trusts caller-
// supplied X-Forwarded-Host / Host headers when building issuer / WWW-
// Authenticate / redirect_uri metadata:
// --parameters mcpPublicUrl=<containerAppUrl from step 1>
//
// When mcpPublicUrl is set, the MCP server uses it verbatim and ignores any
// forwarded-headers from the request. Without it, the server logs a startup
// warning and falls back to header-derived URLs (acceptable for local dev,
// not for production).
@description('Azure region for all resources')
param location string = resourceGroup().location
@description('Name of the Container App')
param containerAppName string = 'autopilotmonitor-mcp'
@description('Name of the Container App Environment')
param environmentName string = 'autopilotmonitor-env'
@description('Name of the Log Analytics Workspace (dedicated for Container Apps)')
param logAnalyticsName string = 'autopilotmonitor-container-logs'
@description('Name of the Azure Container Registry')
param acrName string = 'autopilotmonitoreuacr'
@description('Backend API URL')
param apiUrl string = 'https://autopilotmonitor-api-eu.azurewebsites.net'
@description('Container image tag')
param imageTag string = 'latest'
@description('Entra ID app registration client_id used for the OAuth proxy. Public value (appears in JWT aud claim); not a secret.')
param entraClientId string = '1a400946-62c1-4ab4-aa37-f730ac89704d'
@secure()
@description('Entra ID client secret for OAuth token exchange')
param entraClientSecret string = ''
@description('Comma-separated allowlist of redirect_uri hosts for the OAuth proxy. Loopback is always allowed. Empty = built-in default (Anthropic, OpenAI, Google).')
param mcpAllowedRedirectHosts string = ''
@description('Public URL of the MCP server, used verbatim for OAuth issuer / WWW-Authenticate / redirect metadata. REQUIRED in production: the container refuses to boot without it (host-spoofing defense — see config.ts). Set on the second deploy with the value of the containerAppUrl output from the first deploy. The stage-1 deploy is safe because minReplicas=0 means no container runs until this is pinned. Empty falls back to X-Forwarded-Host headers only outside production (NODE_ENV != production).')
param mcpPublicUrl string = ''
@secure()
// NOTE: avoid characters outside cp1252 here (this one used to say a Unicode
// >=). `az bicep build --stdout` writes through the Windows console codec and
// dies with a UnicodeEncodeError on anything it cannot map. Em dashes are fine.
@description('Base64-encoded HMAC key (>=32 bytes after decode) for signing the OAuth proxy state. Naming and format match the backend\'s PaginationTokenSigningKey for consistency. Empty = generate per-instance random key on startup (acceptable for single-replica deployments since state has 10 min lifetime). Generate with PowerShell: [Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))')
param oAuthStateSigningKey string = ''
@description('Enable structured tool-call logging to stderr (queryable via Container App Logs)')
param toolLoggingEnabled bool = false
// --- Azure Container Registry ---
// Admin user is DISABLED: a registry-wide username/password pair is a static,
// non-rotating credential that would also have to live in a Container App secret.
// The app pulls with its system-assigned identity + AcrPull instead (see below).
resource acr 'Microsoft.ContainerRegistry/registries@2023-07-01' = {
name: acrName
location: location
sku: {
name: 'Basic'
}
properties: {
adminUserEnabled: false
}
}
// AcrPull for the Container App's system-assigned identity — this is what replaces
// the admin credentials. Without it the app cannot pull its own image.
//
// NOTE: this assignment exists on the live registry but could not be enumerated
// during reconciliation (the CLI session lacks Microsoft.Authorization read), so it
// is reproduced here from the documented pattern rather than read back from Azure.
// Verify with:
// az role assignment list --scope <acr-id> --assignee <containerApp principalId>
var acrPullRoleId = '7f951dda-4ed3-4680-a7ca-43fe172d538d'
resource acrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
scope: acr
name: guid(acr.id, containerAppName, acrPullRoleId)
properties: {
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', acrPullRoleId)
principalId: containerApp.identity.principalId
principalType: 'ServicePrincipal'
}
}
// --- Log Analytics Workspace (required by Container App Environment) ---
resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
name: logAnalyticsName
location: location
properties: {
sku: {
name: 'PerGB2018'
}
retentionInDays: 30
}
}
// --- Container App Environment ---
resource environment 'Microsoft.App/managedEnvironments@2024-03-01' = {
name: environmentName
location: location
properties: {
appLogsConfiguration: {
destination: 'log-analytics'
logAnalyticsConfiguration: {
customerId: logAnalytics.properties.customerId
sharedKey: logAnalytics.listKeys().primarySharedKey
}
}
}
}
// --- Container App ---
resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
name: containerAppName
location: location
// System-assigned identity: the credential the app uses to pull from the ACR
// (see the AcrPull assignment above). Replaces the former admin-user secret.
identity: {
type: 'SystemAssigned'
}
properties: {
managedEnvironmentId: environment.id
configuration: {
ingress: {
external: true
targetPort: 8080
transport: 'auto'
allowInsecure: false
}
// 'system' = pull with the app's system-assigned identity. No username, no
// password secret — nothing to rotate and nothing to leak from the app config.
registries: [
{
server: acr.properties.loginServer
identity: 'system'
}
]
secrets: [
{
name: 'entra-client-secret'
value: entraClientSecret
}
{
name: 'oauth-state-signing-key'
value: oAuthStateSigningKey
}
]
activeRevisionsMode: 'Single'
}
template: {
containers: [
{
name: 'mcp-server'
image: '${acr.properties.loginServer}/${containerAppName}:${imageTag}'
// 0.5 vCPU, but NOT for the reason originally given. It was raised from
// 0.25 to shorten the scale-to-zero cold start, on the assumption that
// loading the ONNX embedding model dominated it. Measurement disproved
// that: the app boots in 504 ms in production (see the [boot] marks in
// the container log and docs/mcp/docs-corpus.md), of which the prebaked
// embedder is 173 ms in the background. Of a 31 s cold start, ~98% is
// Container Apps activation — pod scheduling, image mount, container
// start — which no CPU setting touches.
//
// It stays at 0.5 because it plausibly helps CONCURRENT requests
// (maxReplicas is 1, so one replica serves up to 10 in-flight calls) and
// because at scale-to-zero the container bills only while active, making
// the difference negligible. If cold start ever needs to actually go
// away, the only lever is minReplicas: 1. Container Apps accepts fixed
// CPU/memory pairs, so memory moves with the CPU.
resources: {
cpu: json('0.5')
memory: '1.0Gi'
}
env: [
{ name: 'AUTOPILOT_API_URL', value: apiUrl }
{ name: 'PORT', value: '8080' }
{ name: 'AUTOPILOT_ENTRA_CLIENT_ID', value: entraClientId }
{ name: 'AUTOPILOT_ENTRA_CLIENT_SECRET', secretRef: 'entra-client-secret' }
{ name: 'MCP_ALLOWED_REDIRECT_HOSTS', value: mcpAllowedRedirectHosts }
{ name: 'MCP_PUBLIC_URL', value: mcpPublicUrl }
{ name: 'OAuthStateSigningKey', secretRef: 'oauth-state-signing-key' }
{ name: 'MCP_TOOL_LOGGING', value: toolLoggingEnabled ? 'true' : 'false' }
]
probes: [
{
type: 'Liveness'
httpGet: {
path: '/health'
port: 8080
}
periodSeconds: 30
initialDelaySeconds: 10
}
{
type: 'Readiness'
httpGet: {
path: '/health'
port: 8080
}
periodSeconds: 10
initialDelaySeconds: 5
}
]
}
]
scale: {
minReplicas: 0
maxReplicas: 1
rules: [
{
name: 'http-scaler'
http: {
metadata: {
concurrentRequests: '10'
}
}
}
]
}
}
}
}
// --- Outputs ---
output containerAppUrl string = 'https://${containerApp.properties.configuration.ingress.fqdn}'
output mcpEndpoint string = 'https://${containerApp.properties.configuration.ingress.fqdn}/mcp'
output acrLoginServer string = acr.properties.loginServer
+85
View File
@@ -0,0 +1,85 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{ "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 },
{ "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 }
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": ["esp", "reboot", "coalescing", "second-sign-in", "policy-attribution", "user-experience"]
}
+99
View File
@@ -1785,6 +1785,105 @@
"enrollment"
]
},
{
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{
"signal": "multiple_reboot_uris",
"condition": "exists",
"weight": 10
},
{
"signal": "omadm_reboot_flag_set",
"condition": "exists",
"weight": 15
}
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"esp",
"reboot",
"coalescing",
"second-sign-in",
"policy-attribution",
"user-experience"
]
},
{
"ruleId": "ANALYZE-ID-001",
"title": "Expected Machine Certificate Not Found",
+73
View File
@@ -109,6 +109,79 @@
"SELECT * FROM Win32_LogicalDisk",
"SELECT * FROM SoftwareLicensingProduct"
],
"eventLogChannels": [
{
"category": "Core Windows logs",
"channels": [
"Application",
"System",
"Setup",
"Microsoft-Windows-Kernel-Boot",
"Microsoft-Windows-Diagnostics-Performance"
]
},
{
"category": "MDM / Enrollment",
"channels": [
"Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider",
"Microsoft-Windows-ModernDeployment-Diagnostics-Provider",
"Microsoft-Windows-Provisioning-Diagnostics-Provider",
"Microsoft-Windows-AAD",
"Microsoft-Windows-User Device Registration"
]
},
{
"category": "ESP / Shell / Apps",
"channels": [
"Microsoft-Windows-Shell-Core",
"Microsoft-Windows-AppXDeployment",
"Microsoft-Windows-AppXDeploymentServer",
"Microsoft-Windows-AppReadiness",
"Microsoft-Windows-Store"
]
},
{
"category": "Security posture / Crypto",
"channels": [
"Microsoft-Windows-BitLocker-API",
"Microsoft-Windows-BitLocker-DrivePreparationTool",
"Microsoft-Windows-TPM-WMI",
"Microsoft-Windows-CertificateServicesClient-Lifecycle-System"
]
},
{
"category": "Update / Servicing",
"channels": [
"Microsoft-Windows-WindowsUpdateClient"
]
},
{
"category": "Logon / Session",
"channels": [
"Microsoft-Windows-Winlogon",
"Microsoft-Windows-User Profile Service",
"Microsoft-Windows-GroupPolicy",
"Microsoft-Windows-TaskScheduler"
]
},
{
"category": "Network",
"channels": [
"Microsoft-Windows-NetworkProfile",
"Microsoft-Windows-Dhcp-Client",
"Microsoft-Windows-DNS-Client",
"Microsoft-Windows-NCSI",
"Microsoft-Windows-WLAN-AutoConfig",
"Microsoft-Windows-Time-Service"
]
}
],
"blockedEventLogChannels": [
"Security",
"Microsoft-Windows-PowerShell",
"Windows PowerShell",
"Microsoft-Windows-Sysmon"
],
"allowedCommands": [
{
"category": "TPM and Security",
+21 -3
View File
@@ -80,8 +80,8 @@
},
"trigger": {
"type": "string",
"description": "When to collect data.",
"enum": ["startup", "phase_change", "interval", "on_event"]
"description": "When to collect data. \"phase_change\" fires once when triggerPhase is entered, \"phase_exit\" once when it is left.",
"enum": ["startup", "phase_change", "phase_exit", "interval", "on_event"]
},
"intervalSeconds": {
"type": "integer",
@@ -90,12 +90,30 @@
},
"triggerPhase": {
"type": "string",
"description": "Phase to trigger on (only used when trigger = \"phase_change\")."
"description": "Phase to trigger on (used when trigger = \"phase_change\" or \"phase_exit\"). Empty = every phase transition."
},
"triggerEventType": {
"type": "string",
"description": "Event type to trigger on (only used when trigger = \"on_event\")."
},
"activePhases": {
"type": "array",
"description": "Run the rule only while the current enrollment phase is one of these phases (EnrollmentPhase enum names). Empty/absent = unrestricted. Mutually exclusive with activeFromPhase.",
"items": {
"type": "string",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
}
},
"activeFromPhase": {
"type": "string",
"description": "Activate the rule once the enrollment phase first reaches this phase, then keep it active for the rest of the session (sticky latch). Absent = unrestricted. Mutually exclusive with activePhases.",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
},
"emitMode": {
"type": "string",
"description": "Emit behavior: \"always\" (default — emit on every collection) or \"on_change\" (poll on the trigger cadence but emit only when the collected result changes; the first in-scope result always emits).",
"enum": ["always", "on_change"]
},
"outputEventType": {
"type": "string",
"description": "EventType for the emitted event (e.g., \"gather_proxy_settings\")."
+14 -1
View File
@@ -91,6 +91,7 @@ if (fs.existsSync(guardrailsPath)) {
// Flatten categorized lists for validation
const flatRegistryPrefixes = guardrails.registryPrefixes.flatMap((g) => g.prefixes);
const flatCommands = guardrails.allowedCommands.flatMap((g) => g.commands);
const flatEventLogChannels = guardrails.eventLogChannels.flatMap((g) => g.channels);
const ts = `/**
* AUTO-GENERATED from rules/guardrails.json — DO NOT EDIT.
@@ -114,6 +115,10 @@ export const COMMAND_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.allowedCommands, 'commands')}
];
export const EVENT_LOG_CHANNEL_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.eventLogChannels, 'channels')}
];
// ---------------------------------------------------------------------------
// Flat arrays (for validation logic)
// ---------------------------------------------------------------------------
@@ -141,10 +146,18 @@ ${flatArray(guardrails.diagnosticsPathPrefixes)}
export const BLOCKED_FILE_PREFIXES: readonly string[] = [
${flatArray(guardrails.blockedFilePrefixes)}
];
export const ALLOWED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(flatEventLogChannels)}
];
export const BLOCKED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(guardrails.blockedEventLogChannels)}
];
`;
fs.writeFileSync(guardrailsOutput, ts, 'utf8');
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes`);
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes, ${flatEventLogChannels.length} event log channels`);
} else {
console.warn('guardrails.json not found — skipping guardrails generation');
}
@@ -56,6 +56,10 @@
PowerShell 5.1 (IME) reads scripts without BOM as ANSI, corrupting multi-byte chars.
.CHANGELOG
2026-07-20 v2.2 Switched default download endpoint from the legacy blob
(autopilotmonitor.blob.core.windows.net) to the Front Door alias
https://download.autopilotmonitor.com/agent/ (route /agent/*).
version.json is still derived from the same container.
2026-06-25 v2.1 OOBE-relax: detect genuine OOBE via WinRT
SystemSetupInfo.OutOfBoxExperienceState and exempt guards 2+3 when a
single freshly-created profile appears in OOBE (Windows Backup for
@@ -80,7 +84,7 @@
[CmdletBinding()]
param(
[Parameter(Mandatory = $false)]
[string]$AgentDownloadUrl = "https://autopilotmonitor.blob.core.windows.net/agent/AutopilotMonitor-Agent.zip",
[string]$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip",
[Parameter(Mandatory = $false)]
[string]$VersionJsonName = "version.json",
@@ -93,7 +97,7 @@ param(
)
# Script version (bump on meaningful changes; see .CHANGELOG above)
$ScriptVersion = "2.1"
$ScriptVersion = "2.2"
# Configuration - Everything in ProgramData for easy cleanup
$AgentBasePath = "$env:ProgramData\AutopilotMonitor"
@@ -0,0 +1,280 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Allowlist enforcement for the two collectors that used to bypass it:
/// <see cref="LogParserCollector"/> read any path the rule named — including
/// C:\Users, whose hard block every other file collector honours — and
/// <see cref="EventLogCollector"/> read any channel, including Security.
/// A tenant admin can author both through the rules API, so the agent is the
/// only place this can be enforced.
/// </summary>
public sealed class GatherRuleCollectorGuardTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
// %TEMP% itself lives under C:\Users, which is hard-blocked — the positive
// cases need a directory the guard can actually admit, so they run beside
// the test assembly instead.
private readonly string _outsideUsers = Path.Combine(
AppContext.BaseDirectory, "guard-tests-" + Guid.NewGuid().ToString("N"));
public GatherRuleCollectorGuardTests() => Directory.CreateDirectory(_outsideUsers);
public void Dispose()
{
_tmp.Dispose();
try { Directory.Delete(_outsideUsers, recursive: true); } catch { /* best effort */ }
}
private GatherRuleContext Context(bool unrestricted = false, string imeLogPathOverride = null)
=> new GatherRuleContext(
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
"sess", "tenant",
evt => _events.Add(evt),
imeLogPathOverride,
new LogFilePositionTracker())
{
UnrestrictedMode = unrestricted
};
private static GatherRule Rule(string collectorType, string target,
Dictionary<string, string> parameters = null) => new GatherRule
{
RuleId = "GATHER-GUARD-TEST",
Title = "guard test",
CollectorType = collectorType,
Target = target,
Parameters = parameters,
Trigger = "startup",
OutputEventType = "guard_test",
};
private EnrollmentEvent SingleSecurityWarning()
=> Assert.Single(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
// -------------------------------------------------------------------
// Event log channel allowlist
// -------------------------------------------------------------------
[Theory]
[InlineData("System")]
[InlineData("Application")]
[InlineData("Setup")]
[InlineData("Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin")]
[InlineData("Microsoft-Windows-AAD/Operational")]
[InlineData("Microsoft-Windows-Shell-Core/Operational")]
public void EventLogChannel_OnAllowlist_IsAllowed(string channel)
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(channel));
[Theory]
[InlineData("Security")]
[InlineData("Microsoft-Windows-PowerShell/Operational")]
[InlineData("Windows PowerShell")]
[InlineData("Microsoft-Windows-Sysmon/Operational")]
public void EventLogChannel_HardBlocked_StaysBlockedEvenUnrestricted(string channel)
{
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel, unrestrictedMode: true));
}
[Theory]
[InlineData("Microsoft-Windows-AADSomethingElse/Operational")] // prefix spoofing
[InlineData("Microsoft-Windows-TerminalServices-LocalSessionManager/Operational")]
[InlineData("")]
public void EventLogChannel_OffAllowlist_IsBlocked(string channel)
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
[Fact]
public void EventLogChannel_Null_IsBlocked()
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(null));
[Fact]
public void EventLogChannel_OffAllowlist_IsAllowedInUnrestrictedMode()
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(
"Microsoft-Windows-TerminalServices-LocalSessionManager/Operational",
unrestrictedMode: true));
[Fact]
public void EventLogCollector_SecurityChannel_EmitsSecurityWarningAndReturnsNoData()
{
var result = new EventLogCollector().Execute(Rule("eventlog", "Security"), Context());
Assert.Empty(result);
var warning = SingleSecurityWarning();
Assert.Equal(true, warning.Data["blocked"]);
Assert.Equal("Security", warning.Data["target"]);
}
[Fact]
public void EventLogCollector_SecurityChannel_StaysBlockedInUnrestrictedMode()
{
var result = new EventLogCollector().Execute(
Rule("eventlog", "Security"), Context(unrestricted: true));
Assert.Empty(result);
SingleSecurityWarning();
}
// -------------------------------------------------------------------
// Log parser file allowlist
// -------------------------------------------------------------------
private static Dictionary<string, string> TextPattern() => new Dictionary<string, string>
{
["pattern"] = "(?<line>.+)",
["format"] = "text",
["trackPosition"] = "false",
};
[Fact]
public void LogParser_UserProfilePath_IsBlockedAndEmitsSecurityWarning()
{
const string target = @"C:\Users\Public\Documents\secret.txt";
var result = new LogParserCollector().Execute(
Rule("logparser", target, TextPattern()), Context());
Assert.Null(result);
var warning = SingleSecurityWarning();
Assert.Equal(target, warning.Data["target"]);
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_UserProfilePath_StaysBlockedInUnrestrictedMode()
{
// C:\Users is a hard block — unrestricted mode must not lift it.
new LogParserCollector().Execute(
Rule("logparser", @"C:\Users\Public\Documents\secret.txt", TextPattern()),
Context(unrestricted: true));
SingleSecurityWarning();
}
[Fact]
public void LogParser_OffAllowlistPath_IsBlockedEvenWhenFileExists()
{
var path = _tmp.File("evidence.log");
File.WriteAllText(path, "line one\nline two\n");
var result = new LogParserCollector().Execute(
Rule("logparser", path, TextPattern()), Context());
Assert.Null(result);
SingleSecurityWarning();
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_WildcardTarget_IsBlockedByItsDirectory()
{
// Path.GetFullPath rejects '*', so the guard has to check the directory.
File.WriteAllText(_tmp.File("evidence.log"), "line one\n");
new LogParserCollector().Execute(
Rule("logparser", Path.Combine(_tmp.Path, "*.log"), TextPattern()), Context());
SingleSecurityWarning();
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_UnrestrictedMode_ReadsPathOffTheAllowlist()
{
var path = Path.Combine(_outsideUsers, "evidence.log");
File.WriteAllText(path, "line one\nline two\n");
new LogParserCollector().Execute(
Rule("logparser", path, TextPattern()), Context(unrestricted: true));
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
Assert.Equal(2, _events.Count(e => e.EventType == "guard_test"));
}
[Fact]
public void LogParser_LocalImeLogPathOverride_RelaxesTheAllowlist()
{
// --ime-log-path is a local CLI flag, never remote config: the operator
// using it is already a local admin, so it relaxes the allowlist the way
// unrestricted mode does.
File.WriteAllText(Path.Combine(_outsideUsers, "AppWorkload.log"), "line one\n");
new LogParserCollector().Execute(
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
TextPattern()),
Context(imeLogPathOverride: _outsideUsers));
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
Assert.Single(_events, e => e.EventType == "guard_test");
}
// -------------------------------------------------------------------
// Command execution
// -------------------------------------------------------------------
[Fact]
public void Command_OffAllowlist_IsBlockedAndNeverRuns()
{
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Get-Content C:\\Windows\\System32\\config\\SAM"), Context());
Assert.Equal(true, result["blocked"]);
Assert.False(result.ContainsKey("output"));
SingleSecurityWarning();
}
[Fact]
public void Command_HardBlockedPattern_StaysBlockedInUnrestrictedMode()
{
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Invoke-WebRequest https://example.invalid/x"),
Context(unrestricted: true));
Assert.Equal(true, result["blocked"]);
SingleSecurityWarning();
}
[Fact]
public void Command_ThatHangs_IsKilledAtTheTimeoutRatherThanPinningTheWorker()
{
// Regression: reading stdout synchronously blocked until the pipe closed,
// so WaitForExit's timeout was never reached and a hung command pinned the
// worker indefinitely. Unrestricted mode is needed to run an ad-hoc command.
var sw = System.Diagnostics.Stopwatch.StartNew();
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Start-Sleep -Seconds 120"), Context(unrestricted: true));
sw.Stop();
Assert.Equal(true, result["timed_out"]);
Assert.Equal(-1, result["exit_code"]);
// The 30s timeout plus drain margin — nowhere near the command's 120s.
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(60),
$"expected the hung command to be killed at the timeout; took {sw.Elapsed}");
}
[Fact]
public void LogParser_ImeLogPathOverride_StillHonoursTheUsersHardBlock()
{
new LogParserCollector().Execute(
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
TextPattern()),
Context(imeLogPathOverride: @"C:\Users\Public\Documents"));
SingleSecurityWarning();
}
}
}
@@ -0,0 +1,249 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// EmitMode "on_change" on <see cref="GatherRuleExecutor"/>: poll on the trigger cadence but
/// emit only when the collected result changes. The dedup state machine is asserted
/// synchronously via the internal <c>ShouldEmitOnChange</c> / <c>ComputeCanonicalHash</c>
/// (no timer races); the ExecuteRule wiring (hash-before-injection, empty-result no-op,
/// emitOnlyIfExists composition) through the public startup path.
/// </summary>
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorEmitModeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_7e3a1f4b-0000-0000-0000-000000000000";
public GatherRuleExecutorEmitModeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private EnrollmentEvent SingleEvent()
{
lock (_eventsGate) return Assert.Single(_events);
}
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
private static GatherRule OnChangeRule(string id = "GATHER-OC-001") => new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
EmitMode = "on_change",
};
private static Dictionary<string, object> Result(params (string Key, object Value)[] pairs)
{
var d = new Dictionary<string, object>();
foreach (var (key, value) in pairs) d[key] = value;
return d;
}
// ── on_change state machine (synchronous) ──────────────────────────
[Fact]
public void FirstResult_AlwaysEmits_ThenUnchangedIsSuppressed_ThenChangeReemits()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // first
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1")))); // changed
}
[Fact]
public void EmitAfterSuppressionStreak_CarriesSuppressedPollCount_AndResets()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
var changed = Result(("exists", true));
Assert.True(_executor.ShouldEmitOnChange(rule, changed));
Assert.Equal(3, changed["suppressedPolls"]);
Assert.True(changed.ContainsKey("suppressedSinceUtc"));
// Counter must reset: the next change emits without a suppression payload.
var changedAgain = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(rule, changedAgain));
Assert.False(changedAgain.ContainsKey("suppressedPolls"));
Assert.False(changedAgain.ContainsKey("suppressedSinceUtc"));
}
[Fact]
public void SuppressionState_IsIsolatedPerRule()
{
var ruleA = OnChangeRule("GATHER-OC-A");
var ruleB = OnChangeRule("GATHER-OC-B");
var payload = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
// Same content, different rule: B has no hash yet — must emit.
Assert.True(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
}
[Fact]
public void DedupState_SurvivesUpdateRulesRefresh()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
// Config refresh mid-session must not reset the last-emitted hash (that would
// re-emit an unchanged result on every refresh).
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
}
// ── Canonical hashing ──────────────────────────────────────────────
[Fact]
public void CanonicalHash_IsKeyOrderInsensitive()
{
var a = new Dictionary<string, object> { ["alpha"] = 1, ["beta"] = "x" };
var b = new Dictionary<string, object> { ["beta"] = "x", ["alpha"] = 1 };
Assert.Equal(
GatherRuleExecutor.ComputeCanonicalHash(a),
GatherRuleExecutor.ComputeCanonicalHash(b));
}
[Fact]
public void CanonicalHash_DetectsValueChanges_IncludingNested()
{
var baseline = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedScalar = new Dictionary<string, object>
{
["exists"] = false,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedNested = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 2 } },
};
var baseHash = GatherRuleExecutor.ComputeCanonicalHash(baseline);
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedScalar));
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedNested));
}
[Fact]
public void CanonicalHash_DistinguishesTypeShapes()
{
// "1" (string) vs 1 (int) and null vs "<null>"-like strings must not collide.
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "1" }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = 1 }));
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = null }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "<null>" }));
}
// ── ExecuteRule wiring (async via startup path) ────────────────────
[Fact]
public void AlwaysMode_EmitsEveryCollection()
{
// EmitMode absent (legacy) — repeated identical results all emit. Two executions of
// the same collector via two rules sharing the target prove no dedup is applied.
var rule1 = OnChangeRule("GATHER-AL-1"); rule1.EmitMode = null; rule1.Trigger = "startup";
var rule2 = OnChangeRule("GATHER-AL-2"); rule2.EmitMode = null; rule2.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule1, rule2 });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(2));
}
[Fact]
public void OnChange_FirstInScopePoll_EmitsExistsFalse_AsPollingIndicator()
{
// The first result on an absent key IS the visible "we're polling now" indicator.
var rule = OnChangeRule();
rule.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
var evt = SingleEvent();
Assert.Equal(false, evt.Data["exists"]);
Assert.Equal("GATHER-OC-001", evt.Data["ruleId"]); // injection happened after hashing
}
[Fact]
public void OnChange_ComposedWithEmitOnlyIfExists_EmptyResultDoesNotTouchHashState()
{
// emitOnlyIfExists miss → empty result → no emit AND no hash update: the eventual
// first real result must still count as "first emit".
var rule = OnChangeRule();
rule.Trigger = "startup";
rule.Parameters = new Dictionary<string, string> { ["emitOnlyIfExists"] = "true" };
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Thread.Sleep(300);
Assert.Equal(0, EventCount); // zero noise while the key is absent
// When the key appears (simulated via the state machine), the result emits as first.
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1"))));
}
}
}
@@ -0,0 +1,246 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// The <c>phase_exit</c> trigger: a one-shot collection when a phase is LEFT — the closing
/// bookend to <c>phase_change</c> (which fires on ENTER). The distinguishing invariant is that
/// both the TriggerPhase match and the phase-scope gate must evaluate against the phase being
/// left, i.e. before the executor's current-phase state moves forward.
/// Rules use the absent-registry-path pattern so no real system state is touched.
/// </summary>
[Collection("SerialThreading")] // rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorPhaseExitTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_9c4d2a7f-0000-0000-0000-000000000000";
public GatherRuleExecutorPhaseExitTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
private static GatherRule ExitRule(
string id, string triggerPhase, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_exit",
TriggerPhase = triggerPhase,
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
[Fact]
public void FiresOnceWhenTheNamedPhaseIsLeft_NotWhenItIsEntered()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-001", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
// Entering AccountSetup must NOT fire an exit rule.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
// Leaving it does.
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
}
[Fact]
public void DoesNotRefireWhenThePhaseIsLeftTwice()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-002", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
// A phase can be re-entered (ESP re-reporting); the one-shot slot stays consumed.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Thread.Sleep(400);
Assert.Equal(1, EventCount);
}
[Fact]
public void EmptyTriggerPhase_FiresOnEveryExit()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-003", "") });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup); // exit of Unknown — must NOT fire
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup
Assert.True(WaitForEventCount(1));
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup
Assert.True(WaitForEventCount(2));
}
[Fact]
public void NeverFiresOnExitOfUnknown()
{
// Before the first phase signal there is no phase to leave.
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-004", "") });
_executor.OnPhaseChanged(EnrollmentPhase.Start);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void RepeatedSamePhaseSignal_IsNotAnExit()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-005", "") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // same phase — no transition
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void FiresOnTransitionIntoFailed_CapturingStateAtTheFailurePoint()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-006", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.True(WaitForEventCount(1), "leaving a phase into Failed must still snapshot it");
}
[Fact]
public void FiresOnExitOfCompleteOnly_WhenCompleteIsActuallyLeft()
{
// "End of enrollment" is better served by phase_change on Complete / on_event; an exit
// rule on Complete only fires if a later transition happens at all.
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-007", "Complete") });
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
_executor.OnPhaseChanged(EnrollmentPhase.Complete);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void ScopeGate_EvaluatesAgainstThePhaseBeingLeft()
{
// Scope = only during DeviceSetup. Leaving DeviceSetup must fire (the rule is still in
// scope at that instant) even though the NEW phase is outside the scope.
_executor.UpdateRules(new List<GatherRule>
{
ExitRule("GATHER-PX-010", "DeviceSetup", activePhases: new List<string> { "DeviceSetup" })
});
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1),
"the scope gate must see the phase being left, not the one being entered");
}
[Fact]
public void OutOfScopeExit_DoesNotFire_AndDoesNotConsumeTheOneShotSlot()
{
// Scope = only during AccountSetup; the rule exits DeviceSetup while out of scope.
var rule = ExitRule("GATHER-PX-011", "", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup: out of scope
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup: in scope
Assert.True(WaitForEventCount(1));
}
[Fact]
public void ExitAndChangeRulesForTheSamePhase_BothFireIndependently()
{
// The dedup key spaces must not collide: same ruleId shape, same phase, different trigger.
var enter = new GatherRule
{
RuleId = "GATHER-PX-020",
Title = "enter",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_change",
TriggerPhase = "AccountSetup",
OutputEventType = "gather_test",
Enabled = true,
};
var exit = ExitRule("GATHER-PX-021", "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { enter, exit });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1)); // enter fired
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(2)); // exit fired
}
[Fact]
public void ExitRuleWithEmitOnChange_StillEmitsItsFirstResult()
{
var rule = ExitRule("GATHER-PX-030", "AccountSetup");
rule.EmitMode = "on_change";
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
}
}
}
@@ -0,0 +1,326 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Phase scoping on <see cref="GatherRuleExecutor"/> (ActivePhases / ActiveFromPhase):
/// the scope matrix is asserted synchronously via the internal <c>IsRuleInScope</c>
/// (no timer races); the trigger-path wiring (deferred startup, phase_change / on_event
/// gating) is asserted through the public API with positive event waits.
/// Rules use the absent-registry-path pattern from
/// <see cref="RegistryCollectorEmitOnlyIfExistsTests"/> so no real system state is touched.
/// </summary>
[Collection("SerialThreading")] // deferred startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorPhaseScopeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_5b1c8d2e-0000-0000-0000-000000000000";
public GatherRuleExecutorPhaseScopeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
// Bypass the registry allowlist guard for the synthetic test path.
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
/// <summary>Interval rule with a 1h period — the timer never ticks within a test run,
/// so these rules exist purely for synchronous scope-matrix assertions.</summary>
private static GatherRule ScopeProbeRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
private static GatherRule StartupRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "startup",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
// ── Scope matrix (synchronous) ─────────────────────────────────────
[Fact]
public void UnscopedRule_IsInScope_EvenBeforeAnyPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-000");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_InactiveBeforeFirstPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-001", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_ActiveDuringListedPhase_InactiveAfterLeaving()
{
var rule = ScopeProbeRule("GATHER-SCOPE-002", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_MatchIsCaseInsensitive()
{
var rule = ScopeProbeRule("GATHER-SCOPE-003", activePhases: new List<string> { "accountsetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesAtThreshold_AndStaysSticky_IncludingThroughFailed()
{
var rule = ScopeProbeRule("GATHER-SCOPE-010", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule)); // threshold not reached
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule)); // latched
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.True(_executor.IsRuleInScope(rule)); // sticky through Failed
}
[Fact]
public void FromPhase_NoSpuriousActivationViaFailed()
{
// Failed=99 would ordinal-satisfy every threshold — it must never latch.
var rule = ScopeProbeRule("GATHER-SCOPE-011", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesWhenThresholdIsSkippedOver()
{
// The session may never report the exact threshold phase — a later phase latches too.
var rule = ScopeProbeRule("GATHER-SCOPE-012", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_RuleDeliveredByRefreshAfterPhaseReached_LatchesImmediately()
{
_executor.UpdateRules(new List<GatherRule>());
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
// Config refresh delivers a new from-phase rule whose threshold already passed.
var rule = ScopeProbeRule("GATHER-SCOPE-013", fromPhase: "DeviceSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void BothFieldsSet_ActivePhasesWins()
{
// Backend validation rejects this shape; the agent defensively prefers ActivePhases.
var rule = ScopeProbeRule("GATHER-SCOPE-020",
activePhases: new List<string> { "DeviceSetup" }, fromPhase: "Start");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.False(_executor.IsRuleInScope(rule)); // FromPhase(Start) would say yes — list wins
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void IgnorePhaseScope_BypassesScope_WithoutPhaseContext()
{
_executor.IgnorePhaseScope = true;
var during = ScopeProbeRule("GATHER-SCOPE-030", activePhases: new List<string> { "AccountSetup" });
var from = ScopeProbeRule("GATHER-SCOPE-031", fromPhase: "Complete");
_executor.UpdateRules(new List<GatherRule> { during, from });
Assert.True(_executor.IsRuleInScope(during));
Assert.True(_executor.IsRuleInScope(from));
}
// ── Trigger-path wiring (async, positive waits) ────────────────────
[Fact]
public void UnscopedStartupRule_RunsImmediately()
{
_executor.UpdateRules(new List<GatherRule> { StartupRule("GATHER-START-000") });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
[Fact]
public void ScopedStartupRule_IsDeferred_ThenFiresExactlyOnce_WhenScopeActivates()
{
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-001", fromPhase: "AccountSetup")
});
// Not in scope at UpdateRules: nothing queued, latch trivially satisfied.
Assert.True(_executor.WaitForStartupRules(30));
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1), "deferred startup rule should fire when its scope activates");
// Later phase changes must not re-fire it (dedup via _startupRulesExecuted).
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void PhaseChangeRule_OutOfScopePhase_DoesNotFire_InScopePhaseFires()
{
// TriggerPhase empty = fires on every phase change; scope narrows it to AccountSetup.
var rule = new GatherRule
{
RuleId = "GATHER-PC-001",
Title = "pc",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_change",
TriggerPhase = "",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1));
// Same phase again: the (rule, phase) dedup slot is now consumed.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void OnEventRule_GatedByScope()
{
var rule = new GatherRule
{
RuleId = "GATHER-OE-001",
Title = "oe",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "on_event",
TriggerEventType = "app_install_failed",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnEvent("app_install_failed"); // no phase signal yet → out of scope
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnEvent("app_install_failed");
Assert.True(WaitForEventCount(1));
}
[Fact]
public void IgnorePhaseScope_ScopedStartupRule_RunsImmediately()
{
// The --run-gather-rules diagnostic path: scoped rules must execute unconditionally.
_executor.IgnorePhaseScope = true;
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-002", activePhases: new List<string> { "AccountSetup" })
});
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
}
}
@@ -17,6 +17,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Office
/// cadence and stops at its bound, is idempotent, the initial scan raises when binaries already
/// exist, a filesystem create raises, and Dispose cancels the re-probe.
/// </summary>
[Collection("SerialThreading")] // the bounded re-probe runs on the shared ThreadPool
public sealed class OfficeBinaryWatcherTests
{
private static readonly string[] CoreBinaries = { "WINWORD.EXE", "EXCEL.EXE", "POWERPNT.EXE", "OUTLOOK.EXE" };
@@ -0,0 +1,336 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral
/// <c>mdm_policy_reboot_required</c> event (session b2e890c1 rework: per-record events were
/// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an
/// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit
/// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed).
/// The registry probe is always overridden — tests never touch the live registry.
/// </summary>
public sealed class MdmRebootPolicyTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc);
private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation";
private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch";
// Verified real 2800 description shape (session b2e890c1, 2026-07-20).
private const string RealDescription =
"The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly MdmRebootPolicyTracker _tracker;
private long _nextRecordId = 1;
public MdmRebootPolicyTrackerTests()
{
_sink = new FakeSignalIngressSink();
_tracker = MakeTracker(stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private MdmRebootPolicyTracker MakeTracker(string? stateDirectory)
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var tracker = new MdmRebootPolicyTracker(
sessionId: "sess-mdm",
tenantId: "tenant-mdm",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: stateDirectory,
debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly
tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
return tracker;
}
private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null)
{
_tracker.ProcessEvent(
eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? At,
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> Emitted() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.MdmPolicyRebootRequired).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload()
{
Buffer(UriSvchost, timeCreatedUtc: At);
Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120));
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal(2, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
Assert.Equal(UriSvchost, data["firstRebootUri"]);
var uris = Assert.IsType<List<string>>(data["rebootUris"]);
Assert.Equal(2, uris.Count);
Assert.Contains(UriSvchost, uris);
Assert.Contains(UriSystemGuard, uris);
}
[Fact]
public void Message_IsNeutralObservation_NoRebootClaim()
{
// Session b2e890c1: flags raised during AccountSetup have no effect — the raw event
// must state the observation only, never a future consequence.
Buffer(UriSvchost);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var message = s.Payload![SignalPayloadKeys.Message];
Assert.Contains("flagged", message);
Assert.DoesNotContain("will restart", message);
Assert.DoesNotContain("second sign-in", message);
}
[Fact]
public void FlushPending_Empty_EmitsNothing()
{
_tracker.FlushPending();
Assert.Empty(Emitted());
}
[Fact]
public void AggregateTimestamp_IsEarliestRecordTime()
{
var early = At.AddMinutes(-30);
Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true);
Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(early, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
// firstRebootUri follows record-time order, not buffer order.
Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]);
}
[Fact]
public void Backfilled_False_WhenAnyRecordIsLive()
{
Buffer(UriSvchost, isBackfill: true);
Buffer(UriSystemGuard, isBackfill: false);
_tracker.FlushPending();
Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]);
}
[Fact]
public void DuplicateUris_AreDistinctInAggregate()
{
Buffer(UriSvchost);
Buffer(UriSvchost);
_tracker.FlushPending();
var data = Data(Assert.Single(Emitted()));
Assert.Equal(1, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
}
[Fact]
public void UnparsedRecords_CountedWithSampleDescription_StillEmitted()
{
Buffer(rebootUri: null, description: "Some unexpected 2800 wording");
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var data = Data(s);
Assert.False(data.ContainsKey("rebootUris"));
Assert.False(data.ContainsKey("firstRebootUri"));
Assert.Equal(0, data["uriCount"]);
Assert.Equal(1, data["unparsedCount"]);
Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]);
}
[Fact]
public void SameRecordId_BufferedTwice_CountsOnce()
{
Buffer(UriSvchost, recordId: 42);
Buffer(UriSvchost, recordId: 42);
_tracker.FlushPending();
Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void RecordAfterFlush_GoesIntoNextAggregate()
{
Buffer(UriSvchost, recordId: 10);
_tracker.FlushPending();
Buffer(UriSystemGuard, recordId: 11);
_tracker.FlushPending();
Assert.Equal(2, Emitted().Count);
}
[Fact]
public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered()
{
// Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill
// record must not be suppressed by a high-water mark.
Buffer(UriSvchost, recordId: 100);
Buffer(UriSystemGuard, recordId: 50, isBackfill: true);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void NegativeRecordId_IsNeverDeduped()
{
Buffer(UriSvchost, recordId: -1);
Buffer(UriSvchost, recordId: -1);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances()
{
// The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same
// records — a FLUSHED record must be skipped by the next instance.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
first.FlushPending();
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
Assert.Single(Emitted());
}
[Fact]
public void Watermark_NotPersisted_WhenBufferedButNeverFlushed()
{
// Process killed by the very reboot the records announce: nothing was flushed, so the
// next instance must re-read and emit the records.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
// no FlushPending — simulated reboot kill
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(true, Data(s)["backfilled"]);
}
[Fact]
public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt()
{
_tracker.RebootRequiredFlagProbe = () => true;
Buffer(UriSvchost, recordId: 1);
_tracker.FlushPending();
_tracker.RebootRequiredFlagProbe = () => null;
Buffer(UriSystemGuard, recordId: 2);
_tracker.FlushPending();
var emitted = Emitted();
Assert.Equal(2, emitted.Count);
Assert.Equal(true, Data(emitted[0])["omadmRebootRequiredFlag"]);
Assert.False(Data(emitted[1]).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void RebootRequiredFlagProbe_Throwing_IsFailSoft()
{
_tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom");
Buffer(UriSvchost);
_tracker.FlushPending();
Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void BuildXPath_Targets2800()
{
Assert.Equal("*[System[(EventID=2800)]]", MdmRebootPolicyTracker.BuildXPath());
}
// ------------------------------------------------------------------ URI extraction
[Fact]
public void ExtractRebootUri_FromVerifiedRealDescription()
{
// Real 2800 text wraps the URI in parentheses and ends with a period — both must be
// excluded from the extracted URI.
var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription);
Assert.Equal(UriSvchost, uri);
}
[Fact]
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
{
var eventData = new Dictionary<string, string> { ["Data0"] = " " + UriSystemGuard + " " };
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription);
Assert.Equal(UriSystemGuard, uri);
}
[Fact]
public void ExtractRebootUri_ReturnsNull_WhenNothingUriLike()
{
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(
new Dictionary<string, string> { ["Data0"] = "plain" }, "no uri here"));
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(null, null));
}
[Fact]
public void ParseEventData_CollectsNamedAndPositionalData()
{
const string xml =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
"<System><EventID>2800</EventID></System>" +
"<EventData><Data Name='Uri'>./Device/Vendor/MSFT/X</Data><Data>positional</Data></EventData>" +
"</Event>";
var parsed = MdmRebootPolicyTracker.ParseEventData(xml);
Assert.Equal("./Device/Vendor/MSFT/X", parsed["Uri"]);
Assert.Equal("positional", parsed["Data0"]);
}
}
}
@@ -17,13 +17,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
FakeTelemetryTransport transport,
out EventSequenceCounter counter,
string sessionId = "sess-1",
string tenantId = "tenant-1")
string tenantId = "tenant-1",
Func<bool>? traceEventsEnabled = null)
{
var tmp = new TempDirectory();
counter = new EventSequenceCounter(new EventSequencePersistence(tmp.File("seq.json")));
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId);
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId, traceEventsEnabled);
}
private static EnrollmentEvent NewTraceEvent(string eventType = "outbound_ip") =>
new EnrollmentEvent
{
EventType = eventType,
Severity = EventSeverity.Trace,
Source = "test",
Timestamp = At,
Phase = EnrollmentPhase.Unknown,
Message = "trace message",
};
private static EnrollmentEvent NewTestEvent(string eventType = "enrollment_complete") =>
new EnrollmentEvent
{
@@ -49,6 +61,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
Assert.Equal("sess-1", evt.SessionId);
Assert.Equal("tenant-1", evt.TenantId);
Assert.NotNull(item);
Assert.Equal(TelemetryItemKind.Event, item.Kind);
Assert.Equal("tenant-1_sess-1", item.PartitionKey);
Assert.Equal(evt.RowKey, item.RowKey);
@@ -174,5 +187,86 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
Assert.Equal("test-reason", (string?)data["reason"]);
Assert.Equal("Strong", (string?)data["wgConfidence"]);
}
// ============================================================ Trace-event gate
[Fact]
public void Trace_event_is_dropped_when_gate_is_off()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
var item = emitter.Emit(NewTraceEvent());
Assert.Null(item);
Assert.Equal(0, transport.EnqueueCount);
}
[Fact]
public void Trace_event_flows_when_gate_is_on()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => true);
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
[Fact]
public void Trace_event_flows_when_no_gate_is_wired()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _);
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
[Fact]
public void Gate_off_does_not_suppress_non_trace_events()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
foreach (var severity in new[] { EventSeverity.Info, EventSeverity.Warning, EventSeverity.Error })
{
var evt = NewTestEvent();
evt.Severity = severity;
Assert.NotNull(emitter.Emit(evt));
}
Assert.Equal(3, transport.EnqueueCount);
}
[Fact]
public void Dropped_trace_event_does_not_consume_a_sequence_number()
{
var transport = new FakeTelemetryTransport();
var traceAllowed = true;
// ReSharper disable once AccessToModifiedClosure — deliberate: the gate is read
// per event, mirroring a mid-session remote-config merge.
var emitter = Build(transport, out _, traceEventsEnabled: () => traceAllowed);
emitter.Emit(NewTestEvent()); // seq 1
traceAllowed = false;
emitter.Emit(NewTraceEvent()); // dropped — must not burn seq 2
emitter.Emit(NewTestEvent()); // seq 2
Assert.Equal(2, transport.EnqueueCount);
var first = JObject.Parse(transport.Enqueued[0].PayloadJson);
var second = JObject.Parse(transport.Enqueued[1].PayloadJson);
Assert.Equal(1, (long)first["Sequence"]!);
Assert.Equal(2, (long)second["Sequence"]!);
}
[Fact]
public void Throwing_gate_fails_open_so_diagnostics_are_never_silently_lost()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => throw new InvalidOperationException("boom"));
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
}
}
@@ -0,0 +1,628 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> for
/// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog —
/// and emits ONE aggregated <c>mdm_policy_reboot_required</c> event per burst carrying the URI
/// list (<c>rebootUris</c>, <c>uriCount</c>, <c>firstRebootUri</c>).
/// <para>
/// Verified against session b2e890c1 (2026-07-20): the 2800 description reads
/// <c>The following URI has triggered a reboot: (./Device/...).</c> and records arrive in
/// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record
/// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced
/// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond)
/// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL
/// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an
/// actually-observed <c>system_reboot_detected</c> before attributing anything.
/// </para>
/// <para>
/// <b>Backfill + watermark.</b> Device policies can apply before the agent starts, and a
/// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the
/// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical
/// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so
/// a pre-emit death loses nothing and a completed emit is never repeated.
/// </para>
/// </summary>
internal sealed class MdmRebootPolicyTracker : IDisposable
{
internal const string Channel = "Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin";
internal const int EventId_PolicyRebootRequired = 2800;
internal const string WatermarkStateFileName = "mdm-reboot-watermark.json";
/// <summary>Cap for the emitted <c>rebootUris</c> list — <c>uriCount</c> always carries the uncapped distinct total.</summary>
internal const int MaxUrisPerEvent = 20;
/// <summary>Default quiet-period before a burst of 2800 records is flushed as one event.</summary>
internal const int DefaultDebounceMilliseconds = 10000;
/// <summary>Registry key whose <c>RebootRequired</c> value/subkey flags a pending OMA-DM coalesced reboot.</summary>
internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML";
// Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800
// text wraps the URI in parentheses — ')' is deliberately outside the class so the match
// stops cleanly; trailing sentence punctuation is trimmed after the match.
private static readonly Regex UriPattern = new Regex(
@"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled);
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly int _debounceMilliseconds;
private EventLogWatcher _watcher;
private Timer _debounceTimer;
private sealed class PendingRecord
{
public long RecordId;
public DateTime TimeUtc;
public string Uri; // null when extraction failed
public string Description; // may be null
public bool IsBackfill;
}
// Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as
// WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE
// deliberate difference: the watermark is persisted at FLUSH time, not at claim time —
// records buffered but never flushed (process killed by the very reboot they announce)
// must be re-read and emitted by the post-restart backfill.
private readonly List<PendingRecord> _pending = new List<PendingRecord>();
private long _restartWatermark = -1;
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
private long _maxPersistedRecordId = -1;
private readonly object _stateLock = new object();
/// <summary>
/// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown
/// (key absent or unreadable). Settable test seam; the default reads the live registry.
/// </summary>
internal Func<bool?> RebootRequiredFlagProbe { get; set; } = ReadOmadmRebootRequiredFlag;
public MdmRebootPolicyTracker(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger,
bool backfillEnabled = true,
int backfillLookbackMinutes = 60,
string stateDirectory = null,
int debounceMilliseconds = DefaultDebounceMilliseconds)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_debounceMilliseconds = debounceMilliseconds;
}
public void Start()
{
LoadWatermark();
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
{
BackfillRecentEvents();
// The backfill batch is complete — flush immediately instead of waiting out the
// debounce (live records that raced in are simply included).
FlushPending();
}
else
{
_logger.Info("MdmRebootPolicy backfill disabled by config");
}
}
public void Stop()
{
// Flush any un-emitted burst before tearing down (normal agent shutdown path).
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); }
var timer = Interlocked.Exchange(ref _debounceTimer, null);
if (timer != null)
{
try { timer.Dispose(); } catch { }
}
if (_watcher == null) return;
try
{
_watcher.Enabled = false;
_watcher.Dispose();
_logger.Info("MdmRebootPolicy watcher stopped");
}
catch (Exception ex)
{
_logger.Error("Error stopping MdmRebootPolicy watcher", ex);
}
finally
{
_watcher = null;
}
}
public void Dispose() => Stop();
// -----------------------------------------------------------------------
// Watcher lifecycle
// -----------------------------------------------------------------------
private void StartWatcher()
{
try
{
var query = new EventLogQuery(Channel, PathType.LogName, BuildXPath());
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
_watcher.Enabled = true;
_logger.Info($"MdmRebootPolicy watcher started: {Channel} (EventID {EventId_PolicyRebootRequired})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy watcher access denied for {Channel}: {ex.Message}");
}
catch (Exception ex)
{
_logger.Error($"Failed to start MdmRebootPolicy watcher for {Channel}", ex);
// MON-D1: surface a dead watcher as one-shot telemetry, not just a local log line.
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
collectorName: "MdmRebootPolicyTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
}
}
/// <summary>Targeted-EventID XPath filter. Exposed as internal for tests.</summary>
internal static string BuildXPath() =>
$"*[System[(EventID={EventId_PolicyRebootRequired})]]";
// -----------------------------------------------------------------------
// Event processing — buffer + debounce
// -----------------------------------------------------------------------
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
{
if (e.EventRecord == null) return;
try
{
ProcessRecord(e.EventRecord, isBackfill: false);
}
catch (Exception ex)
{
_logger.Error("Error processing MdmRebootPolicy event", ex);
}
}
private void ProcessRecord(EventRecord record, bool isBackfill)
{
var recordId = record.RecordId ?? -1;
// Cheap dedup gate BEFORE the expensive XML/FormatDescription work.
if (IsAlreadyProcessed(recordId))
return;
string xml = null;
try { xml = record.ToXml(); }
catch { /* fall back to description-only parse below */ }
string description = null;
try { description = record.FormatDescription(); }
catch { /* some events lack formatting resources */ }
var rebootUri = ExtractRebootUri(ParseEventData(xml), description);
ProcessEvent(
eventId: record.Id,
recordId: recordId,
timeCreatedUtc: record.TimeCreated?.ToUniversalTime(),
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
/// <summary>
/// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the
/// WindowsUpdateTracker pattern). The aggregated event is emitted by <see cref="FlushPending"/>
/// — after the debounce quiet-period, at backfill completion, or at <see cref="Stop"/>.
/// </summary>
internal void ProcessEvent(
int eventId,
long recordId,
DateTime? timeCreatedUtc,
string rebootUri,
string formattedDescription,
bool isBackfill)
{
lock (_stateLock)
{
if (recordId >= 0)
{
// Already emitted in a prior run, or already buffered/emitted this run.
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return;
}
_pending.Add(new PendingRecord
{
RecordId = recordId,
TimeUtc = timeCreatedUtc ?? DateTime.UtcNow,
Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri,
Description = formattedDescription,
IsBackfill = isBackfill,
});
}
ArmDebounce();
}
private void ArmDebounce()
{
if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly)
var timer = _debounceTimer;
if (timer == null)
{
var created = new Timer(_ =>
{
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); }
}, null, Timeout.Infinite, Timeout.Infinite);
timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created;
if (!ReferenceEquals(timer, created))
{
try { created.Dispose(); } catch { }
}
}
try { timer.Change(_debounceMilliseconds, Timeout.Infinite); }
catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ }
}
/// <summary>
/// Emits the pending burst as ONE aggregated <c>mdm_policy_reboot_required</c> event and
/// persists the watermark. No-op when nothing is pending. Internal for tests.
/// </summary>
internal void FlushPending()
{
List<PendingRecord> batch;
long watermarkToPersist = -1;
lock (_stateLock)
{
if (_pending.Count == 0) return;
batch = new List<PendingRecord>(_pending);
_pending.Clear();
foreach (var r in batch)
{
if (r.RecordId > _maxPersistedRecordId)
_maxPersistedRecordId = r.RecordId;
}
watermarkToPersist = _maxPersistedRecordId;
}
if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist);
batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc));
var distinctUris = batch
.Where(r => r.Uri != null)
.Select(r => r.Uri)
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri;
var unparsedCount = batch.Count(r => r.Uri == null);
var allBackfill = batch.All(r => r.IsBackfill);
var earliest = batch[0].TimeUtc;
var data = new Dictionary<string, object>
{
{ "windowsEventId", EventId_PolicyRebootRequired },
{ "eventLogChannel", Channel },
{ "recordCount", batch.Count },
{ "uriCount", distinctUris.Count },
{ "backfilled", allBackfill },
};
if (distinctUris.Count > 0)
{
data["rebootUris"] = distinctUris
.OrderBy(u => u, StringComparer.OrdinalIgnoreCase)
.Take(MaxUrisPerEvent)
.ToList();
data["firstRebootUri"] = firstUri;
}
if (unparsedCount > 0)
{
data["unparsedCount"] = unparsedCount;
var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description;
if (sample != null)
data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample;
}
var flag = SafeProbeRebootRequiredFlag();
if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value;
string message;
if (distinctUris.Count == 1)
message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}";
else if (distinctUris.Count > 1)
message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)";
else
message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)";
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
// Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts
// must land on the timeline at when the policies were actually flagged.
Timestamp = earliest,
EventType = Constants.EventTypes.MdmPolicyRebootRequired,
// Neutral observation — reboot-required flags outside ESP DeviceSetup have no
// immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an
// actually-observed system_reboot_detected.
Severity = EventSeverity.Info,
Source = "MdmRebootPolicyWatcher",
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
// Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup
// precedes a coalesced reboot that kills the agent.
ImmediateUpload = true,
});
_logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " +
$"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})");
}
/// <summary>
/// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData
/// value that starts with <c>./</c>, (2) a permissive regex over the formatted description
/// (verified shape: <c>The following URI has triggered a reboot: (./Device/...).</c>).
/// Returns null when nothing URI-like is found — the record still counts into the
/// aggregate (<c>unparsedCount</c> + <c>sampleDescription</c>).
/// </summary>
internal static string ExtractRebootUri(Dictionary<string, string> eventData, string description)
{
if (eventData != null)
{
foreach (var value in eventData.Values)
{
var trimmed = value?.Trim();
if (!string.IsNullOrEmpty(trimmed) && trimmed.StartsWith("./", StringComparison.Ordinal))
return trimmed;
}
}
if (!string.IsNullOrEmpty(description))
{
var match = UriPattern.Match(description);
if (match.Success)
{
// The permissive pattern can swallow sentence punctuation trailing the URI.
// OMA-DM URI segments never end in punctuation — trim it.
var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')');
if (candidate.Length > 2) return candidate;
}
}
return null;
}
/// <summary>
/// Extracts <c>&lt;Data Name="..."&gt;value&lt;/Data&gt;</c> pairs from an event's rendered
/// XML, keyed case-insensitively. Also collects UNNAMED Data elements under positional keys
/// (<c>Data0</c>, <c>Data1</c>, …) — DM-Enterprise events frequently carry positional-only
/// payloads. Namespace-agnostic; returns an empty map on null/malformed XML — never throws.
/// </summary>
internal static Dictionary<string, string> ParseEventData(string xml)
{
var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrEmpty(xml)) return result;
try
{
var doc = XDocument.Parse(xml);
var positionalIndex = 0;
foreach (var el in doc.Descendants().Where(e => e.Name.LocalName == "Data"))
{
var nameAttr = el.Attribute("Name");
var key = nameAttr != null && !string.IsNullOrEmpty(nameAttr.Value)
? nameAttr.Value
: "Data" + positionalIndex++;
if (!result.ContainsKey(key))
result[key] = el.Value;
}
}
catch
{
// Malformed / unexpected XML — best effort, return what we have.
}
return result;
}
// -----------------------------------------------------------------------
// OMA-DM RebootRequired flag probe (fail-soft corroboration)
// -----------------------------------------------------------------------
private bool? SafeProbeRebootRequiredFlag()
{
try
{
return RebootRequiredFlagProbe?.Invoke();
}
catch
{
return null; // corroboration only — never let the probe break the emit
}
}
/// <summary>
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. Both a value and a
/// subkey named <c>RebootRequired</c> count as "pending". Fail-soft: null = unknown
/// (key absent / unreadable), matching the EspSkipConfigurationProbe convention.
/// </summary>
internal static bool? ReadOmadmRebootRequiredFlag()
{
try
{
using (var key = Microsoft.Win32.Registry.LocalMachine.OpenSubKey(OmadmSyncMlKeyPath))
{
if (key == null) return null;
if (key.GetValue("RebootRequired") != null) return true;
using (var sub = key.OpenSubKey("RebootRequired"))
{
return sub != null ? (bool?)true : false;
}
}
}
catch
{
return null;
}
}
// -----------------------------------------------------------------------
// Backfill
// -----------------------------------------------------------------------
private void BackfillRecentEvents()
{
try
{
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
var xpath = $"*[System[(EventID={EventId_PolicyRebootRequired}) and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]";
_logger.Info($"MdmRebootPolicy backfill: scanning {Channel} " +
$"(lookback={_backfillLookbackMinutes}min, restartWatermark={_restartWatermark})");
var query = new EventLogQuery(Channel, PathType.LogName, xpath);
int processed = 0;
using (var reader = new EventLogReader(query))
{
EventRecord record;
while ((record = reader.ReadEvent()) != null)
{
using (record)
{
ProcessRecord(record, isBackfill: true);
processed++;
}
}
}
if (processed == 0)
_logger.Debug($"MdmRebootPolicy backfill: no EventID {EventId_PolicyRebootRequired} records in last {_backfillLookbackMinutes} minutes");
else
_logger.Info($"MdmRebootPolicy backfill: scanned {processed} record(s)");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found during backfill: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy backfill access denied: {ex.Message}");
}
catch (Exception ex)
{
_logger.Warning($"MdmRebootPolicy backfill failed: {ex.Message}");
}
}
// -----------------------------------------------------------------------
// Watermark dedup (cross-restart) — persisted at flush, not at claim
// -----------------------------------------------------------------------
private bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_stateLock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
internal void LoadWatermark()
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
if (!File.Exists(filePath)) return;
try
{
var json = File.ReadAllText(filePath);
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
if (state != null)
{
lock (_stateLock)
{
_restartWatermark = state.LastRecordId;
_maxPersistedRecordId = state.LastRecordId;
}
_logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
}
catch (Exception ex)
{
_logger.Warning($"Failed to load MdmRebootPolicy watermark: {ex.Message}");
}
}
private void PersistWatermark(long recordId)
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
try
{
Directory.CreateDirectory(_stateDirectory);
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
var state = new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow };
var json = JsonConvert.SerializeObject(state, Formatting.Indented);
var tempPath = filePath + ".tmp";
File.WriteAllText(tempPath, json);
File.Copy(tempPath, filePath, overwrite: true);
try { File.Delete(tempPath); } catch { }
}
catch (Exception ex)
{
_logger.Warning($"Failed to persist MdmRebootPolicy watermark: {ex.Message}");
}
}
internal sealed class WatermarkState
{
public long LastRecordId { get; set; }
public DateTime PersistedUtc { get; set; }
}
}
}
@@ -17,6 +17,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete
/// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure
///
/// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was
/// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker
/// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves
/// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in
/// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot.
///
/// Raises <see cref="FinalizingSetupPhaseTriggered"/>, <see cref="WhiteGloveCompleted"/>,
/// and <see cref="EspFailureDetected"/>. Cross-notifies the <see cref="HelloTracker"/>
/// on Hello wizard start and ESP exit so Hello timers can react.
@@ -12,6 +12,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
{
public string CollectorType => "command_allowlisted";
private const int CommandTimeoutMs = 30000;
/// <summary>
/// Returns what the reader captured, or an empty string if it did not finish
/// (a killed process can leave the read faulted or still pending).
/// </summary>
private static string DrainOrEmpty(System.Threading.Tasks.Task<string> readTask)
{
try
{
return readTask.Wait(2000) ? readTask.Result : string.Empty;
}
catch
{
return string.Empty;
}
}
public Dictionary<string, object> Execute(GatherRule rule, GatherRuleContext context)
{
var data = new Dictionary<string, object>();
@@ -81,12 +99,29 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
using (var process = Process.Start(psi))
{
var output = process.StandardOutput.ReadToEnd();
var error = process.StandardError.ReadToEnd();
process.WaitForExit(30000); // 30 second timeout
// Read asynchronously: a synchronous ReadToEnd() blocks until the pipe
// closes, which only happens when the process exits — the WaitForExit
// timeout below would never be reached and a hung command would pin
// this worker forever.
var outputTask = process.StandardOutput.ReadToEndAsync();
var errorTask = process.StandardError.ReadToEndAsync();
var exited = process.WaitForExit(CommandTimeoutMs);
if (!exited)
{
context.Logger.Warning(
$"Command timed out after {CommandTimeoutMs / 1000}s, killing: {command} (Rule: {rule.RuleId})");
try { process.Kill(); } catch { /* already gone */ }
// Give the readers a moment to drain once the pipes close.
try { process.WaitForExit(5000); } catch { /* best effort */ }
data["timed_out"] = true;
}
data["command"] = command;
data["exit_code"] = process.ExitCode;
data["exit_code"] = exited ? process.ExitCode : -1;
var output = DrainOrEmpty(outputTask);
var error = DrainOrEmpty(errorTask);
if (!string.IsNullOrWhiteSpace(output))
{
@@ -17,6 +17,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (string.IsNullOrEmpty(logName))
return data;
// Guard: only allow enrollment-relevant channels
if (!GatherRuleGuards.IsEventLogChannelAllowed(logName, context.UnrestrictedMode))
return context.EmitSecurityWarning(rule, "eventlog", logName);
// Optional: escalate severity when matching entries are found
string severityIfFound = null;
rule.Parameters?.TryGetValue("severityIfFound", out severityIfFound);
@@ -34,11 +34,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (filePath == null)
return null; // Token present but no user logged on — skip silently
// Apply IME log path override if set
// Apply IME log path override if set. The override comes from the local
// --ime-log-path flag only (never from remote config), so the operator
// running it is already a local admin — it relaxes the allowlist the same
// way unrestricted mode does, while the hard blocks below still apply.
var localOverride = false;
if (!string.IsNullOrEmpty(context.ImeLogPathOverride))
{
var fileName = Path.GetFileName(filePath);
filePath = Path.Combine(context.ImeLogPathOverride, fileName);
localOverride = true;
}
var relaxAllowlist = context.UnrestrictedMode || localOverride;
// Guard: only allow enrollment-relevant file paths. Checked before resolution
// so a blocked target is reported even when no file matches it. A wildcard
// target is checked by its directory — Path.GetFullPath rejects '*' and '?'.
var guardPath = HasWildcard(Path.GetFileName(filePath))
? Path.GetDirectoryName(filePath)
: filePath;
if (!GatherRuleGuards.IsFilePathAllowed(guardPath, relaxAllowlist, userProfilePath))
{
context.EmitSecurityWarning(rule, "logparser", filePath);
return null;
}
// Get parameters
@@ -87,18 +106,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
foreach (var resolvedPath in resolvedPaths)
{
// Re-check each resolved path: wildcard expansion can surface files
// reached through a junction that the directory check did not cover.
if (!GatherRuleGuards.IsFilePathAllowed(resolvedPath, relaxAllowlist, userProfilePath))
{
context.EmitSecurityWarning(rule, "logparser", resolvedPath);
continue;
}
ProcessLogFile(resolvedPath, rule, pattern, trackPosition, maxLines, isTextMode, context);
}
return null;
}
private static bool HasWildcard(string fileNamePart)
=> !string.IsNullOrEmpty(fileNamePart) &&
(fileNamePart.Contains("*") || fileNamePart.Contains("?"));
private static List<string> ResolveLogPaths(string filePath, string ruleId, GatherRuleContext context)
{
var fileNamePart = Path.GetFileName(filePath);
// No wildcards — single file
if (!fileNamePart.Contains("*") && !fileNamePart.Contains("?"))
if (!HasWildcard(fileNamePart))
{
if (File.Exists(filePath))
return new List<string> { filePath };
@@ -1,6 +1,9 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
@@ -28,6 +31,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
private readonly HashSet<string> _phaseRulesExecuted = new HashSet<string>();
private CountdownEvent _startupRulesLatch; // non-null only while startup rules are pending
// Phase-scope + emit-mode state (ActivePhases / ActiveFromPhase / EmitMode on GatherRule).
// Guarded by _scopeLock; collector work always runs outside the lock. The state deliberately
// survives UpdateRules config refreshes — an agent restart causes at most one re-emit per
// on_change rule (acceptable under the per-enrollment lifecycle).
private readonly object _scopeLock = new object();
private EnrollmentPhase _currentPhase = EnrollmentPhase.Unknown;
private readonly HashSet<string> _fromPhaseLatched = new HashSet<string>();
private readonly Dictionary<string, string> _lastEmittedHash = new Dictionary<string, string>();
private readonly Dictionary<string, (int Count, DateTime SinceUtc)> _suppressed =
new Dictionary<string, (int Count, DateTime SinceUtc)>();
private readonly HashSet<string> _scopeBypassLogged = new HashSet<string>();
/// <summary>
/// When true, phase scoping (ActivePhases / ActiveFromPhase) is bypassed and scoped rules
/// execute unconditionally. Used by the standalone <c>--run-gather-rules</c> diagnostic
/// mode, which has no enrollment phase context. EmitMode is unaffected.
/// </summary>
public bool IgnorePhaseScope { get; set; }
/// <summary>
/// When true, guardrails are relaxed: all registry, WMI, and command targets are allowed.
/// File paths allow everything except C:\Users. Set from tenant configuration.
@@ -89,10 +111,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_activeRules = rules.Where(r => r.Enabled).ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait
var pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait.
// Scoped startup rules that are not yet in scope are deferred: they run once from
// OnPhaseChanged when their scope activates (same _startupRulesExecuted dedup — a
// rule runs either here or there, never both). Unscoped rules behave as before.
List<GatherRule> pendingStartup;
lock (_scopeLock)
{
// A config refresh may deliver new from-phase rules after their phase was
// already reached — evaluate latches against the current phase first.
EvaluateFromPhaseLatchesLocked();
pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.Where(r => IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
if (pendingStartup.Count > 0)
{
@@ -101,7 +140,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
ThreadPool.QueueUserWorkItem(_ =>
{
try { ExecuteRule(rule); }
@@ -110,18 +148,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// Set up interval timers
// Set up interval timers. Timers for scoped rules keep running while out of scope —
// the tick gate makes an out-of-scope tick a free no-op, avoiding start/stop
// lifecycle races on phase transitions.
foreach (var rule in _activeRules.Where(r => r.Trigger == "interval" && r.IntervalSeconds.HasValue))
{
var interval = TimeSpan.FromSeconds(rule.IntervalSeconds.Value);
var intervalRule = rule;
var interval = TimeSpan.FromSeconds(intervalRule.IntervalSeconds.Value);
var timer = new Timer(
_ => ExecuteRule(rule),
_ =>
{
if (!IsRuleInScope(intervalRule))
return;
ExecuteRule(intervalRule);
},
null,
interval, // Initial delay = one interval
interval
);
_intervalTimers[rule.RuleId] = timer;
_logger.Info($" Interval rule {rule.RuleId} scheduled every {rule.IntervalSeconds}s");
_intervalTimers[intervalRule.RuleId] = timer;
_logger.Info($" Interval rule {intervalRule.RuleId} scheduled every {intervalRule.IntervalSeconds}s");
}
_logger.Info($"GatherRuleExecutor: {_activeRules.Count(r => r.Trigger == "startup")} startup, " +
@@ -130,16 +176,95 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// <summary>
/// Called when a phase change event occurs - executes rules triggered by phase changes
/// (<c>phase_change</c> on entering the new phase, <c>phase_exit</c> on leaving the old one).
/// </summary>
public void OnPhaseChanged(EnrollmentPhase newPhase)
{
var phaseName = newPhase.ToString();
// phase_exit runs FIRST and against the phase being LEFT: both the TriggerPhase match
// and the scope gate must see the old phase, so this block has to complete before
// _currentPhase moves below. A transition INTO Failed still fires the exit rules of the
// phase that failed — that snapshot is the whole point at a failure boundary.
List<GatherRule> exitRules = null;
lock (_scopeLock)
{
var previousPhase = _currentPhase;
if (previousPhase != EnrollmentPhase.Unknown && previousPhase != newPhase)
{
var previousPhaseName = previousPhase.ToString();
exitRules = new List<GatherRule>();
foreach (var rule in _activeRules)
{
if (rule.Trigger != "phase_exit") continue;
if (!string.IsNullOrEmpty(rule.TriggerPhase)
&& !string.Equals(rule.TriggerPhase, previousPhaseName, StringComparison.OrdinalIgnoreCase))
continue;
// Scope gate before the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) slot. IsRuleInScopeLocked still sees the old phase.
if (!IsRuleInScopeLocked(rule)) continue;
// "exit:" prefix keeps this key space disjoint from the phase_change keys.
if (!_phaseRulesExecuted.Add($"exit:{rule.RuleId}|{previousPhaseName}"))
{
_logger.Debug($"Phase-exit rule {rule.RuleId} already executed for exit of {previousPhaseName}, skipping");
continue;
}
_logger.Info($"Phase exit triggered rule {rule.RuleId} (left phase: {previousPhaseName}, now: {phaseName})");
exitRules.Add(rule);
}
}
}
if (exitRules != null)
{
foreach (var rule in exitRules)
{
var exitRule = rule;
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(exitRule));
}
}
// Update the phase + from-phase latches so deferred startup activation and
// the phase_change rules below all evaluate against the NEW phase.
List<GatherRule> deferredStartup;
lock (_scopeLock)
{
_currentPhase = newPhase;
EvaluateFromPhaseLatchesLocked();
deferredStartup = _activeRules
.Where(r => r.Trigger == "startup" && HasPhaseScope(r)
&& !_startupRulesExecuted.Contains(r.RuleId)
&& IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in deferredStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
foreach (var rule in deferredStartup)
{
var deferredRule = rule;
_logger.Info($"Phase {phaseName} activated deferred startup rule {deferredRule.RuleId}");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(deferredRule));
}
foreach (var rule in _activeRules.Where(r => r.Trigger == "phase_change"))
{
if (string.IsNullOrEmpty(rule.TriggerPhase) ||
string.Equals(rule.TriggerPhase, phaseName, StringComparison.OrdinalIgnoreCase))
{
// Scope gate BEFORE the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) execution slot.
if (!IsRuleInScope(rule))
continue;
// Deduplicate: only fire once per (ruleId, phase) combination
var deduplicationKey = $"{rule.RuleId}|{phaseName}";
if (!_phaseRulesExecuted.Add(deduplicationKey))
@@ -163,6 +288,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
if (string.Equals(rule.TriggerEventType, eventType, StringComparison.OrdinalIgnoreCase))
{
if (!IsRuleInScope(rule))
continue;
_logger.Info($"Event triggered rule {rule.RuleId} (event: {eventType})");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
}
@@ -192,6 +320,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (result.Count > 0)
{
// EmitMode "on_change": hash the raw collector result (before ruleId/title
// injection) and only emit when it differs from the last emitted one. An
// empty result never reaches this point, so an emitOnlyIfExists miss neither
// emits nor updates the hash — composing to "one event on appearance, then
// only on change".
if (IsOnChangeMode(rule) && !ShouldEmitOnChange(rule, result))
return;
result["ruleId"] = rule.RuleId;
result["ruleTitle"] = rule.Title;
@@ -229,6 +365,199 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// ===== Phase scope (ActivePhases / ActiveFromPhase) =====
private static bool HasPhaseScope(GatherRule rule)
{
return (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
|| !string.IsNullOrEmpty(rule.ActiveFromPhase);
}
// Internal so the scope matrix is testable synchronously (no timer/ThreadPool races).
internal bool IsRuleInScope(GatherRule rule)
{
lock (_scopeLock)
{
return IsRuleInScopeLocked(rule);
}
}
private bool IsRuleInScopeLocked(GatherRule rule)
{
if (!HasPhaseScope(rule))
return true; // unscoped — legacy behavior
if (IgnorePhaseScope)
{
if (_scopeBypassLogged.Add(rule.RuleId))
_logger.Info($"Gather rule {rule.RuleId}: phase scope ignored (diagnostic mode)");
return true;
}
// Both fields set is rejected by backend validation; defensively prefer ActivePhases.
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
{
// No phase signal yet (or scope tokens that never match, e.g. "Unknown"):
// scoped rules stay inactive.
if (_currentPhase == EnrollmentPhase.Unknown)
return false;
var phaseName = _currentPhase.ToString();
foreach (var phase in rule.ActivePhases)
{
if (string.Equals(phase, phaseName, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
return _fromPhaseLatched.Contains(rule.RuleId);
}
/// <summary>
/// Latches every from-phase rule whose threshold the current phase has reached.
/// Never latches on Unknown/Failed (Failed=99 would ordinal-satisfy every threshold);
/// once latched a rule stays active for the session — including through Failed.
/// Caller must hold <see cref="_scopeLock"/>.
/// </summary>
private void EvaluateFromPhaseLatchesLocked()
{
if (_currentPhase == EnrollmentPhase.Unknown || _currentPhase == EnrollmentPhase.Failed)
return;
foreach (var rule in _activeRules)
{
if (string.IsNullOrEmpty(rule.ActiveFromPhase)) continue;
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0) continue; // ActivePhases wins
if (_fromPhaseLatched.Contains(rule.RuleId)) continue;
if (Enum.TryParse<EnrollmentPhase>(rule.ActiveFromPhase, ignoreCase: true, out var fromPhase)
&& fromPhase != EnrollmentPhase.Unknown
&& fromPhase != EnrollmentPhase.Failed
&& (int)_currentPhase >= (int)fromPhase)
{
_fromPhaseLatched.Add(rule.RuleId);
_logger.Info($"Gather rule {rule.RuleId}: from-phase scope latched at {_currentPhase} (activeFromPhase={rule.ActiveFromPhase})");
}
}
}
// ===== Emit mode (on_change) =====
private static bool IsOnChangeMode(GatherRule rule)
{
return string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// on_change decision for a non-empty collector result: true = emit (first result for the
/// rule, or the result changed), false = suppress. Hashes the result BEFORE ruleId/title
/// injection. On emit after a suppression streak, injects <c>suppressedPolls</c> and
/// <c>suppressedSinceUtc</c> into the result so the gap is observable.
/// Internal so the state machine is testable synchronously (no timer/ThreadPool races).
/// </summary>
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result)
{
var hash = ComputeCanonicalHash(result);
bool emit;
lock (_scopeLock)
{
string lastHash;
if (_lastEmittedHash.TryGetValue(rule.RuleId, out lastHash) && lastHash == hash)
{
(int Count, DateTime SinceUtc) streak;
_suppressed[rule.RuleId] = _suppressed.TryGetValue(rule.RuleId, out streak)
? (streak.Count + 1, streak.SinceUtc)
: (1, DateTime.UtcNow);
emit = false;
}
else
{
_lastEmittedHash[rule.RuleId] = hash;
(int Count, DateTime SinceUtc) streak;
if (_suppressed.TryGetValue(rule.RuleId, out streak) && streak.Count > 0)
{
result["suppressedPolls"] = streak.Count;
result["suppressedSinceUtc"] = streak.SinceUtc.ToString("o", CultureInfo.InvariantCulture);
}
_suppressed.Remove(rule.RuleId);
emit = true;
}
}
if (!emit)
_logger.Debug($"Gather rule {rule.RuleId}: result unchanged, emit suppressed (on_change)");
return emit;
}
/// <summary>
/// Canonical hash of a collector result: keys ordinal-sorted at every dictionary level,
/// values rendered with invariant formatting, then SHA-256. Internal for direct test
/// coverage of the canonicalization.
/// </summary>
internal static string ComputeCanonicalHash(Dictionary<string, object> result)
{
var sb = new StringBuilder();
AppendCanonicalValue(sb, result);
using (var sha = SHA256.Create())
{
var hashBytes = sha.ComputeHash(Encoding.UTF8.GetBytes(sb.ToString()));
return Convert.ToBase64String(hashBytes);
}
}
private static void AppendCanonicalValue(StringBuilder sb, object value)
{
if (value == null)
{
sb.Append("<null>");
return;
}
// string first — it is IEnumerable and must not be treated as a sequence
if (value is string str)
{
sb.Append('"').Append(str).Append('"');
return;
}
if (value is System.Collections.IDictionary dict)
{
sb.Append('{');
var keys = dict.Keys.Cast<object>()
.Select(k => k?.ToString() ?? string.Empty)
.OrderBy(k => k, StringComparer.Ordinal);
foreach (var key in keys)
{
sb.Append(key).Append('=');
AppendCanonicalValue(sb, dict[key]);
sb.Append(';');
}
sb.Append('}');
return;
}
if (value is System.Collections.IEnumerable sequence)
{
sb.Append('[');
foreach (var item in sequence)
{
AppendCanonicalValue(sb, item);
sb.Append(';');
}
sb.Append(']');
return;
}
if (value is IFormattable formattable)
{
sb.Append(formattable.ToString(null, CultureInfo.InvariantCulture));
return;
}
sb.Append(value);
}
private EventSeverity ParseSeverity(string severity)
{
return GatherRuleContext.ParseSeverity(severity);
@@ -245,7 +574,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// <summary>
/// Blocks until all startup rules that were queued by the most recent <see cref="UpdateRules"/>
/// call have finished executing, or the timeout elapses.
/// call have finished executing, or the timeout elapses. Deferred scoped startup rules
/// (waiting for their phase scope) are not part of the latch.
/// Returns true if all rules completed within the timeout; false if timed out.
/// </summary>
public bool WaitForStartupRules(int timeoutSeconds = 120)
@@ -33,6 +33,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public static readonly IReadOnlyList<string> AllowedFilePrefixes;
public static readonly IReadOnlyList<string> AllowedWmiQueryPrefixes;
public static readonly IReadOnlyCollection<string> AllowedCommands;
public static readonly IReadOnlyList<string> AllowedEventLogChannels;
// Hard blocks: never allowed, even in unrestricted mode
private static readonly string BlockedUsersPrefix = Path.GetFullPath(@"C:\Users");
@@ -51,6 +52,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// Hard limit: maximum command length (even in unrestricted mode)
private const int MaxCommandLength = 2000;
// Event log channels that are never readable, even in unrestricted mode.
// Security carries the audit trail of user behaviour; the PowerShell channels
// carry script-block logging, which routinely contains secrets in clear text.
// Mirrored in rules/guardrails.json ("blockedEventLogChannels") for the portal
// to display — enforcement lives here so a parse error cannot lift it.
private static readonly string[] HardBlockedEventLogChannels = new[]
{
"Security",
"Microsoft-Windows-PowerShell",
"Windows PowerShell",
"Microsoft-Windows-Sysmon",
};
// Hard-blocked command patterns: never allowed, even in unrestricted mode.
// These prevent privilege escalation, persistence, and data exfiltration.
private static readonly string[] HardBlockedCommandPatterns = new[]
@@ -85,6 +99,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
var commands = FlattenCategorized(obj, "allowedCommands", "commands");
AllowedCommands = new HashSet<string>(commands, StringComparer.OrdinalIgnoreCase);
AllowedEventLogChannels = FlattenCategorized(obj, "eventLogChannels", "channels");
return;
}
}
@@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
AllowedFilePrefixes = new List<string>();
AllowedWmiQueryPrefixes = new List<string>();
AllowedCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
AllowedEventLogChannels = new List<string>();
}
private static string LoadEmbeddedGuardrails()
@@ -257,6 +274,44 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
(trimmed.Length == prefix.Length || char.IsWhiteSpace(trimmed[prefix.Length])));
}
/// <summary>
/// Returns true if the event log channel matches an allowed entry with
/// boundary matching (next char must be '/' or end of string), so that
/// "Microsoft-Windows-AAD" admits "Microsoft-Windows-AAD/Operational"
/// but not "Microsoft-Windows-AADSomethingElse".
/// </summary>
public static bool IsEventLogChannelAllowed(string channel)
=> IsEventLogChannelAllowed(channel, unrestrictedMode: false);
/// <summary>
/// Returns true if the event log channel is allowed.
/// When unrestrictedMode is true, all channels are allowed except the
/// hard-blocked ones (audit trail and script-block logging).
/// </summary>
public static bool IsEventLogChannelAllowed(string channel, bool unrestrictedMode)
{
if (string.IsNullOrEmpty(channel))
return false;
var trimmed = channel.Trim();
// Hard blocks apply even in unrestricted mode
foreach (var blocked in HardBlockedEventLogChannels)
{
if (MatchesChannel(trimmed, blocked))
return false;
}
if (unrestrictedMode)
return true;
return AllowedEventLogChannels.Any(allowed => MatchesChannel(trimmed, allowed));
}
private static bool MatchesChannel(string channel, string prefix)
=> channel.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) &&
(channel.Length == prefix.Length || channel[prefix.Length] == '/');
/// <summary>
/// Returns true if the command (trimmed) exactly matches an entry in the allowlist.
/// Exact matching is intentional — no prefix spoofing possible.
@@ -136,6 +136,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Security
}
watcher.Start();
// Re-check after arming: Dispose() may have run between the lock release above and
// Start() — it then stopped a not-yet-armed watcher and this thread armed it right
// back, leaving a WMI subscription no code path could ever reach again. If disposal
// won that race, tear the watcher down here.
bool disposedDuringArm;
lock (_lock)
{
disposedDuringArm = _disposed;
if (disposedDuringArm) _startWatcher = null;
}
if (disposedDuringArm)
{
try { watcher.EventArrived -= OnProcessStartTrace; } catch { }
try { watcher.Stop(); } catch { }
try { watcher.Dispose(); } catch { }
return;
}
_logger.Info($"[ConsoleBypassWatcher] watching Win32_ProcessStartTrace for " +
$"{string.Join(", ", WatchedProcessNames)} (interactive-session + bare command line)");
}
@@ -335,6 +335,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
osBuildChangedProvider: () => osBuildChangeHost.BuildChanged));
}
// MDM reboot-policy watcher — subscribes to DeviceManagement-Enterprise-Diagnostics-
// Provider/Admin EventID 2800 and attributes the "unexpected reboot + second sign-in"
// pattern to the device-assigned policy URIs that forced the coalesced reboot.
// Agent-native collector (NOT an IAgentAnalyzer); the config flag is an internal
// steering lever only.
if (collectors.MdmRebootPolicyWatcherEnabled)
{
hosts.Add(new MdmRebootPolicyWatcherHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
backfillLookbackMinutes: collectors.MdmRebootPolicyBackfillLookbackMinutes,
stateDirectory: _stateDirectory));
}
// ----- Peripheral hosts (event-only; driven by remote-config toggles) --------------
// V1 parity (PeriodicCollectorManager) — combine Performance + AgentSelfMetrics under
@@ -86,6 +86,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// Production always passes null → Start() creates the real DeadlineScheduler.
private readonly IDeadlineScheduler? _schedulerOverride;
// Live accessor for TenantConfiguration.SendTraceEvents — read per event by the
// TelemetryEventEmitter so a mid-session remote-config merge takes effect. Null
// (tests / legacy callers) means trace events always flow.
private readonly Func<bool>? _traceEventsEnabled;
// Max-lifetime watchdog (M4.6.α). Timer is armed in Start() when _agentMaxLifetime != null.
private System.Threading.Timer? _maxLifetimeTimer;
private int _terminatedFired;
@@ -166,7 +171,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
TimeSpan? terminalDrainTimeout = null,
TimeSpan? agentMaxLifetime = null,
int uploadBatchSize = 100,
IDeadlineScheduler? schedulerOverride = null)
IDeadlineScheduler? schedulerOverride = null,
Func<bool>? traceEventsEnabled = null)
{
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
@@ -206,6 +212,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
_uploadBatchSize = uploadBatchSize;
_schedulerOverride = schedulerOverride;
_traceEventsEnabled = traceEventsEnabled;
}
/// <summary>
@@ -402,7 +409,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// strukturelle Abhängigkeit erlischt damit, Architektur-Baseline-Test gate
// shrinks to exactly two permitted callers.
_eventSequenceCounter = new EventSequenceCounter(_eventSequencePersistence);
var eventEmitter = new TelemetryEventEmitter(_transport, _eventSequenceCounter, _sessionId, _tenantId);
var eventEmitter = new TelemetryEventEmitter(
_transport, _eventSequenceCounter, _sessionId, _tenantId, _traceEventsEnabled);
_signalEmitter = new TelemetrySignalEmitter(_transport, _sessionId, _tenantId);
_transitionEmitter = new TelemetryTransitionEmitter(_transport, _sessionId, _tenantId);
_timelineEmitter = new EventTimelineEmitter(eventEmitter);
@@ -0,0 +1,68 @@
#nullable enable
using System;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Standalone host for the <see cref="MdmRebootPolicyTracker"/> — watches
/// <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> (EventID
/// 2800) for MDM policies that force a coalesced reboot during ESP DeviceSetup and forwards
/// them via the shared single-rail ingress. Mirrors <see cref="WindowsUpdateWatcherHost"/>:
/// builds its own <see cref="InformationalEventPost"/> and delegates Start/Stop to the tracker.
/// </summary>
internal sealed class MdmRebootPolicyWatcherHost : ICollectorHost
{
public string Name => "MdmRebootPolicyTracker";
private readonly MdmRebootPolicyTracker _tracker;
private readonly AgentLogger _logger;
private int _disposed;
public MdmRebootPolicyWatcherHost(
string sessionId,
string tenantId,
AgentLogger logger,
ISignalIngressSink ingress,
IClock clock,
int backfillLookbackMinutes,
string stateDirectory)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
var post = new InformationalEventPost(ingress, clock, logger);
_tracker = new MdmRebootPolicyTracker(
sessionId: sessionId,
tenantId: tenantId,
post: post,
logger: logger,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory);
}
public void Start()
{
_tracker.Start();
_logger.Info("MdmRebootPolicyWatcherHost: started.");
}
public void Stop()
{
try { _tracker.Stop(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicyWatcherHost: stop failed: {ex.Message}"); }
}
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) == 1) return;
Stop();
}
}
}
@@ -30,12 +30,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
private readonly string _sessionId;
private readonly string _tenantId;
private readonly string _partitionKey;
private readonly Func<bool>? _traceEventsEnabled;
public TelemetryEventEmitter(
ITelemetryTransport transport,
EventSequenceCounter sequenceCounter,
string sessionId,
string tenantId)
string tenantId,
Func<bool>? traceEventsEnabled = null)
{
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
@@ -44,13 +46,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
_sessionId = sessionId;
_tenantId = tenantId;
_partitionKey = $"{tenantId}_{sessionId}";
_traceEventsEnabled = traceEventsEnabled;
}
/// <summary>
/// Weist Sequence zu, serialisiert im Legacy-Wire-Format und enqueued via Transport.
/// Thread-safe (EventSequenceCounter + ITelemetryTransport.Enqueue beide unter Lock).
/// <para>
/// Returns <c>null</c> when the event was suppressed by the trace-event gate — the
/// only drop path. Both permitted callers ignore the return value.
/// </para>
/// </summary>
internal TelemetryItem Emit(EnrollmentEvent evt)
internal TelemetryItem? Emit(EnrollmentEvent evt)
{
if (evt == null) throw new ArgumentNullException(nameof(evt));
if (string.IsNullOrEmpty(evt.EventType))
@@ -58,6 +65,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
throw new ArgumentException("EnrollmentEvent.EventType is mandatory.", nameof(evt));
}
// Trace-event gate (TenantConfiguration.SendTraceEvents, default on). Dropped
// before the sequence counter advances, so suppression leaves no gaps in the
// per-session sequence and the timeline stays contiguous. Evaluated per event
// rather than captured at construction: remote config is merged into the live
// AgentConfiguration mid-session, and the gate must follow it.
if (evt.Severity == EventSeverity.Trace && !TraceEventsEnabled())
{
return null;
}
evt.Sequence = _sequenceCounter.Next();
if (string.IsNullOrEmpty(evt.SessionId)) evt.SessionId = _sessionId;
if (string.IsNullOrEmpty(evt.TenantId)) evt.TenantId = _tenantId;
@@ -77,5 +94,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
return _transport.Enqueue(draft);
}
/// <summary>
/// Fail-open: no gate wired (legacy ctor / tests) or a throwing accessor means trace
/// events flow. The gate suppresses diagnostics, so a broken accessor must not be able
/// to silently blind the timeline.
/// </summary>
private bool TraceEventsEnabled()
{
if (_traceEventsEnabled == null) return true;
try
{
return _traceEventsEnabled();
}
catch
{
return true;
}
}
}
}
@@ -224,6 +224,11 @@ namespace AutopilotMonitor.Agent.V2
// Step 4 — run the executor and wait.
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride))
{
// Diagnostic mode has no enrollment phase context — scoped rules
// (activePhases / activeFromPhase) execute unconditionally here.
executor.IgnorePhaseScope = true;
logger.Info("--run-gather-rules: phase scoping bypassed (no phase context in diagnostic mode).");
executor.UpdateRules(rules);
if (consoleMode) Console.WriteLine($"Running {startupRules.Count} rule(s)...");
@@ -132,7 +132,10 @@ namespace AutopilotMonitor.Agent.V2.Runtime
whiteGloveSealingPatternIds: whiteGloveSealingPatternIds,
drainInterval: drainInterval,
agentMaxLifetime: agentMaxLifetime,
uploadBatchSize: uploadBatchSize))
uploadBatchSize: uploadBatchSize,
// Live read: remote config is merged into agentConfig mid-session, so the
// trace-event gate must follow it rather than snapshot at construction.
traceEventsEnabled: () => agentConfig.SendTraceEvents))
{
using (var shutdown = new ManualResetEventSlim(false))
using (var shutdownComplete = new ManualResetEventSlim(false))
@@ -0,0 +1,83 @@
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey → business-time decoders. These are the read-path compensation for rows whose
/// system Timestamp was reset by the 2026-07-18 storage migration: the RowKey is the only
/// time carrier that survives a row rewrite, so the decode must round-trip exactly and
/// must reject every foreign format (falling back instead of mis-dating rows).
/// </summary>
public class BusinessTimestampDecodeTests
{
private static readonly DateTime Ts = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc).AddTicks(4567);
[Fact]
public void Audit_rowkey_roundtrips_tick_exact()
{
var rowKey = TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid());
Assert.True(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("!")]
[InlineData("!123")] // too short
[InlineData("!123456789012345678x_guid")] // non-digit in revtick
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // legacy bare GUID (:N)
[InlineData("0000000000000000000")] // ops-style bare revtick is not an audit key
public void Audit_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out _));
}
[Fact]
public void Ops_rowkey_roundtrips_tick_exact()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
Assert.True(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("123456789012345678")] // 18 digits
[InlineData("12345678901234567890")] // 20 digits
[InlineData("123456789012345678x")] // non-digit
[InlineData("!1234567890123456789")] // audit-prefixed
public void Ops_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out _));
}
[Fact]
public void Event_rowkey_prefix_decodes_to_millisecond_truncated_agent_time()
{
var rowKey = $"{Ts:yyyyMMddHHmmssfff}_{42L:D10}";
Assert.True(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out var decoded));
// The RowKey format truncates sub-millisecond ticks at write time.
var expected = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc);
Assert.Equal(expected, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("20260615103045123")] // no separator/sequence
[InlineData("2026061510304512_0000000042")] // 16-digit prefix
[InlineData("202606151030451234_0000000042")] // 18-digit prefix
[InlineData("2026061510304512x_0000000042")] // non-digit in prefix
[InlineData("20261315103045123_0000000042")] // month 13
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // GUID
public void Event_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out _));
}
}
@@ -0,0 +1,181 @@
using System.Text.RegularExpressions;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey-range date-window / retention clauses. Date filtering moved off the system
/// Timestamp (reset by storage migrations) onto the reverse-tick RowKey; these tests pin
/// (a) tick-exact inclusive/exclusive boundary semantics identical to the old
/// `Timestamp ge/le` clauses, and (b) the GUID-safety invariant: legacy bare-GUID audit
/// rows sort AFTER all '!'-rows, so any lower RowKey bound without the `RowKey lt '"'`
/// guard would match — and a retention sweep would DELETE — every legacy row.
/// </summary>
public class BusinessTimestampFilterClauseTests
{
private static readonly DateTime T = new DateTime(2026, 6, 15, 12, 0, 0, DateTimeKind.Utc).AddTicks(789);
private static readonly string Tenant = "11111111-1111-1111-1111-111111111111";
private static string AuditRk(DateTime ts) => TableStorageService.BuildAuditLogRowKey(ts, Guid.NewGuid());
private static string OpsRk(DateTime ts) => $"{DateTime.MaxValue.Ticks - ts.Ticks:D19}";
/// <summary>Evaluates a clause of the form "RowKey op 'literal' [and RowKey op 'literal']…" against a RowKey.</summary>
private static bool Matches(string clause, string rowKey)
{
var parts = Regex.Matches(clause, @"RowKey (ge|gt|le|lt) '([^']*)'");
Assert.True(parts.Count > 0, $"no RowKey comparisons in clause: {clause}");
foreach (Match m in parts)
{
var cmp = string.CompareOrdinal(rowKey, m.Groups[2].Value);
var ok = m.Groups[1].Value switch
{
"ge" => cmp >= 0,
"gt" => cmp > 0,
"le" => cmp <= 0,
"lt" => cmp < 0,
_ => throw new InvalidOperationException(),
};
if (!ok) return false;
}
return true;
}
// ===== Audit boundaries =====
[Fact]
public void Audit_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateFromClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == from → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → excluded
}
[Fact]
public void Audit_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateToClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == to → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // older → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // 1 tick newer → excluded
}
[Fact]
public void Audit_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.AuditRetentionClause(T);
Assert.False(Matches(clause, AuditRk(T))); // ts == cutoff → kept
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → deleted
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → kept
}
// ===== Audit GUID safety (the single most dangerous invariant of this change) =====
[Fact]
public void Legacy_guid_rowkeys_sort_after_the_time_encoded_guard()
{
for (var i = 0; i < 20; i++)
{
var guidRk = Guid.NewGuid().ToString("N");
Assert.True(string.CompareOrdinal(guidRk, BusinessTimestamp.AuditTimeEncodedUpperBound) > 0);
}
}
[Fact]
public void No_audit_date_or_retention_clause_matches_legacy_guid_rowkeys()
{
var clauses = new[]
{
BusinessTimestamp.AuditDateFromClause(T),
BusinessTimestamp.AuditDateToClause(T),
BusinessTimestamp.AuditDateToClause(DateTime.MaxValue.AddTicks(-1)), // widest possible window
BusinessTimestamp.AuditRetentionClause(T),
BusinessTimestamp.AuditRetentionClause(DateTime.MaxValue.AddTicks(-1)), // widest possible sweep
};
foreach (var clause in clauses)
{
for (var i = 0; i < 20; i++)
{
Assert.False(Matches(clause, Guid.NewGuid().ToString("N")),
$"clause matched a legacy GUID RowKey: {clause}");
}
}
}
[Fact]
public void Audit_lower_bounded_clauses_carry_the_time_encoded_guard()
{
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditDateToClause(T));
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditRetentionClause(T));
}
// ===== Ops boundaries =====
[Fact]
public void Ops_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateFromClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(-1))));
}
[Fact]
public void Ops_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateToClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
[Fact]
public void Ops_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.OpsRetentionClause(T);
Assert.False(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
// ===== Composition into the filter builders =====
[Fact]
public void Audit_filter_builders_emit_rowkey_windows_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var plain = TableStorageService.BuildAuditLogFilter(Tenant, from, to);
var fanOut = TableStorageService.BuildAuditLogFilterWithRowKeyBound(
Tenant, from, to, lastRowKey: "!0123_x", excludeDeletions: false);
foreach (var f in new[] { plain!, fanOut })
{
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.AuditDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.AuditDateToClause(to), f);
}
Assert.Contains("RowKey gt '!0123_x'", fanOut); // fan-out cursor composes with the window
}
[Fact]
public void Ops_filter_builder_emits_rowkey_window_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var f = TableOpsEventRepository.BuildFilter("Security", from, to);
Assert.NotNull(f);
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.OpsDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.OpsDateToClause(to), f);
}
}
@@ -0,0 +1,118 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Read-mapper resolution order for the three affected tables:
/// OccurredUtc column → RowKey decode → system Timestamp. The last hop is the
/// migration-corrupted value (a row rewrite resets it), so it must only ever win when
/// the row carries no better source (audit legacy bare-GUID rows).
/// </summary>
public class BusinessTimestampPrecedenceTests
{
private static readonly DateTime Occurred = new(2026, 5, 10, 8, 0, 0, DateTimeKind.Utc);
private static readonly DateTime RowKeyTime = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
private static readonly DateTimeOffset SystemTs = new(2026, 7, 18, 13, 58, 0, TimeSpan.Zero); // migration moment
private static TableEntity WithSystemTimestamp(TableEntity e)
{
e["Timestamp"] = SystemTs;
return e;
}
// ===== Audit =====
[Fact]
public void Audit_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid())));
Assert.Equal(RowKeyTime, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_legacy_guid_row_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("t", Guid.NewGuid().ToString("N")));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveAuditTimestamp(e));
}
// ===== Ops =====
[Fact]
public void Ops_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}")
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}"));
Assert.Equal(RowKeyTime, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("Security", "not-a-revtick"));
Assert.Equal(SystemTs.UtcDateTime, TableOpsEventRepository.ResolveTimestamp(e));
}
// ===== Events =====
private static string EventRowKey(DateTime ts, long seq = 42) => $"{ts:yyyyMMddHHmmssfff}_{seq:D10}";
[Fact]
public void Event_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_rowkey_prefix_wins_over_system_when_column_missing()
{
// The pre-cutover case: no OccurredUtc column, system Timestamp = migration moment.
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime)));
Assert.Equal(RowKeyTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", "weird-rowkey"));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_occurredutc_read_tolerates_datetime_materialization()
{
// SDK reads surface Edm.DateTime as DateTimeOffset or DateTime depending on payload
// shape — the resolver must accept both (dual-reader lesson).
var e = new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = Occurred, // plain DateTime, not DateTimeOffset
};
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
}
@@ -0,0 +1,283 @@
using AutopilotMonitor.Functions.Functions.Rules;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Azure;
using Azure.Data.Tables;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the three gather-rule scope/emit fields (ActivePhases, ActiveFromPhase, EmitMode):
/// Store↔Map table-serialization roundtrip (table-serialization rule: every model field in BOTH
/// directions), legacy rows without the new columns, the <see cref="GatherRuleService.ContentEquivalent"/>
/// null-tolerance (no reseed churn), and the create/update validation matrix in
/// <see cref="GatherRulesFunction.ValidateScopeAndEmitMode"/>.
/// </summary>
public class GatherRuleScopeFieldsTests
{
// ── Store ↔ Map roundtrip ──────────────────────────────────────────────
private static (TableStorageService service, Func<TableEntity?> lastUpserted) BuildStorageHarness()
{
TableEntity? captured = null;
var tableClient = new Mock<TableClient>();
tableClient
.Setup(c => c.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, TableUpdateMode, CancellationToken>((e, _, _) =>
{
captured = e;
return Task.FromResult(new Mock<Response>().Object);
});
var serviceClient = new Mock<TableServiceClient>();
serviceClient.Setup(c => c.GetTableClient(It.IsAny<string>())).Returns(tableClient.Object);
var service = new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance);
return (service, () => captured);
}
[Fact]
public async Task Roundtrip_ScopeFields_SurviveStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-001",
Title = "Scoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_scoped",
ActivePhases = new List<string> { "AccountSetup", "AppsUser" },
EmitMode = "on_change",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.NotNull(mapped.ActivePhases);
Assert.Equal(new[] { "AccountSetup", "AppsUser" }, mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Equal("on_change", mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_FromPhase_SurvivesStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-002",
Title = "From-phase rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_scoped",
ActiveFromPhase = "AccountSetup",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Equal("AccountSetup", mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_NullScopeFields_StayNull()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-003",
Title = "Unscoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "startup",
OutputEventType = "gather_plain",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public void Map_LegacyRow_WithoutScopeColumns_DefaultsToNull()
{
var (service, _) = BuildStorageHarness();
// A row written before the scope fields existed: none of the new columns present.
var legacy = new TableEntity("global", "GATHER-LEGACY-001")
{
["Title"] = "Legacy rule",
["CollectorType"] = "registry",
["Target"] = "HKLM\\SOFTWARE\\Contoso",
["Trigger"] = "startup",
["OutputEventType"] = "gather_legacy",
};
var mapped = service.MapToGatherRule(legacy);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
// ── ContentEquivalent ──────────────────────────────────────────────────
private static GatherRule BaseRule() => new()
{
RuleId = "GATHER-EQ-001",
Title = "Rule",
Description = "Desc",
Version = "1.0.0",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
TriggerPhase = "",
OutputEventType = "gather_eq",
};
[Fact]
public void ContentEquivalent_NullAndEmptyActivePhases_AreEquivalent()
{
// Pre-existing DB rows map absent columns to null; seeds are also null — and an
// explicitly-empty list means the same "unrestricted". No reseed churn allowed.
var a = BaseRule();
var b = BaseRule();
b.ActivePhases = new List<string>();
Assert.True(GatherRuleService.ContentEquivalent(a, b));
Assert.True(GatherRuleService.ContentEquivalent(b, a));
}
[Fact]
public void ContentEquivalent_DetectsChangesInEachScopeField()
{
var baseline = BaseRule();
var phases = BaseRule();
phases.ActivePhases = new List<string> { "AccountSetup" };
Assert.False(GatherRuleService.ContentEquivalent(baseline, phases));
var from = BaseRule();
from.ActiveFromPhase = "AccountSetup";
Assert.False(GatherRuleService.ContentEquivalent(baseline, from));
var emit = BaseRule();
emit.EmitMode = "on_change";
Assert.False(GatherRuleService.ContentEquivalent(baseline, emit));
}
[Fact]
public void ContentEquivalent_SameScopeFields_AreEquivalent()
{
var a = BaseRule();
a.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
a.EmitMode = "on_change";
var b = BaseRule();
b.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
b.EmitMode = "on_change";
Assert.True(GatherRuleService.ContentEquivalent(a, b));
}
// ── Validation ─────────────────────────────────────────────────────────
private static GatherRule ValidationRule(
List<string>? activePhases = null, string? activeFromPhase = null, string? emitMode = null) => new()
{
RuleId = "GATHER-VAL-001",
Title = "Rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_val",
ActivePhases = activePhases,
ActiveFromPhase = activeFromPhase,
EmitMode = emitMode,
};
[Theory]
[InlineData(null)]
[InlineData("Start")]
[InlineData("DevicePreparation")]
[InlineData("AccountSetup")]
[InlineData("Complete")]
[InlineData("accountsetup")] // case-insensitive
public void Validate_AcceptsValidFromPhase(string? fromPhase)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Theory]
[InlineData("Unknown")]
[InlineData("Failed")]
[InlineData("4")] // numeric enum value — canonical names only
[InlineData("DeviceESP")] // the invalid token the old free-text UI suggested
public void Validate_RejectsInvalidFromPhase(string fromPhase)
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Fact]
public void Validate_AcceptsValidActivePhases_AndRejectsInvalidEntries()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "AppsDevice" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "Failed" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "" })));
}
[Fact]
public void Validate_RejectsBothScopeFieldsSet()
{
var error = GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(
activePhases: new List<string> { "AccountSetup" }, activeFromPhase: "DeviceSetup"));
Assert.NotNull(error);
Assert.Contains("mutually exclusive", error);
}
[Theory]
[InlineData(null)]
[InlineData("always")]
[InlineData("on_change")]
[InlineData("ON_CHANGE")] // case-insensitive
public void Validate_AcceptsValidEmitModes(string? emitMode)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: emitMode)));
}
[Fact]
public void Validate_RejectsUnknownEmitMode()
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: "sometimes")));
}
[Fact]
public void Validate_TogglePartialPayload_PassesThrough()
{
// The portal toggle sends only { enabled, isBuiltIn, isCommunity } — none of the
// scope fields — and must not be rejected.
var toggle = new GatherRule { RuleId = "GATHER-VAL-002", Enabled = false };
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(toggle));
}
[Fact]
public void Validate_EmptyActivePhasesList_IsUnrestricted_AndPasses()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activePhases: new List<string>())));
}
}
@@ -0,0 +1,146 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the custom-rule partial-PUT fix on <see cref="GatherRuleService.UpdateRuleAsync"/>:
/// the portal toggle PUTs only { enabled, isBuiltIn, isCommunity }, and full-storing that
/// payload wiped Title/Target/Trigger/Parameters of the custom rule. Toggle-style partials
/// (empty Title/CollectorType/Target) must merge Enabled into the existing row; full payloads
/// (edit flow) must still replace the row but preserve the original CreatedAt.
/// </summary>
public class GatherRuleUpdatePartialMergeTests
{
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
private const string RuleId = "GATHER-CUST-001";
private static GatherRule ExistingCustomRule() => new()
{
RuleId = RuleId,
Title = "Collect BIOS Config",
Description = "Reads the RealmJoin BIOS key",
Category = "device",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS",
Parameters = new Dictionary<string, string> { ["valueName"] = "Version" },
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_bios_config",
Enabled = true,
IsBuiltIn = false,
IsCommunity = false,
CreatedAt = new DateTime(2026, 1, 15, 12, 0, 0, DateTimeKind.Utc),
ActiveFromPhase = "AccountSetup",
EmitMode = "on_change",
};
private static (GatherRuleService service, Mock<IRuleRepository> repo, List<GatherRule> stored) BuildService(
GatherRule? existingTenantRule)
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
var stored = new List<GatherRule>();
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>());
repo.Setup(r => r.GetGatherRulesAsync(TenantId)).ReturnsAsync(
existingTenantRule == null ? new List<GatherRule>() : new List<GatherRule> { existingTenantRule });
repo.Setup(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), TenantId))
.Callback<GatherRule, string>((rule, _) => stored.Add(rule))
.ReturnsAsync(true);
return (new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance), repo, stored);
}
[Fact]
public async Task TogglePartial_merges_enabled_and_preserves_definition_fields()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
// Exactly what the portal toggle sends: { enabled, isBuiltIn, isCommunity }.
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.False(written.Enabled);
Assert.Equal("Collect BIOS Config", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS", written.Target);
Assert.Equal("registry", written.CollectorType);
Assert.Equal("interval", written.Trigger);
Assert.Equal(60, written.IntervalSeconds);
Assert.Equal("Version", written.Parameters["valueName"]);
Assert.Equal("AccountSetup", written.ActiveFromPhase);
Assert.Equal("on_change", written.EmitMode);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task TogglePartial_for_unknown_custom_rule_fails_without_storing()
{
var (service, repo, stored) = BuildService(existingTenantRule: null);
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.False(ok);
Assert.Empty(stored);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
[Fact]
public async Task FullPayload_replaces_rule_but_preserves_original_CreatedAt()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
var fullPayload = new GatherRule
{
RuleId = RuleId,
Title = "Collect BIOS Config v2",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2",
Trigger = "interval",
IntervalSeconds = 120,
OutputEventType = "gather_bios_config",
Enabled = true,
// Edit flow sends createdAt from the client; a JSON-mode edit may omit it and
// default to UtcNow — either way the stored row must keep the original.
CreatedAt = DateTime.UtcNow,
};
var ok = await service.UpdateRuleAsync(TenantId, fullPayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.Equal("Collect BIOS Config v2", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2", written.Target);
Assert.Equal(120, written.IntervalSeconds);
Assert.False(written.IsBuiltIn);
Assert.False(written.IsCommunity);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task BuiltInRule_toggle_still_goes_through_RuleState_path()
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>
{
new() { RuleId = RuleId, IsBuiltIn = true, Title = "Built-in", Enabled = true }
});
repo.Setup(r => r.StoreRuleStateAsync(TenantId, RuleId, It.IsAny<RuleState>())).ReturnsAsync(true);
var service = new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance);
var ok = await service.UpdateRuleAsync(TenantId, new GatherRule { RuleId = RuleId, Enabled = false });
Assert.True(ok);
repo.Verify(r => r.StoreRuleStateAsync(TenantId, RuleId, It.Is<RuleState>(s => !s.Enabled)), Times.Once);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
}
@@ -129,4 +129,30 @@ public class GetAllTenantConfigurationsFunctionTests
[Fact]
public void DelegatedBareArrayView_Empty_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.DelegatedBareArrayView(new TenantConfiguration[0]));
[Fact]
public void MissingManagedIds_ReturnsOnlyIdsWithoutConfig_CaseInsensitive()
{
var configs = new List<TenantConfiguration>
{
new() { TenantId = TenantB.ToUpperInvariant() }, // case variant still counts as present
};
var missing = GetAllTenantConfigurationsFunction.MissingManagedIds(new[] { TenantB, TenantC }, configs);
var only = Assert.Single(missing);
Assert.Equal(TenantC, only);
}
[Fact]
public void MissingManagedIds_AllPresent_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.MissingManagedIds(
new[] { TenantB },
new List<TenantConfiguration> { new() { TenantId = TenantB } }));
[Fact]
public void MissingConfigCacheKey_LowercasesId_SoCaseVariantsShareOneVerdict()
=> Assert.Equal(
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB),
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB.ToUpperInvariant()));
}
@@ -0,0 +1,107 @@
using System.Collections.Generic;
using System.Net;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Functions.Tests.GraphResolution;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Service-level tests for the Agent Binaries HealthCheck card. The check probes the
/// canonical download alias (download.autopilotmonitor.com) AND the legacy blob keepalive
/// account: alias failure is unhealthy (new installs break), legacy failure is only a
/// warning (existing customers break until they migrate to the alias).
/// </summary>
public class HealthCheckServiceAgentBinariesTests
{
private const string AliasHost = "download.autopilotmonitor.com";
private const string LegacyHost = "autopilotmonitor.blob.core.windows.net";
[Fact]
public async Task Check_AllEndpointsAvailable_ReportsHealthyWithoutUrlDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("Agent Binaries", check.Name);
Assert.Equal("healthy", check.Status);
Assert.Contains("download alias", check.Message);
Assert.Null(check.Details);
// Both hosts must actually have been probed (ZIP + PS1 each).
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(AliasHost)).Count);
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(LegacyHost)).Count);
}
[Fact]
public async Task Check_WithEndpointUrls_SurfacesAliasAndLegacyUrlsInDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: true);
Assert.NotNull(check.Details);
Assert.Equal("https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip", check.Details!["Agent ZIP"]);
Assert.Equal("https://download.autopilotmonitor.com/agent/Install-AutopilotMonitor.ps1", check.Details["Bootstrap script"]);
Assert.Equal("https://autopilotmonitor.blob.core.windows.net/agent", check.Details["Legacy blob (keepalive)"]);
}
[Fact]
public async Task Check_LegacyBlobMissing_ReportsWarningNamingTheLegacyEndpoint()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.NotFound, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("warning", check.Status);
Assert.Contains("legacy", check.Message, System.StringComparison.OrdinalIgnoreCase);
Assert.Contains("404", check.Message);
}
[Fact]
public async Task Check_AliasMissing_ReportsUnhealthy()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.NotFound, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("download alias", check.Message);
}
[Fact]
public async Task Check_EndpointsUnreachable_ReportsUnhealthy()
{
// Un-scripted URLs make the stub throw — models a connection failure.
var check = await BuildService(new StubHttpMessageHandler()).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("unreachable", check.Message);
}
private static HealthCheckService BuildService(StubHttpMessageHandler handler)
{
var config = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string?>())
.Build();
return new HealthCheckService(
NullLogger<HealthCheckService>.Instance,
adminConfigService: null!,
httpClientFactory: new StubHttpClientFactory(handler),
metricsReader: null!,
poisonQueueProbe: null!,
configuration: config);
}
}
@@ -53,6 +53,29 @@ public class HealthCheckServiceMcpServerTests
Assert.DoesNotContain("http", check.Message, System.StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlAlongsideVersion()
{
var svc = BuildService(HttpStatusCode.OK, "{\"status\":\"healthy\",\"version\":\"1.4.0\"}");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
// The Version detail must merge into — not replace — the Server URL detail.
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
Assert.Equal("1.4.0", check.Details["Version"]);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlEvenWhenUnhealthy()
{
var svc = BuildService(HttpStatusCode.ServiceUnavailable, "");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
Assert.Equal("unhealthy", check.Status);
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
}
[Fact]
public async Task Check_TimeoutOrConnectionFailure_ReportsWarningForColdStart()
{
@@ -0,0 +1,61 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Per-row backfill decision (pure function): rows that already carry OccurredUtc are
/// skipped (idempotent re-runs), decodable RowKeys yield the deterministic decoded value,
/// anything else is reported undecodable — never guessed.
/// </summary>
public class OccurredUtcBackfillServiceTests
{
private static readonly DateTime Ts = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
[Fact]
public void Audit_row_with_existing_column_is_skipped()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Audit_time_encoded_row_yields_decoded_write()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()));
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
}
[Fact]
public void Audit_legacy_guid_row_is_undecodable_not_guessed()
{
var e = new TableEntity("t", Guid.NewGuid().ToString("N"));
Assert.Equal(OccurredUtcBackfillService.RowDecision.Undecodable,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Ops_row_yields_decoded_write_and_existing_column_is_skipped()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
var bare = new TableEntity("Security", rowKey);
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, bare, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
var backfilled = new TableEntity("Security", rowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, backfilled, out _));
}
}
@@ -0,0 +1,269 @@
using AutopilotMonitor.Functions.Functions.Admin;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.DataAccess;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the SignalR revoke-enforcement cores of <see cref="DelegatedAdminManagementFunction"/> and
/// <see cref="TenantGroupManagementFunction"/> (extracted seams — the HTTP entry points are intentionally
/// not exercised, same rationale as <see cref="GetAllBlockedDevicesFunctionTests"/>). Locks in three
/// security invariants: (1) a miss (no assignment row) neither audits nor disconnects — a typo must not
/// write false customer-visible "access removed" rows while the real grant stays live; (2) a real
/// revoke/disable/unassign/group-delete always cuts the affected UPN's live streams (group authorization
/// is join-time only); (3) grant validation is fail-closed (GUID tenantId, recognized roles only).
/// </summary>
public class RevokeEnforcementTests
{
private const string Upn = "MSP-Admin@Partner.Example";
private const string UpnLower = "msp-admin@partner.example";
private const string TenantA = "11111111-1111-1111-1111-111111111111";
private const string TenantB = "22222222-2222-2222-2222-222222222222";
private const string GroupId = "tpl-aaaaaaaa";
private static DelegatedAdminService BuildService(Mock<IAdminRepository> repo) =>
new(
repo.Object,
new StubTenantEntitlementService(AutopilotMonitor.Functions.Security.TenantEdition.Enterprise),
new MemoryCache(new MemoryCacheOptions()),
NullLogger<DelegatedAdminService>.Instance);
private static Mock<IMaintenanceRepository> BuildAuditRepo()
{
var audit = new Mock<IMaintenanceRepository>();
audit.Setup(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()))
.ReturnsAsync(true);
return audit;
}
private static void VerifyNoAudit(Mock<IMaintenanceRepository> audit) =>
audit.Verify(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()), Times.Never);
private static (DelegatedAdminManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildAdminFn()
{
var repo = new Mock<IAdminRepository>();
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new DelegatedAdminManagementFunction(
NullLogger<DelegatedAdminManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
private static (TenantGroupManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildGroupFn()
{
var repo = new Mock<IAdminRepository>();
// Loose-mock defaults return a null Task — every repo read the flows touch needs a concrete setup.
repo.Setup(r => r.GetTenantGroupAsync(It.IsAny<string>())).ReturnsAsync((TenantGroup?)null);
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
repo.Setup(r => r.GetGroupAssigneesAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new TenantGroupManagementFunction(
NullLogger<TenantGroupManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
// --- Delegated grant validation (fail-closed) ---
[Fact]
public void ValidateGrant_NullBody_Rejected()
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(null, out _));
[Theory]
[InlineData("", TenantA)]
[InlineData(" ", TenantA)]
[InlineData(Upn, "")]
public void ValidateGrant_MissingFields_Rejected(string upn, string tenantId)
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = upn, TenantId = tenantId }, out _));
[Theory]
[InlineData("contoso.com")]
[InlineData("not-a-guid")]
[InlineData("11111111-1111-1111-1111")]
public void ValidateGrant_NonGuidTenantId_Rejected(string tenantId)
=> Assert.Equal("a valid tenantId (GUID) is required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = tenantId }, out _));
[Fact]
public void ValidateGrant_UnknownRole_Rejected()
{
var error = DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = "GlobalAdmin" }, out _);
Assert.NotNull(error);
Assert.Contains(Constants.DelegatedRoles.DelegatedReader, error);
}
[Fact]
public void ValidateGrant_EmptyRole_DefaultsToLeastPrivilegedReader()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA }, out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedReader, role);
}
[Fact]
public void ValidateGrant_ExplicitDelegatedAdmin_Accepted()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = Constants.DelegatedRoles.DelegatedAdmin },
out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedAdmin, role);
}
// --- Revoke: 404-on-missing-row vs. audit + disconnect ---
[Fact]
public async Task RevokeCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(false);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.False(removed);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task RevokeCore_RemovedRow_AuditsAndDisconnectsLowercasedUpn()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(true);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.True(removed);
// SignalR user ids are lowercased UPNs — a mixed-case route param must still hit the user's connections.
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedAdmin", UpnLower, "ga@vendor.example", null), Times.Once);
}
// --- Disable is a revocation too; enable never disconnects ---
[Fact]
public async Task SetEnabledCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(false);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.False(ok);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Disable_Disconnects()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.True(ok);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Enable_DoesNotDisconnect()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, true)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: true, "ga@vendor.example");
Assert.True(ok);
Assert.Empty(signalR.DisconnectedUsers);
}
// --- Group unassign: 404-on-not-assigned vs. disconnect + per-tenant audit ---
[Fact]
public async Task UnassignCore_NotAssigned_NoAuditNoDisconnect()
{
var (fn, _, audit, signalR) = BuildGroupFn(); // default: no assignments for the UPN
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.False(unassigned);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task UnassignCore_Assigned_DisconnectsAndAuditsPerGroupTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA, TenantB },
AssigneeCount = 1,
});
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(UpnLower))
.ReturnsAsync(new List<TenantGroupAssignment> { new() { Upn = UpnLower, GroupId = GroupId } });
repo.Setup(r => r.UnassignGroupAsync(UpnLower, GroupId)).ReturnsAsync(true);
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.True(unassigned);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
// Lost access is audited under EACH of the group's tenants (customer-visible trail).
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
audit.Verify(a => a.LogAuditEntryAsync(
TenantB, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
// --- Group delete: every (former) assignee's streams are cut ---
[Fact]
public async Task DeleteGroupCore_DisconnectsEveryAssignee_AndAuditsPerTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
var assignees = new List<TenantGroupAssignment>
{
new() { Upn = "one@partner.example", GroupId = GroupId },
new() { Upn = "two@partner.example", GroupId = GroupId },
};
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA },
AssigneeCount = assignees.Count,
});
repo.Setup(r => r.GetGroupAssigneesAsync(GroupId)).ReturnsAsync(assignees);
repo.Setup(r => r.DeleteTenantGroupAsync(GroupId)).ReturnsAsync(true);
await fn.DeleteGroupCoreAsync(GroupId, "ga@vendor.example");
Assert.Equal(new[] { "one@partner.example", "two@partner.example" }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", "*", "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
}
@@ -0,0 +1,25 @@
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// ContactEmail is the address enforcement actions and service notices are sent to.
/// The seed must never overwrite a value the tenant owns — that is the whole contract.
/// </summary>
public class TenantContactEmailTests
{
[Fact]
public void ContactEmail_defaults_to_null_so_absence_is_distinguishable_from_empty()
{
var config = new TenantConfiguration();
Assert.Null(config.ContactEmail);
}
[Fact]
public void ContactEmail_survives_a_round_trip_through_the_model()
{
var config = new TenantConfiguration { ContactEmail = "it-operations@contoso.com" };
Assert.Equal("it-operations@contoso.com", config.ContactEmail);
}
}
@@ -336,6 +336,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "LastUpdated", config.LastUpdated },
{ "UpdatedBy", config.UpdatedBy },
{ "OnboardedBy", config.OnboardedBy },
{ "ContactEmail", config.ContactEmail },
{ "Disabled", config.Disabled },
{ "DisabledReason", config.DisabledReason },
{ "DisabledUntil", config.DisabledUntil },
@@ -432,6 +433,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
LastUpdated = entity.GetDateTime("LastUpdated") ?? DateTime.UtcNow,
UpdatedBy = entity.GetString("UpdatedBy") ?? "Unknown",
OnboardedBy = entity.GetString("OnboardedBy"),
ContactEmail = entity.GetString("ContactEmail"),
Disabled = entity.GetBoolean("Disabled") ?? false,
DisabledReason = entity.GetString("DisabledReason"),
DisabledUntil = entity.GetDateTime("DisabledUntil"),
@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
@@ -43,7 +44,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
["UserId"] = Truncate(entry.UserId, 128),
["Message"] = Truncate(entry.Message, 512),
["Details"] = Truncate(entry.Details, 4096),
["Timestamp"] = entry.Timestamp,
// "Timestamp" is a reserved system property (supplied values are ignored and
// any row rewrite resets it), so the event time needs its own column.
[BusinessTimestamp.OccurredUtcColumn] = entry.Timestamp,
};
await _table.UpsertEntityAsync(entity);
@@ -168,15 +171,16 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
// Filters on the system-managed Timestamp; user-defined "Timestamp" property
// mirrors it on insert via SaveOpsEventAsync.
private static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
// Date windows filter on the RowKey (reverse-tick, index-backed): the system
// Timestamp is reset by storage migrations, and an OccurredUtc property filter
// would exclude rows written before that column existed.
internal static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
{
var clauses = new List<string>();
if (!string.IsNullOrEmpty(category))
@@ -185,11 +189,11 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
}
return clauses.Count == 0 ? null : string.Join(" and ", clauses);
}
@@ -201,27 +205,30 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{
var deleted = 0;
await foreach (var entity in _table.QueryAsync<TableEntity>())
// Server-side RowKey range instead of a full-table scan over the system
// Timestamp: every ops RowKey is a reverse-tick, so age is index-derivable,
// and the system Timestamp lies after storage migrations (it would freeze
// deletions for one retention period, then drop the whole pre-migration
// corpus in a single run).
var filter = BusinessTimestamp.OpsRetentionClause(cutoff);
await foreach (var entity in _table.QueryAsync<TableEntity>(
filter: filter, select: new[] { "PartitionKey", "RowKey" }))
{
var timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue;
if (timestamp < cutoff)
try
{
try
{
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
private static OpsEventEntry MapToEntry(TableEntity entity)
internal static OpsEventEntry MapToEntry(TableEntity entity)
{
return new OpsEventEntry
{
@@ -233,10 +240,22 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
UserId = entity.GetString("UserId"),
Message = entity.GetString("Message") ?? string.Empty,
Details = entity.GetString("Details"),
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue,
Timestamp = ResolveTimestamp(entity),
};
}
// Business time: OccurredUtc column → reverse-tick RowKey decode → system
// Timestamp (last resort; a row rewrite — e.g. storage migration — resets it).
internal static DateTime ResolveTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.MinValue;
}
private static string? Truncate(string? value, int maxLength)
{
if (value == null) return null;
@@ -0,0 +1,103 @@
using System.Net;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Admin
{
/// <summary>
/// Global-Admin trigger for the one-time OccurredUtc backfill on AuditLogs / OpsEvents
/// (business timestamps lost to the system-Timestamp reset of the 2026-07-18 storage
/// migration; values are recovered from the reverse-tick RowKeys). DRY-RUN BY DEFAULT —
/// a real run requires the explicit <c>dryRun=false</c> query parameter, so the operator
/// always sees counts and samples before anything is written.
/// </summary>
public class BackfillOccurredUtcFunction
{
private readonly ILogger<BackfillOccurredUtcFunction> _logger;
private readonly OccurredUtcBackfillService _service;
private const int DefaultMaxRows = 250;
private const int MaxMaxRows = 1000;
public BackfillOccurredUtcFunction(
ILogger<BackfillOccurredUtcFunction> logger,
OccurredUtcBackfillService service)
{
_logger = logger;
_service = service;
}
/// <summary>
/// POST /api/maintenance/backfill-occurred-utc
/// Query parameters:
/// - table: "audit" | "ops" (required).
/// - dryRun: "false" to actually write; anything else (or absent) reports only.
/// - maxRows: rows examined per invocation (default 250, max 1000). Follow
/// nextContinuation to continue; null nextContinuation = table done.
/// - continuation: opaque token from the previous response (URL-encode it).
/// </summary>
[Function("BackfillOccurredUtc")]
public async Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "maintenance/backfill-occurred-utc")] HttpRequestData req,
FunctionContext context)
{
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
var userEmail = TenantHelper.GetUserIdentifier(req);
var table = (req.Query["table"] ?? string.Empty).Trim().ToLowerInvariant();
if (table != OccurredUtcBackfillService.TableAudit && table != OccurredUtcBackfillService.TableOps)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new
{
error = $"table must be '{OccurredUtcBackfillService.TableAudit}' or '{OccurredUtcBackfillService.TableOps}'"
});
return bad;
}
var dryRun = !string.Equals(req.Query["dryRun"], "false", StringComparison.OrdinalIgnoreCase);
var continuation = req.Query["continuation"];
var maxRows = DefaultMaxRows;
if (!string.IsNullOrEmpty(req.Query["maxRows"]))
{
if (!int.TryParse(req.Query["maxRows"], out maxRows) || maxRows < 1)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new { error = $"maxRows must be a positive integer (max {MaxMaxRows})" });
return bad;
}
maxRows = Math.Min(maxRows, MaxMaxRows);
}
_logger.LogInformation(
"OccurredUtc backfill requested by {User}: table={Table} dryRun={DryRun} maxRows={MaxRows} hasContinuation={HasContinuation}",
userEmail, table, dryRun, maxRows, !string.IsNullOrEmpty(continuation));
try
{
var result = await _service.RunAsync(table, dryRun, maxRows, continuation);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
{
success = true,
result,
triggeredBy = userEmail,
triggeredAt = DateTime.UtcNow,
});
return response;
}
catch (Exception ex)
{
_logger.LogError(ex, "OccurredUtc backfill run failed");
var error = req.CreateResponse(HttpStatusCode.InternalServerError);
await error.WriteAsJsonAsync(new { success = false, message = "Internal server error" });
return error;
}
}
}
}
@@ -64,19 +64,9 @@ public class DelegatedAdminManagementFunction
var currentUpn = context.GetRequestContext().UserPrincipalName;
var body = await req.ReadFromJsonAsync<GrantDelegatedAdminRequest>();
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return await Bad(req, "upn and tenantId are required");
// Mirror the tenant-group path: a mistyped tenantId must not create a garbage scope entry
// that pollutes the allow-list and seeds a per-tenant audit trail keyed on the typo.
if (!Guid.TryParse(body.TenantId, out _))
return await Bad(req, "a valid tenantId (GUID) is required");
// Default to the least-privileged role; reject anything we don't recognize (fail-closed) so a typo
// can never silently widen access.
var role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return await Bad(req, $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'");
var validationError = ValidateGrantRequest(body, out var role);
if (validationError != null)
return await Bad(req, validationError);
// Edition note: TARGET tenants may be any edition — an MSP on Enterprise may manage Community
// customers. The Enterprise requirement applies to the delegated admin's HOME tenant and is
@@ -84,7 +74,7 @@ public class DelegatedAdminManagementFunction
// be checked here: at grant time only the UPN is known, and UPN-domain → tenant mapping is not
// reliable (multi-domain tenants). Grants to non-Enterprise-homed admins are simply inert.
var entry = await _delegatedAdminService.UpsertAsync(
body.Upn, body.TenantId, role,
body!.Upn, body.TenantId, role,
Constants.DelegatedStatus.Active, Constants.DelegatedSource.OperatorGranted, currentUpn ?? "");
await _maintenanceRepo.LogAuditEntryAsync(
@@ -112,10 +102,7 @@ public class DelegatedAdminManagementFunction
string upn, string tenantId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
// The service reports whether a row was actually deleted — a mistyped upn/tenantId must not
// 200 and write a false customer-visible "access removed" audit row while the real grant
// stays live (mirrors the tenant-group revoke semantics).
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
var removed = await RevokeCoreAsync(upn, tenantId, currentUpn);
if (!removed)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -123,20 +110,34 @@ public class DelegatedAdminManagementFunction
return notFound;
}
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
// Enforcement: cut the revoked caller's already-joined live streams — group authorization is
// join-time only, so without this they keep receiving tenant telemetry until the connection drops.
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Delegated assignment revoked" });
return response;
}
/// <summary>
/// Testable core of the revoke flow (no HTTP plumbing — see GetAllBlockedDevicesFunctionTests for the
/// convention). Returns false when no assignment row existed (caller 404s): a mistyped upn/tenantId
/// must not 200 and write a false customer-visible "access removed" audit row while the real grant
/// stays live (mirrors the tenant-group revoke semantics). On a real removal writes the audit entry
/// and cuts the revoked UPN's live streams — group authorization is join-time only, so without the
/// kick they keep receiving tenant telemetry until the connection drops.
/// </summary>
internal async Task<bool> RevokeCoreAsync(string upn, string tenantId, string? currentUpn)
{
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
if (!removed)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
return true;
}
/// <summary>PATCH /api/global/delegated-admins/{upn}/{tenantId}/enable — re-enable a disabled assignment. GlobalAdminOnly.</summary>
[Function("EnableDelegatedAdmin")]
[Authorize]
@@ -157,7 +158,7 @@ public class DelegatedAdminManagementFunction
HttpRequestData req, string upn, string tenantId, bool isEnabled, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
var ok = await SetEnabledCoreAsync(upn, tenantId, isEnabled, currentUpn);
if (!ok)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -165,20 +166,56 @@ public class DelegatedAdminManagementFunction
return notFound;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
}
/// <summary>
/// Testable core of enable/disable (no HTTP plumbing). Returns false when no assignment row
/// existed (caller 404s — no audit, no disconnect). Disable is a revocation too, so it cuts the
/// UPN's already-joined live streams (see <see cref="RevokeCoreAsync"/>); enable never disconnects.
/// </summary>
internal async Task<bool> SetEnabledCoreAsync(string upn, string tenantId, bool isEnabled, string? currentUpn)
{
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
if (!ok)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "UPDATE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "",
new Dictionary<string, string> { { "IsEnabled", isEnabled.ToString() } });
// Disable is a revocation too — cut already-joined live streams (see RevokeDelegatedAdmin).
if (!isEnabled)
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated {Action}: {Upn} -> {TenantId} by {By}",
isEnabled ? "enable" : "disable", upn, tenantId, currentUpn);
return true;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
/// <summary>
/// Pure grant-request validation (testable seam). Returns the 400 error message, or null when valid —
/// then <paramref name="role"/> carries the effective role. A mistyped tenantId must not create a
/// garbage scope entry that pollutes the allow-list and seeds a per-tenant audit trail keyed on the
/// typo; the role defaults to least-privileged and unknown roles are rejected (fail-closed) so a typo
/// can never silently widen access.
/// </summary>
internal static string? ValidateGrantRequest(GrantDelegatedAdminRequest? body, out string role)
{
role = Constants.DelegatedRoles.DelegatedReader;
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return "upn and tenantId are required";
if (!Guid.TryParse(body.TenantId, out _))
return "a valid tenantId (GUID) is required";
role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role!;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'";
return null;
}
private static async Task<HttpResponseData> Bad(HttpRequestData req, string error)
@@ -112,15 +112,26 @@ public class TenantGroupManagementFunction
string groupId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
await DeleteGroupCoreAsync(groupId, currentUpn);
// Capture tenants + assignees BEFORE the cascade delete removes them, so we can audit the
// bulk access removal under each affected tenant and cut every assignee's live streams.
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>
/// Testable core of the group cascade delete (no HTTP plumbing). Captures tenants + assignees
/// BEFORE the delete removes them, so the bulk access removal is audited under each affected
/// tenant and EVERY (former) assignee's live streams are cut — group authorization is join-time
/// only, so without the kick they keep receiving the managed tenants' telemetry until their
/// connection drops.
/// </summary>
internal async Task DeleteGroupCoreAsync(string groupId, string? currentUpn)
{
var group = await _delegatedAdminService.GetGroupAsync(groupId);
var assignees = await _delegatedAdminService.GetGroupAssigneesAsync(groupId);
await _delegatedAdminService.DeleteGroupAsync(groupId);
// Enforcement: group authorization is join-time only — without the kick, every (former)
// assignee keeps receiving the managed tenants' live telemetry until their connection drops.
foreach (var assignee in assignees)
await _signalRService.DisconnectUserAsync(assignee.Upn);
@@ -137,10 +148,6 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Tenant group deleted: {GroupId} by {By}", groupId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>POST /api/global/tenant-groups/{groupId}/tenants — add a tenant. GlobalAdminOnly. Body: { "tenantId": "&lt;guid&gt;" }.</summary>
@@ -271,17 +278,31 @@ public class TenantGroupManagementFunction
string groupId, string upn, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var unassigned = await UnassignCoreAsync(groupId, upn, currentUpn);
if (!unassigned)
return await NotFound(req);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
}
/// <summary>
/// Testable core of the unassign flow (no HTTP plumbing). Returns false when the UPN was not
/// actually assigned (caller 404s) — so a mistyped UPN can't 200 and write false "access removed"
/// audit rows, and no disconnect fires. On a real unassign cuts the UPN's already-joined live
/// streams (join-time-only authz) and audits the lost access under each of the group's tenants.
/// </summary>
internal async Task<bool> UnassignCoreAsync(string groupId, string upn, string? currentUpn)
{
var normalizedUpn = upn.ToLowerInvariant();
// Read the group's tenants (unchanged by unassign) so we can audit the UPN's lost access per tenant.
var group = await _delegatedAdminService.GetGroupAsync(groupId);
// The service returns false when the UPN was not actually assigned — so a mistyped UPN can't 200 and
// write false "access removed" audit rows. Only audit after a real unassign.
var unassigned = await _delegatedAdminService.UnassignGroupAsync(normalizedUpn, groupId);
if (!unassigned)
return await NotFound(req);
return false;
// Enforcement: cut the unassigned caller's already-joined live streams (join-time-only authz).
await _signalRService.DisconnectUserAsync(normalizedUpn);
if (group != null)
@@ -296,10 +317,7 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Group {GroupId} unassigned from {Upn} by {By}", groupId, normalizedUpn, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
return true;
}
/// <summary>Logs one audit entry per tenant (the customer-visible "who can read my tenant" trail).</summary>
@@ -270,7 +270,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new AgentConfigResponse
{
ConfigVersion = 33, // Agent endpoint migration on the config channel (MigrateToApiBaseUrl; live-fetch-only, allowlist-validated, one hop, kill wins)
ConfigVersion = 35, // MDM reboot-policy watcher (mdm_policy_reboot_required / esp_reboot_coalescing)
UploadIntervalSeconds = 10,
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
KeepLogFile = tenantConfig.KeepLogFile ?? false,
@@ -314,7 +314,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
DeviceBlocked = killVerdict.IsBlocked,
DeviceKillSignal = killVerdict.IsKill,
UnblockAt = killVerdict.UnblockAt,
MigrateToApiBaseUrl = migrateTarget!,
MigrateToApiBaseUrl = migrateTarget,
});
return response;
@@ -10,6 +10,7 @@ using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Pagination;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
@@ -17,15 +18,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
{
public class GetAllTenantConfigurationsFunction
{
// TTL for the "this managed id has NO config row in any casing" verdict a rescue scan proved.
// Bounds how long a tenant onboarded moments after that scan stays absent from a delegated
// caller's config/all (matches the positive config cache TTL in TenantConfigurationService).
private static readonly TimeSpan MissingConfigNegativeCacheDuration = TimeSpan.FromMinutes(5);
private readonly ILogger<GetAllTenantConfigurationsFunction> _logger;
private readonly TenantConfigurationService _configService;
private readonly IMemoryCache _cache;
public GetAllTenantConfigurationsFunction(
ILogger<GetAllTenantConfigurationsFunction> logger,
TenantConfigurationService configService)
TenantConfigurationService configService,
IMemoryCache cache)
{
_logger = logger;
_configService = configService;
_cache = cache;
}
[Function("GetAllTenantConfigurations")]
@@ -73,9 +82,21 @@ namespace AutopilotMonitor.Functions.Functions.Config
// (agent paths auto-create rows with the caller-supplied id verbatim) — the same caveat
// the delegated sessions fan-out handles case-insensitively. A point-read is exact-case,
// so any id it missed gets ONE scan+filter rescue pass (the pre-point-read behavior).
// Only stale grants or case-variant rows pay for it; the common path stays read-bounded.
// An id a previous scan already proved missing in ANY casing is negative-cached and
// skipped for the TTL — otherwise a single stale grant (offboarded / never-onboarded
// tenant) would make EVERY config/all call pay a full table scan.
if (subset.Count < managedIds.Count)
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
{
var unproven = MissingManagedIds(managedIds, subset)
.Where(id => !_cache.TryGetValue(MissingConfigCacheKey(id), out _))
.ToList();
if (unproven.Count > 0)
{
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
foreach (var id in MissingManagedIds(unproven, subset))
_cache.Set(MissingConfigCacheKey(id), true, MissingConfigNegativeCacheDuration);
}
}
_logger.LogInformation("GetAllTenantConfigurations (delegated subset, {Count} tenants) by {User}",
subset.Count, userIdentifier);
@@ -177,6 +198,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
IEnumerable<TenantConfiguration?> reads)
=> reads.Where(c => c != null).Select(c => c!).ToList();
/// <summary>
/// The managed ids not present (case-insensitively) in <paramref name="configs"/> — before the
/// rescue scan these are the point-read misses, after it the proven-missing ids to negative-cache.
/// Pure + testable.
/// </summary>
internal static List<string> MissingManagedIds(
IEnumerable<string> managedIds, IReadOnlyCollection<TenantConfiguration> configs)
{
var have = new HashSet<string>(configs.Select(c => c.TenantId), StringComparer.OrdinalIgnoreCase);
return managedIds.Where(id => !have.Contains(id)).ToList();
}
/// <summary>Negative-cache key for a managed id a rescue scan proved has no config row.
/// Lowercased so case variants of the same grant share one verdict.</summary>
internal static string MissingConfigCacheKey(string tenantId)
=> $"tenant-config-missing:{tenantId.ToLowerInvariant()}";
/// <summary>
/// Case-variant rescue for managed ids the exact-case point-reads missed: from the full config scan,
/// add every config whose TenantId matches a managed id case-insensitively and is not already in the
@@ -102,17 +102,19 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
// Perform comprehensive health checks
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync();
// SignalR Quota check exposes the subscription/resource-group path of the
// backing SignalR resource, and Poison Queues exposes internal async-worker
// queue topology — restrict both to Global Admins. Non-GA callers
// (Tenant Admins, Operators) get the rest of the report unchanged.
// SignalR Quota exposes the subscription/resource-group path of the backing
// SignalR resource, Poison Queues exposes internal async-worker queue topology,
// and endpoint URLs in the check details expose infrastructure hostnames —
// restrict all of that to Global Admins. Non-GA callers (Tenant Admins,
// Operators) get the rest of the report unchanged.
// The route is registered as AuthenticatedUser, so PolicyEnforcementMiddleware
// does NOT compute IsGlobalAdmin; resolve it directly via GlobalAdminService.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
// Perform comprehensive health checks (endpoint URLs only for Global Admins)
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync(includeEndpointUrls: isGlobalAdmin);
var visibleChecks = isGlobalAdmin
? healthCheckResult.Checks
: healthCheckResult.Checks
@@ -146,10 +148,14 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
/// </summary>
[Function("McpHealthCheck")]
public async Task<HttpResponseData> GetMcpHealthCheck(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req)
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req,
FunctionContext context)
{
// Authentication enforced by PolicyEnforcementMiddleware (AuthenticatedUser).
var check = await _healthCheckService.CheckMcpServerAsync();
// The MCP server URL is infrastructure topology — include it only for Global Admins.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
var check = await _healthCheckService.CheckMcpServerAsync(includeEndpointUrl: isGlobalAdmin);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
@@ -64,6 +64,14 @@ namespace AutopilotMonitor.Functions.Functions.Rules
return badRequest;
}
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
try
{
var success = await _ruleService.CreateRuleAsync(tenantId, rule);
@@ -108,12 +116,71 @@ namespace AutopilotMonitor.Functions.Functions.Rules
rule.RuleId = ruleId;
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
var success = await _ruleService.UpdateRuleAsync(tenantId, rule);
var response = req.CreateResponse(success ? HttpStatusCode.OK : HttpStatusCode.InternalServerError);
await response.WriteAsJsonAsync(new { success, message = success ? "Rule updated" : "Failed to update rule" });
return response;
}
/// <summary>
/// Selectable phase-scope tokens: the <see cref="EnrollmentPhase"/> enum NAMES from
/// Start(0) through Complete(7). Explicit name list (not Enum.TryParse) so numeric
/// tokens ("4") and the non-selectable Unknown/Failed members are rejected — the
/// canonical vocabulary the agent, schema, and portal share.
/// </summary>
private static readonly HashSet<string> ValidScopePhases = new(StringComparer.OrdinalIgnoreCase)
{
nameof(EnrollmentPhase.Start),
nameof(EnrollmentPhase.DevicePreparation),
nameof(EnrollmentPhase.DeviceSetup),
nameof(EnrollmentPhase.AppsDevice),
nameof(EnrollmentPhase.AccountSetup),
nameof(EnrollmentPhase.AppsUser),
nameof(EnrollmentPhase.FinalizingSetup),
nameof(EnrollmentPhase.Complete),
};
/// <summary>
/// Validates the phase-scope + emit-mode fields on create/update.
/// Returns an error message for a 400 response, or null when valid.
/// Toggle-style partial payloads carry none of these fields and pass through.
/// </summary>
internal static string? ValidateScopeAndEmitMode(GatherRule rule)
{
var hasActivePhases = rule.ActivePhases != null && rule.ActivePhases.Count > 0;
var hasFromPhase = !string.IsNullOrEmpty(rule.ActiveFromPhase);
if (hasActivePhases && hasFromPhase)
return "activePhases and activeFromPhase are mutually exclusive — set only one.";
if (hasActivePhases)
{
foreach (var phase in rule.ActivePhases!)
{
if (string.IsNullOrEmpty(phase) || !ValidScopePhases.Contains(phase))
return $"Invalid phase '{phase}' in activePhases. Valid phases: {string.Join(", ", ValidScopePhases)}.";
}
}
if (hasFromPhase && !ValidScopePhases.Contains(rule.ActiveFromPhase!))
return $"Invalid activeFromPhase '{rule.ActiveFromPhase}'. Valid phases: {string.Join(", ", ValidScopePhases)}.";
if (!string.IsNullOrEmpty(rule.EmitMode)
&& !string.Equals(rule.EmitMode, "always", StringComparison.OrdinalIgnoreCase)
&& !string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase))
return "emitMode must be \"always\" or \"on_change\".";
return null;
}
[Function("DeleteGatherRule")]
public async Task<HttpResponseData> DeleteRule(
[HttpTrigger(AuthorizationLevel.Anonymous, "delete", Route = "rules/gather/{ruleId}")] HttpRequestData req,
@@ -0,0 +1,154 @@
using System;
using System.Globalization;
using Azure.Data.Tables;
namespace AutopilotMonitor.Functions.Helpers
{
/// <summary>
/// Business-timestamp handling for tables whose rows historically relied on the
/// Azure Tables system <c>Timestamp</c> (= row write time). That system property is
/// server-managed: supplied values are ignored on write and every row rewrite
/// (storage migration, merge) resets it. The authoritative event time therefore
/// lives in (a) the explicit <see cref="OccurredUtcColumn"/> custom property on
/// rows written after its introduction, and (b) the time encoded in the RowKey
/// for older rows. Readers resolve OccurredUtc → RowKey decode → system Timestamp.
///
/// RowKey schemes decoded here:
/// AuditLogs: "!{MaxTicks - ticks:D19}_{guid:N}" (legacy pre-2026-05 rows are bare GUIDs — not decodable)
/// OpsEvents: "{MaxTicks - ticks:D19}"
/// Events: "{timestamp:yyyyMMddHHmmssfff}_{sequence:D10}"
/// </summary>
internal static class BusinessTimestamp
{
/// <summary>Custom column carrying the business event time. Survives migrations verbatim
/// (unlike the system Timestamp, which any row rewrite resets).</summary>
internal const string OccurredUtcColumn = "OccurredUtc";
internal const string AuditRowKeyPrefix = "!";
/// <summary>
/// '"' (0x22) is the character immediately after '!' (0x21). `RowKey lt '"'` confines a
/// range to time-encoded audit rows: legacy bare-GUID RowKeys start with a hex digit
/// (≥ '0', 0x30) and would otherwise match ANY lower bound of the form `RowKey ge '!…'`.
/// </summary>
internal const string AuditTimeEncodedUpperBound = "\"";
private const string EventRowKeyTimestampFormat = "yyyyMMddHHmmssfff";
/// <summary>
/// Reads a datetime column as UTC regardless of the SDK's materialized type —
/// service reads surface Edm.DateTime as either DateTimeOffset or DateTime
/// depending on payload shape (the pre-existing mappers already read both).
/// Unspecified kinds are treated as UTC (all stored values are UTC).
/// </summary>
internal static DateTime? GetUtcDateTime(TableEntity entity, string column)
{
if (!entity.TryGetValue(column, out var value) || value == null)
return null;
return value switch
{
DateTimeOffset dto => dto.UtcDateTime,
DateTime dt => dt.Kind == DateTimeKind.Local
? dt.ToUniversalTime()
: DateTime.SpecifyKind(dt, DateTimeKind.Utc),
_ => null,
};
}
// ===== RowKey decoders =====
/// <summary>Decodes an audit RowKey ("!{rev:D19}_{guid:N}") to the original UTC write time.
/// Returns false for legacy bare-GUID RowKeys and any other foreign format.</summary>
internal static bool TryDecodeAuditRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length < 21 || rowKey[0] != '!')
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(1, 19), out utc);
}
/// <summary>Decodes an ops-event RowKey (bare "{rev:D19}") to the original UTC write time.</summary>
internal static bool TryDecodeOpsRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length != 19)
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(), out utc);
}
/// <summary>Decodes an enrollment-event RowKey ("{yyyyMMddHHmmssfff}_{seq:D10}") to the
/// (sanitized) agent event time, millisecond precision, Kind=Utc.</summary>
internal static bool TryDecodeEventRowKeyPrefix(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null)
return false;
var separator = rowKey.IndexOf('_');
if (separator != 17)
return false;
if (!DateTime.TryParseExact(rowKey.Substring(0, 17), EventRowKeyTimestampFormat,
CultureInfo.InvariantCulture,
DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out utc))
return false;
return true;
}
private static bool TryDecodeReverseTicks(ReadOnlySpan<char> revTickDigits, out DateTime utc)
{
utc = default;
foreach (var c in revTickDigits)
{
if (c < '0' || c > '9')
return false;
}
if (!long.TryParse(revTickDigits, NumberStyles.None, CultureInfo.InvariantCulture, out var rev))
return false;
var ticks = DateTime.MaxValue.Ticks - rev;
if (ticks < DateTime.MinValue.Ticks || ticks > DateTime.MaxValue.Ticks)
return false;
utc = new DateTime(ticks, DateTimeKind.Utc);
return true;
}
// ===== OData date-window clauses (RowKey ranges) =====
// Date windows filter on the RowKey rather than a property: rows missing a property are
// excluded from OData comparisons (so an OccurredUtc filter would drop every not-yet-
// backfilled row), and RowKey ranges are index-backed. String order equals numeric order
// because the reverse-tick is fixed-width D19. rev(t) = MaxTicks - t.Ticks, so larger
// timestamps have lexically smaller RowKeys (newest-first).
private static long Rev(DateTime utc) => DateTime.MaxValue.Ticks - utc.Ticks;
/// <summary>Audit rows with business time ≥ <paramref name="fromUtc"/> (inclusive):
/// rev ≤ rev(from) &lt; rev(from)+1. The exclusive upper bound also excludes all legacy
/// GUID rows (they sort after every '!' row).</summary>
internal static string AuditDateFromClause(DateTime fromUtc)
=> $"RowKey lt '{AuditRowKeyPrefix}{Rev(fromUtc) + 1:D19}'";
/// <summary>Audit rows with business time ≤ <paramref name="toUtc"/> (inclusive):
/// rev ≥ rev(to). The lower bound alone would match every legacy GUID row, hence the
/// mandatory time-encoded guard.</summary>
internal static string AuditDateToClause(DateTime toUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(toUtc):D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Audit rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// rev ≥ rev(cutoff)+1 ⇔ ts &lt; cutoff. MUST keep the time-encoded guard — without it
/// this clause matches (and a retention sweep would delete) every legacy GUID row.</summary>
internal static string AuditRetentionClause(DateTime cutoffUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(cutoffUtc) + 1:D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Ops rows with business time ≥ <paramref name="fromUtc"/> (inclusive). Bare D19
/// RowKey (no suffix), so `le` is exact.</summary>
internal static string OpsDateFromClause(DateTime fromUtc)
=> $"RowKey le '{Rev(fromUtc):D19}'";
/// <summary>Ops rows with business time ≤ <paramref name="toUtc"/> (inclusive).</summary>
internal static string OpsDateToClause(DateTime toUtc)
=> $"RowKey ge '{Rev(toUtc):D19}'";
/// <summary>Ops rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// `gt` not `ge` — a row at exactly the cutoff is kept, matching `Timestamp lt` semantics.</summary>
internal static string OpsRetentionClause(DateTime cutoffUtc)
=> $"RowKey gt '{Rev(cutoffUtc):D19}'";
}
}
@@ -252,6 +252,7 @@ builder.Services.AddHttpClient<GitHubRuleRepository>()
.AddPolicyHandler((sp, _) => sp.GetRequiredService<ResiliencePolicies>().ExternalDataApi);
builder.Services.AddSingleton<MaintenanceService>();
builder.Services.AddSingleton<LegacyReclassificationService>();
builder.Services.AddSingleton<OccurredUtcBackfillService>();
builder.Services.AddSingleton<
AutopilotMonitor.Functions.Services.Monitoring.IAzureMonitorMetricsReader,
AutopilotMonitor.Functions.Services.Monitoring.AzureMonitorMetricsReader>();
@@ -399,6 +399,9 @@ public static class EndpointAccessPolicyCatalog
// One-time retro-reconcile of misclassified historical sessions (audit 2026-07-16) —
// mutating, dry-run by default, GA only.
new("POST", "maintenance/reclassify-legacy", EndpointPolicy.GlobalAdminOnly),
// One-time OccurredUtc backfill on AuditLogs/OpsEvents (2026-07-18 migration reset
// the system Timestamps) — mutating, dry-run by default, GA only.
new("POST", "maintenance/backfill-occurred-utc", EndpointPolicy.GlobalAdminOnly),
// ── Critical-Table Backup (plan §PR1+§PR2) ─────────────────────────────
new("POST", "global/backups/trigger", EndpointPolicy.GlobalAdminOnly),
@@ -151,10 +151,38 @@ namespace AutopilotMonitor.Functions.Services
return await _ruleRepo.StoreRuleStateAsync(tenantId, rule.RuleId, new RuleState { Enabled = rule.Enabled });
}
// Custom rule. The portal toggle PUTs only { enabled, isBuiltIn, isCommunity } —
// full-storing that payload would wipe Title/Target/Trigger/Parameters. Merge
// toggle-style partials into the existing row; full payloads (edit flow) replace
// the row but keep the original CreatedAt.
var existing = (await _ruleRepo.GetGatherRulesAsync(tenantId))
.FirstOrDefault(r => r.RuleId == rule.RuleId && !r.IsBuiltIn && !r.IsCommunity);
var isTogglePartial = string.IsNullOrEmpty(rule.Title)
&& string.IsNullOrEmpty(rule.CollectorType)
&& string.IsNullOrEmpty(rule.Target);
if (isTogglePartial)
{
if (existing == null)
{
_logger.LogWarning(
"Toggle-style partial update for unknown custom gather rule {RuleId} (tenant {TenantId}); nothing to update",
rule.RuleId, tenantId);
return false;
}
existing.Enabled = rule.Enabled;
existing.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(existing, tenantId);
}
// Ensure custom rules keep correct flags (the incoming payload may
// omit them, causing them to default to IsBuiltIn=true in the model).
rule.IsBuiltIn = false;
rule.IsCommunity = false;
if (existing != null)
rule.CreatedAt = existing.CreatedAt;
rule.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(rule, tenantId);
}
@@ -413,7 +441,22 @@ namespace AutopilotMonitor.Functions.Services
&& a.Title == b.Title
&& a.CollectorType == b.CollectorType
&& a.Trigger == b.Trigger
&& a.TriggerPhase == b.TriggerPhase;
&& a.TriggerPhase == b.TriggerPhase
&& ActivePhasesEquivalent(a.ActivePhases, b.ActivePhases)
&& a.ActiveFromPhase == b.ActiveFromPhase
&& a.EmitMode == b.EmitMode;
}
/// <summary>
/// Null-tolerant ordered comparison for <see cref="GatherRule.ActivePhases"/>:
/// null and empty are both "unrestricted" and therefore equivalent, so pre-existing
/// DB rows (absent column → null) never diff against seeds (also null) — no reseed churn.
/// </summary>
private static bool ActivePhasesEquivalent(List<string>? a, List<string>? b)
{
var left = a ?? new List<string>();
var right = b ?? new List<string>();
return left.SequenceEqual(right, StringComparer.Ordinal);
}
}
}
@@ -34,9 +34,12 @@ public class HealthCheckService
}
/// <summary>
/// Performs all health checks and returns the results
/// Performs all health checks and returns the results.
/// <paramref name="includeEndpointUrls"/> adds the probed/backing endpoint URLs to the
/// per-check details — infrastructure topology, so callers must only enable it for
/// Global Admins.
/// </summary>
public async Task<HealthCheckResult> PerformAllChecksAsync()
public async Task<HealthCheckResult> PerformAllChecksAsync(bool includeEndpointUrls = false)
{
var result = new HealthCheckResult
{
@@ -50,9 +53,9 @@ public class HealthCheckService
// one slow check. It is exposed via its own endpoint (GET health/mcp) that the
// frontend fetches independently and folds into the card grid incrementally.
var checks = await Task.WhenAll(
CheckStorageBackendAsync(),
CheckProcessingBackendAsync(),
CheckAgentBinariesAsync(),
CheckStorageBackendAsync(includeEndpointUrls),
CheckProcessingBackendAsync(includeEndpointUrls),
CheckAgentBinariesAsync(includeEndpointUrls),
CheckSignalRQuotaAsync(),
CheckPoisonQueuesAsync()
);
@@ -73,7 +76,7 @@ public class HealthCheckService
/// <summary>
/// Checks Azure Table Storage connectivity by querying the admin configuration table
/// </summary>
private async Task<HealthCheck> CheckStorageBackendAsync()
private async Task<HealthCheck> CheckStorageBackendAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -97,29 +100,62 @@ public class HealthCheckService
_logger.LogError(ex, "Storage backend health check failed");
}
if (includeEndpointUrls)
{
// Same resolution order as TableStorageService: account name (Managed Identity)
// is the production path, connection string is the local-dev fallback.
var accountName = _configuration["AzureStorageAccountName"];
check.Details = new Dictionary<string, object>
{
["Table endpoint"] = string.IsNullOrWhiteSpace(accountName)
? "connection string (local dev)"
: $"https://{accountName}.table.core.windows.net"
};
}
return check;
}
/// <summary>
/// Checks that the Azure Functions host is running
/// </summary>
private static Task<HealthCheck> CheckProcessingBackendAsync()
private static Task<HealthCheck> CheckProcessingBackendAsync(bool includeEndpointUrls)
{
var uptime = DateTime.UtcNow - System.Diagnostics.Process.GetCurrentProcess().StartTime.ToUniversalTime();
return Task.FromResult(new HealthCheck
var check = new HealthCheck
{
Name = "Processing Backend",
Description = "Application host process",
Status = "healthy",
Message = $"Host running (uptime: {(int)uptime.TotalMinutes}m {uptime.Seconds}s)"
});
};
if (includeEndpointUrls)
{
// WEBSITE_HOSTNAME is stamped by App Service and reflects the host actually
// serving this process — no hardcoded URL to go stale after a migration.
var hostname = Environment.GetEnvironmentVariable("WEBSITE_HOSTNAME");
if (!string.IsNullOrWhiteSpace(hostname))
{
check.Details = new Dictionary<string, object>
{
["API endpoint"] = $"https://{hostname}"
};
}
}
return Task.FromResult(check);
}
/// <summary>
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable in blob storage
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable on the
/// canonical download alias (Front Door → current blob origin) AND on the legacy blob
/// keepalive account that already-deployed customer bootstrap scripts still use.
/// Alias failure is unhealthy (new installs break); legacy failure is a warning
/// (existing customers break until they migrate to the alias).
/// </summary>
private async Task<HealthCheck> CheckAgentBinariesAsync()
internal async Task<HealthCheck> CheckAgentBinariesAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -127,8 +163,20 @@ public class HealthCheckService
Description = "Agent download package availability"
};
var zipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var ps1Url = $"{Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var aliasZipUrl = $"{Constants.AgentDownloadBaseUrl}/{Constants.AgentZipFileName}";
var aliasPs1Url = $"{Constants.AgentDownloadBaseUrl}/{Constants.BootstrapScriptName}";
var legacyZipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var legacyPs1Url = $"{Constants.AgentBlobBaseUrl}/{Constants.BootstrapScriptName}";
if (includeEndpointUrls)
{
check.Details = new Dictionary<string, object>
{
["Agent ZIP"] = aliasZipUrl,
["Bootstrap script"] = aliasPs1Url,
["Legacy blob (keepalive)"] = Constants.AgentBlobBaseUrl
};
}
try
{
@@ -138,40 +186,48 @@ public class HealthCheckService
var sw = System.Diagnostics.Stopwatch.StartNew();
// HEAD requests to check availability without downloading
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasPs1Url)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyPs1Url))
);
sw.Stop();
var zipResponse = results[0];
var ps1Response = results[1];
var aliasZipOk = results[0].IsSuccessStatusCode;
var aliasPs1Ok = results[1].IsSuccessStatusCode;
var legacyZipOk = results[2].IsSuccessStatusCode;
var legacyPs1Ok = results[3].IsSuccessStatusCode;
var zipOk = zipResponse.IsSuccessStatusCode;
var ps1Ok = ps1Response.IsSuccessStatusCode;
var issues = new List<string>();
if (!aliasZipOk) issues.Add($"Agent ZIP (download alias): {(int)results[0].StatusCode}");
if (!aliasPs1Ok) issues.Add($"Bootstrap script (download alias): {(int)results[1].StatusCode}");
if (!legacyZipOk) issues.Add($"Agent ZIP (legacy blob): {(int)results[2].StatusCode}");
if (!legacyPs1Ok) issues.Add($"Bootstrap script (legacy blob): {(int)results[3].StatusCode}");
if (zipOk && ps1Ok)
if (issues.Count == 0)
{
check.Status = "healthy";
check.Message = $"Agent package and bootstrap script available ({sw.ElapsedMilliseconds}ms)";
check.Message = $"Agent package and bootstrap script available via download alias + legacy blob ({sw.ElapsedMilliseconds}ms)";
}
else if (aliasZipOk && aliasPs1Ok)
{
// Alias intact, legacy keepalive degraded — existing customer bootstrap
// scripts still download from the legacy account until they migrate.
check.Status = "warning";
check.Message = $"Download alias OK, but legacy keepalive degraded — {string.Join(", ", issues)}";
}
else
{
check.Status = "unhealthy";
var issues = new List<string>();
if (!zipOk) issues.Add($"Agent ZIP: {(int)zipResponse.StatusCode}");
if (!ps1Ok) issues.Add($"Bootstrap script: {(int)ps1Response.StatusCode}");
check.Message = $"Missing: {string.Join(", ", issues)}";
}
}
catch (Exception ex)
{
check.Status = "unhealthy";
check.Message = $"Blob storage unreachable: {ex.Message}";
check.Message = $"Agent download endpoints unreachable: {ex.Message}";
_logger.LogError(ex, "Agent binaries health check failed");
}
@@ -182,7 +238,8 @@ public class HealthCheckService
/// Probes the MCP server's public <c>/health</c> endpoint (which sits in front of the
/// <c>/mcp</c> access guard, so no auth is required) and surfaces reachability + the
/// deployed MCP build version. Visible to all authenticated users, so the server URL is
/// deliberately NOT included in the message or details.
/// only included in the details when <paramref name="includeEndpointUrl"/> is set —
/// callers must only enable it for Global Admins.
/// <para>
/// The Container App runs with minReplicas=0 and scales to zero on idle. Azure Container
/// Apps holds an inbound request while it activates a cold replica, so this probe doubles
@@ -193,7 +250,7 @@ public class HealthCheckService
/// the blocking all-checks batch, so the dashboard never waits on the wake.
/// </para>
/// </summary>
internal async Task<HealthCheck> CheckMcpServerAsync()
internal async Task<HealthCheck> CheckMcpServerAsync(bool includeEndpointUrl = false)
{
var check = new HealthCheck
{
@@ -203,6 +260,11 @@ public class HealthCheckService
var baseUrl = (_configuration["McpServerUrl"] ?? Constants.McpServerBaseUrl).TrimEnd('/');
var healthUrl = $"{baseUrl}/health";
if (includeEndpointUrl)
{
check.Details = new Dictionary<string, object> { ["Server URL"] = baseUrl };
}
var timeoutSeconds = ParsePositiveInt("McpServerHealthTimeoutSeconds", 30);
// A cold start that takes longer than this is treated as a successful wake rather
@@ -233,7 +295,8 @@ public class HealthCheckService
: $"MCP server reachable ({sw.ElapsedMilliseconds}ms)";
if (!string.IsNullOrWhiteSpace(version))
{
check.Details = new Dictionary<string, object> { ["Version"] = version };
check.Details ??= new Dictionary<string, object>();
check.Details["Version"] = version;
}
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException or HttpRequestException)
@@ -517,6 +517,52 @@ namespace AutopilotMonitor.Functions.Services
/// Removes distress reports older than 14 days. Distress data is unverified
/// and low-volume; short retention keeps Table Storage lean.
/// </summary>
/// <summary>
/// One-off reconciliation for tenants onboarded before <c>ContactEmail</c> existed:
/// copies the preview notification address they already supplied into their tenant
/// configuration, but only where no contact address is set yet.
/// <para>
/// Deliberately never overwrites an existing value — a tenant that has edited its
/// contact address owns it, and a maintenance run must not undo that. Going forward
/// the seed happens when the notification address is saved, so this converges to a
/// no-op and can be removed once every existing tenant has been covered.
/// </para>
/// </summary>
/// <returns>Number of tenants that received a contact address.</returns>
private async Task<int> BackfillTenantContactEmailsAsync()
{
var filled = 0;
try
{
var configs = await _tenantConfigService.GetAllConfigurationsAsync();
foreach (var config in configs)
{
if (config == null || string.IsNullOrWhiteSpace(config.TenantId))
continue;
if (!string.IsNullOrWhiteSpace(config.ContactEmail))
continue;
var email = await _previewWhitelistService.GetNotificationEmailAsync(config.TenantId);
if (string.IsNullOrWhiteSpace(email))
continue;
config.ContactEmail = email.Trim();
await _tenantConfigService.SaveConfigurationAsync(config);
filled++;
}
if (filled > 0)
_logger.LogInformation("Backfilled contact address for {Count} tenant(s)", filled);
}
catch (Exception ex)
{
// Housekeeping, not load-bearing: a failure here must not fail the whole run.
_logger.LogWarning(ex, "Tenant contact address backfill failed");
}
return filled;
}
private async Task CleanupOldDistressReportsAsync()
{
const int retentionDays = 14;
@@ -554,52 +600,53 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Verifies that agent binaries and bootstrap script are available on blob storage.
/// Verifies that agent binaries and bootstrap script are available on the canonical
/// download alias (Front Door → current blob origin) AND on the legacy blob keepalive
/// account that already-deployed customer bootstrap scripts still use.
/// Records an OpsEvent for each missing item so Global Admins see it in the dashboard.
/// </summary>
private async Task CheckAgentBlobStorageAsync()
{
_logger.LogInformation("Checking agent blob storage availability...");
_logger.LogInformation("Checking agent download endpoint availability...");
var zipUrl = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}";
var ps1Url = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var probes = new (string Label, string Url)[]
{
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}"),
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}")
};
try
{
var client = _httpClientFactory.CreateClient();
client.Timeout = TimeSpan.FromSeconds(15);
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
probes.Select(p => client.SendAsync(new HttpRequestMessage(HttpMethod.Head, p.Url)))
);
var zipResponse = results[0];
var ps1Response = results[1];
if (!zipResponse.IsSuccessStatusCode)
var allOk = true;
for (var i = 0; i < probes.Length; i++)
{
_logger.LogError("Agent ZIP not available on blob storage: HTTP {StatusCode}", (int)zipResponse.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Agent ZIP (AutopilotMonitor-Agent.zip)", (int)zipResponse.StatusCode);
if (results[i].IsSuccessStatusCode) continue;
allOk = false;
_logger.LogError("{Item} not available: HTTP {StatusCode}", probes[i].Label, (int)results[i].StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync(probes[i].Label, (int)results[i].StatusCode);
}
if (!ps1Response.IsSuccessStatusCode)
if (allOk)
{
_logger.LogError("Bootstrap script not available on blob storage: HTTP {StatusCode}", (int)ps1Response.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Bootstrap script (Install-AutopilotMonitor.ps1)", (int)ps1Response.StatusCode);
}
if (zipResponse.IsSuccessStatusCode && ps1Response.IsSuccessStatusCode)
{
_logger.LogInformation("Agent blob storage check passed: all binaries available");
_logger.LogInformation("Agent download endpoint check passed: all binaries available on alias + legacy blob");
}
}
catch (Exception ex)
{
_logger.LogError(ex, "Agent blob storage check failed — storage unreachable");
_logger.LogError(ex, "Agent download endpoint check failed — endpoints unreachable");
await _opsEventService.RecordBlobStorageUnreachableAsync(ex.Message);
}
}
@@ -32,6 +32,7 @@ namespace AutopilotMonitor.Functions.Services
public bool DataCleanupExecuted { get; set; }
public bool PlatformStatsRecomputed { get; set; }
public int DevicesBlockedForExcessiveData { get; set; }
public int ContactEmailsBackfilled { get; set; }
}
/// <summary>
@@ -64,6 +65,7 @@ namespace AutopilotMonitor.Functions.Services
private readonly IPoisonQueueProbe _poisonQueueProbe;
private readonly IConfiguration _configuration;
private readonly IHttpClientFactory _httpClientFactory;
private readonly PreviewWhitelistService _previewWhitelistService;
private readonly ILogger<MaintenanceService> _logger;
private const string PlatformStatsAliasFileName = "platform-stats.json";
@@ -92,6 +94,7 @@ namespace AutopilotMonitor.Functions.Services
IPoisonQueueProbe poisonQueueProbe,
IConfiguration configuration,
IHttpClientFactory httpClientFactory,
PreviewWhitelistService previewWhitelistService,
ILogger<MaintenanceService> logger)
{
_maintenanceRepo = maintenanceRepo;
@@ -115,6 +118,7 @@ namespace AutopilotMonitor.Functions.Services
_poisonQueueProbe = poisonQueueProbe;
_configuration = configuration;
_httpClientFactory = httpClientFactory;
_previewWhitelistService = previewWhitelistService;
_logger = logger;
}
@@ -214,6 +218,12 @@ namespace AutopilotMonitor.Functions.Services
// StoreSessionAsync Replace-mode IndexRowKey bug (now fixed).
await _maintenanceRepo.CleanupGhostSessionIndexEntriesAsync();
// One-off: give tenants onboarded before ContactEmail existed the contact
// address they already supplied as a preview notification email. New tenants
// are seeded at the point that address is saved, so this converges to a no-op
// and can be dropped once every existing tenant has been covered.
result.ContactEmailsBackfilled = await BackfillTenantContactEmailsAsync();
// Mirror the timer path: check embedded Intune cert bundle for
// expiring members so manual triggers also exercise the watcher.
await CheckEmbeddedCertExpiryAsync();
@@ -0,0 +1,169 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Shared;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// One-time backfill of the <c>OccurredUtc</c> business-timestamp column on
/// AuditLogs and OpsEvents rows written before the column existed. The value is
/// decoded deterministically from the reverse-tick RowKey, so re-runs are
/// idempotent and the write is a pure MERGE that adds exactly one property
/// (nothing existing is modified or deleted). Audit legacy bare-GUID rows carry
/// no recoverable time and are excluded server-side by the scan filter.
/// Readers do not depend on this backfill (they decode the RowKey on the fly);
/// its value is future-migration robustness and raw-query readability.
/// </summary>
public class OccurredUtcBackfillService
{
public const string TableAudit = "audit";
public const string TableOps = "ops";
private const int MaxSamplesInResponse = 100;
private const int TransactionChunkSize = 100;
private readonly TableStorageService _storage;
private readonly ILogger<OccurredUtcBackfillService> _logger;
public OccurredUtcBackfillService(TableStorageService storage, ILogger<OccurredUtcBackfillService> logger)
{
_storage = storage;
_logger = logger;
}
public class BackfillResult
{
public string Table { get; set; } = string.Empty;
public bool DryRun { get; set; }
public int RowsExamined { get; set; }
public int WouldWrite { get; set; }
public int Written { get; set; }
public int SkippedAlreadySet { get; set; }
public int SkippedUndecodable { get; set; }
public int Errors { get; set; }
public string? NextContinuation { get; set; }
public List<BackfillSample> Samples { get; set; } = new();
}
public class BackfillSample
{
public string PartitionKey { get; set; } = string.Empty;
public string RowKey { get; set; } = string.Empty;
public DateTime DecodedUtc { get; set; }
}
internal enum RowDecision { AlreadySet, Write, Undecodable }
/// <summary>Pure per-row decision, extracted for unit testing.</summary>
internal static RowDecision DecideRow(string table, TableEntity entity, out DateTime occurredUtc)
{
occurredUtc = default;
if (BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn).HasValue)
return RowDecision.AlreadySet;
var decoded = table == TableAudit
? BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out occurredUtc)
: BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out occurredUtc);
return decoded ? RowDecision.Write : RowDecision.Undecodable;
}
public async Task<BackfillResult> RunAsync(string table, bool dryRun, int maxRows, string? continuation)
{
var (tableName, scanFilter) = table switch
{
// Audit: server-side restriction to time-encoded '!'-rows — legacy bare-GUID
// rows (which sort after '"') have nothing to decode and are never touched.
TableAudit => (Constants.TableNames.AuditLogs,
$"RowKey ge '{BusinessTimestamp.AuditRowKeyPrefix}' and RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'"),
TableOps => (Constants.TableNames.OpsEvents, (string?)null),
_ => throw new ArgumentException($"Unknown table '{table}'", nameof(table)),
};
var tableClient = _storage.GetTableClient(tableName);
var result = new BackfillResult { Table = table, DryRun = dryRun };
var (entities, nextToken) = await AzureTablesPaginator.FetchPageAsync<TableEntity>(
client: tableClient,
filter: scanFilter,
pageSize: maxRows,
continuation: continuation,
select: new[] { "PartitionKey", "RowKey", BusinessTimestamp.OccurredUtcColumn });
result.NextContinuation = nextToken;
var pending = new List<(string PartitionKey, string RowKey, DateTime OccurredUtc)>();
foreach (var entity in entities)
{
result.RowsExamined++;
switch (DecideRow(table, entity, out var occurredUtc))
{
case RowDecision.AlreadySet:
result.SkippedAlreadySet++;
break;
case RowDecision.Undecodable:
result.SkippedUndecodable++;
_logger.LogWarning("OccurredUtc backfill: undecodable RowKey {PK}/{RK} in {Table}",
entity.PartitionKey, entity.RowKey, tableName);
break;
case RowDecision.Write:
result.WouldWrite++;
pending.Add((entity.PartitionKey, entity.RowKey, occurredUtc));
if (result.Samples.Count < MaxSamplesInResponse)
{
result.Samples.Add(new BackfillSample
{
PartitionKey = entity.PartitionKey,
RowKey = entity.RowKey,
DecodedUtc = occurredUtc,
});
}
break;
}
}
if (!dryRun && pending.Count > 0)
result.Written = await WritePendingAsync(tableClient, pending, result);
_logger.LogInformation(
"OccurredUtc backfill {Table}: dryRun={DryRun} examined={Examined} wouldWrite={WouldWrite} written={Written} alreadySet={AlreadySet} undecodable={Undecodable} errors={Errors} more={More}",
table, dryRun, result.RowsExamined, result.WouldWrite, result.Written,
result.SkippedAlreadySet, result.SkippedUndecodable, result.Errors, result.NextContinuation != null);
return result;
}
private async Task<int> WritePendingAsync(TableClient tableClient,
List<(string PartitionKey, string RowKey, DateTime OccurredUtc)> pending, BackfillResult result)
{
var written = 0;
foreach (var partitionGroup in pending.GroupBy(p => p.PartitionKey))
{
foreach (var chunk in partitionGroup.Chunk(TransactionChunkSize))
{
// Merge entity carries ONLY the new column; unconditional (ETag *) merge is
// safe because the value is a pure function of the immutable RowKey.
var actions = chunk.Select(p => new TableTransactionAction(
TableTransactionActionType.UpdateMerge,
new TableEntity(p.PartitionKey, p.RowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = p.OccurredUtc,
},
ETag.All)).ToList();
try
{
await tableClient.SubmitTransactionAsync(actions);
written += actions.Count;
}
catch (Exception ex)
{
result.Errors += actions.Count;
_logger.LogError(ex, "OccurredUtc backfill: merge transaction failed for partition {PK} ({Count} rows)",
partitionGroup.Key, actions.Count);
}
}
}
return written;
}
}
}
@@ -102,10 +102,44 @@ public class PreviewWhitelistService
}
/// <summary>
/// Saves (or clears) the notification email for a tenant.
/// Saves (or clears) the notification email for a tenant, and seeds the tenant's
/// contact address from it the first time one is given.
/// </summary>
public async Task SaveNotificationEmailAsync(string tenantId, string? email)
{
await _configRepo.SaveNotificationEmailAsync(tenantId, email);
await SeedContactEmailAsync(tenantId, email);
}
/// <summary>
/// One-way seed: copies the preview notification address into
/// <see cref="TenantConfiguration.ContactEmail"/> only while that field is still empty.
/// Once the tenant owns a contact address — seeded or edited in the portal — this never
/// touches it again, so a later change here cannot silently overwrite the tenant's choice.
/// Best-effort: the notification email has already been persisted by the caller, and a
/// failure to seed must not fail that write.
/// </summary>
private async Task SeedContactEmailAsync(string tenantId, string? email)
{
if (string.IsNullOrWhiteSpace(email))
return;
try
{
var config = await _configRepo.GetTenantConfigurationAsync(tenantId);
if (config == null || !string.IsNullOrWhiteSpace(config.ContactEmail))
return;
config.ContactEmail = email.Trim();
await _configRepo.SaveTenantConfigurationAsync(config);
_logger.LogInformation(
"Seeded tenant contact address for {TenantId} from the preview notification email", tenantId);
}
catch (Exception ex)
{
_logger.LogWarning(ex,
"Could not seed tenant contact address for {TenantId} — the notification email was still saved", tenantId);
}
}
}
@@ -1,5 +1,6 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Security;
using AutopilotMonitor.Shared;
@@ -55,7 +56,10 @@ namespace AutopilotMonitor.Functions.Services
{ "EntityType", entityType },
{ "EntityId", entityId },
{ "PerformedBy", performedBy },
{ "Timestamp", timestamp },
// Business event time. "Timestamp" is a reserved system property (server
// ignores supplied values and resets it on every row rewrite), so the
// event time must live in a regular custom column to survive migrations.
{ BusinessTimestamp.OccurredUtcColumn, timestamp },
{ "Details", details != null ? JsonConvert.SerializeObject(details) : string.Empty }
};
@@ -71,32 +75,26 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Retention cleanup: deletes audit log entries whose Timestamp is older than the cutoff.
/// The AuditLogs table is append-only (one row per admin action) and is otherwise only
/// wiped on tenant offboarding, so without this it grows unbounded. Filters server-side on
/// the per-row "Timestamp" property and selects only PK/RK to keep the scan cheap.
/// Retention cleanup: deletes audit log entries whose business time is older than the
/// cutoff. The AuditLogs table is append-only (one row per admin action) and is otherwise
/// only wiped on tenant offboarding, so without this it grows unbounded.
/// Two passes: (1) time-encoded '!'-rows via an index-backed RowKey range — the system
/// Timestamp cannot be used because storage migrations reset it (post-migration it would
/// delete nothing for a retention period, then the whole pre-migration corpus at once);
/// (2) legacy bare-GUID rows (true event time unrecoverable) via their frozen system
/// Timestamp — they all age out ~180d after the 2026-07-18 migration.
/// TODO: Remove pass 2 after 2027-03-01 (no bare-GUID rows can remain by then).
/// </summary>
public async Task<int> DeleteAuditLogsOlderThanAsync(DateTime cutoffUtc)
{
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.AuditLogs);
var filter = $"Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'";
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
var deleted = 0;
deleted += await DeleteAuditLogsByFilterAsync(tableClient, BusinessTimestamp.AuditRetentionClause(cutoffUtc));
// 'RowKey ge '0'' cannot match any '!'-row (0x21 < 0x30), only legacy GUIDs.
deleted += await DeleteAuditLogsByFilterAsync(tableClient,
$"RowKey ge '0' and Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'");
if (deleted > 0)
_logger.LogInformation("Deleted {Count} audit log entries older than {Cutoff:yyyy-MM-dd}", deleted, cutoffUtc);
@@ -110,6 +108,25 @@ namespace AutopilotMonitor.Functions.Services
}
}
private async Task<int> DeleteAuditLogsByFilterAsync(TableClient tableClient, string filter)
{
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
/// <summary>
/// Gets audit log entries for a tenant within an optional UTC date window.
/// No row cap — returns the full filtered set, sorted newest-first.
@@ -384,9 +401,9 @@ namespace AutopilotMonitor.Functions.Services
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
@@ -398,14 +415,30 @@ namespace AutopilotMonitor.Functions.Services
EntityType = entity.GetString("EntityType") ?? string.Empty,
EntityId = entity.GetString("EntityId") ?? string.Empty,
PerformedBy = entity.GetString("PerformedBy") ?? string.Empty,
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.UtcNow,
Timestamp = ResolveAuditTimestamp(entity),
Details = entity.GetString("Details") ?? string.Empty,
};
// The system Timestamp is auto-managed by Azure on insert and reliably
// sortable; the user-defined "Timestamp" property is set by LogAuditEntry
// for parity with the model. We filter on the system Timestamp since it
// is always indexed.
/// <summary>
/// Business time of an audit row: OccurredUtc column → time decoded from the
/// reverse-tick RowKey → system Timestamp. The system Timestamp is only a last
/// resort (legacy bare-GUID rows): it is reset by any row rewrite, so for rows
/// migrated between storage accounts it holds the migration moment, not the
/// event time.
/// </summary>
internal static DateTime ResolveAuditTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.UtcNow;
}
// Date windows filter on the RowKey (reverse-tick, index-backed) rather than any
// timestamp property: the system Timestamp is reset by migrations, and an
// OccurredUtc property filter would exclude rows that lack the column.
internal static string? BuildAuditLogFilter(string? tenantId, DateTime? dateFrom, DateTime? dateTo,
bool excludeDeletions = false, AuditLogQueryFilters? filters = null)
{
@@ -424,11 +457,11 @@ namespace AutopilotMonitor.Functions.Services
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
}
return string.Join(" and ", clauses);
}
@@ -158,6 +158,9 @@ namespace AutopilotMonitor.Functions.Services
["IntervalSeconds"] = rule.IntervalSeconds,
["TriggerPhase"] = rule.TriggerPhase ?? string.Empty,
["TriggerEventType"] = rule.TriggerEventType ?? string.Empty,
["ActivePhasesJson"] = JsonConvert.SerializeObject(rule.ActivePhases ?? new List<string>()),
["ActiveFromPhase"] = rule.ActiveFromPhase ?? string.Empty,
["EmitMode"] = rule.EmitMode ?? string.Empty,
["OutputEventType"] = rule.OutputEventType ?? string.Empty,
["OutputSeverity"] = rule.OutputSeverity ?? "Info",
["TagsJson"] = JsonConvert.SerializeObject(rule.Tags ?? new string[0]),
@@ -222,7 +225,9 @@ namespace AutopilotMonitor.Functions.Services
}
}
private GatherRule MapToGatherRule(TableEntity entity)
// Internal (not private) so the Store↔Map roundtrip is testable per the
// table-serialization rule: every model field must appear in BOTH directions.
internal GatherRule MapToGatherRule(TableEntity entity)
{
return new GatherRule
{
@@ -244,6 +249,9 @@ namespace AutopilotMonitor.Functions.Services
IntervalSeconds = entity.GetInt32("IntervalSeconds"),
TriggerPhase = entity.GetString("TriggerPhase") ?? string.Empty,
TriggerEventType = entity.GetString("TriggerEventType") ?? string.Empty,
ActivePhases = MapActivePhases(entity.GetString("ActivePhasesJson")),
ActiveFromPhase = string.IsNullOrEmpty(entity.GetString("ActiveFromPhase")) ? null : entity.GetString("ActiveFromPhase"),
EmitMode = string.IsNullOrEmpty(entity.GetString("EmitMode")) ? null : entity.GetString("EmitMode"),
OutputEventType = entity.GetString("OutputEventType") ?? string.Empty,
OutputSeverity = entity.GetString("OutputSeverity") ?? "Info",
Tags = DeserializeJsonArray(entity.GetString("TagsJson")),
@@ -252,6 +260,17 @@ namespace AutopilotMonitor.Functions.Services
};
}
/// <summary>
/// Absent column (pre-existing rows) and the stored-empty "[]" both map to null —
/// null and empty are the same "unrestricted" state on <see cref="GatherRule.ActivePhases"/>,
/// and null keeps <see cref="GatherRuleService.ContentEquivalent"/> reseed-stable.
/// </summary>
private List<string>? MapActivePhases(string? json)
{
var phases = DeserializeJson<List<string>>(json);
return phases.Count == 0 ? null : phases;
}
// ===== RULE STATES METHODS =====
/// <summary>
@@ -681,7 +681,10 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// "Timestamp" is a reserved system property — supplied values are ignored
// and any row rewrite (storage migration) resets it. The sanitized agent
// event time therefore lives in its own column.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -777,7 +780,8 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// Reserved-name caveat: see StoreEventAsync.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -2673,10 +2677,7 @@ namespace AutopilotMonitor.Functions.Services
EventId = entity.GetString("EventId") ?? string.Empty,
SessionId = entity.GetString("SessionId") ?? string.Empty,
TenantId = tenantId ?? partitionKey,
Timestamp = DateTime.SpecifyKind(
entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp")
?? DateTime.UtcNow, DateTimeKind.Utc),
Timestamp = DateTime.SpecifyKind(ResolveEventTimestamp(entity), DateTimeKind.Utc),
EventType = entity.GetString("EventType") ?? string.Empty,
Severity = (EventSeverity)(entity.GetInt32("Severity") ?? 0),
Source = entity.GetString("Source") ?? string.Empty,
@@ -2827,7 +2828,10 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Returns the earliest event timestamp persisted for a session, if any.
/// Events are written with RowKey "{Timestamp}_{Sequence}", so querying the partition
/// and taking the first row yields the earliest event.
/// and taking the first row yields the earliest event. Resolution must NOT rely on
/// the system Timestamp: it is reset by storage migrations, and this value feeds the
/// StartedAt/Duration reconciliation — a migration-reset read would rewrite those
/// session fields with the migration date.
/// </summary>
private async Task<DateTime?> GetEarliestSessionEventTimestampAsync(string tenantId, string sessionId)
{
@@ -2838,13 +2842,12 @@ namespace AutopilotMonitor.Functions.Services
var query = tableClient.QueryAsync<TableEntity>(
filter: $"PartitionKey eq '{partitionKey}'",
maxPerPage: 1,
select: new[] { "Timestamp", "RowKey" }
select: new[] { "RowKey", BusinessTimestamp.OccurredUtcColumn, "Timestamp" }
);
await foreach (var entity in query)
{
return entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp");
return ResolveEventTimestampOrNull(entity);
}
}
catch (Exception ex)
@@ -2855,6 +2858,26 @@ namespace AutopilotMonitor.Functions.Services
return null;
}
/// <summary>
/// Business time of an event row: OccurredUtc column → agent time decoded from the
/// "{yyyyMMddHHmmssfff}_{seq}" RowKey → system Timestamp (last resort — reset by any
/// row rewrite, e.g. a storage migration; for migrated rows it holds the migration
/// moment). Event rows are deliberately never backfilled, so the RowKey decode is the
/// primary path for all pre-OccurredUtc rows.
/// </summary>
internal static DateTime ResolveEventTimestamp(TableEntity entity)
=> ResolveEventTimestampOrNull(entity) ?? DateTime.UtcNow;
internal static DateTime? ResolveEventTimestampOrNull(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeEventRowKeyPrefix(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp");
}
/// <summary>
/// Returns the authoritative (total, reboot) event-row counts for a session, or <c>null</c>
/// if they could not be determined (storage error). Events are written with UpsertReplace
+25 -1
View File
@@ -18,6 +18,22 @@ COPY rules/analyze/ ./rules/analyze/
COPY rules/gather/ ./rules/gather/
COPY rules/ime-log-patterns/ ./rules/ime-log-patterns/
# Published customer documentation (autopilotmonitor-docs) — a SEPARATE public
# repo that the deploy workflow checks out into ./mcp-docs of the build context.
# Same byte-identical requirement as the rules above.
#
# This COPY is deliberately strict: if the workflow's docs checkout is ever
# removed, the build must fail loudly rather than quietly ship an image whose
# search_docs tool simply does not exist. For a local `docker build`, clone the
# bundle first:
# git clone --depth 1 https://github.com/autopilotmonitor/autopilotmonitor-docs mcp-docs
COPY mcp-docs/ ./docs/
# Commit SHA of that docs checkout. Surfaced on /health so a running server can be
# diffed against the docs repo HEAD — the bundle changes independently of this
# repo, so a stale deploy is otherwise invisible.
ARG DOCS_COMMIT=unknown
# Pre-bake the ~23 MB embedding model AND precompute the search index at image
# build time. Embedding the static corpus at boot cost 35-55s of ONNX inference
# on the 0.25 vCPU container (measured); hydrating precomputed vectors is
@@ -35,7 +51,7 @@ RUN mkdir -p "$HF_CACHE_DIR" /app/search-index && \
if [ "$PREBAKE_MODEL" = "1" ]; then \
ok=0; \
for i in 1 2 3 4 5; do \
node dist/precompute-embeddings.js ./rules /app/search-index/search-index.json && { ok=1; break; }; \
node dist/precompute-embeddings.js ./rules /app/search-index/search-index.json ./docs && { ok=1; break; }; \
echo "embedding precompute attempt $i/5 failed; retrying in 15s"; \
sleep 15; \
done && \
@@ -66,8 +82,16 @@ COPY rules/analyze/ ./rules/analyze/
COPY rules/gather/ ./rules/gather/
COPY rules/ime-log-patterns/ ./rules/ime-log-patterns/
# Customer documentation corpus — copied from the build stage rather than the
# context so it is provably the same bytes the embeddings were computed over.
COPY --from=build /app/docs/ ./docs/
ARG DOCS_COMMIT=unknown
ENV DOCS_COMMIT=${DOCS_COMMIT}
ENV PORT=8080
ENV RULES_DIR=/app/rules
ENV DOCS_DIR=/app/docs
ENV SEARCH_INDEX_PATH=/app/search-index/search-index.json
EXPOSE 8080
+52 -22
View File
@@ -1,12 +1,12 @@
{
"name": "autopilot-monitor-mcp",
"version": "1.5.0",
"version": "1.5.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "autopilot-monitor-mcp",
"version": "1.5.0",
"version": "1.5.1",
"dependencies": {
"@huggingface/transformers": "^3.8.1",
"@modelcontextprotocol/sdk": "^1.28.0",
@@ -1730,20 +1730,20 @@
}
},
"node_modules/body-parser": {
"version": "2.2.2",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.2.2.tgz",
"integrity": "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA==",
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz",
"integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==",
"license": "MIT",
"dependencies": {
"bytes": "^3.1.2",
"content-type": "^1.0.5",
"content-type": "^2.0.0",
"debug": "^4.4.3",
"http-errors": "^2.0.0",
"iconv-lite": "^0.7.0",
"http-errors": "^2.0.1",
"iconv-lite": "^0.7.2",
"on-finished": "^2.4.1",
"qs": "^6.14.1",
"raw-body": "^3.0.1",
"type-is": "^2.0.1"
"qs": "^6.15.2",
"raw-body": "^3.0.2",
"type-is": "^2.1.0"
},
"engines": {
"node": ">=18"
@@ -1753,6 +1753,19 @@
"url": "https://opencollective.com/express"
}
},
"node_modules/body-parser/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/boolean": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz",
@@ -3230,9 +3243,9 @@
}
},
"node_modules/protobufjs": {
"version": "7.6.4",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.4.tgz",
"integrity": "sha512-RJJPTTpvFfHcWLkIa2JFWK4XvtSzS0yEWDmunqHXli1h3JlkbcQZXDZdcWxv+JK3Xsl5/UFDPZ0iGm7DAengYw==",
"version": "7.6.5",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.5.tgz",
"integrity": "sha512-/FPD0nUc9jH6rfFjji9IBqOz4pcSE3CsT1m7Ep6Mdb0LxSUMj8hgl6GomOvZzpNpAqqGaXA0P3VSrZLFzIhQrw==",
"hasInstallScript": true,
"license": "BSD-3-Clause",
"dependencies": {
@@ -3674,9 +3687,9 @@
"license": "MIT"
},
"node_modules/tar": {
"version": "7.5.16",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.16.tgz",
"integrity": "sha512-56adEpPMouktRlBLXiaYFFzZ/3+JXa8P9n7WbR+ibIjtviN55mEaOkiysCnPnWm+7kkui1Dn8J9l+g6zV8731w==",
"version": "7.5.20",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.20.tgz",
"integrity": "sha512-9FcyK4PA6+WbzlTM9WhQm6vB5W7cP7dUiPsv1g7YDwEQnQ1CGpK3MGlKk/ITVWMk05kHZuBhmVhiv8LZoy/PFQ==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
@@ -3781,17 +3794,34 @@
}
},
"node_modules/type-is": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.0.1.tgz",
"integrity": "sha512-OZs6gsjF4vMp32qrCbiVSkrFmXtG/AZhY3t0iAMrMBiAZyV9oALtXO8hsrHbMXF9x6L3grlFuwW2oAz7cav+Gw==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
"integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==",
"license": "MIT",
"dependencies": {
"content-type": "^1.0.5",
"content-type": "^2.0.0",
"media-typer": "^1.1.0",
"mime-types": "^3.0.0"
},
"engines": {
"node": ">= 0.6"
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/type-is/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/typescript": {
@@ -0,0 +1,237 @@
/**
* Offline retrieval evaluation for search_docs.
*
* Exists because a single anecdote is not evidence. The WOW6432Node case showed
* that the keyword fallback misses when semantic search returns weak-but-present
* results — but "it fixes that one query" says nothing about how many queries a fix
* BREAKS. This harness answers that: it scores several ranking strategies over the
* same query set and reports, per class, how many queries each strategy fixes and
* how many it regresses relative to the deployed behaviour.
*
* Three query classes, deliberately different in character:
*
* known-item Every chunk's own heading as the query, expecting any chunk with
* that heading back. Easy by construction — its value is as a
* REGRESSION detector, not an absolute score. If a strategy drops
* these, it is breaking retrieval of things the corpus plainly names.
* literal Rare identifiers (few chunks, >=6 chars) as single-word queries,
* expecting the chunks that literally contain them. This is the class
* WOW6432Node belongs to and the one the fallback exists for.
* handwritten Real questions a human would ask, with the page they should reach.
* The reality anchor: unlike the other two, these are not derived from
* the corpus and share little vocabulary with it.
*
* Embeddings are computed ONCE per query and every strategy is a pure function over
* the same semantic + literal candidate lists, so a threshold sweep costs no extra
* inference.
*
* Usage: npx tsx scripts/eval-docs-search.ts [docsDir]
*/
import { loadDocsCorpus } from '../src/docs-corpus.js';
import { VectorSearchProvider } from '../src/vector-search-provider.js';
import { extractDistinctiveTerms } from '../src/tools/search.js';
import type { SearchDocument, SearchResult } from '../src/search-provider.js';
const DOCS_DIR = process.argv[2] ?? 'c:/Code/GitHubRepos/autopilotmonitor-docs';
const TOP_K = 3;
const MIN_SCORE = 0.25;
interface EvalQuery {
klass: 'known-item' | 'literal' | 'handwritten';
query: string;
/** Any of these ids counts as correct. */
expected: Set<string>;
}
// ── Query set ────────────────────────────────────────────────
function buildQueries(docs: SearchDocument[]): EvalQuery[] {
const queries: EvalQuery[] = [];
// known-item: heading → chunks carrying that heading.
const byHeading = new Map<string, Set<string>>();
for (const d of docs) {
const h = d.metadata.heading as string | null;
if (!h) continue;
const leaf = h.split('›').pop()!.trim();
if (leaf.length < 8) continue;
if (!byHeading.has(leaf)) byHeading.set(leaf, new Set());
byHeading.get(leaf)!.add(d.id);
}
for (const [heading, ids] of byHeading) {
queries.push({ klass: 'known-item', query: heading, expected: ids });
}
// literal: rare identifier-ish tokens → chunks literally containing them.
const occurrences = new Map<string, Set<string>>();
for (const d of docs) {
for (const t of new Set(d.text.toLowerCase().match(/[a-z][a-z0-9_-]{5,}/g) ?? [])) {
if (!occurrences.has(t)) occurrences.set(t, new Set());
occurrences.get(t)!.add(d.id);
}
}
for (const [token, ids] of occurrences) {
if (ids.size > 3) continue; // not rare enough to be a known-item probe
// Expected set is every chunk literally containing it, not just the tokenized hits.
const containing = new Set(docs.filter((d) => d.text.toLowerCase().includes(token)).map((d) => d.id));
queries.push({ klass: 'literal', query: token, expected: containing });
}
// handwritten: the reality anchor.
const byPath = (path: string) => new Set(docs.filter((d) => d.metadata.path === path).map((d) => d.id));
const handwritten: Array<[string, string]> = [
['how do I deploy the agent with Intune', 'troubleshooting/faq.md'],
['where is my data stored', 'troubleshooting/faq.md'],
['which role is allowed to delete a session', 'portal-guide/dashboard-and-sessions.md'],
['how do I set up Teams notifications', 'reference/settings.md'],
['what firewall URLs do I need to allow', 'reference/network-endpoints.md'],
['is Autopilot Monitor free', 'troubleshooting/faq.md'],
['how long is my data retained before deletion', 'trust/security-faq.md'],
['what is a gather rule and when does it run', 'rules/gather-rules.md'],
['what can helpdesk staff see in the progress portal', 'portal-guide/progress-portal.md'],
['can an MSP access my tenant', 'trust/security-faq.md'],
['Office Click-to-Run install never finished', 'rules/analyze-rules/built-in-rules.md'],
['proxy configuration blocking app downloads', 'rules/analyze-rules/built-in-rules.md'],
];
for (const [query, path] of handwritten) {
queries.push({ klass: 'handwritten', query, expected: byPath(path) });
}
return queries;
}
// ── Strategies ───────────────────────────────────────────────
interface Candidates {
semantic: SearchResult[]; // sorted desc, unfiltered
literal: SearchResult[]; // corroborated, ranked by term count
}
type Strategy = (c: Candidates) => SearchResult[];
/** Deployed behaviour: semantic, topped up by literal only when short of topK. */
const baseline: Strategy = ({ semantic, literal }) => {
const res = semantic.filter((s) => s.score >= MIN_SCORE).slice(0, TOP_K);
if (res.length < TOP_K) {
const have = new Set(res.map((r) => r.id));
for (const l of literal) {
if (res.length >= TOP_K) break;
if (have.has(l.id)) continue;
have.add(l.id);
res.push(l);
}
}
return res;
};
/**
* Candidate fix: additionally treat a WEAK semantic head as unconvincing and let
* exact literal matches take the front, keeping at least one semantic result so a
* weak-but-right hit is never fully displaced.
*/
function weakTrigger(threshold: number): Strategy {
return ({ semantic, literal }) => {
const res = semantic.filter((s) => s.score >= MIN_SCORE).slice(0, TOP_K);
const have = new Set(res.map((r) => r.id));
const fresh = literal.filter((l) => !have.has(l.id));
if (res.length < TOP_K) {
for (const l of fresh) {
if (res.length >= TOP_K) break;
res.push(l);
}
return res;
}
if (fresh.length > 0 && (res[0]?.score ?? 0) < threshold) {
return [...fresh.slice(0, TOP_K - 1), ...res].slice(0, TOP_K);
}
return res;
};
}
// ── Scoring ──────────────────────────────────────────────────
interface ClassStats {
n: number;
hit: number; // recall@TOP_K
mrr: number;
}
function score(results: SearchResult[], expected: Set<string>): { hit: boolean; rr: number } {
const rank = results.findIndex((r) => expected.has(r.id));
return { hit: rank >= 0, rr: rank >= 0 ? 1 / (rank + 1) : 0 };
}
async function main() {
const docs = await loadDocsCorpus(DOCS_DIR);
console.error(`corpus: ${docs.length} chunks`);
const index = new VectorSearchProvider();
await index.index(docs);
console.error(`indexed: ${index.vectorCount} vectors\n`);
const queries = buildQueries(docs);
const counts = queries.reduce<Record<string, number>>((a, q) => ({ ...a, [q.klass]: (a[q.klass] ?? 0) + 1 }), {});
console.error(`queries: ${queries.length} (${Object.entries(counts).map(([k, v]) => `${k}=${v}`).join(', ')})`);
console.error('embedding queries…');
// One inference pass; strategies are pure functions over these candidates.
const candidates: Candidates[] = [];
for (let i = 0; i < queries.length; i++) {
const q = queries[i];
const semantic = await index.search(q.query, { topK: 10, minScore: -1 });
const terms = extractDistinctiveTerms(q.query);
let literal: SearchResult[] = [];
if (terms.length > 0) {
const required = Math.min(2, terms.length);
literal = index
.lexicalMatch(terms)
.map((hit) => ({ hit, matched: terms.filter((t) => hit.text.toLowerCase().includes(t)).length }))
.filter((x) => x.matched >= required)
.sort((a, b) => b.matched - a.matched)
.map((x) => x.hit);
}
candidates.push({ semantic, literal });
if ((i + 1) % 200 === 0) console.error(` ${i + 1}/${queries.length}`);
}
const strategies: Array<[string, Strategy]> = [
['baseline (deployed)', baseline],
['weak<0.30', weakTrigger(0.30)],
['weak<0.35', weakTrigger(0.35)],
['weak<0.40', weakTrigger(0.40)],
['weak<0.45', weakTrigger(0.45)],
];
// Per-query correctness under the baseline, for the fixed/broken delta.
const baseHit = queries.map((q, i) => score(baseline(candidates[i]), q.expected).hit);
console.log(`\n${'strategy'.padEnd(20)} ${'class'.padEnd(12)} ${'n'.padStart(5)} ${'recall@3'.padStart(9)} ${'MRR'.padStart(7)} ${'fixed'.padStart(6)} ${'broken'.padStart(7)}`);
console.log('-'.repeat(75));
for (const [name, strat] of strategies) {
const stats = new Map<string, ClassStats>();
let fixed = 0;
let broken = 0;
queries.forEach((q, i) => {
const { hit, rr } = score(strat(candidates[i]), q.expected);
const s = stats.get(q.klass) ?? { n: 0, hit: 0, mrr: 0 };
s.n++;
if (hit) s.hit++;
s.mrr += rr;
stats.set(q.klass, s);
if (hit && !baseHit[i]) fixed++;
if (!hit && baseHit[i]) broken++;
});
for (const [klass, s] of [...stats].sort()) {
console.log(
`${name.padEnd(20)} ${klass.padEnd(12)} ${String(s.n).padStart(5)} ` +
`${((s.hit / s.n) * 100).toFixed(1).padStart(8)}% ${(s.mrr / s.n).toFixed(3).padStart(7)} ` +
`${'—'.padStart(6)} ${'—'.padStart(7)}`,
);
}
console.log(`${name.padEnd(20)} ${'TOTAL'.padEnd(12)} ${String(queries.length).padStart(5)} ${''.padStart(9)} ${''.padStart(7)} ${String(fixed).padStart(6)} ${String(broken).padStart(7)}`);
console.log('-'.repeat(75));
}
}
await main();
@@ -0,0 +1,210 @@
/**
* Unit tests for the customer-documentation corpus loader.
*
* The bundle is authored on Windows and ships CRLF, which is the single most
* dangerous property of this input: in JavaScript `.` does not match `\r`, so a
* pattern like /^(#{2,3})\s+(.*)$/ silently matches NOTHING on a CRLF file. The
* failure is not a crash — headings and frontmatter just stop being recognized
* and every page degrades to unlabelled paragraph splits. Several tests below
* therefore run the CRLF form on purpose.
*/
import { describe, it, expect } from 'vitest';
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { loadDocsCorpus, parseFrontmatter, docUrl, slugify, docSections } from '../docs-corpus.js';
async function withBundle<T>(
files: Record<string, string>,
fn: (root: string) => Promise<T>,
): Promise<T> {
const root = await mkdtemp(join(tmpdir(), 'docs-corpus-'));
try {
for (const [rel, content] of Object.entries(files)) {
const full = join(root, rel);
await mkdir(join(full, '..'), { recursive: true });
await writeFile(full, content, 'utf-8');
}
return await fn(root);
} finally {
await rm(root, { recursive: true, force: true });
}
}
describe('parseFrontmatter', () => {
it('reads scalars, folded blocks and inline sequences', () => {
const { frontmatter, body } = parseFrontmatter(
['---', 'type: Concept', 'tags: [security, gdpr]', 'description: >-', ' First line', ' second line', '---', '', '# Title', 'Body.'].join('\n'),
);
expect(frontmatter.type).toBe('Concept');
expect(frontmatter.tags).toEqual(['security', 'gdpr']);
expect(frontmatter.description).toBe('First line second line');
expect(body).toContain('# Title');
expect(body).not.toContain('type: Concept');
});
it('reads block sequences', () => {
const { frontmatter } = parseFrontmatter(['---', 'type: Guide', 'tags:', ' - agent', ' - intune', '---', 'x'].join('\n'));
expect(frontmatter.tags).toEqual(['agent', 'intune']);
});
it('parses CRLF frontmatter identically to LF', () => {
const lf = ['---', 'type: Concept', 'tags: [a, b]', '---', '', '# T', 'x'].join('\n');
expect(parseFrontmatter(lf.replace(/\n/g, '\r\n'))).toEqual(parseFrontmatter(lf));
});
it('returns the whole input as body when there is no frontmatter', () => {
const { frontmatter, body } = parseFrontmatter('# Just a title\n\nText.');
expect(frontmatter).toEqual({ type: undefined, title: undefined, description: undefined, tags: undefined });
expect(body).toBe('# Just a title\n\nText.');
});
});
describe('docUrl', () => {
it('maps a page path to its published URL', () => {
expect(docUrl('trust/security-faq.md')).toBe('https://docs.autopilotmonitor.com/trust/security-faq');
});
it('collapses README.md to its folder', () => {
expect(docUrl('rules/analyze-rules/README.md')).toBe('https://docs.autopilotmonitor.com/rules/analyze-rules');
expect(docUrl('README.md')).toBe('https://docs.autopilotmonitor.com');
});
it('has no URL for index.md — GitBook never publishes the OKF catalog', () => {
expect(docUrl('index.md')).toBeUndefined();
});
});
describe('slugify', () => {
it('strips markdown and punctuation', () => {
expect(slugify('Is the device\'s **IP address** stored?')).toBe('is-the-device-s-ip-address-stored');
});
});
describe('loadDocsCorpus', () => {
const page = (fm: string, body: string) => `---\n${fm}\n---\n\n${body}`;
it('splits on h2/h3 headings and carries a breadcrumb', async () => {
const chunks = await withBundle(
{
'concepts/roles.md': page(
'type: Concept\ntags: [roles]',
['# Roles & Permissions', '', 'Intro prose that is long enough to survive the minimum-length filter for chunks.', '', '## Tenant roles', '', 'A tenant admin can do everything, including managing team members and enabling admin mode.', '', '### Operator', '', 'An operator monitors day to day and cannot perform destructive operations at all.'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
const headings = chunks.map((c) => c.metadata.heading);
expect(headings).toContain('Tenant roles');
expect(headings).toContain('Tenant roles › Operator');
// The page title is prepended to every chunk so a mid-page hit is self-describing.
expect(chunks.every((c) => c.text.startsWith('Roles & Permissions'))).toBe(true);
expect(chunks[0].metadata.section).toBe('concepts');
expect(chunks[0].metadata.tags).toEqual(['roles']);
});
it('produces the SAME chunks for CRLF input as for LF', async () => {
const body = ['# Guide', '', 'Intro paragraph long enough to be kept as its own indexed chunk here.', '', '## Step one', '', 'Do the first thing, which needs a sentence long enough to pass the minimum.', '', '## Step two', '', 'Then do the second thing, also written out at a reasonable length for this.'].join('\n');
const src = page('type: Guide\ntags: [setup]', body);
const lf = await withBundle({ 'getting-started/g.md': src }, (r) => loadDocsCorpus(r));
const crlf = await withBundle({ 'getting-started/g.md': src.replace(/\n/g, '\r\n') }, (r) => loadDocsCorpus(r));
expect(crlf).toEqual(lf);
expect(lf.map((c) => c.metadata.heading)).toEqual([null, 'Step one', 'Step two']);
expect(lf.some((c) => c.text.includes('\r'))).toBe(false);
});
it('treats <details>/<summary> blocks as sections', async () => {
const chunks = await withBundle(
{
'troubleshooting/faq.md': page(
'type: FAQ\ntags: [faq]',
['# FAQ', '', '## General', '', '<details>', '', '<summary>Is Autopilot Monitor free?</summary>', '', 'Yes, the Community plan is free and stays free for everyone who wants to use it.', '', '</details>'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
expect(chunks.map((c) => c.metadata.heading)).toContain('General › Is Autopilot Monitor free?');
});
it('never emits a heading from inside a fenced code block', async () => {
const chunks = await withBundle(
{
'reference/cli.md': page(
'type: Reference\ntags: [cli]',
['# CLI', '', '## Usage', '', '```powershell', '# This is a shell comment, not a heading', '## Neither is this', '```', '', 'Explanatory prose after the block so the section is comfortably long enough.'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
expect(chunks.map((c) => c.metadata.heading)).toEqual(['Usage']);
});
it('skips SUMMARY.md and log.md but keeps index.md', async () => {
const chunks = await withBundle(
{
'SUMMARY.md': '# Table of contents\n\n* [A](a.md)\n',
'log.md': '# Log\n\nSome change record entry that is long enough to be a chunk on its own.\n',
'index.md': page('okf_version: "0.1"', '# Bundle\n\nThe documentation map listing every page available in this bundle here.'),
},
(root) => loadDocsCorpus(root),
);
const paths = [...new Set(chunks.map((c) => c.metadata.path))];
expect(paths).toEqual(['index.md']);
// index.md is indexed for its content but is not a published page.
expect(chunks[0].metadata.url).toBeNull();
});
it('splits an oversized unbroken list so its tail is not truncated away', async () => {
const items = Array.from({ length: 60 }, (_, i) => `* Release note number ${i} describing a change that shipped in this release.`);
const chunks = await withBundle(
{ 'changelog/platform.md': page('type: Changelog\ntags: [changelog]', ['# Platform Changelog', '', '## July 2026', '', ...items].join('\n')) },
(root) => loadDocsCorpus(root),
);
expect(chunks.length).toBeGreaterThan(1);
expect(chunks.every((c) => c.text.length <= 1700)).toBe(true);
// The final item must still be reachable — that is the whole point of splitting.
expect(chunks.some((c) => c.text.includes('number 59'))).toBe(true);
});
it('keeps a markdown table intact rather than splitting mid-table', async () => {
const rows = Array.from({ length: 40 }, (_, i) => `| Setting ${i} | A reasonably wordy description of what this particular setting controls. |`);
const chunks = await withBundle(
{ 'reference/settings.md': page('type: Reference\ntags: [settings]', ['# Settings', '', '## All settings', '', '| Name | Description |', '| --- | --- |', ...rows].join('\n')) },
(root) => loadDocsCorpus(root),
);
const withTable = chunks.filter((c) => c.text.includes('| Setting '));
expect(withTable).toHaveLength(1);
expect(withTable[0].text).toContain('| Setting 39 |');
});
it('assigns unique, stable ids across repeated loads', async () => {
const files = {
'a/one.md': page('type: Guide\ntags: [x]', '# One\n\n## Same\n\nFirst body text that is long enough to be indexed as a chunk.\n\n## Same\n\nSecond body text that is also long enough to be indexed here.'),
};
const first = await withBundle(files, (r) => loadDocsCorpus(r));
const second = await withBundle(files, (r) => loadDocsCorpus(r));
expect(new Set(first.map((c) => c.id)).size).toBe(first.length);
expect(first.map((c) => c.id)).toEqual(second.map((c) => c.id));
});
it('returns an empty corpus for a missing directory instead of throwing', async () => {
await expect(loadDocsCorpus(join(tmpdir(), 'definitely-not-here-9d2f'))).resolves.toEqual([]);
});
});
describe('docSections', () => {
it('lists distinct top-level areas, sorted', async () => {
const chunks = await withBundle(
{
'trust/a.md': '# A\n\nSome prose that is definitely long enough to become an indexed chunk.',
'concepts/b.md': '# B\n\nMore prose that is definitely long enough to become an indexed chunk.',
'root.md': '# R\n\nRoot level prose that is long enough to become an indexed chunk too.',
},
(root) => loadDocsCorpus(root),
);
expect(docSections(chunks)).toEqual(['concepts', 'general', 'trust']);
});
});
@@ -14,8 +14,19 @@ const MODEL = 'Xenova/all-MiniLM-L6-v2';
const doc = (id: string, text: string, metadata: Record<string, unknown> = {}): SearchDocument =>
({ id, text, metadata });
/**
* Fake embeddings still have to satisfy the real contract: indexPrecomputed()
* asserts L2-unit norm, because cosineSimilarity is a bare dot product that would
* otherwise return silently wrong scores. Normalize here rather than hand-writing
* unit vectors, so a test can pass any direction it finds readable.
*/
const unit = (v: number[]): number[] => {
const norm = Math.sqrt(v.reduce((s, x) => s + x * x, 0));
return v.map((x) => x / norm);
};
const withEmbedding = (d: SearchDocument, embedding: number[] = [0.1, 0.2, 0.3]): PrecomputedDocument =>
({ ...d, embedding });
({ ...d, embeddings: [unit(embedding)] });
const KB_DOCS = [doc('rule-1', 'TPM not ready', { type: 'analyze-rule' }), doc('rule-2', 'ESP timeout', { type: 'analyze-rule' })];
const ET_DOCS = [doc('et-1', 'bitlocker status', { eventType: 'bitlocker_status' })];
@@ -99,14 +110,73 @@ describe('validatePrecomputedIndex', () => {
it('rejects inconsistent embedding dimensions', () => {
const f = validFile();
f.knowledgeBase.entries[1].embedding = [0.1, 0.2]; // 2 dims vs 3
f.knowledgeBase.entries[1].embeddings = [[0.1, 0.2]]; // 2 dims vs 3
expect(validatePrecomputedIndex(f, MODEL, KB_DOCS, ET_DOCS)).toMatchObject({
ok: false, reason: expect.stringContaining('dimensions'),
});
});
});
describe('docs section (optional third corpus)', () => {
const DOC_CHUNKS = [doc('docs:trust/security-faq.md#intro', 'Where is my data stored', { type: 'doc', section: 'trust' })];
const withDocs = (): PrecomputedIndexFile => ({
...validFile(),
docs: { docsHash: hashDocs(DOC_CHUNKS), entries: DOC_CHUNKS.map((d) => withEmbedding(d)) },
});
it('returns the docs entries when the corpus matches', () => {
const result = validatePrecomputedIndex(withDocs(), MODEL, KB_DOCS, ET_DOCS, DOC_CHUNKS);
expect(result).toMatchObject({ ok: true });
expect(result.ok && result.docs).toHaveLength(1);
});
it('rejects the whole file when the docs corpus changed since the build', () => {
const stale = [doc('docs:trust/security-faq.md#intro', 'DIFFERENT TEXT', { type: 'doc', section: 'trust' })];
expect(validatePrecomputedIndex(withDocs(), MODEL, KB_DOCS, ET_DOCS, stale)).toMatchObject({
ok: false, reason: expect.stringContaining('docs'),
});
});
it('rejects when the server has docs but the image shipped none — that is staleness, not absence', () => {
expect(validatePrecomputedIndex(validFile(), MODEL, KB_DOCS, ET_DOCS, DOC_CHUNKS)).toMatchObject({
ok: false, reason: expect.stringContaining('docs'),
});
});
it('accepts a file WITHOUT a docs section when the server has no docs corpus', () => {
// A local checkout has no docs repo. That must not throw away the rules and
// event-type vectors too — recomputing those is the expensive half.
const result = validatePrecomputedIndex(validFile(), MODEL, KB_DOCS, ET_DOCS, []);
expect(result).toMatchObject({ ok: true });
expect(result.ok && result.docs).toEqual([]);
});
});
describe('VectorSearchProvider serialize/hydrate roundtrip', () => {
it('rounds embeddings on serialize but stays well inside the unit-norm tolerance', () => {
const p = new VectorSearchProvider();
const raw = unit([0.123456789012345, -0.987654321098765, 0.5555555555555]);
p.indexPrecomputed([{ ...doc('d1', 'text'), embeddings: [raw] }]);
const [out] = p.serialize();
// Rounded to 6 decimals — the point of the exercise (halves the index JSON).
expect(out.embeddings[0].every((v) => v === Number(v.toFixed(6)))).toBe(true);
expect(out.embeddings[0]).not.toEqual(raw);
const norm = Math.sqrt(out.embeddings[0].reduce((s, x) => s + x * x, 0));
expect(Math.abs(norm - 1)).toBeLessThan(1e-5);
// Re-hydrating the rounded vectors must not trip the assertion.
expect(() => new VectorSearchProvider().indexPrecomputed(p.serialize())).not.toThrow();
});
it('rejects a non-unit embedding on hydrate rather than skewing every score silently', () => {
const p = new VectorSearchProvider();
expect(() => p.indexPrecomputed([{ ...doc('d1', 'text'), embeddings: [[0.1, 0.2, 0.3]] }]))
.toThrow(/not L2-unit-normalized/);
});
it('indexPrecomputed restores exactly what serialize exported, without the model', () => {
const source = new VectorSearchProvider();
source.indexPrecomputed(KB_DOCS.map((d, i) => withEmbedding(d, [i + 0.5, i + 1.5])));
@@ -20,16 +20,62 @@
import { describe, it, expect } from 'vitest';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { registerTools } from '../tools.js';
import type { SearchProvider } from '../search-provider.js';
function registeredToolNames(ga: boolean, strictGa: boolean = ga, delegated: boolean = false): string[] {
const server = new McpServer({ name: 'test', version: '0.0.0' });
// knowledgeBase / eventTypeIndex are optional — pass undefined so this stays a
// pure unit test with no search-provider or backend dependency.
registerTools(server, undefined, undefined, ga, strictGa, delegated);
registerTools(server, undefined, undefined, undefined, ga, strictGa, delegated);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
return Object.keys(internal._registeredTools);
}
/** Minimal stand-in for an indexed docs corpus — registration only checks `size`. */
function stubDocsIndex(size = 3): SearchProvider {
return {
name: 'stub',
semanticCapable: true,
size,
index: async () => {},
search: async () => [],
};
}
function namesWithDocs(ga: boolean, strictGa: boolean = ga, delegated: boolean = false): string[] {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, { vector: stubDocsIndex(), sections: ['concepts', 'trust'] }, ga, strictGa, delegated);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
return Object.keys(internal._registeredTools);
}
describe('search_docs registration', () => {
it('is NOT registered when no documentation corpus was baked into the image', () => {
expect(registeredToolNames(true)).not.toContain('search_docs');
});
it('is registered when a corpus is present — for EVERY role, including a plain tenant user', () => {
// Published product documentation carries no tenant data, so unlike the
// platform tools it is deliberately ungated. A tenant user asking "how do I
// deploy the agent" has exactly as much business here as a Global Admin.
expect(namesWithDocs(true, true)).toContain('search_docs');
expect(namesWithDocs(false, false)).toContain('search_docs');
expect(namesWithDocs(false, false, true)).toContain('search_docs');
});
it('is skipped for an empty corpus, so a doc-less build advertises no broken tool', () => {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, { vector: stubDocsIndex(0), sections: [] }, true, true, false);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
expect(Object.keys(internal._registeredTools)).not.toContain('search_docs');
});
it('keeps the catalog alphabetically sorted once it joins', () => {
const names = namesWithDocs(true);
expect(names).toEqual([...names].sort());
});
});
describe('tool catalog ordering', () => {
it('lists tools in alphabetical order for a Global Admin', () => {
const names = registeredToolNames(true);
@@ -27,7 +27,7 @@ const MANAGED = 'aaaa-1111-managed';
/** Build a server for the given caller shape and return a tool's raw handler by name. */
function handlerFor(name: string, opts: { ga?: boolean; strictGa?: boolean; delegated?: boolean }): ToolHandler {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, opts.ga ?? false, opts.strictGa ?? (opts.ga ?? false), opts.delegated ?? false);
registerTools(server, undefined, undefined, undefined, opts.ga ?? false, opts.strictGa ?? (opts.ga ?? false), opts.delegated ?? false);
const registry = (server as unknown as { _registeredTools: Record<string, { handler: ToolHandler }> })._registeredTools;
const tool = registry[name];
if (!tool) throw new Error(`tool ${name} not registered for this caller shape`);
@@ -29,7 +29,7 @@ type ToolHandler = (args: Record<string, unknown>, extra: unknown) => Promise<{
/** Build a strict-GA server and return a tool's raw handler by name. */
function gaHandler(name: string): ToolHandler {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, /*ga*/ true, /*strictGa*/ true, /*delegated*/ false);
registerTools(server, undefined, undefined, undefined, /*ga*/ true, /*strictGa*/ true, /*delegated*/ false);
const registry = (server as unknown as { _registeredTools: Record<string, { handler: ToolHandler }> })._registeredTools;
const tool = registry[name];
if (!tool) throw new Error(`tool ${name} not registered for strict-GA caller`);
@@ -0,0 +1,219 @@
/**
* Unit tests for the search_docs handler — specifically the keyword fallback.
*
* Semantic search misses exact proper nouns that embed poorly. Measured on the real
* corpus, "RealmJoin" occurs in 3 chunks yet scores below the semantic threshold,
* while fuzzy matching at a threshold low enough to surface it also returns 5 hits
* for "SCEPman", a word the corpus does not contain at all. The fallback therefore
* uses literal containment, and the contract under test is:
* appended after semantic hits, flagged, never re-sorted together, never invented.
*
* The provider is a stub: no model, no backend, no corpus on disk.
*/
import { describe, it, expect } from 'vitest';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { registerTools } from '../tools.js';
import { extractDistinctiveTerms } from '../tools/search.js';
import type { SearchDocument, SearchProvider, SearchResult } from '../search-provider.js';
type ToolHandler = (args: Record<string, unknown>) => Promise<{ content: Array<{ text: string }> }>;
const chunk = (id: string, section: string, text = 'body'): SearchResult => ({
id,
text,
metadata: { title: id, heading: null, section, path: `${section}/x.md`, url: null, tags: [] },
score: 0.5,
});
/**
* Provider returning fixed lists, ignoring the query. `lexical` omitted entirely
* models a provider that cannot expose document text (lexicalMatch is optional).
*/
function stubProvider(semantic: SearchResult[], lexical?: SearchResult[]): SearchProvider {
return {
name: 'stub',
semanticCapable: true,
size: Math.max(semantic.length, 1),
index: async (_docs: SearchDocument[]) => {},
search: async () => semantic,
...(lexical ? { lexicalMatch: () => lexical } : {}),
};
}
function searchDocsHandler(provider: SearchProvider): ToolHandler {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, { vector: provider, sections: ['trust', 'reference'] }, true, true, false);
const registry = (server as unknown as { _registeredTools: Record<string, { handler: ToolHandler }> })._registeredTools;
return registry.search_docs.handler;
}
async function run(handler: ToolHandler, args: Record<string, unknown>) {
const res = await handler(args);
return JSON.parse(res.content[0].text) as {
resultCount: number;
keywordFallback?: { used: boolean; added: number };
results: Array<{ id: string; matchType?: string; section: string }>;
};
}
describe('extractDistinctiveTerms', () => {
it('keeps product and setting names, drops filler', () => {
expect(extractDistinctiveTerms('how does the RealmJoin watcher work'))
.toEqual(['realmjoin', 'watcher', 'work']);
});
it('drops words too short to be distinctive', () => {
expect(extractDistinctiveTerms('who can do it')).toEqual([]);
});
it('keeps identifier-shaped tokens intact', () => {
expect(extractDistinctiveTerms('what is CollectorIdleTimeoutMinutes'))
.toContain('collectoridletimeoutminutes');
});
it('deduplicates and caps the term list', () => {
const terms = extractDistinctiveTerms('alpha alpha beta gamma delta epsilon zeta eta theta');
expect(new Set(terms).size).toBe(terms.length);
expect(terms.length).toBeLessThanOrEqual(6);
});
});
describe('search_docs keyword fallback', () => {
it('does not fire when semantic search already fills topK', async () => {
const handler = searchDocsHandler(
stubProvider([chunk('sem-1', 'trust'), chunk('sem-2', 'trust')], [chunk('kw-1', 'trust')]),
);
const body = await run(handler, { query: 'notifications setup', topK: 2 });
expect(body.resultCount).toBe(2);
expect(body.keywordFallback).toBeUndefined();
expect(body.results.every((r) => r.matchType === undefined)).toBe(true);
});
it('tops up a short result set and flags the added hits', async () => {
const both = 'the realmjoin watcher toggle';
const handler = searchDocsHandler(
stubProvider([chunk('sem-1', 'trust')], [chunk('kw-1', 'trust', both), chunk('kw-2', 'trust', both)]),
);
const body = await run(handler, { query: 'realmjoin watcher', topK: 3 });
expect(body.resultCount).toBe(3);
expect(body.keywordFallback).toEqual({ used: true, added: 2 });
// Semantic keeps the head; keyword fills the tail. Never interleaved — literal
// hits carry no cosine similarity to sort against.
expect(body.results.map((r) => r.matchType)).toEqual([undefined, 'keyword', 'keyword']);
});
it('ranks fallback hits by how many query terms they contain', async () => {
const handler = searchDocsHandler(
stubProvider([], [
chunk('two-terms', 'trust', 'mentions realmjoin and the watcher'),
chunk('three-terms', 'trust', 'mentions realmjoin, the watcher and its toggle'),
]),
);
const body = await run(handler, { query: 'realmjoin watcher toggle', topK: 2 });
expect(body.results.map((r) => r.id)).toEqual(['three-terms', 'two-terms']);
});
it('requires corroboration: one incidental term match is not a hit', async () => {
// "bake" is a substring of "baked into the image" — matching ANY single term
// would return a page about LLM providers for a sourdough question. Measured
// on the real corpus before the corroboration rule was added.
const handler = searchDocsHandler(
stubProvider([], [chunk('incidental', 'trust', 'the corpus is baked into the image')]),
);
const body = await run(handler, { query: 'how do I bake sourdough bread', topK: 3 });
expect(body.resultCount).toBe(0);
});
it('never duplicates a document already returned semantically', async () => {
const shared = chunk('same-doc', 'trust', 'realmjoin');
const handler = searchDocsHandler(stubProvider([shared], [shared, chunk('kw-1', 'trust', 'realmjoin')]));
const body = await run(handler, { query: 'realmjoin', topK: 3 });
expect(body.results.map((r) => r.id)).toEqual(['same-doc', 'kw-1']);
expect(body.keywordFallback).toEqual({ used: true, added: 1 });
});
it('honours the section filter for fallback hits too', async () => {
const handler = searchDocsHandler(
stubProvider([], [chunk('kw-other', 'reference', 'realmjoin'), chunk('kw-match', 'trust', 'realmjoin')]),
);
const body = await run(handler, { query: 'realmjoin', topK: 3, section: 'trust' });
expect(body.results.map((r) => r.id)).toEqual(['kw-match']);
expect(body.results.every((r) => r.section === 'trust')).toBe(true);
});
it('lets exact matches take the front when the semantic head is weak', async () => {
// The WOW6432Node case: semantic fills topK with chunks scoring ~0.29 that do
// not contain the term, while the one chunk that does never gets considered.
// Measured over 2465 queries, acting on a weak head fixed 335 and broke 0.
const weak = (id: string) => ({ ...chunk(id, 'trust'), score: 0.29 });
const handler = searchDocsHandler(
stubProvider(
[weak('weak-1'), weak('weak-2'), weak('weak-3')],
[chunk('exact', 'trust', 'mentions wow6432node verbatim')],
),
);
const body = await run(handler, { query: 'wow6432node', topK: 3 });
expect(body.results[0].id).toBe('exact');
expect(body.results[0].matchType).toBe('keyword');
// At least one semantic result survives — a weak-but-correct hit is never
// pushed out entirely.
expect(body.results.map((r) => r.id)).toContain('weak-1');
expect(body.keywordFallback).toEqual({ used: true, added: 1 });
});
it('leaves a confident semantic ranking untouched', async () => {
const strong = (id: string) => ({ ...chunk(id, 'trust'), score: 0.72 });
const handler = searchDocsHandler(
stubProvider(
[strong('sem-1'), strong('sem-2'), strong('sem-3')],
[chunk('exact', 'trust', 'mentions wow6432node verbatim')],
),
);
const body = await run(handler, { query: 'wow6432node', topK: 3 });
expect(body.results.map((r) => r.id)).toEqual(['sem-1', 'sem-2', 'sem-3']);
expect(body.keywordFallback).toBeUndefined();
});
it('returns nothing when neither pass matches — no invented results', async () => {
const handler = searchDocsHandler(stubProvider([], []));
const body = await run(handler, { query: 'how do I bake sourdough bread', topK: 5 });
expect(body.resultCount).toBe(0);
expect(body.results).toEqual([]);
expect(body.keywordFallback).toBeUndefined();
});
it('skips the fallback when the query has no distinctive terms', async () => {
const handler = searchDocsHandler(stubProvider([], [chunk('kw-1', 'trust')]));
const body = await run(handler, { query: 'who can do it', topK: 3 });
expect(body.resultCount).toBe(0);
});
it('works with a provider that exposes no lexicalMatch at all', async () => {
const handler = searchDocsHandler(stubProvider([chunk('sem-1', 'trust')]));
const body = await run(handler, { query: 'realmjoin watcher', topK: 3 });
expect(body.resultCount).toBe(1);
expect(body.keywordFallback).toBeUndefined();
});
});
@@ -5,7 +5,7 @@
* assertUnitNorm directly with canned vectors — no ML model load required.
*/
import { describe, it, expect } from 'vitest';
import { assertUnitNorm } from '../vector-search-provider.js';
import { assertUnitNorm, splitIntoWindows, VectorSearchProvider, embed } from '../vector-search-provider.js';
/** Build an L2-unit vector of the given dimension (all components equal). */
function unitVector(dim: number): number[] {
@@ -46,3 +46,62 @@ describe('assertUnitNorm', () => {
expect(() => assertUnitNorm([5, 0], 'index:doc-42')).toThrow(/index:doc-42/);
});
});
describe('splitIntoWindows', () => {
it('returns short text unchanged as a single window', async () => {
const text = 'Retention is per tenant and configurable, default 90 days.';
await expect(splitIntoWindows(text)).resolves.toEqual([text]);
});
it('splits text beyond the model context into several overlapping windows', async () => {
// ~1200 distinct words — comfortably past the 250-token window.
const long = Array.from({ length: 1200 }, (_, i) => `token${i}`).join(' ');
const windows = await splitIntoWindows(long);
expect(windows.length).toBeGreaterThan(1);
// Overlap: consecutive windows must share content, or a sentence sitting on a
// boundary would be split across two vectors and match neither well.
const tailOfFirst = windows[0].split(/\s+/).slice(-10);
expect(tailOfFirst.some((w) => windows[1].includes(w))).toBe(true);
// Coverage is the whole point: the last token must survive somewhere.
expect(windows.some((w) => w.includes('token1199'))).toBe(true);
});
});
describe('VectorSearchProvider scoring across windows', () => {
const unit3 = (v: number[]) => {
const n = Math.sqrt(v.reduce((s, x) => s + x * x, 0));
return v.map((x) => x / n);
};
it('scores a document by its BEST window, not by its first', async () => {
// Uses the real embedder to obtain a query vector, then plants that exact
// direction as a document's SECOND window. Before windowing, content past the
// first 250 tokens could not be matched at all — this pins that it now can.
const query = 'delivery optimization download timeout';
const target = await embed(query);
const decoy = target.map((_, i) => (i === 0 ? 1 : 0)); // unit, but unrelated direction
const p = new VectorSearchProvider();
p.indexPrecomputed([
{ id: 'match-in-second-window', text: 'long doc', metadata: {}, embeddings: [decoy, target] },
{ id: 'first-window-only', text: 'short doc', metadata: {}, embeddings: [decoy] },
]);
const results = await p.search(query, { topK: 2, minScore: 0.5 });
expect(results[0].id).toBe('match-in-second-window');
// Cosine with itself is 1 — proof the second window, not the first, set the score.
expect(results[0].score).toBeCloseTo(1, 5);
expect(p.vectorCount).toBe(3);
expect(p.size).toBe(2);
});
it('counts vectors separately from documents', () => {
const p = new VectorSearchProvider();
p.indexPrecomputed([
{ id: 'a', text: 'a', metadata: {}, embeddings: [unit3([1, 0, 0]), unit3([0, 1, 0]), unit3([0, 0, 1])] },
]);
expect(p.size).toBe(1);
expect(p.vectorCount).toBe(3);
});
});
@@ -0,0 +1,461 @@
/**
* Loads the published customer documentation bundle (autopilotmonitor-docs) and
* chunks it into SearchDocuments for semantic search via the `search_docs` tool.
*
* This is the product-knowledge corpus — "how do I deploy the agent", "where is
* my data stored", "which roles exist". It is deliberately SEPARATE from the
* rules corpus in knowledge-base.ts: rules answer "why did this session fail",
* docs answer "how does the product work". Indexed together they would compete
* for the same topK and dilute each other.
*
* Like the rules corpus, docs are baked into the image and embedded at Docker
* build time (see precompute-embeddings.ts) — the 0.25-0.5 vCPU container never
* embeds a corpus at boot.
*
* Chunking matters more here than for rules: a rule is already a short, self-
* contained record, whereas a doc page runs to 36 KB (trust/security-faq.md).
* MiniLM-L6-v2 truncates at ~256 tokens, so a whole-page embedding would silently
* represent only the first paragraph. We therefore split on heading boundaries,
* which in this bundle are also the semantic boundaries (one h3 = one question).
*/
import { readdir, readFile } from 'node:fs/promises';
import type { Dirent } from 'node:fs';
import { join, relative, sep } from 'node:path';
import type { SearchDocument } from './search-provider.js';
/** Published base URL of the bundle — used to build a citable link per chunk. */
const DOCS_BASE_URL = 'https://docs.autopilotmonitor.com';
/**
* Upper bound on chunk text. MiniLM-L6-v2's 256-token window is roughly 1000-1200
* characters of English prose; going much beyond that means the tail of the chunk
* contributes nothing to the embedding. We allow a little slack over that so a
* coherent section (or an indivisible table) is not split for the sake of a few
* characters — the overflow is only ever the part that gets truncated.
*/
const MAX_CHUNK_CHARS = 1500;
/**
* A trailing fragment shorter than this is folded back into the preceding chunk
* when splitting an oversized section, so a split never leaves a stranded line.
*/
const MIN_CHUNK_CHARS = 120;
/**
* Minimum body length for a chunk to be indexed at all, EXCLUDING the breadcrumb
* header. Set low on purpose: a short section is very often the most valuable
* kind of hit ("Is Autopilot Monitor free?" → "Yes — the Community plan is free
* and stays free"), and the breadcrumb already supplies the context that a short
* body lacks. This filter only exists to drop structural residue — a stray rule
* line, an empty section left behind by stripped GitBook markers.
*/
const MIN_BODY_CHARS = 25;
/**
* Structural/meta files that are not documentation content.
* `SUMMARY.md` is the GitBook table of contents and `log.md` is the OKF change
* log — both are navigation scaffolding whose text would match queries about
* every topic at once. `index.md` (the OKF bundle catalog) is deliberately KEPT:
* it is the documentation map and answers "what documentation exists".
*/
const SKIP_FILES = new Set(['SUMMARY.md', 'log.md']);
const SKIP_DIRS = new Set(['.git', '.github', '.gitbook', 'node_modules']);
// ── Frontmatter ──────────────────────────────────────────────
/**
* Normalize CRLF/CR to LF before ANY parsing.
*
* Not cosmetic — a correctness precondition. The docs repo is authored on
* Windows and ships CRLF, and in JavaScript `.` does not match `\r`. A pattern
* like /^(#{2,3})\s+(.*)$/ therefore fails on every single heading of a CRLF
* file: `(.*)` stops before the `\r` and `$` never reaches end-of-string. The
* observable symptom is silent, not a crash — headings and frontmatter keys
* simply stop being recognized and the page degrades to unlabelled paragraph
* splits. Normalizing once here is the root fix; the alternative is remembering
* `\r?` in every pattern in this file forever.
*/
function normalizeNewlines(text: string): string {
return text.replace(/\r\n?/g, '\n');
}
export interface DocFrontmatter {
type?: string;
title?: string;
description?: string;
tags?: string[];
}
/**
* Minimal YAML frontmatter parser covering exactly the forms this bundle uses:
* plain scalars, folded blocks (`description: >-`), inline sequences
* (`tags: [a, b]`) and block sequences (`- a`). A full YAML parser would be a new
* runtime dependency in a hardened, dependency-minimal container for the sake of
* four flat keys.
*
* Anything it cannot parse is skipped rather than thrown — a malformed header
* must not cost us the page body, which is the part being searched.
*/
export function parseFrontmatter(input: string): { frontmatter: DocFrontmatter; body: string } {
const raw = normalizeNewlines(input);
if (!raw.startsWith('---')) return { frontmatter: {}, body: raw };
const end = raw.indexOf('\n---', 3);
if (end === -1) return { frontmatter: {}, body: raw };
const header = raw.slice(raw.indexOf('\n') + 1, end);
const body = raw.slice(raw.indexOf('\n', end + 1) + 1);
const fm: Record<string, string | string[]> = {};
const lines = header.split('\n');
for (let i = 0; i < lines.length; i++) {
const match = /^([A-Za-z_][\w-]*):\s*(.*)$/.exec(lines[i]);
if (!match) continue;
const key = match[1];
let value = match[2].trim();
// Folded / literal block scalar: consume the following indented lines.
if (value === '>-' || value === '>' || value === '|' || value === '|-') {
const parts: string[] = [];
while (i + 1 < lines.length && /^\s+\S/.test(lines[i + 1])) {
parts.push(lines[++i].trim());
}
fm[key] = parts.join(' ');
continue;
}
// Block sequence: `tags:` followed by `- item` lines.
if (value === '') {
const items: string[] = [];
while (i + 1 < lines.length && /^\s*-\s+/.test(lines[i + 1])) {
items.push(lines[++i].replace(/^\s*-\s+/, '').trim().replace(/^["']|["']$/g, ''));
}
if (items.length > 0) fm[key] = items;
continue;
}
// Inline sequence: `[a, b, c]`.
if (value.startsWith('[') && value.endsWith(']')) {
fm[key] = value
.slice(1, -1)
.split(',')
.map((s) => s.trim().replace(/^["']|["']$/g, ''))
.filter(Boolean);
continue;
}
fm[key] = value.replace(/^["']|["']$/g, '');
}
const asString = (v: string | string[] | undefined): string | undefined =>
typeof v === 'string' ? v : undefined;
const asArray = (v: string | string[] | undefined): string[] | undefined =>
Array.isArray(v) ? v : typeof v === 'string' && v ? [v] : undefined;
return {
frontmatter: {
type: asString(fm.type),
title: asString(fm.title),
description: asString(fm.description),
tags: asArray(fm.tags),
},
body,
};
}
// ── Markdown normalization ───────────────────────────────────
/**
* Strip GitBook-specific block syntax while keeping the prose inside it.
* `{% hint style="info" %}` wrappers carry no meaning for retrieval but do add
* tokens that dilute the embedding; the sentence inside them frequently IS the
* answer, so the content is preserved and only the markers removed.
*/
function stripGitbookSyntax(text: string): string {
return text
.replace(/^\s*\{%\s*(end)?(hint|tabs|tab|content-ref|embed|code)[^%]*%\}\s*$/gim, '')
.replace(/^\s*\{%\s*end\w+\s*%\}\s*$/gim, '')
.replace(/\n{3,}/g, '\n\n')
.trim();
}
/** URL/anchor-safe slug from heading text. */
export function slugify(text: string): string {
return text
.toLowerCase()
.replace(/[`*_[\]()]/g, '')
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '')
.slice(0, 60);
}
/**
* Public GitBook URL for a bundle-relative path, or undefined when the file is
* not published as a page.
*
* `index.md` is an OKF reserved file that the bundle deliberately keeps out of
* SUMMARY.md, so GitBook never renders it — emitting a link would hand callers a
* 404. Its content is still indexed (it is the documentation map); it just has no
* citable URL.
*/
export function docUrl(relPath: string): string | undefined {
const posix = relPath.split(sep).join('/');
if (posix === 'index.md') return undefined;
const withoutExt = posix.replace(/\.md$/, '');
const path = withoutExt === 'README' ? '' : withoutExt.replace(/\/README$/, '');
return path ? `${DOCS_BASE_URL}/${path}` : DOCS_BASE_URL;
}
// ── Sectioning ───────────────────────────────────────────────
interface Section {
/** Heading trail below the page title, e.g. ['Data Processing', 'What data…']. */
breadcrumb: string[];
/** Anchor of the deepest markdown heading, if this section came from one. */
anchor?: string;
content: string;
}
/**
* Split a page body into heading-delimited sections.
*
* Handles both structures the bundle uses: markdown h2/h3 headings (the common
* case — trust/security-faq.md carries 37 h3s, one per question) and
* `<details><summary>Q</summary>…</details>` blocks (troubleshooting/faq.md),
* where the summary is the question and therefore the natural section title.
*/
function splitIntoSections(body: string): Section[] {
const sections: Section[] = [];
const lines = body.split('\n');
let h2: string | null = null;
let h3: string | null = null;
let detailsTitle: string | null = null;
let buffer: string[] = [];
let inFence = false;
const flush = (anchorSource: string | null) => {
const content = buffer.join('\n').trim();
buffer = [];
if (!content) return;
const breadcrumb = [h2, detailsTitle ?? h3].filter((s): s is string => Boolean(s));
sections.push({
breadcrumb,
anchor: anchorSource ? slugify(anchorSource) : undefined,
content,
});
};
for (const line of lines) {
// Never interpret markdown inside a fenced code block as structure.
if (/^\s*```/.test(line)) {
inFence = !inFence;
buffer.push(line);
continue;
}
if (inFence) {
buffer.push(line);
continue;
}
const summary = /^\s*<summary>(.*?)<\/summary>\s*$/i.exec(line);
if (summary) {
flush(detailsTitle ?? h3 ?? h2);
detailsTitle = summary[1].replace(/<[^>]+>/g, '').trim();
continue;
}
if (/^\s*<\/details>\s*$/i.test(line)) {
flush(null);
detailsTitle = null;
continue;
}
if (/^\s*<details>\s*$/i.test(line)) continue;
const heading = /^(#{2,3})\s+(.*)$/.exec(line);
if (heading) {
flush(detailsTitle ?? h3 ?? h2);
detailsTitle = null;
const text = heading[2].replace(/[*_`]/g, '').trim();
if (heading[1] === '##') {
h2 = text;
h3 = null;
} else {
h3 = text;
}
continue;
}
buffer.push(line);
}
flush(detailsTitle ?? h3 ?? h2);
return sections;
}
/**
* Split an oversized section at paragraph boundaries.
*
* Markdown tables are kept intact: a table split across chunks leaves a body of
* `| --- |` rows with no header, which is both unreadable in a result and
* meaningless to embed. Tables carry real answers in this bundle (the role matrix
* in concepts/roles-and-permissions.md is one), so an over-long table is allowed
* to exceed MAX_CHUNK_CHARS rather than be broken.
*/
function splitOversized(content: string): string[] {
if (content.length <= MAX_CHUNK_CHARS) return [content];
const blocks = content.split(/\n\s*\n/);
const chunks: string[] = [];
let current: string[] = [];
let currentLen = 0;
for (const block of blocks) {
const isTable = /^\s*\|/.test(block);
if (currentLen > 0 && currentLen + block.length > MAX_CHUNK_CHARS && !isTable) {
chunks.push(current.join('\n\n'));
current = [];
currentLen = 0;
}
current.push(block);
currentLen += block.length + 2;
}
if (current.length > 0) chunks.push(current.join('\n\n'));
// A single block can still exceed the budget on its own — the changelog pages
// are one unbroken bullet list per release with no blank lines, so paragraph
// splitting cannot touch them. Left whole, everything past the model's ~256
// tokens would be truncated away and become unsearchable, so fall back to
// packing whole LINES. Contiguous table rows stay together (see above).
const final: string[] = [];
for (const chunk of chunks) {
if (chunk.length <= MAX_CHUNK_CHARS) {
final.push(chunk);
continue;
}
let buf: string[] = [];
let len = 0;
for (const line of chunk.split('\n')) {
const isTableRow = /^\s*\|/.test(line);
if (len > 0 && len + line.length > MAX_CHUNK_CHARS && !isTableRow) {
final.push(buf.join('\n'));
buf = [];
len = 0;
}
buf.push(line);
len += line.length + 1;
}
if (buf.length > 0) final.push(buf.join('\n'));
}
// Fold a runt tail back into its predecessor.
if (final.length > 1 && final[final.length - 1].length < MIN_CHUNK_CHARS) {
const tail = final.pop()!;
final[final.length - 1] += `\n\n${tail}`;
}
return final;
}
// ── File walking ─────────────────────────────────────────────
async function walkMarkdown(root: string, dir = root): Promise<string[]> {
let entries: Dirent[];
try {
entries = await readdir(dir, { withFileTypes: true, encoding: 'utf-8' });
} catch {
return [];
}
const files: string[] = [];
for (const entry of entries) {
if (entry.isDirectory()) {
if (SKIP_DIRS.has(entry.name)) continue;
files.push(...(await walkMarkdown(root, join(dir, entry.name))));
} else if (entry.name.endsWith('.md') && !SKIP_FILES.has(entry.name)) {
files.push(relative(root, join(dir, entry.name)));
}
}
return files;
}
// ── Public API ───────────────────────────────────────────────
/**
* Load and chunk the customer documentation bundle at `docsRoot`.
*
* Returns an empty array when the directory is absent or holds no markdown —
* a local checkout without the docs repo is a supported state, and the caller
* then simply does not register `search_docs`. A missing corpus must never be
* a boot failure.
*/
export async function loadDocsCorpus(docsRoot: string): Promise<SearchDocument[]> {
const files = (await walkMarkdown(docsRoot)).sort();
const docs: SearchDocument[] = [];
const usedIds = new Set<string>();
for (const relPath of files) {
let raw: string;
try {
raw = await readFile(join(docsRoot, relPath), 'utf-8');
} catch {
continue;
}
const { frontmatter, body } = parseFrontmatter(raw);
const posixPath = relPath.split(sep).join('/');
// Title: frontmatter wins (the technical bundle carries it), then the page's
// own h1 (the customer bundle's only source), then the filename.
const h1 = /^#\s+(.+)$/m.exec(body);
const title =
frontmatter.title ??
h1?.[1].replace(/[*_`]/g, '').trim() ??
posixPath.replace(/\.md$/, '').split('/').pop()!;
// Top-level directory doubles as the bundle's section taxonomy
// (getting-started, concepts, portal-guide, …). Root files are 'general'.
const section = posixPath.includes('/') ? posixPath.split('/')[0] : 'general';
const url = docUrl(relPath);
// Drop the h1 line itself — it is carried in every chunk's header instead.
const withoutH1 = body.replace(/^#\s+.+$/m, '');
const sections = splitIntoSections(stripGitbookSyntax(withoutH1));
for (const sec of sections) {
for (const piece of splitOversized(sec.content)) {
const trail = [title, ...sec.breadcrumb].join(' › ');
// The breadcrumb is prepended to the embedded text so a chunk lifted out
// of the middle of a page still states which page and section it is from —
// both for the embedding and for a human reading the tool result.
if (piece.replace(/[^A-Za-z0-9]/g, '').length < MIN_BODY_CHARS) continue;
const text = `${trail}\n\n${piece}`.trim();
const base = `docs:${posixPath}#${slugify(sec.breadcrumb.join('-')) || 'intro'}`;
let id = base;
for (let n = 2; usedIds.has(id); n++) id = `${base}-${n}`;
usedIds.add(id);
docs.push({
id,
text,
metadata: {
type: 'doc',
title,
section,
path: posixPath,
heading: sec.breadcrumb.join(' › ') || null,
docType: frontmatter.type ?? null,
tags: frontmatter.tags ?? [],
url: url ? (sec.anchor ? `${url}#${sec.anchor}` : url) : null,
},
});
}
}
}
return docs;
}
/** Distinct top-level sections present in a loaded corpus — drives the tool's filter enum. */
export function docSections(docs: SearchDocument[]): string[] {
return [...new Set(docs.map((d) => String(d.metadata.section)))].sort();
}
@@ -9,6 +9,7 @@ import { registerTools } from './tools.js';
import { registerResources } from './resources.js';
import { registerPrompts } from './prompts.js';
import { loadKnowledgeDocs } from './knowledge-base.js';
import { loadDocsCorpus, docSections } from './docs-corpus.js';
import { createSearchProvider, resolveBackend } from './search-factory.js';
import type { SearchBackend, SearchProvider } from './search-provider.js';
import { MODEL_NAME, VectorSearchProvider, embed } from './vector-search-provider.js';
@@ -24,9 +25,34 @@ const __dirname = dirname(fileURLToPath(import.meta.url));
const pkg = JSON.parse(readFileSync(resolve(__dirname, '..', 'package.json'), 'utf-8')) as { version: string };
const SERVER_VERSION: string = pkg.version;
/**
* Boot stopwatch. minReplicas=0 means every idle period ends in a cold start a user
* actually waits through — measured at 24.5s to first byte, against 60ms warm.
* Where those seconds go was NOT answerable from outside: Log Analytics stamps all
* console lines of one ingestion batch with the same time, so the boot sequence
* collapses to a single instant no matter how it is queried.
*
* So the server times itself. `process.uptime()` counts from process start, which
* includes Node's own startup and module loading — the part an in-process timer
* started at the top of this file would miss.
*/
const bootMark = (): string => `+${(process.uptime() * 1000).toFixed(0)}ms`;
const PORT = parseInt(process.env.PORT ?? '8080', 10);
const RULES_DIR = process.env.RULES_DIR ?? resolve(__dirname, '..', '..', '..', '..', 'rules');
// Published customer documentation bundle (autopilotmonitor-docs), baked into the
// image by the deploy workflow. Unset in a plain local checkout — the docs live in
// a separate repo — in which case the corpus is empty and search_docs is simply not
// registered. Point DOCS_DIR at a local clone to develop against it.
const DOCS_DIR = process.env.DOCS_DIR ?? '';
// Commit of the docs bundle this image was built from. The docs repo changes
// independently of this one, so a deployed image can silently serve stale
// documentation; surfacing the SHA on /health makes that checkable instead of a
// thing to remember.
const DOCS_COMMIT = process.env.DOCS_COMMIT ?? 'unknown';
// Surface the MCP_PUBLIC_URL state at boot. In production a missing pin is a
// hard boot failure (config.ts throws — host-spoofing defense); this dev-only
// notice flags the forwarded-header fallback. Two-stage deploy expectation:
@@ -46,13 +72,18 @@ console.error(`[startup] Backend API base URL: ${API_BASE_URL}`);
// --- Load shared knowledge base (reused across all sessions) ---
console.error('Loading knowledge base documents…');
console.error(`[boot ${bootMark()}] node started, loading corpora from disk…`);
const docs = await loadKnowledgeDocs(RULES_DIR);
const eventTypeDocs = buildEventTypeSearchDocs();
const docsDocs = DOCS_DIR ? await loadDocsCorpus(DOCS_DIR) : [];
if (DOCS_DIR && docsDocs.length === 0) {
console.error(`[startup] DOCS_DIR=${DOCS_DIR} yielded no documents — search_docs will NOT be registered.`);
}
async function buildSearchIndexes(backend?: SearchBackend): Promise<{
knowledgeBase: SearchProvider;
eventTypeIndex: SearchProvider;
docsIndex: SearchProvider | undefined;
}> {
const kb = await createSearchProvider(backend);
await kb.index(docs);
@@ -61,7 +92,15 @@ async function buildSearchIndexes(backend?: SearchBackend): Promise<{
// the embedder singleton with the knowledge base, so this adds ~no memory.
const et = await createSearchProvider(backend);
await et.index(eventTypeDocs);
return { knowledgeBase: kb, eventTypeIndex: et };
// Product documentation — a third corpus, deliberately not merged into the
// knowledge base (rules answer "why did this fail", docs answer "how does this
// work"; sharing one topK would dilute both).
let dx: SearchProvider | undefined;
if (docsDocs.length > 0) {
dx = await createSearchProvider(backend);
await dx.index(docsDocs);
}
return { knowledgeBase: kb, eventTypeIndex: et, docsIndex: dx };
}
/**
@@ -72,7 +111,12 @@ async function buildSearchIndexes(backend?: SearchBackend): Promise<{
*/
const SEARCH_INDEX_PATH = process.env.SEARCH_INDEX_PATH ?? resolve(__dirname, '..', 'search-index.json');
function tryLoadPrecomputedIndexes(): { knowledgeBase: SearchProvider; eventTypeIndex: SearchProvider } | null {
function tryLoadPrecomputedIndexes(): {
knowledgeBase: SearchProvider;
eventTypeIndex: SearchProvider;
docsIndex: SearchProvider | undefined;
} | null {
console.error(`[boot ${bootMark()}] corpora loaded, reading precomputed index…`);
let raw: string;
try {
raw = readFileSync(SEARCH_INDEX_PATH, 'utf-8');
@@ -81,7 +125,7 @@ function tryLoadPrecomputedIndexes(): { knowledgeBase: SearchProvider; eventType
return null;
}
try {
const validated = validatePrecomputedIndex(JSON.parse(raw), MODEL_NAME, docs, eventTypeDocs);
const validated = validatePrecomputedIndex(JSON.parse(raw), MODEL_NAME, docs, eventTypeDocs, docsDocs);
if (!validated.ok) {
console.error(`[startup] Precomputed search index rejected (${validated.reason}) — computing embeddings at boot.`);
return null;
@@ -90,7 +134,13 @@ function tryLoadPrecomputedIndexes(): { knowledgeBase: SearchProvider; eventType
kb.indexPrecomputed(validated.knowledgeBase);
const et = new VectorSearchProvider();
et.indexPrecomputed(validated.eventTypes);
return { knowledgeBase: kb, eventTypeIndex: et };
let dx: SearchProvider | undefined;
if (validated.docs.length > 0) {
const provider = new VectorSearchProvider();
provider.indexPrecomputed(validated.docs);
dx = provider;
}
return { knowledgeBase: kb, eventTypeIndex: et, docsIndex: dx };
} catch (err) {
console.error('[startup] Failed to read precomputed search index — computing embeddings at boot:', err);
return null;
@@ -106,27 +156,44 @@ function tryLoadPrecomputedIndexes(): { knowledgeBase: SearchProvider; eventType
console.error(`Initializing search provider (${docs.length} documents)…`);
let knowledgeBase: SearchProvider;
let eventTypeIndex: SearchProvider;
let docsIndex: SearchProvider | undefined;
const precomputed = (await resolveBackend()) === 'vector' ? tryLoadPrecomputedIndexes() : null;
if (precomputed) {
({ knowledgeBase, eventTypeIndex } = precomputed);
({ knowledgeBase, eventTypeIndex, docsIndex } = precomputed);
// Serving precomputed vectors does not need the embedder — only incoming
// queries do. Warm it in the background so neither boot nor the readiness
// probe waits on the model load; a search arriving first awaits the same
// memoized load instead of failing.
embed('embedder warmup').then(
() => console.error('Query embedder warm.'),
() => console.error(`[boot ${bootMark()}] Query embedder warm (background; only the first SEARCH waits on this).`),
(err) => console.error('[startup] Query embedder warmup failed — semantic ranking degrades until it loads:', err),
);
} else {
try {
({ knowledgeBase, eventTypeIndex } = await buildSearchIndexes());
({ knowledgeBase, eventTypeIndex, docsIndex } = await buildSearchIndexes());
} catch (err) {
console.error('[startup] Search index initialization failed (embedding model unavailable?) — falling back to the keyword (fuse) backend:', err);
({ knowledgeBase, eventTypeIndex } = await buildSearchIndexes('fuse'));
({ knowledgeBase, eventTypeIndex, docsIndex } = await buildSearchIndexes('fuse'));
}
}
console.error(`Search provider ready: ${knowledgeBase.name} — ${knowledgeBase.size} documents indexed${precomputed ? ' (precomputed)' : ''}.`);
console.error(`Event-type index ready: ${eventTypeIndex.name} — ${eventTypeIndex.size} types indexed${precomputed ? ' (precomputed)' : ''}.`);
if (docsIndex) {
// Vectors exceed chunks whenever a chunk spans several token windows — surfacing
// both makes a regression in the windowing visible in the boot log.
const vectors = (docsIndex as { vectorCount?: number }).vectorCount;
console.error(
`[boot ${bootMark()}] Docs index ready: ${docsIndex.name} — ${docsIndex.size} chunks` +
`${vectors ? ` / ${vectors} vectors` : ''} indexed${precomputed ? ' (precomputed)' : ''} ` +
`(docs commit ${DOCS_COMMIT}).`,
);
} else {
console.error('Docs index: not available — search_docs is not registered.');
}
// Computed once: the tool catalog is rebuilt per request, and this is a pure
// function of the (immutable) corpus.
const DOCS = docsIndex ? { vector: docsIndex, sections: docSections(docsDocs) } : undefined;
// Server-level guidance. The host surfaces this once per connection, so it is
// the right home for cross-cutting strategy that would otherwise be duplicated
@@ -154,6 +221,11 @@ function buildInstructions(ga: boolean, delegated: boolean, managedTenants: stri
'Autopilot-Monitor is a READ-ONLY telemetry server for Windows Autopilot enrollment sessions.',
'',
'Investigating one session: call get_session_summary FIRST (status, filtered timeline, stats, rule analysis in one call), then drill in.',
...(docsIndex
? ['Product questions ("how do I…", "what does X mean", "where is my data stored"): use search_docs — the ' +
'published customer documentation. search_knowledge is a DIFFERENT corpus (analysis rules and IME log ' +
'patterns) and answers why an enrollment failed, not how the product works.']
: []),
'Searching events: use search_events (hybrid keyword+semantic ranking; depth="fast" then "deep" for exhaustive recall) for ranked hits, or get_session_events / query_raw_events for the raw unranked stream.',
'Counting / aggregating: pass a lean `fields=` projection and use `agentVersionPrefix=`/`imeAgentVersionPrefix=` sweeps to stay under the per-response size cap.',
'Pagination: when a response carries `nextLink`, pass that whole string back as `continuation`; stop when it is absent. Results are never silently truncated.',
@@ -173,7 +245,7 @@ function createMcpServer(ga: boolean, strictGa: boolean, delegated: boolean, man
{ name: 'Autopilot-Monitor', version: SERVER_VERSION },
{ instructions: buildInstructions(ga, delegated, managedTenants, homeTenantId) },
);
registerTools(s, knowledgeBase, eventTypeIndex, ga, strictGa, delegated);
registerTools(s, knowledgeBase, eventTypeIndex, DOCS, ga, strictGa, delegated);
registerResources(s);
// A delegated caller has no platform scope, so prompts get the tenant-user surface (ga=false) —
// the cross-tenant prompt wording would be misleading for a tenant-bounded MSP user.
@@ -227,8 +299,19 @@ app.use(createOAuthRouter());
// The version is the same value already advertised in the MCP handshake
// (createMcpServer → { name, version }), so surfacing it here leaks nothing
// new; it lets the backend health dashboard show the deployed MCP build.
//
// The docs block exists for one reason: the documentation bundle lives in its own
// repo and is baked in at image build time, so editing the docs does NOT update a
// running server — only a redeploy does. Reporting the bundle's commit and chunk
// count turns "did we remember to redeploy?" into a diff against
// `git rev-parse HEAD` in autopilotmonitor-docs. Both values are public
// (the bundle is a public repo published at docs.autopilotmonitor.com).
app.get('/health', (_req, res) => {
res.json({ status: 'healthy', version: SERVER_VERSION });
res.json({
status: 'healthy',
version: SERVER_VERSION,
docs: { commit: DOCS_COMMIT, chunks: docsIndex?.size ?? 0, sections: docSections(docsDocs) },
});
});
// Access guard for /mcp — validates JWT, checks backend whitelist, enforces rate limits
@@ -348,7 +431,9 @@ const bodyErrorHandler: ErrorRequestHandler = (err, req, res, next) => {
app.use(bodyErrorHandler);
const server = app.listen(PORT, '0.0.0.0', () => {
console.error(`Autopilot-Monitor MCP Server running on port ${PORT}`);
// The number a user actually waits for on a cold start, minus the platform's
// own activation (pod scheduling, container start), which is not visible here.
console.error(`[boot ${bootMark()}] Autopilot-Monitor MCP Server listening on port ${PORT} — ready to serve.`);
});
// --- Graceful shutdown ---
@@ -7,18 +7,19 @@
* 0.25 vCPU container. Running the real pipeline here also doubles as the
* model prefetch into HF_CACHE_DIR (no HuggingFace CDN dependency at runtime).
*
* Usage: node dist/precompute-embeddings.js <rulesDir> <outFile>
* Usage: node dist/precompute-embeddings.js <rulesDir> <outFile> [docsDir]
*/
import { writeFileSync } from 'node:fs';
import { loadKnowledgeDocs } from './knowledge-base.js';
import { loadDocsCorpus } from './docs-corpus.js';
import { buildEventTypeSearchDocs } from './resource-catalog.js';
import { MODEL_NAME, VectorSearchProvider } from './vector-search-provider.js';
import { hashDocs, type PrecomputedIndexFile } from './precomputed-index.js';
const [rulesDir, outFile] = process.argv.slice(2);
const [rulesDir, outFile, docsDir] = process.argv.slice(2);
if (!rulesDir || !outFile) {
console.error('usage: node dist/precompute-embeddings.js <rulesDir> <outFile>');
console.error('usage: node dist/precompute-embeddings.js <rulesDir> <outFile> [docsDir]');
process.exit(2);
}
@@ -31,6 +32,17 @@ if (knowledgeDocs.length === 0) {
}
const eventTypeDocs = buildEventTypeSearchDocs();
// Docs are optional: the production image always passes docsDir (the workflow
// checks the docs repo out into the build context), but a local build without it
// must still produce a usable index rather than fail. Warn loudly instead —
// a silently doc-less production image would leave search_docs unregistered.
const docsDocs = docsDir ? await loadDocsCorpus(docsDir) : [];
if (docsDir && docsDocs.length === 0) {
console.error(`WARNING: docs dir ${docsDir} yielded no documents — search_docs will not be registered`);
} else if (!docsDir) {
console.error('no docsDir argument — building an index without the documentation corpus');
}
const knowledgeBase = new VectorSearchProvider();
await knowledgeBase.index(knowledgeDocs);
const eventTypeIndex = new VectorSearchProvider();
@@ -41,7 +53,15 @@ const file: PrecomputedIndexFile = {
knowledgeBase: { docsHash: hashDocs(knowledgeDocs), entries: knowledgeBase.serialize() },
eventTypes: { docsHash: hashDocs(eventTypeDocs), entries: eventTypeIndex.serialize() },
};
if (docsDocs.length > 0) {
const docsIndex = new VectorSearchProvider();
await docsIndex.index(docsDocs);
file.docs = { docsHash: hashDocs(docsDocs), entries: docsIndex.serialize() };
}
writeFileSync(outFile, JSON.stringify(file));
console.error(
`precomputed embeddings: ${knowledgeDocs.length} knowledge docs + ${eventTypeDocs.length} event types (${MODEL_NAME}) → ${outFile}`,
`precomputed embeddings: ${knowledgeDocs.length} knowledge docs + ${eventTypeDocs.length} event types + ` +
`${docsDocs.length} doc chunks (${MODEL_NAME}) → ${outFile}`,
);
@@ -29,6 +29,12 @@ export interface PrecomputedIndexFile {
model: string;
knowledgeBase: PrecomputedSection;
eventTypes: PrecomputedSection;
/**
* Chunked customer documentation (see docs-corpus.ts). Optional in the file
* format: a build without the docs repo checked out ships no docs section, and
* that must not invalidate the other two.
*/
docs?: PrecomputedSection;
}
/**
@@ -51,7 +57,12 @@ export function hashDocs(docs: SearchDocument[]): string {
}
export type PrecomputedValidation =
| { ok: true; knowledgeBase: PrecomputedDocument[]; eventTypes: PrecomputedDocument[] }
| {
ok: true;
knowledgeBase: PrecomputedDocument[];
eventTypes: PrecomputedDocument[];
docs: PrecomputedDocument[];
}
| { ok: false; reason: string };
function validateSection(
@@ -67,12 +78,20 @@ function validateSection(
if (section.entries.length !== sourceDocs.length) {
return `${name}: entry count ${section.entries.length} != corpus size ${sourceDocs.length}`;
}
const dims = section.entries[0]?.embedding?.length ?? 0;
// Each entry carries one vector per token window (see splitIntoWindows) — usually
// one, more for long documents. Dimensionality must be uniform across every window
// of every entry: a ragged file would otherwise produce silently wrong dot products.
const dims = section.entries[0]?.embeddings?.[0]?.length ?? 0;
if (dims === 0) return `${name}: first entry has no embedding`;
for (const e of section.entries) {
if (typeof e.id !== 'string' || typeof e.text !== 'string') return `${name}: entry missing id/text`;
if (!Array.isArray(e.embedding) || e.embedding.length !== dims) {
return `${name}: inconsistent embedding dimensions`;
if (!Array.isArray(e.embeddings) || e.embeddings.length === 0) {
return `${name}: entry ${e.id} has no embeddings`;
}
for (const vec of e.embeddings) {
if (!Array.isArray(vec) || vec.length !== dims) {
return `${name}: inconsistent embedding dimensions`;
}
}
}
return null;
@@ -88,6 +107,7 @@ export function validatePrecomputedIndex(
expectedModel: string,
knowledgeDocs: SearchDocument[],
eventTypeDocs: SearchDocument[],
docsDocs: SearchDocument[] = [],
): PrecomputedValidation {
if (parsed === null || typeof parsed !== 'object') return { ok: false, reason: 'not an object' };
const file = parsed as Partial<PrecomputedIndexFile>;
@@ -98,5 +118,24 @@ export function validatePrecomputedIndex(
if (kbError) return { ok: false, reason: kbError };
const etError = validateSection(file.eventTypes, 'eventTypes', eventTypeDocs);
if (etError) return { ok: false, reason: etError };
return { ok: true, knowledgeBase: file.knowledgeBase!.entries, eventTypes: file.eventTypes!.entries };
// Docs are an OPTIONAL corpus: a checkout without the docs repo loads zero doc
// documents, and the matching image ships no docs section. Only validate when
// the booting server actually has a docs corpus to check against — otherwise a
// dev build would throw away the rules and event-type vectors too, which is the
// expensive half. When the server HAS docs but the file does not, that is a
// genuine staleness signal and must be rejected like any other mismatch.
let docs: PrecomputedDocument[] = [];
if (docsDocs.length > 0) {
const docsError = validateSection(file.docs, 'docs', docsDocs);
if (docsError) return { ok: false, reason: docsError };
docs = file.docs!.entries;
}
return {
ok: true,
knowledgeBase: file.knowledgeBase!.entries,
eventTypes: file.eventTypes!.entries,
docs,
};
}
@@ -24,6 +24,7 @@ export const EVENT_TYPES_CATALOG = {
'esp_failure_settle_started',
'esp_failure_settle_recovered',
'esp_exiting',
'mdm_policy_reboot_required',
'esp_config_detected',
'completion_check',
'completion_waiting',
@@ -90,3 +90,16 @@ export function scanLexical(documents: SearchDocument[], needles: string[]): Sea
// ── Provider identifiers ─────────────────────────────────────
export type SearchBackend = 'vector' | 'fuse';
/**
* What `search_docs` runs on, as one value rather than a growing tail of positional
* parameters through registerTools → registerSearchTools.
*
* The keyword fallback needs no second provider: it uses `vector.lexicalMatch`,
* the literal-containment scan every provider already exposes.
*/
export interface DocsSearchBundle {
vector: SearchProvider;
/** Top-level documentation areas present in the corpus — the `section` filter hint. */
sections: string[];
}
@@ -3,6 +3,7 @@ import type { SearchProvider } from './search-provider.js';
import { registerSessionTools } from './tools/sessions.js';
import { registerSearchTools } from './tools/search.js';
import { registerAdminTools } from './tools/admin.js';
import type { DocsSearchBundle } from './search-provider.js';
/**
* Registers the tool catalog for a single request, tailored to the caller's role.
@@ -26,12 +27,13 @@ export function registerTools(
server: McpServer,
knowledgeBase: SearchProvider | undefined,
eventTypeIndex: SearchProvider | undefined,
docs: DocsSearchBundle | undefined,
ga: boolean,
strictGa: boolean = ga,
delegated: boolean = false,
): void {
registerSessionTools(server, ga, delegated);
registerSearchTools(server, knowledgeBase, eventTypeIndex, ga, delegated);
registerSearchTools(server, knowledgeBase, eventTypeIndex, docs, ga, delegated);
registerAdminTools(server, ga, strictGa, delegated);
sortToolCatalog(server);
}
@@ -2,7 +2,7 @@ import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';
import { apiFetch, buildQuery, enforceDelegatedTenant, followNextLink, pickGlobalOrTenantPath } from '../client.js';
import { withToolTelemetry } from '../telemetry.js';
import type { SearchProvider } from '../search-provider.js';
import type { SearchProvider, DocsSearchBundle } from '../search-provider.js';
import { READ_ONLY, MAX_RESULT_SIZE_CHARS, toolResultText, SessionIdSchema, isBenignHealthDetectionReport, tenantIdDescription } from './shared.js';
import { toolError } from './error-handler.js';
import { ALL_EVENT_TYPES } from '../resource-catalog.js';
@@ -959,12 +959,49 @@ export function diversifyBySession<T extends { event: EventEntry }>(
return picked;
}
/**
* Terms from a natural-language query that are worth matching literally.
*
* Only distinctive words: short and common ones ("how", "the", "does") appear in
* nearly every chunk, so matching on them would return the whole corpus in
* arbitrary order. What is left is the vocabulary literal matching is actually good
* at — product names, setting names, identifiers — which is exactly what embeddings
* are bad at.
*/
const KEYWORD_STOPWORDS = new Set([
'about', 'after', 'again', 'also', 'been', 'before', 'being', 'between', 'both', 'can',
'could', 'does', 'doing', 'during', 'each', 'from', 'have', 'having', 'here', 'how',
'into', 'just', 'more', 'most', 'much', 'must', 'need', 'only', 'other', 'over',
'same', 'should', 'some', 'such', 'than', 'that', 'their', 'them', 'then', 'there',
'these', 'they', 'this', 'those', 'through', 'under', 'until', 'very', 'were', 'what',
'when', 'where', 'which', 'while', 'will', 'with', 'would', 'your',
]);
/**
* Below this best-hit cosine score, the semantic pass is treated as having found
* nothing convincing, and exact keyword matches are allowed to take the front.
* Chosen by measurement, not intuition — see scripts/eval-docs-search.ts.
*/
export const WEAK_SEMANTIC_SCORE = 0.35;
export function extractDistinctiveTerms(query: string): string[] {
const seen = new Set<string>();
for (const raw of query.toLowerCase().split(/[^a-z0-9_.-]+/)) {
const term = raw.replace(/^[.-]+|[.-]+$/g, '');
if (term.length < 4 || KEYWORD_STOPWORDS.has(term)) continue;
seen.add(term);
if (seen.size >= 6) break;
}
return [...seen];
}
// ── Registration ────────────────────────────────────────────────────────
export function registerSearchTools(
server: McpServer,
knowledgeBase: SearchProvider | undefined,
eventTypeIndex: SearchProvider | undefined,
docs: DocsSearchBundle | undefined,
ga: boolean,
delegated: boolean = false,
): void {
@@ -1099,6 +1136,8 @@ export function registerSearchTools(
'Use natural language queries like "app install timeout", "BitLocker issues", "detection script failure". ' +
'Returns the most relevant rules and patterns ranked by similarity. ' +
'Great for finding remediation steps, understanding error patterns, or discovering relevant diagnostic rules. ' +
'QUERY IN ENGLISH: the rules and the embedding model are English-only, so a query in another language ' +
'matches nothing at all. Translate the user\'s question first and answer them in their own language. ' +
'ERROR CODES: a query containing an HRESULT/Win32 hex code (e.g. "0x87D1041C", "0x80070002") also triggers a ' +
'literal substring fallback — any rule that names the code verbatim is returned regardless of minScore, since ' +
'such opaque codes embed poorly and the semantic score alone would miss them. Those hits are flagged `matchType: "error-code"`.',
@@ -1176,4 +1215,173 @@ export function registerSearchTools(
})
);
// Tool 11: search_docs — semantic search over the published customer documentation.
//
// Registered for EVERY role, unlike the platform tools: this is public product
// documentation (docs.autopilotmonitor.com), carries no tenant data, and a
// normal tenant user has exactly as much business asking "how do I deploy the
// agent" as a Global Admin. It is skipped entirely when no corpus was baked
// into the image, so a doc-less build advertises no broken tool.
if (docs && docs.vector.size > 0) {
const docsIndex = docs.vector;
server.registerTool(
'search_docs',
{
title: 'Search Product Documentation',
description:
'PRODUCT DOCUMENTATION SEARCH — the published Autopilot Monitor customer documentation ' +
'(docs.autopilotmonitor.com), chunked by section and ranked semantically. ' +
'Use this for questions about how the PRODUCT works rather than what a specific enrollment did: ' +
'setup and onboarding, agent deployment, portal features, roles and permissions, settings, ' +
'notifications, network endpoints, security/privacy/data-residency, plans, and troubleshooting guidance. ' +
'PREFER THIS OVER FETCHING OR WEB-SEARCHING THE DOCS SITE: it returns the matching SECTIONS ' +
'(~1k tokens for topK=3) instead of whole pages (~9k tokens raw, ~600k rendered), it is the exact ' +
'version this server ships rather than a cached or third-party copy, and it matches on meaning — ' +
'"where is my data stored" finds the page that says "Germany West Central" with no shared words, ' +
'which keyword or web search cannot do. ' +
'Each result carries the page `title`, its `heading` breadcrumb and a citable `url` — quote the url when ' +
'answering so the user can verify. ' +
'If semantic matching returns too few hits, exact keyword matches are APPENDED (flagged ' +
'`matchType: "keyword"`, and their scores are on a different scale — rank order across the two is ' +
'not comparable). ' +
'QUERY IN ENGLISH: the documentation and the embedding model are English-only, so a query in another ' +
'language matches nothing at all (it returns zero results rather than poor ones). Translate the ' +
'user\'s question first and answer them in their own language. ' +
'NOT the same corpus as search_knowledge: that one holds analysis/gather rules and IME log patterns and ' +
'explains why an enrollment failed. Rule of thumb — "how does X work?" → search_docs; ' +
'"why did this session fail?" → search_knowledge or get_session_summary.',
inputSchema: {
query: z.string().describe('Natural language question (e.g. "how do I deploy the agent with Intune", "where is my data stored", "which roles exist")'),
topK: z.coerce.number().min(1).max(20).optional().default(5).describe('Number of results to return (1-20, default 5)'),
section: z.string().optional()
.describe('Restrict to one documentation area, matching the bundle\'s top-level folder ' +
`(one of: ${docs.sections.join(', ')}). Omit to search everything.`),
minScore: z.coerce.number().min(0).max(1).optional().default(0.25)
.describe('Minimum similarity score (0-1, default 0.25). Short keyword queries score low on ' +
'all-MiniLM embeddings, so the default is tuned to keep the relevant-but-marginal band.'),
},
annotations: READ_ONLY,
},
async (args) => withToolTelemetry('search_docs', async () => {
try {
const { query, topK, section, minScore } = args;
// Over-fetch before filtering by section, exactly as search_knowledge does
// for `type` — filtering a topK-sized result set would otherwise return
// fewer than topK hits whenever the top matches sit in other sections.
const fetchK = section ? topK * 4 : topK;
let results = await docsIndex.search(query, { topK: fetchK, minScore });
// Same rationale as search_knowledge: opaque HRESULT/Win32 tokens embed
// poorly, and the troubleshooting pages name them verbatim.
const needles = extractErrorCodeNeedles(query);
const errorCodeHitIds = new Set<string>();
if (needles.length > 0 && docsIndex.lexicalMatch) {
const byId = new Map(results.map((r) => [r.id, r] as const));
for (const hit of docsIndex.lexicalMatch(needles)) {
errorCodeHitIds.add(hit.id);
const existing = byId.get(hit.id);
if (!existing || hit.score > existing.score) byId.set(hit.id, hit);
}
results = [...byId.values()].sort((a, b) => b.score - a.score);
}
if (section) {
results = results.filter((r) => r.metadata.section === section);
}
results = results.slice(0, topK);
// Keyword fallback: exact, literal term matching to top up a short result set.
//
// Proper nouns embed poorly — measured on this corpus, "RealmJoin" appears in
// 3 chunks yet scores below the semantic threshold. Fuzzy matching is the wrong
// remedy: at a threshold low enough to surface "RealmJoin" (0.28), Fuse also
// returns 5 hits for "SCEPman", a word that appears in the corpus zero times.
// Literal containment has neither problem — it finds the first and returns
// nothing for the second.
//
// APPENDED, never merged: these carry no cosine similarity, so semantic hits
// keep the head of the list and keyword hits fill the tail, flagged for the
// caller. Ranked among themselves by how many query terms they contain.
const keywordHitIds = new Set<string>();
const terms = extractDistinctiveTerms(query);
// A weak semantic head means "nothing here really matches" — measured, that
// is where exact matches belong in front. Topping up only when the list is
// SHORT misses this: a pasted identifier like "WOW6432Node" returns three
// chunks at 0.29 that do not contain the word, while the one chunk that does
// never gets considered. Over 2465 evaluation queries, acting on a weak head
// below this threshold fixed 335 and broke 0 (see scripts/eval-docs-search.ts);
// raising it to 0.40 fixed more but started demoting hand-written questions,
// so it stays here.
const semanticIsWeak = (results[0]?.score ?? 0) < WEAK_SEMANTIC_SCORE;
if ((results.length < topK || semanticIsWeak) && terms.length > 0 && docsIndex.lexicalMatch) {
// lexicalMatch is ANY-of-needles, which alone is far too generous: for
// "how do I bake sourdough bread" the single word "bake" matches "baked
// into the image" and a page about LLM providers comes back as a hit.
// A multi-term query must therefore corroborate — at least two of its
// terms in the same chunk. A single-term query ("RealmJoin") still needs
// just that one, which is the case this fallback exists for.
const required = Math.min(2, terms.length);
const seen = new Set(results.map((r) => r.id));
const ranked = docsIndex
.lexicalMatch(terms)
.map((hit) => {
const haystack = hit.text.toLowerCase();
return { hit, matched: terms.filter((t) => haystack.includes(t)).length };
})
.filter(({ matched }) => matched >= required)
.sort((a, b) => b.matched - a.matched);
const fresh = ranked
.map(({ hit }) => hit)
.filter((hit) => !seen.has(hit.id) && (!section || hit.metadata.section === section));
if (results.length < topK) {
// Free slots: append, semantic keeps the head.
for (const hit of fresh) {
if (results.length >= topK) break;
keywordHitIds.add(hit.id);
results.push(hit);
}
} else if (semanticIsWeak && fresh.length > 0) {
// No free slots, but nothing semantic is convincing. Exact matches go to
// the front — while keeping at least one semantic result, so a
// weak-but-correct hit is never pushed out entirely.
for (const hit of fresh.slice(0, topK - 1)) keywordHitIds.add(hit.id);
results = [...fresh.slice(0, topK - 1), ...results].slice(0, topK);
}
}
const formatted = results.map((r) => ({
id: r.id,
score: Math.round(r.score * 1000) / 1000,
title: r.metadata.title,
heading: r.metadata.heading,
section: r.metadata.section,
path: r.metadata.path,
url: r.metadata.url,
tags: r.metadata.tags,
content: r.text,
matchType: errorCodeHitIds.has(r.id)
? ('error-code' as const)
: keywordHitIds.has(r.id)
? ('keyword' as const)
: undefined,
}));
return toolResultText({
query,
searchBackend: docsIndex.name,
...(section ? { section } : {}),
resultCount: formatted.length,
...(needles.length > 0 ? { errorCodeFallback: { codes: needles, matchedCount: errorCodeHitIds.size } } : {}),
...(keywordHitIds.size > 0 ? { keywordFallback: { used: true, added: keywordHitIds.size } } : {}),
results: formatted,
}, MAX_RESULT_SIZE_CHARS.small);
} catch (error: unknown) {
return toolError('search_docs', args, error);
}
})
);
}
}
@@ -45,6 +45,68 @@ function getEmbedder(): Promise<FeatureExtractionPipeline> {
return embedderPromise;
}
// ── Token windowing ──────────────────────────────────────────
/**
* The model's context window is 256 tokens INCLUDING the [CLS]/[SEP] specials, so
* windows carry at most 250 content tokens.
*
* Anything past the window is silently dropped by the tokenizer — not an error, just
* a vector that represents the opening of a document and nothing else. Measured on
* the docs corpus, 87 of 324 chunks exceeded it and 13% of all tokens never reached
* the ranking. Splitting long text into several windows and scoring by the best one
* (see search()) puts every token back in play without shrinking chunks, which would
* break up the tables and breadcrumbs the chunker deliberately keeps whole.
*/
const WINDOW_TOKENS = 250;
/**
* Overlap between consecutive windows. A sentence that straddles a boundary would
* otherwise be split across two vectors and match neither well.
*/
const WINDOW_OVERLAP_TOKENS = 40;
// Tokenizer only — NOT the ONNX weights. Needed exclusively on the indexing path
// (build time); the production boot hydrates precomputed vectors and never calls
// index(), so this stays unloaded there. Measured cold load: ~93 ms.
let tokenizerPromise: Promise<{
(text: string, opts?: object): Promise<{ input_ids: { data: BigInt64Array | Int32Array } }>;
decode(ids: number[], opts?: object): string;
}> | null = null;
function getTokenizer() {
if (!tokenizerPromise) {
tokenizerPromise = (async () => {
const { AutoTokenizer, env } = await import('@huggingface/transformers');
if (process.env.HF_CACHE_DIR) env.cacheDir = process.env.HF_CACHE_DIR;
return (await AutoTokenizer.from_pretrained(MODEL_NAME)) as never;
})().catch((err) => {
tokenizerPromise = null;
throw err;
});
}
return tokenizerPromise;
}
/**
* Split `text` into overlapping token windows, returned as decoded strings.
* Short text yields exactly one window (the original string, untouched).
*/
export async function splitIntoWindows(text: string): Promise<string[]> {
const tokenizer = await getTokenizer();
const encoded = await tokenizer(text, { add_special_tokens: false });
const ids = Array.from(encoded.input_ids.data as ArrayLike<bigint | number>, Number);
if (ids.length <= WINDOW_TOKENS) return [text];
const stride = WINDOW_TOKENS - WINDOW_OVERLAP_TOKENS;
const windows: string[] = [];
for (let start = 0; start < ids.length; start += stride) {
windows.push(tokenizer.decode(ids.slice(start, start + WINDOW_TOKENS), { skip_special_tokens: true }));
if (start + WINDOW_TOKENS >= ids.length) break;
}
return windows;
}
/**
* Every embedding this module produces MUST be L2-unit-normalized: cosineSimilarity is a
* bare dot product (see below) that silently returns wrong scores for non-unit inputs. We
@@ -56,6 +118,13 @@ function getEmbedder(): Promise<FeatureExtractionPipeline> {
*/
const UNIT_NORM_EPSILON = 1e-3;
/**
* Decimal places kept when serializing embeddings to the precomputed index.
* Six is comfortably below the tolerance above while roughly halving the file
* size — see VectorSearchProvider.serialize().
*/
const SERIALIZED_PRECISION = 6;
export function assertUnitNorm(vec: number[], context: string): void {
let sumSq = 0;
for (let i = 0; i < vec.length; i++) sumSq += vec[i] * vec[i];
@@ -98,9 +167,16 @@ function cosineSimilarity(a: number[], b: number[]): number {
// ── Provider implementation ──────────────────────────────────
/** An indexed document with its embedding — the unit of build-time serialization. */
/**
* An indexed document with its embeddings — the unit of build-time serialization.
*
* One vector per token window (see splitIntoWindows). Short documents — which is
* every rule and every event type — have exactly one, so the array is not an
* optimization for the common case but the single uniform shape: no branch, no
* "primary vs extra" special case to get wrong.
*/
export interface PrecomputedDocument extends SearchDocument {
embedding: number[];
embeddings: number[][];
}
export class VectorSearchProvider implements SearchProvider {
@@ -119,12 +195,38 @@ export class VectorSearchProvider implements SearchProvider {
* from disk is milliseconds. The embedder is still loaded lazily for queries.
*/
indexPrecomputed(docs: PrecomputedDocument[]): void {
// The serialized vectors are rounded (see serialize()) and arrive from a file
// this process did not produce. Spot-check the first one: cosineSimilarity is
// a bare dot product, so a non-unit vector here would silently skew every
// score rather than fail. One check is enough — the whole section is written
// by a single serialize() call, so they stand or fall together.
if (docs.length > 0 && docs[0].embeddings.length > 0) {
assertUnitNorm(docs[0].embeddings[0], 'indexPrecomputed');
}
this.documents.push(...docs);
}
/** Export the indexed documents with embeddings for build-time serialization. */
/** Total vectors held — exceeds `size` whenever documents span several windows. */
get vectorCount(): number {
return this.documents.reduce((n, d) => n + d.embeddings.length, 0);
}
/**
* Export the indexed documents with embeddings for build-time serialization.
*
* Embeddings are rounded to SERIALIZED_PRECISION decimals. A float64 component
* serializes to ~20 characters of JSON ("0.05234159901738167"), so 384 dims
* cost ~7.8 KB per document; rounding roughly halves that, and the index file
* is JSON.parse'd on every scale-to-zero cold start. The precision loss is
* irrelevant to ranking: the induced L2 norm error is ~1e-5 across 384 dims,
* two orders of magnitude inside UNIT_NORM_EPSILON, and far below the score
* gaps that separate ranked results.
*/
serialize(): PrecomputedDocument[] {
return this.documents;
return this.documents.map((doc) => ({
...doc,
embeddings: doc.embeddings.map((vec) => vec.map((v) => Number(v.toFixed(SERIALIZED_PRECISION)))),
}));
}
async index(docs: SearchDocument[]): Promise<void> {
@@ -134,14 +236,21 @@ export class VectorSearchProvider implements SearchProvider {
const batch = docs.slice(i, i + batchSize);
const embeddings = await Promise.all(
batch.map(async (d) => {
const out = await model(d.text, { pooling: 'mean', normalize: true });
const vec = Array.from(out.data as Float32Array);
assertUnitNorm(vec, `index:${d.id}`);
return vec;
// One vector per token window: text longer than the model's context would
// otherwise be truncated to its opening and the rest never ranked at all.
const windows = await splitIntoWindows(d.text);
return Promise.all(
windows.map(async (window, w) => {
const out = await model(window, { pooling: 'mean', normalize: true });
const vec = Array.from(out.data as Float32Array);
assertUnitNorm(vec, `index:${d.id}#w${w}`);
return vec;
})
);
})
);
for (let j = 0; j < batch.length; j++) {
this.documents.push({ ...batch[j], embedding: embeddings[j] });
this.documents.push({ ...batch[j], embeddings: embeddings[j] });
}
}
}
@@ -150,12 +259,18 @@ export class VectorSearchProvider implements SearchProvider {
const { topK = 5, minScore = 0.3 } = options;
const queryEmbedding = await embed(query);
const scored = this.documents.map((doc) => ({
id: doc.id,
text: doc.text,
metadata: doc.metadata,
score: cosineSimilarity(queryEmbedding, doc.embedding),
}));
// Score a document by its BEST window, not by an average. Averaging would blur a
// long document's several topics into one mediocre vector — the exact dilution
// that makes big context windows worse than chunking. Max keeps a document
// findable by any one of the things it actually talks about.
const scored = this.documents.map((doc) => {
let best = -Infinity;
for (const vec of doc.embeddings) {
const score = cosineSimilarity(queryEmbedding, vec);
if (score > best) best = score;
}
return { id: doc.id, text: doc.text, metadata: doc.metadata, score: best };
});
return scored
.filter((r) => r.score >= minScore)
@@ -73,19 +73,32 @@ namespace AutopilotMonitor.Shared
public const string ApiBaseUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
/// <summary>
/// Public base URL of the MCP server Container App (no trailing slash, no /mcp suffix).
/// Used by the backend health check to probe <c>{McpServerBaseUrl}/health</c>. The
/// Container App runs with minReplicas=0, so the first probe after idle may incur a
/// cold start. Overridable via the <c>McpServerUrl</c> app setting (e.g. for dev).
/// Public base URL of the MCP server (no trailing slash, no /mcp suffix). Stable custom
/// domain in front of the Container App — survives region/environment moves, unlike the
/// generated *.azurecontainerapps.io FQDN. Used by the backend health check to probe
/// <c>{McpServerBaseUrl}/health</c>. The Container App runs with minReplicas=0, so the
/// first probe after idle may incur a cold start. Overridable via the
/// <c>McpServerUrl</c> app setting (e.g. for dev).
/// </summary>
public const string McpServerBaseUrl = "https://autopilotmonitor-mcp.kindwave-58b4b547.westeurope.azurecontainerapps.io";
public const string McpServerBaseUrl = "https://mcp.autopilotmonitor.com";
// -----------------------------------------------------------------------
// Agent self-update
// -----------------------------------------------------------------------
/// <summary>
/// Base URL for agent binaries in Azure Blob Storage (public read)
/// Canonical public base URL for agent binaries: Front Door alias in front of the
/// current blob origin (autopilotmonitoreu, container "agent"). Storage-account moves
/// happen behind this name without touching customers. New bootstrap scripts and all
/// availability probes use this URL.
/// </summary>
public const string AgentDownloadBaseUrl = "https://download.autopilotmonitor.com/agent";
/// <summary>
/// LEGACY base URL for agent binaries: the pre-migration storage account, kept alive
/// (dual-push on every release) because bootstrap scripts already deployed in customer
/// Intune tenants still download from it. Remove only after the customer migration to
/// <see cref="AgentDownloadBaseUrl"/> is complete and the legacy account is torn down.
/// </summary>
public const string AgentBlobBaseUrl = "https://autopilotmonitor.blob.core.windows.net/agent";
@@ -392,6 +405,18 @@ namespace AutopilotMonitor.Shared
public const string OsBuildChanged = "os_build_changed"; // OS build (CurrentBuild.UBR) differs across an agent restart — an update was installed during enrollment
public const string WindowsUpdateChannelCensus = "windows_update_channel_census"; // One-shot EventID histogram of WU Client + UpdateOrchestrator channels when os_build_changed but zero targeted WU events captured
// MDM reboot-required policy flags (MdmRebootPolicyTracker; PMPC "unexpected reboot /
// second sign-in" research). DM-Enterprise EventID 2800 fires once per policy URI that
// matches the OS reboot-required catalog ("The following URI has triggered a reboot:
// (./Device/...)."; verified session b2e890c1). Only URIs applied DURING ESP DeviceSetup
// cause the coalesced mid-ESP reboot — later syncs (AccountSetup+) merely flag, no forced
// reboot. The tracker therefore emits ONE aggregated Info event per burst with the URI
// list; the reboot CLAIM lives in ANALYZE-ESP-005, gated on an actually-observed
// system_reboot_detected. Deliberately DISTINCT from system_reboot_detected: the backend
// RebootCount matches that string exactly (EventIngestProcessor.Classification + terminal
// recount) — this attribution event must never inflate the reboot count.
public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // Aggregated per burst — Data.rebootUris[] + uriCount + firstRebootUri
// TEMPORARY: shadow SM rollout verification — remove when CompletionStateMachine is promoted to primary
public const string ShadowDiscrepancy = "shadow_discrepancy";
@@ -242,7 +242,7 @@ namespace AutopilotMonitor.Shared.Models
/// config cannot chain another migration), and a kill signal always wins over it.
/// Old binaries ignore the field. null = no migration.
/// </summary>
public string MigrateToApiBaseUrl { get; set; }
public string? MigrateToApiBaseUrl { get; set; }
}
/// <summary>
@@ -482,6 +482,28 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public bool WindowsUpdateChannelCensusEnabled { get; set; } = true;
// -----------------------------------------------------------------------
// MDM reboot-policy watcher (DeviceManagement-Enterprise-Diagnostics-Provider/Admin)
// -----------------------------------------------------------------------
/// <summary>
/// Master switch for the MDM reboot-policy watcher. When enabled the agent subscribes to
/// <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> EventID
/// 2800 and emits one <c>mdm_policy_reboot_required</c> event per device-assigned policy
/// URI that forces a coalesced reboot during ESP DeviceSetup (the "unexpected reboot +
/// second sign-in" pattern). Internal steering lever only — not surfaced in any admin UI;
/// disable by pushing config server-side. Default: true.
/// </summary>
public bool MdmRebootPolicyWatcherEnabled { get; set; } = true;
/// <summary>
/// Lookback window in minutes for the MDM reboot-policy backfill scan on startup. Device
/// policies can apply before the agent starts, and after the coalesced reboot the restarted
/// agent must re-scan past the reboot gap (the cross-restart watermark keeps re-reads from
/// re-emitting). 0 = backfill disabled. Default: 60 minutes.
/// </summary>
public int MdmRebootPolicyBackfillLookbackMinutes { get; set; } = 60;
/// <summary>
/// Creates default collector configuration
/// </summary>
@@ -44,6 +44,23 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public string? OnboardedBy { get; set; }
/// <summary>
/// Address used to reach this tenant about the service itself — a technical problem,
/// a security matter, or a change that needs an administrator's attention. Editable by
/// the tenant's own admins under Settings → Tenant → Contact.
/// <para>
/// Seeded once at onboarding from the Private Preview notification address if one was
/// given, and never re-synced afterwards: from that point the value belongs to the
/// tenant, and a later edit must not be overwritten by the onboarding source.
/// </para>
/// <para>
/// Purpose-limited by design — service communication only. It is never used for
/// marketing and never disclosed. Null means we have no way to reach this tenant,
/// which is why enforcement actions cannot promise prior warning.
/// </para>
/// </summary>
public string? ContactEmail { get; set; }
// ===== TENANT STATUS =====
/// <summary>
@@ -107,7 +107,11 @@ namespace AutopilotMonitor.Shared.Models
// ===== WHEN TO COLLECT =====
/// <summary>
/// Trigger type: "startup", "phase_change", "interval", "on_event"
/// Trigger type: "startup", "phase_change", "phase_exit", "interval", "on_event".
/// <para>
/// "phase_change" fires once when the phase in <see cref="TriggerPhase"/> is ENTERED,
/// "phase_exit" once when it is LEFT — the two one-shot bookends of a phase.
/// </para>
/// </summary>
public string Trigger { get; set; } = default!;
@@ -117,8 +121,9 @@ namespace AutopilotMonitor.Shared.Models
public int? IntervalSeconds { get; set; }
/// <summary>
/// Phase to trigger on (only used when Trigger = "phase_change")
/// e.g., "Identity", "MdmEnrollment", "AppInstallation"
/// Phase to trigger on (used when Trigger = "phase_change" or "phase_exit").
/// Canonical tokens are the <see cref="EnrollmentPhase"/> enum names, e.g.
/// "DeviceSetup", "AccountSetup", "Complete". Empty = every phase transition.
/// </summary>
public string TriggerPhase { get; set; } = default!;
@@ -128,6 +133,38 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public string TriggerEventType { get; set; } = default!;
/// <summary>
/// Restricts the rule to run only while the current enrollment phase is one of these
/// phases. Canonical tokens are the <see cref="EnrollmentPhase"/> enum names
/// ("Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup",
/// "AppsUser", "FinalizingSetup", "Complete"); "Unknown" and "Failed" are rejected
/// by backend validation. Null or empty = unrestricted (runs in every phase —
/// legacy behavior). Mutually exclusive with <see cref="ActiveFromPhase"/>;
/// if both are set the agent defensively prefers this list.
/// Applies to ALL trigger types. Before the first phase signal of a session,
/// scoped rules are inactive.
/// </summary>
public List<string>? ActivePhases { get; set; }
/// <summary>
/// Activates the rule once the enrollment phase first reaches this phase
/// (ordinal comparison on <see cref="EnrollmentPhase"/>, ignoring Unknown/Failed),
/// then keeps it active for the rest of the session (sticky latch — including
/// through Failed). Canonical tokens as in <see cref="ActivePhases"/>.
/// Null = unrestricted. Mutually exclusive with <see cref="ActivePhases"/>.
/// </summary>
public string? ActiveFromPhase { get; set; }
/// <summary>
/// Emit behavior for collected results:
/// null / "always" = emit on every collection (legacy behavior);
/// "on_change" = poll on the trigger cadence but emit only when the collected
/// result differs from the last emitted one. The first in-scope result always
/// emits; the suppressed poll count is carried on the next emitted event
/// (suppressedPolls / suppressedSinceUtc in the event data).
/// </summary>
public string? EmitMode { get; set; }
// ===== OUTPUT =====
/// <summary>
@@ -26,9 +26,9 @@ const FEATURES = [
{
title: "Diagnostics Collection",
description:
"Trigger on-demand diagnostic uploads directly from the portal. Collect ETL logs, event logs, IME logs, and system information from any enrolled device without local access.",
"Collect agent logs, IME logs, agent state, and device information as a ZIP bundle at the end of an enrollment — then download it from the session view without touching the device.",
color: "green",
bullets: ["On-demand ZIP bundle upload", "ETL, event & IME log collection", "Configurable log path rules"],
bullets: ["Configurable upload: off, always, or on failure", "Agent, IME & device information bundle", "Configurable additional log paths"],
},
{
title: "Detailed Event Timeline",
@@ -40,7 +40,7 @@ const FEATURES = [
{
title: "Audit Logging & Compliance",
description:
"Complete audit trail of all administrative actions and configuration changes. Meet compliance requirements with detailed, tamper-evident records and configurable data retention.",
"Complete audit trail of all administrative actions and configuration changes. Meet compliance requirements with detailed, tenant-scoped records and configurable data retention.",
color: "red",
bullets: ["Admin action history", "Configurable retention policies", "Tenant-scoped audit log"],
},
@@ -138,7 +138,7 @@ export default function AboutPage() {
{
title: "MSPs & Enterprise Teams",
description:
"Autopilot Monitor supports multi-tenant deployments — each customer tenant runs its own isolated instance. MSPs with login access to a customer environment can use the portal for that tenant independently.",
"Autopilot Monitor is a multi-tenant service with strict per-tenant data isolation — telemetry, configuration, and diagnostics are partitioned and access-scoped to each tenant. MSPs can be granted delegated read access across several customer tenants from a single login.",
},
].map((item) => (
<div key={item.title} className="rounded-lg bg-blue-50 border border-blue-100 p-4">
@@ -165,7 +165,7 @@ export default function AboutPage() {
{ step: "2", text: "The bootstrapper installs the Autopilot Monitor Agent on each enrolling device." },
{ step: "3", text: "The agent captures live enrollment events and uploads them to the backend pipeline." },
{ step: "4", text: "The portal displays real-time session data, analyze rule results, and fleet health metrics." },
{ step: "5", text: "On completion, the agent optionally self-destructs and uploads a diagnostics bundle." },
{ step: "5", text: "On completion, the agent uploads a diagnostics bundle if configured, then removes itself." },
].map((item) => (
<li key={item.step} className="flex items-start gap-3 text-sm text-gray-700">
<span className="inline-flex h-6 w-6 shrink-0 items-center justify-center rounded-full bg-blue-600 text-white text-xs font-bold">
@@ -189,7 +189,7 @@ export default function AboutPage() {
{
title: "Backend",
items: [
"Azure Functions (.NET 8 Isolated) — serverless, scalable API",
"Azure Functions (.NET 10 Isolated, Flex Consumption) — serverless, scalable API",
"Azure Table Storage — high-throughput event ingestion",
"Azure Blob Storage — diagnostics and log bundle storage",
"Azure SignalR Service — real-time push to the portal",
@@ -198,10 +198,10 @@ export default function AboutPage() {
{
title: "Portal (Web Frontend)",
items: [
"Next.js 18 + TypeScript — fast, server-rendered React app",
"Next.js 15 (React 18) + TypeScript — fast, server-rendered React app",
"Microsoft Entra ID (MSAL) — secure authentication",
"Role-based access control (Admin / Operator)",
"Multi-tenant architecture",
"Role-based access control (Admin / Operator / Viewer)",
"Multi-tenant architecture with delegated MSP access",
],
},
{
@@ -209,16 +209,16 @@ export default function AboutPage() {
items: [
".NET binary — lightweight, low-overhead monitoring",
"Runs via scheduled task (no Windows service — easy, residue-free removal)",
"Deployed via Intune bootstrapper script (Win32 app)",
"Client certificate authentication",
"Optional self-destruct on enrollment completion",
"Deployed via an Intune platform script (PowerShell bootstrapper)",
"Mutual TLS using the existing Intune MDM device certificate",
"Self-destruct on enrollment completion (on by default — removes task and files)",
],
},
{
title: "Integrations",
items: [
"Microsoft Intune — agent deployment target",
"Microsoft Teams — enrollment failure notifications",
"Microsoft Teams, Slack & generic JSON webhooks — start, success, failure and SLA alerts",
"Intune Management Extension (IME) log — event source for log pattern detection",
"WMI & Registry — extended data gather rules",
],
@@ -249,11 +249,17 @@ export default function AboutPage() {
extended by the wider Windows Autopilot community.
</p>
<p className="text-gray-700 leading-relaxed mb-4">
Autopilot Monitor was created by{" "}
Autopilot Monitor was created and is maintained by{" "}
<strong>Oliver Kieselbach</strong>, a Microsoft MVP and long-time contributor to the Windows
Autopilot and Microsoft Intune community. The project is driven by real-world enterprise
deployment experience and community feedback.
</p>
<p className="text-gray-700 leading-relaxed mb-4">
The hosted service is operated by <strong>glueckkanja AG</strong> — see the{" "}
<a href="https://www.glueckkanja.com/en/imprint" target="_blank" rel="noopener noreferrer" className="text-blue-600 hover:text-blue-800 underline">Imprint</a> for company
details and the <Link href="/terms" className="text-blue-600 hover:text-blue-800 underline">Terms of Use</Link>{" "}
for what each plan does and does not commit to.
</p>
<div className="flex flex-wrap gap-3 mt-5">
<a
href="https://github.com/okieselbach/Autopilot-Monitor"

Some files were not shown because too many files have changed in this diff Show More