Compare commits

...

42 Commits

Author SHA1 Message Date
Oliver Kieselbach 3f9576d4a9 fix(agent): enforce the gather-rule allowlist in every collector
LogParserCollector and EventLogCollector called no guard at all, while
File/Json/Xml did — and rules/gather-rules.md already promised customers
that log-parser paths were allowlisted. A tenant admin could author a
logparser rule against C:\Users via POST /api/rules/gather and have up to
1000 lines uploaded as telemetry, bypassing the hard C:\Users block the
same page guarantees; eventlog could read any channel, Security included.
The backend validates no collectorType, target, or parameters, so the
agent is the only place this can be enforced.

Both now guard. logparser cannot check rule.Target directly (GetFullPath
throws on a wildcard), so it checks the directory for wildcard targets and
re-checks each resolved path. Event log channels get an allowlist matched
on a "/" boundary. Hard-blocked channels live in a C# constant rather than
guardrails.json: a parse error drops the allowlists to empty, which is
fail-closed, but would have lifted a block sourced from the same file.

Also fixes an inert timeout in CommandCollector: ReadToEnd() blocks until
the pipe closes, so WaitForExit(30000) was only reached once the process
had already exited and a hung command pinned its worker indefinitely.

Adds the first real CI gate. No workflow ran tests on a pull request — the
deploy workflows build without testing and build-agent.yml covered only
DecisionCore.Tests on manual dispatch — while trust/security-faq.md
already told customers automated tests run in CI. ci.yml now runs all six
suites on every PR and merge, plus a job that fails on generated-guardrail
drift, alongside CodeQL and Dependabot. OfficeBinaryWatcherTests is marked
SerialThreading; its bounded re-probe races the shared ThreadPool and
failed once under full-suite load.

Verified: 3101 backend, 1911 agent V2, 474 decision core, 44 dialog, 572
web, 447 MCP. The timeout fix is verified by runtime, not assertion — a
Start-Sleep 120 command now returns after ~30s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 00:28:04 +02:00
Oliver Kieselbach ee58242d33 fix(mcp): tell callers to query the English-only corpora in English
A German question returns zero results from search_docs, not poor ones: both the
documentation and the rules are English, and all-MiniLM-L6-v2 is an English-only
model, so the query vector lands nowhere near any chunk. For a German-speaking
admin driving an MCP client in their own language, the tool is effectively blind
and gives no hint why.

The caller is itself a language model, so the cheapest correct fix is to say so:
both tool descriptions now instruct it to translate the question to English before
searching and to answer the user in their own language. No infrastructure, no
model change, no ranking impact.

A multilingual embedding model would handle it natively and was considered, but it
is substantially larger — which matters on a scale-to-zero container — and it would
change English retrieval as well, so it would have to be measured against
scripts/eval-docs-search.ts rather than swapped in on the assumption that more
languages cannot hurt.

Verified: 447 tests pass, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:42:56 +02:00
Oliver Kieselbach eca528b9b3 docs(mcp): attribute the cold start — the app is 0.5s of it, the platform is the rest
The boot marks answered the question on the first production cold start, and the
answer overturns the reasoning behind an infrastructure change made earlier today.

  +244ms  node started (startup + module loading)
  +431ms  corpora read from disk          (+187ms)
  +491ms  index read, validated, hydrated  (+60ms for 3.12 MB)
  +504ms  listening
  +677ms  query embedder warm             (background)

Against a 31.5s cold start, the application accounts for half a second. The other
~98% is Container Apps activation: KEDA reacting, pod scheduling, mounting the
236 MB image, starting the container. The container also boots faster than a
development laptop (504ms vs 1273ms), so nothing here is CPU-starved.

Consequences recorded rather than quietly dropped:

- Raising the container to 0.5 vCPU was justified by the embedding model load on
  the cold-start path. Prebaked, that load is 173ms and runs in the background
  where only the first search could wait on it. The justification does not survive
  measurement. The setting stays — it plausibly helps concurrent requests, and at
  scale-to-zero the cost difference is negligible — but the Bicep comment now says
  what is true instead of what was assumed, so nobody re-derives the wrong reason.
- Index size is irrelevant to startup (60ms for 3.12 MB), so the corpus can grow
  freely, and the embedding rounding earns its keep on file size alone.
- The only lever that would actually remove the cold start is minReplicas: 1, at
  roughly 7x the monthly free consumption grant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:34:24 +02:00
Oliver Kieselbach 8305f5c850 feat(mcp): time the boot so the cold start can be attributed
A cold start after scale-to-zero measures 24.5s to first byte against 60ms warm,
and nothing outside the process could say why. Log Analytics stamps every console
line of one ingestion batch with the same time, so the boot sequence collapses to
a single instant however it is queried, and piping stdout locally buffers until
exit. So the server times itself: `[boot +Nms]` marks derived from
process.uptime(), which unlike an in-process timer also covers Node's own startup
and module loading.

Measuring the precomputed path locally already contradicts two assumptions worth
recording:

  +860ms   node started (startup + module loading)
  +1199ms  corpora read from disk         (+339ms)
  +1242ms  index read, validated, hydrated (+43ms for 3.12 MB)
  +1273ms  listening                       (+31ms)
  +1497ms  query embedder warm             (+224ms, background)

The index is 43ms, not seconds — its size is not the cold-start cost, and growing
the corpus is cheap. The prebaked embedder is 224ms and loads in the background,
so only the first search could ever wait on it. That was the stated reason for
raising the container to 0.5 vCPU, which now looks like a weaker justification
than it did; the production marks will settle it. If they show the app listening
after a few seconds, the rest of the 24.5s is platform activation that neither CPU
nor index changes can touch.

Verified: 447 tests pass, marks appear on both boot paths locally. Incidentally
the run also confirmed the corpus hash guard and the staleness signal both work —
the local docs repo has moved to 6333d56 while the deployed image reports d8cb7f4,
so the precomputed index was correctly rejected and recomputed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 22:14:49 +02:00
Oliver Kieselbach c2c02b929c fix(web): show gather-rule scope/emit controls only where they change behavior
The form offered trigger, phase scope and emit mode unconditionally, which
produced combinations that cannot do anything plus one silently wrong state:

- A scope on a Phase Start / Phase End rule that names a concrete phase can only
  ever suppress the single firing — the phase IS the moment. Same for emit mode:
  a one-shot has no second result to dedup against.
- Selecting "From a phase onwards" (or "Only during specific phases") and not
  filling the detail dropdown serialised to null, i.e. the rule ran everywhere
  while the form still read "From a phase onwards".

supportsPhaseScope / supportsEmitMode now drive visibility; buildScopeFields
nulls hidden-control state so a value from an earlier trigger choice cannot leak
into the payload; validateScopeSelection rejects an unfilled scope mode on both
create and save. Startup is the deliberate exception — a scope there defers the
one-shot instead of suppressing it, so the scope control stays and only emit mode
is hidden. One-shot triggers now state in-form when they collect.

Sub-dropdown hints no longer claim "No selection = all phases" (they turn amber
and name the required action instead).

Tests: app/gather-rules/__tests__/gatherRuleFormLogic.test.ts (15) pins the
applicability matrix, the payload nulling incl. the trigger-switch leak, the
unfilled-scope rejection, and the JSON-mode scopeMode derivation.
vitest 572/572; tsc silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 22:06:50 +02:00
Oliver Kieselbach 1821af6f05 fix(mcp): fire the search_docs keyword fallback on a weak semantic head
The live test found a third recall gap. The fallback only topped up when semantic
search returned FEWER than topK results — but the more common failure is a full
list of unconvincing ones. Querying "WOW6432Node" returned three chunks scoring
~0.29, none containing the word, while the single chunk that does was never
considered at all.

The fix is a ranking change, so it was measured rather than reasoned about. Adds
scripts/eval-docs-search.ts, which scores strategies over 2465 queries in three
deliberately different classes: every chunk heading (easy by construction, so a
regression detector), rare identifiers (the class this fallback serves), and
hand-written questions (the reality anchor — not derived from the corpus, so it
shares little vocabulary with it). Embeddings are computed once and each strategy
is a pure function over the same candidates, so sweeping a threshold costs no
extra inference.

                        known-item  literal  handwritten MRR  fixed  broken
  append-only (before)       71.9%    66.2%            1.000      -       -
  weak < 0.30                71.9%    72.6%            1.000    142       0
  weak < 0.35                73.7%    81.1%            1.000    335       0
  weak < 0.40                76.3%    88.2%            0.944    502       3
  weak < 0.45                77.2%    93.0%            0.889    613       5

0.35 is strictly dominant — better in every class, worse in none — so that is
what ships. Higher thresholds keep repairing identifier queries but start
demoting hand-written questions, the class that most resembles real use, so the
gain stops being free. When the head is weak, exact matches move to the front
while at least one semantic result is kept, so a weak-but-correct hit is never
pushed out entirely.

The measurement also resized the problem: identifier queries were succeeding only
66.2% of the time, so the reported case was one of roughly 750, not an outlier.
That is the part a single anecdote could not have shown, and the reason the
harness exists before the fix rather than after it.

Verified: 447 tests pass, including new cases for the weak path and for leaving a
confident ranking untouched. On the real corpus "WOW6432Node" now returns the
chunk that contains it at rank 1, while "how do I deploy the agent with Intune"
(0.819) and "what firewall URLs do I need to allow" (0.374) are unchanged and the
negative controls stay empty.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:39:26 +02:00
Oliver Kieselbach 62599d972c fix(mcp): put every doc token back into search_docs ranking
Two measured recall gaps in search_docs, both silent.

1. Truncation. MiniLM's context is 256 tokens; 87 of 324 chunks exceeded it and
13% of all tokens never reached the ranking — the tokenizer drops the tail
without complaint. Concretely, in built-in-rules.md :: Apps (734 tokens) the
strings ANALYZE-APP-013, ANALYZE-OFFICE-001, ANALYZE-CORR-003 and Click-to-Run
all sit past the cutoff, so "proxy configuration blocking app downloads" did not
retrieve that section at all.

Shrinking chunks cannot fix this: the corpus ranges 2.43 to 5.81 characters per
token, so no character budget bounds tokens, and a budget safe for the worst case
(~620 chars) would shred prose and tables alike. So the chunk stays whole and the
embedding is split instead — index() cuts long text into 250-token windows with
40 overlap, a document carries embeddings[][], and search() scores it by its BEST
window. Max rather than mean: averaging would blur a long section's several
topics into one mediocre direction, the same dilution that makes large context
windows worse than chunking.

The tokenizer is a build-time dependency only (vocabulary, not ONNX weights,
~93 ms); production boots from precomputed vectors and never loads it.

2. No general fallback. Proper nouns embed poorly — "RealmJoin" occurs in 3
chunks yet scores under the threshold. Fuzzy matching was built, measured and
rejected: at a threshold low enough to surface RealmJoin (0.28), Fuse also
returns 5 hits for "SCEPman", a word this corpus contains zero times. Literal
containment has neither problem, and every provider already exposes it. So a
short result set is now topped up via lexicalMatch on distinctive query terms,
requiring two matching terms as corroboration — without that rule "how do I bake
sourdough bread" matched a page about LLM providers, because "bake" is a
substring of "baked into the image". Keyword hits are appended and flagged, never
sorted together with cosine scores.

Also sharpens the tool description with the measured reason to prefer this over
fetching the docs site (~1k tokens for topK=3 against ~9k raw / ~600k rendered,
version-matched, and matching on meaning), and replaces the growing tail of
positional docs parameters with a DocsSearchBundle.

Verified: token coverage 87% -> 100%; "Office Click-to-Run install never
finished" 0.318 -> 0.442; "proxy configuration blocking app downloads" absent ->
0.559 at rank 1; the ten reference queries keep byte-identical scores and rank 1
(short chunks are one window, so the same vector as before — no long-chunk bias).
The fallback is not a dead path: across 904 rare corpus tokens queried alone it
contributes hits for 431, while negative controls stay empty. Precompute output
is accepted by the boot validator and a mutated corpus still rejected; index
2.63 -> 3.12 MB for 324 chunks / 424 vectors. The rules corpus gains the same
treatment unplanned: 128 rules now carry 172 vectors. 445 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 21:17:42 +02:00
Oliver Kieselbach 21b751be64 changed sidebar tenant settings entries ordering 2026-07-21 20:02:28 +02:00
Oliver Kieselbach 63ea2e3cee feat(mcp): search_docs — customer documentation as a third search corpus
The MCP server could answer telemetry questions ("why did this session fail")
but not product ones ("how do I deploy the agent", "where is my data stored").
Those answers already exist in the autopilotmonitor-docs bundle; search_docs
makes them semantically searchable.

Deliberately a separate index from search_knowledge: rules and prose answer
different questions, and sharing one topK would dilute both. Registered for
every role — published documentation carries no tenant data — and skipped
entirely when no corpus was baked in, so a doc-less build advertises no
broken tool.

Corpus handling:
- Chunked at h2/h3 and <details>/<summary>, each chunk prefixed with a
  "Page > Section > Subsection" breadcrumb so a mid-page hit is self-describing.
- Newlines are normalized before any parsing. This is a correctness
  precondition, not cosmetics: the bundle ships CRLF and JS `.` does not match
  \r, so /^(#{2,3})\s+(.*)$/ matched zero headings — silently, degrading every
  page to unlabelled paragraph splits.
- Two size exceptions: tables are never split (a split table leaves `| --- |`
  rows with no header), and unbroken changelog lists split at line level (left
  whole, their tails fall outside the model window and become unsearchable).
- Short sections are kept — a one-line answer is often the best hit, and the
  breadcrumb supplies the context a short body lacks.

Build and delivery:
- Docs are embedded at image build time into an optional third
  PrecomputedSection. Optional so a checkout without the docs repo does not
  invalidate the rules and event-type vectors; the reverse (server has docs,
  file does not) is staleness and is rejected.
- The deploy workflow checks the public docs repo into the build context and
  passes its SHA as DOCS_COMMIT, which /health reports alongside the chunk
  count — editing the docs does not update a running server, so staleness is
  made checkable rather than memorable.
- Embeddings are rounded to 6 decimals on serialize: 55% smaller index (2.63
  vs 5.88 MB) for a ~1e-5 norm error against a 1e-3 tolerance. indexPrecomputed
  now asserts unit norm, since the hydrate path never checked it.

Infra: the Bicep template was two changes behind the live deployment (EU
cutover and ACR hardening, both applied directly in Azure). Reconciled against
the running resources — managed-identity image pull replaces the admin
credentials and the acr-password secret — and infra/ is no longer gitignored so
it stops drifting. Container raised to 0.5 vCPU / 1.0 GiB for the cold-start
query-embedder load. The template does not model the mcp.autopilotmonitor.com
custom domain, so a full deployment would remove it; the header warns about it.

Verified: 428 tests pass; 45 pages produce 324 chunks; sample queries hit the
right page top-1/2; all 8 derived URLs return 200 live; the precompute output
is accepted by the boot validator and a mutated corpus is rejected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:48:28 +02:00
Oliver Kieselbach 93dcac44a4 fix(web): add Contact to the tenant settings sidebar
The section was registered for routing but not in the sidebar, so it was
reachable only by typing the URL. Sidebar entries come from
globalNavConfig.tsx, which is not derived from tenantNavSections.ts -
both have to be touched for a section to exist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:19:44 +02:00
Oliver Kieselbach 72be8ac89f feat(config): tenant contact address, seeded once from the preview email
Enforcement and service notices need somewhere to go. Until now the only
address on file was the Private Preview notification email, which is
onboarding data in the PreviewWhitelist table and not something a tenant
can maintain.

Adds TenantConfiguration.ContactEmail, owned and edited by the tenant's
own admins under Settings -> Tenant -> Contact:

- Seeded once when a preview notification address is saved, and only
  while ContactEmail is still empty. It is never re-synced, so an edit in
  the portal cannot be overwritten later by the onboarding source.
- Backfilled for tenants onboarded before the field existed, as a
  manual-trigger maintenance task alongside the other backfills. It skips
  any tenant that already has an address, so it converges to a no-op and
  can be dropped once everyone is covered.
- Surfaced read-only in Admin -> Tenants next to the preview address, so
  an operator who needs to warn a tenant does not have to read the table.

Purpose is stated where it is entered and in the Privacy Policy: service
matters only - a technical problem, a security matter, or a change
needing an administrator's attention. Never marketing, never shared.

The Terms already say a warning is sent "where a tenant contact address
is on file"; this is the field that makes that promise meaningful.

Wired through the full tenant-config round-trip (types, context state,
hydrate, save payload, reset, provider) so a save from any other section
cannot clobber it.

Verified: solution builds clean, 3101 backend tests and 557 web tests
pass, tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 18:00:18 +02:00
Oliver Kieselbach cd570c8b3e feat(web): add IP/licence terms, enforcement ladder, security reporting
- Intellectual Property & Licensing section: the repository licence
  governs the code, not the hosted service, its infrastructure, its data,
  or the name and branding. Customer data stays with the customer.
- Acceptable Use now states the enforcement ladder (warning, reduced rate
  limits, suspension, termination) instead of an unqualified right to
  suspend, and routes security research to GitHub Security Advisories
  with explicit testing boundaries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:38:52 +02:00
Oliver Kieselbach 6dfc564be6 fix(web): align privacy and terms with the DPA-owns-the-register split
Privacy: "Sub-processors and Outbound Data Flows" becomes "External
Services". It still names who receives data, which is a controller's own
duty, but drops the Art. 28 vocabulary and the promise to announce new
sub-processors - that commitment lives in the data processing agreement.
Points at the renamed docs page and notes the DPA is available on
request.

Terms: the MSP clause now says a managing organization is responsible for
having the necessary data protection agreements with its customers,
instead of classifying it as a processor or sub-processor.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 17:25:36 +02:00
Oliver Kieselbach afecd18123 fix(web): patch js-yaml and brace-expansion DoS advisories
Dependabot alerts #63/#64/#65 (all high): brace-expansion exponential-time
expansion of consecutive non-expanding {} groups (GHSA-3jxr-9vmj-r5cp,
both the 1.x and 2.x trees) and js-yaml quadratic CPU consumption via
YAML merge-key chains (GHSA-52cp-r559-cp3m).

All three are transitive dev-only dependencies of the eslint toolchain
(eslint -> @eslint/eslintrc -> js-yaml; eslint/typescript-estree ->
minimatch -> brace-expansion) and resolve within their existing semver
ranges, so only the lockfile changes.

npx tsc --noEmit clean, vitest 557 passed, npm audit reports 0
vulnerabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 15:53:28 +02:00
Oliver Kieselbach 5a294ac4f6 fix(mcp): patch transitive protobufjs/tar/body-parser advisories
Dependabot alert #62 (protobufjs 7.6.4, DoS via infinite loop in .proto
option parsing, GHSA fixed in 7.6.5) plus the tar (critical) and
body-parser (low) advisories npm audit surfaced alongside it. All three
are transitive under @huggingface/transformers and resolve within their
existing semver ranges, so only the lockfile changes.

Lockfile also picks up the pending 1.5.0 -> 1.5.1 package version sync.

npx tsc --noEmit clean, vitest 400 passed / 41 skipped, npm audit
reports 0 vulnerabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 15:50:51 +02:00
Oliver Kieselbach dbf6e50e2d feat(agent,web,shared): phase_exit gather trigger — one-shot collection at a phase's end
Follow-up to gather-rule phase scoping. One-shot collection at a phase START
already existed (phase_change dedups per (rule, phase)) and at enrollment END
(on_event + enrollment_complete); the END of a specific phase had no trigger.

New trigger value "phase_exit": fires once when TriggerPhase is LEFT (empty =
every phase transition). Together with phase_change it forms the two one-shot
bookends of a phase.

Agent (GatherRuleExecutor.OnPhaseChanged):
- Exit rules are evaluated BEFORE _currentPhase advances, so both the
  TriggerPhase match and the phase-scope gate see the phase being LEFT. A rule
  scoped to DeviceSetup therefore fires on its own exit even though the phase
  being entered is out of scope.
- A transition into Failed fires the exit rules of the failing phase — the
  snapshot at a failure boundary is the point of the trigger.
- Exit of Unknown never fires; a repeated identical phase signal is not a
  transition.
- Dedup key "exit:{ruleId}|{phase}" shares _phaseRulesExecuted with the
  phase_change keys but stays in a disjoint key space, so an enter-rule and an
  exit-rule on the same phase cannot consume each other's slot.

Portal: trigger labels are now "Phase Start" / "Phase End"; the canonical phase
dropdown and the create/save payloads cover both via PHASE_TRIGGERS; the card
renders "entering X" / "leaving X".

No ConfigVersion bump: no new field is transported. Old agents do not know the
value and simply never fire the rule (silent no-op).

Tests: GatherRuleExecutorPhaseExitTests (11) — enter-vs-exit, no refire on
re-entry, empty-triggerPhase, exit-of-Unknown, same-phase-signal, into-Failed,
Complete, scope-against-left-phase, out-of-scope slot preservation, key-space
disjointness, on_change interplay. Build 0 Errors; V2.Core.Tests 1884/1884;
Functions.Tests 3099/3099; vitest 557/557; tsc silent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-21 15:37:48 +02:00
Oliver Kieselbach 062962ae41 docs(claude): dedupe and condense agent directives
CLAUDE.md ships with every request. Removed the example from the
operator-only-infrastructure rule - rules state the rule, not an
instance - and applied the same yardstick to the rest of the file.

Merged what said the same thing twice: "would a staff engineer approve"
and "what would a perfectionist reviewer reject"; Demand Elegance and
Code Quality and Core Principles; Task Management's lessons item and the
Self-Improvement Loop; Task Management's plan steps and Plan Mode
Default. Dropped motivational framing that steers no decision, and a
stray heading level.

Every actual rule survives: the 3+ step plan threshold, stop-and-re-plan,
subagent usage, verify-before-done with a diff against main, autonomous
bug fixing, lessons capture, 500 LOC chunked reads, no absence claims
from one grep, the structural-fix mandate, and the Codex review note.

425 words, down from 614 at the start of the session - and that now
includes the customer-facing claims section, which did not exist then.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:15:14 +02:00
Oliver Kieselbach be08660976 docs(claude): condense customer-facing claims and OKF sections
CLAUDE.md ships with every request, so length is a running cost. Cut the
new claims section from 12 lines to 4 and tightened the OKF section,
dropping the spec URL and restating the rules imperatively. Net effect:
the file is now smaller than before the claims section was added
(654 words vs 856).

No rules were removed - verify-before-writing, the change triggers, the
durable-phrasing rule, and the operator-only-infra caveat all remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:09:48 +02:00
Oliver Kieselbach b032005f5b docs(claude): add customer-facing claims to the release checklist
Terms, Privacy, the About page and the trust docs make statements that
customers and security reviewers rely on, but nothing opened them during
a refactor, so they drifted. The July 2026 sweep found a ".NET 8" backend
that was .NET 10, a "Next.js 18" that does not exist, ETL log collection
that never happened, and "each tenant runs its own isolated instance" for
a shared multi-tenant service.

Records which surfaces exist, which kinds of change must update them in
the same commit, and that claims are verified against the code rather
than carried forward.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 14:01:23 +02:00
Oliver Kieselbach 0ab568f479 fix(web): correct stale About claims; open Community plan to any tenant
Terms: replace the blanket business-use-only clause. It excluded exactly
the wrong group - a registered association is not a consumer under German
law anyway (only natural persons can be), while the clause did shut out
private individuals running their own lab or personal tenant, who are a
real and intended audience for a free community edition. Community is now
stated as open to anyone with an eligible Entra ID/Intune tenant;
business-only stays where it means something, on the Enterprise contract.

About page - claims verified against the code and corrected:
- "each customer tenant runs its own isolated instance" was wrong and
  contradicted the same page two sections earlier. It is one shared
  multi-tenant service with partition-key isolation and delegated MSP
  access.
- Diagnostics: no ETL tracing and no Windows event log export happens;
  the bundle is agent logs, IME logs, agent state and device info. Upload
  is tenant-configured (off/always/on failure) at the end of enrollment,
  not triggered on demand from the portal - the portal downloads it.
- .NET 8 Isolated -> .NET 10 Isolated on Flex Consumption.
- "Next.js 18" -> Next.js 15 (React 18); 18 was React's version.
- Roles: Admin/Operator -> Admin/Operator/Viewer plus delegated MSP.
- Notifications: Teams only -> Teams, Slack, generic JSON webhooks, with
  start/success/failure and SLA triggers. Telegram stays unlisted; it is
  operator-only alerting, not a tenant feature.
- Agent deployment: Win32 app -> Intune platform script; self-destruct is
  on by default, not optional-sounding.
- Dropped the "tamper-evident" audit claim - records are ordinary table
  entities with no hash chain, signing, or immutability policy.

Confirmed while verifying: no tenant-configurable email channel exists
anywhere in the codebase, so the published privacy statement that Resend
serves only the onboarding mail is accurate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 13:58:07 +02:00
Oliver Kieselbach fced858a7a feat(legal,agent): professionalize Terms/Privacy; wire SendTraceEvents gate
Rework the public Terms of Use and Privacy Policy from their early-days
wording to match how the service actually runs today, and fix a setting
that looked like a privacy control but gated nothing.

Legal positioning:
- glueckkanja AG operates the service and is the counterparty on BOTH
  plans (Community/Enterprise is a flag in one system, one subscription).
  Oliver Kieselbach is named as creator and maintainer of the open-source
  project and the Community edition, acting on behalf of glueckkanja AG
  and explicitly not a separate contracting party.
- Community: free, "AS-IS", no availability or support commitment.
  Enterprise: written agreement incl. DPA, higher limits, MSP delegation.
- Added business-use-only clause, simplified governing law (dropped the
  CISG exclusion - it governs sale of goods, not SaaS), split liability
  per plan, added an MSP delegation section and a rules-contribution
  clause covering the community rule pool.
- Imprint links to the glueckkanja AG imprint, whose scope clause covers
  every service linking to it - no duplicated company data to go stale.
  Reachable from the landing footer and both help menus.

Privacy Policy corrections (previous claims were inaccurate):
- "No personal data is transmitted" - backend telemetry carries the UPN;
  only the browser SDK is cookie- and UPN-free.
- "Not shared with third parties" - qualified for MSP delegation,
  customer-configured notification channels, the customer's own LLM via
  MCP, and agent-side geolocation.
- Removed "data subject rights are not guaranteed"; GDPR rights are not
  optional. Added region, sub-processors, retention and IP handling.

Agent - SendTraceEvents gate:
The setting was plumbed from TenantConfiguration through to
AgentConfiguration but never consumed; Trace severity only hides an event
from the timeline view. The outbound IP was therefore always collected
when geolocation is on (which is the default). Gate is now enforced in
TelemetryEventEmitter.Emit, the single choke point before the transport
enqueue. Dropped before the sequence counter advances so suppression
leaves no gaps. Evaluated per event via Func<bool> so a mid-session
remote-config merge takes effect. Fails open: an unwired or throwing
accessor lets trace events flow, because the gate suppresses diagnostics
and must not silently blind the timeline.

Default stays on and the switch remains operator-only via the config
table - deliberately not surfaced in the UI or documentation.

Verified: solution builds clean, 1873 agent core tests pass,
tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 13:42:03 +02:00
Oliver Kieselbach ee947e2ca9 updated docs links 2026-07-20 20:52:56 +02:00
Oliver Kieselbach 5f08a6ec68 updated dev script 2026-07-20 19:52:54 +02:00
Oliver Kieselbach a97888ebd5 fix(agent): neutral aggregated 2800 observations; reboot claim gated on observed reboot
Rework after the first real enrollment (session b2e890c1) exposed three
defects in a654c730:

- Shell-Core "RebootCoalescing" match was a false positive by construction:
  the token appears in the ROUTINE SubcategoryProcessing_Started bootstrap
  marker ("Starting subcategory DeviceSetup.RebootCoalescing...") on EVERY
  enrollment. esp_reboot_coalescing removed entirely (branch, constant,
  catalog, tests).
- 2800 reboot-required flags fire at every policy sync; only DeviceSetup-time
  flags cause the coalesced reboot. The session's flags fired during
  AccountSetup - no reboot - yet our Warning events claimed "device will
  restart". Raw event is now a neutral Info observation; the reboot CLAIM
  moved to ANALYZE-ESP-005 with a REQUIRED system_reboot_detected >= 1
  condition (no observed reboot -> advisory silent).
- One event per 2800 record was spam: records are now buffered and debounced
  (10 s) into ONE aggregated event per burst (rebootUris[]/uriCount/
  firstRebootUri/recordCount/unparsedCount). Watermark is persisted at FLUSH,
  not claim - a pre-emit reboot kill loses nothing, the post-restart backfill
  re-emits the burst at its historical time.

Verified 2800 description shape recorded in code, tests, and concept doc:
"The following URI has triggered a reboot: (./Device/...)."; extraction
covered by a real-text regression test. Lesson added to tasks/lessons.md
(observation-level events vs outcome-gated claims; substring tokens appear
in framework bookkeeping records).

Tests: Agent.V2.Core 1867/1867, Functions 3099/3099, MCP 400 passed,
solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:41:47 +02:00
Oliver Kieselbach a654c7309e feat(agent): attribute mid-ESP coalesced reboots to MDM policy URIs (second sign-in detection)
Device-assigned MDM policies whose CSP URIs match the OS reboot-required
catalog force a coalesced reboot at the end of ESP DeviceSetup - the user
sees an unexpected restart and a second sign-in before AccountSetup.
Invisible in the Intune console; admins simply lived with it. Now attributed
and actionable (PMPC research).

- MdmRebootPolicyTracker (+host): watches DeviceManagement-Enterprise-
  Diagnostics-Provider/Admin EventID 2800, emits one mdm_policy_reboot_required
  per reboot-forcing URI (Warning, ImmediateUpload; tolerant EventData-first
  URI extraction with regex fallback; fail-soft OMADM RebootRequired probe).
  Cross-restart RecordId watermark (mdm-reboot-watermark.json) - mandatory,
  the coalesced reboot restarts the agent and the backfill re-reads the log.
- ShellCoreTracker: RebootCoalescing branch on the already-watched 62407
  stream -> esp_reboot_coalescing corroboration event; deliberate deviation:
  emits on backfill too (live emit races the reboot kill).
- ANALYZE-ESP-005 advisory (warning): names the first URI via {{rebootUri}},
  remediation = reassign profiles device -> user groups; +20 confidence on
  coalescing corroboration, +10 on >=2 URIs.
- Config: Collectors.MdmRebootPolicyWatcherEnabled (default ON, internal
  steering only - agent-native collector, NOT an IAgentAnalyzer, no admin UI),
  MdmRebootPolicyBackfillLookbackMinutes=60; ConfigVersion 35.
- Event types deliberately distinct from system_reboot_detected so the
  backend RebootCount (exact-string match, 2 sites) stays uninflated.
- Docs: docs/agent/mdm-reboot-coalescing.md (+index/log). Exact 2800/62407
  message texts are unverified until E2E VM validation (tolerant matching).

Tests: Agent.V2.Core 1868/1868 (21 new), Functions 3099/3099, MCP 400 passed
(event-types drift), solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:05:11 +02:00
Oliver Kieselbach 6d1ff8d7f0 fix: code-review follow-ups (watcher dispose race, revoke-enforcement tests, scan guard)
- ConsoleBypassWatcher: re-check _disposed after watcher.Start() — a Dispose()
  racing the background arm could leave an armed WMI subscription no code path
  could ever stop (leaked until process exit).
- SignalRContext: collapse rapid accessRevoked pushes into one restart via an
  in-flight guard — a second start() on a non-Disconnected connection throws
  and burns the retry budget.
- config/all: negative-cache managed ids a rescue scan proved missing (5 min
  TTL) — one stale grant (offboarded tenant) no longer makes every delegated
  config/all call pay a full table scan.
- DelegatedAdmin/TenantGroup functions: extract testable revoke-enforcement
  cores (RevokeCoreAsync, SetEnabledCoreAsync, UnassignCoreAsync,
  DeleteGroupCoreAsync, ValidateGrantRequest) and cover the security flows:
  404-on-missing-row writes no audit row and fires no disconnect; real
  revoke/disable/unassign/group-delete always cuts live streams; grant
  validation is fail-closed (GUID tenantId, recognized roles only).
- dashboard/page.tsx: drop unused next/link import left by the banner removal.

Tests: backend 3099 passed, web 557 passed, agent ConsoleBypassWatcher suite
passed, tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 13:13:26 +02:00
Oliver Kieselbach 5c8b273f37 chore: bump skills submodule (system-Timestamp warning in raw-query skills)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 12:38:04 +02:00
Oliver Kieselbach da958b5398 fix(backend): business timestamps survive storage migrations (OccurredUtc + RowKey decode)
The 2026-07-18 storage migration reset every row's system Timestamp,
exposing that AuditLogs, OpsEvents and Events displayed, filtered and
retention-swept on it (custom "Timestamp" writes were silent no-ops —
reserved system property). All migrated rows read as 2026-07-18.

- New Helpers/BusinessTimestamp: RowKey decoders (audit !revtick_guid,
  ops bare revtick, event yyyyMMddHHmmssfff_seq), RowKey-range OData
  clause builders (tick-exact, with the mandatory RowKey lt '"' guard
  so no lower bound can ever match legacy bare-GUID audit rows), and
  dual DateTimeOffset/DateTime column reads.
- Writes in all three tables now persist OccurredUtc (custom column,
  copied verbatim by future migrations); no-op ["Timestamp"] removed.
- Read mappers resolve OccurredUtc -> RowKey decode -> system Timestamp;
  GetEarliestSessionEventTimestampAsync uses the same resolution so
  StartedAt/Duration reconciliation can no longer absorb the migration
  date on migrated sessions.
- Date-window filters and retention moved off the system Timestamp onto
  RowKey ranges; ops retention loses its full-table scan and the
  post-migration mass-delete cliff; audit retention runs a temporary
  second pass for legacy GUID rows (TODO: remove after 2027-03-01).
- New GlobalAdmin backfill endpoint POST /api/maintenance/
  backfill-occurred-utc (dry-run by default, batched, idempotent MERGE
  of only OccurredUtc, table=audit|ops) — Events rows are deliberately
  never mutated; their read path decodes the RowKey instead.
- Tests: decoders, clause boundary proofs incl. GUID-safety pins,
  mapper precedence, backfill row decisions (Functions.Tests 3078).
- Docs: docs/backend/business-timestamps.md (system Timestamp is never
  authoritative; migration checklist rule).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 11:29:53 +02:00
Oliver Kieselbach e17eb607ca docs: gather-rule phase scoping + emit-on-change concept doc (OKF)
New rules/ section in the knowledge bundle: semantics table, from-phase latch
behavior (never via Failed, sticky through it), on_change x emitOnlyIfExists
composition, restart re-emit, no-phase-signal caveat, old-agent degradation,
and the prerequisite toggle-wipe fix. Index + log updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:14 +02:00
Oliver Kieselbach 71000a5dd2 feat(web): gather-rule scope + emit-mode UI, phase dropdowns, edit tags wipe fix
Step 4: portal round-trip for the new fields.

- types.ts: GatherRule + activePhases/activeFromPhase/emitMode; NewRuleForm +
  scopeMode ("always" | "during" | "from") as UI-only view state; canonical
  GATHER_PHASES list (EnrollmentPhase names Start..Complete with display
  labels) + EMIT_MODES; new rules default to emitMode "on_change".
- withDerivedScopeMode: JSON-mode merges rule-shaped JSON (no scopeMode key)
  over form state — the helper derives scopeMode from the data in all four
  JSON paths (create apply/save, edit apply, card edit save) so pasted scope
  fields survive into the payload instead of being silently dropped.
- GatherRuleFormFields: "Active During" mode select + phase multi-select /
  from-phase dropdown (mode switch clears the other mode's fields), "Emit
  Mode" select with helper text; triggerPhase free-text replaced by the same
  canonical dropdown (placeholder used to suggest invalid tokens like
  "DeviceESP"; legacy stored values remain selectable).
- page.tsx: create/save payloads carry the three fields; startEditing maps
  them back (existing rules load as "always", the on_change default is for
  NEW rules only); handleSaveEdit now also sends tags — same wipe class as
  the toggle bug, the full-replace PUT dropped them on every edit.
- GatherRuleCard: trigger section shows the scope + emit mode, e.g.
  "Interval every 60 seconds - from Account Setup - emit on change".

npx tsc --noEmit silent; vitest 557/557.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:04 +02:00
Oliver Kieselbach a88db34af5 feat(agent): gather-rule phase scoping + on_change emit dedup in GatherRuleExecutor
Step 3: the executor honors the new GatherRule fields.

Phase scope (ActivePhases / ActiveFromPhase), applied to ALL trigger types:
- interval: timers keep running, an out-of-scope tick is a free early-return
  (no start/stop lifecycle races on phase transitions).
- startup: scoped rules not yet in scope at UpdateRules are deferred and run
  exactly once from OnPhaseChanged when their scope activates (same
  _startupRulesExecuted dedup; not part of the WaitForStartupRules latch).
- phase_change: evaluated against the NEW phase; scope gate runs BEFORE the
  (rule, phase) dedup add so an out-of-scope pass does not consume the slot.
- on_event: gated by current phase.
- From-phase latch: phase != Unknown/Failed && ordinal >= threshold; sticky for
  the session incl. through Failed; re-evaluated on config refresh so a rule
  delivered after its threshold passed latches immediately.
- Before the first phase signal, scoped rules are inactive.
- IgnorePhaseScope bypass for --run-gather-rules (no phase context; logged).

EmitMode on_change:
- Canonical hash (keys ordinal-sorted at every level, invariant formatting,
  SHA-256) of the raw collector result BEFORE ruleId/ruleTitle injection.
- First non-empty result always emits; unchanged results increment a per-rule
  suppression counter; the next emitted event carries suppressedPolls +
  suppressedSinceUtc. Empty results (emitOnlyIfExists miss) touch nothing, so
  the combo yields zero noise, one event on appearance, then only on change.
- State survives UpdateRules refreshes (agent restart => at most one re-emit).

Tests: GatherRuleExecutorPhaseScopeTests (scope matrix synchronous via internal
IsRuleInScope; trigger wiring async with positive waits),
GatherRuleExecutorEmitModeTests (state machine via internal ShouldEmitOnChange,
canonical-hash properties, ExecuteRule wiring). V2.Core.Tests 1847/1847.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:50 +02:00
Oliver Kieselbach 21dd35f787 feat(backend,shared): gather-rule phase scoping + emit-mode fields, custom-rule toggle wipe fix
Step 0-2 of the gather-rule anti-spam plan:
- GatherRuleService.UpdateRuleAsync: toggle-style partial PUTs ({enabled,
  isBuiltIn, isCommunity}) previously full-replaced the stored custom rule and
  wiped Title/Target/Trigger/Parameters. Partials now merge Enabled into the
  existing row; full edit payloads preserve the original CreatedAt.
- GatherRule: new opt-in fields ActivePhases (run only during listed phases),
  ActiveFromPhase (sticky latch from a phase onwards), EmitMode (always |
  on_change). Absent fields = legacy behavior; schema updated accordingly.
- TableStorageService.Rules: Store+Map for the three columns (absent column /
  stored-empty -> null); MapToGatherRule internal for roundtrip tests.
- ContentEquivalent covers the new fields, null-tolerant for ActivePhases so
  pre-existing rows never diff against seeds (no reseed churn).
- GatherRulesFunction.ValidateScopeAndEmitMode: canonical EnrollmentPhase name
  list Start..Complete (Unknown/Failed and numeric tokens rejected), mutual
  exclusion of the two scope fields, emitMode whitelist; 400 on violation.
- ConfigVersion 33 -> 34 (33 was taken by MigrateToApiBaseUrl).

Tests: GatherRuleUpdatePartialMergeTests (merge/replace/unknown-rule/built-in
path), GatherRuleScopeFieldsTests (Store<->Map roundtrip incl. legacy rows,
ContentEquivalent matrix, validation accept/reject). Functions.Tests 3029/3029.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:35 +02:00
Oliver Kieselbach 9933c039ed refactor(shared,backend): MigrateToApiBaseUrl nullable, drop null-forgiving operator
null = no migration is the documented contract and the agent already
assigns null when stripping the field from the on-disk cache; annotate
the property as string? so the backend assignment needs no ! suppression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 13:33:57 +02:00
Oliver Kieselbach ef67dec263 fix(backend): system-health URLs post EU-migration + GA endpoint visibility
- Agent Binaries check + ops watcher now probe the canonical download alias
  (download.autopilotmonitor.com) AND the legacy blob keepalive; alias failure
  is unhealthy, legacy failure a warning
- McpServerBaseUrl fallback moved from the stale kindwave WEU ACA FQDN to the
  stable custom domain mcp.autopilotmonitor.com (CNAME to the new GWC instance)
- health/detailed and health/mcp include the probed endpoint URLs in check
  details for Global Admins only (storage table endpoint, WEBSITE_HOSTNAME,
  alias/legacy download URLs, MCP server URL)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:42:40 +02:00
Oliver Kieselbach b005c20598 feat(agent,backend,web): config-channel agent endpoint migration (MigrateToApiBaseUrl)
Re-home agents to a new API base URL through the agent/config response
(backend move or per-tenant EU->US region move) - the lesson from the
2026-07-18 WEU->GWC cutover, where the compiled-in API hostname hard-cut
every in-flight agent (no DNS indirection possible: mTLS breaks behind a
proxy, Flex custom-domain TLS is broken).

Backend (ConfigVersion 33):
- AdminConfiguration.AgentMigrateApiBaseUrl (global) +
  AgentMigrateTenantOverridesJson (tenantId->URL; empty value pins a
  tenant against the global target; override wins)
- GetAgentConfigFunction.ResolveMigrateTarget serves the validated target;
  AgentMigrateServed / AgentMigrateRejected warnings as delivery evidence
- TableConfigRepository Store+Map; admin portal Global Settings round-trip

Agent (kill-switch-mirrored hardening):
- Shared AgentEndpointMigrationRules: https-only, *.azurewebsites.net on a
  label boundary, no path/port/userinfo - enforced on BOTH sides
- Live fetch only: RemoteConfigService cache strips the field on write AND
  read; EndpointMigration.ResolveTarget re-checks the fetch outcome
- AgentRuntimeConfig.Resolve returns before merge/integrity/bootstrap
  cleanup on a pending migration (no spurious IntegrityCheckFailed against
  a stale migration stub), Program.RunAgent rebuilds Phase 3+4 against the
  new URL - single hop, kill signal wins before and after the hop

Tests: AgentEndpointMigrationTests (backend rules + resolution),
EndpointMigrationTests + cache-hardening strip (agent). Full suites green
(Functions 2992, Agent Core 1822, vitest 557, tsc silent).

Doc: docs/agent/endpoint-migration.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 17:48:48 +02:00
Oliver Kieselbach e2d78c6b06 chore(ci): rename SWA workflow to deploy-web.yml with neutral deploy-token secret
The brave-grass filename and secret name referred to the pre-migration
Static Web App hostname. Workflow now matches the deploy-backend/-mcp
naming scheme and uses AZURE_SWA_DEPLOY_TOKEN (already set to the new
SWA's token). The old secret is removed after the first green run.
2026-07-18 16:51:39 +02:00
Oliver Kieselbach 3d06a689cc feat(web): rename Known Issues to Service Announcements, drop maintenance banners
Menu/dashboard/tip labels now match the docs page title. The temporary
maintenance banners on landing and dashboard are removed — the completion
notice lives in Service Announcements. Also checks in the migration
follow-up plan and the runbook outcome addendum.
2026-07-18 16:43:50 +02:00
Oliver Kieselbach 2d5f87140f chore: retarget SignalR quota watcher to Standard S1 limits + CI build hygiene
- SignalR quota watcher/health card: defaults 20/20k (Free tier) -> 1000/1M
  (Standard S1 x1 unit); settings renamed SignalRFreeTier* -> SignalRConnectionLimit/
  SignalRDailyMessageLimit (no Azure overrides were set); all operator-facing
  texts tier-neutral (connections 429 hard, message overage is billed not throttled)
- web: drop obsolete edge runtime from opengraph/twitter/apple-icon image routes
  (next/og runs on Node since 14; images now statically generated at build)
- web: update caniuse-lite browser db (silences 6-months-old Browserslist warning)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:38:15 +02:00
Oliver Kieselbach ce1ca3dbf4 updated SignalR limits on system health 2026-07-18 16:22:49 +02:00
Oliver Kieselbach b1a8e371ad chore(build): retarget DecisionCore.Tests to net10.0, drop 8.0.x SDK from agent CI
Nothing required net8.0 -- the test runner just predated the .NET 10
migration. Agent projects stay net48 (Intune/OOBE constraint).
474/474 tests green on net10.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:01:39 +02:00
Oliver Kieselbach b698a87bc8 chore(build): align local and CI builds on .NET 10 SDK
- global.json pins SDK 10.0.2xx (latestFeature) for local and CI parity
- build-agent.yml: SDK 10 as build toolchain, 8.0.x kept for the
  net8.0 DecisionCore.Tests runtime
- build.ps1: stale .NET 8 hints updated to .NET 10

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 15:59:23 +02:00
Oliver Kieselbach ac31ee980e feat(ci): mirror agent artifacts to legacy storage (transition until download.-alias migration; fail-soft, counter stays on eu only) 2026-07-18 15:57:40 +02:00
151 changed files with 10785 additions and 611 deletions
+25 -67
View File
@@ -1,79 +1,37 @@
## Workflow Orchestration
# Agent Directives: Mechanical Overrides
# Agent Directives
You are operating within a constrained context window and strict system prompts. To produce production-grade code, you MUST adhere to these overrides:
- **Plan first.** Enter plan mode for any non-trivial task (3+ steps or an architectural decision) and write the spec before building. If something goes sideways, stop and re-plan instead of pushing on.
- **Use subagents liberally** for research, exploration, and parallel analysis — one focus each — to keep the main context clean.
- **Verify before claiming done.** Run the tests, check the logs, show the evidence. Diff against `main` where behaviour could regress.
- **Fix bugs autonomously.** Given a report, a failing test, or an error, resolve it — no hand-holding.
- **Capture corrections.** After any correction from the user, record the pattern and a preventing rule in `tasks/lessons.md`; review it at session start.
- Read files over 500 LOC in chunks. Never treat one grep as proof of absence.
Codex reviews your output once you are done.
## Code Quality
### 1. Plan Mode Default
- Enter plan mode for ANY non-trivial task (3+ steps or architectural decisions)
- If something goes sideways, STOP and re-plan immediately — don't keep pushing
- Use plan mode for specification steps, not just building
- Write detailed specs upfront to reduce ambiguity
### 2. Subagent Strategy
- Use subagents liberally to keep main context window clean
- Offload research, exploration, and parallel analysis to subagents
- For complex problems, throw more compute at it via subagents
- One tack per subagent for focused execution
### 3. Self-Improvement Loop
- After ANY correction from the user: update `tasks/lessons.md` with the pattern
- Write rules for yourself that prevent the same mistake
- Ruthlessly iterate on these lessons until mistake rate drops
- Review lessons at session start for relevant project
### 4. Verification Before Done
- Never mark a task complete without proving it works
- Diff behavior between main and your changes when relevant
- Ask yourself: "Would a staff engineer approve this?"
- Run tests, check logs, demonstrate correctness
### 5. Demand Elegance (Balanced)
- For non-trivial changes: pause and ask "is there a more elegant way?"
- If a fix feels hacky: "Knowing everything I know now, implement the elegant solution"
- Skip this for simple, obvious fixes — don't over-engineer
- Challenge your own work before presenting it
### 6. Autonomous Bug Fixing
- When given a bug report: just fix it. Don't ask for hand-holding
- Point at logs, errors, failing tests — then resolve them
- Zero context switching required from the user
- Go fix failing CI tests without being told how
### Tools Execution
- Read any file over 500 LOC in chunks using offset and limit parameters.
- Do not assume a single grep caught everything.
### Code Quality
- If architecture is flawed, state is duplicated, or patterns are inconsistent - propose and implement structural fixes. Ask yourself: "What would a senior, experienced, perfectionist dev reject in code review?" Fix all of it.
Codex will review your output once you are done!
- Simplicity first: minimal, targeted changes. No temporary fixes — find the root cause.
- Where architecture is flawed, state is duplicated, or patterns are inconsistent, propose and implement the structural fix. Ask what a perfectionist reviewer would reject, and fix that too.
- On non-trivial changes, ask whether there is a more elegant way before presenting. Skip it for simple fixes — don't over-engineer.
## Task Management
1. **Plan First**: Write plan to `tasks/todo.md` with checkable items
2. **Verify Plan**: Check in before starting implementation
3. **Track Progress**: Mark items complete as you go
4. **Explain Changes**: High-level summary at each step
5. **Document Results**: Add review section to `tasks/todo.md`
6. **Capture Lessons**: Update `tasks/lessons.md` after corrections
Keep the plan in `tasks/todo.md` as checkable items, confirm it before implementing, tick items off as you go, and add a review section when done.
## Core Principles
## Customer-Facing Claims
- **Simplicity First**: Make every change as simple as possible. Impact minimal code.
- **No Laziness**: Find root causes. No temporary fixes. Senior developer standards.
- **Minimal Impact**: Changes should only touch what's necessary. Avoid introducing bugs.
`src/Web/autopilot-monitor-web/app/{about,terms,privacy}/page.tsx` and `autopilotmonitor-docs/trust/*` state facts customers and security reviewers rely on. They drift silently.
## Technical Docs — OKF Knowledge Bundle (`docs/`)
- Verify every claim against the code before writing it — never carry one forward because it was already on the page.
- Update them in the same change as: runtime/framework versions, roles, notification providers, agent deployment or lifecycle, diagnostics payload, isolation/delegation model, retention caps, sub-processors, or any default governing what is collected.
- Describe only what customers can actually use; operator-only infrastructure is not a feature.
- Trust pages: durable phrasing over exact figures, and bump their "Last reviewed" date.
`docs/` is an [Open Knowledge Format (OKF) v0.1](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) knowledge bundle for contributor/AI-facing technical documentation (customer docs live in the separate autopilotmonitor-docs repo).
## Technical Docs — OKF Bundle (`docs/`)
- **Consume first**: start at `docs/index.md` for progressive disclosure before opening individual documents.
- **When new durable technical knowledge emerges** (architecture decisions, flows, non-obvious mechanisms), capture it as an OKF concept document in `docs/` — markdown with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp` (recommended).
- **Maintain the bundle**: add every new document to `docs/index.md`, note changes in `docs/log.md` (grouped by ISO date). Use standard RELATIVE markdown links between docs — never the OKF `/`-prefixed bundle-absolute form (GitHub resolves those from the repo root and navigation breaks). `index.md` and `log.md` are reserved filenames.
- Documents are English, structural markdown over prose (`# Schema`, `# Examples`, `# Citations` sections where applicable).
Contributor/AI-facing docs (customer docs live in the `autopilotmonitor-docs` repo). Start at `docs/index.md` before opening individual files.
- New durable knowledge (architecture decisions, flows, non-obvious mechanisms) → an OKF concept doc with YAML frontmatter: `type` (mandatory), plus `title`, `description`, `resource`, `tags`, `timestamp`.
- Register it in `docs/index.md`, note the change in `docs/log.md` (ISO date). `index.md` and `log.md` are reserved names.
- English, structural markdown (`# Schema`, `# Examples`, `# Citations`). Links between docs are RELATIVE — never `/`-prefixed, that breaks GitHub navigation.
+37
View File
@@ -0,0 +1,37 @@
version: 2
updates:
- package-ecosystem: nuget
directory: /
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
dotnet:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: npm
directory: /src/Web/autopilot-monitor-web
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
web:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: npm
directory: /src/McpServer/autopilot-monitor-mcp
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
mcp:
patterns: ['*']
update-types: [minor, patch]
- package-ecosystem: github-actions
directory: /
schedule:
interval: monthly
+16 -1
View File
@@ -58,10 +58,11 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v5
# SDK 10 = single build toolchain (parity with local build.ps1 / global.json).
- name: Setup .NET
uses: actions/setup-dotnet@v5
with:
dotnet-version: '8.0.x'
dotnet-version: '10.0.x'
# Build numbers come from the shared counter blob (single source of truth for CI
# AND local build.ps1). Reserve-before-build with ETag-CAS: concurrent builds can
@@ -265,6 +266,12 @@ jobs:
run: |
$containerUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent"
$sasToken = "${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}".TrimStart('?')
# Legacy mirror (transition until customers moved to the download.-alias):
# blobs are ALSO pushed to the old account so the bootstrap URL baked into
# customers' Intune scripts keeps serving current bits. Fail-soft -- a
# legacy hiccup must never fail a release. Counter lives ONLY on eu.
$legacyContainerUrl = "https://autopilotmonitor.blob.core.windows.net/agent"
$legacySasToken = "${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}".TrimStart('?')
function Send-Blob {
param([string]$Local, [string]$BlobName, [string]$ContentType)
@@ -279,6 +286,14 @@ jobs:
Write-Host "Uploading $BlobName ($([math]::Round($bytes.Length / 1MB, 2)) MB)..."
Invoke-RestMethod -Uri $url -Method Put -Headers $headers -Body $bytes
Write-Host " uploaded $BlobName"
if (-not [string]::IsNullOrEmpty($legacySasToken)) {
try {
Invoke-RestMethod -Uri "$legacyContainerUrl/$BlobName`?$legacySasToken" -Method Put -Headers $headers -Body $bytes
Write-Host " mirrored $BlobName to legacy storage"
} catch {
Write-Host "::warning::legacy mirror failed for $BlobName -- $($_.Exception.Message)"
}
}
}
# Versioned namespace (always)
+116
View File
@@ -0,0 +1,116 @@
name: CI
# Runs the full test suite on every pull request and every push to main.
# The deploy workflows build but do not test — this is the gate that decides
# whether a change is allowed to be correct.
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
dotnet:
name: .NET tests (backend + agent)
# windows-latest is required: the agent test projects target net48.
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-dotnet@v5
with:
dotnet-version: '10.0.x'
- name: Restore
run: dotnet restore AutopilotMonitor.sln
- name: Build
run: dotnet build AutopilotMonitor.sln --configuration Release --no-restore
- name: Backend tests
run: dotnet test src/Backend/AutopilotMonitor.Functions.Tests --configuration Release --no-build --verbosity quiet
- name: Agent V2 Core tests
run: dotnet test src/Agent/AutopilotMonitor.Agent.V2.Core.Tests --configuration Release --no-build --verbosity quiet
- name: Decision Core tests
run: dotnet test src/Agent/AutopilotMonitor.DecisionCore.Tests --configuration Release --no-build --verbosity quiet
- name: Summary Dialog tests
run: dotnet test src/Agent/AutopilotMonitor.SummaryDialog.Tests --configuration Release --no-build --verbosity quiet
web:
name: Web (typecheck + tests)
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/Web/autopilot-monitor-web
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
cache: npm
cache-dependency-path: src/Web/autopilot-monitor-web/package-lock.json
- run: npm ci
- name: Typecheck
run: npx tsc --noEmit
- name: Tests
run: npm test
mcp:
name: MCP server tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/McpServer/autopilot-monitor-mcp
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
cache: npm
cache-dependency-path: src/McpServer/autopilot-monitor-mcp/package-lock.json
- run: npm ci
- name: Typecheck
run: npx tsc --noEmit
- name: Tests
run: npm test
guardrails-in-sync:
name: Generated guardrails match rules/guardrails.json
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
with:
node-version: '20'
- name: Regenerate
run: node rules/scripts/combine.js
- name: Fail if the generated files drifted
run: |
if ! git diff --quiet; then
echo "::error::Generated files are out of date. Run 'node rules/scripts/combine.js' and commit the result."
git diff --stat
exit 1
fi
+58
View File
@@ -0,0 +1,58 @@
name: CodeQL
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
# Weekly, so advisories published after a merge still surface.
- cron: '17 3 * * 1'
permissions:
contents: read
jobs:
analyze:
name: Analyze ${{ matrix.language }}
runs-on: ${{ matrix.runner }}
permissions:
security-events: write
packages: read
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
# csharp needs the net48 agent projects to build, so Windows.
- language: csharp
runner: windows-latest
build-mode: manual
- language: javascript-typescript
runner: ubuntu-latest
build-mode: none
steps:
- uses: actions/checkout@v5
- uses: actions/setup-dotnet@v5
if: matrix.language == 'csharp'
with:
dotnet-version: '10.0.x'
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- name: Build
if: matrix.build-mode == 'manual'
run: dotnet build AutopilotMonitor.sln --configuration Release
- name: Analyze
uses: github/codeql-action/analyze@v3
with:
category: /language:${{ matrix.language }}
+17
View File
@@ -21,6 +21,22 @@ jobs:
steps:
- uses: actions/checkout@v5
# The published customer documentation lives in its own PUBLIC repo and is
# baked into the image as the search_docs corpus (see the Dockerfile). No
# token is needed. Checking it out here rather than vendoring it keeps the
# docs single-sourced — but it also means EDITING THE DOCS DOES NOT UPDATE A
# RUNNING SERVER; this workflow must be re-run. /health reports the SHA
# captured below so that staleness is checkable.
- name: Check out documentation bundle
uses: actions/checkout@v5
with:
repository: autopilotmonitor/autopilotmonitor-docs
path: mcp-docs
- name: Resolve docs commit
id: docs
run: echo "sha=$(git -C mcp-docs rev-parse HEAD)" >> "$GITHUB_OUTPUT"
- name: Login to Azure (OIDC)
uses: azure/login@v3
with:
@@ -35,6 +51,7 @@ jobs:
run: |
docker build \
-f src/McpServer/autopilot-monitor-mcp/Dockerfile \
--build-arg DOCS_COMMIT=${{ steps.docs.outputs.sha }} \
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:${{ env.IMAGE_TAG }} \
-t ${{ env.ACR_NAME }}.azurecr.io/${{ env.CONTAINER_APP_NAME }}:latest \
.
@@ -1,4 +1,4 @@
name: Azure Static Web Apps CI/CD
name: Deploy Web (Static Web App)
on:
push:
@@ -6,7 +6,7 @@ on:
- main
paths:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/azure-static-web-apps-brave-grass-0070b5703.yml'
- '.github/workflows/deploy-web.yml'
workflow_dispatch:
pull_request:
types: [opened, synchronize, reopened, closed]
@@ -14,7 +14,7 @@ on:
- main
paths:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/azure-static-web-apps-brave-grass-0070b5703.yml'
- '.github/workflows/deploy-web.yml'
jobs:
build_and_deploy_job:
@@ -37,7 +37,7 @@ jobs:
NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL: ${{ vars.NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL }}
NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING: ${{ secrets.NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING }}
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_GRASS_0070B5703 }}
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: "upload"
###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
@@ -56,7 +56,7 @@ jobs:
id: closepullrequest
uses: Azure/static-web-apps-deploy@v1
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_GRASS_0070B5703 }}
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
action: "close"
+5 -2
View File
@@ -117,8 +117,11 @@ scripts/Deployment/
# Migration tooling (local only, vorerst) — cross-tenant storage/cert migration scripts
scripts/Migration/
# Infra (local only)
infra/
# Infra: tracked. The Bicep templates carry no credentials — every secret is a
# @secure() parameter with an empty default, injected at deploy time, and image
# pull runs on managed identity. Keep it that way: never commit a resolved secret
# value or a *.parameters.json holding one.
infra/*.parameters.json
# M2 — local raw session fixtures before anonymization (plan §4 M2)
tests/fixtures/enrollment-sessions-local/
+1 -1
View File
@@ -34,7 +34,7 @@ For detailed information about the system architecture, components, and data flo
## Documentation
Full documentation is available at **[autopilotmonitor.com/docs](https://www.autopilotmonitor.com/docs)**
Full admin documentation is available at **[docs.autopilotmonitor.com](https://docs.autopilotmonitor.com)**
## License
+2 -2
View File
@@ -53,7 +53,7 @@ try {
Write-Host " ✓ .NET SDK version: $dotnetVersion" -ForegroundColor Green
}
catch {
Write-Host " ✗ .NET SDK not found. Please install .NET 8 SDK." -ForegroundColor Red
Write-Host " ✗ .NET SDK not found. Please install .NET 10 SDK." -ForegroundColor Red
exit 1
}
@@ -145,7 +145,7 @@ Write-Host "============================================" -ForegroundColor Cyan
Write-Host ""
Write-Host "Output locations:" -ForegroundColor White
Write-Host " Agent: src\Agent\AutopilotMonitor.Agent.V2\bin\$Configuration\net48\" -ForegroundColor Gray
Write-Host " Functions: src\Backend\AutopilotMonitor.Functions\bin\$Configuration\net8.0\" -ForegroundColor Gray
Write-Host " Functions: src\Backend\AutopilotMonitor.Functions\bin\$Configuration\net10.0\" -ForegroundColor Gray
if ($IncludeWeb) {
Write-Host " Web UI: src\Web\autopilot-monitor-web\.next\" -ForegroundColor Gray
+88
View File
@@ -0,0 +1,88 @@
---
type: Concept
title: Agent Endpoint Migration — Config-Channel Re-Home
description: How the backend re-homes agents to a new API base URL (backend move or per-tenant region move) via MigrateToApiBaseUrl on the agent config response, and why HTTP redirects were rejected.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/EndpointMigration.cs
tags:
- agent
- backend
- config
- migration
- security
timestamp: 2026-07-18T00:00:00+02:00
---
# Problem
The agent's API base URL is compiled into the binary (`Constants.ApiBaseUrl`). No DNS/domain
indirection is possible in front of the API: mTLS client-cert auth does not survive a reverse
proxy (Front Door), and Flex Consumption custom-domain TLS is broken. A backend move (like the
2026-07-18 WEU→GWC subscription migration) therefore hard-cuts every deployed agent still
pointing at the old hostname. In-flight agents self-terminate safely (local MaxLifetime watchdog
+ CleanupService), but their sessions are lost and a per-tenant region move (EU→US) has no
mechanism at all.
# Mechanism
`AgentConfigResponse.MigrateToApiBaseUrl` — served on `GET /api/agent/config` (incl. the
bootstrap wrapper), the same control channel as the kill switch, fetched at every agent
process start.
Agent flow (`Program.RunAgent`, Phase 4):
1. Config fetched from the compiled-in (old) base URL.
2. Kill-switch check runs FIRST — **kill wins over migrate**.
3. `AgentRuntimeConfig.Resolve` detects the migration signal directly after the fetch and
returns early, BEFORE merge / binary-integrity / bootstrap-cleanup — none of those side
effects may run against a config from the backend being abandoned (a migration stub may
advertise stale agent hashes; a spurious `IntegrityCheckFailed` + force-update would be
harmful).
4. `Program.RunAgent` swaps `AgentConfiguration.ApiBaseUrl`, disposes the old client stack,
rebuilds Phase 3 (auth clients) and re-runs Phase 4 against the new URL —
`allowEndpointMigration: false`, so the new backend's config **cannot chain a second hop**.
5. Kill-switch check runs again on the NEW backend's config; startup continues normally
(Phase 5+ builds mTLS/telemetry clients from the updated base URL).
# Hardening (mirrors the kill switch)
* **Live fetch only** — the on-disk config cache strips `MigrateToApiBaseUrl` on write AND
read (`RemoteConfigService.CacheConfig` / `LoadCachedConfig`); `EndpointMigration.ResolveTarget`
re-checks `LastFetchOutcome == Succeeded` as defence-in-depth. A planted cache file can
never re-home an agent.
* **Allowlist validated on BOTH sides** — `AgentEndpointMigrationRules` (Shared, single
source): https only, host must end in `.azurewebsites.net` on a label boundary, no
path/query/userinfo/port. The backend refuses to serve an invalid admin value
(`AgentMigrateRejected` warning); the agent refuses to honour one even if the backend was
compromised.
* **One hop, no loops** — second Resolve pass never migrates; a target equal to the current
base URL is a no-op (steady state during the migration window).
# Operation
Global admin settings (`AdminConfiguration`, admin portal → Global Settings):
* `AgentMigrateApiBaseUrl` — global target, set on the backend being **abandoned**; clear it
after the migration window.
* `AgentMigrateTenantOverridesJson` — JSON object `tenantId → URL` for per-tenant moves.
An entry with an **empty string** pins that tenant against the global target (staged rollout).
Overrides win over the global value.
Delivery evidence: every served target logs `AgentMigrateServed` (Warning — reaches App
Insights) with tenant, serial, agent version and target. Agent-side, the re-home is logged as
`Endpoint migration: backend re-homed this agent from … to …`.
Reach limitation: agents that cannot complete an authenticated config fetch (e.g. cert-broken
devices) never see the signal — they fall back to the local MaxLifetime/emergency-break
lifecycle. During a migration window the old backend must keep running as a serving stub;
keeping the old `*.azurewebsites.net` name parked afterwards also prevents hostname
re-registration by third parties while stragglers drain.
# Citations
* `src/Shared/AutopilotMonitor.Shared/Services/AgentEndpointMigrationRules.cs` — shared validation rules
* `src/Shared/AutopilotMonitor.Shared/Models/Config/AgentConfigResponse.cs` — wire field
* `src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs` — `ResolveMigrateTarget` + serving (ConfigVersion 33)
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/EndpointMigration.cs` — agent-side evaluation
* `src/Agent/AutopilotMonitor.Agent.V2/Runtime/AgentRuntimeConfig.cs` — early return before merge/integrity
* `src/Agent/AutopilotMonitor.Agent.V2/Program.cs` — one-hop rebuild in `RunAgent`
* Tests: `AgentEndpointMigrationTests` (backend), `EndpointMigrationTests` + `RemoteConfigServiceCacheHardeningTests` (agent)
+1
View File
@@ -7,5 +7,6 @@ How the on-device agent works:
* [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine.
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
Reading order for newcomers: overview → decision engine → persistence.
+100
View File
@@ -0,0 +1,100 @@
---
type: Concept
title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs
description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
tags:
- agent
- esp
- reboot
- policy-attribution
- second-sign-in
timestamp: 2026-07-20T00:00:00+02:00
---
# MDM Reboot Coalescing
Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog
(`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced
reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a
**second sign-in screen** before AccountSetup. Invisible in the Intune console; most
admins live with it, not knowing that reassigning the profiles to user groups removes the
reboot entirely. Source research: Patch My PC,
["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/).
The session already counts reboots (`system_reboot_detected` → `session.rebootCount`);
this feature adds the **cause attribution**.
# Verified signal facts (session b2e890c1, 2026-07-20)
* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`,
one record per URI, verified description shape:
`The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).`
Records arrive in sub-second bursts during policy sync.
* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs
applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the
verified session the flags fired during AccountSetup (after DeviceSetup completed) —
**no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that
claims "reboot incoming" on every 2800 is therefore wrong.
* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the
ROUTINE bootstrap marker on every enrollment:
`Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'`
— the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a
reboot is needed). A substring match on it is a false positive by construction. If a
real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be
added then.
# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`)
Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup
backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then
flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst:
* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as
reboot-required: … (+N more)") — the raw event never claims a reboot.
* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct),
`firstRebootUri` (record-time order, drives rule interpolation), `recordCount`,
`unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill),
fail-soft `omadmRebootRequiredFlag` probe of
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown).
* `Timestamp` = earliest record time (backfilled bursts land at their historical instant);
`ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a
reboot that kills the agent).
* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the
coalesced reboot kills the process before the debounced emit, nothing is marked — the
post-restart backfill re-reads and emits the burst. A flushed burst is never repeated.
URI extraction: EventData values starting with `./` preferred, permissive regex over the
description as fallback (trailing punctuation trimmed — the verified text wraps the URI in
parens and ends with a period). Unparsed records still count into the aggregate.
Config (internal steering only, no admin UI — agent-native collector, NOT an
`IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true),
`Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35.
# Advisory — ANALYZE-ESP-005
The reboot **claim** lives here, and only here. Required conditions (BOTH):
1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation)
2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot**
Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional
confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 /
threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or
exclude the Autopilot device group), or accept + communicate the restart.
# Non-goals / invariants
* Event type **deliberately distinct from `system_reboot_detected`** — the backend
RebootCount matches that string exactly in two places
(`EventIngestProcessor.Classification` + terminal recount); attribution must never
inflate the reboot count.
* No session-level field / SessionInfoCard change in this iteration (follow-up candidate).
* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry
the actually-triggered URIs) are consumed.
# Citations
* [Patch My PC — Avoid the second sign-in screen after Autopilot](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/)
* [Microsoft Learn — Enrollment Status Page tracking](https://learn.microsoft.com/en-us/autopilot/enrollment-status)
+95
View File
@@ -0,0 +1,95 @@
---
type: Concept
title: Business Timestamps vs. the Azure Tables System Timestamp
description: Why the system Timestamp is never authoritative, how the 2026-07-18 migration exposed that, and the OccurredUtc + RowKey-decode compensation (mappers, RowKey-range date filters, retention, backfill).
resource: /src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs
tags:
- backend
- table-storage
- audit-log
- ops-events
- events
- migration
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Azure Table Storage's `Timestamp` is a **system property**: the server ignores any supplied
value on write and stamps the row's *write* time; every row rewrite (storage-account
migration, MERGE) resets it. Three tables nevertheless treated it as the business event
time — writes like `["Timestamp"] = evt.Timestamp` were silent no-ops, unnoticed because
write time ≈ event time in normal operation.
The 2026-07-18 EU storage migration rewrote every row, so all migrated rows in
**AuditLogs**, **OpsEvents**, and **Events** read as "2026-07-18 ~13:58 UTC": displays were
wrong, date-window filters matched the wrong rows, and retention froze (nothing older than
the migration moment) with a mass-delete cliff one retention period later.
# Rule
**The system Timestamp is never authoritative.** Every table that needs an event time
stores it in the custom column `OccurredUtc` (`BusinessTimestamp.OccurredUtcColumn`).
Custom columns are copied verbatim by `scripts/Migration/Copy-TableStorage.ps1`, so future
migrations preserve chronology. Migration checklist: any table whose reads/filters depend
on a timestamp must carry it as a custom column — audit this before every storage move.
# Recovery sources per table
The original time survives migrations inside the RowKey:
| Table | RowKey scheme | Decodable |
|---|---|---|
| AuditLogs | `!{MaxTicks-ticks:D19}_{guid:N}` (since 2026-05-06) | Yes; legacy bare-GUID rows (pre-2026-05) are lost |
| OpsEvents | `{MaxTicks-ticks:D19}` | Yes, all rows |
| Events | `{ts:yyyyMMddHHmmssfff}_{seq:D10}` (sanitized agent time) | Yes, all rows (ms precision) |
# Mechanism
`BusinessTimestamp` (Functions/Helpers) is the single source of truth: RowKey decoders,
the OData clause builders, and `GetUtcDateTime` (datetime columns materialize as
`DateTimeOffset` *or* `DateTime` depending on payload shape — always read via this helper).
1. **Read mappers** resolve `OccurredUtc` → RowKey decode → system Timestamp (last
resort; only audit legacy GUID rows end there). Resolvers:
`TableStorageService.ResolveAuditTimestamp`, `TableOpsEventRepository.ResolveTimestamp`,
`TableStorageService.ResolveEventTimestamp` (also used by
`GetEarliestSessionEventTimestampAsync`, which feeds StartedAt/Duration reconciliation —
a system-Timestamp read there would rewrite migrated sessions' derived fields).
2. **Date-window filters and retention use RowKey ranges**, not property comparisons:
OData excludes rows missing a compared property (an `OccurredUtc` filter would drop
every not-yet-backfilled row), and RowKey ranges are index-backed. Boundary semantics
are tick-exact equivalents of the old `Timestamp ge/le` clauses.
**Trap:** audit legacy GUID RowKeys (hex first char) sort *after* all `!`-rows, so
every lower bound `RowKey ge '!…'` must be paired with `RowKey lt '"'`
(`AuditTimeEncodedUpperBound`) — without the guard, a retention sweep would delete all
legacy rows. Pinned by `BusinessTimestampFilterClauseTests`.
3. **Writes** in all three tables now set `OccurredUtc`; the no-op `["Timestamp"]`
entries were removed.
4. **Backfill** (`POST /api/maintenance/backfill-occurred-utc`, GlobalAdminOnly, dry-run
by default, `table=audit|ops`, batched via `maxRows`/`continuation`): MERGE that adds
only `OccurredUtc` decoded from the RowKey — deterministic and idempotent. **Events
rows are deliberately never backfilled** (read-path decode is sufficient; no bulk
mutation of enrollment evidence). Note: a MERGE resets the row's system Timestamp —
irrelevant once nothing depends on it.
5. **Audit retention** runs two passes until ~2027-01: RowKey-range for `!`-rows plus a
`RowKey ge '0' and Timestamp lt …` pass that ages legacy GUID rows out via their frozen
migration-date system Timestamp (remove after 2027-03-01).
# Consequences
* Audit legacy GUID rows have "unknown date" semantics: excluded from date-filtered
views, shown with the (wrong but honest) 2026-07-18 fallback in unfiltered views.
* Raw endpoints (`/api/raw/*`, `query_table`) intentionally keep showing the raw system
Timestamp — they are storage-state inspectors, not business views.
* Event timestamps for pre-cutover rows resolve to the RowKey's sanitized agent time
(ms precision); `ReceivedAt` (server ingest time, rows since 2026-03-19) and
`OriginalTimestamp` (pre-clamp agent time, clamped rows only) remain separate fields.
# Citations
* `src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs` — decoders + clause builders
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Maintenance.cs` — audit write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/DataAccess/TableStorage/TableOpsEventRepository.cs` — ops write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Sessions.cs` — event writes, `ResolveEventTimestamp`, earliest-event read
* `src/Backend/AutopilotMonitor.Functions/Services/OccurredUtcBackfillService.cs` + `Functions/Admin/BackfillOccurredUtcFunction.cs` — backfill
+15
View File
@@ -17,6 +17,21 @@ https://docs.autopilotmonitor.com (separate repository).
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
# Rules
* [Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
* [Gather Rule Guardrails](rules/gather-rule-guardrails.md) - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
# Backend
* [Business Timestamps](backend/business-timestamps.md) - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
# MCP Server
* [MCP Docs Corpus](mcp/docs-corpus.md) - How the published customer documentation becomes the `search_docs` corpus: CRLF-safe heading chunking, build-time embedding as a third precomputed section, and why a docs edit needs an MCP redeploy.
# Security & Identity
+30
View File
@@ -1,5 +1,35 @@
# Log
## 2026-07-22
* **Creation**: Added `rules/gather-rule-guardrails.md` — the gather-rule allowlists are enforced only on the agent (the backend validates no `collectorType`, `target`, or `parameters`), so a collector that forgets to call its guard *is* the vulnerability. Written because two had: `LogParserCollector` read any path in `rule.Target`, bypassing even the `C:\Users` hard block that every other file collector honours, and `EventLogCollector` read any channel including `Security` — while the customer-facing documentation already promised both were allowlisted. Both now guard, with `logparser` checking the directory for wildcard targets (`Path.GetFullPath` throws on `*`) and re-checking each resolved path. Added an event-log channel allowlist (`eventLogChannels`) plus hard-blocked channels held in C# rather than JSON, so a parse error cannot lift them. Also fixed the command timeout, which was inert: `ReadToEnd()` blocks until the pipe closes, so `WaitForExit(30000)` was only reached after the process had already exited and a hung command pinned its worker forever — now read asynchronously and killed at the timeout.
* **Update**: Added `.github/workflows/ci.yml` — the full suite (backend, agent V2 Core, DecisionCore, SummaryDialog, web, MCP) now runs on every pull request and every push to main, plus a job that fails if the generated guardrail files drift from `rules/guardrails.json`. Previously **no** workflow ran tests on a PR: the deploy workflows build without testing, and `build-agent.yml` tested only `DecisionCore.Tests` on manual dispatch — while `trust/security-faq.md` already told customers that automated tests run in CI. Added CodeQL (C# and TypeScript) and Dependabot (nuget, npm ×2, actions). Marked `OfficeBinaryWatcherTests` `[Collection("SerialThreading")]`: its bounded re-probe races the shared ThreadPool and failed once under full-suite load.
## 2026-07-21
* **Update**: `search_docs` and `search_knowledge` now state in their tool descriptions that queries must be in English. Both corpora and `all-MiniLM-L6-v2` are English-only, and the failure is total rather than gradual — a German question returns zero results, not weak ones. The caller is a language model, so translating the question before the call and answering in the user's language costs nothing; a multilingual embedding model would be the alternative but is much larger and would have to be measured against the eval harness first, since it changes English retrieval too.
* **Update**: Added `scripts/eval-docs-search.ts`, an offline retrieval evaluation harness (2465 queries in three classes: chunk headings as a regression detector, rare identifiers, and hand-written questions as the reality anchor), and used it to close a third `search_docs` gap found during the live test. The keyword fallback only fired when semantic search returned *fewer* than `topK`; a pasted identifier like `WOW6432Node` instead returned three chunks at ~0.29 that did not contain the word while the one chunk that did was never considered. It now also fires on a weak semantic head (<0.35), letting exact matches take the front while keeping one semantic result. The threshold is measured, not guessed: 0.35 fixes 335 queries and breaks 0 (identifier recall 66.2% → 81.1%), while 0.40 fixes more but starts demoting hand-written questions. The measurement also resized the problem — identifier queries were only succeeding 66.2% of the time, so the reported case was one of roughly 750.
* **Update**: Closed the two measured recall gaps in `search_docs` (`mcp/docs-corpus.md`). (1) **Token windows** — 87 of 324 chunks exceeded the model's 256-token context and 13% of all tokens never reached the ranking; a character budget cannot fix this (the corpus ranges 2.43–5.81 chars/token), so long text is now embedded as 250-token windows with 40 overlap and scored by its BEST window (`embeddings: number[][]`, max not mean). Token coverage 87% → 100%; "proxy configuration blocking app downloads" went from absent to 0.559 at rank 1; the ten reference queries kept byte-identical scores. Costs +31% vectors, index 2.63 → 3.12 MB. The rules corpus gained the same treatment (128 rules → 172 vectors). (2) **Keyword fallback** — when the semantic pass returns fewer than `topK`, results are topped up via literal `lexicalMatch` on distinctive query terms, requiring two matching terms for corroboration (one incidental substring match returned a page about LLM providers for a sourdough query). Fuzzy matching was measured and rejected: at a threshold low enough to surface "RealmJoin" it invents 5 hits for "SCEPman", which the corpus never mentions. Also sharpened the tool description with the measured context argument for preferring it over fetching the docs site.
* **Creation**: Added `mcp/docs-corpus.md` (new `mcp/` section) — the MCP server gains `search_docs`, a third semantic corpus built from the published customer documentation repository (`autopilotmonitor-docs`, checked out into the Docker build context by the deploy workflow). Documents the h2/h3 + `<details>` chunking with breadcrumb prefixes, the two deliberate size exceptions (tables never split, unbroken changelog lists split at line level), the CRLF precondition (`.` does not match `\r` in JS — headings silently stop matching, which produced 286 wrong-but-plausible chunks before it was found), the optional `docs` section in `search-index.json` and its validation asymmetry, embedding rounding to 6 decimals, and the cross-repo coupling: a docs edit reaches clients only after an MCP redeploy, made checkable via `DOCS_COMMIT` on `/health`. Container resources raised to 0.5 vCPU / 1.0 GiB for the cold-start query-embedder load.
## 2026-07-20
* **Update**: `rules/gather-rule-phase-scoping.md` — the gather-rule form now shows the phase-scope and emit-mode controls only where they can change behavior (`supportsPhaseScope` / `supportsEmitMode`), nulls hidden-control state out of the payload, and rejects a scope mode selected but left unfilled. Offering all three "when" controls unconditionally allowed combinations that could only suppress a one-shot rule, plus a silent no-op state.
* **Update**: Extended `rules/gather-rule-phase-scoping.md` with the `phase_exit` trigger — the closing bookend to `phase_change`, for one-shot collection when a phase is LEFT (e.g. "state at the end of Account Setup"). Documents the evaluation-before-phase-advance rule (triggerPhase match and scope gate see the phase being left), firing into `Failed`, the disjoint `exit:` dedup key space, and why no ConfigVersion bump is needed (no new field — old agents simply never fire the rule).
* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.
* **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated.
* **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.
* **Creation**: Added `rules/gather-rule-phase-scoping.md` (new `rules/` section) — gather rules gain `activePhases` (run only during listed phases), `activeFromPhase` (sticky latch from a phase onwards, never via Failed) and `emitMode: "on_change"` (canonical-hash result dedup with `suppressedPolls` observability), ConfigVersion 34. Includes the prerequisite backend fix: the custom-rule toggle PUT no longer wipes the stored rule definition (partial-merge in `GatherRuleService.UpdateRuleAsync`).
## 2026-07-18
* **Creation**: Added `agent/endpoint-migration.md` — config-channel agent re-home (`MigrateToApiBaseUrl` on `GET agent/config`, ConfigVersion 33): backend or per-tenant API-URL moves without DNS indirection (impossible due to mTLS/Flex-TLS). Live-fetch-only with cache strip, `AgentEndpointMigrationRules` allowlist validated on both sides, one hop, kill-switch wins. Built as the lesson from the 2026-07-18 WEU→GWC cutover, which hard-cut all in-flight agents.
## 2026-07-17
* **Creation**: Added `agent/build-counter-blob.md` — agent build numbers now come from a shared public counter blob (`buildcounter-v{N}.txt`, reserve-before-build with ETag-CAS) so local `build.ps1` and CI `build-agent.yml` builds can never mint the same number. Documents the CI parity fixes (SummaryDialog.exe.config in the ZIP, EXE-vs-manifest version check, hash-oracle update gated on stable cutover) and the Sigstore provenance attestation on CI-built ZIPs.
+305
View File
@@ -0,0 +1,305 @@
---
type: Concept
title: MCP Docs Corpus — search_docs and the Cross-Repo Build Coupling
description: How the published customer documentation becomes a third semantic search corpus in the MCP server — CRLF-safe heading chunking, build-time embedding, the corpus hash guard, and why editing the docs does not update a running server.
resource: /src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts
tags:
- mcp
- search
- embeddings
- documentation
- build
timestamp: 2026-07-21T19:30:00+02:00
---
# Purpose
The MCP server answers two different kinds of question, and they need two
different corpora.
* *"Why did this enrollment fail?"* → analysis rules, gather rules, IME log
patterns. That is `search_knowledge`, backed by `knowledge-base.ts`.
* *"How do I deploy the agent?" / "Where is my data stored?"* → the published
customer documentation. That is `search_docs`, backed by `docs-corpus.ts`.
They are deliberately **separate indexes**. Merged into one, they would compete
for the same `topK` and dilute each other: a rules query would surface prose
pages, and a product question would surface rule definitions.
# Schema
## Corpus source
The corpus is the `autopilotmonitor-docs` repository — a public OKF bundle
published at `docs.autopilotmonitor.com`. It is **not** vendored into this repo.
The deploy workflow checks it out into `mcp-docs/` of the Docker build context;
the Dockerfile copies it to `/app/docs` in both the build and the final stage
(byte-identical, or the boot-time corpus hash rejects the precomputed index).
`SUMMARY.md` and `log.md` are skipped as navigation scaffolding. `index.md` is
kept — it is the documentation map and answers "what documentation exists" — but
gets no `url`, because the bundle deliberately keeps it out of `SUMMARY.md` and
GitBook therefore never renders it.
## Chunking
A whole page cannot be one document: `trust/security-faq.md` is 36 KB, while
MiniLM-L6-v2 truncates at ~256 tokens. An un-chunked page would silently embed
only its first paragraph.
Chunks are cut at **h2/h3 headings**, which in this bundle are also the semantic
boundaries — `security-faq.md` carries 37 h3s, one per question.
`<details><summary>Q</summary>` blocks (used by `troubleshooting/faq.md`) are
treated the same way, with the summary as the section title. Each chunk is
prefixed with a `Page Title › Section › Subsection` breadcrumb so a hit lifted
out of the middle of a page is self-describing to both the embedder and the
reader.
Two deliberate exceptions to the ~1500-character budget:
* **Markdown tables are never split.** A table broken across chunks leaves
`| --- |` rows with no header — unreadable in a result and meaningless to
embed. Tables carry real answers here (the role matrix, the settings
reference), so an over-long table is allowed to exceed the budget.
* **Unbroken lists split at line level.** The changelog pages are one bullet
list per release with no blank lines, so paragraph splitting cannot touch
them; left whole, everything past the model's window would be truncated away
and become unsearchable.
Short sections are **kept**, not filtered. A one-line answer ("Is it free?" →
"Yes, the Community plan is free") is often the most valuable hit, and the
breadcrumb supplies the context the short body lacks. Only structural residue
(under 25 alphanumeric characters) is dropped.
## CRLF is a correctness precondition
The docs repo is authored on Windows and ships CRLF. In JavaScript `.` does not
match `\r`, so a pattern like `/^(#{2,3})\s+(.*)$/` matches **nothing** on a CRLF
file: `(.*)` stops before the `\r` and `$` never reaches end-of-string.
This was observed during development, and the failure mode is what makes it
dangerous: no crash, no error — headings and frontmatter simply stop being
recognized and every page silently degrades to unlabelled paragraph splits. The
first working build produced 286 plausible-looking chunks that were almost all
wrong; after normalizing, `security-faq.md` went from 29 blind splits to 46 real
question sections.
`normalizeNewlines()` runs before any parsing. Do not remove it, and do not
"fix" a future parsing bug here by adding `\r?` to one pattern.
## Token windows: why a chunk can hold several vectors
Chunking at heading boundaries keeps sections coherent, but it does not bound the
result by *tokens* — and it cannot. Measured on this corpus the ratio ranges from
**2.43 to 5.81 characters per token**, a factor of 2.4, so no character budget maps
reliably onto the model's 256-token window. A budget safe for the worst case
(~620 characters) would shred prose and tables alike.
Left alone, 87 of 324 chunks exceeded the window and **13% of all tokens never
reached the ranking** — silently, since the tokenizer truncates without complaint.
The effect was concrete: in `built-in-rules.md :: Apps` (734 tokens), the strings
`ANALYZE-APP-013`, `ANALYZE-OFFICE-001`, `ANALYZE-CORR-003` and `Click-to-Run` all
sit past the cutoff, so *"proxy configuration blocking app downloads"* did not
retrieve that section at all.
So the *chunk* stays whole and the *embedding* is split instead:
`VectorSearchProvider.index()` cuts long text into 250-token windows with 40 tokens
of overlap and embeds each. A document therefore carries `embeddings: number[][]`,
and `search()` scores it by its **best** window.
- **Max, not mean.** Averaging window vectors would blur a long section's several
topics into one mediocre direction — the same dilution that makes large context
windows worse than chunking. Max keeps a document findable by any one thing it
discusses.
- **The tokenizer is a build-time dependency only.** `index()` runs during the
Docker build; production boots from precomputed vectors and never loads it. It is
also only the vocabulary, not the ONNX weights (~93 ms).
- Cost: 324 chunks → 424 vectors (+31%), index 2.63 → 3.12 MB, and 424 instead of
324 dot products per query — still microseconds.
- The rules corpus benefits too, unplanned: 128 rules now carry 172 vectors.
Measured effect: token coverage 87% → **100%**, *"Office Click-to-Run install never
finished"* 0.318 → **0.442**, *"proxy configuration blocking app downloads"* absent
→ **0.559 at rank 1**. The ten reference queries kept byte-identical scores — their
chunks are short, hence one window, hence the same vector as before.
## Keyword fallback
Semantic ranking alone leaves a gap: proper nouns embed poorly. `RealmJoin` occurs
in 3 chunks yet scores under the threshold.
Fuzzy matching is the wrong remedy, and the corpus says so. At a threshold low
enough to surface `RealmJoin` (0.28), Fuse also returns 5 hits for `SCEPman` — a
word this corpus contains **zero** times. Literal containment has neither problem.
So when the semantic pass returns fewer than `topK`, `search_docs` tops the list up
via `lexicalMatch`, the substring scan every provider already exposes:
- Query terms are filtered to distinctive ones (≥4 characters, non-stopword, max 6).
- A hit needs **corroboration**: at least two matching terms, unless the query has
only one. Without this rule, "how do I bake sourdough bread" matched a page about
LLM providers, because "bake" is a substring of "baked into the image".
- Hits are **appended, never merged**: they carry no cosine similarity, so semantic
results keep the head of the list and keyword hits fill the tail, flagged
`matchType: "keyword"` with `keywordFallback` in the response.
### Language
The corpus and `all-MiniLM-L6-v2` are both English-only, and the failure is total
rather than gradual: "wo werden meine Daten gespeichert" returns **zero** results,
not weak ones, because the query vector lands nowhere near any English chunk.
The fix is a sentence in the tool description telling the caller to query in English
and answer in the user's language — the caller is a language model, so translating
the question before the call costs nothing. A multilingual embedding model would
handle it natively but is substantially larger and would need to be measured against
`scripts/eval-docs-search.ts` first, since it changes English retrieval too.
`search_knowledge` carries the same note for the same reason.
### When it fires
Two triggers, and the second one exists because the first was not enough:
1. **Fewer than `topK` semantic hits** — the obvious case.
2. **A weak semantic head** (best score below `WEAK_SEMANTIC_SCORE`, 0.35). Topping
up only a short list misses the more common failure: a pasted identifier like
`WOW6432Node` returns three chunks at ~0.29 that do not contain the word, while
the single chunk that does never gets considered. When nothing semantic is
convincing, exact matches take the front — keeping at least one semantic result,
so a weak-but-correct hit is never pushed out entirely.
The threshold was chosen by measurement, not intuition. `scripts/eval-docs-search.ts`
scores strategies over 2465 queries in three classes — every chunk heading
(known-item, a regression detector), rare identifiers (the class this fallback
serves), and hand-written questions (the reality anchor, not derived from the
corpus):
| strategy | known-item | literal | handwritten MRR | fixed | broken |
| --- | --- | --- | --- | --- | --- |
| append-only (previous) | 71.9% | 66.2% | 1.000 | — | — |
| weak < 0.30 | 71.9% | 72.6% | 1.000 | 142 | 0 |
| **weak < 0.35** | **73.7%** | **81.1%** | **1.000** | **335** | **0** |
| weak < 0.40 | 76.3% | 88.2% | 0.944 | 502 | 3 |
| weak < 0.45 | 77.2% | 93.0% | 0.889 | 613 | 5 |
0.35 is strictly dominant: better in every class, worse in none. Higher thresholds
keep repairing more identifier queries but start demoting hand-written questions —
the class that most resembles real use — so the gain is not free past that point.
The measurement also corrected the size of the problem: identifier queries were
succeeding only 66.2% of the time, so `WOW6432Node` was not an outlier but one of
roughly 750 such failures.
## Build-time embedding
Like the rules corpus, docs are embedded once at Docker build time by
`precompute-embeddings.js <rulesDir> <outFile> [docsDir]`, into a third
`PrecomputedSection` (`docs`) of `search-index.json`. The container never embeds
a corpus at boot — only the incoming query.
The `docs` section is **optional** in the file format, which is what makes a
local checkout without the docs repo work: `validatePrecomputedIndex` skips it
when the booting server loaded zero doc chunks. The asymmetry is intentional —
if the server *has* a docs corpus and the file does *not*, that is staleness and
is rejected like any other mismatch.
Embeddings are rounded to 6 decimals on serialize. A float64 component
serializes to ~20 characters of JSON, so 384 dims cost ~7.8 KB per vector;
rounding roughly halves the index file, which is `JSON.parse`d on every
scale-to-zero cold start. The induced L2 norm error (~1e-5 over 384 dims) sits
two orders of magnitude inside `UNIT_NORM_EPSILON`.
## Registration
`search_docs` is registered for **every role**, unlike the platform tools: this
is public documentation carrying no tenant data, and a tenant user asking "how do
I deploy the agent" has as much business here as a Global Admin. It is skipped
entirely when the corpus is empty, so a doc-less build advertises no broken tool.
# Cold start
`minReplicas: 0` means every idle period ends in a cold start someone waits through.
Measured from outside: **24.5 s** to first byte after `ScaledToZero`, against **60 ms**
warm.
Where those seconds go could not be answered from outside the process. Log Analytics
stamps every console line of one ingestion batch with the same time, so the boot
sequence collapses to a single instant however it is queried. The server therefore
times itself — `[boot +Nms]` marks from `process.uptime()`, which includes Node's own
startup and module loading.
Measured in production on a real cold start (31.5 s to first byte that time):
```
+244ms node started (startup + module loading)
+431ms corpora read from disk (+187ms)
+491ms index read, validated, hydrated (+60ms <- for 3.12 MB)
+504ms listening
+677ms query embedder warm (background)
```
**The application accounts for half a second of it.** The other ~31 s — 98% — is
Container Apps activation: KEDA reacting to the request, scheduling a pod, mounting
the 236 MB image, starting the container. The container also boots *faster* than a
development laptop (504 ms against 1273 ms), so nothing here is CPU-starved.
Three conclusions, and two of them overturn assumptions this file previously stated:
- **Index size is irrelevant to startup.** 3.12 MB hydrates in 60 ms. Growing the
corpus is effectively free, and the embedding rounding earns its keep on file size
alone, not on boot time.
- **Raising the container to 0.5 vCPU was not justified by cold start.** The stated
reason was the embedding model load; prebaked, that is 173 ms and runs in the
background, where only the first *search* could ever wait on it. The extra vCPU may
still help concurrent requests, but the cold-start argument does not survive
measurement.
- **The only real lever is `minReplicas: 1`**, which removes activation entirely at
roughly 7× the monthly free consumption grant. Shrinking the image would shave the
mount, but that is a fraction of platform time, not the bulk of it.
# Cross-repo coupling
**Editing the documentation does not update a running MCP server.** The corpus is
baked into the image, so only re-running the *Deploy MCP Server* workflow picks
up doc changes.
That coupling is made checkable rather than memorable: the build passes the docs
checkout SHA as `--build-arg DOCS_COMMIT`, and `/health` reports
`docs: { commit, chunks, sections }`. Comparing that commit against
`git rev-parse HEAD` in `autopilotmonitor-docs` answers "is the deployed
documentation stale?" directly.
Automatic redeploy on a docs push was considered and rejected: it would need a
PAT secret in the docs repo and would turn every typo fix into a container
deployment.
# Examples
Retrieval measured against the real bundle (323 chunks, all-MiniLM-L6-v2):
```
"where is my data stored" → 0.566 troubleshooting/faq.md :: General › Where is my data stored?
"how do I deploy the agent with Intune" → 0.819 troubleshooting/faq.md :: Setup & Agent › How do I deploy the agent?
"which roles exist and what can they do" → 0.500 trust/security-faq.md :: Identity and Access › What roles exist?
"how do I set up Teams notifications" → 0.592 reference/settings.md :: Tenant › Notifications
"what firewall URLs do I need to allow" → 0.374 reference/network-endpoints.md :: (intro)
```
The 0.25 `minScore` default is tuned to that observed band — short keyword
queries on all-MiniLM score low, and a relevant-but-marginal hit lands around
0.34.
# Citations
* `/src/McpServer/autopilot-monitor-mcp/src/docs-corpus.ts` — loader, frontmatter
parser, chunker
* `/src/McpServer/autopilot-monitor-mcp/src/tools/search.ts` — `search_docs`
registration
* `/src/McpServer/autopilot-monitor-mcp/src/precomputed-index.ts` — `docs`
section and its validation asymmetry
* `/src/McpServer/autopilot-monitor-mcp/Dockerfile` — `mcp-docs/` copy,
`DOCS_COMMIT`
* `/.github/workflows/deploy-mcp.yml` — docs checkout and SHA capture
* `/src/McpServer/autopilot-monitor-mcp/scripts/eval-docs-search.ts` — retrieval
evaluation harness; run it before changing anything about ranking
* [MCP OAuth Flow](../mcp-oauth-flow.md) — the other MCP concept document
+81
View File
@@ -0,0 +1,81 @@
---
type: Concept
title: Gather Rule Guardrails
description: >-
How gather-rule collectors are fenced in — the allowlists in rules/guardrails.json,
the hard blocks that survive unrestricted mode, and why every collector must call a
guard rather than trusting rule.Target.
resource: src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather
tags: [gather-rules, security, guardrails, agent]
timestamp: 2026-07-22
---
# Concept
A gather rule is authored by a **tenant administrator** through `POST /api/rules/gather` and executed by the agent as **SYSTEM** on an enrolling device. The backend does not validate `collectorType`, `target`, or `parameters` at all — it only enforces scope/emit-mode fields, a 1 MB body cap, and that the rule lands in the caller's own tenant partition. Everything that decides *what a rule may reach* is therefore enforced **on the agent**, in `GatherRuleGuards`.
This is the security boundary. The portal's `ValidationIndicator` is a convenience: it does not block submit, and the API is reachable directly.
# Schema
Allowlists come from `rules/guardrails.json`, embedded into the agent assembly as
`AutopilotMonitor.Agent.V2.Core.Resources.guardrails.json` (see the `EmbeddedResource`
item in `AutopilotMonitor.Agent.V2.Core.csproj`) and regenerated into the portal's
`utils/guardrails.generated.ts` by `node rules/scripts/combine.js`.
| Guard | Source list | Match rule |
| --- | --- | --- |
| `IsRegistryPathAllowed` | `registryPrefixes` | prefix, boundary `\` or end |
| `IsFilePathAllowed` | `filePrefixes` | `Path.GetFullPath` first, then prefix + boundary |
| `IsWmiQueryAllowed` | `wmiQueryPrefixes` | prefix, boundary whitespace or end |
| `IsCommandAllowed` | `allowedCommands` | **exact** match, trimmed — never prefix |
| `IsEventLogChannelAllowed` | `eventLogChannels` | prefix, boundary `/` or end |
**Hard blocks live in code, not in the JSON**, so a parse error cannot lift them — on a
parse failure the allowlists fall back to empty and everything is blocked (fail-closed),
but the hard blocks must hold regardless of configuration:
* `BlockedUsersPrefix` (`C:\Users`) and `AdditionalHardBlockedPathPrefixes`
(`C:\Windows\System32\config`) — exception only for `%LOGGED_ON_USER_PROFILE%`
under `AppData\Local` / `AppData\Roaming`.
* `HardBlockedCommandPatterns` — download, user/group creation, boot config,
persistence, destructive operations; plus `MaxCommandLength`.
* `HardBlockedEventLogChannels` — `Security` (audit trail of user behaviour) and the
PowerShell channels (script-block logging routinely contains secrets in clear text).
`rules/guardrails.json` mirrors the blocked lists (`blockedFilePrefixes`,
`blockedEventLogChannels`) **for display only**. Enforcement is the C# constant.
# Examples
Every collector must call its guard and return `context.EmitSecurityWarning(...)` on
refusal — that emits the `security_warning` event that makes a block visible in the
timeline instead of failing silently.
```csharp
// FileCollector / JsonCollector / XmlCollector
if (!GatherRuleGuards.IsFilePathAllowed(filePath, context.UnrestrictedMode, userProfilePath))
return context.EmitSecurityWarning(rule, "file", filePath);
```
Two collectors were added to this contract on 2026-07-22 after shipping without it:
* **`LogParserCollector`** read any path in `rule.Target`, bypassing even the `C:\Users`
hard block. It cannot guard `rule.Target` directly — a wildcard filename makes
`Path.GetFullPath` throw — so it guards **the directory** when the filename contains
`*`/`?`, and re-guards **each resolved path** after wildcard expansion (a junction can
surface a file the directory check did not cover).
* **`EventLogCollector`** read any channel, including `Security`.
`ImeLogPathOverride` comes only from the local `--ime-log-path` CLI flag, never from
remote config. The operator using it is already a local admin, so it relaxes the
allowlist exactly the way unrestricted mode does — hard blocks still apply.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleGuards.cs`
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleContext.cs` — `EmitSecurityWarning`
* `src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/Gather/GatherRuleCollectorGuardTests.cs`
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — what the backend does *not* validate
* `rules/guardrails.json`, `rules/scripts/combine.js`
* Customer-facing counterpart: `autopilotmonitor-docs/rules/gather-rules.md` (§ Security guardrails)
+136
View File
@@ -0,0 +1,136 @@
---
type: Concept
title: Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change
description: How gather rules are restricted to enrollment phases (activePhases / activeFromPhase), collect once at a phase boundary (phase_change / phase_exit), and how emitMode "on_change" suppresses repeated identical results.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs
tags:
- agent
- backend
- gather-rules
- config
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Interval gather rules (e.g. a registry poll every 60 s) ran during **all** enrollment phases
and emitted an event on **every** tick — including `exists=false` events while the target key
did not exist yet. A rule needed only from Account Setup onwards produced hundreds of
identical noise events per session and wasted collector runs.
# Schema
Three opt-in fields on `GatherRule` (Shared model, `rules/schema/gather-rule.schema.json`,
delivered to the agent via `GET agent/config`; ConfigVersion 34). Absent fields = legacy
behavior — full back-compat.
| Field | Type | Semantics |
|---|---|---|
| `activePhases` | `string[]` \| null | Rule runs only while the current phase is in the list. Null/empty = unrestricted. |
| `activeFromPhase` | `string` \| null | Sticky latch: activates when the phase first reaches the threshold, then stays active for the rest of the session — including through `Failed`. Null = unrestricted. |
| `emitMode` | `"always"` \| `"on_change"` \| null | `on_change`: poll on the trigger cadence, emit only when the collected result changes. Null/`always` = emit every collection (legacy). |
Canonical phase tokens are the `EnrollmentPhase` enum **names** `Start`(0) …`Complete`(7).
Backend validation (`GatherRulesFunction.ValidateScopeAndEmitMode`) rejects `Unknown`/`Failed`,
numeric tokens, unknown emit modes, and both scope fields set at once (400). The agent
defensively prefers `activePhases` if both arrive anyway.
# One-shot collection at phase boundaries
Scoping answers "while *may* this rule run". The trigger answers "when does it fire". For
one-shot collection the two phase triggers are the bookends of a phase:
| Trigger | Fires |
|---|---|
| `phase_change` | Once when `triggerPhase` is **entered** (dedup key `{ruleId}\|{phase}`). |
| `phase_exit` | Once when `triggerPhase` is **left** (dedup key `exit:{ruleId}\|{phase}`). |
Empty `triggerPhase` means every transition. Both keys live in the same
`_phaseRulesExecuted` set; the `exit:` prefix keeps the two key spaces disjoint, so an
enter-rule and an exit-rule on the same phase never cannibalise each other's slot.
`phase_exit` semantics that are easy to get wrong:
* It is evaluated **before** `_currentPhase` advances, so both the `triggerPhase` match and
the phase-scope gate see the phase being **left**. A rule scoped to `DeviceSetup` therefore
still fires on its own exit, even though the phase being entered is outside that scope.
* A transition into `Failed` **does** fire the exit rules of the failing phase — capturing
state at the failure boundary is the point.
* Exit of `Unknown` never fires (nothing was entered yet), and a repeated identical phase
signal is not a transition.
* An exit rule on `Complete` only fires if something transitions *away* from Complete. For
"at the end of enrollment", use `phase_change` on `Complete` or `on_event` with
`enrollment_complete`.
* Old agents do not know the trigger value and simply never fire the rule (silent no-op) —
which is why `phase_exit` needs no ConfigVersion bump: no new field is transported.
# Scope semantics (agent, `GatherRuleExecutor`)
* Scope applies to **all trigger types**:
* `interval` — timers keep running; an out-of-scope tick is a free early-return (no
start/stop lifecycle races on phase transitions).
* `startup` — a scoped startup rule not yet in scope at `UpdateRules` is **deferred**: it
runs exactly once from `OnPhaseChanged` when its scope activates (same
`_startupRulesExecuted` dedup; it is not part of the `WaitForStartupRules` latch).
* `phase_change` — evaluated against the **new** phase; the scope gate runs before the
per-(rule, phase) dedup so an out-of-scope pass does not consume the execution slot.
* `on_event` — gated by the current phase.
* **Before the first phase signal** of a session (`_currentPhase == Unknown`) scoped rules are
inactive. Sessions that never produce phase signals never run them (documented behavior).
* **From-phase latch**: activates when `phase != Unknown && phase != Failed && (int)phase >=
(int)fromPhase`. `Failed=99` would ordinal-satisfy every threshold, hence the explicit
exclusion — but once latched, a rule stays active through `Failed`. A config refresh that
delivers a from-phase rule after its threshold passed latches it immediately.
* `--run-gather-rules` (diagnostic CLI) sets `IgnorePhaseScope = true` — no phase context
exists there, scoped rules execute unconditionally (log line marks the bypass).
# Emit-on-change semantics
* The collector result dict is canonically hashed (keys ordinal-sorted at every nesting
level, invariant value formatting, SHA-256) **before** `ruleId`/`ruleTitle` injection.
* First in-scope non-empty result always emits — on an absent registry key that first
`exists=false` event *is* the visible "we're polling now" indicator.
* Unchanged results increment a per-rule suppression counter; the next emitted event carries
`suppressedPolls` + `suppressedSinceUtc` in its `Data` so the gap is observable.
* Composition with `emitOnlyIfExists=true` (collector-level): a miss returns an empty dict —
no emit AND no hash update. Result: zero noise while absent, one event on appearance, then
only on change.
* State is in-memory and survives `UpdateRules` config refreshes; an agent restart causes at
most one re-emit per rule (acceptable under the per-enrollment lifecycle).
* Volatile collectors (eventlog timestamps, logparser positions) degrade `on_change` to
`always` — every poll hashes differently. The option stays visible for them since some
outputs are stable; this is documented, not hidden.
# Portal & delivery
* Create form defaults new rules to `emitMode: "on_change"`; existing rules load as
`always`. The scope UI is a three-way mode (always / during phases / from phase) backed by
a canonical `GATHER_PHASES` list — the same list replaced the free-text `triggerPhase`
input, whose placeholder used to suggest invalid tokens.
* **Controls are shown only where they change behavior** (`supportsPhaseScope` /
`supportsEmitMode` in `types.ts`). Offering all three "when" controls unconditionally
produced meaningless combinations — a scope on a `phase_change` rule naming a concrete
phase can only ever suppress the single firing, and `on_change` has no second result to
compare against. Hidden controls are nulled by `buildScopeFields` so state from an earlier
trigger choice cannot leak into the payload, and `validateScopeSelection` rejects a scope
mode that was selected but left unfilled (which previously serialised to null and silently
ran the rule everywhere while the form still read "From a phase onwards").
Startup is the deliberate exception: a scope there defers the one-shot rather than
suppressing it, so the scope control stays while emit mode is hidden.
* The custom-rule toggle PUT (`{enabled, isBuiltIn, isCommunity}`) previously full-replaced
the stored rule and wiped Title/Target/Trigger — `GatherRuleService.UpdateRuleAsync` now
merges toggle-style partials (empty Title/CollectorType/Target) into the existing row and
preserves `CreatedAt` on full edits. This fix is a prerequisite: without it every toggle
would silently clear the new fields.
* Old agents (Newtonsoft) ignore unknown config properties: scoped rules run everywhere,
`emitMode` is ignored — graceful degradation until the fleet rolls forward.
* `ContentEquivalent` treats null and empty `activePhases` as equal, so pre-existing DB rows
(absent columns → null) never diff against seeds — no reseed churn.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs` — scope gates, latch, canonical hash
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — `ValidateScopeAndEmitMode`
* `src/Backend/AutopilotMonitor.Functions/Services/GatherRuleService.cs` — partial-PUT merge, `ContentEquivalent`
* `src/Shared/AutopilotMonitor.Shared/Models/Rules/GatherRule.cs` — field contracts
* Tests: `GatherRuleExecutorPhaseScopeTests`, `GatherRuleExecutorEmitModeTests` (agent); `GatherRuleUpdatePartialMergeTests`, `GatherRuleScopeFieldsTests` (backend)
+6
View File
@@ -0,0 +1,6 @@
{
"sdk": {
"version": "10.0.200",
"rollForward": "latestFeature"
}
}
+268
View File
@@ -0,0 +1,268 @@
// Autopilot-Monitor MCP Server — Azure Container App Infrastructure
//
// Reconciled against the live deployment on 2026-07-21 (rg-autopilotmonitor-prd-gwc,
// Germany West Central). The template had drifted behind two changes made directly
// in Azure: the EU cutover (registry/API renamed) and the ACR hardening (admin user
// disabled, image pull moved to the app's system-assigned identity).
//
// !! BEFORE RE-DEPLOYING THIS TEMPLATE, READ THIS !!
// The live app carries an ingress custom-domain binding for mcp.autopilotmonitor.com
// (SNI-enabled, with a managed certificate). Custom domains are NOT modelled below,
// because the certificate is bound out-of-band. A full template deployment would
// therefore REMOVE that binding and take the public MCP endpoint offline.
// For routine changes (CPU/memory, env vars, image) prefer the surgical CLI form:
// az containerapp update -n autopilotmonitor-mcp -g rg-autopilotmonitor-prd-gwc --cpu 0.5 --memory 1.0Gi
// Treat this file as the documented desired state and as a rebuild-from-scratch
// recipe — not as a routinely-applied deployment.
//
// On a FROM-SCRATCH deploy the AcrPull assignment below is created only after the
// Container App exists (it needs the app's principalId), so the very first revision
// can fail to pull its image. Re-run the deployment, or restart the revision, once
// the role assignment has propagated. This does not affect updates to a live app.
//
// Deploy (two-stage; required to set the OAuth public URL safely):
// 1. First deploy without mcpPublicUrl:
// az deployment group create \
// --resource-group <rg-name> \
// --template-file infra/mcp-server.bicep \
// --parameters apiUrl=https://autopilotmonitor-api.azurewebsites.net \
// entraClientSecret=<secret-value>
// This emits `containerAppUrl` as an output (e.g. https://autopilotmonitor-mcp.<env-suffix>.<region>.azurecontainerapps.io).
//
// 2. Re-deploy with that URL pinned, so the OAuth proxy never trusts caller-
// supplied X-Forwarded-Host / Host headers when building issuer / WWW-
// Authenticate / redirect_uri metadata:
// --parameters mcpPublicUrl=<containerAppUrl from step 1>
//
// When mcpPublicUrl is set, the MCP server uses it verbatim and ignores any
// forwarded-headers from the request. Without it, the server logs a startup
// warning and falls back to header-derived URLs (acceptable for local dev,
// not for production).
@description('Azure region for all resources')
param location string = resourceGroup().location
@description('Name of the Container App')
param containerAppName string = 'autopilotmonitor-mcp'
@description('Name of the Container App Environment')
param environmentName string = 'autopilotmonitor-env'
@description('Name of the Log Analytics Workspace (dedicated for Container Apps)')
param logAnalyticsName string = 'autopilotmonitor-container-logs'
@description('Name of the Azure Container Registry')
param acrName string = 'autopilotmonitoreuacr'
@description('Backend API URL')
param apiUrl string = 'https://autopilotmonitor-api-eu.azurewebsites.net'
@description('Container image tag')
param imageTag string = 'latest'
@description('Entra ID app registration client_id used for the OAuth proxy. Public value (appears in JWT aud claim); not a secret.')
param entraClientId string = '1a400946-62c1-4ab4-aa37-f730ac89704d'
@secure()
@description('Entra ID client secret for OAuth token exchange')
param entraClientSecret string = ''
@description('Comma-separated allowlist of redirect_uri hosts for the OAuth proxy. Loopback is always allowed. Empty = built-in default (Anthropic, OpenAI, Google).')
param mcpAllowedRedirectHosts string = ''
@description('Public URL of the MCP server, used verbatim for OAuth issuer / WWW-Authenticate / redirect metadata. REQUIRED in production: the container refuses to boot without it (host-spoofing defense — see config.ts). Set on the second deploy with the value of the containerAppUrl output from the first deploy. The stage-1 deploy is safe because minReplicas=0 means no container runs until this is pinned. Empty falls back to X-Forwarded-Host headers only outside production (NODE_ENV != production).')
param mcpPublicUrl string = ''
@secure()
// NOTE: avoid characters outside cp1252 here (this one used to say a Unicode
// >=). `az bicep build --stdout` writes through the Windows console codec and
// dies with a UnicodeEncodeError on anything it cannot map. Em dashes are fine.
@description('Base64-encoded HMAC key (>=32 bytes after decode) for signing the OAuth proxy state. Naming and format match the backend\'s PaginationTokenSigningKey for consistency. Empty = generate per-instance random key on startup (acceptable for single-replica deployments since state has 10 min lifetime). Generate with PowerShell: [Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))')
param oAuthStateSigningKey string = ''
@description('Enable structured tool-call logging to stderr (queryable via Container App Logs)')
param toolLoggingEnabled bool = false
// --- Azure Container Registry ---
// Admin user is DISABLED: a registry-wide username/password pair is a static,
// non-rotating credential that would also have to live in a Container App secret.
// The app pulls with its system-assigned identity + AcrPull instead (see below).
resource acr 'Microsoft.ContainerRegistry/registries@2023-07-01' = {
name: acrName
location: location
sku: {
name: 'Basic'
}
properties: {
adminUserEnabled: false
}
}
// AcrPull for the Container App's system-assigned identity — this is what replaces
// the admin credentials. Without it the app cannot pull its own image.
//
// NOTE: this assignment exists on the live registry but could not be enumerated
// during reconciliation (the CLI session lacks Microsoft.Authorization read), so it
// is reproduced here from the documented pattern rather than read back from Azure.
// Verify with:
// az role assignment list --scope <acr-id> --assignee <containerApp principalId>
var acrPullRoleId = '7f951dda-4ed3-4680-a7ca-43fe172d538d'
resource acrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
scope: acr
name: guid(acr.id, containerAppName, acrPullRoleId)
properties: {
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', acrPullRoleId)
principalId: containerApp.identity.principalId
principalType: 'ServicePrincipal'
}
}
// --- Log Analytics Workspace (required by Container App Environment) ---
resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
name: logAnalyticsName
location: location
properties: {
sku: {
name: 'PerGB2018'
}
retentionInDays: 30
}
}
// --- Container App Environment ---
resource environment 'Microsoft.App/managedEnvironments@2024-03-01' = {
name: environmentName
location: location
properties: {
appLogsConfiguration: {
destination: 'log-analytics'
logAnalyticsConfiguration: {
customerId: logAnalytics.properties.customerId
sharedKey: logAnalytics.listKeys().primarySharedKey
}
}
}
}
// --- Container App ---
resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
name: containerAppName
location: location
// System-assigned identity: the credential the app uses to pull from the ACR
// (see the AcrPull assignment above). Replaces the former admin-user secret.
identity: {
type: 'SystemAssigned'
}
properties: {
managedEnvironmentId: environment.id
configuration: {
ingress: {
external: true
targetPort: 8080
transport: 'auto'
allowInsecure: false
}
// 'system' = pull with the app's system-assigned identity. No username, no
// password secret — nothing to rotate and nothing to leak from the app config.
registries: [
{
server: acr.properties.loginServer
identity: 'system'
}
]
secrets: [
{
name: 'entra-client-secret'
value: entraClientSecret
}
{
name: 'oauth-state-signing-key'
value: oAuthStateSigningKey
}
]
activeRevisionsMode: 'Single'
}
template: {
containers: [
{
name: 'mcp-server'
image: '${acr.properties.loginServer}/${containerAppName}:${imageTag}'
// 0.5 vCPU, but NOT for the reason originally given. It was raised from
// 0.25 to shorten the scale-to-zero cold start, on the assumption that
// loading the ONNX embedding model dominated it. Measurement disproved
// that: the app boots in 504 ms in production (see the [boot] marks in
// the container log and docs/mcp/docs-corpus.md), of which the prebaked
// embedder is 173 ms in the background. Of a 31 s cold start, ~98% is
// Container Apps activation — pod scheduling, image mount, container
// start — which no CPU setting touches.
//
// It stays at 0.5 because it plausibly helps CONCURRENT requests
// (maxReplicas is 1, so one replica serves up to 10 in-flight calls) and
// because at scale-to-zero the container bills only while active, making
// the difference negligible. If cold start ever needs to actually go
// away, the only lever is minReplicas: 1. Container Apps accepts fixed
// CPU/memory pairs, so memory moves with the CPU.
resources: {
cpu: json('0.5')
memory: '1.0Gi'
}
env: [
{ name: 'AUTOPILOT_API_URL', value: apiUrl }
{ name: 'PORT', value: '8080' }
{ name: 'AUTOPILOT_ENTRA_CLIENT_ID', value: entraClientId }
{ name: 'AUTOPILOT_ENTRA_CLIENT_SECRET', secretRef: 'entra-client-secret' }
{ name: 'MCP_ALLOWED_REDIRECT_HOSTS', value: mcpAllowedRedirectHosts }
{ name: 'MCP_PUBLIC_URL', value: mcpPublicUrl }
{ name: 'OAuthStateSigningKey', secretRef: 'oauth-state-signing-key' }
{ name: 'MCP_TOOL_LOGGING', value: toolLoggingEnabled ? 'true' : 'false' }
]
probes: [
{
type: 'Liveness'
httpGet: {
path: '/health'
port: 8080
}
periodSeconds: 30
initialDelaySeconds: 10
}
{
type: 'Readiness'
httpGet: {
path: '/health'
port: 8080
}
periodSeconds: 10
initialDelaySeconds: 5
}
]
}
]
scale: {
minReplicas: 0
maxReplicas: 1
rules: [
{
name: 'http-scaler'
http: {
metadata: {
concurrentRequests: '10'
}
}
}
]
}
}
}
}
// --- Outputs ---
output containerAppUrl string = 'https://${containerApp.properties.configuration.ingress.fqdn}'
output mcpEndpoint string = 'https://${containerApp.properties.configuration.ingress.fqdn}/mcp'
output acrLoginServer string = acr.properties.loginServer
+85
View File
@@ -0,0 +1,85 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{ "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 },
{ "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 }
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": ["esp", "reboot", "coalescing", "second-sign-in", "policy-attribution", "user-experience"]
}
+99
View File
@@ -1785,6 +1785,105 @@
"enrollment"
]
},
{
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{
"signal": "multiple_reboot_uris",
"condition": "exists",
"weight": 10
},
{
"signal": "omadm_reboot_flag_set",
"condition": "exists",
"weight": 15
}
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"esp",
"reboot",
"coalescing",
"second-sign-in",
"policy-attribution",
"user-experience"
]
},
{
"ruleId": "ANALYZE-ID-001",
"title": "Expected Machine Certificate Not Found",
+73
View File
@@ -109,6 +109,79 @@
"SELECT * FROM Win32_LogicalDisk",
"SELECT * FROM SoftwareLicensingProduct"
],
"eventLogChannels": [
{
"category": "Core Windows logs",
"channels": [
"Application",
"System",
"Setup",
"Microsoft-Windows-Kernel-Boot",
"Microsoft-Windows-Diagnostics-Performance"
]
},
{
"category": "MDM / Enrollment",
"channels": [
"Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider",
"Microsoft-Windows-ModernDeployment-Diagnostics-Provider",
"Microsoft-Windows-Provisioning-Diagnostics-Provider",
"Microsoft-Windows-AAD",
"Microsoft-Windows-User Device Registration"
]
},
{
"category": "ESP / Shell / Apps",
"channels": [
"Microsoft-Windows-Shell-Core",
"Microsoft-Windows-AppXDeployment",
"Microsoft-Windows-AppXDeploymentServer",
"Microsoft-Windows-AppReadiness",
"Microsoft-Windows-Store"
]
},
{
"category": "Security posture / Crypto",
"channels": [
"Microsoft-Windows-BitLocker-API",
"Microsoft-Windows-BitLocker-DrivePreparationTool",
"Microsoft-Windows-TPM-WMI",
"Microsoft-Windows-CertificateServicesClient-Lifecycle-System"
]
},
{
"category": "Update / Servicing",
"channels": [
"Microsoft-Windows-WindowsUpdateClient"
]
},
{
"category": "Logon / Session",
"channels": [
"Microsoft-Windows-Winlogon",
"Microsoft-Windows-User Profile Service",
"Microsoft-Windows-GroupPolicy",
"Microsoft-Windows-TaskScheduler"
]
},
{
"category": "Network",
"channels": [
"Microsoft-Windows-NetworkProfile",
"Microsoft-Windows-Dhcp-Client",
"Microsoft-Windows-DNS-Client",
"Microsoft-Windows-NCSI",
"Microsoft-Windows-WLAN-AutoConfig",
"Microsoft-Windows-Time-Service"
]
}
],
"blockedEventLogChannels": [
"Security",
"Microsoft-Windows-PowerShell",
"Windows PowerShell",
"Microsoft-Windows-Sysmon"
],
"allowedCommands": [
{
"category": "TPM and Security",
+21 -3
View File
@@ -80,8 +80,8 @@
},
"trigger": {
"type": "string",
"description": "When to collect data.",
"enum": ["startup", "phase_change", "interval", "on_event"]
"description": "When to collect data. \"phase_change\" fires once when triggerPhase is entered, \"phase_exit\" once when it is left.",
"enum": ["startup", "phase_change", "phase_exit", "interval", "on_event"]
},
"intervalSeconds": {
"type": "integer",
@@ -90,12 +90,30 @@
},
"triggerPhase": {
"type": "string",
"description": "Phase to trigger on (only used when trigger = \"phase_change\")."
"description": "Phase to trigger on (used when trigger = \"phase_change\" or \"phase_exit\"). Empty = every phase transition."
},
"triggerEventType": {
"type": "string",
"description": "Event type to trigger on (only used when trigger = \"on_event\")."
},
"activePhases": {
"type": "array",
"description": "Run the rule only while the current enrollment phase is one of these phases (EnrollmentPhase enum names). Empty/absent = unrestricted. Mutually exclusive with activeFromPhase.",
"items": {
"type": "string",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
}
},
"activeFromPhase": {
"type": "string",
"description": "Activate the rule once the enrollment phase first reaches this phase, then keep it active for the rest of the session (sticky latch). Absent = unrestricted. Mutually exclusive with activePhases.",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
},
"emitMode": {
"type": "string",
"description": "Emit behavior: \"always\" (default — emit on every collection) or \"on_change\" (poll on the trigger cadence but emit only when the collected result changes; the first in-scope result always emits).",
"enum": ["always", "on_change"]
},
"outputEventType": {
"type": "string",
"description": "EventType for the emitted event (e.g., \"gather_proxy_settings\")."
+14 -1
View File
@@ -91,6 +91,7 @@ if (fs.existsSync(guardrailsPath)) {
// Flatten categorized lists for validation
const flatRegistryPrefixes = guardrails.registryPrefixes.flatMap((g) => g.prefixes);
const flatCommands = guardrails.allowedCommands.flatMap((g) => g.commands);
const flatEventLogChannels = guardrails.eventLogChannels.flatMap((g) => g.channels);
const ts = `/**
* AUTO-GENERATED from rules/guardrails.json — DO NOT EDIT.
@@ -114,6 +115,10 @@ export const COMMAND_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.allowedCommands, 'commands')}
];
export const EVENT_LOG_CHANNEL_CATEGORIES: readonly GuardrailCategory[] = [
${categorizedArray(guardrails.eventLogChannels, 'channels')}
];
// ---------------------------------------------------------------------------
// Flat arrays (for validation logic)
// ---------------------------------------------------------------------------
@@ -141,10 +146,18 @@ ${flatArray(guardrails.diagnosticsPathPrefixes)}
export const BLOCKED_FILE_PREFIXES: readonly string[] = [
${flatArray(guardrails.blockedFilePrefixes)}
];
export const ALLOWED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(flatEventLogChannels)}
];
export const BLOCKED_EVENT_LOG_CHANNELS: readonly string[] = [
${flatArray(guardrails.blockedEventLogChannels)}
];
`;
fs.writeFileSync(guardrailsOutput, ts, 'utf8');
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes`);
console.log(`guardrails.generated.ts: ${flatRegistryPrefixes.length} registry, ${flatCommands.length} commands, ${guardrails.filePrefixes.length} file prefixes, ${flatEventLogChannels.length} event log channels`);
} else {
console.warn('guardrails.json not found — skipping guardrails generation');
}
@@ -56,6 +56,10 @@
PowerShell 5.1 (IME) reads scripts without BOM as ANSI, corrupting multi-byte chars.
.CHANGELOG
2026-07-20 v2.2 Switched default download endpoint from the legacy blob
(autopilotmonitor.blob.core.windows.net) to the Front Door alias
https://download.autopilotmonitor.com/agent/ (route /agent/*).
version.json is still derived from the same container.
2026-06-25 v2.1 OOBE-relax: detect genuine OOBE via WinRT
SystemSetupInfo.OutOfBoxExperienceState and exempt guards 2+3 when a
single freshly-created profile appears in OOBE (Windows Backup for
@@ -80,7 +84,7 @@
[CmdletBinding()]
param(
[Parameter(Mandatory = $false)]
[string]$AgentDownloadUrl = "https://autopilotmonitor.blob.core.windows.net/agent/AutopilotMonitor-Agent.zip",
[string]$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip",
[Parameter(Mandatory = $false)]
[string]$VersionJsonName = "version.json",
@@ -93,7 +97,7 @@ param(
)
# Script version (bump on meaningful changes; see .CHANGELOG above)
$ScriptVersion = "2.1"
$ScriptVersion = "2.2"
# Configuration - Everything in ProgramData for easy cleanup
$AgentBasePath = "$env:ProgramData\AutopilotMonitor"
@@ -0,0 +1,99 @@
using AutopilotMonitor.Agent.V2.Core.Configuration;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
{
/// <summary>
/// Pins the agent-side endpoint-migration contract (<see cref="EndpointMigration"/>):
/// only a LIVE fetch is honoured, the target must pass the shared allowlist rules even
/// though the backend already validated it, and serving the agent its own current URL is
/// a no-op — the exact mirror of the kill-switch live-fetch hardening.
/// </summary>
public sealed class EndpointMigrationTests
{
private const string CurrentUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
private const string TargetUrl = "https://autopilotmonitor-api-us.azurewebsites.net";
private static AgentConfigResponse Config(string migrateUrl) =>
new AgentConfigResponse { MigrateToApiBaseUrl = migrateUrl };
[Fact]
public void Live_fetch_with_valid_differing_target_migrates()
{
var target = EndpointMigration.ResolveTarget(
Config(TargetUrl), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null);
Assert.Equal(TargetUrl, target);
}
[Fact]
public void Target_is_normalized_to_lowercase_host_without_trailing_slash()
{
var target = EndpointMigration.ResolveTarget(
Config("https://AutopilotMonitor-API-US.azurewebsites.net/"),
RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null);
Assert.Equal(TargetUrl, target);
}
[Theory]
[InlineData(RemoteConfigFetchOutcome.FromCache)]
[InlineData(RemoteConfigFetchOutcome.UsedDefaults)]
[InlineData(RemoteConfigFetchOutcome.NotAttempted)]
public void Non_live_fetch_is_never_honoured(RemoteConfigFetchOutcome outcome)
{
Assert.Null(EndpointMigration.ResolveTarget(
Config(TargetUrl), outcome, CurrentUrl, logger: null));
}
[Fact]
public void Null_config_and_empty_signal_are_noops()
{
Assert.Null(EndpointMigration.ResolveTarget(
null, RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(null), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(" "), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Fact]
public void Target_equal_to_current_url_is_a_noop()
{
// Steady state during a migration window: agent already runs on the target
// (case/trailing-slash differences must not cause a rebuild loop).
Assert.Null(EndpointMigration.ResolveTarget(
Config(CurrentUrl), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(CurrentUrl.ToUpperInvariant() + "/"), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Theory]
[InlineData("http://autopilotmonitor-api-us.azurewebsites.net")] // https only
[InlineData("https://attacker.example.com")] // host not allowlisted
[InlineData("https://evil-azurewebsites.net")] // suffix must sit on a label boundary
[InlineData("https://.azurewebsites.net")] // empty host prefix
[InlineData("https://api.azurewebsites.net:8443")] // non-default port
[InlineData("https://user:pw@api.azurewebsites.net")] // userinfo
[InlineData("https://api.azurewebsites.net/some/path")] // base URL only
[InlineData("https://api.azurewebsites.net?x=1")] // no query
[InlineData("not a url")]
public void Invalid_targets_are_rejected(string candidate)
{
Assert.Null(EndpointMigration.ResolveTarget(
Config(candidate), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Fact]
public void Dev_api_url_override_still_migrates_to_valid_target()
{
// The running base URL may be a non-allowlisted dev override (--api-url); the
// comparison falls back to plain normalization and the migration still applies.
var target = EndpointMigration.ResolveTarget(
Config(TargetUrl), RemoteConfigFetchOutcome.Succeeded, "http://localhost:7071", logger: null);
Assert.Equal(TargetUrl, target);
}
}
}
@@ -62,6 +62,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
DeviceBlocked = true,
DeviceKillSignal = true,
UnblockAt = new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc),
MigrateToApiBaseUrl = "https://attacker.azurewebsites.net",
};
probe.DoCache(config);
@@ -76,9 +77,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.False(persisted.DeviceBlocked);
Assert.False(persisted.DeviceKillSignal);
Assert.Null(persisted.UnblockAt);
Assert.Null(persisted.MigrateToApiBaseUrl);
// The attacker EXE hash must not appear anywhere in the persisted bytes.
// The attacker EXE hash / re-home host must not appear anywhere in the persisted bytes.
Assert.DoesNotContain("deadbeef", raw);
Assert.DoesNotContain("attacker.azurewebsites.net", raw);
// Benign fields survive the strip.
Assert.Equal(77, persisted.ConfigVersion);
@@ -92,6 +95,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.True(config.DeviceBlocked);
Assert.True(config.DeviceKillSignal);
Assert.Equal(new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc), config.UnblockAt);
Assert.Equal("https://attacker.azurewebsites.net", config.MigrateToApiBaseUrl);
}
// ── Read path: LoadCachedConfig strips a planted (attacker) cache file ─
@@ -103,6 +107,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
[InlineData("DeviceBlocked")]
[InlineData("DeviceKillSignal")]
[InlineData("UnblockAt")]
[InlineData("MigrateToApiBaseUrl")]
public void LoadCachedConfig_strips_planted_security_sensitive_field(string field)
{
var probe = NewProbe();
@@ -119,6 +124,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
DeviceBlocked = true,
DeviceKillSignal = true,
UnblockAt = new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc),
MigrateToApiBaseUrl = "https://attacker.azurewebsites.net",
};
File.WriteAllText(_cachePath, JsonConvert.SerializeObject(attacker, Formatting.Indented));
@@ -145,6 +151,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
case "UnblockAt":
Assert.Null(loaded.UnblockAt);
break;
case "MigrateToApiBaseUrl":
Assert.Null(loaded.MigrateToApiBaseUrl);
break;
default:
throw new InvalidOperationException($"unknown field {field}");
}
@@ -0,0 +1,280 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Allowlist enforcement for the two collectors that used to bypass it:
/// <see cref="LogParserCollector"/> read any path the rule named — including
/// C:\Users, whose hard block every other file collector honours — and
/// <see cref="EventLogCollector"/> read any channel, including Security.
/// A tenant admin can author both through the rules API, so the agent is the
/// only place this can be enforced.
/// </summary>
public sealed class GatherRuleCollectorGuardTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
// %TEMP% itself lives under C:\Users, which is hard-blocked — the positive
// cases need a directory the guard can actually admit, so they run beside
// the test assembly instead.
private readonly string _outsideUsers = Path.Combine(
AppContext.BaseDirectory, "guard-tests-" + Guid.NewGuid().ToString("N"));
public GatherRuleCollectorGuardTests() => Directory.CreateDirectory(_outsideUsers);
public void Dispose()
{
_tmp.Dispose();
try { Directory.Delete(_outsideUsers, recursive: true); } catch { /* best effort */ }
}
private GatherRuleContext Context(bool unrestricted = false, string imeLogPathOverride = null)
=> new GatherRuleContext(
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
"sess", "tenant",
evt => _events.Add(evt),
imeLogPathOverride,
new LogFilePositionTracker())
{
UnrestrictedMode = unrestricted
};
private static GatherRule Rule(string collectorType, string target,
Dictionary<string, string> parameters = null) => new GatherRule
{
RuleId = "GATHER-GUARD-TEST",
Title = "guard test",
CollectorType = collectorType,
Target = target,
Parameters = parameters,
Trigger = "startup",
OutputEventType = "guard_test",
};
private EnrollmentEvent SingleSecurityWarning()
=> Assert.Single(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
// -------------------------------------------------------------------
// Event log channel allowlist
// -------------------------------------------------------------------
[Theory]
[InlineData("System")]
[InlineData("Application")]
[InlineData("Setup")]
[InlineData("Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin")]
[InlineData("Microsoft-Windows-AAD/Operational")]
[InlineData("Microsoft-Windows-Shell-Core/Operational")]
public void EventLogChannel_OnAllowlist_IsAllowed(string channel)
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(channel));
[Theory]
[InlineData("Security")]
[InlineData("Microsoft-Windows-PowerShell/Operational")]
[InlineData("Windows PowerShell")]
[InlineData("Microsoft-Windows-Sysmon/Operational")]
public void EventLogChannel_HardBlocked_StaysBlockedEvenUnrestricted(string channel)
{
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel, unrestrictedMode: true));
}
[Theory]
[InlineData("Microsoft-Windows-AADSomethingElse/Operational")] // prefix spoofing
[InlineData("Microsoft-Windows-TerminalServices-LocalSessionManager/Operational")]
[InlineData("")]
public void EventLogChannel_OffAllowlist_IsBlocked(string channel)
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(channel));
[Fact]
public void EventLogChannel_Null_IsBlocked()
=> Assert.False(GatherRuleGuards.IsEventLogChannelAllowed(null));
[Fact]
public void EventLogChannel_OffAllowlist_IsAllowedInUnrestrictedMode()
=> Assert.True(GatherRuleGuards.IsEventLogChannelAllowed(
"Microsoft-Windows-TerminalServices-LocalSessionManager/Operational",
unrestrictedMode: true));
[Fact]
public void EventLogCollector_SecurityChannel_EmitsSecurityWarningAndReturnsNoData()
{
var result = new EventLogCollector().Execute(Rule("eventlog", "Security"), Context());
Assert.Empty(result);
var warning = SingleSecurityWarning();
Assert.Equal(true, warning.Data["blocked"]);
Assert.Equal("Security", warning.Data["target"]);
}
[Fact]
public void EventLogCollector_SecurityChannel_StaysBlockedInUnrestrictedMode()
{
var result = new EventLogCollector().Execute(
Rule("eventlog", "Security"), Context(unrestricted: true));
Assert.Empty(result);
SingleSecurityWarning();
}
// -------------------------------------------------------------------
// Log parser file allowlist
// -------------------------------------------------------------------
private static Dictionary<string, string> TextPattern() => new Dictionary<string, string>
{
["pattern"] = "(?<line>.+)",
["format"] = "text",
["trackPosition"] = "false",
};
[Fact]
public void LogParser_UserProfilePath_IsBlockedAndEmitsSecurityWarning()
{
const string target = @"C:\Users\Public\Documents\secret.txt";
var result = new LogParserCollector().Execute(
Rule("logparser", target, TextPattern()), Context());
Assert.Null(result);
var warning = SingleSecurityWarning();
Assert.Equal(target, warning.Data["target"]);
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_UserProfilePath_StaysBlockedInUnrestrictedMode()
{
// C:\Users is a hard block — unrestricted mode must not lift it.
new LogParserCollector().Execute(
Rule("logparser", @"C:\Users\Public\Documents\secret.txt", TextPattern()),
Context(unrestricted: true));
SingleSecurityWarning();
}
[Fact]
public void LogParser_OffAllowlistPath_IsBlockedEvenWhenFileExists()
{
var path = _tmp.File("evidence.log");
File.WriteAllText(path, "line one\nline two\n");
var result = new LogParserCollector().Execute(
Rule("logparser", path, TextPattern()), Context());
Assert.Null(result);
SingleSecurityWarning();
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_WildcardTarget_IsBlockedByItsDirectory()
{
// Path.GetFullPath rejects '*', so the guard has to check the directory.
File.WriteAllText(_tmp.File("evidence.log"), "line one\n");
new LogParserCollector().Execute(
Rule("logparser", Path.Combine(_tmp.Path, "*.log"), TextPattern()), Context());
SingleSecurityWarning();
Assert.DoesNotContain(_events, e => e.EventType == "guard_test");
}
[Fact]
public void LogParser_UnrestrictedMode_ReadsPathOffTheAllowlist()
{
var path = Path.Combine(_outsideUsers, "evidence.log");
File.WriteAllText(path, "line one\nline two\n");
new LogParserCollector().Execute(
Rule("logparser", path, TextPattern()), Context(unrestricted: true));
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
Assert.Equal(2, _events.Count(e => e.EventType == "guard_test"));
}
[Fact]
public void LogParser_LocalImeLogPathOverride_RelaxesTheAllowlist()
{
// --ime-log-path is a local CLI flag, never remote config: the operator
// using it is already a local admin, so it relaxes the allowlist the way
// unrestricted mode does.
File.WriteAllText(Path.Combine(_outsideUsers, "AppWorkload.log"), "line one\n");
new LogParserCollector().Execute(
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
TextPattern()),
Context(imeLogPathOverride: _outsideUsers));
Assert.DoesNotContain(_events, e => e.EventType == Constants.EventTypes.SecurityWarning);
Assert.Single(_events, e => e.EventType == "guard_test");
}
// -------------------------------------------------------------------
// Command execution
// -------------------------------------------------------------------
[Fact]
public void Command_OffAllowlist_IsBlockedAndNeverRuns()
{
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Get-Content C:\\Windows\\System32\\config\\SAM"), Context());
Assert.Equal(true, result["blocked"]);
Assert.False(result.ContainsKey("output"));
SingleSecurityWarning();
}
[Fact]
public void Command_HardBlockedPattern_StaysBlockedInUnrestrictedMode()
{
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Invoke-WebRequest https://example.invalid/x"),
Context(unrestricted: true));
Assert.Equal(true, result["blocked"]);
SingleSecurityWarning();
}
[Fact]
public void Command_ThatHangs_IsKilledAtTheTimeoutRatherThanPinningTheWorker()
{
// Regression: reading stdout synchronously blocked until the pipe closed,
// so WaitForExit's timeout was never reached and a hung command pinned the
// worker indefinitely. Unrestricted mode is needed to run an ad-hoc command.
var sw = System.Diagnostics.Stopwatch.StartNew();
var result = new CommandCollector().Execute(
Rule("command_allowlisted", "Start-Sleep -Seconds 120"), Context(unrestricted: true));
sw.Stop();
Assert.Equal(true, result["timed_out"]);
Assert.Equal(-1, result["exit_code"]);
// The 30s timeout plus drain margin — nowhere near the command's 120s.
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(60),
$"expected the hung command to be killed at the timeout; took {sw.Elapsed}");
}
[Fact]
public void LogParser_ImeLogPathOverride_StillHonoursTheUsersHardBlock()
{
new LogParserCollector().Execute(
Rule("logparser", @"C:\ProgramData\Microsoft\IntuneManagementExtension\Logs\AppWorkload.log",
TextPattern()),
Context(imeLogPathOverride: @"C:\Users\Public\Documents"));
SingleSecurityWarning();
}
}
}
@@ -0,0 +1,249 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// EmitMode "on_change" on <see cref="GatherRuleExecutor"/>: poll on the trigger cadence but
/// emit only when the collected result changes. The dedup state machine is asserted
/// synchronously via the internal <c>ShouldEmitOnChange</c> / <c>ComputeCanonicalHash</c>
/// (no timer races); the ExecuteRule wiring (hash-before-injection, empty-result no-op,
/// emitOnlyIfExists composition) through the public startup path.
/// </summary>
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorEmitModeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_7e3a1f4b-0000-0000-0000-000000000000";
public GatherRuleExecutorEmitModeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private EnrollmentEvent SingleEvent()
{
lock (_eventsGate) return Assert.Single(_events);
}
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
private static GatherRule OnChangeRule(string id = "GATHER-OC-001") => new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
EmitMode = "on_change",
};
private static Dictionary<string, object> Result(params (string Key, object Value)[] pairs)
{
var d = new Dictionary<string, object>();
foreach (var (key, value) in pairs) d[key] = value;
return d;
}
// ── on_change state machine (synchronous) ──────────────────────────
[Fact]
public void FirstResult_AlwaysEmits_ThenUnchangedIsSuppressed_ThenChangeReemits()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // first
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1")))); // changed
}
[Fact]
public void EmitAfterSuppressionStreak_CarriesSuppressedPollCount_AndResets()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
var changed = Result(("exists", true));
Assert.True(_executor.ShouldEmitOnChange(rule, changed));
Assert.Equal(3, changed["suppressedPolls"]);
Assert.True(changed.ContainsKey("suppressedSinceUtc"));
// Counter must reset: the next change emits without a suppression payload.
var changedAgain = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(rule, changedAgain));
Assert.False(changedAgain.ContainsKey("suppressedPolls"));
Assert.False(changedAgain.ContainsKey("suppressedSinceUtc"));
}
[Fact]
public void SuppressionState_IsIsolatedPerRule()
{
var ruleA = OnChangeRule("GATHER-OC-A");
var ruleB = OnChangeRule("GATHER-OC-B");
var payload = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
// Same content, different rule: B has no hash yet — must emit.
Assert.True(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
}
[Fact]
public void DedupState_SurvivesUpdateRulesRefresh()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
// Config refresh mid-session must not reset the last-emitted hash (that would
// re-emit an unchanged result on every refresh).
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
}
// ── Canonical hashing ──────────────────────────────────────────────
[Fact]
public void CanonicalHash_IsKeyOrderInsensitive()
{
var a = new Dictionary<string, object> { ["alpha"] = 1, ["beta"] = "x" };
var b = new Dictionary<string, object> { ["beta"] = "x", ["alpha"] = 1 };
Assert.Equal(
GatherRuleExecutor.ComputeCanonicalHash(a),
GatherRuleExecutor.ComputeCanonicalHash(b));
}
[Fact]
public void CanonicalHash_DetectsValueChanges_IncludingNested()
{
var baseline = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedScalar = new Dictionary<string, object>
{
["exists"] = false,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedNested = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 2 } },
};
var baseHash = GatherRuleExecutor.ComputeCanonicalHash(baseline);
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedScalar));
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedNested));
}
[Fact]
public void CanonicalHash_DistinguishesTypeShapes()
{
// "1" (string) vs 1 (int) and null vs "<null>"-like strings must not collide.
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "1" }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = 1 }));
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = null }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "<null>" }));
}
// ── ExecuteRule wiring (async via startup path) ────────────────────
[Fact]
public void AlwaysMode_EmitsEveryCollection()
{
// EmitMode absent (legacy) — repeated identical results all emit. Two executions of
// the same collector via two rules sharing the target prove no dedup is applied.
var rule1 = OnChangeRule("GATHER-AL-1"); rule1.EmitMode = null; rule1.Trigger = "startup";
var rule2 = OnChangeRule("GATHER-AL-2"); rule2.EmitMode = null; rule2.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule1, rule2 });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(2));
}
[Fact]
public void OnChange_FirstInScopePoll_EmitsExistsFalse_AsPollingIndicator()
{
// The first result on an absent key IS the visible "we're polling now" indicator.
var rule = OnChangeRule();
rule.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
var evt = SingleEvent();
Assert.Equal(false, evt.Data["exists"]);
Assert.Equal("GATHER-OC-001", evt.Data["ruleId"]); // injection happened after hashing
}
[Fact]
public void OnChange_ComposedWithEmitOnlyIfExists_EmptyResultDoesNotTouchHashState()
{
// emitOnlyIfExists miss → empty result → no emit AND no hash update: the eventual
// first real result must still count as "first emit".
var rule = OnChangeRule();
rule.Trigger = "startup";
rule.Parameters = new Dictionary<string, string> { ["emitOnlyIfExists"] = "true" };
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Thread.Sleep(300);
Assert.Equal(0, EventCount); // zero noise while the key is absent
// When the key appears (simulated via the state machine), the result emits as first.
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1"))));
}
}
}
@@ -0,0 +1,246 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// The <c>phase_exit</c> trigger: a one-shot collection when a phase is LEFT — the closing
/// bookend to <c>phase_change</c> (which fires on ENTER). The distinguishing invariant is that
/// both the TriggerPhase match and the phase-scope gate must evaluate against the phase being
/// left, i.e. before the executor's current-phase state moves forward.
/// Rules use the absent-registry-path pattern so no real system state is touched.
/// </summary>
[Collection("SerialThreading")] // rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorPhaseExitTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_9c4d2a7f-0000-0000-0000-000000000000";
public GatherRuleExecutorPhaseExitTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
private static GatherRule ExitRule(
string id, string triggerPhase, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_exit",
TriggerPhase = triggerPhase,
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
[Fact]
public void FiresOnceWhenTheNamedPhaseIsLeft_NotWhenItIsEntered()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-001", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
// Entering AccountSetup must NOT fire an exit rule.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
// Leaving it does.
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
}
[Fact]
public void DoesNotRefireWhenThePhaseIsLeftTwice()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-002", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
// A phase can be re-entered (ESP re-reporting); the one-shot slot stays consumed.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Thread.Sleep(400);
Assert.Equal(1, EventCount);
}
[Fact]
public void EmptyTriggerPhase_FiresOnEveryExit()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-003", "") });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup); // exit of Unknown — must NOT fire
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup
Assert.True(WaitForEventCount(1));
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup
Assert.True(WaitForEventCount(2));
}
[Fact]
public void NeverFiresOnExitOfUnknown()
{
// Before the first phase signal there is no phase to leave.
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-004", "") });
_executor.OnPhaseChanged(EnrollmentPhase.Start);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void RepeatedSamePhaseSignal_IsNotAnExit()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-005", "") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // same phase — no transition
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void FiresOnTransitionIntoFailed_CapturingStateAtTheFailurePoint()
{
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-006", "AccountSetup") });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.True(WaitForEventCount(1), "leaving a phase into Failed must still snapshot it");
}
[Fact]
public void FiresOnExitOfCompleteOnly_WhenCompleteIsActuallyLeft()
{
// "End of enrollment" is better served by phase_change on Complete / on_event; an exit
// rule on Complete only fires if a later transition happens at all.
_executor.UpdateRules(new List<GatherRule> { ExitRule("GATHER-PX-007", "Complete") });
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
_executor.OnPhaseChanged(EnrollmentPhase.Complete);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
}
[Fact]
public void ScopeGate_EvaluatesAgainstThePhaseBeingLeft()
{
// Scope = only during DeviceSetup. Leaving DeviceSetup must fire (the rule is still in
// scope at that instant) even though the NEW phase is outside the scope.
_executor.UpdateRules(new List<GatherRule>
{
ExitRule("GATHER-PX-010", "DeviceSetup", activePhases: new List<string> { "DeviceSetup" })
});
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1),
"the scope gate must see the phase being left, not the one being entered");
}
[Fact]
public void OutOfScopeExit_DoesNotFire_AndDoesNotConsumeTheOneShotSlot()
{
// Scope = only during AccountSetup; the rule exits DeviceSetup while out of scope.
var rule = ExitRule("GATHER-PX-011", "", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup); // exit of DeviceSetup: out of scope
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser); // exit of AccountSetup: in scope
Assert.True(WaitForEventCount(1));
}
[Fact]
public void ExitAndChangeRulesForTheSamePhase_BothFireIndependently()
{
// The dedup key spaces must not collide: same ruleId shape, same phase, different trigger.
var enter = new GatherRule
{
RuleId = "GATHER-PX-020",
Title = "enter",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_change",
TriggerPhase = "AccountSetup",
OutputEventType = "gather_test",
Enabled = true,
};
var exit = ExitRule("GATHER-PX-021", "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { enter, exit });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1)); // enter fired
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(2)); // exit fired
}
[Fact]
public void ExitRuleWithEmitOnChange_StillEmitsItsFirstResult()
{
var rule = ExitRule("GATHER-PX-030", "AccountSetup");
rule.EmitMode = "on_change";
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.True(WaitForEventCount(1));
}
}
}
@@ -0,0 +1,326 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Phase scoping on <see cref="GatherRuleExecutor"/> (ActivePhases / ActiveFromPhase):
/// the scope matrix is asserted synchronously via the internal <c>IsRuleInScope</c>
/// (no timer races); the trigger-path wiring (deferred startup, phase_change / on_event
/// gating) is asserted through the public API with positive event waits.
/// Rules use the absent-registry-path pattern from
/// <see cref="RegistryCollectorEmitOnlyIfExistsTests"/> so no real system state is touched.
/// </summary>
[Collection("SerialThreading")] // deferred startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorPhaseScopeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_5b1c8d2e-0000-0000-0000-000000000000";
public GatherRuleExecutorPhaseScopeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
// Bypass the registry allowlist guard for the synthetic test path.
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
/// <summary>Interval rule with a 1h period — the timer never ticks within a test run,
/// so these rules exist purely for synchronous scope-matrix assertions.</summary>
private static GatherRule ScopeProbeRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
private static GatherRule StartupRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "startup",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
// ── Scope matrix (synchronous) ─────────────────────────────────────
[Fact]
public void UnscopedRule_IsInScope_EvenBeforeAnyPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-000");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_InactiveBeforeFirstPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-001", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_ActiveDuringListedPhase_InactiveAfterLeaving()
{
var rule = ScopeProbeRule("GATHER-SCOPE-002", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_MatchIsCaseInsensitive()
{
var rule = ScopeProbeRule("GATHER-SCOPE-003", activePhases: new List<string> { "accountsetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesAtThreshold_AndStaysSticky_IncludingThroughFailed()
{
var rule = ScopeProbeRule("GATHER-SCOPE-010", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule)); // threshold not reached
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule)); // latched
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.True(_executor.IsRuleInScope(rule)); // sticky through Failed
}
[Fact]
public void FromPhase_NoSpuriousActivationViaFailed()
{
// Failed=99 would ordinal-satisfy every threshold — it must never latch.
var rule = ScopeProbeRule("GATHER-SCOPE-011", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesWhenThresholdIsSkippedOver()
{
// The session may never report the exact threshold phase — a later phase latches too.
var rule = ScopeProbeRule("GATHER-SCOPE-012", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_RuleDeliveredByRefreshAfterPhaseReached_LatchesImmediately()
{
_executor.UpdateRules(new List<GatherRule>());
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
// Config refresh delivers a new from-phase rule whose threshold already passed.
var rule = ScopeProbeRule("GATHER-SCOPE-013", fromPhase: "DeviceSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void BothFieldsSet_ActivePhasesWins()
{
// Backend validation rejects this shape; the agent defensively prefers ActivePhases.
var rule = ScopeProbeRule("GATHER-SCOPE-020",
activePhases: new List<string> { "DeviceSetup" }, fromPhase: "Start");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.False(_executor.IsRuleInScope(rule)); // FromPhase(Start) would say yes — list wins
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void IgnorePhaseScope_BypassesScope_WithoutPhaseContext()
{
_executor.IgnorePhaseScope = true;
var during = ScopeProbeRule("GATHER-SCOPE-030", activePhases: new List<string> { "AccountSetup" });
var from = ScopeProbeRule("GATHER-SCOPE-031", fromPhase: "Complete");
_executor.UpdateRules(new List<GatherRule> { during, from });
Assert.True(_executor.IsRuleInScope(during));
Assert.True(_executor.IsRuleInScope(from));
}
// ── Trigger-path wiring (async, positive waits) ────────────────────
[Fact]
public void UnscopedStartupRule_RunsImmediately()
{
_executor.UpdateRules(new List<GatherRule> { StartupRule("GATHER-START-000") });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
[Fact]
public void ScopedStartupRule_IsDeferred_ThenFiresExactlyOnce_WhenScopeActivates()
{
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-001", fromPhase: "AccountSetup")
});
// Not in scope at UpdateRules: nothing queued, latch trivially satisfied.
Assert.True(_executor.WaitForStartupRules(30));
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1), "deferred startup rule should fire when its scope activates");
// Later phase changes must not re-fire it (dedup via _startupRulesExecuted).
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void PhaseChangeRule_OutOfScopePhase_DoesNotFire_InScopePhaseFires()
{
// TriggerPhase empty = fires on every phase change; scope narrows it to AccountSetup.
var rule = new GatherRule
{
RuleId = "GATHER-PC-001",
Title = "pc",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_change",
TriggerPhase = "",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1));
// Same phase again: the (rule, phase) dedup slot is now consumed.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void OnEventRule_GatedByScope()
{
var rule = new GatherRule
{
RuleId = "GATHER-OE-001",
Title = "oe",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "on_event",
TriggerEventType = "app_install_failed",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnEvent("app_install_failed"); // no phase signal yet → out of scope
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnEvent("app_install_failed");
Assert.True(WaitForEventCount(1));
}
[Fact]
public void IgnorePhaseScope_ScopedStartupRule_RunsImmediately()
{
// The --run-gather-rules diagnostic path: scoped rules must execute unconditionally.
_executor.IgnorePhaseScope = true;
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-002", activePhases: new List<string> { "AccountSetup" })
});
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
}
}
@@ -17,6 +17,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Office
/// cadence and stops at its bound, is idempotent, the initial scan raises when binaries already
/// exist, a filesystem create raises, and Dispose cancels the re-probe.
/// </summary>
[Collection("SerialThreading")] // the bounded re-probe runs on the shared ThreadPool
public sealed class OfficeBinaryWatcherTests
{
private static readonly string[] CoreBinaries = { "WINWORD.EXE", "EXCEL.EXE", "POWERPNT.EXE", "OUTLOOK.EXE" };
@@ -0,0 +1,336 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral
/// <c>mdm_policy_reboot_required</c> event (session b2e890c1 rework: per-record events were
/// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an
/// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit
/// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed).
/// The registry probe is always overridden — tests never touch the live registry.
/// </summary>
public sealed class MdmRebootPolicyTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc);
private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation";
private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch";
// Verified real 2800 description shape (session b2e890c1, 2026-07-20).
private const string RealDescription =
"The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly MdmRebootPolicyTracker _tracker;
private long _nextRecordId = 1;
public MdmRebootPolicyTrackerTests()
{
_sink = new FakeSignalIngressSink();
_tracker = MakeTracker(stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private MdmRebootPolicyTracker MakeTracker(string? stateDirectory)
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var tracker = new MdmRebootPolicyTracker(
sessionId: "sess-mdm",
tenantId: "tenant-mdm",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: stateDirectory,
debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly
tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
return tracker;
}
private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null)
{
_tracker.ProcessEvent(
eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? At,
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> Emitted() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.MdmPolicyRebootRequired).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload()
{
Buffer(UriSvchost, timeCreatedUtc: At);
Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120));
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal(2, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
Assert.Equal(UriSvchost, data["firstRebootUri"]);
var uris = Assert.IsType<List<string>>(data["rebootUris"]);
Assert.Equal(2, uris.Count);
Assert.Contains(UriSvchost, uris);
Assert.Contains(UriSystemGuard, uris);
}
[Fact]
public void Message_IsNeutralObservation_NoRebootClaim()
{
// Session b2e890c1: flags raised during AccountSetup have no effect — the raw event
// must state the observation only, never a future consequence.
Buffer(UriSvchost);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var message = s.Payload![SignalPayloadKeys.Message];
Assert.Contains("flagged", message);
Assert.DoesNotContain("will restart", message);
Assert.DoesNotContain("second sign-in", message);
}
[Fact]
public void FlushPending_Empty_EmitsNothing()
{
_tracker.FlushPending();
Assert.Empty(Emitted());
}
[Fact]
public void AggregateTimestamp_IsEarliestRecordTime()
{
var early = At.AddMinutes(-30);
Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true);
Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(early, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
// firstRebootUri follows record-time order, not buffer order.
Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]);
}
[Fact]
public void Backfilled_False_WhenAnyRecordIsLive()
{
Buffer(UriSvchost, isBackfill: true);
Buffer(UriSystemGuard, isBackfill: false);
_tracker.FlushPending();
Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]);
}
[Fact]
public void DuplicateUris_AreDistinctInAggregate()
{
Buffer(UriSvchost);
Buffer(UriSvchost);
_tracker.FlushPending();
var data = Data(Assert.Single(Emitted()));
Assert.Equal(1, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
}
[Fact]
public void UnparsedRecords_CountedWithSampleDescription_StillEmitted()
{
Buffer(rebootUri: null, description: "Some unexpected 2800 wording");
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var data = Data(s);
Assert.False(data.ContainsKey("rebootUris"));
Assert.False(data.ContainsKey("firstRebootUri"));
Assert.Equal(0, data["uriCount"]);
Assert.Equal(1, data["unparsedCount"]);
Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]);
}
[Fact]
public void SameRecordId_BufferedTwice_CountsOnce()
{
Buffer(UriSvchost, recordId: 42);
Buffer(UriSvchost, recordId: 42);
_tracker.FlushPending();
Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void RecordAfterFlush_GoesIntoNextAggregate()
{
Buffer(UriSvchost, recordId: 10);
_tracker.FlushPending();
Buffer(UriSystemGuard, recordId: 11);
_tracker.FlushPending();
Assert.Equal(2, Emitted().Count);
}
[Fact]
public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered()
{
// Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill
// record must not be suppressed by a high-water mark.
Buffer(UriSvchost, recordId: 100);
Buffer(UriSystemGuard, recordId: 50, isBackfill: true);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void NegativeRecordId_IsNeverDeduped()
{
Buffer(UriSvchost, recordId: -1);
Buffer(UriSvchost, recordId: -1);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances()
{
// The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same
// records — a FLUSHED record must be skipped by the next instance.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
first.FlushPending();
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
Assert.Single(Emitted());
}
[Fact]
public void Watermark_NotPersisted_WhenBufferedButNeverFlushed()
{
// Process killed by the very reboot the records announce: nothing was flushed, so the
// next instance must re-read and emit the records.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
// no FlushPending — simulated reboot kill
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(true, Data(s)["backfilled"]);
}
[Fact]
public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt()
{
_tracker.RebootRequiredFlagProbe = () => true;
Buffer(UriSvchost, recordId: 1);
_tracker.FlushPending();
_tracker.RebootRequiredFlagProbe = () => null;
Buffer(UriSystemGuard, recordId: 2);
_tracker.FlushPending();
var emitted = Emitted();
Assert.Equal(2, emitted.Count);
Assert.Equal(true, Data(emitted[0])["omadmRebootRequiredFlag"]);
Assert.False(Data(emitted[1]).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void RebootRequiredFlagProbe_Throwing_IsFailSoft()
{
_tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom");
Buffer(UriSvchost);
_tracker.FlushPending();
Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void BuildXPath_Targets2800()
{
Assert.Equal("*[System[(EventID=2800)]]", MdmRebootPolicyTracker.BuildXPath());
}
// ------------------------------------------------------------------ URI extraction
[Fact]
public void ExtractRebootUri_FromVerifiedRealDescription()
{
// Real 2800 text wraps the URI in parentheses and ends with a period — both must be
// excluded from the extracted URI.
var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription);
Assert.Equal(UriSvchost, uri);
}
[Fact]
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
{
var eventData = new Dictionary<string, string> { ["Data0"] = " " + UriSystemGuard + " " };
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription);
Assert.Equal(UriSystemGuard, uri);
}
[Fact]
public void ExtractRebootUri_ReturnsNull_WhenNothingUriLike()
{
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(
new Dictionary<string, string> { ["Data0"] = "plain" }, "no uri here"));
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(null, null));
}
[Fact]
public void ParseEventData_CollectsNamedAndPositionalData()
{
const string xml =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
"<System><EventID>2800</EventID></System>" +
"<EventData><Data Name='Uri'>./Device/Vendor/MSFT/X</Data><Data>positional</Data></EventData>" +
"</Event>";
var parsed = MdmRebootPolicyTracker.ParseEventData(xml);
Assert.Equal("./Device/Vendor/MSFT/X", parsed["Uri"]);
Assert.Equal("positional", parsed["Data0"]);
}
}
}
@@ -17,13 +17,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
FakeTelemetryTransport transport,
out EventSequenceCounter counter,
string sessionId = "sess-1",
string tenantId = "tenant-1")
string tenantId = "tenant-1",
Func<bool>? traceEventsEnabled = null)
{
var tmp = new TempDirectory();
counter = new EventSequenceCounter(new EventSequencePersistence(tmp.File("seq.json")));
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId);
return new TelemetryEventEmitter(transport, counter, sessionId, tenantId, traceEventsEnabled);
}
private static EnrollmentEvent NewTraceEvent(string eventType = "outbound_ip") =>
new EnrollmentEvent
{
EventType = eventType,
Severity = EventSeverity.Trace,
Source = "test",
Timestamp = At,
Phase = EnrollmentPhase.Unknown,
Message = "trace message",
};
private static EnrollmentEvent NewTestEvent(string eventType = "enrollment_complete") =>
new EnrollmentEvent
{
@@ -49,6 +61,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
Assert.Equal("sess-1", evt.SessionId);
Assert.Equal("tenant-1", evt.TenantId);
Assert.NotNull(item);
Assert.Equal(TelemetryItemKind.Event, item.Kind);
Assert.Equal("tenant-1_sess-1", item.PartitionKey);
Assert.Equal(evt.RowKey, item.RowKey);
@@ -174,5 +187,86 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Telemetry.Events
Assert.Equal("test-reason", (string?)data["reason"]);
Assert.Equal("Strong", (string?)data["wgConfidence"]);
}
// ============================================================ Trace-event gate
[Fact]
public void Trace_event_is_dropped_when_gate_is_off()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
var item = emitter.Emit(NewTraceEvent());
Assert.Null(item);
Assert.Equal(0, transport.EnqueueCount);
}
[Fact]
public void Trace_event_flows_when_gate_is_on()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => true);
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
[Fact]
public void Trace_event_flows_when_no_gate_is_wired()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _);
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
[Fact]
public void Gate_off_does_not_suppress_non_trace_events()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => false);
foreach (var severity in new[] { EventSeverity.Info, EventSeverity.Warning, EventSeverity.Error })
{
var evt = NewTestEvent();
evt.Severity = severity;
Assert.NotNull(emitter.Emit(evt));
}
Assert.Equal(3, transport.EnqueueCount);
}
[Fact]
public void Dropped_trace_event_does_not_consume_a_sequence_number()
{
var transport = new FakeTelemetryTransport();
var traceAllowed = true;
// ReSharper disable once AccessToModifiedClosure — deliberate: the gate is read
// per event, mirroring a mid-session remote-config merge.
var emitter = Build(transport, out _, traceEventsEnabled: () => traceAllowed);
emitter.Emit(NewTestEvent()); // seq 1
traceAllowed = false;
emitter.Emit(NewTraceEvent()); // dropped — must not burn seq 2
emitter.Emit(NewTestEvent()); // seq 2
Assert.Equal(2, transport.EnqueueCount);
var first = JObject.Parse(transport.Enqueued[0].PayloadJson);
var second = JObject.Parse(transport.Enqueued[1].PayloadJson);
Assert.Equal(1, (long)first["Sequence"]!);
Assert.Equal(2, (long)second["Sequence"]!);
}
[Fact]
public void Throwing_gate_fails_open_so_diagnostics_are_never_silently_lost()
{
var transport = new FakeTelemetryTransport();
var emitter = Build(transport, out _, traceEventsEnabled: () => throw new InvalidOperationException("boom"));
Assert.NotNull(emitter.Emit(NewTraceEvent()));
Assert.Equal(1, transport.EnqueueCount);
}
}
}
@@ -0,0 +1,57 @@
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Services;
namespace AutopilotMonitor.Agent.V2.Core.Configuration
{
/// <summary>
/// Agent-side evaluation of the config-channel endpoint migration signal
/// (<see cref="AgentConfigResponse.MigrateToApiBaseUrl"/>). Mirrors the kill-switch
/// hardening: only a LIVE fetch is honoured (the on-disk config cache strips the field on
/// write and read, and this check re-validates the outcome as defence-in-depth), and the
/// target must pass <see cref="AgentEndpointMigrationRules"/> even though the backend
/// already validated it — a compromised or misconfigured backend must not be able to
/// re-home agents to an arbitrary host.
/// </summary>
public static class EndpointMigration
{
/// <summary>
/// Returns the normalized migration target when the config carries a valid, live-fetched
/// re-home signal that differs from <paramref name="currentBaseUrl"/>; otherwise null.
/// Never throws — every rejection is logged and degrades to "no migration".
/// </summary>
public static string ResolveTarget(
AgentConfigResponse remoteConfig,
RemoteConfigFetchOutcome fetchOutcome,
string currentBaseUrl,
AgentLogger logger)
{
if (remoteConfig == null || string.IsNullOrWhiteSpace(remoteConfig.MigrateToApiBaseUrl))
return null;
if (fetchOutcome != RemoteConfigFetchOutcome.Succeeded)
{
logger?.Warning(
$"Endpoint migration signal ignored — config source is {fetchOutcome}, not a live fetch.");
return null;
}
if (!AgentEndpointMigrationRules.IsEffectiveMigration(
remoteConfig.MigrateToApiBaseUrl, currentBaseUrl, out var target))
{
// Either the target failed validation (scheme/host allowlist) or it equals the
// URL the agent is already using (steady state during a migration window once
// the new backend's own config echoes nothing).
if (target == null && !AgentEndpointMigrationRules.TryNormalizeTarget(
remoteConfig.MigrateToApiBaseUrl, out _))
{
logger?.Warning(
$"Endpoint migration signal REJECTED — target failed validation: '{remoteConfig.MigrateToApiBaseUrl}'");
}
return null;
}
return target;
}
}
}
@@ -249,18 +249,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
// - DeviceBlocked/DeviceKillSignal/UnblockAt: kill is only honoured from a
// live fetch — a planted cached kill would self-destruct every future
// session, a stale cached kill would outlive the admin removing the rule
// - MigrateToApiBaseUrl: endpoint re-home is only honoured from a live
// fetch — a planted cached URL would silently redirect every future
// session's telemetry to an attacker-chosen host
var liveUnrestricted = config.UnrestrictedMode;
var liveAllowDowngrade = config.AllowAgentDowngrade;
var liveExeHash = config.LatestAgentExeSha256;
var liveBlocked = config.DeviceBlocked;
var liveKill = config.DeviceKillSignal;
var liveUnblockAt = config.UnblockAt;
var liveMigrateUrl = config.MigrateToApiBaseUrl;
config.UnrestrictedMode = false;
config.AllowAgentDowngrade = false;
config.LatestAgentExeSha256 = null;
config.DeviceBlocked = false;
config.DeviceKillSignal = false;
config.UnblockAt = null;
config.MigrateToApiBaseUrl = null;
var json = JsonConvert.SerializeObject(config, Formatting.Indented);
config.UnrestrictedMode = liveUnrestricted;
config.AllowAgentDowngrade = liveAllowDowngrade;
@@ -268,6 +273,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
config.DeviceBlocked = liveBlocked;
config.DeviceKillSignal = liveKill;
config.UnblockAt = liveUnblockAt;
config.MigrateToApiBaseUrl = liveMigrateUrl;
File.WriteAllText(_cacheFilePath, json);
_logger.Debug("Remote config cached to disk");
@@ -299,6 +305,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
config.DeviceBlocked = false;
config.DeviceKillSignal = false;
config.UnblockAt = null;
config.MigrateToApiBaseUrl = null;
}
return config;
}
@@ -0,0 +1,628 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> for
/// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog —
/// and emits ONE aggregated <c>mdm_policy_reboot_required</c> event per burst carrying the URI
/// list (<c>rebootUris</c>, <c>uriCount</c>, <c>firstRebootUri</c>).
/// <para>
/// Verified against session b2e890c1 (2026-07-20): the 2800 description reads
/// <c>The following URI has triggered a reboot: (./Device/...).</c> and records arrive in
/// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record
/// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced
/// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond)
/// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL
/// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an
/// actually-observed <c>system_reboot_detected</c> before attributing anything.
/// </para>
/// <para>
/// <b>Backfill + watermark.</b> Device policies can apply before the agent starts, and a
/// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the
/// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical
/// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so
/// a pre-emit death loses nothing and a completed emit is never repeated.
/// </para>
/// </summary>
internal sealed class MdmRebootPolicyTracker : IDisposable
{
internal const string Channel = "Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin";
internal const int EventId_PolicyRebootRequired = 2800;
internal const string WatermarkStateFileName = "mdm-reboot-watermark.json";
/// <summary>Cap for the emitted <c>rebootUris</c> list — <c>uriCount</c> always carries the uncapped distinct total.</summary>
internal const int MaxUrisPerEvent = 20;
/// <summary>Default quiet-period before a burst of 2800 records is flushed as one event.</summary>
internal const int DefaultDebounceMilliseconds = 10000;
/// <summary>Registry key whose <c>RebootRequired</c> value/subkey flags a pending OMA-DM coalesced reboot.</summary>
internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML";
// Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800
// text wraps the URI in parentheses — ')' is deliberately outside the class so the match
// stops cleanly; trailing sentence punctuation is trimmed after the match.
private static readonly Regex UriPattern = new Regex(
@"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled);
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly int _debounceMilliseconds;
private EventLogWatcher _watcher;
private Timer _debounceTimer;
private sealed class PendingRecord
{
public long RecordId;
public DateTime TimeUtc;
public string Uri; // null when extraction failed
public string Description; // may be null
public bool IsBackfill;
}
// Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as
// WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE
// deliberate difference: the watermark is persisted at FLUSH time, not at claim time —
// records buffered but never flushed (process killed by the very reboot they announce)
// must be re-read and emitted by the post-restart backfill.
private readonly List<PendingRecord> _pending = new List<PendingRecord>();
private long _restartWatermark = -1;
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
private long _maxPersistedRecordId = -1;
private readonly object _stateLock = new object();
/// <summary>
/// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown
/// (key absent or unreadable). Settable test seam; the default reads the live registry.
/// </summary>
internal Func<bool?> RebootRequiredFlagProbe { get; set; } = ReadOmadmRebootRequiredFlag;
public MdmRebootPolicyTracker(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger,
bool backfillEnabled = true,
int backfillLookbackMinutes = 60,
string stateDirectory = null,
int debounceMilliseconds = DefaultDebounceMilliseconds)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_debounceMilliseconds = debounceMilliseconds;
}
public void Start()
{
LoadWatermark();
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
{
BackfillRecentEvents();
// The backfill batch is complete — flush immediately instead of waiting out the
// debounce (live records that raced in are simply included).
FlushPending();
}
else
{
_logger.Info("MdmRebootPolicy backfill disabled by config");
}
}
public void Stop()
{
// Flush any un-emitted burst before tearing down (normal agent shutdown path).
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); }
var timer = Interlocked.Exchange(ref _debounceTimer, null);
if (timer != null)
{
try { timer.Dispose(); } catch { }
}
if (_watcher == null) return;
try
{
_watcher.Enabled = false;
_watcher.Dispose();
_logger.Info("MdmRebootPolicy watcher stopped");
}
catch (Exception ex)
{
_logger.Error("Error stopping MdmRebootPolicy watcher", ex);
}
finally
{
_watcher = null;
}
}
public void Dispose() => Stop();
// -----------------------------------------------------------------------
// Watcher lifecycle
// -----------------------------------------------------------------------
private void StartWatcher()
{
try
{
var query = new EventLogQuery(Channel, PathType.LogName, BuildXPath());
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
_watcher.Enabled = true;
_logger.Info($"MdmRebootPolicy watcher started: {Channel} (EventID {EventId_PolicyRebootRequired})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy watcher access denied for {Channel}: {ex.Message}");
}
catch (Exception ex)
{
_logger.Error($"Failed to start MdmRebootPolicy watcher for {Channel}", ex);
// MON-D1: surface a dead watcher as one-shot telemetry, not just a local log line.
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
collectorName: "MdmRebootPolicyTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
}
}
/// <summary>Targeted-EventID XPath filter. Exposed as internal for tests.</summary>
internal static string BuildXPath() =>
$"*[System[(EventID={EventId_PolicyRebootRequired})]]";
// -----------------------------------------------------------------------
// Event processing — buffer + debounce
// -----------------------------------------------------------------------
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
{
if (e.EventRecord == null) return;
try
{
ProcessRecord(e.EventRecord, isBackfill: false);
}
catch (Exception ex)
{
_logger.Error("Error processing MdmRebootPolicy event", ex);
}
}
private void ProcessRecord(EventRecord record, bool isBackfill)
{
var recordId = record.RecordId ?? -1;
// Cheap dedup gate BEFORE the expensive XML/FormatDescription work.
if (IsAlreadyProcessed(recordId))
return;
string xml = null;
try { xml = record.ToXml(); }
catch { /* fall back to description-only parse below */ }
string description = null;
try { description = record.FormatDescription(); }
catch { /* some events lack formatting resources */ }
var rebootUri = ExtractRebootUri(ParseEventData(xml), description);
ProcessEvent(
eventId: record.Id,
recordId: recordId,
timeCreatedUtc: record.TimeCreated?.ToUniversalTime(),
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
/// <summary>
/// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the
/// WindowsUpdateTracker pattern). The aggregated event is emitted by <see cref="FlushPending"/>
/// — after the debounce quiet-period, at backfill completion, or at <see cref="Stop"/>.
/// </summary>
internal void ProcessEvent(
int eventId,
long recordId,
DateTime? timeCreatedUtc,
string rebootUri,
string formattedDescription,
bool isBackfill)
{
lock (_stateLock)
{
if (recordId >= 0)
{
// Already emitted in a prior run, or already buffered/emitted this run.
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return;
}
_pending.Add(new PendingRecord
{
RecordId = recordId,
TimeUtc = timeCreatedUtc ?? DateTime.UtcNow,
Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri,
Description = formattedDescription,
IsBackfill = isBackfill,
});
}
ArmDebounce();
}
private void ArmDebounce()
{
if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly)
var timer = _debounceTimer;
if (timer == null)
{
var created = new Timer(_ =>
{
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); }
}, null, Timeout.Infinite, Timeout.Infinite);
timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created;
if (!ReferenceEquals(timer, created))
{
try { created.Dispose(); } catch { }
}
}
try { timer.Change(_debounceMilliseconds, Timeout.Infinite); }
catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ }
}
/// <summary>
/// Emits the pending burst as ONE aggregated <c>mdm_policy_reboot_required</c> event and
/// persists the watermark. No-op when nothing is pending. Internal for tests.
/// </summary>
internal void FlushPending()
{
List<PendingRecord> batch;
long watermarkToPersist = -1;
lock (_stateLock)
{
if (_pending.Count == 0) return;
batch = new List<PendingRecord>(_pending);
_pending.Clear();
foreach (var r in batch)
{
if (r.RecordId > _maxPersistedRecordId)
_maxPersistedRecordId = r.RecordId;
}
watermarkToPersist = _maxPersistedRecordId;
}
if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist);
batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc));
var distinctUris = batch
.Where(r => r.Uri != null)
.Select(r => r.Uri)
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri;
var unparsedCount = batch.Count(r => r.Uri == null);
var allBackfill = batch.All(r => r.IsBackfill);
var earliest = batch[0].TimeUtc;
var data = new Dictionary<string, object>
{
{ "windowsEventId", EventId_PolicyRebootRequired },
{ "eventLogChannel", Channel },
{ "recordCount", batch.Count },
{ "uriCount", distinctUris.Count },
{ "backfilled", allBackfill },
};
if (distinctUris.Count > 0)
{
data["rebootUris"] = distinctUris
.OrderBy(u => u, StringComparer.OrdinalIgnoreCase)
.Take(MaxUrisPerEvent)
.ToList();
data["firstRebootUri"] = firstUri;
}
if (unparsedCount > 0)
{
data["unparsedCount"] = unparsedCount;
var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description;
if (sample != null)
data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample;
}
var flag = SafeProbeRebootRequiredFlag();
if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value;
string message;
if (distinctUris.Count == 1)
message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}";
else if (distinctUris.Count > 1)
message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)";
else
message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)";
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
// Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts
// must land on the timeline at when the policies were actually flagged.
Timestamp = earliest,
EventType = Constants.EventTypes.MdmPolicyRebootRequired,
// Neutral observation — reboot-required flags outside ESP DeviceSetup have no
// immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an
// actually-observed system_reboot_detected.
Severity = EventSeverity.Info,
Source = "MdmRebootPolicyWatcher",
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
// Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup
// precedes a coalesced reboot that kills the agent.
ImmediateUpload = true,
});
_logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " +
$"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})");
}
/// <summary>
/// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData
/// value that starts with <c>./</c>, (2) a permissive regex over the formatted description
/// (verified shape: <c>The following URI has triggered a reboot: (./Device/...).</c>).
/// Returns null when nothing URI-like is found — the record still counts into the
/// aggregate (<c>unparsedCount</c> + <c>sampleDescription</c>).
/// </summary>
internal static string ExtractRebootUri(Dictionary<string, string> eventData, string description)
{
if (eventData != null)
{
foreach (var value in eventData.Values)
{
var trimmed = value?.Trim();
if (!string.IsNullOrEmpty(trimmed) && trimmed.StartsWith("./", StringComparison.Ordinal))
return trimmed;
}
}
if (!string.IsNullOrEmpty(description))
{
var match = UriPattern.Match(description);
if (match.Success)
{
// The permissive pattern can swallow sentence punctuation trailing the URI.
// OMA-DM URI segments never end in punctuation — trim it.
var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')');
if (candidate.Length > 2) return candidate;
}
}
return null;
}
/// <summary>
/// Extracts <c>&lt;Data Name="..."&gt;value&lt;/Data&gt;</c> pairs from an event's rendered
/// XML, keyed case-insensitively. Also collects UNNAMED Data elements under positional keys
/// (<c>Data0</c>, <c>Data1</c>, …) — DM-Enterprise events frequently carry positional-only
/// payloads. Namespace-agnostic; returns an empty map on null/malformed XML — never throws.
/// </summary>
internal static Dictionary<string, string> ParseEventData(string xml)
{
var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrEmpty(xml)) return result;
try
{
var doc = XDocument.Parse(xml);
var positionalIndex = 0;
foreach (var el in doc.Descendants().Where(e => e.Name.LocalName == "Data"))
{
var nameAttr = el.Attribute("Name");
var key = nameAttr != null && !string.IsNullOrEmpty(nameAttr.Value)
? nameAttr.Value
: "Data" + positionalIndex++;
if (!result.ContainsKey(key))
result[key] = el.Value;
}
}
catch
{
// Malformed / unexpected XML — best effort, return what we have.
}
return result;
}
// -----------------------------------------------------------------------
// OMA-DM RebootRequired flag probe (fail-soft corroboration)
// -----------------------------------------------------------------------
private bool? SafeProbeRebootRequiredFlag()
{
try
{
return RebootRequiredFlagProbe?.Invoke();
}
catch
{
return null; // corroboration only — never let the probe break the emit
}
}
/// <summary>
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. Both a value and a
/// subkey named <c>RebootRequired</c> count as "pending". Fail-soft: null = unknown
/// (key absent / unreadable), matching the EspSkipConfigurationProbe convention.
/// </summary>
internal static bool? ReadOmadmRebootRequiredFlag()
{
try
{
using (var key = Microsoft.Win32.Registry.LocalMachine.OpenSubKey(OmadmSyncMlKeyPath))
{
if (key == null) return null;
if (key.GetValue("RebootRequired") != null) return true;
using (var sub = key.OpenSubKey("RebootRequired"))
{
return sub != null ? (bool?)true : false;
}
}
}
catch
{
return null;
}
}
// -----------------------------------------------------------------------
// Backfill
// -----------------------------------------------------------------------
private void BackfillRecentEvents()
{
try
{
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
var xpath = $"*[System[(EventID={EventId_PolicyRebootRequired}) and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]";
_logger.Info($"MdmRebootPolicy backfill: scanning {Channel} " +
$"(lookback={_backfillLookbackMinutes}min, restartWatermark={_restartWatermark})");
var query = new EventLogQuery(Channel, PathType.LogName, xpath);
int processed = 0;
using (var reader = new EventLogReader(query))
{
EventRecord record;
while ((record = reader.ReadEvent()) != null)
{
using (record)
{
ProcessRecord(record, isBackfill: true);
processed++;
}
}
}
if (processed == 0)
_logger.Debug($"MdmRebootPolicy backfill: no EventID {EventId_PolicyRebootRequired} records in last {_backfillLookbackMinutes} minutes");
else
_logger.Info($"MdmRebootPolicy backfill: scanned {processed} record(s)");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found during backfill: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy backfill access denied: {ex.Message}");
}
catch (Exception ex)
{
_logger.Warning($"MdmRebootPolicy backfill failed: {ex.Message}");
}
}
// -----------------------------------------------------------------------
// Watermark dedup (cross-restart) — persisted at flush, not at claim
// -----------------------------------------------------------------------
private bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_stateLock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
internal void LoadWatermark()
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
if (!File.Exists(filePath)) return;
try
{
var json = File.ReadAllText(filePath);
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
if (state != null)
{
lock (_stateLock)
{
_restartWatermark = state.LastRecordId;
_maxPersistedRecordId = state.LastRecordId;
}
_logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
}
catch (Exception ex)
{
_logger.Warning($"Failed to load MdmRebootPolicy watermark: {ex.Message}");
}
}
private void PersistWatermark(long recordId)
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
try
{
Directory.CreateDirectory(_stateDirectory);
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
var state = new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow };
var json = JsonConvert.SerializeObject(state, Formatting.Indented);
var tempPath = filePath + ".tmp";
File.WriteAllText(tempPath, json);
File.Copy(tempPath, filePath, overwrite: true);
try { File.Delete(tempPath); } catch { }
}
catch (Exception ex)
{
_logger.Warning($"Failed to persist MdmRebootPolicy watermark: {ex.Message}");
}
}
internal sealed class WatermarkState
{
public long LastRecordId { get; set; }
public DateTime PersistedUtc { get; set; }
}
}
}
@@ -17,6 +17,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete
/// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure
///
/// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was
/// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker
/// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves
/// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in
/// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot.
///
/// Raises <see cref="FinalizingSetupPhaseTriggered"/>, <see cref="WhiteGloveCompleted"/>,
/// and <see cref="EspFailureDetected"/>. Cross-notifies the <see cref="HelloTracker"/>
/// on Hello wizard start and ESP exit so Hello timers can react.
@@ -12,6 +12,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
{
public string CollectorType => "command_allowlisted";
private const int CommandTimeoutMs = 30000;
/// <summary>
/// Returns what the reader captured, or an empty string if it did not finish
/// (a killed process can leave the read faulted or still pending).
/// </summary>
private static string DrainOrEmpty(System.Threading.Tasks.Task<string> readTask)
{
try
{
return readTask.Wait(2000) ? readTask.Result : string.Empty;
}
catch
{
return string.Empty;
}
}
public Dictionary<string, object> Execute(GatherRule rule, GatherRuleContext context)
{
var data = new Dictionary<string, object>();
@@ -81,12 +99,29 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
using (var process = Process.Start(psi))
{
var output = process.StandardOutput.ReadToEnd();
var error = process.StandardError.ReadToEnd();
process.WaitForExit(30000); // 30 second timeout
// Read asynchronously: a synchronous ReadToEnd() blocks until the pipe
// closes, which only happens when the process exits — the WaitForExit
// timeout below would never be reached and a hung command would pin
// this worker forever.
var outputTask = process.StandardOutput.ReadToEndAsync();
var errorTask = process.StandardError.ReadToEndAsync();
var exited = process.WaitForExit(CommandTimeoutMs);
if (!exited)
{
context.Logger.Warning(
$"Command timed out after {CommandTimeoutMs / 1000}s, killing: {command} (Rule: {rule.RuleId})");
try { process.Kill(); } catch { /* already gone */ }
// Give the readers a moment to drain once the pipes close.
try { process.WaitForExit(5000); } catch { /* best effort */ }
data["timed_out"] = true;
}
data["command"] = command;
data["exit_code"] = process.ExitCode;
data["exit_code"] = exited ? process.ExitCode : -1;
var output = DrainOrEmpty(outputTask);
var error = DrainOrEmpty(errorTask);
if (!string.IsNullOrWhiteSpace(output))
{
@@ -17,6 +17,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (string.IsNullOrEmpty(logName))
return data;
// Guard: only allow enrollment-relevant channels
if (!GatherRuleGuards.IsEventLogChannelAllowed(logName, context.UnrestrictedMode))
return context.EmitSecurityWarning(rule, "eventlog", logName);
// Optional: escalate severity when matching entries are found
string severityIfFound = null;
rule.Parameters?.TryGetValue("severityIfFound", out severityIfFound);
@@ -34,11 +34,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (filePath == null)
return null; // Token present but no user logged on — skip silently
// Apply IME log path override if set
// Apply IME log path override if set. The override comes from the local
// --ime-log-path flag only (never from remote config), so the operator
// running it is already a local admin — it relaxes the allowlist the same
// way unrestricted mode does, while the hard blocks below still apply.
var localOverride = false;
if (!string.IsNullOrEmpty(context.ImeLogPathOverride))
{
var fileName = Path.GetFileName(filePath);
filePath = Path.Combine(context.ImeLogPathOverride, fileName);
localOverride = true;
}
var relaxAllowlist = context.UnrestrictedMode || localOverride;
// Guard: only allow enrollment-relevant file paths. Checked before resolution
// so a blocked target is reported even when no file matches it. A wildcard
// target is checked by its directory — Path.GetFullPath rejects '*' and '?'.
var guardPath = HasWildcard(Path.GetFileName(filePath))
? Path.GetDirectoryName(filePath)
: filePath;
if (!GatherRuleGuards.IsFilePathAllowed(guardPath, relaxAllowlist, userProfilePath))
{
context.EmitSecurityWarning(rule, "logparser", filePath);
return null;
}
// Get parameters
@@ -87,18 +106,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
foreach (var resolvedPath in resolvedPaths)
{
// Re-check each resolved path: wildcard expansion can surface files
// reached through a junction that the directory check did not cover.
if (!GatherRuleGuards.IsFilePathAllowed(resolvedPath, relaxAllowlist, userProfilePath))
{
context.EmitSecurityWarning(rule, "logparser", resolvedPath);
continue;
}
ProcessLogFile(resolvedPath, rule, pattern, trackPosition, maxLines, isTextMode, context);
}
return null;
}
private static bool HasWildcard(string fileNamePart)
=> !string.IsNullOrEmpty(fileNamePart) &&
(fileNamePart.Contains("*") || fileNamePart.Contains("?"));
private static List<string> ResolveLogPaths(string filePath, string ruleId, GatherRuleContext context)
{
var fileNamePart = Path.GetFileName(filePath);
// No wildcards — single file
if (!fileNamePart.Contains("*") && !fileNamePart.Contains("?"))
if (!HasWildcard(fileNamePart))
{
if (File.Exists(filePath))
return new List<string> { filePath };
@@ -1,6 +1,9 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
@@ -28,6 +31,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
private readonly HashSet<string> _phaseRulesExecuted = new HashSet<string>();
private CountdownEvent _startupRulesLatch; // non-null only while startup rules are pending
// Phase-scope + emit-mode state (ActivePhases / ActiveFromPhase / EmitMode on GatherRule).
// Guarded by _scopeLock; collector work always runs outside the lock. The state deliberately
// survives UpdateRules config refreshes — an agent restart causes at most one re-emit per
// on_change rule (acceptable under the per-enrollment lifecycle).
private readonly object _scopeLock = new object();
private EnrollmentPhase _currentPhase = EnrollmentPhase.Unknown;
private readonly HashSet<string> _fromPhaseLatched = new HashSet<string>();
private readonly Dictionary<string, string> _lastEmittedHash = new Dictionary<string, string>();
private readonly Dictionary<string, (int Count, DateTime SinceUtc)> _suppressed =
new Dictionary<string, (int Count, DateTime SinceUtc)>();
private readonly HashSet<string> _scopeBypassLogged = new HashSet<string>();
/// <summary>
/// When true, phase scoping (ActivePhases / ActiveFromPhase) is bypassed and scoped rules
/// execute unconditionally. Used by the standalone <c>--run-gather-rules</c> diagnostic
/// mode, which has no enrollment phase context. EmitMode is unaffected.
/// </summary>
public bool IgnorePhaseScope { get; set; }
/// <summary>
/// When true, guardrails are relaxed: all registry, WMI, and command targets are allowed.
/// File paths allow everything except C:\Users. Set from tenant configuration.
@@ -89,10 +111,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_activeRules = rules.Where(r => r.Enabled).ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait
var pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait.
// Scoped startup rules that are not yet in scope are deferred: they run once from
// OnPhaseChanged when their scope activates (same _startupRulesExecuted dedup — a
// rule runs either here or there, never both). Unscoped rules behave as before.
List<GatherRule> pendingStartup;
lock (_scopeLock)
{
// A config refresh may deliver new from-phase rules after their phase was
// already reached — evaluate latches against the current phase first.
EvaluateFromPhaseLatchesLocked();
pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.Where(r => IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
if (pendingStartup.Count > 0)
{
@@ -101,7 +140,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
ThreadPool.QueueUserWorkItem(_ =>
{
try { ExecuteRule(rule); }
@@ -110,18 +148,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// Set up interval timers
// Set up interval timers. Timers for scoped rules keep running while out of scope —
// the tick gate makes an out-of-scope tick a free no-op, avoiding start/stop
// lifecycle races on phase transitions.
foreach (var rule in _activeRules.Where(r => r.Trigger == "interval" && r.IntervalSeconds.HasValue))
{
var interval = TimeSpan.FromSeconds(rule.IntervalSeconds.Value);
var intervalRule = rule;
var interval = TimeSpan.FromSeconds(intervalRule.IntervalSeconds.Value);
var timer = new Timer(
_ => ExecuteRule(rule),
_ =>
{
if (!IsRuleInScope(intervalRule))
return;
ExecuteRule(intervalRule);
},
null,
interval, // Initial delay = one interval
interval
);
_intervalTimers[rule.RuleId] = timer;
_logger.Info($" Interval rule {rule.RuleId} scheduled every {rule.IntervalSeconds}s");
_intervalTimers[intervalRule.RuleId] = timer;
_logger.Info($" Interval rule {intervalRule.RuleId} scheduled every {intervalRule.IntervalSeconds}s");
}
_logger.Info($"GatherRuleExecutor: {_activeRules.Count(r => r.Trigger == "startup")} startup, " +
@@ -130,16 +176,95 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// <summary>
/// Called when a phase change event occurs - executes rules triggered by phase changes
/// (<c>phase_change</c> on entering the new phase, <c>phase_exit</c> on leaving the old one).
/// </summary>
public void OnPhaseChanged(EnrollmentPhase newPhase)
{
var phaseName = newPhase.ToString();
// phase_exit runs FIRST and against the phase being LEFT: both the TriggerPhase match
// and the scope gate must see the old phase, so this block has to complete before
// _currentPhase moves below. A transition INTO Failed still fires the exit rules of the
// phase that failed — that snapshot is the whole point at a failure boundary.
List<GatherRule> exitRules = null;
lock (_scopeLock)
{
var previousPhase = _currentPhase;
if (previousPhase != EnrollmentPhase.Unknown && previousPhase != newPhase)
{
var previousPhaseName = previousPhase.ToString();
exitRules = new List<GatherRule>();
foreach (var rule in _activeRules)
{
if (rule.Trigger != "phase_exit") continue;
if (!string.IsNullOrEmpty(rule.TriggerPhase)
&& !string.Equals(rule.TriggerPhase, previousPhaseName, StringComparison.OrdinalIgnoreCase))
continue;
// Scope gate before the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) slot. IsRuleInScopeLocked still sees the old phase.
if (!IsRuleInScopeLocked(rule)) continue;
// "exit:" prefix keeps this key space disjoint from the phase_change keys.
if (!_phaseRulesExecuted.Add($"exit:{rule.RuleId}|{previousPhaseName}"))
{
_logger.Debug($"Phase-exit rule {rule.RuleId} already executed for exit of {previousPhaseName}, skipping");
continue;
}
_logger.Info($"Phase exit triggered rule {rule.RuleId} (left phase: {previousPhaseName}, now: {phaseName})");
exitRules.Add(rule);
}
}
}
if (exitRules != null)
{
foreach (var rule in exitRules)
{
var exitRule = rule;
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(exitRule));
}
}
// Update the phase + from-phase latches so deferred startup activation and
// the phase_change rules below all evaluate against the NEW phase.
List<GatherRule> deferredStartup;
lock (_scopeLock)
{
_currentPhase = newPhase;
EvaluateFromPhaseLatchesLocked();
deferredStartup = _activeRules
.Where(r => r.Trigger == "startup" && HasPhaseScope(r)
&& !_startupRulesExecuted.Contains(r.RuleId)
&& IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in deferredStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
foreach (var rule in deferredStartup)
{
var deferredRule = rule;
_logger.Info($"Phase {phaseName} activated deferred startup rule {deferredRule.RuleId}");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(deferredRule));
}
foreach (var rule in _activeRules.Where(r => r.Trigger == "phase_change"))
{
if (string.IsNullOrEmpty(rule.TriggerPhase) ||
string.Equals(rule.TriggerPhase, phaseName, StringComparison.OrdinalIgnoreCase))
{
// Scope gate BEFORE the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) execution slot.
if (!IsRuleInScope(rule))
continue;
// Deduplicate: only fire once per (ruleId, phase) combination
var deduplicationKey = $"{rule.RuleId}|{phaseName}";
if (!_phaseRulesExecuted.Add(deduplicationKey))
@@ -163,6 +288,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
if (string.Equals(rule.TriggerEventType, eventType, StringComparison.OrdinalIgnoreCase))
{
if (!IsRuleInScope(rule))
continue;
_logger.Info($"Event triggered rule {rule.RuleId} (event: {eventType})");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
}
@@ -192,6 +320,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (result.Count > 0)
{
// EmitMode "on_change": hash the raw collector result (before ruleId/title
// injection) and only emit when it differs from the last emitted one. An
// empty result never reaches this point, so an emitOnlyIfExists miss neither
// emits nor updates the hash — composing to "one event on appearance, then
// only on change".
if (IsOnChangeMode(rule) && !ShouldEmitOnChange(rule, result))
return;
result["ruleId"] = rule.RuleId;
result["ruleTitle"] = rule.Title;
@@ -229,6 +365,199 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// ===== Phase scope (ActivePhases / ActiveFromPhase) =====
private static bool HasPhaseScope(GatherRule rule)
{
return (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
|| !string.IsNullOrEmpty(rule.ActiveFromPhase);
}
// Internal so the scope matrix is testable synchronously (no timer/ThreadPool races).
internal bool IsRuleInScope(GatherRule rule)
{
lock (_scopeLock)
{
return IsRuleInScopeLocked(rule);
}
}
private bool IsRuleInScopeLocked(GatherRule rule)
{
if (!HasPhaseScope(rule))
return true; // unscoped — legacy behavior
if (IgnorePhaseScope)
{
if (_scopeBypassLogged.Add(rule.RuleId))
_logger.Info($"Gather rule {rule.RuleId}: phase scope ignored (diagnostic mode)");
return true;
}
// Both fields set is rejected by backend validation; defensively prefer ActivePhases.
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
{
// No phase signal yet (or scope tokens that never match, e.g. "Unknown"):
// scoped rules stay inactive.
if (_currentPhase == EnrollmentPhase.Unknown)
return false;
var phaseName = _currentPhase.ToString();
foreach (var phase in rule.ActivePhases)
{
if (string.Equals(phase, phaseName, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
return _fromPhaseLatched.Contains(rule.RuleId);
}
/// <summary>
/// Latches every from-phase rule whose threshold the current phase has reached.
/// Never latches on Unknown/Failed (Failed=99 would ordinal-satisfy every threshold);
/// once latched a rule stays active for the session — including through Failed.
/// Caller must hold <see cref="_scopeLock"/>.
/// </summary>
private void EvaluateFromPhaseLatchesLocked()
{
if (_currentPhase == EnrollmentPhase.Unknown || _currentPhase == EnrollmentPhase.Failed)
return;
foreach (var rule in _activeRules)
{
if (string.IsNullOrEmpty(rule.ActiveFromPhase)) continue;
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0) continue; // ActivePhases wins
if (_fromPhaseLatched.Contains(rule.RuleId)) continue;
if (Enum.TryParse<EnrollmentPhase>(rule.ActiveFromPhase, ignoreCase: true, out var fromPhase)
&& fromPhase != EnrollmentPhase.Unknown
&& fromPhase != EnrollmentPhase.Failed
&& (int)_currentPhase >= (int)fromPhase)
{
_fromPhaseLatched.Add(rule.RuleId);
_logger.Info($"Gather rule {rule.RuleId}: from-phase scope latched at {_currentPhase} (activeFromPhase={rule.ActiveFromPhase})");
}
}
}
// ===== Emit mode (on_change) =====
private static bool IsOnChangeMode(GatherRule rule)
{
return string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// on_change decision for a non-empty collector result: true = emit (first result for the
/// rule, or the result changed), false = suppress. Hashes the result BEFORE ruleId/title
/// injection. On emit after a suppression streak, injects <c>suppressedPolls</c> and
/// <c>suppressedSinceUtc</c> into the result so the gap is observable.
/// Internal so the state machine is testable synchronously (no timer/ThreadPool races).
/// </summary>
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result)
{
var hash = ComputeCanonicalHash(result);
bool emit;
lock (_scopeLock)
{
string lastHash;
if (_lastEmittedHash.TryGetValue(rule.RuleId, out lastHash) && lastHash == hash)
{
(int Count, DateTime SinceUtc) streak;
_suppressed[rule.RuleId] = _suppressed.TryGetValue(rule.RuleId, out streak)
? (streak.Count + 1, streak.SinceUtc)
: (1, DateTime.UtcNow);
emit = false;
}
else
{
_lastEmittedHash[rule.RuleId] = hash;
(int Count, DateTime SinceUtc) streak;
if (_suppressed.TryGetValue(rule.RuleId, out streak) && streak.Count > 0)
{
result["suppressedPolls"] = streak.Count;
result["suppressedSinceUtc"] = streak.SinceUtc.ToString("o", CultureInfo.InvariantCulture);
}
_suppressed.Remove(rule.RuleId);
emit = true;
}
}
if (!emit)
_logger.Debug($"Gather rule {rule.RuleId}: result unchanged, emit suppressed (on_change)");
return emit;
}
/// <summary>
/// Canonical hash of a collector result: keys ordinal-sorted at every dictionary level,
/// values rendered with invariant formatting, then SHA-256. Internal for direct test
/// coverage of the canonicalization.
/// </summary>
internal static string ComputeCanonicalHash(Dictionary<string, object> result)
{
var sb = new StringBuilder();
AppendCanonicalValue(sb, result);
using (var sha = SHA256.Create())
{
var hashBytes = sha.ComputeHash(Encoding.UTF8.GetBytes(sb.ToString()));
return Convert.ToBase64String(hashBytes);
}
}
private static void AppendCanonicalValue(StringBuilder sb, object value)
{
if (value == null)
{
sb.Append("<null>");
return;
}
// string first — it is IEnumerable and must not be treated as a sequence
if (value is string str)
{
sb.Append('"').Append(str).Append('"');
return;
}
if (value is System.Collections.IDictionary dict)
{
sb.Append('{');
var keys = dict.Keys.Cast<object>()
.Select(k => k?.ToString() ?? string.Empty)
.OrderBy(k => k, StringComparer.Ordinal);
foreach (var key in keys)
{
sb.Append(key).Append('=');
AppendCanonicalValue(sb, dict[key]);
sb.Append(';');
}
sb.Append('}');
return;
}
if (value is System.Collections.IEnumerable sequence)
{
sb.Append('[');
foreach (var item in sequence)
{
AppendCanonicalValue(sb, item);
sb.Append(';');
}
sb.Append(']');
return;
}
if (value is IFormattable formattable)
{
sb.Append(formattable.ToString(null, CultureInfo.InvariantCulture));
return;
}
sb.Append(value);
}
private EventSeverity ParseSeverity(string severity)
{
return GatherRuleContext.ParseSeverity(severity);
@@ -245,7 +574,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// <summary>
/// Blocks until all startup rules that were queued by the most recent <see cref="UpdateRules"/>
/// call have finished executing, or the timeout elapses.
/// call have finished executing, or the timeout elapses. Deferred scoped startup rules
/// (waiting for their phase scope) are not part of the latch.
/// Returns true if all rules completed within the timeout; false if timed out.
/// </summary>
public bool WaitForStartupRules(int timeoutSeconds = 120)
@@ -33,6 +33,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public static readonly IReadOnlyList<string> AllowedFilePrefixes;
public static readonly IReadOnlyList<string> AllowedWmiQueryPrefixes;
public static readonly IReadOnlyCollection<string> AllowedCommands;
public static readonly IReadOnlyList<string> AllowedEventLogChannels;
// Hard blocks: never allowed, even in unrestricted mode
private static readonly string BlockedUsersPrefix = Path.GetFullPath(@"C:\Users");
@@ -51,6 +52,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// Hard limit: maximum command length (even in unrestricted mode)
private const int MaxCommandLength = 2000;
// Event log channels that are never readable, even in unrestricted mode.
// Security carries the audit trail of user behaviour; the PowerShell channels
// carry script-block logging, which routinely contains secrets in clear text.
// Mirrored in rules/guardrails.json ("blockedEventLogChannels") for the portal
// to display — enforcement lives here so a parse error cannot lift it.
private static readonly string[] HardBlockedEventLogChannels = new[]
{
"Security",
"Microsoft-Windows-PowerShell",
"Windows PowerShell",
"Microsoft-Windows-Sysmon",
};
// Hard-blocked command patterns: never allowed, even in unrestricted mode.
// These prevent privilege escalation, persistence, and data exfiltration.
private static readonly string[] HardBlockedCommandPatterns = new[]
@@ -85,6 +99,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
var commands = FlattenCategorized(obj, "allowedCommands", "commands");
AllowedCommands = new HashSet<string>(commands, StringComparer.OrdinalIgnoreCase);
AllowedEventLogChannels = FlattenCategorized(obj, "eventLogChannels", "channels");
return;
}
}
@@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
AllowedFilePrefixes = new List<string>();
AllowedWmiQueryPrefixes = new List<string>();
AllowedCommands = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
AllowedEventLogChannels = new List<string>();
}
private static string LoadEmbeddedGuardrails()
@@ -257,6 +274,44 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
(trimmed.Length == prefix.Length || char.IsWhiteSpace(trimmed[prefix.Length])));
}
/// <summary>
/// Returns true if the event log channel matches an allowed entry with
/// boundary matching (next char must be '/' or end of string), so that
/// "Microsoft-Windows-AAD" admits "Microsoft-Windows-AAD/Operational"
/// but not "Microsoft-Windows-AADSomethingElse".
/// </summary>
public static bool IsEventLogChannelAllowed(string channel)
=> IsEventLogChannelAllowed(channel, unrestrictedMode: false);
/// <summary>
/// Returns true if the event log channel is allowed.
/// When unrestrictedMode is true, all channels are allowed except the
/// hard-blocked ones (audit trail and script-block logging).
/// </summary>
public static bool IsEventLogChannelAllowed(string channel, bool unrestrictedMode)
{
if (string.IsNullOrEmpty(channel))
return false;
var trimmed = channel.Trim();
// Hard blocks apply even in unrestricted mode
foreach (var blocked in HardBlockedEventLogChannels)
{
if (MatchesChannel(trimmed, blocked))
return false;
}
if (unrestrictedMode)
return true;
return AllowedEventLogChannels.Any(allowed => MatchesChannel(trimmed, allowed));
}
private static bool MatchesChannel(string channel, string prefix)
=> channel.StartsWith(prefix, StringComparison.OrdinalIgnoreCase) &&
(channel.Length == prefix.Length || channel[prefix.Length] == '/');
/// <summary>
/// Returns true if the command (trimmed) exactly matches an entry in the allowlist.
/// Exact matching is intentional — no prefix spoofing possible.
@@ -136,6 +136,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Security
}
watcher.Start();
// Re-check after arming: Dispose() may have run between the lock release above and
// Start() — it then stopped a not-yet-armed watcher and this thread armed it right
// back, leaving a WMI subscription no code path could ever reach again. If disposal
// won that race, tear the watcher down here.
bool disposedDuringArm;
lock (_lock)
{
disposedDuringArm = _disposed;
if (disposedDuringArm) _startWatcher = null;
}
if (disposedDuringArm)
{
try { watcher.EventArrived -= OnProcessStartTrace; } catch { }
try { watcher.Stop(); } catch { }
try { watcher.Dispose(); } catch { }
return;
}
_logger.Info($"[ConsoleBypassWatcher] watching Win32_ProcessStartTrace for " +
$"{string.Join(", ", WatchedProcessNames)} (interactive-session + bare command line)");
}
@@ -335,6 +335,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
osBuildChangedProvider: () => osBuildChangeHost.BuildChanged));
}
// MDM reboot-policy watcher — subscribes to DeviceManagement-Enterprise-Diagnostics-
// Provider/Admin EventID 2800 and attributes the "unexpected reboot + second sign-in"
// pattern to the device-assigned policy URIs that forced the coalesced reboot.
// Agent-native collector (NOT an IAgentAnalyzer); the config flag is an internal
// steering lever only.
if (collectors.MdmRebootPolicyWatcherEnabled)
{
hosts.Add(new MdmRebootPolicyWatcherHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
backfillLookbackMinutes: collectors.MdmRebootPolicyBackfillLookbackMinutes,
stateDirectory: _stateDirectory));
}
// ----- Peripheral hosts (event-only; driven by remote-config toggles) --------------
// V1 parity (PeriodicCollectorManager) — combine Performance + AgentSelfMetrics under
@@ -86,6 +86,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// Production always passes null → Start() creates the real DeadlineScheduler.
private readonly IDeadlineScheduler? _schedulerOverride;
// Live accessor for TenantConfiguration.SendTraceEvents — read per event by the
// TelemetryEventEmitter so a mid-session remote-config merge takes effect. Null
// (tests / legacy callers) means trace events always flow.
private readonly Func<bool>? _traceEventsEnabled;
// Max-lifetime watchdog (M4.6.α). Timer is armed in Start() when _agentMaxLifetime != null.
private System.Threading.Timer? _maxLifetimeTimer;
private int _terminatedFired;
@@ -166,7 +171,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
TimeSpan? terminalDrainTimeout = null,
TimeSpan? agentMaxLifetime = null,
int uploadBatchSize = 100,
IDeadlineScheduler? schedulerOverride = null)
IDeadlineScheduler? schedulerOverride = null,
Func<bool>? traceEventsEnabled = null)
{
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
@@ -206,6 +212,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
_uploadBatchSize = uploadBatchSize;
_schedulerOverride = schedulerOverride;
_traceEventsEnabled = traceEventsEnabled;
}
/// <summary>
@@ -402,7 +409,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// strukturelle Abhängigkeit erlischt damit, Architektur-Baseline-Test gate
// shrinks to exactly two permitted callers.
_eventSequenceCounter = new EventSequenceCounter(_eventSequencePersistence);
var eventEmitter = new TelemetryEventEmitter(_transport, _eventSequenceCounter, _sessionId, _tenantId);
var eventEmitter = new TelemetryEventEmitter(
_transport, _eventSequenceCounter, _sessionId, _tenantId, _traceEventsEnabled);
_signalEmitter = new TelemetrySignalEmitter(_transport, _sessionId, _tenantId);
_transitionEmitter = new TelemetryTransitionEmitter(_transport, _sessionId, _tenantId);
_timelineEmitter = new EventTimelineEmitter(eventEmitter);
@@ -0,0 +1,68 @@
#nullable enable
using System;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Standalone host for the <see cref="MdmRebootPolicyTracker"/> — watches
/// <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> (EventID
/// 2800) for MDM policies that force a coalesced reboot during ESP DeviceSetup and forwards
/// them via the shared single-rail ingress. Mirrors <see cref="WindowsUpdateWatcherHost"/>:
/// builds its own <see cref="InformationalEventPost"/> and delegates Start/Stop to the tracker.
/// </summary>
internal sealed class MdmRebootPolicyWatcherHost : ICollectorHost
{
public string Name => "MdmRebootPolicyTracker";
private readonly MdmRebootPolicyTracker _tracker;
private readonly AgentLogger _logger;
private int _disposed;
public MdmRebootPolicyWatcherHost(
string sessionId,
string tenantId,
AgentLogger logger,
ISignalIngressSink ingress,
IClock clock,
int backfillLookbackMinutes,
string stateDirectory)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
var post = new InformationalEventPost(ingress, clock, logger);
_tracker = new MdmRebootPolicyTracker(
sessionId: sessionId,
tenantId: tenantId,
post: post,
logger: logger,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory);
}
public void Start()
{
_tracker.Start();
_logger.Info("MdmRebootPolicyWatcherHost: started.");
}
public void Stop()
{
try { _tracker.Stop(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicyWatcherHost: stop failed: {ex.Message}"); }
}
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) == 1) return;
Stop();
}
}
}
@@ -30,12 +30,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
private readonly string _sessionId;
private readonly string _tenantId;
private readonly string _partitionKey;
private readonly Func<bool>? _traceEventsEnabled;
public TelemetryEventEmitter(
ITelemetryTransport transport,
EventSequenceCounter sequenceCounter,
string sessionId,
string tenantId)
string tenantId,
Func<bool>? traceEventsEnabled = null)
{
if (string.IsNullOrEmpty(sessionId)) throw new ArgumentException("SessionId is mandatory.", nameof(sessionId));
if (string.IsNullOrEmpty(tenantId)) throw new ArgumentException("TenantId is mandatory.", nameof(tenantId));
@@ -44,13 +46,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
_sessionId = sessionId;
_tenantId = tenantId;
_partitionKey = $"{tenantId}_{sessionId}";
_traceEventsEnabled = traceEventsEnabled;
}
/// <summary>
/// Weist Sequence zu, serialisiert im Legacy-Wire-Format und enqueued via Transport.
/// Thread-safe (EventSequenceCounter + ITelemetryTransport.Enqueue beide unter Lock).
/// <para>
/// Returns <c>null</c> when the event was suppressed by the trace-event gate — the
/// only drop path. Both permitted callers ignore the return value.
/// </para>
/// </summary>
internal TelemetryItem Emit(EnrollmentEvent evt)
internal TelemetryItem? Emit(EnrollmentEvent evt)
{
if (evt == null) throw new ArgumentNullException(nameof(evt));
if (string.IsNullOrEmpty(evt.EventType))
@@ -58,6 +65,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
throw new ArgumentException("EnrollmentEvent.EventType is mandatory.", nameof(evt));
}
// Trace-event gate (TenantConfiguration.SendTraceEvents, default on). Dropped
// before the sequence counter advances, so suppression leaves no gaps in the
// per-session sequence and the timeline stays contiguous. Evaluated per event
// rather than captured at construction: remote config is merged into the live
// AgentConfiguration mid-session, and the gate must follow it.
if (evt.Severity == EventSeverity.Trace && !TraceEventsEnabled())
{
return null;
}
evt.Sequence = _sequenceCounter.Next();
if (string.IsNullOrEmpty(evt.SessionId)) evt.SessionId = _sessionId;
if (string.IsNullOrEmpty(evt.TenantId)) evt.TenantId = _tenantId;
@@ -77,5 +94,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Telemetry.Events
return _transport.Enqueue(draft);
}
/// <summary>
/// Fail-open: no gate wired (legacy ctor / tests) or a throwing accessor means trace
/// events flow. The gate suppresses diagnostics, so a broken accessor must not be able
/// to silently blind the timeline.
/// </summary>
private bool TraceEventsEnabled()
{
if (_traceEventsEnabled == null) return true;
try
{
return _traceEventsEnabled();
}
catch
{
return true;
}
}
}
}
@@ -224,6 +224,11 @@ namespace AutopilotMonitor.Agent.V2
// Step 4 — run the executor and wait.
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride))
{
// Diagnostic mode has no enrollment phase context — scoped rules
// (activePhases / activeFromPhase) execute unconditionally here.
executor.IgnorePhaseScope = true;
logger.Info("--run-gather-rules: phase scoping bypassed (no phase context in diagnostic mode).");
executor.UpdateRules(rules);
if (consoleMode) Console.WriteLine($"Running {startupRules.Count} rule(s)...");
@@ -213,6 +213,8 @@ namespace AutopilotMonitor.Agent.V2
// Control-channel kill-switch: a live-fetch DeviceKillSignal terminates the agent
// BEFORE mTLS/registration spin-up. Covers agents the telemetry-channel kill can
// never reach (paused drain, empty spool) at their next process start / boot.
// Checked BEFORE any endpoint migration — a kill from the current backend is
// terminal, not re-homed.
if (CheckConfigKillSignal(
runtimeConfig.RemoteConfig,
runtimeConfig.RemoteConfigService.LastFetchOutcome,
@@ -223,6 +225,37 @@ namespace AutopilotMonitor.Agent.V2
return 0;
}
// Endpoint migration (config-channel re-home): the backend served a validated
// MigrateToApiBaseUrl on a live fetch. Rebuild Phase 3 clients against the new base
// URL and re-run Phase 4 there (single hop — the second Resolve cannot migrate
// again), then re-check the kill signal against the NEW backend's config.
if (runtimeConfig.PendingMigrationTarget != null)
{
logger.Warning(
$"Endpoint migration: backend re-homed this agent from {bootstrap.AgentConfig.ApiBaseUrl} " +
$"to {runtimeConfig.PendingMigrationTarget} — rebuilding backend clients.");
bootstrap.AgentConfig.ApiBaseUrl = runtimeConfig.PendingMigrationTarget;
// Best-effort disposal of the old-URL client stack before it goes out of scope.
try { auth.BackendApiClient?.Dispose(); } catch { }
auth = Runtime.BackendClientFactory.BuildAuthClients(bootstrap.AgentConfig, GetAgentVersion(), logger);
runtimeConfig = Runtime.AgentRuntimeConfig.Resolve(
bootstrap.AgentConfig, auth, GetAgentVersion(), consoleMode, logger,
allowEndpointMigration: false);
if (CheckConfigKillSignal(
runtimeConfig.RemoteConfig,
runtimeConfig.RemoteConfigService.LastFetchOutcome,
dataDirectory, stateSubdir,
() => new CleanupService(bootstrap.AgentConfig, logger),
logger, consoleMode))
{
return 0;
}
}
// Phase 5 (mTLS HttpClient + BackendTelemetryUploader) is encapsulated in
// BackendClientFactory. Construction failures map to V1-parity exit codes
// (4 = mTLS, 5 = uploader). See Runtime/BackendClientFactory.cs.
@@ -32,7 +32,8 @@ namespace AutopilotMonitor.Agent.V2.Runtime
BackendAuthBundle auth,
string agentVersion,
bool consoleMode,
AgentLogger logger)
AgentLogger logger,
bool allowEndpointMigration = true)
{
if (agentConfig == null) throw new ArgumentNullException(nameof(agentConfig));
if (auth == null) throw new ArgumentNullException(nameof(auth));
@@ -49,6 +50,25 @@ namespace AutopilotMonitor.Agent.V2.Runtime
.FetchConfigAsync(retryOnTransientErrors: true)
.GetAwaiter().GetResult();
// Endpoint migration (config-channel re-home): decided BEFORE merge / binary-
// integrity / bootstrap-cleanup so none of those side effects run against a config
// from the backend being abandoned — a migration stub may advertise stale agent
// hashes, and a spurious IntegrityCheckFailed + force-update against them would be
// actively harmful. RunAgent rebuilds Phase 3 clients on the new base URL and calls
// Resolve again with allowEndpointMigration:false (single hop — the new backend's
// config cannot chain another migration).
if (allowEndpointMigration)
{
var migrationTarget = EndpointMigration.ResolveTarget(
remoteConfig, remoteConfigService.LastFetchOutcome, agentConfig.ApiBaseUrl, logger);
if (migrationTarget != null)
{
return new RuntimeConfigBundle(
remoteConfigService, remoteConfig, mergeResult: null,
pendingMigrationTarget: migrationTarget);
}
}
// Project remote tenant-controlled knobs onto the runtime AgentConfiguration so that
// downstream consumers (CleanupService, SummaryDialogLauncher, StartupEnvironmentProbes,
// DiagnosticsPackageService, EnrollmentTerminationHandler, the logger, the watchdog)
@@ -71,7 +91,7 @@ namespace AutopilotMonitor.Agent.V2.Runtime
TryCleanupBootstrapConfig(agentConfig, agentVersion, logger);
return new RuntimeConfigBundle(remoteConfigService, remoteConfig, configMergeResult);
return new RuntimeConfigBundle(remoteConfigService, remoteConfig, configMergeResult, pendingMigrationTarget: null);
}
/// <summary>
@@ -145,21 +165,30 @@ namespace AutopilotMonitor.Agent.V2.Runtime
/// <c>rotate_config</c> ServerAction callback), the fetched <see cref="AgentConfigResponse"/>
/// and the <see cref="RemoteConfigMergeResult"/> diff snapshot (used by the
/// <c>EmitUnrestrictedModeAuditIfChanged</c> lifecycle hook).
/// <para>
/// When <see cref="PendingMigrationTarget"/> is non-null the bundle is a partial result:
/// the fetch found a live endpoint-migration signal and Resolve returned before merge /
/// integrity / bootstrap-cleanup (<see cref="MergeResult"/> is null). RunAgent must
/// rebuild the backend clients on the target URL and Resolve again.
/// </para>
/// </summary>
internal sealed class RuntimeConfigBundle
{
public RemoteConfigService RemoteConfigService { get; }
public AgentConfigResponse RemoteConfig { get; }
public RemoteConfigMergeResult MergeResult { get; }
public string PendingMigrationTarget { get; }
public RuntimeConfigBundle(
RemoteConfigService remoteConfigService,
AgentConfigResponse remoteConfig,
RemoteConfigMergeResult mergeResult)
RemoteConfigMergeResult mergeResult,
string pendingMigrationTarget = null)
{
RemoteConfigService = remoteConfigService;
RemoteConfig = remoteConfig;
MergeResult = mergeResult;
PendingMigrationTarget = pendingMigrationTarget;
}
}
}
@@ -132,7 +132,10 @@ namespace AutopilotMonitor.Agent.V2.Runtime
whiteGloveSealingPatternIds: whiteGloveSealingPatternIds,
drainInterval: drainInterval,
agentMaxLifetime: agentMaxLifetime,
uploadBatchSize: uploadBatchSize))
uploadBatchSize: uploadBatchSize,
// Live read: remote config is merged into agentConfig mid-session, so the
// trace-event gate must follow it rather than snapshot at construction.
traceEventsEnabled: () => agentConfig.SendTraceEvents))
{
using (var shutdown = new ManualResetEventSlim(false))
using (var shutdownComplete = new ManualResetEventSlim(false))
@@ -2,11 +2,11 @@
<PropertyGroup>
<!--
Plan §4 M2: xUnit, .NET-8-Testrunner gegen netstandard2.0-Lib.
DecisionCore itself targets netstandard2.0, but the test runner uses .NET 8
so we benefit from modern xUnit + coverlet tooling.
Plan §4 M2: xUnit test runner against the netstandard2.0 lib.
DecisionCore itself targets netstandard2.0; the runner tracks the
repo-wide .NET version (global.json) for modern xUnit + coverlet tooling.
-->
<TargetFramework>net8.0</TargetFramework>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
@@ -0,0 +1,160 @@
using AutopilotMonitor.Functions.Functions.Config;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Services;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the endpoint-migration serving contract: the shared allowlist rules
/// (<see cref="AgentEndpointMigrationRules"/>, used verbatim by the agent too) and the
/// backend-side per-tenant resolution (<see cref="GetAgentConfigFunction.ResolveMigrateTarget"/> —
/// tenant override wins, empty override pins, invalid values are never served).
/// </summary>
public class AgentEndpointMigrationTests
{
private const string TenantA = "11111111-2222-3333-4444-555555555555";
private const string TenantB = "66666666-7777-8888-9999-000000000000";
private const string UsUrl = "https://autopilotmonitor-api-us.azurewebsites.net";
private const string EuUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
// ── Shared validation rules ─────────────────────────────────────────────
[Theory]
[InlineData("https://autopilotmonitor-api-us.azurewebsites.net", "https://autopilotmonitor-api-us.azurewebsites.net")]
[InlineData("https://AutopilotMonitor-API-US.AzureWebsites.NET/", "https://autopilotmonitor-api-us.azurewebsites.net")]
[InlineData(" https://x.azurewebsites.net ", "https://x.azurewebsites.net")]
public void TryNormalizeTarget_accepts_and_normalizes(string candidate, string expected)
{
Assert.True(AgentEndpointMigrationRules.TryNormalizeTarget(candidate, out var normalized));
Assert.Equal(expected, normalized);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
[InlineData("http://x.azurewebsites.net")] // https only
[InlineData("https://attacker.example.com")] // not allowlisted
[InlineData("https://evil-azurewebsites.net")] // label-boundary bypass attempt
[InlineData("https://.azurewebsites.net")] // bare suffix
[InlineData("https://x.azurewebsites.net:8443")] // non-default port
[InlineData("https://u:p@x.azurewebsites.net")] // userinfo
[InlineData("https://x.azurewebsites.net/api")] // path
[InlineData("https://x.azurewebsites.net?a=1")] // query
[InlineData("https://x.azurewebsites.net#frag")] // fragment
[InlineData("ftp://x.azurewebsites.net")]
[InlineData("not a url")]
public void TryNormalizeTarget_rejects(string? candidate)
{
Assert.False(AgentEndpointMigrationRules.TryNormalizeTarget(candidate!, out var normalized));
Assert.Null(normalized);
}
[Fact]
public void TryNormalizeTarget_rejects_overlong_candidate()
{
var candidate = "https://" + new string('a', AgentEndpointMigrationRules.MaxTargetLength) + ".azurewebsites.net";
Assert.False(AgentEndpointMigrationRules.TryNormalizeTarget(candidate, out _));
}
[Fact]
public void IsEffectiveMigration_rejects_target_equal_to_current()
{
Assert.False(AgentEndpointMigrationRules.IsEffectiveMigration(EuUrl, EuUrl + "/", out var target));
Assert.Null(target);
Assert.True(AgentEndpointMigrationRules.IsEffectiveMigration(UsUrl, EuUrl, out target));
Assert.Equal(UsUrl, target);
}
// ── Backend resolution: global vs per-tenant override ──────────────────
[Fact]
public void No_configuration_resolves_to_null()
{
var config = new AdminConfiguration();
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Null(rejected);
}
[Fact]
public void Global_target_applies_to_every_tenant()
{
var config = new AdminConfiguration { AgentMigrateApiBaseUrl = UsUrl };
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Tenant_override_wins_over_global()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = $"{{\"{TenantA}\": \"{EuUrl}\"}}"
};
Assert.Equal(EuUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Empty_tenant_override_pins_tenant_against_global_target()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = $"{{\"{TenantA}\": \"\"}}"
};
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Null(rejected); // pinned, not rejected
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Tenant_override_key_is_case_insensitive()
{
var config = new AdminConfiguration
{
AgentMigrateTenantOverridesJson = $"{{\"{TenantA.ToUpperInvariant()}\": \"{UsUrl}\"}}"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
[Fact]
public void Invalid_configured_target_is_never_served_and_reported_as_rejected()
{
var config = new AdminConfiguration { AgentMigrateApiBaseUrl = "https://attacker.example.com" };
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Equal("https://attacker.example.com", rejected);
}
[Fact]
public void Malformed_overrides_json_degrades_to_global_target()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = "{not json"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
[Fact]
public void Served_target_is_normalized()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = "https://AutopilotMonitor-API-US.azurewebsites.net/"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
}
@@ -0,0 +1,83 @@
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey → business-time decoders. These are the read-path compensation for rows whose
/// system Timestamp was reset by the 2026-07-18 storage migration: the RowKey is the only
/// time carrier that survives a row rewrite, so the decode must round-trip exactly and
/// must reject every foreign format (falling back instead of mis-dating rows).
/// </summary>
public class BusinessTimestampDecodeTests
{
private static readonly DateTime Ts = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc).AddTicks(4567);
[Fact]
public void Audit_rowkey_roundtrips_tick_exact()
{
var rowKey = TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid());
Assert.True(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("!")]
[InlineData("!123")] // too short
[InlineData("!123456789012345678x_guid")] // non-digit in revtick
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // legacy bare GUID (:N)
[InlineData("0000000000000000000")] // ops-style bare revtick is not an audit key
public void Audit_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out _));
}
[Fact]
public void Ops_rowkey_roundtrips_tick_exact()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
Assert.True(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("123456789012345678")] // 18 digits
[InlineData("12345678901234567890")] // 20 digits
[InlineData("123456789012345678x")] // non-digit
[InlineData("!1234567890123456789")] // audit-prefixed
public void Ops_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out _));
}
[Fact]
public void Event_rowkey_prefix_decodes_to_millisecond_truncated_agent_time()
{
var rowKey = $"{Ts:yyyyMMddHHmmssfff}_{42L:D10}";
Assert.True(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out var decoded));
// The RowKey format truncates sub-millisecond ticks at write time.
var expected = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc);
Assert.Equal(expected, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("20260615103045123")] // no separator/sequence
[InlineData("2026061510304512_0000000042")] // 16-digit prefix
[InlineData("202606151030451234_0000000042")] // 18-digit prefix
[InlineData("2026061510304512x_0000000042")] // non-digit in prefix
[InlineData("20261315103045123_0000000042")] // month 13
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // GUID
public void Event_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out _));
}
}
@@ -0,0 +1,181 @@
using System.Text.RegularExpressions;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey-range date-window / retention clauses. Date filtering moved off the system
/// Timestamp (reset by storage migrations) onto the reverse-tick RowKey; these tests pin
/// (a) tick-exact inclusive/exclusive boundary semantics identical to the old
/// `Timestamp ge/le` clauses, and (b) the GUID-safety invariant: legacy bare-GUID audit
/// rows sort AFTER all '!'-rows, so any lower RowKey bound without the `RowKey lt '"'`
/// guard would match — and a retention sweep would DELETE — every legacy row.
/// </summary>
public class BusinessTimestampFilterClauseTests
{
private static readonly DateTime T = new DateTime(2026, 6, 15, 12, 0, 0, DateTimeKind.Utc).AddTicks(789);
private static readonly string Tenant = "11111111-1111-1111-1111-111111111111";
private static string AuditRk(DateTime ts) => TableStorageService.BuildAuditLogRowKey(ts, Guid.NewGuid());
private static string OpsRk(DateTime ts) => $"{DateTime.MaxValue.Ticks - ts.Ticks:D19}";
/// <summary>Evaluates a clause of the form "RowKey op 'literal' [and RowKey op 'literal']…" against a RowKey.</summary>
private static bool Matches(string clause, string rowKey)
{
var parts = Regex.Matches(clause, @"RowKey (ge|gt|le|lt) '([^']*)'");
Assert.True(parts.Count > 0, $"no RowKey comparisons in clause: {clause}");
foreach (Match m in parts)
{
var cmp = string.CompareOrdinal(rowKey, m.Groups[2].Value);
var ok = m.Groups[1].Value switch
{
"ge" => cmp >= 0,
"gt" => cmp > 0,
"le" => cmp <= 0,
"lt" => cmp < 0,
_ => throw new InvalidOperationException(),
};
if (!ok) return false;
}
return true;
}
// ===== Audit boundaries =====
[Fact]
public void Audit_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateFromClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == from → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → excluded
}
[Fact]
public void Audit_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateToClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == to → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // older → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // 1 tick newer → excluded
}
[Fact]
public void Audit_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.AuditRetentionClause(T);
Assert.False(Matches(clause, AuditRk(T))); // ts == cutoff → kept
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → deleted
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → kept
}
// ===== Audit GUID safety (the single most dangerous invariant of this change) =====
[Fact]
public void Legacy_guid_rowkeys_sort_after_the_time_encoded_guard()
{
for (var i = 0; i < 20; i++)
{
var guidRk = Guid.NewGuid().ToString("N");
Assert.True(string.CompareOrdinal(guidRk, BusinessTimestamp.AuditTimeEncodedUpperBound) > 0);
}
}
[Fact]
public void No_audit_date_or_retention_clause_matches_legacy_guid_rowkeys()
{
var clauses = new[]
{
BusinessTimestamp.AuditDateFromClause(T),
BusinessTimestamp.AuditDateToClause(T),
BusinessTimestamp.AuditDateToClause(DateTime.MaxValue.AddTicks(-1)), // widest possible window
BusinessTimestamp.AuditRetentionClause(T),
BusinessTimestamp.AuditRetentionClause(DateTime.MaxValue.AddTicks(-1)), // widest possible sweep
};
foreach (var clause in clauses)
{
for (var i = 0; i < 20; i++)
{
Assert.False(Matches(clause, Guid.NewGuid().ToString("N")),
$"clause matched a legacy GUID RowKey: {clause}");
}
}
}
[Fact]
public void Audit_lower_bounded_clauses_carry_the_time_encoded_guard()
{
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditDateToClause(T));
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditRetentionClause(T));
}
// ===== Ops boundaries =====
[Fact]
public void Ops_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateFromClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(-1))));
}
[Fact]
public void Ops_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateToClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
[Fact]
public void Ops_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.OpsRetentionClause(T);
Assert.False(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
// ===== Composition into the filter builders =====
[Fact]
public void Audit_filter_builders_emit_rowkey_windows_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var plain = TableStorageService.BuildAuditLogFilter(Tenant, from, to);
var fanOut = TableStorageService.BuildAuditLogFilterWithRowKeyBound(
Tenant, from, to, lastRowKey: "!0123_x", excludeDeletions: false);
foreach (var f in new[] { plain!, fanOut })
{
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.AuditDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.AuditDateToClause(to), f);
}
Assert.Contains("RowKey gt '!0123_x'", fanOut); // fan-out cursor composes with the window
}
[Fact]
public void Ops_filter_builder_emits_rowkey_window_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var f = TableOpsEventRepository.BuildFilter("Security", from, to);
Assert.NotNull(f);
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.OpsDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.OpsDateToClause(to), f);
}
}
@@ -0,0 +1,118 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Read-mapper resolution order for the three affected tables:
/// OccurredUtc column → RowKey decode → system Timestamp. The last hop is the
/// migration-corrupted value (a row rewrite resets it), so it must only ever win when
/// the row carries no better source (audit legacy bare-GUID rows).
/// </summary>
public class BusinessTimestampPrecedenceTests
{
private static readonly DateTime Occurred = new(2026, 5, 10, 8, 0, 0, DateTimeKind.Utc);
private static readonly DateTime RowKeyTime = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
private static readonly DateTimeOffset SystemTs = new(2026, 7, 18, 13, 58, 0, TimeSpan.Zero); // migration moment
private static TableEntity WithSystemTimestamp(TableEntity e)
{
e["Timestamp"] = SystemTs;
return e;
}
// ===== Audit =====
[Fact]
public void Audit_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid())));
Assert.Equal(RowKeyTime, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_legacy_guid_row_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("t", Guid.NewGuid().ToString("N")));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveAuditTimestamp(e));
}
// ===== Ops =====
[Fact]
public void Ops_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}")
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}"));
Assert.Equal(RowKeyTime, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("Security", "not-a-revtick"));
Assert.Equal(SystemTs.UtcDateTime, TableOpsEventRepository.ResolveTimestamp(e));
}
// ===== Events =====
private static string EventRowKey(DateTime ts, long seq = 42) => $"{ts:yyyyMMddHHmmssfff}_{seq:D10}";
[Fact]
public void Event_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_rowkey_prefix_wins_over_system_when_column_missing()
{
// The pre-cutover case: no OccurredUtc column, system Timestamp = migration moment.
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime)));
Assert.Equal(RowKeyTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", "weird-rowkey"));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_occurredutc_read_tolerates_datetime_materialization()
{
// SDK reads surface Edm.DateTime as DateTimeOffset or DateTime depending on payload
// shape — the resolver must accept both (dual-reader lesson).
var e = new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = Occurred, // plain DateTime, not DateTimeOffset
};
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
}
@@ -0,0 +1,283 @@
using AutopilotMonitor.Functions.Functions.Rules;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Azure;
using Azure.Data.Tables;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the three gather-rule scope/emit fields (ActivePhases, ActiveFromPhase, EmitMode):
/// Store↔Map table-serialization roundtrip (table-serialization rule: every model field in BOTH
/// directions), legacy rows without the new columns, the <see cref="GatherRuleService.ContentEquivalent"/>
/// null-tolerance (no reseed churn), and the create/update validation matrix in
/// <see cref="GatherRulesFunction.ValidateScopeAndEmitMode"/>.
/// </summary>
public class GatherRuleScopeFieldsTests
{
// ── Store ↔ Map roundtrip ──────────────────────────────────────────────
private static (TableStorageService service, Func<TableEntity?> lastUpserted) BuildStorageHarness()
{
TableEntity? captured = null;
var tableClient = new Mock<TableClient>();
tableClient
.Setup(c => c.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, TableUpdateMode, CancellationToken>((e, _, _) =>
{
captured = e;
return Task.FromResult(new Mock<Response>().Object);
});
var serviceClient = new Mock<TableServiceClient>();
serviceClient.Setup(c => c.GetTableClient(It.IsAny<string>())).Returns(tableClient.Object);
var service = new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance);
return (service, () => captured);
}
[Fact]
public async Task Roundtrip_ScopeFields_SurviveStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-001",
Title = "Scoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_scoped",
ActivePhases = new List<string> { "AccountSetup", "AppsUser" },
EmitMode = "on_change",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.NotNull(mapped.ActivePhases);
Assert.Equal(new[] { "AccountSetup", "AppsUser" }, mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Equal("on_change", mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_FromPhase_SurvivesStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-002",
Title = "From-phase rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_scoped",
ActiveFromPhase = "AccountSetup",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Equal("AccountSetup", mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_NullScopeFields_StayNull()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-003",
Title = "Unscoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "startup",
OutputEventType = "gather_plain",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public void Map_LegacyRow_WithoutScopeColumns_DefaultsToNull()
{
var (service, _) = BuildStorageHarness();
// A row written before the scope fields existed: none of the new columns present.
var legacy = new TableEntity("global", "GATHER-LEGACY-001")
{
["Title"] = "Legacy rule",
["CollectorType"] = "registry",
["Target"] = "HKLM\\SOFTWARE\\Contoso",
["Trigger"] = "startup",
["OutputEventType"] = "gather_legacy",
};
var mapped = service.MapToGatherRule(legacy);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
// ── ContentEquivalent ──────────────────────────────────────────────────
private static GatherRule BaseRule() => new()
{
RuleId = "GATHER-EQ-001",
Title = "Rule",
Description = "Desc",
Version = "1.0.0",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
TriggerPhase = "",
OutputEventType = "gather_eq",
};
[Fact]
public void ContentEquivalent_NullAndEmptyActivePhases_AreEquivalent()
{
// Pre-existing DB rows map absent columns to null; seeds are also null — and an
// explicitly-empty list means the same "unrestricted". No reseed churn allowed.
var a = BaseRule();
var b = BaseRule();
b.ActivePhases = new List<string>();
Assert.True(GatherRuleService.ContentEquivalent(a, b));
Assert.True(GatherRuleService.ContentEquivalent(b, a));
}
[Fact]
public void ContentEquivalent_DetectsChangesInEachScopeField()
{
var baseline = BaseRule();
var phases = BaseRule();
phases.ActivePhases = new List<string> { "AccountSetup" };
Assert.False(GatherRuleService.ContentEquivalent(baseline, phases));
var from = BaseRule();
from.ActiveFromPhase = "AccountSetup";
Assert.False(GatherRuleService.ContentEquivalent(baseline, from));
var emit = BaseRule();
emit.EmitMode = "on_change";
Assert.False(GatherRuleService.ContentEquivalent(baseline, emit));
}
[Fact]
public void ContentEquivalent_SameScopeFields_AreEquivalent()
{
var a = BaseRule();
a.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
a.EmitMode = "on_change";
var b = BaseRule();
b.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
b.EmitMode = "on_change";
Assert.True(GatherRuleService.ContentEquivalent(a, b));
}
// ── Validation ─────────────────────────────────────────────────────────
private static GatherRule ValidationRule(
List<string>? activePhases = null, string? activeFromPhase = null, string? emitMode = null) => new()
{
RuleId = "GATHER-VAL-001",
Title = "Rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_val",
ActivePhases = activePhases,
ActiveFromPhase = activeFromPhase,
EmitMode = emitMode,
};
[Theory]
[InlineData(null)]
[InlineData("Start")]
[InlineData("DevicePreparation")]
[InlineData("AccountSetup")]
[InlineData("Complete")]
[InlineData("accountsetup")] // case-insensitive
public void Validate_AcceptsValidFromPhase(string? fromPhase)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Theory]
[InlineData("Unknown")]
[InlineData("Failed")]
[InlineData("4")] // numeric enum value — canonical names only
[InlineData("DeviceESP")] // the invalid token the old free-text UI suggested
public void Validate_RejectsInvalidFromPhase(string fromPhase)
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Fact]
public void Validate_AcceptsValidActivePhases_AndRejectsInvalidEntries()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "AppsDevice" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "Failed" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "" })));
}
[Fact]
public void Validate_RejectsBothScopeFieldsSet()
{
var error = GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(
activePhases: new List<string> { "AccountSetup" }, activeFromPhase: "DeviceSetup"));
Assert.NotNull(error);
Assert.Contains("mutually exclusive", error);
}
[Theory]
[InlineData(null)]
[InlineData("always")]
[InlineData("on_change")]
[InlineData("ON_CHANGE")] // case-insensitive
public void Validate_AcceptsValidEmitModes(string? emitMode)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: emitMode)));
}
[Fact]
public void Validate_RejectsUnknownEmitMode()
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: "sometimes")));
}
[Fact]
public void Validate_TogglePartialPayload_PassesThrough()
{
// The portal toggle sends only { enabled, isBuiltIn, isCommunity } — none of the
// scope fields — and must not be rejected.
var toggle = new GatherRule { RuleId = "GATHER-VAL-002", Enabled = false };
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(toggle));
}
[Fact]
public void Validate_EmptyActivePhasesList_IsUnrestricted_AndPasses()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activePhases: new List<string>())));
}
}
@@ -0,0 +1,146 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the custom-rule partial-PUT fix on <see cref="GatherRuleService.UpdateRuleAsync"/>:
/// the portal toggle PUTs only { enabled, isBuiltIn, isCommunity }, and full-storing that
/// payload wiped Title/Target/Trigger/Parameters of the custom rule. Toggle-style partials
/// (empty Title/CollectorType/Target) must merge Enabled into the existing row; full payloads
/// (edit flow) must still replace the row but preserve the original CreatedAt.
/// </summary>
public class GatherRuleUpdatePartialMergeTests
{
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
private const string RuleId = "GATHER-CUST-001";
private static GatherRule ExistingCustomRule() => new()
{
RuleId = RuleId,
Title = "Collect BIOS Config",
Description = "Reads the RealmJoin BIOS key",
Category = "device",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS",
Parameters = new Dictionary<string, string> { ["valueName"] = "Version" },
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_bios_config",
Enabled = true,
IsBuiltIn = false,
IsCommunity = false,
CreatedAt = new DateTime(2026, 1, 15, 12, 0, 0, DateTimeKind.Utc),
ActiveFromPhase = "AccountSetup",
EmitMode = "on_change",
};
private static (GatherRuleService service, Mock<IRuleRepository> repo, List<GatherRule> stored) BuildService(
GatherRule? existingTenantRule)
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
var stored = new List<GatherRule>();
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>());
repo.Setup(r => r.GetGatherRulesAsync(TenantId)).ReturnsAsync(
existingTenantRule == null ? new List<GatherRule>() : new List<GatherRule> { existingTenantRule });
repo.Setup(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), TenantId))
.Callback<GatherRule, string>((rule, _) => stored.Add(rule))
.ReturnsAsync(true);
return (new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance), repo, stored);
}
[Fact]
public async Task TogglePartial_merges_enabled_and_preserves_definition_fields()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
// Exactly what the portal toggle sends: { enabled, isBuiltIn, isCommunity }.
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.False(written.Enabled);
Assert.Equal("Collect BIOS Config", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS", written.Target);
Assert.Equal("registry", written.CollectorType);
Assert.Equal("interval", written.Trigger);
Assert.Equal(60, written.IntervalSeconds);
Assert.Equal("Version", written.Parameters["valueName"]);
Assert.Equal("AccountSetup", written.ActiveFromPhase);
Assert.Equal("on_change", written.EmitMode);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task TogglePartial_for_unknown_custom_rule_fails_without_storing()
{
var (service, repo, stored) = BuildService(existingTenantRule: null);
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.False(ok);
Assert.Empty(stored);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
[Fact]
public async Task FullPayload_replaces_rule_but_preserves_original_CreatedAt()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
var fullPayload = new GatherRule
{
RuleId = RuleId,
Title = "Collect BIOS Config v2",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2",
Trigger = "interval",
IntervalSeconds = 120,
OutputEventType = "gather_bios_config",
Enabled = true,
// Edit flow sends createdAt from the client; a JSON-mode edit may omit it and
// default to UtcNow — either way the stored row must keep the original.
CreatedAt = DateTime.UtcNow,
};
var ok = await service.UpdateRuleAsync(TenantId, fullPayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.Equal("Collect BIOS Config v2", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2", written.Target);
Assert.Equal(120, written.IntervalSeconds);
Assert.False(written.IsBuiltIn);
Assert.False(written.IsCommunity);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task BuiltInRule_toggle_still_goes_through_RuleState_path()
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>
{
new() { RuleId = RuleId, IsBuiltIn = true, Title = "Built-in", Enabled = true }
});
repo.Setup(r => r.StoreRuleStateAsync(TenantId, RuleId, It.IsAny<RuleState>())).ReturnsAsync(true);
var service = new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance);
var ok = await service.UpdateRuleAsync(TenantId, new GatherRule { RuleId = RuleId, Enabled = false });
Assert.True(ok);
repo.Verify(r => r.StoreRuleStateAsync(TenantId, RuleId, It.Is<RuleState>(s => !s.Enabled)), Times.Once);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
}
@@ -129,4 +129,30 @@ public class GetAllTenantConfigurationsFunctionTests
[Fact]
public void DelegatedBareArrayView_Empty_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.DelegatedBareArrayView(new TenantConfiguration[0]));
[Fact]
public void MissingManagedIds_ReturnsOnlyIdsWithoutConfig_CaseInsensitive()
{
var configs = new List<TenantConfiguration>
{
new() { TenantId = TenantB.ToUpperInvariant() }, // case variant still counts as present
};
var missing = GetAllTenantConfigurationsFunction.MissingManagedIds(new[] { TenantB, TenantC }, configs);
var only = Assert.Single(missing);
Assert.Equal(TenantC, only);
}
[Fact]
public void MissingManagedIds_AllPresent_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.MissingManagedIds(
new[] { TenantB },
new List<TenantConfiguration> { new() { TenantId = TenantB } }));
[Fact]
public void MissingConfigCacheKey_LowercasesId_SoCaseVariantsShareOneVerdict()
=> Assert.Equal(
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB),
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB.ToUpperInvariant()));
}
@@ -0,0 +1,107 @@
using System.Collections.Generic;
using System.Net;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Functions.Tests.GraphResolution;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Service-level tests for the Agent Binaries HealthCheck card. The check probes the
/// canonical download alias (download.autopilotmonitor.com) AND the legacy blob keepalive
/// account: alias failure is unhealthy (new installs break), legacy failure is only a
/// warning (existing customers break until they migrate to the alias).
/// </summary>
public class HealthCheckServiceAgentBinariesTests
{
private const string AliasHost = "download.autopilotmonitor.com";
private const string LegacyHost = "autopilotmonitor.blob.core.windows.net";
[Fact]
public async Task Check_AllEndpointsAvailable_ReportsHealthyWithoutUrlDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("Agent Binaries", check.Name);
Assert.Equal("healthy", check.Status);
Assert.Contains("download alias", check.Message);
Assert.Null(check.Details);
// Both hosts must actually have been probed (ZIP + PS1 each).
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(AliasHost)).Count);
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(LegacyHost)).Count);
}
[Fact]
public async Task Check_WithEndpointUrls_SurfacesAliasAndLegacyUrlsInDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: true);
Assert.NotNull(check.Details);
Assert.Equal("https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip", check.Details!["Agent ZIP"]);
Assert.Equal("https://download.autopilotmonitor.com/agent/Install-AutopilotMonitor.ps1", check.Details["Bootstrap script"]);
Assert.Equal("https://autopilotmonitor.blob.core.windows.net/agent", check.Details["Legacy blob (keepalive)"]);
}
[Fact]
public async Task Check_LegacyBlobMissing_ReportsWarningNamingTheLegacyEndpoint()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.NotFound, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("warning", check.Status);
Assert.Contains("legacy", check.Message, System.StringComparison.OrdinalIgnoreCase);
Assert.Contains("404", check.Message);
}
[Fact]
public async Task Check_AliasMissing_ReportsUnhealthy()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.NotFound, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("download alias", check.Message);
}
[Fact]
public async Task Check_EndpointsUnreachable_ReportsUnhealthy()
{
// Un-scripted URLs make the stub throw — models a connection failure.
var check = await BuildService(new StubHttpMessageHandler()).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("unreachable", check.Message);
}
private static HealthCheckService BuildService(StubHttpMessageHandler handler)
{
var config = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string?>())
.Build();
return new HealthCheckService(
NullLogger<HealthCheckService>.Instance,
adminConfigService: null!,
httpClientFactory: new StubHttpClientFactory(handler),
metricsReader: null!,
poisonQueueProbe: null!,
configuration: config);
}
}
@@ -53,6 +53,29 @@ public class HealthCheckServiceMcpServerTests
Assert.DoesNotContain("http", check.Message, System.StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlAlongsideVersion()
{
var svc = BuildService(HttpStatusCode.OK, "{\"status\":\"healthy\",\"version\":\"1.4.0\"}");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
// The Version detail must merge into — not replace — the Server URL detail.
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
Assert.Equal("1.4.0", check.Details["Version"]);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlEvenWhenUnhealthy()
{
var svc = BuildService(HttpStatusCode.ServiceUnavailable, "");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
Assert.Equal("unhealthy", check.Status);
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
}
[Fact]
public async Task Check_TimeoutOrConnectionFailure_ReportsWarningForColdStart()
{
@@ -3,7 +3,7 @@ using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Tier-boundary + percent-math tests for the SignalR free-tier quota watcher.
/// Tier-boundary + percent-math tests for the SignalR quota watcher.
/// Mirrors the cert-expiry tests' approach: pure-function coverage of the
/// boundary risks (off-by-one, floor vs ceil, zero-division). Full-service
/// smoke tests would require constructing 16 dependencies; the percent +
@@ -0,0 +1,61 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Per-row backfill decision (pure function): rows that already carry OccurredUtc are
/// skipped (idempotent re-runs), decodable RowKeys yield the deterministic decoded value,
/// anything else is reported undecodable — never guessed.
/// </summary>
public class OccurredUtcBackfillServiceTests
{
private static readonly DateTime Ts = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
[Fact]
public void Audit_row_with_existing_column_is_skipped()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Audit_time_encoded_row_yields_decoded_write()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()));
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
}
[Fact]
public void Audit_legacy_guid_row_is_undecodable_not_guessed()
{
var e = new TableEntity("t", Guid.NewGuid().ToString("N"));
Assert.Equal(OccurredUtcBackfillService.RowDecision.Undecodable,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Ops_row_yields_decoded_write_and_existing_column_is_skipped()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
var bare = new TableEntity("Security", rowKey);
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, bare, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
var backfilled = new TableEntity("Security", rowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, backfilled, out _));
}
}
@@ -0,0 +1,269 @@
using AutopilotMonitor.Functions.Functions.Admin;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.DataAccess;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the SignalR revoke-enforcement cores of <see cref="DelegatedAdminManagementFunction"/> and
/// <see cref="TenantGroupManagementFunction"/> (extracted seams — the HTTP entry points are intentionally
/// not exercised, same rationale as <see cref="GetAllBlockedDevicesFunctionTests"/>). Locks in three
/// security invariants: (1) a miss (no assignment row) neither audits nor disconnects — a typo must not
/// write false customer-visible "access removed" rows while the real grant stays live; (2) a real
/// revoke/disable/unassign/group-delete always cuts the affected UPN's live streams (group authorization
/// is join-time only); (3) grant validation is fail-closed (GUID tenantId, recognized roles only).
/// </summary>
public class RevokeEnforcementTests
{
private const string Upn = "MSP-Admin@Partner.Example";
private const string UpnLower = "msp-admin@partner.example";
private const string TenantA = "11111111-1111-1111-1111-111111111111";
private const string TenantB = "22222222-2222-2222-2222-222222222222";
private const string GroupId = "tpl-aaaaaaaa";
private static DelegatedAdminService BuildService(Mock<IAdminRepository> repo) =>
new(
repo.Object,
new StubTenantEntitlementService(AutopilotMonitor.Functions.Security.TenantEdition.Enterprise),
new MemoryCache(new MemoryCacheOptions()),
NullLogger<DelegatedAdminService>.Instance);
private static Mock<IMaintenanceRepository> BuildAuditRepo()
{
var audit = new Mock<IMaintenanceRepository>();
audit.Setup(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()))
.ReturnsAsync(true);
return audit;
}
private static void VerifyNoAudit(Mock<IMaintenanceRepository> audit) =>
audit.Verify(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()), Times.Never);
private static (DelegatedAdminManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildAdminFn()
{
var repo = new Mock<IAdminRepository>();
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new DelegatedAdminManagementFunction(
NullLogger<DelegatedAdminManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
private static (TenantGroupManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildGroupFn()
{
var repo = new Mock<IAdminRepository>();
// Loose-mock defaults return a null Task — every repo read the flows touch needs a concrete setup.
repo.Setup(r => r.GetTenantGroupAsync(It.IsAny<string>())).ReturnsAsync((TenantGroup?)null);
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
repo.Setup(r => r.GetGroupAssigneesAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new TenantGroupManagementFunction(
NullLogger<TenantGroupManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
// --- Delegated grant validation (fail-closed) ---
[Fact]
public void ValidateGrant_NullBody_Rejected()
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(null, out _));
[Theory]
[InlineData("", TenantA)]
[InlineData(" ", TenantA)]
[InlineData(Upn, "")]
public void ValidateGrant_MissingFields_Rejected(string upn, string tenantId)
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = upn, TenantId = tenantId }, out _));
[Theory]
[InlineData("contoso.com")]
[InlineData("not-a-guid")]
[InlineData("11111111-1111-1111-1111")]
public void ValidateGrant_NonGuidTenantId_Rejected(string tenantId)
=> Assert.Equal("a valid tenantId (GUID) is required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = tenantId }, out _));
[Fact]
public void ValidateGrant_UnknownRole_Rejected()
{
var error = DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = "GlobalAdmin" }, out _);
Assert.NotNull(error);
Assert.Contains(Constants.DelegatedRoles.DelegatedReader, error);
}
[Fact]
public void ValidateGrant_EmptyRole_DefaultsToLeastPrivilegedReader()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA }, out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedReader, role);
}
[Fact]
public void ValidateGrant_ExplicitDelegatedAdmin_Accepted()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = Constants.DelegatedRoles.DelegatedAdmin },
out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedAdmin, role);
}
// --- Revoke: 404-on-missing-row vs. audit + disconnect ---
[Fact]
public async Task RevokeCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(false);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.False(removed);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task RevokeCore_RemovedRow_AuditsAndDisconnectsLowercasedUpn()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(true);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.True(removed);
// SignalR user ids are lowercased UPNs — a mixed-case route param must still hit the user's connections.
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedAdmin", UpnLower, "ga@vendor.example", null), Times.Once);
}
// --- Disable is a revocation too; enable never disconnects ---
[Fact]
public async Task SetEnabledCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(false);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.False(ok);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Disable_Disconnects()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.True(ok);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Enable_DoesNotDisconnect()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, true)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: true, "ga@vendor.example");
Assert.True(ok);
Assert.Empty(signalR.DisconnectedUsers);
}
// --- Group unassign: 404-on-not-assigned vs. disconnect + per-tenant audit ---
[Fact]
public async Task UnassignCore_NotAssigned_NoAuditNoDisconnect()
{
var (fn, _, audit, signalR) = BuildGroupFn(); // default: no assignments for the UPN
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.False(unassigned);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task UnassignCore_Assigned_DisconnectsAndAuditsPerGroupTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA, TenantB },
AssigneeCount = 1,
});
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(UpnLower))
.ReturnsAsync(new List<TenantGroupAssignment> { new() { Upn = UpnLower, GroupId = GroupId } });
repo.Setup(r => r.UnassignGroupAsync(UpnLower, GroupId)).ReturnsAsync(true);
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.True(unassigned);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
// Lost access is audited under EACH of the group's tenants (customer-visible trail).
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
audit.Verify(a => a.LogAuditEntryAsync(
TenantB, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
// --- Group delete: every (former) assignee's streams are cut ---
[Fact]
public async Task DeleteGroupCore_DisconnectsEveryAssignee_AndAuditsPerTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
var assignees = new List<TenantGroupAssignment>
{
new() { Upn = "one@partner.example", GroupId = GroupId },
new() { Upn = "two@partner.example", GroupId = GroupId },
};
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA },
AssigneeCount = assignees.Count,
});
repo.Setup(r => r.GetGroupAssigneesAsync(GroupId)).ReturnsAsync(assignees);
repo.Setup(r => r.DeleteTenantGroupAsync(GroupId)).ReturnsAsync(true);
await fn.DeleteGroupCoreAsync(GroupId, "ga@vendor.example");
Assert.Equal(new[] { "one@partner.example", "two@partner.example" }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", "*", "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
}
@@ -0,0 +1,25 @@
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// ContactEmail is the address enforcement actions and service notices are sent to.
/// The seed must never overwrite a value the tenant owns — that is the whole contract.
/// </summary>
public class TenantContactEmailTests
{
[Fact]
public void ContactEmail_defaults_to_null_so_absence_is_distinguishable_from_empty()
{
var config = new TenantConfiguration();
Assert.Null(config.ContactEmail);
}
[Fact]
public void ContactEmail_survives_a_round_trip_through_the_model()
{
var config = new TenantConfiguration { ContactEmail = "it-operations@contoso.com" };
Assert.Equal("it-operations@contoso.com", config.ContactEmail);
}
}
@@ -336,6 +336,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "LastUpdated", config.LastUpdated },
{ "UpdatedBy", config.UpdatedBy },
{ "OnboardedBy", config.OnboardedBy },
{ "ContactEmail", config.ContactEmail },
{ "Disabled", config.Disabled },
{ "DisabledReason", config.DisabledReason },
{ "DisabledUntil", config.DisabledUntil },
@@ -432,6 +433,7 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
LastUpdated = entity.GetDateTime("LastUpdated") ?? DateTime.UtcNow,
UpdatedBy = entity.GetString("UpdatedBy") ?? "Unknown",
OnboardedBy = entity.GetString("OnboardedBy"),
ContactEmail = entity.GetString("ContactEmail"),
Disabled = entity.GetBoolean("Disabled") ?? false,
DisabledReason = entity.GetString("DisabledReason"),
DisabledUntil = entity.GetDateTime("DisabledUntil"),
@@ -582,7 +584,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "FeedbackMinTenantAgeDays", config.FeedbackMinTenantAgeDays },
{ "FeedbackCooldownDays", config.FeedbackCooldownDays },
// MCP access control
{ "McpAccessPolicy", config.McpAccessPolicy ?? "WhitelistOnly" }
{ "McpAccessPolicy", config.McpAccessPolicy ?? "WhitelistOnly" },
// Agent endpoint migration (config-channel re-home)
{ "AgentMigrateApiBaseUrl", config.AgentMigrateApiBaseUrl ?? string.Empty },
{ "AgentMigrateTenantOverridesJson", config.AgentMigrateTenantOverridesJson ?? string.Empty }
};
return entity;
@@ -643,7 +648,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
FeedbackMinTenantAgeDays = entity.GetInt32("FeedbackMinTenantAgeDays") ?? 14,
FeedbackCooldownDays = entity.GetInt32("FeedbackCooldownDays") ?? 60,
// MCP access control
McpAccessPolicy = entity.GetString("McpAccessPolicy") ?? "WhitelistOnly"
McpAccessPolicy = entity.GetString("McpAccessPolicy") ?? "WhitelistOnly",
// Agent endpoint migration (config-channel re-home)
AgentMigrateApiBaseUrl = entity.GetString("AgentMigrateApiBaseUrl") ?? string.Empty,
AgentMigrateTenantOverridesJson = entity.GetString("AgentMigrateTenantOverridesJson") ?? string.Empty
};
}
@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
@@ -43,7 +44,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
["UserId"] = Truncate(entry.UserId, 128),
["Message"] = Truncate(entry.Message, 512),
["Details"] = Truncate(entry.Details, 4096),
["Timestamp"] = entry.Timestamp,
// "Timestamp" is a reserved system property (supplied values are ignored and
// any row rewrite resets it), so the event time needs its own column.
[BusinessTimestamp.OccurredUtcColumn] = entry.Timestamp,
};
await _table.UpsertEntityAsync(entity);
@@ -168,15 +171,16 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
// Filters on the system-managed Timestamp; user-defined "Timestamp" property
// mirrors it on insert via SaveOpsEventAsync.
private static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
// Date windows filter on the RowKey (reverse-tick, index-backed): the system
// Timestamp is reset by storage migrations, and an OccurredUtc property filter
// would exclude rows written before that column existed.
internal static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
{
var clauses = new List<string>();
if (!string.IsNullOrEmpty(category))
@@ -185,11 +189,11 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
}
return clauses.Count == 0 ? null : string.Join(" and ", clauses);
}
@@ -201,27 +205,30 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{
var deleted = 0;
await foreach (var entity in _table.QueryAsync<TableEntity>())
// Server-side RowKey range instead of a full-table scan over the system
// Timestamp: every ops RowKey is a reverse-tick, so age is index-derivable,
// and the system Timestamp lies after storage migrations (it would freeze
// deletions for one retention period, then drop the whole pre-migration
// corpus in a single run).
var filter = BusinessTimestamp.OpsRetentionClause(cutoff);
await foreach (var entity in _table.QueryAsync<TableEntity>(
filter: filter, select: new[] { "PartitionKey", "RowKey" }))
{
var timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue;
if (timestamp < cutoff)
try
{
try
{
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
private static OpsEventEntry MapToEntry(TableEntity entity)
internal static OpsEventEntry MapToEntry(TableEntity entity)
{
return new OpsEventEntry
{
@@ -233,10 +240,22 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
UserId = entity.GetString("UserId"),
Message = entity.GetString("Message") ?? string.Empty,
Details = entity.GetString("Details"),
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue,
Timestamp = ResolveTimestamp(entity),
};
}
// Business time: OccurredUtc column → reverse-tick RowKey decode → system
// Timestamp (last resort; a row rewrite — e.g. storage migration — resets it).
internal static DateTime ResolveTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.MinValue;
}
private static string? Truncate(string? value, int maxLength)
{
if (value == null) return null;
@@ -0,0 +1,103 @@
using System.Net;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Admin
{
/// <summary>
/// Global-Admin trigger for the one-time OccurredUtc backfill on AuditLogs / OpsEvents
/// (business timestamps lost to the system-Timestamp reset of the 2026-07-18 storage
/// migration; values are recovered from the reverse-tick RowKeys). DRY-RUN BY DEFAULT —
/// a real run requires the explicit <c>dryRun=false</c> query parameter, so the operator
/// always sees counts and samples before anything is written.
/// </summary>
public class BackfillOccurredUtcFunction
{
private readonly ILogger<BackfillOccurredUtcFunction> _logger;
private readonly OccurredUtcBackfillService _service;
private const int DefaultMaxRows = 250;
private const int MaxMaxRows = 1000;
public BackfillOccurredUtcFunction(
ILogger<BackfillOccurredUtcFunction> logger,
OccurredUtcBackfillService service)
{
_logger = logger;
_service = service;
}
/// <summary>
/// POST /api/maintenance/backfill-occurred-utc
/// Query parameters:
/// - table: "audit" | "ops" (required).
/// - dryRun: "false" to actually write; anything else (or absent) reports only.
/// - maxRows: rows examined per invocation (default 250, max 1000). Follow
/// nextContinuation to continue; null nextContinuation = table done.
/// - continuation: opaque token from the previous response (URL-encode it).
/// </summary>
[Function("BackfillOccurredUtc")]
public async Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "maintenance/backfill-occurred-utc")] HttpRequestData req,
FunctionContext context)
{
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
var userEmail = TenantHelper.GetUserIdentifier(req);
var table = (req.Query["table"] ?? string.Empty).Trim().ToLowerInvariant();
if (table != OccurredUtcBackfillService.TableAudit && table != OccurredUtcBackfillService.TableOps)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new
{
error = $"table must be '{OccurredUtcBackfillService.TableAudit}' or '{OccurredUtcBackfillService.TableOps}'"
});
return bad;
}
var dryRun = !string.Equals(req.Query["dryRun"], "false", StringComparison.OrdinalIgnoreCase);
var continuation = req.Query["continuation"];
var maxRows = DefaultMaxRows;
if (!string.IsNullOrEmpty(req.Query["maxRows"]))
{
if (!int.TryParse(req.Query["maxRows"], out maxRows) || maxRows < 1)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new { error = $"maxRows must be a positive integer (max {MaxMaxRows})" });
return bad;
}
maxRows = Math.Min(maxRows, MaxMaxRows);
}
_logger.LogInformation(
"OccurredUtc backfill requested by {User}: table={Table} dryRun={DryRun} maxRows={MaxRows} hasContinuation={HasContinuation}",
userEmail, table, dryRun, maxRows, !string.IsNullOrEmpty(continuation));
try
{
var result = await _service.RunAsync(table, dryRun, maxRows, continuation);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
{
success = true,
result,
triggeredBy = userEmail,
triggeredAt = DateTime.UtcNow,
});
return response;
}
catch (Exception ex)
{
_logger.LogError(ex, "OccurredUtc backfill run failed");
var error = req.CreateResponse(HttpStatusCode.InternalServerError);
await error.WriteAsJsonAsync(new { success = false, message = "Internal server error" });
return error;
}
}
}
}
@@ -64,19 +64,9 @@ public class DelegatedAdminManagementFunction
var currentUpn = context.GetRequestContext().UserPrincipalName;
var body = await req.ReadFromJsonAsync<GrantDelegatedAdminRequest>();
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return await Bad(req, "upn and tenantId are required");
// Mirror the tenant-group path: a mistyped tenantId must not create a garbage scope entry
// that pollutes the allow-list and seeds a per-tenant audit trail keyed on the typo.
if (!Guid.TryParse(body.TenantId, out _))
return await Bad(req, "a valid tenantId (GUID) is required");
// Default to the least-privileged role; reject anything we don't recognize (fail-closed) so a typo
// can never silently widen access.
var role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return await Bad(req, $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'");
var validationError = ValidateGrantRequest(body, out var role);
if (validationError != null)
return await Bad(req, validationError);
// Edition note: TARGET tenants may be any edition — an MSP on Enterprise may manage Community
// customers. The Enterprise requirement applies to the delegated admin's HOME tenant and is
@@ -84,7 +74,7 @@ public class DelegatedAdminManagementFunction
// be checked here: at grant time only the UPN is known, and UPN-domain → tenant mapping is not
// reliable (multi-domain tenants). Grants to non-Enterprise-homed admins are simply inert.
var entry = await _delegatedAdminService.UpsertAsync(
body.Upn, body.TenantId, role,
body!.Upn, body.TenantId, role,
Constants.DelegatedStatus.Active, Constants.DelegatedSource.OperatorGranted, currentUpn ?? "");
await _maintenanceRepo.LogAuditEntryAsync(
@@ -112,10 +102,7 @@ public class DelegatedAdminManagementFunction
string upn, string tenantId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
// The service reports whether a row was actually deleted — a mistyped upn/tenantId must not
// 200 and write a false customer-visible "access removed" audit row while the real grant
// stays live (mirrors the tenant-group revoke semantics).
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
var removed = await RevokeCoreAsync(upn, tenantId, currentUpn);
if (!removed)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -123,20 +110,34 @@ public class DelegatedAdminManagementFunction
return notFound;
}
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
// Enforcement: cut the revoked caller's already-joined live streams — group authorization is
// join-time only, so without this they keep receiving tenant telemetry until the connection drops.
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Delegated assignment revoked" });
return response;
}
/// <summary>
/// Testable core of the revoke flow (no HTTP plumbing — see GetAllBlockedDevicesFunctionTests for the
/// convention). Returns false when no assignment row existed (caller 404s): a mistyped upn/tenantId
/// must not 200 and write a false customer-visible "access removed" audit row while the real grant
/// stays live (mirrors the tenant-group revoke semantics). On a real removal writes the audit entry
/// and cuts the revoked UPN's live streams — group authorization is join-time only, so without the
/// kick they keep receiving tenant telemetry until the connection drops.
/// </summary>
internal async Task<bool> RevokeCoreAsync(string upn, string tenantId, string? currentUpn)
{
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
if (!removed)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
return true;
}
/// <summary>PATCH /api/global/delegated-admins/{upn}/{tenantId}/enable — re-enable a disabled assignment. GlobalAdminOnly.</summary>
[Function("EnableDelegatedAdmin")]
[Authorize]
@@ -157,7 +158,7 @@ public class DelegatedAdminManagementFunction
HttpRequestData req, string upn, string tenantId, bool isEnabled, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
var ok = await SetEnabledCoreAsync(upn, tenantId, isEnabled, currentUpn);
if (!ok)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -165,20 +166,56 @@ public class DelegatedAdminManagementFunction
return notFound;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
}
/// <summary>
/// Testable core of enable/disable (no HTTP plumbing). Returns false when no assignment row
/// existed (caller 404s — no audit, no disconnect). Disable is a revocation too, so it cuts the
/// UPN's already-joined live streams (see <see cref="RevokeCoreAsync"/>); enable never disconnects.
/// </summary>
internal async Task<bool> SetEnabledCoreAsync(string upn, string tenantId, bool isEnabled, string? currentUpn)
{
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
if (!ok)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "UPDATE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "",
new Dictionary<string, string> { { "IsEnabled", isEnabled.ToString() } });
// Disable is a revocation too — cut already-joined live streams (see RevokeDelegatedAdmin).
if (!isEnabled)
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated {Action}: {Upn} -> {TenantId} by {By}",
isEnabled ? "enable" : "disable", upn, tenantId, currentUpn);
return true;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
/// <summary>
/// Pure grant-request validation (testable seam). Returns the 400 error message, or null when valid —
/// then <paramref name="role"/> carries the effective role. A mistyped tenantId must not create a
/// garbage scope entry that pollutes the allow-list and seeds a per-tenant audit trail keyed on the
/// typo; the role defaults to least-privileged and unknown roles are rejected (fail-closed) so a typo
/// can never silently widen access.
/// </summary>
internal static string? ValidateGrantRequest(GrantDelegatedAdminRequest? body, out string role)
{
role = Constants.DelegatedRoles.DelegatedReader;
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return "upn and tenantId are required";
if (!Guid.TryParse(body.TenantId, out _))
return "a valid tenantId (GUID) is required";
role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role!;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'";
return null;
}
private static async Task<HttpResponseData> Bad(HttpRequestData req, string error)
@@ -7,10 +7,11 @@ using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Admin
{
/// <summary>
/// Timer-triggered watcher for the Azure SignalR Free-tier limits (20
/// concurrent connections / 20,000 messages per UTC day). Runs every full
/// hour and emits OpsEvents when the configured thresholds are crossed so
/// operators get a Telegram ping before clients start getting 429'd.
/// Timer-triggered watcher for the Azure SignalR plan limits (Standard S1
/// x1 unit: 1,000 concurrent connections / 1M messages per UTC day). Runs
/// every full hour and emits OpsEvents when the configured thresholds are
/// crossed so operators get a Telegram ping before clients start getting
/// 429'd or message overage is billed.
///
/// Separate from the 2h MaintenanceFunction: the SignalR limits can be
/// breached by a brief connection spike, so the cadence is tighter. Cost
@@ -112,15 +112,26 @@ public class TenantGroupManagementFunction
string groupId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
await DeleteGroupCoreAsync(groupId, currentUpn);
// Capture tenants + assignees BEFORE the cascade delete removes them, so we can audit the
// bulk access removal under each affected tenant and cut every assignee's live streams.
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>
/// Testable core of the group cascade delete (no HTTP plumbing). Captures tenants + assignees
/// BEFORE the delete removes them, so the bulk access removal is audited under each affected
/// tenant and EVERY (former) assignee's live streams are cut — group authorization is join-time
/// only, so without the kick they keep receiving the managed tenants' telemetry until their
/// connection drops.
/// </summary>
internal async Task DeleteGroupCoreAsync(string groupId, string? currentUpn)
{
var group = await _delegatedAdminService.GetGroupAsync(groupId);
var assignees = await _delegatedAdminService.GetGroupAssigneesAsync(groupId);
await _delegatedAdminService.DeleteGroupAsync(groupId);
// Enforcement: group authorization is join-time only — without the kick, every (former)
// assignee keeps receiving the managed tenants' live telemetry until their connection drops.
foreach (var assignee in assignees)
await _signalRService.DisconnectUserAsync(assignee.Upn);
@@ -137,10 +148,6 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Tenant group deleted: {GroupId} by {By}", groupId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>POST /api/global/tenant-groups/{groupId}/tenants — add a tenant. GlobalAdminOnly. Body: { "tenantId": "&lt;guid&gt;" }.</summary>
@@ -271,17 +278,31 @@ public class TenantGroupManagementFunction
string groupId, string upn, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var unassigned = await UnassignCoreAsync(groupId, upn, currentUpn);
if (!unassigned)
return await NotFound(req);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
}
/// <summary>
/// Testable core of the unassign flow (no HTTP plumbing). Returns false when the UPN was not
/// actually assigned (caller 404s) — so a mistyped UPN can't 200 and write false "access removed"
/// audit rows, and no disconnect fires. On a real unassign cuts the UPN's already-joined live
/// streams (join-time-only authz) and audits the lost access under each of the group's tenants.
/// </summary>
internal async Task<bool> UnassignCoreAsync(string groupId, string upn, string? currentUpn)
{
var normalizedUpn = upn.ToLowerInvariant();
// Read the group's tenants (unchanged by unassign) so we can audit the UPN's lost access per tenant.
var group = await _delegatedAdminService.GetGroupAsync(groupId);
// The service returns false when the UPN was not actually assigned — so a mistyped UPN can't 200 and
// write false "access removed" audit rows. Only audit after a real unassign.
var unassigned = await _delegatedAdminService.UnassignGroupAsync(normalizedUpn, groupId);
if (!unassigned)
return await NotFound(req);
return false;
// Enforcement: cut the unassigned caller's already-joined live streams (join-time-only authz).
await _signalRService.DisconnectUserAsync(normalizedUpn);
if (group != null)
@@ -296,10 +317,7 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Group {GroupId} unassigned from {Upn} by {By}", groupId, normalizedUpn, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
return true;
}
/// <summary>Logs one audit entry per tenant (the customer-visible "who can read my tenant" trail).</summary>
@@ -157,6 +157,39 @@ namespace AutopilotMonitor.Functions.Functions.Config
return (line.ZipSha256, line.ExeSha256);
}
/// <summary>
/// Resolves the endpoint-migration target for a tenant: per-tenant override wins over
/// the global value; an override entry with an empty value pins the tenant (no
/// migration even while the global target is set — staged rollout). The winning value
/// is validated against <see cref="AutopilotMonitor.Shared.Services.AgentEndpointMigrationRules"/>;
/// invalid values resolve to null (fail-safe: better to strand an agent on the old
/// backend than to serve a broken or non-allowlisted URL).
/// </summary>
internal static string? ResolveMigrateTarget(
AutopilotMonitor.Shared.Models.AdminConfiguration adminConfig,
string tenantId,
out string? rejectedCandidate)
{
rejectedCandidate = null;
string? candidate = adminConfig.AgentMigrateApiBaseUrl;
var overrides = adminConfig.GetAgentMigrateTenantOverrides();
if (!string.IsNullOrEmpty(tenantId) && overrides.TryGetValue(tenantId, out var tenantTarget))
candidate = tenantTarget; // empty string = pinned, handled below
if (string.IsNullOrWhiteSpace(candidate))
return null;
if (AutopilotMonitor.Shared.Services.AgentEndpointMigrationRules
.TryNormalizeTarget(candidate, out var normalized))
{
return normalized;
}
rejectedCandidate = candidate;
return null;
}
/// <summary>
/// Core config logic: fetch tenant + admin config, gather rules, IME patterns.
/// Called by both the cert-auth Run() method and the bootstrap wrapper.
@@ -215,10 +248,29 @@ namespace AutopilotMonitor.Functions.Functions.Config
// so agent code is unchanged across all major lines.
var (latestAgentSha256, latestAgentExeSha256) = SelectAgentHashesForClient(req, adminConfig);
// Endpoint migration on the control channel: serve the (validated) re-home target
// so agents still bound to this backend's compiled-in URL move themselves to the
// new deployment at their next start. LogWarning (not Information) because worker
// logs below Warning never reach App Insights — this line is the delivery evidence
// during a migration window.
var migrateTarget = ResolveMigrateTarget(adminConfig, tenantId, out var rejectedMigrateCandidate);
if (migrateTarget != null)
{
_logger.LogWarning(
"AgentMigrateServed: tenant={TenantId} serial={Serial} agentVersion={AgentVersion} target={Target}",
tenantId, serialNumberHeader, agentVersionHeader, migrateTarget);
}
else if (rejectedMigrateCandidate != null)
{
_logger.LogWarning(
"AgentMigrateRejected: configured migration target failed validation and is NOT served. tenant={TenantId} candidate={Candidate}",
tenantId, rejectedMigrateCandidate);
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new AgentConfigResponse
{
ConfigVersion = 32, // V2 OOBE-console / Shift+F10 detection portal toggle (EnableConsoleBypassDetection; tenant-scoped, default ON / opt-out; gates ConsoleBypass host + ConsolePrefetchScanner)
ConfigVersion = 35, // MDM reboot-policy watcher (mdm_policy_reboot_required / esp_reboot_coalescing)
UploadIntervalSeconds = 10,
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
KeepLogFile = tenantConfig.KeepLogFile ?? false,
@@ -262,6 +314,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
DeviceBlocked = killVerdict.IsBlocked,
DeviceKillSignal = killVerdict.IsKill,
UnblockAt = killVerdict.UnblockAt,
MigrateToApiBaseUrl = migrateTarget,
});
return response;
@@ -10,6 +10,7 @@ using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Pagination;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
@@ -17,15 +18,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
{
public class GetAllTenantConfigurationsFunction
{
// TTL for the "this managed id has NO config row in any casing" verdict a rescue scan proved.
// Bounds how long a tenant onboarded moments after that scan stays absent from a delegated
// caller's config/all (matches the positive config cache TTL in TenantConfigurationService).
private static readonly TimeSpan MissingConfigNegativeCacheDuration = TimeSpan.FromMinutes(5);
private readonly ILogger<GetAllTenantConfigurationsFunction> _logger;
private readonly TenantConfigurationService _configService;
private readonly IMemoryCache _cache;
public GetAllTenantConfigurationsFunction(
ILogger<GetAllTenantConfigurationsFunction> logger,
TenantConfigurationService configService)
TenantConfigurationService configService,
IMemoryCache cache)
{
_logger = logger;
_configService = configService;
_cache = cache;
}
[Function("GetAllTenantConfigurations")]
@@ -73,9 +82,21 @@ namespace AutopilotMonitor.Functions.Functions.Config
// (agent paths auto-create rows with the caller-supplied id verbatim) — the same caveat
// the delegated sessions fan-out handles case-insensitively. A point-read is exact-case,
// so any id it missed gets ONE scan+filter rescue pass (the pre-point-read behavior).
// Only stale grants or case-variant rows pay for it; the common path stays read-bounded.
// An id a previous scan already proved missing in ANY casing is negative-cached and
// skipped for the TTL — otherwise a single stale grant (offboarded / never-onboarded
// tenant) would make EVERY config/all call pay a full table scan.
if (subset.Count < managedIds.Count)
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
{
var unproven = MissingManagedIds(managedIds, subset)
.Where(id => !_cache.TryGetValue(MissingConfigCacheKey(id), out _))
.ToList();
if (unproven.Count > 0)
{
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
foreach (var id in MissingManagedIds(unproven, subset))
_cache.Set(MissingConfigCacheKey(id), true, MissingConfigNegativeCacheDuration);
}
}
_logger.LogInformation("GetAllTenantConfigurations (delegated subset, {Count} tenants) by {User}",
subset.Count, userIdentifier);
@@ -177,6 +198,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
IEnumerable<TenantConfiguration?> reads)
=> reads.Where(c => c != null).Select(c => c!).ToList();
/// <summary>
/// The managed ids not present (case-insensitively) in <paramref name="configs"/> — before the
/// rescue scan these are the point-read misses, after it the proven-missing ids to negative-cache.
/// Pure + testable.
/// </summary>
internal static List<string> MissingManagedIds(
IEnumerable<string> managedIds, IReadOnlyCollection<TenantConfiguration> configs)
{
var have = new HashSet<string>(configs.Select(c => c.TenantId), StringComparer.OrdinalIgnoreCase);
return managedIds.Where(id => !have.Contains(id)).ToList();
}
/// <summary>Negative-cache key for a managed id a rescue scan proved has no config row.
/// Lowercased so case variants of the same grant share one verdict.</summary>
internal static string MissingConfigCacheKey(string tenantId)
=> $"tenant-config-missing:{tenantId.ToLowerInvariant()}";
/// <summary>
/// Case-variant rescue for managed ids the exact-case point-reads missed: from the full config scan,
/// add every config whose TenantId matches a managed id case-insensitively and is not already in the
@@ -102,17 +102,19 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
// Perform comprehensive health checks
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync();
// SignalR Quota check exposes the subscription/resource-group path of the
// backing SignalR resource, and Poison Queues exposes internal async-worker
// queue topology — restrict both to Global Admins. Non-GA callers
// (Tenant Admins, Operators) get the rest of the report unchanged.
// SignalR Quota exposes the subscription/resource-group path of the backing
// SignalR resource, Poison Queues exposes internal async-worker queue topology,
// and endpoint URLs in the check details expose infrastructure hostnames —
// restrict all of that to Global Admins. Non-GA callers (Tenant Admins,
// Operators) get the rest of the report unchanged.
// The route is registered as AuthenticatedUser, so PolicyEnforcementMiddleware
// does NOT compute IsGlobalAdmin; resolve it directly via GlobalAdminService.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
// Perform comprehensive health checks (endpoint URLs only for Global Admins)
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync(includeEndpointUrls: isGlobalAdmin);
var visibleChecks = isGlobalAdmin
? healthCheckResult.Checks
: healthCheckResult.Checks
@@ -146,10 +148,14 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
/// </summary>
[Function("McpHealthCheck")]
public async Task<HttpResponseData> GetMcpHealthCheck(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req)
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req,
FunctionContext context)
{
// Authentication enforced by PolicyEnforcementMiddleware (AuthenticatedUser).
var check = await _healthCheckService.CheckMcpServerAsync();
// The MCP server URL is infrastructure topology — include it only for Global Admins.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
var check = await _healthCheckService.CheckMcpServerAsync(includeEndpointUrl: isGlobalAdmin);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
@@ -64,6 +64,14 @@ namespace AutopilotMonitor.Functions.Functions.Rules
return badRequest;
}
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
try
{
var success = await _ruleService.CreateRuleAsync(tenantId, rule);
@@ -108,12 +116,71 @@ namespace AutopilotMonitor.Functions.Functions.Rules
rule.RuleId = ruleId;
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
var success = await _ruleService.UpdateRuleAsync(tenantId, rule);
var response = req.CreateResponse(success ? HttpStatusCode.OK : HttpStatusCode.InternalServerError);
await response.WriteAsJsonAsync(new { success, message = success ? "Rule updated" : "Failed to update rule" });
return response;
}
/// <summary>
/// Selectable phase-scope tokens: the <see cref="EnrollmentPhase"/> enum NAMES from
/// Start(0) through Complete(7). Explicit name list (not Enum.TryParse) so numeric
/// tokens ("4") and the non-selectable Unknown/Failed members are rejected — the
/// canonical vocabulary the agent, schema, and portal share.
/// </summary>
private static readonly HashSet<string> ValidScopePhases = new(StringComparer.OrdinalIgnoreCase)
{
nameof(EnrollmentPhase.Start),
nameof(EnrollmentPhase.DevicePreparation),
nameof(EnrollmentPhase.DeviceSetup),
nameof(EnrollmentPhase.AppsDevice),
nameof(EnrollmentPhase.AccountSetup),
nameof(EnrollmentPhase.AppsUser),
nameof(EnrollmentPhase.FinalizingSetup),
nameof(EnrollmentPhase.Complete),
};
/// <summary>
/// Validates the phase-scope + emit-mode fields on create/update.
/// Returns an error message for a 400 response, or null when valid.
/// Toggle-style partial payloads carry none of these fields and pass through.
/// </summary>
internal static string? ValidateScopeAndEmitMode(GatherRule rule)
{
var hasActivePhases = rule.ActivePhases != null && rule.ActivePhases.Count > 0;
var hasFromPhase = !string.IsNullOrEmpty(rule.ActiveFromPhase);
if (hasActivePhases && hasFromPhase)
return "activePhases and activeFromPhase are mutually exclusive — set only one.";
if (hasActivePhases)
{
foreach (var phase in rule.ActivePhases!)
{
if (string.IsNullOrEmpty(phase) || !ValidScopePhases.Contains(phase))
return $"Invalid phase '{phase}' in activePhases. Valid phases: {string.Join(", ", ValidScopePhases)}.";
}
}
if (hasFromPhase && !ValidScopePhases.Contains(rule.ActiveFromPhase!))
return $"Invalid activeFromPhase '{rule.ActiveFromPhase}'. Valid phases: {string.Join(", ", ValidScopePhases)}.";
if (!string.IsNullOrEmpty(rule.EmitMode)
&& !string.Equals(rule.EmitMode, "always", StringComparison.OrdinalIgnoreCase)
&& !string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase))
return "emitMode must be \"always\" or \"on_change\".";
return null;
}
[Function("DeleteGatherRule")]
public async Task<HttpResponseData> DeleteRule(
[HttpTrigger(AuthorizationLevel.Anonymous, "delete", Route = "rules/gather/{ruleId}")] HttpRequestData req,
@@ -0,0 +1,154 @@
using System;
using System.Globalization;
using Azure.Data.Tables;
namespace AutopilotMonitor.Functions.Helpers
{
/// <summary>
/// Business-timestamp handling for tables whose rows historically relied on the
/// Azure Tables system <c>Timestamp</c> (= row write time). That system property is
/// server-managed: supplied values are ignored on write and every row rewrite
/// (storage migration, merge) resets it. The authoritative event time therefore
/// lives in (a) the explicit <see cref="OccurredUtcColumn"/> custom property on
/// rows written after its introduction, and (b) the time encoded in the RowKey
/// for older rows. Readers resolve OccurredUtc → RowKey decode → system Timestamp.
///
/// RowKey schemes decoded here:
/// AuditLogs: "!{MaxTicks - ticks:D19}_{guid:N}" (legacy pre-2026-05 rows are bare GUIDs — not decodable)
/// OpsEvents: "{MaxTicks - ticks:D19}"
/// Events: "{timestamp:yyyyMMddHHmmssfff}_{sequence:D10}"
/// </summary>
internal static class BusinessTimestamp
{
/// <summary>Custom column carrying the business event time. Survives migrations verbatim
/// (unlike the system Timestamp, which any row rewrite resets).</summary>
internal const string OccurredUtcColumn = "OccurredUtc";
internal const string AuditRowKeyPrefix = "!";
/// <summary>
/// '"' (0x22) is the character immediately after '!' (0x21). `RowKey lt '"'` confines a
/// range to time-encoded audit rows: legacy bare-GUID RowKeys start with a hex digit
/// (≥ '0', 0x30) and would otherwise match ANY lower bound of the form `RowKey ge '!…'`.
/// </summary>
internal const string AuditTimeEncodedUpperBound = "\"";
private const string EventRowKeyTimestampFormat = "yyyyMMddHHmmssfff";
/// <summary>
/// Reads a datetime column as UTC regardless of the SDK's materialized type —
/// service reads surface Edm.DateTime as either DateTimeOffset or DateTime
/// depending on payload shape (the pre-existing mappers already read both).
/// Unspecified kinds are treated as UTC (all stored values are UTC).
/// </summary>
internal static DateTime? GetUtcDateTime(TableEntity entity, string column)
{
if (!entity.TryGetValue(column, out var value) || value == null)
return null;
return value switch
{
DateTimeOffset dto => dto.UtcDateTime,
DateTime dt => dt.Kind == DateTimeKind.Local
? dt.ToUniversalTime()
: DateTime.SpecifyKind(dt, DateTimeKind.Utc),
_ => null,
};
}
// ===== RowKey decoders =====
/// <summary>Decodes an audit RowKey ("!{rev:D19}_{guid:N}") to the original UTC write time.
/// Returns false for legacy bare-GUID RowKeys and any other foreign format.</summary>
internal static bool TryDecodeAuditRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length < 21 || rowKey[0] != '!')
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(1, 19), out utc);
}
/// <summary>Decodes an ops-event RowKey (bare "{rev:D19}") to the original UTC write time.</summary>
internal static bool TryDecodeOpsRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length != 19)
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(), out utc);
}
/// <summary>Decodes an enrollment-event RowKey ("{yyyyMMddHHmmssfff}_{seq:D10}") to the
/// (sanitized) agent event time, millisecond precision, Kind=Utc.</summary>
internal static bool TryDecodeEventRowKeyPrefix(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null)
return false;
var separator = rowKey.IndexOf('_');
if (separator != 17)
return false;
if (!DateTime.TryParseExact(rowKey.Substring(0, 17), EventRowKeyTimestampFormat,
CultureInfo.InvariantCulture,
DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out utc))
return false;
return true;
}
private static bool TryDecodeReverseTicks(ReadOnlySpan<char> revTickDigits, out DateTime utc)
{
utc = default;
foreach (var c in revTickDigits)
{
if (c < '0' || c > '9')
return false;
}
if (!long.TryParse(revTickDigits, NumberStyles.None, CultureInfo.InvariantCulture, out var rev))
return false;
var ticks = DateTime.MaxValue.Ticks - rev;
if (ticks < DateTime.MinValue.Ticks || ticks > DateTime.MaxValue.Ticks)
return false;
utc = new DateTime(ticks, DateTimeKind.Utc);
return true;
}
// ===== OData date-window clauses (RowKey ranges) =====
// Date windows filter on the RowKey rather than a property: rows missing a property are
// excluded from OData comparisons (so an OccurredUtc filter would drop every not-yet-
// backfilled row), and RowKey ranges are index-backed. String order equals numeric order
// because the reverse-tick is fixed-width D19. rev(t) = MaxTicks - t.Ticks, so larger
// timestamps have lexically smaller RowKeys (newest-first).
private static long Rev(DateTime utc) => DateTime.MaxValue.Ticks - utc.Ticks;
/// <summary>Audit rows with business time ≥ <paramref name="fromUtc"/> (inclusive):
/// rev ≤ rev(from) &lt; rev(from)+1. The exclusive upper bound also excludes all legacy
/// GUID rows (they sort after every '!' row).</summary>
internal static string AuditDateFromClause(DateTime fromUtc)
=> $"RowKey lt '{AuditRowKeyPrefix}{Rev(fromUtc) + 1:D19}'";
/// <summary>Audit rows with business time ≤ <paramref name="toUtc"/> (inclusive):
/// rev ≥ rev(to). The lower bound alone would match every legacy GUID row, hence the
/// mandatory time-encoded guard.</summary>
internal static string AuditDateToClause(DateTime toUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(toUtc):D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Audit rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// rev ≥ rev(cutoff)+1 ⇔ ts &lt; cutoff. MUST keep the time-encoded guard — without it
/// this clause matches (and a retention sweep would delete) every legacy GUID row.</summary>
internal static string AuditRetentionClause(DateTime cutoffUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(cutoffUtc) + 1:D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Ops rows with business time ≥ <paramref name="fromUtc"/> (inclusive). Bare D19
/// RowKey (no suffix), so `le` is exact.</summary>
internal static string OpsDateFromClause(DateTime fromUtc)
=> $"RowKey le '{Rev(fromUtc):D19}'";
/// <summary>Ops rows with business time ≤ <paramref name="toUtc"/> (inclusive).</summary>
internal static string OpsDateToClause(DateTime toUtc)
=> $"RowKey ge '{Rev(toUtc):D19}'";
/// <summary>Ops rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// `gt` not `ge` — a row at exactly the cutoff is kept, matching `Timestamp lt` semantics.</summary>
internal static string OpsRetentionClause(DateTime cutoffUtc)
=> $"RowKey gt '{Rev(cutoffUtc):D19}'";
}
}
@@ -252,6 +252,7 @@ builder.Services.AddHttpClient<GitHubRuleRepository>()
.AddPolicyHandler((sp, _) => sp.GetRequiredService<ResiliencePolicies>().ExternalDataApi);
builder.Services.AddSingleton<MaintenanceService>();
builder.Services.AddSingleton<LegacyReclassificationService>();
builder.Services.AddSingleton<OccurredUtcBackfillService>();
builder.Services.AddSingleton<
AutopilotMonitor.Functions.Services.Monitoring.IAzureMonitorMetricsReader,
AutopilotMonitor.Functions.Services.Monitoring.AzureMonitorMetricsReader>();
@@ -399,6 +399,9 @@ public static class EndpointAccessPolicyCatalog
// One-time retro-reconcile of misclassified historical sessions (audit 2026-07-16) —
// mutating, dry-run by default, GA only.
new("POST", "maintenance/reclassify-legacy", EndpointPolicy.GlobalAdminOnly),
// One-time OccurredUtc backfill on AuditLogs/OpsEvents (2026-07-18 migration reset
// the system Timestamps) — mutating, dry-run by default, GA only.
new("POST", "maintenance/backfill-occurred-utc", EndpointPolicy.GlobalAdminOnly),
// ── Critical-Table Backup (plan §PR1+§PR2) ─────────────────────────────
new("POST", "global/backups/trigger", EndpointPolicy.GlobalAdminOnly),
@@ -151,10 +151,38 @@ namespace AutopilotMonitor.Functions.Services
return await _ruleRepo.StoreRuleStateAsync(tenantId, rule.RuleId, new RuleState { Enabled = rule.Enabled });
}
// Custom rule. The portal toggle PUTs only { enabled, isBuiltIn, isCommunity } —
// full-storing that payload would wipe Title/Target/Trigger/Parameters. Merge
// toggle-style partials into the existing row; full payloads (edit flow) replace
// the row but keep the original CreatedAt.
var existing = (await _ruleRepo.GetGatherRulesAsync(tenantId))
.FirstOrDefault(r => r.RuleId == rule.RuleId && !r.IsBuiltIn && !r.IsCommunity);
var isTogglePartial = string.IsNullOrEmpty(rule.Title)
&& string.IsNullOrEmpty(rule.CollectorType)
&& string.IsNullOrEmpty(rule.Target);
if (isTogglePartial)
{
if (existing == null)
{
_logger.LogWarning(
"Toggle-style partial update for unknown custom gather rule {RuleId} (tenant {TenantId}); nothing to update",
rule.RuleId, tenantId);
return false;
}
existing.Enabled = rule.Enabled;
existing.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(existing, tenantId);
}
// Ensure custom rules keep correct flags (the incoming payload may
// omit them, causing them to default to IsBuiltIn=true in the model).
rule.IsBuiltIn = false;
rule.IsCommunity = false;
if (existing != null)
rule.CreatedAt = existing.CreatedAt;
rule.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(rule, tenantId);
}
@@ -413,7 +441,22 @@ namespace AutopilotMonitor.Functions.Services
&& a.Title == b.Title
&& a.CollectorType == b.CollectorType
&& a.Trigger == b.Trigger
&& a.TriggerPhase == b.TriggerPhase;
&& a.TriggerPhase == b.TriggerPhase
&& ActivePhasesEquivalent(a.ActivePhases, b.ActivePhases)
&& a.ActiveFromPhase == b.ActiveFromPhase
&& a.EmitMode == b.EmitMode;
}
/// <summary>
/// Null-tolerant ordered comparison for <see cref="GatherRule.ActivePhases"/>:
/// null and empty are both "unrestricted" and therefore equivalent, so pre-existing
/// DB rows (absent column → null) never diff against seeds (also null) — no reseed churn.
/// </summary>
private static bool ActivePhasesEquivalent(List<string>? a, List<string>? b)
{
var left = a ?? new List<string>();
var right = b ?? new List<string>();
return left.SequenceEqual(right, StringComparer.Ordinal);
}
}
}
@@ -34,9 +34,12 @@ public class HealthCheckService
}
/// <summary>
/// Performs all health checks and returns the results
/// Performs all health checks and returns the results.
/// <paramref name="includeEndpointUrls"/> adds the probed/backing endpoint URLs to the
/// per-check details — infrastructure topology, so callers must only enable it for
/// Global Admins.
/// </summary>
public async Task<HealthCheckResult> PerformAllChecksAsync()
public async Task<HealthCheckResult> PerformAllChecksAsync(bool includeEndpointUrls = false)
{
var result = new HealthCheckResult
{
@@ -50,9 +53,9 @@ public class HealthCheckService
// one slow check. It is exposed via its own endpoint (GET health/mcp) that the
// frontend fetches independently and folds into the card grid incrementally.
var checks = await Task.WhenAll(
CheckStorageBackendAsync(),
CheckProcessingBackendAsync(),
CheckAgentBinariesAsync(),
CheckStorageBackendAsync(includeEndpointUrls),
CheckProcessingBackendAsync(includeEndpointUrls),
CheckAgentBinariesAsync(includeEndpointUrls),
CheckSignalRQuotaAsync(),
CheckPoisonQueuesAsync()
);
@@ -73,7 +76,7 @@ public class HealthCheckService
/// <summary>
/// Checks Azure Table Storage connectivity by querying the admin configuration table
/// </summary>
private async Task<HealthCheck> CheckStorageBackendAsync()
private async Task<HealthCheck> CheckStorageBackendAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -97,29 +100,62 @@ public class HealthCheckService
_logger.LogError(ex, "Storage backend health check failed");
}
if (includeEndpointUrls)
{
// Same resolution order as TableStorageService: account name (Managed Identity)
// is the production path, connection string is the local-dev fallback.
var accountName = _configuration["AzureStorageAccountName"];
check.Details = new Dictionary<string, object>
{
["Table endpoint"] = string.IsNullOrWhiteSpace(accountName)
? "connection string (local dev)"
: $"https://{accountName}.table.core.windows.net"
};
}
return check;
}
/// <summary>
/// Checks that the Azure Functions host is running
/// </summary>
private static Task<HealthCheck> CheckProcessingBackendAsync()
private static Task<HealthCheck> CheckProcessingBackendAsync(bool includeEndpointUrls)
{
var uptime = DateTime.UtcNow - System.Diagnostics.Process.GetCurrentProcess().StartTime.ToUniversalTime();
return Task.FromResult(new HealthCheck
var check = new HealthCheck
{
Name = "Processing Backend",
Description = "Application host process",
Status = "healthy",
Message = $"Host running (uptime: {(int)uptime.TotalMinutes}m {uptime.Seconds}s)"
});
};
if (includeEndpointUrls)
{
// WEBSITE_HOSTNAME is stamped by App Service and reflects the host actually
// serving this process — no hardcoded URL to go stale after a migration.
var hostname = Environment.GetEnvironmentVariable("WEBSITE_HOSTNAME");
if (!string.IsNullOrWhiteSpace(hostname))
{
check.Details = new Dictionary<string, object>
{
["API endpoint"] = $"https://{hostname}"
};
}
}
return Task.FromResult(check);
}
/// <summary>
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable in blob storage
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable on the
/// canonical download alias (Front Door → current blob origin) AND on the legacy blob
/// keepalive account that already-deployed customer bootstrap scripts still use.
/// Alias failure is unhealthy (new installs break); legacy failure is a warning
/// (existing customers break until they migrate to the alias).
/// </summary>
private async Task<HealthCheck> CheckAgentBinariesAsync()
internal async Task<HealthCheck> CheckAgentBinariesAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -127,8 +163,20 @@ public class HealthCheckService
Description = "Agent download package availability"
};
var zipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var ps1Url = $"{Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var aliasZipUrl = $"{Constants.AgentDownloadBaseUrl}/{Constants.AgentZipFileName}";
var aliasPs1Url = $"{Constants.AgentDownloadBaseUrl}/{Constants.BootstrapScriptName}";
var legacyZipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var legacyPs1Url = $"{Constants.AgentBlobBaseUrl}/{Constants.BootstrapScriptName}";
if (includeEndpointUrls)
{
check.Details = new Dictionary<string, object>
{
["Agent ZIP"] = aliasZipUrl,
["Bootstrap script"] = aliasPs1Url,
["Legacy blob (keepalive)"] = Constants.AgentBlobBaseUrl
};
}
try
{
@@ -138,40 +186,48 @@ public class HealthCheckService
var sw = System.Diagnostics.Stopwatch.StartNew();
// HEAD requests to check availability without downloading
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasPs1Url)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyPs1Url))
);
sw.Stop();
var zipResponse = results[0];
var ps1Response = results[1];
var aliasZipOk = results[0].IsSuccessStatusCode;
var aliasPs1Ok = results[1].IsSuccessStatusCode;
var legacyZipOk = results[2].IsSuccessStatusCode;
var legacyPs1Ok = results[3].IsSuccessStatusCode;
var zipOk = zipResponse.IsSuccessStatusCode;
var ps1Ok = ps1Response.IsSuccessStatusCode;
var issues = new List<string>();
if (!aliasZipOk) issues.Add($"Agent ZIP (download alias): {(int)results[0].StatusCode}");
if (!aliasPs1Ok) issues.Add($"Bootstrap script (download alias): {(int)results[1].StatusCode}");
if (!legacyZipOk) issues.Add($"Agent ZIP (legacy blob): {(int)results[2].StatusCode}");
if (!legacyPs1Ok) issues.Add($"Bootstrap script (legacy blob): {(int)results[3].StatusCode}");
if (zipOk && ps1Ok)
if (issues.Count == 0)
{
check.Status = "healthy";
check.Message = $"Agent package and bootstrap script available ({sw.ElapsedMilliseconds}ms)";
check.Message = $"Agent package and bootstrap script available via download alias + legacy blob ({sw.ElapsedMilliseconds}ms)";
}
else if (aliasZipOk && aliasPs1Ok)
{
// Alias intact, legacy keepalive degraded — existing customer bootstrap
// scripts still download from the legacy account until they migrate.
check.Status = "warning";
check.Message = $"Download alias OK, but legacy keepalive degraded — {string.Join(", ", issues)}";
}
else
{
check.Status = "unhealthy";
var issues = new List<string>();
if (!zipOk) issues.Add($"Agent ZIP: {(int)zipResponse.StatusCode}");
if (!ps1Ok) issues.Add($"Bootstrap script: {(int)ps1Response.StatusCode}");
check.Message = $"Missing: {string.Join(", ", issues)}";
}
}
catch (Exception ex)
{
check.Status = "unhealthy";
check.Message = $"Blob storage unreachable: {ex.Message}";
check.Message = $"Agent download endpoints unreachable: {ex.Message}";
_logger.LogError(ex, "Agent binaries health check failed");
}
@@ -182,7 +238,8 @@ public class HealthCheckService
/// Probes the MCP server's public <c>/health</c> endpoint (which sits in front of the
/// <c>/mcp</c> access guard, so no auth is required) and surfaces reachability + the
/// deployed MCP build version. Visible to all authenticated users, so the server URL is
/// deliberately NOT included in the message or details.
/// only included in the details when <paramref name="includeEndpointUrl"/> is set —
/// callers must only enable it for Global Admins.
/// <para>
/// The Container App runs with minReplicas=0 and scales to zero on idle. Azure Container
/// Apps holds an inbound request while it activates a cold replica, so this probe doubles
@@ -193,7 +250,7 @@ public class HealthCheckService
/// the blocking all-checks batch, so the dashboard never waits on the wake.
/// </para>
/// </summary>
internal async Task<HealthCheck> CheckMcpServerAsync()
internal async Task<HealthCheck> CheckMcpServerAsync(bool includeEndpointUrl = false)
{
var check = new HealthCheck
{
@@ -203,6 +260,11 @@ public class HealthCheckService
var baseUrl = (_configuration["McpServerUrl"] ?? Constants.McpServerBaseUrl).TrimEnd('/');
var healthUrl = $"{baseUrl}/health";
if (includeEndpointUrl)
{
check.Details = new Dictionary<string, object> { ["Server URL"] = baseUrl };
}
var timeoutSeconds = ParsePositiveInt("McpServerHealthTimeoutSeconds", 30);
// A cold start that takes longer than this is treated as a successful wake rather
@@ -233,7 +295,8 @@ public class HealthCheckService
: $"MCP server reachable ({sw.ElapsedMilliseconds}ms)";
if (!string.IsNullOrWhiteSpace(version))
{
check.Details = new Dictionary<string, object> { ["Version"] = version };
check.Details ??= new Dictionary<string, object>();
check.Details["Version"] = version;
}
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException or HttpRequestException)
@@ -290,7 +353,7 @@ public class HealthCheckService
var check = new HealthCheck
{
Name = "SignalR Quota",
Description = "Free-tier connection + daily message usage"
Description = "Connection + daily message usage vs. plan limits"
};
var resourceId = _configuration["SignalRResourceId"];
@@ -302,9 +365,9 @@ public class HealthCheckService
}
var connectionLimit = ParsePositiveInt(
"SignalRFreeTierConnectionLimit", MaintenanceService.DefaultSignalRConnectionLimit);
"SignalRConnectionLimit", MaintenanceService.DefaultSignalRConnectionLimit);
var messageLimit = ParsePositiveLong(
"SignalRFreeTierDailyMessageLimit", MaintenanceService.DefaultSignalRDailyMessageLimit);
"SignalRDailyMessageLimit", MaintenanceService.DefaultSignalRDailyMessageLimit);
var nowUtc = DateTimeOffset.UtcNow;
var startOfDayUtc = new DateTimeOffset(nowUtc.UtcDateTime.Date, TimeSpan.Zero);
@@ -372,9 +435,9 @@ public class HealthCheckService
check.Message = worstTier switch
{
MaintenanceService.SignalRQuotaTier.Critical => "Free-tier limit nearly saturated — scale to Standard before clients get 429'd",
MaintenanceService.SignalRQuotaTier.Warning => "Approaching free-tier limit — consider scaling to Standard",
_ => "Within free-tier limits"
MaintenanceService.SignalRQuotaTier.Critical => "Plan limit nearly saturated — add SignalR units before clients get 429'd",
MaintenanceService.SignalRQuotaTier.Warning => "Approaching plan limit — consider adding SignalR units",
_ => "Within plan limits"
};
check.Details = details;
return check;
@@ -517,6 +517,52 @@ namespace AutopilotMonitor.Functions.Services
/// Removes distress reports older than 14 days. Distress data is unverified
/// and low-volume; short retention keeps Table Storage lean.
/// </summary>
/// <summary>
/// One-off reconciliation for tenants onboarded before <c>ContactEmail</c> existed:
/// copies the preview notification address they already supplied into their tenant
/// configuration, but only where no contact address is set yet.
/// <para>
/// Deliberately never overwrites an existing value — a tenant that has edited its
/// contact address owns it, and a maintenance run must not undo that. Going forward
/// the seed happens when the notification address is saved, so this converges to a
/// no-op and can be removed once every existing tenant has been covered.
/// </para>
/// </summary>
/// <returns>Number of tenants that received a contact address.</returns>
private async Task<int> BackfillTenantContactEmailsAsync()
{
var filled = 0;
try
{
var configs = await _tenantConfigService.GetAllConfigurationsAsync();
foreach (var config in configs)
{
if (config == null || string.IsNullOrWhiteSpace(config.TenantId))
continue;
if (!string.IsNullOrWhiteSpace(config.ContactEmail))
continue;
var email = await _previewWhitelistService.GetNotificationEmailAsync(config.TenantId);
if (string.IsNullOrWhiteSpace(email))
continue;
config.ContactEmail = email.Trim();
await _tenantConfigService.SaveConfigurationAsync(config);
filled++;
}
if (filled > 0)
_logger.LogInformation("Backfilled contact address for {Count} tenant(s)", filled);
}
catch (Exception ex)
{
// Housekeeping, not load-bearing: a failure here must not fail the whole run.
_logger.LogWarning(ex, "Tenant contact address backfill failed");
}
return filled;
}
private async Task CleanupOldDistressReportsAsync()
{
const int retentionDays = 14;
@@ -554,52 +600,53 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Verifies that agent binaries and bootstrap script are available on blob storage.
/// Verifies that agent binaries and bootstrap script are available on the canonical
/// download alias (Front Door → current blob origin) AND on the legacy blob keepalive
/// account that already-deployed customer bootstrap scripts still use.
/// Records an OpsEvent for each missing item so Global Admins see it in the dashboard.
/// </summary>
private async Task CheckAgentBlobStorageAsync()
{
_logger.LogInformation("Checking agent blob storage availability...");
_logger.LogInformation("Checking agent download endpoint availability...");
var zipUrl = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}";
var ps1Url = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var probes = new (string Label, string Url)[]
{
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}"),
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}")
};
try
{
var client = _httpClientFactory.CreateClient();
client.Timeout = TimeSpan.FromSeconds(15);
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
probes.Select(p => client.SendAsync(new HttpRequestMessage(HttpMethod.Head, p.Url)))
);
var zipResponse = results[0];
var ps1Response = results[1];
if (!zipResponse.IsSuccessStatusCode)
var allOk = true;
for (var i = 0; i < probes.Length; i++)
{
_logger.LogError("Agent ZIP not available on blob storage: HTTP {StatusCode}", (int)zipResponse.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Agent ZIP (AutopilotMonitor-Agent.zip)", (int)zipResponse.StatusCode);
if (results[i].IsSuccessStatusCode) continue;
allOk = false;
_logger.LogError("{Item} not available: HTTP {StatusCode}", probes[i].Label, (int)results[i].StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync(probes[i].Label, (int)results[i].StatusCode);
}
if (!ps1Response.IsSuccessStatusCode)
if (allOk)
{
_logger.LogError("Bootstrap script not available on blob storage: HTTP {StatusCode}", (int)ps1Response.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Bootstrap script (Install-AutopilotMonitor.ps1)", (int)ps1Response.StatusCode);
}
if (zipResponse.IsSuccessStatusCode && ps1Response.IsSuccessStatusCode)
{
_logger.LogInformation("Agent blob storage check passed: all binaries available");
_logger.LogInformation("Agent download endpoint check passed: all binaries available on alias + legacy blob");
}
}
catch (Exception ex)
{
_logger.LogError(ex, "Agent blob storage check failed — storage unreachable");
_logger.LogError(ex, "Agent download endpoint check failed — endpoints unreachable");
await _opsEventService.RecordBlobStorageUnreachableAsync(ex.Message);
}
}
@@ -9,11 +9,13 @@ using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// SignalR Free-tier quota watcher. Reads ConnectionCount (max over the last
/// hour) and MessageCount (total since UTC midnight) from Azure Monitor and
/// emits tiered OpsEvents when either approaches the documented Free-tier
/// caps - 20 concurrent connections and 20,000 messages per day. The
/// operator can then scale to Standard before clients get 429'd.
/// SignalR quota watcher. Reads ConnectionCount (max over the last hour)
/// and MessageCount (total since UTC midnight) from Azure Monitor and
/// emits tiered OpsEvents when either approaches the plan caps - on
/// Standard S1 that is 1,000 concurrent connections per unit (hard limit,
/// excess connections are rejected) and 1M messages per unit per day
/// (included quota; overage is billed, not throttled). The operator can
/// then add units before clients get 429'd or overage costs accumulate.
///
/// Globally scoped (TenantId = null) - SignalR is one shared resource for
/// the whole Function App, not a per-tenant concern.
@@ -25,12 +27,12 @@ namespace AutopilotMonitor.Functions.Services
internal const int SignalRQuotaCriticalPercent = 95;
internal const int SignalRQuotaWarningPercent = 80;
// Default Free-tier caps as documented by Microsoft. Overridable via
// app settings so an operator who has scaled to a paid tier can re-target
// the watcher without a code change (or disable it by setting the cap
// very high).
internal const int DefaultSignalRConnectionLimit = 20;
internal const long DefaultSignalRDailyMessageLimit = 20_000;
// Default caps for Standard S1 with one unit, as documented by Microsoft.
// Overridable via app settings so an operator who adds units or changes
// tier can re-target the watcher without a code change (or disable it by
// setting the cap very high).
internal const int DefaultSignalRConnectionLimit = 1_000;
internal const long DefaultSignalRDailyMessageLimit = 1_000_000;
// Dedup prefix - all SignalR EventTypes start with this, so the seen-index
// can be built with a single Security-category read.
@@ -59,7 +61,7 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Reads SignalR ConnectionCount + MessageCount metrics from Azure Monitor
/// and emits Warning/Error OpsEvents when the Free-tier limits are
/// and emits Warning/Error OpsEvents when the plan limits are
/// approached. Dedup: one event per EventType per UTC day. Fail-soft -
/// missing config or Azure Monitor errors yield a single log line, never
/// a thrown exception.
@@ -75,9 +77,9 @@ namespace AutopilotMonitor.Functions.Services
}
var connectionLimit = ParseIntSetting(
"SignalRFreeTierConnectionLimit", DefaultSignalRConnectionLimit);
"SignalRConnectionLimit", DefaultSignalRConnectionLimit);
var messageLimit = ParseLongSetting(
"SignalRFreeTierDailyMessageLimit", DefaultSignalRDailyMessageLimit);
"SignalRDailyMessageLimit", DefaultSignalRDailyMessageLimit);
var seenToday = await BuildSignalRQuotaSeenIndexAsync();
@@ -32,6 +32,7 @@ namespace AutopilotMonitor.Functions.Services
public bool DataCleanupExecuted { get; set; }
public bool PlatformStatsRecomputed { get; set; }
public int DevicesBlockedForExcessiveData { get; set; }
public int ContactEmailsBackfilled { get; set; }
}
/// <summary>
@@ -64,6 +65,7 @@ namespace AutopilotMonitor.Functions.Services
private readonly IPoisonQueueProbe _poisonQueueProbe;
private readonly IConfiguration _configuration;
private readonly IHttpClientFactory _httpClientFactory;
private readonly PreviewWhitelistService _previewWhitelistService;
private readonly ILogger<MaintenanceService> _logger;
private const string PlatformStatsAliasFileName = "platform-stats.json";
@@ -92,6 +94,7 @@ namespace AutopilotMonitor.Functions.Services
IPoisonQueueProbe poisonQueueProbe,
IConfiguration configuration,
IHttpClientFactory httpClientFactory,
PreviewWhitelistService previewWhitelistService,
ILogger<MaintenanceService> logger)
{
_maintenanceRepo = maintenanceRepo;
@@ -115,6 +118,7 @@ namespace AutopilotMonitor.Functions.Services
_poisonQueueProbe = poisonQueueProbe;
_configuration = configuration;
_httpClientFactory = httpClientFactory;
_previewWhitelistService = previewWhitelistService;
_logger = logger;
}
@@ -214,6 +218,12 @@ namespace AutopilotMonitor.Functions.Services
// StoreSessionAsync Replace-mode IndexRowKey bug (now fixed).
await _maintenanceRepo.CleanupGhostSessionIndexEntriesAsync();
// One-off: give tenants onboarded before ContactEmail existed the contact
// address they already supplied as a preview notification email. New tenants
// are seeded at the point that address is saved, so this converges to a no-op
// and can be dropped once every existing tenant has been covered.
result.ContactEmailsBackfilled = await BackfillTenantContactEmailsAsync();
// Mirror the timer path: check embedded Intune cert bundle for
// expiring members so manual triggers also exercise the watcher.
await CheckEmbeddedCertExpiryAsync();
@@ -0,0 +1,169 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Shared;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// One-time backfill of the <c>OccurredUtc</c> business-timestamp column on
/// AuditLogs and OpsEvents rows written before the column existed. The value is
/// decoded deterministically from the reverse-tick RowKey, so re-runs are
/// idempotent and the write is a pure MERGE that adds exactly one property
/// (nothing existing is modified or deleted). Audit legacy bare-GUID rows carry
/// no recoverable time and are excluded server-side by the scan filter.
/// Readers do not depend on this backfill (they decode the RowKey on the fly);
/// its value is future-migration robustness and raw-query readability.
/// </summary>
public class OccurredUtcBackfillService
{
public const string TableAudit = "audit";
public const string TableOps = "ops";
private const int MaxSamplesInResponse = 100;
private const int TransactionChunkSize = 100;
private readonly TableStorageService _storage;
private readonly ILogger<OccurredUtcBackfillService> _logger;
public OccurredUtcBackfillService(TableStorageService storage, ILogger<OccurredUtcBackfillService> logger)
{
_storage = storage;
_logger = logger;
}
public class BackfillResult
{
public string Table { get; set; } = string.Empty;
public bool DryRun { get; set; }
public int RowsExamined { get; set; }
public int WouldWrite { get; set; }
public int Written { get; set; }
public int SkippedAlreadySet { get; set; }
public int SkippedUndecodable { get; set; }
public int Errors { get; set; }
public string? NextContinuation { get; set; }
public List<BackfillSample> Samples { get; set; } = new();
}
public class BackfillSample
{
public string PartitionKey { get; set; } = string.Empty;
public string RowKey { get; set; } = string.Empty;
public DateTime DecodedUtc { get; set; }
}
internal enum RowDecision { AlreadySet, Write, Undecodable }
/// <summary>Pure per-row decision, extracted for unit testing.</summary>
internal static RowDecision DecideRow(string table, TableEntity entity, out DateTime occurredUtc)
{
occurredUtc = default;
if (BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn).HasValue)
return RowDecision.AlreadySet;
var decoded = table == TableAudit
? BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out occurredUtc)
: BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out occurredUtc);
return decoded ? RowDecision.Write : RowDecision.Undecodable;
}
public async Task<BackfillResult> RunAsync(string table, bool dryRun, int maxRows, string? continuation)
{
var (tableName, scanFilter) = table switch
{
// Audit: server-side restriction to time-encoded '!'-rows — legacy bare-GUID
// rows (which sort after '"') have nothing to decode and are never touched.
TableAudit => (Constants.TableNames.AuditLogs,
$"RowKey ge '{BusinessTimestamp.AuditRowKeyPrefix}' and RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'"),
TableOps => (Constants.TableNames.OpsEvents, (string?)null),
_ => throw new ArgumentException($"Unknown table '{table}'", nameof(table)),
};
var tableClient = _storage.GetTableClient(tableName);
var result = new BackfillResult { Table = table, DryRun = dryRun };
var (entities, nextToken) = await AzureTablesPaginator.FetchPageAsync<TableEntity>(
client: tableClient,
filter: scanFilter,
pageSize: maxRows,
continuation: continuation,
select: new[] { "PartitionKey", "RowKey", BusinessTimestamp.OccurredUtcColumn });
result.NextContinuation = nextToken;
var pending = new List<(string PartitionKey, string RowKey, DateTime OccurredUtc)>();
foreach (var entity in entities)
{
result.RowsExamined++;
switch (DecideRow(table, entity, out var occurredUtc))
{
case RowDecision.AlreadySet:
result.SkippedAlreadySet++;
break;
case RowDecision.Undecodable:
result.SkippedUndecodable++;
_logger.LogWarning("OccurredUtc backfill: undecodable RowKey {PK}/{RK} in {Table}",
entity.PartitionKey, entity.RowKey, tableName);
break;
case RowDecision.Write:
result.WouldWrite++;
pending.Add((entity.PartitionKey, entity.RowKey, occurredUtc));
if (result.Samples.Count < MaxSamplesInResponse)
{
result.Samples.Add(new BackfillSample
{
PartitionKey = entity.PartitionKey,
RowKey = entity.RowKey,
DecodedUtc = occurredUtc,
});
}
break;
}
}
if (!dryRun && pending.Count > 0)
result.Written = await WritePendingAsync(tableClient, pending, result);
_logger.LogInformation(
"OccurredUtc backfill {Table}: dryRun={DryRun} examined={Examined} wouldWrite={WouldWrite} written={Written} alreadySet={AlreadySet} undecodable={Undecodable} errors={Errors} more={More}",
table, dryRun, result.RowsExamined, result.WouldWrite, result.Written,
result.SkippedAlreadySet, result.SkippedUndecodable, result.Errors, result.NextContinuation != null);
return result;
}
private async Task<int> WritePendingAsync(TableClient tableClient,
List<(string PartitionKey, string RowKey, DateTime OccurredUtc)> pending, BackfillResult result)
{
var written = 0;
foreach (var partitionGroup in pending.GroupBy(p => p.PartitionKey))
{
foreach (var chunk in partitionGroup.Chunk(TransactionChunkSize))
{
// Merge entity carries ONLY the new column; unconditional (ETag *) merge is
// safe because the value is a pure function of the immutable RowKey.
var actions = chunk.Select(p => new TableTransactionAction(
TableTransactionActionType.UpdateMerge,
new TableEntity(p.PartitionKey, p.RowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = p.OccurredUtc,
},
ETag.All)).ToList();
try
{
await tableClient.SubmitTransactionAsync(actions);
written += actions.Count;
}
catch (Exception ex)
{
result.Errors += actions.Count;
_logger.LogError(ex, "OccurredUtc backfill: merge transaction failed for partition {PK} ({Count} rows)",
partitionGroup.Key, actions.Count);
}
}
}
return written;
}
}
}
@@ -310,25 +310,25 @@ namespace AutopilotMonitor.Functions.Services
public Task RecordSignalRConnectionsHighAsync(int observed, int limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRConnectionsHigh", OpsEventSeverity.Warning,
$"SignalR concurrent connections at {percent}% of free-tier limit ({observed}/{limit}) - watch for 429s; consider scaling to Standard before saturation",
$"SignalR concurrent connections at {percent}% of plan limit ({observed}/{limit}) - watch for 429s; consider adding units before saturation",
null, "System.Monitoring",
new { metric = "ConnectionCount", aggregation = "Maximum", windowMinutes = 60, observed, limit, percent, resourceId });
public Task RecordSignalRConnectionsCriticalAsync(int observed, int limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRConnectionsCritical", OpsEventSeverity.Error,
$"CRITICAL: SignalR concurrent connections at {percent}% of free-tier limit ({observed}/{limit}) - new client connections will be 429'd at 100%; scale to Standard now",
$"CRITICAL: SignalR concurrent connections at {percent}% of plan limit ({observed}/{limit}) - new client connections will be 429'd at 100%; add units now",
null, "System.Monitoring",
new { metric = "ConnectionCount", aggregation = "Maximum", windowMinutes = 60, observed, limit, percent, resourceId });
public Task RecordSignalRMessagesHighAsync(long observed, long limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRMessagesHigh", OpsEventSeverity.Warning,
$"SignalR daily message count at {percent}% of free-tier limit ({observed}/{limit}) - quota resets at 00:00 UTC; consider scaling if pattern persists",
$"SignalR daily message count at {percent}% of included plan quota ({observed}/{limit}) - resets at 00:00 UTC; overage is billed per extra million messages",
null, "System.Monitoring",
new { metric = "MessageCount", aggregation = "Total", windowDay = "UTC", observed, limit, percent, resourceId });
public Task RecordSignalRMessagesCriticalAsync(long observed, long limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRMessagesCritical", OpsEventSeverity.Error,
$"CRITICAL: SignalR daily message count at {percent}% of free-tier limit ({observed}/{limit}) - hub will throttle at 100% until 00:00 UTC reset; scale to Standard now",
$"CRITICAL: SignalR daily message count at {percent}% of included plan quota ({observed}/{limit}) - overage beyond 100% is billed per extra million messages; review traffic or add units",
null, "System.Monitoring",
new { metric = "MessageCount", aggregation = "Total", windowDay = "UTC", observed, limit, percent, resourceId });
@@ -102,10 +102,44 @@ public class PreviewWhitelistService
}
/// <summary>
/// Saves (or clears) the notification email for a tenant.
/// Saves (or clears) the notification email for a tenant, and seeds the tenant's
/// contact address from it the first time one is given.
/// </summary>
public async Task SaveNotificationEmailAsync(string tenantId, string? email)
{
await _configRepo.SaveNotificationEmailAsync(tenantId, email);
await SeedContactEmailAsync(tenantId, email);
}
/// <summary>
/// One-way seed: copies the preview notification address into
/// <see cref="TenantConfiguration.ContactEmail"/> only while that field is still empty.
/// Once the tenant owns a contact address — seeded or edited in the portal — this never
/// touches it again, so a later change here cannot silently overwrite the tenant's choice.
/// Best-effort: the notification email has already been persisted by the caller, and a
/// failure to seed must not fail that write.
/// </summary>
private async Task SeedContactEmailAsync(string tenantId, string? email)
{
if (string.IsNullOrWhiteSpace(email))
return;
try
{
var config = await _configRepo.GetTenantConfigurationAsync(tenantId);
if (config == null || !string.IsNullOrWhiteSpace(config.ContactEmail))
return;
config.ContactEmail = email.Trim();
await _configRepo.SaveTenantConfigurationAsync(config);
_logger.LogInformation(
"Seeded tenant contact address for {TenantId} from the preview notification email", tenantId);
}
catch (Exception ex)
{
_logger.LogWarning(ex,
"Could not seed tenant contact address for {TenantId} — the notification email was still saved", tenantId);
}
}
}
@@ -1,5 +1,6 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Security;
using AutopilotMonitor.Shared;
@@ -55,7 +56,10 @@ namespace AutopilotMonitor.Functions.Services
{ "EntityType", entityType },
{ "EntityId", entityId },
{ "PerformedBy", performedBy },
{ "Timestamp", timestamp },
// Business event time. "Timestamp" is a reserved system property (server
// ignores supplied values and resets it on every row rewrite), so the
// event time must live in a regular custom column to survive migrations.
{ BusinessTimestamp.OccurredUtcColumn, timestamp },
{ "Details", details != null ? JsonConvert.SerializeObject(details) : string.Empty }
};
@@ -71,32 +75,26 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Retention cleanup: deletes audit log entries whose Timestamp is older than the cutoff.
/// The AuditLogs table is append-only (one row per admin action) and is otherwise only
/// wiped on tenant offboarding, so without this it grows unbounded. Filters server-side on
/// the per-row "Timestamp" property and selects only PK/RK to keep the scan cheap.
/// Retention cleanup: deletes audit log entries whose business time is older than the
/// cutoff. The AuditLogs table is append-only (one row per admin action) and is otherwise
/// only wiped on tenant offboarding, so without this it grows unbounded.
/// Two passes: (1) time-encoded '!'-rows via an index-backed RowKey range — the system
/// Timestamp cannot be used because storage migrations reset it (post-migration it would
/// delete nothing for a retention period, then the whole pre-migration corpus at once);
/// (2) legacy bare-GUID rows (true event time unrecoverable) via their frozen system
/// Timestamp — they all age out ~180d after the 2026-07-18 migration.
/// TODO: Remove pass 2 after 2027-03-01 (no bare-GUID rows can remain by then).
/// </summary>
public async Task<int> DeleteAuditLogsOlderThanAsync(DateTime cutoffUtc)
{
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.AuditLogs);
var filter = $"Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'";
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
var deleted = 0;
deleted += await DeleteAuditLogsByFilterAsync(tableClient, BusinessTimestamp.AuditRetentionClause(cutoffUtc));
// 'RowKey ge '0'' cannot match any '!'-row (0x21 < 0x30), only legacy GUIDs.
deleted += await DeleteAuditLogsByFilterAsync(tableClient,
$"RowKey ge '0' and Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'");
if (deleted > 0)
_logger.LogInformation("Deleted {Count} audit log entries older than {Cutoff:yyyy-MM-dd}", deleted, cutoffUtc);
@@ -110,6 +108,25 @@ namespace AutopilotMonitor.Functions.Services
}
}
private async Task<int> DeleteAuditLogsByFilterAsync(TableClient tableClient, string filter)
{
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
/// <summary>
/// Gets audit log entries for a tenant within an optional UTC date window.
/// No row cap — returns the full filtered set, sorted newest-first.
@@ -384,9 +401,9 @@ namespace AutopilotMonitor.Functions.Services
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
@@ -398,14 +415,30 @@ namespace AutopilotMonitor.Functions.Services
EntityType = entity.GetString("EntityType") ?? string.Empty,
EntityId = entity.GetString("EntityId") ?? string.Empty,
PerformedBy = entity.GetString("PerformedBy") ?? string.Empty,
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.UtcNow,
Timestamp = ResolveAuditTimestamp(entity),
Details = entity.GetString("Details") ?? string.Empty,
};
// The system Timestamp is auto-managed by Azure on insert and reliably
// sortable; the user-defined "Timestamp" property is set by LogAuditEntry
// for parity with the model. We filter on the system Timestamp since it
// is always indexed.
/// <summary>
/// Business time of an audit row: OccurredUtc column → time decoded from the
/// reverse-tick RowKey → system Timestamp. The system Timestamp is only a last
/// resort (legacy bare-GUID rows): it is reset by any row rewrite, so for rows
/// migrated between storage accounts it holds the migration moment, not the
/// event time.
/// </summary>
internal static DateTime ResolveAuditTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.UtcNow;
}
// Date windows filter on the RowKey (reverse-tick, index-backed) rather than any
// timestamp property: the system Timestamp is reset by migrations, and an
// OccurredUtc property filter would exclude rows that lack the column.
internal static string? BuildAuditLogFilter(string? tenantId, DateTime? dateFrom, DateTime? dateTo,
bool excludeDeletions = false, AuditLogQueryFilters? filters = null)
{
@@ -424,11 +457,11 @@ namespace AutopilotMonitor.Functions.Services
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
}
return string.Join(" and ", clauses);
}
@@ -158,6 +158,9 @@ namespace AutopilotMonitor.Functions.Services
["IntervalSeconds"] = rule.IntervalSeconds,
["TriggerPhase"] = rule.TriggerPhase ?? string.Empty,
["TriggerEventType"] = rule.TriggerEventType ?? string.Empty,
["ActivePhasesJson"] = JsonConvert.SerializeObject(rule.ActivePhases ?? new List<string>()),
["ActiveFromPhase"] = rule.ActiveFromPhase ?? string.Empty,
["EmitMode"] = rule.EmitMode ?? string.Empty,
["OutputEventType"] = rule.OutputEventType ?? string.Empty,
["OutputSeverity"] = rule.OutputSeverity ?? "Info",
["TagsJson"] = JsonConvert.SerializeObject(rule.Tags ?? new string[0]),
@@ -222,7 +225,9 @@ namespace AutopilotMonitor.Functions.Services
}
}
private GatherRule MapToGatherRule(TableEntity entity)
// Internal (not private) so the Store↔Map roundtrip is testable per the
// table-serialization rule: every model field must appear in BOTH directions.
internal GatherRule MapToGatherRule(TableEntity entity)
{
return new GatherRule
{
@@ -244,6 +249,9 @@ namespace AutopilotMonitor.Functions.Services
IntervalSeconds = entity.GetInt32("IntervalSeconds"),
TriggerPhase = entity.GetString("TriggerPhase") ?? string.Empty,
TriggerEventType = entity.GetString("TriggerEventType") ?? string.Empty,
ActivePhases = MapActivePhases(entity.GetString("ActivePhasesJson")),
ActiveFromPhase = string.IsNullOrEmpty(entity.GetString("ActiveFromPhase")) ? null : entity.GetString("ActiveFromPhase"),
EmitMode = string.IsNullOrEmpty(entity.GetString("EmitMode")) ? null : entity.GetString("EmitMode"),
OutputEventType = entity.GetString("OutputEventType") ?? string.Empty,
OutputSeverity = entity.GetString("OutputSeverity") ?? "Info",
Tags = DeserializeJsonArray(entity.GetString("TagsJson")),
@@ -252,6 +260,17 @@ namespace AutopilotMonitor.Functions.Services
};
}
/// <summary>
/// Absent column (pre-existing rows) and the stored-empty "[]" both map to null —
/// null and empty are the same "unrestricted" state on <see cref="GatherRule.ActivePhases"/>,
/// and null keeps <see cref="GatherRuleService.ContentEquivalent"/> reseed-stable.
/// </summary>
private List<string>? MapActivePhases(string? json)
{
var phases = DeserializeJson<List<string>>(json);
return phases.Count == 0 ? null : phases;
}
// ===== RULE STATES METHODS =====
/// <summary>
@@ -681,7 +681,10 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// "Timestamp" is a reserved system property — supplied values are ignored
// and any row rewrite (storage migration) resets it. The sanitized agent
// event time therefore lives in its own column.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -777,7 +780,8 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// Reserved-name caveat: see StoreEventAsync.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -2673,10 +2677,7 @@ namespace AutopilotMonitor.Functions.Services
EventId = entity.GetString("EventId") ?? string.Empty,
SessionId = entity.GetString("SessionId") ?? string.Empty,
TenantId = tenantId ?? partitionKey,
Timestamp = DateTime.SpecifyKind(
entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp")
?? DateTime.UtcNow, DateTimeKind.Utc),
Timestamp = DateTime.SpecifyKind(ResolveEventTimestamp(entity), DateTimeKind.Utc),
EventType = entity.GetString("EventType") ?? string.Empty,
Severity = (EventSeverity)(entity.GetInt32("Severity") ?? 0),
Source = entity.GetString("Source") ?? string.Empty,
@@ -2827,7 +2828,10 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Returns the earliest event timestamp persisted for a session, if any.
/// Events are written with RowKey "{Timestamp}_{Sequence}", so querying the partition
/// and taking the first row yields the earliest event.
/// and taking the first row yields the earliest event. Resolution must NOT rely on
/// the system Timestamp: it is reset by storage migrations, and this value feeds the
/// StartedAt/Duration reconciliation — a migration-reset read would rewrite those
/// session fields with the migration date.
/// </summary>
private async Task<DateTime?> GetEarliestSessionEventTimestampAsync(string tenantId, string sessionId)
{
@@ -2838,13 +2842,12 @@ namespace AutopilotMonitor.Functions.Services
var query = tableClient.QueryAsync<TableEntity>(
filter: $"PartitionKey eq '{partitionKey}'",
maxPerPage: 1,
select: new[] { "Timestamp", "RowKey" }
select: new[] { "RowKey", BusinessTimestamp.OccurredUtcColumn, "Timestamp" }
);
await foreach (var entity in query)
{
return entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp");
return ResolveEventTimestampOrNull(entity);
}
}
catch (Exception ex)
@@ -2855,6 +2858,26 @@ namespace AutopilotMonitor.Functions.Services
return null;
}
/// <summary>
/// Business time of an event row: OccurredUtc column → agent time decoded from the
/// "{yyyyMMddHHmmssfff}_{seq}" RowKey → system Timestamp (last resort — reset by any
/// row rewrite, e.g. a storage migration; for migrated rows it holds the migration
/// moment). Event rows are deliberately never backfilled, so the RowKey decode is the
/// primary path for all pre-OccurredUtc rows.
/// </summary>
internal static DateTime ResolveEventTimestamp(TableEntity entity)
=> ResolveEventTimestampOrNull(entity) ?? DateTime.UtcNow;
internal static DateTime? ResolveEventTimestampOrNull(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeEventRowKeyPrefix(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp");
}
/// <summary>
/// Returns the authoritative (total, reboot) event-row counts for a session, or <c>null</c>
/// if they could not be determined (storage error). Events are written with UpsertReplace
@@ -6,9 +6,9 @@
"AzureTableStorageConnectionString": "UseDevelopmentStorage=true",
"AzureBlobStorageConnectionString": "UseDevelopmentStorage=true",
"// SignalR free-tier quota watcher": "Optional. Set to the full Azure resource id of the SignalR Service to enable hourly OpsEvents when the 20-connections / 20k-messages-per-day free-tier limits are approached. Function App's managed identity needs 'Monitoring Reader' on this resource. Leave unset to skip the watcher.",
"// SignalR quota watcher": "Optional. Set to the full Azure resource id of the SignalR Service to enable hourly OpsEvents when the plan limits are approached (defaults: Standard S1 x1 unit = 1000 connections / 1M messages per day; override the limits when adding units or changing tier). Function App's managed identity needs 'Monitoring Reader' on this resource. Leave unset to skip the watcher.",
"SignalRResourceId": "",
"SignalRFreeTierConnectionLimit": "20",
"SignalRFreeTierDailyMessageLimit": "20000"
"SignalRConnectionLimit": "1000",
"SignalRDailyMessageLimit": "1000000"
}
}
+25 -1
View File
@@ -18,6 +18,22 @@ COPY rules/analyze/ ./rules/analyze/
COPY rules/gather/ ./rules/gather/
COPY rules/ime-log-patterns/ ./rules/ime-log-patterns/
# Published customer documentation (autopilotmonitor-docs) — a SEPARATE public
# repo that the deploy workflow checks out into ./mcp-docs of the build context.
# Same byte-identical requirement as the rules above.
#
# This COPY is deliberately strict: if the workflow's docs checkout is ever
# removed, the build must fail loudly rather than quietly ship an image whose
# search_docs tool simply does not exist. For a local `docker build`, clone the
# bundle first:
# git clone --depth 1 https://github.com/autopilotmonitor/autopilotmonitor-docs mcp-docs
COPY mcp-docs/ ./docs/
# Commit SHA of that docs checkout. Surfaced on /health so a running server can be
# diffed against the docs repo HEAD — the bundle changes independently of this
# repo, so a stale deploy is otherwise invisible.
ARG DOCS_COMMIT=unknown
# Pre-bake the ~23 MB embedding model AND precompute the search index at image
# build time. Embedding the static corpus at boot cost 35-55s of ONNX inference
# on the 0.25 vCPU container (measured); hydrating precomputed vectors is
@@ -35,7 +51,7 @@ RUN mkdir -p "$HF_CACHE_DIR" /app/search-index && \
if [ "$PREBAKE_MODEL" = "1" ]; then \
ok=0; \
for i in 1 2 3 4 5; do \
node dist/precompute-embeddings.js ./rules /app/search-index/search-index.json && { ok=1; break; }; \
node dist/precompute-embeddings.js ./rules /app/search-index/search-index.json ./docs && { ok=1; break; }; \
echo "embedding precompute attempt $i/5 failed; retrying in 15s"; \
sleep 15; \
done && \
@@ -66,8 +82,16 @@ COPY rules/analyze/ ./rules/analyze/
COPY rules/gather/ ./rules/gather/
COPY rules/ime-log-patterns/ ./rules/ime-log-patterns/
# Customer documentation corpus — copied from the build stage rather than the
# context so it is provably the same bytes the embeddings were computed over.
COPY --from=build /app/docs/ ./docs/
ARG DOCS_COMMIT=unknown
ENV DOCS_COMMIT=${DOCS_COMMIT}
ENV PORT=8080
ENV RULES_DIR=/app/rules
ENV DOCS_DIR=/app/docs
ENV SEARCH_INDEX_PATH=/app/search-index/search-index.json
EXPOSE 8080
+52 -22
View File
@@ -1,12 +1,12 @@
{
"name": "autopilot-monitor-mcp",
"version": "1.5.0",
"version": "1.5.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "autopilot-monitor-mcp",
"version": "1.5.0",
"version": "1.5.1",
"dependencies": {
"@huggingface/transformers": "^3.8.1",
"@modelcontextprotocol/sdk": "^1.28.0",
@@ -1730,20 +1730,20 @@
}
},
"node_modules/body-parser": {
"version": "2.2.2",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.2.2.tgz",
"integrity": "sha512-oP5VkATKlNwcgvxi0vM0p/D3n2C3EReYVX+DNYs5TjZFn/oQt2j+4sVJtSMr18pdRr8wjTcBl6LoV+FUwzPmNA==",
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/body-parser/-/body-parser-2.3.0.tgz",
"integrity": "sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==",
"license": "MIT",
"dependencies": {
"bytes": "^3.1.2",
"content-type": "^1.0.5",
"content-type": "^2.0.0",
"debug": "^4.4.3",
"http-errors": "^2.0.0",
"iconv-lite": "^0.7.0",
"http-errors": "^2.0.1",
"iconv-lite": "^0.7.2",
"on-finished": "^2.4.1",
"qs": "^6.14.1",
"raw-body": "^3.0.1",
"type-is": "^2.0.1"
"qs": "^6.15.2",
"raw-body": "^3.0.2",
"type-is": "^2.1.0"
},
"engines": {
"node": ">=18"
@@ -1753,6 +1753,19 @@
"url": "https://opencollective.com/express"
}
},
"node_modules/body-parser/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/boolean": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz",
@@ -3230,9 +3243,9 @@
}
},
"node_modules/protobufjs": {
"version": "7.6.4",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.4.tgz",
"integrity": "sha512-RJJPTTpvFfHcWLkIa2JFWK4XvtSzS0yEWDmunqHXli1h3JlkbcQZXDZdcWxv+JK3Xsl5/UFDPZ0iGm7DAengYw==",
"version": "7.6.5",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.5.tgz",
"integrity": "sha512-/FPD0nUc9jH6rfFjji9IBqOz4pcSE3CsT1m7Ep6Mdb0LxSUMj8hgl6GomOvZzpNpAqqGaXA0P3VSrZLFzIhQrw==",
"hasInstallScript": true,
"license": "BSD-3-Clause",
"dependencies": {
@@ -3674,9 +3687,9 @@
"license": "MIT"
},
"node_modules/tar": {
"version": "7.5.16",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.16.tgz",
"integrity": "sha512-56adEpPMouktRlBLXiaYFFzZ/3+JXa8P9n7WbR+ibIjtviN55mEaOkiysCnPnWm+7kkui1Dn8J9l+g6zV8731w==",
"version": "7.5.20",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.20.tgz",
"integrity": "sha512-9FcyK4PA6+WbzlTM9WhQm6vB5W7cP7dUiPsv1g7YDwEQnQ1CGpK3MGlKk/ITVWMk05kHZuBhmVhiv8LZoy/PFQ==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
@@ -3781,17 +3794,34 @@
}
},
"node_modules/type-is": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.0.1.tgz",
"integrity": "sha512-OZs6gsjF4vMp32qrCbiVSkrFmXtG/AZhY3t0iAMrMBiAZyV9oALtXO8hsrHbMXF9x6L3grlFuwW2oAz7cav+Gw==",
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/type-is/-/type-is-2.1.0.tgz",
"integrity": "sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==",
"license": "MIT",
"dependencies": {
"content-type": "^1.0.5",
"content-type": "^2.0.0",
"media-typer": "^1.1.0",
"mime-types": "^3.0.0"
},
"engines": {
"node": ">= 0.6"
"node": ">= 18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/type-is/node_modules/content-type": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/content-type/-/content-type-2.0.0.tgz",
"integrity": "sha512-j/O/d7GcZCyNl7/hwZAb606rzqkyvaDctLmckbxLzHvFBzTJHuGEdodATcP3yIRoDrLHkIATJuvzbFlp/ki2cQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/typescript": {
@@ -0,0 +1,237 @@
/**
* Offline retrieval evaluation for search_docs.
*
* Exists because a single anecdote is not evidence. The WOW6432Node case showed
* that the keyword fallback misses when semantic search returns weak-but-present
* results — but "it fixes that one query" says nothing about how many queries a fix
* BREAKS. This harness answers that: it scores several ranking strategies over the
* same query set and reports, per class, how many queries each strategy fixes and
* how many it regresses relative to the deployed behaviour.
*
* Three query classes, deliberately different in character:
*
* known-item Every chunk's own heading as the query, expecting any chunk with
* that heading back. Easy by construction — its value is as a
* REGRESSION detector, not an absolute score. If a strategy drops
* these, it is breaking retrieval of things the corpus plainly names.
* literal Rare identifiers (few chunks, >=6 chars) as single-word queries,
* expecting the chunks that literally contain them. This is the class
* WOW6432Node belongs to and the one the fallback exists for.
* handwritten Real questions a human would ask, with the page they should reach.
* The reality anchor: unlike the other two, these are not derived from
* the corpus and share little vocabulary with it.
*
* Embeddings are computed ONCE per query and every strategy is a pure function over
* the same semantic + literal candidate lists, so a threshold sweep costs no extra
* inference.
*
* Usage: npx tsx scripts/eval-docs-search.ts [docsDir]
*/
import { loadDocsCorpus } from '../src/docs-corpus.js';
import { VectorSearchProvider } from '../src/vector-search-provider.js';
import { extractDistinctiveTerms } from '../src/tools/search.js';
import type { SearchDocument, SearchResult } from '../src/search-provider.js';
const DOCS_DIR = process.argv[2] ?? 'c:/Code/GitHubRepos/autopilotmonitor-docs';
const TOP_K = 3;
const MIN_SCORE = 0.25;
interface EvalQuery {
klass: 'known-item' | 'literal' | 'handwritten';
query: string;
/** Any of these ids counts as correct. */
expected: Set<string>;
}
// ── Query set ────────────────────────────────────────────────
function buildQueries(docs: SearchDocument[]): EvalQuery[] {
const queries: EvalQuery[] = [];
// known-item: heading → chunks carrying that heading.
const byHeading = new Map<string, Set<string>>();
for (const d of docs) {
const h = d.metadata.heading as string | null;
if (!h) continue;
const leaf = h.split('›').pop()!.trim();
if (leaf.length < 8) continue;
if (!byHeading.has(leaf)) byHeading.set(leaf, new Set());
byHeading.get(leaf)!.add(d.id);
}
for (const [heading, ids] of byHeading) {
queries.push({ klass: 'known-item', query: heading, expected: ids });
}
// literal: rare identifier-ish tokens → chunks literally containing them.
const occurrences = new Map<string, Set<string>>();
for (const d of docs) {
for (const t of new Set(d.text.toLowerCase().match(/[a-z][a-z0-9_-]{5,}/g) ?? [])) {
if (!occurrences.has(t)) occurrences.set(t, new Set());
occurrences.get(t)!.add(d.id);
}
}
for (const [token, ids] of occurrences) {
if (ids.size > 3) continue; // not rare enough to be a known-item probe
// Expected set is every chunk literally containing it, not just the tokenized hits.
const containing = new Set(docs.filter((d) => d.text.toLowerCase().includes(token)).map((d) => d.id));
queries.push({ klass: 'literal', query: token, expected: containing });
}
// handwritten: the reality anchor.
const byPath = (path: string) => new Set(docs.filter((d) => d.metadata.path === path).map((d) => d.id));
const handwritten: Array<[string, string]> = [
['how do I deploy the agent with Intune', 'troubleshooting/faq.md'],
['where is my data stored', 'troubleshooting/faq.md'],
['which role is allowed to delete a session', 'portal-guide/dashboard-and-sessions.md'],
['how do I set up Teams notifications', 'reference/settings.md'],
['what firewall URLs do I need to allow', 'reference/network-endpoints.md'],
['is Autopilot Monitor free', 'troubleshooting/faq.md'],
['how long is my data retained before deletion', 'trust/security-faq.md'],
['what is a gather rule and when does it run', 'rules/gather-rules.md'],
['what can helpdesk staff see in the progress portal', 'portal-guide/progress-portal.md'],
['can an MSP access my tenant', 'trust/security-faq.md'],
['Office Click-to-Run install never finished', 'rules/analyze-rules/built-in-rules.md'],
['proxy configuration blocking app downloads', 'rules/analyze-rules/built-in-rules.md'],
];
for (const [query, path] of handwritten) {
queries.push({ klass: 'handwritten', query, expected: byPath(path) });
}
return queries;
}
// ── Strategies ───────────────────────────────────────────────
interface Candidates {
semantic: SearchResult[]; // sorted desc, unfiltered
literal: SearchResult[]; // corroborated, ranked by term count
}
type Strategy = (c: Candidates) => SearchResult[];
/** Deployed behaviour: semantic, topped up by literal only when short of topK. */
const baseline: Strategy = ({ semantic, literal }) => {
const res = semantic.filter((s) => s.score >= MIN_SCORE).slice(0, TOP_K);
if (res.length < TOP_K) {
const have = new Set(res.map((r) => r.id));
for (const l of literal) {
if (res.length >= TOP_K) break;
if (have.has(l.id)) continue;
have.add(l.id);
res.push(l);
}
}
return res;
};
/**
* Candidate fix: additionally treat a WEAK semantic head as unconvincing and let
* exact literal matches take the front, keeping at least one semantic result so a
* weak-but-right hit is never fully displaced.
*/
function weakTrigger(threshold: number): Strategy {
return ({ semantic, literal }) => {
const res = semantic.filter((s) => s.score >= MIN_SCORE).slice(0, TOP_K);
const have = new Set(res.map((r) => r.id));
const fresh = literal.filter((l) => !have.has(l.id));
if (res.length < TOP_K) {
for (const l of fresh) {
if (res.length >= TOP_K) break;
res.push(l);
}
return res;
}
if (fresh.length > 0 && (res[0]?.score ?? 0) < threshold) {
return [...fresh.slice(0, TOP_K - 1), ...res].slice(0, TOP_K);
}
return res;
};
}
// ── Scoring ──────────────────────────────────────────────────
interface ClassStats {
n: number;
hit: number; // recall@TOP_K
mrr: number;
}
function score(results: SearchResult[], expected: Set<string>): { hit: boolean; rr: number } {
const rank = results.findIndex((r) => expected.has(r.id));
return { hit: rank >= 0, rr: rank >= 0 ? 1 / (rank + 1) : 0 };
}
async function main() {
const docs = await loadDocsCorpus(DOCS_DIR);
console.error(`corpus: ${docs.length} chunks`);
const index = new VectorSearchProvider();
await index.index(docs);
console.error(`indexed: ${index.vectorCount} vectors\n`);
const queries = buildQueries(docs);
const counts = queries.reduce<Record<string, number>>((a, q) => ({ ...a, [q.klass]: (a[q.klass] ?? 0) + 1 }), {});
console.error(`queries: ${queries.length} (${Object.entries(counts).map(([k, v]) => `${k}=${v}`).join(', ')})`);
console.error('embedding queries…');
// One inference pass; strategies are pure functions over these candidates.
const candidates: Candidates[] = [];
for (let i = 0; i < queries.length; i++) {
const q = queries[i];
const semantic = await index.search(q.query, { topK: 10, minScore: -1 });
const terms = extractDistinctiveTerms(q.query);
let literal: SearchResult[] = [];
if (terms.length > 0) {
const required = Math.min(2, terms.length);
literal = index
.lexicalMatch(terms)
.map((hit) => ({ hit, matched: terms.filter((t) => hit.text.toLowerCase().includes(t)).length }))
.filter((x) => x.matched >= required)
.sort((a, b) => b.matched - a.matched)
.map((x) => x.hit);
}
candidates.push({ semantic, literal });
if ((i + 1) % 200 === 0) console.error(` ${i + 1}/${queries.length}`);
}
const strategies: Array<[string, Strategy]> = [
['baseline (deployed)', baseline],
['weak<0.30', weakTrigger(0.30)],
['weak<0.35', weakTrigger(0.35)],
['weak<0.40', weakTrigger(0.40)],
['weak<0.45', weakTrigger(0.45)],
];
// Per-query correctness under the baseline, for the fixed/broken delta.
const baseHit = queries.map((q, i) => score(baseline(candidates[i]), q.expected).hit);
console.log(`\n${'strategy'.padEnd(20)} ${'class'.padEnd(12)} ${'n'.padStart(5)} ${'recall@3'.padStart(9)} ${'MRR'.padStart(7)} ${'fixed'.padStart(6)} ${'broken'.padStart(7)}`);
console.log('-'.repeat(75));
for (const [name, strat] of strategies) {
const stats = new Map<string, ClassStats>();
let fixed = 0;
let broken = 0;
queries.forEach((q, i) => {
const { hit, rr } = score(strat(candidates[i]), q.expected);
const s = stats.get(q.klass) ?? { n: 0, hit: 0, mrr: 0 };
s.n++;
if (hit) s.hit++;
s.mrr += rr;
stats.set(q.klass, s);
if (hit && !baseHit[i]) fixed++;
if (!hit && baseHit[i]) broken++;
});
for (const [klass, s] of [...stats].sort()) {
console.log(
`${name.padEnd(20)} ${klass.padEnd(12)} ${String(s.n).padStart(5)} ` +
`${((s.hit / s.n) * 100).toFixed(1).padStart(8)}% ${(s.mrr / s.n).toFixed(3).padStart(7)} ` +
`${'—'.padStart(6)} ${'—'.padStart(7)}`,
);
}
console.log(`${name.padEnd(20)} ${'TOTAL'.padEnd(12)} ${String(queries.length).padStart(5)} ${''.padStart(9)} ${''.padStart(7)} ${String(fixed).padStart(6)} ${String(broken).padStart(7)}`);
console.log('-'.repeat(75));
}
}
await main();
@@ -0,0 +1,210 @@
/**
* Unit tests for the customer-documentation corpus loader.
*
* The bundle is authored on Windows and ships CRLF, which is the single most
* dangerous property of this input: in JavaScript `.` does not match `\r`, so a
* pattern like /^(#{2,3})\s+(.*)$/ silently matches NOTHING on a CRLF file. The
* failure is not a crash — headings and frontmatter just stop being recognized
* and every page degrades to unlabelled paragraph splits. Several tests below
* therefore run the CRLF form on purpose.
*/
import { describe, it, expect } from 'vitest';
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { loadDocsCorpus, parseFrontmatter, docUrl, slugify, docSections } from '../docs-corpus.js';
async function withBundle<T>(
files: Record<string, string>,
fn: (root: string) => Promise<T>,
): Promise<T> {
const root = await mkdtemp(join(tmpdir(), 'docs-corpus-'));
try {
for (const [rel, content] of Object.entries(files)) {
const full = join(root, rel);
await mkdir(join(full, '..'), { recursive: true });
await writeFile(full, content, 'utf-8');
}
return await fn(root);
} finally {
await rm(root, { recursive: true, force: true });
}
}
describe('parseFrontmatter', () => {
it('reads scalars, folded blocks and inline sequences', () => {
const { frontmatter, body } = parseFrontmatter(
['---', 'type: Concept', 'tags: [security, gdpr]', 'description: >-', ' First line', ' second line', '---', '', '# Title', 'Body.'].join('\n'),
);
expect(frontmatter.type).toBe('Concept');
expect(frontmatter.tags).toEqual(['security', 'gdpr']);
expect(frontmatter.description).toBe('First line second line');
expect(body).toContain('# Title');
expect(body).not.toContain('type: Concept');
});
it('reads block sequences', () => {
const { frontmatter } = parseFrontmatter(['---', 'type: Guide', 'tags:', ' - agent', ' - intune', '---', 'x'].join('\n'));
expect(frontmatter.tags).toEqual(['agent', 'intune']);
});
it('parses CRLF frontmatter identically to LF', () => {
const lf = ['---', 'type: Concept', 'tags: [a, b]', '---', '', '# T', 'x'].join('\n');
expect(parseFrontmatter(lf.replace(/\n/g, '\r\n'))).toEqual(parseFrontmatter(lf));
});
it('returns the whole input as body when there is no frontmatter', () => {
const { frontmatter, body } = parseFrontmatter('# Just a title\n\nText.');
expect(frontmatter).toEqual({ type: undefined, title: undefined, description: undefined, tags: undefined });
expect(body).toBe('# Just a title\n\nText.');
});
});
describe('docUrl', () => {
it('maps a page path to its published URL', () => {
expect(docUrl('trust/security-faq.md')).toBe('https://docs.autopilotmonitor.com/trust/security-faq');
});
it('collapses README.md to its folder', () => {
expect(docUrl('rules/analyze-rules/README.md')).toBe('https://docs.autopilotmonitor.com/rules/analyze-rules');
expect(docUrl('README.md')).toBe('https://docs.autopilotmonitor.com');
});
it('has no URL for index.md — GitBook never publishes the OKF catalog', () => {
expect(docUrl('index.md')).toBeUndefined();
});
});
describe('slugify', () => {
it('strips markdown and punctuation', () => {
expect(slugify('Is the device\'s **IP address** stored?')).toBe('is-the-device-s-ip-address-stored');
});
});
describe('loadDocsCorpus', () => {
const page = (fm: string, body: string) => `---\n${fm}\n---\n\n${body}`;
it('splits on h2/h3 headings and carries a breadcrumb', async () => {
const chunks = await withBundle(
{
'concepts/roles.md': page(
'type: Concept\ntags: [roles]',
['# Roles & Permissions', '', 'Intro prose that is long enough to survive the minimum-length filter for chunks.', '', '## Tenant roles', '', 'A tenant admin can do everything, including managing team members and enabling admin mode.', '', '### Operator', '', 'An operator monitors day to day and cannot perform destructive operations at all.'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
const headings = chunks.map((c) => c.metadata.heading);
expect(headings).toContain('Tenant roles');
expect(headings).toContain('Tenant roles › Operator');
// The page title is prepended to every chunk so a mid-page hit is self-describing.
expect(chunks.every((c) => c.text.startsWith('Roles & Permissions'))).toBe(true);
expect(chunks[0].metadata.section).toBe('concepts');
expect(chunks[0].metadata.tags).toEqual(['roles']);
});
it('produces the SAME chunks for CRLF input as for LF', async () => {
const body = ['# Guide', '', 'Intro paragraph long enough to be kept as its own indexed chunk here.', '', '## Step one', '', 'Do the first thing, which needs a sentence long enough to pass the minimum.', '', '## Step two', '', 'Then do the second thing, also written out at a reasonable length for this.'].join('\n');
const src = page('type: Guide\ntags: [setup]', body);
const lf = await withBundle({ 'getting-started/g.md': src }, (r) => loadDocsCorpus(r));
const crlf = await withBundle({ 'getting-started/g.md': src.replace(/\n/g, '\r\n') }, (r) => loadDocsCorpus(r));
expect(crlf).toEqual(lf);
expect(lf.map((c) => c.metadata.heading)).toEqual([null, 'Step one', 'Step two']);
expect(lf.some((c) => c.text.includes('\r'))).toBe(false);
});
it('treats <details>/<summary> blocks as sections', async () => {
const chunks = await withBundle(
{
'troubleshooting/faq.md': page(
'type: FAQ\ntags: [faq]',
['# FAQ', '', '## General', '', '<details>', '', '<summary>Is Autopilot Monitor free?</summary>', '', 'Yes, the Community plan is free and stays free for everyone who wants to use it.', '', '</details>'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
expect(chunks.map((c) => c.metadata.heading)).toContain('General › Is Autopilot Monitor free?');
});
it('never emits a heading from inside a fenced code block', async () => {
const chunks = await withBundle(
{
'reference/cli.md': page(
'type: Reference\ntags: [cli]',
['# CLI', '', '## Usage', '', '```powershell', '# This is a shell comment, not a heading', '## Neither is this', '```', '', 'Explanatory prose after the block so the section is comfortably long enough.'].join('\n'),
),
},
(root) => loadDocsCorpus(root),
);
expect(chunks.map((c) => c.metadata.heading)).toEqual(['Usage']);
});
it('skips SUMMARY.md and log.md but keeps index.md', async () => {
const chunks = await withBundle(
{
'SUMMARY.md': '# Table of contents\n\n* [A](a.md)\n',
'log.md': '# Log\n\nSome change record entry that is long enough to be a chunk on its own.\n',
'index.md': page('okf_version: "0.1"', '# Bundle\n\nThe documentation map listing every page available in this bundle here.'),
},
(root) => loadDocsCorpus(root),
);
const paths = [...new Set(chunks.map((c) => c.metadata.path))];
expect(paths).toEqual(['index.md']);
// index.md is indexed for its content but is not a published page.
expect(chunks[0].metadata.url).toBeNull();
});
it('splits an oversized unbroken list so its tail is not truncated away', async () => {
const items = Array.from({ length: 60 }, (_, i) => `* Release note number ${i} describing a change that shipped in this release.`);
const chunks = await withBundle(
{ 'changelog/platform.md': page('type: Changelog\ntags: [changelog]', ['# Platform Changelog', '', '## July 2026', '', ...items].join('\n')) },
(root) => loadDocsCorpus(root),
);
expect(chunks.length).toBeGreaterThan(1);
expect(chunks.every((c) => c.text.length <= 1700)).toBe(true);
// The final item must still be reachable — that is the whole point of splitting.
expect(chunks.some((c) => c.text.includes('number 59'))).toBe(true);
});
it('keeps a markdown table intact rather than splitting mid-table', async () => {
const rows = Array.from({ length: 40 }, (_, i) => `| Setting ${i} | A reasonably wordy description of what this particular setting controls. |`);
const chunks = await withBundle(
{ 'reference/settings.md': page('type: Reference\ntags: [settings]', ['# Settings', '', '## All settings', '', '| Name | Description |', '| --- | --- |', ...rows].join('\n')) },
(root) => loadDocsCorpus(root),
);
const withTable = chunks.filter((c) => c.text.includes('| Setting '));
expect(withTable).toHaveLength(1);
expect(withTable[0].text).toContain('| Setting 39 |');
});
it('assigns unique, stable ids across repeated loads', async () => {
const files = {
'a/one.md': page('type: Guide\ntags: [x]', '# One\n\n## Same\n\nFirst body text that is long enough to be indexed as a chunk.\n\n## Same\n\nSecond body text that is also long enough to be indexed here.'),
};
const first = await withBundle(files, (r) => loadDocsCorpus(r));
const second = await withBundle(files, (r) => loadDocsCorpus(r));
expect(new Set(first.map((c) => c.id)).size).toBe(first.length);
expect(first.map((c) => c.id)).toEqual(second.map((c) => c.id));
});
it('returns an empty corpus for a missing directory instead of throwing', async () => {
await expect(loadDocsCorpus(join(tmpdir(), 'definitely-not-here-9d2f'))).resolves.toEqual([]);
});
});
describe('docSections', () => {
it('lists distinct top-level areas, sorted', async () => {
const chunks = await withBundle(
{
'trust/a.md': '# A\n\nSome prose that is definitely long enough to become an indexed chunk.',
'concepts/b.md': '# B\n\nMore prose that is definitely long enough to become an indexed chunk.',
'root.md': '# R\n\nRoot level prose that is long enough to become an indexed chunk too.',
},
(root) => loadDocsCorpus(root),
);
expect(docSections(chunks)).toEqual(['concepts', 'general', 'trust']);
});
});
@@ -14,8 +14,19 @@ const MODEL = 'Xenova/all-MiniLM-L6-v2';
const doc = (id: string, text: string, metadata: Record<string, unknown> = {}): SearchDocument =>
({ id, text, metadata });
/**
* Fake embeddings still have to satisfy the real contract: indexPrecomputed()
* asserts L2-unit norm, because cosineSimilarity is a bare dot product that would
* otherwise return silently wrong scores. Normalize here rather than hand-writing
* unit vectors, so a test can pass any direction it finds readable.
*/
const unit = (v: number[]): number[] => {
const norm = Math.sqrt(v.reduce((s, x) => s + x * x, 0));
return v.map((x) => x / norm);
};
const withEmbedding = (d: SearchDocument, embedding: number[] = [0.1, 0.2, 0.3]): PrecomputedDocument =>
({ ...d, embedding });
({ ...d, embeddings: [unit(embedding)] });
const KB_DOCS = [doc('rule-1', 'TPM not ready', { type: 'analyze-rule' }), doc('rule-2', 'ESP timeout', { type: 'analyze-rule' })];
const ET_DOCS = [doc('et-1', 'bitlocker status', { eventType: 'bitlocker_status' })];
@@ -99,14 +110,73 @@ describe('validatePrecomputedIndex', () => {
it('rejects inconsistent embedding dimensions', () => {
const f = validFile();
f.knowledgeBase.entries[1].embedding = [0.1, 0.2]; // 2 dims vs 3
f.knowledgeBase.entries[1].embeddings = [[0.1, 0.2]]; // 2 dims vs 3
expect(validatePrecomputedIndex(f, MODEL, KB_DOCS, ET_DOCS)).toMatchObject({
ok: false, reason: expect.stringContaining('dimensions'),
});
});
});
describe('docs section (optional third corpus)', () => {
const DOC_CHUNKS = [doc('docs:trust/security-faq.md#intro', 'Where is my data stored', { type: 'doc', section: 'trust' })];
const withDocs = (): PrecomputedIndexFile => ({
...validFile(),
docs: { docsHash: hashDocs(DOC_CHUNKS), entries: DOC_CHUNKS.map((d) => withEmbedding(d)) },
});
it('returns the docs entries when the corpus matches', () => {
const result = validatePrecomputedIndex(withDocs(), MODEL, KB_DOCS, ET_DOCS, DOC_CHUNKS);
expect(result).toMatchObject({ ok: true });
expect(result.ok && result.docs).toHaveLength(1);
});
it('rejects the whole file when the docs corpus changed since the build', () => {
const stale = [doc('docs:trust/security-faq.md#intro', 'DIFFERENT TEXT', { type: 'doc', section: 'trust' })];
expect(validatePrecomputedIndex(withDocs(), MODEL, KB_DOCS, ET_DOCS, stale)).toMatchObject({
ok: false, reason: expect.stringContaining('docs'),
});
});
it('rejects when the server has docs but the image shipped none — that is staleness, not absence', () => {
expect(validatePrecomputedIndex(validFile(), MODEL, KB_DOCS, ET_DOCS, DOC_CHUNKS)).toMatchObject({
ok: false, reason: expect.stringContaining('docs'),
});
});
it('accepts a file WITHOUT a docs section when the server has no docs corpus', () => {
// A local checkout has no docs repo. That must not throw away the rules and
// event-type vectors too — recomputing those is the expensive half.
const result = validatePrecomputedIndex(validFile(), MODEL, KB_DOCS, ET_DOCS, []);
expect(result).toMatchObject({ ok: true });
expect(result.ok && result.docs).toEqual([]);
});
});
describe('VectorSearchProvider serialize/hydrate roundtrip', () => {
it('rounds embeddings on serialize but stays well inside the unit-norm tolerance', () => {
const p = new VectorSearchProvider();
const raw = unit([0.123456789012345, -0.987654321098765, 0.5555555555555]);
p.indexPrecomputed([{ ...doc('d1', 'text'), embeddings: [raw] }]);
const [out] = p.serialize();
// Rounded to 6 decimals — the point of the exercise (halves the index JSON).
expect(out.embeddings[0].every((v) => v === Number(v.toFixed(6)))).toBe(true);
expect(out.embeddings[0]).not.toEqual(raw);
const norm = Math.sqrt(out.embeddings[0].reduce((s, x) => s + x * x, 0));
expect(Math.abs(norm - 1)).toBeLessThan(1e-5);
// Re-hydrating the rounded vectors must not trip the assertion.
expect(() => new VectorSearchProvider().indexPrecomputed(p.serialize())).not.toThrow();
});
it('rejects a non-unit embedding on hydrate rather than skewing every score silently', () => {
const p = new VectorSearchProvider();
expect(() => p.indexPrecomputed([{ ...doc('d1', 'text'), embeddings: [[0.1, 0.2, 0.3]] }]))
.toThrow(/not L2-unit-normalized/);
});
it('indexPrecomputed restores exactly what serialize exported, without the model', () => {
const source = new VectorSearchProvider();
source.indexPrecomputed(KB_DOCS.map((d, i) => withEmbedding(d, [i + 0.5, i + 1.5])));
@@ -20,16 +20,62 @@
import { describe, it, expect } from 'vitest';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { registerTools } from '../tools.js';
import type { SearchProvider } from '../search-provider.js';
function registeredToolNames(ga: boolean, strictGa: boolean = ga, delegated: boolean = false): string[] {
const server = new McpServer({ name: 'test', version: '0.0.0' });
// knowledgeBase / eventTypeIndex are optional — pass undefined so this stays a
// pure unit test with no search-provider or backend dependency.
registerTools(server, undefined, undefined, ga, strictGa, delegated);
registerTools(server, undefined, undefined, undefined, ga, strictGa, delegated);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
return Object.keys(internal._registeredTools);
}
/** Minimal stand-in for an indexed docs corpus — registration only checks `size`. */
function stubDocsIndex(size = 3): SearchProvider {
return {
name: 'stub',
semanticCapable: true,
size,
index: async () => {},
search: async () => [],
};
}
function namesWithDocs(ga: boolean, strictGa: boolean = ga, delegated: boolean = false): string[] {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, { vector: stubDocsIndex(), sections: ['concepts', 'trust'] }, ga, strictGa, delegated);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
return Object.keys(internal._registeredTools);
}
describe('search_docs registration', () => {
it('is NOT registered when no documentation corpus was baked into the image', () => {
expect(registeredToolNames(true)).not.toContain('search_docs');
});
it('is registered when a corpus is present — for EVERY role, including a plain tenant user', () => {
// Published product documentation carries no tenant data, so unlike the
// platform tools it is deliberately ungated. A tenant user asking "how do I
// deploy the agent" has exactly as much business here as a Global Admin.
expect(namesWithDocs(true, true)).toContain('search_docs');
expect(namesWithDocs(false, false)).toContain('search_docs');
expect(namesWithDocs(false, false, true)).toContain('search_docs');
});
it('is skipped for an empty corpus, so a doc-less build advertises no broken tool', () => {
const server = new McpServer({ name: 'test', version: '0.0.0' });
registerTools(server, undefined, undefined, { vector: stubDocsIndex(0), sections: [] }, true, true, false);
const internal = server as unknown as { _registeredTools: Record<string, unknown> };
expect(Object.keys(internal._registeredTools)).not.toContain('search_docs');
});
it('keeps the catalog alphabetically sorted once it joins', () => {
const names = namesWithDocs(true);
expect(names).toEqual([...names].sort());
});
});
describe('tool catalog ordering', () => {
it('lists tools in alphabetical order for a Global Admin', () => {
const names = registeredToolNames(true);

Some files were not shown because too many files have changed in this diff Show More