Compare commits

...

27 Commits

Author SHA1 Message Date
Oliver Kieselbach d8c84f41c1 fix(agent): spool pressure trigger on pending backlog, not file size
99/100 spool_pressure_detected events in the field were file-size-only
triggers on a near-empty backlog (append-only spool.jsonl grows past 5 MB
in healthy long sessions). TelemetrySpool now tracks PendingBytes (exact
serialized size of the not-yet-uploaded tail, maintained on enqueue /
mark-uploaded / restart rehydrate) and the collector thresholds on that
instead of SpoolFileSizeBytes, which stays as an informational metric.
Snapshot gains spool_pending_bytes; pressure event data carries
pendingBytes + pendingBytesThreshold.

Also: startup recovery scan streams via File.ReadLines instead of
materializing the whole spool with File.ReadAllLines (corrupt-tail
break semantics unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 17:43:15 +02:00
Oliver Kieselbach 12b51bbc24 feat(landing): scenario coverage chips in hero + MSI channel in about claims
Hero gets a quiet checkmark chip row (user-driven, pre-provisioning,
self-deploying, Device Preparation, Windows 365, Entra ID & Hybrid join);
about page deployment claim now includes the MSI LOB channel for WDP.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 15:47:15 +02:00
Oliver Kieselbach a369ff99cd feat(wdp): scenario gates for Device Preparation + completion backstop
afee7ae0 audit: WDP sessions ran the full Classic/ESP apparatus. Gates key on
the deterministic BootstrapperAgent ExecutionContext marker only (CloudAssigned*
fallbacks keep Classic behavior).

Paket A (event hygiene): no autopilot_profile/-missing + ZTD query/probe on WDP;
esp_config_detected null-gate (no more unknown/unknown); starved sweep skipped
on WDP (data stays in app_tracking_summary/final-status).

Paket B (resources): EspPolicyProviderStallHost + ProvisioningStatusTracker not
started on WDP; ModernDeployment noise IDs 100/417/418/1005 fully suppressed
(Critical always passes) + Autopilot error backfill skipped; StallProbe skips
ESP category scan and treats noise IDs as harmless.

Paket C (DecisionCore): enrollmentTypeDeterministic payload key seeds
DevicePreparation at High confidence (fallbacks stay Medium); completion_waiting
drops the unsatisfiable account_setup_provisioning_complete gate on WDP; arm D
promotes on real-user desktop; new device_prep_completion 30-min backstop closes
the park-until-watchdog hole; parked tripwire gets a WDP branch.

17 new tests; suites green (532/2307/4287). OKF doc agent/wdp-scenario-gates.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 13:12:09 +02:00
Oliver Kieselbach 7a131af74d agent(summary): DevicePrep app-wave settle wait + Office/CSP row in summary dialog
Session bc803955 (DevicePrep, Succeeded) rendered "No application data
available" although the M365 CSP package installed during prep and IME
delivered 12 required user apps 9 seconds after final-status was written.

- EnrollmentTerminationHandler: bounded settle wait (20s, 250ms poll) before
  the dialog snapshot - DevicePreparation-only and only while the tracker has
  zero packages, so Classic pays nothing. Root cause of the lost race is IME
  AppWorkload.log flush latency (~10-20s), not the 100ms tracker poll.
  Dialog, starved sweep and app_tracking_summary all see settled data.
- FinalStatusBuilder: synthetic "Microsoft 365 Apps" row from the Office C2R
  detector's persisted lifecycle state (Office CSP installs never flow through
  the IME Win32 pipeline). Prepended to the Device bucket, counted in
  appSummary; Preinstalled state deliberately excluded. No SummaryDialog.exe
  change needed - it renders packageStatesByPhase generically.
- OfficeInstallStateData: CompletedAtUtc + VersionReached stamped on terminal
  persists (duration/version for the row); older state files load fail-soft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:57:46 +02:00
Oliver Kieselbach e750cfe9dd feat(web): gap badge on timeline rows after multi-hour/day time gaps
Timeline rows render time-of-day only, so a large gap between adjacent
events (and the date change it implies) was invisible without expanding
event details. Rows following a gap >= 1h now show an amber chip
(e.g. "+3h" / "+19d"); hovering reveals the full date. Applies to both
the phase timeline and raw mode, using the same display timestamp as
the row (backfilled events keep their event-log time).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:39:36 +02:00
Oliver Kieselbach 3629728d93 feat(time-attribution): PriorEnrollmentResidue quality flag (v2) + WG-signal comment fix
Session f475e697: a device re-enrolled without a wipe (IME log residue from a
previous enrollment) completes its pre-installed apps as instant detections,
pulling the phase anchors ~6.5 min ahead of the real ESP page — the breakdown
was mechanically exact but semantically wrong, and entered fleet stats as clean.

- New duration-critical flag PriorEnrollmentResidue, set on
  historic_ime_replay_detected; excluded from fleet segment stats via the
  existing DurationCriticalFlags mask. registry_app_baseline successes are
  deliberately NOT a trigger (by-design normal for WDP Batch-1) — pinned by test.
- AttributionVersion 1 -> 2 so the maintenance sweep recomputes in-retention
  breakdowns and flags old residue sessions retroactively (rule 8).
- Web: flag label in TimeAttributionLane; docs: quality-flags section.
- Stale comment fix: SaveWhiteGloveSuccessResult is NOT a WhiteGlove indicator
  on Win11 25H2+ (validated 2026-08-18 against user-driven sessions) —
  observability only, consumed nowhere for classification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:39:11 +02:00
Oliver Kieselbach e521fd5cf4 fix(ime-archive): heal provenance.json lost behind the write-once 409 path
Daily-review finding: the MSI upload is write-once (If-None-Match:*), so a
first attempt that died between the MSI and the provenance upload funneled
every retry into the 409 short-circuit — which returned "Archived" without
ever writing provenance.json (permanently lost). The 409 path now checks for
the sidecar and, when missing, rehashes the ARCHIVED bytes and writes it;
heal failures ride the normal retryable ladder.

Also closes the review's test gaps: ImeMsiArchiveQueueWorker + producer
worker-level tests (internal QueueClient test ctors, SessionDeletionWorker
pattern), CorporateIdentifierValidator null/whitespace + OData special-char
+ page-cap-transient tests, registry observer helper tests
(TerminalOutcome/ChangedFields/ClassifyStatusServiceStatus).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:21:26 +02:00
Oliver Kieselbach 0ed1e6f542 Merge branch 'main' of https://github.com/okieselbach/Autopilot-Monitor 2026-08-18 03:21:54 +02:00
dependabot[bot] 522911c5a7 chore(deps-dev): bump tsx (#130)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [tsx](https://github.com/privatenumber/tsx).


Updates `tsx` from 4.23.11 to 4.23.12
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.23.11...v4.23.12)

---
updated-dependencies:
- dependency-name: tsx
  dependency-version: 4.23.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: mcp
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-18 03:20:26 +02:00
dependabot[bot] 0bfe1e327b chore(deps): bump the web group (#131)
Bumps the web group in /src/Web/autopilot-monitor-web with 4 updates: [@xyflow/react](https://github.com/xyflow/xyflow/tree/HEAD/packages/react), [next](https://github.com/vercel/next.js), [@next/bundle-analyzer](https://github.com/vercel/next.js/tree/HEAD/packages/next-bundle-analyzer) and [eslint-config-next](https://github.com/vercel/next.js/tree/HEAD/packages/eslint-config-next).


Updates `@xyflow/react` from 12.11.2 to 12.11.3
- [Release notes](https://github.com/xyflow/xyflow/releases)
- [Changelog](https://github.com/xyflow/xyflow/blob/main/packages/react/CHANGELOG.md)
- [Commits](https://github.com/xyflow/xyflow/commits/@xyflow/react@12.11.3/packages/react)

Updates `next` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/compare/v16.3.0...v16.3.1)

Updates `@next/bundle-analyzer` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v16.3.1/packages/next-bundle-analyzer)

Updates `eslint-config-next` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v16.3.1/packages/eslint-config-next)

---
updated-dependencies:
- dependency-name: "@xyflow/react"
  dependency-version: 12.11.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: next
  dependency-version: 16.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: "@next/bundle-analyzer"
  dependency-version: 16.3.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: eslint-config-next
  dependency-version: 16.3.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-18 03:20:10 +02:00
Oliver Kieselbach 2cf7216be5 Token-failure grace window + registry_app_baseline summary
Live findings from the first two 2.0.1402 sessions (Oliver, 2026-08-18):

1. "Failed to get AAD token" is logged routinely during the device phase
   (no user token exists yet) - both test sessions raised the Warning
   although nothing was wrong. The failure is now held pending: any
   IME-TOKEN-SUCCESS match clears it (staleness-gated so a replayed
   success from a previous enrollment cannot mask a current outage), and
   ime_token_failure fires only when no success follows for the 30-min
   grace window, then deduped per errorCode. Genuine auth outages run
   long enough to hit the window; routine device-phase failures never do.

2. New one-shot registry_app_baseline event (Info, only when non-empty):
   summary of the Win32Apps state that predates the agent run - counts
   by enforcement class + capped 50-app typed payload incl.
   LastUpdatedTimeUtc. On WDP the DPP-policy apps install in Batch 1
   before the bootstrap-script agent exists; their outcomes live only in
   this baseline, closing the pre-agent app gap without log backfill.
   The diff rail stays silent on baseline state (anti-replay unchanged).

Event type dual-registered (Constants + MCP catalog + manifests regen).
Tests: grace-window cycle (resolved / unresolved / re-arm dedup),
baseline summary (counts, one-shot, empty). All suites green
(Agent 2274, Functions 4262, MCP 561, web 762), 0 warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:16:58 +02:00
Oliver Kieselbach 587adc5ddf ci(msi): build from project dir - wix resolves File/@Source against cwd
WIX0103 on the first run: Invoke-BootstrapDownload.ps1 not found because
wix build resolves File/@Source relative to the working directory, not
the wxs location. Matches the local build documented in Package.wxs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:11:12 +02:00
Oliver Kieselbach 84bf2bc2fd commands: bump to 3f98090 (log-name kebab-case in analysis skills) 2026-08-18 03:07:42 +02:00
Oliver Kieselbach dcf1b1c227 ci: Bootstrap-MSI build workflow - blob publish behind the download alias
workflow_dispatch-only, mirroring the agent release contract: validate
the runner script (PS parse + ASCII-only, it runs as SYSTEM on every
enrolling device), wix 5 build, upload versioned + stable blob names to
the agent container (no-cache, no legacy mirror - the MSI channel never
existed there), HEAD-verify the public alias serves the fresh bytes,
Sigstore provenance attestation. MSI version stays source-controlled in
Package.wxs - bumped to 1.0.2 (runner: encoding fix + bootstrap-msi.log
rename). Customer download:
https://download.autopilotmonitor.com/agent/AutopilotMonitor-Bootstrap.msi

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:07:08 +02:00
Oliver Kieselbach 6ebdfa41af refactor(agent): unified kebab-case log file naming + date-less agent.log
The agent log directory mixed three conventions (agent_20260817.log,
bootstrap_agent.log, msi-bootstrap.log) while everything else in the
diagnostics ZIP is already kebab-case. All log files now follow
<component>-<content>.log:

  agent.log               (was agent_YYYYMMDD.log; segments agent-NNN.log)
  bootstrap-script.log    (was bootstrap_agent.log; script v2.4)
  bootstrap-msi.log       (was msi-bootstrap.log)
  crash-<utc>.log         (was crash_<utc>.log)
  gather-rules-debug.log  (was gather_rules_debug.log)
  ime-pattern-matches.log (was ime_pattern_matches.log)

The date is dropped from the main log: the agent only lives for one
enrollment, and date rotation split the chronology at local midnight -
which can fall mid-enrollment while the clock is still on the OOBE
default timezone. The existing 50MB size rotation stays as the growth
bound. Legacy dated files are ignored by the rotation probe; crash
detection still picks up legacy crash_*.log once so an in-place
self-update cannot lose a pending crash classification.

Diag collectors glob *.log and need no change. MCP zip-map and docs
note the legacy names for archives from pre-rename agents. Settings UI
hint paths updated. MSI rebuild + bootstrap script publish (v2.4) +
MCP redeploy required to ship.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:55:06 +02:00
Oliver Kieselbach a2c4949473 commands: bump to b431556 (ime-decompile blob-first + import fixes)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:36:57 +02:00
Oliver Kieselbach 81f4a55d88 Bootstrap MSI 1.0.1: version bump for Intune re-upload + log encoding fix
Intune LOB requires a higher ProductVersion to accept a re-upload. Also
includes the runner logging fix: PS 5.1 *>> redirection writes UTF-16LE
into the otherwise ANSI msi-bootstrap.log - route through Add-Content for
a single encoding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:12:11 +02:00
Oliver Kieselbach 4d3d3c56f4 gitignore: restore ime-files/ - local binaries scratch of /ime-decompile
Still needed: the skill extracts each MSI here (msiexec /a payload for grepping
in binaries); only decompilates go to the ime-decompiles repo and MSIs to the
ime-archive blob container.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:51:26 +02:00
Oliver Kieselbach 41898d745a updated commands 2026-08-18 01:33:44 +02:00
Oliver Kieselbach c05ea4d63b Registry second pillar: IME app state from HKLM with log reconciliation
P2 of the 2026-08-17 IME code audit. IME log wording is an unstable
contract (audit found 9 silently dead patterns, reworded lines, quote
drift, [DO TEL] removal); the registry is IME's internal product
contract - the ESP page and Intune reporting consume it.

ImeRegistryAppStateObserver + ImeRegistryAppStateHost (always-on
observability, no DecisionEngine coupling):
- one recursive RegistryWatcher on HKLM\...\IntuneManagementExtension
  covers Win32Apps (EnforcementStateMessage JSON, ExitCode, Intent),
  EspTrackingWin32Apps and SideCarPolicies\StatusServiceReports;
  2 s debounced snapshot-and-diff + 60 s periodic fallback tick
  (poll-only degraded mode if the watcher fails to arm)
- silent startup baseline: Win32Apps keys survive re-enrollments,
  pre-existing state never replays as fresh events
- registry_app_state (Info, state-change-only, 200/session cap with
  one-shot cap notice)
- app_state_reconciliation (Warning, once per app, immediate upload):
  a changed entry terminal for >= 90 s is compared against the
  log-derived AppPackageState - divergence is the built-in
  log-pattern drift alarm (registry = truth, logs = narrative)

Event types dual-registered (Constants + MCP resource-catalog + shared
manifests regenerated). OKF concept doc agent/registry-app-state-pillar.md
incl. WDP Paket-3 reuse notes. Tests: pure parse/diff/divergence logic +
tick flow via snapshot override (baseline silence, emission, dwell,
fire-once, cap). All suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:25:54 +02:00
Oliver Kieselbach 68eb2da283 IME tracker: tail AppActionProcessor.log; GRS + token-failure signals
P0b/P1 of the 2026-08-17 IME code audit (agent release required):

- Tail AppActionProcessor.log (+ rollover glob) in ImeLogTracker and the
  --run-ime-matching CLI. IME routes every [Win32App][ActionProcessor]
  line there (enforcement decision engine), so IME-NO-ACTION and
  IME-NOT-TARGETED-1/2 could never fire before, and GRS skips were
  invisible. Live session 0fe2c562 carried 16 no-action and 64
  classification lines in that file.

- New patterns (wordings verified in decompiled IME 1.97 AND 1.104):
  IME-GRS-SKIP ("is in GRS. The app will not be enforced." -> Postponed;
  THE canonical silent non-install reason), IME-GRS-CLEARED (early
  subgraph termination, retry next check-in -> Postponed),
  IME-TOKEN-FAILURE ("Failed to get AAD token ... errorCode = N").

- New action imeTokenFailure -> ime_token_failure Warning event (new
  EventType: Constants + MCP resource-catalog + shared manifests
  regenerated). Tracker dedups per distinct errorCode; included in
  the historic-replay guard so stale lines from a previous enrollment
  cannot emit a misleading warning. Complements IME-TOKEN-SUCCESS:
  an auth outage previously left check-ins silently doing nothing.

Tests: adapter emission + unknown-code + tracker dedup; known-actions
catalog extended; all suites green (Functions 4262, Agent 2252, web 762).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:01:02 +02:00
Oliver Kieselbach ab0f27b5ab Download times: event-window fallback for IME 1.104 (DO TEL removed)
IME 1.104 no longer logs the [DO TEL] line whose JSON carried the
DO-reported download duration - the Download Progress card lost all
times (per-app and total).

Durations now fall back to the observed event window (first download
event -> first completion-marking event) whenever the DO-reported value
is absent, and completed items show "in <duration>" inline. Works
retroactively for already-stored sessions, no agent release required;
the DO-reported value still wins when present (older IME versions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 00:40:53 +02:00
Oliver Kieselbach 32d8151185 updated .gitignore 2026-08-18 00:28:29 +02:00
Oliver Kieselbach f3749ccc16 updated commands 2026-08-18 00:24:17 +02:00
Oliver Kieselbach ca26cb4f7c backend: auto-archive IME installer on first sighting of a new version
When RecordImeVersionAsync reports a genuinely new IME version (~monthly),
EventIngestProcessor now also enqueues an ime-msi-archive job. The worker
(QueuePollingWorker, heartbeat, pause on ImeMsiArchivingEnabled=false)
downloads the installer and stores it write-once (If-None-Match:*) in the
permanent ime-archive container ({version}/IntuneWindowsAgent.msi +
provenance.json), then merges MsiArchiveStatus/BlobPath/Sha256/Bytes/SourceUrl
onto the ImeVersionHistory row - so every fleet-observed IME build stays
available for decompilation/diffing after Microsoft's versionless CDN moves on.

The download URL comes from agent telemetry (msiDownloadUrl enrichment) and is
treated as untrusted: https-only *.manage.microsoft.com allowlist with exact
filename match, canonical versionless fallback, no redirects, size cap
(MaxImeMsiDownloadSizeMB, default 250, preflight + mid-stream) enforced by a
SHA-256-hashing pass-through stream. Archiver never throws; the worker retries
only transient statuses (download/timeout/error), permanent rejections
complete with their Failed:* status persisted.

Admin settings ImeMsiArchivingEnabled (default true) + MaxImeMsiDownloadSizeMB
via the full 4-file web chain; archive fields on the version-history API are
GA-only (non-GA projection unchanged, redaction tests extended). Shared
manifests regenerated. Tests: archiver status branches via recording subclass
(allowlist, caps, 409 idempotency, cancellation propagation), config defaults.
Backend suite 4262 green, web tsc + vitest 762 green, solution 0 warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 00:20:53 +02:00
Oliver Kieselbach 8ceab7a064 updated commands 2026-08-17 23:51:56 +02:00
Oliver Kieselbach ae85dc325c gitignore: exclude local ime-files/ working directory
Machine-local reference files, kept out of version control.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 23:51:08 +02:00
109 changed files with 5446 additions and 250 deletions
+139
View File
@@ -0,0 +1,139 @@
name: Build & Publish Bootstrap MSI
# Thin MDM LOB delivery vehicle for WDP (Autopilot v2) — see
# src/Agent/AutopilotMonitor.BootstrapMsi/Package.wxs. The MSI carries no bootstrap
# logic (only the runner script that downloads the CURRENT server-hosted
# Install-AutopilotMonitor.ps1), so it is rebuilt rarely: only when the runner script
# or the WiX package changes. Version lives in Package.wxs — bump it there before
# dispatching so Intune LOB update detection sees a new ProductVersion.
on:
workflow_dispatch:
inputs:
upload_blob:
description: 'Upload to blob (customer-facing via download.autopilotmonitor.com/agent/)'
required: true
type: boolean
default: true
permissions:
contents: read
id-token: write # keyless Sigstore signing (artifact attestation)
attestations: write # store the provenance attestation
env:
MSI_PROJECT_DIR: src/Agent/AutopilotMonitor.BootstrapMsi
RUNNER_SCRIPT: src/Agent/AutopilotMonitor.BootstrapMsi/Invoke-BootstrapDownload.ps1
STABLE_MSI: AutopilotMonitor-Bootstrap.msi
jobs:
build:
runs-on: windows-latest
name: Build Bootstrap MSI
steps:
- name: Checkout repository
uses: actions/checkout@v7
# The runner executes as SYSTEM on every enrolling device — a parse error or
# non-ASCII mojibake (PS 5.1 codepage roulette) must fail the build, not the fleet.
- name: Validate runner script (parse + ASCII-only)
shell: pwsh
run: |
$errs = $null
[System.Management.Automation.Language.Parser]::ParseFile((Resolve-Path $env:RUNNER_SCRIPT), [ref]$null, [ref]$errs) | Out-Null
if ($errs) { $errs | ForEach-Object { Write-Error $_.ToString() }; exit 1 }
if ((Get-Content $env:RUNNER_SCRIPT -Raw) -match '[^\x00-\x7F]') {
Write-Error "Runner script contains non-ASCII characters"
exit 1
}
Write-Host "Runner script parse + ASCII check passed"
- name: Read MSI version from Package.wxs
id: version
shell: pwsh
run: |
$wxs = Get-Content "$env:MSI_PROJECT_DIR/Package.wxs" -Raw
if ($wxs -notmatch 'Version="(\d+\.\d+\.\d+)"') {
Write-Error "Could not parse Version from Package.wxs"
exit 1
}
$version = $Matches[1]
Write-Host "MSI version (from Package.wxs): $version"
"version=$version" >> $env:GITHUB_OUTPUT
- name: Install WiX toolset
run: dotnet tool install --global wix --version "5.*"
- name: Build MSI
shell: pwsh
run: |
# wix resolves File/@Source relative to the working directory, not the wxs —
# build from inside the project dir (same as the local build documented in the wxs).
$out = Join-Path (Get-Location) $env:STABLE_MSI
Push-Location $env:MSI_PROJECT_DIR
wix build Package.wxs -arch x64 -o $out
Pop-Location
if (-not (Test-Path $env:STABLE_MSI)) { Write-Error "wix build produced no MSI"; exit 1 }
$sha256 = (Get-FileHash $env:STABLE_MSI -Algorithm SHA256).Hash.ToLowerInvariant()
Write-Host "Built $env:STABLE_MSI ($([math]::Round((Get-Item $env:STABLE_MSI).Length / 1KB)) KB)"
Write-Host "SHA-256: $sha256"
- name: Upload to Azure Blob Storage
if: inputs.upload_blob
shell: pwsh
run: |
$containerUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent"
$sasToken = "${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}".TrimStart('?')
$version = "${{ steps.version.outputs.version }}"
# Same Send-Blob contract as build-agent.yml: no-cache on every blob because the
# stable name rotates in place behind the download alias. No legacy mirror — the
# MSI channel never existed on the legacy account.
function Send-Blob {
param([string]$Local, [string]$BlobName)
$bytes = [System.IO.File]::ReadAllBytes($Local)
$headers = @{
"x-ms-blob-type" = "BlockBlob"
"Content-Type" = "application/octet-stream"
"x-ms-blob-cache-control" = "no-cache"
}
$md5 = [System.Security.Cryptography.MD5]::Create()
$headers["Content-MD5"] = [Convert]::ToBase64String($md5.ComputeHash($bytes))
$md5.Dispose()
Write-Host "Uploading $BlobName ($([math]::Round($bytes.Length / 1KB)) KB)..."
Invoke-RestMethod -Uri "$containerUrl/$BlobName`?$sasToken" -Method Put -Headers $headers -Body $bytes
Write-Host " uploaded $BlobName"
}
# Versioned copy for provenance/rollback + stable customer-facing name.
Send-Blob -Local $env:STABLE_MSI -BlobName "AutopilotMonitor-Bootstrap-v$version.msi"
Send-Blob -Local $env:STABLE_MSI -BlobName $env:STABLE_MSI
# E2E: the Front Door route /agent/* must serve the fresh bytes on the public alias.
- name: Verify download alias serves the MSI
if: inputs.upload_blob
shell: pwsh
run: |
$localSize = (Get-Item $env:STABLE_MSI).Length
$resp = Invoke-WebRequest -Uri "https://download.autopilotmonitor.com/agent/$env:STABLE_MSI" -Method Head -UseBasicParsing
if ($resp.StatusCode -ne 200) { Write-Error "Alias HEAD returned $($resp.StatusCode)"; exit 1 }
$remoteSize = [long]$resp.Headers['Content-Length'][0]
if ($remoteSize -ne $localSize) {
Write-Error "Size mismatch: alias serves $remoteSize bytes, local MSI is $localSize bytes"
exit 1
}
Write-Host "Alias verified: https://download.autopilotmonitor.com/agent/$env:STABLE_MSI ($remoteSize bytes)"
# Sigstore keyless provenance, same as the agent ZIP. Verify:
# gh attestation verify AutopilotMonitor-Bootstrap.msi --repo okieselbach/AutopilotMonitor
- name: Attest build provenance (MSI)
uses: actions/attest-build-provenance@v4
with:
subject-path: ${{ env.STABLE_MSI }}
- name: Upload build artifact
uses: actions/upload-artifact@v7
with:
name: AutopilotMonitor-Bootstrap-v${{ steps.version.outputs.version }}
path: ${{ env.STABLE_MSI }}
+6
View File
@@ -158,3 +158,9 @@ docs/~$Introduction-AutopilotMonitor.pptx
# Web build stamp: generated by scripts/generate-version.js on every build (buildUtc
# changes each run) and served at /version.json from the static export.
src/Web/autopilot-monitor-web/public/version.json
# Local IME binaries scratch used by the /ime-decompile skill (MSI + msiexec-
# extracted payload per version, for grepping in binaries). Decompilates live in
# the private ime-decompiles repo, MSIs in the ime-archive blob - this stays local.
ime-files/
+1 -1
View File
@@ -44,7 +44,7 @@ tenant-config report.
`RemoteConfigMerger` projects the bool onto a runtime path (same pattern as
`EnableImeMatchLog`): `true` → `Constants.GatherRuleDebugLogPath` =
`%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`; `false` → null, unless a
`%ProgramData%\AutopilotMonitor\Logs\gather-rules-debug.log`; `false` → null, unless a
`--gather-debug-log <path>` CLI override supplied a non-default path. The field is not
security-sensitive and deliberately **survives the offline config cache**.
+1 -1
View File
@@ -53,7 +53,7 @@ every line; the spool fsyncs only items flagged `RequiresImmediateFlush`.
| Event sequence | `State\event-sequence.json` | `{ LastAssignedSequence }` | atomic | none; survives WhiteGlove Part-2 resume |
| Spool | `Spool\spool.jsonl` | JSONL (`TelemetryItem`/line) | only `RequiresImmediateFlush` items | none — never truncated, cursor skips uploaded items |
| Upload cursor | `Spool\upload-cursor.json` | `{ LastUploadedItemId }` | atomic | idempotent, never regresses |
| Agent trace log | `Logs\agent_YYYYMMDD[_NNN].log` | text `[utc] [LEVEL] msg` | no | 50 MB → suffix rotation; consumed by `/diag-agent-log` |
| Agent trace log | `Logs\agent.log` (segments `agent-NNN.log`) | text `[utc] [LEVEL] msg` | no | 50 MB → suffix rotation; consumed by `/diag-agent-log` |
| Crash dump + record | `Crashes\<stem>.dmp` + `.json` | MiniDump + JSON | n/a (process dying) | 5 files / 7 days |
| Startup-event gate | `State\startup-event-state.json` | JSON dict | no (atomic) | cross-reboot dedup of one-shot probes |
| Bandwidth state | `State\bandwidth-state.json` | JSON samples/counters | no (atomic) | reboot-survivor for the DO bandwidth estimate |
+1 -1
View File
@@ -146,7 +146,7 @@ snapshots) are executed by the `EffectRunner` — the reducer itself is pure. De
| `\Agent\` | deployed binaries |
| `\State\` | decision persistence (SignalLog, Journal, Snapshot), `enrollment-complete.marker`, `final-status.json`, self-update state, `.quarantine/`, `.part1-<ts>/` |
| `\Spool\` | telemetry spool (offline queue) |
| `\Logs\` | agent trace log, `crash_*.log`, `ime_pattern_matches.log` |
| `\Logs\` | agent trace log (`agent.log`), `crash-*.log`, `ime-pattern-matches.log` |
| `\Config\` | `remote-config.json` cache |
| `\Agent-Update\`, `\Updates\` | self-update staging/download |
+89
View File
@@ -0,0 +1,89 @@
---
type: concept
title: Registry App-State Second Pillar
description: Why the agent reads IME's authoritative per-app state straight from the registry (Win32Apps / EspTrackingWin32Apps / StatusServiceReports), how the snapshot-and-diff observer works, and how registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
resource: agent
tags: [ime, registry, win32apps, observability, reconciliation, pattern-drift, wdp]
timestamp: 2026-08-18
---
# Schema
## Why a second pillar
The 2026-08-17 audit of the decompiled IME (1.97.107 / 1.104.102) proved that IME log
wording is an unstable contract: Microsoft removed the old V1/V2 processing pipeline
(killing 9 patterns silently), reworded the post-remediation line, changed the
report-delta quoting between 1.97 and 1.104, and deleted `[DO TEL]` entirely. The
registry, by contrast, is IME's *internal product contract* — the ESP page and Intune
reporting consume it — and cannot drift casually.
Architecture decision (Oliver, 2026-08-17): **hybrid**. Logs stay the narrative
(downloads, retries, exit codes, scripts — the timeline). The registry is the stable
truth for terminal state. Divergence between the two IS the alarm that log patterns
drifted — turning silent data loss into a Warning event.
## Observed surfaces
All under `HKLM\SOFTWARE\Microsoft\IntuneManagementExtension` (64-bit view), verified
against decompiled IME source:
| Surface | Content |
|---|---|
| `Win32Apps\<userGuid>\<appGuid>_<rev>` | `EnforcementStateMessage` JSON (EnforcementState + ErrorCode), `ExitCode`, `Intent`. Device context = `Guid.Empty` user. Non-GUID subkeys (GRS, OperationalState, Reporting, ProvisioningProgress) are skipped. |
| `EspTrackingWin32Apps\<userGuid>\<appGuid>_<rev>` | Which apps IME registered for ESP tracking, `EspTrackingWin32AppPhase`. |
| `SideCarPolicies\StatusServiceReports\<userGuid>\<appId>` | The `AppInstallStatusReport` the ESP page renders (`Status`: 1000=Installed, 1001=Installing, 3000=Failed, 2000er=NotApplicable). |
## Mechanism
`ImeRegistryAppStateObserver` + `ImeRegistryAppStateHost` (always-on, no config gate —
precedent `EspPolicyProviderStallHost`):
- **One recursive `RegistryWatcher`** on the IME root covers all three surfaces;
changes are debounced (2 s trailing edge) into a tick. A 60-s periodic tick is both
the reconciliation-settle evaluator and the poll-only fallback when the watcher
fails to arm (`collector_degraded`, once).
- **Snapshot-and-diff**: `RegNotifyChangeKeyValue` gives coalesced key-scope edges
only, so every tick re-reads the surfaces and diffs against the previous snapshot.
- **Silent baseline**: the state present at agent start is captured without events.
Win32Apps keys survive re-enrollments — replaying them would be the registry twin
of the historic-IME-replay bug. Only apps that *change during the session* are
reported or judged.
- **`registry_app_state`** (Info): emitted per real field change
(enforcementState/errorCode/exitCode/statusServiceStatus/espTracked/espPhase),
capped at 200/session (cap announced once as Warning).
- **`app_state_reconciliation`** (Warning, once per app, immediate upload): after a
changed entry has been terminal (EnforcementState 1000er=success / 5000er=error,
or StatusService 1000er/3000) for ≥ 90 s, its outcome is compared against the
log-derived `AppPackageState` from `ImeLogHost.AllKnownPackageStates`. Divergence
reasons: `registry_error_log_installed`, `registry_success_log_error`,
`app_unknown_to_log_tracking` (only when the tracker is actively tracking apps —
an idle tracker is not judgeable).
- **Pure observability**: nothing feeds the DecisionEngine (same rule as
oobe-state observability).
## WDP reuse (Paket 3)
The same host archetype (RegistryWatcher subtree + debounce + snapshot-diff) is the
intended vehicle for WDP live progress: watch
`HKLM\SOFTWARE\Microsoft\Provisioning\AutopilotSettings\DevicePreparation\BootstrapperAgent`
(`ProgressUpdates` rows, `ResumeContext.ProvisioningResult` terminal marker) with the
RealmJoinWatcher appearance-watcher pattern for not-yet-existing keys, plus an
Event-142 channel watcher (`Microsoft-Autopilot-BootstrapperAgent/BootstrapperAgentServiceLogProvider`,
channel handling precedent in `DiagnosticsPackageService`). Win32Apps keys are written
on WDP devices too — the pillar works on both rails unchanged.
# Examples
Timeline evidence of a drifted pattern set: apps install fine (registry says
EnforcementState 1000) but the timeline shows no `app_install_completed` — each app
raises one `app_state_reconciliation` with `reason=app_unknown_to_log_tracking`.
Operator response: run `/ime-pattern-validate` against the session, fix patterns,
backend-deploy (no agent release needed for pattern fixes).
# Citations
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ImeRegistryAppStateObserver.cs`
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/ImeRegistryAppStateHost.cs`
- `tasks/ime-v1-code-audit-2026-08-17.md` (decompiled-source audit; registry surface catalog)
- Decompile archive: `okieselbach/ime-decompiles` (private), `C:\Code\GitHubRepos\ime-decompiles\<version>\decompiled`
+108
View File
@@ -0,0 +1,108 @@
---
type: concept
title: WDP Scenario Gates & Completion Backstop
description: How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation (WDP) sessions and how a WDP session resolves without any ESP signal.
resource: agent
tags: [wdp, device-preparation, decision-engine, collectors, completion]
timestamp: 2026-08-18
---
# Schema
Windows Device Preparation (Autopilot v2, "WDP") has **no deployment profile and no ESP**.
Before the afee7ae0 audit (2026-08-18) the agent ran the full Classic apparatus anyway:
misleading `autopilot_profile_missing` warnings, a ZTD event-log query + HTTP probe,
`esp_config_detected` "unknown/unknown" noise, an Autopilot-channel noise flood
(EventIDs 100/417/418/1005, 900+ occurrences per session), a subtree registry watcher
woken by every WDP progress write, and — worst — no completion path when Hello never
resolved (park until the 360-min watchdog).
## Detection: deterministic vs. fallback
`EnrollmentRegistryDetector` classifies v2 through three rules. Only the first is
**deterministic WDP**:
1. `AutopilotSettings\DevicePreparation\BootstrapperAgent` subkey with non-empty
`ExecutionContext` → the WDP policy orchestration tree, present before the agent
starts. Exposed as `EnrollmentRegistryDetector.IsDeterministicDevicePreparation()`
and the `DevicePreparationExecutionContextRule` constant.
2./3. `CloudAssignedDeviceRegistration == 2` / `CloudAssignedEspEnabled == 0` —
profile-derived legacy signals. They imply a REAL Autopilot profile (just ESP-less),
so every gate below deliberately ignores them: **gates require the deterministic
marker and fail toward Classic behavior.**
## Agent-side gates (all keyed to the deterministic marker)
* `DeviceInfoCollector.CollectAutopilotProfile`: no `autopilot_profile` event, no
`autopilot_profile_missing` warning, no ZTD event-log query, no ZTD HTTP probe.
`enrollment_type_detected` still fires. (WDP devices DO carry an
`AutopilotPolicyCache` with `ProfileAvailable=0` / error 807 `ZtdDeviceIsNotRegistered`
— Windows runs the ZTD download regardless — so the probe fired for real before.)
* `CollectEspConfigurationLocked`: `esp_config_detected` is emitted only when at least
one FirstSync value or ESP tracking list exists (generic null-gate, mirrors the
orchestrator bootstrap's signal gate) — fixes "unknown/unknown" for WDP AND plain
Entra joins.
* `DefaultComponentFactory`: `EspPolicyProviderStallHost` (60-s poll of the ESP-only
`EnrollmentStatusTracking` key) is not created on WDP.
* `EspAndHelloTracker.Start`: `ProvisioningStatusTracker` is not started on WDP — its
subtree watcher sits on `AutopilotSettings`, exactly where WDP's BootstrapperAgent
writes progress, so it woke on every write only to read three ESP category values
that never exist there.
* `ModernDeploymentTracker`: `DevicePreparationNoiseEventIds` {100, 417, 418, 1005}
are suppressed entirely on WDP (before `FormatDescription()`); Critical (level 1)
always passes; genuine WDP diagnostics (e.g. 408) keep flowing. The Autopilot error
backfill (807/809/815/908 replay) is skipped — 807 is the EXPECTED state on WDP.
The watcher itself stays on: WDP's provisioning stack logs into the same channels.
* `StallProbeCollector`: skips the ESP category registry scan on WDP and unions the
noise IDs into its harmless set; the channel scans remain (they carry genuine WDP
diagnostics during a stall).
* `EnrollmentTerminationHandler`: the `app_install_starved` terminal sweep is skipped
on WDP — post-desktop installs are the DESIGNED order there (only DPP-policy apps
run during provisioning); per-app data still travels via `app_tracking_summary` and
final-status.json.
## DecisionCore: WDP completion without ESP facts
* **Seeding**: `EnrollmentFactsObserved` carries `enrollmentTypeDeterministic`;
the deterministic marker seeds `DevicePreparation` at **High** confidence (same
registry-determinism standard as the self-deploying OobeConfig bits), the fallback
rules stay Medium. The three explicit downgrade guards from d0ab3eee remain as
defence in depth.
* **Arm D** (`ShouldTransitionToAwaitingHello`): on WDP the DAD-validated real-user
desktop is a legitimate promotion basis — arms A–C are unsatisfiable by construction
(no provisioning categories, no Shell-Core 62407 final exit). With a disabled Hello
policy the fast-path now completes at desktop arrival.
* **Prerequisites**: `completion_waiting` never lists
`account_setup_provisioning_complete` on WDP (unsatisfiable; mirrors the
`skipUserEsp` exemption).
* **Backstop**: `DeadlineNames.DevicePrepCompletion` (30 min, replay-safe base) is
armed by `HandleDesktopArrivedV1` when a WDP session parks desktop-first with Hello
pending. On fire it mirrors the HelloSafety timeout: synthetic
`HelloOutcome=Timeout` + completion through Finalizing. This closes the
"no Hello → park until watchdog" hole; a timer surviving into a terminal stage is
swallowed by the post-terminal dispatch guard.
* **Tripwire**: `DecisionStepProcessor.IsParkedWithoutDeadline` has a WDP branch
(dead-end zone = desktop arrived) so a regression in the backstop becomes visible
as `session_parked_without_deadline`.
# Examples
Session afee7ae0 (the audit trigger): Succeeded in 9:32 min, but its timeline carried
`autopilot_profile` (807), an `autopilot_profile_missing` WARN, `esp_config_detected`
unknown/unknown, 30 `modern_deployment_log` rollup events and 10 `app_install_starved`
entries — all structurally meaningless on WDP. After the gates, the same session shape
emits none of these; completion rode Hello-skipped + desktop as before.
Deliberately NOT changed (open follow-ups, see the WDP audit topic memory): the ESP
phase vocabulary (`esp_phase_changed`/stage names ride the Classic rails on WDP),
subscription to the `BootstrapperAgentServiceLogProvider` event channel, and
`registry_app_state` as the primary WDP app source.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Security/EnrollmentRegistryDetector.cs` — deterministic rule + helper
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/DeviceInfo/DeviceInfoCollector.NetworkAndSecurity.cs` — profile/ESP emission gates
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ModernDeploymentTracker.cs` — `DevicePreparationNoiseEventIds`
* `src/Shared/AutopilotMonitor.DecisionCore/Engine/DecisionEngine.DevicePreparation.cs` — backstop deadline
* `src/Shared/AutopilotMonitor.DecisionCore/State/EnrollmentScenarioProfileUpdater.cs` — High/Medium seeding
* Tests: `DevicePreparationEngineTests`, `ModernDeploymentTrackerDevicePrepTests`, `EnrollmentTerminationHandlerTests.StarvedApps_skipped_on_device_preparation`, `DeviceInfoCollectorEspRefreshTests.NoEspEvidenceAtAll_SuppressesTheEvent`
+16 -1
View File
@@ -58,6 +58,21 @@ timestamp runs backward are dropped (`ClockSkewDropped`).
| Complete, `desktop_arrived` event | `desktop_handoff` |
| Failed | attribution ends — tail stays unattributed |
## Quality flags (AttributionVersion 2)
Duration-critical flags (`ClockSkewDropped`, `PartialObservation`,
`WhiteGloveAnchorsIncomplete`, `PriorEnrollmentResidue`) exclude a breakdown from fleet
segment statistics with a disclosed count; the blocking-set flags (`BlockingSetUnknown`,
`BlockingSetTruncated`) only limit per-app evidence and never gate fleet stats.
`PriorEnrollmentResidue` (v2) fires on `historic_ime_replay_detected`: the IME log on disk
predates the enrollment, so the device was re-enrolled without a wipe — pre-installed apps
complete as instant detections and pull the phase anchors ahead of the real ESP page
(session f475e697: AccountSetup anchor 6.5 min early, 457 s misattributed to
`identity_hello`). `registry_app_baseline` successes are deliberately NOT a trigger — they
are by-design normal for Windows Device Preparation (DPP Batch-1 apps install before the
agent exists) and would starve that class's aggregates.
## ESP-blocking apps (positive evidence only)
Per-app intervals use EVENT timestamps (first started/download event → LAST terminal
@@ -117,7 +132,7 @@ gap before the first part-2 phase declaration is unattributed, not guessed.
# Citations
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — calculator + semantics (AttributionVersion 1).
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — calculator + semantics (AttributionVersion 2).
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.TimeAttribution.cs` — persistence.
* `src/Backend/AutopilotMonitor.Functions/Services/MaintenanceService.Aggregation.cs` — sweep + aggregation.
* `src/Backend/AutopilotMonitor.Functions.Tests/TimeAttributionCalculatorTests.cs` — golden fixtures pinning the invariant.
+2
View File
@@ -25,6 +25,8 @@ https://docs.autopilotmonitor.com (separate repository).
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
* [Secure Boot CA-2023 Detection](agent/secureboot-ca2023-detection.md) - Why ANALYZE-SEC-001 v3 reads the UEFI db/KEK variables directly instead of trusting the SecureBoot\Servicing registry, and how the one-sided uefiCa2023FirmwareConfirmed marker plus a not_exists precondition give absence-tolerant suppression across old and new agents.
* [Continue-Anyway Observation Mode](agent/continue-anyway-observation.md) - EnableEspContinueAnywayObservation (ConfigVersion 37, operator-set): Device-phase ESP terminal failures on Continue-Anyway profiles get a bounded 60-min observation instead of an immediate hard fail; the DAD-validated real-user desktop + Hello gate detect the true end, the session completes Succeeded + EspSoftFailure (amber "with issues" badge), expiry un-defangs to the original esp_terminal_failure.
* [Registry App-State Second Pillar](agent/registry-app-state-pillar.md) - IME's authoritative per-app state read straight from the registry (Win32Apps EnforcementStateMessage, EspTrackingWin32Apps, StatusServiceReports) via a snapshot-and-diff watcher; registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
* [WDP Scenario Gates & Completion Backstop](agent/wdp-scenario-gates.md) - How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation sessions (deterministic ExecutionContext marker, fail-toward-Classic), which noise it suppresses (ZTD probe, esp_config unknown, ModernDeployment 100/417/418/1005, starved sweep), and the WDP completion model: High-confidence seed, desktop-based arm D, and the 30-min device_prep_completion backstop.
* [ESP Policy-Provider Stall Detection](agent/esp-policy-provider-stall.md) - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.
# Rules
+8
View File
@@ -1,5 +1,13 @@
# Log
## 2026-08-18
* **New**: `agent/wdp-scenario-gates.md` — WDP scenario gates (afee7ae0 audit): deterministic ExecutionContext marker as the only gate key (fail-toward-Classic), suppressed Autopilot/ESP-only emissions and hosts (autopilot_profile*/ZTD probe, esp_config unknown null-gate, EspPolicyProviderStallHost, ProvisioningStatusTracker, ModernDeployment noise IDs 100/417/418/1005 + error-backfill skip, starved sweep), and the WDP completion model (High-confidence v2 seed via `enrollmentTypeDeterministic`, desktop-based arm D, 30-min `device_prep_completion` backstop, parked-tripwire WDP branch).
* **Update**: `backend/time-attribution.md` — AttributionVersion 1 → 2: new duration-critical quality flag `PriorEnrollmentResidue` (fires on `historic_ime_replay_detected`; a re-enrollment without wipe pulls the phase anchors ahead of the real ESP page, so the segments are excluded from fleet stats; `registry_app_baseline` successes deliberately not a trigger — by-design normal for WDP). Added the quality-flags section.
* **Update**: `agent/logs-and-persistence.md`, `agent/overview.md`, `agent/gather-rule-debug-log.md` — log files renamed to unified kebab-case: `agent.log` (date rotation dropped; size segments `agent-NNN.log`), `bootstrap-script.log` (was `bootstrap_agent.log`), `bootstrap-msi.log` (was `msi-bootstrap.log`), `crash-*.log`, `gather-rules-debug.log`, `ime-pattern-matches.log`. Old names still appear in archives from pre-rename agents.
* **New**: `agent/registry-app-state-pillar.md` — the registry second pillar for IME app state (audit follow-up): snapshot-and-diff watcher on `HKLM\...\IntuneManagementExtension` (Win32Apps / EspTrackingWin32Apps / StatusServiceReports), `registry_app_state` + `app_state_reconciliation` events, silent startup baseline, WDP reuse notes.
## 2026-08-17
* **Update**: `mcp/tool-telemetry.md` — the `tool_call` stderr line now carries `errorMessage` (capped at 300 chars; thrown message or the leading `toolError` text, absent on success), so error drilldowns name the failure class instead of guessing from args. Same change: `get_rule_stats` omits the per-rule daily `trend` rows by default (they drove 50% of calls over the response cap at platform scope); `includeTrends=true` opts back in, the response carries a `trendsNote` marker when stripped.
+24
View File
@@ -358,6 +358,22 @@
"description": "Marks the current app as failed when its exit code maps to Failed via the admin-defined return-code table ('[Win32App] lpExitCode is defined as <Failed|Success|SoftReboot|HardReboot|Retry>', Win32AppInstaller.cs). This is the NORMAL mapped-exit-code outcome; the previously matched 'Setting enforcementState as ... Error' line only fires on the rare no-mapping fallback path. The numeric exit code is captured separately by IME-EXITCODE, which precedes this line.",
"enabled": true
},
{
"patternId": "IME-GRS-CLEARED",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ExecutionActionHandler\\] Encountered early subgraph termination\\. Clearing GRS for app with id: {GUID} so it can be retried during the next check-in\\.",
"action": "updateStatePostponed",
"description": "IME aborted the app's subgraph before enforcement finished and cleared its GRS entry so the app retries on the next check-in (ExecutionActionHandler, AppWorkload.log). Maps to Postponed: the install did not complete this pass but is scheduled to run again - previously this looked like an unexplained gap. Wording verified in IME 1.97 and 1.104.",
"enabled": true
},
{
"patternId": "IME-GRS-SKIP",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ActionProcessor\\] App with id: {GUID} to (?:install|uninstall) is in GRS\\. The app will not be enforced\\.",
"action": "updateStatePostponed",
"description": "GRS (Global Retry Schedule) skip: the app failed recently and IME will not re-enforce it this check-in ('App with id: <guid> to install is in GRS. The app will not be enforced.', EnforcementActionProcessor / MixedEnforcementActionProcessor). THE canonical reason an app silently does nothing for up to 24h - previously invisible. Lives in AppActionProcessor.log, which the agent tails since the 2026-08-18 release; wording verified in IME 1.97 and 1.104.",
"enabled": true
},
{
"patternId": "IME-HRESULT",
"category": "currentPhase",
@@ -608,6 +624,14 @@
"description": "Detects when V3Processor starts a new subgraph, cancelling any stuck app and setting the new app as current.",
"enabled": true
},
{
"patternId": "IME-TOKEN-FAILURE",
"category": "always",
"pattern": "Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"enabled": true
},
{
"patternId": "IME-TOKEN-SUCCESS",
"category": "always",
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-GRS-CLEARED",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ExecutionActionHandler\\] Encountered early subgraph termination\\. Clearing GRS for app with id: {GUID} so it can be retried during the next check-in\\.",
"action": "updateStatePostponed",
"description": "IME aborted the app's subgraph before enforcement finished and cleared its GRS entry so the app retries on the next check-in (ExecutionActionHandler, AppWorkload.log). Maps to Postponed: the install did not complete this pass but is scheduled to run again - previously this looked like an unexplained gap. Wording verified in IME 1.97 and 1.104.",
"enabled": true
}
+9
View File
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-GRS-SKIP",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ActionProcessor\\] App with id: {GUID} to (?:install|uninstall) is in GRS\\. The app will not be enforced\\.",
"action": "updateStatePostponed",
"description": "GRS (Global Retry Schedule) skip: the app failed recently and IME will not re-enforce it this check-in ('App with id: <guid> to install is in GRS. The app will not be enforced.', EnforcementActionProcessor / MixedEnforcementActionProcessor). THE canonical reason an app silently does nothing for up to 24h - previously invisible. Lives in AppActionProcessor.log, which the agent tails since the 2026-08-18 release; wording verified in IME 1.97 and 1.104.",
"enabled": true
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-TOKEN-FAILURE",
"category": "always",
"pattern": "Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"enabled": true
}
+1
View File
@@ -61,6 +61,7 @@
"cancelStuckAndSetCurrent",
"imeSessionChange",
"imeImpersonation",
"imeTokenFailure",
"enrollmentCompleted",
"updateDoTelemetry",
"scriptStarted",
@@ -95,6 +95,8 @@
Organizations). SKIP-safe when the API is absent (< 1809).
Removed legacy Content-MD5 fallback; SHA-256 manifest verification is
now mandatory (no hash -> bootstrap aborts, no unverified install).
2026-08-18 v2.4 Renamed log file bootstrap_agent.log -> bootstrap-script.log
(unified kebab-case log naming across the agent log directory).
2026-05-09 v2.0 Generic bootstrap: agent owns its own defaults (e.g. 600 s
TenantId-wait), so the script only calls `--install` plain.
Hardened post-install with a 10 s runtime-process verify.
@@ -126,13 +128,13 @@ param(
)
# Script version (bump on meaningful changes; see .CHANGELOG above)
$ScriptVersion = "2.3"
$ScriptVersion = "2.4"
# Configuration - Everything in ProgramData for easy cleanup
$AgentBasePath = "$env:ProgramData\AutopilotMonitor"
$AgentBinPath = "$AgentBasePath\Agent"
$AgentLogPath = "$AgentBasePath\Logs"
$LogFile = "$AgentLogPath\bootstrap_agent.log"
$LogFile = "$AgentLogPath\bootstrap-script.log"
# Write-Host (NOT Write-Output): log calls happen inside functions that return
# decision objects; Write-Output would leak every log line into those return values.
@@ -61,6 +61,41 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.DeviceInfo
return Assert.IsAssignableFrom<IReadOnlyList<string>>(data[key]);
}
[Fact]
public void NoEspEvidenceAtAll_SuppressesTheEvent()
{
// afee7ae0 (WDP / plain Entra join): FirstSync has no values and the ESP tracking
// key does not exist — an event saying only "SkipUser=unknown, SkipDevice=unknown"
// is noise and must not be emitted (mirrors the orchestrator bootstrap's null-gate).
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
var ingress = new FakeSignalIngressSink();
var clock = new VirtualClock(T0);
var collector = new DeviceInfoCollector(
"session-nullgate", "tenant-1",
new InformationalEventPost(ingress, clock), logger,
signalIngress: null, clock: clock,
startupGate: new StartupEventGate(tmp.Path, logger));
using var skip = new EspSkipConfigurationProbe.ScopedFullOverride(
_ => new EspFirstSyncSnapshot(
skipUser: null, skipDevice: null,
blockInStatusPage: null, syncFailureTimeoutMinutes: null));
using (new EspTrackingInfoProbe.ScopedOverride(_ => default))
{
collector.CollectEspConfiguration();
}
Assert.Empty(EspConfigEvents(ingress));
// A later refresh where real evidence appeared must still emit normally.
using (new EspTrackingInfoProbe.ScopedOverride(_ => Snapshot(new[] { DeviceApp1 }, Array.Empty<string>())))
{
collector.RefreshEspConfiguration(DeviceInfoHost.EspConfigTriggerAppsDevice);
}
Assert.Single(EspConfigEvents(ingress));
}
[Fact]
public void Refresh_reemits_when_lists_grew_and_stays_silent_when_unchanged()
{
@@ -15,7 +15,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
/// </summary>
public sealed class AgentLoggerTests
{
private static string TodayBaseName => $"agent_{DateTime.Now:yyyyMMdd}";
private const string BaseName = "agent";
[Fact]
public void Log_writes_to_unsuffixed_base_file_when_dir_is_empty()
@@ -25,7 +25,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
logger.Info("hello");
var basePath = Path.Combine(tmp.Path, TodayBaseName + ".log");
var basePath = Path.Combine(tmp.Path, BaseName + ".log");
Assert.True(File.Exists(basePath), $"expected base file at {basePath}");
Assert.Contains("hello", File.ReadAllText(basePath));
}
@@ -39,17 +39,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
// Each line ~80 bytes after timestamp/severity envelope; 30 lines clears 1 KB
// even at the smallest payload. The first lines fill the base file; once the
// FileInfo.Length read sees >= 1024 the next call rotates to _002.
// FileInfo.Length read sees >= 1024 the next call rotates to -002.
for (int i = 0; i < 60; i++)
{
logger.Info($"line-{i:D4}-padded-with-some-extra-context-to-cross-1KB-quickly");
}
var files = Directory.GetFiles(tmp.Path, TodayBaseName + "*.log").OrderBy(s => s).ToArray();
var files = Directory.GetFiles(tmp.Path, BaseName + "*.log").OrderBy(s => s).ToArray();
Assert.True(files.Length >= 2, $"expected at least 2 segments, got: {string.Join(",", files.Select(Path.GetFileName))}");
// The unsuffixed file is the first segment; rotated segments use _002 onward.
var rotated = files.FirstOrDefault(p => Path.GetFileNameWithoutExtension(p).EndsWith("_002"));
// The unsuffixed file is the first segment; rotated segments use -002 onward.
var rotated = files.FirstOrDefault(p => Path.GetFileNameWithoutExtension(p).EndsWith("-002"));
Assert.NotNull(rotated);
}
@@ -64,8 +64,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
logger.Info($"pad-{i:D4}-content-content-content-content-content-content");
}
var rotatedPath = Path.Combine(tmp.Path, TodayBaseName + "_002.log");
Assert.True(File.Exists(rotatedPath), "expected agent_YYYYMMDD_002.log to exist");
var rotatedPath = Path.Combine(tmp.Path, BaseName + "-002.log");
Assert.True(File.Exists(rotatedPath), "expected agent-002.log to exist");
var first = File.ReadAllLines(rotatedPath).FirstOrDefault();
Assert.NotNull(first);
Assert.Contains("AgentLogger: rotated", first);
@@ -75,23 +75,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
public void Log_continues_existing_rotation_on_restart_without_overwriting()
{
using var tmp = new TempDirectory();
// Pre-seed: looks like an earlier process already rotated up to _003.
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + ".log"), "older-base\n");
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + "_002.log"), "older-002\n");
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + "_003.log"), "older-003\n");
// Pre-seed: looks like an earlier process already rotated up to -003.
File.WriteAllText(Path.Combine(tmp.Path, BaseName + ".log"), "older-base\n");
File.WriteAllText(Path.Combine(tmp.Path, BaseName + "-002.log"), "older-002\n");
File.WriteAllText(Path.Combine(tmp.Path, BaseName + "-003.log"), "older-003\n");
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info, maxFileSizeBytes: 1024L * 1024);
logger.Info("after-restart");
// New entries must land in the highest existing segment (_003), not in a fresh
// New entries must land in the highest existing segment (-003), not in a fresh
// base file. Older content in _003 must be preserved.
var resumed = File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + "_003.log"));
var resumed = File.ReadAllText(Path.Combine(tmp.Path, BaseName + "-003.log"));
Assert.Contains("older-003", resumed);
Assert.Contains("after-restart", resumed);
// Older segments are left alone.
Assert.Equal("older-base\n", File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + ".log")));
Assert.Equal("older-002\n", File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + "_002.log")));
Assert.Equal("older-base\n", File.ReadAllText(Path.Combine(tmp.Path, BaseName + ".log")));
Assert.Equal("older-002\n", File.ReadAllText(Path.Combine(tmp.Path, BaseName + "-002.log")));
}
[Fact]
@@ -105,7 +105,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
// as plain ASCII, while genuine non-ASCII payload stays untouched.
logger.Info("check failed \u2014 range 1\u20132 \u2192 done\u2026 \u2018a\u2019 \u201Cb\u201D nb\u00A0sp f\u00FCr M\u00FCller");
var text = File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + ".log"));
var text = File.ReadAllText(Path.Combine(tmp.Path, BaseName + ".log"));
Assert.Contains("check failed - range 1-2 -> done... 'a' \"b\" nb sp f\u00FCr M\u00FCller", text);
}
@@ -115,8 +115,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info, maxFileSizeBytes: 1024L * 1024);
// Hold an exclusive lock on today's file to force AppendAllText to throw.
var lockedPath = Path.Combine(tmp.Path, TodayBaseName + ".log");
// Hold an exclusive lock on the log file to force AppendAllText to throw.
var lockedPath = Path.Combine(tmp.Path, BaseName + ".log");
File.WriteAllText(lockedPath, ""); // ensure exists
using var holder = new FileStream(lockedPath, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
@@ -25,7 +25,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
public void Dispose() => _tmp.Dispose();
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather_rules_debug.log");
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather-rules-debug.log");
[Fact]
public void Write_produces_pipe_delimited_line_and_creates_directory_lazily()
@@ -73,7 +73,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
for (int i = 0; i < 20; i++)
log.Write("RULE-1", GatherRuleDebugLog.StageExec, new string('x', 100));
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather_rules_debug.old.log");
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather-rules-debug.old.log");
Assert.True(File.Exists(oldPath), "expected one .old rotation generation");
Assert.True(File.Exists(LogPath));
Assert.Contains("-- rotated", File.ReadAllText(LogPath) + File.ReadAllText(oldPath));
@@ -28,7 +28,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
public GatherRuleExecutorDebugLogTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_debugLogPath = Path.Combine(_tmp.Path, "gather-rules-debug.log");
_executor = new GatherRuleExecutor(
"sess", "tenant", _ => { }, logger,
debugLogPath: _debugLogPath)
@@ -32,7 +32,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
Directory.CreateDirectory(_outsideUsers);
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_debugLogPath = Path.Combine(_tmp.Path, "gather-rules-debug.log");
_context = new GatherRuleContext(
logger, "sess", "tenant",
evt => _events.Add(evt),
@@ -56,6 +56,42 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Office
Assert.Equal(1, loaded.PeakDo.DownloadMode);
}
[Fact]
public void Save_then_load_roundtrips_terminal_metadata()
{
// Session bc803955 — CompletedAtUtc/VersionReached feed the summary dialog's
// synthetic Office row (duration + version).
using var tmp = new TempDirectory();
var sut = new OfficeInstallStatePersistence(tmp.Path, NewLogger(tmp.Path));
var completedAt = new DateTime(2026, 6, 8, 12, 3, 22, DateTimeKind.Utc);
sut.Save(new OfficeInstallStateData
{
State = OfficeInstallStateData.StateCompleted,
CompletedAtUtc = completedAt,
VersionReached = "16.0.20228.20190",
});
var loaded = sut.Load();
Assert.Equal(completedAt, loaded!.CompletedAtUtc);
Assert.Equal("16.0.20228.20190", loaded.VersionReached);
}
[Fact]
public void Load_tolerates_state_files_from_older_agents_without_terminal_metadata()
{
using var tmp = new TempDirectory();
var sut = new OfficeInstallStatePersistence(tmp.Path, NewLogger(tmp.Path));
// Pre-terminal-metadata wire format (verbatim shape from a live 2.0.1404 session).
File.WriteAllText(sut.StateFilePath,
"{\"State\":\"Completed\",\"StartedAtUtc\":\"2026-08-18T07:56:16.8208141Z\",\"StartedTrigger\":\"registry\"}");
var loaded = sut.Load();
Assert.Equal(OfficeInstallStateData.StateCompleted, loaded!.State);
Assert.Null(loaded.CompletedAtUtc);
Assert.Null(loaded.VersionReached);
}
[Fact]
public void Save_overwrites_a_previous_state()
{
@@ -0,0 +1,377 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Registry second pillar (audit 2026-08-17): pure parsing/diff logic plus the
/// tick flow (baseline silence, state-change emission, reconciliation dwell)
/// driven through the snapshot override — no live registry.
/// </summary>
public sealed class ImeRegistryAppStateObserverTests
{
private const string App1 = "11111111-1111-1111-1111-111111111111";
private const string App2 = "22222222-2222-2222-2222-222222222222";
private static readonly DateTime T0 = new DateTime(2026, 8, 18, 8, 0, 0, DateTimeKind.Utc);
// ── pure helpers ────────────────────────────────────────────────────────
[Theory]
[InlineData("11111111-1111-1111-1111-111111111111_1", "11111111-1111-1111-1111-111111111111")]
[InlineData("11111111-1111-1111-1111-111111111111", "11111111-1111-1111-1111-111111111111")]
[InlineData("GRS", null)]
[InlineData("", null)]
[InlineData("not-a-guid_1", null)]
public void ExtractAppId_parses_app_keys_and_rejects_non_app_keys(string keyName, string? expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ExtractAppId(keyName));
[Fact]
public void ParseEnforcementStateMessage_reads_state_and_error()
{
var (state, error) = ImeRegistryAppStateObserver.ParseEnforcementStateMessage(
"{\"EnforcementState\":1000,\"ErrorCode\":0}");
Assert.Equal(1000, state);
Assert.Equal(0L, error);
Assert.Equal((null, null), ImeRegistryAppStateObserver.ParseEnforcementStateMessage("not json"));
Assert.Equal((null, null), ImeRegistryAppStateObserver.ParseEnforcementStateMessage("[]"));
}
[Theory]
[InlineData(1000, "success")]
[InlineData(1004, "success")]
[InlineData(2009, "inProgress")]
[InlineData(3000, "requirementsNotMet")]
[InlineData(5003, "error")]
[InlineData(6001, "notAttempted")]
[InlineData(42, "unknown")]
public void ClassifyEnforcementState_bands(int state, string expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ClassifyEnforcementState(state));
[Theory]
[InlineData(1000, "installed")]
[InlineData(1999, "installed")]
[InlineData(3000, "failed")]
[InlineData(2000, "notApplicable")]
[InlineData(2999, "notApplicable")]
[InlineData(0, null)]
[InlineData(999, null)]
[InlineData(3001, null)]
[InlineData(4000, null)]
public void ClassifyStatusServiceStatus_bands(int status, string? expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ClassifyStatusServiceStatus(status));
[Fact]
public void TerminalOutcome_prefers_enforcement_state_then_status_service()
{
// Enforcement state alone decides when terminal.
Assert.Equal("success", ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 1000)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 5003)));
// Non-terminal enforcement bands defer to the StatusService pillar.
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 2009)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(
Entry(enforcementState: 2009, statusServiceStatus: 3000)));
Assert.Equal("success", ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 1000)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 3000)));
// requirementsNotMet / notApplicable / empty entries are not judgeable.
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 3000)));
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 2500)));
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry()));
}
[Fact]
public void ChangedFields_reports_each_field_and_ignores_espPhase_casing()
{
var prev = Entry(enforcementState: 2009, errorCode: null, exitCode: null,
statusServiceStatus: null, espTracked: false, espPhase: "DevicePreparation");
var next = Entry(enforcementState: 1000, errorCode: 0, exitCode: 0,
statusServiceStatus: 1000, espTracked: true, espPhase: "devicepreparation");
var changed = ImeRegistryAppStateObserver.ChangedFields(prev, next);
Assert.Equal(
new[] { "enforcementState", "errorCode", "exitCode", "statusServiceStatus", "espTracked" },
changed);
// Case-only espPhase difference is not a change; identity diffs to empty.
Assert.DoesNotContain("espPhase", changed);
Assert.Empty(ImeRegistryAppStateObserver.ChangedFields(next, next));
}
[Fact]
public void ChangedFields_null_previous_reports_populated_fields_of_new_entry()
{
var fresh = Entry(enforcementState: 2000, espTracked: true, espPhase: "AccountSetup");
var changed = ImeRegistryAppStateObserver.ChangedFields(null, fresh);
Assert.Contains("enforcementState", changed);
Assert.Contains("espTracked", changed);
Assert.Contains("espPhase", changed);
// Fields that are null on both sides are not "changes" for a brand-new entry.
Assert.DoesNotContain("errorCode", changed);
Assert.DoesNotContain("exitCode", changed);
Assert.DoesNotContain("statusServiceStatus", changed);
}
[Fact]
public void DiffSnapshots_reports_only_changed_fields()
{
var prev = Snapshot((App1, 2009, null));
var next = Snapshot((App1, 1000, 0), (App2, 2000, null));
var changes = ImeRegistryAppStateObserver.DiffSnapshots(prev, next);
Assert.Equal(2, changes.Count);
var app1Change = changes.Single(c => c.Entry.AppId == App1);
Assert.Contains("enforcementState", app1Change.ChangedFields);
Assert.Contains("errorCode", app1Change.ChangedFields);
Assert.False(app1Change.IsNew);
Assert.True(changes.Single(c => c.Entry.AppId == App2).IsNew);
Assert.Empty(ImeRegistryAppStateObserver.DiffSnapshots(next, next));
}
[Fact]
public void IsDivergent_rules()
{
var installed = PackageState(App1, AppInstallationState.Installed);
var errored = PackageState(App1, AppInstallationState.Error);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("error", installed, true, out var r1));
Assert.Equal("registry_error_log_installed", r1);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("success", errored, true, out var r2));
Assert.Equal("registry_success_log_error", r2);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("success", null, true, out var r3));
Assert.Equal("app_unknown_to_log_tracking", r3);
// Tracker idle (no apps at all) — not judgeable, no false alarm.
Assert.False(ImeRegistryAppStateObserver.IsDivergent("success", null, false, out _));
// Agreement.
Assert.False(ImeRegistryAppStateObserver.IsDivergent("success", installed, true, out _));
Assert.False(ImeRegistryAppStateObserver.IsDivergent("error", errored, true, out _));
}
// ── tick flow ───────────────────────────────────────────────────────────
[Fact]
public void Baseline_is_silent_then_changes_emit_state_events()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
current = Snapshot((App1, 1000, 0));
observer.Tick("registry_change");
}
var evt = Assert.Single(Events(sink, SharedEventTypes.RegistryAppState));
Assert.Equal(App1, evt.Payload!["appId"]);
Assert.Equal("1000", evt.Payload["enforcementState"]);
Assert.Equal("success", evt.Payload["enforcementClass"]);
}
[Fact]
public void NonEmpty_baseline_emits_one_summary_with_counts()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 1000, 0), (App2, 5003, 101));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
observer.Tick("periodic"); // summary is one-shot, not per tick
}
var summary = Assert.Single(Events(sink, SharedEventTypes.RegistryAppBaseline));
Assert.Equal("2", summary.Payload!["totalApps"]);
Assert.Equal("1", summary.Payload["successCount"]);
Assert.Equal("1", summary.Payload["errorCount"]);
// The baseline stays silent on the diff rail.
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
}
[Fact]
public void Empty_baseline_emits_no_summary()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = new ImeRegistrySnapshot();
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
}
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppBaseline));
}
[Fact]
public void Unchanged_snapshot_emits_nothing()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
observer.Tick("periodic");
observer.Tick("periodic");
}
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
}
[Fact]
public void Reconciliation_fires_once_after_settle_delay_on_divergence()
{
var tracker = new List<AppPackageState> { PackageState(App1, AppInstallationState.Installed) };
var (sink, clock, observer) = CreateObserver(trackerApps: tracker);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = Snapshot((App1, 5003, 101)); // registry error, log says Installed
observer.Tick("registry_change");
// Before the settle delay: no reconciliation yet.
Assert.Empty(Events(sink, SharedEventTypes.AppStateReconciliation));
clock.Advance(ImeRegistryAppStateObserver.ReconcileSettleDelay + TimeSpan.FromSeconds(1));
observer.Tick("periodic");
observer.Tick("periodic"); // second pass must not re-emit
}
var rec = Assert.Single(Events(sink, SharedEventTypes.AppStateReconciliation));
Assert.Equal("error", rec.Payload!["registryOutcome"]);
Assert.Equal("registry_error_log_installed", rec.Payload["reason"]);
Assert.Equal("Installed", rec.Payload["logState"]);
}
[Fact]
public void Reconciliation_skips_agreeing_and_baseline_only_apps()
{
var tracker = new List<AppPackageState> { PackageState(App1, AppInstallationState.Installed) };
var (sink, clock, observer) = CreateObserver(trackerApps: tracker);
// App2 is terminal in the BASELINE (pre-existing from an earlier enrollment) and
// never changes — it must never be judged. App1 changes and agrees with the log.
var current = Snapshot((App1, 2000, null), (App2, 5000, 1));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = Snapshot((App1, 1000, 0), (App2, 5000, 1));
observer.Tick("registry_change");
clock.Advance(ImeRegistryAppStateObserver.ReconcileSettleDelay + TimeSpan.FromSeconds(1));
observer.Tick("periodic");
}
Assert.Empty(Events(sink, SharedEventTypes.AppStateReconciliation));
}
[Fact]
public void State_event_cap_emits_single_warning_then_suppresses()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var baseline = new ImeRegistrySnapshot();
var flooded = new ImeRegistrySnapshot();
for (var i = 0; i < ImeRegistryAppStateObserver.MaxStateEventsPerSession + 25; i++)
{
var appId = Guid.NewGuid().ToString("D");
baseline.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId).EnforcementState = 2000;
flooded.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId).EnforcementState = 1000;
}
var current = baseline;
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = flooded;
observer.Tick("registry_change");
}
var events = Events(sink, SharedEventTypes.RegistryAppState);
// Cap payload-carrying events + exactly one cap-notice Warning.
Assert.Equal(ImeRegistryAppStateObserver.MaxStateEventsPerSession + 1, events.Count);
Assert.Single(events, e => e.Payload != null
&& e.Payload.TryGetValue(SignalPayloadKeys.Message, out var m)
&& m.Contains("cap reached"));
}
// ── harness ─────────────────────────────────────────────────────────────
private static (FakeSignalIngressSink sink, VirtualClock clock, ImeRegistryAppStateObserver observer)
CreateObserver(IReadOnlyList<AppPackageState>? trackerApps)
{
var sink = new FakeSignalIngressSink();
var clock = new VirtualClock(T0);
var post = new InformationalEventPost(sink, clock);
var observer = new ImeRegistryAppStateObserver(
post,
logger: null,
clock: clock,
trackerStateProbe: trackerApps == null ? null : () => trackerApps);
return (sink, clock, observer);
}
private static AppRegistryEntry Entry(
int? enforcementState = null, long? errorCode = null, int? exitCode = null,
int? statusServiceStatus = null, bool espTracked = false, string? espPhase = null)
{
var entry = new AppRegistryEntry($"{ImeRegistrySnapshot.DeviceContext}|{App1}", ImeRegistrySnapshot.DeviceContext, App1);
entry.EnforcementState = enforcementState;
entry.ErrorCode = errorCode;
entry.ExitCode = exitCode;
entry.StatusServiceStatus = statusServiceStatus;
entry.EspTracked = espTracked;
entry.EspPhase = espPhase;
return entry;
}
private static ImeRegistrySnapshot Snapshot(params (string appId, int enforcementState, long? errorCode)[] apps)
{
var snapshot = new ImeRegistrySnapshot();
foreach (var (appId, state, error) in apps)
{
var entry = snapshot.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId);
entry.EnforcementState = state;
entry.ErrorCode = error;
}
return snapshot;
}
private static AppPackageState PackageState(string id, AppInstallationState state)
{
var pkg = new AppPackageState(id, listPos: 0);
if (state == AppInstallationState.Installed)
{
// Route through Installing first — a bare Installed would trip the
// inverse-detection auto-downgrade heuristic (Installed w/o activity -> Skipped).
pkg.UpdateState(AppInstallationState.Installing);
}
pkg.UpdateState(state);
return pkg;
}
private static IReadOnlyList<FakeSignalIngressSink.PostedSignal> Events(FakeSignalIngressSink sink, string eventType) =>
sink.Posted.Where(p =>
p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
}
}
@@ -0,0 +1,130 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Session afee7ae0 (2026-08-18) — Device Preparation structural-noise suppression.
/// A nine-minute WDP session shipped 900+ occurrences of WIL error 1005 plus 400+
/// duplicate-workload 417/418 records and "Autopilot policy not found" 100s — all
/// structural on WDP (no deployment profile exists; the BootstrapperAgent's SLDM
/// progress poll produces the 417/418 chatter). On WDP those IDs are suppressed
/// entirely (<see cref="ModernDeploymentTracker.DevicePreparationNoiseEventIds"/>);
/// genuine diagnostics (e.g. 408 provisioning warnings) and Critical records keep
/// flowing, and Classic sessions keep the harmless-downgrade + rollup path.
/// </summary>
public sealed class ModernDeploymentTrackerDevicePrepTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 8, 18, 9, 10, 0, DateTimeKind.Utc);
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly AgentLogger _logger;
public ModernDeploymentTrackerDevicePrepTests()
{
_sink = new FakeSignalIngressSink();
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
}
public void Dispose() => _tmp.Dispose();
private ModernDeploymentTracker BuildTracker(bool isDevicePreparation) =>
new ModernDeploymentTracker(
sessionId: "sess-wdp-noise",
tenantId: "tenant-wdp-noise",
post: new InformationalEventPost(_sink, new VirtualClock(At)),
logger: _logger,
backfillEnabled: false,
isDevicePreparation: isDevicePreparation);
private void Process(ModernDeploymentTracker tracker, int eventId, int level, string shortName, string channel)
{
tracker.ProcessEvent(
eventId: eventId,
level: level,
levelDisplayName: level == 1 ? "Critical" : level == 2 ? "Error" : "Warning",
providerName: "ModernDeployment-Diagnostics-Provider",
timeCreatedUtc: At,
formattedDescription: $"EventID {eventId} record.",
shortName: shortName,
channelName: channel,
isBackfill: false);
}
private List<FakeSignalIngressSink.PostedSignal> PostedModernDeployment() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et.StartsWith("modern_deployment", StringComparison.Ordinal)).ToList();
[Theory]
[InlineData(100, 3, "Autopilot")] // policy not found — no profile exists on WDP
[InlineData(417, 3, "Autopilot")] // duplicate workload — SLDM poll chatter
[InlineData(418, 3, "Autopilot")] // duplicate batch — SLDM poll chatter
[InlineData(1005, 2, "ManagementService")] // WIL error storm
public void WdpNoiseIds_AreFullySuppressed_OnDevicePreparation(int eventId, int level, string shortName)
{
var tracker = BuildTracker(isDevicePreparation: true);
var channel = shortName == "Autopilot"
? ModernDeploymentTracker.AutopilotChannel
: ModernDeploymentTracker.ManagementChannel;
for (var i = 0; i < 10; i++)
{
Process(tracker, eventId, level, shortName, channel);
}
Assert.Empty(PostedModernDeployment());
}
[Fact]
public void GenuineWdpWarning_408_KeepsFlowing_OnDevicePreparation()
{
// 408 "Autopilot Provisioning reported a warning" carries the real WDP session
// context — it is NOT in the noise set and must keep flowing.
var tracker = BuildTracker(isDevicePreparation: true);
Process(tracker, 408, level: 3, shortName: "Autopilot", channel: ModernDeploymentTracker.AutopilotChannel);
var posted = Assert.Single(PostedModernDeployment());
Assert.Equal("modern_deployment_warning", posted.Payload![SignalPayloadKeys.EventType]);
}
[Fact]
public void CriticalRecords_AreNeverSuppressed_EvenForNoiseIds()
{
var tracker = BuildTracker(isDevicePreparation: true);
Process(tracker, 1005, level: 1, shortName: "ManagementService", channel: ModernDeploymentTracker.ManagementChannel);
var posted = Assert.Single(PostedModernDeployment());
Assert.Equal("modern_deployment_error", posted.Payload![SignalPayloadKeys.EventType]);
}
[Fact]
public void ClassicSessions_KeepTheHarmlessDowngradePath_ForTheSameIds()
{
// Classic regression guard: without the WDP flag EventID 100 still flows through
// the harmless-downgrade path (individual Debug emissions up to the rollup limit).
var tracker = BuildTracker(isDevicePreparation: false);
for (var i = 0; i < 2; i++)
{
Process(tracker, 100, level: 3, shortName: "Autopilot", channel: ModernDeploymentTracker.AutopilotChannel);
}
Assert.Equal(2, PostedModernDeployment().Count);
}
}
}
@@ -53,7 +53,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
using var tmp = new TempDirectory();
var logDir = Path.Combine(tmp.Path, "logs");
Directory.CreateDirectory(logDir);
var crashPath = Path.Combine(logDir, "crash_20260421_100000.log");
var crashPath = Path.Combine(logDir, "crash-20260421-100000.log");
File.WriteAllText(crashPath, "[2026-04-21T10:00:00Z] FATAL: InvalidOperationException: simulated crash");
var summary = AutopilotMonitor.Agent.V2.Program.DetectPreviousExit(tmp.Path, logDir);
@@ -312,7 +312,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
AutopilotMonitor.Agent.V2.Program.WriteCrashLog(logDir, new InvalidOperationException("boom"));
var files = Directory.GetFiles(logDir, "crash_*.log");
var files = Directory.GetFiles(logDir, "crash-*.log");
Assert.Single(files);
var content = File.ReadAllText(files[0]);
Assert.Contains("FATAL:", content);
@@ -462,6 +462,64 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
Assert.Equal("IME Agent version: 1.101.109.0", info.Payload[SignalPayloadKeys.Message]);
}
[Fact]
public void ImeTokenFailure_emits_warning_event_with_error_code()
{
using var f = new ImeLogTrackerAdapterFixture();
using var adapter = new ImeLogTrackerAdapter(f.Tracker, f.Ingress, f.Clock);
adapter.TriggerImeTokenFailureFromTest("3399614467",
"Failed to get AAD token. len = 0 using client id x and resource id y, errorCode = 3399614467");
var info = Assert.Single(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
Assert.Equal("ImeLogTracker", info.Payload![SignalPayloadKeys.Source]);
Assert.Equal("3399614467", info.Payload["errorCode"]);
Assert.Contains("errorCode=3399614467", info.Payload[SignalPayloadKeys.Message]);
Assert.Contains("Failed to get AAD token", info.Payload["logLine"]);
}
[Fact]
public void ImeTokenFailure_grace_window_emits_only_without_token_success()
{
// "Failed to get AAD token" appears routinely during the device phase (no user
// token yet) — the Warning must fire only when NO success follows for the whole
// grace window.
using var f = new ImeLogTrackerAdapterFixture();
using var adapter = new ImeLogTrackerAdapter(f.Tracker, f.Ingress, f.Clock);
var t0 = DateTime.UtcNow;
// Failure resolved by a token success: never emitted, not even after the window.
f.Tracker.HandleImeTokenFailureForTest("100", "line a");
f.Tracker.NoteTokenSuccessForTest();
f.Tracker.CheckPendingTokenFailure(t0 + ImeLogTracker.TokenFailureGraceWindow + TimeSpan.FromMinutes(1));
Assert.Empty(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
// Unresolved failure: silent before the window, one Warning after it.
f.Tracker.HandleImeTokenFailureForTest("100", "line b");
f.Tracker.CheckPendingTokenFailure(t0 + TimeSpan.FromMinutes(5));
Assert.Empty(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
f.Tracker.CheckPendingTokenFailure(t0 + ImeLogTracker.TokenFailureGraceWindow + TimeSpan.FromHours(1));
Assert.Single(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
// Same code again: re-armed window expires but the per-code dedup suppresses.
f.Tracker.HandleImeTokenFailureForTest("100", "line c");
f.Tracker.CheckPendingTokenFailure(t0 + TimeSpan.FromHours(3));
Assert.Single(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
}
[Fact]
public void ImeTokenFailure_empty_error_code_reports_unknown()
{
using var f = new ImeLogTrackerAdapterFixture();
using var adapter = new ImeLogTrackerAdapter(f.Tracker, f.Ingress, f.Clock);
adapter.TriggerImeTokenFailureFromTest("", null!);
var info = Assert.Single(f.InfoEvents(SharedEventTypes.ImeTokenFailure));
Assert.Equal("unknown", info.Payload!["errorCode"]);
Assert.False(info.Payload.ContainsKey("logLine"));
}
[Fact]
public void ImeAgentVersion_null_or_empty_is_skipped()
{
@@ -165,6 +165,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
public IReadOnlyCollection<string> StarvedAlreadyReportedOverride { get; set; } = Array.Empty<string>();
public List<string> TerminationActionLog { get; } = new List<string>();
public TimeSpan SpoolDrainPeriodOverride { get; set; } = TimeSpan.Zero;
// Session bc803955 — DevicePrep app-wave settle wait. Zero (default) disables the
// wait so the existing timing-sensitive tests stay untouched; the dedicated settle
// tests opt in with a real budget.
public TimeSpan DevicePrepAppSettleWaitOverride { get; set; } = TimeSpan.Zero;
// WG Part-1 straggler-ordering fix — the production wiring passes
// orchestrator.StopCollectorHosts, whose collectors emit their one-shot
// stop-time stragglers (network_bandwidth_estimate, …) through Post during
@@ -210,7 +214,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
stopPeripheralCollectors: StopPeripheralCollectorsHook,
// Zero out the timing ceremony for tests — production paths are covered by
// the dedicated V1-parity tests below which opt back in via their own Rig.
lateEventGracePeriod: TimeSpan.Zero);
lateEventGracePeriod: TimeSpan.Zero,
devicePrepAppSettleWait: DevicePrepAppSettleWaitOverride);
}
public void Dispose() => Tmp.Dispose();
@@ -317,6 +322,91 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
Assert.True(File.Exists(statusPath), "final-status.json should be written even when ShowEnrollmentSummary=false");
}
// ---- Session bc803955 — DevicePrep app-wave settle wait ----
private static DecisionState DevicePrepState(SessionStage stage) =>
new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1"))
{
Stage = stage,
ScenarioProfile = new EnrollmentScenarioProfile(
mode: EnrollmentMode.DevicePreparation,
joinMode: EnrollmentJoinMode.Unknown,
espConfig: EspConfig.Unknown,
preProvisioningSide: PreProvisioningSide.None,
confidence: ProfileConfidence.Medium,
evidenceOrdinal: 1,
reason: "test"),
}.Build();
[Fact]
public void Handle_devicePrep_settle_wait_catches_late_app_wave_for_dialog()
{
using var rig = new Rig();
rig.State = DevicePrepState(SessionStage.Completed);
rig.DevicePrepAppSettleWaitOverride = TimeSpan.FromSeconds(10);
// Simulate the IME user-context sync landing AFTER termination handling began
// (AppWorkload.log flush latency) — the wave arrives 500ms in, well before the
// 10s budget; the settle loop must pick it up and the dialog must render it.
var waveThread = new Thread(() =>
{
Thread.Sleep(500);
var app = new AppPackageState("app-late", 0);
app.UpdateState(AppInstallationState.Downloading);
TestHelpers.SetTargeted(app, AppTargeted.User);
rig.Packages.Add(app);
});
waveThread.Start();
var sw = System.Diagnostics.Stopwatch.StartNew();
rig.Build().Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.Completed));
sw.Stop();
waveThread.Join();
var statusJson = File.ReadAllText(Path.Combine(rig.StateDir, SummaryDialogLauncher.FinalStatusFileName));
Assert.Contains("\"totalApps\": 1", statusJson);
// Early exit: the wave arrived at ~500ms — the handler must not sit out the full 10s.
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(8),
$"settle wait should exit early once packages arrive (took {sw.Elapsed.TotalSeconds:0.0}s)");
}
[Fact]
public void Handle_devicePrep_settle_wait_is_bounded_when_no_apps_arrive()
{
using var rig = new Rig();
rig.State = DevicePrepState(SessionStage.Completed);
rig.DevicePrepAppSettleWaitOverride = TimeSpan.FromMilliseconds(600);
var sw = System.Diagnostics.Stopwatch.StartNew();
rig.Build().Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.Completed));
sw.Stop();
// Budget elapsed (>=600ms) and the dialog still rendered — just without app data.
Assert.True(sw.Elapsed >= TimeSpan.FromMilliseconds(600),
$"bounded wait should consume its budget when no data arrives (took {sw.Elapsed.TotalMilliseconds:0}ms)");
var statusJson = File.ReadAllText(Path.Combine(rig.StateDir, SummaryDialogLauncher.FinalStatusFileName));
Assert.Contains("\"totalApps\": 0", statusJson);
}
[Fact]
public void Handle_non_devicePrep_does_not_wait_even_with_empty_packages()
{
using var rig = new Rig();
// Classic/unknown mode + empty package list — the settle wait must not engage.
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.Completed }.Build();
rig.DevicePrepAppSettleWaitOverride = TimeSpan.FromSeconds(10);
var sw = System.Diagnostics.Stopwatch.StartNew();
rig.Build().Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.Completed));
sw.Stop();
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(8),
$"non-DevicePrep termination must not settle-wait (took {sw.Elapsed.TotalSeconds:0.0}s)");
}
[Fact]
public void Handle_writes_enrollment_complete_marker_on_non_whiteglove_terminal()
{
@@ -1407,6 +1497,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
Assert.DoesNotContain("app_install_starved", rig.EmittedEventTypes);
}
[Fact]
public void StarvedApps_skipped_on_device_preparation()
{
// afee7ae0: WDP has no ESP apps gate — post-desktop installs are the designed
// order, so the sweep would report every pending app as noise (10+ events on a
// normal session). The per-app data still travels via app_tracking_summary.
using var rig = new Rig();
rig.State = DevicePrepState(SessionStage.Completed);
rig.StarvedAppsOverride = new[]
{
BuildStarvedApp("app-pending-1", "Contoso Backgrounds"),
BuildStarvedApp("app-pending-2", "Contoso Tools"),
};
rig.Build().Handle(sender: null!,
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.Completed));
Assert.DoesNotContain("app_install_starved", rig.EmittedEventTypes);
}
[Fact]
public void PromoteActiveInstalls_skipped_on_effect_infrastructure_failure()
{
@@ -1,8 +1,10 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Termination;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
@@ -508,6 +510,121 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
Assert.Null(status.SignalTimestamps);
}
// ---- Session bc803955 — synthetic Office row from the C2R detector's persisted state ----
private static FinalStatus BuildWithOffice(OfficeInstallStateData? officeState, IReadOnlyList<AppPackageState>? packages = null) =>
FinalStatusBuilder.Build(StateWith(SessionStage.Completed),
Args(EnrollmentTerminationReason.DecisionTerminalStage, EnrollmentTerminationOutcome.Succeeded, SessionStage.Completed),
packageStates: packages, agentStartTimeUtc: StartUtc, officeState: officeState);
[Fact]
public void Build_office_completed_adds_device_row_with_duration_and_counts()
{
var office = new OfficeInstallStateData
{
State = OfficeInstallStateData.StateCompleted,
StartedAtUtc = StartUtc.AddMinutes(1),
CompletedAtUtc = StartUtc.AddMinutes(4).AddSeconds(22),
VersionReached = "16.0.20228.20190",
};
var status = BuildWithOffice(office);
var row = Assert.Single(status.PackageStatesByPhase["Device"]);
Assert.Equal("Microsoft 365 Apps (16.0.20228.20190)", row.AppName);
Assert.Equal("Installed", row.State);
Assert.True(row.IsCompleted);
Assert.False(row.IsError);
Assert.Equal(202, row.DurationSeconds);
Assert.Equal(office.StartedAtUtc.Value.ToString("o"), row.StartedAt);
Assert.Equal(1, status.AppSummary.TotalApps);
Assert.Equal(1, status.AppSummary.CompletedApps);
Assert.Equal(0, status.AppSummary.ErrorCount);
Assert.Equal(1, status.AppSummary.AppsByPhase["Device"]);
}
[Fact]
public void Build_office_active_row_is_installing_without_completion()
{
var office = new OfficeInstallStateData
{
State = OfficeInstallStateData.StateActive,
StartedAtUtc = StartUtc.AddMinutes(1),
};
var status = BuildWithOffice(office);
var row = Assert.Single(status.PackageStatesByPhase["Device"]);
Assert.Equal("Microsoft 365 Apps", row.AppName);
Assert.Equal("Installing", row.State);
Assert.False(row.IsCompleted);
Assert.Null(row.DurationSeconds);
Assert.Equal(1, status.AppSummary.TotalApps);
Assert.Equal(0, status.AppSummary.CompletedApps);
}
[Fact]
public void Build_office_failed_row_counts_device_error()
{
var office = new OfficeInstallStateData
{
State = OfficeInstallStateData.StateFailed,
StartedAtUtc = StartUtc.AddMinutes(1),
CompletedAtUtc = StartUtc.AddMinutes(2),
};
var status = BuildWithOffice(office);
var row = Assert.Single(status.PackageStatesByPhase["Device"]);
Assert.Equal("Error", row.State);
Assert.True(row.IsError);
Assert.True(row.IsCompleted); // Error is terminal — mirrors IsCompleted()
Assert.Equal(1, status.AppSummary.ErrorCount);
Assert.Equal(1, status.AppSummary.DeviceErrors);
}
[Theory]
[InlineData(OfficeInstallStateData.StatePreinstalled)]
[InlineData("garbage")]
[InlineData(null)]
public void Build_office_preinstalled_or_unknown_adds_no_row(string? state)
{
var office = state == null ? null : new OfficeInstallStateData { State = state };
var status = BuildWithOffice(office);
Assert.Empty(status.PackageStatesByPhase);
Assert.Equal(0, status.AppSummary.TotalApps);
}
[Fact]
public void Build_office_row_is_prepended_before_ime_device_apps()
{
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
var list = new AppPackageStateList(logger);
var imeApp = new AppPackageState("app-1", 0);
imeApp.UpdateState(AppInstallationState.Installed);
TestHelpers.SetTargeted(imeApp, AppTargeted.Device);
list.Add(imeApp);
var office = new OfficeInstallStateData
{
State = OfficeInstallStateData.StateCompleted,
StartedAtUtc = StartUtc,
CompletedAtUtc = StartUtc.AddMinutes(3),
};
var status = BuildWithOffice(office, list);
Assert.Equal(2, status.PackageStatesByPhase["Device"].Count);
Assert.StartsWith("Microsoft 365 Apps", status.PackageStatesByPhase["Device"][0].AppName);
Assert.Equal(2, status.AppSummary.TotalApps);
Assert.Equal(2, status.AppSummary.CompletedApps);
Assert.Equal(2, status.AppSummary.AppsByPhase["Device"]);
}
}
/// <summary>Mini helper for reflection-based field setters — AppPackageState.Targeted is a private-set property.</summary>
@@ -221,6 +221,81 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Transport
Assert.Equal(1, System.Threading.Volatile.Read(ref signalled));
}
[Fact]
public void PendingBytes_tracks_enqueue_and_shrinks_to_zero_when_all_uploaded()
{
using var tmp = new TempDirectory();
var spool = new TelemetrySpool(tmp.Path, Clock());
Assert.Equal(0, spool.PendingBytes);
spool.Enqueue(EventDraft("a"));
spool.Enqueue(EventDraft("b"));
spool.Enqueue(EventDraft("c"));
// PendingBytes must equal the full file size while nothing is uploaded yet
// (every line written by Enqueue is still pending).
Assert.Equal(spool.SpoolFileSizeBytes, spool.PendingBytes);
spool.MarkUploaded(2);
Assert.Equal(0, spool.PendingBytes);
// File keeps its append-only size — the two measures MUST diverge here,
// otherwise the pressure trigger regresses to the file-size false positive.
Assert.True(spool.SpoolFileSizeBytes > 0);
}
[Fact]
public void PendingBytes_after_partial_upload_equals_remaining_tail_line_bytes()
{
using var tmp = new TempDirectory();
var spool = new TelemetrySpool(tmp.Path, Clock());
spool.Enqueue(EventDraft("a"));
spool.Enqueue(EventDraft("b"));
spool.Enqueue(EventDraft("c-longer-rowkey-for-distinct-length"));
spool.MarkUploaded(1);
// Expected = on-disk byte length of the last line incl. its trailing \n.
var lines = File.ReadAllLines(Path.Combine(tmp.Path, "spool.jsonl"), Encoding.UTF8);
var expected = Encoding.UTF8.GetByteCount(lines[2]) + 1;
Assert.Equal(expected, spool.PendingBytes);
}
[Fact]
public void PendingBytes_is_rehydrated_on_restart_from_pending_tail_only()
{
using var tmp = new TempDirectory();
var s1 = new TelemetrySpool(tmp.Path, Clock());
s1.Enqueue(EventDraft("a"));
s1.Enqueue(EventDraft("b"));
s1.Enqueue(EventDraft("c"));
s1.MarkUploaded(1);
var expectedPendingBytes = s1.PendingBytes;
Assert.True(expectedPendingBytes > 0);
// Restart: rehydrated PendingBytes must match the pre-restart value exactly
// (uploaded prefix excluded), not the full file size.
var s2 = new TelemetrySpool(tmp.Path, Clock());
Assert.Equal(expectedPendingBytes, s2.PendingBytes);
Assert.True(s2.PendingBytes < s2.SpoolFileSizeBytes);
}
[Fact]
public void PendingBytes_rehydrate_ignores_corrupt_tail_line()
{
using var tmp = new TempDirectory();
var s1 = new TelemetrySpool(tmp.Path, Clock());
s1.Enqueue(EventDraft("a"));
var cleanPendingBytes = s1.PendingBytes;
File.AppendAllText(
Path.Combine(tmp.Path, "spool.jsonl"),
"{\"Kind\":\"Event\",\"PartitionKey\":\"corrupt-no-",
Encoding.UTF8);
var s2 = new TelemetrySpool(tmp.Path, Clock());
Assert.Equal(cleanPendingBytes, s2.PendingBytes);
}
[Fact]
public void Enqueue_without_RequiresImmediateFlush_does_not_raise_ImmediateFlushRequested()
{
@@ -34,9 +34,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Logging
private readonly long _maxFileSizeBytes;
private readonly string _logDirectory;
private readonly string _logFileBaseName; // "agent_20260425"
private string _logFilePath; // mutable — rotates to "_002", "_003" ...
private int _rotationSuffix; // 0 = base file, 2+ = "_002" etc.
private readonly string _logFileBaseName; // "agent"
private string _logFilePath; // mutable — rotates to "-002", "-003" ...
private int _rotationSuffix; // 0 = base file, 2+ = "-002" etc.
/// <summary>
/// When true, log entries are also written to Console.Out (same format as the log file).
@@ -55,14 +55,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Logging
Directory.CreateDirectory(logDirectory);
}
_logFileBaseName = $"agent_{DateTime.Now:yyyyMMdd}";
// One canonical name, no date: the agent only lives for the duration of an
// enrollment, and a date-based split rotates at local midnight — which can fall
// mid-enrollment (local time may still be on the OOBE default timezone). Size
// rotation below bounds pathological growth. Legacy `agent_YYYYMMDD*.log` files
// from a pre-rename agent are ignored by the suffix probe and left in place.
_logFileBaseName = "agent";
_rotationSuffix = ProbeNextRotationSuffix(logDirectory, _logFileBaseName);
_logFilePath = BuildLogFilePath();
}
// Picks up the highest existing rotation suffix on the same day so a process restart
// doesn't overwrite previous segments. Returns 0 when no log file exists yet — the
// first segment uses the unsuffixed name `agent_YYYYMMDD.log`.
// Picks up the highest existing rotation suffix so a process restart doesn't
// overwrite previous segments. Returns 0 when no log file exists yet — the
// first segment uses the unsuffixed name `agent.log`.
private static int ProbeNextRotationSuffix(string dir, string baseName)
{
if (!Directory.Exists(dir)) return 0;
@@ -80,7 +85,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Logging
continue;
}
if (name.Length > baseName.Length + 1 &&
name.StartsWith(baseName + "_", StringComparison.OrdinalIgnoreCase) &&
name.StartsWith(baseName + "-", StringComparison.OrdinalIgnoreCase) &&
int.TryParse(name.Substring(baseName.Length + 1), out var suffix) &&
suffix > max)
{
@@ -97,7 +102,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Logging
{
var name = _rotationSuffix < 2
? $"{_logFileBaseName}.log"
: $"{_logFileBaseName}_{_rotationSuffix:D3}.log";
: $"{_logFileBaseName}-{_rotationSuffix:D3}.log";
return Path.Combine(_logDirectory, name);
}
@@ -25,6 +25,71 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private DateTime _impersonationLastRollupUtc = DateTime.MinValue;
private static readonly TimeSpan ImpersonationRollupInterval = TimeSpan.FromSeconds(60);
// "Failed to get AAD token" is logged ROUTINELY during the device phase (no user
// token exists yet — field-confirmed: the line appears en masse in every enrollment,
// Oliver 2026-08-18). A failure is therefore only meaningful when NO token success
// follows: the failure is held pending, any IME-TOKEN-SUCCESS match clears it, and
// the Warning fires only after the grace window elapses without a success. Genuine
// auth outages (WAM broken, conditional access, network) run long enough to hit it.
// Not persisted across restarts — failures keep arriving, the window re-arms.
internal static readonly TimeSpan TokenFailureGraceWindow = TimeSpan.FromMinutes(30);
private sealed class PendingTokenFailure
{
public string Code = "unknown";
public string Excerpt = string.Empty;
public DateTime FirstFailureUtc;
}
private PendingTokenFailure _pendingTokenFailure;
// Dedup per distinct error code so a persistent outage yields one Warning total.
private readonly HashSet<string> _reportedTokenFailureCodes = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private void HandleImeTokenFailure(string errorCode, string message)
{
if (_pendingTokenFailure != null) return; // earliest unresolved failure wins
var code = string.IsNullOrEmpty(errorCode) ? "unknown" : errorCode.Trim();
var excerpt = message ?? string.Empty;
if (excerpt.Length > 300) excerpt = excerpt.Substring(0, 300);
_pendingTokenFailure = new PendingTokenFailure
{
Code = code,
Excerpt = excerpt,
// Source-line time so a late-starting agent replaying an ongoing outage
// ages the window correctly; a success later in the same log still clears
// before the expiry check runs (lines are drained first, expiry after).
FirstFailureUtc = LastMatchedLogTimestamp ?? DateTime.UtcNow,
};
}
private void ClearPendingTokenFailure()
{
if (_pendingTokenFailure == null) return;
_logger.Debug($"ImeLogTracker: token success observed — pending token failure (errorCode={_pendingTokenFailure.Code}) cleared");
_pendingTokenFailure = null;
}
/// <summary>Called once per poll cycle after the log drain (and testable directly).</summary>
internal void CheckPendingTokenFailure(DateTime nowUtc)
{
var pending = _pendingTokenFailure;
if (pending == null) return;
if (nowUtc - pending.FirstFailureUtc < TokenFailureGraceWindow) return;
_pendingTokenFailure = null; // next failure line re-arms a fresh window
if (!_reportedTokenFailureCodes.Add(pending.Code)) return;
_logger.Warning($"ImeLogTracker: IME token acquisition failing for >= {TokenFailureGraceWindow.TotalMinutes:0} min without a success (errorCode={pending.Code})");
OnImeTokenFailure?.Invoke(pending.Code, pending.Excerpt);
}
// Test seams: drive the pending/clear/expiry cycle without the regex pipeline.
internal void HandleImeTokenFailureForTest(string errorCode, string message)
=> HandleImeTokenFailure(errorCode, message);
internal void NoteTokenSuccessForTest() => ClearPendingTokenFailure();
private void HandleImeImpersonation(string user)
{
var current = string.IsNullOrEmpty(user) ? "(unknown)" : user;
@@ -214,6 +214,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
"capturedetectionresult", "esptrackstatus", "policiesdiscovered",
"ignorecompletedapp", "updatename", "updatewin32appstate",
"cancelstuckandsetcurrent", "updatedotelemetry",
// Not app-mutating, but a stale token-failure replayed from a previous
// enrollment's log would emit a misleading Warning for THIS session.
"imetokenfailure",
};
private void HandlePatternMatch(CompiledPattern pattern, Match match, string message, CmTraceLogEntry entry)
@@ -241,6 +244,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
return;
}
// A token success resolves any pending token failure (grace-window model, see
// HandleImeTokenFailure). Matched on our own stable pattern ID — the line also
// drives espPhaseDetected, whose handler must stay unaware of token semantics.
// Placed after the staleness gate so a success replayed from a previous
// enrollment's log cannot clear a genuine current outage.
if (!isStaleReplayLine && string.Equals(pattern.PatternId, "IME-TOKEN-SUCCESS", StringComparison.OrdinalIgnoreCase))
ClearPendingTokenFailure();
try
{
var id = match.Groups["id"]?.Value;
@@ -296,6 +307,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
OnImeAgentVersion?.Invoke(version);
break;
case "imetokenfailure":
HandleImeTokenFailure(match.Groups["errorCode"]?.Value, message);
break;
case "imeimpersonation":
// PR3-A2: dedup. The same user triggers ~24 identical lines per session;
// log on first/changed user, otherwise count and emit a single rollup
@@ -119,6 +119,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
"IntuneManagementExtension-????????-??????.log",
"AppWorkload.log",
"AppWorkload-????????-??????.log",
// Enforcement decision engine (V3 ActionProcessor). IME routes every
// [Win32App][ActionProcessor] line here (app.config TraceSource mapping) —
// NO-ACTION / NOT-TARGETED classifications and the GRS "will not be
// enforced" skips are ONLY visible in this file (audit 2026-08-17).
"AppActionProcessor.log",
"AppActionProcessor-????????-??????.log",
// PowerShell script tracking
"AgentExecutor.log",
"AgentExecutor-????????-??????.log",
@@ -194,6 +200,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
public Action OnAllAppsCompleted { get; set; }
public Action OnUserSessionCompleted { get; set; }
public Action<string> OnImeSessionChange { get; set; }
// Args: errorCode (may be empty when IME logged none), raw failure line excerpt.
// Fired once per distinct errorCode per tracker lifetime (see HandleImeTokenFailure).
public Action<string, string> OnImeTokenFailure { get; set; }
// Second arg: the [DO TEL] source log line's timestamp when the DO data came from a
// log line, null when the DO collector observed the completion live via polling. The
// consumer must stamp poll-driven telemetry with the clock — inheriting an unrelated
@@ -598,6 +607,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// line never arrived within the grace period. Runs on this same loop
// thread (no locking) so it observes the buffer right after parsing.
FlushStalePlatformScriptResults(DateTime.UtcNow);
// Token-failure grace window: fires only when the failure stayed
// unresolved (no IME-TOKEN-SUCCESS) for the whole window. Runs after
// the drain so a success later in the same batch clears first.
CheckPendingTokenFailure(DateTime.UtcNow);
}
catch (OperationCanceledException) { break; }
catch (Exception ex)
@@ -37,6 +37,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly int _modernDeploymentBackfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly int[] _modernDeploymentHarmlessEventIds;
private readonly bool _isDevicePreparation;
private readonly Func<(bool? skipUser, bool? skipDevice)> _skipConfigProbe;
private readonly Func<bool> _accountSetupActivityProbe;
private readonly Func<bool> _userEspAppsSettledProbe;
@@ -194,7 +195,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// <summary>
/// True when DeviceSetup contains a SaveWhiteGloveSuccessResult subcategory with state "succeeded".
/// This is a definitive WhiteGlove (Pre-Provisioning) confirmation signal from the ESP registry.
/// NOT a WhiteGlove indicator on Win11 25H2+ — Windows writes it in every enrollment
/// (see <see cref="ProvisioningStatusTracker.HasSaveWhiteGloveSuccessResult"/>). Observability only.
/// </summary>
public bool HasSaveWhiteGloveSuccessResult => _provisioningTracker?.HasSaveWhiteGloveSuccessResult ?? false;
@@ -210,6 +212,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
int modernDeploymentBackfillLookbackMinutes = 30,
string stateDirectory = null,
int[] modernDeploymentHarmlessEventIds = null,
bool isDevicePreparation = false,
Func<(bool? skipUser, bool? skipDevice)> skipConfigProbe = null,
Func<bool> accountSetupActivityProbe = null,
Func<bool> userEspAppsSettledProbe = null,
@@ -227,6 +230,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_modernDeploymentBackfillLookbackMinutes = modernDeploymentBackfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_modernDeploymentHarmlessEventIds = modernDeploymentHarmlessEventIds;
_isDevicePreparation = isDevicePreparation;
// Test seam: allow injection of a fake skip-config reader; defaults to the real
// registry probe so production never has to think about this parameter.
_skipConfigProbe = skipConfigProbe ?? (() => EspSkipConfigurationProbe.Read(_logger));
@@ -347,16 +351,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_shellCoreTracker.HelloWizardStarted += OnHelloWizardStarted;
_shellCoreTracker.Start();
_provisioningTracker = new ProvisioningStatusTracker(
_sessionId,
_tenantId,
_post,
_logger,
packageStatesProbe: _packageStatesProbe);
_provisioningTracker.EspFailureDetected += OnProvisioningEspFailureDetected;
_provisioningTracker.DeviceSetupProvisioningComplete += OnDeviceSetupProvisioningComplete;
_provisioningTracker.AccountSetupProvisioningComplete += OnAccountSetupProvisioningComplete;
_provisioningTracker.Start();
// WDP gate (afee7ae0, 2026-08-18): Device Preparation writes its BootstrapperAgent
// progress under the very AutopilotSettings key this tracker subtree-watches, so on
// WDP the watcher woke up on every provisioning write only to read three ESP
// category values that never exist there. The ESP categories are the tracker's
// ONLY subject — skip it entirely. All consumers already null-guard the tracker.
if (!_isDevicePreparation)
{
_provisioningTracker = new ProvisioningStatusTracker(
_sessionId,
_tenantId,
_post,
_logger,
packageStatesProbe: _packageStatesProbe);
_provisioningTracker.EspFailureDetected += OnProvisioningEspFailureDetected;
_provisioningTracker.DeviceSetupProvisioningComplete += OnDeviceSetupProvisioningComplete;
_provisioningTracker.AccountSetupProvisioningComplete += OnAccountSetupProvisioningComplete;
_provisioningTracker.Start();
}
else
{
_logger.Info("EspAndHelloTracker: Device Preparation session — ProvisioningStatusTracker (ESP categories) not started");
}
if (_modernDeploymentWatcherEnabled)
{
@@ -369,7 +385,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_modernDeploymentBackfillEnabled,
_modernDeploymentBackfillLookbackMinutes,
_stateDirectory,
_modernDeploymentHarmlessEventIds);
_modernDeploymentHarmlessEventIds,
_isDevicePreparation);
_modernDeploymentTracker.Start();
}
}
@@ -0,0 +1,643 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text.Json;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.Shared.Models;
using Microsoft.Win32;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Registry "second pillar" for IME app state (audit 2026-08-17, architecture decision:
/// registry = stable truth, IME logs = narrative). IME persists its authoritative per-app
/// state under <c>HKLM\SOFTWARE\Microsoft\IntuneManagementExtension</c>:
/// <list type="bullet">
/// <item><c>Win32Apps\&lt;userGuid&gt;\&lt;appGuid&gt;_&lt;rev&gt;</c> — EnforcementStateMessage JSON
/// (EnforcementState + ErrorCode), ExitCode, Intent. Device context = Guid.Empty user.</item>
/// <item><c>EspTrackingWin32Apps\&lt;userGuid&gt;\&lt;appGuid&gt;_&lt;rev&gt;</c> — which apps IME
/// registered for ESP tracking and in which phase.</item>
/// <item><c>SideCarPolicies\StatusServiceReports\&lt;userGuid&gt;\&lt;appId&gt;</c> — the exact
/// AppInstallStatusReport the ESP page renders (Status 1000=Installed .. 3000=Failed).</item>
/// </list>
/// The observer is snapshot-and-diff driven (RegistryWatcher gives key-scope edges only,
/// coalesced — a per-write parse is impossible by design): every tick re-reads the three
/// surfaces and emits <c>registry_app_state</c> on real field changes. Pre-existing state at
/// agent start is the silent baseline — Win32Apps keys survive re-enrollments, so replaying
/// them as fresh events would be the registry twin of the historic-IME-replay bug.
///
/// Reconciliation (= built-in log-pattern drift alarm): once a registry entry that changed
/// during THIS session has been terminal for <see cref="ReconcileSettleDelay"/>, its outcome
/// is compared against the log-derived <see cref="AppPackageState"/>. A divergence means the
/// IME log patterns missed or mis-classified the app — emitted once per app as
/// <c>app_state_reconciliation</c> (Warning). Pure observability: nothing here feeds the
/// DecisionEngine.
/// </summary>
internal sealed class ImeRegistryAppStateObserver
{
public const string SourceLabel = "RegistryAppState";
internal const string ImeRootKeyPath = @"SOFTWARE\Microsoft\IntuneManagementExtension";
// Safety cap: a pathological churn (or a wrong diff) must not flood the timeline.
// 200 covers ~25 apps x 8 state transitions; the cap itself is announced once.
internal const int MaxStateEventsPerSession = 200;
// The log tracker legitimately lags the registry (log write + 100 ms poll + phase
// gating), so divergence is only evaluated after the registry state has been stable
// terminal for this long.
internal static readonly TimeSpan ReconcileSettleDelay = TimeSpan.FromSeconds(90);
private readonly InformationalEventPost _post;
private readonly AgentLogger? _logger;
private readonly IClock _clock;
private readonly Func<IReadOnlyList<AppPackageState>>? _trackerStateProbe;
private readonly object _sync = new object();
private ImeRegistrySnapshot? _last;
private readonly Dictionary<string, DateTime> _terminalSinceUtc =
new Dictionary<string, DateTime>(StringComparer.OrdinalIgnoreCase);
private readonly HashSet<string> _changedDuringSession =
new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private readonly HashSet<string> _reconciliationEmitted =
new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private int _stateEventsEmitted;
private bool _capNoticeEmitted;
// Test seam: replaces the live registry walk (same convention as EspTrackingInfoProbe).
internal static Func<ImeRegistrySnapshot>? SnapshotOverride;
internal sealed class ScopedSnapshotOverride : IDisposable
{
private readonly Func<ImeRegistrySnapshot>? _previous;
private int _disposed;
public ScopedSnapshotOverride(Func<ImeRegistrySnapshot> replacement)
{
_previous = SnapshotOverride;
SnapshotOverride = replacement;
}
public void Dispose()
{
if (System.Threading.Interlocked.Exchange(ref _disposed, 1) != 0) return;
SnapshotOverride = _previous;
}
}
public ImeRegistryAppStateObserver(
InformationalEventPost post,
AgentLogger? logger,
IClock clock,
Func<IReadOnlyList<AppPackageState>>? trackerStateProbe)
{
_post = post ?? throw new ArgumentNullException(nameof(post));
_clock = clock ?? throw new ArgumentNullException(nameof(clock));
_logger = logger;
_trackerStateProbe = trackerStateProbe;
}
/// <summary>
/// One observation pass: snapshot → diff → emit → reconcile. Called from the debounced
/// registry-change edge AND a periodic fallback tick (the settle-delay evaluation must
/// happen even when the registry goes quiet). Fail-soft: never throws.
/// </summary>
public void Tick(string reason)
{
try
{
lock (_sync)
{
var snapshot = SnapshotOverride?.Invoke() ?? ReadSnapshot();
var nowUtc = _clock.UtcNow;
if (_last == null)
{
// Silent baseline for the DIFF stream — see class doc. But the baseline
// itself is valuable evidence: on WDP the DPP-policy apps install in
// Batch 1 BEFORE the agent (bootstrap = Batch 2), so their terminal
// states exist ONLY here. One capped summary event closes that
// pre-agent gap without any log backfill (Oliver, 2026-08-18).
_last = snapshot;
_logger?.Info($"RegistryAppState: baseline captured ({snapshot.Entries.Count} app entries, trigger={reason})");
if (snapshot.Entries.Count > 0)
EmitBaselineSummary(snapshot, nowUtc);
return;
}
var changes = DiffSnapshots(_last, snapshot);
foreach (var change in changes)
{
_changedDuringSession.Add(change.Entry.Key);
EmitStateEvent(change, nowUtc);
}
TrackTerminalTransitions(snapshot, nowUtc);
EvaluateReconciliations(snapshot, nowUtc);
_last = snapshot;
}
}
catch (Exception ex)
{
_logger?.Debug($"RegistryAppState: tick failed (fail-soft): {ex.Message}");
}
}
// ── snapshot ────────────────────────────────────────────────────────────
internal static ImeRegistrySnapshot ReadSnapshot()
{
var snapshot = new ImeRegistrySnapshot();
using var baseKey = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64);
using var root = baseKey.OpenSubKey(ImeRootKeyPath);
if (root == null) return snapshot;
ReadWin32Apps(root, snapshot);
ReadEspTracking(root, snapshot);
ReadStatusServiceReports(root, snapshot);
return snapshot;
}
private static void ReadWin32Apps(RegistryKey root, ImeRegistrySnapshot snapshot)
{
using var win32Apps = root.OpenSubKey("Win32Apps");
if (win32Apps == null) return;
foreach (var userName in win32Apps.GetSubKeyNames())
{
if (!LooksLikeGuid(userName)) continue; // skips OperationalState / Reporting / ProvisioningProgress
using var userKey = win32Apps.OpenSubKey(userName);
if (userKey == null) continue;
foreach (var appKeyName in userKey.GetSubKeyNames())
{
var appId = ExtractAppId(appKeyName);
if (appId == null) continue; // skips GRS + non-app keys
using var appKey = userKey.OpenSubKey(appKeyName);
if (appKey == null) continue;
var entry = snapshot.GetOrAdd(userName, appId);
entry.ExitCode = TryReadInt(appKey.GetValue("ExitCode"));
entry.Intent = TryReadInt(appKey.GetValue("Intent"));
entry.LastUpdatedUtc = appKey.GetValue("LastUpdatedTimeUtc") as string;
using var esm = appKey.OpenSubKey("EnforcementStateMessage");
var json = esm?.GetValue("EnforcementStateMessage") as string;
if (!string.IsNullOrEmpty(json))
{
var (state, error) = ParseEnforcementStateMessage(json!);
entry.EnforcementState = state;
entry.ErrorCode = error;
}
}
}
}
private static void ReadEspTracking(RegistryKey root, ImeRegistrySnapshot snapshot)
{
using var esp = root.OpenSubKey("EspTrackingWin32Apps");
if (esp == null) return;
foreach (var userName in esp.GetSubKeyNames())
{
if (!LooksLikeGuid(userName)) continue;
using var userKey = esp.OpenSubKey(userName);
if (userKey == null) continue;
foreach (var appKeyName in userKey.GetSubKeyNames())
{
var appId = ExtractAppId(appKeyName);
if (appId == null) continue;
using var appKey = userKey.OpenSubKey(appKeyName);
var entry = snapshot.GetOrAdd(userName, appId);
entry.EspTracked = true;
entry.EspPhase = appKey?.GetValue("EspTrackingWin32AppPhase") as string;
}
}
}
private static void ReadStatusServiceReports(RegistryKey root, ImeRegistrySnapshot snapshot)
{
using var reports = root.OpenSubKey(@"SideCarPolicies\StatusServiceReports");
if (reports == null) return;
foreach (var userName in reports.GetSubKeyNames())
{
if (!LooksLikeGuid(userName)) continue;
using var userKey = reports.OpenSubKey(userName);
if (userKey == null) continue;
foreach (var appKeyName in userKey.GetSubKeyNames())
{
var appId = ExtractAppId(appKeyName);
if (appId == null) continue;
using var appKey = userKey.OpenSubKey(appKeyName);
if (appKey == null) continue;
var entry = snapshot.GetOrAdd(userName, appId);
entry.StatusServiceStatus = TryReadInt(appKey.GetValue("Status"));
}
}
}
// ── pure helpers (unit-tested directly) ─────────────────────────────────
internal static bool LooksLikeGuid(string name) =>
!string.IsNullOrEmpty(name) && Guid.TryParse(name, out _);
/// <summary>
/// Win32Apps app keys are <c>&lt;appGuid&gt;_&lt;revision&gt;</c>; StatusServiceReports keys
/// are the bare GUID. Returns the lowercase app GUID or null for non-app keys (GRS, ...).
/// </summary>
internal static string? ExtractAppId(string keyName)
{
if (string.IsNullOrEmpty(keyName)) return null;
var candidate = keyName.Length >= 36 ? keyName.Substring(0, 36) : keyName;
return Guid.TryParse(candidate, out var g) ? g.ToString("D") : null;
}
internal static (int? state, long? errorCode) ParseEnforcementStateMessage(string json)
{
try
{
using var doc = JsonDocument.Parse(json);
int? state = null;
long? error = null;
if (doc.RootElement.ValueKind == JsonValueKind.Object)
{
if (doc.RootElement.TryGetProperty("EnforcementState", out var s) && s.ValueKind == JsonValueKind.Number)
state = s.GetInt32();
if (doc.RootElement.TryGetProperty("ErrorCode", out var e) && e.ValueKind == JsonValueKind.Number)
error = e.GetInt64();
}
return (state, error);
}
catch
{
return (null, null);
}
}
internal static int? TryReadInt(object? value)
{
switch (value)
{
case null: return null;
case int i: return i;
case long l: return unchecked((int)l);
case string s when int.TryParse(s, NumberStyles.Integer, CultureInfo.InvariantCulture, out var parsed):
return parsed;
default: return null;
}
}
/// <summary>StateMessageEnforcementState bands (verified against decompiled IME 1.97/1.104).</summary>
internal static string ClassifyEnforcementState(int state)
{
if (state >= 1000 && state < 2000) return "success";
if (state >= 2000 && state < 3000) return "inProgress";
if (state >= 3000 && state < 4000) return "requirementsNotMet";
if (state >= 5000 && state < 6000) return "error";
if (state >= 6000 && state < 7000) return "notAttempted";
return "unknown";
}
/// <summary>AppInstallStatus (StatusService/ESP page): 1000er = installed, 3000 = failed.</summary>
internal static string? ClassifyStatusServiceStatus(int status)
{
if (status >= 1000 && status < 2000) return "installed";
if (status == 3000) return "failed";
if (status >= 2000 && status < 3000) return "notApplicable";
return null;
}
internal static List<AppRegistryChange> DiffSnapshots(ImeRegistrySnapshot previous, ImeRegistrySnapshot next)
{
var changes = new List<AppRegistryChange>();
foreach (var kv in next.Entries)
{
previous.Entries.TryGetValue(kv.Key, out var prev);
var changed = ChangedFields(prev, kv.Value);
if (changed.Count > 0)
changes.Add(new AppRegistryChange(kv.Value, changed, isNew: prev == null));
}
return changes;
}
internal static List<string> ChangedFields(AppRegistryEntry? prev, AppRegistryEntry next)
{
var changed = new List<string>();
if (prev?.EnforcementState != next.EnforcementState) changed.Add("enforcementState");
if (prev?.ErrorCode != next.ErrorCode) changed.Add("errorCode");
if (prev?.ExitCode != next.ExitCode) changed.Add("exitCode");
if (prev?.StatusServiceStatus != next.StatusServiceStatus) changed.Add("statusServiceStatus");
if ((prev?.EspTracked ?? false) != next.EspTracked) changed.Add("espTracked");
if (!string.Equals(prev?.EspPhase, next.EspPhase, StringComparison.OrdinalIgnoreCase)) changed.Add("espPhase");
return changed;
}
/// <summary>"success" / "error" when the registry outcome is terminal, else null.</summary>
internal static string? TerminalOutcome(AppRegistryEntry entry)
{
if (entry.EnforcementState is int es)
{
var cls = ClassifyEnforcementState(es);
if (cls == "success") return "success";
if (cls == "error") return "error";
}
if (entry.StatusServiceStatus is int ss)
{
var cls = ClassifyStatusServiceStatus(ss);
if (cls == "installed") return "success";
if (cls == "failed") return "error";
}
return null;
}
/// <summary>
/// Divergence rules (observability, conservative — prefer false-negative over noise):
/// registry error vs log Installed, registry success vs log Error, or registry terminal
/// while the tracker (actively tracking other apps) never saw the app at all.
/// </summary>
internal static bool IsDivergent(
string registryOutcome,
AppPackageState? logState,
bool trackerHasAnyApps,
out string reasonCode)
{
if (logState == null)
{
if (trackerHasAnyApps)
{
reasonCode = "app_unknown_to_log_tracking";
return true;
}
reasonCode = string.Empty; // tracker idle (e.g. pre-ESP) — not judgeable
return false;
}
if (registryOutcome == "error" && logState.InstallationState == AppInstallationState.Installed)
{
reasonCode = "registry_error_log_installed";
return true;
}
if (registryOutcome == "success" && logState.InstallationState == AppInstallationState.Error)
{
reasonCode = "registry_success_log_error";
return true;
}
reasonCode = string.Empty;
return false;
}
// ── emission ────────────────────────────────────────────────────────────
private void EmitStateEvent(AppRegistryChange change, DateTime nowUtc)
{
if (_stateEventsEmitted >= MaxStateEventsPerSession)
{
if (!_capNoticeEmitted)
{
_capNoticeEmitted = true;
_post.Emit(
eventType: SharedEventTypes.RegistryAppState,
source: SourceLabel,
message: $"registry_app_state cap reached ({MaxStateEventsPerSession}) — further registry app-state changes are logged locally only",
severity: EventSeverity.Warning,
occurredAtUtc: nowUtc);
}
_logger?.Debug($"RegistryAppState: cap reached, suppressing {change.Entry.AppId}");
return;
}
_stateEventsEmitted++;
var e = change.Entry;
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["appId"] = e.AppId,
["userContext"] = e.UserContext,
["changedFields"] = string.Join(",", change.ChangedFields),
};
if (e.EnforcementState is int es)
{
data["enforcementState"] = es.ToString(CultureInfo.InvariantCulture);
data["enforcementClass"] = ClassifyEnforcementState(es);
}
if (e.ErrorCode is long ec) data["errorCode"] = ec.ToString(CultureInfo.InvariantCulture);
if (e.ExitCode is int xc) data["exitCode"] = xc.ToString(CultureInfo.InvariantCulture);
if (e.Intent is int intent) data["intent"] = intent.ToString(CultureInfo.InvariantCulture);
if (e.StatusServiceStatus is int ss)
{
data["statusServiceStatus"] = ss.ToString(CultureInfo.InvariantCulture);
var cls = ClassifyStatusServiceStatus(ss);
if (cls != null) data["statusServiceClass"] = cls;
}
if (e.EspTracked) data["espTracked"] = "true";
if (!string.IsNullOrEmpty(e.EspPhase)) data["espPhase"] = e.EspPhase!;
var summary = data.TryGetValue("enforcementClass", out var encls) ? encls
: data.TryGetValue("statusServiceClass", out var sscls) ? sscls
: "updated";
_post.Emit(
eventType: SharedEventTypes.RegistryAppState,
source: SourceLabel,
message: $"Registry app state: {e.AppId} ({(e.UserContext == ImeRegistrySnapshot.DeviceContext ? "device" : "user")}) -> {summary}",
severity: EventSeverity.Info,
data: data,
occurredAtUtc: nowUtc);
}
// Baseline entries in the summary's typed payload are capped — a long-lived device
// with many historical Win32Apps entries must not produce an oversized event.
internal const int MaxBaselineSummaryApps = 50;
private void EmitBaselineSummary(ImeRegistrySnapshot snapshot, DateTime nowUtc)
{
int success = 0, error = 0, inProgress = 0, other = 0;
var apps = new List<Dictionary<string, string>>();
foreach (var entry in snapshot.Entries.Values)
{
var cls = entry.EnforcementState is int es ? ClassifyEnforcementState(es) : null;
switch (cls)
{
case "success": success++; break;
case "error": error++; break;
case "inProgress": inProgress++; break;
default: other++; break;
}
if (apps.Count >= MaxBaselineSummaryApps) continue;
var app = new Dictionary<string, string>(StringComparer.Ordinal)
{
["appId"] = entry.AppId,
["userContext"] = entry.UserContext,
};
if (entry.EnforcementState is int state)
{
app["enforcementState"] = state.ToString(CultureInfo.InvariantCulture);
app["enforcementClass"] = ClassifyEnforcementState(state);
}
if (entry.ErrorCode is long ec) app["errorCode"] = ec.ToString(CultureInfo.InvariantCulture);
if (entry.ExitCode is int xc) app["exitCode"] = xc.ToString(CultureInfo.InvariantCulture);
if (!string.IsNullOrEmpty(entry.LastUpdatedUtc)) app["lastUpdatedUtc"] = entry.LastUpdatedUtc!;
if (entry.EspTracked) app["espTracked"] = "true";
apps.Add(app);
}
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["totalApps"] = snapshot.Entries.Count.ToString(CultureInfo.InvariantCulture),
["successCount"] = success.ToString(CultureInfo.InvariantCulture),
["errorCount"] = error.ToString(CultureInfo.InvariantCulture),
["inProgressCount"] = inProgress.ToString(CultureInfo.InvariantCulture),
["otherCount"] = other.ToString(CultureInfo.InvariantCulture),
};
if (snapshot.Entries.Count > MaxBaselineSummaryApps)
data["appListTruncated"] = "true";
_post.Emit(
eventType: SharedEventTypes.RegistryAppBaseline,
source: SourceLabel,
message: $"Registry app baseline at agent start: {snapshot.Entries.Count} apps ({success} success, {error} error, {inProgress} in progress) — state that predates this agent run",
severity: EventSeverity.Info,
data: data,
occurredAtUtc: nowUtc,
typedPayload: new { apps });
}
private void TrackTerminalTransitions(ImeRegistrySnapshot snapshot, DateTime nowUtc)
{
foreach (var kv in snapshot.Entries)
{
if (!_changedDuringSession.Contains(kv.Key)) continue; // baseline state — not ours to judge
var outcome = TerminalOutcome(kv.Value);
if (outcome == null)
{
_terminalSinceUtc.Remove(kv.Key); // regressed to non-terminal — reset dwell
continue;
}
if (!_terminalSinceUtc.ContainsKey(kv.Key))
_terminalSinceUtc[kv.Key] = nowUtc;
}
}
private void EvaluateReconciliations(ImeRegistrySnapshot snapshot, DateTime nowUtc)
{
if (_trackerStateProbe == null) return;
List<AppPackageState>? trackerStates = null;
foreach (var kv in _terminalSinceUtc.ToList())
{
if (_reconciliationEmitted.Contains(kv.Key)) continue;
if (nowUtc - kv.Value < ReconcileSettleDelay) continue;
if (!snapshot.Entries.TryGetValue(kv.Key, out var entry)) continue;
var outcome = TerminalOutcome(entry);
if (outcome == null) continue;
trackerStates ??= (_trackerStateProbe() ?? Array.Empty<AppPackageState>()).ToList();
var logState = trackerStates.FirstOrDefault(p =>
string.Equals(p.Id, entry.AppId, StringComparison.OrdinalIgnoreCase));
if (!IsDivergent(outcome, logState, trackerStates.Count > 0, out var reasonCode))
{
_reconciliationEmitted.Add(kv.Key); // agreed (or not judgeable) — done with this app
continue;
}
_reconciliationEmitted.Add(kv.Key);
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["appId"] = entry.AppId,
["userContext"] = entry.UserContext,
["registryOutcome"] = outcome,
["reason"] = reasonCode,
["logState"] = logState?.InstallationState.ToString() ?? "unknown",
};
if (logState != null && !string.IsNullOrEmpty(logState.Name)) data["appName"] = logState.Name;
if (entry.EnforcementState is int es) data["enforcementState"] = es.ToString(CultureInfo.InvariantCulture);
if (entry.ErrorCode is long ec) data["errorCode"] = ec.ToString(CultureInfo.InvariantCulture);
_post.Emit(
eventType: SharedEventTypes.AppStateReconciliation,
source: SourceLabel,
message: $"Registry/log divergence for app {entry.AppId}: registry={outcome}, log={data["logState"]} ({reasonCode}) — IME log patterns may have drifted",
severity: EventSeverity.Warning,
immediateUpload: true,
data: data,
occurredAtUtc: nowUtc);
_logger?.Warning($"RegistryAppState: divergence for {entry.AppId} — registry={outcome}, log={data["logState"]} ({reasonCode})");
}
}
}
/// <summary>Immutable-ish snapshot of the three IME registry surfaces, keyed per user-context + app.</summary>
internal sealed class ImeRegistrySnapshot
{
public const string DeviceContext = "00000000-0000-0000-0000-000000000000";
public Dictionary<string, AppRegistryEntry> Entries { get; } =
new Dictionary<string, AppRegistryEntry>(StringComparer.OrdinalIgnoreCase);
public AppRegistryEntry GetOrAdd(string userContext, string appId)
{
var normalizedUser = Guid.TryParse(userContext, out var g) ? g.ToString("D") : userContext;
var key = normalizedUser + "|" + appId;
if (!Entries.TryGetValue(key, out var entry))
{
entry = new AppRegistryEntry(key, normalizedUser, appId);
Entries[key] = entry;
}
return entry;
}
}
internal sealed class AppRegistryEntry
{
public AppRegistryEntry(string key, string userContext, string appId)
{
Key = key;
UserContext = userContext;
AppId = appId;
}
public string Key { get; }
public string UserContext { get; }
public string AppId { get; }
public string? LastUpdatedUtc { get; set; }
public int? EnforcementState { get; set; }
public long? ErrorCode { get; set; }
public int? ExitCode { get; set; }
public int? Intent { get; set; }
public int? StatusServiceStatus { get; set; }
public bool EspTracked { get; set; }
public string? EspPhase { get; set; }
}
internal sealed class AppRegistryChange
{
public AppRegistryChange(AppRegistryEntry entry, List<string> changedFields, bool isNew)
{
Entry = entry;
ChangedFields = changedFields;
IsNew = isNew;
}
public AppRegistryEntry Entry { get; }
public List<string> ChangedFields { get; }
public bool IsNew { get; }
}
}
@@ -66,6 +66,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
internal const string AutopilotErrorBackfillStateFileName = "autopilot-error-backfill.json";
/// <summary>
/// EventIDs that are STRUCTURAL noise on a Device Preparation session and are
/// suppressed there entirely (afee7ae0: 900+ occurrences of 1005 plus 400+ of
/// 417/418 in a nine-minute session, all expected): 100 "Autopilot policy not
/// found" (no deployment profile exists on WDP by design), 417/418 "duplicate
/// deployment workload/batch" (BootstrapperAgent SLDM progress-poll chatter),
/// 1005 ManagementService WIL error storm. Genuine WDP diagnostics (e.g. 408
/// provisioning warnings) keep flowing; Critical (level 1) records are never
/// suppressed. Classic sessions are unaffected — they keep the configurable
/// harmless-downgrade + rollup path instead.
/// </summary>
internal static readonly HashSet<int> DevicePreparationNoiseEventIds = new HashSet<int> { 100, 417, 418, 1005 };
/// <summary>
/// Per-EventId emission caps for harmless-downgraded events (session 8bc1180f,
/// 2026-06-12). Windows can dump hundreds of identical EventID-100 "Autopilot policy
@@ -93,6 +106,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly HashSet<int> _harmlessEventIds;
private readonly bool _isDevicePreparation;
private EventLogWatcher _autopilotWatcher;
private EventLogWatcher _managementWatcher;
@@ -131,7 +145,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
bool backfillEnabled = true,
int backfillLookbackMinutes = 30,
string stateDirectory = null,
int[] harmlessEventIds = null)
int[] harmlessEventIds = null,
bool isDevicePreparation = false)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
@@ -140,6 +155,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_logLevelMax = logLevelMax;
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_isDevicePreparation = isDevicePreparation;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_harmlessEventIds = harmlessEventIds != null && harmlessEventIds.Length > 0
? new HashSet<int>(harmlessEventIds)
@@ -166,7 +182,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
if (_backfillEnabled)
{
BackfillTargetedEvents();
BackfillAutopilotErrors();
// WDP gate (afee7ae0): the error backfill replays ZTD registration /
// attestation errors (807/809/815/908) — on Device Preparation the 807
// ZtdDeviceIsNotRegistered answer is the EXPECTED state, not a failure.
if (!_isDevicePreparation)
{
BackfillAutopilotErrors();
}
else
{
_logger.Info("ModernDeployment: Device Preparation session — Autopilot error backfill skipped (ZTD errors are structural there)");
}
}
else
{
@@ -393,6 +419,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
var effectiveLevel = level ?? 4;
// WDP structural-noise gate — see DevicePreparationNoiseEventIds. Sits before
// the severity mapping so suppressed records also skip FormatDescription().
// Critical (1) and LogAlways (0) records always pass.
if (_isDevicePreparation
&& effectiveLevel >= 2
&& DevicePreparationNoiseEventIds.Contains(eventId))
{
return EventVerdict.Suppress;
}
switch (effectiveLevel)
{
case 1:
@@ -107,7 +107,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private bool _deviceSetupProvisioningCompleteFired;
// Fire-once guard for AccountSetupProvisioningComplete event (session 330f73f3 fix)
private bool _accountSetupProvisioningCompleteFired;
// WhiteGlove confirmation: DeviceSetup registry contains SaveWhiteGloveSuccessResult=succeeded
// DeviceSetup registry contains SaveWhiteGloveSuccessResult=succeeded. NOT a WhiteGlove
// indicator on Win11 25H2+ (see HasSaveWhiteGloveSuccessResult) — observability only.
private bool _saveWhiteGloveSuccessResultSeen;
// Track SaveWhiteGloveSuccessResult state transitions for observability (null → notStarted → succeeded)
private string _lastSaveWhiteGloveState;
@@ -226,8 +227,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// <summary>
/// True when DeviceSetup registry JSON contains a SaveWhiteGloveSuccessResult property
/// with subcategoryState=succeeded. This is a definitive WhiteGlove (Pre-Provisioning)
/// confirmation signal — Windows only writes this property during White Glove flows.
/// with subcategoryState=succeeded. NOT a WhiteGlove indicator: on Win11 25H2 (build
/// 26200+) Windows writes this property as a trivially-succeeding step in EVERY
/// enrollment (validated 2026-08-18 — user-driven sessions f475e697/c601a24b/4219203f
/// all carry it). Kept for observability only (whiteglove_signal_* raw dumps); must not
/// be used for classification. Reliable WhiteGlove anchors are ModernDeployment
/// EventID 509 and the Shell-Core WhiteGlove-success events.
/// </summary>
public bool HasSaveWhiteGloveSuccessResult
{
@@ -891,10 +896,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string categoryStatusMessage = SafeGetString(root, "categoryStatusMessage");
var subcategories = ParseSubcategories(root);
// Detect WhiteGlove signal in DeviceSetup category.
// SaveWhiteGloveSuccessResult is NOT a *Subcategory-suffixed property, so
// ParseSubcategories skips it. We scan the raw JSON explicitly.
// Track state transitions for full observability (notStarted → succeeded).
// Track the SaveWhiteGloveSuccessResult transitions in the DeviceSetup
// category (observability only — NOT a WhiteGlove indicator on 25H2+, see
// HasSaveWhiteGloveSuccessResult). It is NOT a *Subcategory-suffixed
// property, so ParseSubcategories skips it; we scan the raw JSON explicitly.
if (string.Equals(categoryLabel, "DeviceSetup", StringComparison.OrdinalIgnoreCase))
{
foreach (var prop in root.EnumerateObject())
@@ -919,7 +924,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
_saveWhiteGloveSuccessResultSeen = true;
_logger.Info("ProvisioningStatusTracker: SaveWhiteGloveSuccessResult=succeeded " +
"— WhiteGlove confirmation signal");
"(observability only — not a WhiteGlove indicator on 25H2+)");
}
}
break;
@@ -52,6 +52,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly HashSet<string> _sources;
private readonly int _sessionStalledAfterProbeIndex;
private readonly HashSet<int> _harmlessModernDeploymentEventIds;
private readonly bool _isDevicePreparation;
private readonly object _stateLock = new object();
private readonly HashSet<int> _firedProbeIndices = new HashSet<int>();
@@ -151,7 +152,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
int[] traceIndices,
string[] sources,
int sessionStalledAfterProbeIndex,
int[] harmlessModernDeploymentEventIds = null)
int[] harmlessModernDeploymentEventIds = null,
bool isDevicePreparation = false)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
@@ -161,10 +163,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_traceIndices = new HashSet<int>(traceIndices ?? new int[0]);
_sources = new HashSet<string>(sources ?? new string[0], StringComparer.OrdinalIgnoreCase);
_sessionStalledAfterProbeIndex = sessionStalledAfterProbeIndex;
_isDevicePreparation = isDevicePreparation;
_harmlessModernDeploymentEventIds = new HashSet<int>(
harmlessModernDeploymentEventIds != null && harmlessModernDeploymentEventIds.Length > 0
? harmlessModernDeploymentEventIds
: DefaultHarmlessModernDeploymentEventIds);
// WDP gate (afee7ae0): the structural WDP noise IDs (417/418 duplicate-workload
// chatter, 1005 WIL storm, ...) must never count as stall anomalies on Device
// Preparation, independent of what the tenant-configured harmless list contains.
// The channels themselves stay scanned — WDP's provisioning stack logs genuine
// diagnostics (e.g. 408) into the very same Autopilot channel.
if (_isDevicePreparation)
{
_harmlessModernDeploymentEventIds.UnionWith(ModernDeploymentTracker.DevicePreparationNoiseEventIds);
}
}
/// <summary>
@@ -244,7 +257,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
result.DurationMs = (int)(DateTime.UtcNow - started).TotalMilliseconds;
// Always log locally for diagnostics — the agent_YYYYMMDD.log is the always-on observability.
// Always log locally for diagnostics — agent.log is the always-on observability.
var anomaliesStr = result.Anomalies.Count > 0
? $"anomalies=[{string.Join(",", result.Anomalies)}]"
: "quiet";
@@ -368,6 +381,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private void ScanProvisioningRegistry(ProbeResult result)
{
// Device Preparation writes no ESP category values — the three Status JSONs
// below are ESP-only constructs, so the scan can never find anything there.
if (_isDevicePreparation)
return;
using (var key = Registry.LocalMachine.OpenSubKey(@"SOFTWARE\Microsoft\Provisioning\AutopilotSettings", writable: false))
{
if (key == null)
@@ -972,7 +972,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
/// <summary>
/// Appends a log line to the main agent log file (logger isn't initialized yet at update time).
/// Uses the same date-based naming and format as AgentLogger: agent_YYYYMMDD.log
/// Uses the same naming and format as AgentLogger: agent.log
/// </summary>
private static void LogToFile(string message)
{
@@ -980,8 +980,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
{
var logDir = Environment.ExpandEnvironmentVariables(Constants.LogDirectory);
Directory.CreateDirectory(logDir);
var logFileName = $"agent_{DateTime.Now:yyyyMMdd}.log";
var logPath = Path.Combine(logDir, logFileName);
var logPath = Path.Combine(logDir, "agent.log");
File.AppendAllText(logPath, $"[{DateTime.UtcNow:yyyy-MM-dd HH:mm:ss.fff}] [INFO] {message}{Environment.NewLine}");
}
catch
@@ -275,14 +275,34 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
// Include enrollment type in autopilot_profile event data
data["enrollmentType"] = detectedType;
// Second diagnostic surface (independent of AutopilotPolicyCache): Microsoft's
// documented Diagnostics\Autopilot key (IsAutopilotDisabled, TenantMatched,
// CloudAssignedTenantDomain/Id, ...). See docs/agent/autopilot-ztd-diagnostics.md.
var diagnosticsRegistry = ZtdEvidence.ReadDiagnosticsRegistry(_logger);
if (diagnosticsRegistry != null)
data["diagnosticsRegistry"] = diagnosticsRegistry;
// Device Preparation (WDP) has no Autopilot deployment profile by design:
// Windows still runs the ZTD policy download during OOBE and caches the 807
// ZtdDeviceIsNotRegistered answer, so without this gate every WDP session
// emits an autopilot_profile dump plus a misleading "no deployment profile
// was assigned" warning and burns a ZTD event-log query + HTTP probe on
// evidence that answers a question nobody asked (afee7ae0). Gated on the
// deterministic WDP rule only — the CloudAssigned* fallback rules imply a
// real profile (just ESP-less), where the dump stays meaningful. The
// enrollment_type_detected event below still carries the rail decision.
var isDevicePreparation = detectionRule ==
AutopilotMonitor.Agent.V2.Core.Security.EnrollmentRegistryDetector.DevicePreparationExecutionContextRule;
EmitDeviceInfoEvent(Constants.EventTypes.AutopilotProfile, "Autopilot profile configuration", data);
Dictionary<string, object> diagnosticsRegistry = null;
if (!isDevicePreparation)
{
// Second diagnostic surface (independent of AutopilotPolicyCache): Microsoft's
// documented Diagnostics\Autopilot key (IsAutopilotDisabled, TenantMatched,
// CloudAssignedTenantDomain/Id, ...). See docs/agent/autopilot-ztd-diagnostics.md.
diagnosticsRegistry = ZtdEvidence.ReadDiagnosticsRegistry(_logger);
if (diagnosticsRegistry != null)
data["diagnosticsRegistry"] = diagnosticsRegistry;
EmitDeviceInfoEvent(Constants.EventTypes.AutopilotProfile, "Autopilot profile configuration", data);
}
else
{
_logger.Info("EnrollmentTracker: Device Preparation session — skipping autopilot_profile/ZTD evidence (no deployment profile exists on WDP)");
}
// ProfileAvailable=0 means the ZTD service returned no profile for this device
// during OOBE. Sessions only exist after the backend's device-registration
@@ -293,7 +313,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
// Autopilot session — surface the edge case explicitly, with a ZTD verdict
// backed by the ModernDeployment Autopilot event log plus an endpoint probe
// (see docs/agent/autopilot-ztd-diagnostics.md for the evidence model).
if (IsAutopilotProfileMissing(data))
if (!isDevicePreparation && IsAutopilotProfileMissing(data))
{
// On Windows 365 Cloud PCs a missing profile is the expected state — Cloud PCs
// are provisioned by the Windows 365 service, not by Autopilot — so the event
@@ -787,10 +807,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.DeviceInfo
data["espTrackedModernCount"] = tracking.ModernCount;
}
var summary = BuildSummary(snapshot, tracking);
_logger.Info($"EnrollmentTracker: ESP configuration detected — {summary}");
EmitDeviceInfoEvent(Constants.EventTypes.EspConfigDetected, $"ESP configuration: {summary}", data);
PostEspConfigDetectedSignal(snapshot);
// No FirstSync value and no ESP tracking list means there is no ESP evidence at
// all (Device Preparation, plain Entra join, pre-sync timing) — an event that
// says nothing but "unknown/unknown" is noise, so skip it. Mirrors the null-gate
// the orchestrator bootstrap already applies to the decision signal
// (EnrollmentOrchestrator.PostEspConfigDetectedBootstrap).
var hasAnyEspEvidence = snapshot.SkipUser.HasValue
|| snapshot.SkipDevice.HasValue
|| snapshot.BlockInStatusPage.HasValue
|| snapshot.SyncFailureTimeoutMinutes.HasValue
|| tracking.HasData;
if (hasAnyEspEvidence)
{
var summary = BuildSummary(snapshot, tracking);
_logger.Info($"EnrollmentTracker: ESP configuration detected — {summary}");
EmitDeviceInfoEvent(Constants.EventTypes.EspConfigDetected, $"ESP configuration: {summary}", data);
PostEspConfigDetectedSignal(snapshot);
}
else
{
_logger.Debug("EnrollmentTracker: no ESP configuration evidence (FirstSync empty, no tracking lists) — esp_config_detected suppressed");
}
}
catch (Exception ex)
{
@@ -300,7 +300,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office
{
_state = DetectorState.Terminal;
EmitLifecycle(Constants.EventTypes.OfficeInstallCompleted, snap, EventSeverity.Info, "Completed", isTerminal: true, coreBinariesPresent: true);
PersistState(OfficeInstallStateData.StateCompleted);
PersistState(OfficeInstallStateData.StateCompleted, snap);
return CompletionOutcome.Completed;
}
@@ -405,7 +405,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office
if (_state == DetectorState.Terminal) return;
_state = DetectorState.Terminal;
EmitLifecycle(Constants.EventTypes.OfficeInstallFailed, snap, EventSeverity.Error, "Failed", isTerminal: true);
PersistState(OfficeInstallStateData.StateFailed);
PersistState(OfficeInstallStateData.StateFailed, snap);
}
/// <summary>
@@ -444,15 +444,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office
/// (the resume is what delivers a completion missed across a mid-install reboot).
/// Caller holds the lock.
/// </summary>
private void PersistState(string state)
private void PersistState(string state, OfficeC2RSnapshot? snap = null)
{
if (_statePersistence == null) return;
_lastStatePersistUtc = _clock.UtcNow;
var isTerminal = state == OfficeInstallStateData.StateCompleted
|| state == OfficeInstallStateData.StateFailed;
_statePersistence.Save(new OfficeInstallStateData
{
State = state,
StartedAtUtc = _startedAtUtc,
StartedTrigger = _startedTrigger,
// Terminal-only metadata for the summary dialog's Office row (duration + version).
CompletedAtUtc = isTerminal ? _clock.UtcNow : (DateTime?)null,
VersionReached = isTerminal ? snap?.VersionToReport : null,
PeakDo = OfficeDoPeakData.FromSample(_peakDo),
});
}
@@ -105,6 +105,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office
public DateTime? StartedAtUtc { get; set; }
public string? StartedTrigger { get; set; }
/// <summary>Stamped only on the two terminal persists (Completed/Failed) — basis for the
/// summary dialog's Office row duration. Null on Active/Preinstalled and on state files
/// written by older agents (fail-soft deserialization).</summary>
public DateTime? CompletedAtUtc { get; set; }
/// <summary>Office build reached at the terminal transition (snapshot's VersionToReport).
/// Null when the snapshot had none or on older state files.</summary>
public string? VersionReached { get; set; }
/// <summary>Highest-bytes DO sample seen — basis for the doSummary on a resumed completion.</summary>
public OfficeDoPeakData? PeakDo { get; set; }
@@ -22,11 +22,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// Pressure thresholds — once either is crossed within a session, a one-shot
// `spool_pressure_detected` event is emitted with RequiresImmediateFlush=true
// so the backend can surface the condition without waiting for the next snapshot
// to be queried out of Storage. Sized so a healthy 6h provisioning session never
// trips them; tripping is a strong "upload backlog is growing or downstream stalled"
// signal worth investigating in the field.
// to be queried out of Storage. Both measure the PENDING backlog (items not yet
// marked uploaded), never the append-only spool file size — a healthy long session
// grows the file past any fixed size while its backlog stays near zero (field data
// 2026-08: 99/100 pressure events were file-size-only false positives). Tripping is
// therefore a genuine "upload is stalled or falling behind" signal.
internal const int PressurePendingItemThreshold = 2000;
internal const long PressureFileSizeBytesThreshold = 5L * 1024 * 1024; // 5 MB
internal const long PressurePendingBytesThreshold = 5L * 1024 * 1024; // 5 MB
private readonly string _agentVersion;
private readonly NetworkMetrics _networkMetrics;
@@ -77,11 +79,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
protected override void Collect()
{
// Full path writes 19 keys: agent_version + process metrics (5: cpu, ws, private,
// threads, handles) + spool stats (4) + network delta (9: requests, failures,
// bytes_up/down, avg_latency, total_up/down/requests/latency). cap=19 →
// HashHelpers.GetPrime(19)=23 buckets → no resize on the 19th key.
var data = new Dictionary<string, object>(capacity: 19, StringComparer.Ordinal)
// Full path writes 20 keys: agent_version + process metrics (5: cpu, ws, private,
// threads, handles) + spool stats (5) + network delta (9: requests, failures,
// bytes_up/down, avg_latency, total_up/down/requests/latency). cap=20 →
// HashHelpers.GetPrime(20)=23 buckets → no resize on the 20th key.
var data = new Dictionary<string, object>(capacity: 20, StringComparer.Ordinal)
{
{ "agent_version", _agentVersion }
};
@@ -126,17 +128,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
// EnrollmentOrchestrator.Start into IComponentFactory.CreateCollectorHosts
// (ARCH-F4). Null only on test fakes that don't construct a real spool.
int pendingItemCount = 0;
long pendingBytes = 0L;
long spoolFileSizeBytes = 0L;
if (_telemetrySpool != null)
{
try
{
pendingItemCount = _telemetrySpool.PendingItemCount;
pendingBytes = _telemetrySpool.PendingBytes;
spoolFileSizeBytes = _telemetrySpool.SpoolFileSizeBytes;
var peakPending = _telemetrySpool.PeakPendingItemCount;
var totalEnqueued = _telemetrySpool.LastAssignedItemId + 1; // -1 sentinel → 0
data["spool_pending_item_count"] = pendingItemCount;
data["spool_pending_bytes"] = pendingBytes;
data["spool_peak_pending_item_count"] = peakPending;
data["spool_file_size_bytes"] = spoolFileSizeBytes;
data["spool_total_enqueued_count"] = totalEnqueued;
@@ -194,20 +199,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
});
}
// One-shot pressure event \u2014 fires once per session when the spool grows past
// either threshold. ImmediateUpload=true so it shows up promptly on the
// backend without waiting for the next batch drain.
// One-shot pressure event \u2014 fires once per session when the pending backlog
// grows past either threshold. ImmediateUpload=true so it shows up promptly on
// the backend without waiting for the next batch drain.
if (_telemetrySpool != null
&& (pendingItemCount > PressurePendingItemThreshold
|| spoolFileSizeBytes > PressureFileSizeBytesThreshold)
|| pendingBytes > PressurePendingBytesThreshold)
&& Interlocked.CompareExchange(ref _pressureEmitted, 1, 0) == 0)
{
var pressureData = new Dictionary<string, object>(capacity: 8, StringComparer.Ordinal)
{
{ "pendingItemCount", pendingItemCount },
{ "fileSizeBytes", spoolFileSizeBytes },
{ "pendingBytes", pendingBytes },
{ "fileSizeBytes", spoolFileSizeBytes }, // informational only, not a trigger
{ "pendingThreshold", PressurePendingItemThreshold },
{ "fileSizeThresholdBytes", PressureFileSizeBytesThreshold },
{ "pendingBytesThreshold", PressurePendingBytesThreshold },
{ "totalEnqueuedCount", _telemetrySpool.LastAssignedItemId + 1 },
{ "lastUploadedItemId", _telemetrySpool.LastUploadedItemId },
{ "ImmediateUpload", true }
@@ -222,15 +228,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
Severity = EventSeverity.Warning,
Source = "AgentSelfMetricsCollector",
Phase = EnrollmentPhase.Unknown,
Message = $"Telemetry spool pressure detected: pending={pendingItemCount}, " +
$"fileBytes={spoolFileSizeBytes} \u2014 upload may be stalled or session " +
$"is unusually long.",
Message = $"Telemetry spool pressure detected: pending={pendingItemCount} items / " +
$"{pendingBytes} bytes not yet uploaded \u2014 upload is stalled or " +
$"falling behind.",
Data = pressureData
});
Logger.Warning(
$"AgentSelfMetricsCollector: spool pressure detected " +
$"(pending={pendingItemCount}, fileBytes={spoolFileSizeBytes}). " +
$"(pending={pendingItemCount}, pendingBytes={pendingBytes}). " +
$"Emitted spool_pressure_detected (one-shot per session).");
}
}
@@ -404,12 +404,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
if (state.Stage.IsTerminal()) return false;
if (state.Stage == SessionStage.Unknown || state.Stage == SessionStage.SessionStarted) return false;
if (state.AccountSetupEnteredUtc == null) return false;
// WDP (afee7ae0): Device Preparation has no ESP, so neither the AccountSetup
// anchor nor an ESP final exit is guaranteed to exist — its dead-end zone
// begins at the real-user desktop instead (from there only Hello or a deadline
// can move the session). Normally the DevicePrepCompletion backstop armed at
// desktop arrival satisfies the invariant; this branch makes a regression visible.
var isDevicePreparation = state.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation;
if (state.AccountSetupEnteredUtc == null && !isDevicePreparation) return false;
var deadEndZoneEntered =
state.EspAdvisoryFailureRecordedUtc != null
|| (state.EspFinalExitUtc != null
&& state.EspFinalExitUtc.Value >= state.AccountSetupEnteredUtc.Value);
|| (state.AccountSetupEnteredUtc != null
&& state.EspFinalExitUtc != null
&& state.EspFinalExitUtc.Value >= state.AccountSetupEnteredUtc.Value)
|| (isDevicePreparation && state.DesktopArrivedUtc != null);
if (!deadEndZoneEntered) return false;
foreach (var deadline in state.Deadlines)
@@ -431,8 +440,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
["stage"] = state.Stage.ToString(),
["stepIndex"] = state.StepIndex.ToString(CultureInfo.InvariantCulture),
["signalOrdinal"] = signal.SessionSignalOrdinal.ToString(CultureInfo.InvariantCulture),
["accountSetupEnteredUtc"] = state.AccountSetupEnteredUtc!.Value
.ToString("o", CultureInfo.InvariantCulture),
// Null on the WDP branch of IsParkedWithoutDeadline (no ESP anchor there).
["accountSetupEnteredUtc"] = state.AccountSetupEnteredUtc?.Value
.ToString("o", CultureInfo.InvariantCulture) ?? "",
["signalsSeen"] = string.Join(",", census.SignalsSeen),
["armedDeadlines"] = string.Join(",", state.Deadlines.Select(d => d.Name)),
["dwellSeconds"] = _parkedTripwireDwell.TotalSeconds.ToString("F0", CultureInfo.InvariantCulture),
@@ -109,6 +109,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
var hosts = new List<ICollectorHost>();
var collectors = _remoteConfig.Collectors ?? CollectorConfiguration.CreateDefault();
// WDP scenario gate (afee7ae0, 2026-08-18): Device Preparation has no ESP, so the
// ESP-only machinery below (provisioning-status registry watcher, ESP policy-provider
// stall tripwire, Autopilot-channel noise) burns cycles on registry keys that never
// exist there — or worse, wakes up on every WDP progress write. Gated on the
// DETERMINISTIC WDP marker only (BootstrapperAgent ExecutionContext, present before
// the agent starts); the CloudAssigned* fallback rules keep full Classic behavior.
var isDevicePreparation = Security.EnrollmentRegistryDetector.IsDeterministicDevicePreparation();
if (isDevicePreparation)
{
logger.Info("DefaultComponentFactory: Device Preparation session (deterministic marker) — ESP-only hosts and Autopilot-channel noise are gated");
}
// ----- Kernel hosts (always-on; they produce decision signals) --------------------
// Single-rail refactor (plan §5.8) — DeviceInfoCollector existed in V2.Core but had
@@ -151,6 +163,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
modernDeploymentBackfillLookbackMinutes: collectors.ModernDeploymentBackfillLookbackMinutes,
modernDeploymentHarmlessEventIds: collectors.ModernDeploymentHarmlessEventIds,
stateDirectory: _stateDirectory,
isDevicePreparation: isDevicePreparation,
userEspAppsSettledProbe: () => imeLogHostRef?.AreUserEspAppsSettled() == true,
// Liveness plan PR3: starved-apps probe over the same lazily-assigned IME host.
starvedUserEspAppsProbe: () =>
@@ -308,7 +321,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
traceIndices: collectors.StallProbeTraceIndices,
sources: collectors.StallProbeSources,
sessionStalledAfterProbeIndex: collectors.SessionStalledAfterProbeIndex,
harmlessModernDeploymentEventIds: collectors.ModernDeploymentHarmlessEventIds));
harmlessModernDeploymentEventIds: collectors.ModernDeploymentHarmlessEventIds,
isDevicePreparation: isDevicePreparation));
}
// ESP policy-provider stall tripwire — always-on kernel host (no config gate, like
@@ -317,14 +331,32 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// TrackingPoliciesCreated=1 the ESP ignores foreign provider names), parks the user
// ESP at "Apps (Identifying)" WITHOUT any OS timeout. Wall-clock dwell, deliberately
// outside the StallProbe gate: the idle clock there resets on any session activity,
// and the stall must accrue regardless of it.
hosts.Add(new EspPolicyProviderStallHost(
// and the stall must accrue regardless of it. Not created on Device Preparation —
// the EnrollmentStatusTracking key it polls every 60s is an ESP-only construct.
if (!isDevicePreparation)
{
hosts.Add(new EspPolicyProviderStallHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
startupGate: _startupEventGate));
}
// Registry second pillar (audit 2026-08-17): IME's authoritative per-app state from
// HKLM\...\IntuneManagementExtension (Win32Apps EnforcementStateMessage,
// EspTrackingWin32Apps, StatusServiceReports) via snapshot-and-diff. Emits
// registry_app_state on real changes and app_state_reconciliation when the
// registry outcome diverges from the IME-log-derived state (= built-in pattern-drift
// alarm). Always-on observability host; the tracker probe is read-only.
hosts.Add(new ImeRegistryAppStateHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
startupGate: _startupEventGate));
trackerStateProbe: () => imeLogHost.AllKnownPackageStates));
// Deterministic update corroboration — compares the persisted OS build (CurrentBuild.UBR)
// across agent restarts and emits os_build_changed when it differs (session 7443317c:
@@ -70,6 +70,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
int modernDeploymentBackfillLookbackMinutes,
int[]? modernDeploymentHarmlessEventIds,
string stateDirectory,
bool isDevicePreparation = false,
Func<bool>? userEspAppsSettledProbe = null,
Func<System.Collections.Generic.IReadOnlyList<Monitoring.Enrollment.Ime.AppPackageState>>? starvedUserEspAppsProbe = null,
Func<System.Collections.Generic.IReadOnlyList<Monitoring.Enrollment.Ime.AppPackageState>>? packageStatesProbe = null)
@@ -89,6 +90,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
modernDeploymentBackfillLookbackMinutes: modernDeploymentBackfillLookbackMinutes,
stateDirectory: stateDirectory,
modernDeploymentHarmlessEventIds: modernDeploymentHarmlessEventIds,
isDevicePreparation: isDevicePreparation,
userEspAppsSettledProbe: userEspAppsSettledProbe,
starvedUserEspAppsProbe: starvedUserEspAppsProbe,
packageStatesProbe: packageStatesProbe);
@@ -0,0 +1,149 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Microsoft.Win32;
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Hosts the <see cref="ImeRegistryAppStateObserver"/> (registry second pillar, audit
/// 2026-08-17): one recursive <see cref="RegistryWatcher"/> on the IME root — all three
/// observed surfaces (Win32Apps, EspTrackingWin32Apps, SideCarPolicies\StatusServiceReports)
/// live under it — debounced into snapshot-and-diff ticks, plus a 60-s periodic fallback
/// tick. The fallback both evaluates the reconciliation settle-delay when the registry goes
/// quiet AND keeps the pillar alive if the watcher fails to arm (poll-only degraded mode,
/// announced once via <c>collector_degraded</c>). Always-on observability host, no config
/// gate — precedent: <see cref="EspPolicyProviderStallHost"/>.
/// </summary>
internal sealed class ImeRegistryAppStateHost : ICollectorHost
{
public string Name => "ImeRegistryAppState";
private static readonly TimeSpan DebounceDelay = TimeSpan.FromSeconds(2);
private static readonly TimeSpan PeriodicInterval = TimeSpan.FromSeconds(60);
private readonly ImeRegistryAppStateObserver _observer;
private readonly InformationalEventPost _post;
private readonly AgentLogger? _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private RegistryWatcher? _watcher;
private Timer? _debounceTimer;
private Timer? _periodicTimer;
private int _stopped;
public ImeRegistryAppStateHost(
string sessionId,
string tenantId,
AgentLogger? logger,
ISignalIngressSink ingress,
IClock clock,
Func<IReadOnlyList<AppPackageState>>? trackerStateProbe)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
_sessionId = sessionId;
_tenantId = tenantId;
_logger = logger;
_post = new InformationalEventPost(ingress, clock);
_observer = new ImeRegistryAppStateObserver(_post, logger, clock, trackerStateProbe);
}
public void Start()
{
// Baseline immediately: pre-existing registry state must be captured silently
// BEFORE the first change edge, or stale entries would replay as fresh events.
_observer.Tick("baseline");
try
{
_watcher = new RegistryWatcher(
RegistryHive.LocalMachine,
ImeRegistryAppStateObserver.ImeRootKeyPath,
watchSubtree: true,
view: RegistryView.Registry64,
filter: RegistryNativeMethods.RegChangeNotifyFilter.Name
| RegistryNativeMethods.RegChangeNotifyFilter.LastSet
| RegistryNativeMethods.RegChangeNotifyFilter.ThreadAgnostic,
trace: msg => _logger?.Debug($"ImeRegistryAppState watcher: {msg}"));
_watcher.Changed += (_, __) => ScheduleDebouncedTick();
_watcher.Error += (_, ex) =>
{
_logger?.Warning($"ImeRegistryAppState watcher error (poll-only fallback continues): {ex.Message}");
};
_watcher.Start();
}
catch (Exception ex)
{
// Watcher arm failure degrades to poll-only (periodic tick below) — announce once.
_logger?.Warning($"ImeRegistryAppState: watcher arm failed, running poll-only: {ex.Message}");
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId, Name, "watcher_arm_failed", ex);
_watcher?.Dispose();
_watcher = null;
}
_periodicTimer = new Timer(_ => SafeTick("periodic"), null, PeriodicInterval, PeriodicInterval);
_logger?.Info($"ImeRegistryAppStateHost started (watcher={(_watcher != null ? "armed" : "poll-only")})");
}
private void ScheduleDebouncedTick()
{
if (Volatile.Read(ref _stopped) != 0) return;
// One trailing-edge tick per change burst; RegistryWatcher coalesces anyway and the
// observer snapshot-diffs, so collapsing bursts loses nothing.
var timer = _debounceTimer;
if (timer == null)
{
var created = new Timer(_ => SafeTick("registry_change"), null, DebounceDelay, Timeout.InfiniteTimeSpan);
if (Interlocked.CompareExchange(ref _debounceTimer, created, null) != null)
created.Dispose();
else
return;
timer = _debounceTimer;
}
try { timer?.Change(DebounceDelay, Timeout.InfiniteTimeSpan); }
catch (ObjectDisposedException) { /* stopping */ }
}
private void SafeTick(string reason)
{
if (Volatile.Read(ref _stopped) != 0) return;
try
{
_observer.Tick(reason);
}
catch (Exception ex)
{
// Observer.Tick is itself fail-soft; this is the last-resort belt.
_logger?.Debug($"ImeRegistryAppState tick failed: {ex.Message}");
}
}
public void Stop()
{
if (Interlocked.Exchange(ref _stopped, 1) != 0) return;
// Never Stop() the watcher from inside its own callback — we're outside here,
// but RequestStop first keeps the contract obvious.
try { _watcher?.RequestStop(); } catch { /* fail-soft */ }
try { _watcher?.Dispose(); } catch { /* fail-soft */ }
_watcher = null;
_debounceTimer?.Dispose();
_debounceTimer = null;
_periodicTimer?.Dispose();
_periodicTimer = null;
}
public void Dispose() => Stop();
}
}
@@ -57,7 +57,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
int[]? traceIndices,
string[]? sources,
int sessionStalledAfterProbeIndex,
int[]? harmlessModernDeploymentEventIds)
int[]? harmlessModernDeploymentEventIds,
bool isDevicePreparation = false)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
@@ -72,7 +73,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
traceIndices: traceIndices ?? new[] { 2 },
sources: sources ?? new[] { "provisioning_registry", "diagnostics_registry", "eventlog", "appworkload_log" },
sessionStalledAfterProbeIndex: sessionStalledAfterProbeIndex,
harmlessModernDeploymentEventIds: harmlessModernDeploymentEventIds);
harmlessModernDeploymentEventIds: harmlessModernDeploymentEventIds,
isDevicePreparation: isDevicePreparation);
_adapter = new StallProbeCollectorAdapter(_collector, ingress, clock);
_observableIngress = ingress as SignalIngress;
@@ -188,6 +188,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Security
/// </summary>
public static string DetectEnrollmentType() => DetectEnrollmentTypeWithRule().enrollmentType;
/// <summary>
/// Rule string returned when the deterministic Device Preparation marker fired
/// (the WDP policy orchestration tree — see <see cref="DetectEnrollmentTypeWithRule"/>).
/// </summary>
public const string DevicePreparationExecutionContextRule =
@"DevicePreparation\BootstrapperAgent\ExecutionContext present";
/// <summary>
/// True only when the deterministic Device Preparation marker (BootstrapperAgent
/// ExecutionContext) is present. Scenario gates that DISABLE Autopilot/ESP-only
/// machinery must use this instead of the "v2" string: the CloudAssigned* fallback
/// rules describe a real Autopilot profile (just without ESP) where that machinery
/// still has data to observe — gates fail toward Classic behavior by design.
/// </summary>
public static bool IsDeterministicDevicePreparation()
{
var (enrollmentType, detectionRule) = DetectEnrollmentTypeWithRule();
return enrollmentType == "v2" && detectionRule == DevicePreparationExecutionContextRule;
}
/// <summary>
/// Same as <see cref="DetectEnrollmentType"/> but also returns WHICH rule fired, so the
/// enrollment_type_detected event can name its evidence instead of re-implementing the
@@ -203,7 +223,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Security
{
var executionContext = wdpKey?.GetValue("ExecutionContext")?.ToString();
if (!string.IsNullOrEmpty(executionContext))
return ("v2", @"DevicePreparation\BootstrapperAgent\ExecutionContext present");
return ("v2", DevicePreparationExecutionContextRule);
}
using (var key = Registry.LocalMachine.OpenSubKey(AutopilotSettingsKey))
@@ -76,6 +76,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
private readonly Action<AppPackageState, DateTime?>? _prevOnDoTelemetryReceived;
private readonly Action<ScriptExecutionState>? _prevOnScriptCompleted;
private readonly Action<ScriptStartedInfo>? _prevOnScriptStarted;
private readonly Action<string, string>? _prevOnImeTokenFailure;
// Our own delegate instances — stored once so Dispose can compare by reference.
private readonly Action<string> _ourOnEspPhaseChanged;
@@ -86,6 +87,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
private readonly Action<AppPackageState, DateTime?> _ourOnDoTelemetryReceived;
private readonly Action<ScriptExecutionState> _ourOnScriptCompleted;
private readonly Action<ScriptStartedInfo> _ourOnScriptStarted;
private readonly Action<string, string> _ourOnImeTokenFailure;
// Dedup state for DecisionSignals.
private string? _lastEspPhase;
@@ -175,6 +177,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_prevOnDoTelemetryReceived = _tracker.OnDoTelemetryReceived;
_prevOnScriptCompleted = _tracker.OnScriptCompleted;
_prevOnScriptStarted = _tracker.OnScriptStarted;
_prevOnImeTokenFailure = _tracker.OnImeTokenFailure;
// Store our delegate instances once — implicit method-group conversions
// create a new delegate each time, which would break Dispose's reference check.
@@ -186,6 +189,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_ourOnDoTelemetryReceived = OnDoTelemetryReceived;
_ourOnScriptCompleted = OnScriptCompleted;
_ourOnScriptStarted = OnScriptStarted;
_ourOnImeTokenFailure = OnImeTokenFailure;
_tracker.OnEspPhaseChanged = _ourOnEspPhaseChanged;
_tracker.OnUserSessionCompleted = _ourOnUserSessionCompleted;
@@ -195,6 +199,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_tracker.OnDoTelemetryReceived = _ourOnDoTelemetryReceived;
_tracker.OnScriptCompleted = _ourOnScriptCompleted;
_tracker.OnScriptStarted = _ourOnScriptStarted;
_tracker.OnImeTokenFailure = _ourOnImeTokenFailure;
}
public void Dispose()
@@ -217,6 +222,8 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
_tracker.OnScriptCompleted = _prevOnScriptCompleted;
if (ReferenceEquals(_tracker.OnScriptStarted, _ourOnScriptStarted))
_tracker.OnScriptStarted = _prevOnScriptStarted;
if (ReferenceEquals(_tracker.OnImeTokenFailure, _ourOnImeTokenFailure))
_tracker.OnImeTokenFailure = _prevOnImeTokenFailure;
}
private void OnEspPhaseChanged(string phase)
@@ -267,6 +274,12 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
EmitScriptStarted(info);
}
private void OnImeTokenFailure(string errorCode, string message)
{
_prevOnImeTokenFailure?.Invoke(errorCode, message);
EmitImeTokenFailure(errorCode, message);
}
internal void TriggerEspPhaseFromTest(string phase) => EmitEspPhase(phase);
internal void TriggerUserSessionCompletedFromTest() => EmitUserSessionCompleted();
internal void TriggerAppStateFromTest(AppPackageState app, AppInstallationState oldState, AppInstallationState newState) =>
@@ -277,6 +290,7 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
EmitDoTelemetry(app, sourceTimestampUtc);
internal void TriggerScriptCompletedFromTest(ScriptExecutionState script) => EmitScriptCompleted(script);
internal void TriggerScriptStartedFromTest(ScriptStartedInfo info) => EmitScriptStarted(info);
internal void TriggerImeTokenFailureFromTest(string errorCode, string message) => EmitImeTokenFailure(errorCode, message);
/// <summary>
/// Prefer the CMTrace log entry timestamp recorded by the most recent pattern match
@@ -899,6 +913,34 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
: $"ImeAdapter: IME agent version detected: {version}");
}
// IME token-acquisition failure ("Failed to get AAD token ... errorCode = N",
// IntuneTokenManager). Previously we only matched the success line — an outage
// (WAM broken, conditional access, network) was invisible while every check-in
// silently did nothing. Tracker-side dedup: once per distinct errorCode.
private void EmitImeTokenFailure(string errorCode, string message)
{
var now = ResolveOccurredAt(out var derivedFromClock, out var rawSourceTs);
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["errorCode"] = string.IsNullOrEmpty(errorCode) ? "unknown" : errorCode,
};
if (!string.IsNullOrEmpty(message))
data["logLine"] = message;
var patternId = _tracker.LastMatchedPatternId;
if (!string.IsNullOrEmpty(patternId))
data["patternId"] = patternId!;
TagDerivedTimestamp(data, derivedFromClock, rawSourceTs);
_post.Emit(
eventType: SharedEventTypes.ImeTokenFailure,
source: SourceLabel,
message: $"IME token acquisition failed (errorCode={data["errorCode"]})",
data: data,
severity: EventSeverity.Warning,
occurredAtUtc: now);
}
private ImeMsiInstallSource ResolveMsiInstallSource(string version)
{
if (_msiSourceCachedForVersion == version) return _msiSourceCache;
@@ -8,6 +8,7 @@ using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Interop;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Runtime;
using AutopilotMonitor.Agent.V2.Core.Security;
@@ -56,6 +57,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
public sealed class EnrollmentTerminationHandler
{
private static readonly TimeSpan DefaultLateEventGrace = TimeSpan.FromMilliseconds(2000);
/// <summary>
/// Session bc803955 — on DevicePreparation the IME user-context app sync starts only
/// after desktop arrival and its log lines reach disk with 10-20s flush latency, so the
/// dialog snapshot raced it and rendered an empty Applications section 9 seconds before
/// the tracker learned about 12 pending apps. Bounded settle wait before the summary
/// build; DevicePrep-only and only while the tracker has zero packages, so Classic (data
/// long settled) and populated DevicePrep sessions pay nothing. Installs are never
/// delayed — this only holds the dialog snapshot.
/// </summary>
private static readonly TimeSpan DefaultDevicePrepAppSettleWait = TimeSpan.FromSeconds(20);
private static readonly TimeSpan AppSettlePollInterval = TimeSpan.FromMilliseconds(250);
// Option 1 (WG Part 1 graceful-exit hardening, 2026-04-30): poll cadence for the
// active spool-empty drain. 50ms is short enough that we exit the bounded wait
// within ~one cadence after the last upload is acknowledged, but long enough that
@@ -90,6 +103,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
private readonly AgentAnalyzerManager _analyzerManager;
private readonly InformationalEventPost _post;
private readonly TimeSpan _lateEventGracePeriod;
private readonly TimeSpan _devicePrepAppSettleWait;
// L4 (delta review 2026-07-02): epoch anchor for boot-time derivation. Captured as a
// (monotonic tick, wall clock) pair at construction so the boot timestamp used by the
@@ -113,7 +127,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
AgentAnalyzerManager analyzerManager = null,
InformationalEventPost post = null,
Action stopPeripheralCollectors = null,
TimeSpan? lateEventGracePeriod = null)
TimeSpan? lateEventGracePeriod = null,
TimeSpan? devicePrepAppSettleWait = null)
{
_session = session ?? throw new ArgumentNullException(nameof(session));
_appTracking = appTracking ?? throw new ArgumentNullException(nameof(appTracking));
@@ -126,6 +141,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
_post = post;
_stopPeripheralCollectors = stopPeripheralCollectors;
_lateEventGracePeriod = lateEventGracePeriod ?? DefaultLateEventGrace;
_devicePrepAppSettleWait = devicePrepAppSettleWait ?? DefaultDevicePrepAppSettleWait;
_configuration = session.Configuration;
_stateDirectory = session.StateDirectory;
@@ -182,6 +198,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
}
else
{
// Session bc803955 — give the IME user-context app wave a bounded chance
// to land BEFORE the dialog snapshot, the starved sweep and the
// app_tracking_summary below all read package states. DevicePrep-only,
// no-op when the tracker already has data.
WaitForImeAppSettleIfDevicePrep(state);
RunBuildAndLaunchDialog(state, args);
}
}
@@ -207,7 +228,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
// Runs on every outcome; the live (esp_exited) path's dedupe set is
// consulted so no app is reported twice. Promoted likely-stuck apps are
// Error by now and excluded by the probe itself.
EmitStarvedUserEspApps(args);
EmitStarvedUserEspApps(state, args);
// Session 6b4993e5 / fc48c71a — on an ESP terminal Apps-subcategory failure,
// name the in-flight device app(s) (e.g. stuck Downloading at 0%) as the likely
@@ -378,6 +399,59 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
}
}
/// <summary>
/// Bounded settle wait for the IME app wave on DevicePreparation sessions (see
/// <see cref="DefaultDevicePrepAppSettleWait"/>). Returns immediately for every other
/// scenario mode and whenever the tracker already reports packages. Poll is cheap
/// (in-memory read-model access), so a fine 250ms interval keeps the early exit tight.
/// </summary>
private void WaitForImeAppSettleIfDevicePrep(DecisionState state)
{
try
{
if (state == null || _devicePrepAppSettleWait <= TimeSpan.Zero) return;
if (state.ScenarioProfile.Mode != EnrollmentMode.DevicePreparation) return;
var packages = TryGetPackageStates();
if (packages != null && packages.Count > 0) return;
_logger.Info(
$"EnrollmentTerminationHandler: DevicePreparation session with no IME app data yet — waiting up to " +
$"{_devicePrepAppSettleWait.TotalSeconds:0}s for the user-context app wave before building the summary " +
"(installs are not delayed; this only holds the dialog snapshot).");
var deadline = DateTime.UtcNow + _devicePrepAppSettleWait;
while (DateTime.UtcNow < deadline)
{
Thread.Sleep(AppSettlePollInterval);
packages = TryGetPackageStates();
if (packages != null && packages.Count > 0)
{
_logger.Info($"EnrollmentTerminationHandler: IME app data arrived during settle wait ({packages.Count} package(s)).");
return;
}
}
_logger.Info("EnrollmentTerminationHandler: app settle wait elapsed without IME app data — summary reflects Office/CSP state only.");
}
catch (Exception ex)
{
_logger.Warning($"EnrollmentTerminationHandler: app settle wait failed: {ex.Message}");
}
}
/// <summary>
/// Fail-soft read of the Office C2R lifecycle state persisted by
/// <see cref="OfficeInstallDetector"/> — feeds the summary dialog's synthetic
/// "Microsoft 365 Apps" row (Office CSP installs never flow through the IME Win32
/// pipeline, so the package-state list cannot represent them).
/// </summary>
private OfficeInstallStateData TryLoadOfficeInstallState()
{
try { return new OfficeInstallStatePersistence(_stateDirectory, _logger).Load(); }
catch { return null; }
}
private void RunBuildAndLaunchDialog(DecisionState state, EnrollmentTerminatedEventArgs args)
{
try
@@ -385,7 +459,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
var packages = TryGetPackageStates();
var timings = TryGetAppTimings();
var status = FinalStatusBuilder.Build(state, args, packages, _agentStartTimeUtc, timings,
deviceBootUtc: ObservationCoverage.DeviceBootUtc(_bootAnchorTickMs, _bootAnchorUtc));
deviceBootUtc: ObservationCoverage.DeviceBootUtc(_bootAnchorTickMs, _bootAnchorUtc),
officeState: TryLoadOfficeInstallState());
SummaryDialogLauncher.WriteAndLaunch(status, _configuration, _stateDirectory, _logger, _dialogExePathOverride);
if (_configuration.ShowEnrollmentSummary)
@@ -482,7 +557,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
/// wording; Failed/TimedOut keep the Warning + "starved" verdict.
/// </para>
/// </summary>
private void EmitStarvedUserEspApps(EnrollmentTerminatedEventArgs args)
private void EmitStarvedUserEspApps(DecisionState state, EnrollmentTerminatedEventArgs args)
{
if (_post == null) return;
@@ -491,6 +566,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
var starved = _appTracking.GetStarvedUserEspApps();
if (starved == null || starved.Count == 0) return;
// Device Preparation has no ESP apps gate — installing after the desktop is
// the DESIGNED order there (only DPP-policy apps run during provisioning, the
// rest follow via IME). Reporting every pending app as "starved" turned a
// normal WDP session into 10+ noise events (afee7ae0); the same per-app data
// still reaches the backend via app_tracking_summary and final-status.json.
if (state?.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation)
{
_logger.Info(
$"EnrollmentTerminationHandler: {starved.Count} app(s) pending at termination on a " +
"DevicePreparation session — expected post-desktop install order, app_install_starved sweep skipped.");
return;
}
var succeeded = args.Outcome == EnrollmentTerminationOutcome.Succeeded;
foreach (var app in starved)
@@ -2,6 +2,7 @@
using System;
using System.Collections.Generic;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Office;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.State;
@@ -30,7 +31,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
IReadOnlyList<AppPackageState>? packageStates,
DateTime agentStartTimeUtc,
IReadOnlyDictionary<string, AppInstallTiming>? appTimings = null,
DateTime? deviceBootUtc = null)
DateTime? deviceBootUtc = null,
OfficeInstallStateData? officeState = null)
{
if (state == null) throw new ArgumentNullException(nameof(state));
if (terminated == null) throw new ArgumentNullException(nameof(terminated));
@@ -78,9 +80,85 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
PackageStatesByPhase = BuildPackageStatesByPhase(packageStates, timings),
};
AddOfficeRow(status, officeState);
return status;
}
/// <summary>
/// Session bc803955 — the Office C2R install (Office CSP app type / M365 "CSP" package)
/// never flows through the IME Win32 pipeline, so <see cref="BuildAppSummary"/> cannot see
/// it and a DevicePreparation session whose only app activity is Office rendered
/// "No application data available". Synthesize one package row from the
/// <see cref="OfficeInstallDetector"/>'s persisted lifecycle state instead. Preinstalled
/// is deliberately skipped — that state marks resident OEM/consumer Office, not an
/// enrollment install.
/// </summary>
private static void AddOfficeRow(FinalStatus status, OfficeInstallStateData? officeState)
{
if (officeState == null) return;
string rowState;
bool isCompleted = false, isError = false;
switch (officeState.State)
{
case OfficeInstallStateData.StateActive:
rowState = AppInstallationState.Installing.ToString();
break;
case OfficeInstallStateData.StateCompleted:
rowState = AppInstallationState.Installed.ToString();
isCompleted = true;
break;
case OfficeInstallStateData.StateFailed:
rowState = AppInstallationState.Error.ToString();
isError = true;
isCompleted = true; // mirrors IsCompleted(): Error counts as terminal
break;
default:
return; // Preinstalled / unknown — not an enrollment install
}
double? duration = officeState.StartedAtUtc.HasValue && officeState.CompletedAtUtc.HasValue
? Math.Max(0, (officeState.CompletedAtUtc.Value - officeState.StartedAtUtc.Value).TotalSeconds)
: (double?)null;
var name = string.IsNullOrEmpty(officeState.VersionReached)
? "Microsoft 365 Apps"
: $"Microsoft 365 Apps ({officeState.VersionReached})";
var row = new FinalStatusPackageInfo
{
AppName = name,
State = rowState,
IsError = isError,
IsCompleted = isCompleted,
Targeted = AppTargeted.Device.ToString(),
StartedAt = officeState.StartedAtUtc?.ToString("o"),
CompletedAt = officeState.CompletedAtUtc?.ToString("o"),
DurationSeconds = duration,
};
var phaseKey = AppTargeted.Device.ToString();
if (!status.PackageStatesByPhase.TryGetValue(phaseKey, out var bucket))
{
bucket = new List<FinalStatusPackageInfo>();
status.PackageStatesByPhase[phaseKey] = bucket;
}
// Office installs during device prep / device ESP — before any IME-tracked app.
bucket.Insert(0, row);
var summary = status.AppSummary;
summary.TotalApps++;
if (isCompleted) summary.CompletedApps++;
if (isError)
{
summary.ErrorCount++;
summary.DeviceErrors++;
}
if (!summary.AppsByPhase.TryGetValue(phaseKey, out var cnt)) cnt = 0;
summary.AppsByPhase[phaseKey] = cnt + 1;
}
/// <summary>
/// Schema 2 — derives a human-readable explanation rendered as the dialog's failure
/// banner. Returns <c>null</c> for successful terminal outcomes (succeeded /
@@ -59,10 +59,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
/// </summary>
int PeakPendingItemCount { get; }
/// <summary>
/// Serialized on-disk byte size of the pending (not-yet-uploaded) tail — the actual
/// upload backlog. Unlike <see cref="SpoolFileSizeBytes"/>, which grows append-only
/// over the whole session regardless of upload progress, this shrinks back to 0 as
/// <see cref="MarkUploaded"/> advances the cursor, so it is the correct input for
/// back-pressure thresholds.
/// </summary>
long PendingBytes { get; }
/// <summary>
/// Current size of the on-disk <c>spool.jsonl</c> file in bytes. Returns 0 when the
/// file does not yet exist. Cheap (single <c>FileInfo.Length</c> call) and safe to
/// poll from periodic collectors.
/// poll from periodic collectors. Append-only cumulative — informational only, NOT a
/// backlog measure (use <see cref="PendingBytes"/> for that).
/// </summary>
long SpoolFileSizeBytes { get; }
@@ -56,9 +56,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
// Populated from disk in the ctor and kept in sync by Enqueue / MarkUploaded. Peek
// reads from here so the periodic drain loop is O(batchSize) instead of O(N) per
// call. Disk JSONL stays the source of truth — the cache is rebuilt on every
// process start.
private readonly LinkedList<TelemetryItem> _pending = new LinkedList<TelemetryItem>();
// process start. Each entry carries its on-disk line length (incl. trailing \n) so
// MarkUploaded can decrement _pendingBytes exactly.
private readonly LinkedList<PendingEntry> _pending = new LinkedList<PendingEntry>();
private int _peakPendingCount;
private long _pendingBytes;
private readonly struct PendingEntry
{
public PendingEntry(TelemetryItem item, int byteLength)
{
Item = item;
ByteLength = byteLength;
}
public TelemetryItem Item { get; }
public int ByteLength { get; }
}
public TelemetrySpool(string directoryPath, IClock clock, Logging.AgentLogger? logger = null)
{
@@ -105,6 +119,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
get { lock (_lock) return _peakPendingCount; }
}
public long PendingBytes
{
get { lock (_lock) return _pendingBytes; }
}
public long SpoolFileSizeBytes
{
get
@@ -171,7 +190,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
}
_lastAssignedItemId = itemId;
_pending.AddLast(item);
_pending.AddLast(new PendingEntry(item, bytes.Length));
_pendingBytes += bytes.Length;
if (_pending.Count > _peakPendingCount)
_peakPendingCount = _pending.Count;
@@ -209,7 +229,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
var node = _pending.First;
while (node != null && batch.Count < max)
{
batch.Add(node.Value);
batch.Add(node.Value.Item);
node = node.Next;
}
return batch;
@@ -233,25 +253,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
// Trim the in-memory tail. The list is in TelemetryItemId-monotonic order
// (Enqueue only appends), so we can drop from the front until the first
// remaining item is past the cursor.
while (_pending.First != null && _pending.First.Value.TelemetryItemId <= upToItemIdInclusive)
while (_pending.First != null && _pending.First.Value.Item.TelemetryItemId <= upToItemIdInclusive)
{
_pendingBytes -= _pending.First.Value.ByteLength;
_pending.RemoveFirst();
}
}
}
/// <summary>
/// Single-pass scan of <c>spool.jsonl</c>: returns the highest <c>TelemetryItemId</c>
/// found AND populates <see cref="_pending"/> with every item past
/// <see cref="_lastUploadedItemId"/>. Called once from the ctor — replaces the
/// previous per-Peek <c>File.ReadAllLines</c> hot path.
/// Single-pass streaming scan of <c>spool.jsonl</c> (<c>File.ReadLines</c> — never
/// materializes the whole file): returns the highest <c>TelemetryItemId</c> found AND
/// populates <see cref="_pending"/> / <see cref="_pendingBytes"/> with every item past
/// <see cref="_lastUploadedItemId"/>. Called once from the ctor.
/// </summary>
private long ScanSpoolAndRehydratePending()
{
if (!File.Exists(_spoolPath)) return -1;
long highest = -1;
foreach (var line in File.ReadAllLines(_spoolPath, Encoding.UTF8))
foreach (var line in File.ReadLines(_spoolPath, Encoding.UTF8))
{
if (string.IsNullOrWhiteSpace(line)) continue;
TelemetryItem item;
@@ -270,7 +291,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Transport.Telemetry
if (item.TelemetryItemId > highest) highest = item.TelemetryItemId;
if (item.TelemetryItemId > _lastUploadedItemId)
{
_pending.AddLast(item);
// +1 = the trailing \n Enqueue writes after every line, so the rehydrated
// byte count matches what Enqueue would have accumulated.
var byteLength = Encoding.UTF8.GetByteCount(line) + 1;
_pending.AddLast(new PendingEntry(item, byteLength));
_pendingBytes += byteLength;
}
}
return highest;
@@ -26,7 +26,7 @@ namespace AutopilotMonitor.Agent.V2
/// cleanup and exit.</item>
/// <item><b>Session-age emergency break</b> — absolute cap (<c>AbsoluteMaxSessionHours</c>)
/// across restarts prevents zombie agents from persisting forever.</item>
/// <item><b>Crash log writer</b> — writes a last-resort crash_*.log next to the agent log
/// <item><b>Crash log writer</b> — writes a last-resort crash-*.log next to the agent log
/// directory when an uncaught exception escapes <c>Main</c>.</item>
/// </list>
/// </summary>
@@ -34,7 +34,10 @@ namespace AutopilotMonitor.Agent.V2
{
internal const string CleanExitMarkerFileName = "clean-exit.marker";
internal const string EnrollmentCompleteMarkerFileName = "enrollment-complete.marker";
private const string CrashLogPrefix = "crash_";
private const string CrashLogPrefix = "crash-";
// Pre-rename agents (< 2026-08-18) wrote crash_*.log; still detected once so an
// in-place self-update does not lose a pending crash classification.
private const string LegacyCrashLogPrefix = "crash_";
// ----------------------------------------------------------------- Multi-instance guard
@@ -102,7 +105,7 @@ namespace AutopilotMonitor.Agent.V2
}
/// <summary>
/// Reads <c>clean-exit.marker</c> + <c>crash_*.log</c> + session files + (if needed) the
/// Reads <c>clean-exit.marker</c> + <c>crash-*.log</c> + session files + (if needed) the
/// Windows event log to classify the previous exit. Deletes the marker and crash logs so
/// the next cycle starts fresh. Legacy parity — same rules, same outputs.
/// </summary>
@@ -114,6 +117,8 @@ namespace AutopilotMonitor.Agent.V2
var hadCleanExit = File.Exists(cleanExitMarker);
var crashLogs = Directory.Exists(logDirectory)
? Directory.GetFiles(logDirectory, CrashLogPrefix + "*.log")
.Concat(Directory.GetFiles(logDirectory, LegacyCrashLogPrefix + "*.log"))
.ToArray()
: Array.Empty<string>();
if (hadCleanExit)
@@ -406,7 +411,7 @@ namespace AutopilotMonitor.Agent.V2
try
{
Directory.CreateDirectory(logDirectory);
var crashPath = Path.Combine(logDirectory, $"{CrashLogPrefix}{DateTime.UtcNow:yyyyMMdd_HHmmss}.log");
var crashPath = Path.Combine(logDirectory, $"{CrashLogPrefix}{DateTime.UtcNow:yyyyMMdd-HHmmss}.log");
File.WriteAllText(crashPath, $"[{DateTime.UtcNow:u}] FATAL: {ex}");
}
catch { /* nowhere left to log */ }
@@ -30,6 +30,8 @@ namespace AutopilotMonitor.Agent.V2
"IntuneManagementExtension-????????-??????.log",
"AppWorkload.log",
"AppWorkload-????????-??????.log",
"AppActionProcessor.log",
"AppActionProcessor-????????-??????.log",
"AgentExecutor.log",
"AgentExecutor-????????-??????.log",
"HealthScripts.log",
@@ -306,7 +306,12 @@ namespace AutopilotMonitor.Agent.V2.Runtime
{
try
{
var enrollmentType = EnrollmentRegistryDetector.DetectEnrollmentType();
var (enrollmentType, detectionRule) = EnrollmentRegistryDetector.DetectEnrollmentTypeWithRule();
// High-confidence WDP seed only for the deterministic marker — the
// CloudAssigned* fallback rules keep the Medium seed (see the payload
// key's doc on SignalPayloadKeys.EnrollmentTypeDeterministic).
var deterministicWdp = enrollmentType == "v2"
&& detectionRule == EnrollmentRegistryDetector.DevicePreparationExecutionContextRule;
var isHybridJoin = EnrollmentRegistryDetector.DetectHybridJoin();
var isSelfDeploying = EnrollmentRegistryDetector.DetectSelfDeployingProfile();
var isCloudPc = CloudPcDetector.DetectIsCloudPc();
@@ -314,6 +319,7 @@ namespace AutopilotMonitor.Agent.V2.Runtime
var payload = new Dictionary<string, string>(StringComparer.Ordinal)
{
["enrollmentType"] = enrollmentType,
["enrollmentTypeDeterministic"] = deterministicWdp ? "true" : "false",
["isHybridJoin"] = isHybridJoin ? "true" : "false",
["isSelfDeployingProfile"] = isSelfDeploying ? "true" : "false",
["isCloudPc"] = isCloudPc ? "true" : "false",
@@ -7,14 +7,14 @@
# server-updatable without re-shipping the MSI.
#
# Fail-soft by design: the runner always exits 0 so a transient network failure never
# marks the LOB install as failed; diagnostics go to msi-bootstrap.log.
# marks the LOB install as failed; diagnostics go to bootstrap-msi.log.
# ASCII only - no special characters in this file.
$ErrorActionPreference = 'Stop'
$baseDir = Join-Path $env:ProgramData 'AutopilotMonitor'
$logDir = Join-Path $baseDir 'Logs'
New-Item -ItemType Directory -Force -Path $logDir | Out-Null
$log = Join-Path $logDir 'msi-bootstrap.log'
$log = Join-Path $logDir 'bootstrap-msi.log'
function Write-Log([string]$message) {
Add-Content -Path $log -Value ("[{0}] {1}" -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $message)
@@ -13,7 +13,7 @@
<Wix xmlns="http://wixtoolset.org/schemas/v4/wxs">
<Package Name="Autopilot Monitor Bootstrap"
Manufacturer="Autopilot Monitor"
Version="1.0.0"
Version="1.0.2"
UpgradeCode="1fbe5515-d346-445c-b8b8-14ee79f97522"
Scope="perMachine"
Compressed="yes">
@@ -39,7 +39,7 @@
<!-- Deferred + no-impersonate: runs as SYSTEM after files land. Return=ignore keeps
the LOB install green even if the runner hits a transient error (the runner is
additionally fail-soft itself); diagnostics live in msi-bootstrap.log. -->
additionally fail-soft itself); diagnostics live in bootstrap-msi.log. -->
<CustomAction Id="RunBootstrap"
Directory="INSTALLFOLDER"
ExeCommand="&quot;[System64Folder]WindowsPowerShell\v1.0\powershell.exe&quot; -NoProfile -ExecutionPolicy Bypass -File &quot;[INSTALLFOLDER]Invoke-BootstrapDownload.ps1&quot;"
@@ -0,0 +1,230 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using Xunit;
namespace AutopilotMonitor.DecisionCore.Tests
{
/// <summary>
/// Device Preparation (WDP) engine semantics — afee7ae0 audit (2026-08-18).
/// WDP has no ESP, so the ESP-derived arms/gates/deadlines can never fire there.
/// Coverage:
/// <list type="bullet">
/// <item>Profile seeding: the deterministic WDP marker seeds High confidence; the
/// CloudAssigned* fallback rules stay Medium.</item>
/// <item><c>completion_waiting</c> never lists <c>account_setup_provisioning_complete</c>
/// on WDP (unsatisfiable by construction — no ESP categories).</item>
/// <item>Desktop-first arming of the <see cref="DeadlineNames.DevicePrepCompletion"/>
/// backstop, its fire semantics (synthetic Hello timeout + Finalizing), and the
/// no-rearm/no-classic-leak guards.</item>
/// <item>Arm D: the Hello-disabled fast-path completes at desktop arrival on WDP.</item>
/// </list>
/// </summary>
public sealed class DevicePreparationEngineTests
{
private static readonly DateTime T0 = new DateTime(2026, 8, 18, 9, 0, 0, DateTimeKind.Utc);
private static DecisionSignal MakeSignal(
long ordinal,
DecisionSignalKind kind,
DateTime occurredAtUtc,
IReadOnlyDictionary<string, string>? payload = null)
{
return new DecisionSignal(
sessionSignalOrdinal: ordinal,
sessionTraceOrdinal: ordinal,
kind: kind,
kindSchemaVersion: 1,
occurredAtUtc: occurredAtUtc,
sourceOrigin: "test",
evidence: new Evidence(EvidenceKind.Synthetic, $"{kind}-{ordinal}", "test"),
payload: payload);
}
private static DecisionSignal DeadlineFired(long ordinal, DateTime occurredAtUtc, string deadlineName) =>
MakeSignal(ordinal, DecisionSignalKind.DeadlineFired, occurredAtUtc,
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = deadlineName });
private static List<DecisionEffect> CompletionWaitingEffects(DecisionStep step) =>
step.Effects.Where(e =>
e.Kind == DecisionEffectKind.EmitEventTimelineEntry
&& e.Parameters != null
&& e.Parameters.TryGetValue("eventType", out var et)
&& et == "completion_waiting").ToList();
/// <summary>Session seeded as WDP via the EnrollmentFactsObserved signal rail.</summary>
private static DecisionState SeedWdpSession(DecisionEngine engine, bool deterministic = true)
{
var state = DecisionState.CreateInitial("sess-wdp", "tenant-wdp", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
state = engine.Reduce(state, MakeSignal(
1, DecisionSignalKind.EnrollmentFactsObserved, T0,
new Dictionary<string, string>
{
[SignalPayloadKeys.EnrollmentType] = "v2",
[SignalPayloadKeys.EnrollmentTypeDeterministic] = deterministic ? "true" : "false",
})).NewState;
return state;
}
// ===================================================== profile seeding
[Fact]
public void DeterministicV2Seed_SetsDevicePreparationAtHighConfidence()
{
var engine = new DecisionEngine();
var state = SeedWdpSession(engine, deterministic: true);
Assert.Equal(EnrollmentMode.DevicePreparation, state.ScenarioProfile.Mode);
Assert.Equal(ProfileConfidence.High, state.ScenarioProfile.Confidence);
}
[Fact]
public void FallbackV2Seed_StaysAtMediumConfidence()
{
// CloudAssigned* fallback rules (no deterministic marker) — Medium, as before
// d0ab3eee: an ESP-less classic profile could in principle trip them.
var engine = new DecisionEngine();
var state = SeedWdpSession(engine, deterministic: false);
Assert.Equal(EnrollmentMode.DevicePreparation, state.ScenarioProfile.Mode);
Assert.Equal(ProfileConfidence.Medium, state.ScenarioProfile.Confidence);
}
[Fact]
public void LegacyV2Payload_WithoutDeterministicKey_StaysAtMediumConfidence()
{
// Older agents post enrollmentType=v2 without the new key — must behave like
// the fallback (Medium), not throw or over-promote.
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial("sess-legacy", "tenant-wdp", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
state = engine.Reduce(state, MakeSignal(
1, DecisionSignalKind.EnrollmentFactsObserved, T0,
new Dictionary<string, string> { [SignalPayloadKeys.EnrollmentType] = "v2" })).NewState;
Assert.Equal(EnrollmentMode.DevicePreparation, state.ScenarioProfile.Mode);
Assert.Equal(ProfileConfidence.Medium, state.ScenarioProfile.Confidence);
}
// ===================================================== desktop-first backstop
[Fact]
public void DesktopFirst_OnWdp_ArmsDevicePrepCompletionBackstop()
{
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
var step = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9)));
// Hello policy unknown → no fast-path; stage parked, backstop armed.
var backstop = Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.DevicePrepCompletion);
Assert.Equal(T0.AddMinutes(9).AddMinutes(30), backstop.DueAtUtc);
Assert.Contains(step.Effects,
e => e.Kind == DecisionEffectKind.ScheduleDeadline && e.Deadline?.Name == DeadlineNames.DevicePrepCompletion);
}
[Fact]
public void DesktopFirst_OnWdp_WaitingListsOnlyHello_NeverTheEspProvisioningGate()
{
// account_setup_provisioning_complete is unsatisfiable on WDP (no ESP categories)
// and must not appear as a missing prerequisite (afee7ae0 listed it).
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
var step = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9)));
var waiting = Assert.Single(CompletionWaitingEffects(step));
Assert.Equal("hello_resolution", waiting.Parameters!["missingPrerequisites"]);
}
[Fact]
public void DuplicateDesktopSignal_DoesNotRearmTheBackstop()
{
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
state = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9))).NewState;
var second = engine.Reduce(state, MakeSignal(11, DecisionSignalKind.DesktopArrived, T0.AddMinutes(10)));
Assert.Single(second.NewState.Deadlines, d => d.Name == DeadlineNames.DevicePrepCompletion);
Assert.DoesNotContain(second.Effects,
e => e.Kind == DecisionEffectKind.ScheduleDeadline && e.Deadline?.Name == DeadlineNames.DevicePrepCompletion);
}
[Fact]
public void ClassicDesktopFirst_DoesNotArmTheBackstop()
{
// Classic regression guard: a non-WDP session parked desktop-first keeps the
// pre-change deadline picture (nothing armed by this handler).
var engine = new DecisionEngine();
var state = DecisionState.CreateInitial("sess-classic", "tenant-classic", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
var step = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9)));
Assert.DoesNotContain(step.NewState.Deadlines, d => d.Name == DeadlineNames.DevicePrepCompletion);
}
// ===================================================== backstop fire semantics
[Fact]
public void BackstopFired_WithoutHello_SynthesizesTimeoutAndCompletesThroughFinalizing()
{
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
state = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9))).NewState;
var fired = engine.Reduce(state, DeadlineFired(20, T0.AddMinutes(39), DeadlineNames.DevicePrepCompletion));
Assert.Equal(SessionStage.Finalizing, fired.NewState.Stage);
Assert.NotNull(fired.NewState.HelloResolvedUtc);
Assert.Equal("Timeout", fired.NewState.HelloOutcome!.Value);
Assert.DoesNotContain(fired.NewState.Deadlines, d => d.Name == DeadlineNames.DevicePrepCompletion);
Assert.Contains(fired.NewState.Deadlines, d => d.Name == DeadlineNames.FinalizingGrace);
// FinalizingGrace completes the session — the full WDP no-Hello resolution chain.
var completed = engine.Reduce(fired.NewState,
DeadlineFired(21, T0.AddMinutes(39).AddSeconds(5), DeadlineNames.FinalizingGrace));
Assert.Equal(SessionStage.Completed, completed.NewState.Stage);
}
[Fact]
public void HelloResolved_BeforeBackstopFires_CompletesNormally()
{
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
state = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9))).NewState;
var resolved = engine.Reduce(state, MakeSignal(
20, DecisionSignalKind.HelloResolved, T0.AddMinutes(10),
new Dictionary<string, string> { [SignalPayloadKeys.HelloOutcome] = "completed" }));
Assert.Equal(SessionStage.Finalizing, resolved.NewState.Stage);
Assert.Equal("completed", resolved.NewState.HelloOutcome!.Value);
}
// ===================================================== arm D fast-path
[Fact]
public void HelloDisabledFastPath_OnWdp_CompletesAtDesktopArrival()
{
// Arm D: no ESP facts exist on WDP, yet a disabled Hello policy + real-user
// desktop is a legitimate completion — previously arms A–C all failed and the
// session parked despite Hello being disabled.
var engine = new DecisionEngine();
var state = SeedWdpSession(engine);
state = engine.Reduce(state, MakeSignal(
5, DecisionSignalKind.HelloPolicyDetected, T0.AddMinutes(1),
new Dictionary<string, string> { [SignalPayloadKeys.HelloEnabled] = "false" })).NewState;
var step = engine.Reduce(state, MakeSignal(10, DecisionSignalKind.DesktopArrived, T0.AddMinutes(9)));
Assert.Equal(SessionStage.Finalizing, step.NewState.Stage);
Assert.Equal("Skipped", step.NewState.HelloOutcome!.Value);
}
}
}
@@ -0,0 +1,44 @@
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Default-value tests for the IME-installer archiving settings. The flag gates the
/// ime-msi-archive queue worker (pause, not drop); the size cap guards the download.
/// Defaulting the flag to TRUE is deliberate: archiving is operator infrastructure with
/// ~one small download per Microsoft IME release, and a version missed while the flag is
/// off is gone for good (the versionless CDN moves on).
/// </summary>
public class AdminConfigurationImeMsiArchivingTests
{
[Fact]
public void ImeMsiArchivingEnabled_defaults_to_true_on_new_config()
{
var cfg = new AdminConfiguration();
Assert.True(cfg.ImeMsiArchivingEnabled);
}
[Fact]
public void ImeMsiArchivingEnabled_defaults_to_true_on_CreateDefault()
{
var cfg = AdminConfiguration.CreateDefault();
Assert.True(cfg.ImeMsiArchivingEnabled);
}
[Fact]
public void MaxImeMsiDownloadSizeMB_defaults_to_250()
{
// 250 MB: ~20x today's MSI so a legitimately grown installer is never missed,
// while a tampered URL still cannot stream gigabytes (Oliver, 2026-08-18).
Assert.Equal(250, new AdminConfiguration().MaxImeMsiDownloadSizeMB);
Assert.Equal(250, AdminConfiguration.CreateDefault().MaxImeMsiDownloadSizeMB);
}
[Fact]
public void Settings_persist_on_the_config()
{
var cfg = new AdminConfiguration { ImeMsiArchivingEnabled = false, MaxImeMsiDownloadSizeMB = 50 };
Assert.False(cfg.ImeMsiArchivingEnabled);
Assert.Equal(50, cfg.MaxImeMsiDownloadSizeMB);
}
}
@@ -0,0 +1,124 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Azure;
using Azure.Storage.Queues;
using Azure.Storage.Queues.Models;
using AutopilotMonitor.Functions.Services.Ime;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Newtonsoft.Json;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Producer tests for the ime-msi-archive queue (daily review 2026-08-18: previously
/// untested). Contract: the caller is the ingest path's fire-and-forget continuation, so
/// NOTHING may throw — version-less envelopes are skipped, send failures are swallowed,
/// and a failed CreateIfNotExists is retried on the next enqueue instead of latching.
/// </summary>
public class AzureQueueImeMsiArchiveProducerTests
{
private const string Version = "1.104.102.0";
[Fact]
public async Task EnqueueAsync_sends_envelope_json_and_ensures_queue_once()
{
var (producer, queue, sentBodies) = BuildSut();
await producer.EnqueueAsync(Envelope());
await producer.EnqueueAsync(Envelope());
Assert.Equal(2, sentBodies.Count);
var roundTripped = JsonConvert.DeserializeObject<ImeMsiArchiveEnvelope>(sentBodies[0])!;
Assert.Equal(Version, roundTripped.Version);
Assert.Equal(ImeMsiArchiver.CanonicalMsiUrl, roundTripped.MsiDownloadUrl);
Assert.Equal("11111111-1111-1111-1111-111111111111", roundTripped.TenantId);
// Queue existence is ensured once, then latched — not re-checked per message.
queue.Verify(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()),
Times.Once);
}
[Fact]
public async Task EnqueueAsync_skips_envelope_without_version()
{
var (producer, queue, sentBodies) = BuildSut();
await producer.EnqueueAsync(new ImeMsiArchiveEnvelope { Version = string.Empty });
await producer.EnqueueAsync(null!);
Assert.Empty(sentBodies);
queue.Verify(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task EnqueueAsync_send_failure_is_swallowed()
{
var (producer, queue, _) = BuildSut();
queue.Setup(q => q.SendMessageAsync(It.IsAny<string>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(503, "queue busy"));
// Fire-and-forget continuation on the ingest path — must never throw.
await producer.EnqueueAsync(Envelope());
}
[Fact]
public async Task EnqueueAsync_retries_queue_creation_after_failure()
{
var (producer, queue, sentBodies) = BuildSut();
queue.SetupSequence(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()))
.ThrowsAsync(new RequestFailedException(500, "storage down"))
.ReturnsAsync((Response?)null);
await producer.EnqueueAsync(Envelope()); // ensure fails — not latched
await producer.EnqueueAsync(Envelope()); // ensure retried and succeeds
queue.Verify(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()),
Times.Exactly(2));
// Both messages were still attempted (fail-soft send after a failed ensure).
Assert.Equal(2, sentBodies.Count);
}
// ============================================================ Helpers ====
private static ImeMsiArchiveEnvelope Envelope() => new()
{
Version = Version,
MsiDownloadUrl = ImeMsiArchiver.CanonicalMsiUrl,
MsiMatchedBy = "productVersion",
TenantId = "11111111-1111-1111-1111-111111111111",
SessionId = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
EnqueuedAt = DateTime.UtcNow,
};
private static (AzureQueueImeMsiArchiveProducer Producer, Mock<QueueClient> Queue, List<string> SentBodies) BuildSut()
{
var queue = new Mock<QueueClient>();
var sentBodies = new List<string>();
queue.Setup(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync((Response?)null);
queue.Setup(q => q.SendMessageAsync(It.IsAny<string>(), It.IsAny<CancellationToken>()))
.Returns<string, CancellationToken>((body, _) =>
{
sentBodies.Add(body);
var receipt = QueuesModelFactory.SendReceipt(
"msg", DateTimeOffset.UtcNow, DateTimeOffset.UtcNow.AddMinutes(7),
"pop", DateTimeOffset.UtcNow);
return Task.FromResult(Response.FromValue(receipt, new Mock<Response>().Object));
});
var producer = new AzureQueueImeMsiArchiveProducer(
queue.Object, NullLogger<AzureQueueImeMsiArchiveProducer>.Instance);
return (producer, queue, sentBodies);
}
}
@@ -171,7 +171,7 @@ public class BuiltInRulesTests
"updateStatePostponed", "espPhaseDetected", "imeStarted",
"policiesDiscovered", "ignoreCompletedApp", "imeAgentVersion",
"espTrackStatus", "updateName", "updateWin32AppState",
"cancelStuckAndSetCurrent", "imeSessionChange", "imeImpersonation",
"cancelStuckAndSetCurrent", "imeSessionChange", "imeImpersonation", "imeTokenFailure",
"enrollmentCompleted", "updateDoTelemetry", "scriptStarted",
"scriptContext", "scriptExitCode", "scriptOutput", "scriptCompleted",
"resetPlatformScriptContext",
@@ -247,4 +247,64 @@ public class CorporateIdentifierValidatorTests
Assert.False(result.IsValid);
Assert.True(result.IsTransient);
}
[Theory]
[InlineData(null, "Virtual Machine", Serial)]
[InlineData("Microsoft Corporation", null, Serial)]
[InlineData("Microsoft Corporation", "Virtual Machine", null)]
[InlineData("", "Virtual Machine", Serial)]
[InlineData("Microsoft Corporation", "", Serial)]
[InlineData("Microsoft Corporation", "Virtual Machine", "")]
[InlineData(" ", "Virtual Machine", Serial)]
[InlineData("Microsoft Corporation", "Virtual Machine", " ")]
public async Task ValidateAsync_missingHeaderComponent_isDefinitive_withoutGraphCall(
string? manufacturer, string? model, string? serial)
{
// Agents that fail to read a hardware property send the header empty — that is a
// definitive rejection (retries cannot conjure the value), and Graph must not be hit.
var (sut, handler) = BuildSut(Json("""{"value":[]}"""));
var result = await sut.ValidateAsync(TenantId, manufacturer, model, serial);
Assert.False(result.IsValid);
Assert.False(result.IsTransient);
Assert.Contains("not provided", result.ErrorMessage);
Assert.Empty(handler.Requests);
}
[Fact]
public async Task ValidateAsync_specialCharsInSerial_neverReachOdataFilter()
{
// The contains() narrowing uses the NORMALIZED (alphanumeric-only) serial, so an
// OData-breaking quote in a raw serial can neither corrupt nor inject into the filter.
const string hostileSerial = "O'Brien-123'&$top=9999";
var (sut, handler) = BuildSut(Json("""{"value":[]}"""));
var result = await sut.ValidateAsync(TenantId, "Microsoft Corporation", "Virtual Machine", hostileSerial);
Assert.False(result.IsValid);
var url = Uri.UnescapeDataString(Assert.Single(handler.Requests).RequestUri!.ToString());
Assert.Contains("contains(importedDeviceIdentifier,'OBRIEN123TOP9999')", url);
Assert.DoesNotContain("O'Brien", url);
}
[Fact]
public async Task ValidateAsync_pageCapExceeded_isTransient_andNotCached()
{
// Stopping mid-list must NOT be classified "not found" — a cached false negative would
// block a legitimately registered device for the negative-cache TTL.
var (sut, handler) = BuildSut(Json("""
{"value":[{"importedDeviceIdentityType":"manufacturerModelSerial","importedDeviceIdentifier":"DELL,XPS,OTHER1"}],
"@odata.nextLink":"https://graph.microsoft.com/beta/deviceManagement/importedDeviceIdentities?$skiptoken=more"}
"""));
var result = await sut.ValidateAsync(TenantId, "Microsoft Corporation", "Virtual Machine", Serial);
Assert.False(result.IsValid);
Assert.True(result.IsTransient);
Assert.Contains("page budget", result.ErrorMessage);
// Filtered scan pages up to its cap of 5, and the in-call retry runs a second sweep —
// proof the transient path went through the retry loop instead of caching.
Assert.Equal(10, handler.Requests.Count);
}
}
@@ -39,6 +39,26 @@ public class GetImeVersionHistoryRedactionTests
Assert.DoesNotContain("firstSeenSessionId", json, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// The MSI-archive columns (ImeMsiArchiver) are operator infrastructure — blob paths,
/// hashes and source URLs of the platform's own archive have no business in a tenant
/// member's payload. Pinned like the identifiers so the boundary survives refactors.
/// </summary>
[Fact]
public void NonGlobalCaller_DropsMsiArchiveFields()
{
var json = SerializeFor(hasGlobalScope: false, MakeEntry());
Assert.DoesNotContain("msiArchiveStatus", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("msiArchiveBlobPath", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("msiSha256", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("msiBytes", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("msiSourceUrl", json, StringComparison.OrdinalIgnoreCase);
// Values too, not just keys (same rationale as the identifier-value test below).
Assert.DoesNotContain("4d6b85fd3c49cf227ab4711308bca1f3c701cad9bff1799da4c6b6616066bb5e", json, StringComparison.OrdinalIgnoreCase);
Assert.DoesNotContain("imeswdb-afd-secondary", json, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// The values, not just the property names: a rename of the model property must not let the
/// identifier survive under a different key. Asserting on the GUIDs catches that.
@@ -101,7 +121,12 @@ public class GetImeVersionHistoryRedactionTests
var keys = doc.RootElement[0].EnumerateObject().Select(p => p.Name).OrderBy(n => n).ToArray();
Assert.Equal(
new[] { "firstSeenAt", "firstSeenSessionId", "firstSeenTenantId", "lastSeenAt", "sessionCount", "version" },
new[]
{
"firstSeenAt", "firstSeenSessionId", "firstSeenTenantId", "lastSeenAt",
"msiArchiveBlobPath", "msiArchiveStatus", "msiBytes", "msiSha256", "msiSourceUrl",
"sessionCount", "version",
},
keys);
}
@@ -185,5 +210,10 @@ public class GetImeVersionHistoryRedactionTests
FirstSeenTenantId = "57f34dd1-6a42-47f9-9bc0-b921fa6caa30",
LastSeenAt = Last,
SessionCount = 3109,
MsiArchiveStatus = "Archived",
MsiArchiveBlobPath = $"{version}/IntuneWindowsAgent.msi",
MsiSha256 = "4d6b85fd3c49cf227ab4711308bca1f3c701cad9bff1799da4c6b6616066bb5e",
MsiBytes = 13_107_200,
MsiSourceUrl = "https://imeswdb-afd-secondary.manage.microsoft.com/IntuneWindowsAgent.msi",
};
}
@@ -0,0 +1,261 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Azure;
using Azure.Storage.Queues;
using Azure.Storage.Queues.Models;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Functions.Services.Ime;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Newtonsoft.Json;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Worker-level tests for the ime-msi-archive queue consumer (daily review 2026-08-18:
/// previously untested). The worker's contract: merge the archiver's outcome onto the
/// ImeVersionHistory row on EVERY attempt (also failures, so operators see a stuck
/// download), delete the message for terminal outcomes, rethrow for retryable ones
/// (visibility-timeout ladder), pause — not drop — while ImeMsiArchivingEnabled is off,
/// and drop version-less envelopes without invoking the archiver.
/// </summary>
public class ImeMsiArchiveQueueWorkerTests
{
private const string Version = "1.104.102.0";
[Fact]
public async Task Worker_success_merges_row_and_deletes_message()
{
var harness = new Harness();
harness.Archiver.Result = new ImeMsiArchiveResult(
true, ImeMsiArchiver.Statuses.Archived, Retryable: false,
$"{Version}/IntuneWindowsAgent.msi", "abc123", 42L, ImeMsiArchiver.CanonicalMsiUrl);
harness.EnqueueMessage(JsonConvert.SerializeObject(Envelope()));
await harness.RunUntilAsync(() => harness.MainQueueDeleted());
harness.Repo.Verify(r => r.UpdateImeVersionArchiveInfoAsync(
Version, ImeMsiArchiver.Statuses.Archived,
$"{Version}/IntuneWindowsAgent.msi", "abc123", 42L, ImeMsiArchiver.CanonicalMsiUrl),
Times.Once);
harness.MainQueue.Verify(q => q.DeleteMessageAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()),
Times.Once);
}
[Fact]
public async Task Worker_retryable_failure_merges_row_but_leaves_message()
{
var harness = new Harness();
harness.Archiver.Result = new ImeMsiArchiveResult(
false, ImeMsiArchiver.Statuses.FailedDownload, Retryable: true,
null, null, null, ImeMsiArchiver.CanonicalMsiUrl);
harness.EnqueueMessage(JsonConvert.SerializeObject(Envelope()));
await harness.RunUntilAsync(() => harness.Archiver.Calls > 0);
// Row always tells the truth — the failure status IS merged...
harness.Repo.Verify(r => r.UpdateImeVersionArchiveInfoAsync(
Version, ImeMsiArchiver.Statuses.FailedDownload,
null, null, null, ImeMsiArchiver.CanonicalMsiUrl),
Times.AtLeastOnce);
// ...but the message stays for the visibility-timeout retry → poison ladder.
harness.MainQueue.Verify(q => q.DeleteMessageAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task Worker_permanent_failure_merges_row_and_completes_message()
{
var harness = new Harness();
harness.Archiver.Result = new ImeMsiArchiveResult(
false, ImeMsiArchiver.Statuses.FailedTooLarge, Retryable: false,
null, null, 999L, ImeMsiArchiver.CanonicalMsiUrl);
harness.EnqueueMessage(JsonConvert.SerializeObject(Envelope()));
await harness.RunUntilAsync(() => harness.MainQueueDeleted());
harness.Repo.Verify(r => r.UpdateImeVersionArchiveInfoAsync(
Version, ImeMsiArchiver.Statuses.FailedTooLarge,
null, null, 999L, ImeMsiArchiver.CanonicalMsiUrl),
Times.Once);
harness.MainQueue.Verify(q => q.DeleteMessageAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()),
Times.Once);
}
[Fact]
public async Task Worker_pauses_while_archiving_disabled()
{
var harness = new Harness(archivingEnabled: false);
harness.EnqueueMessage(JsonConvert.SerializeObject(Envelope()));
await harness.RunForAsync(TimeSpan.FromMilliseconds(500));
// Pause, not drop: no receive, no archive attempt — the message parks in the queue.
harness.MainQueue.Verify(q => q.ReceiveMessagesAsync(
It.IsAny<int>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()),
Times.Never);
Assert.Equal(0, harness.Archiver.Calls);
}
[Fact]
public async Task Worker_drops_envelope_without_version_before_archiver()
{
var harness = new Harness();
harness.EnqueueMessage(JsonConvert.SerializeObject(new ImeMsiArchiveEnvelope
{
Version = string.Empty,
EnqueuedAt = DateTime.UtcNow,
}));
await harness.RunUntilAsync(() => harness.MainQueueDeleted());
Assert.Equal(0, harness.Archiver.Calls);
harness.Repo.Verify(r => r.UpdateImeVersionArchiveInfoAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string?>(),
It.IsAny<string?>(), It.IsAny<long?>(), It.IsAny<string?>()),
Times.Never);
}
// ============================================================ Helpers ====
private static ImeMsiArchiveEnvelope Envelope() => new()
{
Version = Version,
MsiDownloadUrl = ImeMsiArchiver.CanonicalMsiUrl,
MsiMatchedBy = "productVersion",
TenantId = "11111111-1111-1111-1111-111111111111",
SessionId = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
EnqueuedAt = DateTime.UtcNow,
};
/// <summary>Scripted archiver — the worker only consumes the classified result.</summary>
private sealed class StubArchiver : ImeMsiArchiver
{
private int _calls;
public ImeMsiArchiveResult Result { get; set; } = new(
true, Statuses.Archived, Retryable: false, null, null, null, null);
public int Calls => Volatile.Read(ref _calls);
public override Task<ImeMsiArchiveResult> ArchiveAsync(
ImeMsiArchiveEnvelope envelope, CancellationToken cancellationToken = default)
{
Interlocked.Increment(ref _calls);
return Task.FromResult(Result);
}
}
/// <summary>Mock-queue poll-loop harness, same shape as SessionDeletionWorkerTests.</summary>
private sealed class Harness
{
public Mock<QueueClient> MainQueue { get; }
public Mock<QueueClient> PoisonQueue { get; }
public StubArchiver Archiver { get; } = new();
public Mock<ISessionRepository> Repo { get; } = new();
public ImeMsiArchiveQueueWorker Sut { get; }
private readonly Queue<QueueMessage> _pendingMessages = new();
public Harness(bool archivingEnabled = true)
{
MainQueue = new Mock<QueueClient>();
PoisonQueue = new Mock<QueueClient>();
MainQueue.Setup(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync((Response?)null);
PoisonQueue.Setup(q => q.CreateIfNotExistsAsync(
It.IsAny<IDictionary<string, string>>(), It.IsAny<CancellationToken>()))
.ReturnsAsync((Response?)null);
MainQueue.Setup(q => q.ReceiveMessagesAsync(
It.IsAny<int>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.Returns<int, TimeSpan?, CancellationToken>((maxMessages, _, _) =>
{
var batch = new List<QueueMessage>();
lock (_pendingMessages)
{
while (batch.Count < maxMessages && _pendingMessages.Count > 0)
batch.Add(_pendingMessages.Dequeue());
}
return Task.FromResult(Response.FromValue(batch.ToArray(), new Mock<Response>().Object));
});
MainQueue.Setup(q => q.DeleteMessageAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(new Mock<Response>().Object);
MainQueue.Setup(q => q.UpdateMessageAsync(
It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<TimeSpan>(), It.IsAny<CancellationToken>()))
.Returns<string, string, string, TimeSpan, CancellationToken>((_, _, _, vis, _) =>
{
var updated = QueuesModelFactory.UpdateReceipt(
"pop-extended-" + Guid.NewGuid().ToString("N"),
DateTimeOffset.UtcNow.Add(vis));
return Task.FromResult(Response.FromValue(updated, new Mock<Response>().Object));
});
var adminConfig = new Mock<AdminConfigurationService>(
Mock.Of<IConfigRepository>(),
NullLogger<AdminConfigurationService>.Instance,
new MemoryCache(new MemoryCacheOptions()));
adminConfig.Setup(a => a.GetConfigurationAsync())
.ReturnsAsync(new AdminConfiguration { ImeMsiArchivingEnabled = archivingEnabled });
Sut = new ImeMsiArchiveQueueWorker(
MainQueue.Object, PoisonQueue.Object,
Archiver, Repo.Object, adminConfig.Object,
NullLogger<ImeMsiArchiveQueueWorker>.Instance,
pollInterval: TimeSpan.FromMilliseconds(50));
}
public void EnqueueMessage(string body)
{
var msg = QueuesModelFactory.QueueMessage(
messageId: "msg-" + Guid.NewGuid().ToString("N"),
popReceipt: "pop-" + Guid.NewGuid().ToString("N"),
body: new BinaryData(body),
dequeueCount: 1);
lock (_pendingMessages) _pendingMessages.Enqueue(msg);
}
public async Task RunForAsync(TimeSpan duration)
{
using var cts = new CancellationTokenSource(duration);
try { await Sut.StartAsync(cts.Token); }
catch (OperationCanceledException) { }
try { await Task.Delay(duration, cts.Token); }
catch (OperationCanceledException) { }
try { await Sut.StopAsync(CancellationToken.None); }
catch (OperationCanceledException) { }
}
public async Task RunUntilAsync(Func<bool> condition, TimeSpan? timeout = null)
{
var deadline = DateTime.UtcNow + (timeout ?? TimeSpan.FromSeconds(10));
using var cts = new CancellationTokenSource();
try { await Sut.StartAsync(cts.Token); }
catch (OperationCanceledException) { }
while (!condition() && DateTime.UtcNow < deadline)
await Task.Delay(25);
cts.Cancel();
try { await Sut.StopAsync(CancellationToken.None); }
catch (OperationCanceledException) { }
}
public bool MainQueueDeleted() =>
MainQueue.Invocations.Any(i => i.Method.Name == nameof(QueueClient.DeleteMessageAsync));
}
}
@@ -0,0 +1,445 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Security.Cryptography;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services.Ime;
using AutopilotMonitor.Shared.Models;
using Azure;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// The IME-installer archiver downloads a newly sighted IME build into the permanent
/// ime-archive container. Its contract: NEVER throw (worker decides retry from the result),
/// treat the agent-supplied URL as untrusted (allowlist, else canonical fallback), enforce
/// the size cap both at Content-Length preflight and mid-stream, and stay idempotent under
/// queue re-delivery (409 on the write-once upload == archived). Tests pin every status
/// branch via a recording subclass — no Azurite, no HTTP.
/// </summary>
public class ImeMsiArchiverTests
{
private const string Version = "1.104.102.0";
private const string EventUrl = "https://imeswdb-afd-secondary.manage.microsoft.com/IntuneWindowsAgent.msi";
// =========================================================================
// URL allowlist (static — the SSRF boundary)
// =========================================================================
[Theory]
[InlineData(EventUrl, true)]
[InlineData("https://swda01-mscdn.manage.microsoft.com/IntuneWindowsAgent.msi", true)]
[InlineData("https://manage.microsoft.com/IntuneWindowsAgent.msi", true)] // apex host
[InlineData("https://x.manage.microsoft.com/path/IntuneWindowsAgent.msi?sv=1&sig=2", true)] // query ignored
[InlineData("https://x.manage.microsoft.com/intunewindowsagent.MSI", true)] // case-insensitive
[InlineData("http://imeswdb-afd-secondary.manage.microsoft.com/IntuneWindowsAgent.msi", false)] // https only
[InlineData("https://evil.example.com/IntuneWindowsAgent.msi", false)]
[InlineData("https://manage.microsoft.com.evil.com/IntuneWindowsAgent.msi", false)] // suffix spoof
[InlineData("https://xmanage.microsoft.com/IntuneWindowsAgent.msi", false)] // needs the dot
[InlineData("https://x.manage.microsoft.com/SomethingElse.msi", false)] // wrong filename
[InlineData("https://x.manage.microsoft.com/IntuneWindowsAgent.msi.exe", false)]
[InlineData("not a url", false)]
[InlineData("", false)]
[InlineData(null, false)]
public void IsAllowedMsiUrl_cases(string? url, bool expected)
{
Assert.Equal(expected, ImeMsiArchiver.IsAllowedMsiUrl(url));
}
// =========================================================================
// Version guard (blob path is built from untrusted input)
// =========================================================================
[Theory]
[InlineData("")]
[InlineData("1.104.102.0.1")] // too many components
[InlineData("1.104../escape")]
[InlineData("latest")]
[InlineData("1.104.102.0 ")]
public async Task ArchiveAsync_BadVersion_PermanentFailure_NoDownload(string version)
{
var archiver = new RecordingArchiver();
var result = await archiver.ArchiveAsync(Envelope(version: version));
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedBadVersion, result.Status);
Assert.False(result.Retryable);
Assert.Equal(0, archiver.DownloadCalls + archiver.UploadCalls);
}
// =========================================================================
// URL selection
// =========================================================================
[Fact]
public async Task ArchiveAsync_AllowedEventUrl_IsUsed()
{
var archiver = new RecordingArchiver();
var result = await archiver.ArchiveAsync(Envelope(url: EventUrl));
Assert.True(result.Success);
Assert.Equal(EventUrl, archiver.DownloadLastUrl);
Assert.Equal(EventUrl, result.SourceUrl);
}
[Theory]
[InlineData("https://evil.example.com/IntuneWindowsAgent.msi")] // allowlist reject
[InlineData(null)] // pre-enrichment agent
public async Task ArchiveAsync_MissingOrRejectedEventUrl_FallsBackToCanonical(string? url)
{
var archiver = new RecordingArchiver();
var result = await archiver.ArchiveAsync(Envelope(url: url));
Assert.True(result.Success);
Assert.Equal(ImeMsiArchiver.CanonicalMsiUrl, archiver.DownloadLastUrl);
Assert.Equal(ImeMsiArchiver.CanonicalMsiUrl, result.SourceUrl);
}
// =========================================================================
// Happy path
// =========================================================================
[Fact]
public async Task ArchiveAsync_HappyPath_UploadsMsiAndProvenance_WithHashAndSize()
{
var payload = new byte[3 * 1024 * 1024];
new Random(42).NextBytes(payload);
var expectedSha = Convert.ToHexString(SHA256.HashData(payload)).ToLowerInvariant();
var archiver = new RecordingArchiver { Payload = payload };
var result = await archiver.ArchiveAsync(Envelope());
Assert.True(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.Archived, result.Status);
Assert.Equal($"{Version}/IntuneWindowsAgent.msi", result.BlobPath);
Assert.Equal(expectedSha, result.Sha256);
Assert.Equal(payload.Length, result.SizeBytes);
Assert.Equal(1, archiver.UploadCalls);
Assert.Equal($"{Version}/IntuneWindowsAgent.msi", archiver.UploadLastPath);
Assert.Equal(payload.Length, archiver.UploadedByteCount);
Assert.Equal(1, archiver.ProvenanceCalls);
Assert.Equal($"{Version}/provenance.json", archiver.ProvenanceLastPath);
Assert.Contains(expectedSha, archiver.ProvenanceLastJson);
Assert.Contains(Version, archiver.ProvenanceLastJson);
}
// =========================================================================
// Size cap — preflight and mid-stream
// =========================================================================
[Fact]
public async Task ArchiveAsync_ContentLengthAboveCap_Rejected_NoUpload()
{
var archiver = new RecordingArchiver
{
MaxDownloadSizeMB = 1,
Payload = new byte[2 * 1024 * 1024],
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedTooLarge, result.Status);
Assert.False(result.Retryable);
Assert.Equal(0, archiver.UploadCalls);
}
[Fact]
public async Task ArchiveAsync_UnknownContentLength_CapStillEnforcedMidStream()
{
// A lying/absent Content-Length must not bypass the cap — the hashing wrapper
// aborts the stream as soon as the byte count crosses it.
var archiver = new RecordingArchiver
{
MaxDownloadSizeMB = 1,
Payload = new byte[2 * 1024 * 1024],
ReportContentLength = false,
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedTooLarge, result.Status);
Assert.False(result.Retryable);
Assert.Equal(0, archiver.ProvenanceCalls);
}
[Fact]
public async Task ArchiveAsync_CapZero_MeansUnlimited()
{
var archiver = new RecordingArchiver
{
MaxDownloadSizeMB = 0,
Payload = new byte[2 * 1024 * 1024],
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.True(result.Success);
}
// =========================================================================
// Idempotency under queue re-delivery
// =========================================================================
[Fact]
public async Task ArchiveAsync_BlobAlreadyExists_TreatedAsArchived_NotRetryable()
{
var archiver = new RecordingArchiver
{
UploadException = new RequestFailedException(409, "BlobAlreadyExists"),
ProvenanceExists = true,
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.True(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.Archived, result.Status);
Assert.False(result.Retryable);
Assert.Equal($"{Version}/IntuneWindowsAgent.msi", result.BlobPath);
// Sidecar already there — the re-delivery must not rewrite it.
Assert.Equal(0, archiver.ProvenanceCalls);
}
// =========================================================================
// Provenance-write gap (daily review 2026-08-18): MSI upload is write-once,
// so a first attempt that dies between the MSI and the provenance upload
// funnels every retry into the 409 path — which must heal the sidecar.
// =========================================================================
[Fact]
public async Task ArchiveAsync_ProvenanceUploadFails_RetryableError()
{
var archiver = new RecordingArchiver
{
ProvenanceException = new RequestFailedException(500, "storage hiccup"),
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedError, result.Status);
Assert.True(result.Retryable);
}
[Fact]
public async Task ArchiveAsync_RedeliveryWithMissingProvenance_HealsSidecarFromArchivedBytes()
{
// The archived blob's content deliberately differs from this attempt's re-download —
// the healed provenance must describe the ARCHIVED bytes, not the fresh download.
var archivedPayload = new byte[1024];
new Random(7).NextBytes(archivedPayload);
var expectedSha = Convert.ToHexString(SHA256.HashData(archivedPayload)).ToLowerInvariant();
var archiver = new RecordingArchiver
{
UploadException = new RequestFailedException(409, "BlobAlreadyExists"),
ProvenanceExists = false,
ArchivedPayload = archivedPayload,
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.True(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.Archived, result.Status);
Assert.False(result.Retryable);
Assert.Equal(expectedSha, result.Sha256);
Assert.Equal(archivedPayload.Length, result.SizeBytes);
Assert.Equal(1, archiver.ProvenanceCalls);
Assert.Equal($"{Version}/provenance.json", archiver.ProvenanceLastPath);
Assert.Contains(expectedSha, archiver.ProvenanceLastJson);
Assert.Contains(Version, archiver.ProvenanceLastJson);
}
[Fact]
public async Task ArchiveAsync_ProvenanceFailure_ThenRedelivery_EndsHealed()
{
// THE regression scenario end to end on one instance: attempt 1 uploads the MSI but
// the provenance write dies; attempt 2 (queue re-delivery) gets 409 on the write-once
// MSI and must still produce the sidecar.
var archiver = new RecordingArchiver
{
ProvenanceException = new RequestFailedException(500, "storage hiccup"),
};
var first = await archiver.ArchiveAsync(Envelope());
Assert.False(first.Success);
Assert.True(first.Retryable);
archiver.ProvenanceException = null;
archiver.UploadException = new RequestFailedException(409, "BlobAlreadyExists");
archiver.ProvenanceExists = false; // attempt 1 never landed the sidecar
var second = await archiver.ArchiveAsync(Envelope());
Assert.True(second.Success);
Assert.Equal(ImeMsiArchiver.Statuses.Archived, second.Status);
Assert.Equal($"{Version}/provenance.json", archiver.ProvenanceLastPath);
Assert.NotNull(second.Sha256);
}
[Fact]
public async Task ArchiveAsync_HealFails_RetryableError()
{
// Heal errors ride the normal retry ladder — the next re-delivery tries again.
var archiver = new RecordingArchiver
{
UploadException = new RequestFailedException(409, "BlobAlreadyExists"),
ProvenanceExists = false,
ArchivedOpenException = new RequestFailedException(500, "read failed"),
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedError, result.Status);
Assert.True(result.Retryable);
Assert.Equal(0, archiver.ProvenanceCalls);
}
// =========================================================================
// Transient failures — retryable, never thrown
// =========================================================================
[Fact]
public async Task ArchiveAsync_DownloadFails_RetryableDownloadStatus()
{
var archiver = new RecordingArchiver
{
DownloadException = new HttpRequestException("boom"),
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedDownload, result.Status);
Assert.True(result.Retryable);
}
[Fact]
public async Task ArchiveAsync_Timeout_RetryableTimeoutStatus()
{
var archiver = new RecordingArchiver
{
DownloadException = new OperationCanceledException(),
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedTimeout, result.Status);
Assert.True(result.Retryable);
}
[Fact]
public async Task ArchiveAsync_UnexpectedUploadError_RetryableErrorStatus()
{
var archiver = new RecordingArchiver
{
UploadException = new InvalidOperationException("storage hiccup"),
};
var result = await archiver.ArchiveAsync(Envelope());
Assert.False(result.Success);
Assert.Equal(ImeMsiArchiver.Statuses.FailedError, result.Status);
Assert.True(result.Retryable);
}
[Fact]
public async Task ArchiveAsync_CallerCancellation_Propagates()
{
// Host shutdown must reach the worker's dispatch loop so the message stays queued.
var archiver = new RecordingArchiver();
using var cts = new CancellationTokenSource();
cts.Cancel();
archiver.DownloadException = new OperationCanceledException(cts.Token);
await Assert.ThrowsAnyAsync<OperationCanceledException>(
() => archiver.ArchiveAsync(Envelope(), cts.Token));
}
// =========================================================================
// Helpers
// =========================================================================
private static ImeMsiArchiveEnvelope Envelope(string version = Version, string? url = EventUrl) =>
new()
{
Version = version,
MsiDownloadUrl = url,
MsiMatchedBy = "productVersion",
TenantId = "11111111-1111-1111-1111-111111111111",
SessionId = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
EnqueuedAt = DateTime.UtcNow,
};
private sealed class RecordingArchiver : ImeMsiArchiver
{
public byte[] Payload { get; set; } = new byte[2048];
public bool ReportContentLength { get; set; } = true;
public int MaxDownloadSizeMB { get; set; } = 250;
public Exception? DownloadException { get; set; }
public Exception? UploadException { get; set; }
public Exception? ProvenanceException { get; set; }
public bool ProvenanceExists { get; set; } = true;
public byte[]? ArchivedPayload { get; set; }
public Exception? ArchivedOpenException { get; set; }
public int DownloadCalls { get; private set; }
public string? DownloadLastUrl { get; private set; }
public int UploadCalls { get; private set; }
public string? UploadLastPath { get; private set; }
public long UploadedByteCount { get; private set; }
public int ProvenanceCalls { get; private set; }
public string? ProvenanceLastPath { get; private set; }
public string ProvenanceLastJson { get; private set; } = string.Empty;
protected override Task<AdminConfiguration> GetAdminConfigurationAsync()
=> Task.FromResult(new AdminConfiguration { MaxImeMsiDownloadSizeMB = MaxDownloadSizeMB });
protected override Task<(Stream Content, long ContentLength)> OpenDownloadAsync(
string url, CancellationToken cancellationToken)
{
DownloadCalls++;
DownloadLastUrl = url;
if (DownloadException is not null) throw DownloadException;
return Task.FromResult(
((Stream)new MemoryStream(Payload), ReportContentLength ? Payload.LongLength : -1L));
}
protected override async Task UploadMsiAsync(string blobPath, Stream content, CancellationToken cancellationToken)
{
UploadCalls++;
UploadLastPath = blobPath;
// Drain the stream like the real blob upload would — this is what drives the
// hashing/cap wrapper (and mid-stream aborts surface here as exceptions).
var buffer = new byte[64 * 1024];
int read;
while ((read = await content.ReadAsync(buffer, cancellationToken)) > 0)
UploadedByteCount += read;
if (UploadException is not null) throw UploadException;
}
protected override Task UploadProvenanceAsync(string blobPath, string json, CancellationToken cancellationToken)
{
if (ProvenanceException is not null) throw ProvenanceException;
ProvenanceCalls++;
ProvenanceLastPath = blobPath;
ProvenanceLastJson = json;
return Task.CompletedTask;
}
protected override Task<bool> ProvenanceExistsAsync(string blobPath, CancellationToken cancellationToken)
=> Task.FromResult(ProvenanceExists);
protected override Task<Stream> OpenArchivedMsiAsync(string blobPath, CancellationToken cancellationToken)
{
if (ArchivedOpenException is not null) throw ArchivedOpenException;
return Task.FromResult<Stream>(new MemoryStream(ArchivedPayload ?? Payload));
}
}
}
@@ -197,6 +197,53 @@ public class TimeAttributionCalculatorTests
Assert.True(b.QualityFlags.HasFlag(TimeAttributionFlags.PartialObservation));
}
// ── prior-enrollment residue ────────────────────────────────────────────
[Fact]
public void HistoricImeReplay_SetsPriorEnrollmentResidue_AndExcludesFromFleetStats()
{
// Re-enrollment without a wipe (session f475e697): the surviving IME log pulls the
// phase anchors ahead of the real ESP page — the segments must be flagged as unfit.
var events = new List<EnrollmentEvent>
{
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
Evt(T0.AddSeconds(2), "historic_ime_replay_detected"),
Evt(T0.AddMinutes(1), "esp_phase_changed", EnrollmentPhase.AppsDevice),
Evt(T0.AddMinutes(10), "enrollment_complete"),
};
var b = TimeAttributionCalculator.Compute(
Input(events, completedAt: T0.AddMinutes(10), durationSeconds: 600))!;
AssertExactPartition(b);
Assert.True(b.QualityFlags.HasFlag(TimeAttributionFlags.PriorEnrollmentResidue));
Assert.True(TimeAttributionFlagQuality.ExcludesFromFleetStats(b.QualityFlags));
}
[Fact]
public void RegistryAppBaseline_Alone_DoesNotSetResidueFlag()
{
// A registry_app_baseline with successes is by-design normal for WDP (DPP Batch-1
// apps install before the agent exists) — treating it as residue would starve the
// device_preparation class's fleet aggregates. Only the IME-log replay signal counts.
var events = new List<EnrollmentEvent>
{
Evt(T0, "agent_started", EnrollmentPhase.DeviceSetup),
Evt(T0.AddSeconds(2), "registry_app_baseline", data: new Dictionary<string, object>
{
["totalApps"] = "3", ["successCount"] = "3",
["errorCount"] = "0", ["inProgressCount"] = "0",
}),
Evt(T0.AddMinutes(10), "enrollment_complete"),
};
var b = TimeAttributionCalculator.Compute(
Input(events, completedAt: T0.AddMinutes(10), durationSeconds: 600))!;
AssertExactPartition(b);
Assert.False(b.QualityFlags.HasFlag(TimeAttributionFlags.PriorEnrollmentResidue));
}
// ── WhiteGlove: two windows in ONE session, pause never attributed ──────
[Fact]
@@ -818,6 +818,8 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "VulnerabilityCorrelationEnabled", config.VulnerabilityCorrelationEnabled },
{ "EnableIndexDualWrite", config.EnableIndexDualWrite },
{ "AutoApproveNewTenants", config.AutoApproveNewTenants },
{ "ImeMsiArchivingEnabled", config.ImeMsiArchivingEnabled },
{ "MaxImeMsiDownloadSizeMB", config.MaxImeMsiDownloadSizeMB },
{ "SelfServiceAppHomingEnabled", config.SelfServiceAppHomingEnabled },
{ "SessionDeletionKillSwitch", config.SessionDeletionKillSwitch },
{ "VulnerabilityDataLastSyncUtc", config.VulnerabilityDataLastSyncUtc },
@@ -885,6 +887,8 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
VulnerabilityCorrelationEnabled = entity.GetBoolean("VulnerabilityCorrelationEnabled") ?? true,
EnableIndexDualWrite = entity.GetBoolean("EnableIndexDualWrite") ?? false,
AutoApproveNewTenants = entity.GetBoolean("AutoApproveNewTenants") ?? false,
ImeMsiArchivingEnabled = entity.GetBoolean("ImeMsiArchivingEnabled") ?? true,
MaxImeMsiDownloadSizeMB = entity.GetInt32("MaxImeMsiDownloadSizeMB") ?? 250,
SelfServiceAppHomingEnabled = entity.GetBoolean("SelfServiceAppHomingEnabled") ?? false,
SessionDeletionKillSwitch = entity.GetBoolean("SessionDeletionKillSwitch") ?? false,
VulnerabilityDataLastSyncUtc = entity.GetString("VulnerabilityDataLastSyncUtc"),
@@ -172,6 +172,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
public Task<List<ImeVersionHistoryEntry>> GetImeVersionHistoryAsync()
=> _storage.GetImeVersionHistoryAsync();
public Task UpdateImeVersionArchiveInfoAsync(
string version, string status, string? blobPath, string? sha256, long? sizeBytes, string? sourceUrl)
=> _storage.UpdateImeVersionArchiveInfoAsync(version, status, blobPath, sha256, sizeBytes, sourceUrl);
public async Task<bool> StoreEventAsync(EnrollmentEvent evt)
{
var result = await _storage.StoreEventAsync(evt);
@@ -70,7 +70,9 @@ public sealed class TimeAttributionInput
public static class TimeAttributionCalculator
{
/// <summary>Bump on any semantic change so aggregates never mix definitions (truthfulness rule 8).</summary>
public const int CurrentVersion = 1;
// v2: PriorEnrollmentResidue flag — sessions whose phase anchors are distorted by
// pre-enrollment state on disk are now flagged and excluded from fleet segment stats.
public const int CurrentVersion = 2;
/// <summary>
/// Enrollment class for fleet aggregation — classes are NEVER mixed in one aggregate (a
@@ -164,6 +166,15 @@ public static class TimeAttributionCalculator
if (events.Any(e => e.EventType == Constants.EventTypes.AgentLateStart))
flags |= TimeAttributionFlags.PartialObservation;
// Prior-enrollment residue (session f475e697): an IME log predating this enrollment
// means the disk was not wiped — pre-installed apps complete as instant detections and
// pull the phase anchors ahead of the real ESP page, so the segment durations lie.
// historic_ime_replay_detected is the ONLY trigger on purpose: registry_app_baseline
// with successes is by-design normal for WDP (DPP Batch-1 apps install before the
// agent exists) and would starve that class's aggregates (see DurationCriticalFlags).
if (events.Any(e => e.EventType == Constants.EventTypes.HistoricImeReplayDetected))
flags |= TimeAttributionFlags.PriorEnrollmentResidue;
return new SessionTimeBreakdown
{
TenantId = input.TenantId,
@@ -334,6 +334,26 @@ builder.Services.AddSingleton<
builder.Services.AddHostedService<
AutopilotMonitor.Functions.Services.Vulnerability.VulnerabilityCorrelateQueueWorker>();
// IME-installer archiving on first fleet-wide sighting of a new IME version (~monthly).
// Producer sits in EventIngestProcessor's new-version continuation; the worker downloads
// the MSI from the CSP-reported (allowlisted) URL into the permanent ime-archive container
// and merges the outcome onto the ImeVersionHistory row. Typed HttpClient: no redirects
// (SSRF posture — the allowlisted host must serve directly), long timeout owned by the
// archiver's own CTS, transient retry via the shared external-data policy.
builder.Services.AddHttpClient<AutopilotMonitor.Functions.Services.Ime.ImeMsiArchiver>()
.ConfigurePrimaryHttpMessageHandler(() => new System.Net.Http.SocketsHttpHandler
{
AllowAutoRedirect = false,
ConnectTimeout = TimeSpan.FromSeconds(10),
})
.ConfigureHttpClient(client => client.Timeout = Timeout.InfiniteTimeSpan)
.AddPolicyHandler((sp, _) => sp.GetRequiredService<ResiliencePolicies>().ExternalDataApi);
builder.Services.AddSingleton<
AutopilotMonitor.Functions.Services.Ime.IImeMsiArchiveProducer,
AutopilotMonitor.Functions.Services.Ime.AzureQueueImeMsiArchiveProducer>();
builder.Services.AddHostedService<
AutopilotMonitor.Functions.Services.Ime.ImeMsiArchiveQueueWorker>();
// Delayed tenant auto-approve (public availability). Producer fires at first-login signup
// with a ~1-minute visibility delay; the worker activates the tenant via the shared
// TenantApprovalService only if AdminConfiguration.AutoApproveNewTenants is enabled at
@@ -43,6 +43,7 @@ namespace AutopilotMonitor.Functions.Services
private readonly AutopilotMonitor.Functions.Services.Analyze.IAnalyzeOnEnrollmentEndProducer _analyzeProducer;
private readonly AutopilotMonitor.Functions.Services.Analyze.InterimTriggerRegistry _interimTriggerRegistry;
private readonly IVulnerabilityCorrelateProducer _vulnProducer;
private readonly Ime.IImeMsiArchiveProducer _imeMsiArchiveProducer;
public EventIngestProcessor(
ILogger<EventIngestProcessor> logger,
@@ -58,7 +59,8 @@ namespace AutopilotMonitor.Functions.Services
TelemetryClient telemetryClient,
AutopilotMonitor.Functions.Services.Analyze.IAnalyzeOnEnrollmentEndProducer analyzeProducer,
AutopilotMonitor.Functions.Services.Analyze.InterimTriggerRegistry interimTriggerRegistry,
IVulnerabilityCorrelateProducer vulnProducer)
IVulnerabilityCorrelateProducer vulnProducer,
Ime.IImeMsiArchiveProducer imeMsiArchiveProducer)
{
_logger = logger;
_sessionRepo = sessionRepo;
@@ -74,6 +76,7 @@ namespace AutopilotMonitor.Functions.Services
_analyzeProducer = analyzeProducer;
_interimTriggerRegistry = interimTriggerRegistry;
_vulnProducer = vulnProducer;
_imeMsiArchiveProducer = imeMsiArchiveProducer;
}
/// <summary>
@@ -129,6 +132,14 @@ namespace AutopilotMonitor.Functions.Services
.ContinueWith(t => _logger.LogWarning(t.Exception?.InnerException,
"ImeAgentVersion update failed (non-fatal)"), TaskContinuationOptions.OnlyOnFaulted);
// Captured here (not inside the continuation) purely for clarity — the agent's
// CSP-registry enrichment; absent on agent builds before 2026-08-17.
string? msiDownloadUrl = null, msiMatchedBy = null;
if (imeVersionEvent.Data.TryGetValue("msiDownloadUrl", out var urlObj))
msiDownloadUrl = urlObj?.ToString();
if (imeVersionEvent.Data.TryGetValue("msiMatchedBy", out var matchedByObj))
msiMatchedBy = matchedByObj?.ToString();
_ = _sessionRepo.RecordImeVersionAsync(imeVersion, request.TenantId, request.SessionId)
.ContinueWith(async t =>
{
@@ -141,6 +152,19 @@ namespace AutopilotMonitor.Functions.Services
{
await _opsEventService.RecordNewImeVersionDetectedAsync(
imeVersion, request.TenantId, request.SessionId);
// First fleet-wide sighting → archive the installer binary while
// Microsoft's versionless CDN still serves exactly this build.
// Producer is fail-soft; worker pauses on ImeMsiArchivingEnabled=false.
await _imeMsiArchiveProducer.EnqueueAsync(new ImeMsiArchiveEnvelope
{
Version = imeVersion,
MsiDownloadUrl = msiDownloadUrl,
MsiMatchedBy = msiMatchedBy,
TenantId = request.TenantId,
SessionId = request.SessionId,
EnqueuedAt = DateTime.UtcNow,
});
}
}, TaskScheduler.Default);
}
@@ -0,0 +1,96 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Azure.Storage.Queues;
using AutopilotMonitor.Functions.Services.Queueing;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
namespace AutopilotMonitor.Functions.Services.Ime
{
/// <summary>
/// Azure Storage Queue implementation of <see cref="IImeMsiArchiveProducer"/>. Mirrors
/// <see cref="Analyze.AzureQueueAnalyzeOnEnrollmentEndProducer"/>: Managed Identity
/// preferred, connection-string fallback, Base64 encoding (matches the consumer worker).
/// <para>
/// <b>Fault tolerance:</b> queue-side exceptions are logged but never rethrown — the
/// caller is the ingest path's fire-and-forget continuation. A missed enqueue means the
/// version's installer is not auto-archived; the /ime-decompile skill's blob backfill
/// is the manual fallback.
/// </para>
/// </summary>
public sealed class AzureQueueImeMsiArchiveProducer : IImeMsiArchiveProducer
{
private readonly QueueClient _queueClient;
private readonly ILogger<AzureQueueImeMsiArchiveProducer> _logger;
private int _queueEnsured; // 0 = not yet ensured, 1 = CreateIfNotExistsAsync has run
public AzureQueueImeMsiArchiveProducer(
QueueClientFactory queueFactory,
ILogger<AzureQueueImeMsiArchiveProducer> logger)
{
_logger = logger;
// Base64 encoding matches the consumer (BackgroundService) so messages round-trip.
_queueClient = queueFactory.Create(Constants.QueueNames.ImeMsiArchive);
}
/// <summary>
/// Test seam: construct directly with a mock <see cref="QueueClient"/>. Mirrors
/// <see cref="Deletion.SessionDeletionProducer"/>'s internal test ctor.
/// </summary>
internal AzureQueueImeMsiArchiveProducer(
QueueClient queueClient,
ILogger<AzureQueueImeMsiArchiveProducer> logger)
{
_queueClient = queueClient;
_logger = logger;
}
public async Task EnqueueAsync(ImeMsiArchiveEnvelope envelope, CancellationToken cancellationToken = default)
{
if (envelope is null) return;
if (string.IsNullOrEmpty(envelope.Version))
{
_logger.LogWarning("ImeMsiArchive enqueue skipped — missing Version");
return;
}
await EnsureQueueExistsAsync(cancellationToken).ConfigureAwait(false);
try
{
var body = JsonConvert.SerializeObject(envelope);
await _queueClient.SendMessageAsync(body, cancellationToken: cancellationToken).ConfigureAwait(false);
_logger.LogInformation(
"ImeMsiArchive enqueued (version={Version} urlPresent={UrlPresent} tenant={Tenant} session={Session})",
envelope.Version, !string.IsNullOrEmpty(envelope.MsiDownloadUrl), envelope.TenantId, envelope.SessionId);
}
catch (Exception ex)
{
// Fail-soft — ingest continuation. Skill-side blob backfill is the fallback.
_logger.LogWarning(ex,
"ImeMsiArchive enqueue failed (version={Version}) — /ime-decompile blob backfill is the fallback",
envelope.Version);
}
}
private async Task EnsureQueueExistsAsync(CancellationToken cancellationToken)
{
if (_queueEnsured == 1) return;
try
{
await _queueClient.CreateIfNotExistsAsync(cancellationToken: cancellationToken).ConfigureAwait(false);
Interlocked.Exchange(ref _queueEnsured, 1);
}
catch (Exception ex)
{
// Leave _queueEnsured=0 so a later call retries.
_logger.LogWarning(ex, "ImeMsiArchive queue CreateIfNotExists failed — will retry next enqueue");
}
}
}
}
@@ -0,0 +1,16 @@
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Functions.Services.Ime
{
/// <summary>
/// Enqueues an IME-installer archive job when a new IME version is first sighted.
/// Implementations must be fail-soft: the caller sits on the ingest hot path's
/// fire-and-forget continuation and must never observe an exception.
/// </summary>
public interface IImeMsiArchiveProducer
{
Task EnqueueAsync(ImeMsiArchiveEnvelope envelope, CancellationToken cancellationToken = default);
}
}
@@ -0,0 +1,107 @@
using AutopilotMonitor.Functions.Services.Queueing;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services.Ime
{
/// <summary>
/// Background worker for the <c>ime-msi-archive</c> queue: downloads a newly sighted IME
/// installer into the permanent <c>ime-archive</c> container and merges the outcome onto
/// the ImeVersionHistory row. Volume is ~one message per Microsoft IME release.
/// <para>
/// Retry semantics: the archiver never throws — it classifies. Permanent rejections
/// (bad version, size cap) complete the message with their <c>Failed:*</c> status
/// persisted; transient failures (download/timeout) rethrow so the visibility-timeout
/// retry → poison ladder applies, with the last status visible on the row.
/// <see cref="UseHeartbeat"/> keeps the message invisible across the up-to-10-minute
/// download window. The status row-merge happens on EVERY attempt so operators can see
/// a stuck download as <c>Failed:Download</c> instead of nothing.
/// </para>
/// </summary>
public sealed class ImeMsiArchiveQueueWorker : QueuePollingWorker<ImeMsiArchiveEnvelope>
{
private readonly ImeMsiArchiver _archiver;
private readonly ISessionRepository _sessionRepo;
private readonly AdminConfigurationService _adminConfigService;
public ImeMsiArchiveQueueWorker(
QueueClientFactory queueFactory,
ImeMsiArchiver archiver,
ISessionRepository sessionRepo,
AdminConfigurationService adminConfigService,
ILogger<ImeMsiArchiveQueueWorker> logger)
: base(queueFactory, Constants.QueueNames.ImeMsiArchive, logger)
{
_archiver = archiver ?? throw new ArgumentNullException(nameof(archiver));
_sessionRepo = sessionRepo ?? throw new ArgumentNullException(nameof(sessionRepo));
_adminConfigService = adminConfigService ?? throw new ArgumentNullException(nameof(adminConfigService));
}
/// <summary>
/// Test seam: construct directly with mock <see cref="Azure.Storage.Queues.QueueClient"/>
/// instances so the poll loop can be exercised without Azurite. Mirrors
/// <see cref="Deletion.SessionDeletionWorker"/>'s internal test ctor.
/// </summary>
internal ImeMsiArchiveQueueWorker(
Azure.Storage.Queues.QueueClient mainQueue,
Azure.Storage.Queues.QueueClient poisonQueue,
ImeMsiArchiver archiver,
ISessionRepository sessionRepo,
AdminConfigurationService adminConfigService,
ILogger<ImeMsiArchiveQueueWorker> logger,
TimeSpan? pollInterval = null)
: base(mainQueue, poisonQueue, logger, pollInterval)
{
_archiver = archiver ?? throw new ArgumentNullException(nameof(archiver));
_sessionRepo = sessionRepo ?? throw new ArgumentNullException(nameof(sessionRepo));
_adminConfigService = adminConfigService ?? throw new ArgumentNullException(nameof(adminConfigService));
}
/// <summary>Download can legitimately take minutes — extend visibility while working.</summary>
protected override bool UseHeartbeat => true;
/// <summary>
/// Paused while <see cref="AdminConfiguration.ImeMsiArchivingEnabled"/> is off —
/// messages stay parked (not dropped) and are processed when the flag returns.
/// Uses the cached config read: this is a pause, not a kill-switch, and the worker
/// only wakes for rare messages anyway.
/// </summary>
protected override async ValueTask<bool> ShouldPauseAsync(CancellationToken ct)
{
try
{
var config = await _adminConfigService.GetConfigurationAsync();
return !config.ImeMsiArchivingEnabled;
}
catch
{
return false; // config unreadable → keep working (archive is fail-soft anyway)
}
}
protected override bool TryValidate(ImeMsiArchiveEnvelope envelope)
=> !string.IsNullOrEmpty(envelope.Version);
protected override string DescribeForLog(ImeMsiArchiveEnvelope envelope)
=> $"version {envelope.Version}";
protected override async Task HandleAsync(ImeMsiArchiveEnvelope envelope, CancellationToken ct)
{
var result = await _archiver.ArchiveAsync(envelope, ct);
// Merge the outcome first — also for failures, so the row always tells the truth.
await _sessionRepo.UpdateImeVersionArchiveInfoAsync(
envelope.Version, result.Status, result.BlobPath, result.Sha256, result.SizeBytes, result.SourceUrl);
if (!result.Success && result.Retryable)
{
// Visibility-timeout retry → poison after MaxDequeueCount. The thrown message
// lands in the worker's handler-failure log line.
throw new InvalidOperationException(
$"IME MSI archive attempt failed with {result.Status} for version {envelope.Version}");
}
}
}
}
@@ -0,0 +1,441 @@
using System.Security.Cryptography;
using System.Text;
using System.Text.RegularExpressions;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Azure;
using Azure.Storage.Blobs.Models;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
namespace AutopilotMonitor.Functions.Services.Ime
{
/// <summary>
/// Outcome of an IME-installer archive attempt. <see cref="Status"/> is one of the
/// <see cref="ImeMsiArchiver.Statuses"/> constants and is merged verbatim onto the
/// ImeVersionHistory row. <see cref="Retryable"/> tells the queue worker whether to
/// trigger a visibility-timeout retry (transient download problems) or complete the
/// message (permanent rejections).
/// </summary>
public sealed record ImeMsiArchiveResult(
bool Success, string Status, bool Retryable,
string? BlobPath, string? Sha256, long? SizeBytes, string? SourceUrl);
/// <summary>
/// Downloads a newly sighted IME installer and archives it into the permanent
/// <see cref="Constants.BlobContainers.ImeArchive"/> container
/// (<c>{version}/IntuneWindowsAgent.msi</c> + <c>{version}/provenance.json</c>).
/// Modeled on <see cref="Diagnostics.SessionReportDiagnosticsArchiveCopier"/>:
/// fail-soft by contract (never throws), Statuses constants persisted verbatim,
/// admin-configured size cap, typed catch ladder, <c>protected virtual</c> test seams.
/// <para>
/// The download URL comes from agent telemetry and is therefore untrusted: only HTTPS
/// URLs on <c>*.manage.microsoft.com</c> whose path filename is exactly
/// <c>IntuneWindowsAgent.msi</c> are accepted; anything else falls back to the canonical
/// versionless CDN URL — safe for a NEW version, because "new" is by definition what
/// that URL currently serves. The blob write uses If-None-Match:* so a queue re-delivery
/// can never overwrite an existing archive entry; the 409 path additionally heals a
/// provenance sidecar the first attempt failed to write (see
/// <see cref="CompleteExistingArchiveAsync"/>).
/// </para>
/// </summary>
public class ImeMsiArchiver
{
/// <summary>Persisted MsiArchiveStatus values.</summary>
public static class Statuses
{
public const string Archived = "Archived";
public const string FailedBadVersion = "Failed:BadVersion";
public const string FailedTooLarge = "Failed:TooLarge";
public const string FailedDownload = "Failed:Download";
public const string FailedTimeout = "Failed:Timeout";
public const string FailedError = "Failed:Error";
}
/// <summary>
/// The CSP's versionless distribution endpoint — always serves the currently rolled-out
/// build. Fallback when the event carried no (acceptable) URL.
/// </summary>
public const string CanonicalMsiUrl = "https://imeswdb-afd-secondary.manage.microsoft.com/IntuneWindowsAgent.msi";
private const string MsiFileName = "IntuneWindowsAgent.msi";
private const string AllowedHostSuffix = ".manage.microsoft.com";
/// <summary>Hard ceiling for one download attempt; the worker's heartbeat covers it.</summary>
private static readonly TimeSpan DownloadTimeout = TimeSpan.FromMinutes(10);
/// <summary>Blob folder name = version — must be plain dotted digits (untrusted input).</summary>
internal static readonly Regex VersionRegex = new(@"^\d+(\.\d+){1,3}$", RegexOptions.Compiled);
private readonly HttpClient _httpClient;
private readonly BlobStorageService _blobStorage;
private readonly AdminConfigurationService _adminConfigService;
private readonly ILogger<ImeMsiArchiver> _logger;
public ImeMsiArchiver(
HttpClient httpClient,
BlobStorageService blobStorage,
AdminConfigurationService adminConfigService,
ILogger<ImeMsiArchiver> logger)
{
_httpClient = httpClient;
_blobStorage = blobStorage;
_adminConfigService = adminConfigService;
_logger = logger;
}
/// <summary>
/// Test seam: Azure-free construction for recording subclasses that override the
/// virtual download/upload/config seams below.
/// </summary>
protected ImeMsiArchiver()
{
_httpClient = null!;
_blobStorage = null!;
_adminConfigService = null!;
_logger = NullLogger<ImeMsiArchiver>.Instance;
}
/// <summary>
/// True when <paramref name="url"/> is an HTTPS URL on <c>*.manage.microsoft.com</c>
/// whose path filename is exactly <c>IntuneWindowsAgent.msi</c> (query ignored).
/// </summary>
public static bool IsAllowedMsiUrl(string? url)
{
if (string.IsNullOrEmpty(url)) return false;
if (!Uri.TryCreate(url, UriKind.Absolute, out var uri)) return false;
if (uri.Scheme != Uri.UriSchemeHttps) return false;
var host = uri.Host;
var hostAllowed =
host.EndsWith(AllowedHostSuffix, StringComparison.OrdinalIgnoreCase) ||
host.Equals(AllowedHostSuffix.TrimStart('.'), StringComparison.OrdinalIgnoreCase);
if (!hostAllowed) return false;
var fileName = uri.Segments.Length > 0 ? uri.Segments[^1] : string.Empty;
return fileName.Equals(MsiFileName, StringComparison.OrdinalIgnoreCase);
}
public virtual async Task<ImeMsiArchiveResult> ArchiveAsync(
ImeMsiArchiveEnvelope envelope, CancellationToken cancellationToken = default)
{
if (envelope is null || !VersionRegex.IsMatch(envelope.Version ?? string.Empty))
{
_logger.LogWarning(
"ImeMsiArchiver: rejecting version {Version} — not a plain dotted version string",
envelope?.Version ?? "(null)");
return new ImeMsiArchiveResult(false, Statuses.FailedBadVersion, Retryable: false,
null, null, null, null);
}
var version = envelope.Version!;
var blobPath = $"{version}/{MsiFileName}";
string sourceUrl;
if (IsAllowedMsiUrl(envelope.MsiDownloadUrl))
{
sourceUrl = envelope.MsiDownloadUrl!;
}
else
{
if (!string.IsNullOrEmpty(envelope.MsiDownloadUrl))
{
_logger.LogWarning(
"ImeMsiArchiver: event URL rejected by allowlist for version {Version} — using canonical URL",
version);
}
sourceUrl = CanonicalMsiUrl;
}
try
{
var adminConfig = await GetAdminConfigurationAsync();
var maxSizeBytes = (long)adminConfig.MaxImeMsiDownloadSizeMB * 1024 * 1024;
using var cts = new CancellationTokenSource(DownloadTimeout);
using var linked = CancellationTokenSource.CreateLinkedTokenSource(cts.Token, cancellationToken);
var (content, contentLength) = await OpenDownloadAsync(sourceUrl, linked.Token);
string sha256;
long totalBytes;
using (content)
{
// Same fast-reject as the diagnostics proxy: cap of 0 means unlimited.
if (maxSizeBytes > 0 && contentLength > maxSizeBytes)
{
_logger.LogWarning(
"ImeMsiArchiver: download for {Version} rejected — Content-Length {SizeBytes} exceeds limit {MaxSizeBytes}",
version, contentLength, maxSizeBytes);
return new ImeMsiArchiveResult(false, Statuses.FailedTooLarge, Retryable: false,
null, null, contentLength, sourceUrl);
}
// Hash + count while streaming into the blob; the wrapper enforces the cap
// mid-stream too (a lying/absent Content-Length cannot bypass it).
using var guarded = new HashingCappedReadStream(content, maxSizeBytes);
try
{
await UploadMsiAsync(blobPath, guarded, linked.Token);
}
catch (RequestFailedException ex) when (ex.Status == 409)
{
// Queue re-delivery after a successful upload: the archive already has
// this version — that IS the desired end state. But if the first
// attempt died between the MSI and the provenance upload, the sidecar
// is missing and this 409 pre-empts the happy path's provenance write
// on every future retry — so it must be healed here.
return await CompleteExistingArchiveAsync(
envelope, version, blobPath, sourceUrl, linked.Token);
}
sha256 = guarded.Sha256Hex;
totalBytes = guarded.TotalBytes;
}
await WriteProvenanceAsync(envelope, version, sourceUrl, sha256, totalBytes, linked.Token);
_logger.LogInformation(
"ImeMsiArchiver: archived {Version} ({SizeBytes} bytes, sha256 {Sha256}) from {SourceUrl}",
version, totalBytes, sha256, sourceUrl);
return new ImeMsiArchiveResult(true, Statuses.Archived, Retryable: false,
blobPath, sha256, totalBytes, sourceUrl);
}
catch (SizeCapExceededException)
{
_logger.LogWarning(
"ImeMsiArchiver: download for {Version} exceeded the size cap mid-stream — aborted",
version);
return new ImeMsiArchiveResult(false, Statuses.FailedTooLarge, Retryable: false,
null, null, null, sourceUrl);
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
_logger.LogWarning("ImeMsiArchiver: download for {Version} timed out", version);
return new ImeMsiArchiveResult(false, Statuses.FailedTimeout, Retryable: true,
null, null, null, sourceUrl);
}
catch (HttpRequestException ex)
{
_logger.LogWarning(ex, "ImeMsiArchiver: download for {Version} failed", version);
return new ImeMsiArchiveResult(false, Statuses.FailedDownload, Retryable: true,
null, null, null, sourceUrl);
}
catch (OperationCanceledException)
{
// Host shutdown — let the worker's dispatch loop see it and leave the message.
throw;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "ImeMsiArchiver: archiving {Version} failed", version);
return new ImeMsiArchiveResult(false, Statuses.FailedError, Retryable: true,
null, null, null, sourceUrl);
}
}
/// <summary>
/// Re-delivery landed on an already-archived version (409 on the write-once MSI
/// upload). Normally the provenance sidecar exists too and there is nothing to do —
/// but when the first attempt died between the MSI and the provenance upload
/// (transient storage failure, host shutdown), the sidecar would otherwise be lost
/// FOREVER: the 409 short-circuit pre-empts the happy path's provenance write on
/// every retry. Hash and size are recomputed from the ARCHIVED bytes, not this
/// attempt's re-download, so the provenance describes the blob it sits next to.
/// Failures propagate into the caller's catch ladder (retryable), so the queue
/// retries the heal until it lands.
/// </summary>
private async Task<ImeMsiArchiveResult> CompleteExistingArchiveAsync(
ImeMsiArchiveEnvelope envelope, string version, string blobPath, string sourceUrl,
CancellationToken cancellationToken)
{
if (await ProvenanceExistsAsync($"{version}/provenance.json", cancellationToken))
{
_logger.LogInformation(
"ImeMsiArchiver: blob {BlobPath} already exists — treating re-delivery as archived",
blobPath);
return new ImeMsiArchiveResult(true, Statuses.Archived, Retryable: false,
blobPath, null, null, sourceUrl);
}
string sha256;
long totalBytes;
using (var archived = await OpenArchivedMsiAsync(blobPath, cancellationToken))
using (var hashing = new HashingCappedReadStream(archived, maxBytes: 0))
{
await hashing.CopyToAsync(Stream.Null, cancellationToken);
sha256 = hashing.Sha256Hex;
totalBytes = hashing.TotalBytes;
}
await WriteProvenanceAsync(envelope, version, sourceUrl, sha256, totalBytes, cancellationToken);
_logger.LogInformation(
"ImeMsiArchiver: healed missing provenance for already-archived {Version} ({SizeBytes} bytes, sha256 {Sha256})",
version, totalBytes, sha256);
return new ImeMsiArchiveResult(true, Statuses.Archived, Retryable: false,
blobPath, sha256, totalBytes, sourceUrl);
}
private async Task WriteProvenanceAsync(
ImeMsiArchiveEnvelope envelope, string version, string sourceUrl,
string sha256, long totalBytes, CancellationToken cancellationToken)
{
var provenance = new
{
version,
url = sourceUrl,
urlFromEvent = sourceUrl != CanonicalMsiUrl,
msiMatchedBy = envelope.MsiMatchedBy,
sha256,
msiBytes = totalBytes,
firstSeenTenantId = envelope.TenantId,
firstSeenSessionId = envelope.SessionId,
archivedAtUtc = DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ"),
};
await UploadProvenanceAsync(
$"{version}/provenance.json",
System.Text.Json.JsonSerializer.Serialize(provenance,
new System.Text.Json.JsonSerializerOptions { WriteIndented = true }),
cancellationToken);
}
// -------- Test seams (production defaults hit Azure / the network) --------
/// <summary>Fetches the platform admin configuration (size cap).</summary>
protected virtual Task<AdminConfiguration> GetAdminConfigurationAsync()
=> _adminConfigService.GetConfigurationAsync();
/// <summary>Opens the installer download stream (headers-read, no buffering).</summary>
protected virtual async Task<(Stream Content, long ContentLength)> OpenDownloadAsync(
string url, CancellationToken cancellationToken)
{
var response = await _httpClient.GetAsync(
url, HttpCompletionOption.ResponseHeadersRead, cancellationToken);
response.EnsureSuccessStatusCode();
var content = await response.Content.ReadAsStreamAsync(cancellationToken);
return (content, response.Content.Headers.ContentLength ?? -1);
}
/// <summary>
/// Uploads the installer stream write-once (If-None-Match:*). Implementations must
/// surface a conflict as <see cref="RequestFailedException"/> with status 409.
/// </summary>
protected virtual async Task UploadMsiAsync(string blobPath, Stream content, CancellationToken cancellationToken)
{
var containerClient = _blobStorage.GetContainerClient(Constants.BlobContainers.ImeArchive);
await containerClient.CreateIfNotExistsAsync(cancellationToken: cancellationToken);
var blobClient = containerClient.GetBlobClient(blobPath);
await blobClient.UploadAsync(content, new BlobUploadOptions
{
HttpHeaders = new BlobHttpHeaders { ContentType = "application/octet-stream" },
Conditions = new BlobRequestConditions { IfNoneMatch = ETag.All }
}, cancellationToken);
}
/// <summary>True when the provenance sidecar already exists in the archive container.</summary>
protected virtual async Task<bool> ProvenanceExistsAsync(string blobPath, CancellationToken cancellationToken)
{
var containerClient = _blobStorage.GetContainerClient(Constants.BlobContainers.ImeArchive);
return (await containerClient.GetBlobClient(blobPath).ExistsAsync(cancellationToken)).Value;
}
/// <summary>Opens the already-archived installer blob for the provenance-heal rehash.</summary>
protected virtual async Task<Stream> OpenArchivedMsiAsync(string blobPath, CancellationToken cancellationToken)
{
var containerClient = _blobStorage.GetContainerClient(Constants.BlobContainers.ImeArchive);
return await containerClient.GetBlobClient(blobPath).OpenReadAsync(cancellationToken: cancellationToken);
}
/// <summary>Uploads the provenance sidecar (overwrite allowed — it is derived data).</summary>
protected virtual async Task UploadProvenanceAsync(string blobPath, string json, CancellationToken cancellationToken)
{
var containerClient = _blobStorage.GetContainerClient(Constants.BlobContainers.ImeArchive);
var blobClient = containerClient.GetBlobClient(blobPath);
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(json));
await blobClient.UploadAsync(stream, new BlobUploadOptions
{
HttpHeaders = new BlobHttpHeaders { ContentType = "application/json" }
}, cancellationToken);
}
// -------- Streaming helpers --------
private sealed class SizeCapExceededException : Exception
{
}
/// <summary>
/// Read-only pass-through stream that SHA-256-hashes and counts everything read and
/// throws once the byte count exceeds the cap (0 = unlimited). Non-seekable on
/// purpose — the blob SDK then stages blocks instead of demanding Length upfront.
/// </summary>
private sealed class HashingCappedReadStream : Stream
{
private readonly Stream _inner;
private readonly long _maxBytes;
private readonly IncrementalHash _hash = IncrementalHash.CreateHash(HashAlgorithmName.SHA256);
private string? _sha256Hex;
public HashingCappedReadStream(Stream inner, long maxBytes)
{
_inner = inner;
_maxBytes = maxBytes;
}
public long TotalBytes { get; private set; }
public string Sha256Hex => _sha256Hex ??=
Convert.ToHexString(_hash.GetHashAndReset()).ToLowerInvariant();
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => throw new NotSupportedException();
public override long Position
{
get => TotalBytes;
set => throw new NotSupportedException();
}
public override int Read(byte[] buffer, int offset, int count)
{
var read = _inner.Read(buffer, offset, count);
Account(buffer.AsSpan(offset, read));
return read;
}
public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
{
var read = await _inner.ReadAsync(buffer, cancellationToken);
Account(buffer.Span[..read]);
return read;
}
public override Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
=> ReadAsync(buffer.AsMemory(offset, count), cancellationToken).AsTask();
private void Account(ReadOnlySpan<byte> data)
{
if (data.IsEmpty) return;
TotalBytes += data.Length;
if (_maxBytes > 0 && TotalBytes > _maxBytes)
throw new SizeCapExceededException();
_hash.AppendData(data);
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
protected override void Dispose(bool disposing)
{
if (disposing) _hash.Dispose();
base.Dispose(disposing);
}
}
}
}
@@ -3316,7 +3316,12 @@ namespace AutopilotMonitor.Functions.Services
FirstSeenSessionId = entity.GetString("FirstSeenSessionId") ?? string.Empty,
FirstSeenTenantId = entity.GetString("FirstSeenTenantId") ?? string.Empty,
LastSeenAt = entity.GetDateTime("LastSeenAt") ?? DateTime.MinValue,
SessionCount = entity.GetInt32("SessionCount") ?? 0
SessionCount = entity.GetInt32("SessionCount") ?? 0,
MsiArchiveStatus = entity.GetString("MsiArchiveStatus"),
MsiArchiveBlobPath = entity.GetString("MsiArchiveBlobPath"),
MsiSha256 = entity.GetString("MsiSha256"),
MsiBytes = entity.GetInt64("MsiBytes"),
MsiSourceUrl = entity.GetString("MsiSourceUrl")
});
}
}
@@ -3328,6 +3333,38 @@ namespace AutopilotMonitor.Functions.Services
return results.OrderByDescending(e => e.FirstSeenAt).ToList();
}
/// <summary>
/// Merges the installer-archiving outcome into the version's ImeVersionHistory row
/// (same Merge idiom as the LastSeen update above). Fail-soft: the archive job
/// already logged its own result; a lost status column must never fail the worker.
/// </summary>
public async Task UpdateImeVersionArchiveInfoAsync(
string version, string status, string? blobPath, string? sha256, long? sizeBytes, string? sourceUrl)
{
if (string.IsNullOrEmpty(version) || string.IsNullOrEmpty(status)) return;
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.ImeVersionHistory);
var mergeEntity = new TableEntity("Global", version)
{
["MsiArchiveStatus"] = status
};
if (!string.IsNullOrEmpty(blobPath)) mergeEntity["MsiArchiveBlobPath"] = blobPath;
if (!string.IsNullOrEmpty(sha256)) mergeEntity["MsiSha256"] = sha256;
if (sizeBytes.HasValue) mergeEntity["MsiBytes"] = sizeBytes.Value;
if (!string.IsNullOrEmpty(sourceUrl)) mergeEntity["MsiSourceUrl"] = sourceUrl;
await tableClient.UpsertEntityAsync(mergeEntity, TableUpdateMode.Merge);
}
catch (Exception ex)
{
_logger.LogWarning(ex,
"Failed to update ImeVersionHistory archive info for version {Version} (status {Status})",
version, status);
}
}
#endregion
#region Excessive-Event Detection
+4 -4
View File
@@ -24,7 +24,7 @@
"@types/compression": "^1.8.1",
"@types/express": "^5.0.6",
"@types/node": "^22.20.1",
"tsx": "^4.23.11",
"tsx": "^4.23.12",
"typescript": "^5.0.0",
"vitest": "^4.1.10"
},
@@ -3823,9 +3823,9 @@
"optional": true
},
"node_modules/tsx": {
"version": "4.23.11",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.11.tgz",
"integrity": "sha512-Ry2oTEUnhBdeEdWIztY8kf3/nBGnPnjMLVGL0YfdRXMORuPER5NlKmayqxtxRxwB1xBN+RivRaJfe7PM1rtiyw==",
"version": "4.23.12",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.12.tgz",
"integrity": "sha512-FDf4L4sYzKtzWYhU/Xm0AQFdTjdIxNo9ElTf2mxXM6k8YMHXzYUe4yODVaXP4V9uMFbVg8c0qyBccK2OOxb45Q==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -32,7 +32,7 @@
"@types/compression": "^1.8.1",
"@types/express": "^5.0.6",
"@types/node": "^22.20.1",
"tsx": "^4.23.11",
"tsx": "^4.23.12",
"typescript": "^5.0.0",
"vitest": "^4.1.10"
},
@@ -28,8 +28,8 @@ export const DIAG_ZIP_MAP = {
appliesTo: 'v2',
},
{
path: 'AgentLogs/agent_YYYYMMDD.log',
what: 'Full client chronology: startup → config → collectors → enrollment tracking → completion → upload. Rotated per day on multi-day enrollments — read every dated file.',
path: 'AgentLogs/agent.log',
what: 'Full client chronology: startup → config → collectors → enrollment tracking → completion → upload. Size-rotated segments agent-002.log onward. Pre-2026-08-18 agents wrote date-rotated agent_YYYYMMDD.log instead — glob agent*.log to catch both.',
read: 'grep-errors-first',
grepHints: ['"\\[WARN\\]\\|\\[ERROR\\]"', '-A 5 -B 2 "\\[ERROR\\]"', '"DecisionStep: ord="', '"EventUploadService"', '"EnrollmentTracker"', '"DiagnosticsCollector"'],
priority: 2,
@@ -57,7 +57,7 @@ export const DIAG_ZIP_MAP = {
appliesTo: 'v2',
},
{
path: 'AgentLogs/ime_pattern_matches.log (v2) / ime-matches.log (v1)',
path: 'AgentLogs/ime-pattern-matches.log (older v2 agents: ime_pattern_matches.log; v1: ime-matches.log)',
what: 'IME pattern matches. Format: [sourceFileName] [patternId] rawLogLine. Read when the question is about apps / ESP phases / IME patterns.',
read: 'full',
priority: 6,
@@ -113,5 +113,6 @@ export const DIAG_ZIP_MAP = {
'V1 agent archives have NO AgentState/ or AgentSpool/ folders (DecisionCore is v2-only).',
'Pre-provisioned (White Glove) blobs carry a -preprov suffix — that is normal.',
'No archive ≠ a problem: upload mode may be Off or OnFailure (so successful sessions have none).',
'Log naming is kebab-case since 2026-08-18: agent.log, bootstrap-script.log (was bootstrap_agent.log), bootstrap-msi.log (was msi-bootstrap.log), crash-*.log (was crash_*.log), gather-rules-debug.log, ime-pattern-matches.log. Archives from older agents/bootstraps carry the old names.',
],
} as const;
@@ -146,6 +146,10 @@ export const EVENT_TYPES_CATALOG = {
'agent_shutdown',
'agent_version_check',
'ime_agent_version',
'ime_token_failure',
'registry_app_state',
'registry_app_baseline',
'app_state_reconciliation',
'ime_user_session_completed',
'ime_session_change',
'ime_process_exited',
@@ -64,6 +64,18 @@ namespace AutopilotMonitor.DecisionCore.Engine
/// of failing.
/// </summary>
public const string AdvisoryCompletion = "advisory_completion";
/// <summary>
/// Device Preparation completion backstop (afee7ae0, 2026-08-18). WDP has no ESP,
/// so none of the ESP-derived resolution deadlines (HelloSafety, AdvisoryCompletion)
/// can ever arm there — a WDP session whose Hello never resolves would park until
/// the agent max-lifetime watchdog with no verdict. Armed by
/// <c>HandleDesktopArrivedV1</c> when the desktop arrives on a DevicePreparation
/// session without completing in the same step; on fire the session resolves like
/// the HelloSafety timeout (synthetic <c>HelloOutcome=Timeout</c> + completion —
/// the desktop is present by construction of the arming site).
/// </summary>
public const string DevicePrepCompletion = "device_prep_completion";
}
/// <summary>
@@ -130,6 +142,15 @@ namespace AutopilotMonitor.DecisionCore.Engine
/// <summary>On <c>SessionStarted</c>: literal "v1" / "v2" from <c>EnrollmentRegistryDetector.DetectEnrollmentType()</c>.</summary>
public const string EnrollmentType = "enrollmentType";
/// <summary>
/// On <c>EnrollmentFactsObserved</c>: "true" when the v2 verdict came from the
/// deterministic Device Preparation marker (BootstrapperAgent ExecutionContext)
/// rather than the CloudAssigned* fallback rules. Lifts the DevicePreparation
/// profile seed to High confidence — the fallback rules stay Medium because they
/// can in principle fire on ESP-less classic profiles.
/// </summary>
public const string EnrollmentTypeDeterministic = "enrollmentTypeDeterministic";
/// <summary>On <c>SessionStarted</c>: "true" / "false" from <c>EnrollmentRegistryDetector.DetectHybridJoin()</c>.</summary>
public const string IsHybridJoin = "isHybridJoin";
@@ -657,12 +657,27 @@ namespace AutopilotMonitor.DecisionCore.Engine
// HelloResolved arrives.
builder.WithStage(state.Stage);
// WDP completion backstop (afee7ae0): no ESP → HelloSafety / AdvisoryCompletion can
// never arm on Device Preparation, so a Hello that never resolves would park the
// session until the max-lifetime watchdog. Desktop is here, Hello is not — bound
// the wait. Armed once (name-keyed dedupe is belt; the explicit check keeps the
// audit trail free of re-arm noise on duplicate desktop signals).
var effects = new List<DecisionEffect>(2);
if (state.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation
&& !HasDevicePrepCompletionDeadline(state))
{
var backstop = BuildDevicePrepCompletionDeadline(state, signal);
builder.AddDeadline(backstop);
effects.Add(new DecisionEffect(DecisionEffectKind.ScheduleDeadline, deadline: backstop));
}
// Liveness plan PR2: Desktop is in but completion is still blocked (Hello pending,
// or the Hello-disabled fast-path's strong gate is not satisfied) — surface the
// missing prerequisites. Covers both the gate-false fast-path and the plain
// desktop-first ordering; the fingerprint dedupes against earlier emissions.
var waitingEffect = BuildCompletionWaitingEffect(
state, builder, signal, trigger: nameof(DecisionSignalKind.DesktopArrived));
if (waitingEffect != null) effects.Add(waitingEffect);
var newState = builder.Build();
@@ -676,7 +691,7 @@ namespace AutopilotMonitor.DecisionCore.Engine
return new DecisionStep(
newState,
transition,
waitingEffect != null ? new[] { waitingEffect } : Array.Empty<DecisionEffect>());
effects.Count > 0 ? effects.ToArray() : Array.Empty<DecisionEffect>());
}
/// <summary>
@@ -226,7 +226,8 @@ namespace AutopilotMonitor.DecisionCore.Engine
state.AccountSetupEnteredUtc,
state.EspFinalExitUtc,
state.ImeUserSessionCompletedUtc,
state.DesktopArrivedUtc));
state.DesktopArrivedUtc),
devicePreparation: state.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation);
/// <summary>
/// Arm C of <c>ShouldTransitionToAwaitingHello</c> (session a4537c36) restated over raw
@@ -257,10 +258,16 @@ namespace AutopilotMonitor.DecisionCore.Engine
bool helloPolicyDisabled,
bool desktopArrived,
bool realmJoinGateOpen,
bool postEspUserSessionEvidence = false)
bool postEspUserSessionEvidence = false,
bool devicePreparation = false)
{
var missing = new List<string>(4);
if (!accountSetupProvisioned && !skipUserEsp && !postEspUserSessionEvidence)
// Device Preparation has no ESP: AccountSetupCategory.Status is never written
// there, so the provisioning gate is unsatisfiable by construction and must not
// be reported as a missing prerequisite (afee7ae0 — mirrors the skipUserEsp
// exemption; completion on WDP rides on Hello + Desktop, see arm D of
// ShouldTransitionToAwaitingHello and the DevicePrepCompletion backstop).
if (!accountSetupProvisioned && !skipUserEsp && !postEspUserSessionEvidence && !devicePreparation)
missing.Add(CompletionPrerequisites.AccountSetupProvisioningComplete);
if (!helloResolved && !helloPolicyDisabled)
missing.Add(CompletionPrerequisites.HelloResolution);
@@ -302,7 +309,8 @@ namespace AutopilotMonitor.DecisionCore.Engine
builder.AccountSetupEnteredUtc,
builder.EspFinalExitUtc,
builder.ImeUserSessionCompletedUtc,
builder.DesktopArrivedUtc));
builder.DesktopArrivedUtc),
devicePreparation: builder.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation);
if (missing.Count == 0) return null;
var fingerprint = string.Join(",", missing);
@@ -0,0 +1,108 @@
using System;
using System.Collections.Generic;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
namespace AutopilotMonitor.DecisionCore.Engine
{
// Device Preparation (WDP) — completion backstop (afee7ae0, 2026-08-18).
//
// WDP has no ESP, so every ESP-derived resolution deadline is structurally unarmable
// there: HelloSafety arms only on EspExiting / AccountSetupProvisioningComplete / arm C,
// AdvisoryCompletion only on an EspTerminalFailure or a guard-blocked post-AccountSetup
// esp_exiting — none of which exist on Device Preparation. A WDP session whose Hello
// never resolves therefore used to park until the agent max-lifetime watchdog with no
// verdict. The engine otherwise deliberately rides the Classic rails for WDP (the
// Hello + Desktop conjunction and arm D of ShouldTransitionToAwaitingHello); this
// partial adds only the WDP-specific bounded wait.
public sealed partial class DecisionEngine
{
/// <summary>
/// Resolution window from real-user desktop arrival on a Device Preparation session.
/// Deliberately the same order of magnitude as <c>AdvisoryCompletion</c> (30 min),
/// not <c>HelloSafety</c> (300 s): the user may legitimately still be walking
/// through a configured Hello wizard, and the tracker's own Hello completion
/// timers normally resolve long before this fires — it is the last net, not the
/// expected path.
/// </summary>
private static readonly TimeSpan s_devicePrepCompletionWindow = TimeSpan.FromMinutes(30);
/// <summary>True when <see cref="DeadlineNames.DevicePrepCompletion"/> is armed in <paramref name="state"/>.</summary>
private static bool HasDevicePrepCompletionDeadline(DecisionState state)
{
foreach (var d in state.Deadlines)
{
if (d.Name == DeadlineNames.DevicePrepCompletion) return true;
}
return false;
}
/// <summary>
/// Build the <see cref="DeadlineNames.DevicePrepCompletion"/> backstop deadline.
/// Replay-safe via <see cref="EffectiveDeadlineBase"/> (floored at AgentBootUtc).
/// </summary>
internal static ActiveDeadline BuildDevicePrepCompletionDeadline(DecisionState state, DecisionSignal signal) =>
new ActiveDeadline(
name: DeadlineNames.DevicePrepCompletion,
dueAtUtc: EffectiveDeadlineBase(state, signal).Add(s_devicePrepCompletionWindow),
firesSignalKind: DecisionSignalKind.DeadlineFired,
firesPayload: new Dictionary<string, string>
{
[SignalPayloadKeys.Deadline] = DeadlineNames.DevicePrepCompletion,
});
/// <summary>
/// <see cref="DeadlineNames.DevicePrepCompletion"/> fired: the WDP session reached
/// the real-user desktop but Hello never resolved within the window. Mirror of
/// <see cref="HandleHelloSafetyDeadlineFired"/>: record the synthetic
/// <see cref="SyntheticHelloOutcomeTimeout"/> (unless a real resolution raced the
/// timer) and complete through Finalizing. Desktop is present by construction of
/// the arming site (<c>HandleDesktopArrivedV1</c>); the AwaitingDesktop fallback
/// only keeps the handler total. A timer surviving into a terminal stage is
/// swallowed by the post-terminal guard in <see cref="Dispatch"/> — no explicit
/// cancel wiring at the completion sites is needed for correctness.
/// </summary>
private DecisionStep HandleDevicePrepCompletionDeadlineFired(DecisionState state, DecisionSignal signal)
{
var nextStep = state.StepIndex + 1;
var builder = state.ToBuilder()
.WithStepIndex(nextStep)
.WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal)
.CancelDeadline(DeadlineNames.DevicePrepCompletion);
if (state.HelloResolvedUtc == null)
{
builder.HelloResolvedUtc = new SignalFact<DateTime>(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
builder.HelloOutcome = new SignalFact<string>(SyntheticHelloOutcomeTimeout, signal.SessionSignalOrdinal);
}
if (state.DesktopArrivedUtc != null)
{
return CompleteThroughFinalizingOrDefer(
state: state,
signal: signal,
preparedBuilder: builder,
nextStepIndex: nextStep,
trigger: $"DeadlineFired:{DeadlineNames.DevicePrepCompletion}");
}
builder.WithStage(SessionStage.AwaitingDesktop);
var waitingEffect = BuildCompletionWaitingEffect(
state, builder, signal, trigger: $"DeadlineFired:{DeadlineNames.DevicePrepCompletion}");
var newState = builder.Build();
var transition = BuildTakenTransition(
before: state,
signal: signal,
toStage: SessionStage.AwaitingDesktop,
nextStepIndex: nextStep,
trigger: $"DeadlineFired:{DeadlineNames.DevicePrepCompletion}");
return new DecisionStep(
newState,
transition,
waitingEffect != null ? new[] { waitingEffect } : Array.Empty<DecisionEffect>());
}
}
}
@@ -362,6 +362,8 @@ namespace AutopilotMonitor.DecisionCore.Engine
return HandleRealmJoinTimeoutDeadlineFired(state, signal);
case DeadlineNames.AdvisoryCompletion:
return HandleAdvisoryCompletionDeadlineFired(state, signal);
case DeadlineNames.DevicePrepCompletion:
return HandleDevicePrepCompletionDeadlineFired(state, signal);
default:
// Deadline name not recognized in this sub-milestone. Cancel it from state
// and record a neutral taken transition — M3.3+ adds handlers for
@@ -784,6 +786,16 @@ namespace AutopilotMonitor.DecisionCore.Engine
{
return true;
}
// Arm D (afee7ae0, 2026-08-18) — Device Preparation: there is no ESP, so arms A–C
// are unsatisfiable by construction (no provisioning categories, no Shell-Core
// final exit, IME anchors are incidental pattern hits). The DAD-validated
// real-user desktop IS the end of WDP provisioning — the DPP page has closed and
// the user is signed in; only the Hello gate remains after this promotion.
if (state.ScenarioProfile.Mode == EnrollmentMode.DevicePreparation
&& (desktopArrivedInFlight || state.DesktopArrivedUtc != null))
{
return true;
}
return false;
}
@@ -99,8 +99,19 @@ namespace AutopilotMonitor.DecisionCore.State
if (string.Equals(rawType, "v2", StringComparison.OrdinalIgnoreCase))
{
// Deterministic WDP marker (BootstrapperAgent ExecutionContext) seeds
// High — the marker is as registry-deterministic as the self-deploying
// OobeConfig bits below, and a High seed lets the generic monotonic
// guard protect the mode instead of relying solely on the explicit
// DevicePreparation checks in the downstream promotion handlers.
// The CloudAssigned* fallback rules stay Medium (ESP-less classic
// profiles could in principle trip them).
var deterministic = signal.Payload.TryGetValue(SignalPayloadKeys.EnrollmentTypeDeterministic, out var rawDeterministic)
&& bool.TryParse(rawDeterministic, out var isDeterministic)
&& isDeterministic;
mode = EnrollmentMode.DevicePreparation;
confidence = Max(confidence, ProfileConfidence.Medium);
confidence = Max(confidence, deterministic ? ProfileConfidence.High : ProfileConfidence.Medium);
reason = newReason;
changed = true;
}
@@ -60,14 +60,14 @@ namespace AutopilotMonitor.Shared
/// <summary>
/// Default path for the IME pattern match log file (debugging/diagnostics)
/// </summary>
public const string ImeMatchLogPath = @"%ProgramData%\AutopilotMonitor\Logs\ime_pattern_matches.log";
public const string ImeMatchLogPath = @"%ProgramData%\AutopilotMonitor\Logs\ime-pattern-matches.log";
/// <summary>
/// Default path for the gather rule debug trace file (debugging/diagnostics).
/// Written when the tenant enables EnableGatherRuleDebugLog; explains every
/// gather-rule evaluation decision, including outcomes that produce no event.
/// </summary>
public const string GatherRuleDebugLogPath = @"%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log";
public const string GatherRuleDebugLogPath = @"%ProgramData%\AutopilotMonitor\Logs\gather-rules-debug.log";
/// <summary>
/// Scheduled Task name used to run the agent as SYSTEM
@@ -416,6 +416,13 @@ namespace AutopilotMonitor.Shared
// IME log tracker — app / device / script telemetry (V2 single-rail plan §5.9)
public const string EspPhaseChanged = "esp_phase_changed";
public const string ImeAgentVersion = "ime_agent_version";
public const string ImeTokenFailure = "ime_token_failure"; // Warning, once per distinct errorCode: IME token acquisition failed (IntuneTokenManager "Failed to get AAD token ... errorCode = N"). Complements IME-TOKEN-SUCCESS: an auth outage previously left every check-in silently doing nothing with no timeline evidence.
// Registry second pillar (ImeRegistryAppStateObserver, audit 2026-08-17): IME's authoritative
// per-app state read straight from HKLM\...\IntuneManagementExtension. Observability only —
// never consumed by the DecisionEngine.
public const string RegistryAppState = "registry_app_state"; // Info, state-change-only (snapshot-diff vs silent startup baseline; capped 200/session, cap announced once as Warning): per-app EnforcementState/ErrorCode/ExitCode/StatusService status/ESP-tracking from the registry.
public const string RegistryAppBaseline = "registry_app_baseline"; // Info, one-shot at agent start (only when non-empty): summary of the Win32Apps state that PREDATES this agent run. Closes the WDP pre-agent gap — DPP-policy apps install in Batch 1 before the bootstrap-script agent exists, so their outcomes live only in this baseline (typed payload capped at 50 apps).
public const string AppStateReconciliation = "app_state_reconciliation"; // Warning, once per app, immediate upload: registry terminal outcome (settled >= 90 s) diverges from the IME-log-derived app state — the built-in log-pattern drift alarm (registry = truth, logs = narrative).
public const string ImeUserSessionCompleted = "ime_user_session_completed";
public const string DoTelemetry = "do_telemetry";
public const string AllAppsCompleted = "all_apps_completed";
@@ -1295,6 +1302,19 @@ namespace AutopilotMonitor.Shared
/// settings (Soft-Delete + Versioning) provide defence-in-depth.
/// </summary>
public const string CriticalTableBackups = "critical-table-backups";
/// <summary>
/// Permanent platform archive of IME (Intune Management Extension) installer
/// binaries, one virtual folder per version:
/// <c>{version}/IntuneWindowsAgent.msi</c> + <c>{version}/provenance.json</c>.
/// Written by <c>ImeMsiArchiver</c> when a new IME version is first seen across
/// the fleet (companion of <see cref="TableNames.ImeVersionHistory"/>, which is
/// equally KeptByDesign). NOT tenant-scoped — blob paths are version-keyed, so
/// tenant offboarding (Phase 2.E) deliberately never touches this container.
/// No lifecycle rule: the archive is the point (build-to-build diffs of
/// Microsoft's rollouts). Private by default (SDK CreateIfNotExists).
/// </summary>
public const string ImeArchive = "ime-archive";
}
/// <summary>
@@ -1392,6 +1412,18 @@ namespace AutopilotMonitor.Shared
/// <summary>Poison sibling of <see cref="SessionDeletionMaintenance"/>.</summary>
public const string SessionDeletionMaintenancePoison = "session-deletion-maintenance-poison";
/// <summary>
/// IME-MSI archive fan-out. Producer = <c>EventIngestProcessor</c> when
/// <c>RecordImeVersionAsync</c> reports a genuinely NEW IME version (first sighting
/// across the fleet — roughly monthly); consumer = <c>ImeMsiArchiveQueueWorker</c>,
/// which downloads the installer from the CSP-reported
/// <c>msiDownloadUrl</c> (host-allowlisted, size-capped) into
/// <see cref="BlobContainers.ImeArchive"/> and merges the outcome into the
/// <see cref="TableNames.ImeVersionHistory"/> row. Poison suffix <c>-poison</c>,
/// max-dequeue 5.
/// </summary>
public const string ImeMsiArchive = "ime-msi-archive";
}
/// <summary>
@@ -176,6 +176,14 @@ namespace AutopilotMonitor.Shared.DataAccess
Task<bool> RecordImeVersionAsync(string version, string tenantId, string sessionId);
Task<List<ImeVersionHistoryEntry>> GetImeVersionHistoryAsync();
/// <summary>
/// Merges the installer-archiving outcome (<c>ImeMsiArchiver</c>) into the version's
/// <c>ImeVersionHistory</c> row. Fail-soft: storage errors are logged, never thrown.
/// Blob path / hash / size / URL are only written on success (null leaves them absent).
/// </summary>
Task UpdateImeVersionArchiveInfoAsync(
string version, string status, string? blobPath, string? sha256, long? sizeBytes, string? sourceUrl);
// --- Events ---
Task<bool> StoreEventAsync(EnrollmentEvent evt);
Task<List<EnrollmentEvent>> StoreEventsBatchAsync(List<EnrollmentEvent> events);
@@ -460,6 +460,24 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public bool AutoApproveNewTenants { get; set; } = false;
/// <summary>
/// When true, the first fleet-wide sighting of a new IME version enqueues an
/// archive job (<c>ime-msi-archive</c> queue) that downloads the installer from the
/// CSP-reported URL into the <c>ime-archive</c> blob container. Roughly one download
/// per Microsoft IME release (~monthly). Default true — flip off to pause archiving
/// (queued messages then wait, they are not dropped). Round-tripped via the 4-file
/// web chain (memory <c>feedback_admin_config_ui_roundtrip</c>).
/// </summary>
public bool ImeMsiArchivingEnabled { get; set; } = true;
/// <summary>
/// Size cap for the IME installer download in MB (Content-Length preflight AND
/// streamed-byte guard). The MSI is ~12 MB today; the generous cap exists only so a
/// tampered/wrong URL cannot stream gigabytes into the archive while still never
/// missing a legitimately grown installer. 0 = no limit. Default: 250 MB.
/// </summary>
public int MaxImeMsiDownloadSizeMB { get; set; } = 250;
/// <summary>
/// When true, the dual app-registration self-service homing flip is active: the Graph
/// consent flow funnels legacy-homed tenants to the primary (new) app registration and
@@ -0,0 +1,44 @@
using System;
namespace AutopilotMonitor.Shared.Models
{
/// <summary>
/// Queue-message envelope for the <c>ime-msi-archive</c> queue. Enqueued by
/// <c>EventIngestProcessor</c> when an <c>ime_agent_version</c> event carries a version
/// the platform has never seen before (RecordImeVersionAsync insert succeeded). The
/// worker downloads the IME installer and archives it into the <c>ime-archive</c> blob
/// container so every fleet-observed IME build stays available for later decompilation
/// and build-to-build diffing, even after Microsoft's versionless CDN URL has moved on.
/// </summary>
public sealed class ImeMsiArchiveEnvelope
{
/// <summary>Schema version — bump on breaking envelope changes so consumers can reject or migrate.</summary>
public string EnvelopeVersion { get; set; } = "1";
/// <summary>The newly sighted IME version, e.g. <c>1.104.102.0</c> (blob folder name).</summary>
public string Version { get; set; } = string.Empty;
/// <summary>
/// The EnterpriseDesktopAppManagement-CSP <c>CurrentDownloadUrl</c> from the event's
/// <c>msiDownloadUrl</c> enrichment. Null when the reporting agent predates the
/// enrichment — the worker then falls back to the canonical versionless CDN URL
/// (safe: a brand-new version is by definition what that URL currently serves).
/// </summary>
public string? MsiDownloadUrl { get; set; }
/// <summary>
/// The event's <c>msiMatchedBy</c> companion (<c>productVersion</c> = URL is
/// version-authoritative, <c>fileName</c> = registry drift). Provenance only.
/// </summary>
public string? MsiMatchedBy { get; set; }
/// <summary>First-seen tenant — provenance only, never used for auth or paths.</summary>
public string TenantId { get; set; } = string.Empty;
/// <summary>First-seen session — provenance only.</summary>
public string SessionId { get; set; } = string.Empty;
/// <summary>UTC time the producer enqueued the message; useful for measuring queue lag.</summary>
public DateTime EnqueuedAt { get; set; }
}
}
@@ -61,7 +61,17 @@ namespace AutopilotMonitor.Shared.Models
/// event, missing ResumedAt, or a derived part-1 length that contradicts the stored
/// combined duration) — the two-window split is best-effort rather than event-anchored.
/// </summary>
WhiteGloveAnchorsIncomplete = 16
WhiteGloveAnchorsIncomplete = 16,
/// <summary>
/// The device carries state from a previous enrollment (<c>historic_ime_replay_detected</c>:
/// the IME log on disk predates this enrollment by more than the staleness threshold — the
/// machine was re-enrolled without a wipe). Pre-installed apps then complete as instant
/// detections, which pulls the phase anchors far ahead of the real ESP page (session
/// f475e697: the AccountSetup anchor fired 6.5 min before device setup actually finished),
/// so the segment durations misrepresent the enrollment.
/// </summary>
PriorEnrollmentResidue = 32
}
/// <summary>
@@ -74,7 +84,8 @@ namespace AutopilotMonitor.Shared.Models
{
/// <summary>
/// Flags that make the SEGMENT DURATIONS themselves untrustworthy — dropped anchors,
/// underobserved early phases, best-effort WhiteGlove window boundaries. Only these
/// underobserved early phases, best-effort WhiteGlove window boundaries, phase anchors
/// distorted by prior-enrollment residue. Only these
/// exclude a breakdown from fleet segment statistics. The blocking-set flags
/// (<see cref="TimeAttributionFlags.BlockingSetUnknown"/> / BlockingSetTruncated) say
/// nothing about the measured spans — they only limit per-app blocking evidence, which
@@ -85,7 +96,8 @@ namespace AutopilotMonitor.Shared.Models
public const TimeAttributionFlags DurationCriticalFlags =
TimeAttributionFlags.ClockSkewDropped |
TimeAttributionFlags.PartialObservation |
TimeAttributionFlags.WhiteGloveAnchorsIncomplete;
TimeAttributionFlags.WhiteGloveAnchorsIncomplete |
TimeAttributionFlags.PriorEnrollmentResidue;
/// <summary>True when the breakdown's segment durations are unfit for fleet statistics.</summary>
public static bool ExcludesFromFleetStats(TimeAttributionFlags flags)
@@ -575,5 +575,25 @@ namespace AutopilotMonitor.Shared.Models
public string FirstSeenTenantId { get; set; } = string.Empty;
public DateTime LastSeenAt { get; set; }
public int SessionCount { get; set; }
/// <summary>
/// Outcome of the automatic installer archiving (<c>ImeMsiArchiver</c>):
/// <c>Archived</c> or a <c>Failed:*</c> status; null for versions sighted before the
/// feature existed or while the archive job is still queued. Like the FirstSeen*
/// identifiers, the archive fields are only serialized for Global Admin callers.
/// </summary>
public string? MsiArchiveStatus { get; set; }
/// <summary>Blob path inside the <c>ime-archive</c> container, e.g. <c>1.104.102.0/IntuneWindowsAgent.msi</c>.</summary>
public string? MsiArchiveBlobPath { get; set; }
/// <summary>SHA-256 of the archived installer (hex, lowercase).</summary>
public string? MsiSha256 { get; set; }
/// <summary>Size of the archived installer in bytes.</summary>
public long? MsiBytes { get; set; }
/// <summary>The URL the installer was downloaded from (CSP-reported or canonical fallback).</summary>
public string? MsiSourceUrl { get; set; }
}
}
@@ -226,7 +226,7 @@ export default function AboutPage() {
items: [
".NET binary — lightweight, low-overhead monitoring",
"Runs via scheduled task (no Windows service — easy, residue-free removal)",
"Deployed via an Intune platform script (PowerShell bootstrapper)",
"Deployed via an Intune platform script (PowerShell bootstrapper); for Autopilot Device Preparation additionally as a thin MSI line-of-business app",
"Mutual TLS using the existing Intune MDM device certificate",
"Self-destruct on enrollment completion (on by default — removes task and files)",
],
@@ -50,6 +50,10 @@ interface AdminConfigContextValue {
setAutoApproveNewTenants: (value: boolean) => void;
selfServiceAppHomingEnabled: boolean;
setSelfServiceAppHomingEnabled: (value: boolean) => void;
imeMsiArchivingEnabled: boolean;
setImeMsiArchivingEnabled: (value: boolean) => void;
maxImeMsiDownloadSizeMB: number;
setMaxImeMsiDownloadSizeMB: (value: number) => void;
// Diagnostics log paths
globalDiagPaths: DiagnosticsLogPath[];
@@ -152,6 +156,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
const [sessionDeletionKillSwitch, setSessionDeletionKillSwitch] = useState(false);
const [autoApproveNewTenants, setAutoApproveNewTenants] = useState(false);
const [selfServiceAppHomingEnabled, setSelfServiceAppHomingEnabled] = useState(false);
const [imeMsiArchivingEnabled, setImeMsiArchivingEnabled] = useState(true);
const [maxImeMsiDownloadSizeMB, setMaxImeMsiDownloadSizeMB] = useState(250);
// Diagnostics Log Paths state
const [globalDiagPaths, setGlobalDiagPaths] = useState<DiagnosticsLogPath[]>([]);
@@ -215,6 +221,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
setSessionDeletionKillSwitch(data.sessionDeletionKillSwitch ?? false);
setAutoApproveNewTenants(data.autoApproveNewTenants ?? false);
setSelfServiceAppHomingEnabled(data.selfServiceAppHomingEnabled ?? false);
setImeMsiArchivingEnabled(data.imeMsiArchivingEnabled ?? true);
setMaxImeMsiDownloadSizeMB(data.maxImeMsiDownloadSizeMB ?? 250);
try {
setGlobalDiagPaths(data.diagnosticsGlobalLogPathsJson ? JSON.parse(data.diagnosticsGlobalLogPathsJson) : []);
} catch {
@@ -332,6 +340,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
sessionDeletionKillSwitch,
autoApproveNewTenants,
selfServiceAppHomingEnabled,
imeMsiArchivingEnabled,
maxImeMsiDownloadSizeMB,
};
const response = await authenticatedFetch(api.globalConfig.get(), getAccessToken, {
@@ -358,7 +368,7 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
} finally {
setSavingConfig(false);
}
}, [isGlobalAdmin, adminConfig, globalRateLimit, userRateLimit, globalAdminRateLimit, platformStatsBlobSasUrl, agentMigrateApiBaseUrl, agentMigrateTenantOverridesJson, collectorIdleTimeoutMinutes, desktopDetectorNoCandidateTimeoutMinutes, opsEventRetentionDays, slaNotificationCooldownHours, allowAgentDowngrade, modernDeploymentHarmlessEventIds, enableIndexDualWrite, sessionDeletionKillSwitch, autoApproveNewTenants, selfServiceAppHomingEnabled, getAccessToken]);
}, [isGlobalAdmin, adminConfig, globalRateLimit, userRateLimit, globalAdminRateLimit, platformStatsBlobSasUrl, agentMigrateApiBaseUrl, agentMigrateTenantOverridesJson, collectorIdleTimeoutMinutes, desktopDetectorNoCandidateTimeoutMinutes, opsEventRetentionDays, slaNotificationCooldownHours, allowAgentDowngrade, modernDeploymentHarmlessEventIds, enableIndexDualWrite, sessionDeletionKillSwitch, autoApproveNewTenants, selfServiceAppHomingEnabled, imeMsiArchivingEnabled, maxImeMsiDownloadSizeMB, getAccessToken]);
// Reset admin config
const handleResetAdminConfig = useCallback(() => {
@@ -379,6 +389,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
setSessionDeletionKillSwitch(adminConfig.sessionDeletionKillSwitch ?? false);
setAutoApproveNewTenants(adminConfig.autoApproveNewTenants ?? false);
setSelfServiceAppHomingEnabled(adminConfig.selfServiceAppHomingEnabled ?? false);
setImeMsiArchivingEnabled(adminConfig.imeMsiArchivingEnabled ?? true);
setMaxImeMsiDownloadSizeMB(adminConfig.maxImeMsiDownloadSizeMB ?? 250);
try {
setGlobalDiagPaths(adminConfig.diagnosticsGlobalLogPathsJson ? JSON.parse(adminConfig.diagnosticsGlobalLogPathsJson) : []);
} catch {
@@ -525,6 +537,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
sessionDeletionKillSwitch, setSessionDeletionKillSwitch,
autoApproveNewTenants, setAutoApproveNewTenants,
selfServiceAppHomingEnabled, setSelfServiceAppHomingEnabled,
imeMsiArchivingEnabled, setImeMsiArchivingEnabled,
maxImeMsiDownloadSizeMB, setMaxImeMsiDownloadSizeMB,
globalDiagPaths, setGlobalDiagPaths, savingDiagPaths,
opsAlertRules, setOpsAlertRules,
opsAlertTelegramEnabled, setOpsAlertTelegramEnabled,
@@ -36,6 +36,10 @@ interface AdminConfigSettingsSectionProps {
setAutoApproveNewTenants: (value: boolean) => void;
selfServiceAppHomingEnabled: boolean;
setSelfServiceAppHomingEnabled: (value: boolean) => void;
imeMsiArchivingEnabled: boolean;
setImeMsiArchivingEnabled: (value: boolean) => void;
maxImeMsiDownloadSizeMB: number;
setMaxImeMsiDownloadSizeMB: (value: number) => void;
onSave: () => Promise<void>;
onReset: () => void;
}
@@ -74,6 +78,10 @@ export function AdminConfigSettingsSection({
setAutoApproveNewTenants,
selfServiceAppHomingEnabled,
setSelfServiceAppHomingEnabled,
imeMsiArchivingEnabled,
setImeMsiArchivingEnabled,
maxImeMsiDownloadSizeMB,
setMaxImeMsiDownloadSizeMB,
onSave,
onReset,
}: AdminConfigSettingsSectionProps) {
@@ -375,6 +383,47 @@ export function AdminConfigSettingsSection({
</label>
</div>
<div>
<label className="flex items-start space-x-3 cursor-pointer">
<input
type="checkbox"
checked={imeMsiArchivingEnabled}
onChange={(e) => setImeMsiArchivingEnabled(e.target.checked)}
className="mt-1 h-5 w-5 rounded border-indigo-300 dark:border-indigo-600 text-indigo-600 accent-indigo-600 focus:ring-indigo-500"
/>
<span>
<span className="text-indigo-900 dark:text-indigo-100 font-medium">IME installer archiving</span>
<p className="text-sm text-indigo-800 dark:text-gray-300 mt-1">
When <strong>on</strong>, the first fleet-wide sighting of a new IME version downloads
the installer (from the CSP-reported, allowlisted URL) into the permanent
<code className="ml-1 text-xs bg-indigo-100 dark:bg-indigo-900 dark:text-indigo-200 px-1 rounded">ime-archive</code> blob
container — roughly one small download per Microsoft IME release. The outcome is
visible per version in the IME version history. When <strong>off</strong>, queued
archive jobs wait (they are not dropped). Default <strong>on</strong>.
</p>
</span>
</label>
</div>
<div>
<label className="block">
<span className="text-indigo-900 dark:text-indigo-100 font-medium">IME installer download size cap (MB)</span>
<p className="text-sm text-indigo-800 dark:text-gray-300 mb-2">
Safety cap for the IME installer download (the MSI is ~12 MB today). Generous on
purpose so a legitimately grown installer is never missed, while a wrong URL can
still not stream gigabytes into the archive. 0 = unlimited. Default 250.
</p>
<input
type="number"
min="0"
max="10000"
value={maxImeMsiDownloadSizeMB}
onChange={(e) => setMaxImeMsiDownloadSizeMB(parseInt(e.target.value) || 0)}
className="mt-1 block w-full px-4 py-2 border border-indigo-300 dark:border-indigo-600 rounded-lg bg-white dark:bg-gray-700 text-gray-900 dark:text-gray-100 placeholder-gray-400 dark:placeholder-gray-500 focus:outline-none focus:ring-2 focus:ring-green-500 focus:border-green-500 transition-colors"
/>
</label>
</div>
<div>
<label className="flex items-start space-x-3 cursor-pointer">
<input
@@ -24,6 +24,8 @@ export function SectionGlobalSettings() {
sessionDeletionKillSwitch, setSessionDeletionKillSwitch,
autoApproveNewTenants, setAutoApproveNewTenants,
selfServiceAppHomingEnabled, setSelfServiceAppHomingEnabled,
imeMsiArchivingEnabled, setImeMsiArchivingEnabled,
maxImeMsiDownloadSizeMB, setMaxImeMsiDownloadSizeMB,
handleSaveAdminConfig, handleResetAdminConfig,
} = useAdminConfig();
@@ -66,6 +68,10 @@ export function SectionGlobalSettings() {
setAutoApproveNewTenants={setAutoApproveNewTenants}
selfServiceAppHomingEnabled={selfServiceAppHomingEnabled}
setSelfServiceAppHomingEnabled={setSelfServiceAppHomingEnabled}
imeMsiArchivingEnabled={imeMsiArchivingEnabled}
setImeMsiArchivingEnabled={setImeMsiArchivingEnabled}
maxImeMsiDownloadSizeMB={maxImeMsiDownloadSizeMB}
setMaxImeMsiDownloadSizeMB={setMaxImeMsiDownloadSizeMB}
onSave={handleSaveAdminConfig}
onReset={handleResetAdminConfig}
/>
@@ -180,8 +180,12 @@ export default function EventTimeline({
<div className="bg-white shadow rounded-lg p-4">
<h2 className="text-sm font-semibold text-gray-700 mb-3">Raw Events ({sortedBySequence.length})</h2>
<div className="divide-y divide-gray-100">
{sortedBySequence.map((ev) => (
<RawEventRow key={ev.eventId || `${ev.sessionId}-${ev.sequence}`} event={ev} />
{sortedBySequence.map((ev, i) => (
<RawEventRow
key={ev.eventId || `${ev.sessionId}-${ev.sequence}`}
event={ev}
prevDisplayTime={i > 0 ? getDisplayTime(sortedBySequence[i - 1]) : null}
/>
))}
</div>
</div>
@@ -341,11 +345,12 @@ function PhaseSection({
{isExpanded && (
<div className="space-y-3">
{events.map((event) => (
{events.map((event, i) => (
<EventRow
key={event.eventId || `${event.sessionId}-${event.sequence}`}
event={event}
showScriptOutput={showScriptOutput}
prevDisplayTime={i > 0 ? getDisplayTime(events[i - 1]) : null}
/>
))}
</div>
@@ -372,7 +377,32 @@ const BACKFILL_TOOLTIP =
"If enrollment progressed afterwards, this error was already resolved by a retry. " +
"The timestamp shown is the original event-log time.";
function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScriptOutput?: boolean }) {
// Display timestamp of a row: backfilled events show the original event-log time.
function getDisplayTime(event: EnrollmentEvent): Date {
return getBackfillInfo(event).recordedAt ?? new Date(event.timestamp);
}
// Rows render time-of-day only, so a multi-hour/multi-day gap to the previous row
// (and the date change it implies) is invisible without expanding event details.
// Shown on the row AFTER the gap; hover reveals the full date.
function GapBadge({ prevTime, eventTime }: { prevTime: Date | null; eventTime: Date }) {
if (!prevTime) return null;
const gapMs = eventTime.getTime() - prevTime.getTime();
if (gapMs < 60 * 60 * 1000) return null;
const label = gapMs >= 48 * 60 * 60 * 1000
? `+${Math.round(gapMs / (24 * 60 * 60 * 1000))}d`
: `+${Math.round(gapMs / (60 * 60 * 1000))}h`;
return (
<span
className="px-1.5 py-0.5 rounded text-xs font-medium bg-amber-50 text-amber-700 whitespace-nowrap flex-shrink-0"
title={`${label} after the previous event — ${eventTime.toLocaleString()}`}
>
⏱ {label}
</span>
);
}
function EventRow({ event, showScriptOutput, prevDisplayTime }: { event: EnrollmentEvent; showScriptOutput?: boolean; prevDisplayTime?: Date | null }) {
const [showDetails, setShowDetails] = useState(false);
const [showRaw, setShowRaw] = useState(false);
const [copied, setCopied] = useState(false);
@@ -458,6 +488,7 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
<span className="text-gray-400 font-sans"> (reported {new Date(event.timestamp).toLocaleTimeString()})</span>
)}
</span>
<GapBadge prevTime={prevDisplayTime ?? null} eventTime={recordedAt ?? new Date(event.timestamp)} />
<SeverityBadge severity={event.severity} />
{isBackfilled && (
<span
@@ -786,7 +817,7 @@ function EventRow({ event, showScriptOutput }: { event: EnrollmentEvent; showScr
);
}
function RawEventRow({ event }: { event: EnrollmentEvent }) {
function RawEventRow({ event, prevDisplayTime }: { event: EnrollmentEvent; prevDisplayTime?: Date | null }) {
const [expanded, setExpanded] = useState(false);
const detailData = useMemo(() => normalizeEventDataForDisplay(event.data), [event.data]);
const hasDetails = detailData && Object.keys(detailData).length > 0;
@@ -806,6 +837,7 @@ function RawEventRow({ event }: { event: EnrollmentEvent }) {
<div className="flex items-start gap-2">
<span className="text-gray-400 w-8 text-right flex-shrink-0">{event.sequence}</span>
<span className="text-gray-500 flex-shrink-0">{(recordedAt ?? new Date(event.timestamp)).toLocaleTimeString()}</span>
<GapBadge prevTime={prevDisplayTime ?? null} eventTime={recordedAt ?? new Date(event.timestamp)} />
<span className={`flex-shrink-0 w-14 ${sevColor[event.severity] || "text-gray-500"}`}>{event.severity}</span>
{isBackfilled && (
<span className="text-slate-500 flex-shrink-0" title={BACKFILL_TOOLTIP}>⟲</span>

Some files were not shown because too many files have changed in this diff Show More