fix(agent): neutral aggregated 2800 observations; reboot claim gated on observed reboot

Rework after the first real enrollment (session b2e890c1) exposed three
defects in a654c730:

- Shell-Core "RebootCoalescing" match was a false positive by construction:
  the token appears in the ROUTINE SubcategoryProcessing_Started bootstrap
  marker ("Starting subcategory DeviceSetup.RebootCoalescing...") on EVERY
  enrollment. esp_reboot_coalescing removed entirely (branch, constant,
  catalog, tests).
- 2800 reboot-required flags fire at every policy sync; only DeviceSetup-time
  flags cause the coalesced reboot. The session's flags fired during
  AccountSetup - no reboot - yet our Warning events claimed "device will
  restart". Raw event is now a neutral Info observation; the reboot CLAIM
  moved to ANALYZE-ESP-005 with a REQUIRED system_reboot_detected >= 1
  condition (no observed reboot -> advisory silent).
- One event per 2800 record was spam: records are now buffered and debounced
  (10 s) into ONE aggregated event per burst (rebootUris[]/uriCount/
  firstRebootUri/recordCount/unparsedCount). Watermark is persisted at FLUSH,
  not claim - a pre-emit reboot kill loses nothing, the post-restart backfill
  re-emits the burst at its historical time.

Verified 2800 description shape recorded in code, tests, and concept doc:
"The following URI has triggered a reboot: (./Device/...)."; extraction
covered by a real-text regression test. Lesson added to tasks/lessons.md
(observation-level events vs outcome-gated claims; substring tokens appear
in framework bookkeeping records).

Tests: Agent.V2.Core 1867/1867, Functions 3099/3099, MCP 400 passed,
solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-07-20 18:41:47 +02:00
parent a654c7309e
commit a97888ebd5
12 changed files with 504 additions and 551 deletions
+1 -1
View File
@@ -7,6 +7,6 @@ How the on-device agent works:
* [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine.
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
Reading order for newcomers: overview → decision engine → persistence.
+57 -66
View File
@@ -1,7 +1,7 @@
---
type: Concept
title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs
description: How the agent detects device-assigned MDM policies that force a coalesced reboot during ESP DeviceSetup (DM-Enterprise EventID 2800 + Shell-Core 62407 RebootCoalescing), and the ANALYZE-ESP-005 advisory that makes the invisible "second sign-in" fixable.
description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
tags:
- agent
@@ -17,91 +17,82 @@ timestamp: 2026-07-20T00:00:00+02:00
Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog
(`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced
reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a
**second sign-in screen** before AccountSetup. This is expected OS behavior, invisible in
the Intune console; most admins simply live with it, not knowing that reassigning the
profiles to user groups removes the reboot entirely. Source research: Patch My PC,
**second sign-in screen** before AccountSetup. Invisible in the Intune console; most
admins live with it, not knowing that reassigning the profiles to user groups removes the
reboot entirely. Source research: Patch My PC,
["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/).
The session already counts reboots (`system_reboot_detected` → `session.rebootCount`);
this feature adds the **cause attribution**.
# Signal chain
# Verified signal facts (session b2e890c1, 2026-07-20)
1. **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`
— logged by the SyncML engine once per policy URI that matched the reboot-required
catalog, during device-policy application in DeviceSetup.
2. **EventID 62407**, `Microsoft-Windows-Shell-Core/Operational`, description containing
`RebootCoalescing` — ESP/CloudExperienceHost initiating the coalesced reboot at the end
of DeviceSetup.
3. The reboot kills the agent; the post-reboot restart re-observes both channels via
backfill (the .evtx persists across the reboot).
* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`,
one record per URI, verified description shape:
`The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).`
Records arrive in sub-second bursts during policy sync.
* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs
applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the
verified session the flags fired during AccountSetup (after DeviceSetup completed) —
**no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that
claims "reboot incoming" on every 2800 is therefore wrong.
* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the
ROUTINE bootstrap marker on every enrollment:
`Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'`
— the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a
reboot is needed). A substring match on it is a false positive by construction. If a
real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be
added then.
# Detectors
# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`)
## `MdmRebootPolicyTracker` (new collector, `MdmRebootPolicyWatcherHost`)
Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup
backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then
flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst:
Structural clone of `WindowsUpdateTracker`: live `EventLogWatcher` on the DM-Enterprise
Admin channel (EventID 2800) + startup backfill (default 60 min lookback) + **cross-restart
RecordId watermark** (`mdm-reboot-watermark.json`). The watermark is mandatory here: the
coalesced reboot restarts the agent, and without it every reboot-forcing URI would be
re-emitted after the very reboot it caused.
* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as
reboot-required: … (+N more)") — the raw event never claims a reboot.
* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct),
`firstRebootUri` (record-time order, drives rule interpolation), `recordCount`,
`unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill),
fail-soft `omadmRebootRequiredFlag` probe of
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown).
* `Timestamp` = earliest record time (backfilled bursts land at their historical instant);
`ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a
reboot that kills the agent).
* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the
coalesced reboot kills the process before the debounced emit, nothing is marked — the
post-restart backfill re-reads and emits the burst. A flushed burst is never repeated.
Per 2800 record it emits one `mdm_policy_reboot_required` event (Warning,
`ImmediateUpload=true` — the announced reboot will kill the agent, flush now) with
`Data.rebootUri` (the CSP URI), `recordId`, `backfilled`, `description` (truncated 1000),
and a fail-soft `omadmRebootRequiredFlag` corroboration probe of
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown, never
blocks the emit). Per-URI emission (not aggregated) is deliberate: an aggregate would need
a flush point that races the reboot kill.
URI extraction: EventData values starting with `./` preferred, permissive regex over the
description as fallback (trailing punctuation trimmed — the verified text wraps the URI in
parens and ends with a period). Unparsed records still count into the aggregate.
**Tolerant matching**: the exact 2800 message text is unverified — emission is gated on
the EventID only. URI extraction prefers structured EventData values starting with `./`
and falls back to a permissive regex over the description (trailing punctuation trimmed).
Events with an unparseable URI are still emitted (timeline-visible, rule stays silent, and
the captured description tells us how to fix the extraction). Exact texts are validated
post-deploy on the E2E VM and recorded here.
Config (internal steering only, no admin UI — this is an agent-native collector, NOT an
Config (internal steering only, no admin UI — agent-native collector, NOT an
`IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true),
`Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35.
## `ShellCoreTracker` — `RebootCoalescing` branch
The already-watched 62407 stream gets a fourth branch (checked AFTER
WhiteGlove/failure/exiting so their semantics stay untouched): description containing
`RebootCoalescing` → one `esp_reboot_coalescing` event (Warning, ImmediateUpload), pure
informational — no `FinalizingSetup` transition, no C# event raise, no DecisionCore
involvement.
**Deliberate deviation**: unlike the other 62407 branches, the backfill path DOES emit
this event. The live emit races the very reboot that kills the process; the post-restart
5-min backfill re-emission is the only reliable delivery. A live+backfill cross-process
duplicate is possible and accepted (fire-once bool resets per process; both copies carry
the identical historical timestamp; the backfill copy is marked `backfilled:true`; the
analyze rule uses exists-semantics). The shared 5-min lookback is NOT enlarged — the
rule's required condition is the watermarked 2800 event; coalescing is optional
corroboration only.
# Advisory — ANALYZE-ESP-005
"MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)", severity `warning`,
`markSessionAsFailedDefault: false`. Required: `mdm_policy_reboot_required.rebootUri`
exists (first URI interpolates as `{{rebootUri}}`). Optional corroboration:
`esp_reboot_coalescing` observed (+20 confidence) and ≥2 reboot URIs (+10) on base 70 /
threshold 70. Remediation: map each URI to its Intune profile and reassign device → user
groups (or exclude the Autopilot device group), or accept + communicate the restart.
Because `{{token}}` interpolation is scalar-only, the explanation points at the timeline
for the complete URI list — each URI is its own event.
The reboot **claim** lives here, and only here. Required conditions (BOTH):
1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation)
2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot**
Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional
confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 /
threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or
exclude the Autopilot device group), or accept + communicate the restart.
# Non-goals / invariants
* The new event types are **deliberately distinct from `system_reboot_detected`** — the
backend RebootCount matches that string exactly in two places
(`EventIngestProcessor.Classification` + the terminal recount); attribution events must
never inflate the reboot count.
* Event type **deliberately distinct from `system_reboot_detected`** — the backend
RebootCount matches that string exactly in two places
(`EventIngestProcessor.Classification` + terminal recount); attribution must never
inflate the reboot count.
* No session-level field / SessionInfoCard change in this iteration (follow-up candidate).
* The `RebootRequiredURIs` catalog itself is not read (format unverified); only the 2800
events (which carry the actually-triggered URIs) are consumed.
* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry
the actually-triggered URIs) are consumed.
# Citations
+1 -1
View File
@@ -18,7 +18,7 @@ https://docs.autopilotmonitor.com (separate repository).
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: DM-Enterprise 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
# Rules
+1
View File
@@ -2,6 +2,7 @@
## 2026-07-20
* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.
* **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated.
* **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.
+24 -13
View File
@@ -2,10 +2,10 @@
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.0.0",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
@@ -15,37 +15,48 @@
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "rebootUri",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_uri_count",
"signal": "reboot_observed",
"source": "event_count",
"eventType": "mdm_policy_reboot_required",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "esp_coalesced_reboot_observed",
"source": "event_type",
"eventType": "esp_reboot_coalescing",
"operator": "exists",
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{ "signal": "esp_coalesced_reboot_observed", "condition": "exists", "weight": 20 },
{ "signal": "mdm_policy_reboot_required", "condition": "count >= 2", "weight": 10 }
{ "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 },
{ "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 }
],
"confidenceThreshold": 70,
"explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)",
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
+27 -16
View File
@@ -1788,10 +1788,10 @@
{
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.0.0",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
@@ -1801,45 +1801,56 @@
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "rebootUri",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_uri_count",
"signal": "reboot_observed",
"source": "event_count",
"eventType": "mdm_policy_reboot_required",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "esp_coalesced_reboot_observed",
"source": "event_type",
"eventType": "esp_reboot_coalescing",
"operator": "exists",
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{
"signal": "esp_coalesced_reboot_observed",
"signal": "multiple_reboot_uris",
"condition": "exists",
"weight": 20
"weight": 10
},
{
"signal": "mdm_policy_reboot_required",
"condition": "count >= 2",
"weight": 10
"signal": "omadm_reboot_flag_set",
"condition": "exists",
"weight": 15
}
],
"confidenceThreshold": 70,
"explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)",
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
@@ -1,195 +0,0 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
{
/// <summary>
/// Shell-Core 62407 "RebootCoalescing" branch — ESP-initiated coalesced reboot (MDM policies
/// forced a mid-ESP restart → second sign-in). Pure informational corroboration for the
/// MdmRebootPolicyTracker's per-URI 2800 attribution: no FinalizingSetup transition, no C#
/// event raise, and — as a DELIBERATE deviation from the other 62407 backfill branches — the
/// backfill path DOES emit (the live emit races the very reboot that kills the process).
/// </summary>
public sealed class ShellCoreTrackerRebootCoalescingTests
{
private static readonly DateTime ClockNow = new DateTime(2026, 7, 20, 10, 5, 0, DateTimeKind.Utc);
private const string CoalescingDescription =
"CloudExperienceHost Web App Event 2. Name: 'CommercialOOBE_DeviceSetup_RebootCoalescing', Value: '...'.";
private static (ShellCoreTracker tracker, FakeSignalIngressSink sink) MakeTracker(TempDirectory tmp)
{
var sink = new FakeSignalIngressSink();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
var tracker = new ShellCoreTracker(
sessionId: "S1",
tenantId: "T1",
post: new InformationalEventPost(sink, new VirtualClock(ClockNow)),
logger: logger,
helloTracker: null);
return (tracker, sink);
}
private static IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(FakeSignalIngressSink sink, string eventType) =>
sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void ProcessEvent_62407_RebootCoalescing_EmitsWarning_ImmediateUpload()
{
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
var sourceTime = ClockNow.AddSeconds(-5);
tracker.ProcessEvent(
eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent,
description: CoalescingDescription,
timestamp: sourceTime,
providerName: "Microsoft-Windows-Shell-Core",
isBackfill: false);
var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
Assert.Equal(sourceTime, s.OccurredAtUtc);
Assert.Equal(CoalescingDescription, Data(s)["description"]);
}
}
[Fact]
public void ProcessEvent_62407_RebootCoalescing_FiresOncePerRun()
{
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false);
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
CoalescingDescription, ClockNow.AddSeconds(1), "Microsoft-Windows-Shell-Core", isBackfill: false);
Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
}
}
[Fact]
public void ProcessEvent_62407_RebootCoalescing_DoesNotTriggerEspSemantics()
{
// The coalescing record is corroboration only — it must not trip the ESP-exit /
// FinalizingSetup / failure machinery.
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
var raised = new List<string>();
tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}");
tracker.EspExited += (_, _) => raised.Add("espExited");
tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}");
tracker.WhiteGloveCompleted += (_, _) => raised.Add("whiteGlove");
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false);
Assert.Empty(raised);
Assert.False(tracker.IsEspExitedForTest);
Assert.Empty(ByType(sink, Constants.EventTypes.EspExiting));
Assert.Empty(ByType(sink, Constants.EventTypes.EspFailure));
}
}
[Fact]
public void ProcessEvent_62407_ExistingBranches_Unaffected()
{
// Ordering regression guard: WhiteGlove/failure/exiting still classify as before —
// the RebootCoalescing check comes AFTER them.
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
"Name: 'CommercialOOBE_ESPProgress_Page_Exiting'", ClockNow, "p", isBackfill: false);
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
"Name: 'CommercialOOBE_ESPProgress_WhiteGlove_Success'", ClockNow, "p", isBackfill: false);
Assert.Single(ByType(sink, Constants.EventTypes.EspExiting));
Assert.Single(ByType(sink, Constants.EventTypes.WhiteGloveComplete));
Assert.Empty(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
}
}
[Fact]
public void Backfill_RebootCoalescing_Emits_WithBackfilledTrue_AndHistoricalTimestamp()
{
// Deliberate deviation from the other 62407 backfill branches: after the coalesced
// reboot restarts the agent, the backfill re-emission is the only reliable delivery.
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
var historical = ClockNow.AddMinutes(-3);
tracker.HandleBackfillRecord(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, historical);
var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
Assert.Equal(historical, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
}
}
[Fact]
public void Backfill_RebootCoalescing_AfterLiveFire_DoesNotDoubleEmit_SameProcess()
{
// Same-process dedup via the fire-once flag. (Cross-process live+backfill duplication
// is possible and accepted — exists-semantics rule, identical historical timestamp.)
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent,
CoalescingDescription, ClockNow, "p", isBackfill: false);
tracker.HandleBackfillRecord(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-1));
Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing));
}
}
[Fact]
public void Backfill_RebootCoalescing_DoesNotFallThroughToExitOrFailureHandling()
{
using var tmp = new TempDirectory();
var (tracker, sink) = MakeTracker(tmp);
using (tracker)
{
var raised = new List<string>();
tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}");
tracker.EspExited += (_, _) => raised.Add("espExited");
tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}");
tracker.HandleBackfillRecord(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-2));
Assert.Empty(raised);
Assert.False(tracker.IsEspExitedForTest);
}
}
}
}
@@ -15,16 +15,22 @@ using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// MdmRebootPolicyTracker — attributes the "unexpected reboot + second sign-in" pattern to the
/// device-assigned policy URIs that forced the coalesced reboot (DM-Enterprise EventID 2800).
/// Drives the primitive <see cref="MdmRebootPolicyTracker.ProcessEvent"/> test seam (no real
/// EventRecord, which is abstract + Windows-only), mirroring the WindowsUpdateTracker pattern.
/// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral
/// <c>mdm_policy_reboot_required</c> event (session b2e890c1 rework: per-record events were
/// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an
/// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit
/// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed).
/// The registry probe is always overridden — tests never touch the live registry.
/// </summary>
public sealed class MdmRebootPolicyTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 7, 20, 9, 30, 0, DateTimeKind.Utc);
private const string SampleUri = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/EnableVirtualizationBasedSecurity";
private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc);
private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation";
private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch";
// Verified real 2800 description shape (session b2e890c1, 2026-07-20).
private const string RealDescription =
"The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
@@ -34,21 +40,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
public MdmRebootPolicyTrackerTests()
{
_sink = new FakeSignalIngressSink();
_tracker = MakeTracker(stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private MdmRebootPolicyTracker MakeTracker(string? stateDirectory)
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_tracker = new MdmRebootPolicyTracker(
var tracker = new MdmRebootPolicyTracker(
sessionId: "sess-mdm",
tenantId: "tenant-mdm",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: null); // in-memory dedup for most tests
_tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
stateDirectory: stateDirectory,
debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly
tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
return tracker;
}
public void Dispose() => _tmp.Dispose();
private void Emit(string? rebootUri = SampleUri, string? description = null, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null)
{
_tracker.ProcessEvent(
eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired,
@@ -70,104 +83,192 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void ProcessEvent_EmitsWarning_WithUri_AndImmediateUpload()
public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload()
{
Emit();
Buffer(UriSvchost, timeCreatedUtc: At);
Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120));
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal(SampleUri, data["rebootUri"]);
Assert.Equal(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, data["windowsEventId"]);
Assert.Equal(MdmRebootPolicyTracker.Channel, data["eventLogChannel"]);
Assert.Equal(2, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
Assert.Equal(UriSvchost, data["firstRebootUri"]);
var uris = Assert.IsType<List<string>>(data["rebootUris"]);
Assert.Equal(2, uris.Count);
Assert.Contains(UriSvchost, uris);
Assert.Contains(UriSystemGuard, uris);
}
[Fact]
public void ProcessEvent_NoUri_StillEmits_WithoutRebootUriField()
public void Message_IsNeutralObservation_NoRebootClaim()
{
// Tolerant contract: an unparsed URI must stay timeline-visible (with the captured
// description telling us how to fix the extraction) — the analyze rule keys on
// "rebootUri exists" and simply stays silent.
Emit(rebootUri: null, description: "Some unexpected 2800 wording");
// Session b2e890c1: flags raised during AccountSetup have no effect — the raw event
// must state the observation only, never a future consequence.
Buffer(UriSvchost);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var message = s.Payload![SignalPayloadKeys.Message];
Assert.Contains("flagged", message);
Assert.DoesNotContain("will restart", message);
Assert.DoesNotContain("second sign-in", message);
}
[Fact]
public void FlushPending_Empty_EmitsNothing()
{
_tracker.FlushPending();
Assert.Empty(Emitted());
}
[Fact]
public void AggregateTimestamp_IsEarliestRecordTime()
{
var early = At.AddMinutes(-30);
Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true);
Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(early, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
// firstRebootUri follows record-time order, not buffer order.
Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]);
}
[Fact]
public void Backfilled_False_WhenAnyRecordIsLive()
{
Buffer(UriSvchost, isBackfill: true);
Buffer(UriSystemGuard, isBackfill: false);
_tracker.FlushPending();
Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]);
}
[Fact]
public void DuplicateUris_AreDistinctInAggregate()
{
Buffer(UriSvchost);
Buffer(UriSvchost);
_tracker.FlushPending();
var data = Data(Assert.Single(Emitted()));
Assert.Equal(1, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
}
[Fact]
public void UnparsedRecords_CountedWithSampleDescription_StillEmitted()
{
Buffer(rebootUri: null, description: "Some unexpected 2800 wording");
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var data = Data(s);
Assert.False(data.ContainsKey("rebootUri"));
Assert.Equal("Some unexpected 2800 wording", data["description"]);
Assert.False(data.ContainsKey("rebootUris"));
Assert.False(data.ContainsKey("firstRebootUri"));
Assert.Equal(0, data["uriCount"]);
Assert.Equal(1, data["unparsedCount"]);
Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]);
}
[Fact]
public void BackfilledEvent_UsesEventTimeForTimelineTimestamp_AndMarksBackfilled()
public void SameRecordId_BufferedTwice_CountsOnce()
{
var eventTime = At.AddMinutes(-25); // policy applied well before agent start
Emit(isBackfill: true, timeCreatedUtc: eventTime);
Buffer(UriSvchost, recordId: 42);
Buffer(UriSvchost, recordId: 42);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(eventTime, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void SameRecordId_ProcessedTwice_EmitsOnce()
public void RecordAfterFlush_GoesIntoNextAggregate()
{
Emit(recordId: 42);
Emit(recordId: 42);
Assert.Single(Emitted());
}
[Fact]
public void BackfillEventWithLowerRecordId_AfterLiveEvent_IsStillEmitted()
{
// Same contract as WindowsUpdateTracker: the live watcher is armed BEFORE the backfill
// scan, so a live record with a higher RecordId can arrive first — the older,
// never-emitted backfill record must not be suppressed by a high-water mark.
Emit(recordId: 100);
Emit(recordId: 50, isBackfill: true);
Buffer(UriSvchost, recordId: 10);
_tracker.FlushPending();
Buffer(UriSystemGuard, recordId: 11);
_tracker.FlushPending();
Assert.Equal(2, Emitted().Count);
}
[Fact]
public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered()
{
// Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill
// record must not be suppressed by a high-water mark.
Buffer(UriSvchost, recordId: 100);
Buffer(UriSystemGuard, recordId: 50, isBackfill: true);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void NegativeRecordId_IsNeverDeduped()
{
Emit(recordId: -1);
Emit(recordId: -1);
Buffer(UriSvchost, recordId: -1);
Buffer(UriSvchost, recordId: -1);
_tracker.FlushPending();
Assert.Equal(2, Emitted().Count);
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void Watermark_PersistsAcrossTrackerInstances()
public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances()
{
// The coalesced reboot restarts the agent and the post-reboot backfill re-reads the
// same 2800 records — the persisted watermark must make that re-read a no-op.
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var first = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
first.RebootRequiredFlagProbe = () => null;
// The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same
// records — a FLUSHED record must be skipped by the next instance.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: false);
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
first.FlushPending();
var second = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
second.RebootRequiredFlagProbe = () => null;
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: true);
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
Assert.Single(Emitted());
}
[Fact]
public void Watermark_NotPersisted_WhenBufferedButNeverFlushed()
{
// Process killed by the very reboot the records announce: nothing was flushed, so the
// next instance must re-read and emit the records.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
// no FlushPending — simulated reboot kill
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(true, Data(s)["backfilled"]);
}
[Fact]
public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt()
{
_tracker.RebootRequiredFlagProbe = () => true;
Emit(recordId: 1);
Buffer(UriSvchost, recordId: 1);
_tracker.FlushPending();
_tracker.RebootRequiredFlagProbe = () => null;
Emit(recordId: 2);
Buffer(UriSystemGuard, recordId: 2);
_tracker.FlushPending();
var emitted = Emitted();
Assert.Equal(2, emitted.Count);
@@ -179,10 +280,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
public void RebootRequiredFlagProbe_Throwing_IsFailSoft()
{
_tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom");
Emit();
Buffer(UriSvchost);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.False(Data(s).ContainsKey("omadmRebootRequiredFlag"));
Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
@@ -194,20 +295,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
// ------------------------------------------------------------------ URI extraction
[Fact]
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
public void ExtractRebootUri_FromVerifiedRealDescription()
{
var eventData = new Dictionary<string, string> { ["Data0"] = " " + SampleUri + " " };
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, "unrelated ./Other/Uri text");
Assert.Equal(SampleUri, uri);
// Real 2800 text wraps the URI in parentheses and ends with a period — both must be
// excluded from the extracted URI.
var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription);
Assert.Equal(UriSvchost, uri);
}
[Fact]
public void ExtractRebootUri_RegexFallback_FromDescription()
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
{
var uri = MdmRebootPolicyTracker.ExtractRebootUri(
new Dictionary<string, string> { ["Data0"] = "not a uri" },
$"The following URI has triggered a reboot: {SampleUri}.");
Assert.Equal(SampleUri, uri);
var eventData = new Dictionary<string, string> { ["Data0"] = " " + UriSystemGuard + " " };
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription);
Assert.Equal(UriSystemGuard, uri);
}
[Fact]
@@ -4,6 +4,7 @@ using System.Diagnostics.Eventing.Reader;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
@@ -15,31 +16,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> for
/// EventID 2800 — logged once per device-assigned MDM policy whose CSP URI matches the OS
/// reboot-required catalog — and forwards each as a <c>mdm_policy_reboot_required</c> event
/// carrying the reboot-forcing URI.
/// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog —
/// and emits ONE aggregated <c>mdm_policy_reboot_required</c> event per burst carrying the URI
/// list (<c>rebootUris</c>, <c>uriCount</c>, <c>firstRebootUri</c>).
/// <para>
/// This attributes the "unexpected reboot + second sign-in screen" pattern (PMPC research):
/// device-assigned policies applied during ESP DeviceSetup can demand a reboot; Windows
/// coalesces them into one restart at the end of DeviceSetup, forcing the user through a second
/// sign-in before AccountSetup. Admins are blind to this in the Intune console — the attributed
/// URIs make it actionable (reassign the profiles to user groups and the reboot disappears).
/// Verified against session b2e890c1 (2026-07-20): the 2800 description reads
/// <c>The following URI has triggered a reboot: (./Device/...).</c> and records arrive in
/// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record
/// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced
/// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond)
/// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL
/// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an
/// actually-observed <c>system_reboot_detected</c> before attributing anything.
/// </para>
/// <para>
/// <b>Backfill.</b> Device policies can apply before the agent starts, so a startup backfill
/// scans the lookback window. The .evtx persists across the coalesced reboot, so the
/// post-reboot agent restart re-reads the same records — which is why the
/// <b>cross-restart RecordId watermark</b> (WindowsUpdateTracker pattern) is mandatory here:
/// without it every reboot-forcing URI would be re-emitted after the very reboot it caused.
/// </para>
/// <para>
/// <b>Tolerant matching.</b> The exact 2800 message text is unverified (validated post-deploy on
/// the E2E VM): emission is gated on the EventID only, never on message wording. URI extraction
/// prefers structured EventData values that look like an OMA-DM URI (<c>./…</c>) and falls back
/// to a permissive regex over the formatted description. Events with an unparseable URI are
/// still emitted (with the captured description) so the timeline shows them and the captured
/// text tells us how to fix the extraction — the analyze rule keys on <c>rebootUri exists</c>
/// and simply stays silent for those.
/// <b>Backfill + watermark.</b> Device policies can apply before the agent starts, and a
/// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the
/// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical
/// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so
/// a pre-emit death loses nothing and a completed emit is never repeated.
/// </para>
/// </summary>
internal sealed class MdmRebootPolicyTracker : IDisposable
@@ -49,12 +44,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
internal const string WatermarkStateFileName = "mdm-reboot-watermark.json";
/// <summary>Cap for the emitted <c>rebootUris</c> list — <c>uriCount</c> always carries the uncapped distinct total.</summary>
internal const int MaxUrisPerEvent = 20;
/// <summary>Default quiet-period before a burst of 2800 records is flushed as one event.</summary>
internal const int DefaultDebounceMilliseconds = 10000;
/// <summary>Registry key whose <c>RebootRequired</c> value/subkey flags a pending OMA-DM coalesced reboot.</summary>
internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML";
// Permissive OMA-DM URI shape: "./Device/Vendor/MSFT/...", "./User/...", "./Vendor/...".
// Kept deliberately broad — the exact 2800 text is unverified; post-deploy E2E validates
// and tightens this against real records if needed.
// Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800
// text wraps the URI in parentheses — ')' is deliberately outside the class so the match
// stops cleanly; trailing sentence punctuation is trimmed after the match.
private static readonly Regex UriPattern = new Regex(
@"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled);
@@ -65,16 +66,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly int _debounceMilliseconds;
private EventLogWatcher _watcher;
private Timer _debounceTimer;
// Dedup — same contract as WindowsUpdateTracker (see its field docs): immutable
// cross-restart boundary from disk + intra-run HashSet (NOT a high-water mark, because the
// live watcher is armed before the backfill scan and can deliver higher RecordIds first).
private sealed class PendingRecord
{
public long RecordId;
public DateTime TimeUtc;
public string Uri; // null when extraction failed
public string Description; // may be null
public bool IsBackfill;
}
// Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as
// WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE
// deliberate difference: the watermark is persisted at FLUSH time, not at claim time —
// records buffered but never flushed (process killed by the very reboot they announce)
// must be re-read and emitted by the post-restart backfill.
private readonly List<PendingRecord> _pending = new List<PendingRecord>();
private long _restartWatermark = -1;
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
private long _maxEmittedRecordId = -1;
private readonly object _watermarkLock = new object();
private long _maxPersistedRecordId = -1;
private readonly object _stateLock = new object();
/// <summary>
/// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown
@@ -89,7 +104,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
AgentLogger logger,
bool backfillEnabled = true,
int backfillLookbackMinutes = 60,
string stateDirectory = null)
string stateDirectory = null,
int debounceMilliseconds = DefaultDebounceMilliseconds)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
@@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_debounceMilliseconds = debounceMilliseconds;
}
public void Start()
@@ -106,13 +123,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
{
BackfillRecentEvents();
// The backfill batch is complete — flush immediately instead of waiting out the
// debounce (live records that raced in are simply included).
FlushPending();
}
else
{
_logger.Info("MdmRebootPolicy backfill disabled by config");
}
}
public void Stop()
{
// Flush any un-emitted burst before tearing down (normal agent shutdown path).
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); }
var timer = Interlocked.Exchange(ref _debounceTimer, null);
if (timer != null)
{
try { timer.Dispose(); } catch { }
}
if (_watcher == null) return;
try
{
@@ -168,7 +202,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
$"*[System[(EventID={EventId_PolicyRebootRequired})]]";
// -----------------------------------------------------------------------
// Event processing
// Event processing — buffer + debounce
// -----------------------------------------------------------------------
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
@@ -212,9 +246,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
/// <summary>
/// Core processing extracted to primitive inputs so tests can drive it without synthesizing
/// an abstract, Windows-only <see cref="EventRecord"/>. Mirrors the WindowsUpdateTracker
/// test-seam pattern.
/// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the
/// WindowsUpdateTracker pattern). The aggregated event is emitted by <see cref="FlushPending"/>
/// — after the debounce quiet-period, at backfill completion, or at <see cref="Stop"/>.
/// </summary>
internal void ProcessEvent(
int eventId,
@@ -224,53 +258,155 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string formattedDescription,
bool isBackfill)
{
if (!MarkProcessed(recordId))
return; // already emitted (cross-restart or duplicate delivery)
lock (_stateLock)
{
if (recordId >= 0)
{
// Already emitted in a prior run, or already buffered/emitted this run.
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return;
}
_pending.Add(new PendingRecord
{
RecordId = recordId,
TimeUtc = timeCreatedUtc ?? DateTime.UtcNow,
Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri,
Description = formattedDescription,
IsBackfill = isBackfill,
});
}
ArmDebounce();
}
private void ArmDebounce()
{
if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly)
var timer = _debounceTimer;
if (timer == null)
{
var created = new Timer(_ =>
{
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); }
}, null, Timeout.Infinite, Timeout.Infinite);
timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created;
if (!ReferenceEquals(timer, created))
{
try { created.Dispose(); } catch { }
}
}
try { timer.Change(_debounceMilliseconds, Timeout.Infinite); }
catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ }
}
/// <summary>
/// Emits the pending burst as ONE aggregated <c>mdm_policy_reboot_required</c> event and
/// persists the watermark. No-op when nothing is pending. Internal for tests.
/// </summary>
internal void FlushPending()
{
List<PendingRecord> batch;
long watermarkToPersist = -1;
lock (_stateLock)
{
if (_pending.Count == 0) return;
batch = new List<PendingRecord>(_pending);
_pending.Clear();
foreach (var r in batch)
{
if (r.RecordId > _maxPersistedRecordId)
_maxPersistedRecordId = r.RecordId;
}
watermarkToPersist = _maxPersistedRecordId;
}
if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist);
batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc));
var distinctUris = batch
.Where(r => r.Uri != null)
.Select(r => r.Uri)
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri;
var unparsedCount = batch.Count(r => r.Uri == null);
var allBackfill = batch.All(r => r.IsBackfill);
var earliest = batch[0].TimeUtc;
var data = new Dictionary<string, object>
{
{ "windowsEventId", eventId },
{ "backfilled", isBackfill },
{ "windowsEventId", EventId_PolicyRebootRequired },
{ "eventLogChannel", Channel },
{ "recordCount", batch.Count },
{ "uriCount", distinctUris.Count },
{ "backfilled", allBackfill },
};
if (recordId >= 0) data["recordId"] = recordId;
if (timeCreatedUtc.HasValue) data["eventTime"] = timeCreatedUtc.Value.ToString("o");
if (!string.IsNullOrEmpty(rebootUri)) data["rebootUri"] = rebootUri;
if (!string.IsNullOrEmpty(formattedDescription))
if (distinctUris.Count > 0)
{
data["description"] = formattedDescription.Length > 1000
? formattedDescription.Substring(0, 1000) + "…"
: formattedDescription;
data["rebootUris"] = distinctUris
.OrderBy(u => u, StringComparer.OrdinalIgnoreCase)
.Take(MaxUrisPerEvent)
.ToList();
data["firstRebootUri"] = firstUri;
}
if (unparsedCount > 0)
{
data["unparsedCount"] = unparsedCount;
var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description;
if (sample != null)
data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample;
}
var flag = SafeProbeRebootRequiredFlag();
if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value;
string message;
if (distinctUris.Count == 1)
message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}";
else if (distinctUris.Count > 1)
message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)";
else
message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)";
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
// The record's own time, not UtcNow: backfilled pre-agent (and post-reboot re-read)
// records must land on the timeline at when the policy actually demanded the reboot.
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
// Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts
// must land on the timeline at when the policies were actually flagged.
Timestamp = earliest,
EventType = Constants.EventTypes.MdmPolicyRebootRequired,
Severity = EventSeverity.Warning,
// Neutral observation — reboot-required flags outside ESP DeviceSetup have no
// immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an
// actually-observed system_reboot_detected.
Severity = EventSeverity.Info,
Source = "MdmRebootPolicyWatcher",
Phase = EnrollmentPhase.Unknown,
Message = !string.IsNullOrEmpty(rebootUri)
? $"MDM policy requires a reboot during ESP: {rebootUri}"
: "MDM policy requires a reboot during ESP (URI not parsed from event — see description)",
Message = message,
Data = data,
// The reboot this event announces will kill the agent — flush now.
// Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup
// precedes a coalesced reboot that kills the agent.
ImmediateUpload = true,
});
_logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " +
$"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})");
}
/// <summary>
/// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData
/// value that starts with <c>./</c> (the CSP-URI shape), (2) a permissive regex over the
/// formatted description. Returns null when nothing URI-like is found — the caller still
/// emits, just without the <c>rebootUri</c> field.
/// value that starts with <c>./</c>, (2) a permissive regex over the formatted description
/// (verified shape: <c>The following URI has triggered a reboot: (./Device/...).</c>).
/// Returns null when nothing URI-like is found — the record still counts into the
/// aggregate (<c>unparsedCount</c> + <c>sampleDescription</c>).
/// </summary>
internal static string ExtractRebootUri(Dictionary<string, string> eventData, string description)
{
@@ -289,8 +425,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
var match = UriPattern.Match(description);
if (match.Success)
{
// The permissive pattern can swallow sentence punctuation trailing the URI
// ("...Security."). OMA-DM URI segments never end in punctuation — trim it.
// The permissive pattern can swallow sentence punctuation trailing the URI.
// OMA-DM URI segments never end in punctuation — trim it.
var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')');
if (candidate.Length > 2) return candidate;
}
@@ -349,8 +485,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
/// <summary>
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. The exact shape is
/// unverified (PMPC research names the key, not the value type), so both a value and a
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. Both a value and a
/// subkey named <c>RebootRequired</c> count as "pending". Fail-soft: null = unknown
/// (key absent / unreadable), matching the EspSkipConfigurationProbe convention.
/// </summary>
@@ -424,44 +559,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
// -----------------------------------------------------------------------
// Watermark dedup (cross-restart) — WindowsUpdateTracker pattern verbatim
// Watermark dedup (cross-restart) — persisted at flush, not at claim
// -----------------------------------------------------------------------
private bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_watermarkLock)
lock (_stateLock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
/// <summary>
/// Atomically claims <paramref name="recordId"/>. Returns true if this is the first time we
/// see it (caller should emit), false if it was already processed — either in a PRIOR run
/// (at/below the restart watermark) or already this run (in the seen-set). Records without
/// a RecordId (-1) are always emitted, never tracked or persisted.
/// </summary>
private bool MarkProcessed(long recordId)
{
if (recordId < 0) return true;
long toPersist = -1;
lock (_watermarkLock)
{
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return false;
if (recordId > _maxEmittedRecordId)
{
_maxEmittedRecordId = recordId;
toPersist = _maxEmittedRecordId;
}
}
if (toPersist >= 0) PersistWatermark(toPersist);
return true;
}
internal void LoadWatermark()
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
@@ -475,10 +584,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
if (state != null)
{
lock (_watermarkLock)
lock (_stateLock)
{
_restartWatermark = state.LastRecordId;
_maxEmittedRecordId = state.LastRecordId;
_maxPersistedRecordId = state.LastRecordId;
}
_logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
@@ -16,9 +16,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// CommercialOOBE_ESPProgress_Page_Exiting — normal ESP exit
/// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete
/// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure
/// "RebootCoalescing" (DeviceSetup) — ESP-initiated coalesced reboot
/// (MDM policies forced a mid-ESP restart → second sign-in; corroborates the
/// MdmRebootPolicyTracker's per-URI 2800 attribution events)
///
/// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was
/// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker
/// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves
/// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in
/// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot.
///
/// Raises <see cref="FinalizingSetupPhaseTriggered"/>, <see cref="WhiteGloveCompleted"/>,
/// and <see cref="EspFailureDetected"/>. Cross-notifies the <see cref="HelloTracker"/>
@@ -50,7 +53,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private bool _espExitDetected;
private bool _whiteGloveDetected;
private bool _helloWizardStartDetected;
private bool _rebootCoalescingDetected;
private readonly object _stateLock = new object();
/// <summary>
@@ -251,27 +253,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_logger.Info("ESP phase exit detected - detected via Shell-Core event 62407");
}
else if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0)
{
// ESP-initiated coalesced reboot (end of DeviceSetup): MDM policies forced a
// restart, the user will see a second sign-in screen. Checked AFTER the
// WhiteGlove/failure/exiting patterns so their semantics stay untouched even
// if a description ever carried both tokens. Fire-once per process run — the
// reboot restarts the agent, and the fresh run re-observes it via backfill.
lock (_stateLock)
{
if (_rebootCoalescingDetected) return;
_rebootCoalescingDetected = true;
}
eventType = Constants.EventTypes.EspRebootCoalescing;
severity = EventSeverity.Warning;
message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in";
// No FinalizingSetup transition, no C# event raise — pure informational
// corroboration for the MdmRebootPolicyTracker's per-URI attribution.
_logger.Warning("ESP coalesced reboot detected via Shell-Core event 62407 (RebootCoalescing)");
}
else
{
return;
@@ -446,29 +427,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
return;
}
if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0)
{
// DELIBERATE deviation from the other 62407 backfill branches (which only raise C#
// events): this is a pure informational type with no reducer/subscriber — the
// emission IS the signal, and the live emit races the very reboot that kills the
// process. Re-emitting after the restart is the only reliable delivery path. A
// live+backfill cross-process duplicate is possible (no watermark here); accepted:
// the analyze rule uses exists-semantics, both copies carry the identical historical
// timestamp, and the backfill copy is marked backfilled:true.
bool shouldEmit;
lock (_stateLock)
{
shouldEmit = !_rebootCoalescingDetected;
_rebootCoalescingDetected = true;
}
if (shouldEmit)
{
_logger.Info($"Backfill: ESP coalesced reboot (RebootCoalescing) found in recent Shell-Core logs (originalAt={occurredAtUtc:o})");
EmitRebootCoalescing(description, occurredAtUtc, isBackfill: true);
}
return; // a RebootCoalescing record is not an ESP exit/failure — stop here
}
if (EspExitingPattern.IsMatch(description))
{
bool shouldNotify = false;
@@ -507,39 +465,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
}
/// <summary>
/// Emits the <c>esp_reboot_coalescing</c> event with the standard Shell-Core eventData
/// shape. Used by the backfill path (the live path flows through the shared emit tail in
/// <see cref="ProcessEvent"/>); <paramref name="isBackfill"/> adds the
/// <c>backfilled</c> marker that distinguishes an accepted post-restart re-emission.
/// </summary>
private void EmitRebootCoalescing(string description, DateTime occurredAtUtc, bool isBackfill)
{
var eventData = new Dictionary<string, object>
{
{ "windowsEventId", EventId_ShellCore_WebAppEvent },
{ "providerName", "" },
{ "description", description },
{ "eventLogChannel", ShellCoreEventLogChannel },
{ "eventTime", occurredAtUtc.ToString("o") },
{ "backfilled", isBackfill }
};
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
Timestamp = occurredAtUtc,
EventType = Constants.EventTypes.EspRebootCoalescing,
Severity = EventSeverity.Warning,
Source = "EspAndHelloTracker",
Phase = EnrollmentPhase.Unknown,
Message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in",
Data = eventData,
ImmediateUpload = true
});
}
// =====================================================================
// Pattern helpers
// =====================================================================
@@ -24,7 +24,6 @@ export const EVENT_TYPES_CATALOG = {
'esp_failure_settle_started',
'esp_failure_settle_recovered',
'esp_exiting',
'esp_reboot_coalescing',
'mdm_policy_reboot_required',
'esp_config_detected',
'completion_check',
+11 -11
View File
@@ -405,17 +405,17 @@ namespace AutopilotMonitor.Shared
public const string OsBuildChanged = "os_build_changed"; // OS build (CurrentBuild.UBR) differs across an agent restart — an update was installed during enrollment
public const string WindowsUpdateChannelCensus = "windows_update_channel_census"; // One-shot EventID histogram of WU Client + UpdateOrchestrator channels when os_build_changed but zero targeted WU events captured
// MDM-policy-forced coalesced reboot during ESP (MdmRebootPolicyTracker + ShellCoreTracker;
// PMPC "unexpected reboot / second sign-in" research). Device-assigned policies whose CSP
// URIs match the OS reboot-required catalog force a coalesced reboot at the end of
// DeviceSetup — the user sees an unexpected restart and a second sign-in screen before
// AccountSetup. 2800 (DeviceManagement-Enterprise-Diagnostics-Provider/Admin) fires once
// per reboot-forcing URI; Shell-Core 62407 "RebootCoalescing" records ESP initiating the
// reboot. Deliberately DISTINCT from system_reboot_detected: the backend RebootCount
// matches that string exactly (EventIngestProcessor.Classification + terminal recount) —
// these attribution events must never inflate the reboot count.
public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // DM-Enterprise EventID 2800 — Data.rebootUri = CSP URI that demands the reboot (one event per URI)
public const string EspRebootCoalescing = "esp_reboot_coalescing"; // Shell-Core 62407 "RebootCoalescing" — ESP-initiated coalesced reboot imminent (second sign-in follows)
// MDM reboot-required policy flags (MdmRebootPolicyTracker; PMPC "unexpected reboot /
// second sign-in" research). DM-Enterprise EventID 2800 fires once per policy URI that
// matches the OS reboot-required catalog ("The following URI has triggered a reboot:
// (./Device/...)."; verified session b2e890c1). Only URIs applied DURING ESP DeviceSetup
// cause the coalesced mid-ESP reboot — later syncs (AccountSetup+) merely flag, no forced
// reboot. The tracker therefore emits ONE aggregated Info event per burst with the URI
// list; the reboot CLAIM lives in ANALYZE-ESP-005, gated on an actually-observed
// system_reboot_detected. Deliberately DISTINCT from system_reboot_detected: the backend
// RebootCount matches that string exactly (EventIngestProcessor.Classification + terminal
// recount) — this attribution event must never inflate the reboot count.
public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // Aggregated per burst — Data.rebootUris[] + uriCount + firstRebootUri
// TEMPORARY: shadow SM rollout verification — remove when CompletionStateMachine is promoted to primary
public const string ShadowDiscrepancy = "shadow_discrepancy";