diff --git a/docs/agent/index.md b/docs/agent/index.md index 984f0320..b744a477 100644 --- a/docs/agent/index.md +++ b/docs/agent/index.md @@ -7,6 +7,6 @@ How the on-device agent works: * [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine. * [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read. * [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map. -* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory. +* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot. Reading order for newcomers: overview → decision engine → persistence. diff --git a/docs/agent/mdm-reboot-coalescing.md b/docs/agent/mdm-reboot-coalescing.md index 3faca9e5..210d8eea 100644 --- a/docs/agent/mdm-reboot-coalescing.md +++ b/docs/agent/mdm-reboot-coalescing.md @@ -1,7 +1,7 @@ --- type: Concept title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs -description: How the agent detects device-assigned MDM policies that force a coalesced reboot during ESP DeviceSetup (DM-Enterprise EventID 2800 + Shell-Core 62407 RebootCoalescing), and the ANALYZE-ESP-005 advisory that makes the invisible "second sign-in" fixable. +description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed. resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals tags: - agent @@ -17,91 +17,82 @@ timestamp: 2026-07-20T00:00:00+02:00 Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog (`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a -**second sign-in screen** before AccountSetup. This is expected OS behavior, invisible in -the Intune console; most admins simply live with it, not knowing that reassigning the -profiles to user groups removes the reboot entirely. Source research: Patch My PC, +**second sign-in screen** before AccountSetup. Invisible in the Intune console; most +admins live with it, not knowing that reassigning the profiles to user groups removes the +reboot entirely. Source research: Patch My PC, ["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/). The session already counts reboots (`system_reboot_detected` → `session.rebootCount`); this feature adds the **cause attribution**. -# Signal chain +# Verified signal facts (session b2e890c1, 2026-07-20) -1. **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin` - — logged by the SyncML engine once per policy URI that matched the reboot-required - catalog, during device-policy application in DeviceSetup. -2. **EventID 62407**, `Microsoft-Windows-Shell-Core/Operational`, description containing - `RebootCoalescing` — ESP/CloudExperienceHost initiating the coalesced reboot at the end - of DeviceSetup. -3. The reboot kills the agent; the post-reboot restart re-observes both channels via - backfill (the .evtx persists across the reboot). +* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`, + one record per URI, verified description shape: + `The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).` + Records arrive in sub-second bursts during policy sync. +* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs + applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the + verified session the flags fired during AccountSetup (after DeviceSetup completed) — + **no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that + claims "reboot incoming" on every 2800 is therefore wrong. +* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the + ROUTINE bootstrap marker on every enrollment: + `Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'` + — the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a + reboot is needed). A substring match on it is a false positive by construction. If a + real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be + added then. -# Detectors +# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`) -## `MdmRebootPolicyTracker` (new collector, `MdmRebootPolicyWatcherHost`) +Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup +backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then +flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst: -Structural clone of `WindowsUpdateTracker`: live `EventLogWatcher` on the DM-Enterprise -Admin channel (EventID 2800) + startup backfill (default 60 min lookback) + **cross-restart -RecordId watermark** (`mdm-reboot-watermark.json`). The watermark is mandatory here: the -coalesced reboot restarts the agent, and without it every reboot-forcing URI would be -re-emitted after the very reboot it caused. +* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as + reboot-required: … (+N more)") — the raw event never claims a reboot. +* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct), + `firstRebootUri` (record-time order, drives rule interpolation), `recordCount`, + `unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill), + fail-soft `omadmRebootRequiredFlag` probe of + `HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown). +* `Timestamp` = earliest record time (backfilled bursts land at their historical instant); + `ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a + reboot that kills the agent). +* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the + coalesced reboot kills the process before the debounced emit, nothing is marked — the + post-restart backfill re-reads and emits the burst. A flushed burst is never repeated. -Per 2800 record it emits one `mdm_policy_reboot_required` event (Warning, -`ImmediateUpload=true` — the announced reboot will kill the agent, flush now) with -`Data.rebootUri` (the CSP URI), `recordId`, `backfilled`, `description` (truncated 1000), -and a fail-soft `omadmRebootRequiredFlag` corroboration probe of -`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown, never -blocks the emit). Per-URI emission (not aggregated) is deliberate: an aggregate would need -a flush point that races the reboot kill. +URI extraction: EventData values starting with `./` preferred, permissive regex over the +description as fallback (trailing punctuation trimmed — the verified text wraps the URI in +parens and ends with a period). Unparsed records still count into the aggregate. -**Tolerant matching**: the exact 2800 message text is unverified — emission is gated on -the EventID only. URI extraction prefers structured EventData values starting with `./` -and falls back to a permissive regex over the description (trailing punctuation trimmed). -Events with an unparseable URI are still emitted (timeline-visible, rule stays silent, and -the captured description tells us how to fix the extraction). Exact texts are validated -post-deploy on the E2E VM and recorded here. - -Config (internal steering only, no admin UI — this is an agent-native collector, NOT an +Config (internal steering only, no admin UI — agent-native collector, NOT an `IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true), `Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35. -## `ShellCoreTracker` — `RebootCoalescing` branch - -The already-watched 62407 stream gets a fourth branch (checked AFTER -WhiteGlove/failure/exiting so their semantics stay untouched): description containing -`RebootCoalescing` → one `esp_reboot_coalescing` event (Warning, ImmediateUpload), pure -informational — no `FinalizingSetup` transition, no C# event raise, no DecisionCore -involvement. - -**Deliberate deviation**: unlike the other 62407 branches, the backfill path DOES emit -this event. The live emit races the very reboot that kills the process; the post-restart -5-min backfill re-emission is the only reliable delivery. A live+backfill cross-process -duplicate is possible and accepted (fire-once bool resets per process; both copies carry -the identical historical timestamp; the backfill copy is marked `backfilled:true`; the -analyze rule uses exists-semantics). The shared 5-min lookback is NOT enlarged — the -rule's required condition is the watermarked 2800 event; coalescing is optional -corroboration only. - # Advisory — ANALYZE-ESP-005 -"MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)", severity `warning`, -`markSessionAsFailedDefault: false`. Required: `mdm_policy_reboot_required.rebootUri` -exists (first URI interpolates as `{{rebootUri}}`). Optional corroboration: -`esp_reboot_coalescing` observed (+20 confidence) and ≥2 reboot URIs (+10) on base 70 / -threshold 70. Remediation: map each URI to its Intune profile and reassign device → user -groups (or exclude the Autopilot device group), or accept + communicate the restart. -Because `{{token}}` interpolation is scalar-only, the explanation points at the timeline -for the complete URI list — each URI is its own event. +The reboot **claim** lives here, and only here. Required conditions (BOTH): + +1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation) +2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot** + +Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional +confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 / +threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or +exclude the Autopilot device group), or accept + communicate the restart. # Non-goals / invariants -* The new event types are **deliberately distinct from `system_reboot_detected`** — the - backend RebootCount matches that string exactly in two places - (`EventIngestProcessor.Classification` + the terminal recount); attribution events must - never inflate the reboot count. +* Event type **deliberately distinct from `system_reboot_detected`** — the backend + RebootCount matches that string exactly in two places + (`EventIngestProcessor.Classification` + terminal recount); attribution must never + inflate the reboot count. * No session-level field / SessionInfoCard change in this iteration (follow-up candidate). -* The `RebootRequiredURIs` catalog itself is not read (format unverified); only the 2800 - events (which carry the actually-triggered URIs) are consumed. +* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry + the actually-triggered URIs) are consumed. # Citations diff --git a/docs/index.md b/docs/index.md index cadebde6..49178c41 100644 --- a/docs/index.md +++ b/docs/index.md @@ -18,7 +18,7 @@ https://docs.autopilotmonitor.com (separate repository). * [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map). * [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore). * [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins. -* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: DM-Enterprise 2800 watcher with cross-restart watermark, Shell-Core RebootCoalescing corroboration, ANALYZE-ESP-005 advisory. +* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot. # Rules diff --git a/docs/log.md b/docs/log.md index 574d4095..f07a33dc 100644 --- a/docs/log.md +++ b/docs/log.md @@ -2,6 +2,7 @@ ## 2026-07-20 +* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded. * **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated. * **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule. diff --git a/rules/analyze/ANALYZE-ESP-005.json b/rules/analyze/ANALYZE-ESP-005.json index 4cb16873..87f93a26 100644 --- a/rules/analyze/ANALYZE-ESP-005.json +++ b/rules/analyze/ANALYZE-ESP-005.json @@ -2,10 +2,10 @@ "$schema": "../schema/analyze-rule.schema.json", "ruleId": "ANALYZE-ESP-005", "title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)", - "description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.", + "description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.", "severity": "warning", "category": "esp", - "version": "1.0.0", + "version": "1.1.0", "author": "Autopilot Monitor", "enabled": true, "trigger": "single", @@ -15,37 +15,48 @@ "signal": "reboot_forcing_policy_uri", "source": "event_data", "eventType": "mdm_policy_reboot_required", - "dataField": "rebootUri", + "dataField": "firstRebootUri", "operator": "exists", "required": true }, { - "signal": "reboot_uri_count", + "signal": "reboot_observed", "source": "event_count", - "eventType": "mdm_policy_reboot_required", + "eventType": "system_reboot_detected", "operator": "count_gte", "value": "1", + "required": true + }, + { + "signal": "multiple_reboot_uris", + "source": "event_data", + "eventType": "mdm_policy_reboot_required", + "dataField": "uriCount", + "operator": "gte", + "value": "2", "required": false }, { - "signal": "esp_coalesced_reboot_observed", - "source": "event_type", - "eventType": "esp_reboot_coalescing", - "operator": "exists", + "signal": "omadm_reboot_flag_set", + "source": "event_data", + "eventType": "mdm_policy_reboot_required", + "dataField": "omadmRebootRequiredFlag", + "operator": "equals", + "value": "true", "required": false } ], "confidenceFactors": [ - { "signal": "esp_coalesced_reboot_observed", "condition": "exists", "weight": 20 }, - { "signal": "mdm_policy_reboot_required", "condition": "count >= 2", "weight": 10 } + { "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 }, + { "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 } ], "confidenceThreshold": 70, - "explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.", + "explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.", "remediation": [ { "title": "Reassign the reboot-forcing policies to user groups", "steps": [ - "Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)", + "Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)", "Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area", "Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup", "Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment" diff --git a/rules/dist/analyze-rules.json b/rules/dist/analyze-rules.json index f4143bf0..feea87cd 100644 --- a/rules/dist/analyze-rules.json +++ b/rules/dist/analyze-rules.json @@ -1788,10 +1788,10 @@ { "ruleId": "ANALYZE-ESP-005", "title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)", - "description": "Detects the 'unexpected reboot + second sign-in screen' pattern: one or more device-assigned MDM policies matched the OS reboot-required catalog during ESP DeviceSetup, and Windows coalesced them into a restart at the end of DeviceSetup — forcing the user through a second sign-in before AccountSetup. Attributes the reboot to the specific CSP policy URIs so the profiles can be reassigned.", + "description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.", "severity": "warning", "category": "esp", - "version": "1.0.0", + "version": "1.1.0", "author": "Autopilot Monitor", "enabled": true, "trigger": "single", @@ -1801,45 +1801,56 @@ "signal": "reboot_forcing_policy_uri", "source": "event_data", "eventType": "mdm_policy_reboot_required", - "dataField": "rebootUri", + "dataField": "firstRebootUri", "operator": "exists", "required": true }, { - "signal": "reboot_uri_count", + "signal": "reboot_observed", "source": "event_count", - "eventType": "mdm_policy_reboot_required", + "eventType": "system_reboot_detected", "operator": "count_gte", "value": "1", + "required": true + }, + { + "signal": "multiple_reboot_uris", + "source": "event_data", + "eventType": "mdm_policy_reboot_required", + "dataField": "uriCount", + "operator": "gte", + "value": "2", "required": false }, { - "signal": "esp_coalesced_reboot_observed", - "source": "event_type", - "eventType": "esp_reboot_coalescing", - "operator": "exists", + "signal": "omadm_reboot_flag_set", + "source": "event_data", + "eventType": "mdm_policy_reboot_required", + "dataField": "omadmRebootRequiredFlag", + "operator": "equals", + "value": "true", "required": false } ], "confidenceFactors": [ { - "signal": "esp_coalesced_reboot_observed", + "signal": "multiple_reboot_uris", "condition": "exists", - "weight": 20 + "weight": 10 }, { - "signal": "mdm_policy_reboot_required", - "condition": "count >= 2", - "weight": 10 + "signal": "omadm_reboot_flag_set", + "condition": "exists", + "weight": 15 } ], "confidenceThreshold": 70, - "explanation": "During ESP DeviceSetup, a device-assigned MDM policy matched the OS reboot-required catalog and Windows scheduled a **coalesced reboot** at the end of DeviceSetup — the user saw an unexpected restart and a **second sign-in screen** before AccountSetup continued.\n\nFirst reboot-forcing policy URI: `{{rebootUri}}`\n\nWhen several policies force a reboot they are coalesced into one restart; **each** reboot-forcing policy appears as its own `mdm_policy_reboot_required` event on the session timeline — check the timeline for the complete URI list.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.", + "explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.", "remediation": [ { "title": "Reassign the reboot-forcing policies to user groups", "steps": [ - "Open each `mdm_policy_reboot_required` timeline event and note the CSP URI (e.g. `./Device/Vendor/MSFT/...`)", + "Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)", "Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area", "Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup", "Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment" diff --git a/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/ShellCoreTrackerRebootCoalescingTests.cs b/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/ShellCoreTrackerRebootCoalescingTests.cs deleted file mode 100644 index 033c046e..00000000 --- a/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/ShellCoreTrackerRebootCoalescingTests.cs +++ /dev/null @@ -1,195 +0,0 @@ -#nullable enable -using System; -using System.Collections.Generic; -using System.Linq; -using AutopilotMonitor.Agent.V2.Core.Logging; -using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals; -using AutopilotMonitor.Agent.V2.Core.Orchestration; -using AutopilotMonitor.Agent.V2.Core.Tests.Harness; -using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration; -using AutopilotMonitor.DecisionCore.Engine; -using AutopilotMonitor.DecisionCore.Signals; -using AutopilotMonitor.Shared; -using Xunit; - -namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring -{ - /// - /// Shell-Core 62407 "RebootCoalescing" branch — ESP-initiated coalesced reboot (MDM policies - /// forced a mid-ESP restart → second sign-in). Pure informational corroboration for the - /// MdmRebootPolicyTracker's per-URI 2800 attribution: no FinalizingSetup transition, no C# - /// event raise, and — as a DELIBERATE deviation from the other 62407 backfill branches — the - /// backfill path DOES emit (the live emit races the very reboot that kills the process). - /// - public sealed class ShellCoreTrackerRebootCoalescingTests - { - private static readonly DateTime ClockNow = new DateTime(2026, 7, 20, 10, 5, 0, DateTimeKind.Utc); - private const string CoalescingDescription = - "CloudExperienceHost Web App Event 2. Name: 'CommercialOOBE_DeviceSetup_RebootCoalescing', Value: '...'."; - - private static (ShellCoreTracker tracker, FakeSignalIngressSink sink) MakeTracker(TempDirectory tmp) - { - var sink = new FakeSignalIngressSink(); - var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info); - var tracker = new ShellCoreTracker( - sessionId: "S1", - tenantId: "T1", - post: new InformationalEventPost(sink, new VirtualClock(ClockNow)), - logger: logger, - helloTracker: null); - return (tracker, sink); - } - - private static IReadOnlyList ByType(FakeSignalIngressSink sink, string eventType) => - sink.Posted.Where(p => - p.Kind == DecisionSignalKind.InformationalEvent - && p.Payload != null - && p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et) - && et == eventType).ToList(); - - private static IReadOnlyDictionary Data(FakeSignalIngressSink.PostedSignal s) => - (IReadOnlyDictionary)s.TypedPayload!; - - [Fact] - public void ProcessEvent_62407_RebootCoalescing_EmitsWarning_ImmediateUpload() - { - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - var sourceTime = ClockNow.AddSeconds(-5); - tracker.ProcessEvent( - eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent, - description: CoalescingDescription, - timestamp: sourceTime, - providerName: "Microsoft-Windows-Shell-Core", - isBackfill: false); - - var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing)); - Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]); - Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]); - Assert.Equal(sourceTime, s.OccurredAtUtc); - Assert.Equal(CoalescingDescription, Data(s)["description"]); - } - } - - [Fact] - public void ProcessEvent_62407_RebootCoalescing_FiresOncePerRun() - { - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false); - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - CoalescingDescription, ClockNow.AddSeconds(1), "Microsoft-Windows-Shell-Core", isBackfill: false); - - Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing)); - } - } - - [Fact] - public void ProcessEvent_62407_RebootCoalescing_DoesNotTriggerEspSemantics() - { - // The coalescing record is corroboration only — it must not trip the ESP-exit / - // FinalizingSetup / failure machinery. - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - var raised = new List(); - tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}"); - tracker.EspExited += (_, _) => raised.Add("espExited"); - tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}"); - tracker.WhiteGloveCompleted += (_, _) => raised.Add("whiteGlove"); - - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - CoalescingDescription, ClockNow, "Microsoft-Windows-Shell-Core", isBackfill: false); - - Assert.Empty(raised); - Assert.False(tracker.IsEspExitedForTest); - Assert.Empty(ByType(sink, Constants.EventTypes.EspExiting)); - Assert.Empty(ByType(sink, Constants.EventTypes.EspFailure)); - } - } - - [Fact] - public void ProcessEvent_62407_ExistingBranches_Unaffected() - { - // Ordering regression guard: WhiteGlove/failure/exiting still classify as before — - // the RebootCoalescing check comes AFTER them. - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - "Name: 'CommercialOOBE_ESPProgress_Page_Exiting'", ClockNow, "p", isBackfill: false); - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - "Name: 'CommercialOOBE_ESPProgress_WhiteGlove_Success'", ClockNow, "p", isBackfill: false); - - Assert.Single(ByType(sink, Constants.EventTypes.EspExiting)); - Assert.Single(ByType(sink, Constants.EventTypes.WhiteGloveComplete)); - Assert.Empty(ByType(sink, Constants.EventTypes.EspRebootCoalescing)); - } - } - - [Fact] - public void Backfill_RebootCoalescing_Emits_WithBackfilledTrue_AndHistoricalTimestamp() - { - // Deliberate deviation from the other 62407 backfill branches: after the coalesced - // reboot restarts the agent, the backfill re-emission is the only reliable delivery. - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - var historical = ClockNow.AddMinutes(-3); - tracker.HandleBackfillRecord( - ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, historical); - - var s = Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing)); - Assert.Equal(historical, s.OccurredAtUtc); - Assert.Equal(true, Data(s)["backfilled"]); - Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]); - } - } - - [Fact] - public void Backfill_RebootCoalescing_AfterLiveFire_DoesNotDoubleEmit_SameProcess() - { - // Same-process dedup via the fire-once flag. (Cross-process live+backfill duplication - // is possible and accepted — exists-semantics rule, identical historical timestamp.) - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, - CoalescingDescription, ClockNow, "p", isBackfill: false); - tracker.HandleBackfillRecord( - ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-1)); - - Assert.Single(ByType(sink, Constants.EventTypes.EspRebootCoalescing)); - } - } - - [Fact] - public void Backfill_RebootCoalescing_DoesNotFallThroughToExitOrFailureHandling() - { - using var tmp = new TempDirectory(); - var (tracker, sink) = MakeTracker(tmp); - using (tracker) - { - var raised = new List(); - tracker.FinalizingSetupPhaseTriggered += (_, reason) => raised.Add($"finalizing:{reason}"); - tracker.EspExited += (_, _) => raised.Add("espExited"); - tracker.EspFailureDetected += (_, type) => raised.Add($"failure:{type}"); - - tracker.HandleBackfillRecord( - ShellCoreTracker.EventId_ShellCore_WebAppEvent, CoalescingDescription, ClockNow.AddMinutes(-2)); - - Assert.Empty(raised); - Assert.False(tracker.IsEspExitedForTest); - } - } - } -} diff --git a/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/SystemSignals/MdmRebootPolicyTrackerTests.cs b/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/SystemSignals/MdmRebootPolicyTrackerTests.cs index a67394d6..947115f1 100644 --- a/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/SystemSignals/MdmRebootPolicyTrackerTests.cs +++ b/src/Agent/AutopilotMonitor.Agent.V2.Core.Tests/Monitoring/SystemSignals/MdmRebootPolicyTrackerTests.cs @@ -15,16 +15,22 @@ using Xunit; namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals { /// - /// MdmRebootPolicyTracker — attributes the "unexpected reboot + second sign-in" pattern to the - /// device-assigned policy URIs that forced the coalesced reboot (DM-Enterprise EventID 2800). - /// Drives the primitive test seam (no real - /// EventRecord, which is abstract + Windows-only), mirroring the WindowsUpdateTracker pattern. + /// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral + /// mdm_policy_reboot_required event (session b2e890c1 rework: per-record events were + /// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an + /// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit + /// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed). /// The registry probe is always overridden — tests never touch the live registry. /// public sealed class MdmRebootPolicyTrackerTests : IDisposable { - private static readonly DateTime At = new DateTime(2026, 7, 20, 9, 30, 0, DateTimeKind.Utc); - private const string SampleUri = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/EnableVirtualizationBasedSecurity"; + private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc); + private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation"; + private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch"; + + // Verified real 2800 description shape (session b2e890c1, 2026-07-20). + private const string RealDescription = + "The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation)."; private readonly TempDirectory _tmp = new TempDirectory(); private readonly FakeSignalIngressSink _sink; @@ -34,21 +40,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals public MdmRebootPolicyTrackerTests() { _sink = new FakeSignalIngressSink(); + _tracker = MakeTracker(stateDirectory: null); + } + + public void Dispose() => _tmp.Dispose(); + + private MdmRebootPolicyTracker MakeTracker(string? stateDirectory) + { var post = new InformationalEventPost(_sink, new VirtualClock(At)); var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info); - _tracker = new MdmRebootPolicyTracker( + var tracker = new MdmRebootPolicyTracker( sessionId: "sess-mdm", tenantId: "tenant-mdm", post: post, logger: logger, backfillEnabled: false, - stateDirectory: null); // in-memory dedup for most tests - _tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default + stateDirectory: stateDirectory, + debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly + tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default + return tracker; } - public void Dispose() => _tmp.Dispose(); - - private void Emit(string? rebootUri = SampleUri, string? description = null, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null) + private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null) { _tracker.ProcessEvent( eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired, @@ -70,104 +83,192 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals (IReadOnlyDictionary)s.TypedPayload!; [Fact] - public void ProcessEvent_EmitsWarning_WithUri_AndImmediateUpload() + public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload() { - Emit(); + Buffer(UriSvchost, timeCreatedUtc: At); + Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120)); + _tracker.FlushPending(); var s = Assert.Single(Emitted()); - Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]); + Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]); Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]); var data = Data(s); - Assert.Equal(SampleUri, data["rebootUri"]); - Assert.Equal(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, data["windowsEventId"]); - Assert.Equal(MdmRebootPolicyTracker.Channel, data["eventLogChannel"]); + Assert.Equal(2, data["uriCount"]); + Assert.Equal(2, data["recordCount"]); + Assert.Equal(UriSvchost, data["firstRebootUri"]); + var uris = Assert.IsType>(data["rebootUris"]); + Assert.Equal(2, uris.Count); + Assert.Contains(UriSvchost, uris); + Assert.Contains(UriSystemGuard, uris); } [Fact] - public void ProcessEvent_NoUri_StillEmits_WithoutRebootUriField() + public void Message_IsNeutralObservation_NoRebootClaim() { - // Tolerant contract: an unparsed URI must stay timeline-visible (with the captured - // description telling us how to fix the extraction) — the analyze rule keys on - // "rebootUri exists" and simply stays silent. - Emit(rebootUri: null, description: "Some unexpected 2800 wording"); + // Session b2e890c1: flags raised during AccountSetup have no effect — the raw event + // must state the observation only, never a future consequence. + Buffer(UriSvchost); + _tracker.FlushPending(); + + var s = Assert.Single(Emitted()); + var message = s.Payload![SignalPayloadKeys.Message]; + Assert.Contains("flagged", message); + Assert.DoesNotContain("will restart", message); + Assert.DoesNotContain("second sign-in", message); + } + + [Fact] + public void FlushPending_Empty_EmitsNothing() + { + _tracker.FlushPending(); + Assert.Empty(Emitted()); + } + + [Fact] + public void AggregateTimestamp_IsEarliestRecordTime() + { + var early = At.AddMinutes(-30); + Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true); + Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true); + _tracker.FlushPending(); + + var s = Assert.Single(Emitted()); + Assert.Equal(early, s.OccurredAtUtc); + Assert.Equal(true, Data(s)["backfilled"]); + // firstRebootUri follows record-time order, not buffer order. + Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]); + } + + [Fact] + public void Backfilled_False_WhenAnyRecordIsLive() + { + Buffer(UriSvchost, isBackfill: true); + Buffer(UriSystemGuard, isBackfill: false); + _tracker.FlushPending(); + + Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]); + } + + [Fact] + public void DuplicateUris_AreDistinctInAggregate() + { + Buffer(UriSvchost); + Buffer(UriSvchost); + _tracker.FlushPending(); + + var data = Data(Assert.Single(Emitted())); + Assert.Equal(1, data["uriCount"]); + Assert.Equal(2, data["recordCount"]); + } + + [Fact] + public void UnparsedRecords_CountedWithSampleDescription_StillEmitted() + { + Buffer(rebootUri: null, description: "Some unexpected 2800 wording"); + _tracker.FlushPending(); var s = Assert.Single(Emitted()); var data = Data(s); - Assert.False(data.ContainsKey("rebootUri")); - Assert.Equal("Some unexpected 2800 wording", data["description"]); + Assert.False(data.ContainsKey("rebootUris")); + Assert.False(data.ContainsKey("firstRebootUri")); + Assert.Equal(0, data["uriCount"]); + Assert.Equal(1, data["unparsedCount"]); + Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]); } [Fact] - public void BackfilledEvent_UsesEventTimeForTimelineTimestamp_AndMarksBackfilled() + public void SameRecordId_BufferedTwice_CountsOnce() { - var eventTime = At.AddMinutes(-25); // policy applied well before agent start - Emit(isBackfill: true, timeCreatedUtc: eventTime); + Buffer(UriSvchost, recordId: 42); + Buffer(UriSvchost, recordId: 42); + _tracker.FlushPending(); - var s = Assert.Single(Emitted()); - Assert.Equal(eventTime, s.OccurredAtUtc); - Assert.Equal(true, Data(s)["backfilled"]); + Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]); } [Fact] - public void SameRecordId_ProcessedTwice_EmitsOnce() + public void RecordAfterFlush_GoesIntoNextAggregate() { - Emit(recordId: 42); - Emit(recordId: 42); - - Assert.Single(Emitted()); - } - - [Fact] - public void BackfillEventWithLowerRecordId_AfterLiveEvent_IsStillEmitted() - { - // Same contract as WindowsUpdateTracker: the live watcher is armed BEFORE the backfill - // scan, so a live record with a higher RecordId can arrive first — the older, - // never-emitted backfill record must not be suppressed by a high-water mark. - Emit(recordId: 100); - Emit(recordId: 50, isBackfill: true); + Buffer(UriSvchost, recordId: 10); + _tracker.FlushPending(); + Buffer(UriSystemGuard, recordId: 11); + _tracker.FlushPending(); Assert.Equal(2, Emitted().Count); } + [Fact] + public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered() + { + // Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill + // record must not be suppressed by a high-water mark. + Buffer(UriSvchost, recordId: 100); + Buffer(UriSystemGuard, recordId: 50, isBackfill: true); + _tracker.FlushPending(); + + Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]); + } + [Fact] public void NegativeRecordId_IsNeverDeduped() { - Emit(recordId: -1); - Emit(recordId: -1); + Buffer(UriSvchost, recordId: -1); + Buffer(UriSvchost, recordId: -1); + _tracker.FlushPending(); - Assert.Equal(2, Emitted().Count); + Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]); } [Fact] - public void Watermark_PersistsAcrossTrackerInstances() + public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances() { - // The coalesced reboot restarts the agent and the post-reboot backfill re-reads the - // same 2800 records — the persisted watermark must make that re-read a no-op. - var post = new InformationalEventPost(_sink, new VirtualClock(At)); - var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info); - - var first = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path); - first.RebootRequiredFlagProbe = () => null; + // The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same + // records — a FLUSHED record must be skipped by the next instance. + var first = MakeTracker(stateDirectory: _tmp.Path); first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500, - timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: false); + timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false); + first.FlushPending(); - var second = new MdmRebootPolicyTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path); - second.RebootRequiredFlagProbe = () => null; + var second = MakeTracker(stateDirectory: _tmp.Path); second.LoadWatermark(); second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500, - timeCreatedUtc: At, rebootUri: SampleUri, formattedDescription: null, isBackfill: true); + timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true); + second.FlushPending(); Assert.Single(Emitted()); } + [Fact] + public void Watermark_NotPersisted_WhenBufferedButNeverFlushed() + { + // Process killed by the very reboot the records announce: nothing was flushed, so the + // next instance must re-read and emit the records. + var first = MakeTracker(stateDirectory: _tmp.Path); + first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500, + timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false); + // no FlushPending — simulated reboot kill + + var second = MakeTracker(stateDirectory: _tmp.Path); + second.LoadWatermark(); + second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500, + timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true); + second.FlushPending(); + + var s = Assert.Single(Emitted()); + Assert.Equal(true, Data(s)["backfilled"]); + } + [Fact] public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt() { _tracker.RebootRequiredFlagProbe = () => true; - Emit(recordId: 1); + Buffer(UriSvchost, recordId: 1); + _tracker.FlushPending(); + _tracker.RebootRequiredFlagProbe = () => null; - Emit(recordId: 2); + Buffer(UriSystemGuard, recordId: 2); + _tracker.FlushPending(); var emitted = Emitted(); Assert.Equal(2, emitted.Count); @@ -179,10 +280,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals public void RebootRequiredFlagProbe_Throwing_IsFailSoft() { _tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom"); - Emit(); + Buffer(UriSvchost); + _tracker.FlushPending(); - var s = Assert.Single(Emitted()); - Assert.False(Data(s).ContainsKey("omadmRebootRequiredFlag")); + Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag")); } [Fact] @@ -194,20 +295,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals // ------------------------------------------------------------------ URI extraction [Fact] - public void ExtractRebootUri_PrefersEventDataValue_OverDescription() + public void ExtractRebootUri_FromVerifiedRealDescription() { - var eventData = new Dictionary { ["Data0"] = " " + SampleUri + " " }; - var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, "unrelated ./Other/Uri text"); - Assert.Equal(SampleUri, uri); + // Real 2800 text wraps the URI in parentheses and ends with a period — both must be + // excluded from the extracted URI. + var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription); + Assert.Equal(UriSvchost, uri); } [Fact] - public void ExtractRebootUri_RegexFallback_FromDescription() + public void ExtractRebootUri_PrefersEventDataValue_OverDescription() { - var uri = MdmRebootPolicyTracker.ExtractRebootUri( - new Dictionary { ["Data0"] = "not a uri" }, - $"The following URI has triggered a reboot: {SampleUri}."); - Assert.Equal(SampleUri, uri); + var eventData = new Dictionary { ["Data0"] = " " + UriSystemGuard + " " }; + var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription); + Assert.Equal(UriSystemGuard, uri); } [Fact] diff --git a/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/MdmRebootPolicyTracker.cs b/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/MdmRebootPolicyTracker.cs index f6508bfb..b625209a 100644 --- a/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/MdmRebootPolicyTracker.cs +++ b/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/MdmRebootPolicyTracker.cs @@ -4,6 +4,7 @@ using System.Diagnostics.Eventing.Reader; using System.IO; using System.Linq; using System.Text.RegularExpressions; +using System.Threading; using System.Xml.Linq; using AutopilotMonitor.Agent.V2.Core.Logging; using AutopilotMonitor.Agent.V2.Core.Orchestration; @@ -15,31 +16,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals { /// /// Watches Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin for - /// EventID 2800 — logged once per device-assigned MDM policy whose CSP URI matches the OS - /// reboot-required catalog — and forwards each as a mdm_policy_reboot_required event - /// carrying the reboot-forcing URI. + /// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog — + /// and emits ONE aggregated mdm_policy_reboot_required event per burst carrying the URI + /// list (rebootUris, uriCount, firstRebootUri). /// - /// This attributes the "unexpected reboot + second sign-in screen" pattern (PMPC research): - /// device-assigned policies applied during ESP DeviceSetup can demand a reboot; Windows - /// coalesces them into one restart at the end of DeviceSetup, forcing the user through a second - /// sign-in before AccountSetup. Admins are blind to this in the Intune console — the attributed - /// URIs make it actionable (reassign the profiles to user groups and the reboot disappears). + /// Verified against session b2e890c1 (2026-07-20): the 2800 description reads + /// The following URI has triggered a reboot: (./Device/...). and records arrive in + /// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record + /// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced + /// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond) + /// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL + /// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an + /// actually-observed system_reboot_detected before attributing anything. /// /// - /// Backfill. Device policies can apply before the agent starts, so a startup backfill - /// scans the lookback window. The .evtx persists across the coalesced reboot, so the - /// post-reboot agent restart re-reads the same records — which is why the - /// cross-restart RecordId watermark (WindowsUpdateTracker pattern) is mandatory here: - /// without it every reboot-forcing URI would be re-emitted after the very reboot it caused. - /// - /// - /// Tolerant matching. The exact 2800 message text is unverified (validated post-deploy on - /// the E2E VM): emission is gated on the EventID only, never on message wording. URI extraction - /// prefers structured EventData values that look like an OMA-DM URI (./…) and falls back - /// to a permissive regex over the formatted description. Events with an unparseable URI are - /// still emitted (with the captured description) so the timeline shows them and the captured - /// text tells us how to fix the extraction — the analyze rule keys on rebootUri exists - /// and simply stays silent for those. + /// Backfill + watermark. Device policies can apply before the agent starts, and a + /// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the + /// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical + /// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so + /// a pre-emit death loses nothing and a completed emit is never repeated. /// /// internal sealed class MdmRebootPolicyTracker : IDisposable @@ -49,12 +44,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals internal const string WatermarkStateFileName = "mdm-reboot-watermark.json"; + /// Cap for the emitted rebootUris list — uriCount always carries the uncapped distinct total. + internal const int MaxUrisPerEvent = 20; + + /// Default quiet-period before a burst of 2800 records is flushed as one event. + internal const int DefaultDebounceMilliseconds = 10000; + /// Registry key whose RebootRequired value/subkey flags a pending OMA-DM coalesced reboot. internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML"; - // Permissive OMA-DM URI shape: "./Device/Vendor/MSFT/...", "./User/...", "./Vendor/...". - // Kept deliberately broad — the exact 2800 text is unverified; post-deploy E2E validates - // and tightens this against real records if needed. + // Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800 + // text wraps the URI in parentheses — ')' is deliberately outside the class so the match + // stops cleanly; trailing sentence punctuation is trimmed after the match. private static readonly Regex UriPattern = new Regex( @"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled); @@ -65,16 +66,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals private readonly bool _backfillEnabled; private readonly int _backfillLookbackMinutes; private readonly string _stateDirectory; + private readonly int _debounceMilliseconds; private EventLogWatcher _watcher; + private Timer _debounceTimer; - // Dedup — same contract as WindowsUpdateTracker (see its field docs): immutable - // cross-restart boundary from disk + intra-run HashSet (NOT a high-water mark, because the - // live watcher is armed before the backfill scan and can deliver higher RecordIds first). + private sealed class PendingRecord + { + public long RecordId; + public DateTime TimeUtc; + public string Uri; // null when extraction failed + public string Description; // may be null + public bool IsBackfill; + } + + // Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as + // WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE + // deliberate difference: the watermark is persisted at FLUSH time, not at claim time — + // records buffered but never flushed (process killed by the very reboot they announce) + // must be re-read and emitted by the post-restart backfill. + private readonly List _pending = new List(); private long _restartWatermark = -1; private readonly HashSet _seenThisRun = new HashSet(); - private long _maxEmittedRecordId = -1; - private readonly object _watermarkLock = new object(); + private long _maxPersistedRecordId = -1; + private readonly object _stateLock = new object(); /// /// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown @@ -89,7 +104,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals AgentLogger logger, bool backfillEnabled = true, int backfillLookbackMinutes = 60, - string stateDirectory = null) + string stateDirectory = null, + int debounceMilliseconds = DefaultDebounceMilliseconds) { _sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId)); _tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId)); @@ -98,6 +114,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals _backfillEnabled = backfillEnabled; _backfillLookbackMinutes = backfillLookbackMinutes; _stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null; + _debounceMilliseconds = debounceMilliseconds; } public void Start() @@ -106,13 +123,30 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals StartWatcher(); if (_backfillEnabled && _backfillLookbackMinutes > 0) + { BackfillRecentEvents(); + // The backfill batch is complete — flush immediately instead of waiting out the + // debounce (live records that raced in are simply included). + FlushPending(); + } else + { _logger.Info("MdmRebootPolicy backfill disabled by config"); + } } public void Stop() { + // Flush any un-emitted burst before tearing down (normal agent shutdown path). + try { FlushPending(); } + catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); } + + var timer = Interlocked.Exchange(ref _debounceTimer, null); + if (timer != null) + { + try { timer.Dispose(); } catch { } + } + if (_watcher == null) return; try { @@ -168,7 +202,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals $"*[System[(EventID={EventId_PolicyRebootRequired})]]"; // ----------------------------------------------------------------------- - // Event processing + // Event processing — buffer + debounce // ----------------------------------------------------------------------- private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e) @@ -212,9 +246,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals } /// - /// Core processing extracted to primitive inputs so tests can drive it without synthesizing - /// an abstract, Windows-only . Mirrors the WindowsUpdateTracker - /// test-seam pattern. + /// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the + /// WindowsUpdateTracker pattern). The aggregated event is emitted by + /// — after the debounce quiet-period, at backfill completion, or at . /// internal void ProcessEvent( int eventId, @@ -224,53 +258,155 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals string formattedDescription, bool isBackfill) { - if (!MarkProcessed(recordId)) - return; // already emitted (cross-restart or duplicate delivery) + lock (_stateLock) + { + if (recordId >= 0) + { + // Already emitted in a prior run, or already buffered/emitted this run. + if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId)) + return; + } + + _pending.Add(new PendingRecord + { + RecordId = recordId, + TimeUtc = timeCreatedUtc ?? DateTime.UtcNow, + Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri, + Description = formattedDescription, + IsBackfill = isBackfill, + }); + } + + ArmDebounce(); + } + + private void ArmDebounce() + { + if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly) + + var timer = _debounceTimer; + if (timer == null) + { + var created = new Timer(_ => + { + try { FlushPending(); } + catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); } + }, null, Timeout.Infinite, Timeout.Infinite); + + timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created; + if (!ReferenceEquals(timer, created)) + { + try { created.Dispose(); } catch { } + } + } + + try { timer.Change(_debounceMilliseconds, Timeout.Infinite); } + catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ } + } + + /// + /// Emits the pending burst as ONE aggregated mdm_policy_reboot_required event and + /// persists the watermark. No-op when nothing is pending. Internal for tests. + /// + internal void FlushPending() + { + List batch; + long watermarkToPersist = -1; + + lock (_stateLock) + { + if (_pending.Count == 0) return; + batch = new List(_pending); + _pending.Clear(); + + foreach (var r in batch) + { + if (r.RecordId > _maxPersistedRecordId) + _maxPersistedRecordId = r.RecordId; + } + watermarkToPersist = _maxPersistedRecordId; + } + + if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist); + + batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc)); + + var distinctUris = batch + .Where(r => r.Uri != null) + .Select(r => r.Uri) + .Distinct(StringComparer.OrdinalIgnoreCase) + .ToList(); + var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri; + var unparsedCount = batch.Count(r => r.Uri == null); + var allBackfill = batch.All(r => r.IsBackfill); + var earliest = batch[0].TimeUtc; var data = new Dictionary { - { "windowsEventId", eventId }, - { "backfilled", isBackfill }, + { "windowsEventId", EventId_PolicyRebootRequired }, { "eventLogChannel", Channel }, + { "recordCount", batch.Count }, + { "uriCount", distinctUris.Count }, + { "backfilled", allBackfill }, }; - if (recordId >= 0) data["recordId"] = recordId; - if (timeCreatedUtc.HasValue) data["eventTime"] = timeCreatedUtc.Value.ToString("o"); - if (!string.IsNullOrEmpty(rebootUri)) data["rebootUri"] = rebootUri; - if (!string.IsNullOrEmpty(formattedDescription)) + if (distinctUris.Count > 0) { - data["description"] = formattedDescription.Length > 1000 - ? formattedDescription.Substring(0, 1000) + "…" - : formattedDescription; + data["rebootUris"] = distinctUris + .OrderBy(u => u, StringComparer.OrdinalIgnoreCase) + .Take(MaxUrisPerEvent) + .ToList(); + data["firstRebootUri"] = firstUri; + } + if (unparsedCount > 0) + { + data["unparsedCount"] = unparsedCount; + var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description; + if (sample != null) + data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample; } var flag = SafeProbeRebootRequiredFlag(); if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value; + string message; + if (distinctUris.Count == 1) + message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}"; + else if (distinctUris.Count > 1) + message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)"; + else + message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)"; + _post.Emit(new EnrollmentEvent { SessionId = _sessionId, TenantId = _tenantId, - // The record's own time, not UtcNow: backfilled pre-agent (and post-reboot re-read) - // records must land on the timeline at when the policy actually demanded the reboot. - Timestamp = timeCreatedUtc ?? DateTime.UtcNow, + // Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts + // must land on the timeline at when the policies were actually flagged. + Timestamp = earliest, EventType = Constants.EventTypes.MdmPolicyRebootRequired, - Severity = EventSeverity.Warning, + // Neutral observation — reboot-required flags outside ESP DeviceSetup have no + // immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an + // actually-observed system_reboot_detected. + Severity = EventSeverity.Info, Source = "MdmRebootPolicyWatcher", Phase = EnrollmentPhase.Unknown, - Message = !string.IsNullOrEmpty(rebootUri) - ? $"MDM policy requires a reboot during ESP: {rebootUri}" - : "MDM policy requires a reboot during ESP (URI not parsed from event — see description)", + Message = message, Data = data, - // The reboot this event announces will kill the agent — flush now. + // Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup + // precedes a coalesced reboot that kills the agent. ImmediateUpload = true, }); + + _logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " + + $"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})"); } /// /// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData - /// value that starts with ./ (the CSP-URI shape), (2) a permissive regex over the - /// formatted description. Returns null when nothing URI-like is found — the caller still - /// emits, just without the rebootUri field. + /// value that starts with ./, (2) a permissive regex over the formatted description + /// (verified shape: The following URI has triggered a reboot: (./Device/...).). + /// Returns null when nothing URI-like is found — the record still counts into the + /// aggregate (unparsedCount + sampleDescription). /// internal static string ExtractRebootUri(Dictionary eventData, string description) { @@ -289,8 +425,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals var match = UriPattern.Match(description); if (match.Success) { - // The permissive pattern can swallow sentence punctuation trailing the URI - // ("...Security."). OMA-DM URI segments never end in punctuation — trim it. + // The permissive pattern can swallow sentence punctuation trailing the URI. + // OMA-DM URI segments never end in punctuation — trim it. var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')'); if (candidate.Length > 2) return candidate; } @@ -349,8 +485,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals } /// - /// Reads the pending-reboot flag under . The exact shape is - /// unverified (PMPC research names the key, not the value type), so both a value and a + /// Reads the pending-reboot flag under . Both a value and a /// subkey named RebootRequired count as "pending". Fail-soft: null = unknown /// (key absent / unreadable), matching the EspSkipConfigurationProbe convention. /// @@ -424,44 +559,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals } // ----------------------------------------------------------------------- - // Watermark dedup (cross-restart) — WindowsUpdateTracker pattern verbatim + // Watermark dedup (cross-restart) — persisted at flush, not at claim // ----------------------------------------------------------------------- private bool IsAlreadyProcessed(long recordId) { if (recordId < 0) return false; // no RecordId → cannot dedup, let it through - lock (_watermarkLock) + lock (_stateLock) { return recordId <= _restartWatermark || _seenThisRun.Contains(recordId); } } - /// - /// Atomically claims . Returns true if this is the first time we - /// see it (caller should emit), false if it was already processed — either in a PRIOR run - /// (at/below the restart watermark) or already this run (in the seen-set). Records without - /// a RecordId (-1) are always emitted, never tracked or persisted. - /// - private bool MarkProcessed(long recordId) - { - if (recordId < 0) return true; - - long toPersist = -1; - lock (_watermarkLock) - { - if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId)) - return false; - - if (recordId > _maxEmittedRecordId) - { - _maxEmittedRecordId = recordId; - toPersist = _maxEmittedRecordId; - } - } - if (toPersist >= 0) PersistWatermark(toPersist); - return true; - } - internal void LoadWatermark() { if (string.IsNullOrEmpty(_stateDirectory)) return; @@ -475,10 +584,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals var state = JsonConvert.DeserializeObject(json); if (state != null) { - lock (_watermarkLock) + lock (_stateLock) { _restartWatermark = state.LastRecordId; - _maxEmittedRecordId = state.LastRecordId; + _maxPersistedRecordId = state.LastRecordId; } _logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})"); } diff --git a/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ShellCoreTracker.cs b/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ShellCoreTracker.cs index c93cf113..0f875428 100644 --- a/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ShellCoreTracker.cs +++ b/src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ShellCoreTracker.cs @@ -16,9 +16,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals /// CommercialOOBE_ESPProgress_Page_Exiting — normal ESP exit /// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete /// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure - /// "RebootCoalescing" (DeviceSetup) — ESP-initiated coalesced reboot - /// (MDM policies forced a mid-ESP restart → second sign-in; corroborates the - /// MdmRebootPolicyTracker's per-URI 2800 attribution events) + /// + /// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was + /// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker + /// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves + /// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in + /// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot. /// /// Raises , , /// and . Cross-notifies the @@ -50,7 +53,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals private bool _espExitDetected; private bool _whiteGloveDetected; private bool _helloWizardStartDetected; - private bool _rebootCoalescingDetected; private readonly object _stateLock = new object(); /// @@ -251,27 +253,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals _logger.Info("ESP phase exit detected - detected via Shell-Core event 62407"); } - else if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0) - { - // ESP-initiated coalesced reboot (end of DeviceSetup): MDM policies forced a - // restart, the user will see a second sign-in screen. Checked AFTER the - // WhiteGlove/failure/exiting patterns so their semantics stay untouched even - // if a description ever carried both tokens. Fire-once per process run — the - // reboot restarts the agent, and the fresh run re-observes it via backfill. - lock (_stateLock) - { - if (_rebootCoalescingDetected) return; - _rebootCoalescingDetected = true; - } - - eventType = Constants.EventTypes.EspRebootCoalescing; - severity = EventSeverity.Warning; - message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in"; - // No FinalizingSetup transition, no C# event raise — pure informational - // corroboration for the MdmRebootPolicyTracker's per-URI attribution. - - _logger.Warning("ESP coalesced reboot detected via Shell-Core event 62407 (RebootCoalescing)"); - } else { return; @@ -446,29 +427,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals return; } - if (description.IndexOf("RebootCoalescing", StringComparison.OrdinalIgnoreCase) >= 0) - { - // DELIBERATE deviation from the other 62407 backfill branches (which only raise C# - // events): this is a pure informational type with no reducer/subscriber — the - // emission IS the signal, and the live emit races the very reboot that kills the - // process. Re-emitting after the restart is the only reliable delivery path. A - // live+backfill cross-process duplicate is possible (no watermark here); accepted: - // the analyze rule uses exists-semantics, both copies carry the identical historical - // timestamp, and the backfill copy is marked backfilled:true. - bool shouldEmit; - lock (_stateLock) - { - shouldEmit = !_rebootCoalescingDetected; - _rebootCoalescingDetected = true; - } - if (shouldEmit) - { - _logger.Info($"Backfill: ESP coalesced reboot (RebootCoalescing) found in recent Shell-Core logs (originalAt={occurredAtUtc:o})"); - EmitRebootCoalescing(description, occurredAtUtc, isBackfill: true); - } - return; // a RebootCoalescing record is not an ESP exit/failure — stop here - } - if (EspExitingPattern.IsMatch(description)) { bool shouldNotify = false; @@ -507,39 +465,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals } } - /// - /// Emits the esp_reboot_coalescing event with the standard Shell-Core eventData - /// shape. Used by the backfill path (the live path flows through the shared emit tail in - /// ); adds the - /// backfilled marker that distinguishes an accepted post-restart re-emission. - /// - private void EmitRebootCoalescing(string description, DateTime occurredAtUtc, bool isBackfill) - { - var eventData = new Dictionary - { - { "windowsEventId", EventId_ShellCore_WebAppEvent }, - { "providerName", "" }, - { "description", description }, - { "eventLogChannel", ShellCoreEventLogChannel }, - { "eventTime", occurredAtUtc.ToString("o") }, - { "backfilled", isBackfill } - }; - - _post.Emit(new EnrollmentEvent - { - SessionId = _sessionId, - TenantId = _tenantId, - Timestamp = occurredAtUtc, - EventType = Constants.EventTypes.EspRebootCoalescing, - Severity = EventSeverity.Warning, - Source = "EspAndHelloTracker", - Phase = EnrollmentPhase.Unknown, - Message = "ESP initiated a coalesced reboot (RebootCoalescing) — device will restart and show a second sign-in", - Data = eventData, - ImmediateUpload = true - }); - } - // ===================================================================== // Pattern helpers // ===================================================================== diff --git a/src/McpServer/autopilot-monitor-mcp/src/resource-catalog.ts b/src/McpServer/autopilot-monitor-mcp/src/resource-catalog.ts index 98bdf6a1..ab988117 100644 --- a/src/McpServer/autopilot-monitor-mcp/src/resource-catalog.ts +++ b/src/McpServer/autopilot-monitor-mcp/src/resource-catalog.ts @@ -24,7 +24,6 @@ export const EVENT_TYPES_CATALOG = { 'esp_failure_settle_started', 'esp_failure_settle_recovered', 'esp_exiting', - 'esp_reboot_coalescing', 'mdm_policy_reboot_required', 'esp_config_detected', 'completion_check', diff --git a/src/Shared/AutopilotMonitor.Shared/Constants.cs b/src/Shared/AutopilotMonitor.Shared/Constants.cs index c609bf06..96ee9533 100644 --- a/src/Shared/AutopilotMonitor.Shared/Constants.cs +++ b/src/Shared/AutopilotMonitor.Shared/Constants.cs @@ -405,17 +405,17 @@ namespace AutopilotMonitor.Shared public const string OsBuildChanged = "os_build_changed"; // OS build (CurrentBuild.UBR) differs across an agent restart — an update was installed during enrollment public const string WindowsUpdateChannelCensus = "windows_update_channel_census"; // One-shot EventID histogram of WU Client + UpdateOrchestrator channels when os_build_changed but zero targeted WU events captured - // MDM-policy-forced coalesced reboot during ESP (MdmRebootPolicyTracker + ShellCoreTracker; - // PMPC "unexpected reboot / second sign-in" research). Device-assigned policies whose CSP - // URIs match the OS reboot-required catalog force a coalesced reboot at the end of - // DeviceSetup — the user sees an unexpected restart and a second sign-in screen before - // AccountSetup. 2800 (DeviceManagement-Enterprise-Diagnostics-Provider/Admin) fires once - // per reboot-forcing URI; Shell-Core 62407 "RebootCoalescing" records ESP initiating the - // reboot. Deliberately DISTINCT from system_reboot_detected: the backend RebootCount - // matches that string exactly (EventIngestProcessor.Classification + terminal recount) — - // these attribution events must never inflate the reboot count. - public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // DM-Enterprise EventID 2800 — Data.rebootUri = CSP URI that demands the reboot (one event per URI) - public const string EspRebootCoalescing = "esp_reboot_coalescing"; // Shell-Core 62407 "RebootCoalescing" — ESP-initiated coalesced reboot imminent (second sign-in follows) + // MDM reboot-required policy flags (MdmRebootPolicyTracker; PMPC "unexpected reboot / + // second sign-in" research). DM-Enterprise EventID 2800 fires once per policy URI that + // matches the OS reboot-required catalog ("The following URI has triggered a reboot: + // (./Device/...)."; verified session b2e890c1). Only URIs applied DURING ESP DeviceSetup + // cause the coalesced mid-ESP reboot — later syncs (AccountSetup+) merely flag, no forced + // reboot. The tracker therefore emits ONE aggregated Info event per burst with the URI + // list; the reboot CLAIM lives in ANALYZE-ESP-005, gated on an actually-observed + // system_reboot_detected. Deliberately DISTINCT from system_reboot_detected: the backend + // RebootCount matches that string exactly (EventIngestProcessor.Classification + terminal + // recount) — this attribution event must never inflate the reboot count. + public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // Aggregated per burst — Data.rebootUris[] + uriCount + firstRebootUri // TEMPORARY: shadow SM rollout verification — remove when CompletionStateMachine is promoted to primary public const string ShadowDiscrepancy = "shadow_discrepancy";