Compare commits

...

70 Commits

Author SHA1 Message Date
Oliver Kieselbach 6b92e4e28c bump pointer: internal (D-316/D-317, concept docs, plan review, B-qrz/B-cwy/B-shx/B-96m), .claude/commands (platform-health marker)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:20:45 +02:00
Oliver Kieselbach f392819b97 Portal, MCP and backend apply script_output_reconciliation: a corrected platform run shows its own output and "exit n/a" (D-316)
Web: the script panel applies the correction to the run its runId names: the saved output replaces the emitted one, a foreign end block's exit code shows as n/a with an explanation, the state follows the corrected data instead of the emitted event type, and the output carries a short "from IME's saved result" note. The timeline marks the original row as corrected later, and the correction event falls under the script-output setting like the other script events. The session page passes the event to the panel; the gather-rule event list knows the type.

MCP: get_session_summary no longer counts a script_failed as an error, ranks it as one or attaches the exit code's catalog text when a foreign correction with IME result Success names its run.

Backend: the correction never heals a stalled session and never counts as a script run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:15:07 +02:00
Oliver Kieselbach 672570df04 Platform script output is checked against IME's saved result and corrected when the log attributed another script's end block (D-316)
After reporting a script batch, IME saves every run under HKLM\SOFTWARE\Microsoft\IntuneManagementExtension\Policies\<user>\<policy>; ResultDetails carries the stdout IME read from that policy's own result file (the stderr when it failed with error code 99). The agent now checks each emitted platform run against it: a match confirms the output and the exit code of the same end block, a contradiction emits script_output_reconciliation (Warning, outcome "foreign": the saved output replaces the emitted one, the exit code is unverified), and output the log delivered incomplete is completed (Info, outcome "repaired").

A save is paired with the latest run of its user and policy only after the tracker completed a pass that started two seconds after the save was seen, so a stalled poll loop cannot pair it with an older run. Platform completions carry a runId the correction names. Two restart-safe counters (verified, corrected) ride agent_metrics_snapshot and ime_pattern_hits. The new event type is registered in the shared constants, the generated manifests and the MCP event catalog.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:08:22 +02:00
Oliver Kieselbach 6f7ececa1f Platform scripts no longer take another executor's output and exit code: a cut line in AgentExecutor.log ends the invocation in front of it (D-317)
Two AgentExecutor processes that open the log at the same end write their start blocks over each other. The end block of the one whose start block lost then sat behind the other executor's start block, and position-based ownership handed a Win32 detection script's stdout and exit code to the platform script.

An AgentExecutor.log entry that is not exactly one CMTrace record (a fragment, a merged line or an empty line) is where one writer's bytes end and another's begin: it now closes the platform invocation in front of it and is never matched. Invocation markers live as long as the bytes of their entry: a rewind sets them aside and gives an entry that reads unchanged its markers back.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 15:08:21 +02:00
Oliver Kieselbach af7e9d3650 bump pointer: internal (D-315 live, B-crw field verify)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:51:00 +02:00
Oliver Kieselbach b3b122a23c bump pointer: internal (session start anchor D-315, plan, B-crw)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:41:19 +02:00
Oliver Kieselbach 064b670c75 Update and servicing rows backfilled from before the agent no longer set the session start (D-315)
Since agent 2.0.1473 the Windows Update and servicing watchers backfill rows with their event-log
time up to 60 min before the agent started, and the session anchor only skipped the system
timeline watcher, so session starts and durations grew by 22 min at the median.

- Shared: Constants.EventSources.SessionAnchorIneligible and IsSessionAnchorEligible are the one
  rule; the agent writes the same constants as Source.
- Backend: the ingest batch (ClassifyEvents, now internal static) and the Events-table probe use it.
- Web: the session duration reads the list from shared-manifests.json.
- Recorded sessions keep their stored start; the rule applies to new ingests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:40:54 +02:00
Oliver Kieselbach e5eb6dc4fb bump pointer: internal (OOBE update field measurement 2, B-cqa session start regression, B-93v)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 13:05:18 +02:00
Oliver Kieselbach 9f46adcdfa ci: the storage auth helper masks its token in the run log, like the Key Vault token
GitHub masks only secrets.* by itself. The bearer token Get-StorageAuthHeaders.ps1
mints at run time was never registered as a mask, unlike the Key Vault signing token,
which build-agent, build-msi and publish-scripts mask before exporting it. No caller
prints it today -- every call site captures the hashtable and passes it only as
-Headers -- but this repository's run logs are public, and a single debugging line
would have published it: an uncaptured call of the helper, or Get-Error /
Format-List on a failed request, whose TargetObject is the HttpRequestMessage
including the Authorization header (reproduced locally with a dummy token).

The helper now emits ::add-mask:: for every token it hands out, which covers all six
workflows in one place. Write-Host, never Write-Output: the output stream is the
script's return value. Verified with a mocked az that the result stays one hashtable
and that a nested caller in the Request-BuildNumber.ps1 pattern still returns only
its build number. Only under GITHUB_ACTIONS -- locally the line would print the token
instead of hiding it.

Prompted by an external AI review of the public scripts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 12:52:01 +02:00
Claude cd1319a486 The GA tenant editor's Members card drops its icon column, so its content lines up with the other cards
- The Members card was the only card in the edit modal with a separate
  icon column: its title, add form and member list sat 32px in from the
  card edge, which on a phone also took 32px from the email field and the
  member rows.
- It now uses the layout of its sibling cards: the title on top, the
  content at full card width.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNLaZbFQTdhQJU9jh37Hn6
2026-10-03 09:17:08 +00:00
Claude ca10f36624 Landing stats band takes the versioned stats file from the browser cache again; the build log names the baked file
Since e537a4b the backend writes a new versioned file per publish, so its
immutable header holds again. The browser revalidates only the manifest
(no-cache) and serves the file it names from its cache (default mode),
instead of revalidating both, the workaround in 762fef4 for the per-day name.

The build log names the baked file ("Platform stats baked into the landing
page from platform-stats.<date>T<time>Z.json (as of ...)"), so every web
deploy shows which publish it carries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4ddfosgdeCwXo7NvB9xd2
2026-10-03 04:57:38 +00:00
Claude e537a4b074 Platform stats publish writes a new versioned file each run, so its immutable header holds again
The versioned stats file is served public, max-age=31536000, immutable, and
the manifest platform-stats.json names the current one. Its per-day name
(19e9713a) was unique while maintenance ran once a night; since 82919d47 it
runs every 2 hours and rewrote that file up to 12 times a day, so caches kept
the first copy of the day. The name now carries the publish instant to the
second in invariant culture (platform-stats.2026-10-03T060004Z.json).

Headers and access tier travel with the upload instead of two follow-up
calls, which left the blob briefly without Cache-Control and content type,
and kept it that way (with the manifest still on the old file) when one of
them failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4ddfosgdeCwXo7NvB9xd2
2026-10-03 04:54:54 +00:00
Claude 762fef4e24 Landing page bakes the platform stats of the deploy into its static HTML; browsers then swap in the live numbers
Crawlers and LLM fetchers run no JS and saw only the skeleton of the stats
band. app/page.tsx now awaits one read of the public stats blob during
next build (lib/platformStats.ts: manifest, then the versioned file it
names) and renders the numbers into the export. StatsBand starts from that
snapshot and replaces it with the live numbers; a failed live read keeps
the baked numbers instead of hiding the band.

The read is awaited at page level, not under Suspense: React moves a
resolved boundary of a page this size into a hidden div at the end of the
body. It uses no-store (route force-static), so Next's fetch cache cannot
hand a later build the previous numbers. Fail-soft: without a snapshot the
page ships the skeleton and loads live as before, and deploy-web.yml warns.

The browser now revalidates both files (no-cache instead of force-cache):
the backend rewrites the per-day file every 2 hours under immutable, so a
copy cached earlier that day would have made the numbers drop after load.

The page body moved to components/landing/LandingPage.tsx, which the
marketing guards render synchronously; StatsBand left the .json() ratchet
baseline; the sitemap lastmod of / includes the body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N4ddfosgdeCwXo7NvB9xd2
2026-10-03 03:27:20 +00:00
Oliver Kieselbach bca492ba05 bump pointer: internal (agent v2.0.1474 released, B-6dx field verify, time-attribution deploy steps)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 23:06:40 +02:00
Oliver Kieselbach e7dd962133 bump pointer: internal (time attribution v4 D-314, OOBE update page agent fixes, backlog B-q3z/B-6dx/B-dbf; member list lesson)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:57:10 +02:00
Oliver Kieselbach 14a67ef5af Time attribution shows the OOBE quality update as Windows Update and the wait after its restart as Waiting for sign-in (D-314)
TimeAttributionCalculator v4 cuts two segments out of the phase partition:
os_update from the OobeNDUP page start whose visit holds update evidence
(a servicing step or a name only an update in progress writes) to the last
update evidence before the user is back; awaiting_sign_in from there to the
first evidence of the user, only after a restart. Both replace the phase
span they overlap; without evidence the partition is v3's. OsUpdates names
the KBs and restarts. Reboot gaps no longer start or end on the clock-sync
row logged at boot. The sweep recomputes the last 30 days.

Aggregates carry eight segments; the session lane and Fleet Health show
"Windows Update" (cyan) and "Waiting for sign-in" (sky) with the KBs and
restarts in legend and tooltip; MCP get_time_attribution description;
manifest and wire types regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:55:45 +02:00
Oliver Kieselbach 2371855b77 OOBE update page: fields from event properties in every display language, page start and stop never capped, no duplicates after a restart
ShellCoreTracker reads CXID, result, name and value from the event properties
(62404..62407 %1/%2), live and in the backfill. The formatted message is
localized: German devices lost result and value, French devices reported no
update page at all. The text patterns stay as the fallback and accept
guillemets. The Hello-wizard start reads the CXID property too.

OobeUpdateTelemetry: page start and stop sit outside the run budget (at most
3 per page), names share a budget of 150 (was 50; the cloud page logs ~45
init names in its first 40 s). ReportPage returns whether it reported.

A RecordId watermark (shellcore-page-watermark.json) claims every update-page
record and persists reported ones, so the restart backfill does not send the
minutes before the restart again. EspAndHelloTracker passes the state
directory.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:55:39 +02:00
Oliver Kieselbach 3a50de83ef Add-member input keeps a minimum width and wraps instead of shrinking (D-313)
The form no longer lays out by viewport breakpoint: a wide window can host a
narrow column next to the settings sidebar, where the input shrank to a few
characters. The input now grows from an 18rem basis and wraps below the type
switch when the row has no room; role and Add wrap together.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 22:51:28 +02:00
Oliver Kieselbach 24dfd247e9 bump pointer: internal (viewer app role + member list live, D-313, lesson)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 21:18:14 +02:00
Oliver Kieselbach a640039518 Member role changes need an explicit Save; the GA add form gives the input its own line (D-312)
- An open member row keeps role and bootstrap permission as a draft:
  Save writes it, Discard or closing the row drops it, so a stray select
  change no longer alters anyone's access.
- AddMemberForm takes `stacked` for narrow hosts; the GA tenant editor
  modal uses it, so the email field spans the full width there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 21:13:11 +02:00
Oliver Kieselbach c3e9011905 bump pointer: internal (D-312, plan viewer-app-role-and-member-list, B-8gf/B-7w2)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 20:36:37 +02:00
Oliver Kieselbach 8f6926a945 Viewer can come from an Entra app role; Access Management lists one line per member (D-312)
- EntraAppRoleResolver maps the Viewer app role next to Admin and Operator
  (the highest role wins); a claim Viewer gets no bootstrap-token rights.
- The tenant settings page and the GA tenant editor share components/members:
  one line per member, a click opens who added it and the actions, Remove
  asks in the row, role chips with counts, a search, 25 rows per page. A
  vanished chip or a shrunken list moves the selection, so the list never
  shows an empty page and never snaps back.
- The add form sits above the list, About Roles is collapsed, and a tenant
  with Entra app roles enabled sees that those users are not listed.
- The EntraAppRolesEnabled doc names Viewer; manifest and generated wire
  types regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 20:32:54 +02:00
Oliver Kieselbach 01323fdf6d ProgressAccessHint test matches the docs link as a substring, not a RegExp
CodeQL js/incomplete-hostname-regexp (alert #96) traced DOCS_URL into the
RegExp built in the "Learn more" assertion. The dots were escaped with
[.], which CodeQL does not recognize as a sanitizer. A substring match
on the exact markup needs no escaping and keeps the same checks: a space
after the sentence, the exact href with target/rel, and "Learn more" as
the link text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 20:17:46 +02:00
Oliver Kieselbach c566ed818b bump pointer: internal (access hint released, B-u5a verify only)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 17:01:38 +02:00
Oliver Kieselbach 3a40361bb6 bump pointer: internal (access-hint follow-up, docs-corpus URL invariant)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:54:52 +02:00
Oliver Kieselbach 1b988d2943 Docs links use the URL GitBook publishes: access hint via the docs path registry, MCP citations from SUMMARY.md groups
GitBook publishes a page under the slug of its SUMMARY.md group, not its
folder: troubleshooting/ is /troubleshooting-and-support/, trust/ is
/trust-and-security/. A folder URL works only for pages that predate the
group rename (307); for a newer page it is a 404.

- Web: the Progress Portal access hint linked the new troubleshooting
  article by folder path (404 live). It now uses DOCS_PATHS.progressPortalOnly.
- MCP: docUrl built folder URLs for every search_docs citation. Live census
  over all 58 SUMMARY pages: 47 direct, 10 via redirect, 1 dead.
  summaryFolderSlugs now derives the segments from SUMMARY.md; all 60 page
  URLs of the real corpus answer 200.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:54:40 +02:00
Oliver Kieselbach e88a0e455c Progress Portal access hint links to its troubleshooting article
Both variants end with a muted "Learn more" to
troubleshooting/progress-portal-only in the customer docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:35:37 +02:00
Oliver Kieselbach 0695f7943c bump pointers: internal (D-311, plan progress-portal-access-hint, B-zr9/B-u5a), commands (platform-health marker)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:20:50 +02:00
Oliver Kieselbach 6f8e7d48b2 Progress Portal tells members without a role whom to ask for access; activation lands every role where sign-in would
A member without a portal role only ever sees the Progress Portal and never
learned that their organization already runs the full product. Below the
search they now see one muted line: ask your Autopilot Monitor admin for a
role. When the organization holds no session and signed up more than 14 days
ago, the line names the signup date and the support mailbox instead.

- GET progress/tenant-status (AuthenticatedUser, JWT tenant only, no session
  data): unused flag plus signup date. Both reads are strict, so a storage
  failure is a 500 and never reads as "unused".
- ISessionRepository.HasAnySessionStrictAsync: key-only probe on
  SessionsIndex with the primary-table fallback, no catch.
- ProgressTenantStatusResponse + wire parity facts; regenerated manifests and
  web/MCP wire types.
- lib/landingTarget.ts: one landing decision for the landing gate and the
  activation page. Leaving activation no longer sends Operators, Viewers,
  Global Readers and delegated admins to the Progress Portal, and it keeps
  the deep link the landing gate saved during activation.
- lib/supportContact.ts: one source for the support address.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:20:38 +02:00
Oliver Kieselbach 0248d14738 bump pointer: internal (IME diff notes 1.106.106.0 + 1.105.154.0, B-4v2)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 16:01:05 +02:00
Oliver Kieselbach dc59fea6e5 bump pointer: internal (OOBE update phase: DEV-004 live, agent v2.0.1473 in Shadow)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 15:28:34 +02:00
Oliver Kieselbach 08455cc5f9 OOBE quality update: install events from the System log, servicing steps, a bounded engine phase in Shadow mode (D-310)
Report 3a2207978e4c: a session was failed 30 min after the ESP exit while Windows installed its
OOBE quality update. The update watcher had been blind since July and the engine had no notion
of the phase.

- Windows Update client: 19/20/43/44 come from the System log (provider-filtered XPath), as the
  provider manifest routes them; 25/26/31/41 from Operational via their own config list.
  Updates are classified os/defender/store/other: only Windows/.NET updates become rows, the
  rest is counted into windows_update_activity_summary. Per-channel watermarks persist a record
  only after its row and engine signal were handed to the ingress.
- Servicing: CBS steps from the Setup log (windows_update_servicing), client UpdateAgentLCU.
- Engine: OsUpdateActivity and OsUpdateFacts (event-time ordered) anchor an OOBE update phase
  after the ESP exit. Each state has its own bound: the update runs at most 3 h (then Failed
  oobe_update_not_finished), after it the user has 60 min to sign in (then the engine stops
  waiting without a verdict), after the sign-in one window of grace.
- The phase acts only in OobeUpdatePhaseMode=Active. Shadow, the default, reports what it would
  decide as oobe_update_phase_shadow right before today's verdict. The agent stamps the mode on
  EnrollmentFactsObserved at every start; a later stamp can take Active away, never grant it.
- Agent: a stop without a verdict ends the session like the max-lifetime watchdog
  (agent_shutting_down reason=stopped_waiting); the backend classifies it honestly (verdict
  origin stopwait), never Failed.
- Measurement: oobe_update_page (the update page and its ExpeditedUpdate_* names; a 62407 value
  only as boolean, integer or HRESULT) and oobe_update_state (NDUP and ESP-policy registry
  values), both Debug and bounded per run.
- ANALYZE-DEV-004 1.2.0 counts install and download failures only, never scan failures.
- Event types in Constants, MCP catalog and web; shared manifests regenerated; privacy page
  names Windows updates installed during the enrollment.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-02 15:09:50 +02:00
Oliver Kieselbach 87c5bf018c bump pointer: internal (gather-rule state released, B-hx3 field verify)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 22:26:18 +02:00
Oliver Kieselbach 0d6c03d64b Public texts no longer name a customer domain or IME decompiles
- Two code comments of the gather-rule timeline feed named a customer's domain next to the
  session id; the session id stays as the regression anchor.
- The get_ime_pattern_health description and the IME Pattern Health admin text no longer send
  the reader to "the IME decompile"; the IME-TOKEN-FAILURE and IME-DO-TEL pattern descriptions
  say "verified against IME x" / "absent from the x builds" (rules/dist regenerated).
- Code comments that cited decompiled IME builds cite the IME version instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 22:00:52 +02:00
Oliver Kieselbach b14866c9a6 bump pointer: internal (gather-rule state committed, B-hx3)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 21:46:44 +02:00
Oliver Kieselbach 25b133d3d0 bump pointer: internal (gather-rule state across restarts, D-309)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 21:45:29 +02:00
Oliver Kieselbach 684167292f Gather rules keep their place across agent restarts; byte-exact logparser reader; MCP filters by gather event types
- Logparser positions per (rule, file) and the on_change values persist in
  State\gather-rule-state.json, bound to the session id. A reboot, hard kill or WhiteGlove
  Part 2 no longer replays a log from byte 0 or re-emits an unchanged on_change result. A
  replaced log (fingerprint of its first 4 KB) is read from the beginning. Positions are saved
  after the events went out; restoring the file is fail-soft. --run-gather-rules stays in memory.
- LogLineReader: a maxLines stop loses no line, an unfinished last line is never split
  (interval rules wait one run, one-shot triggers read it again next run), UTF-16/UTF-32 logs
  are read by their byte order mark from any position, line ends as StreamReader.ReadLine,
  logLineNumber counts from the start of the file, lines over 32 MB are skipped. A maxLines
  value that is no positive number falls back to 1000.
- UserProfileResolver no longer caches "no user signed in yet" (detection retried at most
  every 30 s); guarded callers expand the token with the profile they checked.
- MCP get_session_events, search_sessions_by_event and query_raw_events accept a gather
  rule's own outputEventType; an empty first-page result for an uncatalogued type carries
  eventTypeNote with the closest built-in types instead of being rejected up front.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 21:45:10 +02:00
Oliver Kieselbach 6474860a57 bump pointer: internal (lesson from the vulnerabilities feedback)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 20:15:56 +02:00
Oliver Kieselbach d23f2eae3e bump pointer: internal (vulnerabilities feedback fixes, D-308)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 20:14:40 +02:00
Oliver Kieselbach fae6aa0b92 Vulnerabilities band filters always find entries, list controls stay put, first load shows the progress dialog
The summary endpoint takes perBand: above 0 it adds bandCves and bandSoftware, each severity and
priority band's top entries outside the lists. The portal asks for 10 per band and shows them only
under a filter, so Attend lists its 7 CVEs instead of nothing; the MCP tool never asks and keeps its
size. The Most exposed controls sit below the title in every state, bands with 0 entries are
disabled, and the first load shows CalculatingDialog (the Geographic Performance dialog without its
page) with separate estimates for all tenants and one tenant. D-308.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 20:13:47 +02:00
Oliver Kieselbach 43dfb731f1 bump pointer: internal (vulnerabilities tab redesign, D-307)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 19:38:45 +02:00
Oliver Kieselbach dd5534b0c9 Vulnerabilities tab shows a severity donut, the priority ladder and the most exposed software
The exposure summary groups CVE index rows by product: every new row carries the finding's
CPE vendor:product as Product, and a row without it borrows the product of a row with the
same title. The summary adds topSoftware, distinctSoftware and kevAffectedSessions, and
known-exploited CVEs and products stay listed below the topN cut (capped so the MCP result
stays under its 50k cap). The shared panel is rebuilt: KPI tiles, a severity donut and the
Act/Attend/Track ladder as list filters, and a Most exposed card with Software and CVEs
lenses. get_vulnerability_summary describes the new fields. D-307.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 19:35:09 +02:00
Oliver Kieselbach 54334de378 Rule pages point to AI-assisted rule authoring
The community box on the gather and analyze rule pages gets a line that an
assistant connected via MCP can draft, validate and test a rule, linking the
AI-assisted rule authoring docs page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 19:11:56 +02:00
Oliver Kieselbach 9671191eef bump pointer: internal (/plans nav marking, slash links, MCP docs, D-306)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 18:43:41 +02:00
Oliver Kieselbach c5a4feb762 Internal links on the public pages end with a slash
The host answers /plans, /about, /privacy and the other page paths with a
301 to the slash path first. The navbar, the footer, the landing and
get-started calls to action, the terms page and two portal links to
/settings now point at the slash path directly. A guard renders every
public page with the build's trailing-slash setting and fails on an
internal link without it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 18:41:22 +02:00
Oliver Kieselbach 2253fca533 bump pointers: internal, .claude/commands (AI page and marketing click counts, D-305)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 18:16:42 +02:00
Oliver Kieselbach ff9e79dcc7 Landing gets an AI page (/ai) and counts clicks and section views
New public page /ai, linked from the navbar (AI), the Then just ask section
and the footer. It shows what an AI client can answer from the structured
enrollment data: a question gallery whose answer panel switches between chat
app, VS Code, terminal and self-hosted, the data path (where the model runs,
what reaches us) and how to connect. A test keeps every MCP tool and argument
on the page in line with the server's registrations for tenant users.

The landing, /get-started and /ai send anonymous click and section-view
counts (marketing_cta_clicked, marketing_section_viewed); every link and
button carries a data-track id. Layouts with their own OpenGraph metadata
keep the shared image and the large Twitter card again. The navbar tightens
between lg and xl so the new entry fits at 1024 px. The FAQ names the
documented MCP clients, and the terminal demo uses the real search_sessions
argument (deviceName).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 18:15:15 +02:00
Oliver Kieselbach e54b4585b8 bump pointers: internal, .claude/commands (event follow-up page live, D-304)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 17:17:41 +02:00
Oliver Kieselbach 7894d61e0b Event follow-up page for the Workplace Ninja Summit 2026
Unlisted page /events/<slug>/ for the link in the post-event mail: try it,
request a walkthrough, read the docs, plus the free community editions of
SCEPman and MyWorkID. It is noindex and kept out of the sitemap, robots.txt
and llms.txt. Every call to action sends one anonymous click event
(event_followup_clicked). The customer quote is now shared with the landing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 17:11:48 +02:00
Oliver Kieselbach 090f47d4fb Merge branch 'main' of https://github.com/okieselbach/AutopilotMonitor 2026-10-01 13:14:44 +02:00
Oliver Kieselbach c2d68b6537 bump pointer: internal (landing GitHub stars, D-303)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 13:13:35 +02:00
Oliver Kieselbach 5b0dfaad99 Landing shows the GitHub star count and a customer quote
The navbar gets a "Star" pill with the repository's star count from sm up;
phones keep the bare GitHub mark. A new section before the closing CTA
shows one customer quote with a star ask underneath. The footer's
"Star us on GitHub" line is gone, so the page asks twice, not three times.

The count is read once per web deploy (deploy-web.yml, own step, job token
only there) and inlined by next build as NEXT_PUBLIC_GITHUB_STARS, so
visitors' browsers never contact GitHub and the CSP stays as it is.
Without a value the pill renders without a counter (D-303).

The desktop links, bell and sign-in now start at lg instead of md: the bar
already overflowed between 768 and ~1000 px and the pill widened it further.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 13:13:33 +02:00
dependabot[bot] 50d29e61e8 chore(deps): bump the web-security group across 1 directory with 3 updates (#170)
Bumps the web-security group with 3 updates in the /src/Web/autopilot-monitor-web directory: [next](https://github.com/vercel/next.js), [brace-expansion](https://github.com/juliangruber/brace-expansion) and [dompurify](https://github.com/cure53/DOMPurify).


Updates `next` from 16.3.5 to 16.3.6
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/compare/v16.3.5...v16.3.6)

Updates `brace-expansion` from 1.1.18 to 1.1.21
- [Release notes](https://github.com/juliangruber/brace-expansion/releases)
- [Commits](https://github.com/juliangruber/brace-expansion/compare/v1.1.18...v1.1.21)

Updates `dompurify` from 3.4.14 to 3.4.16
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.4.14...3.4.16)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 16.3.6
  dependency-type: direct:production
  dependency-group: web-security
- dependency-name: brace-expansion
  dependency-version: 1.1.21
  dependency-type: indirect
  dependency-group: web-security
- dependency-name: dompurify
  dependency-version: 3.4.16
  dependency-type: indirect
  dependency-group: web-security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 11:16:00 +02:00
dependabot[bot] 4b03ba4e19 chore(deps): bump fast-uri (#169)
Bumps the mcp-security group with 1 update in the /src/McpServer/autopilot-monitor-mcp directory: [fast-uri](https://github.com/fastify/fast-uri).


Updates `fast-uri` from 3.1.7 to 3.1.8
- [Release notes](https://github.com/fastify/fast-uri/releases)
- [Commits](https://github.com/fastify/fast-uri/compare/v3.1.7...v3.1.8)

---
updated-dependencies:
- dependency-name: fast-uri
  dependency-version: 3.1.8
  dependency-type: indirect
  dependency-group: mcp-security
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 11:15:42 +02:00
dependabot[bot] f0aec031f3 chore(deps): bump the actions group with 2 updates (#168)
Bumps the actions group with 2 updates: [github/codeql-action/init](https://github.com/github/codeql-action) and [github/codeql-action/analyze](https://github.com/github/codeql-action).


Updates `github/codeql-action/init` from 4.38.0 to 4.38.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)

Updates `github/codeql-action/analyze` from 4.38.0 to 4.38.2
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b96794f015dfd88f77b49b1c93e0fa7110f94c63...2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2)

---
updated-dependencies:
- dependency-name: github/codeql-action/init
  dependency-version: 4.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: github/codeql-action/analyze
  dependency-version: 4.38.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-01 09:28:08 +02:00
Oliver Kieselbach 66208b3767 bump pointer: internal (rule notification placeholders, D-302)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 18:25:11 +02:00
Oliver Kieselbach c60e07dfd4 Rule notifications resolve {{token}} placeholders before they are sent
A stored rule result keeps the rule's template; only the web and the MCP
server resolved it. The notification builders sent the explanation as
stored, so a channel card showed {{reason}}, {{appName}},
{{failedSubcategory}} and {{errorCode}} although the session had recorded
all four values.

- RuleTemplateInterpolator: the backend copy of the resolution, applied in
  BuildRuleFiredAlert and AddRuleResultSections before the text is cut.
  In a notification a list item whose value was not recorded is dropped
  and a token in running text becomes an en dash.
- The cut lands on a line, sentence or word boundary instead of a
  character count; the limits are unchanged.
- tests/fixtures/rule-template-interpolation/cases.json: one case file
  run by the backend, the web and the MCP server.
- The engine's evidence auto-field list is one constant. The web, the MCP
  interpolator and the validate_rule token lint gain timeoutHours, which
  the engine already recorded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 18:12:02 +02:00
Oliver Kieselbach f564ee6a06 bump pointer: internal (hello wait timeout released)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 15:50:29 +02:00
Oliver Kieselbach ec025686ce bump pointer: internal (hello wait timeout plan, D-301)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 15:38:30 +02:00
Oliver Kieselbach 821c41e2ba Tenant Hello wait timeout up to 1 hour (#167)
* feat: tenant Hello wait timeout up to 1 hour

The "Hello Wait Timeout" setting (30..300 s in the UI) had no effect in the
V2 agent: its only consumer, HelloTracker.StartHelloWaitTimer, has no
production caller, while the decision engine armed a fixed 300 s
hello_safety deadline after the ESP exit and the tracker ran a fixed 300 s
completion timer after the wizard start.

The setting is now the real Hello budget, clamped to 300..3600 s
(HelloWaitTimeout in AutopilotMonitor.Shared). Every value that was valid
before (<= 300, including the old 30 s default) keeps today's 5 minutes.

- Engine: the agent stamps helloWaitTimeoutSeconds on EnrollmentFactsObserved
  (only when > 300), recorded set-once in EnrollmentScenarioObservations.
  All six hello_safety arm sites share one builder with the tenant window;
  a fact that arrives after hello_safety was armed moves it out. The WDP
  backstop is never shorter than the tenant window; the signal census
  surfaces hello_wait_extended.
- Agent: the Hello tracker's wizard completion timer uses the same budget;
  the stall probe skips session_stalled while hello_safety is pending.
- Backend: ValidateModel rejects values outside 300..3600; stored rows are
  read as their effective value, so legacy 30 s rows never block unrelated
  saves or backup reverts. Default 30 -> 300; shared manifest and wire types
  regenerated.
- Web: input 300..3600, bounds pinned against the C# constants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S17pipjWqjZ8ULAu1JrFyG

* fix: keep the Hello wait's 30 s minimum and default, raise only the maximum

Product decision: the setting keeps its UI contract (30 s minimum and
default, same description); only the maximum moves to 3600 s. Configured
values up to 300 s keep the built-in 5-minute Hello window, larger values
extend it.

- HelloWaitTimeout: MinSeconds/DefaultSeconds 30; BuiltInSeconds 300 is the
  floor of EffectiveSeconds.
- Backend validates only a changed value (30..3600), like the retention cap;
  the read-time normalization is gone again.
- The agent stamps the tenant window only above BuiltInSeconds.
- Web keeps the original text and default; only the maximum and its hint
  change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S17pipjWqjZ8ULAu1JrFyG

* fix: Hello wait default 300 s, stale-fire guard, deferred stall report

Product decision: default 300 s (what V2 has always waited), minimum 30 s
still accepted, maximum 3600 s. Values up to 300 s keep the built-in
5-minute window, which the settings hint now says.

Review follow-ups:
- Engine: a hello_safety fire whose incarnation was replaced (armed due
  later than the fire's due time) is dead-ended, same guard as RealmJoin —
  a late tenant window could otherwise be undercut by the queued fire of
  the built-in deadline. The tenant fact is recorded only when it extends
  the built-in window.
- Stall probe: session_stalled is held back while a tenant-extended Hello
  wait (hello_safety or the WDP backstop) is pending and emitted once it is
  over and the session is still idle; real activity drops it. Default
  tenants never hit the guard.
- Comments: WDP backstop wording, 5-min reference, AgentConfiguration.
- Web pins the default against the shared manifest; tracker constructor
  defaults use HelloWaitTimeout.DefaultSeconds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S17pipjWqjZ8ULAu1JrFyG

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-09-30 15:36:36 +02:00
Oliver Kieselbach ec3eaa6437 fix(web): sitemap and llms.txt list public pages at their canonical /page/ URL (#166) 2026-09-29 14:13:59 +02:00
Oliver Kieselbach d36c9a659d Merge branch 'main' of https://github.com/okieselbach/AutopilotMonitor 2026-09-29 12:02:55 +02:00
Oliver Kieselbach c27dbf2aa2 bump pointer: internal (client role gates plan)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:46:31 +02:00
Oliver Kieselbach 28443d11fa fix(web): don't send SignalR joins and bootstrap reads the backend refuses by role
Every visit by a user without a tenant role joined the tenant broadcast
group before the dashboard redirected to /progress, and the settings
area loaded the bootstrap session list for every role. The backend
refused both by rule: 134 AddToGroup 403s, 19 serial-less session
joins and 27 bootstrap-list 403s in 30 days, each one also an
api_request_failed event.

- lib/signalrGroupAccess mirrors the backend's same-tenant join rules
  (broadcast and member-notify groups need an own role or platform
  scope, the admin-notify group the Admin or a GA, a session group a
  serial for roleless callers, global-admins platform scope, an
  unparseable name is a 400). Cross-tenant joins stay with the
  backend's platform/delegated admission. The gate keys on
  hasOwnTenantOrPlatformRole, not hasTenantReadScope: a delegated admin
  without an own role is refused the home tenant's groups too.
- SignalRContext.joinGroup applies it before sending: a refused join
  reaches onDenied with the server's status and is never held for the
  reconnect rejoin. joinGroup depends on the caller's role fields as
  primitives, so its identity changes only when auth/me resolves, the
  same commit that delivers the tenant id consumers join with. That
  also ends the "tenant-" join the dashboard sent before the tenant id
  was known (the 400s).
- TenantConfigContext loads bootstrap sessions only for
  canManageBootstrapSessions (mirror of BootstrapManagerOrGA: GA, Admin,
  Operator with the bootstrap grant).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:46:05 +02:00
Oliver Kieselbach 0c41b0f449 bump pointer: .claude/commands (ux-insights rejoin baseline)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:22:44 +02:00
Oliver Kieselbach 7799fdb725 bump pointer: .claude/commands (platform-health baselines 2026-09-29)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:04:44 +02:00
dependabot[bot] 0c1a2a4192 chore(deps-dev): bump tsx (#165)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [tsx](https://github.com/privatenumber/tsx).


Updates `tsx` from 4.23.13 to 4.23.15
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.23.13...v4.23.15)

---
updated-dependencies:
- dependency-name: tsx
  dependency-version: 4.23.15
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: mcp
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-29 11:00:30 +02:00
Oliver Kieselbach 0356770ae8 bump pointer: internal (operator group customer label, D-300)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:58:39 +02:00
Oliver Kieselbach 3a9a074be1 Operator groups can carry a customer label; customers still never see the group name
Under D-267 a managed tenant saw every operator-provisioned access as
"Platform support". A customer in two operator groups run by different teams
got two identical entries under "Who can read this tenant" and could not tell
them apart.

A Global Admin can now give an operator group a customer label (at most 80
characters) with the existing PATCH global/tenant-groups/{groupId}, which takes
only the fields to change: name and/or customerLabel, where an empty label
clears it. The customer sees the label as the entry name; a group without one
and direct grants keep "Platform support". Owned (self-service) groups refuse a
label with 409, because their customers see the managing tenant's name.
RenameTenantGroupAsync becomes UpdateTenantGroupAsync, one read-modify-write on
the meta row.

DelegationSelfService.CustomerNameOf is now the one source of the name a
customer sees for a group. The managers view and the Group detail of the audit
rows written under every member tenant (assign, unassign, group delete) both use
it. Those audit rows carried the internal group name to the customer's audit
page before this change.

Admin, Tenant Groups: operator groups show "Customers see: ..." and get a
"Customer label" button next to Rename. The customer page is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:58:25 +02:00
Oliver Kieselbach a3870e7dcc bump pointer: internal (pre-provisioning seal + registration failure reason live)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 01:59:09 +02:00
350 changed files with 22897 additions and 2509 deletions
@@ -11,6 +11,7 @@
Identity comes from whatever `az login` established: the federated GitHub OIDC principal
in CI (azure/login, subject repo:okieselbach/AutopilotMonitor:ref:refs/heads/main), the
operator's own account for the local build. No secret is read, stored or passed around.
Under GitHub Actions the token is masked in the run log before it is returned.
x-ms-version is MANDATORY here, not decoration: Entra-authorized requests are rejected
below 2017-11-09, and Table requests without a SAS must carry the header at all. 2021-08-06
@@ -45,6 +46,15 @@ if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($token)) {
'on the table -- a missing role shows up later as HTTP 403 AuthorizationPermissionMismatch.')
}
# GitHub masks only secrets.* on its own; a token minted at run time is clear text in the
# (public) run log the moment anything prints it -- an uncaptured call of this script, or
# Get-Error / Format-List on a failed request, whose TargetObject carries this header.
# Write-Host, never Write-Output: the output stream is this script's return value. Only
# under Actions -- anywhere else the line would print the token instead of hiding it.
if ($env:GITHUB_ACTIONS -eq 'true') {
Write-Host "::add-mask::$token"
}
return @{
'Authorization' = "Bearer $token"
'x-ms-version' = '2021-08-06'
+2 -2
View File
@@ -114,7 +114,7 @@ jobs:
dotnet-version: '10.0.x'
- name: Initialize CodeQL
uses: github/codeql-action/init@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/init@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
@@ -125,6 +125,6 @@ jobs:
run: dotnet build AutopilotMonitor.sln --configuration Release
- name: Analyze
uses: github/codeql-action/analyze@b96794f015dfd88f77b49b1c93e0fa7110f94c63 # v4.38.0
uses: github/codeql-action/analyze@2892aa5e19bbd11bc0cff5427e3b750a04d9e3c2 # v4.38.2
with:
category: /language:${{ matrix.language }}
+24
View File
@@ -78,6 +78,23 @@ jobs:
working-directory: ${{ env.WEB_DIR }}
run: npm ci
# Star counter for the GitHub star link (utils/githubStars.ts). Baked into the static HTML
# so visitors' browsers never call GitHub; stale until the next web deploy. The token stays
# in this step, never in the build. Fail-soft: without a number the link has no counter.
- name: Read GitHub star count (fail-soft)
id: stars
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: |
count=$(gh api "repos/$REPO" --jq .stargazers_count 2>/dev/null || true)
if [[ "$count" =~ ^[0-9]+$ ]]; then
echo "count=$count" >> "$GITHUB_OUTPUT"
echo "GitHub stars: $count"
else
echo "::warning::GitHub star count unavailable - the star link renders without a counter"
fi
# next.config output:'export' -- `next build` writes the static site to out/
# (prebuild: sync-error-codes, generate-lastmod, generate-version, whats-new).
- name: Build
@@ -91,6 +108,8 @@ jobs:
NEXT_PUBLIC_ENTRA_REDIRECT_URI: ${{ vars.NEXT_PUBLIC_ENTRA_REDIRECT_URI }}
NEXT_PUBLIC_ENTRA_POST_LOGOUT_REDIRECT_URI: ${{ vars.NEXT_PUBLIC_ENTRA_POST_LOGOUT_REDIRECT_URI }}
NEXT_PUBLIC_API_BASE_URL: ${{ vars.NEXT_PUBLIC_API_BASE_URL }}
# Browsers load the live stats through it, and the build reads it once to bake the
# numbers of this moment into the landing page's static HTML (lib/platformStats.ts).
NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL: ${{ vars.NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL }}
NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING: ${{ secrets.NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING }}
# Build stamp for public/version.json (scripts/generate-version.js in prebuild);
@@ -98,10 +117,15 @@ jobs:
GITHUB_SHA: ${{ github.sha }}
# Docs checkout for public/whats-new.json; the prebuild fails in CI without it.
WHATS_NEW_DOCS_DIR: ${{ github.workspace }}/whats-new-docs
# Star counter from the step above; empty when the lookup failed.
NEXT_PUBLIC_GITHUB_STARS: ${{ steps.stars.outputs.count }}
run: |
npm run build
test -f out/version.json || { echo "out/version.json missing - export did not run"; exit 1; }
test -f out/whats-new.json || { echo "out/whats-new.json missing - whats-new prebuild did not run"; exit 1; }
# Fail-soft like the star count: browsers still load the stats live; only crawlers and
# LLM fetchers, which run no JS, see no numbers until the next deploy.
grep -q 'data-stats-snapshot=' out/index.html || echo "::warning::Platform stats were not baked into the landing page (reason in the build log) - crawlers and LLM fetchers see no numbers until the next deploy"
# One archive of the exact bytes that get uploaded, so the attestation has a single subject.
- name: Package export
+16 -3
View File
@@ -2,10 +2,10 @@
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-DEV-004",
"title": "Windows Update Failed During Enrollment",
"description": "Detects a Windows quality/cumulative update that FAILED to install during the OOBE / ESP enrollment window (WindowsUpdateClient EventID 20). A cumulative update failing mid-OOBE is a known cause of enrollment errors and is otherwise invisible in the Intune console.",
"description": "Detects a Windows quality/cumulative update that FAILED to install or download during the OOBE / ESP enrollment window (WindowsUpdateClient EventID 20 from the System log, 31 from the Operational channel). A cumulative update failing mid-OOBE is a known cause of enrollment errors and is otherwise invisible in the Intune console.",
"severity": "high",
"category": "device",
"version": "1.1.0",
"version": "1.2.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
@@ -18,10 +18,14 @@
"conditions": [
{
"signal": "wu_install_failed",
"source": "event_type",
"source": "event_data",
"eventType": "windows_update_failed",
"dataField": "wuEventId",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": true
},
{
@@ -31,6 +35,9 @@
"dataField": "updateTitle",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
@@ -40,6 +47,9 @@
"dataField": "hresult",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
@@ -49,6 +59,9 @@
"dataField": "hresultSymbol",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
+16 -3
View File
@@ -1385,10 +1385,10 @@
{
"ruleId": "ANALYZE-DEV-004",
"title": "Windows Update Failed During Enrollment",
"description": "Detects a Windows quality/cumulative update that FAILED to install during the OOBE / ESP enrollment window (WindowsUpdateClient EventID 20). A cumulative update failing mid-OOBE is a known cause of enrollment errors and is otherwise invisible in the Intune console.",
"description": "Detects a Windows quality/cumulative update that FAILED to install or download during the OOBE / ESP enrollment window (WindowsUpdateClient EventID 20 from the System log, 31 from the Operational channel). A cumulative update failing mid-OOBE is a known cause of enrollment errors and is otherwise invisible in the Intune console.",
"severity": "high",
"category": "device",
"version": "1.1.0",
"version": "1.2.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
@@ -1401,10 +1401,14 @@
"conditions": [
{
"signal": "wu_install_failed",
"source": "event_type",
"source": "event_data",
"eventType": "windows_update_failed",
"dataField": "wuEventId",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": true
},
{
@@ -1414,6 +1418,9 @@
"dataField": "updateTitle",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
@@ -1423,6 +1430,9 @@
"dataField": "hresult",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
@@ -1432,6 +1442,9 @@
"dataField": "hresultSymbol",
"operator": "exists",
"value": "",
"filterField": "wuPhase",
"filterOperator": "in",
"filterValue": "install,download",
"required": false
},
{
+2 -2
View File
@@ -203,7 +203,7 @@
"category": "currentPhase",
"pattern": "^\\[DO TEL\\] = (?<doTelJson>\\{.*\\})",
"action": "updateDoTelemetry",
"description": "Captures Delivery Optimization telemetry JSON after a DO download completes. Contains file size, peer caching stats, download mode, and duration. IME removed the [DO TEL] emitter in 1.104 (absent in 1.104.102.0 and 1.105.103.0 decompiles, present in 1.97.107.0); kept while 1.101/1.103 devices are in the fleet — retire once those versions are gone.",
"description": "Captures Delivery Optimization telemetry JSON after a DO download completes. Contains file size, peer caching stats, download mode, and duration. IME removed the [DO TEL] emitter in 1.104 (absent from the 1.104.102.0 and 1.105.103.0 builds, present in 1.97.107.0); kept while 1.101/1.103 devices are in the fleet — retire once those versions are gone.",
"enabled": true
},
{
@@ -584,7 +584,7 @@
"category": "always",
"pattern": "^Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104.",
"enabled": true
},
{
+1 -1
View File
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "^\\[DO TEL\\] = (?<doTelJson>\\{.*\\})",
"action": "updateDoTelemetry",
"description": "Captures Delivery Optimization telemetry JSON after a DO download completes. Contains file size, peer caching stats, download mode, and duration. IME removed the [DO TEL] emitter in 1.104 (absent in 1.104.102.0 and 1.105.103.0 decompiles, present in 1.97.107.0); kept while 1.101/1.103 devices are in the fleet — retire once those versions are gone.",
"description": "Captures Delivery Optimization telemetry JSON after a DO download completes. Contains file size, peer caching stats, download mode, and duration. IME removed the [DO TEL] emitter in 1.104 (absent from the 1.104.102.0 and 1.105.103.0 builds, present in 1.97.107.0); kept while 1.101/1.103 devices are in the fleet — retire once those versions are gone.",
"enabled": true
}
@@ -4,6 +4,6 @@
"category": "always",
"pattern": "^Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104.",
"enabled": true
}
@@ -41,15 +41,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
}
private GatherRuleContext Context(bool unrestricted = false, string? imeLogPathOverride = null)
=> new GatherRuleContext(
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
return new GatherRuleContext(
logger,
"sess", "tenant",
evt => _events.Add(evt),
imeLogPathOverride,
new LogFilePositionTracker())
new GatherRuleSessionState("sess", null, logger))
{
UnrestrictedMode = unrestricted
};
}
private static GatherRule Rule(string collectorType, string target,
Dictionary<string, string>? parameters = null) => new GatherRule
@@ -94,7 +94,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
public void Context_DebugLog_is_noop_without_writer()
{
var context = new GatherRuleContext(
_logger, "sess", "tenant", _ => { }, null, new LogFilePositionTracker());
_logger, "sess", "tenant", _ => { }, null, new GatherRuleSessionState("sess", null, _logger));
context.DebugLog("RULE-1", GatherRuleDebugLog.StageScope, "should not throw, should not write");
@@ -0,0 +1,220 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Newtonsoft.Json;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// An agent restart inside one enrollment session — a new <see cref="GatherRuleExecutor"/> on the
/// same state directory — continues where the previous run stopped: logparser rules do not
/// replay their log, on_change rules do not re-emit an unchanged result. Regression anchor:
/// session c8106021, where every restart replayed the whole HP Image Assistant log (70 events).
/// </summary>
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorRestartTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly TempDirectory _stateDir = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly List<GatherRuleExecutor> _executors = new List<GatherRuleExecutor>();
// %TEMP% lives under C:\Users (hard-blocked) — log targets sit beside the test assembly.
private readonly string _logDir = Path.Combine(
AppContext.BaseDirectory, "gather-restart-tests-" + Guid.NewGuid().ToString("N"));
public GatherRuleExecutorRestartTests()
{
Directory.CreateDirectory(_logDir);
}
public void Dispose()
{
foreach (var executor in _executors)
executor.Dispose();
_tmp.Dispose();
_stateDir.Dispose();
try { Directory.Delete(_logDir, recursive: true); } catch { /* best effort */ }
}
private string StateFile => Path.Combine(_stateDir.Path, GatherRuleStatePersistence.FileName);
private List<EnrollmentEvent> Events
{
get { lock (_eventsGate) return _events.ToList(); }
}
/// <summary>One agent run: a fresh executor whose startup rules run once. Not disposed — a hard kill.</summary>
private GatherRuleExecutor AgentRun(GatherRule rule, string sessionId = "session-a", string? stateDirectory = "default",
Action<EnrollmentEvent>? onEvent = null, string? debugLogPath = null)
{
var executor = new GatherRuleExecutor(
sessionId, "tenant",
evt =>
{
onEvent?.Invoke(evt);
lock (_eventsGate) _events.Add(evt);
},
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
debugLogPath: debugLogPath,
stateDirectory: stateDirectory == "default" ? _stateDir.Path : stateDirectory)
{
UnrestrictedMode = true
};
_executors.Add(executor);
executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(executor.WaitForStartupRules(30));
return executor;
}
private static GatherRule LogRule(string path, string id = "hpia-log-collect") => new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "logparser",
Target = path,
Parameters = new Dictionary<string, string> { ["pattern"] = "^(?<status>Update .+)$", ["format"] = "text" },
Trigger = "startup",
OutputEventType = "HPiA-UpdateStatus",
Enabled = true,
};
private static GatherRule OnChangeFileRule(string path) => new GatherRule
{
RuleId = "GATHER-OC-FILE",
Title = "file presence",
CollectorType = "file",
Target = path,
Trigger = "startup",
OutputEventType = "gather_file_presence",
Enabled = true,
EmitMode = "on_change",
};
private string WriteLog(string name, string content)
{
var path = Path.Combine(_logDir, name);
File.WriteAllText(path, content);
return path;
}
private int StatusEvents => Events.Count(e => e.EventType == "HPiA-UpdateStatus");
[Fact]
public void Restart_does_not_replay_the_log_and_new_lines_arrive_once()
{
var path = WriteLog("HP Image Assistant.log", "Update one\nUpdate two\nnoise\n");
AgentRun(LogRule(path));
Assert.Equal(2, StatusEvents);
AgentRun(LogRule(path)); // reboot, nothing new
Assert.Equal(2, StatusEvents);
File.AppendAllText(path, "Update three\n");
AgentRun(LogRule(path)); // next restart picks up only the new line
Assert.Equal(3, StatusEvents);
Assert.Equal("Update three", Events.Last(e => e.EventType == "HPiA-UpdateStatus").Data["status"]);
}
[Fact]
public void Position_is_on_disk_right_after_the_run_without_a_clean_shutdown()
{
var path = WriteLog("hardkill.log", "Update one\n");
AgentRun(LogRule(path)); // never disposed
var saved = JsonConvert.DeserializeObject<GatherRuleStateData>(File.ReadAllText(StateFile))!;
var position = Assert.Single(saved.LogPositions!);
Assert.Equal(new FileInfo(path).Length, position.Position);
Assert.Equal("session-a", saved.SessionId);
}
[Fact]
public void Position_is_saved_only_after_the_events_went_out()
{
var path = WriteLog("ordering.log", "Update one\n");
var savedDuringEmit = new List<bool>();
AgentRun(LogRule(path), onEvent: _ => savedDuringEmit.Add(File.Exists(StateFile)));
Assert.Equal(new[] { false }, savedDuringEmit);
Assert.True(File.Exists(StateFile));
}
[Fact]
public void A_new_session_reads_the_log_from_the_beginning()
{
var path = WriteLog("rotated.log", "Update one\nUpdate two\n");
AgentRun(LogRule(path), sessionId: "session-a");
AgentRun(LogRule(path), sessionId: "session-b"); // session rotated on the same device
Assert.Equal(4, StatusEvents);
}
[Fact]
public void A_log_replaced_during_the_reboot_is_read_from_the_beginning()
{
var path = WriteLog("replaced.log", "Update one\n");
AgentRun(LogRule(path, "replaced"));
// The tool writes a new log after the reboot; it is already longer than the old position.
File.WriteAllText(path, "Update fresh A\nUpdate fresh B\nUpdate fresh C\n");
AgentRun(LogRule(path, "replaced"));
Assert.Equal(new[] { "Update one", "Update fresh A", "Update fresh B", "Update fresh C" },
Events.Where(e => e.EventType == "HPiA-UpdateStatus").Select(e => (string)e.Data["status"]).ToArray());
}
[Fact]
public void An_unchanged_on_change_result_stays_silent_after_a_restart()
{
var watched = Path.Combine(_logDir, "marker.txt");
AgentRun(OnChangeFileRule(watched)); // absent → first result emits
AgentRun(OnChangeFileRule(watched)); // reboot, still absent → silent
Assert.Single(Events, e => e.EventType == "gather_file_presence");
File.WriteAllText(watched, "present");
AgentRun(OnChangeFileRule(watched)); // changed → emits
Assert.Equal(2, Events.Count(e => e.EventType == "gather_file_presence"));
}
[Fact]
public void Without_a_state_directory_every_run_starts_over_and_writes_nothing()
{
var path = WriteLog("diagnostic.log", "Update one\n");
AgentRun(LogRule(path), stateDirectory: null);
AgentRun(LogRule(path), stateDirectory: null);
Assert.Equal(2, StatusEvents);
Assert.False(File.Exists(StateFile));
}
[Fact]
public void Debug_trace_says_what_was_restored_and_where_reading_continues()
{
var path = WriteLog("traced.log", "Update one\n");
var trace = Path.Combine(_tmp.Path, "gather-rules-debug.log");
AgentRun(LogRule(path));
File.AppendAllText(path, "Update two\n");
AgentRun(LogRule(path), debugLogPath: trace);
var text = File.ReadAllText(trace);
Assert.Contains("gather state: restored 1 logparser position(s) for 1 rule(s)", text);
Assert.Contains("hpia-log-collect | config | logparser position restored: traced.log", text);
Assert.Contains("traced.log: continuing at position", text);
}
}
}
@@ -0,0 +1,278 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using Newtonsoft.Json;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// <see cref="GatherRuleSessionState"/> and <see cref="GatherRuleStatePersistence"/>: what is
/// written, what a restart restores, what it refuses, and when the file is (not) written.
/// </summary>
public sealed class GatherRuleSessionStateTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly TempDirectory _stateDir = new TempDirectory();
private readonly AgentLogger _logger;
public GatherRuleSessionStateTests()
{
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
}
public void Dispose()
{
_tmp.Dispose();
_stateDir.Dispose();
}
private string StateFile => Path.Combine(_stateDir.Path, GatherRuleStatePersistence.FileName);
private GatherRuleSessionState NewState(string sessionId = "session-a") =>
new GatherRuleSessionState(sessionId, new GatherRuleStatePersistence(_stateDir.Path, _logger), _logger);
private static LogFileHead Head(string text) => LogFileHead.FromBytes(Encoding.UTF8.GetBytes(text));
// ── restore ─────────────────────────────────────────────────────────
[Fact]
public void Positions_and_emitted_hashes_survive_a_restart_of_the_same_session()
{
var first = NewState();
first.CommitLogPosition("RULE-1", @"C:\Logs\a.log", 120, 7, Head("2026-09-28 a.log start"), heldTailStart: -1);
first.CommitEmittedHash("RULE-2", "hash-value");
var second = NewState();
var position = second.PeekLogPosition("RULE-1", @"C:\Logs\a.log");
Assert.NotNull(position);
Assert.Equal(120, position!.Position);
Assert.Equal(7, position.LineNumber);
Assert.True(position.FromPreviousRun);
Assert.Equal("hash-value", second.GetEmittedHashes()["RULE-2"]);
Assert.StartsWith("restored 1 logparser position(s) for 1 rule(s) and 1 on_change value(s)", second.RestoreSummary);
}
[Fact]
public void State_of_another_session_is_ignored()
{
NewState("session-a").CommitLogPosition("RULE-1", @"C:\Logs\a.log", 120, 7, Head("start"), -1);
var rotated = NewState("session-b");
Assert.Null(rotated.PeekLogPosition("RULE-1", @"C:\Logs\a.log"));
Assert.Contains("saved for session session-a, current session is session-b", rotated.RestoreSummary);
}
[Fact]
public void Missing_file_is_a_first_run_and_corrupt_file_a_fresh_start()
{
Assert.StartsWith("no saved state", NewState().RestoreSummary);
File.WriteAllText(StateFile, "not json {{{");
var afterCorruption = NewState();
Assert.StartsWith("saved state discarded", afterCorruption.RestoreSummary);
Assert.Empty(afterCorruption.GetEmittedHashes());
}
[Fact]
public void Tampered_entries_are_dropped_and_the_rest_restored()
{
var data = new GatherRuleStateData
{
SessionId = "session-a",
SavedAtUtc = DateTime.UtcNow,
LogPositions = new List<GatherLogPositionData>
{
new GatherLogPositionData { RuleId = "OK", Path = @"C:\a.log", Position = 10, LineNumber = 1, HeadLength = 10, HeadHash = "h" },
new GatherLogPositionData { RuleId = "NEG", Path = @"C:\b.log", Position = -5, HeadLength = 10, HeadHash = "h" },
new GatherLogPositionData { RuleId = "", Path = @"C:\c.log", Position = 1, HeadLength = 1, HeadHash = "h" },
new GatherLogPositionData { RuleId = "BIGHEAD", Path = @"C:\d.log", Position = 1, HeadLength = 5000, HeadHash = "h" },
new GatherLogPositionData { RuleId = "NOHEAD", Path = @"C:\e.log", Position = 50, HeadLength = 0 },
},
OnChangeHashes = new Dictionary<string, string> { ["GOOD"] = "x", ["EMPTY"] = "" },
};
File.WriteAllText(StateFile, JsonConvert.SerializeObject(data));
var state = NewState();
Assert.NotNull(state.PeekLogPosition("OK", @"C:\a.log"));
Assert.Null(state.PeekLogPosition("NEG", @"C:\b.log"));
Assert.Null(state.PeekLogPosition("BIGHEAD", @"C:\d.log"));
Assert.Null(state.PeekLogPosition("NOHEAD", @"C:\e.log"));
Assert.Equal(new[] { "GOOD" }, state.GetEmittedHashes().Keys.ToArray());
Assert.Contains("5 invalid entries dropped", state.RestoreSummary);
}
[Fact]
public void A_path_no_real_file_can_have_is_dropped_and_never_stops_the_executor()
{
// Path APIs throw on these characters under .NET Framework; a throw here would keep
// every collector host down at every restart of the session.
var data = new GatherRuleStateData
{
SessionId = "session-a",
LogPositions = new List<GatherLogPositionData>
{
new GatherLogPositionData { RuleId = "BAD", Path = "C:\\Logs\\x.|og", Position = 1, HeadLength = 1, HeadHash = "h" },
new GatherLogPositionData { RuleId = "WILD", Path = "C:\\Logs\\*.log", Position = 1, HeadLength = 1, HeadHash = "h" },
new GatherLogPositionData { RuleId = "OK", Path = "C:\\Logs\\ok.log", Position = 1, HeadLength = 1, HeadHash = "h" },
},
};
File.WriteAllText(StateFile, JsonConvert.SerializeObject(data));
var state = NewState();
Assert.NotNull(state.PeekLogPosition("OK", "C:\\Logs\\ok.log"));
Assert.Null(state.PeekLogPosition("BAD", "C:\\Logs\\x.|og"));
Assert.Contains("2 invalid entries dropped", state.RestoreSummary);
}
// ── where reading starts ────────────────────────────────────────────
[Fact]
public void Reading_continues_while_the_file_head_still_matches()
{
var state = NewState();
state.CommitLogPosition("R", @"C:\a.log", 30, 3, Head("2026-09-28 first line\n"), -1);
var start = GatherRuleSessionState.ResolveStart(state.PeekLogPosition("R", @"C:\a.log"), 200,
Head("2026-09-28 first line\nsecond line\nthird"), out var how);
Assert.Equal(30, start);
Assert.Equal(LogReadStart.Continue, how);
}
[Fact]
public void A_shrunk_file_and_a_replaced_file_are_read_from_the_beginning()
{
var state = NewState();
state.CommitLogPosition("R", @"C:\a.log", 30, 3, Head("2026-09-28 first line\n"), -1);
var known = state.PeekLogPosition("R", @"C:\a.log");
Assert.Equal(0, GatherRuleSessionState.ResolveStart(known, 20, Head("2026-09-28 fir"), out var shrunk));
Assert.Equal(LogReadStart.Shrunk, shrunk);
// Replaced and already longer than the old position: only the head tells.
Assert.Equal(0, GatherRuleSessionState.ResolveStart(known, 500, Head("2026-09-30 a new log entirely, much longer"), out var replaced));
Assert.Equal(LogReadStart.Replaced, replaced);
// Shorter than the recorded head although not shorter than the position.
state.CommitLogPosition("S", @"C:\b.log", 5, 1, Head("0123456789abcdef"), -1);
Assert.Equal(0, GatherRuleSessionState.ResolveStart(state.PeekLogPosition("S", @"C:\b.log"), 8, Head("01234567"), out var cut));
Assert.Equal(LogReadStart.Replaced, cut);
}
[Fact]
public void Positions_are_kept_per_rule_for_the_same_file()
{
var state = NewState();
state.CommitLogPosition("RULE-A", @"C:\shared.log", 100, 10, Head("x"), -1);
Assert.Null(state.PeekLogPosition("RULE-B", @"C:\shared.log"));
Assert.NotNull(state.PeekLogPosition("RULE-A", @"C:\SHARED.LOG")); // paths compare case-insensitively
}
// ── when the file is written ────────────────────────────────────────
[Fact]
public void An_unchanged_commit_does_not_write_the_file()
{
var state = NewState();
state.CommitLogPosition("R", @"C:\a.log", 10, 1, Head("head"), -1);
File.Delete(StateFile);
state.CommitLogPosition("R", @"C:\a.log", 10, 1, Head("head"), -1);
state.CommitEmittedHash("H", "v");
File.Delete(StateFile);
state.CommitEmittedHash("H", "v");
Assert.False(File.Exists(StateFile));
}
[Fact]
public void A_failed_save_is_retried_at_the_next_commit()
{
var blocked = Path.Combine(_stateDir.Path, "blocked");
File.WriteAllText(blocked, "a file where the state directory should be");
var state = new GatherRuleSessionState("session-a", new GatherRuleStatePersistence(blocked, _logger), _logger);
state.CommitEmittedHash("H", "v"); // Directory.CreateDirectory fails
File.Delete(blocked);
state.CommitEmittedHash("H", "v"); // unchanged, but the earlier save is still owed
Assert.True(File.Exists(Path.Combine(blocked, GatherRuleStatePersistence.FileName)));
}
[Fact]
public void A_closed_state_no_longer_writes()
{
var state = NewState();
state.Close();
state.CommitEmittedHash("H", "v");
Assert.False(File.Exists(StateFile));
}
[Fact]
public void Without_persistence_nothing_is_written()
{
var state = new GatherRuleSessionState("session-a", null, _logger);
state.CommitLogPosition("R", @"C:\a.log", 10, 1, Head("head"), -1);
Assert.StartsWith("persistence off", state.RestoreSummary);
Assert.Empty(Directory.GetFiles(_stateDir.Path));
}
// ── bounds and concurrency ──────────────────────────────────────────
[Fact]
public void A_rule_forgets_its_least_recently_touched_file_beyond_the_cap()
{
var state = NewState();
for (var i = 0; i < GatherRuleSessionState.MaxLogFilesPerRule; i++)
state.CommitLogPosition("R", $@"C:\logs\{i}.log", 1, 1, Head("h"), -1);
state.PeekLogPosition("R", @"C:\logs\0.log"); // touched again — survives
state.CommitLogPosition("R", @"C:\logs\new.log", 1, 1, Head("h"), -1);
Assert.NotNull(state.PeekLogPosition("R", @"C:\logs\0.log"));
Assert.Null(state.PeekLogPosition("R", @"C:\logs\1.log"));
Assert.NotNull(state.PeekLogPosition("R", @"C:\logs\new.log"));
}
[Fact]
public void Parallel_commits_leave_a_consistent_file()
{
var state = NewState();
Parallel.For(0, 8, worker =>
{
for (var i = 0; i < 50; i++)
{
state.CommitLogPosition($"RULE-{worker}", $@"C:\logs\{worker}.log", i + 1, i, Head("h"), -1);
state.PeekLogPosition($"RULE-{worker}", $@"C:\logs\{worker}.log");
state.CommitEmittedHash($"RULE-{worker}", "hash-" + i);
}
});
var restored = NewState();
for (var worker = 0; worker < 8; worker++)
{
Assert.Equal(50, restored.PeekLogPosition($"RULE-{worker}", $@"C:\logs\{worker}.log")!.Position);
Assert.Equal("hash-49", restored.GetEmittedHashes()[$"RULE-{worker}"]);
}
}
}
}
@@ -0,0 +1,188 @@
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// <see cref="LogLineReader"/>, <see cref="LogTextEncoding"/> and <see cref="LogFileHead"/>:
/// exact byte positions, StreamReader-compatible line ends, byte order marks only at offset 0,
/// and the code-unit view of UTF-16/UTF-32 files.
/// </summary>
public sealed class LogLineReaderTests
{
private static byte[] Bytes(Encoding encoding, string text, bool withBom = true)
{
var body = encoding.GetBytes(text);
return withBom ? encoding.GetPreamble().Concat(body).ToArray() : body;
}
private static (List<string> Lines, List<long> EndPositions, List<bool> Terminated) ReadAll(
byte[] file, long start = 0, int maxLineBytes = 1024)
{
var encoding = LogFileHead.FromBytes(file.Take(LogFileHead.MaxBytes).ToArray()).Encoding;
using (var stream = new MemoryStream(file))
{
stream.Seek(start, SeekOrigin.Begin);
var reader = new LogLineReader(stream, encoding, maxLineBytes);
var lines = new List<string>();
var ends = new List<long>();
var terminated = new List<bool>();
string line;
while ((line = reader.ReadLine()) != null)
{
lines.Add(line);
ends.Add(reader.Position);
terminated.Add(reader.LastLineTerminated);
}
return (lines, ends, terminated);
}
}
[Fact]
public void Line_ends_match_StreamReader_including_a_lone_carriage_return()
{
var file = Encoding.UTF8.GetBytes("one\ntwo\r\nthree\rfour");
var result = ReadAll(file);
Assert.Equal(new[] { "one", "two", "three", "four" }, result.Lines);
Assert.Equal(new long[] { 4, 9, 15, 19 }, result.EndPositions);
Assert.Equal(new[] { true, true, true, false }, result.Terminated);
}
[Fact]
public void Position_after_each_line_is_the_offset_to_resume_from()
{
var file = Encoding.UTF8.GetBytes("alpha\nbeta\ngamma\n");
var first = ReadAll(file);
var resumed = ReadAll(file, start: first.EndPositions[0]);
Assert.Equal(new[] { "beta", "gamma" }, resumed.Lines);
Assert.Equal(file.Length, resumed.EndPositions.Last());
}
[Fact]
public void Utf8_byte_order_mark_is_skipped_only_at_offset_0()
{
var file = Bytes(Encoding.UTF8, "first\nsecond\n");
var result = ReadAll(file);
Assert.Equal(new[] { "first", "second" }, result.Lines);
Assert.Equal(LogTextEncoding.Utf8WithBom, LogFileHead.FromBytes(file).Encoding);
}
[Theory]
[InlineData("utf-16le")]
[InlineData("utf-16be")]
[InlineData("utf-32le")]
[InlineData("utf-32be")]
public void Unicode_files_are_read_in_their_code_units_from_any_resume_point(string name)
{
var encoding = name == "utf-16le" ? Encoding.Unicode
: name == "utf-16be" ? Encoding.BigEndianUnicode
: name == "utf-32le" ? Encoding.UTF32
: new UTF32Encoding(bigEndian: true, byteOrderMark: true);
var file = Bytes(encoding, "ERROR first\r\nERROR zweite Zeile mit Umlaut äöü\r\n");
var whole = ReadAll(file);
var resumed = ReadAll(file, start: whole.EndPositions[0]);
Assert.Equal(name, LogFileHead.FromBytes(file).Encoding.Name);
Assert.Equal(new[] { "ERROR first", "ERROR zweite Zeile mit Umlaut äöü" }, whole.Lines);
Assert.Equal(new[] { "ERROR zweite Zeile mit Umlaut äöü" }, resumed.Lines);
Assert.Equal(file.Length, whole.EndPositions.Last());
}
[Fact]
public void File_without_byte_order_mark_is_utf8_and_ansi_bytes_decode_as_before()
{
var file = new byte[] { (byte)'M', (byte)'a', (byte)'n', 0xFC, (byte)'\n' }; // "Man" + Windows-1252 'ü'
var result = ReadAll(file);
Assert.Equal(LogTextEncoding.Utf8, LogFileHead.FromBytes(file).Encoding);
Assert.Equal(new StreamReader(new MemoryStream(file), Encoding.UTF8).ReadLine(), result.Lines.Single());
}
[Fact]
public void Carriage_return_at_end_of_file_ends_the_line_and_its_late_line_feed_is_skipped()
{
var firstWrite = Encoding.UTF8.GetBytes("done\r");
var later = Encoding.UTF8.GetBytes("done\r\nnext\n");
var first = ReadAll(firstWrite);
var resumed = ReadAll(later, start: first.EndPositions.Last());
Assert.Equal(new[] { "done" }, first.Lines);
Assert.True(first.Terminated.Single());
Assert.Equal(new[] { "next" }, resumed.Lines);
}
[Fact]
public void Overlong_line_is_consumed_without_being_kept_and_reading_continues_behind_it()
{
var file = Encoding.UTF8.GetBytes(new string('x', 50) + "\nshort\n");
using (var stream = new MemoryStream(file))
{
var reader = new LogLineReader(stream, LogTextEncoding.Utf8, maxLineBytes: 10);
Assert.Equal(string.Empty, reader.ReadLine());
Assert.True(reader.LastLineTruncated);
Assert.Equal(51, reader.Position);
Assert.Equal("short", reader.ReadLine());
Assert.False(reader.LastLineTruncated);
Assert.Null(reader.ReadLine());
}
}
[Fact]
public void Line_spanning_many_read_buffers_is_returned_whole()
{
var longLine = new string('y', 200_000);
var file = Encoding.UTF8.GetBytes(longLine + "\nend\n");
var result = ReadAll(file, maxLineBytes: 1_000_000);
Assert.Equal(new[] { longLine, "end" }, result.Lines);
}
[Fact]
public void Half_written_utf16_character_at_end_of_file_stays_in_the_unterminated_line()
{
var full = Bytes(Encoding.Unicode, "ok\r\nhalf");
var file = full.Concat(new byte[] { 0x41 }).ToArray(); // first byte of the next UTF-16 unit
var result = ReadAll(file);
Assert.Equal("ok", result.Lines[0]);
Assert.False(result.Terminated.Last());
Assert.Equal(file.Length, result.EndPositions.Last());
}
[Fact]
public void AlignDown_snaps_to_code_units_behind_the_byte_order_mark()
{
Assert.Equal(0, LogTextEncoding.Utf16LittleEndian.AlignDown(1));
Assert.Equal(2, LogTextEncoding.Utf16LittleEndian.AlignDown(3));
Assert.Equal(6, LogTextEncoding.Utf16LittleEndian.AlignDown(7));
Assert.Equal(8, LogTextEncoding.Utf32LittleEndian.AlignDown(11));
Assert.Equal(7, LogTextEncoding.Utf8.AlignDown(7));
}
[Fact]
public void Head_hash_of_a_prefix_matches_the_hash_of_the_shorter_file()
{
var shortFile = LogFileHead.FromBytes(Encoding.UTF8.GetBytes("2026-09-28 start"));
var grown = LogFileHead.FromBytes(Encoding.UTF8.GetBytes("2026-09-28 start\nmore lines"));
Assert.Equal(shortFile.Hash(shortFile.Length), grown.Hash(shortFile.Length));
Assert.NotEqual(shortFile.Hash(shortFile.Length), grown.Hash(grown.Length));
}
}
}
@@ -37,7 +37,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
logger, "sess", "tenant",
evt => _events.Add(evt),
null,
new LogFilePositionTracker(),
new GatherRuleSessionState("sess", null, logger),
new GatherRuleDebugLog(_debugLogPath, logger))
{
UnrestrictedMode = true
@@ -0,0 +1,225 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Reading semantics of <see cref="LogParserCollector"/> across runs on one context: every
/// line of a growing log is matched exactly once — no line lost at a maxLines cut, no line
/// split by a writer that is mid-line, UTF-16 logs readable after the first run, positions
/// kept per rule, line numbers counted from the start of the file.
/// </summary>
public sealed class LogParserCollectorReadingTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly GatherRuleContext _context;
// %TEMP% lives under C:\Users (hard-blocked) — parse targets sit beside the
// test assembly instead, admitted via UnrestrictedMode.
private readonly string _logDir = Path.Combine(
AppContext.BaseDirectory, "gather-reading-tests-" + Guid.NewGuid().ToString("N"));
public LogParserCollectorReadingTests()
{
Directory.CreateDirectory(_logDir);
_context = NewContext();
}
public void Dispose()
{
_tmp.Dispose();
try { Directory.Delete(_logDir, recursive: true); } catch { /* best effort */ }
}
private GatherRuleContext NewContext()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
return new GatherRuleContext(logger, "sess", "tenant",
evt => _events.Add(evt), null, new GatherRuleSessionState("sess", null, logger))
{
UnrestrictedMode = true
};
}
private string LogPath(string name) => Path.Combine(_logDir, name);
private static GatherRule TextRule(string ruleId, string target, string pattern,
string trigger = "interval", int? maxLines = null)
{
var parameters = new Dictionary<string, string> { ["pattern"] = pattern, ["format"] = "text" };
if (maxLines.HasValue) parameters["maxLines"] = maxLines.Value.ToString();
return new GatherRule
{
RuleId = ruleId,
Title = ruleId,
CollectorType = "logparser",
Target = target,
Parameters = parameters,
Trigger = trigger,
IntervalSeconds = trigger == "interval" ? 30 : (int?)null,
OutputEventType = "logparser_reading_test",
};
}
private List<string> Captured(string group) =>
_events.Select(e => e.Data.TryGetValue(group, out var v) ? (string)v : "<missing>").ToList();
[Fact]
public void Two_rules_on_one_file_each_read_the_whole_file()
{
var path = LogPath("shared.log");
File.WriteAllText(path, "ERROR one\nWARN two\nERROR three\n");
var collector = new LogParserCollector();
collector.Execute(TextRule("RULE-ERRORS", path, "^ERROR (?<what>\\w+)"), _context);
collector.Execute(TextRule("RULE-WARNINGS", path, "^WARN (?<what>\\w+)"), _context);
Assert.Equal(2, _events.Count(e => (string)e.Data["ruleId"] == "RULE-ERRORS"));
Assert.Single(_events, e => (string)e.Data["ruleId"] == "RULE-WARNINGS");
}
[Fact]
public void MaxLines_cut_loses_no_line_across_runs()
{
var path = LogPath("long.log");
var sb = new StringBuilder();
for (var i = 1; i <= 30; i++)
sb.Append("hit ").Append(i.ToString("D3")).Append(' ').Append('x', 90).Append('\n');
File.WriteAllText(path, sb.ToString());
var rule = TextRule("RULE-LONG", path, "^hit (?<n>\\d{3})", maxLines: 10);
var collector = new LogParserCollector();
for (var run = 0; run < 6; run++)
collector.Execute(rule, _context);
var expected = Enumerable.Range(1, 30).Select(i => i.ToString("D3")).ToList();
Assert.Equal(expected, Captured("n"));
}
[Fact]
public void Line_written_in_two_parts_is_matched_once_and_whole()
{
var path = LogPath("partial.log");
File.WriteAllText(path, "ERR");
var rule = TextRule("RULE-PARTIAL", path, "^ERROR (?<code>0x[0-9A-F]+)$");
var collector = new LogParserCollector();
collector.Execute(rule, _context); // the writer is mid-line
File.AppendAllText(path, "OR 0x1F\n");
collector.Execute(rule, _context);
Assert.Equal(new List<string> { "0x1F" }, Captured("code"));
}
[Fact]
public void Interval_rule_reads_a_line_still_unfinished_on_its_second_run_once()
{
var path = LogPath("never-terminated.log");
File.WriteAllText(path, "ERROR 0x2A");
var rule = TextRule("RULE-UNFINISHED", path, "^ERROR (?<code>0x[0-9A-F]+)$");
var collector = new LogParserCollector();
collector.Execute(rule, _context); // held: the writer may still be in the line
collector.Execute(rule, _context); // still unfinished — taken as it is
collector.Execute(rule, _context); // nothing new
Assert.Equal(new List<string> { "0x2A" }, Captured("code"));
}
[Fact]
public void One_shot_rule_reads_an_unfinished_line_now_and_whole_on_its_next_run()
{
var path = LogPath("startup.log");
File.WriteAllText(path, "ok\nERR");
var rule = TextRule("RULE-STARTUP", path, "^ERR(?<rest>.*)$", trigger: "startup");
var collector = new LogParserCollector();
collector.Execute(rule, _context); // agent start: the writer is mid-line
File.AppendAllText(path, "OR 0x1F\n");
collector.Execute(rule, _context); // next agent start
// Read twice, never split: the second read sees the whole line.
Assert.Equal(new List<string> { "", "OR 0x1F" }, Captured("rest"));
Assert.Equal(new long[] { 2, 2 }, _events.Select(e => Convert.ToInt64(e.Data["logLineNumber"])).ToArray());
}
[Fact]
public void Line_feed_written_after_a_trailing_carriage_return_is_consumed_without_a_line()
{
var path = LogPath("cr-at-end.log");
File.WriteAllText(path, "hit\r");
var rule = TextRule("RULE-CR", path, "^hit$");
var collector = new LogParserCollector();
collector.Execute(rule, _context);
File.AppendAllText(path, "\n");
collector.Execute(rule, _context);
Assert.Single(_events);
var position = _context.SessionState.PeekLogPosition("RULE-CR", path);
Assert.Equal(new FileInfo(path).Length, position!.Position);
Assert.Equal(1, position.LineNumber);
}
[Fact]
public void Utf16_log_appended_after_the_first_run_is_still_matched()
{
var path = LogPath("transcript.log");
File.WriteAllText(path, "ERROR first\r\n", Encoding.Unicode); // UTF-16 LE with BOM
var rule = TextRule("RULE-UTF16", path, "^ERROR (?<what>\\w+)$");
var collector = new LogParserCollector();
collector.Execute(rule, _context);
using (var stream = new FileStream(path, FileMode.Append, FileAccess.Write))
{
var bytes = new UnicodeEncoding(bigEndian: false, byteOrderMark: false).GetBytes("ERROR second\r\n");
stream.Write(bytes, 0, bytes.Length);
}
collector.Execute(rule, _context);
Assert.Equal(new List<string> { "first", "second" }, Captured("what"));
}
[Theory]
[InlineData("abc")]
[InlineData("0")]
[InlineData("-5")]
public void MaxLines_that_is_no_positive_number_falls_back_to_the_default(string maxLines)
{
var path = LogPath("misconfigured.log");
File.WriteAllText(path, "hit 1\nhit 2\n");
var rule = TextRule("RULE-MISCONFIGURED", path, "^hit (?<n>\\d)$");
rule.Parameters["maxLines"] = maxLines;
new LogParserCollector().Execute(rule, _context);
Assert.Equal(new List<string> { "1", "2" }, Captured("n"));
}
[Fact]
public void Line_numbers_count_from_the_start_of_the_file()
{
var path = LogPath("numbers.log");
File.WriteAllText(path, "a\nhit\nb\n");
var rule = TextRule("RULE-NUMBERS", path, "^hit$");
var collector = new LogParserCollector();
collector.Execute(rule, _context);
File.AppendAllText(path, "c\nhit\n");
collector.Execute(rule, _context);
Assert.Equal(new long[] { 2, 5 }, _events.Select(e => Convert.ToInt64(e.Data["logLineNumber"])).ToArray());
}
}
}
@@ -35,7 +35,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
tenantId: "tenant",
onEventCollected: _ => { },
imeLogPathOverride: null,
filePositionTracker: new LogFilePositionTracker())
sessionState: new GatherRuleSessionState("sess", null, logger))
{
// Bypass the registry allowlist guard for the synthetic test path.
UnrestrictedMode = true
@@ -0,0 +1,71 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
{
/// <summary>
/// The completion timer a started Hello wizard runs against is the tenant's Hello wait
/// (<c>HelloWaitTimeoutSeconds</c>), resolved to 300..3600 s — the same budget the engine's
/// hello_safety window spends from the ESP exit, so a tenant allowing an hour is not cut off
/// by a fixed 5-min tracker timer while the user is still inside the wizard.
/// </summary>
public sealed class HelloTrackerCompletionBudgetTests
{
private static readonly DateTime Fixed = new DateTime(2026, 9, 29, 8, 0, 0, DateTimeKind.Utc);
[Theory]
[InlineData(30, 300)] // the old default, still stored for many tenants → the built-in window
[InlineData(0, 300)]
[InlineData(300, 300)]
[InlineData(1800, 1800)]
[InlineData(3600, 3600)]
[InlineData(86400, 3600)] // clamped to the ceiling
public void Completion_budget_is_the_resolved_tenant_hello_wait(int configured, int expected)
{
using var tmp = new TempDirectory();
using var tracker = NewTracker(tmp, new FakeSignalIngressSink(), configured);
Assert.Equal(expected, tracker.CompletionTimeoutSecondsForTest);
}
[Fact]
public void Completion_timeout_event_reports_the_tenant_budget()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
using var tracker = NewTracker(tmp, ingress, helloWaitTimeoutSeconds: 2400);
tracker.NotifyHelloWizardStarted();
Assert.True(tracker.IsCompletionTimerActiveForTest);
tracker.TriggerCompletionTimeoutForTest();
var timeout = ingress.Posted.Single(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.HelloCompletionTimeout);
var data = Assert.IsAssignableFrom<IDictionary<string, object>>(timeout.TypedPayload);
Assert.Equal(2400, data["timeoutSeconds"]);
Assert.Equal("timeout", tracker.HelloOutcome);
}
private static HelloTracker NewTracker(TempDirectory tmp, FakeSignalIngressSink ingress, int helloWaitTimeoutSeconds) =>
new HelloTracker(
sessionId: "S1",
tenantId: "T1",
post: new InformationalEventPost(ingress, new VirtualClock(Fixed)),
logger: new AgentLogger(tmp.Path, AgentLogLevel.Info),
helloWaitTimeoutSeconds: helloWaitTimeoutSeconds);
}
}
@@ -337,7 +337,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
"Launch powershell executor in machine session",
new[] { "context=machine" }
};
// Wording from the IME decompile (ContentDownloaderDeliveryOptimization, identical 1.97–1.105).
// IME wording, identical from 1.97 to 1.105.
yield return new object[]
{
"IME-DO-TIMEOUT-2",
@@ -140,6 +140,22 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
Assert.Equal(1, fired); // the persisted flag suppressed the second emission
}
[Fact]
public void Registry_reconciliation_counters_survive_a_restart()
{
using var h = new Harness();
h.Tracker.ScriptRuns.CountVerified();
h.Tracker.ScriptRuns.CountVerified();
h.Tracker.ScriptRuns.CountCorrected();
h.Tracker.SaveStateForTest();
h.Restart();
var health = h.Tracker.GetHealthSnapshot();
Assert.Equal(2, health.PlatformOutputVerified);
Assert.Equal(1, health.PlatformOutputCorrected);
}
[Fact]
public async Task Handler_time_does_not_count_against_the_line_budget()
{
@@ -0,0 +1,368 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// Session 756970cf: IME launched a Win32 detection executor and a platform-script executor 15 ms apart, and both
/// opened AgentExecutor.log at the same end-of-file offset. The platform executor's shorter start block overwrote
/// the detection executor's, whose identity line the tracker never saw; the detection's end block (exit 1 and its
/// stdout), written at its own position behind its own start block, then sat behind the PLATFORM start block, and
/// "the nearest preceding marker owns the line" handed another script's output and exit code to the platform
/// script. These tests pin the writer boundary: an entry that is not exactly one CMTrace record is where one
/// writer's bytes end and another's begin, and nothing behind it belongs to the invocation in front of it.
/// </summary>
public sealed partial class ImeLogTrackerOverwriteRewindTests
{
private const string RaceUserId = "00000000-0000-0000-0000-000000000000";
private const string RacePlatformId = "1d6f0a52-3c4e-4b7a-9e21-5f8c7d2b9a10";
private const string RaceDetectionAppId = "8b2e4c71-6a9d-4f03-b5e8-2c7a1d9f4e36";
private const string RaceFillerAppId = "5e1c2b7d-0a4f-4c8e-9d36-7b2f1e8a4c50";
private const string ImeDir = @"C:\Program Files (x86)\Microsoft Intune Management Extension";
private const string DetectionStdout = "VERBOSE: === Sample Suite Detection Started ===\r\nWARNING: === DETECTION FAILED: Sample Suite is not installed ===";
private const string PlatformStdout = "=== bootstrap loader started ===\r\nInstaller finished (exit code 0).";
private static string EntryAt(string time, string message)
=> $"<![LOG[{message}]LOG]!><time=\"{time}\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">\r\n";
/// <summary>The four lines every AgentExecutor invocation starts with — identical length for every executor.</summary>
private static string ExecutorBanner(string time)
=> EntryAt(time, "ExecutorLog AgentExecutor gets invoked")
+ EntryAt(time, "Creating command line parser, name delimiter is - and value separator is .")
+ EntryAt(time, "Getting Ordered Parameters")
+ EntryAt(time, "Parsing Ordered Parameters.");
private static string ExecutorLaunch(string time, string scriptPath, string pid)
=> EntryAt(time, "Prepare to run Powershell Script ..")
+ EntryAt(time, $"cmd line for running powershell is -NoProfile -executionPolicy bypass -file \"{scriptPath}\" ")
+ EntryAt(time, "runAs32BitOn64 = False, so Disable Wow64FsRedirection")
+ EntryAt(time, @"PowerShell path is C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe")
+ EntryAt(time, $"[Executor] created powershell with process id {pid}");
private static string PlatformStartBlock(string time, string policyId, string pid)
{
var path = $@"{ImeDir}\Policies\Scripts\{RaceUserId}_{policyId}.ps1";
var results = $@"{ImeDir}\Policies\Results\{RaceUserId}_{policyId}";
return ExecutorBanner(time)
+ EntryAt(time, $"Adding argument powershell with value {path} to the named argument list.")
+ EntryAt(time, "Powershell option gets invoked")
+ EntryAt(time, path)
+ EntryAt(time, results + ".output")
+ EntryAt(time, results + ".error")
+ EntryAt(time, results + ".timeout")
+ ExecutorLaunch(time, path, pid);
}
private static string DetectionStartBlock(string time, string appId, string pid)
{
var path = $@"{ImeDir}\Content\DetectionScripts\{appId}_2.ps1";
return ExecutorBanner(time)
+ EntryAt(time, $"Adding argument powershellDetection with value {path} to the named argument list.")
+ EntryAt(time, "PowershellDetection option gets invoked")
+ EntryAt(time, path)
+ EntryAt(time, path + "quotedResultFilePath.txt")
+ EntryAt(time, path + "quotedErrorFilePath.txt")
+ EntryAt(time, path + "quotedTimeoutFilePath.txt")
+ EntryAt(time, path + "quotedExitCodeFilePath.txt")
+ ExecutorLaunch(time, path, pid);
}
private static string ExecutorEndBlock(string time, int exitCode, string stdout)
=> EntryAt(time, $"Powershell exit code is {exitCode}")
+ EntryAt(time, $"length of out={stdout.Length}")
+ EntryAt(time, "length of error=0")
+ EntryAt(time, "error from script =")
+ EntryAt(time, exitCode == 0 ? "Powershell script is successfully executed." : "Powershell script is failed to execute")
+ EntryAt(time, $"write output done. output = {stdout}, error = ")
+ EntryAt(time, "Revert Wow64FsRedirection")
+ EntryAt(time, "Agent executor completed.");
private static string PlatformResultLine(string policyId)
=> $"[PowerShell] User Id = {RaceUserId}, Policy id = {policyId}, policy result = Success";
/// <summary>The file offsets at which the entries of an ASCII <paramref name="block"/> start when it is written at <paramref name="at"/>.</summary>
private static HashSet<long> EntryStarts(string block, long at)
{
Assert.True(block.All(c => c < 128), "offset arithmetic assumes ASCII test content");
var starts = new HashSet<long>();
for (var i = block.IndexOf("<![LOG[", StringComparison.Ordinal); i >= 0; i = block.IndexOf("<![LOG[", i + 1, StringComparison.Ordinal))
starts.Add(at + i);
return starts;
}
[Fact]
public async Task Executor_that_opened_at_the_same_offset_never_lends_its_end_block_to_the_platform_script()
{
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
// An earlier, finished detection run: the race starts mid-file like in the field log.
h.Append(DetectionStartBlock("12:00:00.0000000", RaceFillerAppId, "1548") + ExecutorEndBlock("12:00:00.5000000", 0, "detected"));
await h.Pass();
var x = h.Length;
// Detection executor C and platform executor D open the file at the same end x. C writes first; the tracker
// reads C's banner, then D writes its own start block over C's.
var cStart = DetectionStartBlock("12:00:01.0000000", RaceDetectionAppId, "6416");
var dStart = PlatformStartBlock("12:00:01.2000000", RacePlatformId, "2524");
var cHead = cStart.Substring(0, cStart.IndexOf("<![LOG[Getting Ordered Parameters", StringComparison.Ordinal));
h.Append(cHead);
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Append(cStart.Substring(cHead.Length));
h.Overwrite(x, dStart);
// The field geometry: D's start block is shorter and ends inside one of C's lines, so the tail of that line
// sits between D's start block and C's remaining bytes.
Assert.True(dStart.Length < cStart.Length);
Assert.DoesNotContain(x + dStart.Length, EntryStarts(cStart, x));
h.Now = h.Now.AddMilliseconds(150);
await h.Pass();
Assert.True(h.Health.OverwriteRewinds >= 1);
// C ends: its end block lands at its own position, right behind its own (overwritten) start block.
Assert.Equal(x + cStart.Length, h.Length);
var cEnd = ExecutorEndBlock("12:00:07.0000000", 1, DetectionStdout);
h.Append(cEnd);
h.Now = h.Now.AddSeconds(6);
await h.Pass();
// IME's result for D reaches the tracker before D's own end block: the completion must wait for it.
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(RacePlatformId), sourceFileName: "IntuneManagementExtension.log");
Assert.Empty(h.Completed);
// D ends: its end block lands at ITS position, over the fragment and C's bytes; C's last line survives.
var dEnd = ExecutorEndBlock("12:00:09.5000000", 0, PlatformStdout);
Assert.True(dEnd.Length < cStart.Length - dStart.Length + cEnd.Length);
h.Overwrite(x + dStart.Length, dEnd);
h.Now = h.Now.AddMilliseconds(130);
await h.Pass();
h.Tracker.FlushPendingPlatformScriptResults(h.Now);
var script = Assert.Single(h.Completed);
Assert.Equal(RacePlatformId, script.PolicyId);
Assert.Equal("Success", script.Result);
Assert.Equal(0, script.ExitCode);
Assert.Equal(PlatformStdout.Replace("\r\n", "\n"), script.Stdout);
Assert.DoesNotContain(h.Completed, s => (s.Stdout ?? string.Empty).Contains("Detection"));
// The run is known to the registry reconciliation under the user id of its own lines (D-316).
Assert.Equal(RaceUserId, script.UserId);
Assert.False(string.IsNullOrEmpty(script.RunId));
h.Tracker.ScriptRuns.NotePassCompleted(DateTime.MaxValue.AddDays(-1));
Assert.Equal(PlatformRunClaim.Claimed, h.Tracker.ScriptRuns.TryClaim(RaceUserId, RacePlatformId, h.Now, out var run));
Assert.Equal(script.RunId, run!.RunId);
}
/// <summary>
/// The mirror image: a foreign executor whose stream position lies inside the platform start block (it opened
/// at the same end with a shorter start block) writes its end block there. Its first line merges with the head of
/// a platform line — cut inside the message or inside the trailer, which the parser would otherwise read up to
/// the foreign trailer — and its output line follows without any marker in between.
/// </summary>
[Theory]
[InlineData(15)]
[InlineData(-30)]
public async Task Foreign_end_block_merged_into_the_platform_start_block_never_reaches_the_platform_script(int cut)
{
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
h.Append(DetectionStartBlock("12:00:00.0000000", RaceFillerAppId, "1548") + ExecutorEndBlock("12:00:00.5000000", 0, "detected"));
await h.Pass();
var x = h.Length;
var dStart = PlatformStartBlock("12:00:01.0000000", RacePlatformId, "2524");
h.Append(dStart);
await h.Pass();
// The foreign end block starts inside the line after D's argument line.
var lineAfterArgument = x + dStart.IndexOf("<![LOG[Powershell option gets invoked", StringComparison.Ordinal);
var lineLength = EntryAt("12:00:01.0000000", "Powershell option gets invoked").Length;
var at = lineAfterArgument + (cut > 0 ? cut : lineLength + cut);
var foreignEnd = ExecutorEndBlock("12:00:03.0000000", 1, "foreign output");
Assert.True(at + foreignEnd.Length < x + dStart.Length, "the foreign block must end inside D's start block");
h.Overwrite(at, foreignEnd);
h.Tracker.RequestOverwriteCheckForTest();
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(RacePlatformId), sourceFileName: "IntuneManagementExtension.log");
h.Now = h.Now.Add(ImeLogTracker.PlatformScriptEndBlockGrace);
h.Tracker.FlushPendingPlatformScriptResults(h.Now);
// No owner is provable behind the cut: the completion goes out without end-block data, never with the foreign one.
var script = Assert.Single(h.Completed);
Assert.Equal("Success", script.Result);
Assert.Null(script.ExitCode);
Assert.Null(script.Stdout);
}
[Fact]
public async Task Entry_read_unchanged_after_a_rewind_keeps_its_platform_marker()
{
const string platformId = "2b7d9e41-8c3a-4f60-a5d2-9e1b7c4f3a08";
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
// Detection W waits for its script; detection X runs completely behind it; platform F starts behind X.
var wStart = DetectionStartBlock("12:00:00.0000000", RaceFillerAppId, "1111");
var xRun = DetectionStartBlock("12:00:00.2000000", RaceDetectionAppId, "2222") + ExecutorEndBlock("12:00:00.4000000", 0, "x detected");
h.Append(wStart + xRun + PlatformStartBlock("12:00:01.0000000", platformId, "3333"));
await h.Pass();
// W ends at its own position, over the head of X's run; F's start block keeps its bytes and offset.
var wEnd = ExecutorEndBlock("12:00:02.0000000", 0, "w detected");
Assert.True(wEnd.Length < xRun.Length);
h.Overwrite(wStart.Length, wEnd);
h.Tracker.RequestOverwriteCheckForTest();
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
Assert.Equal(1, h.Health.OverwriteRewinds);
// F's end block lands behind its start block: still F's, because F's marker survived with its bytes.
h.Append(ExecutorEndBlock("12:00:05.0000000", 7, "Hello from F"));
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(platformId), sourceFileName: "IntuneManagementExtension.log");
var script = Assert.Single(h.Completed);
Assert.Equal(platformId, script.PolicyId);
Assert.Equal(7, script.ExitCode);
Assert.Equal("Hello from F", script.Stdout);
}
[Fact]
public async Task Writer_boundary_read_unchanged_after_a_rewind_still_closes_the_platform_invocation()
{
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
// Detection Q waits for its script while detection R runs completely behind it.
var qStart = DetectionStartBlock("12:00:00.0000000", RaceFillerAppId, "1111");
var rRun = DetectionStartBlock("12:00:00.2000000", "6c3f8a20-1d5e-4b97-8f4a-0e2d6b9c7a13", "2222") + ExecutorEndBlock("12:00:00.4000000", 0, "r detected");
h.Append(qStart + rRun);
await h.Pass();
var x = h.Length;
// The race of the field case behind them: the boundary closes D.
var cStart = DetectionStartBlock("12:00:01.0000000", RaceDetectionAppId, "6416");
var dStart = PlatformStartBlock("12:00:01.2000000", RacePlatformId, "2524");
h.Append(cStart);
h.Overwrite(x, dStart);
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
// Q ends at its own position, over R's run: the rewind starts in front of the race, whose bytes read unchanged.
var qEnd = ExecutorEndBlock("12:00:02.0000000", 0, "q detected");
Assert.True(qEnd.Length < rRun.Length);
h.Overwrite(qStart.Length, qEnd);
h.Tracker.RequestOverwriteCheckForTest();
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
// C's end block behind the boundary stays unowned; D's own end block completes D.
var cEnd = ExecutorEndBlock("12:00:07.0000000", 1, DetectionStdout);
h.Append(cEnd);
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(RacePlatformId), sourceFileName: "IntuneManagementExtension.log");
Assert.Empty(h.Completed);
h.Overwrite(x + dStart.Length, ExecutorEndBlock("12:00:09.5000000", 0, PlatformStdout));
h.Now = h.Now.AddMilliseconds(130);
await h.Pass();
h.Tracker.FlushPendingPlatformScriptResults(h.Now);
var script = Assert.Single(h.Completed);
Assert.Equal(0, script.ExitCode);
Assert.Equal(PlatformStdout.Replace("\r\n", "\n"), script.Stdout);
}
[Fact]
public async Task Raw_text_in_the_executor_log_is_never_matched_and_closes_the_platform_invocation()
{
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
h.Append(PlatformStartBlock("12:00:00.0000000", RacePlatformId, "2524"));
await h.Pass();
// The tail of a cut line that happens to read like an exit line, then a foreign end block.
h.Append("Powershell exit code is 5\r\n" + ExecutorEndBlock("12:00:02.0000000", 6, "foreign output"));
// A fragment right at the bookmark stops the first pass for a check; the second one reads it.
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(RacePlatformId), sourceFileName: "IntuneManagementExtension.log");
h.Now = h.Now.Add(ImeLogTracker.PlatformScriptEndBlockGrace);
h.Tracker.FlushPendingPlatformScriptResults(h.Now);
var script = Assert.Single(h.Completed);
Assert.Null(script.ExitCode);
Assert.Null(script.Stdout);
}
[Fact]
public async Task Fragment_in_the_ime_log_does_not_close_the_platform_run_context()
{
using var h = new Harness("IntuneManagementExtension.log", ScriptPatterns());
// In the IME log the service writes the platform marker and the context line itself; the fragment in
// between is the tail of an executor's telemetry line and separates nothing.
h.Append(Entry(@"Script file C:\Program Files (x86)\Microsoft Intune Management Extension\Policies\Scripts\" + RaceUserId + "_" + RacePlatformId + ".ps1 is generated.", "17")
+ "ing mananger...]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">\r\n"
+ Entry("Launch powershell executor in user session", "17"));
await h.Pass();
h.Tracker.ProcessLogMessageForTest(PlatformResultLine(RacePlatformId));
h.Now = h.Now.Add(ImeLogTracker.PlatformScriptEndBlockGrace);
h.Tracker.FlushPendingPlatformScriptResults(h.Now);
var script = Assert.Single(h.Completed);
Assert.Equal("User", script.RunContext);
}
[Fact]
public async Task Cut_multi_line_output_adds_one_boundary_not_one_per_line()
{
using var h = new Harness("AgentExecutor.log", ScriptPatterns());
h.Append(PlatformStartBlock("12:00:00.0000000", RacePlatformId, "2524"));
await h.Pass();
var markersBefore = h.Tracker.InvocationMarkerCountForTest("AgentExecutor.log");
var lines = string.Concat(Enumerable.Range(1, 600).Select(i => $"orphaned output line {i}\r\n"));
h.Append(lines);
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
h.Now = h.Now.AddMilliseconds(100);
await h.Pass();
Assert.Equal(markersBefore + 1, h.Tracker.InvocationMarkerCountForTest("AgentExecutor.log"));
}
[Theory]
[InlineData("<![LOG[Powershell exit code is 0]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", true)]
[InlineData("<![LOG[write output done. output = line one\nline two, error = ]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", true)]
[InlineData("\uFEFF<![LOG[ExecutorLog AgentExecutor gets invoked]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", true)]
[InlineData("t code is 3]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", false)]
[InlineData("", false)]
[InlineData("<![LOG[Powe<![LOG[Powershell exit code is 1]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", false)]
[InlineData("<![LOG[Powershell option gets invoked]LOG]!><time=\"12:00:00.0000000\" da<![LOG[Powershell exit code is 1]LOG]!><time=\"12:00:01.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", false)]
[InlineData("<![LOG[write output done. output = line one\nline<![LOG[Agent executor completed.]LOG]!><time=\"12:00:00.0000000\" date=\"9-10-2026\" component=\"AgentExecutor\" context=\"\" type=\"1\" thread=\"1\" file=\"\">", false)]
public void Single_cmtrace_record_is_told_apart_from_the_cut_of_two_writers(string text, bool expected)
{
Assert.Equal(expected, ImeLogTracker.IsSingleCmTraceRecord(text));
}
[Theory]
[InlineData("AgentExecutor.log", true)]
[InlineData("AgentExecutor-20261003-233348.log", true)]
[InlineData("IntuneManagementExtension.log", false)]
[InlineData("IntuneManagementExtension-20261003-233348.log", false)]
[InlineData("HealthScripts.log", false)]
[InlineData("AppWorkload.log", false)]
public void Writer_boundaries_apply_to_the_executor_log_live_and_rotated(string fileName, bool expected)
{
Assert.Equal(expected, ImeLogTracker.IsWriterBoundaryLogFile(fileName));
}
}
}
@@ -25,7 +25,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
/// contract: unchanged bytes never rewind, changed bytes are processed exactly once, and the
/// executor lines a rewind uncovers belong to the invocation that owns their file position.
/// </summary>
public sealed class ImeLogTrackerOverwriteRewindTests
public sealed partial class ImeLogTrackerOverwriteRewindTests
{
private static readonly DateTime T0 = new DateTime(2026, 9, 10, 19, 30, 0, DateTimeKind.Utc);
@@ -42,7 +42,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
private static readonly string[] ScriptPatternIds =
{
"PS-AGENT-INVOCATION", "PS-AGENT-ARG", "PS-AGENT-SCRIPT-START", "PS-AGENT-EXITCODE",
"PS-AGENT-OUTPUT", "PS-AGENT-COMPLETED", "PS-SCRIPT-GENERATED", "PS-SCRIPT-CONTEXT", "PS-SCRIPT-RESULT",
"PS-AGENT-OUTPUT", "PS-AGENT-ERROR", "PS-AGENT-COMPLETED", "PS-SCRIPT-GENERATED", "PS-SCRIPT-CONTEXT", "PS-SCRIPT-RESULT",
};
/// <summary>The shipped pattern JSON — the same source combine.js embeds — so the contract is the real one.</summary>
@@ -19,6 +19,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
/// completion markers (whiteglove.complete, clean-exit) — V1 sessions had only AgentLogs/
/// and ImeLogs/, which left forensics blind to decision-engine state and pending uploads.
/// </summary>
[Collection(UserProfileResolverCollection.Name)] // replaces UserProfileResolver's static state
public sealed class DiagnosticsPackageServiceTests
{
private static AgentConfiguration Cfg(string sessionId = "S1") => new AgentConfiguration
@@ -0,0 +1,96 @@
using System;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
{
/// <summary>
/// Serialises the test classes that replace <see cref="UserProfileResolver"/>'s static state.
/// </summary>
[CollectionDefinition(Name)]
public sealed class UserProfileResolverCollection
{
public const string Name = "UserProfileResolver";
}
/// <summary>
/// The agent starts before anyone has signed in. A lookup in that window must not pin
/// "no user" for the rest of the process — the %LOGGED_ON_USER_PROFILE% gather rules and the
/// RealmJoin user logs of the diagnostics package would never resolve.
/// </summary>
[Collection(UserProfileResolverCollection.Name)]
public sealed class UserProfileResolverTests : IDisposable
{
private static readonly DateTime Start = new DateTime(2026, 9, 28, 12, 0, 0, DateTimeKind.Utc);
public void Dispose() => UserProfileResolver.Reset();
[Fact]
public void User_who_signs_in_after_the_first_lookup_is_found_later()
{
string? signedIn = null;
var now = Start;
UserProfileResolver.SetDetectorForTesting(() => signedIn, () => now);
Assert.Null(UserProfileResolver.GetLoggedOnUserProfilePath());
signedIn = @"C:\Users\enduser";
now += UserProfileResolver.RetryInterval;
Assert.Equal(@"C:\Users\enduser", UserProfileResolver.GetLoggedOnUserProfilePath());
}
[Fact]
public void No_user_is_detected_again_only_after_the_retry_interval()
{
var detections = 0;
var now = Start;
UserProfileResolver.SetDetectorForTesting(() => { detections++; return null; }, () => now);
UserProfileResolver.GetLoggedOnUserProfilePath();
now += TimeSpan.FromSeconds(10);
UserProfileResolver.GetLoggedOnUserProfilePath();
UserProfileResolver.GetLoggedOnUserProfilePath();
Assert.Equal(1, detections);
now += UserProfileResolver.RetryInterval;
UserProfileResolver.GetLoggedOnUserProfilePath();
Assert.Equal(2, detections);
}
[Fact]
public void A_clock_stepping_backwards_does_not_delay_the_next_detection()
{
var detections = 0;
var now = Start;
UserProfileResolver.SetDetectorForTesting(() => { detections++; return null; }, () => now);
UserProfileResolver.GetLoggedOnUserProfilePath();
now -= TimeSpan.FromHours(1); // OOBE time sync
UserProfileResolver.GetLoggedOnUserProfilePath();
Assert.Equal(2, detections);
}
[Fact]
public void A_found_user_is_kept_without_detecting_again()
{
var detections = 0;
UserProfileResolver.SetDetectorForTesting(() => { detections++; return @"C:\Users\enduser"; }, () => Start);
UserProfileResolver.GetLoggedOnUserProfilePath();
UserProfileResolver.GetLoggedOnUserProfilePath();
Assert.Equal(1, detections);
}
[Fact]
public void Pinned_no_user_stays_null_without_real_detection()
{
UserProfileResolver.SetForTesting(null!);
Assert.Null(UserProfileResolver.GetLoggedOnUserProfilePath());
Assert.Null(UserProfileResolver.ExpandCustomTokens(@"%LOGGED_ON_USER_PROFILE%\AppData\Local\x.log"));
}
}
}
@@ -0,0 +1,445 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Newtonsoft.Json;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
{
/// <summary>
/// CloudExperienceHost navigation (D-310): one agent.log line per page start/stop and per
/// update-page event name, and for the OOBE update page itself (<c>OobeNDUP</c>,
/// <c>RebootNDUP</c>) the <c>oobe_update_page</c> telemetry plus the registry state at its live
/// start and stop. Descriptions follow the Shell-Core manifest templates; a 62407 value
/// (URLs with tenant ids) leaves the tracker only as a boolean, an integer or an HRESULT.
/// </summary>
public sealed class ShellCoreTrackerBreadcrumbTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 10, 2, 9, 0, 0, DateTimeKind.Utc);
private const string NdupStarted = "CloudExperienceHost Web App Activity started. CXID: 'OobeNDUP'.";
private const string EspPageStarted = "CloudExperienceHost Web App Activity started. CXID: 'MDMProgressAfterAADRefactored'.";
private const string PageStopped = "CloudExperienceHost Web App Activity stopped. Result: 'success'.";
private const string ScanStartedName = "CloudExperienceHost Web App Event 1. Name: 'ExpeditedUpdate_startWUScanStarted'.";
private const string NavigateWithTenantUrl =
"CloudExperienceHost Web App Event 2. Name: 'Navigate', Value: '{\"httpMethod\":\"GET\",\"url\":\"https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/authorize\"}'.";
private const string DownloadSucceededWithValue =
"CloudExperienceHost Web App Event 2. Name: 'ExpeditedUpdate_commitExpeditionDownloadInstallAsyncSucceeded', Value: '{\"url\":\"https://login.microsoftonline.com/x\"}'.";
private const string CspGateTrue =
"CloudExperienceHost Web App Event 2. Name: 'ExpeditedUpdate_isNDUPAllowedByCSPSucceeded', Value: 'true'.";
// A French Windows labels the same fields "Résultat", "Nom" and "valeur" and quotes them
// in guillemets; the event properties carry the values unchanged in every language.
private const string FrenchNdupStarted = "Activité d’application web CloudExperienceHost démarrée. CXID : « OobeNDUP ».";
private const string FrenchPageStopped = "Activité d’application web CloudExperienceHost arrêtée. Résultat : « success ».";
private const string FrenchCspGate =
"Événement d’application web CloudExperienceHost 2. Nom : « ExpeditedUpdate_isNDUPAllowedByCSPSucceeded », valeur : « true ».";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
private IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState> _registry = Array.Empty<OobeUpdateRegistrySnapshot.KeyState>();
public void Dispose() => _tmp.Dispose();
private ShellCoreTracker Build(string? stateDirectory = null) =>
new ShellCoreTracker("S1", "T1",
new InformationalEventPost(_sink, new VirtualClock(At)),
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
helloTracker: null,
oobeUpdateRegistryReader: () => _registry,
stateDirectory: stateDirectory);
private string StateDirectory => Path.Combine(_tmp.Path, "state");
private static void Live(ShellCoreTracker tracker, int eventId, string description, DateTime at, long recordId) =>
tracker.ProcessEvent(eventId, description, at, "Microsoft-Windows-Shell-Core", isBackfill: false, recordId: recordId);
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal posted) =>
(IReadOnlyDictionary<string, object>)posted.TypedPayload!;
private static OobeUpdateRegistrySnapshot.KeyState NdupKey(string line) =>
new OobeUpdateRegistrySnapshot.KeyState("setupOobeNdup", @"SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OOBE\NDUP",
exists: true, hasValues: true, line: line);
// ------------------------------------------------------------- breadcrumbs ----
[Fact]
public void PageStart_NamesTheCxid()
{
Assert.Equal("CXH page started: OobeNDUP",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, lastPage: null));
}
[Fact]
public void PageStop_IsAttributedToTheLastStartedPage()
{
// 62405 names no page — it ends the one started last.
Assert.Equal("CXH page stopped: OobeNDUP (result=success)",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, lastPage: "OobeNDUP"));
Assert.Equal("CXH page stopped: ? (result=success)",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, lastPage: null));
}
[Fact]
public void UpdatePageEventNames_AreLogged_FromBothEventIds()
{
Assert.Equal("CXH event: ExpeditedUpdate_startWUScanStarted",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, lastPage: null));
Assert.Equal("CXH event: ExpeditedUpdate_commitExpeditionDownloadInstallAsyncSucceeded",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppEvent, DownloadSucceededWithValue, lastPage: null));
}
[Fact]
public void UnsafeEventValues_NeverReachTheLog()
{
var crumb = ShellCoreTracker.FormatCxhBreadcrumb(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, DownloadSucceededWithValue, lastPage: null);
Assert.NotNull(crumb);
Assert.DoesNotContain("microsoftonline", crumb);
Assert.DoesNotContain("url", crumb!, StringComparison.OrdinalIgnoreCase);
}
[Fact]
public void SafeEventValues_AreLogged()
{
Assert.Equal("CXH event: ExpeditedUpdate_isNDUPAllowedByCSPSucceeded (value=true)",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppEvent, CspGateTrue, lastPage: null));
}
[Fact]
public void OtherEventNames_AreNotLogged()
{
// Ordinary page traffic (Navigate, Visible, ESP progress) stays out of the log.
Assert.Null(ShellCoreTracker.FormatCxhBreadcrumb(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, NavigateWithTenantUrl, lastPage: "OobeNDUP"));
Assert.Null(ShellCoreTracker.FormatCxhBreadcrumb(
ShellCoreTracker.EventId_ShellCore_WebAppEvent, "CommercialOOBE_ESPProgress_Page_Exiting", lastPage: null));
}
[Theory]
[InlineData("")]
[InlineData("CloudExperienceHost Web App Activity started.")]
public void PageStartWithoutCxid_IsNoBreadcrumb(string description)
{
Assert.Null(ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppStarted, description, lastPage: null));
}
[Fact]
public void LongTokens_AreBounded()
{
var longCxid = new string('X', 500);
var crumb = ShellCoreTracker.FormatCxhBreadcrumb(
ShellCoreTracker.EventId_ShellCore_WebAppStarted, $"CloudExperienceHost Web App Activity started. CXID: '{longCxid}'.", lastPage: null);
Assert.NotNull(crumb);
Assert.True(crumb!.Length < 200, crumb);
Assert.EndsWith("…", crumb);
}
[Theory]
[InlineData("OobeNDUP", true)]
[InlineData("RebootNDUP", true)]
[InlineData("EarlyRebootNDUP", true)]
[InlineData("ExpeditedUpdate_startWUScanStarted", true)]
[InlineData("AADHello", false)]
[InlineData("MDMProgressAfterAADRefactored", false)]
[InlineData(null, false)]
public void UpdatePageTokens_AreRecognized(string? token, bool expected)
{
Assert.Equal(expected, ShellCoreTracker.IsUpdatePageToken(token!));
}
// ------------------------------------------------------ update page records ----
[Fact]
public void UpdatePageRecords_ArePageStartsStopsAndNames_OfTheUpdatePageOnly()
{
var start = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, lastPage: null);
Assert.Equal(OobeUpdatePageRecord.PageStarted, start!.CxhEvent);
Assert.Equal("OobeNDUP", start.Page);
var stop = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, lastPage: "RebootNDUP");
Assert.Equal(OobeUpdatePageRecord.PageStopped, stop!.CxhEvent);
Assert.Equal("RebootNDUP", stop.Page);
Assert.Equal("success", stop.Result);
var name = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, lastPage: null);
Assert.Equal(OobeUpdatePageRecord.EventName, name!.CxhEvent);
Assert.Equal("ExpeditedUpdate_startWUScanStarted", name.Name);
Assert.Null(name.Value);
// Other pages and names are no update-page records.
Assert.Null(ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppStarted, EspPageStarted, lastPage: null));
Assert.Null(ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, lastPage: "MDMProgressAfterAADRefactored"));
Assert.Null(ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppEvent, NavigateWithTenantUrl, lastPage: "OobeNDUP"));
}
[Fact]
public void UpdatePageRecord_CarriesOnlyASafeValue()
{
var unsafeValue = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppEvent, DownloadSucceededWithValue, lastPage: null);
Assert.Null(unsafeValue!.Value);
var gate = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppEvent, CspGateTrue, lastPage: null);
Assert.Equal("true", gate!.Value);
}
[Theory]
[InlineData("true", "true")]
[InlineData("FALSE", "FALSE")]
[InlineData("0", "0")]
[InlineData("-2147024891", "-2147024891")]
[InlineData("0x80240022", "0x80240022")]
[InlineData("12345678901", null)] // more than ten digits
[InlineData("0x123456789", null)] // more than eight hex digits
[InlineData("https://login.microsoftonline.com/x", null)]
[InlineData("{\"result\":0}", null)]
[InlineData("true and more", null)]
[InlineData("", null)]
public void SafeValues_AreBooleansIntegersAndHresults(string value, string? expected)
{
var description = $"CloudExperienceHost Web App Event 2. Name: 'ExpeditedUpdate_x', Value: '{value}'.";
Assert.Equal(expected, ShellCoreTracker.ExtractSafeValue(description));
}
[Fact]
public void LiveUpdatePage_IsReported_WithTheRegistryState_AndRaisesNothing()
{
_registry = new[] { NdupKey("EnableExpeditedUpdate=1") };
using var tracker = Build();
var raised = 0;
tracker.EspExited += (_, _) => raised++;
tracker.EspFailureDetected += (_, _) => raised++;
tracker.FinalizingSetupPhaseTriggered += (_, _) => raised++;
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, "Microsoft-Windows-Shell-Core", isBackfill: false);
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, At.AddMinutes(1), "Microsoft-Windows-Shell-Core", isBackfill: false);
_registry = new[] { NdupKey("EnableExpeditedUpdate=1, EnableExpeditedUpdateSyncInstallCompleted=1") };
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, At.AddMinutes(30), "Microsoft-Windows-Shell-Core", isBackfill: false);
var pages = ByType(Constants.EventTypes.OobeUpdatePage);
Assert.Equal(new object[] { "page_started", "event_name", "page_stopped" }, pages.Select(p => Data(p)["cxhEvent"]).ToArray());
Assert.All(pages, p => Assert.Equal(false, Data(p)["backfill"]));
var states = ByType(Constants.EventTypes.OobeUpdateState);
Assert.Equal(new object[] { "update_page_started", "update_page_stopped" }, states.Select(s => Data(s)["moment"]).ToArray());
Assert.All(states, s => Assert.Equal("OobeNDUP", Data(s)["page"]));
// Telemetry only — no ESP signal, no other event.
Assert.Equal(0, raised);
Assert.False(tracker.IsEspExitedForTest);
Assert.Equal(pages.Count + states.Count, _sink.Posted.Count);
}
[Fact]
public void OtherPages_AreNotReported()
{
_registry = new[] { NdupKey("EnableExpeditedUpdate=1") };
using var tracker = Build();
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStarted, EspPageStarted, At, "Microsoft-Windows-Shell-Core", isBackfill: false);
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, At.AddMinutes(1), "Microsoft-Windows-Shell-Core", isBackfill: false);
Assert.Empty(_sink.Posted);
}
[Fact]
public void Backfill_ReportsTheUpdatePage_WithItsEventTimes_ButReadsNoRegistry()
{
// The registry shows the state now, not at the record's time.
var reads = 0;
using var tracker = new ShellCoreTracker("S1", "T1",
new InformationalEventPost(_sink, new VirtualClock(At)),
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
helloTracker: null,
oobeUpdateRegistryReader: () => { reads++; return new[] { NdupKey("EnableExpeditedUpdate=1") }; });
tracker.ReplayBackfillRecords(new[]
{
(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At.AddMinutes(-4)),
(ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, At.AddMinutes(-3)),
(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, At.AddMinutes(-2)),
});
var pages = ByType(Constants.EventTypes.OobeUpdatePage);
Assert.Equal(3, pages.Count);
Assert.All(pages, p => Assert.Equal(true, Data(p)["backfill"]));
Assert.Equal(At.AddMinutes(-4), pages[0].OccurredAtUtc);
Assert.Equal(0, reads);
// Nothing else: no ESP exit or failure was in the window, so no skipped-records trace.
Assert.Equal(3, _sink.Posted.Count);
}
// ------------------------------------------------------------ localization ----
[Fact]
public void LocalizedRecords_AreReadFromTheirProperties()
{
using var tracker = Build();
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStarted, FrenchNdupStarted, At, "Microsoft-Windows-Shell-Core",
isBackfill: false, properties: new[] { "OobeNDUP" });
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppEvent, FrenchCspGate, At.AddSeconds(5), "Microsoft-Windows-Shell-Core",
isBackfill: false, properties: new[] { "ExpeditedUpdate_isNDUPAllowedByCSPSucceeded", "true" });
tracker.ProcessEvent(ShellCoreTracker.EventId_ShellCore_WebAppStopped, FrenchPageStopped, At.AddMinutes(1), "Microsoft-Windows-Shell-Core",
isBackfill: false, properties: new[] { "success" });
var pages = ByType(Constants.EventTypes.OobeUpdatePage).Select(Data).ToList();
Assert.Equal(new object[] { "page_started", "event_name", "page_stopped" }, pages.Select(d => d["cxhEvent"]).ToArray());
Assert.Equal("OobeNDUP", pages[0]["page"]);
Assert.Equal("ExpeditedUpdate_isNDUPAllowedByCSPSucceeded", pages[1]["name"]);
Assert.Equal("true", pages[1]["value"]);
Assert.Equal("OobeNDUP", pages[2]["page"]);
Assert.Equal("success", pages[2]["result"]);
}
[Fact]
public void LocalizedMessage_WithoutProperties_DoesNotYieldItsLabelledFields()
{
// Why the properties are read: the French labels are not the manifest's.
Assert.Null(ShellCoreTracker.ReadCxhFields(ShellCoreTracker.EventId_ShellCore_WebAppStopped, FrenchPageStopped, properties: null).Primary);
Assert.Null(ShellCoreTracker.ReadCxhFields(ShellCoreTracker.EventId_ShellCore_WebAppEvent, FrenchCspGate, properties: null).Primary);
Assert.Equal("success",
ShellCoreTracker.ReadCxhFields(ShellCoreTracker.EventId_ShellCore_WebAppStopped, FrenchPageStopped, new[] { "success" }).Primary);
}
[Theory]
[InlineData("CloudExperienceHost Web App Event 2. Name: 'ExpeditedUpdate_x', Value: 'true'.")]
[InlineData("CloudExperienceHost Web App Event 2. Name : « ExpeditedUpdate_x », Value : « true ».")]
[InlineData("CloudExperienceHost Web App Event 2. Name\u00A0: «\u202FExpeditedUpdate_x\u202F», Value\u00A0: «\u00A0true\u00A0».")]
public void MessageFallback_AcceptsQuotesAndGuillemets(string description)
{
var fields = ShellCoreTracker.ReadCxhFields(ShellCoreTracker.EventId_ShellCore_WebAppEvent, description, properties: null);
Assert.Equal("ExpeditedUpdate_x", fields.Primary);
Assert.Equal("true", fields.Value);
}
[Fact]
public void PropertyValue_LeavesTheTrackerOnlyInASafeForm()
{
var fields = ShellCoreTracker.ReadCxhFields(ShellCoreTracker.EventId_ShellCore_WebAppEvent, "",
new[] { "ExpeditedUpdate_commitExpeditionDownloadInstallAsyncSucceeded", "{\"url\":\"https://login.microsoftonline.com/x\"}" });
var record = ShellCoreTracker.ParseUpdatePageRecord(ShellCoreTracker.EventId_ShellCore_WebAppEvent, fields, lastPage: null);
Assert.Equal("ExpeditedUpdate_commitExpeditionDownloadInstallAsyncSucceeded", record!.Name);
Assert.Null(record.Value);
Assert.DoesNotContain("microsoftonline",
ShellCoreTracker.FormatCxhBreadcrumb(ShellCoreTracker.EventId_ShellCore_WebAppEvent, fields, lastPage: null));
}
// ------------------------------------------------------- restart watermark ----
[Fact]
public void RestartBackfill_SkipsUpdatePageRecords_AnEarlierRunReported()
{
using (var first = Build(StateDirectory))
{
Live(first, ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, recordId: 500);
Live(first, ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, At.AddMinutes(1), recordId: 501);
}
// The restart reads the minutes before it again — plus the stop the first run missed.
using var second = Build(StateDirectory);
second.LoadPageWatermark();
second.ReplayBackfillRecords(new[]
{
new ShellCoreRecord(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, recordId: 500),
new ShellCoreRecord(ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, At.AddMinutes(1), recordId: 501),
new ShellCoreRecord(ShellCoreTracker.EventId_ShellCore_WebAppStopped, PageStopped, At.AddMinutes(2), recordId: 502),
});
var pages = ByType(Constants.EventTypes.OobeUpdatePage).Select(Data).ToList();
Assert.Equal(new object[] { "page_started", "event_name", "page_stopped" }, pages.Select(d => d["cxhEvent"]).ToArray());
Assert.Equal(new object[] { false, false, true }, pages.Select(d => d["backfill"]).ToArray());
// The skipped start still names the page the stop belongs to.
Assert.Equal("OobeNDUP", pages[2]["page"]);
}
[Fact]
public void SameRecord_FromTheLiveWatcherAndTheBackfill_IsReportedOnce()
{
// The live watcher is armed before the backfill runs, so both can deliver a record.
using var tracker = Build(StateDirectory);
Live(tracker, ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, recordId: 700);
tracker.ReplayBackfillRecords(new[]
{
new ShellCoreRecord(ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, recordId: 700),
});
Assert.Single(ByType(Constants.EventTypes.OobeUpdatePage));
}
[Fact]
public void Watermark_FollowsReportedRecordsOnly()
{
using var tracker = Build(StateDirectory);
for (var i = 0; i <= OobeUpdateTelemetry.MaxPageEventsPerKey; i++)
Live(tracker, ShellCoreTracker.EventId_ShellCore_WebAppEventName, ScanStartedName, At.AddSeconds(i), recordId: 800 + i);
// The repeat over the per-name cap is claimed for this run, not written down.
Assert.Equal(OobeUpdateTelemetry.MaxPageEventsPerKey, ByType(Constants.EventTypes.OobeUpdatePage).Count);
var state = JsonConvert.DeserializeObject<EventRecordWatermark.WatermarkState>(
File.ReadAllText(Path.Combine(StateDirectory, ShellCoreTracker.PageWatermarkStateFileName)));
Assert.Equal(800 + OobeUpdateTelemetry.MaxPageEventsPerKey - 1, state!.LastRecordId);
}
[Fact]
public void RecordsWithoutARecordId_AreNotDeduplicated()
{
using var tracker = Build(StateDirectory);
Live(tracker, ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At, recordId: -1);
Live(tracker, ShellCoreTracker.EventId_ShellCore_WebAppStarted, NdupStarted, At.AddMinutes(5), recordId: -1);
Assert.Equal(2, ByType(Constants.EventTypes.OobeUpdatePage).Count);
Assert.False(File.Exists(Path.Combine(StateDirectory, ShellCoreTracker.PageWatermarkStateFileName)));
}
[Fact]
public void AgentStartAndStop_ReportTheRegistryState_StopOnce()
{
_registry = new[] { NdupKey("EnableExpeditedUpdate=1") };
var tracker = Build();
tracker.Start();
_registry = new[] { NdupKey("EnableExpeditedUpdate=1, ExpeditedUpdateStatus=2") };
tracker.Stop();
tracker.Stop();
tracker.Dispose();
var states = ByType(Constants.EventTypes.OobeUpdateState);
Assert.Equal(new object[] { "agent_start", "agent_stop" }, states.Select(s => Data(s)["moment"]).ToArray());
}
[Fact]
public void TrackerThatNeverStarted_ReportsNothingAtStop()
{
_registry = new[] { NdupKey("EnableExpeditedUpdate=1") };
var tracker = Build();
tracker.Dispose();
Assert.Empty(_sink.Posted);
}
}
}
@@ -84,6 +84,55 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
Assert.False(raised);
}
[Fact]
public void ProcessEvent_62404_NGC_in_a_localized_message_is_read_from_the_cxid_property()
{
// A French Windows quotes the CXID in guillemets — the "'NGC'" substring never matched.
using var tmp = new TempDirectory();
using var tracker = MakeTracker(tmp, new VirtualClock(ClockNow));
var raised = 0;
tracker.HelloWizardStarted += (_, _) => raised++;
tracker.ProcessEvent(
eventId: ShellCoreTracker.EventId_ShellCore_WebAppStarted,
description: "Activité d’application web CloudExperienceHost démarrée. CXID : « NGC ».",
timestamp: ClockNow,
providerName: "Microsoft-Windows-Shell-Core",
isBackfill: false,
properties: new[] { "NGC" });
Assert.Equal(1, raised);
}
[Theory]
[InlineData("AADHello", true)]
[InlineData("NGC", true)]
[InlineData("NGCPin", false)]
[InlineData("OobeNDUP", false)]
public void HelloWizardStart_is_decided_by_the_cxid(string cxid, bool expected)
{
Assert.Equal(expected, ShellCoreTracker.IsHelloWizardStart(cxid, description: "CXID: 'AADHello'"));
}
[Fact]
public void Backfill_62404_reads_the_cxid_property()
{
using var tmp = new TempDirectory();
using var tracker = MakeTracker(tmp, new VirtualClock(ClockNow));
var captured = new List<HelloWizardStartedEventArgs>();
tracker.HelloWizardStarted += (_, args) => captured.Add(args);
var historical = ClockNow.AddMinutes(-3);
tracker.ReplayBackfillRecords(new[]
{
new ShellCoreRecord(ShellCoreTracker.EventId_ShellCore_WebAppStarted,
"Activité d’application web CloudExperienceHost démarrée. CXID : « NGC ».", historical,
recordId: 42, properties: new[] { "NGC" }),
});
Assert.Equal(historical, Assert.Single(captured).OccurredAtUtc);
}
[Fact]
public void Backfill_62404_raises_once_with_historical_timestamp()
{
@@ -61,7 +61,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
// IME AppInstallStatus values (StatusServiceReports\..\Status), stable IME 1.50 .. 1.106.
// 1001/1003 sit inside the 1000 band but are NOT terminal — IME maps every InProgress
// enforcement state to 1001 (verified in the decompiled 1.106 StatusServiceHelpers).
// enforcement state to 1001 (verified against IME 1.106).
[Theory]
[InlineData(1000, "installed")]
[InlineData(1002, "installed")]
@@ -0,0 +1,411 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using Xunit;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// D-316: IME saves every platform script's result under Policies\&lt;user&gt;\&lt;policy&gt; after reporting the
/// batch; the observer checks the tracker's emitted run against it. The field case (session 756970cf): the run
/// had been given a Win32 detection script's end block — exit 1 and "Detection Started" output — while IME saved
/// the bootstrap output of the platform script itself.
/// </summary>
public sealed class ImeRegistryScriptResultObserverTests
{
private const string DeviceUser = "00000000-0000-0000-0000-000000000000";
private const string PolicyA = "1d6f0a52-3c4e-4b7a-9e21-5f8c7d2b9a10";
private const string PolicyB = "2b7d9e41-8c3a-4f60-a5d2-9e1b7c4f3a08";
private const string OwnStdout = "=== bootstrap loader started ===\nInstaller finished (exit code 0).";
private const string ForeignStdout = "VERBOSE: === Sample Suite Detection Started ===\nWARNING: === DETECTION FAILED ===";
private static readonly DateTime T0 = new DateTime(2026, 10, 3, 20, 59, 0, DateTimeKind.Utc);
// ── PlatformOutputJudge ─────────────────────────────────────────────────
private static PlatformScriptRun Run(string? stdout, string? stderr = null, int? exitCode = 0, string result = "Success", string resultSource = "ime_policy_result")
=> new PlatformScriptRun { PolicyId = PolicyA, UserId = DeviceUser, RunId = "run-1", Emitted = true, Result = result, ResultSource = resultSource, ExitCode = exitCode, Stdout = stdout, Stderr = stderr };
private static string Details(string executionMessage)
=> "{\"Version\":1,\"SigningCode\":15,\"EncryptionCode\":19,\"ExecutionMsg\":" + System.Text.Json.JsonSerializer.Serialize(executionMessage) + "}";
private static SavedScriptResult Saved(string? executionMessage, string result = "Success", int errorCode = 0, string user = DeviceUser, string policy = PolicyA, string stamp = "10/3/2026 9:00:00 PM")
=> new SavedScriptResult
{
UserId = user,
PolicyId = policy,
Result = result,
ErrorCode = errorCode,
ResultDetails = executionMessage == null ? null : Details(executionMessage),
LastUpdatedTimeUtc = stamp,
};
[Fact]
public void Field_case_detection_output_on_the_platform_run_is_foreign_and_the_saved_stdout_replaces_it()
{
var judgement = PlatformOutputJudge.Judge(Run(ForeignStdout, string.Empty, exitCode: 1), Saved(OwnStdout.Replace("\n", "\r\n")));
Assert.Equal(PlatformOutputVerdict.Foreign, judgement.Verdict);
Assert.Equal("stdout_differs", judgement.Reason);
Assert.Equal(OwnStdout, judgement.Stdout);
Assert.Equal(string.Empty, judgement.Stderr);
}
[Fact]
public void Matching_output_is_verified_across_line_ending_and_trim_differences()
{
var judgement = PlatformOutputJudge.Judge(Run(OwnStdout, string.Empty), Saved(" " + OwnStdout.Replace("\n", "\r\n") + "\r\n"));
Assert.Equal(PlatformOutputVerdict.Verified, judgement.Verdict);
}
[Fact]
public void Output_cut_by_the_tracker_is_verified_against_the_longer_saved_text()
{
var full = string.Concat(Enumerable.Repeat("line of output\n", 1000)).TrimEnd();
var emitted = ImeLogTracker.TruncateOutput(full);
Assert.EndsWith(ImeLogTracker.OutputTruncationMarker, emitted);
Assert.Equal(PlatformOutputVerdict.Verified, PlatformOutputJudge.Judge(Run(emitted), Saved(full)).Verdict);
}
[Fact]
public void Both_empty_is_verified_but_stderr_on_a_success_is_another_executors()
{
Assert.Equal(PlatformOutputVerdict.Verified, PlatformOutputJudge.Judge(Run(string.Empty), Saved(string.Empty)).Verdict);
var judgement = PlatformOutputJudge.Judge(Run(string.Empty, "Get-Item : not found"), Saved(string.Empty));
Assert.Equal(PlatformOutputVerdict.Foreign, judgement.Verdict);
Assert.Equal("stderr_on_success", judgement.Reason);
}
[Fact]
public void Explicitly_empty_emitted_output_against_saved_text_is_foreign()
{
Assert.Equal(PlatformOutputVerdict.Foreign, PlatformOutputJudge.Judge(Run(string.Empty), Saved(OwnStdout)).Verdict);
Assert.Equal(PlatformOutputVerdict.Foreign, PlatformOutputJudge.Judge(Run(OwnStdout), Saved(string.Empty)).Verdict);
}
[Fact]
public void Output_never_read_is_completed_from_the_saved_text_and_the_exit_code_stands()
{
var judgement = PlatformOutputJudge.Judge(Run(null), Saved(OwnStdout));
Assert.Equal(PlatformOutputVerdict.Repaired, judgement.Verdict);
Assert.Equal("stdout_missing", judgement.Reason);
Assert.Equal(OwnStdout, judgement.Stdout);
Assert.Equal(PlatformOutputVerdict.NotComparable, PlatformOutputJudge.Judge(Run(null), Saved(string.Empty)).Verdict);
}
[Fact]
public void Stdout_containing_the_output_error_separator_is_repaired_not_foreign()
{
// "write output done. output = X, error = " with X = "Copying files, error = 0 errors\nDone": the
// lazy split leaves "Copying files" as stdout and the rest as stderr.
var judgement = PlatformOutputJudge.Judge(
Run("Copying files", "0 errors\nDone, error ="),
Saved("Copying files, error = 0 errors\r\nDone"));
Assert.Equal(PlatformOutputVerdict.Repaired, judgement.Verdict);
Assert.Equal("stdout_split", judgement.Reason);
Assert.Equal("Copying files, error = 0 errors\nDone", judgement.Stdout);
Assert.Equal(string.Empty, judgement.Stderr);
}
[Fact]
public void A_prefix_without_the_split_explanation_is_foreign()
{
Assert.Equal(PlatformOutputVerdict.Foreign, PlatformOutputJudge.Judge(Run("Detected"), Saved("Detected and repaired")).Verdict);
}
[Fact]
public void Failed_with_stderr_is_judged_on_stderr()
{
Assert.Equal(PlatformOutputVerdict.Verified,
PlatformOutputJudge.Judge(Run("partial", "Access denied", 1, "Failed"), Saved("Access denied\r\n", "Failed", 99)).Verdict);
var missing = PlatformOutputJudge.Judge(Run(null, null, 1, "Failed"), Saved("Access denied", "Failed", 99));
Assert.Equal(PlatformOutputVerdict.Repaired, missing.Verdict);
Assert.Null(missing.Stdout);
Assert.Equal("Access denied", missing.Stderr);
var foreign = PlatformOutputJudge.Judge(Run("Detected", string.Empty, 0, "Failed"), Saved("Access denied", "Failed", 99));
Assert.Equal(PlatformOutputVerdict.Foreign, foreign.Verdict);
Assert.Equal(string.Empty, foreign.Stdout);
Assert.Equal("Access denied", foreign.Stderr);
}
[Fact]
public void Results_without_saved_output_or_from_another_run_are_not_compared()
{
Assert.Equal("no_saved_output", PlatformOutputJudge.Judge(Run(OwnStdout, null, 1, "Failed"), Saved("1", "Failed", 1)).Reason);
var hashMismatch = new SavedScriptResult { UserId = DeviceUser, PolicyId = PolicyA, Result = "Failed", ErrorCode = 2, ResultDetails = "Hash mismatch" };
Assert.Equal("no_execution_message", PlatformOutputJudge.Judge(Run(OwnStdout, null, 1, "Failed"), hashMismatch).Reason);
Assert.Equal("result_differs", PlatformOutputJudge.Judge(Run(OwnStdout), Saved("Access denied", "Failed", 99)).Reason);
// A result derived from the exit code (fallback) is not IME's — the saved one is.
Assert.Equal(PlatformOutputVerdict.Verified,
PlatformOutputJudge.Judge(Run(OwnStdout, null, 1, "Failed", "agentexecutor_fallback"), Saved(OwnStdout)).Verdict);
}
// ── PlatformScriptRunRegister ───────────────────────────────────────────
private static ScriptExecutionState Emitted(string policyId, string? stdout, int? exitCode = 0, string? userId = DeviceUser, string runId = "run-1")
=> new ScriptExecutionState { PolicyId = policyId, ScriptType = "platform", UserId = userId, RunId = runId, Result = "Success", ResultSource = "ime_policy_result", ExitCode = exitCode, Stdout = stdout };
[Fact]
public void A_save_is_paired_only_after_the_tracker_read_past_it_and_the_run_was_emitted()
{
var runs = new PlatformScriptRunRegister();
runs.NoteStarted(DeviceUser, PolicyA, T0);
var seen = T0.AddSeconds(30);
Assert.Equal(PlatformRunClaim.Wait, runs.TryClaim(DeviceUser, PolicyA, seen, out _));
runs.NotePassCompleted(seen.AddSeconds(1));
Assert.Equal(PlatformRunClaim.Wait, runs.TryClaim(DeviceUser, PolicyA, seen, out _)); // pass started too early
runs.NotePassCompleted(seen + PlatformScriptRunRegister.TrackerCatchUp);
Assert.Equal(PlatformRunClaim.Wait, runs.TryClaim(DeviceUser, PolicyA, seen, out _)); // run not emitted yet
runs.NoteEmitted(Emitted(PolicyA, OwnStdout), T0.AddSeconds(40));
Assert.Equal(PlatformRunClaim.Claimed, runs.TryClaim(DeviceUser, PolicyA, seen, out var run));
Assert.Equal("run-1", run!.RunId);
Assert.Equal(OwnStdout, run.Stdout);
Assert.Equal(PlatformRunClaim.None, runs.TryClaim(DeviceUser, PolicyA, seen, out _)); // never twice
}
[Fact]
public void A_new_start_replaces_the_latest_run_of_its_key_and_other_users_stay_apart()
{
var runs = new PlatformScriptRunRegister();
runs.NoteStarted(DeviceUser, PolicyA, T0);
runs.NoteEmitted(Emitted(PolicyA, "first run"), T0.AddSeconds(5));
runs.NoteStarted(DeviceUser, PolicyA, T0.AddMinutes(30));
runs.NoteEmitted(Emitted(PolicyA, "second run", runId: "run-2"), T0.AddMinutes(30).AddSeconds(5));
const string user = "6c3f8a20-1d5e-4b97-8f4a-0e2d6b9c7a13";
runs.NoteStarted(user, PolicyA, T0.AddMinutes(31));
runs.NotePassCompleted(T0.AddHours(1));
Assert.Equal(PlatformRunClaim.Claimed, runs.TryClaim(DeviceUser, PolicyA, T0.AddMinutes(31), out var run));
Assert.Equal("second run", run!.Stdout);
Assert.Equal(PlatformRunClaim.Wait, runs.TryClaim(user, PolicyA, T0.AddMinutes(32), out _));
Assert.Equal(PlatformRunClaim.None, runs.TryClaim(DeviceUser, PolicyB, T0.AddMinutes(32), out _));
}
[Fact]
public void A_run_without_a_readable_user_id_is_claimed_by_its_policy()
{
var runs = new PlatformScriptRunRegister();
runs.NoteStarted(null, PolicyA, T0);
runs.NoteEmitted(Emitted(PolicyA, OwnStdout, userId: null), T0.AddSeconds(5));
runs.NotePassCompleted(T0.AddMinutes(5));
Assert.Equal(PlatformRunClaim.Claimed, runs.TryClaim(DeviceUser, PolicyA, T0.AddMinutes(1), out _));
}
[Fact]
public void Register_keeps_the_newest_runs_and_its_counters_report_each_change_once()
{
var runs = new PlatformScriptRunRegister();
for (var i = 0; i <= PlatformScriptRunRegister.MaxRuns; i++)
runs.NoteStarted(DeviceUser, Guid.NewGuid().ToString("D"), T0.AddSeconds(i));
runs.NoteStarted(DeviceUser, PolicyA, T0.AddHours(1));
runs.NoteEmitted(Emitted(PolicyA, OwnStdout), T0.AddHours(1));
runs.NotePassCompleted(T0.AddHours(2));
Assert.Equal(PlatformRunClaim.Claimed, runs.TryClaim(DeviceUser, PolicyA, T0.AddHours(1), out _));
Assert.False(runs.ConsumeCountersChanged());
runs.CountVerified();
runs.CountCorrected();
Assert.True(runs.ConsumeCountersChanged());
Assert.False(runs.ConsumeCountersChanged());
Assert.Equal(1, runs.Verified);
runs.RestoreCounters(7, 3);
Assert.Equal(7, runs.Verified);
Assert.Equal(3, runs.Corrected);
}
// ── observer flow ───────────────────────────────────────────────────────
private sealed class Fixture
{
public readonly FakeSignalIngressSink Sink = new FakeSignalIngressSink();
public readonly VirtualClock Clock = new VirtualClock(T0);
public readonly PlatformScriptRunRegister Runs = new PlatformScriptRunRegister();
public Dictionary<string, SavedScriptResult> Registry = new Dictionary<string, SavedScriptResult>(StringComparer.OrdinalIgnoreCase);
public readonly ImeRegistryScriptResultObserver Observer;
public Fixture()
{
Observer = new ImeRegistryScriptResultObserver(Runs, new InformationalEventPost(Sink, Clock), null, Clock,
() => new Dictionary<string, SavedScriptResult>(Registry, StringComparer.OrdinalIgnoreCase));
}
public void Save(SavedScriptResult saved) => Registry[ImeRegistryScriptResultObserver.KeyOf(saved.UserId, saved.PolicyId)] = saved;
/// <summary>The tracker emitted a run, then kept polling past the save.</summary>
public void EmitRun(string policyId, string? stdout, int? exitCode, string runId)
{
Runs.NoteStarted(DeviceUser, policyId, Clock.UtcNow);
Runs.NoteEmitted(Emitted(policyId, stdout, exitCode, runId: runId), Clock.UtcNow);
}
public void TrackerCaughtUp() => Runs.NotePassCompleted(Clock.UtcNow);
public IReadOnlyList<FakeSignalIngressSink.PostedSignal> Corrections()
=> Sink.Posted.Where(p => p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == SharedEventTypes.ScriptOutputReconciliation).ToList();
}
[Fact]
public void Field_case_end_to_end_the_save_corrects_the_run_with_a_warning_naming_it()
{
var f = new Fixture();
f.Save(Saved("an older run of another policy", policy: PolicyB, stamp: "10/2/2026 8:00:00 AM"));
f.Observer.Tick("baseline");
f.EmitRun(PolicyA, ForeignStdout, 1, "run-a");
f.Save(Saved(OwnStdout));
f.Clock.Advance(TimeSpan.FromSeconds(1));
f.Observer.Tick("registry_change");
Assert.Empty(f.Corrections()); // the tracker has not read past the save yet
f.Clock.Advance(PlatformScriptRunRegister.TrackerCatchUp);
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
var correction = Assert.Single(f.Corrections());
var data = correction.Payload!;
Assert.Equal(PolicyA, data["policyId"]);
Assert.Equal("platform", data["scriptType"]);
Assert.Equal("run-a", data["runId"]);
Assert.Equal("foreign", data["outcome"]);
Assert.Equal("stdout_differs", data["reason"]);
Assert.Equal("Success", data["result"]);
Assert.Equal(OwnStdout, data["stdout"]);
Assert.Equal(string.Empty, data["stderr"]);
Assert.Equal("1", data["previousExitCode"]);
Assert.Equal("0", data["registryErrorCode"]);
Assert.Equal("Warning", data[SignalPayloadKeys.Severity]);
Assert.Equal("true", data[SignalPayloadKeys.ImmediateUpload]);
Assert.DoesNotContain("exitCode", data.Keys);
Assert.Equal(1, f.Runs.Corrected);
Assert.Equal(0, f.Runs.Verified);
// The same save read again changes nothing.
f.Clock.Advance(TimeSpan.FromMinutes(1));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Single(f.Corrections());
}
[Fact]
public void Confirmed_output_is_counted_and_emits_nothing()
{
var f = new Fixture();
f.Observer.Tick("baseline");
f.EmitRun(PolicyA, OwnStdout, 0, "run-a");
f.Save(Saved(OwnStdout));
f.Clock.Advance(TimeSpan.FromSeconds(1));
f.Observer.Tick("registry_change");
f.Clock.Advance(TimeSpan.FromSeconds(3));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Empty(f.Corrections());
Assert.Equal(1, f.Runs.Verified);
}
[Fact]
public void Completed_output_is_an_info_event_that_keeps_the_exit_code()
{
var f = new Fixture();
f.Observer.Tick("baseline");
f.EmitRun(PolicyA, null, 0, "run-a");
f.Save(Saved(OwnStdout));
f.Clock.Advance(TimeSpan.FromSeconds(1));
f.Observer.Tick("registry_change");
f.Clock.Advance(TimeSpan.FromSeconds(3));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
var data = Assert.Single(f.Corrections()).Payload!;
Assert.Equal("repaired", data["outcome"]);
Assert.Equal("Info", data[SignalPayloadKeys.Severity]);
Assert.DoesNotContain("previousExitCode", data.Keys);
}
[Fact]
public void A_save_seen_before_its_run_was_emitted_waits_for_the_emit()
{
var f = new Fixture();
f.Observer.Tick("baseline");
f.Runs.NoteStarted(DeviceUser, PolicyA, f.Clock.UtcNow); // result held for its end block
f.Save(Saved(OwnStdout));
f.Observer.Tick("registry_change");
f.Clock.Advance(TimeSpan.FromSeconds(3));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Equal(0, f.Runs.Verified + f.Runs.Corrected);
f.Runs.NoteEmitted(Emitted(PolicyA, ForeignStdout, 1, runId: "run-a"), f.Clock.UtcNow);
f.Clock.Advance(TimeSpan.FromSeconds(60));
f.Observer.Tick("periodic");
Assert.Equal("foreign", Assert.Single(f.Corrections()).Payload!["outcome"]);
}
[Fact]
public void A_save_no_run_claims_within_the_expiry_is_dropped()
{
var f = new Fixture();
f.Observer.Tick("baseline");
f.Save(Saved(OwnStdout));
f.Observer.Tick("registry_change");
f.Clock.Advance(ImeRegistryScriptResultObserver.PendingExpiry);
f.TrackerCaughtUp();
f.Runs.NoteStarted(DeviceUser, PolicyA, f.Clock.UtcNow);
f.Observer.Tick("periodic"); // waits for the emit — and expires
f.Runs.NoteEmitted(Emitted(PolicyA, ForeignStdout, 1, runId: "run-a"), f.Clock.UtcNow);
f.Clock.Advance(TimeSpan.FromSeconds(60));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Empty(f.Corrections());
}
[Fact]
public void Saves_from_before_the_agent_started_are_the_silent_baseline()
{
var f = new Fixture();
f.EmitRun(PolicyA, ForeignStdout, 1, "run-a");
f.Save(Saved(OwnStdout));
f.TrackerCaughtUp();
f.Observer.Tick("baseline");
f.Clock.Advance(TimeSpan.FromMinutes(1));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Empty(f.Corrections());
Assert.Equal(0, f.Runs.Verified + f.Runs.Corrected);
}
[Fact]
public void Corrections_beyond_the_cap_are_counted_but_not_emitted()
{
var f = new Fixture();
f.Observer.Tick("baseline");
var policies = Enumerable.Range(0, ImeRegistryScriptResultObserver.MaxCorrectionsPerSession + 3).Select(_ => Guid.NewGuid().ToString("D")).ToList();
foreach (var policy in policies)
{
f.EmitRun(policy, ForeignStdout, 1, "run-" + policy);
f.Save(Saved(OwnStdout, policy: policy));
}
f.Observer.Tick("registry_change");
f.Clock.Advance(TimeSpan.FromSeconds(3));
f.TrackerCaughtUp();
f.Observer.Tick("periodic");
Assert.Equal(ImeRegistryScriptResultObserver.MaxCorrectionsPerSession, f.Corrections().Count);
Assert.Equal(policies.Count, f.Runs.Corrected);
}
}
}
@@ -0,0 +1,128 @@
#nullable enable
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// The OOBE update registry snapshot (D-310, S4/S5): one readable, bounded line per key, for
/// agent.log and the <c>oobe_update_state</c> event. The registry read itself is a thin wrapper.
/// </summary>
public sealed class OobeUpdateRegistrySnapshotTests
{
private static KeyValuePair<string, object> Entry(string name, object value) => new KeyValuePair<string, object>(name, value);
[Fact]
public void Values_AreFormattedByType()
{
Assert.Equal("1", OobeUpdateRegistrySnapshot.FormatValue(1));
Assert.Equal("5000000000", OobeUpdateRegistrySnapshot.FormatValue(5000000000L));
Assert.Equal("Succeeded", OobeUpdateRegistrySnapshot.FormatValue("Succeeded"));
Assert.Equal("a|b", OobeUpdateRegistrySnapshot.FormatValue(new[] { "a", "b" }));
Assert.Equal("(binary 3 bytes)", OobeUpdateRegistrySnapshot.FormatValue(new byte[] { 1, 2, 3 }));
Assert.Equal("(null)", OobeUpdateRegistrySnapshot.FormatValue(null!));
}
[Fact]
public void LongValues_AreTruncated()
{
var formatted = OobeUpdateRegistrySnapshot.FormatValue(new string('x', 500));
Assert.Equal(OobeUpdateRegistrySnapshot.MaxValueLength + 1, formatted.Length);
Assert.EndsWith("…", formatted);
}
[Fact]
public void Entries_AreOrderedByName()
{
var line = OobeUpdateRegistrySnapshot.FormatEntries(new[]
{
Entry("ExpeditedUpdateStatus", 2),
Entry("EnableExpeditedUpdate", 1),
Entry("", "default"),
}, oobeOnly: false);
Assert.Equal("(default)=default, EnableExpeditedUpdate=1, ExpeditedUpdateStatus=2", line);
}
[Fact]
public void PolicyManagerSystemKey_KeepsOnlyOobeValues()
{
// That key holds every System policy of the device; only the OOBE ones matter here.
var line = OobeUpdateRegistrySnapshot.FormatEntries(new[]
{
Entry("AllowTelemetry", 1),
Entry("AllowOOBEUpdates", 1),
Entry("AllowOOBEUpdates_ProviderSet", 1),
}, oobeOnly: true);
Assert.Equal("AllowOOBEUpdates=1, AllowOOBEUpdates_ProviderSet=1", line);
Assert.Equal("(no OOBE values)", OobeUpdateRegistrySnapshot.FormatEntries(new[] { Entry("AllowTelemetry", 1) }, oobeOnly: true));
Assert.Equal("(no values)", OobeUpdateRegistrySnapshot.FormatEntries(new KeyValuePair<string, object>[0], oobeOnly: false));
}
[Fact]
public void ManyEntries_AreCapped_AndSaySo()
{
var entries = Enumerable.Range(0, OobeUpdateRegistrySnapshot.MaxEntries + 5)
.Select(i => Entry($"Value{i:000}", i));
var line = OobeUpdateRegistrySnapshot.FormatEntries(entries, oobeOnly: false);
Assert.EndsWith("… (+5 more)", line);
}
[Fact]
public void Describe_CountsOnlyOobeValues_OfThePolicyManagerSystemKey()
{
var policy = OobeUpdateRegistrySnapshot.Describe("policyManagerSystem", OobeUpdateRegistrySnapshot.PolicyManagerSystemKey,
new[] { Entry("AllowTelemetry", 1) }, new string[0]);
Assert.True(policy.Exists);
Assert.False(policy.HasValues);
var oobe = OobeUpdateRegistrySnapshot.Describe("policyManagerSystem", OobeUpdateRegistrySnapshot.PolicyManagerSystemKey,
new[] { Entry("AllowTelemetry", 1), Entry("AllowOOBEUpdates", 1) }, new string[0]);
Assert.True(oobe.HasValues);
Assert.Equal("AllowOOBEUpdates=1", oobe.Line);
}
[Fact]
public void Describe_SubkeysAlone_AreNoValues()
{
var state = OobeUpdateRegistrySnapshot.Describe("ndup", @"SOFTWARE\Microsoft\Windows\CurrentVersion\NDUP",
new KeyValuePair<string, object>[0], new[] { "Updates" });
Assert.False(state.HasValues);
Assert.Equal("(no values) | subkeys: Updates", state.Line);
}
[Fact]
public void Keys_HaveDistinctAliases()
{
// The aliases are the data keys of oobe_update_state.
Assert.Equal(OobeUpdateRegistrySnapshot.Keys.Length,
OobeUpdateRegistrySnapshot.Keys.Select(k => k.Alias).Distinct().Count());
}
[Fact]
public void Read_ReturnsEveryKey_FromTheRealRegistry()
{
var states = OobeUpdateRegistrySnapshot.Read();
Assert.Equal(OobeUpdateRegistrySnapshot.Keys.Select(k => k.Path), states.Select(s => s.Path));
Assert.All(states.Where(s => !s.Exists), s => Assert.False(s.HasValues));
}
[Fact]
public void Log_ToleratesMissingLoggerOrStates()
{
using var tmp = new TempDirectory();
OobeUpdateRegistrySnapshot.Log(new AgentLogger(tmp.Path, AgentLogLevel.Info), "test", OobeUpdateRegistrySnapshot.Read());
OobeUpdateRegistrySnapshot.Log(null!, "test", OobeUpdateRegistrySnapshot.Read());
OobeUpdateRegistrySnapshot.Log(new AgentLogger(tmp.Path, AgentLogLevel.Info), "test", null!);
}
}
}
@@ -0,0 +1,232 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Measurement telemetry of the OOBE update page (D-310): <c>oobe_update_page</c> per
/// update-page record and <c>oobe_update_state</c> for the registry behind it — Debug, bounded
/// per run, the state only when a key holds values and the content changed.
/// </summary>
public sealed class OobeUpdateTelemetryTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 10, 2, 9, 0, 0, DateTimeKind.Utc);
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
private IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState> _registry = Array.Empty<OobeUpdateRegistrySnapshot.KeyState>();
public void Dispose() => _tmp.Dispose();
private OobeUpdateTelemetry Build(Func<IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState>>? reader = null) =>
new OobeUpdateTelemetry("sess-oobe", "tenant-oobe",
new InformationalEventPost(_sink, new VirtualClock(At)),
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
reader ?? (() => _registry));
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal posted) =>
(IReadOnlyDictionary<string, object>)posted.TypedPayload!;
private static OobeUpdateRegistrySnapshot.KeyState Key(string alias, string? line) =>
line == null
? new OobeUpdateRegistrySnapshot.KeyState(alias, $@"SOFTWARE\{alias}", exists: false, hasValues: false, line: null)
: new OobeUpdateRegistrySnapshot.KeyState(alias, $@"SOFTWARE\{alias}", exists: true, hasValues: true, line: line);
// ------------------------------------------------------------- oobe_update_page ----
[Fact]
public void Page_IsReported_WithItsEventTime_AndFields()
{
Build().ReportPage(
new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStopped, 62405, page: "OobeNDUP", result: "success"),
At.AddMinutes(-3), isBackfill: true);
var page = Assert.Single(ByType(Constants.EventTypes.OobeUpdatePage));
Assert.Equal(At.AddMinutes(-3), page.OccurredAtUtc);
Assert.Equal("Debug", page.Payload![SignalPayloadKeys.Severity]);
var data = Data(page);
Assert.Equal("page_stopped", data["cxhEvent"]);
Assert.Equal("OobeNDUP", data["page"]);
Assert.Equal("success", data["result"]);
Assert.Equal(true, data["backfill"]);
Assert.Equal(1, data["occurrence"]);
Assert.False(data.ContainsKey("value"));
}
[Fact]
public void EventName_CarriesItsSafeValue()
{
Build().ReportPage(
new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, 62407, name: "ExpeditedUpdate_isNDUPAllowedByCSPSucceeded", value: "true"),
At, isBackfill: false);
var data = Data(Assert.Single(ByType(Constants.EventTypes.OobeUpdatePage)));
Assert.Equal("ExpeditedUpdate_isNDUPAllowedByCSPSucceeded", data["name"]);
Assert.Equal("true", data["value"]);
}
[Fact]
public void SamePageOrName_IsReportedAtMostThreeTimes()
{
var telemetry = Build();
for (var i = 0; i < 10; i++)
{
telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, 62406, name: "ExpeditedUpdate_progress"), At.AddSeconds(i), isBackfill: false);
}
telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStopped, 62405, page: "OobeNDUP", result: "success"), At.AddMinutes(1), isBackfill: false);
var pages = ByType(Constants.EventTypes.OobeUpdatePage);
Assert.Equal(OobeUpdateTelemetry.MaxPageEventsPerKey + 1, pages.Count);
Assert.Equal(new object[] { 1, 2, 3 }, pages.Take(3).Select(p => Data(p)["occurrence"]).ToArray());
// The repeats do not crowd out the page end.
Assert.Equal("page_stopped", Data(pages.Last())["cxhEvent"]);
}
[Fact]
public void PageEvents_AreCappedPerRun()
{
var telemetry = Build();
for (var i = 0; i < OobeUpdateTelemetry.MaxPageEventsPerRun + 20; i++)
{
telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, 62406, name: $"ExpeditedUpdate_step{i}"), At, isBackfill: false);
}
Assert.Equal(OobeUpdateTelemetry.MaxPageEventsPerRun, ByType(Constants.EventTypes.OobeUpdatePage).Count);
}
[Fact]
public void PageStartAndStop_AreReported_AfterTheNameBudgetIsSpent()
{
// The time attribution reads the update's span from the page start and stop.
var telemetry = Build();
for (var i = 0; i < OobeUpdateTelemetry.MaxPageEventsPerRun; i++)
{
Assert.True(telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, 62406, name: $"ExpeditedUpdate_step{i}"), At, isBackfill: false));
}
Assert.False(telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, 62406, name: "ExpeditedUpdate_late"), At, isBackfill: false));
Assert.True(telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStarted, 62404, page: "RebootNDUP"), At.AddMinutes(1), isBackfill: false));
Assert.True(telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStopped, 62405, page: "RebootNDUP", result: "success"), At.AddMinutes(2), isBackfill: false));
var pages = ByType(Constants.EventTypes.OobeUpdatePage);
Assert.Equal(OobeUpdateTelemetry.MaxPageEventsPerRun + 2, pages.Count);
Assert.Equal(new object[] { "page_started", "page_stopped" },
pages.Skip(OobeUpdateTelemetry.MaxPageEventsPerRun).Select(p => Data(p)["cxhEvent"]).ToArray());
}
[Fact]
public void SamePageStart_IsReportedAtMostThreeTimes()
{
// Outside the run budget, but a page that restarts in a loop still cannot flood.
var telemetry = Build();
var reported = Enumerable.Range(0, 10)
.Count(i => telemetry.ReportPage(new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStarted, 62404, page: "OobeNDUP"), At.AddMinutes(i), isBackfill: false));
Assert.Equal(OobeUpdateTelemetry.MaxPageEventsPerKey, reported);
}
// ------------------------------------------------------------ oobe_update_state ----
[Fact]
public void State_WithoutValues_IsNotReported()
{
_registry = new[] { Key("setupOobeNdup", null), Key("policiesOobe", null) };
Build().ReportState(OobeUpdateTelemetry.Moments.AgentStart);
Assert.Empty(ByType(Constants.EventTypes.OobeUpdateState));
}
[Fact]
public void State_ReportsOnlyKeysWithValues_UnderTheirAlias()
{
_registry = new[]
{
Key("setupOobeNdup", "EnableExpeditedUpdate=1, EnableExpeditedUpdateSyncDownloadCompleted=1"),
Key("ndup", null),
new OobeUpdateRegistrySnapshot.KeyState("policyManagerSystem", "x", exists: true, hasValues: false, line: "(no OOBE values)"),
Key("espSetupPolicy", "InstallQualityUpdates=1"),
};
Build().ReportState(OobeUpdateTelemetry.Moments.UpdatePageStopped, "OobeNDUP");
var state = Assert.Single(ByType(Constants.EventTypes.OobeUpdateState));
Assert.Equal("Debug", state.Payload![SignalPayloadKeys.Severity]);
var data = Data(state);
Assert.Equal("update_page_stopped", data["moment"]);
Assert.Equal("OobeNDUP", data["page"]);
Assert.Equal(2, data["keysWithValues"]);
Assert.Equal("EnableExpeditedUpdate=1, EnableExpeditedUpdateSyncDownloadCompleted=1", data["setupOobeNdup"]);
Assert.Equal("InstallQualityUpdates=1", data["espSetupPolicy"]);
Assert.False(data.ContainsKey("ndup"));
Assert.False(data.ContainsKey("policyManagerSystem"));
}
[Fact]
public void UnchangedState_IsReportedOnce_AChangeAgain()
{
var telemetry = Build();
_registry = new[] { Key("espSetupPolicy", "InstallQualityUpdates=1") };
telemetry.ReportState(OobeUpdateTelemetry.Moments.AgentStart);
telemetry.ReportState(OobeUpdateTelemetry.Moments.UpdatePageStarted, "OobeNDUP");
_registry = new[] { Key("espSetupPolicy", "InstallQualityUpdates=1"), Key("setupOobeNdup", "EnableExpeditedUpdate=1") };
telemetry.ReportState(OobeUpdateTelemetry.Moments.UpdatePageStopped, "OobeNDUP");
telemetry.ReportState(OobeUpdateTelemetry.Moments.AgentStop);
var states = ByType(Constants.EventTypes.OobeUpdateState);
Assert.Equal(new object[] { "agent_start", "update_page_stopped" }, states.Select(s => Data(s)["moment"]).ToArray());
}
[Fact]
public void StateEvents_AreCappedPerRun()
{
var telemetry = Build();
for (var i = 0; i < OobeUpdateTelemetry.MaxStateEventsPerRun + 4; i++)
{
_registry = new[] { Key("setupOobeNdup", $"ExpeditedUpdateStatus={i}") };
telemetry.ReportState(OobeUpdateTelemetry.Moments.UpdatePageStarted, "OobeNDUP");
}
Assert.Equal(OobeUpdateTelemetry.MaxStateEventsPerRun, ByType(Constants.EventTypes.OobeUpdateState).Count);
}
[Fact]
public void LongLines_AreTruncated()
{
_registry = new[] { Key("ndupUpdates", new string('x', 3000)) };
Build().ReportState(OobeUpdateTelemetry.Moments.AgentStart);
var line = (string)Data(Assert.Single(ByType(Constants.EventTypes.OobeUpdateState)))["ndupUpdates"];
Assert.Equal(OobeUpdateTelemetry.MaxStateLineLength + 1, line.Length);
Assert.EndsWith("…", line);
}
[Fact]
public void FailingRegistryRead_ReportsNothing_AndDoesNotThrow()
{
Build(() => throw new UnauthorizedAccessException("denied")).ReportState(OobeUpdateTelemetry.Moments.AgentStart);
Assert.Empty(_sink.Posted);
}
}
}
@@ -0,0 +1,289 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// ServicingTracker (D-310) — the CBS servicing steps of OS updates from the Setup log. The
/// sequence and XML shapes are those Windows 11 writes for a cumulative update installed by
/// <c>UpdateAgentLCU</c>: initiate Absent → Staged, Staged reached, initiate Staged → Installed,
/// restart required, Installed reached after the restart.
/// </summary>
public sealed class ServicingTrackerTests : IDisposable
{
private static readonly DateTime T0 = new DateTime(2026, 9, 25, 18, 52, 37, DateTimeKind.Utc);
private const string Kb = "KB5099999";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
private readonly UpdateActivityTally _tally = new UpdateActivityTally();
private readonly List<(string Step, string Update, DateTime At)> _osActivity = new List<(string, string, DateTime)>();
private readonly ServicingTracker _tracker;
private long _nextRecordId = 1200;
public ServicingTrackerTests()
{
_tracker = NewTracker(stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private ServicingTracker NewTracker(string? stateDirectory) =>
new ServicingTracker(
sessionId: "sess-cbs",
tenantId: "tenant-cbs",
post: new InformationalEventPost(_sink, new VirtualClock(T0)),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
backfillEnabled: false,
stateDirectory: stateDirectory,
tally: _tally,
onOsUpdateActivity: (step, update, at) => _osActivity.Add((step, update, at)));
private void Process(
int eventId,
DateTime at,
string? initialState = null,
string? targetState = "Installed",
string? errorCode = null,
string? client = "UpdateAgentLCU",
string? package = Kb,
long? recordId = null,
ServicingTracker? tracker = null,
bool isBackfill = false)
{
(tracker ?? _tracker).ProcessEvent(
eventId: eventId,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: at,
package: package,
initialState: initialState,
targetState: targetState,
errorCode: errorCode,
client: client,
isBackfill: isBackfill);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> Servicing() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.WindowsUpdateServicing).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void CumulativeUpdateSequence_ListsEveryStep_AndReportsTheEngineSteps()
{
Process(ServicingTracker.EventId_InitiatingChanges, T0, initialState: "Absent", targetState: "Staged");
Process(ServicingTracker.EventId_StateReached, T0.AddMinutes(4), targetState: "Staged", errorCode: "0x0");
Process(ServicingTracker.EventId_InitiatingChanges, T0.AddMinutes(29), initialState: "Staged", targetState: "Installed");
Process(ServicingTracker.EventId_RebootRequired, T0.AddMinutes(33), targetState: "Installed", errorCode: "0x0");
// Written after the restart.
Process(ServicingTracker.EventId_StateReached, T0.AddMinutes(41), targetState: "Installed", errorCode: "0x0");
var rows = Servicing();
Assert.Equal(
new[] { "initiating", "state_reached", "initiating", "reboot_required", "state_reached" },
rows.Select(s => (string)Data(s)["step"]).ToArray());
Assert.All(rows, s => Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]));
Assert.All(rows, s => Assert.Equal("ServicingWatcher", s.Payload![SignalPayloadKeys.Source]));
Assert.All(rows, s => Assert.Equal(Kb, Data(s)["package"]));
Assert.All(rows, s => Assert.Equal("UpdateAgentLCU", Data(s)["client"]));
// ErrorCode 0x0 is success, not a decoded HRESULT.
Assert.All(rows, s => Assert.False(Data(s).ContainsKey("hresult")));
Assert.Equal("Absent", Data(rows[0])["initialState"]);
Assert.Equal("Staged", Data(rows[0])["targetState"]);
Assert.Equal(T0.AddMinutes(33), rows[3].OccurredAtUtc);
Assert.Contains("needs a restart", rows[3].Payload![SignalPayloadKeys.Message]);
Assert.Equal(
new[]
{
OsUpdateSteps.StagingStarted, OsUpdateSteps.Staged, OsUpdateSteps.InstallStarted,
OsUpdateSteps.RebootRequired, OsUpdateSteps.Installed,
},
_osActivity.Select(a => a.Step).ToArray());
Assert.All(_osActivity, a => Assert.Equal(Kb, a.Update));
Assert.Equal(T0.AddMinutes(29), _osActivity[2].At);
}
[Theory]
[InlineData("CbsTask")]
[InlineData("DISM Package Manager Provider")]
[InlineData("")]
[InlineData(null)]
public void NonOsClients_AreCountedNotListed(string? client)
{
// CBS maintenance writes hundreds of these in seconds; language packs and features on
// demand installed by apps are not the OOBE update either.
Process(ServicingTracker.EventId_InitiatingChanges, T0, initialState: "Staged", targetState: "Installed",
client: client, package: "Microsoft-Windows-Telnet-Client-FOD-en-US");
Process(ServicingTracker.EventId_StateReached, T0, targetState: "Installed", errorCode: "0x0",
client: client, package: "Microsoft-Windows-Telnet-Client-FOD-en-US");
Assert.Empty(_sink.Posted);
Assert.Empty(_osActivity);
var counts = _tally.Snapshot().ToDictionary(kv => kv.Key, kv => kv.Value);
Assert.Equal(1, counts["servicing_other_1"]);
Assert.Equal(1, counts["servicing_other_2"]);
}
[Fact]
public void WindowsUpdateAgentClient_IsAnOsClient()
{
Process(ServicingTracker.EventId_RebootRequired, T0, client: "WindowsUpdateAgent", errorCode: "0x0");
Assert.Single(Servicing());
Assert.Equal(OsUpdateSteps.RebootRequired, Assert.Single(_osActivity).Step);
}
[Fact]
public void ChangeFailed_EmitsWarning_WithDecodedHResult()
{
Process(ServicingTracker.EventId_ChangeFailed, T0, targetState: "Installed", errorCode: "0x800f0922");
var s = Assert.Single(Servicing());
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
var data = Data(s);
Assert.Equal("failed", data["step"]);
Assert.Equal("0x800F0922", data["hresult"]);
Assert.Equal("CBS_E_INSTALLERS_FAILED", data["hresultSymbol"]);
Assert.Contains("FAILED", s.Payload![SignalPayloadKeys.Message]);
Assert.Equal(OsUpdateSteps.Failed, Assert.Single(_osActivity).Step);
}
[Fact]
public void PartiallyInstalled_IsAFailure_AndSaysSo()
{
Process(ServicingTracker.EventId_PartiallyInstalled, T0, targetState: "Installed", errorCode: "0x800f0922");
var s = Assert.Single(Servicing());
Assert.Equal("failed", Data(s)["step"]);
Assert.Contains("partially installed", s.Payload![SignalPayloadKeys.Message]);
Assert.Equal(OsUpdateSteps.Failed, Assert.Single(_osActivity).Step);
}
[Fact]
public void ChangeTowardsAnotherState_IsListed_ButIsNoEngineStep()
{
// Removing a superseded package is servicing, not progress of the update install.
Process(ServicingTracker.EventId_InitiatingChanges, T0, initialState: "Installed", targetState: "Absent");
Assert.Single(Servicing());
Assert.Empty(_osActivity);
}
[Fact]
public void BackfilledRecord_KeepsItsEventTime()
{
var earlier = T0.AddMinutes(-20);
Process(ServicingTracker.EventId_InitiatingChanges, earlier, initialState: "Absent", targetState: "Staged", isBackfill: true);
var s = Assert.Single(Servicing());
Assert.Equal(earlier, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
Assert.Equal(earlier, Assert.Single(_osActivity).At);
}
[Fact]
public void Watermark_SkipsEmittedRecordsAfterRestart()
{
var first = NewTracker(_tmp.Path);
Process(ServicingTracker.EventId_RebootRequired, T0, errorCode: "0x0", recordId: 1205, tracker: first);
var second = NewTracker(_tmp.Path);
second.LoadWatermark();
Process(ServicingTracker.EventId_RebootRequired, T0, errorCode: "0x0", recordId: 1205, tracker: second, isBackfill: true);
Process(ServicingTracker.EventId_StateReached, T0.AddMinutes(8), errorCode: "0x0", recordId: 1207, tracker: second);
Assert.Equal(new[] { "reboot_required", "state_reached" }, Servicing().Select(s => (string)Data(s)["step"]).ToArray());
}
[Fact]
public void SameRecordDeliveredTwice_IsListedOnce()
{
Process(ServicingTracker.EventId_RebootRequired, T0, errorCode: "0x0", recordId: 1205);
Process(ServicingTracker.EventId_RebootRequired, T0, errorCode: "0x0", recordId: 1205);
Assert.Single(Servicing());
Assert.Single(_osActivity);
}
[Fact]
public void ParseUserData_ReadsBothCbsPayloadShapes()
{
const string initiate =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Servicing' Guid='{bd12f3b8-fc40-4a61-a307-b7a013a069c1}'/>" +
"<EventID>1</EventID><Version>0</Version><Level>0</Level><Task>1</Task><Opcode>0</Opcode><Keywords>0x8000000000000000</Keywords>" +
"<TimeCreated SystemTime='2026-09-25T19:21:42.4486782Z'/><EventRecordID>1203</EventRecordID><Channel>Setup</Channel><Computer>DESKTOP-TEST</Computer>" +
"<Security UserID='S-1-5-18'/></System><UserData><CbsPackageInitiateChanges xmlns='http://manifests.microsoft.com/win/2004/08/windows/setup_provider'>" +
"<PackageIdentifier>KB5099999</PackageIdentifier><InitialPackageState>5064</InitialPackageState><InitialPackageStateTextized>Staged</InitialPackageStateTextized>" +
"<IntendedPackageState>5112</IntendedPackageState><IntendedPackageStateTextized>Installed</IntendedPackageStateTextized><Client>UpdateAgentLCU</Client>" +
"</CbsPackageInitiateChanges></UserData></Event>";
const string changeState =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Servicing' Guid='{bd12f3b8-fc40-4a61-a307-b7a013a069c1}'/>" +
"<EventID>4</EventID><TimeCreated SystemTime='2026-09-25T19:26:07.1381462Z'/><EventRecordID>1205</EventRecordID><Channel>Setup</Channel></System>" +
"<UserData><CbsPackageChangeState xmlns='http://manifests.microsoft.com/win/2004/08/windows/setup_provider'><PackageIdentifier>KB5099999</PackageIdentifier>" +
"<IntendedPackageState>5112</IntendedPackageState><IntendedPackageStateTextized>Installed</IntendedPackageStateTextized><ErrorCode>0x0</ErrorCode>" +
"<Client>UpdateAgentLCU</Client></CbsPackageChangeState></UserData></Event>";
var a = ServicingTracker.ParseUserData(initiate);
Assert.Equal("KB5099999", a["PackageIdentifier"]);
Assert.Equal("Staged", a["InitialPackageStateTextized"]);
Assert.Equal("Installed", a["IntendedPackageStateTextized"]);
Assert.Equal("UpdateAgentLCU", a["client"]); // case-insensitive
Assert.False(a.ContainsKey("ErrorCode"));
var b = ServicingTracker.ParseUserData(changeState);
Assert.Equal("Installed", b["IntendedPackageStateTextized"]);
Assert.Equal("0x0", b["ErrorCode"]);
Assert.False(b.ContainsKey("InitialPackageStateTextized"));
}
[Fact]
public void ParseUserData_MalformedOrWithoutUserData_ReturnsEmpty_NeverThrows()
{
Assert.Empty(ServicingTracker.ParseUserData(null));
Assert.Empty(ServicingTracker.ParseUserData(""));
Assert.Empty(ServicingTracker.ParseUserData("<not-xml"));
Assert.Empty(ServicingTracker.ParseUserData("<Event><System><EventID>1</EventID></System></Event>"));
}
[Fact]
public void Watermark_PersistsOnlyAfterTheEngineSignalWasHandedOver()
{
// "Reboot required" is written right before the update's restart: its engine signal
// must be on its way before the watermark tells the next run's backfill to skip it.
var path = System.IO.Path.Combine(_tmp.Path, ServicingTracker.WatermarkStateFileName);
bool? watermarkOnDiskAtSignal = null;
var tracker = new ServicingTracker(
sessionId: "sess-cbs",
tenantId: "tenant-cbs",
post: new InformationalEventPost(_sink, new VirtualClock(T0)),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
backfillEnabled: false,
stateDirectory: _tmp.Path,
tally: _tally,
onOsUpdateActivity: (step, update, at) => watermarkOnDiskAtSignal = System.IO.File.Exists(path));
Process(ServicingTracker.EventId_RebootRequired, T0, errorCode: "0x0", tracker: tracker);
Assert.False(watermarkOnDiskAtSignal);
Assert.True(System.IO.File.Exists(path));
}
}
}
@@ -0,0 +1,123 @@
#nullable enable
using System;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// A tenant-extended Hello wait runs up to 60 min — the default session_stalled threshold. While
/// it is pending the session waits on a bounded timer that resolves it either way, so the stall
/// probe holds the report back instead of flagging the session Stalled; once the wait is over
/// and the session is still idle the report fires, and real activity drops it. The collector
/// runs with no scan sources, so only the session_stalled decision is exercised.
/// </summary>
public sealed class StallProbeHelloWaitGuardTests
{
private static readonly DateTime Fixed = new DateTime(2026, 9, 29, 8, 0, 0, DateTimeKind.Utc);
private const double IdleAtStallThreshold = 60;
[Fact]
public void Session_stalled_is_held_back_while_the_hello_wait_is_pending()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
var collector = NewCollector(tmp, ingress, helloWaitPending: () => true);
collector.CheckAndRunProbes(IdleAtStallThreshold);
collector.CheckAndRunProbes(IdleAtStallThreshold + 1);
Assert.Equal(0, StalledCount(ingress));
}
[Fact]
public void Held_back_report_fires_once_the_hello_wait_is_over_and_the_session_is_still_idle()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
var pending = true;
var collector = NewCollector(tmp, ingress, helloWaitPending: () => pending);
collector.CheckAndRunProbes(IdleAtStallThreshold);
Assert.Equal(0, StalledCount(ingress));
// hello_safety resolved (a deadline fire is no activity, so no ResetProbes): the next
// tick emits the held-back report — once.
pending = false;
collector.CheckAndRunProbes(IdleAtStallThreshold + 1);
collector.CheckAndRunProbes(IdleAtStallThreshold + 2);
Assert.Equal(1, StalledCount(ingress));
}
[Fact]
public void Real_activity_drops_the_held_back_report()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
var pending = true;
var collector = NewCollector(tmp, ingress, helloWaitPending: () => pending);
collector.CheckAndRunProbes(IdleAtStallThreshold);
pending = false;
collector.ResetProbes();
collector.CheckAndRunProbes(1);
Assert.Equal(0, StalledCount(ingress));
// A new idle window reaches the threshold again — the fire-once report is still unused.
collector.CheckAndRunProbes(IdleAtStallThreshold);
Assert.Equal(1, StalledCount(ingress));
}
[Fact]
public void Without_a_hello_wait_probe_session_stalled_fires_as_before()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
var collector = NewCollector(tmp, ingress, helloWaitPending: null);
collector.CheckAndRunProbes(IdleAtStallThreshold);
Assert.Equal(1, StalledCount(ingress));
}
[Fact]
public void A_throwing_hello_wait_probe_never_costs_the_stall_report()
{
using var tmp = new TempDirectory();
var ingress = new FakeSignalIngressSink();
var collector = NewCollector(tmp, ingress, helloWaitPending: () => throw new InvalidOperationException("probe down"));
collector.CheckAndRunProbes(IdleAtStallThreshold);
Assert.Equal(1, StalledCount(ingress));
}
private static StallProbeCollector NewCollector(TempDirectory tmp, FakeSignalIngressSink ingress, Func<bool>? helloWaitPending) =>
new StallProbeCollector(
sessionId: "S1",
tenantId: "T1",
post: new InformationalEventPost(ingress, new VirtualClock(Fixed)),
logger: new AgentLogger(tmp.Path, AgentLogLevel.Info),
thresholdsMinutes: new[] { 60 },
traceIndices: Array.Empty<int>(),
sources: Array.Empty<string>(),
sessionStalledAfterProbeIndex: 1,
helloWaitPending: helloWaitPending);
private static int StalledCount(FakeSignalIngressSink ingress) =>
ingress.Posted.Count(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.SessionStalled);
}
}
@@ -0,0 +1,85 @@
#nullable enable
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// WindowsUpdateClassifier (D-310) — decides which Windows Update client events are listed one
/// by one (OS updates) and which are only counted (Store, Defender, everything else). The
/// titles are the shapes the WU client writes on Windows 11 GA and Insider builds.
/// </summary>
public sealed class WindowsUpdateClassifierTests
{
[Theory]
[InlineData("2026-09 Cumulative Update for Windows 11, version 25H2 for x64-based Systems (KB5099999)")]
[InlineData("2026-09 Cumulative Update Preview for Windows 11, version 24H2 for x64-based Systems (KB5099998)")]
[InlineData("2026-09 Dynamic Update for Windows 11, version 25H2 for x64-based Systems (KB5099997)")]
[InlineData("2026-09 Servicing Stack Update for Windows 11, version 25H2 for x64-based Systems (KB5099996)")]
[InlineData("Windows 11 Insider Preview Quality Update (26220.9568)")]
[InlineData("Windows 11 Insider Preview Feature Update (26220.9022)")]
[InlineData("Windows 11, version 25H2 Upgrade")]
[InlineData("2026-09 .NET Framework Security Update (KB5099995)")]
[InlineData("2026-08 .NET Framework Preview Update (KB5099994)")]
[InlineData("Windows Malicious Software Removal Tool x64 - v5.145 (KB890830)")]
public void WindowsAndDotNetFrameworkUpdates_AreOs(string title)
{
Assert.Equal(WindowsUpdateClassifier.Os, WindowsUpdateClassifier.Classify(title, serviceGuid: null));
}
[Theory]
[InlineData("Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.459.505.0) - Current Channel (Broad)")]
[InlineData("Update for Microsoft Defender Antivirus antimalware platform - KB4052623 (Version 4.18.26080.4) - Current Channel (Staged)")]
// Names "Windows" and "Update" — Defender must still win.
[InlineData("Windows 10/11 - Update for Microsoft Defender for Endpoint - KB5005292 (Version 10.8838.26060.15013) - Current Channel (Staged)")]
public void DefenderUpdates_AreDefender(string title)
{
Assert.Equal(WindowsUpdateClassifier.Defender, WindowsUpdateClassifier.Classify(title, serviceGuid: null));
}
[Theory]
[InlineData("9NSTH9KHZDLQ-Microsoft.UI.Xaml.2.8")]
[InlineData("9NBLGGH4NNS1-Microsoft.DesktopAppInstaller")]
// Upper-case "WINDOWS" inside the package name must not make it an OS update.
[InlineData("9WZDNCRFHVN5-MICROSOFT.WINDOWSCALCULATOR")]
[InlineData("9WZDNCRFHWKN-MICROSOFT.WINDOWSSOUNDRECORDER")]
[InlineData("ApplicationSet-9WZDNCRFJ3Q2-Microsoft.BingWeather")]
public void StoreProductTitles_AreStore(string title)
{
Assert.Equal(WindowsUpdateClassifier.Store, WindowsUpdateClassifier.Classify(title, serviceGuid: null));
}
[Theory]
[InlineData("{855e8a7c-ecb4-4ca3-b045-1dfa50104289}")]
[InlineData("{855E8A7C-ECB4-4CA3-B045-1DFA50104289}")]
[InlineData("855e8a7c-ecb4-4ca3-b045-1dfa50104289")]
public void StoreServiceGuid_IsStore_WhateverTheTitle(string serviceGuid)
{
Assert.Equal(WindowsUpdateClassifier.Store,
WindowsUpdateClassifier.Classify("2026-09 Cumulative Update for Windows 11 (KB5099999)", serviceGuid));
}
[Fact]
public void OtherServiceGuid_LeavesTheTitleInCharge()
{
Assert.Equal(WindowsUpdateClassifier.Os,
WindowsUpdateClassifier.Classify("2026-09 Cumulative Update for Windows 11 (KB5099999)", "{7971f918-a847-4430-9279-4a52d1efe18d}"));
}
[Theory]
[InlineData("Intel net Driver Update (24.50.0.4)")]
[InlineData("PowerShell v7.6.6 (x64)")]
[InlineData("Remote help 5.2.1040.0")]
[InlineData("Microsoft Edge-Stable Channel Version 140 Update for x64 based Edge-Chromium (Build 140.0.3485.54)")]
[InlineData("Microsoft .NET 8.0.20 Security Update for x64 Client (KB5099993)")]
[InlineData("Windows Feature Experience Pack 1000.26100.234.0")]
[InlineData("KB5099999")]
[InlineData("")]
[InlineData(" ")]
[InlineData(null)]
public void EverythingElse_IsOther(string? title)
{
Assert.Equal(WindowsUpdateClassifier.Other, WindowsUpdateClassifier.Classify(title!, serviceGuid: null));
}
}
}
@@ -1,6 +1,7 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
@@ -9,55 +10,81 @@ using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared;
using Newtonsoft.Json.Linq;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// WindowsUpdateTracker — surfaces quality/cumulative update activity during OOBE from the
/// WindowsUpdateClient/Operational channel. Drives the primitive <see cref="WindowsUpdateTracker.ProcessEvent"/>
/// WindowsUpdateClient provider: install events from the System log, scan/download events from
/// the Operational channel (D-310). Drives the primitive <see cref="WindowsUpdateTracker.ProcessEvent"/>
/// test seam (no real EventRecord, which is abstract + Windows-only), mirroring the
/// ModernDeploymentTracker test pattern.
/// ModernDeploymentTracker test pattern. Update titles are the shapes the provider writes.
/// </summary>
public sealed class WindowsUpdateTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 7, 8, 10, 15, 0, DateTimeKind.Utc);
private const string Lcu = "2026-09 Cumulative Update for Windows 11, version 25H2 for x64-based Systems (KB5099999)";
private const string StoreApp = "9WZDNCRFHVN5-MICROSOFT.WINDOWSCALCULATOR";
private const string DefenderIntelligence = "Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.459.505.0) - Current Channel (Broad)";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly WindowsUpdateTracker _tracker;
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
private readonly UpdateActivityTally _tally = new UpdateActivityTally();
private readonly List<(string Step, string Update, DateTime At)> _osActivity = new List<(string, string, DateTime)>();
private readonly WindowsUpdateTracker _system;
private readonly WindowsUpdateTracker _operational;
private long _nextRecordId = 1;
public WindowsUpdateTrackerTests()
{
_sink = new FakeSignalIngressSink();
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_tracker = new WindowsUpdateTracker(
sessionId: "sess-wu",
tenantId: "tenant-wu",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: null); // in-memory dedup for most tests
// In-memory dedup for most tests.
_system = NewTracker(WindowsUpdateTracker.SystemChannel, stateDirectory: null);
_operational = NewTracker(channel: null, stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private void Emit(int eventId, string? errorCode = null, string? updateTitle = "Cumulative Update KB5099999", long? recordId = null)
private WindowsUpdateTracker NewTracker(string? channel, string? stateDirectory) =>
new WindowsUpdateTracker(
sessionId: "sess-wu",
tenantId: "tenant-wu",
post: new InformationalEventPost(_sink, new VirtualClock(At)),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
backfillEnabled: false,
stateDirectory: stateDirectory,
channel: channel,
tally: _tally,
onOsUpdateActivity: (step, update, at) => _osActivity.Add((step, update, at)));
private void Emit(
WindowsUpdateTracker tracker,
int eventId,
string? updateTitle = Lcu,
string? errorCode = null,
long? recordId = null,
string? serviceGuid = null,
string? updateCount = null,
DateTime? at = null,
bool isBackfill = false)
{
_tracker.ProcessEvent(
tracker.ProcessEvent(
eventId: eventId,
level: 4,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: At,
timeCreatedUtc: at ?? At,
updateTitle: updateTitle,
updateGuid: "{8b1c8726-1111-2222-3333-444455556666}",
updateRevisionNumber: "200",
errorCode: errorCode,
formattedDescription: null,
isBackfill: false);
isBackfill: isBackfill,
serviceGuid: serviceGuid,
updateCount: updateCount);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
@@ -70,10 +97,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
private int Counted(string key) =>
_tally.Snapshot().Where(kv => kv.Key == key).Select(kv => kv.Value).SingleOrDefault();
// ---------------------------------------------------------------------------
// Install events (System log)
// ---------------------------------------------------------------------------
[Fact]
public void FailureEvent_EmitsWindowsUpdateFailed_WithDecodedHResult_AndImmediateUpload()
{
Emit(WindowsUpdateTracker.EventId_InstallFailure, errorCode: "0x80240022");
Emit(_system, WindowsUpdateTracker.EventId_InstallFailure, errorCode: "0x80240022");
var failed = ByType(Constants.EventTypes.WindowsUpdateFailed);
var s = Assert.Single(failed);
@@ -83,37 +117,44 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
var data = Data(s);
Assert.Equal("0x80240022", data["hresult"]);
Assert.Equal("WU_E_ALL_UPDATES_FAILED", data["hresultSymbol"]);
Assert.Equal("Cumulative Update KB5099999", data["updateTitle"]);
Assert.Equal(Lcu, data["updateTitle"]);
Assert.Equal(20, data["wuEventId"]);
Assert.Equal("system", data["wuChannel"]);
Assert.Equal("install", data["wuPhase"]);
Assert.Equal(WindowsUpdateClassifier.Os, data["updateClass"]);
}
[Fact]
public void SuccessEvent_EmitsWindowsUpdateSucceeded_InfoSeverity()
{
Emit(WindowsUpdateTracker.EventId_InstallSuccess);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess);
var ok = ByType(Constants.EventTypes.WindowsUpdateSucceeded);
var s = Assert.Single(ok);
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("false", s.Payload![SignalPayloadKeys.ImmediateUpload]);
Assert.Equal("installed", Data(s)["wuPhase"]);
}
[Fact]
public void StartedAndDownloadEvents_EmitDebugStarted()
public void StartedAndDownloadEvents_EmitInfoStarted_WithTheirPhase()
{
Emit(WindowsUpdateTracker.EventId_InstallStarted);
Emit(WindowsUpdateTracker.EventId_DownloadStarted);
// Info, not Debug: with only OS updates listed they are the visible progress of a
// long OOBE update.
Emit(_system, WindowsUpdateTracker.EventId_InstallStarted);
Emit(_system, WindowsUpdateTracker.EventId_DownloadStarted);
var started = ByType(Constants.EventTypes.WindowsUpdateStarted);
Assert.Equal(2, started.Count);
Assert.All(started, s => Assert.Equal("Debug", s.Payload![SignalPayloadKeys.Severity]));
Assert.All(started, s => Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]));
Assert.Equal(new[] { "install", "download" }, started.Select(s => (string)Data(s)["wuPhase"]).ToArray());
}
[Fact]
public void SameRecordId_ProcessedTwice_EmitsOnce()
{
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: 42);
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: 42);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 42);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 42);
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateSucceeded));
}
@@ -125,8 +166,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
// with a higher RecordId can be processed first. An older, never-emitted backfill event
// (lower RecordId) must NOT be suppressed as "already processed" — these early pre-agent
// OOBE updates are the whole point of the feature. A high-water-mark dedup dropped them.
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: 100); // live, high RecordId
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: 50); // backfill, older, never seen
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 100); // live, high RecordId
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 50); // backfill, older, never seen
Assert.Equal(2, ByType(Constants.EventTypes.WindowsUpdateSucceeded).Count);
}
@@ -138,53 +179,212 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
// time, not at agent start (the ctor's default UtcNow). InformationalEventPost forwards
// EnrollmentEvent.Timestamp as the signal's OccurredAtUtc, which drives the timeline entry.
var eventTime = new DateTime(2026, 7, 8, 7, 5, 0, DateTimeKind.Utc); // well before `At`
_tracker.ProcessEvent(
eventId: WindowsUpdateTracker.EventId_InstallFailure,
level: 4,
recordId: 7,
timeCreatedUtc: eventTime,
updateTitle: "KB5099999",
updateGuid: null,
updateRevisionNumber: null,
errorCode: "0x80240022",
formattedDescription: null,
isBackfill: true);
Emit(_system, WindowsUpdateTracker.EventId_InstallFailure, errorCode: "0x80240022",
recordId: 7, at: eventTime, isBackfill: true);
var s = Assert.Single(ByType(Constants.EventTypes.WindowsUpdateFailed));
Assert.Equal(eventTime, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
}
[Fact]
public void NegativeRecordId_IsNeverDeduped()
{
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: -1);
Emit(WindowsUpdateTracker.EventId_InstallSuccess, recordId: -1);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: -1);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: -1);
Assert.Equal(2, ByType(Constants.EventTypes.WindowsUpdateSucceeded).Count);
}
// ---------------------------------------------------------------------------
// Scan/download events (Operational channel)
// ---------------------------------------------------------------------------
[Fact]
public void Watermark_PersistsAcrossTrackerInstances()
public void Downloaded_EmitsInfoStarted_WithDownloadedPhase()
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
Emit(_operational, WindowsUpdateTracker.EventId_Downloaded);
var first = new WindowsUpdateTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
first.ProcessEvent(WindowsUpdateTracker.EventId_InstallSuccess, 4, recordId: 500, timeCreatedUtc: At,
updateTitle: "KB1", updateGuid: null, updateRevisionNumber: null, errorCode: null,
formattedDescription: null, isBackfill: false);
var s = Assert.Single(ByType(Constants.EventTypes.WindowsUpdateStarted));
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("downloaded", Data(s)["wuPhase"]);
Assert.Equal("operational", Data(s)["wuChannel"]);
}
// A fresh tracker (simulating an agent restart) must load the watermark and skip the
// already-emitted record when the OOBE backfill re-reads it.
var second = new WindowsUpdateTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(WindowsUpdateTracker.EventId_InstallSuccess, 4, recordId: 500, timeCreatedUtc: At,
updateTitle: "KB1", updateGuid: null, updateRevisionNumber: null, errorCode: null,
formattedDescription: null, isBackfill: true);
[Fact]
public void DownloadFailure_OfAnOsUpdate_EmitsWarning_WithDownloadPhase()
{
Emit(_operational, WindowsUpdateTracker.EventId_DownloadFailed, errorCode: "0x80248007");
var s = Assert.Single(ByType(Constants.EventTypes.WindowsUpdateFailed));
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("false", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal("download", data["wuPhase"]);
Assert.Equal("0x80248007", data["hresult"]);
}
[Fact]
public void ScanFound_EmitsOnlyChangedCounts_AndCountsTheRepeats()
{
foreach (var count in new[] { "3", "3", "5", "5", "0" })
Emit(_operational, WindowsUpdateTracker.EventId_ScanFound, updateTitle: null, updateCount: count);
var found = ByType(Constants.EventTypes.WindowsUpdateStarted);
Assert.Equal(new[] { "3", "5", "0" }, found.Select(s => (string)Data(s)["updateCount"]).ToArray());
Assert.All(found, s => Assert.Equal("Debug", s.Payload![SignalPayloadKeys.Severity]));
Assert.All(found, s => Assert.Equal("scan", Data(s)["wuPhase"]));
Assert.All(found, s => Assert.False(Data(s).ContainsKey("updateClass")));
Assert.Contains("scan found 3 update(s)", found[0].Payload![SignalPayloadKeys.Message]);
Assert.Equal(2, Counted("scan_found_repeat"));
Assert.Empty(_osActivity); // a scan result is not an update step
}
[Fact]
public void ScanFound_IsCappedPerRun()
{
for (var i = 0; i < WindowsUpdateTracker.MaxScanFoundEmissions + 2; i++)
Emit(_operational, WindowsUpdateTracker.EventId_ScanFound, updateTitle: null, updateCount: i.ToString());
Assert.Equal(WindowsUpdateTracker.MaxScanFoundEmissions, ByType(Constants.EventTypes.WindowsUpdateStarted).Count);
Assert.Equal(2, Counted("scan_found_repeat"));
}
[Fact]
public void ScanFailed_EmitsEachErrorCodeOnce()
{
Emit(_operational, WindowsUpdateTracker.EventId_ScanFailed, updateTitle: null, errorCode: "0x8024401c");
Emit(_operational, WindowsUpdateTracker.EventId_ScanFailed, updateTitle: null, errorCode: "0x8024401c");
Emit(_operational, WindowsUpdateTracker.EventId_ScanFailed, updateTitle: null, errorCode: "0x80072EE2");
Emit(_operational, WindowsUpdateTracker.EventId_ScanFailed, updateTitle: null, errorCode: "0x8024401C");
var failed = ByType(Constants.EventTypes.WindowsUpdateFailed);
Assert.Equal(2, failed.Count);
Assert.All(failed, s => Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]));
Assert.All(failed, s => Assert.Equal("scan", Data(s)["wuPhase"]));
Assert.Equal(2, Counted("scan_failed_repeat"));
Assert.Empty(_osActivity);
}
[Fact]
public void ScanFailed_IsCappedPerRun()
{
for (var i = 0; i < WindowsUpdateTracker.MaxScanFailedEmissions + 2; i++)
Emit(_operational, WindowsUpdateTracker.EventId_ScanFailed, updateTitle: null, errorCode: $"0x8024000{i}");
Assert.Equal(WindowsUpdateTracker.MaxScanFailedEmissions, ByType(Constants.EventTypes.WindowsUpdateFailed).Count);
Assert.Equal(2, Counted("scan_failed_repeat"));
}
// ---------------------------------------------------------------------------
// Classification — only OS updates are listed, the rest is counted
// ---------------------------------------------------------------------------
[Theory]
[InlineData(StoreApp, null, WindowsUpdateTracker.EventId_InstallSuccess, "store_19")]
[InlineData("Some App", "{855e8a7c-ecb4-4ca3-b045-1dfa50104289}", WindowsUpdateTracker.EventId_Downloaded, "store_41")]
[InlineData(DefenderIntelligence, null, WindowsUpdateTracker.EventId_InstallStarted, "defender_43")]
[InlineData("Intel net Driver Update (24.50.0.4)", null, WindowsUpdateTracker.EventId_DownloadStarted, "other_44")]
// A real download failure names no update at all; it must not reach ANALYZE-DEV-004 as
// a failed OS update.
[InlineData("", null, WindowsUpdateTracker.EventId_DownloadFailed, "other_31")]
[InlineData("Some App", "{855e8a7c-ecb4-4ca3-b045-1dfa50104289}", WindowsUpdateTracker.EventId_InstallFailure, "store_20")]
public void NonOsUpdates_AreCountedNotEmitted(string title, string? serviceGuid, int eventId, string expectedKey)
{
var tracker = WindowsUpdateTracker.IsSystemChannelEventId(eventId) ? _system : _operational;
Emit(tracker, eventId, updateTitle: title, serviceGuid: serviceGuid, errorCode: "0x80248007");
Assert.Empty(_sink.Posted);
Assert.Empty(_osActivity);
Assert.Equal(1, Counted(expectedKey));
// The census still sees the channel working.
Assert.Equal(1, tracker.EmittedThisRun);
}
[Fact]
public void CountedRecord_DeliveredTwice_IsCountedOnce()
{
// Live watcher and backfill overlap: the seen-set dedups counted records too.
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, updateTitle: StoreApp, recordId: 7);
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, updateTitle: StoreApp, recordId: 7);
Assert.Equal(1, Counted("store_19"));
}
[Fact]
public void OsUpdateSteps_AreReportedToTheEngineCallback()
{
var t0 = At;
Emit(_system, WindowsUpdateTracker.EventId_DownloadStarted, at: t0);
Emit(_operational, WindowsUpdateTracker.EventId_Downloaded, at: t0.AddMinutes(4));
Emit(_system, WindowsUpdateTracker.EventId_InstallStarted, at: t0.AddMinutes(5));
Emit(_system, WindowsUpdateTracker.EventId_InstallSuccess, at: t0.AddMinutes(31));
Emit(_system, WindowsUpdateTracker.EventId_InstallFailure, errorCode: "0x800F0922", at: t0.AddMinutes(32));
Emit(_operational, WindowsUpdateTracker.EventId_DownloadFailed, errorCode: "0x80248007", at: t0.AddMinutes(33));
Assert.Equal(
new[]
{
OsUpdateSteps.DownloadStarted, OsUpdateSteps.Downloaded, OsUpdateSteps.InstallStarted,
OsUpdateSteps.Installed, OsUpdateSteps.Failed, OsUpdateSteps.Failed,
},
_osActivity.Select(a => a.Step).ToArray());
Assert.All(_osActivity, a => Assert.Equal(Lcu, a.Update));
Assert.Equal(t0.AddMinutes(31), _osActivity[3].At);
}
[Fact]
public void ThrowingCallback_NeverCostsTheEvent()
{
var tracker = new WindowsUpdateTracker(
"s", "t", new InformationalEventPost(_sink, new VirtualClock(At)),
new AgentLogger(_tmp.Path, AgentLogLevel.Info),
backfillEnabled: false,
channel: WindowsUpdateTracker.SystemChannel,
onOsUpdateActivity: (step, update, at) => throw new InvalidOperationException("boom"));
Emit(tracker, WindowsUpdateTracker.EventId_InstallSuccess);
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateSucceeded));
}
// ---------------------------------------------------------------------------
// Channels and XPath
// ---------------------------------------------------------------------------
[Fact]
public void DefaultEventIds_FollowTheProviderManifest()
{
Assert.Equal(WindowsUpdateTracker.SystemChannel, _system.WatchedChannel);
Assert.Equal(new[] { 19, 20, 43, 44 }, _system.TargetedEventIds.OrderBy(id => id).ToArray());
Assert.Equal(WindowsUpdateTracker.Channel, _operational.WatchedChannel);
Assert.Equal(new[] { 25, 26, 31, 41 }, _operational.TargetedEventIds.OrderBy(id => id).ToArray());
}
[Theory]
[InlineData(16, true)]
[InlineData(19, true)]
[InlineData(20, true)]
[InlineData(24, true)]
[InlineData(27, true)]
[InlineData(28, true)]
[InlineData(32, true)]
[InlineData(33, true)]
[InlineData(43, true)]
[InlineData(44, true)]
[InlineData(212, true)]
[InlineData(218, true)]
[InlineData(25, false)]
[InlineData(26, false)]
[InlineData(31, false)]
[InlineData(41, false)]
[InlineData(15, false)]
[InlineData(219, false)]
public void IsSystemChannelEventId_MatchesTheManifest(int eventId, bool expected)
{
Assert.Equal(expected, WindowsUpdateTracker.IsSystemChannelEventId(eventId));
}
[Fact]
public void BuildXPath_ContainsAllTargetedIds_Ordered()
{
@@ -192,6 +392,113 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
Assert.Equal("*[System[(EventID=19 or EventID=20 or EventID=43 or EventID=44)]]", xpath);
}
[Fact]
public void BuildXPath_SharedSystemLog_FiltersTheProvider_AndTheLookback()
{
// Verified against a real System log: the provider clause keeps the other providers'
// EventIDs 19/20/43/44 out of the shared channel.
var ids = new HashSet<int> { 44, 19, 20, 43 };
Assert.Equal(
"*[System[Provider[@Name='Microsoft-Windows-WindowsUpdateClient'] and (EventID=19 or EventID=20 or EventID=43 or EventID=44)]]",
WindowsUpdateTracker.BuildXPath(ids, WindowsUpdateTracker.ProviderName));
Assert.Equal(
"*[System[Provider[@Name='Microsoft-Windows-WindowsUpdateClient'] and (EventID=19 or EventID=20 or EventID=43 or EventID=44) and TimeCreated[timediff(@SystemTime) <= 3600000]]]",
WindowsUpdateTracker.BuildXPath(ids, WindowsUpdateTracker.ProviderName, lookbackMs: 3600000));
Assert.Equal(
"*[System[(EventID=25 or EventID=26) and TimeCreated[timediff(@SystemTime) <= 60000]]]",
WindowsUpdateTracker.BuildXPath(new HashSet<int> { 26, 25 }, providerName: null, lookbackMs: 60000));
}
// ---------------------------------------------------------------------------
// Watermark
// ---------------------------------------------------------------------------
[Fact]
public void Watermark_PersistsAcrossTrackerInstances()
{
var first = NewTracker(WindowsUpdateTracker.SystemChannel, _tmp.Path);
Emit(first, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 500);
// A fresh tracker (simulating an agent restart) must load the watermark and skip the
// already-emitted record when the OOBE backfill re-reads it.
var second = NewTracker(WindowsUpdateTracker.SystemChannel, _tmp.Path);
second.LoadWatermark();
Emit(second, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 500, isBackfill: true);
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateSucceeded));
}
[Fact]
public void Watermark_IsPerChannel()
{
// RecordIds are per channel: System record 500 and Operational record 500 are two
// different events and must not dedup each other — before or after a restart.
Emit(NewTracker(WindowsUpdateTracker.SystemChannel, _tmp.Path), WindowsUpdateTracker.EventId_InstallSuccess, recordId: 500);
Emit(NewTracker(null, _tmp.Path), WindowsUpdateTracker.EventId_Downloaded, recordId: 500);
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateSucceeded));
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateStarted));
Assert.Equal(500L, ReadWatermark(WindowsUpdateTracker.SystemWatermarkStateFileName));
Assert.Equal(500L, ReadWatermark(WindowsUpdateTracker.WatermarkStateFileName));
var system = NewTracker(WindowsUpdateTracker.SystemChannel, _tmp.Path);
var operational = NewTracker(null, _tmp.Path);
system.LoadWatermark();
operational.LoadWatermark();
Emit(system, WindowsUpdateTracker.EventId_InstallSuccess, recordId: 500, isBackfill: true);
Emit(operational, WindowsUpdateTracker.EventId_Downloaded, recordId: 500, isBackfill: true);
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateSucceeded));
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateStarted));
}
[Fact]
public void CountedRecords_WriteNoStateFile()
{
// A Store/Defender burst must not cost a state-file write per record; only emitted
// records move the persisted watermark.
var tracker = NewTracker(WindowsUpdateTracker.SystemChannel, _tmp.Path);
Emit(tracker, WindowsUpdateTracker.EventId_InstallSuccess, updateTitle: StoreApp, recordId: 600);
Emit(tracker, WindowsUpdateTracker.EventId_InstallStarted, updateTitle: DefenderIntelligence, recordId: 601);
Assert.False(File.Exists(Path.Combine(_tmp.Path, WindowsUpdateTracker.SystemWatermarkStateFileName)));
Emit(tracker, WindowsUpdateTracker.EventId_InstallStarted, recordId: 602);
Assert.Equal(602L, ReadWatermark(WindowsUpdateTracker.SystemWatermarkStateFileName));
}
[Fact]
public void Watermark_PersistsOnlyAfterTheEngineSignalWasHandedOver()
{
// A restart's backfill skips a persisted record, so the record's engine signal must be
// on its way before the watermark reaches the disk — never the other way round.
var path = Path.Combine(_tmp.Path, WindowsUpdateTracker.SystemWatermarkStateFileName);
bool? watermarkOnDiskAtSignal = null;
var tracker = new WindowsUpdateTracker(
sessionId: "sess-wu",
tenantId: "tenant-wu",
post: new InformationalEventPost(_sink, new VirtualClock(At)),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
backfillEnabled: false,
stateDirectory: _tmp.Path,
channel: WindowsUpdateTracker.SystemChannel,
tally: _tally,
onOsUpdateActivity: (step, update, at) => watermarkOnDiskAtSignal = File.Exists(path));
Emit(tracker, WindowsUpdateTracker.EventId_InstallStarted, recordId: 700);
Assert.False(watermarkOnDiskAtSignal);
Assert.Equal(700L, ReadWatermark(WindowsUpdateTracker.SystemWatermarkStateFileName));
}
private long ReadWatermark(string fileName) =>
JObject.Parse(File.ReadAllText(Path.Combine(_tmp.Path, fileName)))["LastRecordId"]!.Value<long>();
// ---------------------------------------------------------------------------
// Parsing and HRESULT decoding
// ---------------------------------------------------------------------------
[Fact]
public void ParseEventData_ExtractsNamedFields_NamespaceAgnostic()
{
@@ -212,6 +519,24 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
Assert.True(data.ContainsKey("UPDATETITLE"));
}
[Fact]
public void ParseEventData_ReadsTheScanFields()
{
// Event 26 (scan found N) and 25 (scan failed) carry no update title.
const string xml =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
"<System><EventID>26</EventID></System>" +
"<EventData>" +
"<Data Name='updateCount'>3</Data>" +
"<Data Name='serviceGuid'>{8b24b027-1dee-babb-9a95-3517dfb9c552}</Data>" +
"</EventData></Event>";
var data = WindowsUpdateTracker.ParseEventData(xml);
Assert.Equal("3", data["updateCount"]);
Assert.Equal("{8b24b027-1dee-babb-9a95-3517dfb9c552}", data["serviceGuid"]);
Assert.False(data.ContainsKey("updateTitle"));
}
[Fact]
public void ParseEventData_MalformedXml_ReturnsEmpty_NeverThrows()
{
@@ -303,10 +628,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
var data = Data(s);
Assert.Equal("21=1,25=3", data["wuClientCensus"]);
Assert.Equal("200=2", data["updateOrchestratorCensus"]);
Assert.Equal("19,20,43,44", data["targetedEventIds"]);
Assert.Equal("25,26,31,41", data["targetedEventIds"]);
Assert.False(data.ContainsKey("censusTruncated"));
}
[Fact]
public void Census_ListsTheTargetedIdsOfEveryChannel()
{
// The host runs the census on the Operational instance; the evidence must still say
// the System-log install IDs were watched too.
var tracker = BuildCensusTracker(censusEnabled: true, osBuildChanged: true);
tracker.EmitChannelCensusIfBlind(otherChannelsEmitted: 0, otherTargetedEventIds: new[] { 44, 19, 20, 43 });
var s = Assert.Single(ByType(Constants.EventTypes.WindowsUpdateChannelCensus));
Assert.Equal("19,20,25,26,31,41,43,44", Data(s)["targetedEventIds"]);
}
[Fact]
public void Census_BuildUnchanged_EmitsNothing()
{
@@ -320,17 +658,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
[Fact]
public void Census_TargetedEventsWereEmittedThisRun_EmitsNothing()
{
// The watcher captured the update through its normal path — no blind spot to report.
// The watcher captured activity through its normal path — no blind spot to report.
// Any class counts: a counted Store update proves the channel is read as well.
var tracker = BuildCensusTracker(censusEnabled: true, osBuildChanged: true);
tracker.ProcessEvent(WindowsUpdateTracker.EventId_InstallSuccess, 4, recordId: 9, timeCreatedUtc: At,
updateTitle: "KB1", updateGuid: null, updateRevisionNumber: null, errorCode: null,
formattedDescription: null, isBackfill: false);
Emit(tracker, WindowsUpdateTracker.EventId_Downloaded, updateTitle: StoreApp, recordId: 9);
tracker.EmitChannelCensusIfBlind();
Assert.Empty(ByType(Constants.EventTypes.WindowsUpdateChannelCensus));
}
[Fact]
public void Census_AnotherChannelEmitted_EmitsNothing()
{
// The System-log watcher saw the install — the update was not missed.
var tracker = BuildCensusTracker(censusEnabled: true, osBuildChanged: true);
tracker.EmitChannelCensusIfBlind(otherChannelsEmitted: 1);
Assert.Empty(ByType(Constants.EventTypes.WindowsUpdateChannelCensus));
}
[Fact]
public void Census_DisabledByConfig_EmitsNothing()
{
@@ -153,6 +153,62 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
Assert.Equal(0, callCount);
}
[Fact]
public void Stopped_waiting_fires_callback_once_with_the_stage_still_non_terminal()
{
// D-310: the engine stopped waiting for the user without a verdict. The stage stays
// non-terminal, yet the agent must end the session — the callback carries the state.
using var rig = new Rig();
DecisionState? captured = null;
var callCount = 0;
var sut = rig.Build(DecisionState.CreateInitial("S1", "T1"),
onTerminal: state =>
{
Interlocked.Increment(ref callCount);
captured = state;
});
var builder = new DecisionStateBuilder(sut.CurrentState)
{
Stage = SessionStage.EspAccountSetup,
StepIndex = sut.CurrentState.StepIndex + 1,
StoppedWaitingReason = new SignalFact<string>(StoppedWaitingReasons.OobeUpdateNoSignIn, 1),
};
var stopped = builder.Build();
var step = new DecisionStep(stopped, new DecisionTransition(
stepIndex: stopped.StepIndex, sessionTraceOrdinal: 1, signalOrdinalRef: 1, occurredAtUtc: At,
trigger: "DeadlineFired:advisory_completion:OobeUpdateSignInWaitExpired",
fromStage: SessionStage.SessionStarted, toStage: SessionStage.EspAccountSetup,
taken: true, deadEndReason: null, reducerVersion: "2.0.0.0"), Array.Empty<DecisionEffect>());
sut.ApplyStep(step, Sig(1, At));
// A follow-up step must not fire it again.
sut.ApplyStep(StepToStage(sut.CurrentState, SessionStage.EspAccountSetup, signalOrdinal: 2, at: At.AddSeconds(1)),
Sig(2, At.AddSeconds(1)));
Assert.Equal(1, callCount);
Assert.Equal(SessionStage.EspAccountSetup, captured!.Stage);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, captured.StoppedWaitingReason!.Value);
}
[Fact]
public void Stopped_waiting_on_load_does_not_refire_the_callback()
{
// The orchestrator ends a recovered stopped-waiting session itself at start.
using var rig = new Rig();
var callCount = 0;
var recovered = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1"))
{
Stage = SessionStage.EspAccountSetup,
StoppedWaitingReason = new SignalFact<string>(StoppedWaitingReasons.OobeUpdateNoSignIn, 7),
}.Build();
var sut = rig.Build(recovered, onTerminal: _ => Interlocked.Increment(ref callCount));
sut.ApplyStep(StepToStage(sut.CurrentState, SessionStage.EspAccountSetup, signalOrdinal: 8, at: At), Sig(8, At));
Assert.Equal(0, callCount);
}
[Fact]
public void Terminal_callback_null_is_supported()
{
@@ -0,0 +1,103 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Configuration;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
{
/// <summary>
/// The production factory hands its state directory to the gather executor. Without it every
/// unit test of the gather state passes while the agent keeps replaying logs after each reboot —
/// the one line in <see cref="DefaultComponentFactory"/> is the whole feature.
/// </summary>
[Collection("SerialThreading")] // the startup rule executes on the shared ThreadPool
public sealed class DefaultComponentFactoryGatherStateTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly TempDirectory _stateDir = new TempDirectory();
// %TEMP% lives under C:\Users (hard-blocked) — the log sits beside the test assembly.
private readonly string _logDir = Path.Combine(
AppContext.BaseDirectory, "factory-gather-state-" + Guid.NewGuid().ToString("N"));
public DefaultComponentFactoryGatherStateTests() => Directory.CreateDirectory(_logDir);
public void Dispose()
{
_tmp.Dispose();
_stateDir.Dispose();
try { Directory.Delete(_logDir, recursive: true); } catch { /* best effort */ }
}
[Fact]
public void Gather_host_built_by_the_factory_persists_logparser_positions_in_the_state_directory()
{
var log = Path.Combine(_logDir, "vendor.log");
File.WriteAllText(log, "Update one\n");
var factory = new DefaultComponentFactory(
agentConfig: new AgentConfiguration
{
ApiBaseUrl = "https://example",
TenantId = "t",
SessionId = "s",
UnrestrictedMode = true,
},
remoteConfig: new AgentConfigResponse
{
Collectors = CollectorConfiguration.CreateDefault(),
GatherRules = new List<GatherRule>
{
new GatherRule
{
RuleId = "vendor-log",
Title = "vendor log",
CollectorType = "logparser",
Target = log,
Parameters = new Dictionary<string, string> { ["pattern"] = "^Update", ["format"] = "text" },
Trigger = "startup",
OutputEventType = "vendor_update",
Enabled = true,
},
},
},
networkMetrics: null,
agentVersion: "test",
stateDirectory: _stateDir.Path,
startupEventGate: null);
var hosts = factory.CreateCollectorHosts(
sessionId: "s",
tenantId: "t",
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
whiteGloveSealingPatternIds: Array.Empty<string>(),
ingress: new FakeSignalIngressSink(),
clock: new VirtualClock(new DateTime(2026, 9, 28, 12, 0, 0, DateTimeKind.Utc)),
telemetrySpool: null,
timelineEvents: null).Hosts;
try
{
hosts.OfType<GatherRuleExecutorHost>().Single().Start();
var stateFile = Path.Combine(_stateDir.Path, GatherRuleStatePersistence.FileName);
Assert.True(SpinWait.SpinUntil(() => File.Exists(stateFile), TimeSpan.FromSeconds(10)),
"the gather host did not write its state into the factory's state directory");
}
finally
{
foreach (var host in hosts)
{
try { host.Dispose(); } catch { /* best-effort cleanup */ }
}
}
}
}
}
@@ -16,7 +16,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
/// <summary>
/// GatherRuleExecutorHost feeds its phase_change / phase_exit / on_event triggers from the
/// POST-REDUCE <see cref="TimelineEventStream"/> (emitted timeline events), not from raw
/// signals. Regression anchor: session 32312a32 (rsneuffen.de) — a phase_change rule on
/// signals. Regression anchor: session 32312a32 — a phase_change rule on
/// FinalizingSetup fired at the raw EspPhaseChanged(FinalizingSetup) signal (ESP exit),
/// 7 minutes before the engine's RealmJoin-gated phase_transition(FinalizingSetup), and read
/// a registry key before the RealmJoin package wrote it. With the timeline feed the rule
@@ -174,6 +174,40 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
sut.Stop();
}
[Fact]
public void Recovered_stopped_waiting_state_ends_the_session_at_start()
{
// D-310: a crash between "the engine stopped waiting" and the termination must not
// leave the session running — the restart ends it the same way, with the engine's reason.
using var rig = new Rig();
Directory.CreateDirectory(rig.StateDir);
var stoppedState = DecisionState.CreateInitial("S1", "T1")
.ToBuilder()
.WithStage(SessionStage.EspAccountSetup)
.WithStepIndex(9)
.Build()
.ToBuilder();
stoppedState.StoppedWaitingReason = new SignalFact<string>(StoppedWaitingReasons.OobeUpdateNoSignIn, 42);
new SnapshotPersistence(Path.Combine(rig.StateDir, "snapshot.json")).Save(stoppedState.Build());
var sut = rig.Build();
using var fired = new ManualResetEventSlim(false);
EnrollmentTerminatedEventArgs? captured = null;
sut.Terminated += (_, e) => { captured = e; fired.Set(); };
sut.Start();
Assert.True(fired.Wait(5000), "Terminated did not fire for the recovered stopped-waiting state.");
Assert.Equal(EnrollmentTerminationReason.StoppedWaiting, captured!.Reason);
Assert.Equal(EnrollmentTerminationOutcome.TimedOut, captured.Outcome);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, captured.StopReason);
Assert.Equal(SessionStage.EspAccountSetup.ToString(), captured.StageName);
Assert.Contains("60 minutes", captured.Details);
sut.Stop();
}
// ================================================================= Sonderfall 2: Segment-Quarantine
[Fact]
@@ -0,0 +1,209 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
{
/// <summary>
/// WindowsUpdateWatcherHost (D-310) — routes every configured WU EventID to the channel the
/// provider manifest writes it to, reports counted activity once, and feeds Windows/.NET update
/// steps to the decision engine unless <c>OobeUpdatePhaseMode</c> is Off. The event-log
/// watchers themselves are not armed here; the trackers' ProcessEvent seams drive the host.
/// </summary>
public sealed class WindowsUpdateWatcherHostTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 10, 1, 9, 30, 0, DateTimeKind.Utc);
private const string Lcu = "2026-09 Cumulative Update for Windows 11, version 25H2 for x64-based Systems (KB5099999)";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink = new FakeSignalIngressSink();
private long _nextRecordId = 1;
public void Dispose() => _tmp.Dispose();
private WindowsUpdateWatcherHost Build(bool osUpdateSignalsEnabled, bool servicingWatcherEnabled = true) =>
new WindowsUpdateWatcherHost(
sessionId: "sess-wuhost",
tenantId: "tenant-wuhost",
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
ingress: _sink,
clock: new VirtualClock(At),
targetedEventIds: new[] { 19, 20, 43, 44 },
backfillLookbackMinutes: 0,
stateDirectory: _tmp.Path,
operationalEventIds: new[] { 25, 26, 31, 41 },
servicingWatcherEnabled: servicingWatcherEnabled,
osUpdateSignalsEnabled: osUpdateSignalsEnabled);
private void Wu(WindowsUpdateTracker tracker, int eventId, string title, DateTime at) =>
tracker.ProcessEvent(eventId, 4, _nextRecordId++, at, title, updateGuid: null, updateRevisionNumber: null,
errorCode: null, formattedDescription: null, isBackfill: false);
private void Cbs(ServicingTracker tracker, int eventId, string client, DateTime at) =>
tracker.ProcessEvent(eventId, _nextRecordId++, at, "KB5099999", initialState: null, targetState: "Installed",
errorCode: "0x0", client: client, isBackfill: false);
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> OsUpdateActivity() =>
_sink.Posted.Where(p => p.Kind == DecisionSignalKind.OsUpdateActivity).ToList();
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
// ---------------------------------------------------------------- routing ----
[Fact]
public void RouteEventIds_Defaults_SplitByManifest()
{
var (system, operational) = WindowsUpdateWatcherHost.RouteEventIds(new[] { 19, 20, 43, 44 }, new[] { 25, 26, 31, 41 });
Assert.Equal(new[] { 19, 20, 43, 44 }, system);
Assert.Equal(new[] { 25, 26, 31, 41 }, operational);
}
[Fact]
public void RouteEventIds_MisplacedIds_EndUpOnTheirRealChannel()
{
// An old config that lists 41 under the targeted IDs, or 20 under the operational
// ones, must still watch the channel that carries them.
var (system, operational) = WindowsUpdateWatcherHost.RouteEventIds(new[] { 41, 19 }, new[] { 20, 41 });
Assert.Equal(new[] { 19, 20 }, system);
Assert.Equal(new[] { 41 }, operational);
}
[Fact]
public void RouteEventIds_NothingConfigured_WatchesNothing()
{
var (system, operational) = WindowsUpdateWatcherHost.RouteEventIds(null, Array.Empty<int>());
Assert.Empty(system);
Assert.Empty(operational);
}
[Fact]
public void Host_BuildsOneTrackerPerChannel_AndTheServicingTracker()
{
var host = Build(osUpdateSignalsEnabled: false);
Assert.Equal(WindowsUpdateTracker.SystemChannel, host.SystemTrackerForTest!.WatchedChannel);
Assert.Equal(new[] { 19, 20, 43, 44 }, host.SystemTrackerForTest.TargetedEventIds.OrderBy(id => id).ToArray());
Assert.Equal(WindowsUpdateTracker.Channel, host.OperationalTrackerForTest!.WatchedChannel);
Assert.Equal(new[] { 25, 26, 31, 41 }, host.OperationalTrackerForTest.TargetedEventIds.OrderBy(id => id).ToArray());
Assert.NotNull(host.ServicingTrackerForTest);
}
[Fact]
public void ServicingWatcherDisabled_BuildsNoServicingTracker()
{
Assert.Null(Build(osUpdateSignalsEnabled: false, servicingWatcherEnabled: false).ServicingTrackerForTest);
}
// ------------------------------------------------- engine signal behind flag ----
[Fact]
public void OsUpdateSignalsDisabled_ListsTheUpdate_ButPostsNoEngineSignal()
{
var host = Build(osUpdateSignalsEnabled: false);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallStarted, Lcu, At);
Cbs(host.ServicingTrackerForTest!, ServicingTracker.EventId_RebootRequired, "UpdateAgentLCU", At.AddMinutes(20));
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateStarted));
Assert.Single(ByType(Constants.EventTypes.WindowsUpdateServicing));
Assert.Empty(OsUpdateActivity());
}
[Fact]
public void OsUpdateSignalsEnabled_PostsOsUpdateActivity_FromBothSources()
{
var host = Build(osUpdateSignalsEnabled: true);
var wuAt = DateTime.SpecifyKind(At, DateTimeKind.Unspecified);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallStarted, Lcu, wuAt);
Cbs(host.ServicingTrackerForTest!, ServicingTracker.EventId_RebootRequired, "UpdateAgentLCU", At.AddMinutes(20));
var signals = OsUpdateActivity();
Assert.Equal(2, signals.Count);
var wu = signals[0];
Assert.Equal(OsUpdateSteps.InstallStarted, wu.Payload![OsUpdatePayloadKeys.Step]);
Assert.Equal("wu", wu.Payload![OsUpdatePayloadKeys.Source]);
Assert.Equal(Lcu, wu.Payload![OsUpdatePayloadKeys.Update]);
Assert.Equal("WindowsUpdateWatcher", wu.SourceOrigin);
Assert.Equal(EvidenceKind.Raw, wu.Evidence.Kind);
Assert.Equal("os-update-activity:wu", wu.Evidence.Identifier);
Assert.Equal(At, wu.OccurredAtUtc);
Assert.Equal(DateTimeKind.Utc, wu.OccurredAtUtc.Kind);
Assert.Equal(1, wu.KindSchemaVersion);
var cbs = signals[1];
Assert.Equal(OsUpdateSteps.RebootRequired, cbs.Payload![OsUpdatePayloadKeys.Step]);
Assert.Equal("servicing", cbs.Payload![OsUpdatePayloadKeys.Source]);
Assert.Equal("KB5099999", cbs.Payload![OsUpdatePayloadKeys.Update]);
Assert.Equal("ServicingWatcher", cbs.SourceOrigin);
Assert.Equal("os-update-activity:servicing", cbs.Evidence.Identifier);
Assert.Equal(At.AddMinutes(20), cbs.OccurredAtUtc);
}
[Fact]
public void OsUpdateSignalsEnabled_NonOsActivity_NeverReachesTheEngine()
{
var host = Build(osUpdateSignalsEnabled: true);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallSuccess, "9NSTH9KHZDLQ-Microsoft.UI.Xaml.2.8", At);
Wu(host.OperationalTrackerForTest!, WindowsUpdateTracker.EventId_Downloaded,
"Security Intelligence Update for Microsoft Defender Antivirus - KB2267602 (Version 1.459.505.0) - Current Channel (Broad)", At);
Cbs(host.ServicingTrackerForTest!, ServicingTracker.EventId_StateReached, "CbsTask", At);
Assert.Empty(OsUpdateActivity());
Assert.Empty(_sink.Posted);
}
// ------------------------------------------------------- activity summary ----
[Fact]
public void ActivitySummary_IsEmittedOnce_WhenTheHostStops()
{
var host = Build(osUpdateSignalsEnabled: false);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallSuccess, "9NSTH9KHZDLQ-Microsoft.UI.Xaml.2.8", At);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallSuccess, "9NBLGGH4NNS1-Microsoft.DesktopAppInstaller", At);
Cbs(host.ServicingTrackerForTest!, ServicingTracker.EventId_InitiatingChanges, "CbsTask", At);
// The termination path stops the host explicitly and disposes it afterwards.
host.Stop();
host.Dispose();
var summary = Assert.Single(ByType(Constants.EventTypes.WindowsUpdateActivitySummary));
Assert.Equal("Debug", summary.Payload![SignalPayloadKeys.Severity]);
Assert.Contains("3 event(s)", summary.Payload![SignalPayloadKeys.Message]);
var data = (IReadOnlyDictionary<string, object>)summary.TypedPayload!;
Assert.Equal(2, data["store_19"]);
Assert.Equal(1, data["servicing_other_1"]);
}
[Fact]
public void ActivitySummary_NothingCounted_EmitsNothing()
{
var host = Build(osUpdateSignalsEnabled: false);
Wu(host.SystemTrackerForTest!, WindowsUpdateTracker.EventId_InstallSuccess, Lcu, At);
host.Stop();
Assert.Empty(ByType(Constants.EventTypes.WindowsUpdateActivitySummary));
}
}
}
@@ -4,6 +4,7 @@ using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared.Models;
using Xunit;
@@ -29,7 +30,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
var sink = new FakeSignalIngressSink();
var logger = NewLogger(tmp.Path);
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, logger);
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, HelloWaitTimeout.DefaultSeconds, logger);
Assert.Single(sink.Posted);
var posted = sink.Posted[0];
@@ -47,7 +48,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
var sink = new FakeSignalIngressSink();
var logger = NewLogger(tmp.Path);
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, logger);
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, HelloWaitTimeout.DefaultSeconds, logger);
var payload = sink.Posted[0].Payload;
Assert.NotNull(payload);
@@ -73,6 +74,52 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
Assert.True(cloudPc == "true" || cloudPc == "false");
}
[Theory]
[InlineData(300)] // the default — the payload stays exactly what it was before the key existed
[InlineData(30)] // the old default, still stored for many tenants
[InlineData(0)]
public void Hello_wait_not_extending_the_built_in_window_is_not_stamped(int helloWaitTimeoutSeconds)
{
using var tmp = new TempDirectory();
var sink = new FakeSignalIngressSink();
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, helloWaitTimeoutSeconds, NewLogger(tmp.Path));
Assert.DoesNotContain(SignalPayloadKeys.HelloWaitTimeoutSeconds, (IDictionary<string, string>)sink.Posted[0].Payload!);
}
[Theory]
[InlineData(1800, "1800")]
[InlineData(3600, "3600")]
[InlineData(86400, "3600")] // the effective (clamped) value is what the engine should arm
public void Extended_hello_wait_is_stamped_as_its_effective_value(int helloWaitTimeoutSeconds, string expected)
{
using var tmp = new TempDirectory();
var sink = new FakeSignalIngressSink();
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, helloWaitTimeoutSeconds, NewLogger(tmp.Path));
Assert.Equal(expected, sink.Posted[0].Payload![SignalPayloadKeys.HelloWaitTimeoutSeconds]);
}
[Theory]
[InlineData(null, "Shadow")] // no config value — the default mode
[InlineData("Shadow", "Shadow")]
[InlineData("active", "Active")] // case-insensitive
[InlineData("Off", "Off")]
[InlineData("Enforce", "Shadow")] // unknown — changes no decision, still measures
public void Oobe_update_phase_mode_is_stamped_at_every_start_normalized(string? configured, string expected)
{
// D-310: stamped every time, so a later Off/Shadow can take Active away mid-session.
using var tmp = new TempDirectory();
var sink = new FakeSignalIngressSink();
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(
sink, HelloWaitTimeout.DefaultSeconds, NewLogger(tmp.Path), configured);
Assert.Equal(expected, sink.Posted[0].Payload![SignalPayloadKeys.OobeUpdatePhaseMode]);
}
[Fact]
public void Swallows_sink_exceptions()
{
@@ -82,7 +129,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Program
var logger = NewLogger(tmp.Path);
var ex = Record.Exception(() =>
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, logger));
AutopilotMonitor.Agent.V2.Runtime.LifecycleEmitters.PostEnrollmentFactsObserved(sink, HelloWaitTimeout.DefaultSeconds, logger));
Assert.Null(ex);
}
@@ -0,0 +1,54 @@
using System;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.SignalAdapters;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.SignalAdapters
{
/// <summary>
/// D-316: every emitted platform-script completion carries a run id, and the tracker's run register holds what
/// was emitted for it under the IME user id — what the registry reconciliation pairs IME's saved result with.
/// </summary>
public sealed class ImeLogTrackerAdapterPlatformRunTests
{
private static readonly DateTime ClockNow = new DateTime(2026, 10, 3, 21, 0, 0, DateTimeKind.Utc);
private const string DeviceUser = "00000000-0000-0000-0000-000000000000";
private const string PolicyId = "1d6f0a52-3c4e-4b7a-9e21-5f8c7d2b9a10";
[Fact]
public void Platform_completion_carries_its_run_id_and_the_register_holds_the_emitted_run()
{
using var f = new ImeLogTrackerAdapterFixture(ClockNow);
using var adapter = new ImeLogTrackerAdapter(f.Tracker, f.Ingress, f.Clock);
f.Tracker.LastMatchedLogTimestamp = ClockNow.AddSeconds(-10);
f.Tracker.HandlePlatformScriptStarted(PolicyId, "ime", DeviceUser);
f.Tracker.RecordPlatformScriptExitCodeForTesting(PolicyId, 0, ClockNow.AddSeconds(-1));
f.Tracker.LastMatchedLogTimestamp = ClockNow;
f.Tracker.CompletePlatformScriptFromImeResultForTesting(PolicyId, "Success", ClockNow);
var info = f.InfoEvent(SharedEventTypes.ScriptCompleted);
Assert.True(Guid.TryParse(info.Payload!["runId"], out _));
f.Tracker.ScriptRuns.NotePassCompleted(ClockNow.AddMinutes(1));
Assert.Equal(PlatformRunClaim.Claimed, f.Tracker.ScriptRuns.TryClaim(DeviceUser, PolicyId, ClockNow.AddSeconds(5), out var run));
Assert.Equal(info.Payload["runId"], run!.RunId);
Assert.Equal(0, run.ExitCode);
Assert.Equal("Success", run.Result);
}
[Theory]
[InlineData("[PowerShell] User Id = 00000000-0000-0000-0000-000000000000, Policy id = d94468af-6ebe-4e94-9901-2a62d33ea6c1, policy result = Success", "00000000-0000-0000-0000-000000000000")]
[InlineData(@"Adding argument powershell with value C:\Program Files (x86)\Microsoft Intune Management Extension\Policies\Scripts\6C3F8A20-1D5E-4B97-8F4A-0E2D6B9C7A13_d94468af-6ebe-4e94-9901-2a62d33ea6c1.ps1", "6c3f8a20-1d5e-4b97-8f4a-0e2d6b9c7a13")]
[InlineData(@"Script file C:\Program Files (x86)\Microsoft Intune Management Extension\Policies\Scripts\00000000-0000-0000-0000-000000000000_d94468af-6ebe-4e94-9901-2a62d33ea6c1.ps1 is generated", "00000000-0000-0000-0000-000000000000")]
[InlineData(@"Adding argument powershellDetection with value C:\Program Files (x86)\Microsoft Intune Management Extension\Content\DetectionScripts\8b2e4c71-6a9d-4f03-b5e8-2c7a1d9f4e36_2.ps1", null)]
[InlineData("[PowerShell] User Id = not-a-guid, Policy id = d94468af", null)]
[InlineData("", null)]
public void Ime_user_id_is_read_from_the_result_line_and_the_script_path(string text, string? expected)
{
Assert.Equal(expected, ImeLogTracker.ImeUserIdFrom(text));
}
}
}
@@ -933,6 +933,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Termination
Assert.Equal(EnrollmentTerminationOutcome.Failed.ToString(), (string)data["outcome"]);
}
[Fact]
public void Handle_propagates_stopped_waiting_reason_and_cause_in_agent_shutting_down_data()
{
// D-310: the engine stopped waiting for the user (no sign-in after the OOBE update).
// The backend classifies this shutdown honestly — it keys on the reason tag.
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.EspAccountSetup }.Build();
rig.Build().Handle(sender: null!, new EnrollmentTerminatedEventArgs(
EnrollmentTerminationReason.StoppedWaiting,
EnrollmentTerminationOutcome.TimedOut,
SessionStage.EspAccountSetup.ToString(),
EndUtc,
details: StoppedWaitingReasons.Describe(StoppedWaitingReasons.OobeUpdateNoSignIn),
stopReason: StoppedWaitingReasons.OobeUpdateNoSignIn));
var data = rig.DataOf(Constants.EventTypes.AgentShuttingDown);
Assert.NotNull(data);
Assert.Equal("stopped_waiting", (string)data!["reason"]);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, (string)data["stopReason"]);
Assert.Equal(EnrollmentTerminationOutcome.TimedOut.ToString(), (string)data["outcome"]);
}
[Fact]
public void Handle_omits_stop_reason_for_other_shutdowns()
{
using var rig = new Rig();
rig.State = new DecisionStateBuilder(DecisionState.CreateInitial("S1", "T1")) { Stage = SessionStage.AwaitingHello }.Build();
rig.Build().Handle(sender: null!,
Args(EnrollmentTerminationReason.MaxLifetimeExceeded, EnrollmentTerminationOutcome.TimedOut, SessionStage.AwaitingHello));
var data = rig.DataOf(Constants.EventTypes.AgentShuttingDown);
Assert.NotNull(data);
Assert.False(data!.ContainsKey("stopReason"));
}
[Fact]
public void Handle_includes_uptime_and_agent_version_in_agent_shutting_down()
{
@@ -157,12 +157,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
public double ReplaySpeedFactor { get; set; } = 50;
/// <summary>
/// Wait time in seconds after ESP exit before marking Hello as skipped.
/// When ESP exits, we wait this duration for Hello wizard to start (event 62404).
/// Default: 30 seconds (reasonable for systems under load).
/// If Hello wizard starts within this window, we continue waiting for Hello completion (300/301).
/// Tenant Hello wait in seconds after the ESP exit, merged from RemoteConfig. Stamped on
/// <c>EnrollmentFactsObserved</c> for the decision engine's hello_safety window; the Hello
/// tracker gets the same remote value through the collector configuration. Both resolve it
/// through <see cref="HelloWaitTimeout.EffectiveSeconds"/> (at least the built-in 300 s,
/// at most 3600 s). Default: 300 seconds.
/// </summary>
public int HelloWaitTimeoutSeconds { get; set; } = 30;
public int HelloWaitTimeoutSeconds { get; set; } = HelloWaitTimeout.DefaultSeconds;
/// <summary>
/// Maximum consecutive authentication failures (401/403) before the agent shuts down.
@@ -58,8 +58,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
/// (IME service family and AgentExecutor.log: eras are byte ranges).</description></item>
/// <item><description><see cref="TryResolveByLocalTime"/> — transfer for files written by the
/// SAME process as the IME service log but without their own era marker (AppWorkload.log,
/// AppActionProcessor.log: plugins hosted in IntuneManagementExtension.exe, verified in the
/// decompiled build). Within one process every file shares the zone belief, so a local
/// AppActionProcessor.log: plugins hosted in IntuneManagementExtension.exe, verified against
/// the IME build). Within one process every file shares the zone belief, so a local
/// timestamp places the line in the service era whose local range contains it. Disabled
/// entirely when the service eras' local ranges are not monotonic (a westward zone change
/// between two service starts makes the ranges overlap) — ambiguity must never anchor.</description></item>
@@ -271,14 +271,38 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
}
else
{
HandlePlatformScriptStarted(id, source);
HandlePlatformScriptStarted(id, source, ImeUserIdFrom(match.Value));
}
}
/// <summary>
/// The IME user id on a platform-script line — <c>User Id = {id}</c> on the result line,
/// <c>…\Policies\Scripts\{id}_{policy}.ps1</c> on the start lines — or null. Read from the
/// matched text, not from a capture group, so it works with every pattern pack the agent may
/// be served.
/// </summary>
internal static string ImeUserIdFrom(string text)
{
if (string.IsNullOrEmpty(text)) return null;
const string resultTag = "User Id = ";
const string pathTag = @"\Policies\Scripts\";
var i = text.IndexOf(resultTag, StringComparison.OrdinalIgnoreCase);
if (i >= 0) return GuidAt(text, i + resultTag.Length);
i = text.IndexOf(pathTag, StringComparison.OrdinalIgnoreCase);
return i >= 0 ? GuidAt(text, i + pathTag.Length) : null;
}
private static string GuidAt(string text, int start)
{
if (start + 36 > text.Length) return null;
var candidate = text.Substring(start, 36);
return Guid.TryParse(candidate, out _) ? candidate.ToLowerInvariant() : null;
}
// Platform script: AgentExecutor.log entries create/enrich, IME log entries also create.
// Internal so the stale-slot hardening is directly testable (the public path arrives via
// a regex Match).
internal void HandlePlatformScriptStarted(string id, string source = null)
internal void HandlePlatformScriptStarted(string id, string source = null, string userId = null)
{
var currentStartTs = LastMatchedLogTimestamp ?? UtcNowProvider();
var currentStartProvenance = CaptureLastMatchedProvenance();
@@ -327,6 +351,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
pendingSlot.StartedAtProvenance = currentStartProvenance;
_stateDirty = true;
}
if (pendingSlot.UserId == null && userId != null) pendingSlot.UserId = userId;
}
if (!_pendingPlatformScripts.ContainsKey(id))
{
@@ -340,6 +365,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
PolicyId = id,
ScriptType = "platform",
UserId = userId,
// Prefer the source CMTrace timestamp so a script that started before the
// agent launched (replayed log content) is dated to its real start, not now.
// Set only at slot creation: the start line fires twice (agentexecutor + ime
@@ -347,6 +373,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
StartedAtUtc = currentStartTs,
StartedAtProvenance = currentStartProvenance,
};
ScriptRuns.NoteStarted(userId, id, UtcNowProvider());
// Live "running" indicator — same signal health scripts emit. Gated on slot
// creation so the duplicate start line (agentexecutor + ime source) doesn't
// double-emit for the same execution.
@@ -439,6 +466,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
ApplyExitCode(script, exitCode, LastMatchedLogTimestamp ?? UtcNowProvider(), CaptureLastMatchedProvenance());
else if (ownerPolicyId != null)
_logger.Debug($"ImeLogTracker: exit code {exitCode} belongs to platform script {ownerPolicyId}, whose completion was already emitted — dropped");
else
LogLineBehindWriterBoundary($"exit code {exitCode}");
// An exit code means the executor just wrote its end block and IME is about to
// write the result — both land at a stale stream position when another process
// appended meanwhile. Check the ledgers on the next pass.
@@ -465,7 +494,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
var outputType = parameters != null && parameters.TryGetValue("outputType", out var ot) ? ot : null;
var script = GetCurrentScriptForLineUpdate(parameters);
if (script == null) return;
if (script == null)
{
LogLineBehindWriterBoundary($"script output ({outputType ?? "combined"})");
return;
}
// PS-AGENT-OUTPUT captures both stdout and stderr in one pattern
var output = match.Groups["output"]?.Value;
@@ -507,7 +540,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
if (string.IsNullOrEmpty(id)) return;
CompletePlatformScriptFromImeResult(id, result, LastMatchedLogTimestamp, LastMatchedPatternId, CaptureLastMatchedProvenance());
CompletePlatformScriptFromImeResult(id, result, LastMatchedLogTimestamp, LastMatchedPatternId, CaptureLastMatchedProvenance(), ImeUserIdFrom(match.Value));
}
/// <summary>
@@ -521,7 +554,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
/// line reopened the slot, produced a fallback duplicate (session e7f3c910).
/// <see cref="FlushPendingPlatformScriptResults"/> emits the held completion.
/// </summary>
internal void CompletePlatformScriptFromImeResult(string policyId, string result, DateTime? resultLineTimestampUtc, string patternId, CmTraceLineProvenance resultProvenance)
internal void CompletePlatformScriptFromImeResult(string policyId, string result, DateTime? resultLineTimestampUtc, string patternId, CmTraceLineProvenance resultProvenance, string userId = null)
{
// Dedup: the deadline-based fallback (FlushPendingPlatformScriptResults) may already have
// emitted this script from its AgentExecutor exit code because IME's authoritative
@@ -558,11 +591,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
script = new ScriptExecutionState
{
PolicyId = policyId,
ScriptType = "platform"
ScriptType = "platform",
UserId = userId,
};
_pendingPlatformScripts[policyId] = script;
ScriptRuns.NoteStarted(userId, policyId, UtcNowProvider());
}
if (script.UserId == null && userId != null) script.UserId = userId;
script.Result = result;
script.ResultSource = "ime_policy_result";
script.ResultObservedAtUtc = resultLineTimestampUtc ?? UtcNowProvider();
@@ -591,7 +627,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private void EmitPlatformScriptResult(ScriptExecutionState script, string note = null)
{
_logger.Info($"ImeLogTracker: platform script completed: {script.PolicyId}, result={script.Result}, exit={script.ExitCode}{(note != null ? $" ({note})" : string.Empty)}");
if (string.IsNullOrEmpty(script.RunId)) script.RunId = Guid.NewGuid().ToString("D");
EmitScriptEvent(script);
ScriptRuns.NoteEmitted(script, UtcNowProvider());
// The run's source stamp, capped at the clock: a result line cannot be read before it
// was written, so a stamp ahead of the clock is a wrong zone assumption on that line
// (a rewound result resolved in the reader's zone, session 4377911b: +1 h). Left
@@ -1163,11 +1201,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
internal static readonly TimeSpan HeldScriptStartGrace = TimeSpan.FromMinutes(30);
internal static readonly TimeSpan HeldScriptStartShutdownGrace = TimeSpan.FromMinutes(5);
private static string TruncateOutput(string output)
internal static string TruncateOutput(string output)
{
if (string.IsNullOrEmpty(output) || output.Length <= MaxScriptOutputLength)
return output;
return output.Substring(0, MaxScriptOutputLength) + "...[truncated]";
return output.Substring(0, MaxScriptOutputLength) + OutputTruncationMarker;
}
// -----------------------------------------------------------------------
@@ -123,16 +123,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
var ledger = IsMultiWriterLogFile(fileName) ? GetLedger(fileName) : null;
// Set by a rewind: entries that still read unchanged behind the old bookmark are
// skipped, and nothing below the old bookmark is fresh or a calibration anchor.
HashSet<LedgerEntry> rewoundEntries = null;
// skipped (their invocation markers are given back), and nothing below the old
// bookmark is fresh or a calibration anchor.
RewindState rewind = null;
long oldBookmark = -1;
var writerBoundaries = IsWriterBoundaryLogFile(fileName);
if (ledger != null)
{
var rewindTo = await PrepareLedgerAsync(filePath, fileName, ledger, startPos, passNowUtc, token);
if (rewindTo >= 0)
{
rewoundEntries = ApplyRewind(filePath, fileName, ledger, rewindTo, startPos);
rewind = ApplyRewind(filePath, fileName, ledger, rewindTo, startPos);
oldBookmark = startPos;
startPos = rewindTo;
}
@@ -164,7 +166,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
// Guard zone: the file grew and is open anyway — re-check the entries just behind
// the bookmark, where a stale writer's block lands when it is short.
if (ledger != null && rewoundEntries == null && ledger.Entries.Count > 0)
if (ledger != null && rewind == null && ledger.Entries.Count > 0)
{
var guardFrom = ledger.OffsetOfEntryAtOrAfter(startPos - OverwriteGuardBytes);
if (guardFrom >= 0 && guardFrom < startPos)
@@ -178,7 +180,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
_verifiedBytes += startPos - ledger.StartOffset;
_verifyPasses++;
if (earliest >= 0) divergence = earliest;
rewoundEntries = ApplyRewind(filePath, fileName, ledger, divergence, startPos);
rewind = ApplyRewind(filePath, fileName, ledger, divergence, startPos);
oldBookmark = startPos;
startPos = divergence;
}
@@ -259,7 +261,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// A fragment right at the bookmark of a multi-writer file (the previous
// entry ended exactly here, this one does not open an entry) is the tail of
// a block a stale writer laid over already-read bytes: verify before reading on.
if (ledger != null && rewoundEntries == null && entry.Offset == ledger.EndOffset
if (ledger != null && rewind == null && entry.Offset == ledger.EndOffset
&& ledger.LastFragmentOffset != entry.Offset && !entry.Text.StartsWith("<![LOG["))
{
ledger.VerifyRequested = true;
@@ -270,8 +272,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
}
if (ledger != null) ledger.Record(entry);
if (rewoundEntries != null && rewoundEntries.Contains(new LedgerEntry { Offset = entry.Offset, Hash = entry.Hash }))
continue; // unchanged behind the old bookmark — processed before the rewind
var unchanged = rewind != null && rewind.IsUnchanged(entry);
if (unchanged) RestoreInvocationMarkers(rewind, entry.Offset);
// Writer boundary (see ImeLogTracker.Overwrite.cs): an AgentExecutor.log entry that is
// not exactly one CMTrace record ends the bytes of the executor in front of it. Never
// matched — its text is the cut of two writers. Evaluated for unchanged entries too:
// what owns the position in front of them may have changed with the rewind.
if (writerBoundaries && !IsSingleCmTraceRecord(entry.Text))
{
CloseAtWriterBoundary(fileName, entry.Offset, entry.Text);
continue;
}
if (unchanged) continue; // unchanged behind the old bookmark — processed before the rewind
_currentPassLinesAreFresh = passLinesAreFresh && entry.Offset >= oldBookmark;
@@ -293,13 +293,40 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
}
}
/// <summary>Moves the bookmark back to the first changed entry; returns the entries to skip if they still read unchanged behind it.</summary>
private HashSet<LedgerEntry> ApplyRewind(string filePath, string fileName, FileLedger ledger, long rewindTo, long bookmark)
/// <summary>
/// What one rewind pass carries: the entries to skip if they still read unchanged behind the rewind point, and
/// the invocation markers set aside there until their entries prove unchanged. Pass-local by design — whatever
/// is not given back dies with the pass on every exit path.
/// </summary>
private sealed class RewindState
{
public RewindState(string fileName, HashSet<LedgerEntry> unchanged, List<InvocationMarker> markers, InvocationMarker lastPlatformMarker)
{
FileName = fileName;
Unchanged = unchanged;
Markers = markers;
LastPlatformMarker = lastPlatformMarker;
}
public string FileName { get; }
public HashSet<LedgerEntry> Unchanged { get; }
/// <summary>Set aside at the rewind, ascending by offset.</summary>
public List<InvocationMarker> Markers { get; }
/// <summary>The tracker-wide last platform marker, when it was among the set-aside ones.</summary>
public InvocationMarker LastPlatformMarker { get; }
public int Next { get; set; }
public bool IsUnchanged(AssembledEntry entry) => Unchanged.Contains(new LedgerEntry { Offset = entry.Offset, Hash = entry.Hash });
}
/// <summary>Moves the bookmark back to the first changed entry; returns what the re-read needs to skip and to give back.</summary>
private RewindState ApplyRewind(string filePath, string fileName, FileLedger ledger, long rewindTo, long bookmark)
{
var removed = ledger.TruncateFrom(rewindTo);
_positionTracker.SetPosition(filePath, rewindTo);
_heldTails.Remove(filePath);
InvalidateInvocationMarkersFrom(fileName, rewindTo);
var lastPlatformMarker = _lastPlatformMarker;
var markers = TakeInvocationMarkersFrom(fileName, rewindTo, out var tookLastPlatformMarker);
_overwriteRewinds++;
_overwriteBytesReprocessed += bookmark - rewindTo;
@@ -314,7 +341,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
_logger.Debug(message);
}
return removed;
return new RewindState(fileName, removed, markers, tookLastPlatformMarker ? lastPlatformMarker : null);
}
private long RetentionFloor(string fileName, long bookmark)
@@ -331,20 +358,35 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// -----------------------------------------------------------------------
//
// AgentExecutor.log carries "Powershell exit code is N" / "write output done" without a
// policy id. Their owner is decided by FILE POSITION, not by the order lines were read:
// an executor writes its end block at its own stream position, i.e. directly after its
// own start block in the final bytes, so the nearest preceding invocation marker names
// it — and stays right after a rewind re-reads that region. A marker is a platform start
// (policy id), some other invocation (detection, remediation, requirement: null) or a
// close ("gets invoked" banner, "Agent executor completed").
// policy id. Their owner is decided by FILE POSITION: the nearest preceding marker — a
// platform start (policy id), some other invocation (detection, remediation, requirement:
// null), a close ("gets invoked" banner, "Agent executor completed") or a writer boundary.
//
// Every executor writes its lines contiguously from its own stream position, so its end
// block follows its own start block — but that start block may be gone. Two executors that
// open the file at the same end write their start blocks over each other, and the end block
// of the one that lost then sits behind the OTHER executor's start block (session 756970cf:
// a detection script's exit code and output on a platform script). Where one writer's bytes
// meet another's, the cut leaves an entry that is not exactly one CMTrace record — a
// fragment, a merged line or an empty line — unless the line boundaries of both writers
// coincide byte-exactly. That writer boundary closes the platform invocation in front of it.
// AgentExecutor.log only: in the IME log the service writes the markers and the lines they
// own itself, and its fragments are executor telemetry lines.
//
// A marker lives exactly as long as the bytes of its entry: a rewind sets the markers from
// the rewind point on aside, and an entry that reads unchanged gets its markers back.
private sealed class InvocationMarker
{
public long Offset;
public string PolicyId;
public bool IsClose;
/// <summary>Set on a writer boundary: the platform invocation it closed.</summary>
public string ClosedPolicyId;
}
private const string ExecutorLogFileName = "AgentExecutor.log";
private readonly Dictionary<string, List<InvocationMarker>> _invocationMarkers =
new Dictionary<string, List<InvocationMarker>>(StringComparer.OrdinalIgnoreCase);
private InvocationMarker _lastPlatformMarker;
@@ -354,22 +396,59 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private readonly HashSet<string> _platformMarkerFiles = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private long _testEntryOrdinal;
private void RecordInvocationMarker(string policyId, bool isClose)
/// <summary>
/// The executor log, live or rotated (<c>AgentExecutor-yyyyMMdd-HHmmss.log</c>): its invocations are one process
/// each, so a writer boundary ends an invocation — in the final bytes of a rotation as much as in the live file.
/// </summary>
internal static bool IsWriterBoundaryLogFile(string fileName)
=> string.Equals(fileName, ExecutorLogFileName, StringComparison.OrdinalIgnoreCase)
|| (fileName != null
&& fileName.StartsWith("AgentExecutor-", StringComparison.OrdinalIgnoreCase)
&& fileName.EndsWith(".log", StringComparison.OrdinalIgnoreCase));
/// <summary>
/// True when the entry is exactly one CMTrace record: it opens with <c>&lt;![LOG[</c> (culture-sensitive like
/// <c>CmTraceLogParser.TryParseLine</c>, so a BOM is ignorable) and holds no second one — the parser takes the
/// message up to the LAST trailer, so a line cut anywhere, its trailer included, would swallow the record
/// written over it. An empty entry is the terminator of a cut line. A script whose own output contains the
/// open tag loses that output line: the bytes cannot tell it from a merged line.
/// </summary>
internal static bool IsSingleCmTraceRecord(string text)
{
if (string.IsNullOrEmpty(text) || !text.StartsWith(LogOpenTag)) return false;
var first = text.IndexOf(LogOpenTag, StringComparison.Ordinal);
return first >= 0 && text.IndexOf(LogOpenTag, first + LogOpenTag.Length, StringComparison.Ordinal) < 0;
}
private List<InvocationMarker> GetOrCreateMarkerList(string file)
{
if (_currentEntryOffset < 0) return;
var file = _currentSourceFileName ?? TestSourceFileName;
List<InvocationMarker> list;
if (!_invocationMarkers.TryGetValue(file, out list))
{
list = new List<InvocationMarker>();
_invocationMarkers[file] = list;
}
return list;
}
private static void AppendInvocationMarker(List<InvocationMarker> list, InvocationMarker marker)
{
if (list.Count >= MaxInvocationMarkersPerFile) list.RemoveAt(0);
list.Add(marker);
}
private void RecordInvocationMarker(string policyId, bool isClose)
{
if (_currentEntryOffset < 0) return;
var file = _currentSourceFileName ?? TestSourceFileName;
var list = GetOrCreateMarkerList(file);
if (list.Count > 0 && list[list.Count - 1].Offset == _currentEntryOffset)
{
// Two patterns on one line: the platform start outranks the generic argument line.
// A writer boundary is never upgraded.
var last = list[list.Count - 1];
if (!isClose && policyId != null)
if (!isClose && policyId != null && last.ClosedPolicyId == null)
{
last.PolicyId = policyId;
last.IsClose = false;
@@ -379,9 +458,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
return;
}
if (list.Count >= MaxInvocationMarkersPerFile) list.RemoveAt(0);
var marker = new InvocationMarker { Offset = _currentEntryOffset, PolicyId = policyId, IsClose = isClose };
list.Add(marker);
AppendInvocationMarker(list, marker);
if (!isClose && policyId != null)
{
_lastPlatformMarker = marker;
@@ -390,21 +468,34 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
}
/// <summary>
/// The pending platform script that owns the entry being processed, or null.
/// <paramref name="ownerPolicyId"/> names the owning platform script even when its slot is
/// gone (completion already emitted); null when no platform invocation owns the entry.
/// An entry of <paramref name="fileName"/> at <paramref name="offset"/> that is not exactly one CMTrace record:
/// the bytes behind it were written by another process than the bytes in front of it. Closes the platform
/// invocation that owns the position; records nothing when none is open there (a cut multi-line output yields
/// one fragment per line). Idempotent at the same offset.
/// </summary>
private ScriptExecutionState ResolvePlatformScriptForCurrentEntry(out string ownerPolicyId)
private void CloseAtWriterBoundary(string fileName, long offset, string text)
{
var owner = FindOwningMarker(fileName, offset);
if (owner == null || owner.IsClose || owner.PolicyId == null) return;
var list = GetOrCreateMarkerList(fileName);
if (list.Count > 0 && list[list.Count - 1].Offset >= offset) return;
AppendInvocationMarker(list, new InvocationMarker { Offset = offset, IsClose = true, ClosedPolicyId = owner.PolicyId });
var kind = string.IsNullOrEmpty(text) ? "empty line" : text.StartsWith(LogOpenTag) ? "merged line" : "fragment";
_logger.Debug($"ImeLogTracker: writer boundary in {fileName} at offset {offset} ({kind}, {text?.Length ?? 0} chars) closes platform script {owner.PolicyId} (opened at {owner.Offset}) — the bytes behind it belong to another executor");
}
/// <summary>The marker that owns <paramref name="offset"/> in <paramref name="file"/>, or null.</summary>
private InvocationMarker FindOwningMarker(string file, long offset)
{
ownerPolicyId = null;
var file = _currentSourceFileName ?? TestSourceFileName;
InvocationMarker marker = null;
List<InvocationMarker> list = null;
if (_currentEntryOffset >= 0 && _invocationMarkers.TryGetValue(file, out list))
if (offset >= 0 && _invocationMarkers.TryGetValue(file, out list))
{
for (var i = list.Count - 1; i >= 0; i--)
{
if (list[i].Offset <= _currentEntryOffset)
if (list[i].Offset <= offset)
{
marker = list[i];
break;
@@ -420,6 +511,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// (session 24dc69d1: the health-script worker's exit line pre-filled a platform slot's
// exit code, its launch line the slot's context).
if (marker == null && (list == null || list.Count == 0) && _platformMarkerFiles.Contains(file)) marker = _lastPlatformMarker;
return marker;
}
/// <summary>
/// The pending platform script that owns the entry being processed, or null.
/// <paramref name="ownerPolicyId"/> names the owning platform script even when its slot is
/// gone (completion already emitted); null when no platform invocation owns the entry.
/// </summary>
private ScriptExecutionState ResolvePlatformScriptForCurrentEntry(out string ownerPolicyId)
{
ownerPolicyId = null;
var marker = FindOwningMarker(_currentSourceFileName ?? TestSourceFileName, _currentEntryOffset);
if (marker == null || marker.IsClose || marker.PolicyId == null) return null;
ownerPolicyId = marker.PolicyId;
@@ -427,17 +530,32 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
return _pendingPlatformScripts.TryGetValue(marker.PolicyId, out state) ? state : null;
}
private void InvalidateInvocationMarkersFrom(string fileName, long offset)
/// <summary>Debug trace for an executor line that a writer boundary kept away from a platform script.</summary>
private void LogLineBehindWriterBoundary(string what)
{
var marker = FindOwningMarker(_currentSourceFileName ?? TestSourceFileName, _currentEntryOffset);
if (marker?.ClosedPolicyId == null) return;
_logger.Debug($"ImeLogTracker: {what} at offset {_currentEntryOffset} lies behind the writer boundary at {marker.Offset} that closed platform script {marker.ClosedPolicyId} — another executor's end block, not attributed");
}
/// <summary>Removes the markers from <paramref name="offset"/> on and returns them in offset order.</summary>
private List<InvocationMarker> TakeInvocationMarkersFrom(string fileName, long offset, out bool tookLastPlatformMarker)
{
tookLastPlatformMarker = false;
var taken = new List<InvocationMarker>();
List<InvocationMarker> list;
if (!_invocationMarkers.TryGetValue(fileName, out list) || list.Count == 0) return;
var removedLast = false;
for (var i = list.Count - 1; i >= 0 && list[i].Offset >= offset; i--)
if (!_invocationMarkers.TryGetValue(fileName, out list) || list.Count == 0) return taken;
var from = list.Count;
while (from > 0 && list[from - 1].Offset >= offset) from--;
for (var i = from; i < list.Count; i++)
{
if (ReferenceEquals(list[i], _lastPlatformMarker)) removedLast = true;
list.RemoveAt(i);
if (ReferenceEquals(list[i], _lastPlatformMarker)) tookLastPlatformMarker = true;
taken.Add(list[i]);
}
if (removedLast)
list.RemoveRange(from, list.Count - from);
if (tookLastPlatformMarker)
{
_lastPlatformMarker = null;
for (var i = list.Count - 1; i >= 0; i--)
@@ -449,6 +567,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
}
}
}
return taken;
}
private void InvalidateInvocationMarkersFrom(string fileName, long offset) => TakeInvocationMarkersFrom(fileName, offset, out _);
/// <summary>
/// An entry that reads unchanged after a rewind keeps its markers: gives back the set-aside markers at exactly
/// <paramref name="entryOffset"/>. Set-aside markers in front of it belonged to entries that changed and stay
/// gone with their bytes.
/// </summary>
private void RestoreInvocationMarkers(RewindState rewind, long entryOffset)
{
var markers = rewind.Markers;
while (rewind.Next < markers.Count && markers[rewind.Next].Offset < entryOffset) rewind.Next++;
while (rewind.Next < markers.Count && markers[rewind.Next].Offset == entryOffset)
{
var marker = markers[rewind.Next++];
var list = GetOrCreateMarkerList(rewind.FileName);
if (list.Count > 0 && list[list.Count - 1].Offset >= marker.Offset) continue;
AppendInvocationMarker(list, marker);
if (ReferenceEquals(marker, rewind.LastPlatformMarker)) _lastPlatformMarker = marker;
}
}
private long OldestPendingScriptMarkerOffset(string fileName)
@@ -474,6 +614,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
/// <summary>What a rollover of the file does to its invocation markers.</summary>
internal void ClearInvocationMarkersForTest(string fileName) => InvalidateInvocationMarkersFrom(fileName, 0);
internal int InvocationMarkerCountForTest(string fileName)
=> _invocationMarkers.TryGetValue(fileName, out var list) ? list.Count : 0;
internal int LedgerEntryCountForTest(string fileName)
{
FileLedger ledger;
@@ -182,6 +182,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
private readonly Dictionary<string, ScriptExecutionState> _pendingPlatformScripts =
new Dictionary<string, ScriptExecutionState>(StringComparer.OrdinalIgnoreCase);
/// <summary>
/// The platform runs this tracker started and emitted, for the registry reconciliation that
/// checks them against IME's saved results (D-316). Shared with the registry host's thread.
/// </summary>
internal PlatformScriptRunRegister ScriptRuns { get; } = new PlatformScriptRunRegister();
// Health-script (remediation) line-by-line accumulator — single slot because IME
// executes health scripts SEQUENTIALLY within a session (verified across multiple
// diagnostic captures: ProcessScript → context → exit → stdout → stderr → compliance,
@@ -240,7 +246,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// the 32K-char Table Storage property limit that bounds the event downstream. Growth
// driver is the number of log lines the bootstrap chain writes, so re-measure before
// adding a chatty step rather than raising this again.
private const int MaxScriptOutputLength = 8192;
internal const int MaxScriptOutputLength = 8192;
internal const string OutputTruncationMarker = "...[truncated]";
private const int MaxMultiLineBufferLines = 100;
// Size cap for ONE entry — a single physical line (enforced by BoundedLineReader before
// the line is materialized) and an assembled multiline entry alike. Any process able to
@@ -389,6 +396,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
VerifiedBytes = _verifiedBytes,
PassMaxMs = _passMaxMs,
PassGapMaxMs = _passGapMaxMs,
PlatformOutputVerified = ScriptRuns.Verified,
PlatformOutputCorrected = ScriptRuns.Corrected,
PatternHits = new Dictionary<string, int>(_patternHits, StringComparer.OrdinalIgnoreCase),
};
}
@@ -1034,6 +1043,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
internal async Task RunPollPassAsync(CancellationToken token)
{
var startMs = MonotonicMillisProvider();
var passStartedUtc = UtcNowProvider();
if (_lastPassEndMs >= 0 && startMs - _lastPassEndMs > _passGapMaxMs)
{
_passGapMaxMs = startMs - _lastPassEndMs;
@@ -1045,6 +1055,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
FlushRecurringScripts(UtcNowProvider(), shuttingDown: false);
CheckPendingTokenFailure(UtcNowProvider());
// The lines written before this pass started are read now — a held tail or a fragment
// awaiting its check follows a pass or two later, which TrackerCatchUp covers. The
// registry reconciliation waits for such a pass before it pairs an IME save with a run.
if (!token.IsCancellationRequested) ScriptRuns.NotePassCompleted(passStartedUtc);
if (ScriptRuns.ConsumeCountersChanged()) _stateDirty = true;
var endMs = MonotonicMillisProvider();
if (endMs - startMs > _passMaxMs)
{
@@ -1207,6 +1223,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
_verifiedBytes = state.Health.VerifiedBytes;
_passMaxMs = state.Health.PassMaxMs;
_passGapMaxMs = state.Health.PassGapMaxMs;
ScriptRuns.RestoreCounters(state.Health.PlatformOutputVerified, state.Health.PlatformOutputCorrected);
}
if (state.PatternHits != null)
{
@@ -1283,6 +1300,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
VerifiedBytes = _verifiedBytes,
PassMaxMs = _passMaxMs,
PassGapMaxMs = _passGapMaxMs,
PlatformOutputVerified = ScriptRuns.Verified,
PlatformOutputCorrected = ScriptRuns.Corrected,
};
state.PatternHits = new Dictionary<string, int>(_patternHits, StringComparer.OrdinalIgnoreCase);
}
@@ -81,6 +81,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
/// </summary>
public long PassGapMaxMs { get; set; }
/// <summary>
/// Platform-script runs whose emitted output IME's saved result (registry, after the batch
/// report) confirmed — the exit code of the same end block with it. Together with
/// <see cref="PlatformOutputCorrected"/> the field measure of the registry reconciliation (D-316).
/// </summary>
public long PlatformOutputVerified { get; set; }
/// <summary>Platform-script runs whose emitted output IME's saved result corrected (script_output_reconciliation).</summary>
public long PlatformOutputCorrected { get; set; }
/// <summary>Match count per enabled pattern ID — every enabled pattern is present, zeros included.</summary>
public IReadOnlyDictionary<string, int> PatternHits { get; set; }
@@ -185,6 +185,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
// Poll-loop timing maxima (null-defaulting on older state files).
public long PassMaxMs { get; set; }
public long PassGapMaxMs { get; set; }
// Registry reconciliation of platform-script output (null-defaulting on older state files).
public long PlatformOutputVerified { get; set; }
public long PlatformOutputCorrected { get; set; }
}
public class PackageStateData
@@ -4,9 +4,9 @@ using System.Collections.Generic;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
/// <summary>
/// Tracks file read positions for incremental log file reading.
/// In-memory only - resets on agent restart, which is acceptable since
/// we start fresh per enrollment session.
/// Tracks the IME log tracker's file read positions for incremental log file reading.
/// The bookmarks survive agent restarts through the tracker's own state file
/// (<see cref="GetAllPositions"/> / <see cref="RestorePosition"/>); the freshness timestamps do not.
/// </summary>
public class LogFilePositionTracker
{
@@ -0,0 +1,193 @@
using System;
using System.Collections.Generic;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime
{
/// <summary>What the registry reconciliation may do with a saved IME result right now.</summary>
internal enum PlatformRunClaim
{
/// <summary>The run is handed out for reconciliation; it is never handed out again.</summary>
Claimed,
/// <summary>Not decidable yet: the tracker has not read past the save, or the run is not emitted yet.</summary>
Wait,
/// <summary>No run of this agent owns the save (none seen, or already reconciled).</summary>
None,
}
/// <summary>One platform-script run as the tracker emitted it; the register hands out copies.</summary>
internal sealed class PlatformScriptRun
{
public string PolicyId;
public string UserId;
public string RunId;
public DateTime StartSeenUtc;
public bool Emitted;
public bool Claimed;
public string Result;
public string ResultSource;
public int? ExitCode;
/// <summary>As emitted: null when no output line of the run was read, empty when it said so.</summary>
public string Stdout;
public string Stderr;
public PlatformScriptRun Clone() => (PlatformScriptRun)MemberwiseClone();
}
/// <summary>
/// The platform-script runs the tracker has seen, the latest per (IME user id, policy id), for the
/// registry reconciliation (D-316). IME saves a run's output under
/// <c>Policies\&lt;UserId&gt;\&lt;PolicyId&gt;</c> only after it reported the whole script batch, so the
/// observer reading it must know which run a changed key belongs to and what was emitted for it.
/// Written on the tracker's poll thread, claimed on the registry host's timer thread: every member
/// locks, and nothing calls out while holding the lock.
/// <para>
/// A save belongs to the latest run of its key: IME runs a policy once per batch and saves the
/// batch after its last script, so the next run of the same key starts minutes later at the
/// earliest. A save is paired only after the tracker completed a pass that started
/// <see cref="TrackerCatchUp"/> after the save was seen — every line written before the save, the
/// run's start among them, has been read by then, a stalled poll loop included.
/// </para>
/// </summary>
internal sealed class PlatformScriptRunRegister
{
/// <summary>
/// How far behind a seen save the tracker must have started a completed pass before the save
/// is paired; covers the ledger's verify cadence (1 s) for a start line hidden by an overwrite.
/// </summary>
internal static readonly TimeSpan TrackerCatchUp = TimeSpan.FromSeconds(2);
internal const int MaxRuns = 256;
private readonly object _sync = new object();
private readonly Dictionary<string, PlatformScriptRun> _latest =
new Dictionary<string, PlatformScriptRun>(StringComparer.OrdinalIgnoreCase);
private DateTime _lastCompletedPassStartUtc = DateTime.MinValue;
private long _verified;
private long _corrected;
private bool _countersChanged;
private static string KeyOf(string userId, string policyId) => (userId ?? string.Empty) + "|" + policyId;
/// <summary>A platform run began: its start line, or its result when no start line was seen.</summary>
public void NoteStarted(string userId, string policyId, DateTime seenUtc)
{
if (string.IsNullOrEmpty(policyId)) return;
lock (_sync)
{
_latest[KeyOf(userId, policyId)] = new PlatformScriptRun { PolicyId = policyId, UserId = userId, StartSeenUtc = seenUtc };
TrimLocked();
}
}
/// <summary>The tracker emitted the run with these end-block values.</summary>
public void NoteEmitted(ScriptExecutionState script, DateTime emittedUtc)
{
if (script == null || string.IsNullOrEmpty(script.PolicyId)) return;
lock (_sync)
{
var key = KeyOf(script.UserId, script.PolicyId);
if (!_latest.TryGetValue(key, out var run) || run.Emitted)
{
run = new PlatformScriptRun { PolicyId = script.PolicyId, UserId = script.UserId, StartSeenUtc = emittedUtc };
_latest[key] = run;
}
run.Emitted = true;
run.RunId = script.RunId;
run.Result = script.Result;
run.ResultSource = script.ResultSource;
run.ExitCode = script.ExitCode;
run.Stdout = script.Stdout;
run.Stderr = script.Stderr;
TrimLocked();
}
}
/// <summary>The tracker finished a poll pass that started at <paramref name="passStartedUtc"/>.</summary>
public void NotePassCompleted(DateTime passStartedUtc)
{
lock (_sync)
{
if (passStartedUtc > _lastCompletedPassStartUtc) _lastCompletedPassStartUtc = passStartedUtc;
}
}
/// <summary>
/// Hands out the run a save of (<paramref name="userId"/>, <paramref name="policyId"/>) seen at
/// <paramref name="savedSeenUtc"/> belongs to — once. A run whose user id the tracker could not
/// read is keyed without one.
/// </summary>
public PlatformRunClaim TryClaim(string userId, string policyId, DateTime savedSeenUtc, out PlatformScriptRun run)
{
run = null;
lock (_sync)
{
if (_lastCompletedPassStartUtc < savedSeenUtc + TrackerCatchUp) return PlatformRunClaim.Wait;
if (!_latest.TryGetValue(KeyOf(userId, policyId), out var latest)
&& !_latest.TryGetValue(KeyOf(null, policyId), out latest))
return PlatformRunClaim.None;
if (latest.Claimed) return PlatformRunClaim.None;
if (!latest.Emitted) return PlatformRunClaim.Wait;
latest.Claimed = true;
run = latest.Clone();
return PlatformRunClaim.Claimed;
}
}
public void CountVerified()
{
lock (_sync) { _verified++; _countersChanged = true; }
}
public void CountCorrected()
{
lock (_sync) { _corrected++; _countersChanged = true; }
}
/// <summary>Runs whose emitted output IME's saved result confirmed (restart-safe, session-cumulative).</summary>
public long Verified { get { lock (_sync) return _verified; } }
/// <summary>Runs whose emitted output IME's saved result corrected (restart-safe, session-cumulative).</summary>
public long Corrected { get { lock (_sync) return _corrected; } }
public void RestoreCounters(long verified, long corrected)
{
lock (_sync)
{
_verified = verified;
_corrected = corrected;
}
}
/// <summary>True once after a counter moved — the tracker then persists its state.</summary>
public bool ConsumeCountersChanged()
{
lock (_sync)
{
var changed = _countersChanged;
_countersChanged = false;
return changed;
}
}
private void TrimLocked()
{
while (_latest.Count > MaxRuns)
{
string oldestKey = null;
var oldest = DateTime.MaxValue;
foreach (var kv in _latest)
{
if (kv.Value.StartSeenUtc < oldest)
{
oldest = kv.Value.StartSeenUtc;
oldestKey = kv.Key;
}
}
if (oldestKey == null) return;
_latest.Remove(oldestKey);
}
}
}
}
@@ -15,6 +15,19 @@ public class ScriptExecutionState
/// <summary>"platform" or "remediation".</summary>
public string ScriptType { get; set; }
/// <summary>
/// IME user id a platform script ran for (the all-zero GUID in device context), read from its
/// script path or its result line. Keys IME's saved result
/// (<c>Policies\&lt;UserId&gt;\&lt;PolicyId&gt;</c>) that the registry reconciliation compares against.
/// </summary>
public string UserId { get; set; }
/// <summary>
/// Identity of one emitted platform-script run, surfaced as <c>runId</c> on the event: what a
/// later <c>script_output_reconciliation</c> names when it corrects that run.
/// </summary>
public string RunId { get; set; }
/// <summary>
/// UTC timestamp of the script's first observed start line, taken from the source CMTrace log
/// timestamp (so it dates correctly even when the agent replays historic IME log content that
@@ -241,7 +241,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string tenantId,
InformationalEventPost post,
AgentLogger logger,
int helloWaitTimeoutSeconds = 30,
int helloWaitTimeoutSeconds = HelloWaitTimeout.DefaultSeconds,
bool modernDeploymentWatcherEnabled = true,
int modernDeploymentLogLevelMax = 3,
bool modernDeploymentBackfillEnabled = true,
@@ -413,7 +413,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
_tenantId,
_post,
_logger,
_helloTracker);
_helloTracker,
stateDirectory: _stateDirectory);
_shellCoreTracker.FinalizingSetupPhaseTriggered += OnFinalizingSetupPhaseTriggered;
_shellCoreTracker.WhiteGloveCompleted += OnWhiteGloveCompleted;
_shellCoreTracker.EspFailureDetected += OnShellCoreEspFailureDetected;
@@ -0,0 +1,148 @@
using System;
using System.Collections.Generic;
using System.IO;
using AutopilotMonitor.Agent.V2.Core.Logging;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Cross-restart dedup for event-log watchers that backfill a channel on start. RecordIds are
/// per channel, so every watched channel owns one instance with its own state file.
/// <para>
/// Two boundaries, deliberately separate: the restart watermark is the highest RecordId a
/// PRIOR run emitted (loaded once, immutable for this run); the seen-set holds what THIS run
/// processed. A single high-water mark would be wrong because the live watcher is armed before
/// the backfill runs: a live record with a higher RecordId can arrive first and would then
/// make the backfill skip every older, never-emitted record.
/// </para>
/// </summary>
internal sealed class EventRecordWatermark
{
private readonly string _stateFilePath;
private readonly AgentLogger _logger;
private readonly string _label;
private readonly object _lock = new object();
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
private long _restartWatermark = -1;
private long _maxEmittedRecordId = -1;
/// <param name="stateDirectory">Directory for the state file; null keeps the dedup in memory only.</param>
/// <param name="stateFileName">File name inside <paramref name="stateDirectory"/>.</param>
/// <param name="label">Watcher name for log lines.</param>
public EventRecordWatermark(string stateDirectory, string stateFileName, AgentLogger logger, string label)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_label = label ?? "EventLog";
_stateFilePath = string.IsNullOrEmpty(stateDirectory) || string.IsNullOrEmpty(stateFileName)
? null
: Path.Combine(Environment.ExpandEnvironmentVariables(stateDirectory), stateFileName);
}
/// <summary>Highest RecordId emitted by a prior run; -1 when none was persisted.</summary>
public long RestartWatermark
{
get { lock (_lock) return _restartWatermark; }
}
/// <summary>True when the record was already emitted, by a prior run or by this one.</summary>
public bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_lock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
/// <summary>
/// Atomically claims <paramref name="recordId"/> for this run. True on first sight (the
/// caller processes it), false when it was already processed. Records without a RecordId
/// (-1) are always claimed, never tracked. Claiming persists nothing — only
/// <see cref="MarkEmitted"/> does, so a record that is counted instead of emitted costs no
/// state-file write (CBS maintenance writes hundreds of servicing records in seconds).
/// </summary>
public bool TryClaim(long recordId)
{
if (recordId < 0) return true;
lock (_lock)
{
return recordId > _restartWatermark && _seenThisRun.Add(recordId);
}
}
/// <summary>
/// Persists <paramref name="recordId"/> when it raises the emitted maximum, so a restarted
/// agent's backfill skips it. Called after the record's event and engine signal were
/// handed to the ingress: a crash before that re-emits the record instead of losing it.
/// The ingress makes them durable only when its worker appends them to the SignalLog —
/// normally milliseconds later; a process killed inside that window (e.g. by a reboot)
/// loses the record, and the backfill does not bring it back. Counted records above the
/// persisted maximum are counted again after a restart, which only touches the activity
/// summary.
/// </summary>
public void MarkEmitted(long recordId)
{
if (recordId < 0) return;
long toPersist = -1;
lock (_lock)
{
if (recordId > _maxEmittedRecordId)
{
_maxEmittedRecordId = recordId;
toPersist = recordId;
}
}
if (toPersist >= 0) Persist(toPersist);
}
public void Load()
{
if (_stateFilePath == null || !File.Exists(_stateFilePath)) return;
try
{
var state = JsonConvert.DeserializeObject<WatermarkState>(File.ReadAllText(_stateFilePath));
if (state == null) return;
lock (_lock)
{
_restartWatermark = state.LastRecordId;
_maxEmittedRecordId = state.LastRecordId;
}
_logger.Info($"{_label} watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
catch (Exception ex)
{
_logger.Warning($"Failed to load {_label} watermark: {ex.Message}");
}
}
private void Persist(long recordId)
{
if (_stateFilePath == null) return;
try
{
Directory.CreateDirectory(Path.GetDirectoryName(_stateFilePath));
var json = JsonConvert.SerializeObject(
new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow },
Formatting.Indented);
var tempPath = _stateFilePath + ".tmp";
File.WriteAllText(tempPath, json);
File.Copy(tempPath, _stateFilePath, overwrite: true);
try { File.Delete(tempPath); } catch { }
}
catch (Exception ex)
{
_logger.Warning($"Failed to persist {_label} watermark: {ex.Message}");
}
}
internal sealed class WatermarkState
{
public long LastRecordId { get; set; }
public DateTime PersistedUtc { get; set; }
}
}
}
@@ -47,7 +47,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
internal const string HResult_UserSkippedHello = "0x801C044F";
internal const int HelloCompletionTimeoutSeconds = 300;
internal const int BackfillLookbackMinutes = 5;
private static readonly HashSet<int> TrackedUdrEventIds = new HashSet<int>
@@ -81,6 +80,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly InformationalEventPost _post;
private readonly int _helloWaitTimeoutSeconds;
// How long a started Hello wizard may take to reach a terminal event (300/301/362): the
// tenant's Hello wait resolved through HelloWaitTimeout (300..3600 s). The engine's
// hello_safety window spends the same budget from the ESP exit, so this timer never cuts
// a user off inside the wizard before the tenant's budget is used up.
private readonly int _helloCompletionTimeoutSeconds;
private EventLogWatcher _udrWatcher;
private EventLogWatcher _helloForBusinessWatcher;
private System.Threading.Timer _policyCheckTimer;
@@ -121,13 +126,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string tenantId,
InformationalEventPost post,
AgentLogger logger,
int helloWaitTimeoutSeconds = 30)
int helloWaitTimeoutSeconds = HelloWaitTimeout.DefaultSeconds)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_helloWaitTimeoutSeconds = helloWaitTimeoutSeconds;
_helloCompletionTimeoutSeconds = HelloWaitTimeout.EffectiveSeconds(helloWaitTimeoutSeconds);
}
// =====================================================================
@@ -1094,7 +1100,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
if (policyKnownEnabled)
{
_logger.Info($"Hello wait timeout ({_helloWaitTimeoutSeconds}s) expired but Hello policy is enabled — " +
$"wizard not yet visible, starting long completion timer ({HelloCompletionTimeoutSeconds}s)");
$"wizard not yet visible, starting long completion timer ({_helloCompletionTimeoutSeconds}s)");
_post.Emit(new EnrollmentEvent
{
@@ -1105,7 +1111,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
Source = "EspAndHelloTracker",
Phase = EnrollmentPhase.Unknown,
Message = $"Hello wizard did not start within {_helloWaitTimeoutSeconds}s after ESP exit — " +
$"Hello policy is enabled, waiting up to {HelloCompletionTimeoutSeconds}s for wizard",
$"Hello policy is enabled, waiting up to {_helloCompletionTimeoutSeconds}s for wizard",
Data = new Dictionary<string, object>
{
{ "timeoutSeconds", _helloWaitTimeoutSeconds },
@@ -1205,11 +1211,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
if (_isHelloCompleted) return;
if (_helloCompletionTimer != null) return;
_logger.Info($"Starting Hello completion timer ({HelloCompletionTimeoutSeconds}s) - waiting for terminal Hello event (300/301/362)");
_logger.Info($"Starting Hello completion timer ({_helloCompletionTimeoutSeconds}s) - waiting for terminal Hello event (300/301/362)");
_helloCompletionTimer = new System.Threading.Timer(
OnHelloCompletionTimeout,
null,
TimeSpan.FromSeconds(HelloCompletionTimeoutSeconds),
TimeSpan.FromSeconds(_helloCompletionTimeoutSeconds),
TimeSpan.FromMilliseconds(-1));
}
@@ -1230,7 +1236,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
return;
}
_logger.Warning($"Hello completion timeout ({HelloCompletionTimeoutSeconds}s) expired after wizard start without terminal event");
_logger.Warning($"Hello completion timeout ({_helloCompletionTimeoutSeconds}s) expired after wizard start without terminal event");
_isHelloCompleted = true;
HelloOutcome = _helloWizardStarted ? "timeout" : "wizard_not_started";
StopHelloCompletionTimerLocked();
@@ -1243,10 +1249,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
Severity = EventSeverity.Warning,
Source = "EspAndHelloTracker",
Phase = EnrollmentPhase.Unknown,
Message = $"Hello wizard started but no terminal event (300/301/362) arrived within {HelloCompletionTimeoutSeconds}s",
Message = $"Hello wizard started but no terminal event (300/301/362) arrived within {_helloCompletionTimeoutSeconds}s",
Data = new Dictionary<string, object>
{
{ "timeoutSeconds", HelloCompletionTimeoutSeconds },
{ "timeoutSeconds", _helloCompletionTimeoutSeconds },
{ "helloWizardStarted", _helloWizardStarted }
},
ImmediateUpload = true
@@ -1304,6 +1310,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
internal void TriggerWaitTimeoutForTest() => OnHelloWaitTimeout(null);
internal void TriggerCompletionTimeoutForTest() => OnHelloCompletionTimeout(null);
internal int CompletionTimeoutSecondsForTest => _helloCompletionTimeoutSeconds;
internal bool IsWaitTimerActiveForTest { get { lock (_stateLock) { return _helloWaitTimer != null; } } }
internal bool IsCompletionTimerActiveForTest { get { lock (_stateLock) { return _helloCompletionTimer != null; } } }
internal bool IsHelloWizardStartedForTest { get { lock (_stateLock) { return _helloWizardStarted; } } }
@@ -360,7 +360,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
: (DateTime?)null;
}
/// <summary>StateMessageEnforcementState bands (verified against decompiled IME 1.97/1.104).</summary>
/// <summary>StateMessageEnforcementState bands (verified against IME 1.97/1.104).</summary>
internal static string ClassifyEnforcementState(int state)
{
if (state >= 1000 && state < 2000) return "success";
@@ -0,0 +1,401 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Text.Json;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.Shared.Models;
using Microsoft.Win32;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Registry second pillar for platform scripts (D-316). After IME reported a script batch to
/// Intune it saves every run of the batch under
/// <c>HKLM\SOFTWARE\Microsoft\IntuneManagementExtension\Policies\&lt;UserId&gt;\&lt;PolicyId&gt;</c>:
/// <c>Result</c>, <c>ErrorCode</c> and <c>ResultDetails</c>, whose <c>ExecutionMsg</c> is what IME read
/// from the result file of exactly that policy — the stdout when the run succeeded (IME's Success
/// means an empty stderr file, whatever the exit code), the stderr when it failed with ErrorCode 99.
/// The log tracker assigns a run's exit code and output by file position in AgentExecutor.log, where
/// no policy id anchors them; this observer checks what was emitted against the keyed value and
/// emits <c>script_output_reconciliation</c> when they disagree.
/// <para>
/// Snapshot-and-diff like <see cref="ImeRegistryAppStateObserver"/>, ticked by the same host: the
/// first tick is the silent baseline, a later save is paired with its run through
/// <see cref="PlatformScriptRunRegister"/>. No exit code lives in the registry: a run whose output
/// matches proves the exit code of the same end block; a contradicted run loses its exit code.
/// </para>
/// </summary>
internal sealed class ImeRegistryScriptResultObserver
{
public const string SourceLabel = "RegistryScriptResult";
internal const string PoliciesKeyPath = ImeRegistryAppStateObserver.ImeRootKeyPath + @"\Policies";
/// <summary>A save whose run never became claimable is dropped after this long (send failure of a later batch, agent restart).</summary>
internal static readonly TimeSpan PendingExpiry = TimeSpan.FromMinutes(10);
/// <summary>Safety cap on correction events per agent run; corrections beyond it are counted only.</summary>
internal const int MaxCorrectionsPerSession = 100;
private readonly PlatformScriptRunRegister _runs;
private readonly Func<IReadOnlyDictionary<string, SavedScriptResult>> _snapshotSource;
private readonly InformationalEventPost _post;
private readonly AgentLogger? _logger;
private readonly IClock _clock;
private readonly object _sync = new object();
private IReadOnlyDictionary<string, SavedScriptResult>? _last;
private readonly Dictionary<string, (SavedScriptResult Saved, DateTime SeenUtc)> _pending =
new Dictionary<string, (SavedScriptResult, DateTime)>(StringComparer.OrdinalIgnoreCase);
private int _correctionsEmitted;
/// <param name="snapshotSource">Replaces the live registry walk in tests; per instance, so parallel test classes never share it.</param>
public ImeRegistryScriptResultObserver(
PlatformScriptRunRegister runs,
InformationalEventPost post,
AgentLogger? logger,
IClock clock,
Func<IReadOnlyDictionary<string, SavedScriptResult>>? snapshotSource = null)
{
_runs = runs ?? throw new ArgumentNullException(nameof(runs));
_post = post ?? throw new ArgumentNullException(nameof(post));
_clock = clock ?? throw new ArgumentNullException(nameof(clock));
_logger = logger;
_snapshotSource = snapshotSource ?? ReadSnapshot;
}
/// <summary>One observation pass: snapshot → new saves → pair → judge → correct. Fail-soft: never throws.</summary>
public void Tick(string reason)
{
try
{
lock (_sync)
{
var snapshot = _snapshotSource();
var nowUtc = _clock.UtcNow;
if (_last == null)
{
// Saves that predate this agent run belong to runs it never emitted.
_last = snapshot;
_logger?.Info($"RegistryScriptResult: baseline captured ({snapshot.Count} saved platform-script results, trigger={reason})");
return;
}
foreach (var kv in snapshot)
{
if (_last.TryGetValue(kv.Key, out var previous) && previous.SameSaveAs(kv.Value)) continue;
_pending[kv.Key] = (kv.Value, nowUtc); // a newer save of the key replaces a waiting one
}
_last = snapshot;
ProcessPending(nowUtc);
}
}
catch (Exception ex)
{
_logger?.Debug($"RegistryScriptResult: tick failed (fail-soft): {ex.Message}");
}
}
private void ProcessPending(DateTime nowUtc)
{
foreach (var key in _pending.Keys.ToList())
{
var (saved, seenUtc) = _pending[key];
var claim = _runs.TryClaim(saved.UserId, saved.PolicyId, seenUtc, out var run);
if (claim == PlatformRunClaim.Wait)
{
if (nowUtc - seenUtc >= PendingExpiry) _pending.Remove(key);
continue;
}
_pending.Remove(key);
if (claim == PlatformRunClaim.Claimed && run != null) Reconcile(run, saved, nowUtc);
}
}
private void Reconcile(PlatformScriptRun run, SavedScriptResult saved, DateTime nowUtc)
{
var judgement = PlatformOutputJudge.Judge(run, saved);
switch (judgement.Verdict)
{
case PlatformOutputVerdict.Verified:
_runs.CountVerified();
_logger?.Debug($"RegistryScriptResult: platform script {run.PolicyId} output confirmed by IME's saved result");
return;
case PlatformOutputVerdict.NotComparable:
_logger?.Debug($"RegistryScriptResult: platform script {run.PolicyId} not comparable ({judgement.Reason})");
return;
}
_runs.CountCorrected();
if (_correctionsEmitted >= MaxCorrectionsPerSession)
{
_logger?.Warning($"RegistryScriptResult: correction for {run.PolicyId} not emitted — cap of {MaxCorrectionsPerSession} per agent run reached");
return;
}
_correctionsEmitted++;
var foreign = judgement.Verdict == PlatformOutputVerdict.Foreign;
var culture = CultureInfo.InvariantCulture;
var data = new Dictionary<string, string>(StringComparer.Ordinal)
{
["policyId"] = run.PolicyId,
["scriptType"] = "platform",
["outcome"] = foreign ? "foreign" : "repaired",
["reason"] = judgement.Reason,
};
if (!string.IsNullOrEmpty(run.RunId)) data["runId"] = run.RunId!;
if (!string.IsNullOrEmpty(saved.Result)) data["result"] = saved.Result!;
if (saved.ErrorCode.HasValue) data["registryErrorCode"] = saved.ErrorCode.Value.ToString(culture);
// A present key replaces the emitted value, an empty one clears it; an absent key keeps it.
if (judgement.Stdout != null) data["stdout"] = judgement.Stdout;
if (judgement.Stderr != null) data["stderr"] = judgement.Stderr;
if (foreign && run.ExitCode.HasValue) data["previousExitCode"] = run.ExitCode.Value.ToString(culture);
var shortId = run.PolicyId.Length > 8 ? run.PolicyId.Substring(0, 8) : run.PolicyId;
var message = foreign
? $"Platform script {shortId}: output corrected from IME's saved result; the log had attributed another script's output, exit code unverified"
: $"Platform script {shortId}: output completed from IME's saved result";
_post.Emit(
eventType: SharedEventTypes.ScriptOutputReconciliation,
source: SourceLabel,
message: message,
severity: foreign ? EventSeverity.Warning : EventSeverity.Info,
immediateUpload: foreign,
data: data,
occurredAtUtc: nowUtc);
if (foreign)
_logger?.Warning($"RegistryScriptResult: platform script {run.PolicyId} run {run.RunId} carried another script's end block ({judgement.Reason}) — corrected from IME's saved result");
else
_logger?.Info($"RegistryScriptResult: platform script {run.PolicyId} run {run.RunId} output completed from IME's saved result ({judgement.Reason})");
}
// ── snapshot ────────────────────────────────────────────────────────────
internal static string KeyOf(string userId, string policyId) => userId + "|" + policyId;
internal static IReadOnlyDictionary<string, SavedScriptResult> ReadSnapshot()
{
var snapshot = new Dictionary<string, SavedScriptResult>(StringComparer.OrdinalIgnoreCase);
using var baseKey = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64);
using var policies = baseKey.OpenSubKey(PoliciesKeyPath);
if (policies == null) return snapshot;
foreach (var userId in policies.GetSubKeyNames())
{
if (!Guid.TryParse(userId, out _)) continue;
using var userKey = policies.OpenSubKey(userId);
if (userKey == null) continue;
foreach (var policyId in userKey.GetSubKeyNames())
{
if (!Guid.TryParse(policyId, out _)) continue;
using var policyKey = userKey.OpenSubKey(policyId);
if (policyKey == null) continue;
snapshot[KeyOf(userId, policyId)] = new SavedScriptResult
{
UserId = userId,
PolicyId = policyId,
Result = policyKey.GetValue("Result") as string,
ErrorCode = TryReadInt(policyKey.GetValue("ErrorCode")),
ResultDetails = policyKey.GetValue("ResultDetails") as string,
LastUpdatedTimeUtc = policyKey.GetValue("LastUpdatedTimeUtc") as string,
};
}
}
return snapshot;
}
private static int? TryReadInt(object? value)
{
switch (value)
{
case int i: return i;
case long l when l >= int.MinValue && l <= int.MaxValue: return (int)l;
case string s when int.TryParse(s, NumberStyles.Integer, CultureInfo.InvariantCulture, out var parsed): return parsed;
default: return null;
}
}
}
/// <summary>One platform-script result as IME saved it under <c>Policies\&lt;UserId&gt;\&lt;PolicyId&gt;</c>.</summary>
internal sealed class SavedScriptResult
{
public string UserId { get; set; } = string.Empty;
public string PolicyId { get; set; } = string.Empty;
public string? Result { get; set; }
public int? ErrorCode { get; set; }
public string? ResultDetails { get; set; }
public string? LastUpdatedTimeUtc { get; set; }
/// <summary>The same save: IME stamps every save with a new <c>LastUpdatedTimeUtc</c>.</summary>
public bool SameSaveAs(SavedScriptResult other)
=> string.Equals(LastUpdatedTimeUtc, other.LastUpdatedTimeUtc, StringComparison.Ordinal)
&& string.Equals(Result, other.Result, StringComparison.Ordinal)
&& ErrorCode == other.ErrorCode
&& string.Equals(ResultDetails, other.ResultDetails, StringComparison.Ordinal);
/// <summary>
/// <c>ExecutionMsg</c> of the CryptoResult JSON in <c>ResultDetails</c> (IME 1.73 and later), or null
/// when the value is not that JSON ("Hash mismatch", a launch failure).
/// </summary>
public string? ExecutionMessage()
{
if (string.IsNullOrWhiteSpace(ResultDetails)) return null;
try
{
using var doc = JsonDocument.Parse(ResultDetails!);
if (doc.RootElement.ValueKind != JsonValueKind.Object) return null;
return doc.RootElement.TryGetProperty("ExecutionMsg", out var msg) && msg.ValueKind == JsonValueKind.String
? msg.GetString()
: null;
}
catch (JsonException)
{
return null;
}
}
}
internal enum PlatformOutputVerdict
{
/// <summary>IME's saved result cannot confirm or contradict the run (no output saved, other failure kind).</summary>
NotComparable,
/// <summary>The emitted output is what IME saved: the end block, its exit code included, is the run's own.</summary>
Verified,
/// <summary>The log delivered the run's output incomplete; IME's saved text completes it, the exit code stands.</summary>
Repaired,
/// <summary>The emitted output contradicts IME's saved result: the end block was another executor's.</summary>
Foreign,
}
internal sealed class PlatformOutputJudgement
{
public PlatformOutputVerdict Verdict { get; set; }
public string Reason { get; set; } = string.Empty;
/// <summary>Value to publish (Repaired/Foreign): null keeps the emitted stdout, empty clears it.</summary>
public string? Stdout { get; set; }
/// <summary>Value to publish (Repaired/Foreign): null keeps the emitted stderr, empty clears it.</summary>
public string? Stderr { get; set; }
}
/// <summary>
/// Compares a platform-script run as the tracker emitted it with IME's saved result. Comparable are
/// a Success (ExecutionMsg = stdout; IME's Success means the stderr file was empty) and a Failed with
/// ErrorCode 99 (ExecutionMsg = stderr). Both texts are compared after CR/LF normalisation and Trim;
/// the tracker's own 8192-character cut is a prefix of the saved text.
/// </summary>
internal static class PlatformOutputJudge
{
internal const int ImeStderrErrorCode = 99;
private const string OutputErrorSeparator = ", error =";
public static PlatformOutputJudgement Judge(PlatformScriptRun run, SavedScriptResult saved)
{
var executionMessage = saved.ExecutionMessage();
if (executionMessage == null) return NotComparable("no_execution_message");
// The run's IME result line and the save come from the same IME run.
if (string.Equals(run.ResultSource, "ime_policy_result", StringComparison.Ordinal)
&& !string.Equals(run.Result, saved.Result, StringComparison.OrdinalIgnoreCase))
return NotComparable("result_differs");
if (string.Equals(saved.Result, "Success", StringComparison.OrdinalIgnoreCase))
return JudgeSuccess(run, Normalize(executionMessage)!);
if (string.Equals(saved.Result, "Failed", StringComparison.OrdinalIgnoreCase) && saved.ErrorCode == ImeStderrErrorCode)
return JudgeStderrFailure(run, Normalize(executionMessage)!);
return NotComparable("no_saved_output");
}
private static PlatformOutputJudgement JudgeSuccess(PlatformScriptRun run, string savedStdout)
{
var stdout = NormalizeEmitted(run.Stdout, out var stdoutCut);
var stderr = Normalize(run.Stderr);
var stderrClean = string.IsNullOrEmpty(stderr);
if (!string.IsNullOrEmpty(stdout) && Agrees(stdout!, stdoutCut, savedStdout))
return stderrClean ? Verified() : Correct(PlatformOutputVerdict.Repaired, "stderr_cleared", savedStdout, string.Empty);
if (stdout != null && stdout.Length == 0 && savedStdout.Length == 0)
return stderrClean ? Verified() : Correct(PlatformOutputVerdict.Foreign, "stderr_on_success", string.Empty, string.Empty);
if (stdout == null)
{
if (!stderrClean) return Correct(PlatformOutputVerdict.Foreign, "stderr_on_success", savedStdout, string.Empty);
return savedStdout.Length > 0
? Correct(PlatformOutputVerdict.Repaired, "stdout_missing", savedStdout, string.Empty)
: NotComparable("nothing_read");
}
// "write output done. output = X, error = Y" splits at the first ", error =" — a stdout that
// contains the separator ends up partly in stderr. The saved text shows the cut right there.
if (stdout.Length > 0 && !stdoutCut && !stderrClean
&& savedStdout.StartsWith(stdout, StringComparison.Ordinal)
&& savedStdout.Substring(stdout.Length).TrimStart().StartsWith(OutputErrorSeparator, StringComparison.Ordinal))
return Correct(PlatformOutputVerdict.Repaired, "stdout_split", savedStdout, string.Empty);
return Correct(PlatformOutputVerdict.Foreign, "stdout_differs", savedStdout, string.Empty);
}
private static PlatformOutputJudgement JudgeStderrFailure(PlatformScriptRun run, string savedStderr)
{
var stderr = NormalizeEmitted(run.Stderr, out var stderrCut);
if (!string.IsNullOrEmpty(stderr) && Agrees(stderr!, stderrCut, savedStderr)) return Verified();
// Nothing of the end block read: complete the stderr, the run's stdout stays as emitted.
if (run.Stderr == null && run.Stdout == null)
return Correct(PlatformOutputVerdict.Repaired, "stderr_missing", null, savedStderr);
// The end block read said something else — an empty stderr included: IME read stderr from this
// run's error file. The block's stdout is the other executor's as well.
return Correct(PlatformOutputVerdict.Foreign, "stderr_differs", string.Empty, savedStderr);
}
private static bool Agrees(string emitted, bool emittedCut, string saved)
=> string.Equals(emitted, saved, StringComparison.Ordinal)
|| (emittedCut && saved.StartsWith(emitted, StringComparison.Ordinal));
/// <summary>CR LF and lone CR to LF, then Trim — what the tracker's line reader and handlers make of a text.</summary>
internal static string? Normalize(string? text)
=> text?.Replace("\r\n", "\n").Replace('\r', '\n').Trim();
private static string? NormalizeEmitted(string? text, out bool cut)
{
cut = false;
if (text == null) return null;
if (text.EndsWith(ImeLogTracker.OutputTruncationMarker, StringComparison.Ordinal))
{
cut = true;
text = text.Substring(0, text.Length - ImeLogTracker.OutputTruncationMarker.Length);
}
return Normalize(text);
}
private static PlatformOutputJudgement Verified()
=> new PlatformOutputJudgement { Verdict = PlatformOutputVerdict.Verified, Reason = "match" };
private static PlatformOutputJudgement NotComparable(string reason)
=> new PlatformOutputJudgement { Verdict = PlatformOutputVerdict.NotComparable, Reason = reason };
private static PlatformOutputJudgement Correct(PlatformOutputVerdict verdict, string reason, string? stdout, string? stderr)
=> new PlatformOutputJudgement
{
Verdict = verdict,
Reason = reason,
// The tracker's own output cap: a corrected value is never longer than an emitted one.
Stdout = stdout == null ? null : ImeLogTracker.TruncateOutput(stdout),
Stderr = stderr == null ? null : ImeLogTracker.TruncateOutput(stderr),
};
}
}
@@ -0,0 +1,157 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using Microsoft.Win32;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Reads the registry state behind the OOBE quality update (D-310, S4/S5): the NDUP state
/// Windows keeps while the update page runs, and both policy paths of the ESP setting "Install
/// Windows quality updates". <see cref="OobeUpdateTelemetry"/> logs it and reports it as
/// <c>oobe_update_state</c>. Read-only; the values are update-state flags and policy values,
/// no user data.
/// </summary>
internal static class OobeUpdateRegistrySnapshot
{
/// <summary>The snapshot's keys, each with the alias it carries in <c>oobe_update_state</c>.</summary>
internal static readonly (string Alias, string Path)[] Keys =
{
("setupOobeNdup", @"SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OOBE\NDUP"),
("setupOobeNdupUpdates", @"SOFTWARE\Microsoft\Windows\CurrentVersion\Setup\OOBE\NDUP\Updates"),
("ndup", @"SOFTWARE\Microsoft\Windows\CurrentVersion\NDUP"),
("ndupUpdates", @"SOFTWARE\Microsoft\Windows\CurrentVersion\NDUP\Updates"),
("espSetupPolicy", @"SOFTWARE\Microsoft\Windows\Autopilot\EnrollmentStatusTracking\Device\Setup\Policy"),
("policyManagerSystem", PolicyManagerSystemKey),
("policiesOobe", @"SOFTWARE\Policies\Microsoft\Windows\OOBE"),
};
// Carries every System policy of the device; only the OOBE ones belong in this snapshot.
internal const string PolicyManagerSystemKey = @"SOFTWARE\Microsoft\PolicyManager\current\device\System";
internal const int MaxEntries = 30;
internal const int MaxValueLength = 200;
/// <summary>One key of the snapshot: whether it exists, whether it holds values, and its line.</summary>
internal sealed class KeyState
{
public KeyState(string alias, string path, bool exists, bool hasValues, string line)
{
Alias = alias;
Path = path;
Exists = exists;
HasValues = hasValues;
Line = line;
}
public string Alias { get; }
public string Path { get; }
public bool Exists { get; }
/// <summary>
/// True when the key holds at least one value — for <see cref="PolicyManagerSystemKey"/>
/// one with OOBE in its name. Subkeys alone are no values.
/// </summary>
public bool HasValues { get; }
/// <summary><see cref="FormatEntries"/> plus the subkey names; null when the key is absent.</summary>
public string Line { get; }
}
/// <summary>Reads every key from the 64-bit view. Throws on a registry failure; the caller is fail-soft.</summary>
public static IReadOnlyList<KeyState> Read()
{
var states = new List<KeyState>(Keys.Length);
using (var hklm = RegistryKey.OpenBaseKey(RegistryHive.LocalMachine, RegistryView.Registry64))
{
foreach (var (alias, path) in Keys)
{
using (var key = hklm.OpenSubKey(path))
{
if (key == null)
{
states.Add(new KeyState(alias, path, exists: false, hasValues: false, line: null));
continue;
}
var entries = key.GetValueNames()
.Select(name => new KeyValuePair<string, object>(name, key.GetValue(name)))
.ToList();
states.Add(Describe(alias, path, entries, key.GetSubKeyNames()));
}
}
}
return states;
}
/// <summary>The state of a present key from its values and subkey names.</summary>
internal static KeyState Describe(string alias, string path, IReadOnlyList<KeyValuePair<string, object>> entries, string[] subKeys)
{
var oobeOnly = string.Equals(path, PolicyManagerSystemKey, StringComparison.OrdinalIgnoreCase);
var line = FormatEntries(entries, oobeOnly);
if (subKeys != null && subKeys.Length > 0)
line += $" | subkeys: {string.Join(", ", subKeys.Take(MaxEntries))}";
return new KeyState(alias, path, exists: true,
hasValues: entries.Any(e => !oobeOnly || IsOobeName(e.Key)), line: line);
}
/// <summary>One agent.log line per present key, one naming the absent keys.</summary>
public static void Log(AgentLogger logger, string moment, IReadOnlyList<KeyState> states)
{
if (logger == null || states == null) return;
var absent = new List<string>();
foreach (var state in states)
{
if (!state.Exists)
{
absent.Add(state.Path);
continue;
}
logger.Info($"OOBE update registry ({moment}): HKLM\\{state.Path}: {state.Line}");
}
if (absent.Count > 0)
logger.Info($"OOBE update registry ({moment}): absent: {string.Join(", ", absent)}");
}
/// <summary>
/// <c>name=value</c> pairs ordered by name, bounded in count and length.
/// <paramref name="oobeOnly"/> keeps only names that mention OOBE.
/// </summary>
internal static string FormatEntries(IEnumerable<KeyValuePair<string, object>> entries, bool oobeOnly)
{
var selected = entries
.Where(e => !oobeOnly || IsOobeName(e.Key))
.OrderBy(e => e.Key, StringComparer.OrdinalIgnoreCase)
.ToList();
if (selected.Count == 0) return oobeOnly ? "(no OOBE values)" : "(no values)";
var shown = selected.Take(MaxEntries)
.Select(e => $"{(e.Key.Length == 0 ? "(default)" : e.Key)}={FormatValue(e.Value)}");
var line = string.Join(", ", shown);
return selected.Count > MaxEntries ? $"{line}, … (+{selected.Count - MaxEntries} more)" : line;
}
internal static string FormatValue(object value)
{
string text;
switch (value)
{
case null: text = "(null)"; break;
case int i: text = i.ToString(CultureInfo.InvariantCulture); break;
case long l: text = l.ToString(CultureInfo.InvariantCulture); break;
case string s: text = s; break;
case string[] multi: text = string.Join("|", multi); break;
case byte[] bytes: text = $"(binary {bytes.Length} bytes)"; break;
default: text = Convert.ToString(value, CultureInfo.InvariantCulture) ?? string.Empty; break;
}
return text.Length > MaxValueLength ? text.Substring(0, MaxValueLength) + "…" : text;
}
private static bool IsOobeName(string name) =>
name != null && name.IndexOf("OOBE", StringComparison.OrdinalIgnoreCase) >= 0;
}
}
@@ -0,0 +1,250 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// One CloudExperienceHost record of the OOBE update page, parsed by
/// <see cref="ShellCoreTracker.ParseUpdatePageRecord"/>.
/// </summary>
internal sealed class OobeUpdatePageRecord
{
public const string PageStarted = "page_started";
public const string PageStopped = "page_stopped";
public const string EventName = "event_name";
public OobeUpdatePageRecord(string cxhEvent, int windowsEventId, string page = null, string result = null, string name = null, string value = null)
{
CxhEvent = cxhEvent;
WindowsEventId = windowsEventId;
Page = page;
Result = result;
Name = name;
Value = value;
}
/// <summary><see cref="PageStarted"/>, <see cref="PageStopped"/> or <see cref="EventName"/>.</summary>
public string CxhEvent { get; }
public int WindowsEventId { get; }
/// <summary>The page's CXID (start), or the page a stop is attributed to.</summary>
public string Page { get; }
/// <summary>The page result of a stop.</summary>
public string Result { get; }
/// <summary>The event name of a 62406/62407 record.</summary>
public string Name { get; }
/// <summary>A 62407 value of a safe form (boolean, integer, HRESULT); null otherwise.</summary>
public string Value { get; }
}
/// <summary>
/// Measurement telemetry of the OOBE quality update (D-310): the update page's
/// CloudExperienceHost records as <c>oobe_update_page</c> and the registry state behind it as
/// <c>oobe_update_state</c>. Debug, bounded per agent run, no decisions. Owned by
/// <see cref="ShellCoreTracker"/>, which observes the page.
/// </summary>
internal sealed class OobeUpdateTelemetry
{
internal const int MaxPageEventsPerRun = 150;
internal const int MaxPageEventsPerKey = 3;
internal const int MaxStateEventsPerRun = 6;
internal const int MaxStateLineLength = 1000;
/// <summary>When a registry snapshot is taken (<c>moment</c> of <c>oobe_update_state</c>).</summary>
internal static class Moments
{
public const string AgentStart = "agent_start";
public const string UpdatePageStarted = "update_page_started";
public const string UpdatePageStopped = "update_page_stopped";
public const string AgentStop = "agent_stop";
}
private const string Source = "ShellCoreTracker";
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly AgentLogger _logger;
private readonly Func<IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState>> _readRegistry;
private readonly object _lock = new object();
private readonly Dictionary<string, int> _pageEventsPerKey = new Dictionary<string, int>(StringComparer.Ordinal);
private int _pageEvents;
private bool _pageCapLogged;
private int _stateEvents;
private string _lastStateContent;
public OobeUpdateTelemetry(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger,
Func<IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState>> readRegistry)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_readRegistry = readRegistry ?? throw new ArgumentNullException(nameof(readRegistry));
}
/// <summary>
/// One <c>oobe_update_page</c> per update-page record, at most
/// <see cref="MaxPageEventsPerKey"/> per page/name — a page that logs the same name in a
/// loop cannot flood the session. Event names also share <see cref="MaxPageEventsPerRun"/>
/// per run; a page start or stop is outside that budget, because the time attribution
/// reads the update's span from them. True when the record was reported.
/// </summary>
public bool ReportPage(OobeUpdatePageRecord record, DateTime occurredAtUtc, bool isBackfill)
{
if (record == null) return false;
var isEventName = record.CxhEvent == OobeUpdatePageRecord.EventName;
var key = $"{record.CxhEvent}:{record.Page ?? record.Name}";
int occurrence;
lock (_lock)
{
if (isEventName && _pageEvents >= MaxPageEventsPerRun)
{
if (!_pageCapLogged)
{
_pageCapLogged = true;
_logger.Info($"OOBE update page telemetry: {MaxPageEventsPerRun} event names reached, the rest of this run stays in agent.log");
}
return false;
}
_pageEventsPerKey.TryGetValue(key, out occurrence);
if (occurrence >= MaxPageEventsPerKey) return false;
occurrence++;
_pageEventsPerKey[key] = occurrence;
if (isEventName) _pageEvents++;
}
var data = new Dictionary<string, object>
{
{ "cxhEvent", record.CxhEvent },
{ "windowsEventId", record.WindowsEventId },
{ "eventTime", occurredAtUtc.ToString("o") },
{ "backfill", isBackfill },
{ "occurrence", occurrence },
};
if (record.Page != null) data["page"] = record.Page;
if (record.Result != null) data["result"] = record.Result;
if (record.Name != null) data["name"] = record.Name;
if (record.Value != null) data["value"] = record.Value;
try
{
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
Timestamp = occurredAtUtc,
EventType = Constants.EventTypes.OobeUpdatePage,
Severity = EventSeverity.Debug,
Source = Source,
Phase = EnrollmentPhase.Unknown,
Message = DescribePage(record),
Data = data,
ImmediateUpload = false,
});
return true;
}
catch (Exception ex)
{
_logger.Debug($"OOBE update page telemetry failed: {ex.Message}");
return false;
}
}
/// <summary>
/// Reads the OOBE update registry state, logs it and reports it as <c>oobe_update_state</c>
/// — only when at least one key holds values and the content changed since the last report
/// of this run, at most <see cref="MaxStateEventsPerRun"/> times. Fail-soft.
/// </summary>
public void ReportState(string moment, string page = null)
{
var label = page == null ? moment : $"{moment} {page}";
IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState> states;
try
{
states = _readRegistry() ?? Array.Empty<OobeUpdateRegistrySnapshot.KeyState>();
}
catch (Exception ex)
{
_logger.Debug($"OOBE update registry ({label}): snapshot failed: {ex.Message}");
return;
}
OobeUpdateRegistrySnapshot.Log(_logger, label, states);
var withValues = states.Where(s => s.HasValues).ToList();
if (withValues.Count == 0) return;
var content = string.Join("\n", withValues.Select(s => $"{s.Alias}={s.Line}"));
lock (_lock)
{
if (string.Equals(content, _lastStateContent, StringComparison.Ordinal)) return;
if (_stateEvents >= MaxStateEventsPerRun) return;
_lastStateContent = content;
_stateEvents++;
}
var data = new Dictionary<string, object>
{
{ "moment", moment },
{ "keysWithValues", withValues.Count },
};
if (page != null) data["page"] = page;
foreach (var state in withValues)
{
data[state.Alias] = state.Line.Length > MaxStateLineLength
? state.Line.Substring(0, MaxStateLineLength) + "…"
: state.Line;
}
try
{
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
EventType = Constants.EventTypes.OobeUpdateState,
Severity = EventSeverity.Debug,
Source = Source,
Phase = EnrollmentPhase.Unknown,
Message = $"OOBE update registry state ({label}): {withValues.Count} key(s) with values",
Data = data,
ImmediateUpload = false,
});
}
catch (Exception ex)
{
_logger.Debug($"OOBE update state telemetry failed: {ex.Message}");
}
}
private static string DescribePage(OobeUpdatePageRecord record)
{
switch (record.CxhEvent)
{
case OobeUpdatePageRecord.PageStarted:
return $"OOBE update page started: {record.Page}";
case OobeUpdatePageRecord.PageStopped:
return $"OOBE update page stopped: {record.Page} (result: {record.Result})";
default:
return record.Value == null
? $"OOBE update page event: {record.Name}"
: $"OOBE update page event: {record.Name} = {record.Value}";
}
}
}
}
@@ -0,0 +1,373 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.Linq;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches the CBS servicing steps of OS updates in the <c>Setup</c> log
/// (<c>Microsoft-Windows-Servicing</c>, D-310): 1 initiating changes (current → target state),
/// 2 target state reached, 3 failed, 4 a restart is needed to finish, 6 failed and partially
/// installed. The Windows Update client logs only the start of an installation; these steps
/// show the long part of it — staging, installing, "reboot required" right before the restart.
/// <para>
/// Only packages of the OS update clients (<c>UpdateAgentLCU</c> for the cumulative update,
/// <c>WindowsUpdateAgent</c>) are emitted as <c>windows_update_servicing</c> and reported to
/// the decision engine; language packs, features on demand and DISM operations started by
/// apps are counted into the host's <see cref="UpdateActivityTally"/>.
/// </para>
/// </summary>
internal sealed class ServicingTracker : IDisposable
{
internal const string Channel = "Setup";
internal const string ProviderName = "Microsoft-Windows-Servicing";
internal const int EventId_InitiatingChanges = 1;
internal const int EventId_StateReached = 2;
internal const int EventId_ChangeFailed = 3;
internal const int EventId_RebootRequired = 4;
internal const int EventId_PartiallyInstalled = 6;
internal static readonly int[] TargetedEventIds =
{
EventId_InitiatingChanges, EventId_StateReached, EventId_ChangeFailed, EventId_RebootRequired, EventId_PartiallyInstalled,
};
internal const string WatermarkStateFileName = "servicing-watermark.json";
// CBS client ids of the Windows Update installers (seen on 24H2/25H2: UpdateAgentLCU).
private static readonly HashSet<string> OsUpdateClients = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
"UpdateAgentLCU",
"WindowsUpdateAgent",
};
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly UpdateActivityTally _tally;
private readonly Action<string, string, DateTime> _onOsUpdateActivity;
private readonly EventRecordWatermark _watermark;
private EventLogWatcher _watcher;
public ServicingTracker(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger,
bool backfillEnabled = true,
int backfillLookbackMinutes = 60,
string stateDirectory = null,
UpdateActivityTally tally = null,
Action<string, string, DateTime> onOsUpdateActivity = null)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_tally = tally;
_onOsUpdateActivity = onOsUpdateActivity;
_watermark = new EventRecordWatermark(stateDirectory, WatermarkStateFileName, logger, "Servicing");
}
public void Start()
{
_watermark.Load();
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
BackfillRecentEvents();
else
_logger.Info("Servicing backfill disabled by config");
}
public void Stop()
{
if (_watcher == null) return;
try
{
_watcher.Enabled = false;
_watcher.Dispose();
_logger.Info("Servicing watcher stopped");
}
catch (Exception ex)
{
_logger.Error("Error stopping Servicing watcher", ex);
}
finally
{
_watcher = null;
}
}
public void Dispose() => Stop();
internal void LoadWatermark() => _watermark.Load();
private static HashSet<int> TargetedSet => new HashSet<int>(TargetedEventIds);
private void StartWatcher()
{
try
{
var query = new EventLogQuery(Channel, PathType.LogName,
WindowsUpdateTracker.BuildXPath(TargetedSet, ProviderName));
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
_watcher.Enabled = true;
_logger.Info($"Servicing watcher started: {Channel} ({ProviderName}, targetedIds={string.Join(",", TargetedEventIds)})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"Servicing event log not found: {Channel}");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"Servicing watcher access denied for {Channel}: {ex.Message}");
}
catch (Exception ex)
{
_logger.Error($"Failed to start Servicing watcher for {Channel}", ex);
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
collectorName: "ServicingTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
}
}
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
{
if (e.EventRecord == null) return;
try
{
ProcessRecord(e.EventRecord, isBackfill: false);
}
catch (Exception ex)
{
_logger.Error("Error processing Servicing event", ex);
}
}
private void ProcessRecord(EventRecord record, bool isBackfill)
{
var recordId = record.RecordId ?? -1;
if (_watermark.IsAlreadyProcessed(recordId)) return;
string xml = null;
try { xml = record.ToXml(); }
catch { }
var fields = ParseUserData(xml);
fields.TryGetValue("PackageIdentifier", out var package);
fields.TryGetValue("InitialPackageStateTextized", out var initialState);
fields.TryGetValue("IntendedPackageStateTextized", out var targetState);
fields.TryGetValue("ErrorCode", out var errorCode);
fields.TryGetValue("Client", out var client);
ProcessEvent(
eventId: record.Id,
recordId: recordId,
timeCreatedUtc: record.TimeCreated?.ToUniversalTime(),
package: package,
initialState: initialState,
targetState: targetState,
errorCode: errorCode,
client: client,
isBackfill: isBackfill);
}
/// <summary>Test seam — drives the processing without a Windows-only <see cref="EventRecord"/>.</summary>
internal void ProcessEvent(
int eventId,
long recordId,
DateTime? timeCreatedUtc,
string package,
string initialState,
string targetState,
string errorCode,
string client,
bool isBackfill)
{
if (!_watermark.TryClaim(recordId)) return;
if (string.IsNullOrEmpty(client) || !OsUpdateClients.Contains(client))
{
_tally?.Increment($"servicing_other_{eventId}");
return;
}
var pkg = string.IsNullOrEmpty(package) ? "(unknown package)" : package;
var target = string.IsNullOrEmpty(targetState) ? "?" : targetState;
string step;
string osStep;
string message;
var severity = EventSeverity.Info;
string hresultHex = null;
string hresultSymbol = null;
if (WindowsUpdateTracker.TryNormalizeHResult(errorCode, out var hresultValue, out var hex) && hresultValue != 0)
{
hresultHex = hex;
hresultSymbol = WindowsUpdateTracker.ResolveHResultSymbol(hresultValue);
}
switch (eventId)
{
case EventId_InitiatingChanges:
step = "initiating";
osStep = IsState(targetState, "Installed") ? OsUpdateSteps.InstallStarted
: IsState(targetState, "Staged") ? OsUpdateSteps.StagingStarted
: null;
message = $"Servicing started for {pkg}: {(string.IsNullOrEmpty(initialState) ? "?" : initialState)} → {target}";
break;
case EventId_StateReached:
step = "state_reached";
osStep = IsState(targetState, "Installed") ? OsUpdateSteps.Installed
: IsState(targetState, "Staged") ? OsUpdateSteps.Staged
: null;
message = $"Servicing finished for {pkg}: now {target}";
break;
case EventId_RebootRequired:
step = "reboot_required";
osStep = OsUpdateSteps.RebootRequired;
message = $"Servicing needs a restart to finish {pkg} → {target}";
break;
case EventId_ChangeFailed:
case EventId_PartiallyInstalled:
step = "failed";
osStep = OsUpdateSteps.Failed;
severity = EventSeverity.Warning;
message = $"Servicing FAILED for {pkg} → {target}" +
(hresultHex != null ? $" ({hresultHex} {hresultSymbol})" : string.Empty) +
(eventId == EventId_PartiallyInstalled ? ", package is partially installed" : string.Empty);
break;
default:
return;
}
var data = new Dictionary<string, object>
{
{ "servicingEventId", eventId },
{ "step", step },
{ "package", pkg },
{ "client", client },
{ "backfilled", isBackfill },
};
if (!string.IsNullOrEmpty(initialState)) data["initialState"] = initialState;
if (!string.IsNullOrEmpty(targetState)) data["targetState"] = targetState;
if (hresultHex != null)
{
data["hresult"] = hresultHex;
data["hresultSymbol"] = hresultSymbol;
}
if (recordId >= 0) data["recordId"] = recordId;
if (timeCreatedUtc.HasValue) data["timeCreated"] = timeCreatedUtc.Value.ToString("o");
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
EventType = Constants.EventTypes.WindowsUpdateServicing,
Severity = severity,
Source = Constants.EventSources.ServicingWatcher,
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
ImmediateUpload = false,
});
if (osStep != null && _onOsUpdateActivity != null)
{
try { _onOsUpdateActivity(osStep, pkg, timeCreatedUtc ?? DateTime.UtcNow); }
catch (Exception ex) { _logger.Warning($"Servicing: OS update activity callback failed: {ex.Message}"); }
}
// Last: everything this record produced has been handed to the ingress first.
_watermark.MarkEmitted(recordId);
}
private static bool IsState(string state, string expected) =>
string.Equals(state, expected, StringComparison.OrdinalIgnoreCase);
/// <summary>
/// Child elements of the first element under <c>UserData</c>
/// (<c>CbsPackageInitiateChanges</c> / <c>CbsPackageChangeState</c>), keyed by local name,
/// case-insensitively. Empty on null/malformed XML — never throws.
/// </summary>
internal static Dictionary<string, string> ParseUserData(string xml)
{
var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrEmpty(xml)) return result;
try
{
var doc = XDocument.Parse(xml);
var userData = doc.Descendants().FirstOrDefault(e => e.Name.LocalName == "UserData");
var payload = userData?.Elements().FirstOrDefault();
if (payload == null) return result;
foreach (var el in payload.Elements())
{
if (!result.ContainsKey(el.Name.LocalName))
result[el.Name.LocalName] = el.Value;
}
}
catch
{
// Malformed / unexpected XML — best effort.
}
return result;
}
private void BackfillRecentEvents()
{
try
{
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
var xpath = WindowsUpdateTracker.BuildXPath(TargetedSet, ProviderName, lookbackMs);
_logger.Info($"Servicing backfill: scanning {Channel} (lookback={_backfillLookbackMinutes}min, restartWatermark={_watermark.RestartWatermark})");
int processed = 0;
using (var reader = new EventLogReader(new EventLogQuery(Channel, PathType.LogName, xpath)))
{
EventRecord record;
while ((record = reader.ReadEvent()) != null)
{
using (record)
{
ProcessRecord(record, isBackfill: true);
processed++;
}
}
}
if (processed > 0)
_logger.Info($"Servicing backfill: scanned {processed} event(s)");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"Servicing event log not found during backfill: {Channel}");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"Servicing backfill access denied: {ex.Message}");
}
catch (Exception ex)
{
_logger.Warning($"Servicing backfill failed: {ex.Message}");
}
}
}
}
@@ -1,7 +1,9 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.Globalization;
using System.Text.RegularExpressions;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared;
@@ -31,9 +33,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
internal const string ShellCoreEventLogChannel = "Microsoft-Windows-Shell-Core/Operational";
internal const int EventId_ShellCore_WebAppStarted = 62404;
internal const int EventId_ShellCore_WebAppStopped = 62405;
internal const int EventId_ShellCore_WebAppEventName = 62406;
internal const int EventId_ShellCore_WebAppEvent = 62407;
internal const int BackfillLookbackMinutes = 5;
// 62405 (page stopped) and 62406 (event with a name only) feed the CloudExperienceHost
// navigation breadcrumbs in agent.log and the OOBE update page telemetry (D-310), nothing else.
private const string WatchedEventIdsXPath =
"(EventID=62404 or EventID=62405 or EventID=62406 or EventID=62407)";
/// <summary>
/// Upper bound for the caller-supplied backfill lookback. Matches the agent's max
/// lifetime (360 min) — a window wider than the agent can ever live would only widen
@@ -45,22 +54,53 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private static readonly HashSet<int> TrackedShellCoreEventIds = new HashSet<int>
{
EventId_ShellCore_WebAppStarted,
EventId_ShellCore_WebAppStopped,
EventId_ShellCore_WebAppEventName,
EventId_ShellCore_WebAppEvent
};
private static readonly Regex EspExitingPattern = new Regex(
@"OOBE_ESP.*Exiting", RegexOptions.IgnoreCase | RegexOptions.Compiled);
// Manifest templates: 62404 "... started. CXID: '%1'.", 62405 "... stopped. Result: '%1'.",
// 62406 "... Event 1. Name: '%1'.", 62407 "... Event 2. Name: '%1', Value: '%2'.". The
// fields are read from the event PROPERTIES (%1/%2): the formatted message is localized
// ("Ergebnis: '…'", "Nom : « … »"), so these patterns are only the fallback for a record
// without properties and accept « » and a space before the colon. A 62407 value can carry
// URLs with tenant ids — a value leaves the tracker only in a form that cannot
// (SafeValuePattern: a boolean, an integer, an HRESULT).
private static readonly Regex CxidPattern = new Regex(
@"CXID\s*:\s*['\u00AB]\s*([^'\u00BB]*?)\s*['\u00BB]", RegexOptions.CultureInvariant | RegexOptions.Compiled, TimeSpan.FromMilliseconds(100));
private static readonly Regex ResultPattern = new Regex(
@"Result\s*:\s*['\u00AB]\s*([^'\u00BB]*?)\s*['\u00BB]", RegexOptions.CultureInvariant | RegexOptions.Compiled, TimeSpan.FromMilliseconds(100));
private static readonly Regex NamePattern = new Regex(
@"Name\s*:\s*['\u00AB]\s*([^'\u00BB]*?)\s*['\u00BB]", RegexOptions.CultureInvariant | RegexOptions.Compiled, TimeSpan.FromMilliseconds(100));
private static readonly Regex ValuePattern = new Regex(
@"Value\s*:\s*['\u00AB]\s*([^'\u00BB]*?)\s*['\u00BB]", RegexOptions.CultureInvariant | RegexOptions.Compiled, TimeSpan.FromMilliseconds(100));
private static readonly Regex SafeValuePattern = new Regex(
@"^(?:true|false|-?[0-9]{1,10}|0x[0-9a-f]{1,8})$",
RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.Compiled, TimeSpan.FromMilliseconds(100));
private const int MaxBreadcrumbTokenLength = 128;
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly HelloTracker _helloTracker;
private readonly OobeUpdateTelemetry _oobeUpdate;
// Shell-Core RecordIds of the update-page records already reported: the restart backfill
// must not report a record an earlier run already sent (D-310 field data: the same 20
// records came twice after a restart).
private readonly EventRecordWatermark _pageWatermark;
internal const string PageWatermarkStateFileName = "shellcore-page-watermark.json";
private EventLogWatcher _watcher;
private int _started;
private int _stopReported;
private bool _espExitDetected;
private bool _whiteGloveDetected;
private bool _helloWizardStartDetected;
private string _lastCxhPage; // 62405 names no page, it ends the last one started
private readonly object _stateLock = new object();
/// <summary>
@@ -99,26 +139,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string tenantId,
InformationalEventPost post,
AgentLogger logger,
HelloTracker helloTracker)
HelloTracker helloTracker,
Func<IReadOnlyList<OobeUpdateRegistrySnapshot.KeyState>> oobeUpdateRegistryReader = null,
string stateDirectory = null)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_helloTracker = helloTracker; // nullable — HelloTracker may be unavailable in some test setups
// Test seam: a fake registry reader; production reads the real registry.
_oobeUpdate = new OobeUpdateTelemetry(sessionId, tenantId, post, logger,
oobeUpdateRegistryReader ?? OobeUpdateRegistrySnapshot.Read);
_pageWatermark = new EventRecordWatermark(stateDirectory, PageWatermarkStateFileName, logger, "Shell-Core update page");
}
internal bool IsEspExitedForTest { get { lock (_stateLock) { return _espExitDetected; } } }
internal bool IsWhiteGloveDetectedForTest { get { lock (_stateLock) { return _whiteGloveDetected; } } }
/// <summary>Loads the update-page watermark as <see cref="Start"/> does, without arming the watcher.</summary>
internal void LoadPageWatermark() => _pageWatermark.Load();
public void Start()
{
// The OOBE update registry state at every agent start — so also right after the
// update's restart (D-310).
if (Interlocked.Exchange(ref _started, 1) == 0)
{
_oobeUpdate.ReportState(OobeUpdateTelemetry.Moments.AgentStart);
_pageWatermark.Load();
}
try
{
var query = new EventLogQuery(
ShellCoreEventLogChannel,
PathType.LogName,
"*[System[(EventID=62404 or EventID=62407)]]");
$"*[System[{WatchedEventIdsXPath}]]");
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
@@ -143,6 +200,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
public void Stop()
{
// Stop runs again from Dispose — one snapshot, and only for a tracker that started.
if (Volatile.Read(ref _started) == 1 && Interlocked.Exchange(ref _stopReported, 1) == 0)
{
_oobeUpdate.ReportState(OobeUpdateTelemetry.Moments.AgentStop);
}
if (_watcher == null) return;
try
{
@@ -172,7 +235,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
var description = record.FormatDescription() ?? $"Event ID {record.Id}";
var timestamp = (record.TimeCreated ?? DateTime.UtcNow).ToUniversalTime();
ProcessEvent(record.Id, description, timestamp, record.ProviderName ?? "", isBackfill: false);
ProcessEvent(record.Id, description, timestamp, record.ProviderName ?? "", isBackfill: false,
recordId: record.RecordId ?? -1, properties: ReadProperties(record));
}
catch (Exception ex)
{
@@ -184,8 +248,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// Core event-processing logic. Exposed as internal so tests can drive it without
/// needing to synthesize an <see cref="EventRecord"/> (abstract + Windows-only).
/// </summary>
internal void ProcessEvent(int eventId, string description, DateTime timestamp, string providerName, bool isBackfill)
internal void ProcessEvent(
int eventId,
string description,
DateTime timestamp,
string providerName,
bool isBackfill,
long recordId = -1,
IReadOnlyList<string> properties = null)
{
var fields = ReadCxhFields(eventId, description, properties);
ObserveCxhRecord(eventId, fields, timestamp, isBackfill, recordId);
if (eventId == EventId_ShellCore_WebAppStopped || eventId == EventId_ShellCore_WebAppEventName) return;
string eventType;
EventSeverity severity = EventSeverity.Info;
string message;
@@ -197,7 +272,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
switch (eventId)
{
case EventId_ShellCore_WebAppStarted: // 62404
if (description.Contains("AADHello") || description.Contains("'NGC'"))
if (IsHelloWizardStart(fields.Primary, description))
{
eventType = Constants.EventTypes.HelloWizardStarted;
message = "Windows Hello wizard started (CloudExperienceHost)";
@@ -388,9 +463,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
var query = new EventLogQuery(
ShellCoreEventLogChannel,
PathType.LogName,
$"*[System[(EventID=62404 or EventID=62407) and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]");
$"*[System[{WatchedEventIdsXPath} and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]");
var records = new List<(int Id, string Description, DateTime OccurredAtUtc)>();
var records = new List<ShellCoreRecord>();
using (var reader = new EventLogReader(query))
{
for (EventRecord record = reader.ReadEvent(); record != null; record = reader.ReadEvent())
@@ -402,7 +477,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
// (EspAndHelloTrackerAdapter) can stamp signals with the source time
// rather than collapsing to wall-clock-now.
var timestamp = (record.TimeCreated ?? DateTime.UtcNow).ToUniversalTime();
records.Add((record.Id, description, timestamp));
records.Add(new ShellCoreRecord(record.Id, description, timestamp,
record.RecordId ?? -1, ReadProperties(record)));
}
}
}
@@ -455,6 +531,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// </summary>
internal void ReplayBackfillRecords(
IReadOnlyList<(int Id, string Description, DateTime OccurredAtUtc)> records)
{
if (records == null) return;
var converted = new List<ShellCoreRecord>(records.Count);
foreach (var r in records) converted.Add(new ShellCoreRecord(r.Id, r.Description, r.OccurredAtUtc));
ReplayBackfillRecords(converted);
}
internal void ReplayBackfillRecords(IReadOnlyList<ShellCoreRecord> records)
{
if (records == null || records.Count == 0) return;
@@ -466,13 +550,18 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
foreach (var record in records)
{
var description = record.Description ?? string.Empty;
var fields = ReadCxhFields(record.Id, description, record.Properties);
ObserveCxhRecord(record.Id, fields, record.OccurredAtUtc, isBackfill: true, recordId: record.RecordId);
if (record.Id == EventId_ShellCore_WebAppStarted)
{
HandleBackfillRecord(record.Id, description, record.OccurredAtUtc);
HandleBackfillRecord(record.Id, description, record.OccurredAtUtc, fields.Primary);
continue;
}
// 62405/62406 only feed the breadcrumbs and the update page telemetry above.
if (record.Id != EventId_ShellCore_WebAppEvent) continue;
var isFailure = HasEspFailurePattern(description);
var isExit = !isFailure && EspExitingPattern.IsMatch(description);
if (!isFailure && !isExit) continue;
@@ -535,7 +624,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// (<c>record.TimeCreated</c>); subscribers read it via
/// <see cref="LastEventOccurredAtUtc"/> during their synchronous event handler.
/// </summary>
internal void HandleBackfillRecord(int eventId, string description, DateTime occurredAtUtc)
internal void HandleBackfillRecord(int eventId, string description, DateTime occurredAtUtc, string cxid = null)
{
if (eventId == EventId_ShellCore_WebAppStarted)
{
@@ -543,7 +632,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
// are unrelated. Fire-once so a replayed log tail cannot re-raise (downstream is
// idempotent anyway: HelloTracker once-guard, adapter dedup flag, engine
// set-once fact — this guard just keeps the noise down).
if (!description.Contains("AADHello") && !description.Contains("'NGC'")) return;
if (!IsHelloWizardStart(cxid ?? ReadCxhFields(eventId, description, null).Primary, description)) return;
bool shouldRaiseWizard;
lock (_stateLock)
@@ -569,6 +658,242 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
// =====================================================================
// CloudExperienceHost navigation: breadcrumbs and OOBE update page telemetry (D-310)
// =====================================================================
/// <summary>
/// Writes one agent.log line per CloudExperienceHost page start/stop and per update-page
/// event name, so a diagnostics package shows the OOBE navigation. Records of the update
/// page itself (<c>OobeNDUP</c>, <c>RebootNDUP</c> and its <c>ExpeditedUpdate_*</c> names)
/// are also reported as <c>oobe_update_page</c> — once across agent runs, by RecordId — and
/// at the page's live start and stop the registry state behind it as
/// <c>oobe_update_state</c>. No decision.
/// </summary>
private void ObserveCxhRecord(int eventId, CxhFields fields, DateTime occurredAtUtc, bool isBackfill, long recordId)
{
try
{
string page;
lock (_stateLock)
{
if (eventId == EventId_ShellCore_WebAppStarted && fields.Primary != null)
_lastCxhPage = fields.Primary;
page = _lastCxhPage;
}
var crumb = FormatCxhBreadcrumb(eventId, fields, page);
if (crumb == null) return;
_logger.Info(isBackfill ? $"{crumb} (backfill, at {occurredAtUtc:o})" : crumb);
var record = ParseUpdatePageRecord(eventId, fields, page);
if (record == null) return;
// A record an earlier run (or this one) already reported — the restart backfill
// re-reads the minutes before the restart.
if (!_pageWatermark.TryClaim(recordId)) return;
// Only a reported record moves the state file — at most the run budget plus the
// page boundaries; one over the budget stays claimed for this run only.
if (_oobeUpdate.ReportPage(record, occurredAtUtc, isBackfill))
_pageWatermark.MarkEmitted(recordId);
// The registry is read now — only a live page boundary is the page's moment.
if (!isBackfill && record.CxhEvent != OobeUpdatePageRecord.EventName)
{
_oobeUpdate.ReportState(
record.CxhEvent == OobeUpdatePageRecord.PageStarted
? OobeUpdateTelemetry.Moments.UpdatePageStarted
: OobeUpdateTelemetry.Moments.UpdatePageStopped,
record.Page);
}
}
catch (Exception ex)
{
_logger.Debug($"ShellCoreTracker: CXH record observation failed: {ex.Message}");
}
}
/// <summary>The own fields of a CloudExperienceHost record: CXID, result or name (%1) and the 62407 value (%2).</summary>
internal readonly struct CxhFields
{
public CxhFields(string primary, string value)
{
Primary = primary;
Value = value;
}
/// <summary>62404: CXID; 62405: result; 62406/62407: event name. Null when not readable.</summary>
public string Primary { get; }
/// <summary>62407: the raw value (never logged or sent unless it is of a safe form).</summary>
public string Value { get; }
}
/// <summary>
/// The record's fields from its event properties, which carry them verbatim in every
/// display language. Without properties (a test, a record that failed to render them) the
/// formatted message is parsed — English punctuation and « » only, so a localized label
/// ("Ergebnis", "Nom") yields null there.
/// </summary>
internal static CxhFields ReadCxhFields(int eventId, string description, IReadOnlyList<string> properties)
{
if (properties != null && properties.Count > 0)
{
return new CxhFields(
Bound(properties[0]),
eventId == EventId_ShellCore_WebAppEvent && properties.Count > 1 ? Bound(properties[1]) : null);
}
switch (eventId)
{
case EventId_ShellCore_WebAppStarted:
return new CxhFields(ExtractToken(CxidPattern, description), null);
case EventId_ShellCore_WebAppStopped:
return new CxhFields(ExtractToken(ResultPattern, description), null);
case EventId_ShellCore_WebAppEventName:
return new CxhFields(ExtractToken(NamePattern, description), null);
case EventId_ShellCore_WebAppEvent:
return new CxhFields(ExtractToken(NamePattern, description), ExtractToken(ValuePattern, description));
default:
return default;
}
}
/// <summary>
/// The breadcrumb for one Shell-Core record, or null when it is not one: a page start
/// (62404, its CXID), a page stop (62405, its result, attributed to the last started
/// page), or an update-page event name (62406/62407 — a 62407 value only in a safe form).
/// </summary>
internal static string FormatCxhBreadcrumb(int eventId, string description, string lastPage) =>
FormatCxhBreadcrumb(eventId, ReadCxhFields(eventId, description, null), lastPage);
internal static string FormatCxhBreadcrumb(int eventId, CxhFields fields, string lastPage)
{
switch (eventId)
{
case EventId_ShellCore_WebAppStarted:
return fields.Primary == null ? null : $"CXH page started: {fields.Primary}";
case EventId_ShellCore_WebAppStopped:
return $"CXH page stopped: {lastPage ?? "?"} (result={fields.Primary ?? "?"})";
case EventId_ShellCore_WebAppEventName:
case EventId_ShellCore_WebAppEvent:
{
if (!IsUpdatePageToken(fields.Primary)) return null;
var value = eventId == EventId_ShellCore_WebAppEvent ? SafeValue(fields.Value) : null;
return value == null ? $"CXH event: {fields.Primary}" : $"CXH event: {fields.Primary} (value={value})";
}
default:
return null;
}
}
/// <summary>
/// The record of the OOBE update page, or null for any other CloudExperienceHost record:
/// the start of a page whose CXID names NDUP, the stop of such a page (62405, attributed
/// to the last started page), or an update-page event name from 62406/62407.
/// </summary>
internal static OobeUpdatePageRecord ParseUpdatePageRecord(int eventId, string description, string lastPage) =>
ParseUpdatePageRecord(eventId, ReadCxhFields(eventId, description, null), lastPage);
internal static OobeUpdatePageRecord ParseUpdatePageRecord(int eventId, CxhFields fields, string lastPage)
{
switch (eventId)
{
case EventId_ShellCore_WebAppStarted:
return IsUpdatePageToken(fields.Primary)
? new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStarted, eventId, page: fields.Primary)
: null;
case EventId_ShellCore_WebAppStopped:
return IsUpdatePageToken(lastPage)
? new OobeUpdatePageRecord(OobeUpdatePageRecord.PageStopped, eventId,
page: lastPage, result: fields.Primary ?? "?")
: null;
case EventId_ShellCore_WebAppEventName:
case EventId_ShellCore_WebAppEvent:
{
if (!IsUpdatePageToken(fields.Primary)) return null;
var value = eventId == EventId_ShellCore_WebAppEvent ? SafeValue(fields.Value) : null;
return new OobeUpdatePageRecord(OobeUpdatePageRecord.EventName, eventId, name: fields.Primary, value: value);
}
default:
return null;
}
}
internal static bool IsUpdatePageToken(string token) =>
token != null
&& (token.IndexOf("NDUP", StringComparison.OrdinalIgnoreCase) >= 0
|| token.IndexOf("ExpeditedUpdate", StringComparison.OrdinalIgnoreCase) >= 0);
/// <summary>
/// The Hello wizard's CloudExperienceHost start: CXID <c>AADHello</c> or <c>NGC</c>. With
/// no readable CXID the formatted message decides, as before (an English or German
/// message quotes the CXID as <c>'NGC'</c>).
/// </summary>
internal static bool IsHelloWizardStart(string cxid, string description) =>
cxid != null
? cxid.Contains("AADHello") || string.Equals(cxid, "NGC", StringComparison.Ordinal)
: description != null && (description.Contains("AADHello") || description.Contains("'NGC'"));
/// <summary>
/// The 62407 value when it is a boolean, an integer of up to ten digits or an HRESULT —
/// forms that cannot carry a URL or a tenant id (the CSP gate, error codes). Null for
/// anything else.
/// </summary>
internal static string ExtractSafeValue(string description) =>
SafeValue(ReadCxhFields(EventId_ShellCore_WebAppEvent, description, null).Value);
internal static string SafeValue(string value)
{
if (string.IsNullOrEmpty(value)) return null;
try
{
return SafeValuePattern.IsMatch(value) ? value : null;
}
catch (RegexMatchTimeoutException)
{
return null;
}
}
private static string ExtractToken(Regex pattern, string description)
{
if (string.IsNullOrEmpty(description)) return null;
try
{
var match = pattern.Match(description);
return match.Success ? Bound(match.Groups[1].Value) : null;
}
catch (RegexMatchTimeoutException)
{
return null;
}
}
private static string Bound(string token)
{
if (string.IsNullOrEmpty(token)) return null;
return token.Length > MaxBreadcrumbTokenLength ? token.Substring(0, MaxBreadcrumbTokenLength) + "…" : token;
}
/// <summary>The event's property values as strings (the manifest's %1, %2, …); null when unreadable.</summary>
private static IReadOnlyList<string> ReadProperties(EventRecord record)
{
try
{
var properties = record.Properties;
if (properties == null || properties.Count == 0) return null;
var values = new string[properties.Count];
for (var i = 0; i < properties.Count; i++)
values[i] = Convert.ToString(properties[i]?.Value, CultureInfo.InvariantCulture);
return values;
}
catch (Exception)
{
return null;
}
}
// =====================================================================
// Pattern helpers
// =====================================================================
@@ -608,4 +933,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
return "Unknown_ESP_Failure";
}
}
/// <summary>One Shell-Core record read by the restart backfill.</summary>
internal readonly struct ShellCoreRecord
{
public ShellCoreRecord(int id, string description, DateTime occurredAtUtc, long recordId = -1, IReadOnlyList<string> properties = null)
{
Id = id;
Description = description;
OccurredAtUtc = occurredAtUtc;
RecordId = recordId;
Properties = properties;
}
public int Id { get; }
public string Description { get; }
public DateTime OccurredAtUtc { get; }
/// <summary>The channel's RecordId; -1 when unknown (no cross-run dedup).</summary>
public long RecordId { get; }
/// <summary>The event's property values (%1, %2, …); null when unknown.</summary>
public IReadOnlyList<string> Properties { get; }
}
}
@@ -39,7 +39,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// - Trace-indexed probes: emit a <c>stall_probe_check</c> Trace event on quiet/activeInstalls.
/// - Non-trace probes: emit nothing when no anomaly is found (agent log only).
/// - Anomaly found: always emit <c>stall_probe_result</c> Warning (plus EspFailureDetected if terminal).
/// - Probe index configured as "session-stalled": emit fire-once <c>session_stalled</c> Warning.
/// - Probe index configured as "session-stalled": emit fire-once <c>session_stalled</c> Warning
/// (held back while a tenant-extended Hello wait is pending, emitted once it is over).
/// </summary>
public class StallProbeCollector
{
@@ -54,10 +55,20 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
private readonly HashSet<int> _harmlessModernDeploymentEventIds;
private readonly bool _isDevicePreparation;
// True while a tenant-extended Hello wait (up to 60 min) is pending in the decision
// engine. It is a bounded wait that resolves the session either way — not a stall — so
// session_stalled is held back until it is over.
private readonly Func<bool> _helloWaitPending;
private readonly object _stateLock = new object();
private readonly HashSet<int> _firedProbeIndices = new HashSet<int>();
private bool _sessionStalledFired;
// The session_stalled report held back while the Hello wait was pending (the probe result
// of that idle window). Emitted on a later tick once the wait is over and the session is
// still idle; dropped by real activity (ResetProbes).
private ProbeResult _deferredSessionStalled;
// P2 — monotonic generation, bumped by ResetProbes. RunProbe runs OUTSIDE _stateLock; if a
// real-activity reset lands while a probe is in flight, the post-probe "mark fired" must be
// dropped (its generation is stale) so the next idle period can fire that probe again.
@@ -153,9 +164,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string[] sources,
int sessionStalledAfterProbeIndex,
int[] harmlessModernDeploymentEventIds = null,
bool isDevicePreparation = false)
bool isDevicePreparation = false,
Func<bool> helloWaitPending = null)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_helloWaitPending = helloWaitPending;
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
@@ -192,6 +205,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
// Always bump the generation, even when no probes are currently marked fired — a
// probe may be running right now (outside the lock) and must not re-mark itself.
_probeGeneration++;
_deferredSessionStalled = null;
if (_firedProbeIndices.Count > 0)
{
_logger.Trace($"StallProbeCollector: resetting {_firedProbeIndices.Count} fired probes");
@@ -206,6 +220,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// </summary>
public void CheckAndRunProbes(double idleMinutes)
{
EmitDeferredSessionStalledIfDue(idleMinutes);
// Iterate thresholds from lowest to highest so probes fire in order.
for (int i = 0; i < _thresholdsMinutes.Length; i++)
{
@@ -225,7 +241,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
if (alreadyFired)
continue;
RunProbe(probeIndex, thresholdMinutes, idleMinutes);
RunProbe(probeIndex, thresholdMinutes, idleMinutes, genAtStart);
lock (_stateLock)
{
@@ -237,7 +253,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
}
private void RunProbe(int probeIndex, int thresholdMinutes, double idleMinutes)
private void RunProbe(int probeIndex, int thresholdMinutes, double idleMinutes, int generation)
{
var started = DateTime.UtcNow;
var result = new ProbeResult { ProbeIndex = probeIndex, IdleMinutes = Math.Round(idleMinutes, 1) };
@@ -293,8 +309,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
// session_stalled is fire-once across the whole session. It fires at the configured
// probe index UNLESS activeInstalls is observed (activeInstalls counts as progress).
// While a tenant Hello wait is pending it is held back, not dropped.
if (probeIndex == _sessionStalledAfterProbeIndex && !hasActiveInstalls)
{
if (IsHelloWaitPending())
{
lock (_stateLock)
{
// P2: a reset that landed while this probe ran ended the idle window.
if (!_sessionStalledFired && _probeGeneration == generation)
_deferredSessionStalled = result;
}
_logger.Info($"StallProbeCollector: session_stalled held back at probe {probeIndex} — a tenant Hello wait is still pending");
return;
}
lock (_stateLock)
{
if (_sessionStalledFired)
@@ -305,6 +334,52 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
}
/// <summary>
/// Emits the held-back session_stalled report once the Hello wait is over and the session
/// is still idle (no ResetProbes since), with the idle time of this tick.
/// </summary>
private void EmitDeferredSessionStalledIfDue(double idleMinutes)
{
ProbeResult deferred;
lock (_stateLock)
{
deferred = _deferredSessionStalled;
if (deferred == null || _sessionStalledFired)
return;
}
if (IsHelloWaitPending())
return;
lock (_stateLock)
{
if (_sessionStalledFired || !ReferenceEquals(_deferredSessionStalled, deferred))
return;
_sessionStalledFired = true;
_deferredSessionStalled = null;
}
deferred.IdleMinutes = Math.Round(idleMinutes, 1);
_logger.Info($"StallProbeCollector: Hello wait over, still idle at {deferred.IdleMinutes}min — emitting the held-back session_stalled");
EmitSessionStalledEvent(deferred);
}
private bool IsHelloWaitPending()
{
if (_helloWaitPending == null)
return false;
try
{
return _helloWaitPending();
}
catch (Exception ex)
{
// A failing probe must never cost the stall report — treat as not pending.
_logger.Warning($"StallProbeCollector: Hello-wait probe threw, treating as not pending: {ex.Message}");
return false;
}
}
private void RunWithTimeout(Action<ProbeResult> scan, int timeoutMs, string sourceName, ProbeResult result)
{
// Enforce a real hard timeout: run the source on a dedicated background thread and
@@ -340,7 +340,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
EventType = Constants.EventTypes.SystemClockChanged,
Severity = isLargeStep ? EventSeverity.Warning : EventSeverity.Info,
Source = "SystemTimelineWatcher",
Source = Constants.EventSources.SystemTimelineWatcher,
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
@@ -467,7 +467,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
EventType = Constants.EventTypes.SystemSleepEpisode,
Severity = EventSeverity.Info,
Source = "SystemTimelineWatcher",
Source = Constants.EventSources.SystemTimelineWatcher,
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
@@ -0,0 +1,42 @@
using System;
using System.Collections.Generic;
using System.Linq;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Thread-safe counters for update activity that is counted instead of emitted (D-310): Store,
/// Defender and other non-OS update events, repeated scan results, servicing steps of non-OS
/// packages. Shared by the update watchers of one host, which reports the totals once as
/// <c>windows_update_activity_summary</c> when it stops.
/// </summary>
internal sealed class UpdateActivityTally
{
private readonly object _lock = new object();
private readonly Dictionary<string, int> _counts = new Dictionary<string, int>(StringComparer.Ordinal);
public void Increment(string key)
{
if (string.IsNullOrEmpty(key)) return;
lock (_lock)
{
_counts.TryGetValue(key, out var current);
_counts[key] = current + 1;
}
}
public bool IsEmpty
{
get { lock (_lock) return _counts.Count == 0; }
}
/// <summary>Counts ordered by key, for a stable event payload.</summary>
public IReadOnlyList<KeyValuePair<string, int>> Snapshot()
{
lock (_lock)
{
return _counts.OrderBy(kv => kv.Key, StringComparer.Ordinal).ToList();
}
}
}
}
@@ -0,0 +1,65 @@
using System;
using System.Text.RegularExpressions;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Sorts a Windows Update client event into the update class that decides how it is reported
/// (D-310). Only <see cref="Os"/> updates — Windows quality/cumulative/servicing-stack/feature
/// updates, .NET Framework and the Malicious Software Removal Tool — are enrollment-relevant and
/// emitted one by one. Defender intelligence and Store app updates arrive every few minutes on a
/// fresh device (up to 171 downloads per hour in the field census) and are only counted.
/// </summary>
internal static class WindowsUpdateClassifier
{
public const string Os = "os";
public const string Defender = "defender";
public const string Store = "store";
public const string Other = "other";
/// <summary>The Microsoft Store service; WU events carry it as <c>serviceGuid</c>.</summary>
internal const string StoreServiceGuid = "855e8a7c-ecb4-4ca3-b045-1dfa50104289";
// Store packages are titled "<12-char product id>-<Publisher.App>", e.g. "9NBLGGH4NNS1-Microsoft.DesktopAppInstaller";
// bundles add an "ApplicationSet-" prefix.
private static readonly Regex StoreProductTitle =
new Regex(@"^(ApplicationSet-)?[0-9A-Z]{12}-\S", RegexOptions.CultureInvariant, TimeSpan.FromMilliseconds(100));
private static readonly Regex DefenderTitle =
new Regex(@"Microsoft Defender|Windows Defender", RegexOptions.IgnoreCase | RegexOptions.CultureInvariant, TimeSpan.FromMilliseconds(100));
// "Windows" plus "Update"/"Upgrade" covers cumulative, quality, security, servicing-stack,
// dynamic and feature updates in both the GA and the Insider title forms.
private static readonly Regex WindowsUpdateTitle =
new Regex(@"\bWindows\b.*\b(Update|Upgrade)\b|\b(Update|Upgrade)\b.*\bWindows\b",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant, TimeSpan.FromMilliseconds(100));
private static readonly Regex OsTitleExtras =
new Regex(@"\.NET Framework|Servicing Stack|Malicious Software Removal Tool",
RegexOptions.IgnoreCase | RegexOptions.CultureInvariant, TimeSpan.FromMilliseconds(100));
public static string Classify(string updateTitle, string serviceGuid)
{
if (!string.IsNullOrEmpty(serviceGuid)
&& serviceGuid.Trim('{', '}', ' ').Equals(StoreServiceGuid, StringComparison.OrdinalIgnoreCase))
{
return Store;
}
if (string.IsNullOrWhiteSpace(updateTitle)) return Other;
try
{
if (StoreProductTitle.IsMatch(updateTitle)) return Store;
if (DefenderTitle.IsMatch(updateTitle)) return Defender;
if (OsTitleExtras.IsMatch(updateTitle) || WindowsUpdateTitle.IsMatch(updateTitle)) return Os;
}
catch (RegexMatchTimeoutException)
{
// A pathological title must never cost the event — it is reported as "other".
}
return Other;
}
}
}
@@ -2,100 +2,106 @@ using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Threading;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Services;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches <c>Microsoft-Windows-WindowsUpdateClient/Operational</c> for quality/cumulative
/// update install activity that happens DURING Autopilot OOBE / ESP and forwards it as
/// structured <c>windows_update_succeeded</c> / <c>windows_update_failed</c> /
/// <c>windows_update_started</c> events.
/// Watches ONE channel of the <c>Microsoft-Windows-WindowsUpdateClient</c> provider for update
/// activity during OOBE / ESP and forwards it as <c>windows_update_started</c> /
/// <c>windows_update_succeeded</c> / <c>windows_update_failed</c> events.
/// <para>
/// This is a blind spot no other tool surfaces well: a cumulative update installing mid-OOBE
/// can break the enrollment (r/Intune KB5095189), and the ESP "Install Windows quality updates
/// during OOBE" feature makes it increasingly common. The Intune console shows none of it — it
/// is otherwise deep manual digging on the device.
/// <b>Two channels (D-310).</b> The provider manifest writes the install events — 19 success,
/// 20 failure (HRESULT), 43 install started, 44 download started — to the <b>System</b> log and
/// the scan/download events — 25 scan failed, 26 scan found N, 31 download failed, 41
/// downloaded — to the Operational channel. The host runs one instance per channel; RecordIds
/// are per channel, so each instance keeps its own watermark. The System log is shared with
/// every other provider, hence the provider filter in its XPath.
/// </para>
/// <para>
/// <b>Backfill.</b> OOBE updates run at the early "Getting updates" screen, frequently BEFORE the
/// agent starts. The WindowsUpdateClient/Operational .evtx persists across the mid-OOBE reboot,
/// so a startup backfill with a generous lookback catches those pre-agent events.
/// <b>Classification.</b> Only Windows/.NET updates (<see cref="WindowsUpdateClassifier.Os"/>)
/// are emitted one by one and reported to the decision engine. Store, Defender and other
/// updates arrive every few minutes on a fresh device and are only counted into the host's
/// <see cref="UpdateActivityTally"/>. Repeated scan results (26) and scan failures (25) are
/// deduplicated; their repeats are counted the same way.
/// </para>
/// <para>
/// <b>Cross-restart dedup.</b> Unlike the single-latch WhiteGlove backfill, WU events are many
/// distinct records, so an agent restart would re-read (and re-emit) events already reported in
/// the prior run. We persist the highest processed <see cref="EventRecord.RecordId"/> as a
/// watermark and skip records at/below it. Live records always carry a higher RecordId and pass.
/// </para>
/// <para>
/// <b>Verified EventIDs (build against these):</b> 19=install success, 20=install failure
/// (carries HRESULT in <c>errorCode</c>), 43=install started, 44=download started. IDs whose
/// semantics are unverified across sources (25/26/31, restart-required 21 vs 41) are excluded by
/// default and can be added via <c>CollectorConfiguration.WindowsUpdateTargetedEventIds</c>
/// after validation on a real device trace — no redeploy needed.
/// <b>Backfill.</b> Updates can start before the agent does; the .evtx survives the reboot, so
/// a startup backfill with a generous lookback catches them. Backfilled events keep their own
/// event time on the timeline.
/// </para>
/// </summary>
internal sealed class WindowsUpdateTracker : IDisposable
{
internal const string Channel = "Microsoft-Windows-WindowsUpdateClient/Operational";
internal const string SystemChannel = "System";
internal const string ProviderName = "Microsoft-Windows-WindowsUpdateClient";
internal const string OrchestratorChannel = "Microsoft-Windows-UpdateOrchestrator/Operational";
internal const int EventId_InstallSuccess = 19;
internal const int EventId_InstallFailure = 20;
// System channel (provider manifest).
internal const int EventId_InstallSuccess = 19;
internal const int EventId_InstallFailure = 20;
internal const int EventId_InstallStarted = 43;
internal const int EventId_DownloadStarted = 44;
// Operational channel.
internal const int EventId_ScanFailed = 25;
internal const int EventId_ScanFound = 26;
internal const int EventId_DownloadFailed = 31;
internal const int EventId_Downloaded = 41;
internal const string WatermarkStateFileName = "windows-update-watermark.json";
internal const string SystemWatermarkStateFileName = "windows-update-system-watermark.json";
// Hard bound for the unfiltered census scan — during OOBE both channels carry at most a
// few hundred records in the lookback window; the cap only guards against a pathological
// log. A truncated census says so in its payload (no silent caps).
internal const int CensusRecordCap = 5000;
// Scan results repeat every few minutes; only changes are worth a timeline row.
internal const int MaxScanFoundEmissions = 10;
internal const int MaxScanFailedEmissions = 5;
private static readonly int[] DefaultSystemEventIds = { EventId_InstallSuccess, EventId_InstallFailure, EventId_InstallStarted, EventId_DownloadStarted };
private static readonly int[] DefaultOperationalEventIds = { EventId_ScanFailed, EventId_ScanFound, EventId_DownloadFailed, EventId_Downloaded };
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly string _channel;
private readonly bool _isSystemChannel;
private readonly HashSet<int> _targetedEventIds;
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly bool _channelCensusEnabled;
private readonly Func<bool> _osBuildChangedProvider;
private readonly UpdateActivityTally _tally;
private readonly Action<string, string, DateTime> _onOsUpdateActivity;
private readonly EventRecordWatermark _watermark;
// Count of targeted WU events emitted THIS run (live + backfill). Zero after the backfill
// while the OS build provably changed = the watcher is blind to the update's channel/IDs.
// Targeted events processed THIS run (any class). Zero after the backfill while the OS
// build provably changed = the watcher is blind to the update's channel/IDs.
private int _emittedThisRun;
private readonly object _throttleLock = new object();
private readonly HashSet<string> _scanFailureCodesReported = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
private string _lastScanFoundCount;
private int _scanFoundEmitted;
private EventLogWatcher _watcher;
// Dedup. Guarded because EventRecordWritten callbacks run concurrently with the backfill scan.
//
// Cross-restart boundary loaded from disk: the highest RecordId emitted in a PRIOR run.
// Immutable during this run — events at/below it were already emitted before this process
// started and must not be re-emitted by the backfill scan.
private long _restartWatermark = -1;
// RecordIds emitted in THIS run — the intra-run dedup set. Deliberately a HashSet, NOT a
// high-water mark: the live watcher is armed before the backfill runs, so a live event with a
// HIGHER RecordId can be processed before/during backfill. A high-water mark would then make
// the dedup skip every older, never-emitted backfill event (recordId <= max) — silently
// dropping exactly the early pre-agent OOBE updates that are this feature's whole point.
// WU install events are low-volume (a handful per enrollment), so the set stays tiny.
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
// Highest RecordId emitted so far (max of restart watermark + this run). Persisted so the next
// run's restart watermark is correct.
private long _maxEmittedRecordId = -1;
private readonly object _watermarkLock = new object();
/// <param name="channel">Event log channel; defaults to the Operational channel.</param>
/// <param name="targetedEventIds">IDs to watch on <paramref name="channel"/>; defaults to the manifest set of that channel.</param>
/// <param name="tally">Counter for activity that is counted instead of emitted; null drops the counts.</param>
/// <param name="onOsUpdateActivity">Called for every Windows/.NET update step with (step, update title, event time).</param>
public WindowsUpdateTracker(
string sessionId,
string tenantId,
@@ -106,25 +112,60 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
int backfillLookbackMinutes = 60,
string stateDirectory = null,
bool channelCensusEnabled = true,
Func<bool> osBuildChangedProvider = null)
Func<bool> osBuildChangedProvider = null,
string channel = null,
UpdateActivityTally tally = null,
Action<string, string, DateTime> onOsUpdateActivity = null)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_channel = string.IsNullOrEmpty(channel) ? Channel : channel;
_isSystemChannel = string.Equals(_channel, SystemChannel, StringComparison.OrdinalIgnoreCase);
_targetedEventIds = targetedEventIds != null && targetedEventIds.Length > 0
? new HashSet<int>(targetedEventIds)
: new HashSet<int> { EventId_InstallSuccess, EventId_InstallFailure, EventId_InstallStarted, EventId_DownloadStarted };
: new HashSet<int>(_isSystemChannel ? DefaultSystemEventIds : DefaultOperationalEventIds);
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_channelCensusEnabled = channelCensusEnabled;
_osBuildChangedProvider = osBuildChangedProvider;
_tally = tally;
_onOsUpdateActivity = onOsUpdateActivity;
_watermark = new EventRecordWatermark(
stateDirectory,
_isSystemChannel ? SystemWatermarkStateFileName : WatermarkStateFileName,
logger,
_isSystemChannel ? "WindowsUpdate (System)" : "WindowsUpdate");
}
/// <summary>Channel this instance watches.</summary>
internal string WatchedChannel => _channel;
/// <summary>Targeted events processed this run, any class.</summary>
internal int EmittedThisRun => Volatile.Read(ref _emittedThisRun);
/// <summary>EventIDs this instance watches on <see cref="WatchedChannel"/>.</summary>
internal IReadOnlyCollection<int> TargetedEventIds => _targetedEventIds;
/// <summary>
/// True when the provider manifest writes <paramref name="eventId"/> to the System log
/// (16–24, 27, 28, 32, 33, 43, 44, 212–218); all other IDs live in the Operational channel.
/// </summary>
internal static bool IsSystemChannelEventId(int eventId) =>
(eventId >= 16 && eventId <= 24)
|| eventId == 27 || eventId == 28
|| eventId == 32 || eventId == 33
|| eventId == 43 || eventId == 44
|| (eventId >= 212 && eventId <= 218);
/// <summary>
/// Watches the channel and backfills it. The blind-spot census is NOT run here: it needs
/// the emissions of every channel, so the host runs it after all its trackers started.
/// </summary>
public void Start()
{
LoadWatermark();
_watermark.Load();
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
@@ -133,10 +174,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
else
{
_logger.Info("WindowsUpdate backfill disabled by config");
_logger.Info($"WindowsUpdate backfill disabled by config ({_channel})");
}
EmitChannelCensusIfBlind();
}
public void Stop()
@@ -146,11 +185,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
_watcher.Enabled = false;
_watcher.Dispose();
_logger.Info("WindowsUpdate watcher stopped");
_logger.Info($"WindowsUpdate watcher stopped ({_channel})");
}
catch (Exception ex)
{
_logger.Error("Error stopping WindowsUpdate watcher", ex);
_logger.Error($"Error stopping WindowsUpdate watcher ({_channel})", ex);
}
finally
{
@@ -160,52 +199,64 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
public void Dispose() => Stop();
internal void LoadWatermark() => _watermark.Load();
// -----------------------------------------------------------------------
// Watcher lifecycle
// -----------------------------------------------------------------------
private string ProviderFilter => _isSystemChannel ? ProviderName : null;
private void StartWatcher()
{
if (_targetedEventIds.Count == 0)
{
_logger.Warning("WindowsUpdate watcher not started: no targeted EventIDs configured");
_logger.Warning($"WindowsUpdate watcher not started: no targeted EventIDs configured ({_channel})");
return;
}
try
{
var xpath = BuildXPath(_targetedEventIds);
var query = new EventLogQuery(Channel, PathType.LogName, xpath);
var query = new EventLogQuery(_channel, PathType.LogName, BuildXPath(_targetedEventIds, ProviderFilter));
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
_watcher.Enabled = true;
_logger.Info($"WindowsUpdate watcher started: {Channel} (targetedIds={_targetedEventIds.Count})");
_logger.Info($"WindowsUpdate watcher started: {_channel} (targetedIds={string.Join(",", _targetedEventIds.OrderBy(id => id))})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"WindowsUpdate event log not found: {Channel} (normal on non-Windows 10/11 test environments)");
_logger.Warning($"WindowsUpdate event log not found: {_channel} (normal on non-Windows 10/11 test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"WindowsUpdate watcher access denied for {Channel}: {ex.Message}");
_logger.Warning($"WindowsUpdate watcher access denied for {_channel}: {ex.Message}");
}
catch (Exception ex)
{
_logger.Error($"Failed to start WindowsUpdate watcher for {Channel}", ex);
_logger.Error($"Failed to start WindowsUpdate watcher for {_channel}", ex);
// MON-D1: surface a dead watcher as one-shot telemetry, not just a local log line.
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
collectorName: "WindowsUpdateTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
collectorName: "WindowsUpdateTracker", reason: $"watcher_arm_failed:{_channel}", ex: ex);
}
}
/// <summary>Targeted-EventID XPath filter without a provider clause. Exposed for tests.</summary>
internal static string BuildXPath(HashSet<int> targetedEventIds) =>
BuildXPath(targetedEventIds, providerName: null);
/// <summary>
/// Builds the targeted-EventID XPath filter. Exposed as internal for tests.
/// Targeted-EventID XPath filter, optionally restricted to <paramref name="providerName"/>
/// (mandatory on the shared System log) and to the last <paramref name="lookbackMs"/>.
/// </summary>
internal static string BuildXPath(HashSet<int> targetedEventIds)
internal static string BuildXPath(HashSet<int> targetedEventIds, string providerName, long? lookbackMs = null)
{
var idClauses = string.Join(" or ",
targetedEventIds.OrderBy(id => id).Select(id => $"EventID={id}"));
return $"*[System[({idClauses})]]";
var clauses = new List<string>(3);
if (!string.IsNullOrEmpty(providerName))
clauses.Add($"Provider[@Name='{providerName}']");
clauses.Add("(" + string.Join(" or ", targetedEventIds.OrderBy(id => id).Select(id => $"EventID={id}")) + ")");
if (lookbackMs.HasValue)
clauses.Add($"TimeCreated[timediff(@SystemTime) <= {lookbackMs.Value}]");
return $"*[System[{string.Join(" and ", clauses)}]]";
}
// -----------------------------------------------------------------------
@@ -230,7 +281,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
var recordId = record.RecordId ?? -1;
// Cheap dedup gate BEFORE the expensive XML/FormatDescription work.
if (IsAlreadyProcessed(recordId))
if (_watermark.IsAlreadyProcessed(recordId))
return;
string xml = null;
@@ -242,6 +293,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
eventData.TryGetValue("updateGuid", out var updateGuid);
eventData.TryGetValue("updateRevisionNumber", out var updateRevision);
eventData.TryGetValue("errorCode", out var errorCode);
eventData.TryGetValue("serviceGuid", out var serviceGuid);
eventData.TryGetValue("updateCount", out var updateCount);
string description = null;
try { description = record.FormatDescription(); }
@@ -257,7 +310,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
updateRevisionNumber: updateRevision,
errorCode: errorCode,
formattedDescription: description,
isBackfill: isBackfill);
isBackfill: isBackfill,
serviceGuid: serviceGuid,
updateCount: updateCount);
}
/// <summary>
@@ -275,66 +330,68 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
string updateRevisionNumber,
string errorCode,
string formattedDescription,
bool isBackfill)
bool isBackfill,
string serviceGuid = null,
string updateCount = null)
{
if (!MarkProcessed(recordId))
if (!_watermark.TryClaim(recordId))
return; // already emitted (cross-restart or duplicate delivery)
System.Threading.Interlocked.Increment(ref _emittedThisRun);
Interlocked.Increment(ref _emittedThisRun);
string eventType;
EventSeverity severity;
bool immediateUpload;
ClassifyEventId(eventId, out eventType, out severity, out immediateUpload);
var shape = ShapeFor(eventId);
var title = string.IsNullOrEmpty(updateTitle) ? "(unknown update)" : updateTitle;
// Scan-level events name no update: deduplicate instead of classifying. Every other
// event is classified; only Windows/.NET updates are listed, the rest is counted —
// so windows_update_succeeded/failed always mean an OS update (ANALYZE-DEV-004/005).
string updateClass = null;
if (eventId == EventId_ScanFound)
{
if (!ClaimScanFoundEmission(updateCount)) { _tally?.Increment("scan_found_repeat"); return; }
}
else if (eventId == EventId_ScanFailed)
{
if (!ClaimScanFailedEmission(errorCode)) { _tally?.Increment("scan_failed_repeat"); return; }
}
else
{
updateClass = WindowsUpdateClassifier.Classify(updateTitle, serviceGuid);
if (updateClass != WindowsUpdateClassifier.Os)
{
_tally?.Increment($"{updateClass}_{eventId}");
return;
}
}
var data = new Dictionary<string, object>
{
{ "wuEventId", eventId },
{ "wuChannel", _isSystemChannel ? "system" : "operational" },
{ "wuPhase", shape.Phase },
{ "backfilled", isBackfill },
};
if (updateClass != null) data["updateClass"] = updateClass;
if (recordId >= 0) data["recordId"] = recordId;
if (level.HasValue) data["level"] = level.Value;
if (!string.IsNullOrEmpty(updateTitle)) data["updateTitle"] = updateTitle;
if (!string.IsNullOrEmpty(updateGuid)) data["updateGuid"] = updateGuid;
if (!string.IsNullOrEmpty(updateRevisionNumber)) data["updateRevisionNumber"] = updateRevisionNumber;
if (!string.IsNullOrEmpty(updateCount)) data["updateCount"] = updateCount;
if (timeCreatedUtc.HasValue) data["timeCreated"] = timeCreatedUtc.Value.ToString("o");
// Decode the HRESULT for failure events (and any event that carries a non-empty errorCode).
string hresultHex;
if (TryNormalizeHResult(errorCode, out var hresultValue, out hresultHex))
if (TryNormalizeHResult(errorCode, out var hresultValue, out var hresultHex))
{
data["hresult"] = hresultHex;
data["hresultSymbol"] = ResolveHResultSymbol(hresultValue);
}
var title = string.IsNullOrEmpty(updateTitle) ? "(unknown update)" : updateTitle;
string message;
switch (eventId)
{
case EventId_InstallSuccess:
message = $"Windows Update installed during enrollment: {title}";
break;
case EventId_InstallFailure:
message = data.TryGetValue("hresultSymbol", out var sym)
? $"Windows Update FAILED during enrollment: {title} ({data["hresult"]} {sym})"
: $"Windows Update FAILED during enrollment: {title}";
break;
case EventId_DownloadStarted:
message = $"Windows Update download started during enrollment: {title}";
break;
case EventId_InstallStarted:
message = $"Windows Update install started during enrollment: {title}";
break;
default:
message = $"Windows Update activity (EventID {eventId}) during enrollment: {title}";
break;
}
if (!string.IsNullOrEmpty(formattedDescription))
{
var truncated = formattedDescription.Length > 1000
data["description"] = formattedDescription.Length > 1000
? formattedDescription.Substring(0, 1000) + "…"
: formattedDescription;
data["description"] = truncated;
}
_post.Emit(new EnrollmentEvent
@@ -343,46 +400,107 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
TenantId = _tenantId,
// Use the WU event's own time, not the default UtcNow the ctor stamps: backfilled
// pre-agent OOBE updates must land on the timeline at when the update actually
// installed, not at agent start. InformationalEventPost forwards this as occurredAtUtc,
// happened, not at agent start. InformationalEventPost forwards this as occurredAtUtc,
// which drives the timeline entry's timestamp (EventTimelineEmitter).
Timestamp = timeCreatedUtc ?? DateTime.UtcNow,
EventType = eventType,
Severity = severity,
Source = "WindowsUpdateWatcher",
EventType = shape.EventType,
Severity = shape.Severity,
Source = Constants.EventSources.WindowsUpdateWatcher,
Phase = EnrollmentPhase.Unknown,
Message = message,
Message = BuildMessage(eventId, title, updateCount, data),
Data = data,
ImmediateUpload = immediateUpload,
ImmediateUpload = shape.Immediate,
});
if (updateClass == WindowsUpdateClassifier.Os && shape.OsStep != null && _onOsUpdateActivity != null)
{
try { _onOsUpdateActivity(shape.OsStep, updateTitle, timeCreatedUtc ?? DateTime.UtcNow); }
catch (Exception ex) { _logger.Warning($"WindowsUpdate: OS update activity callback failed: {ex.Message}"); }
}
// Last: everything this record produced has been handed to the ingress first.
_watermark.MarkEmitted(recordId);
}
private static void ClassifyEventId(int eventId, out string eventType, out EventSeverity severity, out bool immediateUpload)
private struct EventShape
{
public string EventType;
public EventSeverity Severity;
public bool Immediate;
public string Phase;
/// <summary>Step reported to the decision engine for OS updates; null = not a step.</summary>
public string OsStep;
}
private static EventShape ShapeFor(int eventId)
{
switch (eventId)
{
case EventId_InstallFailure:
eventType = Constants.EventTypes.WindowsUpdateFailed;
severity = EventSeverity.Error;
immediateUpload = true; // a mid-OOBE update failure is high-signal — surface it fast
break;
case EventId_InstallSuccess:
eventType = Constants.EventTypes.WindowsUpdateSucceeded;
severity = EventSeverity.Info;
immediateUpload = false;
break;
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateSucceeded, Severity = EventSeverity.Info, Phase = "installed", OsStep = OsUpdateSteps.Installed };
case EventId_InstallFailure:
// A mid-OOBE install failure is high-signal — surface it fast.
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateFailed, Severity = EventSeverity.Error, Immediate = true, Phase = "install", OsStep = OsUpdateSteps.Failed };
case EventId_InstallStarted:
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateStarted, Severity = EventSeverity.Info, Phase = "install", OsStep = OsUpdateSteps.InstallStarted };
case EventId_DownloadStarted:
eventType = Constants.EventTypes.WindowsUpdateStarted;
severity = EventSeverity.Debug; // timeline context only
immediateUpload = false;
break;
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateStarted, Severity = EventSeverity.Info, Phase = "download", OsStep = OsUpdateSteps.DownloadStarted };
case EventId_Downloaded:
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateStarted, Severity = EventSeverity.Info, Phase = "downloaded", OsStep = OsUpdateSteps.Downloaded };
case EventId_DownloadFailed:
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateFailed, Severity = EventSeverity.Warning, Phase = "download", OsStep = OsUpdateSteps.Failed };
case EventId_ScanFailed:
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateFailed, Severity = EventSeverity.Warning, Phase = "scan" };
case EventId_ScanFound:
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateStarted, Severity = EventSeverity.Debug, Phase = "scan" };
default:
// A newly-enabled (still-being-validated) EventID: keep it visible (Info, not Debug)
// so it can be assessed on real traces.
eventType = Constants.EventTypes.WindowsUpdateStarted;
severity = EventSeverity.Info;
immediateUpload = false;
break;
// An EventID enabled via config whose meaning has not been mapped: keep it
// visible (Info, not Debug) so it can be assessed on real traces.
return new EventShape { EventType = Constants.EventTypes.WindowsUpdateStarted, Severity = EventSeverity.Info, Phase = "activity" };
}
}
private static string BuildMessage(int eventId, string title, string updateCount, Dictionary<string, object> data)
{
var hresultSuffix = data.TryGetValue("hresultSymbol", out var sym)
? $" ({data["hresult"]} {sym})"
: string.Empty;
switch (eventId)
{
case EventId_InstallSuccess: return $"Windows Update installed during enrollment: {title}";
case EventId_InstallFailure: return $"Windows Update FAILED during enrollment: {title}{hresultSuffix}";
case EventId_InstallStarted: return $"Windows Update install started during enrollment: {title}";
case EventId_DownloadStarted: return $"Windows Update download started during enrollment: {title}";
case EventId_Downloaded: return $"Windows Update downloaded during enrollment: {title}";
case EventId_DownloadFailed: return $"Windows Update download FAILED during enrollment: {title}{hresultSuffix}";
case EventId_ScanFailed: return $"Windows Update scan failed during enrollment{hresultSuffix}";
case EventId_ScanFound:
return string.IsNullOrEmpty(updateCount)
? "Windows Update scan completed during enrollment"
: $"Windows Update scan found {updateCount} update(s) during enrollment";
default: return $"Windows Update activity (EventID {eventId}) during enrollment: {title}";
}
}
private bool ClaimScanFoundEmission(string updateCount)
{
lock (_throttleLock)
{
if (_scanFoundEmitted >= MaxScanFoundEmissions) return false;
if (string.Equals(_lastScanFoundCount, updateCount ?? string.Empty, StringComparison.Ordinal)) return false;
_lastScanFoundCount = updateCount ?? string.Empty;
_scanFoundEmitted++;
return true;
}
}
private bool ClaimScanFailedEmission(string errorCode)
{
lock (_throttleLock)
{
if (_scanFailureCodesReported.Count >= MaxScanFailedEmissions) return false;
return _scanFailureCodesReported.Add(errorCode ?? string.Empty);
}
}
@@ -496,14 +614,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
try
{
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
var idClauses = string.Join(" or ",
_targetedEventIds.OrderBy(id => id).Select(id => $"EventID={id}"));
var xpath = $"*[System[({idClauses}) and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]";
var xpath = BuildXPath(_targetedEventIds, ProviderFilter, lookbackMs);
_logger.Info($"WindowsUpdate backfill: scanning {Channel} " +
$"(lookback={_backfillLookbackMinutes}min, targetedIds={_targetedEventIds.Count}, restartWatermark={_restartWatermark})");
_logger.Info($"WindowsUpdate backfill: scanning {_channel} " +
$"(lookback={_backfillLookbackMinutes}min, targetedIds={_targetedEventIds.Count}, restartWatermark={_watermark.RestartWatermark})");
var query = new EventLogQuery(Channel, PathType.LogName, xpath);
var query = new EventLogQuery(_channel, PathType.LogName, xpath);
int processed = 0;
using (var reader = new EventLogReader(query))
@@ -520,21 +636,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
}
if (processed == 0)
_logger.Debug($"WindowsUpdate backfill: no targeted events in last {_backfillLookbackMinutes} minutes");
_logger.Debug($"WindowsUpdate backfill: no targeted events in last {_backfillLookbackMinutes} minutes ({_channel})");
else
_logger.Info($"WindowsUpdate backfill: scanned {processed} event(s)");
_logger.Info($"WindowsUpdate backfill: scanned {processed} event(s) ({_channel})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"WindowsUpdate event log not found during backfill: {Channel} (normal on non-Windows 10/11 test environments)");
_logger.Warning($"WindowsUpdate event log not found during backfill: {_channel} (normal on non-Windows 10/11 test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"WindowsUpdate backfill access denied: {ex.Message}");
_logger.Warning($"WindowsUpdate backfill access denied ({_channel}): {ex.Message}");
}
catch (Exception ex)
{
_logger.Warning($"WindowsUpdate backfill failed: {ex.Message}");
_logger.Warning($"WindowsUpdate backfill failed ({_channel}): {ex.Message}");
}
}
@@ -544,17 +660,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// <summary>
/// Self-evidencing blind spot diagnosis. Session 7443317c: an OOBE quality update
/// provably installed (OS build changed across the mid-enrollment reboot) yet neither
/// the live watcher nor two 60-min backfills captured a single targeted event — the
/// update logged via a channel/EventID set the watcher is blind to. With diagnostics
/// upload tenant-disabled there is no client-log path to find out which one; so when
/// the blind spot provably occurred (build changed AND zero targeted events emitted
/// this run), emit a one-shot unfiltered EventID histogram of both update channels
/// over the lookback window. The histogram arrives via normal event upload and tells
/// us which IDs to add to <c>WindowsUpdateTargetedEventIds</c> via remote config —
/// no agent redeploy, no diagnostics access needed.
/// provably installed (OS build changed across the mid-enrollment reboot) yet the watcher
/// captured nothing — it read the install IDs from the Operational channel, while the
/// manifest writes them to the System log (D-310). When the blind spot occurs again
/// (build changed AND zero targeted events on every channel this run), emit a one-shot
/// unfiltered EventID histogram of the Operational and UpdateOrchestrator channels.
/// </summary>
internal void EmitChannelCensusIfBlind()
/// <param name="otherChannelsEmitted">Targeted events the host's other update watchers processed this run.</param>
/// <param name="otherTargetedEventIds">EventIDs the host's other update watchers target, listed with this instance's own.</param>
internal void EmitChannelCensusIfBlind(int otherChannelsEmitted = 0, IEnumerable<int> otherTargetedEventIds = null)
{
if (!_channelCensusEnabled) return;
@@ -562,7 +676,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
if (_osBuildChangedProvider == null || !_osBuildChangedProvider())
return;
if (System.Threading.Volatile.Read(ref _emittedThisRun) > 0)
if (EmittedThisRun + otherChannelsEmitted > 0)
return;
var lookback = _backfillLookbackMinutes > 0 ? _backfillLookbackMinutes : 60;
@@ -581,8 +695,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{ "wuClientCensus", wuClientHistogram },
{ "updateOrchestratorCensus", orchestratorHistogram },
{ "lookbackMinutes", lookback },
{ "restartWatermark", _restartWatermark },
{ "targetedEventIds", string.Join(",", _targetedEventIds.OrderBy(id => id)) },
{ "restartWatermark", _watermark.RestartWatermark },
{ "targetedEventIds", string.Join(",", _targetedEventIds.Concat(otherTargetedEventIds ?? Enumerable.Empty<int>()).Distinct().OrderBy(id => id)) },
};
if (wuClient.Truncated || orchestrator.Truncated)
data["censusTruncated"] = true;
@@ -593,7 +707,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
TenantId = _tenantId,
EventType = Constants.EventTypes.WindowsUpdateChannelCensus,
Severity = EventSeverity.Debug,
Source = "WindowsUpdateWatcher",
Source = Constants.EventSources.WindowsUpdateWatcher,
Phase = EnrollmentPhase.Unknown,
Message = "Windows Update channel census: OS build changed across restart but no targeted WU events were captured — EventID histogram attached",
Data = data,
@@ -666,102 +780,5 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// <summary>Compact "id=count" form ordered by EventID, e.g. "19=2,25=1,43=2".</summary>
internal static string FormatHistogram(Dictionary<int, int> histogram) =>
string.Join(",", histogram.OrderBy(kv => kv.Key).Select(kv => $"{kv.Key}={kv.Value}"));
// -----------------------------------------------------------------------
// Watermark dedup (cross-restart)
// -----------------------------------------------------------------------
private bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_watermarkLock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
/// <summary>
/// Atomically claims <paramref name="recordId"/>. Returns true if this is the first time we
/// see it (caller should emit), false if it was already processed — either in a PRIOR run
/// (at/below the restart watermark) or already this run (in the seen-set). Only advances +
/// persists the max on a genuinely-new, higher RecordId. Records without a RecordId (-1) are
/// always emitted, never tracked or persisted.
/// </summary>
private bool MarkProcessed(long recordId)
{
if (recordId < 0) return true;
long toPersist = -1;
lock (_watermarkLock)
{
// Prior run, or already emitted this run. Note: a lower RecordId that was never
// emitted this run (e.g. a backfill record read after a higher live record) is NOT
// in _seenThisRun and is above the restart watermark, so it is correctly emitted.
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return false;
if (recordId > _maxEmittedRecordId)
{
_maxEmittedRecordId = recordId;
toPersist = _maxEmittedRecordId;
}
}
if (toPersist >= 0) PersistWatermark(toPersist);
return true;
}
internal void LoadWatermark()
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
if (!File.Exists(filePath)) return;
try
{
var json = File.ReadAllText(filePath);
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
if (state != null)
{
lock (_watermarkLock)
{
_restartWatermark = state.LastRecordId;
_maxEmittedRecordId = state.LastRecordId;
}
_logger.Info($"WindowsUpdate watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
}
catch (Exception ex)
{
_logger.Warning($"Failed to load WindowsUpdate watermark: {ex.Message}");
}
}
private void PersistWatermark(long recordId)
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
try
{
Directory.CreateDirectory(_stateDirectory);
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
var state = new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow };
var json = JsonConvert.SerializeObject(state, Formatting.Indented);
var tempPath = filePath + ".tmp";
File.WriteAllText(tempPath, json);
File.Copy(tempPath, filePath, overwrite: true);
try { File.Delete(tempPath); } catch { }
}
catch (Exception ex)
{
_logger.Warning($"Failed to persist WindowsUpdate watermark: {ex.Message}");
}
}
internal sealed class WatermarkState
{
public long LastRecordId { get; set; }
public DateTime PersistedUtc { get; set; }
}
}
}
@@ -472,7 +472,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
// Expand BEFORE the guard: the guard must judge the path that is read,
// and an unresolved token is not a path it can judge.
var expandedPath = UserProfileResolver.ExpandCustomTokens(entry.Path);
var expandedPath = UserProfileResolver.ExpandCustomTokens(entry.Path, userProfilePath);
if (expandedPath == null)
{
_logger.Warning($"Diagnostics path skipped (no user session for token): {entry.Path}");
@@ -1,6 +1,7 @@
using System;
using System.Diagnostics;
using System.IO;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
@@ -14,7 +15,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
/// The agent runs as SYSTEM, so standard environment variables like %USERPROFILE% or
/// %LOCALAPPDATA% resolve to the SYSTEM profile — not the logged-on user. This class
/// detects the real user via explorer.exe ownership (WTS session query, WMI fallback —
/// see <see cref="Interop.ProcessOwnerLookup"/>) and caches the result for the agent's lifetime.
/// see <see cref="Interop.ProcessOwnerLookup"/>) and caches the first user it finds for the
/// agent's lifetime. "No user yet" is not cached: the agent usually starts before anyone has
/// signed in, so detection is retried — at most every <see cref="RetryInterval"/>.
///
/// Usage in paths: %LOGGED_ON_USER_PROFILE%\AppData\Local\RealmJoin\Logs\*.log
/// </summary>
@@ -23,9 +26,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
// Spelled once, in Shared: the built-in diagnostics catalog uses the same token.
public const string Token = Shared.Models.DiagnosticsBuiltInSections.UserProfileToken;
/// <summary>Minimum time between two detections while no user has been found.</summary>
internal static readonly TimeSpan RetryInterval = TimeSpan.FromSeconds(30);
private static readonly object Lock = new object();
private static bool _resolved;
private static string _userProfilePath; // e.g. C:\Users\JohnDoe — null if no user detected
private static string _userProfilePath; // e.g. C:\Users\JohnDoe — null until a user is detected
private static DateTime _lastFailedDetectionUtc = DateTime.MinValue;
// Seams for tests; production uses the real detection and clock.
private static Func<string> _detector = DetectLoggedOnUserProfile;
private static Func<DateTime> _utcNow = () => DateTime.UtcNow;
/// <summary>
/// Returns the logged-on user's profile path (e.g. C:\Users\JohnDoe) or null
@@ -34,17 +44,28 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
/// </summary>
public static string GetLoggedOnUserProfilePath()
{
if (_resolved)
return _userProfilePath;
var cached = Volatile.Read(ref _userProfilePath);
if (cached != null)
return cached;
lock (Lock)
{
if (_resolved)
if (_userProfilePath != null)
return _userProfilePath;
_userProfilePath = DetectLoggedOnUserProfile();
_resolved = true;
return _userProfilePath;
// A clock that stepped backwards (common during OOBE) never delays the retry.
var now = _utcNow();
var sinceLastFailure = now - _lastFailedDetectionUtc;
if (_lastFailedDetectionUtc != DateTime.MinValue
&& sinceLastFailure >= TimeSpan.Zero && sinceLastFailure < RetryInterval)
return null;
var detected = _detector();
if (detected != null)
Volatile.Write(ref _userProfilePath, detected);
else
_lastFailedDetectionUtc = now;
return detected;
}
}
@@ -53,18 +74,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
/// Environment.ExpandEnvironmentVariables for standard tokens.
/// Returns null if the path contains the token but no user is logged on.
/// </summary>
public static string ExpandCustomTokens(string rawPath)
public static string ExpandCustomTokens(string rawPath) =>
ExpandCustomTokens(rawPath, ContainsUserProfileToken(rawPath) ? GetLoggedOnUserProfilePath() : null);
/// <summary>
/// Same, with the profile path the caller already resolved for its guard. Callers that
/// guard the expanded path must use this overload: while detection is still retrying, a
/// second lookup can find the user the first one missed, and the guard would then judge a
/// C:\Users path without the profile that admits it.
/// </summary>
public static string ExpandCustomTokens(string rawPath, string userProfilePath)
{
if (string.IsNullOrEmpty(rawPath))
return rawPath;
if (rawPath.IndexOf(Token, StringComparison.OrdinalIgnoreCase) >= 0)
{
var profilePath = GetLoggedOnUserProfilePath();
if (profilePath == null)
if (userProfilePath == null)
return null; // No user detected — caller should skip this path
rawPath = ReplaceCaseInsensitive(rawPath, Token, profilePath);
rawPath = ReplaceCaseInsensitive(rawPath, Token, userProfilePath);
}
return Environment.ExpandEnvironmentVariables(rawPath);
@@ -80,26 +109,37 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Runtime
}
/// <summary>
/// Resets the cached state. Only used for testing.
/// Resets the cached state and the seams. Only used for testing.
/// </summary>
internal static void Reset()
{
lock (Lock)
{
_resolved = false;
_userProfilePath = null;
_lastFailedDetectionUtc = DateTime.MinValue;
_detector = DetectLoggedOnUserProfile;
_utcNow = () => DateTime.UtcNow;
}
}
/// <summary>
/// Allows tests to inject a specific profile path without WMI.
/// Allows tests to inject a specific profile path without WMI. Null pins "no user signed in"
/// — detection keeps answering null instead of finding the developer's own session.
/// </summary>
internal static void SetForTesting(string profilePath)
{
SetDetectorForTesting(() => profilePath, () => DateTime.UtcNow);
}
/// <summary>Replaces detection and clock, clearing what was cached. Only used for testing.</summary>
internal static void SetDetectorForTesting(Func<string> detector, Func<DateTime> utcNow)
{
lock (Lock)
{
_userProfilePath = profilePath;
_resolved = true;
_userProfilePath = null;
_lastFailedDetectionUtc = DateTime.MinValue;
_detector = detector ?? throw new ArgumentNullException(nameof(detector));
_utcNow = utcNow ?? throw new ArgumentNullException(nameof(utcNow));
}
}
@@ -23,7 +23,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
filePath = UserProfileResolver.ExpandCustomTokens(filePath, userProfilePath);
if (filePath == null)
return data; // Token present but no user logged on — skip silently
@@ -24,7 +24,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
filePath = UserProfileResolver.ExpandCustomTokens(filePath, userProfilePath);
if (filePath == null)
return data; // Token present but no user logged on — skip silently
@@ -0,0 +1,313 @@
using System;
using System.IO;
using System.Security.Cryptography;
using System.Text;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
{
/// <summary>
/// Text encoding of a log file as the logparser reads it. Chosen from the byte order mark at
/// the start of the file in <see cref="StreamReader"/>'s detection order; without one the file
/// is UTF-8, which is also how ANSI logs were always read (invalid bytes decode to U+FFFD).
/// </summary>
internal sealed class LogTextEncoding
{
public static readonly LogTextEncoding Utf8 = new LogTextEncoding("utf-8", new UTF8Encoding(false), 1, 0);
public static readonly LogTextEncoding Utf8WithBom = new LogTextEncoding("utf-8 with bom", new UTF8Encoding(false), 1, 3);
public static readonly LogTextEncoding Utf16LittleEndian = new LogTextEncoding("utf-16le", new UnicodeEncoding(false, false), 2, 2);
public static readonly LogTextEncoding Utf16BigEndian = new LogTextEncoding("utf-16be", new UnicodeEncoding(true, false), 2, 2);
public static readonly LogTextEncoding Utf32LittleEndian = new LogTextEncoding("utf-32le", new UTF32Encoding(false, false), 4, 4);
public static readonly LogTextEncoding Utf32BigEndian = new LogTextEncoding("utf-32be", new UTF32Encoding(true, false), 4, 4);
private LogTextEncoding(string name, Encoding encoding, int unitSize, int byteOrderMarkLength)
{
Name = name;
Encoding = encoding;
UnitSize = unitSize;
ByteOrderMarkLength = byteOrderMarkLength;
LineFeed = encoding.GetBytes("\n");
CarriageReturn = encoding.GetBytes("\r");
}
public string Name { get; }
public Encoding Encoding { get; }
/// <summary>Bytes per code unit; every line boundary lies on a unit boundary.</summary>
public int UnitSize { get; }
/// <summary>Length of the byte order mark at offset 0 — skipped, never part of a line.</summary>
public int ByteOrderMarkLength { get; }
public byte[] LineFeed { get; }
public byte[] CarriageReturn { get; }
public static LogTextEncoding Detect(byte[] head)
{
if (StartsWith(head, 0xEF, 0xBB, 0xBF)) return Utf8WithBom;
if (StartsWith(head, 0xFF, 0xFE, 0x00, 0x00)) return Utf32LittleEndian;
if (StartsWith(head, 0xFF, 0xFE)) return Utf16LittleEndian;
if (StartsWith(head, 0xFE, 0xFF)) return Utf16BigEndian;
if (StartsWith(head, 0x00, 0x00, 0xFE, 0xFF)) return Utf32BigEndian;
return Utf8;
}
/// <summary>The last code-unit boundary at or before <paramref name="position"/>.</summary>
public long AlignDown(long position)
{
if (position <= ByteOrderMarkLength) return position < ByteOrderMarkLength ? 0 : position;
return position - (position - ByteOrderMarkLength) % UnitSize;
}
private static bool StartsWith(byte[] bytes, params byte[] prefix)
{
if (bytes.Length < prefix.Length) return false;
for (var i = 0; i < prefix.Length; i++)
if (bytes[i] != prefix[i]) return false;
return true;
}
}
/// <summary>
/// The first bytes of a log file: they carry its byte order mark and identify the file, so a
/// file replaced under the same name is recognised even when it is already longer than the
/// position read so far.
/// </summary>
internal sealed class LogFileHead
{
internal const int MaxBytes = 4096;
private readonly byte[] _bytes;
private LogFileHead(byte[] bytes)
{
_bytes = bytes;
}
/// <summary>Up to <see cref="MaxBytes"/> from offset 0. Leaves the stream at an undefined position.</summary>
public static LogFileHead Read(Stream stream)
{
stream.Seek(0, SeekOrigin.Begin);
var buffer = new byte[(int)Math.Min(MaxBytes, stream.Length)];
var read = 0;
while (read < buffer.Length)
{
var n = stream.Read(buffer, read, buffer.Length - read);
if (n <= 0) break;
read += n;
}
if (read < buffer.Length) Array.Resize(ref buffer, read);
return new LogFileHead(buffer);
}
internal static LogFileHead FromBytes(byte[] bytes) => new LogFileHead(bytes);
public int Length => _bytes.Length;
public LogTextEncoding Encoding => LogTextEncoding.Detect(_bytes);
/// <summary>SHA-256 of the first <paramref name="length"/> bytes, Base64.</summary>
public string Hash(int length)
{
using (var sha = SHA256.Create())
return Convert.ToBase64String(sha.ComputeHash(_bytes, 0, length));
}
}
/// <summary>
/// Byte-exact line reader for the logparser collector. <see cref="Position"/> is the file
/// offset right behind the last line returned, so a caller that stops after any line can
/// resume exactly there — <see cref="StreamReader"/> only exposes its read-ahead boundary.
/// <para>
/// Line ends are those of <see cref="StreamReader.ReadLine"/>: "\r\n", "\n" and a lone "\r",
/// in code units of the file's <see cref="LogTextEncoding"/>. A "\n" directly after the start
/// position is skipped when the unit before it is "\r": the previous run ended on a "\r" whose
/// "\n" had not been written yet. A line at end of file without a line end is returned with
/// <see cref="LastLineTerminated"/> false; the caller decides whether the writer is still in
/// it. A line longer than the cap is consumed without being kept: <see cref="ReadLine"/>
/// returns an empty string and sets <see cref="LastLineTruncated"/>.
/// </para>
/// </summary>
internal sealed class LogLineReader
{
private const int ReadBufferSize = 64 * 1024;
private readonly Stream _stream;
private readonly LogTextEncoding _encoding;
private readonly int _unit;
private readonly int _maxLineBytes;
private readonly byte[] _buffer = new byte[ReadBufferSize];
private int _pos;
private int _len;
private long _bufferOffset;
private bool _eof;
private bool _skipLineFeedAfterCarriageReturn;
private byte[] _line = new byte[4096];
private int _lineLength;
/// <param name="stream">Seekable stream positioned at a code-unit boundary.</param>
public LogLineReader(Stream stream, LogTextEncoding encoding, int maxLineBytes)
{
if (stream == null) throw new ArgumentNullException(nameof(stream));
if (encoding == null) throw new ArgumentNullException(nameof(encoding));
if (maxLineBytes <= 0) throw new ArgumentOutOfRangeException(nameof(maxLineBytes));
_stream = stream;
_encoding = encoding;
_unit = encoding.UnitSize;
_maxLineBytes = maxLineBytes;
_bufferOffset = stream.Position;
_skipLineFeedAfterCarriageReturn = PrecededByCarriageReturn();
}
/// <summary>File offset of the next unread byte, exact.</summary>
public long Position => _bufferOffset + _pos;
/// <summary>File offset of the first byte of the line most recently returned.</summary>
public long LastLineStart { get; private set; }
/// <summary>False for a line that reached end of file without a line end.</summary>
public bool LastLineTerminated { get; private set; }
/// <summary>True when the last line was longer than the cap; it was consumed, not returned.</summary>
public bool LastLineTruncated { get; private set; }
/// <summary>The next line without its line end, or null at end of file.</summary>
public string ReadLine()
{
if (Position == 0 && _encoding.ByteOrderMarkLength > 0 && Ensure(_encoding.ByteOrderMarkLength))
_pos += _encoding.ByteOrderMarkLength;
if (_skipLineFeedAfterCarriageReturn)
{
_skipLineFeedAfterCarriageReturn = false;
if (Ensure(_unit) && UnitAt(_pos, _encoding.LineFeed))
_pos += _unit;
}
LastLineStart = Position;
LastLineTerminated = false;
LastLineTruncated = false;
_lineLength = 0;
while (true)
{
if (!Ensure(_unit))
{
// End of file. A partial code unit (a writer mid-character) belongs to the line.
var rest = _len - _pos;
if (rest > 0)
{
Append(_pos, rest);
_pos = _len;
}
return Position > LastLineStart ? Decode() : null;
}
// Scan the whole code units in the buffer for the next line end and take the text
// before it in one copy.
var end = _pos + (_len - _pos) / _unit * _unit;
var scan = _pos;
while (scan < end && !IsLineEnd(scan))
scan += _unit;
if (scan > _pos)
{
Append(_pos, scan - _pos);
_pos = scan;
}
if (scan == end)
continue; // no line end in the buffer yet — refill
var carriageReturn = !UnitAt(_pos, _encoding.LineFeed);
_pos += _unit;
if (carriageReturn && Ensure(_unit) && UnitAt(_pos, _encoding.LineFeed))
_pos += _unit;
LastLineTerminated = true;
return Decode();
}
}
private bool IsLineEnd(int offset)
{
if (_unit == 1)
{
var b = _buffer[offset];
return b == (byte)'\n' || b == (byte)'\r';
}
return UnitAt(offset, _encoding.LineFeed) || UnitAt(offset, _encoding.CarriageReturn);
}
private bool PrecededByCarriageReturn()
{
var start = _bufferOffset;
if (start - _unit < _encoding.ByteOrderMarkLength) return false;
var previous = new byte[_unit];
_stream.Seek(start - _unit, SeekOrigin.Begin);
var read = 0;
while (read < _unit)
{
var n = _stream.Read(previous, read, _unit - read);
if (n <= 0) break;
read += n;
}
_stream.Seek(start, SeekOrigin.Begin);
if (read < _unit) return false;
for (var i = 0; i < _unit; i++)
if (previous[i] != _encoding.CarriageReturn[i]) return false;
return true;
}
/// <summary>At least <paramref name="count"/> unread bytes in the buffer, unless the file ends first.</summary>
private bool Ensure(int count)
{
while (_len - _pos < count)
{
if (_eof) return false;
var remaining = _len - _pos;
if (_pos > 0)
{
Buffer.BlockCopy(_buffer, _pos, _buffer, 0, remaining);
_bufferOffset += _pos;
_pos = 0;
_len = remaining;
}
var read = _stream.Read(_buffer, _len, _buffer.Length - _len);
if (read <= 0)
{
_eof = true;
return _len - _pos >= count;
}
_len += read;
}
return true;
}
private bool UnitAt(int offset, byte[] unit)
{
for (var i = 0; i < _unit; i++)
if (_buffer[offset + i] != unit[i]) return false;
return true;
}
private void Append(int offset, int count)
{
var room = _maxLineBytes - _lineLength;
if (count > room)
{
LastLineTruncated = true;
count = room;
if (count <= 0) return;
}
if (_lineLength + count > _line.Length)
{
var grown = new byte[Math.Max(_line.Length * 2, _lineLength + count)];
Buffer.BlockCopy(_line, 0, grown, 0, _lineLength);
_line = grown;
}
Buffer.BlockCopy(_buffer, offset, _line, _lineLength, count);
_lineLength += count;
}
private string Decode() =>
_lineLength == 0 || LastLineTruncated ? string.Empty : _encoding.Encoding.GetString(_line, 0, _lineLength);
}
}
@@ -18,6 +18,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
{
public string CollectorType => "logparser";
/// <summary>A wildcard target reads at most this many files per run, the newest first.</summary>
internal const int MaxFilesPerRun = 20;
/// <summary>Lines one run reads from one file unless the rule sets <c>maxLines</c>.</summary>
internal const int DefaultMaxLines = 1000;
/// <summary>The IME tracker's line bound: a longer physical line is skipped, never matched.</summary>
internal const int MaxLineBytes = 32 * 1024 * 1024;
/// <summary>
/// Executes the log parser rule. Returns null because logparser emits events directly
/// via context.OnEventCollected rather than returning a single result dictionary.
@@ -34,7 +43,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
filePath = UserProfileResolver.ExpandCustomTokens(filePath, userProfilePath);
if (filePath == null)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
@@ -84,10 +93,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
if (rule.Parameters.TryGetValue("trackPosition", out trackPositionStr))
bool.TryParse(trackPositionStr, out trackPosition);
// A value that is no positive number keeps the default: a run capped at 0 lines would
// never advance its position, and the rule would read nothing for the whole session.
string maxLinesStr;
int maxLines = 1000;
int maxLines = DefaultMaxLines;
if (rule.Parameters.TryGetValue("maxLines", out maxLinesStr))
int.TryParse(maxLinesStr, out maxLines);
{
int parsedMaxLines;
if (int.TryParse(maxLinesStr, out parsedMaxLines) && parsedMaxLines > 0)
maxLines = parsedMaxLines;
else
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"maxLines '{maxLinesStr}' is not a positive number — using {DefaultMaxLines}");
}
// Determine format: "cmtrace" (default) or "text" for plain text logs
string formatStr;
@@ -144,7 +162,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
/// as the event data). Stops after <see cref="MaxTracedMatchesPerFile"/> matches.
/// </summary>
private static void TraceMatch(GatherRuleContext context, GatherRule rule, string fileName,
int lineNumber, Match match, Regex pattern, int matchNumber)
long lineNumber, Match match, Regex pattern, int matchNumber)
{
if (matchNumber > MaxTracedMatchesPerFile)
{
@@ -195,14 +213,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
try
{
// Return matched files sorted by last write time (newest first), capped at 20
// Return matched files sorted by last write time (newest first), capped at MaxFilesPerRun
var allMatches = Directory.GetFiles(directory, fileNamePart);
if (allMatches.Length > 20)
if (allMatches.Length > MaxFilesPerRun)
context.DebugLog(ruleId, GatherRuleDebugLog.StageLogParser,
$"wildcard matched {allMatches.Length} files — capped to the 20 newest");
$"wildcard matched {allMatches.Length} files — capped to the {MaxFilesPerRun} newest");
return allMatches
.OrderByDescending(f => new FileInfo(f).LastWriteTimeUtc)
.Take(20)
.Take(MaxFilesPerRun)
.ToList();
}
catch (Exception ex)
@@ -216,183 +234,248 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
private static void ProcessLogFile(string filePath, GatherRule rule, Regex pattern,
bool trackPosition, int maxLines, bool isTextMode, GatherRuleContext context)
{
var fileName = Path.GetFileName(filePath);
try
{
var fileInfo = new FileInfo(filePath);
long startPosition = trackPosition
? context.FilePositionTracker.GetSafePosition(filePath, fileInfo.Length)
: 0;
var state = context.SessionState;
var known = trackPosition ? state.PeekLogPosition(rule.RuleId, filePath) : null;
// Nothing new to read
if (startPosition >= fileInfo.Length)
// Nothing new since the last run — decided from the length alone, without opening the file.
var length = new FileInfo(filePath).Length;
if (known != null && known.Position == length)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{Path.GetFileName(filePath)}: no new content (position {startPosition} >= length {fileInfo.Length}) — trackPosition=true, nothing re-read");
$"{fileName}: no new content (position {known.Position} >= length {length}) — trackPosition=true, nothing re-read" +
(known.FromPreviousRun ? "; position saved by a previous agent run" : ""));
return;
}
// A last line without a line end is usually one its writer is still in. An interval
// rule leaves it for its next run, once. A one-shot trigger may have no next run, so it
// reads the line now but saves the position before it: the next run reads it again,
// whole — at worst a match repeats, it is never split.
var isInterval = string.Equals(rule.Trigger, "interval", StringComparison.OrdinalIgnoreCase);
var mayHoldTail = trackPosition && isInterval;
var tailReadAgainNextRun = false;
int matchCount = 0;
int linesRead = 0;
int committedLines = 0;
int parseFailCount = 0;
int timeoutCount = 0;
long endPosition = startPosition;
int oversizedCount = 0;
long startPosition;
long endPosition;
long lineNumberBase;
long fileLength;
long heldTailStart = -1;
LogFileHead head;
LogTextEncoding encoding;
using (var stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
fileLength = stream.Length;
head = LogFileHead.Read(stream);
encoding = head.Encoding;
LogReadStart start;
startPosition = encoding.AlignDown(GatherRuleSessionState.ResolveStart(known, fileLength, head, out start));
lineNumberBase = start == LogReadStart.Continue ? known.LineNumber : 0;
TraceReadStart(context, rule, fileName, known, start, fileLength);
stream.Seek(startPosition, SeekOrigin.Begin);
var reader = new LogLineReader(stream, encoding, MaxLineBytes);
endPosition = startPosition;
using (var reader = new StreamReader(stream, Encoding.UTF8))
while (linesRead < maxLines)
{
string line;
while ((line = reader.ReadLine()) != null && linesRead < maxLines)
var line = reader.ReadLine();
if (line == null)
{
linesRead++;
if (isTextMode)
{
// Text mode: match regex directly against the raw line
Match match;
try
{
match = pattern.Match(line);
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
continue;
var data = new Dictionary<string, object>();
foreach (var groupName in pattern.GetGroupNames())
{
if (groupName == "0") continue;
var group = match.Groups[groupName];
if (group.Success)
data[groupName] = group.Value;
}
data["logLine"] = TruncateMessage(line, 500);
data["logLineNumber"] = linesRead;
data["logFile"] = Path.GetFileName(filePath);
data["ruleId"] = rule.RuleId;
data["ruleTitle"] = rule.Title;
GatherRuleExecutor.StampRuleMarkers(rule, data);
var eventType = !string.IsNullOrEmpty(rule.OutputEventType)
? rule.OutputEventType
: "logparser_match";
var severity = GatherRuleContext.ParseSeverity(rule.OutputSeverity);
context.OnEventCollected(new EnrollmentEvent
{
SessionId = context.SessionId,
TenantId = context.TenantId,
Timestamp = DateTime.UtcNow,
EventType = eventType,
Severity = severity,
Source = GatherRuleExecutor.SourceName,
Message = $"Gather: {rule.Title}",
Data = data
});
matchCount++;
TraceMatch(context, rule, Path.GetFileName(filePath), linesRead, match, pattern, matchCount);
}
else
{
// CMTrace mode: parse line as CMTrace format, match regex against message
CmTraceLogEntry entry;
if (!CmTraceLogParser.TryParseLine(line, out entry))
{
parseFailCount++;
continue;
}
Match match;
try
{
match = pattern.Match(entry.Message);
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
continue;
var data = new Dictionary<string, object>();
foreach (var groupName in pattern.GetGroupNames())
{
if (groupName == "0") continue;
var group = match.Groups[groupName];
if (group.Success)
data[groupName] = group.Value;
}
// Gather-rule logs are arbitrary files this collector does not
// tail, so no writer offset can be measured for them; the
// reader-zone fallback carries its known defect (see
// ResolveUtcAssumingReaderZone). Behaviour unchanged from before
// the parser split.
var entryUtc = entry.TimestampUtc
?? (entry.HasTimestamp
? CmTraceLogParser.ResolveUtcAssumingReaderZone(entry.LocalTimestamp)
: DateTime.UtcNow);
data["logTimestamp"] = entryUtc.ToString("o");
data["logComponent"] = entry.Component;
data["logType"] = entry.Type;
data["logMessage"] = TruncateMessage(entry.Message, 500);
data["logFile"] = Path.GetFileName(filePath);
data["ruleId"] = rule.RuleId;
data["ruleTitle"] = rule.Title;
GatherRuleExecutor.StampRuleMarkers(rule, data);
var eventType = !string.IsNullOrEmpty(rule.OutputEventType)
? rule.OutputEventType
: "logparser_match";
var severity = MapCmTraceTypeToSeverity(entry.Type, rule.OutputSeverity);
context.OnEventCollected(new EnrollmentEvent
{
SessionId = context.SessionId,
TenantId = context.TenantId,
Timestamp = entryUtc,
EventType = eventType,
Severity = severity,
Source = GatherRuleExecutor.SourceName,
Message = $"Gather: {rule.Title}",
Data = data
});
matchCount++;
TraceMatch(context, rule, Path.GetFileName(filePath), linesRead, match, pattern, matchCount);
}
// End of file. A "\n" the reader skipped behind the "\r" the previous run
// ended on is consumed too — unless a kept tail must be read again.
if (!tailReadAgainNextRun)
endPosition = reader.Position;
break;
}
// Capture the final stream position after reading
endPosition = stream.Position;
var unfinished = !reader.LastLineTerminated;
if (unfinished && mayHoldTail
&& (known == null || known.HeldTailStart != reader.LastLineStart))
{
heldTailStart = reader.LastLineStart;
break;
}
linesRead++;
var lineNumber = lineNumberBase + linesRead;
if (unfinished && trackPosition && !isInterval)
{
tailReadAgainNextRun = true;
}
else
{
endPosition = reader.Position;
committedLines = linesRead;
}
if (reader.LastLineTruncated)
{
oversizedCount++;
continue;
}
if (isTextMode)
{
// Text mode: match regex directly against the raw line
Match match;
try
{
match = pattern.Match(line);
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
continue;
var data = new Dictionary<string, object>();
foreach (var groupName in pattern.GetGroupNames())
{
if (groupName == "0") continue;
var group = match.Groups[groupName];
if (group.Success)
data[groupName] = group.Value;
}
data["logLine"] = TruncateMessage(line, 500);
data["logLineNumber"] = lineNumber;
data["logFile"] = fileName;
data["ruleId"] = rule.RuleId;
data["ruleTitle"] = rule.Title;
GatherRuleExecutor.StampRuleMarkers(rule, data);
var eventType = !string.IsNullOrEmpty(rule.OutputEventType)
? rule.OutputEventType
: "logparser_match";
var severity = GatherRuleContext.ParseSeverity(rule.OutputSeverity);
context.OnEventCollected(new EnrollmentEvent
{
SessionId = context.SessionId,
TenantId = context.TenantId,
Timestamp = DateTime.UtcNow,
EventType = eventType,
Severity = severity,
Source = GatherRuleExecutor.SourceName,
Message = $"Gather: {rule.Title}",
Data = data
});
matchCount++;
TraceMatch(context, rule, fileName, lineNumber, match, pattern, matchCount);
}
else
{
// CMTrace mode: parse line as CMTrace format, match regex against message
CmTraceLogEntry entry;
if (!CmTraceLogParser.TryParseLine(line, out entry))
{
parseFailCount++;
continue;
}
Match match;
try
{
match = pattern.Match(entry.Message);
}
catch (RegexMatchTimeoutException)
{
timeoutCount++;
continue;
}
if (!match.Success)
continue;
var data = new Dictionary<string, object>();
foreach (var groupName in pattern.GetGroupNames())
{
if (groupName == "0") continue;
var group = match.Groups[groupName];
if (group.Success)
data[groupName] = group.Value;
}
// Gather-rule logs are arbitrary files this collector does not
// tail, so no writer offset can be measured for them; the
// reader-zone fallback carries its known defect (see
// ResolveUtcAssumingReaderZone). Behaviour unchanged from before
// the parser split.
var entryUtc = entry.TimestampUtc
?? (entry.HasTimestamp
? CmTraceLogParser.ResolveUtcAssumingReaderZone(entry.LocalTimestamp)
: DateTime.UtcNow);
data["logTimestamp"] = entryUtc.ToString("o");
data["logComponent"] = entry.Component;
data["logType"] = entry.Type;
data["logMessage"] = TruncateMessage(entry.Message, 500);
data["logFile"] = fileName;
data["ruleId"] = rule.RuleId;
data["ruleTitle"] = rule.Title;
GatherRuleExecutor.StampRuleMarkers(rule, data);
var eventType = !string.IsNullOrEmpty(rule.OutputEventType)
? rule.OutputEventType
: "logparser_match";
var severity = MapCmTraceTypeToSeverity(entry.Type, rule.OutputSeverity);
context.OnEventCollected(new EnrollmentEvent
{
SessionId = context.SessionId,
TenantId = context.TenantId,
Timestamp = entryUtc,
EventType = eventType,
Severity = severity,
Source = GatherRuleExecutor.SourceName,
Message = $"Gather: {rule.Title}",
Data = data
});
matchCount++;
TraceMatch(context, rule, fileName, lineNumber, match, pattern, matchCount);
}
}
}
// Saved only now, after every event of this file went out: a process that dies
// before this line repeats the file's new lines after the restart, never skips them.
if (trackPosition)
context.FilePositionTracker.SetPosition(filePath, endPosition);
state.CommitLogPosition(rule.RuleId, filePath, endPosition, lineNumberBase + committedLines, head, heldTailStart);
if (matchCount > 0)
context.Logger.Debug($"LogParser rule {rule.RuleId}: {matchCount} matches from {linesRead} lines in {Path.GetFileName(filePath)}");
context.Logger.Debug($"LogParser rule {rule.RuleId}: {matchCount} matches from {linesRead} lines in {fileName}");
// Per-file outcome — always written so a "matched 0" run is visible too.
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{Path.GetFileName(filePath)}: read {linesRead} lines from position {startPosition}->{endPosition}, " +
$"matched {matchCount}, parseFailures={parseFailCount}, regexTimeouts={timeoutCount}, mode={(isTextMode ? "text" : "cmtrace")}");
$"{fileName}: read {linesRead} lines from position {startPosition}->{endPosition}, " +
$"matched {matchCount}, parseFailures={parseFailCount}, regexTimeouts={timeoutCount}, mode={(isTextMode ? "text" : "cmtrace")}, encoding={encoding.Name}" +
(oversizedCount > 0 ? $", skipped {oversizedCount} line(s) over {MaxLineBytes / (1024 * 1024)} MB" : ""));
if (heldTailStart >= 0)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{fileName}: last line has no line end yet — left for the next run, read then even if still unfinished");
else if (tailReadAgainNextRun)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{fileName}: last line has no line end yet — read now and again by the next run, position kept before it");
if (!isTextMode && linesRead > 0 && parseFailCount == linesRead)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
"every line failed CMTrace parsing — if this is a plain-text log, set parameter format=text");
if (linesRead == maxLines)
if (linesRead == maxLines && endPosition < fileLength)
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"stopped at maxLines={maxLines} — remaining content is deferred to the next run");
}
@@ -403,6 +486,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
}
}
private static void TraceReadStart(GatherRuleContext context, GatherRule rule, string fileName,
LogPositionSnapshot known, LogReadStart start, long fileLength)
{
if (start == LogReadStart.Shrunk)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{fileName}: file is shorter than the saved position (length {fileLength} < {known.Position}) — truncated or recreated, reading from the beginning");
}
else if (start == LogReadStart.Replaced)
{
context.Logger.Info($"LogParser rule {rule.RuleId}: {fileName} was replaced since the last read — reading it from the beginning");
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{fileName}: file start changed since the last read (replaced or rewritten) — saved position {known.Position} discarded, reading from the beginning");
}
else if (start == LogReadStart.Continue && known.FromPreviousRun)
{
context.DebugLog(rule.RuleId, GatherRuleDebugLog.StageLogParser,
$"{fileName}: continuing at position {known.Position} (after line {known.LineNumber}) saved by a previous agent run");
}
}
private static EventSeverity MapCmTraceTypeToSeverity(int cmTraceType, string ruleOverride)
{
// If the rule specifies a severity, use it
@@ -26,7 +26,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors
// Expand custom tokens (%LOGGED_ON_USER_PROFILE%) and standard environment variables
var userProfilePath = UserProfileResolver.ContainsUserProfileToken(filePath)
? UserProfileResolver.GetLoggedOnUserProfilePath() : null;
filePath = UserProfileResolver.ExpandCustomTokens(filePath);
filePath = UserProfileResolver.ExpandCustomTokens(filePath, userProfilePath);
if (filePath == null)
return data; // Token present but no user logged on — skip silently
@@ -4,7 +4,6 @@ using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
@@ -19,7 +18,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
public string TenantId { get; }
public Action<EnrollmentEvent> OnEventCollected { get; }
public string ImeLogPathOverride { get; }
public LogFilePositionTracker FilePositionTracker { get; }
/// <summary>Logparser positions per (rule, file) and on_change values; survives agent restarts when persisted.</summary>
public GatherRuleSessionState SessionState { get; }
public bool UnrestrictedMode { get; set; }
/// <summary>
@@ -34,7 +36,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
string tenantId,
Action<EnrollmentEvent> onEventCollected,
string imeLogPathOverride,
LogFilePositionTracker filePositionTracker,
GatherRuleSessionState sessionState,
GatherRuleDebugLog debugLog = null)
{
Logger = logger ?? throw new ArgumentNullException(nameof(logger));
@@ -42,7 +44,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
TenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
OnEventCollected = onEventCollected ?? throw new ArgumentNullException(nameof(onEventCollected));
ImeLogPathOverride = imeLogPathOverride;
FilePositionTracker = filePositionTracker ?? throw new ArgumentNullException(nameof(filePositionTracker));
SessionState = sessionState ?? throw new ArgumentNullException(nameof(sessionState));
DebugWriter = debugLog;
}
@@ -10,7 +10,6 @@ using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
@@ -38,6 +37,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
private readonly AgentLogger _logger;
private readonly GatherRuleDebugLog _debug; // null unless EnableGatherRuleDebugLog / --gather-debug-log
private readonly GatherRuleSessionState _state;
private readonly GatherRuleContext _context;
private readonly Dictionary<string, IGatherRuleCollector> _collectors;
@@ -50,12 +50,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// Phase-scope + emit-mode state (ActivePhases / ActiveFromPhase / EmitMode on GatherRule).
// Guarded by _scopeLock; collector work always runs outside the lock. The state deliberately
// survives UpdateRules config refreshes — an agent restart causes at most one re-emit per
// on_change rule (acceptable under the per-enrollment lifecycle).
// survives UpdateRules config refreshes. Across agent restarts only the last emitted hash
// survives: it is seeded from _state and committed there after each emit, so an unchanged
// on_change result stays silent after a reboot. Phase latches, dedup sets and the
// suppression streak are process-scoped.
private readonly object _scopeLock = new object();
private EnrollmentPhase _currentPhase = EnrollmentPhase.Unknown;
private readonly HashSet<string> _fromPhaseLatched = new HashSet<string>();
private readonly Dictionary<string, string> _lastEmittedHash = new Dictionary<string, string>();
private readonly HashSet<string> _hashFromPreviousRun = new HashSet<string>();
private readonly Dictionary<string, (int Count, DateTime SinceUtc)> _suppressed =
new Dictionary<string, (int Count, DateTime SinceUtc)>();
private readonly HashSet<string> _scopeBypassLogged = new HashSet<string>();
@@ -77,9 +80,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
set { _context.UnrestrictedMode = value; }
}
/// <param name="stateDirectory">
/// Where logparser positions and on_change values are kept across agent restarts. Null keeps
/// them in memory — required for <c>--run-gather-rules</c>, which runs beside the live agent
/// under a throwaway session id and must not overwrite the session's file.
/// </param>
public GatherRuleExecutor(string sessionId, string tenantId, Action<EnrollmentEvent> onEventCollected,
AgentLogger logger, string imeLogPathOverride = null, string debugLogPath = null,
Action<string> debugEcho = null)
Action<string> debugEcho = null, string stateDirectory = null)
{
if (sessionId == null) throw new ArgumentNullException(nameof(sessionId));
if (tenantId == null) throw new ArgumentNullException(nameof(tenantId));
@@ -89,8 +97,19 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_logger = logger;
_debug = string.IsNullOrEmpty(debugLogPath) ? null : new GatherRuleDebugLog(debugLogPath, logger, debugEcho);
var filePositionTracker = new LogFilePositionTracker();
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, filePositionTracker, _debug);
var persistence = string.IsNullOrEmpty(stateDirectory) ? null : new GatherRuleStatePersistence(stateDirectory, logger);
_state = new GatherRuleSessionState(sessionId, persistence, logger);
_context = new GatherRuleContext(logger, sessionId, tenantId, onEventCollected, imeLogPathOverride, _state, _debug);
foreach (var hash in _state.GetEmittedHashes())
{
_lastEmittedHash[hash.Key] = hash.Value;
_hashFromPreviousRun.Add(hash.Key);
}
_logger.Info($"GatherRuleExecutor: gather state {_state.RestoreSummary}");
DebugLog(null, GatherRuleDebugLog.StageConfig, $"gather state: {_state.RestoreSummary}");
foreach (var detail in _state.RestoredDetails)
DebugLog(detail.Key, GatherRuleDebugLog.StageConfig, detail.Value);
// Register all collector strategies
var collectorList = new IGatherRuleCollector[]
@@ -480,14 +499,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
// empty result never reaches this point, so an emitOnlyIfExists miss neither
// emits nor updates the hash — composing to "one event on appearance, then
// only on change".
string emittedHash = null;
if (IsOnChangeMode(rule))
{
string hashPrefix;
int suppressedStreak;
if (!ShouldEmitOnChange(rule, result, out hashPrefix, out suppressedStreak))
bool comparedWithPreviousRun;
if (!ShouldEmitOnChange(rule, result, out emittedHash, out hashPrefix, out suppressedStreak, out comparedWithPreviousRun))
{
DebugLog(rule.RuleId, GatherRuleDebugLog.StageSuppress,
$"result unchanged (hash {hashPrefix}) — emit suppressed (on_change), suppressed streak={suppressedStreak}");
$"result unchanged (hash {hashPrefix}) — emit suppressed (on_change), suppressed streak={suppressedStreak}" +
(comparedWithPreviousRun ? "; compared with the value a previous agent run emitted" : ""));
return;
}
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
@@ -524,6 +546,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
Message = $"Gather: {rule.Title}",
Data = result
});
// Committed only after the event went out: a process that dies in between emits
// the result once more after the restart instead of never.
if (emittedHash != null)
_state.CommitEmittedHash(rule.RuleId, emittedHash);
DebugLog(rule.RuleId, GatherRuleDebugLog.StageEmit,
$"emitted {eventType} (severity={severity}, {result.Count} fields)");
}
@@ -679,9 +707,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result,
out string hashPrefix, out int suppressedStreak)
{
var hash = ComputeCanonicalHash(result);
string hash;
bool comparedWithPreviousRun;
return ShouldEmitOnChange(rule, result, out hash, out hashPrefix, out suppressedStreak, out comparedWithPreviousRun);
}
/// <summary>
/// Full form: <paramref name="hash"/> is the result's canonical hash, committed to the session
/// state once the event went out; <paramref name="comparedWithPreviousRun"/> says a suppression
/// compared against the value a previous agent run emitted.
/// </summary>
private bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result,
out string hash, out string hashPrefix, out int suppressedStreak, out bool comparedWithPreviousRun)
{
hash = ComputeCanonicalHash(result);
hashPrefix = hash.Length > 8 ? hash.Substring(0, 8) : hash;
suppressedStreak = 0;
comparedWithPreviousRun = false;
bool emit;
lock (_scopeLock)
{
@@ -693,11 +735,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
? (streak.Count + 1, streak.SinceUtc)
: (1, DateTime.UtcNow);
suppressedStreak = _suppressed[rule.RuleId].Count;
comparedWithPreviousRun = _hashFromPreviousRun.Contains(rule.RuleId);
emit = false;
}
else
{
_lastEmittedHash[rule.RuleId] = hash;
_hashFromPreviousRun.Remove(rule.RuleId);
(int Count, DateTime SinceUtc) streak;
if (_suppressed.TryGetValue(rule.RuleId, out streak) && streak.Count > 0)
{
@@ -820,6 +864,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
StopAllTimers();
_startupRulesLatch?.Dispose();
_startupRulesLatch = null;
// A rule still finishing on a pool thread keeps its in-memory state but no longer writes:
// the agent may be removing its own folder right now.
_state.Close();
}
}
}
@@ -0,0 +1,399 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
/// <summary>
/// Gather-rule state of one enrollment session that survives agent restarts: where each
/// logparser rule stopped reading each file, and the on_change value each rule last emitted.
/// <para>
/// Positions are kept per (rule, file) — two rules parsing the same log each read all of it.
/// Every position carries a fingerprint of the file's first bytes, so a file replaced under
/// the same name is read from the beginning even when it is already longer than the position.
/// </para>
/// <para>
/// Every commit is written to disk synchronously, AFTER the events it covers were handed on:
/// a process that dies in between repeats that one pass after the restart instead of losing it.
/// The lock guards memory and file write together and is never taken under the executor's
/// scope lock. Without a persistence (diagnostic mode, tests) the state lives in memory only.
/// </para>
/// </summary>
public sealed class GatherRuleSessionState
{
/// <summary>
/// Files remembered per rule. A run reads at most <see cref="LogParserCollector.MaxFilesPerRun"/>
/// files; the least recently touched entry beyond the cap is dropped, and its file — should it
/// come back — is simply read again.
/// </summary>
internal const int MaxLogFilesPerRule = 5 * LogParserCollector.MaxFilesPerRun;
private static readonly char[] WildcardChars = { '*', '?' };
private readonly object _gate = new object();
private readonly string _sessionId;
private readonly GatherRuleStatePersistence _persistence;
private readonly AgentLogger _logger;
private readonly Dictionary<string, Dictionary<string, Entry>> _logPositions =
new Dictionary<string, Dictionary<string, Entry>>(StringComparer.Ordinal);
private readonly Dictionary<string, string> _emittedHashes = new Dictionary<string, string>(StringComparer.Ordinal);
private readonly List<KeyValuePair<string, string>> _restoredDetails = new List<KeyValuePair<string, string>>();
private long _touchCounter;
private bool _unsaved;
private bool _saveFailureLogged;
private bool _closed;
private sealed class Entry
{
public long Position;
public long LineNumber;
public int HeadLength;
public string HeadHash;
public bool FromPreviousRun;
public long HeldTailStart = -1;
public long LastTouched;
}
public GatherRuleSessionState(string sessionId, GatherRuleStatePersistence persistence, AgentLogger logger)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_persistence = persistence;
RestoreSummary = Restore();
}
/// <summary>One line on what the constructor found on disk — for the agent log and the debug trace.</summary>
public string RestoreSummary { get; }
/// <summary>(ruleId, message) per restored position and on_change value — for the debug trace.</summary>
internal IReadOnlyList<KeyValuePair<string, string>> RestoredDetails => _restoredDetails;
// ===== logparser positions =====
/// <summary>The recorded position of <paramref name="filePath"/> for the rule, or null when it has none.</summary>
internal LogPositionSnapshot PeekLogPosition(string ruleId, string filePath)
{
lock (_gate)
{
Dictionary<string, Entry> files;
Entry entry;
if (!_logPositions.TryGetValue(ruleId, out files) || !files.TryGetValue(filePath, out entry))
return null;
entry.LastTouched = ++_touchCounter;
return new LogPositionSnapshot(entry.Position, entry.LineNumber, entry.HeadLength, entry.HeadHash,
entry.FromPreviousRun, entry.HeldTailStart);
}
}
/// <summary>
/// Records where the rule stopped in the file and saves the state when it changed.
/// <paramref name="heldTailStart"/> (in memory only) marks an unfinished last line left for
/// the next run; -1 when none.
/// </summary>
internal void CommitLogPosition(string ruleId, string filePath, long position, long lineNumber,
LogFileHead head, long heldTailStart)
{
var headLength = head.Length;
var headHash = headLength > 0 ? head.Hash(headLength) : null;
lock (_gate)
{
Dictionary<string, Entry> files;
if (!_logPositions.TryGetValue(ruleId, out files))
{
files = new Dictionary<string, Entry>(StringComparer.OrdinalIgnoreCase);
_logPositions[ruleId] = files;
}
Entry entry;
var changed = false;
if (!files.TryGetValue(filePath, out entry))
{
entry = new Entry();
files[filePath] = entry;
changed = true;
}
if (changed || entry.Position != position || entry.LineNumber != lineNumber
|| entry.HeadLength != headLength || !string.Equals(entry.HeadHash, headHash, StringComparison.Ordinal))
{
entry.Position = position;
entry.LineNumber = lineNumber;
entry.HeadLength = headLength;
entry.HeadHash = headHash;
entry.FromPreviousRun = false;
changed = true;
}
entry.HeldTailStart = heldTailStart;
entry.LastTouched = ++_touchCounter;
if (files.Count > MaxLogFilesPerRule)
EvictLeastRecentlyTouched(ruleId, files);
if (changed || _unsaved)
SaveLocked();
}
}
/// <summary>
/// Where to continue reading: the recorded position while the file is still the one it was
/// recorded for, otherwise 0 with the reason.
/// </summary>
internal static long ResolveStart(LogPositionSnapshot known, long fileLength, LogFileHead head, out LogReadStart start)
{
if (known == null)
{
start = LogReadStart.FirstRead;
return 0;
}
if (fileLength < known.Position)
{
start = LogReadStart.Shrunk;
return 0;
}
if (known.HeadLength > 0
&& (head.Length < known.HeadLength
|| !string.Equals(head.Hash(known.HeadLength), known.HeadHash, StringComparison.Ordinal)))
{
start = LogReadStart.Replaced;
return 0;
}
start = LogReadStart.Continue;
return known.Position;
}
// ===== on_change values =====
/// <summary>Rule id → hash of the result the rule last emitted.</summary>
internal Dictionary<string, string> GetEmittedHashes()
{
lock (_gate)
return new Dictionary<string, string>(_emittedHashes, StringComparer.Ordinal);
}
/// <summary>Records the hash of a result that was just emitted and saves the state when it changed.</summary>
internal void CommitEmittedHash(string ruleId, string hash)
{
lock (_gate)
{
string current;
var changed = !_emittedHashes.TryGetValue(ruleId, out current)
|| !string.Equals(current, hash, StringComparison.Ordinal);
_emittedHashes[ruleId] = hash;
if (changed || _unsaved)
SaveLocked();
}
}
/// <summary>Retries a save that failed earlier, then stops writing (executor disposed).</summary>
internal void Close()
{
lock (_gate)
{
if (_unsaved)
SaveLocked();
_closed = true;
}
}
// ===== persistence =====
private void SaveLocked()
{
if (_persistence == null || _closed)
{
_unsaved = false;
return;
}
var data = new GatherRuleStateData
{
SessionId = _sessionId,
SavedAtUtc = DateTime.UtcNow,
LogPositions = _logPositions
.SelectMany(rule => rule.Value.Select(file => new { RuleId = rule.Key, Path = file.Key, Entry = file.Value }))
.OrderBy(x => x.Entry.LastTouched)
.Select(x => new GatherLogPositionData
{
RuleId = x.RuleId,
Path = x.Path,
Position = x.Entry.Position,
LineNumber = x.Entry.LineNumber,
HeadLength = x.Entry.HeadLength,
HeadHash = x.Entry.HeadHash,
})
.ToList(),
OnChangeHashes = new Dictionary<string, string>(_emittedHashes, StringComparer.Ordinal),
};
if (_persistence.Save(data))
{
_unsaved = false;
_saveFailureLogged = false;
return;
}
_unsaved = true;
if (!_saveFailureLogged)
{
_saveFailureLogged = true;
_logger.Warning($"GatherRuleSessionState: saving {_persistence.StateFilePath} failed — kept in memory, retried at the next change");
}
}
private string Restore()
{
if (_persistence == null)
return "persistence off (no state directory) — positions and on_change values live in memory only";
string discardReason;
var data = _persistence.Load(_sessionId, out discardReason);
if (data == null)
{
return discardReason == null
? "no saved state — first agent run of this session"
: $"saved state discarded ({discardReason}) — starting fresh";
}
// The file is input like any other: whatever it holds, the executor must come up. A
// failure here would otherwise stop every collector host, at every restart of the session.
try
{
return RestoreFrom(data);
}
catch (Exception ex)
{
_logPositions.Clear();
_emittedHashes.Clear();
_restoredDetails.Clear();
_logger.Warning($"GatherRuleSessionState: restoring {_persistence.StateFilePath} failed, starting fresh: {ex.Message}");
return $"saved state discarded (restore failed: {ex.Message}) — starting fresh";
}
}
private string RestoreFrom(GatherRuleStateData data)
{
var dropped = 0;
foreach (var position in data.LogPositions ?? new List<GatherLogPositionData>())
{
if (!IsValid(position))
{
dropped++;
continue;
}
Dictionary<string, Entry> files;
if (!_logPositions.TryGetValue(position.RuleId, out files))
{
files = new Dictionary<string, Entry>(StringComparer.OrdinalIgnoreCase);
_logPositions[position.RuleId] = files;
}
files[position.Path] = new Entry
{
Position = position.Position,
LineNumber = position.LineNumber,
HeadLength = position.HeadLength,
HeadHash = position.HeadLength > 0 ? position.HeadHash : null,
FromPreviousRun = true,
LastTouched = ++_touchCounter,
};
if (files.Count > MaxLogFilesPerRule)
EvictLeastRecentlyTouched(position.RuleId, files);
}
foreach (var hash in data.OnChangeHashes ?? new Dictionary<string, string>())
{
if (string.IsNullOrEmpty(hash.Key) || string.IsNullOrEmpty(hash.Value))
{
dropped++;
continue;
}
_emittedHashes[hash.Key] = hash.Value;
}
foreach (var rule in _logPositions)
{
foreach (var file in rule.Value)
{
_restoredDetails.Add(new KeyValuePair<string, string>(rule.Key,
$"logparser position restored: {Path.GetFileName(file.Key)} at byte {file.Value.Position}, after line {file.Value.LineNumber}"));
}
}
foreach (var hash in _emittedHashes)
{
_restoredDetails.Add(new KeyValuePair<string, string>(hash.Key,
$"on_change value restored (hash {Prefix(hash.Value)}) — an unchanged result stays silent"));
}
var positionCount = _logPositions.Sum(rule => rule.Value.Count);
return $"restored {positionCount} logparser position(s) for {_logPositions.Count} rule(s) and {_emittedHashes.Count} on_change value(s), " +
$"saved {data.SavedAtUtc:yyyy-MM-ddTHH:mm:ssZ} by a previous agent run of this session" +
(dropped > 0 ? $"; {dropped} invalid entr{(dropped == 1 ? "y" : "ies")} dropped" : "") +
" — logparser rules continue after their saved position, on_change rules stay silent until their result changes";
}
private static bool IsValid(GatherLogPositionData position)
{
if (position == null || string.IsNullOrEmpty(position.RuleId) || string.IsNullOrEmpty(position.Path))
return false;
// A real resolved path never carries these; Path APIs throw on them under .NET Framework.
if (position.Path.IndexOfAny(Path.GetInvalidPathChars()) >= 0 || position.Path.IndexOfAny(WildcardChars) >= 0)
return false;
if (position.Position < 0 || position.LineNumber < 0)
return false;
if (position.HeadLength < 0 || position.HeadLength > LogFileHead.MaxBytes)
return false;
if (position.HeadLength > 0 && string.IsNullOrEmpty(position.HeadHash))
return false;
// Reading any byte records a non-empty head; a position without one cannot be verified.
return position.Position == 0 || position.HeadLength > 0;
}
private void EvictLeastRecentlyTouched(string ruleId, Dictionary<string, Entry> files)
{
var oldest = files.OrderBy(file => file.Value.LastTouched).First();
files.Remove(oldest.Key);
_logger.Info($"GatherRuleSessionState: rule {ruleId} tracks more than {MaxLogFilesPerRule} files — forgot the position of {Path.GetFileName(oldest.Key)}");
}
private static string Prefix(string hash) => hash.Length > 8 ? hash.Substring(0, 8) : hash;
}
/// <summary>Immutable copy of a recorded logparser position.</summary>
internal sealed class LogPositionSnapshot
{
public LogPositionSnapshot(long position, long lineNumber, int headLength, string headHash,
bool fromPreviousRun, long heldTailStart)
{
Position = position;
LineNumber = lineNumber;
HeadLength = headLength;
HeadHash = headHash;
FromPreviousRun = fromPreviousRun;
HeldTailStart = heldTailStart;
}
public long Position { get; }
public long LineNumber { get; }
public int HeadLength { get; }
public string HeadHash { get; }
/// <summary>Restored from disk and not advanced by this process yet.</summary>
public bool FromPreviousRun { get; }
/// <summary>Start of an unfinished last line left for this run by the previous one; -1 when none.</summary>
public long HeldTailStart { get; }
}
/// <summary>How <see cref="GatherRuleSessionState.ResolveStart"/> decided where reading starts.</summary>
internal enum LogReadStart
{
FirstRead,
Continue,
Shrunk,
Replaced,
}
}
@@ -0,0 +1,131 @@
using System;
using System.Collections.Generic;
using System.IO;
using AutopilotMonitor.Agent.V2.Core.Logging;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
/// <summary>
/// Persists <see cref="GatherRuleSessionState"/> in the agent's state directory so one
/// enrollment's logparser positions and on_change values survive agent restarts, reboots and
/// the WhiteGlove Part 2 resume.
/// <para>
/// The state is only resumed for the SAME session id: the state directory is one fixed folder
/// that a session rotation does not clear, and the new session starts empty in the backend, so
/// it must read its logs from the beginning.
/// </para>
/// Fail-soft like <c>BandwidthStatePersistence</c>: I/O errors never throw; a missing, corrupt
/// or foreign file loads as null (fresh start — the pre-persistence behavior).
/// </summary>
public class GatherRuleStatePersistence
{
public const string FileName = "gather-rule-state.json";
private const string SourceName = "GatherRuleStatePersistence";
private readonly string _stateDirectory;
private readonly string _stateFilePath;
private readonly AgentLogger _logger;
public GatherRuleStatePersistence(string stateDirectory, AgentLogger logger)
{
_stateDirectory = Environment.ExpandEnvironmentVariables(stateDirectory);
_stateFilePath = Path.Combine(_stateDirectory, FileName);
_logger = logger;
}
public string StateFilePath => _stateFilePath;
/// <summary>
/// Loads the persisted state, or null when absent, unreadable, or written for a different
/// session. <paramref name="discardReason"/> is null when there simply was no file.
/// </summary>
public GatherRuleStateData Load(string expectedSessionId, out string discardReason)
{
discardReason = null;
if (!File.Exists(_stateFilePath)) return null;
try
{
var json = File.ReadAllText(_stateFilePath);
var state = JsonConvert.DeserializeObject<GatherRuleStateData>(json);
if (state == null || string.IsNullOrEmpty(state.SessionId))
{
discardReason = "state file empty or invalid";
_logger.Warning($"[{SourceName}] persisted state file was empty or invalid — starting fresh");
return null;
}
if (!string.Equals(state.SessionId, expectedSessionId, StringComparison.OrdinalIgnoreCase))
{
discardReason = $"saved for session {state.SessionId}, current session is {expectedSessionId}";
_logger.Warning($"[{SourceName}] persisted state belongs to session {state.SessionId}, not {expectedSessionId} — starting fresh");
return null;
}
return state;
}
catch (Exception ex)
{
discardReason = $"state file unreadable: {ex.Message}";
_logger.Warning($"[{SourceName}] failed to load persisted state, starting fresh: {ex.Message}");
return null;
}
}
/// <summary>Persists the state (atomic write via temp file). Never throws; false when the write failed.</summary>
public bool Save(GatherRuleStateData state)
{
try
{
Directory.CreateDirectory(_stateDirectory);
var tempPath = _stateFilePath + ".tmp";
File.WriteAllText(tempPath, JsonConvert.SerializeObject(state));
if (File.Exists(_stateFilePath))
{
File.Replace(tempPath, _stateFilePath, destinationBackupFileName: null, ignoreMetadataErrors: true);
}
else
{
File.Move(tempPath, _stateFilePath);
}
return true;
}
catch (Exception ex)
{
_logger.Debug($"[{SourceName}] failed to save state: {ex.Message}");
return false;
}
}
}
// ---- DTOs for state serialization ----
public class GatherRuleStateData
{
public string SessionId { get; set; }
public DateTime SavedAtUtc { get; set; }
/// <summary>Least recently used first, so a restore keeps the eviction order.</summary>
public List<GatherLogPositionData> LogPositions { get; set; }
/// <summary>Rule id → canonical hash of the result the rule last emitted (on_change).</summary>
public Dictionary<string, string> OnChangeHashes { get; set; }
}
public class GatherLogPositionData
{
public string RuleId { get; set; }
public string Path { get; set; }
/// <summary>File offset right behind the last line the rule processed.</summary>
public long Position { get; set; }
/// <summary>Lines before <see cref="Position"/> — the base for <c>logLineNumber</c>.</summary>
public long LineNumber { get; set; }
/// <summary>Bytes from the start of the file the fingerprint covers (0..4096).</summary>
public int HeadLength { get; set; }
/// <summary>SHA-256 of those bytes, Base64; null when <see cref="HeadLength"/> is 0.</summary>
public string HeadHash { get; set; }
}
}
@@ -215,6 +215,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Periodic
data["ime_verified_bytes"] = health.VerifiedBytes;
data["ime_pass_max_ms"] = health.PassMaxMs;
data["ime_pass_gap_max_ms"] = health.PassGapMaxMs;
data["ime_platform_output_verified"] = health.PlatformOutputVerified;
data["ime_platform_output_corrected"] = health.PlatformOutputCorrected;
}
}
catch (Exception ex)
@@ -132,12 +132,21 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
_informationalEvents = informationalEvents;
_parkedTripwireDwell = parkedTripwireDwell ?? DefaultParkedTripwireDwell;
// If recovery loaded a state that already sits on a terminal stage (e.g. a crash
// after a success-path step but before Stop()), treat it as already-notified so we
// do not re-fire the hook.
_terminalNotified = initialState.Stage.IsTerminal();
// If recovery loaded a state that already ends the session (e.g. a crash after a
// success-path step but before Stop()), treat it as already-notified so we do not
// re-fire the hook — the orchestrator handles that state at start.
_terminalNotified = EndsSession(initialState);
}
/// <summary>
/// True when the state ends this agent's session: a terminal stage, or the engine stopped
/// waiting for the user without a verdict (<see cref="DecisionState.StoppedWaitingReason"/>,
/// D-310) — the stage stays non-terminal there, and the agent ends the session like its
/// max-lifetime watchdog.
/// </summary>
internal static bool EndsSession(DecisionState state) =>
state.Stage.IsTerminal() || state.StoppedWaitingReason != null;
public DecisionState CurrentState => _currentState;
/// <summary>Test-Observability — anzahl aufeinanderfolgender Journal-Failures.</summary>
@@ -284,7 +293,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// orchestrator turns this into the public EnrollmentTerminated event so peripheral
// consumers (CleanupService, SummaryDialog, DiagnosticsPackageService) can react
// without touching the kernel state machine.
if (!_terminalNotified && _currentState.Stage.IsTerminal())
if (!_terminalNotified && EndsSession(_currentState))
{
_terminalNotified = true;
try { _onTerminalStageReached?.Invoke(_currentState); }
@@ -405,7 +414,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
/// </summary>
private static bool IsParkedWithoutDeadline(DecisionState state)
{
if (state.Stage.IsTerminal()) return false;
if (EndsSession(state)) return false;
if (state.Stage == SessionStage.Unknown || state.Stage == SessionStage.SessionStarted) return false;
// WDP (afee7ae0): Device Preparation has no ESP, so neither the AccountSetup
@@ -479,6 +479,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
if (collectors.StallProbeEnabled)
{
// A pending tenant Hello wait (up to 60 min) is a bounded wait that resolves the
// session either way — the probe holds session_stalled back until it is over.
Func<bool>? helloWaitPending = null;
if (decisionStateProbe != null)
helloWaitPending = () => DecisionEngine.IsTenantHelloWaitPending(decisionStateProbe(), clock.UtcNow);
hosts.Add(new StallProbeHost(
sessionId: sessionId,
tenantId: tenantId,
@@ -490,7 +496,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
sources: collectors.StallProbeSources,
sessionStalledAfterProbeIndex: collectors.SessionStalledAfterProbeIndex,
harmlessModernDeploymentEventIds: collectors.ModernDeploymentHarmlessEventIds,
isDevicePreparation: isDevicePreparation));
isDevicePreparation: isDevicePreparation,
helloWaitPending: helloWaitPending));
}
// ESP policy-provider stall tripwire — always-on kernel host (no config gate, like
@@ -517,14 +524,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// EspTrackingWin32Apps, StatusServiceReports) via snapshot-and-diff. Emits
// registry_app_state on real changes and app_state_reconciliation when the
// registry outcome diverges from the IME-log-derived state (= built-in pattern-drift
// alarm). Always-on observability host; the tracker probe is read-only.
// alarm). Always-on observability host; the tracker probe is read-only. Under the same
// root IME saves each platform script's result after reporting the batch: the script
// observer checks the tracker's emitted runs against it (script_output_reconciliation).
hosts.Add(new ImeRegistryAppStateHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
trackerStateProbe: () => imeLogHost.AllKnownPackageStates));
trackerStateProbe: () => imeLogHost.AllKnownPackageStates,
scriptRuns: imeLogHost.Tracker.ScriptRuns));
// Deterministic update corroboration — compares the persisted OS build (CurrentBuild.UBR)
// across agent restarts and emits os_build_changed when it differs (session 7443317c:
@@ -540,9 +550,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
stateDirectory: _stateDirectory);
hosts.Add(osBuildChangeHost);
// Windows Update during OOBE watcher — subscribes to WindowsUpdateClient/Operational and
// backfills recent events (OOBE quality updates run before the agent starts). Surfaces
// quality/cumulative updates installing/failing DURING enrollment — otherwise invisible.
// OS update watchers (D-310) — the Windows Update client on the System log (install
// events) and its Operational channel (scan/download), plus the CBS servicing steps on
// the Setup log; each backfills recent events because updates can start before the
// agent does. Surfaces Windows/.NET updates installing/failing DURING enrollment and,
// unless OobeUpdatePhaseMode is Off, feeds them into the engine's OOBE update phase.
if (collectors.WindowsUpdateWatcherEnabled)
{
hosts.Add(new WindowsUpdateWatcherHost(
@@ -555,7 +567,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
backfillLookbackMinutes: collectors.WindowsUpdateBackfillLookbackMinutes,
stateDirectory: _stateDirectory,
channelCensusEnabled: collectors.WindowsUpdateChannelCensusEnabled,
osBuildChangedProvider: () => osBuildChangeHost.BuildChanged));
osBuildChangedProvider: () => osBuildChangeHost.BuildChanged,
operationalEventIds: collectors.WindowsUpdateOperationalEventIds,
servicingWatcherEnabled: collectors.ServicingWatcherEnabled,
osUpdateSignalsEnabled: DecisionCore.State.OobeUpdatePhaseModes.PostsSignals(collectors.OobeUpdatePhaseMode)));
}
// MDM reboot-policy watcher — subscribes to DeviceManagement-Enterprise-Diagnostics-
@@ -729,7 +744,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
imeLogPathOverride: _agentConfig.ImeLogPathOverride,
unrestrictedMode: _agentConfig.UnrestrictedMode,
gatherDebugLogPath: _agentConfig.GatherRuleDebugLogPath,
timelineEvents: timelineEvents));
timelineEvents: timelineEvents,
stateDirectory: _stateDirectory));
}
else if (!string.IsNullOrEmpty(_agentConfig.GatherRuleDebugLogPath))
{
@@ -740,7 +740,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// itself is constructed inside the onIngressReady hook (step 13b), which has
// already fired by this point — so when our recovery-fire propagates through
// Task.Run, the handler is fully wired.
if (initialState.Stage == SessionStage.Completed || initialState.Stage == SessionStage.Failed)
// The same applies to a state in which the engine stopped waiting (D-310): a
// crash between that step and the termination must not leave the session running.
if (initialState.Stage == SessionStage.Completed || initialState.Stage == SessionStage.Failed
|| initialState.StoppedWaitingReason != null)
{
_logger.Warning(
$"EnrollmentOrchestrator: post-recovery initialState already terminal " +
@@ -869,25 +872,40 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// Stop the max-lifetime watchdog — the real terminal arrived before it could trip.
try { _maxLifetimeTimer?.Dispose(); _maxLifetimeTimer = null; } catch { }
var outcome = terminalState.Stage switch
{
SessionStage.Completed or SessionStage.WhiteGloveSealed
=> EnrollmentTerminationOutcome.Succeeded,
SessionStage.Failed => EnrollmentTerminationOutcome.Failed,
_ => EnrollmentTerminationOutcome.TimedOut,
};
// D-310: the engine stopped waiting for the user without a verdict. The stage stays
// non-terminal; the session ends like the max-lifetime watchdog ends it (TimedOut), so
// the backend classifies it honestly — never as a failure.
var stoppedWaiting = !terminalState.Stage.IsTerminal() && terminalState.StoppedWaitingReason != null;
var outcome = stoppedWaiting
? EnrollmentTerminationOutcome.TimedOut
: terminalState.Stage switch
{
SessionStage.Completed or SessionStage.WhiteGloveSealed
=> EnrollmentTerminationOutcome.Succeeded,
SessionStage.Failed => EnrollmentTerminationOutcome.Failed,
_ => EnrollmentTerminationOutcome.TimedOut,
};
var reason = stoppedWaiting
? EnrollmentTerminationReason.StoppedWaiting
: EnrollmentTerminationReason.DecisionTerminalStage;
_logger.Info(
$"EnrollmentOrchestrator: decision terminal stage reached (stage={terminalState.Stage}, outcome={outcome}) — dispatching Terminated(DecisionTerminalStage) off worker.");
$"EnrollmentOrchestrator: decision ended the session (stage={terminalState.Stage}, outcome={outcome}, reason={reason}) — dispatching Terminated off worker.");
string? details = null;
if (stoppedWaiting)
details = StoppedWaitingReasons.Describe(terminalState.StoppedWaitingReason!.Value);
else if (terminalState.Stage.IsPauseBeforePart2())
details = "WhiteGlove Part 1 sealed — session will resume on Part 2 post-reboot; self-destruct suppressed.";
var terminatedArgs = new EnrollmentTerminatedEventArgs(
reason: EnrollmentTerminationReason.DecisionTerminalStage,
reason: reason,
outcome: outcome,
stageName: terminalState.Stage.ToString(),
terminatedAtUtc: _clock.UtcNow,
details: terminalState.Stage.IsPauseBeforePart2()
? "WhiteGlove Part 1 sealed — session will resume on Part 2 post-reboot; self-destruct suppressed."
: null);
details: details,
stopReason: stoppedWaiting ? terminalState.StoppedWaitingReason!.Value : null);
// Codex Finding 2 — off-worker dispatch. Task.Run unblocks the ingress worker
// immediately; the worker keeps draining the channel (including events the
@@ -16,6 +16,13 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
/// longer than the configured cap without reaching a terminal stage.
/// </summary>
MaxLifetimeExceeded,
/// <summary>
/// The DecisionEngine stopped waiting for the user without a verdict — a bounded wait
/// expired (<c>DecisionState.StoppedWaitingReason</c>, D-310). Ends the session like
/// <see cref="MaxLifetimeExceeded"/> (outcome TimedOut), with the engine's reason.
/// </summary>
StoppedWaiting,
}
/// <summary>
@@ -60,18 +67,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
public DateTime TerminatedAtUtc { get; }
public string? Details { get; }
/// <summary>
/// The engine's reason code for <see cref="EnrollmentTerminationReason.StoppedWaiting"/>
/// (e.g. <c>oobe_update_no_sign_in</c>); null for every other reason.
/// </summary>
public string? StopReason { get; }
public EnrollmentTerminatedEventArgs(
EnrollmentTerminationReason reason,
EnrollmentTerminationOutcome outcome,
string? stageName,
DateTime terminatedAtUtc,
string? details = null)
string? details = null,
string? stopReason = null)
{
Reason = reason;
Outcome = outcome;
StageName = stageName;
TerminatedAtUtc = terminatedAtUtc;
Details = details;
StopReason = stopReason;
}
}
}
@@ -23,7 +23,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// V1 fired these from MonitoringService on every emitted EnrollmentEvent; the first V2
// wiring subscribed to SignalIngress.SignalPosted instead, which fires at enqueue time
// with the RAW collector payload — before the reducer has applied gates like the
// RealmJoin completion gate. Consequence (session 32312a32, rsneuffen.de): a phase_change
// RealmJoin completion gate. Consequence (session 32312a32): a phase_change
// rule on FinalizingSetup fired at the raw EspPhaseChanged(FinalizingSetup) signal (ESP
// exit / Hello wizard), 7 minutes before the engine declared phase_transition(FinalizingSetup)
// on the timeline — and before the RealmJoin package wrote the registry key the rule was
@@ -47,7 +47,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
string? imeLogPathOverride,
bool unrestrictedMode = false,
string? gatherDebugLogPath = null,
TimelineEventStream? timelineEvents = null)
TimelineEventStream? timelineEvents = null,
string? stateDirectory = null)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
@@ -66,7 +67,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
var post = new InformationalEventPost(ingress, clock, logger);
_executor = new Monitoring.Telemetry.Gather.GatherRuleExecutor(
sessionId, tenantId, evt => post.Emit(evt), logger, imeLogPathOverride,
debugLogPath: gatherDebugLogPath);
debugLogPath: gatherDebugLogPath, stateDirectory: stateDirectory);
_timelineEvents = timelineEvents;
}
@@ -15,9 +15,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Hosts the <see cref="ImeRegistryAppStateObserver"/> (registry second pillar, audit
/// 2026-08-17): one recursive <see cref="RegistryWatcher"/> on the IME root — all three
/// observed surfaces (Win32Apps, EspTrackingWin32Apps, SideCarPolicies\StatusServiceReports)
/// live under it — debounced into snapshot-and-diff ticks, plus a 60-s periodic fallback
/// 2026-08-17) and the <see cref="ImeRegistryScriptResultObserver"/> (D-316): one recursive
/// <see cref="RegistryWatcher"/> on the IME root — every observed surface (Win32Apps,
/// EspTrackingWin32Apps, SideCarPolicies\StatusServiceReports, Policies) lives under it —
/// debounced into snapshot-and-diff ticks, plus a 60-s periodic fallback
/// tick. The fallback both evaluates the reconciliation settle-delay when the registry goes
/// quiet AND keeps the pillar alive if the watcher fails to arm (poll-only degraded mode,
/// announced once via <c>collector_degraded</c>). Always-on observability host, no config
@@ -31,6 +32,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
private static readonly TimeSpan PeriodicInterval = TimeSpan.FromSeconds(60);
private readonly ImeRegistryAppStateObserver _observer;
// Platform-script results under the same root (D-316); null when no run register was supplied.
private readonly ImeRegistryScriptResultObserver? _scriptObserver;
private readonly InformationalEventPost _post;
private readonly AgentLogger? _logger;
private readonly string _sessionId;
@@ -47,7 +50,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
AgentLogger? logger,
ISignalIngressSink ingress,
IClock clock,
Func<IReadOnlyList<AppPackageState>>? trackerStateProbe)
Func<IReadOnlyList<AppPackageState>>? trackerStateProbe,
PlatformScriptRunRegister? scriptRuns = null)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
@@ -56,6 +60,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
_logger = logger;
_post = new InformationalEventPost(ingress, clock);
_observer = new ImeRegistryAppStateObserver(_post, logger, clock, trackerStateProbe);
if (scriptRuns != null) _scriptObserver = new ImeRegistryScriptResultObserver(scriptRuns, _post, logger, clock);
}
public void Start()
@@ -63,6 +68,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
// Baseline immediately: pre-existing registry state must be captured silently
// BEFORE the first change edge, or stale entries would replay as fresh events.
_observer.Tick("baseline");
_scriptObserver?.Tick("baseline");
try
{
@@ -120,6 +126,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
try
{
_observer.Tick(reason);
_scriptObserver?.Tick(reason);
}
catch (Exception ex)
{
@@ -58,7 +58,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
string[]? sources,
int sessionStalledAfterProbeIndex,
int[]? harmlessModernDeploymentEventIds,
bool isDevicePreparation = false)
bool isDevicePreparation = false,
Func<bool>? helloWaitPending = null)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
@@ -74,7 +75,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
sources: sources ?? new[] { "provisioning_registry", "diagnostics_registry", "eventlog", "appworkload_log" },
sessionStalledAfterProbeIndex: sessionStalledAfterProbeIndex,
harmlessModernDeploymentEventIds: harmlessModernDeploymentEventIds,
isDevicePreparation: isDevicePreparation);
isDevicePreparation: isDevicePreparation,
helloWaitPending: helloWaitPending);
_adapter = new StallProbeCollectorAdapter(_collector, ingress, clock);
_observableIngress = ingress as SignalIngress;
@@ -1,26 +1,41 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using AutopilotMonitor.Shared.Models;
using SharedConstants = AutopilotMonitor.Shared.Constants;
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Standalone host for the <see cref="WindowsUpdateTracker"/> — watches
/// <c>Microsoft-Windows-WindowsUpdateClient/Operational</c> for quality/cumulative update
/// activity during OOBE / ESP and forwards it via the shared single-rail ingress.
/// Mirrors <see cref="StallProbeHost"/>: builds its own <see cref="InformationalEventPost"/>
/// and delegates Start/Stop to the tracker.
/// Hosts the OS-update watchers (D-310): the Windows Update client on the System log
/// (install events) and on its Operational channel (scan/download events), plus the CBS
/// servicing steps on the Setup log. Routes every configured WU EventID to the channel the
/// provider manifest assigns it, runs the blind-spot census across all channels, reports the
/// counted (non-OS) activity once on stop, and — unless <c>OobeUpdatePhaseMode</c> is Off —
/// feeds Windows/.NET update steps into the decision engine as
/// <see cref="DecisionSignalKind.OsUpdateActivity"/>.
/// </summary>
internal sealed class WindowsUpdateWatcherHost : ICollectorHost
{
public string Name => "WindowsUpdateTracker";
private readonly WindowsUpdateTracker _tracker;
private readonly WindowsUpdateTracker? _systemTracker;
private readonly WindowsUpdateTracker? _operationalTracker;
private readonly ServicingTracker? _servicingTracker;
private readonly UpdateActivityTally _tally = new UpdateActivityTally();
private readonly InformationalEventPost _post;
private readonly ISignalIngressSink _ingress;
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private int _summaryEmitted;
private int _disposed;
public WindowsUpdateWatcherHost(
@@ -33,36 +48,125 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
int backfillLookbackMinutes,
string stateDirectory,
bool channelCensusEnabled = true,
Func<bool>? osBuildChangedProvider = null)
Func<bool>? osBuildChangedProvider = null,
int[]? operationalEventIds = null,
bool servicingWatcherEnabled = true,
bool osUpdateSignalsEnabled = false)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
_ingress = ingress ?? throw new ArgumentNullException(nameof(ingress));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = new InformationalEventPost(ingress, clock, logger);
var post = new InformationalEventPost(ingress, clock, logger);
_tracker = new WindowsUpdateTracker(
sessionId: sessionId,
tenantId: tenantId,
post: post,
logger: logger,
targetedEventIds: targetedEventIds,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory,
channelCensusEnabled: channelCensusEnabled,
osBuildChangedProvider: osBuildChangedProvider);
var (systemIds, operationalIds) = RouteEventIds(targetedEventIds, operationalEventIds);
Action<string, string, DateTime>? wuActivity = null;
Action<string, string, DateTime>? servicingActivity = null;
if (osUpdateSignalsEnabled)
{
wuActivity = (step, update, at) => PostOsUpdateActivity(step, "wu", update, at);
servicingActivity = (step, update, at) => PostOsUpdateActivity(step, "servicing", update, at);
}
if (systemIds.Length > 0)
{
_systemTracker = new WindowsUpdateTracker(
sessionId: sessionId,
tenantId: tenantId,
post: _post,
logger: logger,
targetedEventIds: systemIds,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory,
channelCensusEnabled: false,
channel: WindowsUpdateTracker.SystemChannel,
tally: _tally,
onOsUpdateActivity: wuActivity);
}
if (operationalIds.Length > 0)
{
_operationalTracker = new WindowsUpdateTracker(
sessionId: sessionId,
tenantId: tenantId,
post: _post,
logger: logger,
targetedEventIds: operationalIds,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory,
channelCensusEnabled: channelCensusEnabled,
osBuildChangedProvider: osBuildChangedProvider,
channel: WindowsUpdateTracker.Channel,
tally: _tally,
onOsUpdateActivity: wuActivity);
}
if (servicingWatcherEnabled)
{
_servicingTracker = new ServicingTracker(
sessionId: sessionId,
tenantId: tenantId,
post: _post,
logger: logger,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory,
tally: _tally,
onOsUpdateActivity: servicingActivity);
}
}
// Test seams — expose the trackers so unit tests can drive their ProcessEvent seams.
internal WindowsUpdateTracker? SystemTrackerForTest => _systemTracker;
internal WindowsUpdateTracker? OperationalTrackerForTest => _operationalTracker;
internal ServicingTracker? ServicingTrackerForTest => _servicingTracker;
/// <summary>
/// Splits the configured WU EventIDs by the channel the provider manifest writes them to.
/// An ID configured in either list ends up on its real channel, so a misplaced ID still
/// works instead of watching a channel that never carries it.
/// </summary>
internal static (int[] systemIds, int[] operationalIds) RouteEventIds(int[]? targetedEventIds, int[]? operationalEventIds)
{
var all = (targetedEventIds ?? Array.Empty<int>())
.Concat(operationalEventIds ?? Array.Empty<int>())
.Distinct()
.OrderBy(id => id)
.ToArray();
return (
all.Where(WindowsUpdateTracker.IsSystemChannelEventId).ToArray(),
all.Where(id => !WindowsUpdateTracker.IsSystemChannelEventId(id)).ToArray());
}
public void Start()
{
_tracker.Start();
_systemTracker?.Start();
_operationalTracker?.Start();
_servicingTracker?.Start();
// The census asks "did ANY channel see the update?" — only answerable after every
// backfill ran.
_operationalTracker?.EmitChannelCensusIfBlind(
otherChannelsEmitted: _systemTracker?.EmittedThisRun ?? 0,
otherTargetedEventIds: _systemTracker?.TargetedEventIds);
_logger.Info("WindowsUpdateWatcherHost: started.");
}
public void Stop()
{
try { _tracker.Stop(); }
catch (Exception ex) { _logger.Warning($"WindowsUpdateWatcherHost: stop failed: {ex.Message}"); }
try { _systemTracker?.Stop(); }
catch (Exception ex) { _logger.Warning($"WindowsUpdateWatcherHost: system watcher stop failed: {ex.Message}"); }
try { _operationalTracker?.Stop(); }
catch (Exception ex) { _logger.Warning($"WindowsUpdateWatcherHost: operational watcher stop failed: {ex.Message}"); }
try { _servicingTracker?.Stop(); }
catch (Exception ex) { _logger.Warning($"WindowsUpdateWatcherHost: servicing watcher stop failed: {ex.Message}"); }
EmitActivitySummaryOnce();
}
public void Dispose()
@@ -70,5 +174,61 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
if (Interlocked.Exchange(ref _disposed, 1) == 1) return;
Stop();
}
/// <summary>
/// One summary of everything that was counted instead of emitted. Stop runs twice on the
/// termination path (explicit stop, then dispose), hence the once-guard.
/// </summary>
internal void EmitActivitySummaryOnce()
{
if (_tally.IsEmpty) return;
if (Interlocked.Exchange(ref _summaryEmitted, 1) == 1) return;
try
{
var counts = _tally.Snapshot();
var data = new Dictionary<string, object>(StringComparer.Ordinal);
foreach (var kv in counts) data[kv.Key] = kv.Value;
var total = counts.Sum(kv => kv.Value);
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
EventType = SharedConstants.EventTypes.WindowsUpdateActivitySummary,
Severity = EventSeverity.Debug,
Source = SharedConstants.EventSources.WindowsUpdateWatcher,
Phase = EnrollmentPhase.Unknown,
Message = $"Update activity counted instead of listed: {total} event(s) " +
"(Store, Defender and other non-OS updates, repeated scan results, servicing of non-OS packages)",
Data = data,
ImmediateUpload = false,
});
}
catch (Exception ex)
{
_logger.Warning($"WindowsUpdateWatcherHost: activity summary failed: {ex.Message}");
}
}
private void PostOsUpdateActivity(string step, string source, string update, DateTime occurredAtUtc)
{
var payload = new Dictionary<string, string>(StringComparer.Ordinal)
{
[OsUpdatePayloadKeys.Step] = step,
[OsUpdatePayloadKeys.Source] = source,
};
if (!string.IsNullOrEmpty(update)) payload[OsUpdatePayloadKeys.Update] = update;
_ingress.Post(
kind: DecisionSignalKind.OsUpdateActivity,
occurredAtUtc: DateTime.SpecifyKind(occurredAtUtc, DateTimeKind.Utc),
sourceOrigin: source == "servicing" ? SharedConstants.EventSources.ServicingWatcher : SharedConstants.EventSources.WindowsUpdateWatcher,
evidence: new Evidence(
kind: EvidenceKind.Raw,
identifier: $"os-update-activity:{source}",
summary: string.IsNullOrEmpty(update) ? step : $"{step}: {update}"),
payload: payload);
}
}
}
@@ -92,6 +92,15 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
SharedConstants.EventTypes.ImePatternHits,
// Bookkeeping of runs that were deliberately not reported as activity.
SharedConstants.EventTypes.ScriptRecurrenceSummary,
// A correction of an already reported script run from IME's saved result (D-316):
// registry observation after the fact, not progress.
SharedConstants.EventTypes.ScriptOutputReconciliation,
// Same for the update watchers' tally of Store/Defender/other update events (D-310):
// emitted on stop, never progress. Windows/.NET update steps stay activity.
SharedConstants.EventTypes.WindowsUpdateActivitySummary,
// The OOBE update registry snapshot (D-310) is observation at agent start/stop and at
// the update page — the page's own records (oobe_update_page) stay activity.
SharedConstants.EventTypes.OobeUpdateState,
};
/// <summary>
@@ -1263,6 +1263,8 @@ namespace AutopilotMonitor.Agent.V2.Core.SignalAdapters
// logged its result line before the deadline). Lets the UI/MCP flag fallback-grounded
// platform-script completions. Platform scripts only — null for health scripts.
if (!string.IsNullOrEmpty(script.ResultSource)) data["resultSource"] = script.ResultSource!;
// The run a later script_output_reconciliation corrects names it by this id (platform only).
if (!string.IsNullOrEmpty(script.RunId)) data["runId"] = script.RunId!;
if (!string.IsNullOrEmpty(script.ComplianceResult)) data["complianceResult"] = script.ComplianceResult!;
if (script.RemediationStatus.HasValue) data["remediationStatus"] = script.RemediationStatus.Value.ToString(culture);
if (script.TargetType.HasValue) data["targetType"] = script.TargetType.Value.ToString(culture);
@@ -599,6 +599,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
["verifiedBytes"] = health.VerifiedBytes,
["passMaxMs"] = health.PassMaxMs,
["passGapMaxMs"] = health.PassGapMaxMs,
["platformOutputVerified"] = health.PlatformOutputVerified,
["platformOutputCorrected"] = health.PlatformOutputCorrected,
["agentVersion"] = _agentVersion ?? string.Empty,
};
if (!string.IsNullOrEmpty(health.ImeAgentVersion))
@@ -1227,11 +1229,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
case EnrollmentTerminationReason.MaxLifetimeExceeded:
reasonTag = "max_lifetime";
break;
case EnrollmentTerminationReason.StoppedWaiting:
// The backend classifies this shutdown honestly, exactly like max_lifetime.
reasonTag = "stopped_waiting";
break;
default:
reasonTag = args.Reason.ToString();
break;
}
var data = new Dictionary<string, object>
{
{ "reason", reasonTag },
{ "outcome", args.Outcome.ToString() },
{ "stage", args.StageName ?? string.Empty },
{ "uptimeMinutes", uptimeMinutes },
{ "agentVersion", _agentVersion },
};
if (!string.IsNullOrEmpty(args.StopReason)) data["stopReason"] = args.StopReason;
EmitEventSafe(new EnrollmentEvent
{
SessionId = _configuration.SessionId,
@@ -1240,15 +1256,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Termination
Severity = EventSeverity.Info,
Source = "EnrollmentTerminationHandler",
Phase = EnrollmentPhase.Unknown,
Message = $"Agent shutting down (reason={reasonTag}, outcome={args.Outcome}).",
Data = new Dictionary<string, object>
{
{ "reason", reasonTag },
{ "outcome", args.Outcome.ToString() },
{ "stage", args.StageName ?? string.Empty },
{ "uptimeMinutes", uptimeMinutes },
{ "agentVersion", _agentVersion },
},
Message = args.Reason == EnrollmentTerminationReason.StoppedWaiting && !string.IsNullOrEmpty(args.Details)
? $"Agent shutting down (reason={reasonTag}, outcome={args.Outcome}): {args.Details}"
: $"Agent shutting down (reason={reasonTag}, outcome={args.Outcome}).",
Data = data,
ImmediateUpload = true,
});
}
@@ -230,6 +230,9 @@ namespace AutopilotMonitor.Agent.V2
logger.Info($"--run-gather-rules: debug trace -> {gatherDebugLogPath}");
if (consoleMode) Console.WriteLine($"Debug trace: {gatherDebugLogPath}");
// No stateDirectory on purpose: this mode runs beside the live agent under a
// throwaway session id and must neither resume nor overwrite the session's
// gather state (logparser positions, on_change values).
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride,
debugLogPath: gatherDebugLogPath,
debugEcho: consoleMode ? (Action<string>)(line => Console.WriteLine($" {line}")) : null))
@@ -526,7 +526,11 @@ namespace AutopilotMonitor.Agent.V2.Runtime
// analytics. WhiteGlove Part-2 resume runs as a fresh Classic
// enrollment after PR-A's archive-and-reset, so the engine state is
// empty and needs the facts re-seeded — same as a first boot.
LifecycleEmitters.PostEnrollmentFactsObserved(orchestrator.IngressSink, logger);
LifecycleEmitters.PostEnrollmentFactsObserved(
orchestrator.IngressSink,
agentConfig.HelloWaitTimeoutSeconds,
logger,
remoteConfig.Collectors?.OobeUpdatePhaseMode);
// V2 parity — post SessionStarted so the reducer establishes the session
// anchor (HandleSessionStartedV1 in DecisionEngine.Shared.cs). Skipped only
@@ -314,7 +314,9 @@ namespace AutopilotMonitor.Agent.V2.Runtime
/// <see cref="DecisionSignalKind.EnrollmentFactsObserved"/> signal carrying the
/// registry-derived enrollment facts (<c>enrollmentType</c> + <c>isHybridJoin</c>)
/// so the reducer can seed <see cref="State.EnrollmentScenarioProfile"/> via the
/// stage-agnostic <c>HandleEnrollmentFactsObservedV1</c> handler.
/// stage-agnostic <c>HandleEnrollmentFactsObservedV1</c> handler — plus the tenant's
/// Hello wait when it extends the engine's built-in hello_safety window, and the mode of
/// the OOBE update phase (D-310).
/// <para>
/// Posted immediately before <see cref="PostSessionStarted"/> so the Inspector
/// timeline reads naturally (facts → anchor) — but the reducer correctness does
@@ -329,7 +331,9 @@ namespace AutopilotMonitor.Agent.V2.Runtime
/// </summary>
public static void PostEnrollmentFactsObserved(
ISignalIngressSink ingressSink,
AgentLogger logger)
int helloWaitTimeoutSeconds,
AgentLogger logger,
string oobeUpdatePhaseMode = null)
{
try
{
@@ -352,6 +356,22 @@ namespace AutopilotMonitor.Agent.V2.Runtime
["isCloudPc"] = isCloudPc ? "true" : "false",
};
// Tenant Hello wait (remote config, not a registry fact) — stamped only when it
// widens the engine's built-in hello_safety window, so sessions on the default
// post the same payload as before.
var helloWaitSeconds = HelloWaitTimeout.EffectiveSeconds(helloWaitTimeoutSeconds);
if (helloWaitSeconds > HelloWaitTimeout.BuiltInSeconds)
{
payload[SignalPayloadKeys.HelloWaitTimeoutSeconds] =
helloWaitSeconds.ToString(System.Globalization.CultureInfo.InvariantCulture);
}
// OOBE update phase mode (remote config) — stamped at every start, the same value
// the update watchers were wired with: the engine lets the phase act only while
// every start of the session said Active, so Off or Shadow is a kill switch.
var phaseMode = OobeUpdatePhaseModes.Normalize(oobeUpdatePhaseMode);
payload[SignalPayloadKeys.OobeUpdatePhaseMode] = phaseMode;
var evidence = new Evidence(
kind: EvidenceKind.Synthetic,
identifier: "enrollment_registry_facts_read",
@@ -364,7 +384,7 @@ namespace AutopilotMonitor.Agent.V2.Runtime
evidence: evidence,
payload: payload);
logger.Debug($"EnrollmentFactsObserved signal posted (enrollmentType={enrollmentType}, isHybridJoin={isHybridJoin}, isSelfDeployingProfile={isSelfDeploying}, isCloudPc={isCloudPc}).");
logger.Debug($"EnrollmentFactsObserved signal posted (enrollmentType={enrollmentType}, isHybridJoin={isHybridJoin}, isSelfDeployingProfile={isSelfDeploying}, isCloudPc={isCloudPc}, helloWaitSeconds={helloWaitSeconds}, oobeUpdatePhaseMode={phaseMode}).");
}
catch (Exception ex)
{
@@ -117,7 +117,9 @@ namespace AutopilotMonitor.DecisionCore.Tests
// Session 4910a5a5 (2026-07-23) added `espAdvisoryFailureResolvedUtc` (the failed
// category recovered — lifts the advisory's re-arm/rebase guard exemption) and
// `espAdvisoryFailureCategory` (which category the advisory was for) → 20.
Assert.Equal(20, facts.Count);
// D-310 (2026-10-02) added `stoppedWaitingReason` — the engine stopped waiting for
// the user without a verdict (no sign-in after the OOBE update) → 21.
Assert.Equal(21, facts.Count);
Assert.All(facts.Values, v => Assert.Null(v));
}
@@ -242,9 +244,11 @@ namespace AutopilotMonitor.DecisionCore.Tests
// that vetoes/retracts the policy-disabled Hello skip → 18. Session 4910a5a5
// (2026-07-23) added `espAdvisoryFailureResolvedUtc` + `espAdvisoryFailureCategory`
// — advisory-recovery facts that lift the AdvisoryCompletion guard exemption → 20.
// D-310 (2026-10-02) added `stoppedWaitingReason` — the engine stopped waiting for
// the user without a verdict → 21.
// If this number ever changes, both the count expectation AND the actual snapshot
// output need to evolve in lockstep.
Assert.Equal(20, expectedFactKeys.Count);
Assert.Equal(21, expectedFactKeys.Count);
var snapshot = DecisionStateSnapshotBuilder.Build(DecisionState.CreateInitial("s", "t"));
var facts = (Dictionary<string, object?>)snapshot["facts"]!;
@@ -0,0 +1,357 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using Xunit;
namespace AutopilotMonitor.DecisionCore.Tests
{
/// <summary>
/// Tenant-extendable Hello window. The agent stamps the tenant's <c>HelloWaitTimeoutSeconds</c>
/// on <see cref="DecisionSignalKind.EnrollmentFactsObserved"/> when it exceeds the built-in
/// 300 s; every <see cref="DeadlineNames.HelloSafety"/> arm site then uses it, clamped to
/// 3600 s. Without the key the window stays exactly the built-in 300 s.
/// </summary>
public sealed class HelloSafetyTenantWindowTests
{
private static readonly DateTime T0 = new DateTime(2026, 9, 29, 8, 0, 0, DateTimeKind.Utc);
private static readonly DateTime EspExit = T0.AddMinutes(6);
[Theory]
[InlineData(null, 300)]
[InlineData("1800", 1800)]
[InlineData("301", 301)]
[InlineData("3600", 3600)]
[InlineData("86400", 3600)] // beyond the ceiling — clamped
[InlineData("300", 300)] // not an extension — ignored (the agent never stamps it)
[InlineData("120", 300)]
public void Esp_exit_arms_hello_safety_with_the_tenant_window(string? helloWait, int expectedSeconds)
{
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait));
Assert.Equal(SessionStage.AwaitingHello, state.Stage);
var helloSafety = Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.HelloSafety);
Assert.Equal(EspExit.AddSeconds(expectedSeconds), helloSafety.DueAtUtc);
}
[Fact]
public void Tenant_window_is_recorded_set_once_with_its_source_ordinal()
{
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit("1800"));
var fact = state.ScenarioObservations.HelloWaitTimeoutSeconds;
Assert.NotNull(fact);
Assert.Equal(1800, fact!.Value);
Assert.Equal(1, fact.SourceSignalOrdinal);
// A later re-post (every agent start posts the facts again) never re-bases the window.
var repost = engine.Reduce(state, Facts(20, EspExit.AddSeconds(30), "3600"));
Assert.Equal(1800, repost.NewState.ScenarioObservations.HelloWaitTimeoutSeconds!.Value);
Assert.Equal(1, repost.NewState.ScenarioObservations.HelloWaitTimeoutSeconds!.SourceSignalOrdinal);
Assert.Empty(repost.Effects);
Assert.Equal(
EspExit.AddSeconds(1800),
Assert.Single(repost.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
}
[Fact]
public void Census_surfaces_the_tenant_window_only_when_recorded()
{
var engine = new DecisionEngine();
var extended = DecisionStateSignalCensus.Build(ReduceAll(engine, ClassicUntilEspExit("2700")));
Assert.Contains("hello_wait_extended", extended.SignalsSeen);
var evidence = Assert.IsAssignableFrom<IDictionary<string, object>>(extended.SignalEvidence["helloWaitExtended"]);
Assert.Equal(2700, evidence["seconds"]);
var builtIn = DecisionStateSignalCensus.Build(ReduceAll(engine, ClassicUntilEspExit(helloWait: null)));
Assert.DoesNotContain("hello_wait_extended", builtIn.SignalsSeen);
}
[Theory]
[InlineData("abc")]
[InlineData("")]
[InlineData("0")]
[InlineData("-5")]
public void Malformed_tenant_window_is_ignored(string helloWait)
{
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait));
Assert.Null(state.ScenarioObservations.HelloWaitTimeoutSeconds);
Assert.Equal(
EspExit.AddSeconds(300),
Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
}
[Fact]
public void Late_tenant_window_extends_the_armed_hello_safety_from_its_original_anchor()
{
// A backfilled ESP exit armed the built-in window before the facts signal landed.
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait: null));
Assert.Equal(
EspExit.AddSeconds(300),
Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
var step = engine.Reduce(state, Facts(20, EspExit.AddSeconds(10), "1800"));
Assert.Equal(SessionStage.AwaitingHello, step.NewState.Stage);
var extended = Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety);
Assert.Equal(EspExit.AddSeconds(1800), extended.DueAtUtc);
Assert.Equal(DecisionSignalKind.DeadlineFired, extended.FiresSignalKind);
Assert.Equal(DeadlineNames.HelloSafety, extended.FiresPayload![SignalPayloadKeys.Deadline]);
// The live scheduler is told to replace its timer (same name → replaced).
var effect = Assert.Single(step.Effects);
Assert.Equal(DecisionEffectKind.ScheduleDeadline, effect.Kind);
Assert.Equal(EspExit.AddSeconds(1800), effect.Deadline!.DueAtUtc);
}
[Theory]
[InlineData("300")]
[InlineData("120")]
public void Late_tenant_window_not_longer_than_the_built_in_leaves_the_deadline_alone(string helloWait)
{
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait: null));
var step = engine.Reduce(state, Facts(20, EspExit.AddSeconds(10), helloWait));
Assert.Empty(step.Effects);
Assert.Null(step.NewState.ScenarioObservations.HelloWaitTimeoutSeconds);
Assert.Equal(
EspExit.AddSeconds(300),
Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
}
[Fact]
public void Fire_of_the_replaced_built_in_deadline_is_dead_ended_after_the_late_extension()
{
// The old timer fired while the facts signal was still queued ahead of it: the facts
// step moves hello_safety out, then the queued fire of the replaced incarnation
// arrives. It must not time Hello out at the built-in window.
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait: null));
var builtInDue = EspExit.AddSeconds(300);
state = engine.Reduce(state, Facts(20, EspExit.AddSeconds(10), "1800")).NewState;
var stale = engine.Reduce(state, MakeSignal(21, DecisionSignalKind.DeadlineFired, builtInDue.AddSeconds(2),
Fired(DeadlineNames.HelloSafety, builtInDue)));
Assert.False(stale.Transition.Taken);
Assert.Equal("hello_safety_stale_superseded_by_rearm", stale.Transition.DeadEndReason);
Assert.Empty(stale.Effects);
Assert.Null(stale.NewState.HelloResolvedUtc);
Assert.Equal(SessionStage.AwaitingHello, stale.NewState.Stage);
var extendedDue = EspExit.AddSeconds(1800);
Assert.Equal(
extendedDue,
Assert.Single(stale.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
// The fire of the current incarnation still times Hello out (desktop is in → Finalizing).
var current = engine.Reduce(stale.NewState, MakeSignal(22, DecisionSignalKind.DeadlineFired, extendedDue,
Fired(DeadlineNames.HelloSafety, extendedDue)));
Assert.Equal(SessionStage.Finalizing, current.NewState.Stage);
Assert.Equal("Timeout", current.NewState.HelloOutcome!.Value);
}
[Fact]
public void Late_tenant_window_after_the_built_in_expired_leaves_the_pending_fire_to_decide()
{
var engine = new DecisionEngine();
var state = ReduceAll(engine, ClassicUntilEspExit(helloWait: null));
var step = engine.Reduce(state, Facts(20, EspExit.AddSeconds(301), "1800"));
Assert.Empty(step.Effects);
Assert.Equal(
EspExit.AddSeconds(300),
Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
// The fact is still recorded — later arm sites use it.
Assert.Equal(1800, step.NewState.ScenarioObservations.HelloWaitTimeoutSeconds!.Value);
}
[Fact]
public void Late_extension_is_replay_deterministic()
{
var engine = new DecisionEngine();
var signals = ClassicUntilEspExit(helloWait: null)
.Concat(new[] { Facts(20, EspExit.AddSeconds(10), "2400") })
.ToList();
var stepwise = ReduceAll(engine, signals);
var replayed = ReducerReplay.Replay(engine, Seed(), signals);
Assert.Equal(
Assert.Single(stepwise.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc,
Assert.Single(replayed.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
Assert.Equal(EspExit.AddSeconds(2400), replayed.Deadlines.Single(d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
}
[Fact]
public void Tenant_window_also_applies_to_the_AccountSetup_complete_deferred_promote()
{
// Strong gate arrives after the final exit and before the desktop: the deferred
// promote arms hello_safety at the gate signal — with the tenant window.
var engine = new DecisionEngine();
var state = ReduceAll(engine, new[]
{
MakeSignal(0, DecisionSignalKind.SessionStarted, T0),
Facts(1, T0.AddSeconds(5), "2400"),
MakeSignal(2, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1), Phase("DeviceSetup")),
MakeSignal(3, DecisionSignalKind.HelloPolicyDetected, T0.AddMinutes(1).AddSeconds(5),
new Dictionary<string, string> { [SignalPayloadKeys.HelloEnabled] = "true" }),
MakeSignal(4, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(12), Phase("AccountSetup")),
MakeSignal(5, DecisionSignalKind.EspExiting, T0.AddMinutes(24)),
});
Assert.Equal(SessionStage.EspAccountSetup, state.Stage);
var gate = T0.AddMinutes(25);
var step = engine.Reduce(state, MakeSignal(6, DecisionSignalKind.AccountSetupProvisioningComplete, gate));
Assert.Equal(SessionStage.AwaitingHello, step.NewState.Stage);
Assert.Equal(
gate.AddSeconds(2400),
Assert.Single(step.NewState.Deadlines, d => d.Name == DeadlineNames.HelloSafety).DueAtUtc);
}
[Theory]
[InlineData(null, 30 * 60)]
[InlineData("1200", 30 * 60)] // a shorter tenant window never shortens the WDP backstop
[InlineData("3600", 3600)]
public void Device_preparation_backstop_is_never_shorter_than_the_tenant_window(string? helloWait, int expectedSeconds)
{
// WDP has no ESP, so hello_safety never arms; the desktop-armed backstop is the last
// net behind the tracker's Hello completion timer, which runs on the tenant budget.
var engine = new DecisionEngine();
var facts = new Dictionary<string, string>
{
[SignalPayloadKeys.EnrollmentType] = "v2",
[SignalPayloadKeys.EnrollmentTypeDeterministic] = "true",
};
if (helloWait != null)
facts[SignalPayloadKeys.HelloWaitTimeoutSeconds] = helloWait;
var desktop = T0.AddMinutes(9);
var state = ReduceAll(engine, new[]
{
MakeSignal(0, DecisionSignalKind.SessionStarted, T0),
MakeSignal(1, DecisionSignalKind.EnrollmentFactsObserved, T0, facts),
MakeSignal(2, DecisionSignalKind.DesktopArrived, desktop),
});
Assert.Equal(
desktop.AddSeconds(expectedSeconds),
Assert.Single(state.Deadlines, d => d.Name == DeadlineNames.DevicePrepCompletion).DueAtUtc);
}
[Fact]
public void IsTenantHelloWaitPending_is_true_only_while_an_extended_window_is_armed_and_not_yet_due()
{
var engine = new DecisionEngine();
var awaiting = ReduceAll(engine, ClassicUntilEspExit("1800"));
var due = EspExit.AddSeconds(1800);
Assert.True(DecisionEngine.IsTenantHelloWaitPending(awaiting, EspExit.AddMinutes(1)));
Assert.True(DecisionEngine.IsTenantHelloWaitPending(awaiting, due.AddSeconds(-1)));
Assert.False(DecisionEngine.IsTenantHelloWaitPending(awaiting, due));
Assert.False(DecisionEngine.IsTenantHelloWaitPending(awaiting, due.AddSeconds(1)));
Assert.False(DecisionEngine.IsTenantHelloWaitPending(null, EspExit));
Assert.False(DecisionEngine.IsTenantHelloWaitPending(Seed(), EspExit));
// The built-in window never counts: default tenants keep their stall reporting.
var builtIn = ReduceAll(engine, ClassicUntilEspExit(helloWait: null));
Assert.Contains(builtIn.Deadlines, d => d.Name == DeadlineNames.HelloSafety);
Assert.False(DecisionEngine.IsTenantHelloWaitPending(builtIn, EspExit.AddMinutes(1)));
// Hello resolved → the deadline is cancelled → nothing pending any more.
var resolved = engine.Reduce(awaiting, MakeSignal(20, DecisionSignalKind.HelloResolved, EspExit.AddMinutes(10),
new Dictionary<string, string> { [SignalPayloadKeys.HelloOutcome] = "completed" })).NewState;
Assert.False(DecisionEngine.IsTenantHelloWaitPending(resolved, EspExit.AddMinutes(11)));
}
[Fact]
public void IsTenantHelloWaitPending_covers_the_device_preparation_backstop()
{
var engine = new DecisionEngine();
var desktop = T0.AddMinutes(9);
var state = ReduceAll(engine, new[]
{
MakeSignal(0, DecisionSignalKind.SessionStarted, T0),
MakeSignal(1, DecisionSignalKind.EnrollmentFactsObserved, T0, new Dictionary<string, string>
{
[SignalPayloadKeys.EnrollmentType] = "v2",
[SignalPayloadKeys.EnrollmentTypeDeterministic] = "true",
[SignalPayloadKeys.HelloWaitTimeoutSeconds] = "3600",
}),
MakeSignal(2, DecisionSignalKind.DesktopArrived, desktop),
});
Assert.True(DecisionEngine.IsTenantHelloWaitPending(state, desktop.AddMinutes(59)));
Assert.False(DecisionEngine.IsTenantHelloWaitPending(state, desktop.AddMinutes(60)));
}
// ============================================================ helpers
private static DecisionState Seed() => DecisionState.CreateInitial("sess-hello-window", "tenant", T0);
/// <summary>
/// Classic user-driven flow up to the final ESP exit: strong gate and desktop in place,
/// Hello policy unknown — so the exit promotes to AwaitingHello and arms hello_safety.
/// </summary>
private static List<DecisionSignal> ClassicUntilEspExit(string? helloWait)
{
var signals = new List<DecisionSignal> { MakeSignal(0, DecisionSignalKind.SessionStarted, T0) };
if (helloWait != null)
signals.Add(Facts(1, T0.AddSeconds(5), helloWait));
signals.Add(MakeSignal(2, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1), Phase("AccountSetup")));
signals.Add(MakeSignal(3, DecisionSignalKind.AccountSetupProvisioningComplete, T0.AddMinutes(2)));
signals.Add(MakeSignal(4, DecisionSignalKind.DesktopArrived, T0.AddMinutes(3)));
signals.Add(MakeSignal(5, DecisionSignalKind.EspExiting, EspExit));
return signals;
}
private static DecisionState ReduceAll(DecisionEngine engine, IEnumerable<DecisionSignal> signals)
{
var state = Seed();
foreach (var signal in signals)
state = engine.Reduce(state, signal).NewState;
return state;
}
private static DecisionSignal Facts(long ordinal, DateTime occurredAtUtc, string helloWait) =>
MakeSignal(ordinal, DecisionSignalKind.EnrollmentFactsObserved, occurredAtUtc,
new Dictionary<string, string> { [SignalPayloadKeys.HelloWaitTimeoutSeconds] = helloWait });
private static Dictionary<string, string> Fired(string deadlineName, DateTime dueAtUtc) =>
new Dictionary<string, string>
{
[SignalPayloadKeys.Deadline] = deadlineName,
[SignalPayloadKeys.DeadlineDueAtUtc] = dueAtUtc.ToString("O", CultureInfo.InvariantCulture),
};
private static Dictionary<string, string> Phase(string phase) =>
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = phase };
private static DecisionSignal MakeSignal(
long ordinal,
DecisionSignalKind kind,
DateTime occurredAtUtc,
IReadOnlyDictionary<string, string>? payload = null) =>
new DecisionSignal(
sessionSignalOrdinal: ordinal,
sessionTraceOrdinal: ordinal,
kind: kind,
kindSchemaVersion: 1,
occurredAtUtc: occurredAtUtc,
sourceOrigin: "test",
evidence: new Evidence(EvidenceKind.Synthetic, $"t-{kind}-{ordinal}", $"synthetic {kind}"),
payload: payload);
}
}
@@ -0,0 +1,749 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Serialization;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.DecisionCore.State;
using Newtonsoft.Json.Linq;
using Xunit;
namespace AutopilotMonitor.DecisionCore.Tests
{
/// <summary>
/// Report 3a2207978e4c (2026-10-01, D-310) — the OOBE quality update. With the ESP setting
/// "Install Windows quality updates" Windows installs the update after the Device-ESP page
/// closes and before the user signs in: 20–40 minutes plus a restart to the lock screen. The
/// IME had logged its AccountSetup phase line before that handoff exit, so the exit armed the
/// 30-min <c>advisory_completion</c> window, and the session was failed
/// (<c>esp_exit_without_completion_evidence</c>) while the update was still installing.
/// <para>
/// The OOBE update phase holds the window, every state bounded on its own (user decision
/// 2026-10-02): the update runs at most 3 h, then fails as not finished; after the update the
/// user has 60 min to sign in, then the engine stops waiting — no verdict, the backend
/// classifies; after the sign-in the user's own setup gets one window.
/// </para>
/// <para>
/// Only the Active mode acts (measurement first, user decision 2026-10-02). In the Shadow mode
/// — the default, and what an unstamped session counts as — the fire reports what the phase
/// would decide and today's rule decides; Off evaluates nothing.
/// </para>
/// </summary>
public sealed class OobeUpdatePhaseTests
{
private static readonly DateTime T0 = new DateTime(2026, 10, 1, 11, 0, 0, DateTimeKind.Utc);
private const string Lcu = "2026-09 Cumulative Update for Windows 11, version 25H2 for x64-based Systems (KB5099999)";
private static DecisionSignal MakeSignal(
long ordinal,
DecisionSignalKind kind,
DateTime occurredAtUtc,
IReadOnlyDictionary<string, string>? payload = null)
{
return new DecisionSignal(
sessionSignalOrdinal: ordinal,
sessionTraceOrdinal: ordinal,
kind: kind,
kindSchemaVersion: 1,
occurredAtUtc: occurredAtUtc,
sourceOrigin: "test",
evidence: new Evidence(EvidenceKind.Synthetic, $"{kind}-{ordinal}", "test"),
payload: payload);
}
private static DecisionSignal OsUpdate(long ordinal, DateTime at, string step, string? update = Lcu)
{
var payload = new Dictionary<string, string>
{
[OsUpdatePayloadKeys.Step] = step,
[OsUpdatePayloadKeys.Source] = "wu",
};
if (update != null) payload[OsUpdatePayloadKeys.Update] = update;
return MakeSignal(ordinal, DecisionSignalKind.OsUpdateActivity, at, payload);
}
private static DecisionSignal AdvisoryFired(long ordinal, DateTime at) =>
MakeSignal(ordinal, DecisionSignalKind.DeadlineFired, at,
new Dictionary<string, string> { [SignalPayloadKeys.Deadline] = DeadlineNames.AdvisoryCompletion });
private static DecisionSignal Reboot(long ordinal, DateTime at) =>
MakeSignal(ordinal, DecisionSignalKind.SystemRebootObserved, at);
/// <summary>The agent's mode stamp on <c>EnrollmentFactsObserved</c>, posted at every start.</summary>
private static DecisionSignal PhaseMode(long ordinal, DateTime at, string mode) =>
MakeSignal(ordinal, DecisionSignalKind.EnrollmentFactsObserved, at,
new Dictionary<string, string> { [SignalPayloadKeys.OobeUpdatePhaseMode] = mode });
private static bool HasTimelineEffect(DecisionStep step, string eventType) =>
step.Effects.Any(e =>
e.Kind == DecisionEffectKind.EmitEventTimelineEntry
&& e.Parameters != null
&& e.Parameters.TryGetValue("eventType", out var et)
&& et == eventType);
private static ActiveDeadline? FindDeadline(DecisionState state, string name) =>
state.Deadlines.FirstOrDefault(d => d.Name == name);
private static DecisionEffect SingleTimelineEffect(DecisionStep step, string eventType) =>
step.Effects.Single(e =>
e.Kind == DecisionEffectKind.EmitEventTimelineEntry
&& e.Parameters != null
&& e.Parameters.TryGetValue("eventType", out var et)
&& et == eventType);
/// <summary>
/// The report's shape up to the arming exit: Classic user-driven, full ESP, DeviceSetup,
/// the IME's pre-sign-in AccountSetup line, then the Device-ESP handoff exit at T0+21,
/// which arms the window (due T0+51). <paramref name="beforeExit"/> runs before the exit;
/// <paramref name="mode"/> is the agent's mode stamp (null: none, an older agent).
/// </summary>
private static DecisionState HandoffExitArmsTheWindow(
DecisionEngine engine,
Func<DecisionState, DecisionState>? beforeExit = null,
string? mode = OobeUpdatePhaseModes.Active)
{
var state = DecisionState.CreateInitial("sess-3a220797", "tenant-3a220797", T0);
state = engine.Reduce(state, MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
if (mode != null) state = engine.Reduce(state, PhaseMode(1, T0, mode)).NewState;
state = engine.Reduce(state, MakeSignal(
5, DecisionSignalKind.EspConfigDetected, T0.AddMinutes(1),
new Dictionary<string, string>
{
[SignalPayloadKeys.SkipUserEsp] = "false",
[SignalPayloadKeys.SkipDeviceEsp] = "false",
[SignalPayloadKeys.EspAllowContinueAnyway] = "false",
})).NewState;
state = engine.Reduce(state, MakeSignal(
10, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(1),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "DeviceSetup" })).NewState;
state = engine.Reduce(state, MakeSignal(
20, DecisionSignalKind.EspPhaseChanged, T0.AddMinutes(20),
new Dictionary<string, string> { [SignalPayloadKeys.EspPhase] = "AccountSetup" })).NewState;
if (beforeExit != null) state = beforeExit(state);
state = engine.Reduce(state, MakeSignal(30, DecisionSignalKind.EspExiting, T0.AddMinutes(21))).NewState;
var armed = FindDeadline(state, DeadlineNames.AdvisoryCompletion);
Assert.NotNull(armed);
Assert.Equal(T0.AddMinutes(51), armed!.DueAtUtc);
Assert.Null(state.Outcome);
return state;
}
/// <summary>
/// Fires the armed advisory deadline at its due time until a fire does not re-arm it;
/// returns every step. Proves each re-arm lands on time — the last one on the bound.
/// </summary>
private static List<DecisionStep> FireUntilResolved(DecisionEngine engine, DecisionState state, long firstOrdinal)
{
var steps = new List<DecisionStep>();
var ordinal = firstOrdinal;
while (true)
{
var deadline = FindDeadline(state, DeadlineNames.AdvisoryCompletion);
Assert.NotNull(deadline);
var step = engine.Reduce(state, AdvisoryFired(ordinal++, deadline!.DueAtUtc));
steps.Add(step);
state = step.NewState;
if (FindDeadline(state, DeadlineNames.AdvisoryCompletion) == null) return steps;
Assert.True(steps.Count < 50, "the phase never resolved");
}
}
// ================================================================= handler ====
[Fact]
public void ActivityAfterTheHandoffExit_AnchorsThePhase()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
var first = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted));
Assert.True(first.Transition.Taken);
Assert.Equal("OsUpdateActivity:OobeUpdatePhaseStarted", first.Transition.Trigger);
Assert.Equal(state.Stage, first.NewState.Stage);
Assert.Empty(first.Effects);
var facts = first.NewState.OsUpdateFacts;
Assert.Equal(T0.AddMinutes(22), facts.PostExitActivityUtc!.Value);
Assert.Equal(40, facts.PostExitActivityUtc.SourceSignalOrdinal);
Assert.True(DecisionEngine.IsInOobeUpdatePhase(first.NewState));
var second = engine.Reduce(first.NewState, OsUpdate(41, T0.AddMinutes(35), OsUpdateSteps.InstallStarted));
Assert.Equal("OsUpdateActivity", second.Transition.Trigger);
facts = second.NewState.OsUpdateFacts;
Assert.Equal(T0.AddMinutes(22), facts.PostExitActivityUtc!.Value);
Assert.Equal(T0.AddMinutes(35), facts.LastActivityUtc!.Value);
Assert.Equal(OsUpdateSteps.InstallStarted, facts.LastStep!.Value);
Assert.Equal(Lcu, facts.LastUpdate!.Value);
Assert.Equal(2, facts.ActivityCount);
}
[Fact]
public void RebootRequiredAndFailure_AreRecorded()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(30), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(31), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, OsUpdate(42, T0.AddMinutes(32), OsUpdateSteps.Failed, update: null)).NewState;
var facts = state.OsUpdateFacts;
Assert.Equal(T0.AddMinutes(30), facts.RebootRequiredUtc!.Value); // earliest
Assert.Equal(T0.AddMinutes(32), facts.FailedUtc!.Value);
Assert.Equal("KB5099999", facts.LastUpdate!.Value); // a step without an update keeps the last one
}
[Fact]
public void ActivityWithoutStep_IsADeadEnd()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
var step = engine.Reduce(state, MakeSignal(40, DecisionSignalKind.OsUpdateActivity, T0.AddMinutes(22),
new Dictionary<string, string> { [OsUpdatePayloadKeys.Source] = "wu" }));
Assert.False(step.Transition.Taken);
Assert.Equal("os_update_activity_without_step", step.Transition.DeadEndReason);
Assert.Equal(0, step.NewState.OsUpdateFacts.ActivityCount);
}
[Fact]
public void ActivityBeforeTheExit_IsNoPhase()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, beforeExit: s =>
engine.Reduce(s, OsUpdate(25, T0.AddMinutes(15), OsUpdateSteps.Installed)).NewState);
Assert.NotNull(state.OsUpdateFacts.FirstActivityUtc);
Assert.Null(state.OsUpdateFacts.PostExitActivityUtc);
Assert.False(DecisionEngine.IsInOobeUpdatePhase(state));
}
[Fact]
public void BackfilledActivityThatPredatesTheExit_IsNoPhase()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(18), OsUpdateSteps.InstallStarted)).NewState;
Assert.Null(state.OsUpdateFacts.PostExitActivityUtc);
Assert.Equal(1, state.OsUpdateFacts.ActivityCount);
}
[Fact]
public void ActivityAfterTheUserSignedIn_IsNoPhase()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, MakeSignal(35, DecisionSignalKind.HelloWizardStarted, T0.AddMinutes(25))).NewState;
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(30), OsUpdateSteps.InstallStarted)).NewState;
Assert.Null(state.OsUpdateFacts.PostExitActivityUtc);
Assert.False(DecisionEngine.IsInOobeUpdatePhase(state));
}
// ============================================================ restart facts ====
[Fact]
public void RestartDuringThePhase_IsRecorded()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, Reboot(50, T0.AddMinutes(53))).NewState;
Assert.Equal(T0.AddMinutes(53), state.OsUpdateFacts.RestartUtc!.Value);
Assert.Equal(50, state.OsUpdateFacts.RestartUtc.SourceSignalOrdinal);
}
[Fact]
public void RestartBeforeThePhase_OrAfterTheSignIn_IsNotTheUpdatesRestart()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, Reboot(35, T0.AddMinutes(21.5))).NewState; // no anchor yet
Assert.Null(state.OsUpdateFacts.RestartUtc);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, MakeSignal(45, DecisionSignalKind.HelloWizardStarted, T0.AddMinutes(30))).NewState;
state = engine.Reduce(state, Reboot(50, T0.AddMinutes(40))).NewState; // user already there
Assert.Null(state.OsUpdateFacts.RestartUtc);
}
// ======================================================= advisory window ====
[Fact]
public void Report3a2207978e4c_FireDuringTheUpdate_RearmsInsteadOfFailing()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(35), OsUpdateSteps.InstallStarted)).NewState;
var fireAt = T0.AddMinutes(51);
var step = engine.Reduce(state, AdvisoryFired(50, fireAt));
Assert.True(step.Transition.Taken);
Assert.Null(step.NewState.Outcome);
Assert.NotEqual(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdatePhase", step.Transition.Trigger);
var rearmed = FindDeadline(step.NewState, DeadlineNames.AdvisoryCompletion);
Assert.Equal(fireAt.AddMinutes(30), rearmed!.DueAtUtc);
var schedule = step.Effects.Single(e => e.Kind == DecisionEffectKind.ScheduleDeadline);
Assert.Equal(DeadlineNames.AdvisoryCompletion, schedule.Deadline!.Name);
var waiting = SingleTimelineEffect(step, "completion_waiting");
Assert.Contains("installing an update in OOBE", waiting.Parameters!["message"]);
Assert.Equal(OsUpdateSteps.InstallStarted, waiting.Parameters!["osUpdateLastStep"]);
Assert.Equal(Lcu, waiting.Parameters!["osUpdate"]);
Assert.Equal(rearmed.DueAtUtc.ToString("o"), waiting.Parameters!["resolutionDeadlineDueAtUtc"]);
// The update may run 3 h from its first activity.
Assert.Equal(T0.AddMinutes(22).AddHours(3).ToString("o"), waiting.Parameters!["oobeUpdateHoldsUntilUtc"]);
}
[Fact]
public void WithoutUpdateActivity_TheFireStillFails()
{
// Mutation proof: the guard keys on observed update activity, never on its absence.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("esp_exit_without_completion_evidence", SingleTimelineEffect(step, "enrollment_failed").Parameters!["reason"]);
}
[Fact]
public void ActivityOnlyBeforeTheExit_TheFireStillFails()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, beforeExit: s =>
engine.Reduce(s, OsUpdate(25, T0.AddMinutes(15), OsUpdateSteps.Installed)).NewState);
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
}
[Fact]
public void UpdateStillRunningAtTheCap_FailsAsNotFinished()
{
// A hanging update: the last step says it still runs, and no restart ever comes.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(30), OsUpdateSteps.InstallStarted)).NewState;
var steps = FireUntilResolved(engine, state, firstOrdinal: 100);
Assert.All(steps.Take(steps.Count - 1), s =>
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdatePhase", s.Transition.Trigger));
var cap = T0.AddMinutes(22).AddHours(3);
// The last re-arm lands on the cap, not up to 30 min after it.
Assert.Equal(cap, FindDeadline(steps[steps.Count - 2].NewState, DeadlineNames.AdvisoryCompletion)!.DueAtUtc);
var resolved = steps.Last();
Assert.Equal(SessionStage.Failed, resolved.NewState.Stage);
Assert.Equal(SessionOutcome.EnrollmentFailed, resolved.NewState.Outcome);
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdateNotFinished", resolved.Transition.Trigger);
var failed = SingleTimelineEffect(resolved, "enrollment_failed");
Assert.Equal("oobe_update_not_finished", failed.Parameters!["reason"]);
Assert.Equal(OsUpdateSteps.InstallStarted, failed.Parameters!["osUpdateLastStep"]);
Assert.Equal("3", failed.Parameters!["oobeUpdateCapHours"]);
Assert.Null(resolved.NewState.StoppedWaitingReason);
}
[Fact]
public void UpdateWithRestart_WaitsSixtyMinutesForTheSignIn_ThenStopsWaiting()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(40), OsUpdateSteps.InstallStarted)).NewState;
state = engine.Reduce(state, OsUpdate(42, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
var beforeRestart = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdateAwaitingSignIn", beforeRestart.Transition.Trigger);
state = beforeRestart.NewState;
// The update page restarts the device; the lock screen waits for the user.
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState;
Assert.Equal(T0.AddMinutes(83), FindDeadline(state, DeadlineNames.AdvisoryCompletion)!.DueAtUtc); // re-based
var steps = FireUntilResolved(engine, state, firstOrdinal: 100);
var waiting = steps.Take(steps.Count - 1).ToList();
Assert.All(waiting, s => Assert.Equal("DeadlineFired:advisory_completion:OobeUpdateAwaitingSignIn", s.Transition.Trigger));
Assert.Contains("waiting for the user to sign in",
SingleTimelineEffect(waiting[0], "completion_waiting").Parameters!["message"]);
var stopped = steps.Last();
Assert.Equal(T0.AddMinutes(53).AddMinutes(60), stopped.Transition.OccurredAtUtc);
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdateSignInWaitExpired", stopped.Transition.Trigger);
Assert.True(stopped.Transition.Taken);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, stopped.NewState.StoppedWaitingReason!.Value);
// No verdict: the stage stays, nothing is failed, nothing is armed any more.
Assert.Equal(state.Stage, stopped.NewState.Stage);
Assert.Null(stopped.NewState.Outcome);
Assert.Empty(stopped.NewState.Deadlines);
var expired = SingleTimelineEffect(stopped, "completion_wait_expired");
Assert.Equal("Warning", expired.Parameters!["severity"]);
Assert.Equal("true", expired.Parameters!["immediateUpload"]);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, expired.Parameters!["reason"]);
Assert.Equal("60", expired.Parameters!["signInWindowMinutes"]);
Assert.Equal(T0.AddMinutes(53).ToString("o"), expired.Parameters!["oobeUpdateEndedUtc"]);
Assert.Contains("Not a failure", expired.Parameters!["message"]);
}
[Fact]
public void UpdateEndingWithoutRestart_SignInWindowStartsAtItsLastStep()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(40), OsUpdateSteps.Installed)).NewState;
var steps = FireUntilResolved(engine, state, firstOrdinal: 100);
var stopped = steps.Last();
Assert.Equal(T0.AddMinutes(100), stopped.Transition.OccurredAtUtc);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, stopped.NewState.StoppedWaitingReason!.Value);
}
[Fact]
public void BackfilledOlderStepAfterTheRestart_DoesNotReopenTheUpdate()
{
// The watchers backfill after the restart: a record written before it arrives late.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState;
state = engine.Reduce(state, OsUpdate(70, T0.AddMinutes(45), OsUpdateSteps.InstallStarted)).NewState;
Assert.Equal(OsUpdateSteps.RebootRequired, state.OsUpdateFacts.LastStep!.Value);
var phase = DecisionEngine.EvaluateOobeUpdatePhase(state, T0.AddMinutes(83));
Assert.Equal(DecisionEngine.OobeUpdatePhaseKind.AwaitingSignIn, phase.Kind);
Assert.Equal(T0.AddMinutes(113), phase.BoundUtc);
}
[Fact]
public void NewerUpdateStepAfterTheRestart_ReopensTheUpdate_BoundedByTheCap()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState;
state = engine.Reduce(state, OsUpdate(70, T0.AddMinutes(60), OsUpdateSteps.DownloadStarted)).NewState;
var phase = DecisionEngine.EvaluateOobeUpdatePhase(state, T0.AddMinutes(83));
Assert.Equal(DecisionEngine.OobeUpdatePhaseKind.Updating, phase.Kind);
Assert.Equal(T0.AddMinutes(22).AddHours(3), phase.BoundUtc); // the cap counts from the first activity
}
[Fact]
public void UserSignIn_GetsOneFullWindow_ThenTheFireResolvesNormally()
{
// The transition to AwaitingHello does not cancel the window. Without the grace a fire
// right after the sign-in would fail the session inside the Hello wizard.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51))).NewState; // due T0+81
state = engine.Reduce(state, MakeSignal(60, DecisionSignalKind.HelloWizardStarted, T0.AddMinutes(75))).NewState;
Assert.False(DecisionEngine.IsInOobeUpdatePhase(state));
// 6 minutes after the sign-in — inside the grace, which ends at T0+105.
var grace = engine.Reduce(state, AdvisoryFired(70, T0.AddMinutes(81)));
Assert.Null(grace.NewState.Outcome);
Assert.Equal("DeadlineFired:advisory_completion:OobeUpdatePhaseUserArrived", grace.Transition.Trigger);
Assert.Contains("signed in after the OOBE update", SingleTimelineEffect(grace, "completion_waiting").Parameters!["message"]);
Assert.Equal(T0.AddMinutes(105), FindDeadline(grace.NewState, DeadlineNames.AdvisoryCompletion)!.DueAtUtc);
// One full window after the sign-in, still no completion evidence — resolved the normal way.
var resolved = engine.Reduce(grace.NewState, AdvisoryFired(80, T0.AddMinutes(105)));
Assert.Equal(SessionStage.Failed, resolved.NewState.Stage);
Assert.Equal("esp_exit_without_completion_evidence", SingleTimelineEffect(resolved, "enrollment_failed").Parameters!["reason"]);
Assert.Null(resolved.NewState.StoppedWaitingReason);
// Active decides itself — no shadow report.
Assert.False(HasTimelineEffect(resolved, "oobe_update_phase_shadow"));
}
[Fact]
public void AdvisoryVariant_IsExempt()
{
// Mutation proof: an unresolved ESP terminal failure does not un-happen because
// Windows installs an update afterwards.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
Assert.True(DecisionEngine.IsInOobeUpdatePhase(state));
var builder = state.ToBuilder();
builder.EspAdvisoryFailureRecordedUtc = new SignalFact<DateTime>(T0.AddMinutes(21), 30);
state = builder.Build();
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("esp_terminal_failure", SingleTimelineEffect(step, "enrollment_failed").Parameters!["reason"]);
}
[Fact]
public void NoPhase_WithoutAnAnchor()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
Assert.Equal(DecisionEngine.OobeUpdatePhaseKind.None,
DecisionEngine.EvaluateOobeUpdatePhase(state, T0.AddMinutes(51)).Kind);
}
// ================================================= serialization + census ====
[Fact]
public void SnapshotRoundTrip_PreservesTheUpdateFacts_AndTheStop()
{
// The update ends with a restart; the agent restores the engine state from its
// snapshot, so the phase — and a decided stop — must survive it.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState;
var stopped = FireUntilResolved(engine, state, firstOrdinal: 100).Last().NewState;
var roundtripped = StateSerializer.Deserialize(StateSerializer.Serialize(stopped));
var facts = roundtripped.OsUpdateFacts;
Assert.Equal(T0.AddMinutes(22), facts.PostExitActivityUtc!.Value);
Assert.Equal(T0.AddMinutes(48), facts.RebootRequiredUtc!.Value);
Assert.Equal(T0.AddMinutes(53), facts.RestartUtc!.Value);
Assert.Equal(OsUpdateSteps.RebootRequired, facts.LastStep!.Value);
Assert.Equal(StoppedWaitingReasons.OobeUpdateNoSignIn, roundtripped.StoppedWaitingReason!.Value);
Assert.Equal(OobeUpdatePhaseModes.Active, roundtripped.ScenarioObservations.OobeUpdatePhaseMode!.Value);
}
[Fact]
public void LegacySnapshot_WithoutTheNewFacts_DeserializesToEmpty()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
var legacy = JObject.Parse(StateSerializer.Serialize(state));
foreach (var prop in legacy.Properties()
.Where(p => p.Name.IndexOf("OsUpdateFacts", StringComparison.OrdinalIgnoreCase) >= 0
|| p.Name.IndexOf("StoppedWaitingReason", StringComparison.OrdinalIgnoreCase) >= 0)
.ToList())
{
prop.Remove();
}
var roundtripped = StateSerializer.Deserialize(legacy.ToString());
Assert.Same(OsUpdateFacts.Empty, roundtripped.OsUpdateFacts);
Assert.Null(roundtripped.StoppedWaitingReason);
Assert.False(DecisionEngine.IsInOobeUpdatePhase(roundtripped));
}
[Fact]
public void Census_SurfacesThePhase_AndTheStop()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine);
Assert.DoesNotContain("oobe_update_phase", DecisionStateSignalCensus.Build(state).SignalsSeen);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.Installed)).NewState;
var census = DecisionStateSignalCensus.Build(state);
Assert.Contains("oobe_update_phase", census.SignalsSeen);
Assert.Equal(T0.AddMinutes(22).ToString("o"), census.SignalTimestamps["oobeUpdatePhase"]);
Assert.DoesNotContain("stopped_waiting", census.SignalsSeen);
var stopped = FireUntilResolved(engine, state, firstOrdinal: 100).Last().NewState;
Assert.Contains("stopped_waiting", DecisionStateSignalCensus.Build(stopped).SignalsSeen);
}
[Theory]
[InlineData(OobeUpdatePhaseModes.Active, "Active")]
[InlineData(OobeUpdatePhaseModes.Shadow, "Shadow")]
[InlineData(null, "Shadow")] // never stamped
public void Census_NamesTheModeThePhaseRanIn(string? mode, string expected)
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: mode);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
var evidence = (IDictionary<string, object>)DecisionStateSignalCensus.Build(state).SignalEvidence["oobeUpdatePhase"];
Assert.Equal(expected, evidence["mode"]);
}
// ======================================================= measurement mode ====
[Fact]
public void ShadowMode_ReportsWhatThePhaseWouldDecide_AndTodaysRuleDecides()
{
// Report 3a2207978e4c in the Shadow mode: the fire mid-update fails the session as
// before, and says that the phase would have kept waiting.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: OobeUpdatePhaseModes.Shadow);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(35), OsUpdateSteps.InstallStarted)).NewState;
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("DeadlineFired:advisory_completion", step.Transition.Trigger);
Assert.Equal("esp_exit_without_completion_evidence", SingleTimelineEffect(step, "enrollment_failed").Parameters!["reason"]);
var shadow = SingleTimelineEffect(step, "oobe_update_phase_shadow");
var parameters = shadow.Parameters!;
Assert.Equal("Debug", parameters["severity"]);
Assert.Equal(OobeUpdatePhaseModes.Shadow, parameters["phaseMode"]);
Assert.Equal("update_running", parameters["phaseState"]);
Assert.Equal("hold", parameters["wouldDecide"]);
Assert.Equal("esp_exit_without_completion_evidence", parameters["decidedReason"]);
Assert.Equal(T0.AddMinutes(22).AddHours(3).ToString("o"), parameters["phaseBoundUtc"]);
Assert.Equal(T0.AddMinutes(22).ToString("o"), parameters["oobeUpdatePhaseStartedUtc"]);
Assert.Equal(OsUpdateSteps.InstallStarted, parameters["osUpdateLastStep"]);
Assert.Equal(Lcu, parameters["osUpdate"]);
Assert.Contains("still installing an update", parameters["message"]);
// The report precedes the verdict on the timeline.
Assert.True(step.Effects.ToList().IndexOf(shadow) < step.Effects.ToList().IndexOf(SingleTimelineEffect(step, "enrollment_failed")));
}
[Fact]
public void ShadowMode_AfterTheUpdatesRestart_ReportsTheSignInWait()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: OobeUpdatePhaseModes.Shadow);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState; // re-bases the window to T0+83
var step = engine.Reduce(state, AdvisoryFired(70, T0.AddMinutes(83)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
var parameters = SingleTimelineEffect(step, "oobe_update_phase_shadow").Parameters!;
Assert.Equal("awaiting_sign_in", parameters["phaseState"]);
Assert.Equal("hold", parameters["wouldDecide"]);
Assert.Equal(T0.AddMinutes(53).ToString("o"), parameters["oobeUpdateEndedUtc"]);
Assert.Equal(T0.AddMinutes(113).ToString("o"), parameters["phaseBoundUtc"]);
}
[Fact]
public void ShadowMode_HangingUpdate_ReportsTheNotFinishedFailure()
{
// A fire after the cap (a shadow session lives this long only when another guard held
// it, e.g. enforcement progress) reports the not-finished failure the phase would give.
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: OobeUpdatePhaseModes.Shadow);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.InstallStarted)).NewState;
var afterCap = T0.AddMinutes(22).AddHours(3).AddMinutes(5);
var armed = FindDeadline(state, DeadlineNames.AdvisoryCompletion)!;
state = state.ToBuilder()
.CancelDeadline(DeadlineNames.AdvisoryCompletion)
.AddDeadline(new ActiveDeadline(armed.Name, afterCap, armed.FiresSignalKind, armed.FiresPayload))
.Build();
var step = engine.Reduce(state, AdvisoryFired(50, afterCap));
Assert.Equal("esp_exit_without_completion_evidence", SingleTimelineEffect(step, "enrollment_failed").Parameters!["reason"]);
var parameters = SingleTimelineEffect(step, "oobe_update_phase_shadow").Parameters!;
Assert.Equal("update_not_finished", parameters["phaseState"]);
Assert.Equal("fail_oobe_update_not_finished", parameters["wouldDecide"]);
}
[Fact]
public void UnstampedSession_CountsAsShadow()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: null);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("update_running", SingleTimelineEffect(step, "oobe_update_phase_shadow").Parameters!["phaseState"]);
}
[Fact]
public void OffMode_EvaluatesNothing()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: OobeUpdatePhaseModes.Off);
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.False(HasTimelineEffect(step, "oobe_update_phase_shadow"));
}
[Fact]
public void ShadowMode_WithoutPhase_ReportsNothing()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine, mode: OobeUpdatePhaseModes.Shadow);
var step = engine.Reduce(state, AdvisoryFired(50, T0.AddMinutes(51)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.False(HasTimelineEffect(step, "oobe_update_phase_shadow"));
}
[Fact]
public void ModeStamp_ActiveCanOnlyBeTakenAway_NeverGranted()
{
var engine = new DecisionEngine();
var initial = engine.Reduce(DecisionState.CreateInitial("s", "t", T0), MakeSignal(0, DecisionSignalKind.SessionStarted, T0)).NewState;
// A later agent start in Shadow or Off takes Active away (kill switch) ...
var state = engine.Reduce(initial, PhaseMode(1, T0, "active")).NewState;
Assert.Equal(OobeUpdatePhaseModes.Active, state.ScenarioObservations.OobeUpdatePhaseMode!.Value);
state = engine.Reduce(state, PhaseMode(2, T0.AddMinutes(60), OobeUpdatePhaseModes.Active)).NewState;
Assert.Equal(1, state.ScenarioObservations.OobeUpdatePhaseMode!.SourceSignalOrdinal);
state = engine.Reduce(state, PhaseMode(3, T0.AddMinutes(90), OobeUpdatePhaseModes.Off)).NewState;
Assert.Equal(OobeUpdatePhaseModes.Off, state.ScenarioObservations.OobeUpdatePhaseMode!.Value);
Assert.Equal(3, state.ScenarioObservations.OobeUpdatePhaseMode.SourceSignalOrdinal);
// ... and switching it on mid-session applies from the next session only.
state = engine.Reduce(state, PhaseMode(4, T0.AddMinutes(120), OobeUpdatePhaseModes.Active)).NewState;
Assert.Equal(OobeUpdatePhaseModes.Off, state.ScenarioObservations.OobeUpdatePhaseMode!.Value);
var shadow = engine.Reduce(initial, PhaseMode(1, T0, "unknown-mode")).NewState;
Assert.Equal(OobeUpdatePhaseModes.Shadow, shadow.ScenarioObservations.OobeUpdatePhaseMode!.Value);
shadow = engine.Reduce(shadow, PhaseMode(2, T0.AddMinutes(60), OobeUpdatePhaseModes.Active)).NewState;
Assert.Equal(OobeUpdatePhaseModes.Shadow, shadow.ScenarioObservations.OobeUpdatePhaseMode!.Value);
}
[Fact]
public void ActiveTakenAwayAfterTheRestart_TheFireFallsBackToTodaysRule()
{
var engine = new DecisionEngine();
var state = HandoffExitArmsTheWindow(engine); // Active
state = engine.Reduce(state, OsUpdate(40, T0.AddMinutes(22), OsUpdateSteps.DownloadStarted)).NewState;
state = engine.Reduce(state, OsUpdate(41, T0.AddMinutes(48), OsUpdateSteps.RebootRequired, "KB5099999")).NewState;
state = engine.Reduce(state, Reboot(60, T0.AddMinutes(53))).NewState;
// The agent comes back after the update's restart with the phase switched back to Shadow.
state = engine.Reduce(state, PhaseMode(61, T0.AddMinutes(54), OobeUpdatePhaseModes.Shadow)).NewState;
var step = engine.Reduce(state, AdvisoryFired(70, T0.AddMinutes(83)));
Assert.Equal(SessionStage.Failed, step.NewState.Stage);
Assert.Equal("awaiting_sign_in", SingleTimelineEffect(step, "oobe_update_phase_shadow").Parameters!["phaseState"]);
}
}
}
@@ -523,7 +523,8 @@ namespace AutopilotMonitor.DecisionCore.Tests
.WithEspSyncFailureTimeoutMinutes(value: 90, sourceSignalOrdinal: 5)
.WithRegistrySelfDeployingProfile(value: false, sourceSignalOrdinal: 6)
.WithCloudPc(value: true, sourceSignalOrdinal: 7)
.WithAccountSetupProgressAtWhiteGloveSuccess(value: false, sourceSignalOrdinal: 8))
.WithAccountSetupProgressAtWhiteGloveSuccess(value: false, sourceSignalOrdinal: 8)
.WithHelloWaitTimeoutSeconds(value: 1800, sourceSignalOrdinal: 9))
.Build();
var json = StateSerializer.Serialize(state);
@@ -541,6 +542,8 @@ namespace AutopilotMonitor.DecisionCore.Tests
Assert.Equal(7, o.CloudPc!.SourceSignalOrdinal);
Assert.False(o.AccountSetupProgressAtWhiteGloveSuccess!.Value);
Assert.Equal(8, o.AccountSetupProgressAtWhiteGloveSuccess!.SourceSignalOrdinal);
Assert.Equal(1800, o.HelloWaitTimeoutSeconds!.Value);
Assert.Equal(9, o.HelloWaitTimeoutSeconds!.SourceSignalOrdinal);
// Never-observed facts must rehydrate as null (null ≠ veto — session 62e603c9).
Assert.Null(o.WhiteGloveSealingPatternSeen);
@@ -90,6 +90,11 @@ namespace AutopilotMonitor.DecisionCore.Tests.State
lastClassifierVerdictId: "verdict-" + seed));
}
if (type == typeof(RealmJoinFacts)) return RealmJoinFacts.Empty.WithDetected(BaseUtc.AddMinutes(seed), ordinal);
if (type == typeof(OsUpdateFacts))
{
return OsUpdateFacts.Empty.WithActivity(
BaseUtc.AddMinutes(seed), ordinal, OsUpdateSteps.RebootRequired, "KB-" + seed, isPostExit: true);
}
throw new Xunit.Sdk.XunitException(
$"No value factory for builder property type {type}. A new DecisionState field " +
@@ -455,6 +455,25 @@ public class AuthFunctionTests
Assert.Equal(Constants.TenantRoles.Operator, (string)body.role);
}
[Fact]
public void AutoAdmin_NotTriggered_ForClaimDerivedViewer_WhenNoTableMembers()
{
// Same guard for the read-only claim role: an Entra app-role Viewer in a claim-only tenant
// stays a Viewer and is never written into the table as Admin.
var result = AuthFunction.BuildAuthResult(
DefaultConfig(), isGlobalAdmin: false, isGlobalReader: false, isPreviewApproved: true,
memberRole: ViewerRole(), // came from the "roles" claim
mcpCheck: McpDenied(),
hasTenantAdmins: false,
TenantId, Upn, DisplayName, ObjectId);
Assert.True(result.IsSuccess);
Assert.False(result.NeedsAutoAdmin);
var body = ToDynamic(result.Body);
Assert.False((bool)body.isTenantAdmin);
Assert.Equal(Constants.TenantRoles.Viewer, (string)body.role);
}
[Fact]
public void AutoAdmin_NotTriggered_WhenUserAlreadyAdmin()
{

Some files were not shown because too many files have changed in this diff Show More