Compare commits

...

98 Commits

Author SHA1 Message Date
Oliver Kieselbach 977c5fb447 fix(web): headroom + symmetric insets for network-timeline marker labels
A left-edge "Clock -3h 0m" label collided with the HTML legend above the
band (first live fleet session d0c5b672). Marker-label headroom grows to
48 units so long labels stay inside the SVG, and the drawing content is
now inset 64 units on BOTH sides - the left inset mirrors the existing
right one, giving edge labels room and a calmer, symmetric frame.

Verified against the live session carrying the first system_clock_changed
event (backfilled step -> marker only, no rebase, as designed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 20:14:29 +02:00
Oliver Kieselbach 27fdd19568 feat(agent): optional Delivery Optimization GroupId from network fingerprint
EnableDoGroupIdAutoSet (ConfigVersion 40, default off): the agent seeds the
DOGroupId policy value with a deterministic GUID derived from the default
gateway's IPv4 + SendARP MAC, RealmJoin-compatible byte layout so mixed
fleets peer together. Foreign DOGroupId/DOGroupIdSource policies always win
(ownership tracked via StartupEventGate fingerprint); no random-GUID
fallback. New do_group_id_auto_set event, portal toggle, admin report row,
manifest/catalog sync, OKF doc. Gateway heuristic deduplicated into
NetworkInterfaceLocator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 20:09:45 +02:00
Oliver Kieselbach 72f7adcb3d feat(web): clock-era axis + sleep segments in network timeline
Consume the two new agent events:

- system_clock_changed: live-observed steps (timeDeltaMs authoritative,
  >=30s; sub-30s NTP nudges ignored) split the stream into clock eras and
  rebase every later timestamp by the cumulative delta, so the axis shows
  true elapsed time. Backfilled steps (pre-agent, replayed from the event
  log) are marked but never rebased. Each step gets a slanted "Clock +1h"
  lifecycle marker with old->new time and reason in the tooltip.
- system_sleep_episode: episodes (emitted retroactively at wake) carve an
  "Asleep" segment out of the active network instead of attributing the
  quiet period to it; tooltip carries kind (Modern Standby/sleep), wake
  source and AC/battery. Episode boundaries get the same era correction
  as the event that carried them.

Stats row gains "Asleep total" and "Clock changes"; the event list
includes both new types (now titled "Network & system events").

Layout: drawing content stops 64 units short of the right viewBox edge
(CONTENT_W), so right-edge lifecycle labels slant up-right like all
others instead of flipping left.

7 new vitest cases pin era rebasing (live/backfilled/nudge), sleep
overlay incl. era-corrected boundaries, and hotspot confidence.

Live verification against fleet data pending - the emitting agent was
released minutes ago; no session carries the events yet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:57:19 +02:00
Oliver Kieselbach 06cf1c4c12 feat(agent): system timeline watcher — clock steps + sleep episodes as ground truth
Two enrollment blind spots become timeline ground truth from the System event
log via one EventLogWatcher (WindowsUpdateTracker pattern, 24h backfill with
RecordId watermark recovering pre-agent records):

- system_clock_changed (Kernel-General 1): oldTime/newTime/signed timeDeltaMs,
  reason and the setting process. 2s floor suppresses w32time 1ms micro-slews;
  steps >= 5min are Warning + immediate upload. Payload instants are the
  authoritative source (event timestamps can be clamped server-side).
- system_sleep_episode (Power-Troubleshooter 1 classic sleep/hibernate,
  Kernel-Power 507 Modern Standby; 506 deliberately unconsumed): kind,
  enteredAt/exitedAt, durationSeconds, wake source / AC state. 60s real-sleep
  floor filters Modern Standby screen-off scenarios.

ConfigVersion 39 (Collectors.EnableSystemTimelineWatcher +
SystemEventBackfillLookbackMinutes). Both types are non-activity on agent and
backend (idle/stall clocks unaffected; a sleeping device stays Stalled).

Consumers: TimeJumpBadge gains the ground-truth cause "clock-set"
(classifyTimeJump matches backward display steps against recorded clock sets);
EventTimeline badge blocks for both events plus the previously bare
power_state_change; SessionInfoCard keeps the wall-clock duration and adds a
standby note; TimeAttributionCalculator builds SleepSpans (RebootSpan clone,
AttributionVersion 3) persisted as SleepSeconds/SleepSpansJson and rendered in
TimeAttributionLane; ANALYZE-DEV-012 (info, single) fires once per session for
episodes >= 5min - deliberately the only new rule, clock findings stay with
DEV-008 to avoid duplicate-finding flood.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:41:56 +02:00
Oliver Kieselbach e612c69178 feat(web): feature-form default for the grant command + docs link
The generated PowerShell command now uses the readable -Features form
(first catalog feature as the example) instead of raw permission strings;
a note below lists all available values incl. the All meta-feature and
how to fall back to -Permissions via Copy permissions. The section links
to the Optional Graph Permissions customer docs (via DOCS_URL registry),
and the "What this script does" bullet reflects the Cloud Shell
ambient-identity sign-in. Script help no longer claims the admin UI
generates the -Permissions form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:27:51 +02:00
Oliver Kieselbach 829a841d9b feat(web): slanted inline labels for network-timeline lifecycle markers
The gray lifecycle dots (Reboot / Desktop / Completed) were hard to hit
and mute without hovering. Their label is now written directly at the dot,
slanted up-right (flipping up-left near the right edge so it stays inside
the drawing), with the band shifted down to make headroom. The whole
marker group (line, dot, label) shares an enlarged invisible hit target,
and a label is suppressed when its neighbor sits too close - the tooltip
still carries the details.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:09:03 +02:00
Oliver Kieselbach 10eedded13 feat(web): confidence tier for network-timeline hotspot badge
An SSID matching the typical iPhone hotspot name (".. iPhone") is strong
evidence on its own — such networks now get a plain "Hotspot" badge, while
subnet-only evidence keeps the hedged "Hotspot?". Tooltip and footnote
explain the distinction; detection also fires on iPhone-named SSIDs
outside the Apple hotspot subnet.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 19:00:56 +02:00
Oliver Kieselbach a9f6ccd86d feat(web): power + network status chips on the progress portal
Taskbar-style chips in the progress page status header showing the
device's last-reported power state (AC/battery with percent, low-battery
warning) and network state (WiFi with SSID tooltip + signal tiers,
Ethernet, offline). Derived purely client-side by folding the event
stream the portal already downloads (power_state_check/change,
network_interface_info, network_state_change, wifi_signal_info) - no
backend, API, or auth change. Chips render only while InProgress and
disappear gracefully for sessions from agents without these events;
devices without a battery show no power chip (Windows-tray parity).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 18:56:04 +02:00
Oliver Kieselbach 7ba67df1a1 Agent instructions 2026-08-21 18:39:35 +02:00
Oliver Kieselbach e214e8dc1a feat(scripts): -Features All meta-value grants every optional permission at once
All is a script-side convenience that expands to every GraphFeatureCatalog
feature (works with -Revoke too). The catalog sync test now pins All in the
ValidateSet while keeping it out of the catalog comparison, and the admin UI
mentions the shorthand next to the generated grant command.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 18:32:48 +02:00
Oliver Kieselbach f8d0ccd2f5 feat(web): per-session network timeline page + device-details deep link
New /sessions/network-timeline?id=&tenantId= page visualizing every network
the device was on during enrollment: one colored segment per network
(Ethernet / WiFi SSIDs, fixed-order validated palette), hatched offline
gaps, ESP phase context lane, lifecycle + connectivity-check markers with
hover/tap tooltips, stats row and the raw network event list.

Segment model derives from network_interface_info + network_state_change
(before/after), enriches SSIDs from wifi_signal_info, infers missing
post-reconnect SSIDs via matching gateway, and flags probable smartphone
hotspots from vendor-default subnets (Apple 172.20.10.x, Windows
192.168.137.x, legacy Android 192.168.43.x) - always as a heuristic badge.
Axis is honest: linear per segment, minimum width for short segments,
boundary ticks show real times. Mobile scrolls horizontally.

Entry point: "Network timeline" deep link in DeviceDetailsCard's Network
section via new networkTimelineUrl() builder. Route registered in the SWA
config (exact routes before the /sessions/* wildcard, incl. the index.txt
RSC route) and excluded from LegacyPathRedirect's /sessions/{id} rewrite
via the new static-siblings set (test pinned).

Planned follow-up once the agent emits them: clock-change events (rebase
axis into clock eras) and standby/Modern-Standby events (render sleep as
its own segment kind).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 18:26:49 +02:00
Oliver Kieselbach c9701301ea feat(scripts): Cloud Shell ambient-identity sign-in for Grant-AutopilotMonitorAddOn
Try Connect-MgGraph -Identity first (Azure Cloud Shell issues a delegated
token for the signed-in admin, avoiding the device-code flow that
Conditional Access commonly blocks), probe Graph readability, and fall
back to interactive sign-in with explicit scopes otherwise. The
AppRoleAssignment.ReadWrite.All scope guard now applies only to the
interactive path, since ambient-identity tokens carry a fixed scope set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 18:24:31 +02:00
Oliver Kieselbach ba2a390a83 fix(agent): battery-instance fallback for the dead Win32_PowerManagementEvent provider
On Windows 11 24H2/25H2 (observed on builds 26200 and 26220, session
161b838c) the MS_Power_Management_Event_Provider is still registered in the
WMI repository but activating the extrinsic push query fails with
WBEM_E_NOT_FOUND - the legacy provider no longer loads on exactly the OS
builds new devices enroll with, so the power watcher degraded fleet-wide.

Arm chain: try the push event first (still alive on older builds), then
fall back to the intrinsic __InstanceModificationEvent WITHIN 30 on
Win32_Battery. The WITHIN sampling runs inside WinMgmt, not in the agent;
the handler path is unchanged (payload ignored, re-probe + diff), worst-case
detection latency ~35s incl. debounce. collector_degraded now fires only
when BOTH attempts fail; the arm log line names the active mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:49:03 +02:00
Oliver Kieselbach dbd7d2ac6f Page-Size-Option „5" 2026-08-21 15:26:03 +02:00
Oliver Kieselbach 8f24876682 feat(agent): live power-state watcher + battery analyze rules DEV-009/010
Event-driven AC/battery observation during enrollment (no polling): a WMI
Win32_PowerManagementEvent (EventCode 10) push subscription re-probes
GetSystemPowerStatus and diffs snapshots through a pure tracker into
power_state_change events - transition ac_to_battery (Warning, immediate) /
battery_to_ac (Info) / threshold_crossed at 50/30/15% (per-level lifetime
latch, lowest-crossed-only on jumps, arm-time baseline covers sessions
starting low, 5s trailing-edge debounce, emission cap 20). Never armed on
devices without a battery; arm failure surfaces as collector_degraded.
Kill-switch Collectors.EnablePowerStateWatcher (ConfigVersion 38).

power_state_change is classified non-activity on both sides
(SignalActivityClassifier + IsPeriodicOrStallEvent) so battery drain can
neither keep idle collectors alive nor heal a Stalled session.

New analyze rules, both with on_event:power_state_change interim triggers
and deliberately single-required-condition (event_data conditions scan
events independently - no same-instance join):
- ANALYZE-DEV-009 (high): thresholdPercent equals 15
- ANALYZE-DEV-010 (warning): transition equals ac_to_battery

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 15:24:41 +02:00
Oliver Kieselbach 2d5ee887d7 fix(web): make rule JSON paste-import lossless in both rule editors
The JSON editors serialize the FORM object, but what users paste is usually a
rule EXPORT (e.g. received from another tenant). The create/edit whitelists
rebuilt collector parameters and evaluateOn from form fields the export does
not carry, so a pasted gather rule silently lost every collector parameter and
a pasted analyze rule silently reverted to the enrollment-end default.

Both rules pages get a single shape-tolerant mapper in types.ts:

- gatherRuleToForm unflattens the nested parameters object (authoritative when
  present, so edit-merge leftovers cannot leak), maps nullable rule fields to
  form defaults, imports a missing emitMode as 'always' (the rule's actual
  behavior) instead of the empty-form on_change default, and now also backs
  startEditing so edit and paste share one mapping.
- analyze jsonToForm maps evaluateOn onto the eval* checkboxes (an explicit
  evaluateOn wins over stale form flags in edit-mode merges); terminal-only
  exports fall back to the enrollment-end default.
- tags and the folded effective markSessionAsFailed now survive the import:
  both forms carry tags into the create/save payloads, analyze creates carry
  markSessionAsFailedDefault. notify/notifyChannelIds stay excluded — channel
  ids are tenant-specific.

9 new vitest cases pin both shapes, the merge precedence, and the round-trips;
full web suite 824/824, tsc and eslint clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 14:29:08 +02:00
Oliver Kieselbach 81c14bb3ed fix(rules): close the rule-ID namespace gaps around the shared global keyspace
Custom rules are tenant-partitioned and functionally collision-free, but three
surfaces still keyed on the bare non-unique ruleId string:

- RuleStats global aggregate (global_{ruleId}) summed same-ID custom rules of
  unrelated tenants (title/severity last-writer-wins). The global row is now
  catalog-only: analyze gates on IsBuiltIn/IsCommunity, gather gates on the
  reserved-namespace pattern (agent events carry no flag; custom rules can
  never occupy the reserved pattern).
- The 'never ship a built-in in the CUSTOM namespace' commitment from
  RuleIdPolicy was documented but unenforced: combine.js now fails on
  duplicate or non-reserved built-in IDs, BuiltInRuleCatalogPolicyTests pins
  the same invariants on the embedded catalog, and the GitHub reseed rejects
  non-reserved IDs before writing to the global partition.
- DeleteRuleStatesForRuleIdAcrossTenantsAsync (cross-partition delete by bare
  RowKey) now refuses IDs outside the reserved namespace with failed=-1 so
  callers abort the follow-up global catalog delete.

Verified: prod global partitions (45 analyze + 11 gather) are fully inside the
reserved namespace, so the guards touch no existing data. New OKF doc
docs/rules/rule-id-namespace.md captures the identity/collision model.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 14:13:46 +02:00
Oliver Kieselbach 3a0bd74b12 fix(backend): sequence the synthetic vulnerability_report event instead of defaulting to 0
The async queue correlation stored the event with Sequence 0, sorting it to
the top of the timeline (and tripping the time-jump badge) even though it
arrives last. Follow the BuildSessionTimeoutEvent/BuildAgentEmergencyBreakEvent
pattern: max+1 so it sorts last; WhiteGlove Part 1 anchors at the Part-1
terminal markers so a late device_setup report can never cross the web
timeline's sequence-based Part-1/Part-2 split. Also upsert the EventType
index the queue path never wrote, making the event visible to cross-session
search.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 13:57:16 +02:00
Oliver Kieselbach 50e7590060 feat: app install durations measure the status-defining attempt
The IME processes the ESP app list in multiple passes (device-ESP evaluation
ending Skipped, real install up to an hour later), and AppInstallSummary
folded first observation -> last terminal across them: a 23-second app late
in the queue reported 4002 s (session 44862cd3, "Set TimeZone").

Backend ingest: new max-folded LastAttemptStartedAt anchors DurationSeconds
(CompletedAt - last attempt start; legacy rows without the anchor keep the
span fallback), replay-guarded additive InstallPassCount, attempt-scoped
DownloadDurationSeconds pairing, and a weaker-terminal guard (completed with
state Skipped/Postponed never overrides Installed/Error, in-batch and in the
cross-batch reconcile). StartedAt stays the earliest-wins window/version-
ordering key. AppsDashboardProjection carries the new columns.

AppVersionRegressionRadar: cutover gate — only attempt-anchored rows enter
the duration population on BOTH comparison sides, so mixed-semantics medians
can neither mask real regressions nor falsely re-arm active episodes.

Time attribution: per-pass segment pairing replaces the first-start ->
last-terminal hull. An app's seconds = sum of its ACTIVE segments (failed
attempts and evaluation passes count; inter-pass idle gaps don't), occupancy
merges the segments, hull endpoints remain for display and what-if bounds.

Agent: a terminal->active transition re-arms BOTH timing endpoints, so event
payload durationSeconds also describes the attempt.

Web/PDF/MCP: labels and tooltips say "install time (final attempt)", the
apps sessions table gets a xN multi-pass chip, the time-series chart carries
a cutover footnote, MCP tool descriptions document the semantics split
between get_app_install_metrics and get_time_attribution.

No backfill: pre-cutover rows keep span values (documented in the public
changelog). OKF doc: docs/backend/app-install-duration-attempt.md.

Suites: Functions 4383, Agent 2407, MCP 564, Web 814 - all green, build 0/0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 13:40:47 +02:00
Oliver Kieselbach 61e53a2e9b feat: lead duration cards with median + P90 instead of outlier-skewed average
Enrollment durations are heavily right-skewed; a handful of overnight/WhiteGlove
outliers made the Fleet Health and Dashboard average cards read like hours
(541 min) and answer no real question. Both cards now lead with the median
(robust 'typical enrollment' answer) and keep the tail visible via a P90
subtext. AvgDurationMinutes stays in both DTOs for API compatibility.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 12:49:42 +02:00
Oliver Kieselbach 7e7b6f8086 fix(backend): declare LastIngestAt in the SessionsIndex manifest
bbc7a1d8 stamped LastIngestAt into IncrementSessionEventCountAsync's merge
entity intending it as Sessions-only ("deliberately not mirrored"), but that
same entity flows through SyncSessionIndexAsync into the index merge — the
field was already landing in SessionsIndex on every ingest batch, just
undeclared. The fail-soft drift guard has been warning ~1,200x/h in prod
since the 08-20 deploy, and a StartedAt-shift full rebuild would silently
drop the column, making index-served session reads lose lastIngestAt that
Sessions-served reads carry.

Declare it properly instead of suppressing: ConditionallyProjected in
SessionIndexFieldManifest, projected in BuildSessionIndexEntity (read side
MapToSessionSummary already handled it), sample value pinned in the
bidirectional manifest tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 01:54:34 +02:00
Oliver Kieselbach 2e20476559 feat(agent): extend RealmJoin timeout while first deployment is demonstrably active
The 60-min RealmJoinTimeout deadline arms at RJ detection (the RJ agent MSI
install during DeviceSetup) and used to cut off monitoring mid-install: report
55e6afd61c9d (session 46a96078) lost the tail of a large package catalog whose
last observed package completed 3 seconds before the deadline, with the first
deployment only starting 16 minutes into the window.

When the deadline now fires with LastDeploymentPhase 100/101 and deployment
activity (phase change or package observation, persisted restart-safe in
RealmJoinFacts.LastActivityUtc; the detection-time phase capture deliberately
does not count) inside the last 60 min, the handler re-arms to
lastActivity + 60 min instead of timing out — Info event
realmjoin_timeout_extended — clamped at an absolute cap of detection + 4 h.
realmjoin_timeout carries a reason payload (hard_timeout / inactivity /
absolute_cap); a fire of a deadline incarnation superseded by a re-arm
dead-ends via the OccurredAtUtc == DueAtUtc scheduler contract. The idle case
(no activity ever) still times out at 60 min, and phase-200/210 activity never
extends. Packages are only registry-visible at completion, so the inactivity
window keeps the full original 60 min.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 01:11:48 +02:00
Oliver Kieselbach 990641b19e feat: P14 - per-upload-batch send time separates spool delay from clock offset
Agent stamps X-Send-Time-Utc (device clock, ISO-8601 round-trip) per upload
ATTEMPT; ingest parses it (year>=2000 guard, deliberately no upper clamp) and
server-stamps SentAt onto every event of the request. SentAt - OccurredUtc is
pure spool delay, ReceivedAt - SentAt is network + device-vs-server clock
offset - previously indistinguishable and only separable via median statistics.

clock_skew condition: batches carrying SentAt measure the frame directly
(spool-immune by construction - no spread cap, IME/clamped exclusions
irrelevant, IME-only sessions become measurable; evidence sentAtBatchCount);
legacy batches keep the per-event median path unchanged. CmTraceSkewTripwire
deliberately untouched: its differential median math cancels clock offset as
common mode.

Full persistence chain: Store x2 + Map, fields= projection (sentAt), MCP
get_session_events field list + rule-authoring-guide. New OKF doc
backend/send-time-frame-separation.md.

Suites: Functions 4370 (+16), Agent 2407 (+1), MCP vitest 564 + tsc 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 00:44:29 +02:00
Oliver Kieselbach 3ca2f3b499 refactor(web): collapse time provenance behind an inline toggle
The labeled provenance rows duplicated values already visible in the
raw JSON dump and inflated the metadata block for every IME event. The
section is now collapsed by default behind a small "time provenance"
toggle sitting inline on the Created row (whose value it explains) -
zero extra height when collapsed, and the base label column returns to
its original width, so the default detail view is pixel-identical to
before P13. Expanding reveals the same rows, indented under Created.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-21 00:02:26 +02:00
Oliver Kieselbach b9fa760404 feat(web): P13 - CMTrace time provenance visible in the session timeline
The per-line self-anchoring provenance (sourceLocalTs, applied offset +
origin, observational writer offset, derived/rejected markers) reaches
the browser but was buried unlabeled in the raw JSON dump. Sequence
stays THE sort order - without exception (now pinned by a regression
test); the corrected UTC is display-only, and a backwards step of the
displayed time within sequence order is marked as information instead
of confusing the reader.

- lib/timeProvenance.ts: readTimeProvenance (camelCase/snake_case,
  tolerant of truncated payloads) + classifyTimeJump with cause
  precedence derived > rejected-source > era-offset; era-offset
  requires applied offsets on BOTH rows - it never guesses.
- TimeJumpBadge on the row AFTER the jump, computed against the
  previous RENDERED row (filters/phase boundaries respected); sky for
  known causes, amber otherwise; tooltip carries both times, the delta
  and the "true write order (sequence)" sentence. Threshold 5 min,
  field-calibrated on 20 sessions: 12/12 normal sessions show zero
  badges, the era-mixed regression session shows exactly its 6.
- Clamped chip (rare) + TIME PROVENANCE detail section: only the
  applied offset says "(applied)"; the writer offset is explicitly
  observational (sticky after era flip-backs); origin chips with
  forward-compatible fallback for unknown values.
- EnrollmentEvent gains timestampClamped/originalTimestamp (wire
  already carried them); formatUtcOffset joins lib/formatting.

Verified: 814 web tests (19 new), tsc/eslint/build clean; Playwright
against e9753578 (badges match the pre-measurement), 9f475bb2 (no
badge on line-anchored rows, detail section correct, raw mode shows
exactly the one predicted jump).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 23:54:39 +02:00
Oliver Kieselbach ab61839685 feat(backend): P6 split - CMTrace skew tripwire + customer clock_skew rule
Operator side: CmTraceSkewTripwire piggybacks on the terminal counter
reconcile's partition scan (zero extra reads) and compares median
ReceivedAt-OccurredUtc deltas of IME-derived vs other events. A clean
15-min-grid divergence (residual < 2 min, >=20 samples / >=3 distinct
batches per side) emits the CmTraceTimeSkewRegression ops event
(Agent/Warning, goal: never fires). Bias-dominated sessions are
suppressed; measuredWriterOffsetMinutes is never consulted (sticky
after era flip-backs). Kill switch: CmTraceSkewTripwireDisabled.

Customer side: new clock_skew analyze condition source (skewMetric
clock_jump | sustained_offset, threshold seconds, gt/gte magnitude)
behind built-in ANALYZE-DEV-008 (300 s = the Kerberos/Entra 5-min skew
boundary). Batch-median frames with spool-spread filter, plateau step
detection (a gradual backlog-drain ramp cannot fake a jump) and
end-state persistence; IME-derived events are excluded so a CMTrace
anchoring regression can never fire the customer rule. Evidence carries
field=skewSummary/value for {{skewSummary}} interpolation.

Full mirroring chain: schema enum + skewMetric property, shared
manifest + combine.js regeneration, MCP authoring guide + semantic
lint, web SOURCES/form/card, ops-event dual-register, docs. P5 is
subsumed by the tripwire. PercentileMath is now the single nearest-rank
implementation (Apps/PlatformMetrics delegate).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 22:30:10 +02:00
Oliver Kieselbach 34fd40c3ad fix(agent): refuse cross-file calibration anchors, make rejections visible
Field forensics from the 1412 verification run (session 067797d8): the
label/anchor crossing recurred on a provably genuine binary, and the
new anchor-timestamp tripwire delivered — the "+02:00 measured for
IntuneManagementExtension.log" anchor (local 19:51:26.948) does not
exist in ANY log file on the device, and IME.log never produced its
expected first measurement during 24 s of growing passes beforehand.
Mechanism still unexplained; per-line anchoring keeps resolution
immune (all 58 line-anchored events of the session were correct), but
the observational layer must convict the culprit next time:

- CalibrateFrom now receives the iteration's ACTUAL file name next to
  the calibration key. A mismatch — the smoking gun the two field
  occurrences imply but static analysis says is impossible — logs a
  Warning with both names plus the anchor, and refuses the anchor, so
  even the unexplained path can no longer poison the measurement store.
- The first calibrator rejection per file logs at Debug (anchor local
  time + clock). The 24 s of silent non-measurement is exactly what
  made the phenomenon undiagnosable from the agent log.

232 IME tracker tests green against a verified-clean build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 20:18:35 +02:00
Oliver Kieselbach a0eb1b9e35 Merge branch 'main' of https://github.com/okieselbach/Autopilot-Monitor 2026-08-20 19:45:40 +02:00
Oliver Kieselbach 9d677db2a8 docs(okf): CMTrace time resolution — per-line self-anchoring concept
Why the reader-zone assumption and the per-file offset application both
failed, the per-line self-anchoring rule with its strict freshness
contract (LastCheckedUtc on every poll look, 30 s window, no freshness
across restarts), the provenance vocabulary that keeps every event
re-derivable, and the documented limits: the bounded grid-replay edge,
snapshot readers staying on the marked fallback, and the still-open
e9753578 label/anchor crossing with its tripwire.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 19:28:54 +02:00
Oliver Kieselbach c728a0bd41 feat(backend): surface genuine agent binary-integrity mismatches as ops events
IntegrityCheckFailed emergency reports used to land in App Insights
only — session e9753578's agent reported a hash mismatch and no product
surface showed it, which cost a day of mis-attributed root-causing.
They now materialize as an AgentBinaryIntegrityMismatch ops event
(Warning, Agent category, dual-registered in the web alert-rule
catalog per feedback_ops_event_types_dual_register).

With suppression for the KNOWN, accepted release race the operator
correctly called out: around every release, agents compare against
hash snapshots that do not belong to their own binary — an older agent
sees the new release's hash on its next config fetch, and a freshly
installed latest agent can receive the PREVIOUS release's hashes from
the 5-minute AdminConfiguration cache (e9753578: genuine 1410 exe
checked against the cached 1409 hash — proven by the 1409 archive
logging "verified OK" with exactly that hash). Both self-heal via the
forced self-update and are suppressed at materialization time against
the CURRENT config: reported running hash == published exe, or
reported version != latest. What remains — an agent claiming the
latest version whose binary is NOT the published one — is the real
signal: tamper, stale blob, or a build outside the release pipeline.
Missing config fails open to recording; visibility beats suppression.

Ops event only, deliberately no timeline event: the report is about
the binary, not the enrollment, and may precede session registration.

Backend suite 4320 green, web tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 19:28:43 +02:00
Oliver Kieselbach 64336c0c89 feat(agent): resolve fresh CMTrace lines by per-line self-anchoring
The era-aware successor to the reverted per-file offset application
(04b1a7c6). At a 100 ms poll, every line of a provably fresh pass was
written essentially "now", so its own distance to the agent clock,
rounded to the 15-minute offset grid, IS its writer's UTC offset — no
cross-line state, which is what interleaved writer eras break. The line
timestamp still contributes sub-poll precision and ordering; the grid
absorbs poll latency. Origin: LineAnchored → "line-anchored".

Freshness is stricter than "the file grew": every poll look stamps
LastCheckedUtc (empty first sights included, so AgentExecutor.log
anchors from its very first content), a pass anchors only when the gap
since the last look is ≤ 30 s (FreshLineMaxAge — far above the poll,
far below the 13-minute zone where a line's AGE rounds onto the grid),
and a restart-restored bookmark deliberately carries no freshness at
all — downtime backlog must never anchor. Everything not provably
fresh keeps the uniform reader-zone fallback: wrong in absolute terms,
self-consistent for durations, and marked as such (the revert's core
lesson — partially corrected is strictly worse than uniformly wrong).

Accepted, bounded edge (characterization test): a freshly WRITTEN line
carrying a replayed timestamp aged exactly N×15 min ± 2 min anchors to
≈now — an anchored line lands at now ± 2 min by construction, and
sub-24h replays are treated as current by the historic-replay guard.

The per-file measurement stays observational (Info log +
measuredWriterOffsetMinutes) and now logs the anchor's local timestamp
as a tripwire: session e9753578 logged "+02:00 measured for
IntuneManagementExtension.log" although that file provably held no
+2-era line — a label/anchor pairing the committed code could not be
shown to produce (iteration-local anchor, single sequential call site,
green repro of the exact field scenario, no +2-capable chunk in the
window). The mechanism remains open; the resolution path is immune to
it by design, and CmTraceCalibratorFileIsolationTests pins the file
isolation so a real regression can never ship silently.

Tests: 13 new — resolves-to-T across five writer beliefs, interleaved
eras inside one chunk, first-sight/empty-first-sight/poll-gap/restart
freshness boundaries (including the nastiest backlog: exactly 30 min
old, on-grid), the grid-replay edge, and the committed two-writer-era
fixture replayed at production poll cadence across its write silences.
Agent suite 2406 green against a verified-clean build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 19:28:27 +02:00
Oliver Kieselbach 59c944618b test(agent): move CMTrace fixture to tests/fixtures and anonymize it
Two corrections to 3c25172d, both found by asking why the repo suddenly had two
fixture locations.

Location: committed test data lives under tests/fixtures/. I had put this one
inside the agent test project without checking, creating a second parallel home.
It does NOT belong in tests/fixtures/enrollment-sessions/ either — that is a
governed corpus of anonymized DecisionSignal JSONL for the Replay Harness, with a
schema, category prefixes and a paired-scenario-test rule that raw log text cannot
satisfy. So: a sibling corpus, tests/fixtures/cmtrace-logs/, with its own README
stating why it is separate.

Anonymization: the fixture carried three real Intune policy/script GUIDs from a
customer tenant. Point 1 of the existing fixture policy requires tenant-scoped
identifiers to be replaced, and I committed without applying it — the check
belonged BEFORE the commit, not after. Replaced with
policy-anon-NNNN-0000-0000-0000-000000000000, consistent with the tenant-anon /
device-anon / user-anon convention already in use. The all-zero GUID is a literal
in IME's own output and stays; the session ID stays as our diagnostic anchor and
is already referenced from the commit history and tasks/todo.md.

The new README carries the commit policy for this corpus, including the rules that
were implicit before: verbatim content (the parser is tested against real
formatting), a header stating what the fixture pins, versioned filenames so a
pinned test cannot silently change meaning, and .cmtrace rather than .log because
*.log is globally gitignored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:54:55 +02:00
Oliver Kieselbach 3c25172dfb test(agent): real two-writer-era CMTrace fixture from session e9753578
Captured from ImeLogs/AgentExecutor.log of the session that exposed the regression
(agent 2.0.1410; device booted on the Pacific OOBE default, agent set W. Europe at
12:13:22Z).

One file, two writer eras: 371 lines rendered at local hour 05 by a writer still
on PDT (UTC-7) and 195 at hour 14 by one already on CEST (UTC+2) - nine hours
apart, interleaved, and no line carries a bias suffix, so nothing in the file
itself says which is which. Exactly the case the per-file calibration could not
express, and the test that was missing.

Analysing the archive also turned up a SECOND, independent defect the revert
commit did not know about: the agent log reports "IntuneManagementExtension.log
writer offset measured 02:00:00" while that file contains ONLY hour-05 lines
(2022 of them, zero at hour 14). A +2 measurement is impossible from its own
content, so the calibration anchor leaked across file boundaries - files are
walked in sorted order and AgentExecutor.log precedes it. Two-eras-per-file and
anchor-leak are separate bugs; both must be fixed.

Extension is .cmtrace.txt, not .log: *.log is globally gitignored, and a fixture
has no reason to claim that extension.

Fixture only, nothing consumes it yet - the era-aware calibrator is next.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 17:47:15 +02:00
Oliver Kieselbach eda61eaade Fix Tenant Groups layout overflowing on phones (#132) 2026-08-20 15:27:08 +02:00
Oliver Kieselbach 04b1a7c6e7 revert(agent): stop applying the measured CMTrace offset — one offset per file is wrong
Field regression, session e9753578 (agent 2.0.1410, PST-defaulted device in DE).
Five script_started events resolved 9 h early while their script_completed
counterparts resolved correctly, inflating every platform-script duration to
~32,400 s. Reverts the application introduced in e9dba11b.

Root cause: calibration keeps ONE offset per log file, but a single file can hold
lines from two writer eras. IME restarting across a timezone change leaves older
lines in the old zone and newer ones in the new zone, in the same file. The anchor
is the newest line, so its offset was also applied to the older lines, for which
it does not hold. The class doc even claimed "self-healing on writer restart" —
that reasoning only ever covered FUTURE lines, never the ones already on disk.

The more expensive mistake was the second one: correcting ABSOLUTE timestamps
without checking what depends on their DIFFERENCES. Applying the reader zone
uniformly is wrong in absolute terms but self-consistent — both ends of a duration
are wrong by the same amount, so the duration is right. Correcting only some
timestamps produced a mixed frame inside one session, which is strictly worse than
being uniformly wrong. This is the same asymmetry the original analysis described
for AppInstallSummaries (68 min reported against 8 min real) and then reproduced.

What stays, deliberately:
- Bias-carrying lines still resolve via the writer-declared offset; that path was
  never implicated.
- The measurement keeps running. It costs nothing, feeds the Info log, and is the
  field data the era-aware fix has to be built on — the first anlauf got the era
  question wrong from assumption rather than observation.
- Provenance now separates measured from applied (measuredWriterOffsetMinutes vs
  sourceOffsetMinutes). The broken event was tagged "calibrated" while carrying an
  offset that did not hold for its line; that confusion must not be expressible.
- The fallback path reports the offset it actually applied. An event stating no
  offset cannot be recomputed later, which defeats the point of provenance.

Provenance earned its keep here: 57 events measured -420 (the PDT writer) against
5 at +120 made the diagnosis a two-minute query instead of median statistics over
a session.

Next: era-aware calibration keyed on the IME-STARTED marker, which we already
track, plus a test over a log file containing two writer eras — the case that was
missing.

Agent 2393 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:31:50 +02:00
Oliver Kieselbach 3b59dab2df feat(agent): carry timestamp provenance on IME-derived events
Every IME-sourced event now states what the writer wrote, what offset was applied,
and how that offset was obtained:

  sourceLocalTs       raw local time, verbatim, NO zone suffix
  sourceOffsetMinutes the offset subtracted
  sourceOffsetOrigin  bias | calibrated | reader-zone-fallback

Why: until now the only record of a conversion was the converted value, so a wrong
offset was indistinguishable from a right one. Proving the field defect needed the
diagnostics archive — and of the 5,386 sessions that went through a timezone
change, essentially none had one, so the analysis had to fall back to median
statistics across thousands of sessions to establish what one stored value makes
obvious per session. It also makes a stored timestamp recomputable after the fact,
without an agent release.

No zone suffix on sourceLocalTs on purpose: the value genuinely carries none, and
formatting one in would re-introduce the assumption this whole change removes.

sourceOffsetMinutes is normalised to one sense (local = UTC + offset) across both
origins. The wire bias uses the opposite GetTimeZoneInformation convention
(UTC = local + bias), so a "+480" line is reported as -480; without that the field
would flip sign depending on where the offset came from and be useless to compare.

Added centrally in TagDerivedTimestamp, which all 14 emit sites already call, so
this does not scatter through the adapter. DataJson only — no table schema change,
so no Store/Map/seed chain and no backend deploy needed to start collecting it.

Agent 2393 / backend 4313 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:05:32 +02:00
Oliver Kieselbach 389426f8f2 fix(agent): future-skew guard that a one-hour timezone error can actually trip
ResolveOccurredAt rejected a source timestamp when ageHours < -1.0. That bound
could not fire for the single most common timezone error: with a one-hour offset
mistake the real value is parseLag - 3600 s, i.e. just ABOVE -1.0. It sat exactly
on the boundary of the case it was meant to catch, which is why every skewed
session sailed past it.

It was asymmetric too — a past-shifted error had to exceed 24 h before anything
noticed, while the measured field errors reached -17 h.

The bound is now derived rather than guessed: writer and reader share one system
clock, and the tracker only sees a line after it was written, so a correctly
converted source timestamp is always at or behind "now". Anything ahead means the
conversion was wrong. MaxFutureSourceSkew is 2 minutes of jitter tolerance —
deliberately far below the hour a timezone mistake produces.

The rejection log now names which bound tripped and by how much, instead of
printing a bare ageHours that reads the same for both directions.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:05:31 +02:00
Oliver Kieselbach e9dba11b96 fix(agent): resolve IME log timestamps with the writer's measured offset
This is the behaviour change the previous two commits prepared. ImeLogTracker no
longer substitutes its own timezone belief for the IME process's; it measures the
writer's offset from a freshly appended line and converts with that.

Freshness is stricter than "the file grew". GetSafePosition returns 0 for an
unknown file, so the first pass reads the whole file and its newest line may be
hours old — anchoring there would measure the line's age, not the writer's offset.
LogFilePositionTracker.HasSeen(), captured BEFORE GetSafePosition creates the
entry, gates calibration on growth against a size we had already observed.

Calibration runs at the END of a qualifying pass, so that pass still uses the
previously measured offset — at most one poll (100 ms) old, and an offset only
moves when a process restarts. Calibrating first would mean buffering the pass,
which is not viable: the first pass over AppWorkload.log can be hundreds of MB.
The cost is a warm-up of one growing pass, during which lines fall back to the
reader zone.

Bias-carrying lines are never used as anchors: they already state the writer's
offset, so they need no measurement and must not overwrite one.

CalibrateFrom logs at Info when the measured offset disagrees with this process's
own zone, naming the correction applied. That disagreement is precisely the
condition that silently corrupted every IME-derived timestamp until now; agreement
stays at Debug so the normal case does not add noise.

Tests are deliberately machine-independent: the assertion is always "a line
written at instant T resolves to T", asserted across writer beliefs of +2, +1, 0,
-7 and +10. That one property covers both field cases — the writer disagreeing
with us (the bug) and the writer agreeing with us (the majority of sessions, where
two stale beliefs cancel and which a fix to the agent's own cache would have
broken). Plus: no calibration on first sight, following a writer restart into a
new zone, and a declared bias honoured without being measured.

CheckLogFilesAsync is internal as a test seam — calibration depends on the pass
structure, which only a real read cycle exercises.

Agent 2390 / backend 4313 green against a verified-clean build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:49:14 +02:00
Oliver Kieselbach 828cbb5cd0 refactor(shared): CMTrace parser stops guessing the writer's timezone
Behaviour-neutral: 2381 agent and 4313 backend tests pass unchanged.

The parser resolved a bias-less line with DateTimeStyles.AssumeLocal followed by
ToUniversalTime(), i.e. with the READING process's cached zone. That silently
assumes reader and writer agree. They do not — each caches its zone at process
start and neither follows a later tzutil or Windows auto-timezone change — and the
assumption was buried inside a shared helper where no call site could see it.

CmTraceLogEntry now exposes what the line actually contains: LocalTimestamp
(Kind=Unspecified, because the value is meaningless without knowing the writer's
zone) and BiasMinutes. TimestampUtc is populated only when the writer declared its
bias, which is the one case where the offset is genuinely known. Timestamp is
removed rather than deprecated, so the compiler names every site that has to
decide — it immediately turned up ProcessLogMessageForTest, which grep had missed.

The defective path still exists, but now as one named function,
ResolveUtcAssumingReaderZone(), whose documentation states when it is wrong. All
three consumers call it and so keep their current behaviour exactly:
- ImeLogTracker routes every use through ResolveEntryUtc(), the single point where
  the measured offset replaces the fallback in the follow-up change;
- StallProbeCollector and LogParserCollector read file snapshots rather than
  tailing, so they cannot measure an offset and stay on the fallback for now.

The parser also no longer invents a timestamp. An unparseable time used to be
stamped DateTime.UtcNow inside the parser, making a fabricated value
indistinguishable from a real one downstream; it now reports HasTimestamp=false
and leaves the fallback to the caller, which alone knows what is safe in context
(StallProbeCollector still treats such a line as fresh — calling it stale would
hide progress).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:35:43 +02:00
Oliver Kieselbach 93a42ba5e4 feat(agent): measure the CMTrace writer's UTC offset instead of assuming it
A CMTrace line carries local time and no offset — IME 1.104 writes
DateTime.Now.TimeOfDay in both of its trace listeners. Converting it with the
agent's own TimeZoneInfo.Local assumes both processes agree on the zone. They do
not: each caches its zone at process start and neither follows a later tzutil or a
Windows auto-timezone change, so the error is Offset_writer - Offset_agent and is
zero only by coincidence.

Measured over 11,068 sessions: 26 carry a real zone-offset error, +1 h (GMT to
W. Europe, session bfff9b1c) through -17 h (a writer still on the Pacific OOBE
default while the agent had already moved to E. Australia). Note the sign — this
is why "make the agent's own clock honest" is the wrong fix: in the -17 h cases
the agent was correct and the writer was stale, and clearing the agent's cache
would have broken the majority of sessions where both are stale and the two errors
cancel.

The tracker tails at a 100 ms poll, so a line that appeared since the previous
pass was written essentially "now". That makes the writer's offset measurable:
round(lineLocal - agentUtcNow) onto the 15-minute grid every real UTC offset sits
on. Per source file, because IntuneManagementExtension.log and AppWorkload.log are
written by different processes; re-measured on every fresh anchor, so a writer
restart is picked up within one poll.

The freshness requirement is the load-bearing part and cannot be replaced by the
residual guard: a backlog line exactly 30 minutes old is itself a clean multiple
of the grid, rounds with zero residual, and would calibrate the offset half an
hour wrong. RejectsBacklogLine_EvenWhenItRoundsCleanly pins that the calibrator
cannot detect this on its own — only the caller's freshness rule prevents it.

Additive only; no call site uses it yet.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:25:48 +02:00
Oliver Kieselbach aaacc6ecb2 refactor(backend): order rule evaluation by Sequence, not Timestamp
Sequence is a persisted, strictly monotonic per-session counter assigned at emit
time (EventSequenceCounter: locked, saved before return, survives restarts). It is
immune to every clock problem. Timestamp carries the device clock frame plus, for
CMTrace-derived events, the writing process's timezone belief.

EvaluateAppInstallDurationCondition and EvaluateEventCorrelationCondition both
ordered by Timestamp and expressed "A happened before B" as a timestamp
comparison. Both now order by Sequence and compare Sequence for ordering.

Timestamp deliberately stays where an actual ELAPSED DURATION is meant:
durationSeconds, the evidence fields, and TimeWindowSeconds.

Scope, honestly: this is hardening, not a live bug fix. BuiltInAnalyzeRules.cs
contains zero event_correlation conditions, so the cross-frame join can only be
reached by a self-authored rule. And the app-install duration evaluator pairs
app_install_started with app_install_completed — both IME-sourced, one frame — so
its arithmetic was already correct. (The measured 68-vs-8-minute inflation comes
from EventIngestProcessor.AppInstall, which folds DO-collector download_progress
together with IME app_install_completed; that is fixed at the root in the agent.)

The TimeWindowSeconds check stays exposed: the window IS a duration, so there is
no alternative to a timestamp there. A +1 h skew on B makes any window below
3600 s fail to match. Only the agent-side calibration closes that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:13:09 +02:00
Oliver Kieselbach bbc7a1d8b2 fix(backend): agent-silence sweep must compare clock frames it owns
LastEventAt is the maximum AGENT-supplied event timestamp and therefore lives in
the device clock frame. The silence sweep compared it against a server-derived
cutoff, mixing two frames.

Measured over 11,068 sessions: 26 carry a genuine CMTrace timezone skew on their
IME-derived events, from -17 h to +1 h (a further 27 carry plain device-clock
offsets). At -17 h a live agent looks silent the moment it starts and is marked
Stalled with a customer-visible "Agent silent for 1020min"; at +1 h the sweep is
blind for an hour.

Adds SessionSummary.LastIngestAt, stamped from the server clock in
IncrementSessionEventCountAsync — reaching that method IS contact from the agent,
so it is written unconditionally rather than gated on latestEventTimestamp.

The sweep's server-side filter now spans both frames (LastIngestAt lt X or
LastEventAt lt X): filtering on LastIngestAt alone would silently drop every row
written before the field existed, because Table Storage does not match a filter
against a missing property. The authoritative decision moves into
ServerFrameLastContact() (LastIngestAt ?? LastEventAt ?? StartedAt) and is applied
in both stages — before the Stalled marker and in the stage-2 alive guard.

Deliberately not mirrored into SessionsIndex: the sweep reads Sessions, and the
index manifest is pinned bidirectionally by tests.

shared-manifests.json + .generated.ts regenerated — SharedManifestParityTests is
the drift brake and correctly went red on the new field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 13:12:49 +02:00
Oliver Kieselbach 5f00124a86 fix(web): keep PDF section titles with their content, show full exit-code text
Section headings now reserve keep-with-next space (charts reserve their full
height), so a title never strands at a page bottom while its table starts on
the next page. Table cells wrap up to 4 lines instead of single-line ellipsis,
and the exit-code column carries the full error description (the 24-char slice
is gone) - that text is what the app owner acts on. Failure-code column widths
rebalanced (Code 42 / Description 62 / Exit 60 / Count 16).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 09:42:57 +02:00
Oliver Kieselbach d43cfe2bb3 updated last modified page 2026-08-20 09:16:17 +02:00
Oliver Kieselbach 78cc038426 fix(agent): record real bytes-down when AutomaticDecompression strips Content-Length
The pipeline's AutomaticDecompression (GZip|Deflate) removes the response
Content-Length header on virtually every backend JSON response, so
net_total_bytes_down in agent_metrics_snapshot read 0 on healthy sessions.
When the header is absent, buffer the content (LoadIntoBufferAsync, 16 MB cap)
and record the DECOMPRESSED payload size — HttpClient buffers responses anyway
(default ResponseContentRead), so no extra memory cost. Fail-soft: a buffering
error keeps 0 and leaves the failure for the caller to observe.
AutomaticDecompression itself stays untouched (pinned by
MtlsHttpClientFactoryTests).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 01:25:41 +02:00
Oliver Kieselbach ff8b847dc0 fix(backend): fold do* fields from download_progress — DO data survives IME 1.104
IME 1.104 removed the [DO TEL] log line, leaving the agent's DO collector as
the only do_telemetry source — and its gate only fires when a 3s poll itself
observes the completed download. Fast, interrupted or between-poll-completed
downloads never emit one, so DoDownloadMode stayed -1 and DoAggregator dropped
the whole row: DO bytes read as "practically null" on 1.104 fleets even though
download_progress events carried the full do* set on every poll.

- Shared ApplyDoFields for both writers: download_progress folds byte counters
  and file size via Math.Max (monotonic, replay-idempotent) and sets the mode
  only when >= 0; do_telemetry stays authoritative (last-write-wins) but never
  regresses DoDownloadMode to the -1 unset sentinel.
- DownloadBytes now prefers actually-transferred bytes over doFileSize (the
  file size used to inflate the transfer measure).

Intended effect after deploy: DO aggregates count previously-invisible
incomplete downloads, so app counts/bytes rise. No backfill of historic rows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 01:25:28 +02:00
Oliver Kieselbach aab7ae12ff feat(mcp): get_agent_efficiency_metrics tool + honest truncated + full platform pass-through
- New GA-only tool get_agent_efficiency_metrics: near-pass-through of the
  server-side per-version aggregation (120s timeout, default 500 sessions,
  optional tenantId). Added to GA_FULL + PLATFORM_ONLY catalog snapshots.
- platformWindowEcho decides truncated on the backend's sessionsScanned
  (sessions actually covered) and only falls back to the has-snapshots count
  for old backends — the previous comparison read false even when the cap was
  hit exactly, whenever some scanned sessions carried no snapshots.
- get_platform_metrics: 90s timeout (backend fans out one storage query per
  session); spool block, crashRate, deliveryLatency, computedAt,
  computeDurationMs and fromCache are no longer dropped on the MCP side;
  description clarifies totalBytesDown is the agent's own HTTP traffic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 01:25:14 +02:00
Oliver Kieselbach 935f0e8c71 feat(backend): agent-efficiency aggregate + honest scan count + filtered metrics fetch
- New GET /api/global/metrics/agent-efficiency (GlobalReadOrAdmin, ?tenantId=):
  per-agent-version percentiles (CPU, working set, private bytes, threads,
  handles, spool, API latency/requests), crash rates per version, top offender
  sessions per dimension, 5-min cache keyed (days, limit, tenant). First
  backend consumption of agent_thread_count/agent_handle_count.
- PlatformAgentMetricsResponse.SessionsScanned: sessions the scan actually
  covered — the honest basis for the caller's truncated flag (Sessions.Count
  only counts sessions WITH snapshots).
- Per-session fetch no longer drains the full Events partition: new
  GetSessionEventsByTypesAsync (multi-type OData filter + column projection +
  real maxResults enforcement); the 20 delivery-latency sample sessions keep
  the full fetch so latency deltas stay computed over all events.
- Shared AgentMetricsAggregation (crash tally, payload getters, bounded
  fan-out, metrics event types) so the two services cannot drift.
- GetAllSessionsPageAsync gains an optional select projection (default null,
  existing callers unchanged), pinned by an equivalence test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 01:24:59 +02:00
Oliver Kieselbach 541dc32ffd feat(web): PDF export for the app install report + usage telemetry
Adds an "Export PDF" button to the app detail page that renders the full
90/30/7-day install report (KPIs, both charts, failure codes, device model
correlation, version breakdown, detection-rule/regression callouts) as a
branded A4 PDF, drawn data-driven with jsPDF — no DOM capture, so the output
is light-palette and theme-independent. jsPDF stays out of the route bundle
via on-click dynamic import (verified: separate async chunk).

Sample messages and the affected-sessions list are deliberately excluded:
the PDF is meant to be emailed outside the org.

Tracks app_report_pdf_exported in Application Insights (windowDays, scope,
durationMs, pageCount, success/errorName — chunk-load failures included).

Generic chrome (header/footer/tables/KPI cells/callouts) lives in
utils/pdf/pdfPrimitives.ts for reuse by future report exports.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 00:57:39 +02:00
Oliver Kieselbach 022cc0354b fix(engine): let the observed user-ESP skip stand in for the exit evidence
The five stuck sits-d Cloud PCs carried SkipUserStatusPage=True — verified
across three agent processes (10+ registry reads, True every time) and six
hours of live Shell-Core watching with zero 62407: Windows never rendered a
user ESP page, so no exit and no AccountSetup provisioning registry can ever
exist on this flow. Arm B of ShouldTransitionToAwaitingHello was open from
the bootstrap EspConfigDetected onward, but every evaluation site hangs off
a signal the flow cannot produce, and the AdvisoryCompletion backstop arms
exclusively from an EspExiting signal — an open gate nobody knocks on, six
hours to the max-lifetime watchdog. The earlier lost-exit-through-reboot
diagnosis is retired by the same evidence.

HandleImeUserSessionCompletedV1's completion attempt now accepts the
observed skip in place of IsPostAccountSetupFinalExit (anchor + genuine IME
user-session completion + real-user desktop stay mandatory; Device
Preparation excluded). The knock deliberately lives on the IME edge, not on
desktop arrival: Fix 10's AccountSetup bounce-back would undo a desktop-side
promotion two signals later, while the IME phase line always precedes the
user-session-complete line in log order, so a bounced promotion is
re-knocked by the very next re-emission — restart re-parse included.

Docs and comments no longer claim an ESP-exit recovery that does not exist
(stray duplicate summary in ShellCoreTracker, ImeLogHost restore-callback
comment); the OKF doc is reframed on the evidence and names the residual
SkipUser=false exit-in-downtime gap honestly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 00:40:23 +02:00
Oliver Kieselbach 3aa8b95676 fix(agent): narrow the Shell-Core replay to the Hello-wizard rail
Round-four review, confirmed — and the cause sits a layer below where I had been
fixing. The IsBackfill return added last round only guarded the coordinator's own
completion logic. Upstream of it, ShellCoreTracker still raised
FinalizingSetupPhaseTriggered, the coordinator still forwarded EspExited to the
adapter before that return, and the adapter discards the args entirely
(OnEspExited(...) => EmitEspExiting()). So a replayed exit still arrived at the
reducer as an ordinary, unmarked EspExiting.

There it is decisive: HandleEspExitingV1 passes espFinalExitInFlight: true for
every arriving exit, and IsPostAccountSetupFinalExit orders exits by ingest
ordinal rather than timestamp — so a historic exit replayed today reads as
post-AccountSetup by construction. With state restored from the snapshot
(AccountSetup entered, a genuine IME user session, desktop arrived) arm C of
ShouldTransitionToAwaitingHello opens on the Device→Account handoff. The reducer
is itself a completion gate and cannot tell the two apart, so the fix has to be
upstream.

The replay is therefore narrowed to what is provably safe: the Hello-wizard start
(62404). That fact is conservative — it vetoes a premature "Hello is disabled"
skip and can never by itself complete a session, so replaying it can only make the
agent wait longer. It is also the observation session 772fe502 wrote the replay
for. A 62407 is no longer raised at all, neither exit nor failure: the exit
because Windows writes the identical description for the intermediate and the
final one and nothing downstream can order it, the failure because re-injecting it
as fresh can fail a session that recovered on retry (ANALYZE-ESP-006). Skipped
records are reported once as an agent_trace with counts and bounds — silently
dropping evidence is how this class of bug survives.

The cross-layer proof is now a pair, as the review asked:
ClassicEspExitingOnRestoredStateTests pins that the reducer opens on any arriving
exit (including one with an hour-old source timestamp), and
ShellCoreTrackerReplayScopeTests pins that the replay never produces one.

Dead machinery removed with it: EspExitedEventArgs.IsBackfill, allowEspExit, the
newest-exit selection, the IsBackfill return. The method is now called
BackfillRecentHelloWizardStart.

One pre-existing integration test drove EspExited through HandleBackfillRecord
because the replay is easier to call than the event-log watcher, and its class doc
claimed that was the production path — it never was, the replay had no caller when
that test was written. Switched to the live path and the doc corrected.

Agent V2.Core 2361, DecisionCore 538, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:05:14 +02:00
Oliver Kieselbach ec945a93d2 fix(agent): bar replayed ESP exits from the completion gate entirely
Round-three review, confirmed. The previous round stopped a replayed exit from
arming the DEFERRED re-check but left the immediate synthesis attempt right below
it untouched. My justification for that — "pre-existing behaviour" — was simply
wrong: the immediate attempt on a replayed exit never existed, because the
backfill had no caller. I introduced it, and by moving the replay behind the IME
state restore I made the dangerous state reachable by design: if the newest 62407
in the window is only the intermediate DeviceSetup→AccountSetup exit while the
restored app states are already terminal, that attempt opens the strong
AccountSetup gate before the final exit has even happened. A test was pinning the
faulty assumption as expected behaviour.

A replayed exit now returns before touching completion at all — no deferred edge,
no immediate attempt. Nothing available to it can establish its position in time:
Shell-Core writes the identical description for both exits, and every probe that
could order it reads state as it is now rather than at the event's time. The
replay keeps the job it was written for in session 772fe502 — feeding EspExiting,
FinalizingSetup and the Hello wizard to the reducer for the window in which no
agent process was observing. Completion is the exclusive business of exits this
agent watched live.

With the synthesis attempt gone, the replay no longer needs restored app states,
so it moves back into EspAndHelloHost.Start(); that also keeps this host's startup
independent of the IME host's. ImeLogHost.onStateRestored goes back to doing just
the re-check, which is still needed for a live exit observed before the restore
finishes.

The test asserting the faulty assumption is inverted, plus coverage that the
replay is not neutered (EspExiting still forwards) and that a live exit following
a replayed one completes normally.

Agent V2.Core 2364, DecisionCore 533, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:25:22 +02:00
Oliver Kieselbach b36cf8ff36 fix(agent): order the ESP-exit replay by evidence, size its window by measurement
Round-two review, both findings confirmed against the code.

A replayed exit was being confirmed as post-AccountSetup by reading the registry
at replay time. That read reflects NOW, not the event's time, so an agent that was
down across the DeviceSetup→AccountSetup transition would confirm the stale
intermediate exit and, once the apps settled, synthesise a completion while the
AccountSetup page was still up. Windows offers no way to tell the two apart from
the event itself — Shell-Core writes the identical description
CommercialOOBE_ESPProgress_Page_Exiting for both. So the deferred edge is now
armed by LIVE exits only (EspExitedEventArgs.IsBackfill): only for those is the
continuous-observation argument valid. A replayed exit keeps the single immediate
synthesis attempt it always had.

For that attempt to be worth anything it must see real data, so the replay moved
out of EspAndHelloHost.Start() into ReplayEspExitBackfill(), invoked from
ImeLogHost.onStateRestored — after LoadState() has restored the package states.
That is what makes "the agent came back long after the ESP finished" recoverable.

The five-minute window for non-reboot restarts was unsound: the scheduled task
carries a BootTrigger only, with no restart-on-failure, so after an
exception_crash the agent does not return until the next boot — hours later, long
after Windows wrote the final exit. The window is now measured rather than
guessed: back to the last moment the previous run is known to have been alive,
taking the wider of the snapshot mtime (rewritten every decision step, available
for every exit type) and LastBootUtc. The five-minute default is the floor,
ClampLookbackMinutes the ceiling, and a future timestamp from clock skew is
ignored instead of producing a negative window.

Also switched the backfill tests to the real Shell-Core wording — the invented
string matched the regex but hid the fact that both exits look identical.

9 new tests. Agent V2.Core 2362, DecisionCore 533, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:06:14 +02:00
Oliver Kieselbach c6ee7808f1 fix(agent): scope the ESP-exit replay to restart recovery only
Follow-up to the review hardening: the replay ran on EVERY agent start with a
5-minute window, first run included. On a first run there is no gap to recover —
no earlier agent process existed — and a 62407 already sitting in the Shell-Core
log belongs to a phase transition this agent was never meant to observe. Replaying
it pushes an EspExiting into the reducer that the pre-fix agent never saw, i.e. a
happy-path behaviour change for no gain.

ResolveEspExitBackfillLookbackMinutes now keys the window to the previous exit
type: first_run (or unknown) disables the replay entirely, clean/hard_kill/
exception_crash keep the 5-minute default because the process is back within
seconds, and reboot_kill widens to "since LastBootUtc" because that gap spans the
forced restart until the post-reboot logon. The result is still clamped to
BackfillLookbackMaxMinutes, so policy and backstop together cannot build an
unbounded event-log query.

With this the happy path is byte-identical to pre-fix behaviour: on a first run
nothing replays, no edge is remembered, and the deferred re-check returns on a
volatile bool read.

No other backfill is affected — the Hello, MDM-reboot, Windows-Update and
ModernDeployment trackers each own a private backfill invoked from their own
Start(), none of which this change touches.

13 policy tests including clock skew across the reboot, a missing boot timestamp
and the clamp backstop. Agent V2.Core 2353, DecisionCore 533, Backend 4287.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:37:40 +02:00
Oliver Kieselbach b61d7023bc fix(agent): harden post-reboot completion recovery after review
Three real regressions, all confirmed against the code before touching it — two of
them introduced by the recovery itself.

The remembered exit edge was unqualified. Shell-Core raises 62407 at every ESP
phase transition, so OnEspExited also latched the DeviceSetup→AccountSetup one;
the deferred re-check would then open the strong AccountSetup gate as soon as the
last user app settled, while the AccountSetup page was still up and its other
subcategories still running. IsConfirmedPostAccountSetupExit() now demands
positive evidence (AccountSetup activity observed, or SkipUser == true) and is
deliberately stricter than the existing IsIntermediateDeviceEspExit() forward
guard: an unknown SkipUser counts as not confirmed. Erring strict costs the
pre-fix stall; erring loose costs a premature Succeeded. The immediate on-edge
synthesis keeps its previous behaviour verbatim.

The replay picked the oldest exit, not the final one. The reader walks
oldest-first and the exit branch is fire-once, so the widened downtime window —
which routinely holds both the intermediate and the final exit — handed over the
intermediate one and the fire-once guard swallowed the exit that matters.
ReplayBackfillRecords buffers the batch and lets only the newest exit-matching
record through; ESP failures and the Hello-wizard rail keep replaying
chronologically.

The replay ran before the IME state restore. ImeLogTracker.Start() restores the
persisted package states via LoadState() without raising OnAppStateChanged, and
ImeLogHost starts after EspAndHelloHost, so the replay evaluated an empty app
list — leaving the sharpest case unfixed: apps already terminal before the
reboot, only the final exit lost to the downtime. ImeLogHost gained an
onStateRestored callback wired to the re-check, and the host order is now pinned
by DefaultComponentFactoryOrderingTests so a reshuffle fails loudly.

11 new tests. Agent V2.Core 2340, DecisionCore 533, Backend 4287, all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:27:29 +02:00
Oliver Kieselbach e1b03de5b3 fix(agent): recover ESP completion after a forced mid-ESP reboot
Seven sits-d Cloud PCs enrolled the same day with identical configuration
(~138 required apps at the user-ESP gate, DeviceGuard/DmaGuard device-assigned).
The two that finished AccountSetup before the coalesced reboot landed succeeded;
the five the reboot killed in Stage=EspAccountSetup never derived a completion
again and ended Incomplete on healthy devices. Two defects behind that:

ShellCoreTracker.BackfillRecentEspExitEvents() has existed since session
772fe502 but no caller ever invoked it, so the Shell-Core watcher only saw
records written after Start(). Every agent restart silently dropped an ESP exit
(62407) or Hello-wizard start (62404) from its downtime. EspAndHelloHost.Start()
now calls it, and the fixed 5-minute window is sized to the actual downtime when
the previous run died to a reboot (LastBootUtc, clamped to 360 min).

The user-apps-settled synthesis ran only on the EspExited edge. When the ESP page
exits with apps still in flight the single attempt misses and nothing re-checks —
8110e262 and a89aac2d both reached 138/138 apps, 0 failed, and still never
completed. The exit edge is now recorded and the synthesis re-evaluated from
ImeLogTracker.OnAppStateChanged. No new completion path: the gate conditions are
unchanged, this only gives the existing conservative check a second look. The
fire-once claim moves to Interlocked (edge and re-check run on different threads)
and the re-check does not re-emit app_install_starved.

Rules: ANALYZE-ESP-005 gains on_event:session_stalled so a hung session produces
a finding at the 60-minute stall instead of staying finding-less for hours
(verified against 8110e262 — fired, confidence 95). New ANALYZE-APP-016 turns the
already-emitted app_install_starved events into a customer finding and, unlike
ANALYZE-APP-015, fires even on a succeeded session: 3b9291aa was recorded
Succeeded with 99 required apps that never started installing.

ANALYZE-APP-015 and ANALYZE-ESP-001 were considered for the stall trigger and
deliberately left alone — APP-015 is the documented worked example of the
terminal-precondition trap, and ESP-001's phase_duration gate on a still-open
phase is unverified mid-run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:00:59 +02:00
Oliver Kieselbach 25bac34acb fix(ci): upload raw bytes, not a stringified array
The first CI run overwrote all four customer-facing scripts on the download
host and the legacy mirror with their own byte values as decimal text
(Install-AutopilotMonitor.ps1 went from 27331 to 95679 bytes).

PowerShell unrolls an array returned from a function into the output stream, so
the caller receives Object[] rather than byte[]. Invoke-RestMethod serialises an
Object[] body as space-separated decimals. Every hash check stayed green because
[byte[]] parameter coercion repairs the type everywhere except on the wire.
build-agent.yml's Send-Blob was never affected: it read the bytes inside the
function. Proven with a local HttpListener: the old shape puts 50 bytes starting
54,48,32,51,53 ("60 35 13 1...") on the wire where 17 bytes starting 60,35,13,10
("<#\r\n.SYN") are expected.

Fixes:
- ',' on both byte-array returns, plus an explicit [byte[]] at the upload site.
- Read each blob back from storage right after its PUT and abort before the
  mirror and the remaining files. Verifying only at the end left every blob
  corrupted.
- The version-bump guard tolerates an unparseable published copy with a warning
  instead of throwing: that is the state a republish exists to repair, and a
  guard that blocks the repair is worse than no guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 15:31:06 +02:00
Oliver Kieselbach c0be389a95 ci(scripts): version-bump guard compares behaviour, not bytes
Byte comparison made the guard fire on comment fixes, typos in comments and
reflow -- changes no consumer of the version can act on, so forcing a bump for
them is friction without a payoff.

The guard now compares a code fingerprint: the PowerShell token stream with
comments, newlines and line continuations removed. A behaviour change under an
unchanged $ScriptVersion still fails; comment and formatting changes publish
with a warning. That is also the rule the source file's own comment on
$ScriptVersion already states ("bump on meaningful changes").

Tokenising rather than regex-stripping keeps a '#' inside a string literal from
being mistaken for a comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 15:15:48 +02:00
Oliver Kieselbach 91f9d656ae ci(scripts): publish customer scripts on push, decoupled from the agent release
The scripts behind download.autopilotmonitor.com/agent/ rotated only on a
publish_as_stable run of build-agent.yml, so a bootstrap fix sat unpublished
until the next agent cutover -- while the WDP bootstrap MSI downloads the
published copy on every enrolling device.

Publish-BootstrapScripts.ps1 becomes the single owner of the chain: dev render
with anchor check, $ScriptVersion bump guard, uploads (eu + fail-soft legacy
mirror, no-cache), the two version oracles, and verification through the alias.
publish-scripts.yml calls it on every push to main touching the sources; the
publish_as_stable branch of build-agent.yml calls the same code, so a release
still lands a consistent set.

Blobs are published in a deterministic CRLF/UTF-8-without-BOM form. Git
normalises line endings on checkout, so raw working-copy bytes would differ per
environment and make every byte comparison -- guard and verify alike -- depend
on where the workflow ran. The form reproduces the currently published bytes
exactly.

The Pester (PS 5.1) and ASCII gates move into the reusable
bootstrap-script-gates.yml and now block the upload; pester.yml keeps the
pull-request view and drops its push trigger. AdminConfiguration
.LatestBootstrapV2ScriptVersion gets exactly one writer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 15:09:33 +02:00
Oliver Kieselbach b1e7077d0a fix(agent): spool pending-bytes drift guard + WDP backstop fallback test
Codex findings on d8c84f41/a369ff99: (1) MarkUploaded now recomputes
_pendingBytes from the pending list when accounting drifts negative or
stays non-zero on an empty list — a silent negative would disarm the
spool-pressure trigger. (2) New test covers the DevicePrepCompletion
handler''s AwaitingDesktop fallback (stale/replayed timer without a
desktop fact): synthetic Hello timeout, park, and completion on the
later real desktop arrival. Third finding (HashingCappedReadStream
maxBytes:0) refuted — the 0=uncapped contract is implemented in the
same class and exercised by the heal-path tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 12:59:56 +02:00
Oliver Kieselbach d8c84f41c1 fix(agent): spool pressure trigger on pending backlog, not file size
99/100 spool_pressure_detected events in the field were file-size-only
triggers on a near-empty backlog (append-only spool.jsonl grows past 5 MB
in healthy long sessions). TelemetrySpool now tracks PendingBytes (exact
serialized size of the not-yet-uploaded tail, maintained on enqueue /
mark-uploaded / restart rehydrate) and the collector thresholds on that
instead of SpoolFileSizeBytes, which stays as an informational metric.
Snapshot gains spool_pending_bytes; pressure event data carries
pendingBytes + pendingBytesThreshold.

Also: startup recovery scan streams via File.ReadLines instead of
materializing the whole spool with File.ReadAllLines (corrupt-tail
break semantics unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 17:43:15 +02:00
Oliver Kieselbach 12b51bbc24 feat(landing): scenario coverage chips in hero + MSI channel in about claims
Hero gets a quiet checkmark chip row (user-driven, pre-provisioning,
self-deploying, Device Preparation, Windows 365, Entra ID & Hybrid join);
about page deployment claim now includes the MSI LOB channel for WDP.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 15:47:15 +02:00
Oliver Kieselbach a369ff99cd feat(wdp): scenario gates for Device Preparation + completion backstop
afee7ae0 audit: WDP sessions ran the full Classic/ESP apparatus. Gates key on
the deterministic BootstrapperAgent ExecutionContext marker only (CloudAssigned*
fallbacks keep Classic behavior).

Paket A (event hygiene): no autopilot_profile/-missing + ZTD query/probe on WDP;
esp_config_detected null-gate (no more unknown/unknown); starved sweep skipped
on WDP (data stays in app_tracking_summary/final-status).

Paket B (resources): EspPolicyProviderStallHost + ProvisioningStatusTracker not
started on WDP; ModernDeployment noise IDs 100/417/418/1005 fully suppressed
(Critical always passes) + Autopilot error backfill skipped; StallProbe skips
ESP category scan and treats noise IDs as harmless.

Paket C (DecisionCore): enrollmentTypeDeterministic payload key seeds
DevicePreparation at High confidence (fallbacks stay Medium); completion_waiting
drops the unsatisfiable account_setup_provisioning_complete gate on WDP; arm D
promotes on real-user desktop; new device_prep_completion 30-min backstop closes
the park-until-watchdog hole; parked tripwire gets a WDP branch.

17 new tests; suites green (532/2307/4287). OKF doc agent/wdp-scenario-gates.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 13:12:09 +02:00
Oliver Kieselbach 7a131af74d agent(summary): DevicePrep app-wave settle wait + Office/CSP row in summary dialog
Session bc803955 (DevicePrep, Succeeded) rendered "No application data
available" although the M365 CSP package installed during prep and IME
delivered 12 required user apps 9 seconds after final-status was written.

- EnrollmentTerminationHandler: bounded settle wait (20s, 250ms poll) before
  the dialog snapshot - DevicePreparation-only and only while the tracker has
  zero packages, so Classic pays nothing. Root cause of the lost race is IME
  AppWorkload.log flush latency (~10-20s), not the 100ms tracker poll.
  Dialog, starved sweep and app_tracking_summary all see settled data.
- FinalStatusBuilder: synthetic "Microsoft 365 Apps" row from the Office C2R
  detector's persisted lifecycle state (Office CSP installs never flow through
  the IME Win32 pipeline). Prepended to the Device bucket, counted in
  appSummary; Preinstalled state deliberately excluded. No SummaryDialog.exe
  change needed - it renders packageStatesByPhase generically.
- OfficeInstallStateData: CompletedAtUtc + VersionReached stamped on terminal
  persists (duration/version for the row); older state files load fail-soft.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:57:46 +02:00
Oliver Kieselbach e750cfe9dd feat(web): gap badge on timeline rows after multi-hour/day time gaps
Timeline rows render time-of-day only, so a large gap between adjacent
events (and the date change it implies) was invisible without expanding
event details. Rows following a gap >= 1h now show an amber chip
(e.g. "+3h" / "+19d"); hovering reveals the full date. Applies to both
the phase timeline and raw mode, using the same display timestamp as
the row (backfilled events keep their event-log time).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:39:36 +02:00
Oliver Kieselbach 3629728d93 feat(time-attribution): PriorEnrollmentResidue quality flag (v2) + WG-signal comment fix
Session f475e697: a device re-enrolled without a wipe (IME log residue from a
previous enrollment) completes its pre-installed apps as instant detections,
pulling the phase anchors ~6.5 min ahead of the real ESP page — the breakdown
was mechanically exact but semantically wrong, and entered fleet stats as clean.

- New duration-critical flag PriorEnrollmentResidue, set on
  historic_ime_replay_detected; excluded from fleet segment stats via the
  existing DurationCriticalFlags mask. registry_app_baseline successes are
  deliberately NOT a trigger (by-design normal for WDP Batch-1) — pinned by test.
- AttributionVersion 1 -> 2 so the maintenance sweep recomputes in-retention
  breakdowns and flags old residue sessions retroactively (rule 8).
- Web: flag label in TimeAttributionLane; docs: quality-flags section.
- Stale comment fix: SaveWhiteGloveSuccessResult is NOT a WhiteGlove indicator
  on Win11 25H2+ (validated 2026-08-18 against user-driven sessions) —
  observability only, consumed nowhere for classification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:39:11 +02:00
Oliver Kieselbach e521fd5cf4 fix(ime-archive): heal provenance.json lost behind the write-once 409 path
Daily-review finding: the MSI upload is write-once (If-None-Match:*), so a
first attempt that died between the MSI and the provenance upload funneled
every retry into the 409 short-circuit — which returned "Archived" without
ever writing provenance.json (permanently lost). The 409 path now checks for
the sidecar and, when missing, rehashes the ARCHIVED bytes and writes it;
heal failures ride the normal retryable ladder.

Also closes the review's test gaps: ImeMsiArchiveQueueWorker + producer
worker-level tests (internal QueueClient test ctors, SessionDeletionWorker
pattern), CorporateIdentifierValidator null/whitespace + OData special-char
+ page-cap-transient tests, registry observer helper tests
(TerminalOutcome/ChangedFields/ClassifyStatusServiceStatus).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 10:21:26 +02:00
Oliver Kieselbach 0ed1e6f542 Merge branch 'main' of https://github.com/okieselbach/Autopilot-Monitor 2026-08-18 03:21:54 +02:00
dependabot[bot] 522911c5a7 chore(deps-dev): bump tsx (#130)
Bumps the mcp group in /src/McpServer/autopilot-monitor-mcp with 1 update: [tsx](https://github.com/privatenumber/tsx).


Updates `tsx` from 4.23.11 to 4.23.12
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.23.11...v4.23.12)

---
updated-dependencies:
- dependency-name: tsx
  dependency-version: 4.23.12
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: mcp
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-18 03:20:26 +02:00
dependabot[bot] 0bfe1e327b chore(deps): bump the web group (#131)
Bumps the web group in /src/Web/autopilot-monitor-web with 4 updates: [@xyflow/react](https://github.com/xyflow/xyflow/tree/HEAD/packages/react), [next](https://github.com/vercel/next.js), [@next/bundle-analyzer](https://github.com/vercel/next.js/tree/HEAD/packages/next-bundle-analyzer) and [eslint-config-next](https://github.com/vercel/next.js/tree/HEAD/packages/eslint-config-next).


Updates `@xyflow/react` from 12.11.2 to 12.11.3
- [Release notes](https://github.com/xyflow/xyflow/releases)
- [Changelog](https://github.com/xyflow/xyflow/blob/main/packages/react/CHANGELOG.md)
- [Commits](https://github.com/xyflow/xyflow/commits/@xyflow/react@12.11.3/packages/react)

Updates `next` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/compare/v16.3.0...v16.3.1)

Updates `@next/bundle-analyzer` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v16.3.1/packages/next-bundle-analyzer)

Updates `eslint-config-next` from 16.3.0 to 16.3.1
- [Release notes](https://github.com/vercel/next.js/releases)
- [Commits](https://github.com/vercel/next.js/commits/v16.3.1/packages/eslint-config-next)

---
updated-dependencies:
- dependency-name: "@xyflow/react"
  dependency-version: 12.11.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: next
  dependency-version: 16.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: "@next/bundle-analyzer"
  dependency-version: 16.3.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
- dependency-name: eslint-config-next
  dependency-version: 16.3.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: web
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-18 03:20:10 +02:00
Oliver Kieselbach 2cf7216be5 Token-failure grace window + registry_app_baseline summary
Live findings from the first two 2.0.1402 sessions (Oliver, 2026-08-18):

1. "Failed to get AAD token" is logged routinely during the device phase
   (no user token exists yet) - both test sessions raised the Warning
   although nothing was wrong. The failure is now held pending: any
   IME-TOKEN-SUCCESS match clears it (staleness-gated so a replayed
   success from a previous enrollment cannot mask a current outage), and
   ime_token_failure fires only when no success follows for the 30-min
   grace window, then deduped per errorCode. Genuine auth outages run
   long enough to hit the window; routine device-phase failures never do.

2. New one-shot registry_app_baseline event (Info, only when non-empty):
   summary of the Win32Apps state that predates the agent run - counts
   by enforcement class + capped 50-app typed payload incl.
   LastUpdatedTimeUtc. On WDP the DPP-policy apps install in Batch 1
   before the bootstrap-script agent exists; their outcomes live only in
   this baseline, closing the pre-agent app gap without log backfill.
   The diff rail stays silent on baseline state (anti-replay unchanged).

Event type dual-registered (Constants + MCP catalog + manifests regen).
Tests: grace-window cycle (resolved / unresolved / re-arm dedup),
baseline summary (counts, one-shot, empty). All suites green
(Agent 2274, Functions 4262, MCP 561, web 762), 0 warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:16:58 +02:00
Oliver Kieselbach 587adc5ddf ci(msi): build from project dir - wix resolves File/@Source against cwd
WIX0103 on the first run: Invoke-BootstrapDownload.ps1 not found because
wix build resolves File/@Source relative to the working directory, not
the wxs location. Matches the local build documented in Package.wxs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:11:12 +02:00
Oliver Kieselbach 84bf2bc2fd commands: bump to 3f98090 (log-name kebab-case in analysis skills) 2026-08-18 03:07:42 +02:00
Oliver Kieselbach dcf1b1c227 ci: Bootstrap-MSI build workflow - blob publish behind the download alias
workflow_dispatch-only, mirroring the agent release contract: validate
the runner script (PS parse + ASCII-only, it runs as SYSTEM on every
enrolling device), wix 5 build, upload versioned + stable blob names to
the agent container (no-cache, no legacy mirror - the MSI channel never
existed there), HEAD-verify the public alias serves the fresh bytes,
Sigstore provenance attestation. MSI version stays source-controlled in
Package.wxs - bumped to 1.0.2 (runner: encoding fix + bootstrap-msi.log
rename). Customer download:
https://download.autopilotmonitor.com/agent/AutopilotMonitor-Bootstrap.msi

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 03:07:08 +02:00
Oliver Kieselbach 6ebdfa41af refactor(agent): unified kebab-case log file naming + date-less agent.log
The agent log directory mixed three conventions (agent_20260817.log,
bootstrap_agent.log, msi-bootstrap.log) while everything else in the
diagnostics ZIP is already kebab-case. All log files now follow
<component>-<content>.log:

  agent.log               (was agent_YYYYMMDD.log; segments agent-NNN.log)
  bootstrap-script.log    (was bootstrap_agent.log; script v2.4)
  bootstrap-msi.log       (was msi-bootstrap.log)
  crash-<utc>.log         (was crash_<utc>.log)
  gather-rules-debug.log  (was gather_rules_debug.log)
  ime-pattern-matches.log (was ime_pattern_matches.log)

The date is dropped from the main log: the agent only lives for one
enrollment, and date rotation split the chronology at local midnight -
which can fall mid-enrollment while the clock is still on the OOBE
default timezone. The existing 50MB size rotation stays as the growth
bound. Legacy dated files are ignored by the rotation probe; crash
detection still picks up legacy crash_*.log once so an in-place
self-update cannot lose a pending crash classification.

Diag collectors glob *.log and need no change. MCP zip-map and docs
note the legacy names for archives from pre-rename agents. Settings UI
hint paths updated. MSI rebuild + bootstrap script publish (v2.4) +
MCP redeploy required to ship.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:55:06 +02:00
Oliver Kieselbach a2c4949473 commands: bump to b431556 (ime-decompile blob-first + import fixes)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:36:57 +02:00
Oliver Kieselbach 81f4a55d88 Bootstrap MSI 1.0.1: version bump for Intune re-upload + log encoding fix
Intune LOB requires a higher ProductVersion to accept a re-upload. Also
includes the runner logging fix: PS 5.1 *>> redirection writes UTF-16LE
into the otherwise ANSI msi-bootstrap.log - route through Add-Content for
a single encoding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 02:12:11 +02:00
Oliver Kieselbach 4d3d3c56f4 gitignore: restore ime-files/ - local binaries scratch of /ime-decompile
Still needed: the skill extracts each MSI here (msiexec /a payload for grepping
in binaries); only decompilates go to the ime-decompiles repo and MSIs to the
ime-archive blob container.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:51:26 +02:00
Oliver Kieselbach 41898d745a updated commands 2026-08-18 01:33:44 +02:00
Oliver Kieselbach c05ea4d63b Registry second pillar: IME app state from HKLM with log reconciliation
P2 of the 2026-08-17 IME code audit. IME log wording is an unstable
contract (audit found 9 silently dead patterns, reworded lines, quote
drift, [DO TEL] removal); the registry is IME's internal product
contract - the ESP page and Intune reporting consume it.

ImeRegistryAppStateObserver + ImeRegistryAppStateHost (always-on
observability, no DecisionEngine coupling):
- one recursive RegistryWatcher on HKLM\...\IntuneManagementExtension
  covers Win32Apps (EnforcementStateMessage JSON, ExitCode, Intent),
  EspTrackingWin32Apps and SideCarPolicies\StatusServiceReports;
  2 s debounced snapshot-and-diff + 60 s periodic fallback tick
  (poll-only degraded mode if the watcher fails to arm)
- silent startup baseline: Win32Apps keys survive re-enrollments,
  pre-existing state never replays as fresh events
- registry_app_state (Info, state-change-only, 200/session cap with
  one-shot cap notice)
- app_state_reconciliation (Warning, once per app, immediate upload):
  a changed entry terminal for >= 90 s is compared against the
  log-derived AppPackageState - divergence is the built-in
  log-pattern drift alarm (registry = truth, logs = narrative)

Event types dual-registered (Constants + MCP resource-catalog + shared
manifests regenerated). OKF concept doc agent/registry-app-state-pillar.md
incl. WDP Paket-3 reuse notes. Tests: pure parse/diff/divergence logic +
tick flow via snapshot override (baseline silence, emission, dwell,
fire-once, cap). All suites green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:25:54 +02:00
Oliver Kieselbach 68eb2da283 IME tracker: tail AppActionProcessor.log; GRS + token-failure signals
P0b/P1 of the 2026-08-17 IME code audit (agent release required):

- Tail AppActionProcessor.log (+ rollover glob) in ImeLogTracker and the
  --run-ime-matching CLI. IME routes every [Win32App][ActionProcessor]
  line there (enforcement decision engine), so IME-NO-ACTION and
  IME-NOT-TARGETED-1/2 could never fire before, and GRS skips were
  invisible. Live session 0fe2c562 carried 16 no-action and 64
  classification lines in that file.

- New patterns (wordings verified in decompiled IME 1.97 AND 1.104):
  IME-GRS-SKIP ("is in GRS. The app will not be enforced." -> Postponed;
  THE canonical silent non-install reason), IME-GRS-CLEARED (early
  subgraph termination, retry next check-in -> Postponed),
  IME-TOKEN-FAILURE ("Failed to get AAD token ... errorCode = N").

- New action imeTokenFailure -> ime_token_failure Warning event (new
  EventType: Constants + MCP resource-catalog + shared manifests
  regenerated). Tracker dedups per distinct errorCode; included in
  the historic-replay guard so stale lines from a previous enrollment
  cannot emit a misleading warning. Complements IME-TOKEN-SUCCESS:
  an auth outage previously left check-ins silently doing nothing.

Tests: adapter emission + unknown-code + tracker dedup; known-actions
catalog extended; all suites green (Functions 4262, Agent 2252, web 762).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 01:01:02 +02:00
Oliver Kieselbach ab0f27b5ab Download times: event-window fallback for IME 1.104 (DO TEL removed)
IME 1.104 no longer logs the [DO TEL] line whose JSON carried the
DO-reported download duration - the Download Progress card lost all
times (per-app and total).

Durations now fall back to the observed event window (first download
event -> first completion-marking event) whenever the DO-reported value
is absent, and completed items show "in <duration>" inline. Works
retroactively for already-stored sessions, no agent release required;
the DO-reported value still wins when present (older IME versions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 00:40:53 +02:00
Oliver Kieselbach 32d8151185 updated .gitignore 2026-08-18 00:28:29 +02:00
Oliver Kieselbach f3749ccc16 updated commands 2026-08-18 00:24:17 +02:00
Oliver Kieselbach ca26cb4f7c backend: auto-archive IME installer on first sighting of a new version
When RecordImeVersionAsync reports a genuinely new IME version (~monthly),
EventIngestProcessor now also enqueues an ime-msi-archive job. The worker
(QueuePollingWorker, heartbeat, pause on ImeMsiArchivingEnabled=false)
downloads the installer and stores it write-once (If-None-Match:*) in the
permanent ime-archive container ({version}/IntuneWindowsAgent.msi +
provenance.json), then merges MsiArchiveStatus/BlobPath/Sha256/Bytes/SourceUrl
onto the ImeVersionHistory row - so every fleet-observed IME build stays
available for decompilation/diffing after Microsoft's versionless CDN moves on.

The download URL comes from agent telemetry (msiDownloadUrl enrichment) and is
treated as untrusted: https-only *.manage.microsoft.com allowlist with exact
filename match, canonical versionless fallback, no redirects, size cap
(MaxImeMsiDownloadSizeMB, default 250, preflight + mid-stream) enforced by a
SHA-256-hashing pass-through stream. Archiver never throws; the worker retries
only transient statuses (download/timeout/error), permanent rejections
complete with their Failed:* status persisted.

Admin settings ImeMsiArchivingEnabled (default true) + MaxImeMsiDownloadSizeMB
via the full 4-file web chain; archive fields on the version-history API are
GA-only (non-GA projection unchanged, redaction tests extended). Shared
manifests regenerated. Tests: archiver status branches via recording subclass
(allowlist, caps, 409 idempotency, cancellation propagation), config defaults.
Backend suite 4262 green, web tsc + vitest 762 green, solution 0 warnings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-18 00:20:53 +02:00
Oliver Kieselbach 8ceab7a064 updated commands 2026-08-17 23:51:56 +02:00
Oliver Kieselbach ae85dc325c gitignore: exclude local ime-files/ working directory
Machine-local reference files, kept out of version control.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 23:51:08 +02:00
Oliver Kieselbach aeb2be6d64 agent: enrich ime_agent_version with CSP install source (CurrentDownloadUrl)
The EnterpriseDesktopAppManagement CSP records every MSI it enforces under
HKLM\SOFTWARE\Microsoft\EnterpriseDesktopAppManagement\<SID>\MSI\{ProductCode},
including CurrentDownloadUrl and ProductVersion. New ImeMsiInstallSourceProbe
resolves the entry matching the IME version parsed from the IME log (normalized
ProductVersion match first, IntuneWindowsAgent.msi filename fallback) and the
adapter attaches msiDownloadUrl / msiProductCode / msiProductVersion /
msiMatchedBy to the ime_agent_version event - answering which CDN/AFD endpoint
delivered exactly that IME build. Fail-soft, cached per version, additive keys
only (no new event type). Tests: probe selection/normalization + adapter
enrichment; assembly-wide probe default keeps dev machines deterministic.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 23:29:37 +02:00
Oliver Kieselbach 0474223290 Diagnostics ZIP: in-archive packaging manifest
The agent-log snapshot is zipped BEFORE packaging finishes, so packaging
problems (blocked paths, silent no-match patterns, locked files, evtx
export errors) were invisible in the uploaded archive - twice this made a
missing evtx undiagnosable (sessions a11102f4/3ae7528b). Every packaging
decision is now recorded in package-manifest.txt inside the ZIP: ADDED,
NO MATCH, BLOCKED (path guard), SKIPPED (caps/reparse/no user session),
FAILED (copy), EVTX EXPORTED/ERROR/TIMEOUT, plus each configured
additional path with its expanded folder+pattern. Written unconditionally
so absence of the file identifies a pre-feature archive.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 23:04:21 +02:00
Oliver Kieselbach 1bdb71be59 evtx export: resolve channel via LogFilePath, never by naming convention
The %4-encodes-slash file naming is a convention, not a contract: the
Microsoft-Autopilot-BootstrapperAgent/BootstrapperAgentServiceLogProvider
channel writes plain BootstrapperAgentServiceLogProvider.evtx. Deriving the
channel from the file name therefore produced an invalid wevtutil target
(and the wrong advice to rename the tenant diag path cost a WDP test
cycle). The channel is now resolved by scanning registered channels for a
matching LogFilePath, with the %4 convention only as fallback. A failed
export no longer skips the file - it falls through to the raw copy, which
succeeds for inactive/archived evtx files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:48:19 +02:00
Oliver Kieselbach ccf4f2e054 IME patterns: quote-flexible report deltas for IME 1.104 serialization
Live validation against session 0fe2c562 (IME 1.104.102, first session on
the repaired pattern set) showed all report-delta patterns at zero: IME
1.104 logs the delta JSON with single quotes ({"EnforcementState":...)
where older versions logged doubled quotes ({""EnforcementState"":...).

Every literal quote in the seven delta patterns (DETECTED-INSTALLED-1,
ENFORCEMENT-SUCCESS, ERROR-DOWNLOAD, ERROR-REPORT, NOT-APPLICABLE-2/3/4)
is now matched as "{1,2} to cover both serializations.

Verified against the session's AppWorkload.log via validate-ime-patterns:
ENFORCEMENT-SUCCESS 12 (was 0), ERROR-REPORT 14 (was 0); no CRITICALs;
the 2026-08-17 repair set confirmed live (ESP-TRACK-STATUS 24,
ESP-APP-REGISTERED 12, CACHE-MISS 11 in the agent's own match log).

Also observed in 1.104, tracked separately (no pattern change possible):
[DO TEL] line removed entirely - the DO download-duration display feeds
on it; needs an agent-side replacement (DO collector enrichment).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:43:32 +02:00
Oliver Kieselbach d192a8e0fa fix(agent): token-aware runtime launch - Scheduled Task first on stripped MSI tokens
The MDM LOB Bootstrap-MSI chain (EnterpriseDesktopAppManagement CSP ->
msiexec -> runner -> bootstrap -> --install) carries a SYSTEM token with
SeTimeZonePrivilege and SeSystemEnvironmentPrivilege REMOVED. WMI
Win32_Process.Create duplicates the caller token onto the runtime, so
every MSI-delivered agent ran degraded for its whole life: tzutil exit 5
(ACCESS_DENIED) on timezone auto-set and privilege_denied on SEC-001
firmware reads (observed 2026-08-17, sessions a11102f4/0fe2c562/3ae7528b
vs. platform-script sessions 7b072755/c4c1c2d5 on the same binary).

Install mode now probes its own token (TokenPrivilegeProbe, fail-soft to
"held"): with a full token the proven WMI-first chain is unchanged; on a
stripped token the order flips to schtasks /Run first - the Task
Scheduler service starts the task with a fresh full SYSTEM token - with
a 10s process-verification poll (schtasks only reports "queued", the
2026-04-30 silent-failure lesson), WMI as degraded-but-alive fallback,
BootTrigger defer as the final net (full token there anyway).

Also: MSI runner log encoding - PS 5.1 *>> redirection wrote UTF-16LE
into the Add-Content (ANSI) msi-bootstrap.log, garbling every bootstrap
line; script output now routes through the same Add-Content pipeline.
Runner fix needs an MSI rebuild (wix build) to ship.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:39:39 +02:00
Oliver Kieselbach 06d92460ed Dashboard session rows: native open-in-new-tab via invisible cell link overlays
Rows were plain <tr onClick> so the browser offered no context-menu entry.
Each non-interactive cell now contains an invisible absolutely-positioned
Link (prefetch off) giving a real href for right-click/middle/ctrl+click;
visuals and SPA click behavior are unchanged. Cells with own buttons or
title tooltips (status, tenantId, actions) keep the row-click fallback.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:37:48 +02:00
Oliver Kieselbach fc3900429c Security: IME log pattern PUT must be GlobalAdminOnly
PUT rules/ime-log-patterns/{patternId} sat on the TenantAdminOrGA tier
while the handler only checks ?global=true and writes the GLOBAL pattern
with no GA re-gate - any customer tenant admin could mutate platform-wide
patterns via direct API call. The UI gate (isGlobalAdmin && globalAdminMode)
was cosmetic only.

Moved the entry to GlobalAdminOnly next to the reseed route and added a
regression test pinning both IME pattern write routes to that tier.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 22:09:35 +02:00
Oliver Kieselbach 0ffcb2fde2 command update 2026-08-17 21:44:26 +02:00
Oliver Kieselbach a3089753d3 IME patterns: repair drift found by decompiled-source audit (IME 1.97 Dec-25)
Audit vs decompiled IME 1.97.107: ~55/79 patterns verified verbatim, but
several were broken or dead against the current V3 pipeline.

Fixes:
- IME-ESP-TRACK-STATUS: drop trailing anchor (line grew ' with name: <n>.')
- HS-COMPLIANCE: tolerate both 'remdiation' typo and corrected wording;
  new HS-COMPLIANCE-POST for the reworded post-detection line
- IME-DETECTION-RESULT: capture DetectedDifferentVersion (ordered first)
- IME-ENFORCEMENT-SUCCESS: unpin OldValue (Error/PendingReboot/null paths)
- IME-ERROR-REPORT: allow unquoted null OldValue on first write
- Disable 11 dead patterns (emitters removed with IME's old V1/V2 pipeline;
  IME-DETECTED-OLD additionally collided with IME-SKIP-INSTALL-DETECTED)

New signals (existing actions, tailed files only):
- IME-ESP-APP-REGISTERED + IME-CACHE-MISS as app-name sources (replacing
  dead IME-UPDATE-NAME-1/2/3)
- IME-WINGET-ERROR (WinGet failures were invisible)
- IME-EXITCODE-MAPPED-FAILED (normal mapped-exit-code failure path)
- IME-STOP-SIGNAL (service stop != OS shutdown)
- IME-RESTART-STOP-PROCESSING (reboot-gated processing abort)

Backend seed diff now also compares Enabled + Parameters - enable/disable
flips and parameter changes previously never propagated to the table.

All exact wordings verified against decompiled source; regex smoke-tested
via validate-ime-patterns.py (engine=regex, 75/75 compiled, all new
patterns fire on synthetic lines with the exact IME format strings).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 21:33:37 +02:00
Oliver Kieselbach 26cbc897c5 WDP: unify enrollment_type_detected with detector + export locked evtx
Two fixes from the a11102f4 diag analysis:

- enrollment_type_detected re-implemented the old CloudAssigned*-only
  detection and kept reporting "Autopilot v1 (Classic ESP)" on WDP devices
  while the session row already said v2. DeviceInfoCollector now uses
  EnrollmentRegistryDetector.DetectEnrollmentTypeWithRule() - the same
  detector as registration - and the event names the rule that fired.
- Tenant-configured .evtx paths never made it into the diagnostics ZIP:
  active event-log channels hold their file exclusively locked. The
  package service now exports the channel via wevtutil epl (channel name
  derived from the %4-encoded file name) and streams the temp export.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 21:28:43 +02:00
Oliver Kieselbach c88629dcf2 Bootstrap MSI: thin MDM LOB delivery vehicle for WDP
WDP batch order is fixed (User -> SLDM/MDM apps -> Scripts -> Win32Apps),
so a WDP-policy script can never observe the app phase live, and a hard
Batch-1 failure leaves zero visibility. LOB MSIs install via OMA-DM during
the first sync - before the SLDM batch - making them the earliest delivery
channel. The MSI carries no logic: it installs one fail-soft runner script
that downloads and executes the CURRENT server-hosted
Install-AutopilotMonitor.ps1, so all real bootstrap logic stays updatable
without re-shipping the MSI. Skips via the Deployed regkey when another
channel won the race. Build: wix build Package.wxs -arch x64 (WiX 5).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 20:35:37 +02:00
364 changed files with 26982 additions and 1315 deletions
@@ -0,0 +1,367 @@
<#
.SYNOPSIS
Publishes the customer-facing PowerShell scripts behind download.autopilotmonitor.com/agent/.
.DESCRIPTION
Single owner of the script-publishing chain. Called from two places:
* publish-scripts.yml -- on every push to main that touches the sources,
so a script fix reaches customers without waiting
for an agent release.
* build-agent.yml -- on a publish_as_stable run, so a release always
lands a consistent set (ZIP + manifest + scripts).
Both callers run the same code. The scripts on the download alias are the live
copy the WDP bootstrap MSI (Invoke-BootstrapDownload.ps1) downloads on every
enrolling device, so "current" is a correctness property, not cosmetics.
Steps, in order:
1. Parse $ScriptVersion from the bootstrap source.
2. Render the -Dev variant by literal substitution of the two URL/manifest
defaults. A missing anchor is a hard failure -- a silently un-substituted
dev script would point the dev fleet at the stable agent.
3. Version-bump guard: if the published bootstrap differs from the source
but carries the same $ScriptVersion, abort. Otherwise the docs badge,
version.json.bootstrapVersion and the portal's "your script is outdated"
hint would all keep asserting a version that is no longer what ships.
4. Upload every script blob with Cache-Control: no-cache (they rotate in
place), mirrored fail-soft to the legacy account.
5. Reconcile the two version oracles: version.json.bootstrapVersion
(read-modify-write under If-Match, so a concurrent agent release cannot
lose its agent fields) and AdminConfiguration.LatestBootstrapV2ScriptVersion.
6. Verify through the alias -- re-download each blob and compare SHA-256.
.PARAMETER SasToken
Container SAS for the eu storage account with write permission. Leading '?' tolerated.
.PARAMETER LegacySasToken
Container SAS for the legacy account. Optional -- a mirror failure warns, never fails.
.PARAMETER TableSasToken
Table SAS for AdminConfiguration. Optional -- omitted or 'XXX' skips the oracle update.
.PARAMETER RepoRoot
Repository root. Defaults to the parent of .github/scripts.
.PARAMETER DryRun
Run every check and render, upload nothing.
.EXAMPLE
./Publish-BootstrapScripts.ps1 -DryRun
.EXAMPLE
./Publish-BootstrapScripts.ps1 -SasToken $env:SAS -TableSasToken $env:TABLE_SAS
#>
[CmdletBinding()]
param(
[string]$SasToken,
[string]$LegacySasToken,
[string]$TableSasToken,
[string]$RepoRoot,
[switch]$DryRun
)
$ErrorActionPreference = 'Stop'
if (-not $RepoRoot) {
$RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '../..')).Path
}
if (-not $DryRun -and [string]::IsNullOrWhiteSpace($SasToken)) {
throw 'SasToken is required unless -DryRun is used.'
}
$ContainerUrl = 'https://autopilotmonitoreu.blob.core.windows.net/agent'
$LegacyContainerUrl = 'https://autopilotmonitor.blob.core.windows.net/agent'
$AliasUrl = 'https://download.autopilotmonitor.com/agent'
$TableUrl = "https://autopilotmonitoreu.table.core.windows.net/AdminConfiguration(PartitionKey='GlobalConfig',RowKey='config')"
$BootstrapSource = Join-Path $RepoRoot 'scripts/Bootstrap/Install-AutopilotMonitor.ps1'
$BootstrapBlob = 'Install-AutopilotMonitor.ps1'
$DevBlob = 'Install-AutopilotMonitor-Dev.ps1'
$ScriptContentType = 'text/plain; charset=utf-8'
function Get-HttpStatus {
param($ErrorRecord)
if ($ErrorRecord.Exception.PSObject.Properties['Response'] -and $ErrorRecord.Exception.Response) {
return [int]$ErrorRecord.Exception.Response.StatusCode
}
return 0
}
function Get-Sha256 {
param([byte[]]$Bytes)
$sha = [System.Security.Cryptography.SHA256]::Create()
try { return [BitConverter]::ToString($sha.ComputeHash($Bytes)).Replace('-', '').ToLowerInvariant() }
finally { $sha.Dispose() }
}
# Raw bytes of a published blob, or $null when it does not exist yet.
function Get-PublishedBytes {
param([string]$Url)
try {
$resp = Invoke-WebRequest -Uri $Url -Method Get -UseBasicParsing
} catch {
if ((Get-HttpStatus $_) -eq 404) { return $null }
throw
}
# Comma operator: PowerShell unrolls an array returned from a function into the output
# stream and reassembles it as Object[], which would silently defeat every [byte[]] use
# downstream. ',' wraps it so the byte[] survives the return.
if ($resp.Content -is [byte[]]) { return ,$resp.Content }
return ,[System.Text.Encoding]::UTF8.GetBytes([string]$resp.Content)
}
# Deterministic publish form: CRLF, UTF-8 without BOM. Git normalises line endings on
# checkout (a Windows runner yields CRLF, a core.autocrlf=false clone yields LF), so
# publishing the raw working-copy bytes would emit different blobs for identical source
# and make every byte comparison -- guard and verify alike -- environment-dependent.
# CRLF is also what PS 5.1 on the device expects and what is published today.
function Get-PublishBytes {
param([string]$Text)
$normalised = $Text.Replace("`r`n", "`n").Replace("`n", "`r`n")
# ',' keeps this a byte[]: without it PowerShell unrolls the array into the output stream
# and the caller receives Object[]. Invoke-RestMethod then serialises an Object[] body as
# space-separated decimals instead of raw bytes -- a silent corruption of the upload.
return ,(New-Object System.Text.UTF8Encoding $false).GetBytes($normalised)
}
function Get-BootstrapScriptVersion {
param([string]$Content, [string]$Origin, [switch]$Optional)
if ($Content -match '\$ScriptVersion\s*=\s*"([\d\.\-a-zA-Z]+)"') { return $Matches[1] }
# Unreadable SOURCE is fatal. An unreadable PUBLISHED copy must not be: that is exactly
# the state a republish is meant to repair, and a guard that blocks the repair is worse
# than no guard.
if ($Optional) { return $null }
throw "Could not parse ScriptVersion from $Origin"
}
# What the script actually DOES, with comments and layout removed: the PowerShell token
# stream minus comments, newlines and line continuations. Lets the version-bump guard tell
# a behaviour change (must be versioned -- consumers act on the version) apart from a
# comment fix or reflow (nobody needs to know). Tokenising rather than regex-stripping
# keeps '#' inside strings from being mistaken for a comment.
function Get-CodeFingerprint {
param([string]$Text, [string]$Origin)
$tokens = $null
$errors = $null
[System.Management.Automation.Language.Parser]::ParseInput($Text, [ref]$tokens, [ref]$errors) | Out-Null
if ($errors -and $errors.Count -gt 0) {
throw "Parse error in ${Origin}: $($errors[0])"
}
$ignored = @('Comment', 'NewLine', 'LineContinuation', 'EndOfInput')
return (($tokens | Where-Object { $ignored -notcontains $_.Kind.ToString() } | ForEach-Object { $_.Text }) -join "`n")
}
# ------------------------------------------------------------------ 1. source
$sourceContent = Get-Content $BootstrapSource -Raw
$sourceBytes = Get-PublishBytes $sourceContent
$scriptVersion = Get-BootstrapScriptVersion -Content $sourceContent -Origin $BootstrapSource
Write-Host "Bootstrap script version: $scriptVersion"
# ------------------------------------------------------------------ 2. dev render
$stableUrlLiteral = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip"'
$devUrlLiteral = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent-dev.zip"'
$stableManLiteral = '$VersionJsonName = "version.json"'
$devManLiteral = '$VersionJsonName = "version-dev.json"'
if ($sourceContent.IndexOf($stableUrlLiteral) -lt 0 -or $sourceContent.IndexOf($stableManLiteral) -lt 0) {
throw "Bootstrap dev-render: anchor literal missing in $BootstrapSource"
}
$devContent = $sourceContent.Replace($stableUrlLiteral, $devUrlLiteral).Replace($stableManLiteral, $devManLiteral)
Write-Host "Rendered $DevBlob (dev agent URL + version-dev.json)"
# ------------------------------------------------------------------ 3. bump guard
$publishedBytes = Get-PublishedBytes "$AliasUrl/$BootstrapBlob"
if ($null -eq $publishedBytes) {
Write-Host 'No published bootstrap yet -- version-bump guard skipped (first publish)'
} elseif ((Get-Sha256 $publishedBytes) -eq (Get-Sha256 $sourceBytes)) {
Write-Host 'Published bootstrap is already byte-identical to the source'
} else {
$publishedText = [System.Text.Encoding]::UTF8.GetString($publishedBytes)
$publishedVersion = Get-BootstrapScriptVersion -Content $publishedText -Origin "$AliasUrl/$BootstrapBlob" -Optional
if ($null -eq $publishedVersion) {
Write-Host "::warning::The published $BootstrapBlob is not a readable bootstrap script (no ScriptVersion found). Republishing over it."
} elseif ($publishedVersion -ne $scriptVersion) {
Write-Host "Version bump: $publishedVersion -> $scriptVersion"
} elseif ((Get-CodeFingerprint -Text $publishedText -Origin "$AliasUrl/$BootstrapBlob") -eq
(Get-CodeFingerprint -Text $sourceContent -Origin $BootstrapSource)) {
# Comments, typos in comments, reflow -- nothing a consumer of the version can act on.
Write-Host "::warning::$BootstrapBlob changed in comments or formatting only; publishing under the unchanged version $scriptVersion."
} else {
throw ("Version-bump guard: $BootstrapBlob changed behaviour but ScriptVersion is still $scriptVersion. " +
"Bump it in $BootstrapSource -- the docs badge, version.json.bootstrapVersion and the " +
'portal outdated-script hint all read that value, and customers decide from it whether ' +
'to re-upload their Intune copy. Comment-only and formatting changes do not need a bump.')
}
}
# ------------------------------------------------------------------ 4. upload
# Sources beyond the bootstrap pair, read straight from the repo.
$extraSources = @(
@{ Path = 'scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1'; BlobName = 'Test-ShouldBootstrapAgent.ps1' }
# Customer-side Graph add-on grant script -- docs and admin UI link it directly.
@{ Path = 'scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1'; BlobName = 'Grant-AutopilotMonitorAddOn.ps1' }
)
# Uninstall-AutopilotMonitor.ps1 is deliberately NOT published: nothing links it, and
# an unauthenticated uninstall script on the public download host is not a feature.
$publishSet = [System.Collections.Generic.List[object]]::new()
$publishSet.Add([pscustomobject]@{ BlobName = $BootstrapBlob; Bytes = $sourceBytes })
$publishSet.Add([pscustomobject]@{ BlobName = $DevBlob; Bytes = (Get-PublishBytes $devContent) })
foreach ($extra in $extraSources) {
$extraPath = Join-Path $RepoRoot $extra.Path
if (-not (Test-Path $extraPath)) {
throw "Publish source missing: $extraPath"
}
$publishSet.Add([pscustomobject]@{ BlobName = $extra.BlobName; Bytes = (Get-PublishBytes (Get-Content $extraPath -Raw)) })
}
$legacySas = if ($LegacySasToken) { $LegacySasToken.TrimStart('?') } else { '' }
$writeSas = if ($SasToken) { $SasToken.TrimStart('?') } else { '' }
foreach ($item in $publishSet) {
# Explicit type, not a convenience: an Object[] body is uploaded as space-separated
# decimals rather than raw bytes, and every hash check still passes because [byte[]]
# parameter coercion repairs it everywhere EXCEPT the wire.
[byte[]]$bytes = $item.Bytes
$item | Add-Member -NotePropertyName Sha256 -NotePropertyValue (Get-Sha256 $bytes) -Force
if ($DryRun) {
# Doubles as a drift check: says per blob whether the alias already serves this source.
$served = Get-PublishedBytes "$AliasUrl/$($item.BlobName)"
$state = if ($null -eq $served) { 'MISSING on the alias' }
elseif ((Get-Sha256 $served) -eq $item.Sha256) { 'already current' }
else { 'STALE on the alias' }
Write-Host " [dry-run] $($item.BlobName) ($($bytes.Length) bytes, sha256 $($item.Sha256)) -- $state"
continue
}
# no-cache on every blob: they all rotate in place and the download alias fronts them
# with Front Door. Route caching is disabled there, but a stale script paired with a
# fresh manifest would fail the bootstrap SHA check -- this keeps that class of bug
# impossible even if caching is ever re-enabled.
$headers = @{ 'x-ms-blob-type' = 'BlockBlob'; 'Content-Type' = $ScriptContentType; 'x-ms-blob-cache-control' = 'no-cache' }
Invoke-RestMethod -Uri "$ContainerUrl/$($item.BlobName)?$writeSas" -Method Put -Headers $headers -Body $bytes | Out-Null
# Read straight back from the blob (authoritative, no CDN in the way) before touching the
# mirror or the next file. The alias verification at the end would catch a bad body too,
# but only after every blob was already overwritten -- these are live customer downloads.
$written = Get-PublishedBytes "$ContainerUrl/$($item.BlobName)"
if ($null -eq $written -or (Get-Sha256 $written) -ne $item.Sha256) {
$writtenLength = if ($null -eq $written) { 'missing' } else { "$($written.Length) bytes" }
throw ("Upload of $($item.BlobName) did not round-trip: sent $($bytes.Length) bytes, " +
"blob now holds $writtenLength. Publishing stopped before the mirror and the " +
'remaining files, so no further blob was touched.')
}
Write-Host " uploaded $($item.BlobName) ($($bytes.Length) bytes, round-trip verified)"
if ($legacySas) {
try {
Invoke-RestMethod -Uri "$LegacyContainerUrl/$($item.BlobName)?$legacySas" -Method Put -Headers $headers -Body $bytes | Out-Null
Write-Host " mirrored $($item.BlobName) to legacy storage"
} catch {
Write-Host "::warning::legacy mirror failed for $($item.BlobName) -- $($_.Exception.Message)"
}
}
}
# ------------------------------------------------------------------ 5. version oracles
# version.json is the agent manifest; only bootstrapVersion belongs to us. Read from the
# blob (not the alias) for an authoritative ETag, write back under If-Match so a concurrent
# agent release cannot lose its version/sha256 fields.
if (-not $DryRun) {
try {
$manifestResp = Invoke-WebRequest -Uri "$ContainerUrl/version.json" -Method Get -UseBasicParsing
$manifestRaw = if ($manifestResp.Content -is [byte[]]) {
[System.Text.Encoding]::UTF8.GetString($manifestResp.Content)
} else { [string]$manifestResp.Content }
$manifest = $manifestRaw | ConvertFrom-Json
if ($manifest.bootstrapVersion -eq $scriptVersion) {
Write-Host "version.json already reports bootstrapVersion $scriptVersion"
} else {
$etag = '"' + ([string]($manifestResp.Headers['ETag'] | Select-Object -First 1)).Trim('"') + '"'
$manifest | Add-Member -NotePropertyName 'bootstrapVersion' -NotePropertyValue $scriptVersion -Force
$body = [System.Text.Encoding]::UTF8.GetBytes(($manifest | ConvertTo-Json -Compress))
$manifestHeaders = @{
'x-ms-blob-type' = 'BlockBlob'
'Content-Type' = 'application/json'
'x-ms-blob-cache-control' = 'no-cache'
'If-Match' = $etag
}
Invoke-RestMethod -Uri "$ContainerUrl/version.json?$writeSas" -Method Put -Headers $manifestHeaders -Body $body | Out-Null
Write-Host "version.json bootstrapVersion -> $scriptVersion (agent fields untouched)"
if ($legacySas) {
try {
$legacyHeaders = @{ 'x-ms-blob-type' = 'BlockBlob'; 'Content-Type' = 'application/json'; 'x-ms-blob-cache-control' = 'no-cache' }
Invoke-RestMethod -Uri "$LegacyContainerUrl/version.json?$legacySas" -Method Put -Headers $legacyHeaders -Body $body | Out-Null
} catch {
Write-Host "::warning::legacy mirror failed for version.json -- $($_.Exception.Message)"
}
}
}
} catch {
if ((Get-HttpStatus $_) -eq 412) {
throw 'version.json changed while publishing (concurrent agent release). Re-run this workflow.'
}
throw
}
}
$tableSas = if ($TableSasToken) { $TableSasToken.TrimStart('?') } else { '' }
if ($DryRun) {
Write-Host " [dry-run] would set AdminConfiguration.LatestBootstrapV2ScriptVersion = $scriptVersion"
} elseif ([string]::IsNullOrEmpty($tableSas) -or $tableSas -eq 'XXX') {
Write-Host 'SKIPPED: AdminConfiguration update -- set AZURE_TABLE_ADMIN_SAS_TOKEN to enable the portal version oracle'
} else {
$tableHeaders = @{
'Content-Type' = 'application/json'
'Accept' = 'application/json;odata=nometadata'
'If-Match' = '*'
}
$tableBody = @{ LatestBootstrapV2ScriptVersion = $scriptVersion } | ConvertTo-Json
Invoke-RestMethod -Uri "$TableUrl`?$tableSas" -Method Merge -Headers $tableHeaders -Body $tableBody | Out-Null
Write-Host "AdminConfiguration.LatestBootstrapV2ScriptVersion = $scriptVersion"
}
# ------------------------------------------------------------------ 6. verify via alias
if (-not $DryRun) {
$pending = [System.Collections.ArrayList]::new()
$publishSet | ForEach-Object { [void]$pending.Add($_) }
for ($attempt = 1; $attempt -le 6 -and $pending.Count -gt 0; $attempt++) {
if ($attempt -gt 1) { Start-Sleep -Seconds 20 }
foreach ($item in @($pending)) {
$servedBytes = Get-PublishedBytes "$AliasUrl/$($item.BlobName)"
if ($null -ne $servedBytes -and (Get-Sha256 $servedBytes) -eq $item.Sha256) {
Write-Host " verified $($item.BlobName) via $AliasUrl"
$pending.Remove($item)
}
}
if ($pending.Count -gt 0) {
Write-Host " attempt $attempt : $($pending.Count) blob(s) not current on the alias yet"
}
}
if ($pending.Count -gt 0) {
$names = ($pending | ForEach-Object { $_.BlobName }) -join ', '
throw ("Alias still serves stale content for: $names. The blobs were written, so this is a Front Door " +
'cache issue -- purge with: az afd endpoint purge --resource-group rg-autopilotmonitor-prd-gwc ' +
"--profile-name autopilotmonitor-fd --endpoint-name apm-download --content-paths '/agent/*'")
}
}
if ($env:GITHUB_STEP_SUMMARY) {
$mode = if ($DryRun) { 'DRY RUN -- nothing uploaded' } else { 'published + verified via the download alias' }
$summary = @(
"### Bootstrap scripts: $mode",
'',
"Bootstrap script version: **$scriptVersion**",
''
) + ($publishSet | ForEach-Object { "- $($_.BlobName) -- sha256 $($_.Sha256)" })
($summary -join [Environment]::NewLine) | Out-File -FilePath $env:GITHUB_STEP_SUMMARY -Append -Encoding utf8
}
Write-Host 'Done.'
@@ -0,0 +1,63 @@
name: Bootstrap script gates
# Quality gate for the customer-facing PowerShell scripts, factored out so it exists
# once and both callers run the identical checks:
# * pester.yml -- pull requests (reports, does not block the merge button)
# * publish-scripts.yml -- pushes to main, where it BLOCKS the upload to the
# download alias. Devices execute these scripts as SYSTEM
# during enrollment, so a red suite must never ship.
on:
workflow_call:
permissions:
contents: read
jobs:
gates:
name: Bootstrap guard tests (PS 5.1)
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- name: Install Pester 5+
shell: powershell
run: |
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force -Scope CurrentUser | Out-Null
Install-Module Pester -Scope CurrentUser -Force -SkipPublisherCheck -MinimumVersion 5.5.0
# Windows PowerShell 5.1 on purpose: that is the runtime IME uses to execute
# platform scripts, so guard/relax logic is verified without resetting a device.
- name: Run Pester
shell: powershell
run: |
Import-Module Pester -MinimumVersion 5.5.0
$c = New-PesterConfiguration
$c.Run.Path = 'scripts/Bootstrap'
$c.Run.Exit = $true
$c.Output.Verbosity = 'Detailed'
Invoke-Pester -Configuration $c
- name: Guard scripts stay pure ASCII
shell: powershell
run: |
$failed = $false
Get-ChildItem 'scripts/Bootstrap' -Filter '*.ps1' | ForEach-Object {
$bytes = [System.IO.File]::ReadAllBytes($_.FullName)
for ($i = 0; $i -lt $bytes.Length; $i++) {
if ($bytes[$i] -gt 127) {
Write-Host "::error file=scripts/Bootstrap/$($_.Name)::Non-ASCII byte 0x$('{0:X2}' -f $bytes[$i]) at offset $i. PS 5.1 (IME) reads BOM-less files as ANSI and corrupts multi-byte chars."
$failed = $true
break
}
}
}
if ($failed) { exit 1 }
Write-Host 'All bootstrap scripts are pure ASCII.'
# The publish chain renders the -Dev variant by literal substitution and refuses to
# ship when an anchor moved. Running it dry here surfaces that (and any drift against
# the alias) on the PR instead of on the push that would have published.
- name: Publish dry run (dev-render anchors + version-bump guard + drift)
shell: pwsh
run: ./.github/scripts/Publish-BootstrapScripts.ps1 -DryRun
+22 -37
View File
@@ -33,6 +33,13 @@ permissions:
id-token: write # keyless Sigstore signing (artifact attestation)
attestations: write # store the provenance attestation
# Shared with publish-scripts.yml: both write version.json (this workflow the agent fields,
# the script publisher bootstrapVersion under If-Match). Serialising them keeps a script push
# from colliding with a release. Never cancel -- a half-published set is worse than a queue.
concurrency:
group: agent-blob-publish
cancel-in-progress: false
env:
LINE_MAJOR: 2
AGENT_PROJECT: src/Agent/AutopilotMonitor.Agent.V2/AutopilotMonitor.Agent.V2.csproj
@@ -46,8 +53,6 @@ env:
VERSIONED_ZIP: AutopilotMonitor-Agent-v2.zip
STABLE_ZIP: AutopilotMonitor-Agent.zip
STABLE_MANIFEST: version.json
STABLE_BOOTSTRAP: Install-AutopilotMonitor.ps1
STABLE_BOOTSTRAP_DEV: Install-AutopilotMonitor-Dev.ps1
jobs:
build:
@@ -181,6 +186,8 @@ jobs:
# commit/buildUtc are the same two fields the backend and MCP manifests carry —
# additive, and the bootstrap reads this file by field name, so older installers
# ignore them. sha256/bootstrapVersion stay: those are the customer contract.
# bootstrapVersion is seeded here so the published manifest is never briefly
# missing it; Publish-BootstrapScripts.ps1 owns and re-asserts the field afterwards.
$manifest | Add-Member -NotePropertyName "sha256" -NotePropertyValue $sha256Hash -Force
$manifest | Add-Member -NotePropertyName "commit" -NotePropertyValue '${{ github.sha }}'.Substring(0, 7) -Force
$manifest | Add-Member -NotePropertyName "buildUtc" -NotePropertyValue ([DateTime]::UtcNow.ToString('o')) -Force
@@ -190,29 +197,12 @@ jobs:
$manifest | ConvertTo-Json -Compress | Set-Content $env:MANIFEST_NAME -Encoding UTF8
Write-Host "$env:MANIFEST_NAME content: $(Get-Content $env:MANIFEST_NAME)"
# Bootstrap = single generic source. Stable variant = source 1:1.
# Dev variant = literal-string substitution of the two URL/manifest defaults.
# Both are uploaded only on -PublishAsStable runs (versioned namespace doesn't carry bootstrap).
if ('${{ inputs.publish_as_stable }}' -eq 'true') {
$stableUrl = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip"'
$devUrl = '$AgentDownloadUrl = "https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent-dev.zip"'
$stableMan = '$VersionJsonName = "version.json"'
$devMan = '$VersionJsonName = "version-dev.json"'
Copy-Item $env:BOOTSTRAP_SOURCE $env:STABLE_BOOTSTRAP -Force
$content = Get-Content $env:BOOTSTRAP_SOURCE -Raw
if ($content.IndexOf($stableUrl) -lt 0 -or $content.IndexOf($stableMan) -lt 0) {
Write-Error "Bootstrap dev-render: anchor literal missing in $env:BOOTSTRAP_SOURCE"
exit 1
}
$devContent = $content.Replace($stableUrl, $devUrl).Replace($stableMan, $devMan)
[System.IO.File]::WriteAllText("$env:STABLE_BOOTSTRAP_DEV", $devContent, (New-Object System.Text.UTF8Encoding $false))
}
# The customer-facing scripts (stable + dev render, test script, grant script) are
# published by .github/scripts/Publish-BootstrapScripts.ps1 -- see the upload step.
"versioned_zip_release=$versionedZip" >> $env:GITHUB_OUTPUT
"sha256=$sha256Hash" >> $env:GITHUB_OUTPUT
"exe_sha256=$exeSha256Hash" >> $env:GITHUB_OUTPUT
"bootstrap_version=$bootstrapVersion" >> $env:GITHUB_OUTPUT
- name: Upload to Azure Blob Storage
if: inputs.upload_blob
@@ -264,22 +254,21 @@ jobs:
Write-Host "PUBLISH AS STABLE -- writing V2 into stable namespace (customer-facing)" -ForegroundColor Magenta
Send-Blob -Local $env:VERSIONED_ZIP -BlobName $env:STABLE_ZIP -ContentType "application/zip"
Send-Blob -Local $env:MANIFEST_NAME -BlobName $env:STABLE_MANIFEST -ContentType "application/json"
Send-Blob -Local $env:STABLE_BOOTSTRAP -BlobName $env:STABLE_BOOTSTRAP -ContentType "text/plain; charset=utf-8"
Send-Blob -Local $env:STABLE_BOOTSTRAP_DEV -BlobName $env:STABLE_BOOTSTRAP_DEV -ContentType "text/plain; charset=utf-8"
if (Test-Path "scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1") {
Send-Blob -Local "scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1" -BlobName "Test-ShouldBootstrapAgent.ps1" -ContentType "text/plain; charset=utf-8"
}
# Customer-side Graph add-on grant script -- docs and admin UI link
# https://download.autopilotmonitor.com/agent/Grant-AutopilotMonitorAddOn.ps1
if (Test-Path "scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1") {
Send-Blob -Local "scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1" -BlobName "Grant-AutopilotMonitorAddOn.ps1" -ContentType "text/plain; charset=utf-8"
}
# Customer-facing scripts: same publisher publish-scripts.yml runs on every push
# to main, so a release lands a set that is consistent with what is already live
# (dev render, version-bump guard, bootstrapVersion oracles, alias verification).
# Runs AFTER version.json so its bootstrapVersion read sees this build's manifest.
./.github/scripts/Publish-BootstrapScripts.ps1 `
-SasToken '${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}' `
-LegacySasToken '${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}' `
-TableSasToken '${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}'
}
# Update AdminConfiguration LatestAgentV2* -- ONLY on stable cutover. The hash-oracle
# must never point at a build that is not actually in the stable namespace.
# LatestBootstrapV2ScriptVersion is NOT set here: it belongs to the script publisher
# above, which also owns it on script-only pushes. One writer per field.
$tableSasToken = "${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}".TrimStart('?')
if ('${{ inputs.publish_as_stable }}' -ne 'true') {
Write-Host "SKIPPED: AdminConfiguration update (versioned-only run -- stable namespace unchanged)"
@@ -289,22 +278,18 @@ jobs:
$sha256 = "${{ steps.artifact.outputs.sha256 }}"
$exeSha256 = "${{ steps.artifact.outputs.exe_sha256 }}"
$agentVersion = "${{ steps.version.outputs.version }}"
$bootstrapVersion = "${{ steps.artifact.outputs.bootstrap_version }}"
$tableUrl = "https://autopilotmonitoreu.table.core.windows.net/AdminConfiguration(PartitionKey='GlobalConfig',RowKey='config')?$tableSasToken"
$tableBody = @{
LatestAgentV2Version = $agentVersion
LatestAgentV2Sha256 = $sha256
LatestAgentV2ExeSha256 = $exeSha256
}
if ($bootstrapVersion) {
$tableBody["LatestBootstrapV2ScriptVersion"] = $bootstrapVersion
}
Invoke-RestMethod -Uri $tableUrl -Method Merge -Headers @{
"Content-Type" = "application/json"
"Accept" = "application/json;odata=nometadata"
"If-Match" = "*"
} -Body ($tableBody | ConvertTo-Json)
Write-Host "AdminConfiguration updated: V2 version=$agentVersion, sha256=$sha256, exeSha256=$exeSha256, bootstrapVersion=$bootstrapVersion"
Write-Host "AdminConfiguration updated: V2 version=$agentVersion, sha256=$sha256, exeSha256=$exeSha256"
}
# Sigstore keyless provenance: binds the ZIP digest to this workflow run (repo, ref,
+139
View File
@@ -0,0 +1,139 @@
name: Build & Publish Bootstrap MSI
# Thin MDM LOB delivery vehicle for WDP (Autopilot v2) — see
# src/Agent/AutopilotMonitor.BootstrapMsi/Package.wxs. The MSI carries no bootstrap
# logic (only the runner script that downloads the CURRENT server-hosted
# Install-AutopilotMonitor.ps1), so it is rebuilt rarely: only when the runner script
# or the WiX package changes. Version lives in Package.wxs — bump it there before
# dispatching so Intune LOB update detection sees a new ProductVersion.
on:
workflow_dispatch:
inputs:
upload_blob:
description: 'Upload to blob (customer-facing via download.autopilotmonitor.com/agent/)'
required: true
type: boolean
default: true
permissions:
contents: read
id-token: write # keyless Sigstore signing (artifact attestation)
attestations: write # store the provenance attestation
env:
MSI_PROJECT_DIR: src/Agent/AutopilotMonitor.BootstrapMsi
RUNNER_SCRIPT: src/Agent/AutopilotMonitor.BootstrapMsi/Invoke-BootstrapDownload.ps1
STABLE_MSI: AutopilotMonitor-Bootstrap.msi
jobs:
build:
runs-on: windows-latest
name: Build Bootstrap MSI
steps:
- name: Checkout repository
uses: actions/checkout@v7
# The runner executes as SYSTEM on every enrolling device — a parse error or
# non-ASCII mojibake (PS 5.1 codepage roulette) must fail the build, not the fleet.
- name: Validate runner script (parse + ASCII-only)
shell: pwsh
run: |
$errs = $null
[System.Management.Automation.Language.Parser]::ParseFile((Resolve-Path $env:RUNNER_SCRIPT), [ref]$null, [ref]$errs) | Out-Null
if ($errs) { $errs | ForEach-Object { Write-Error $_.ToString() }; exit 1 }
if ((Get-Content $env:RUNNER_SCRIPT -Raw) -match '[^\x00-\x7F]') {
Write-Error "Runner script contains non-ASCII characters"
exit 1
}
Write-Host "Runner script parse + ASCII check passed"
- name: Read MSI version from Package.wxs
id: version
shell: pwsh
run: |
$wxs = Get-Content "$env:MSI_PROJECT_DIR/Package.wxs" -Raw
if ($wxs -notmatch 'Version="(\d+\.\d+\.\d+)"') {
Write-Error "Could not parse Version from Package.wxs"
exit 1
}
$version = $Matches[1]
Write-Host "MSI version (from Package.wxs): $version"
"version=$version" >> $env:GITHUB_OUTPUT
- name: Install WiX toolset
run: dotnet tool install --global wix --version "5.*"
- name: Build MSI
shell: pwsh
run: |
# wix resolves File/@Source relative to the working directory, not the wxs —
# build from inside the project dir (same as the local build documented in the wxs).
$out = Join-Path (Get-Location) $env:STABLE_MSI
Push-Location $env:MSI_PROJECT_DIR
wix build Package.wxs -arch x64 -o $out
Pop-Location
if (-not (Test-Path $env:STABLE_MSI)) { Write-Error "wix build produced no MSI"; exit 1 }
$sha256 = (Get-FileHash $env:STABLE_MSI -Algorithm SHA256).Hash.ToLowerInvariant()
Write-Host "Built $env:STABLE_MSI ($([math]::Round((Get-Item $env:STABLE_MSI).Length / 1KB)) KB)"
Write-Host "SHA-256: $sha256"
- name: Upload to Azure Blob Storage
if: inputs.upload_blob
shell: pwsh
run: |
$containerUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent"
$sasToken = "${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}".TrimStart('?')
$version = "${{ steps.version.outputs.version }}"
# Same Send-Blob contract as build-agent.yml: no-cache on every blob because the
# stable name rotates in place behind the download alias. No legacy mirror — the
# MSI channel never existed on the legacy account.
function Send-Blob {
param([string]$Local, [string]$BlobName)
$bytes = [System.IO.File]::ReadAllBytes($Local)
$headers = @{
"x-ms-blob-type" = "BlockBlob"
"Content-Type" = "application/octet-stream"
"x-ms-blob-cache-control" = "no-cache"
}
$md5 = [System.Security.Cryptography.MD5]::Create()
$headers["Content-MD5"] = [Convert]::ToBase64String($md5.ComputeHash($bytes))
$md5.Dispose()
Write-Host "Uploading $BlobName ($([math]::Round($bytes.Length / 1KB)) KB)..."
Invoke-RestMethod -Uri "$containerUrl/$BlobName`?$sasToken" -Method Put -Headers $headers -Body $bytes
Write-Host " uploaded $BlobName"
}
# Versioned copy for provenance/rollback + stable customer-facing name.
Send-Blob -Local $env:STABLE_MSI -BlobName "AutopilotMonitor-Bootstrap-v$version.msi"
Send-Blob -Local $env:STABLE_MSI -BlobName $env:STABLE_MSI
# E2E: the Front Door route /agent/* must serve the fresh bytes on the public alias.
- name: Verify download alias serves the MSI
if: inputs.upload_blob
shell: pwsh
run: |
$localSize = (Get-Item $env:STABLE_MSI).Length
$resp = Invoke-WebRequest -Uri "https://download.autopilotmonitor.com/agent/$env:STABLE_MSI" -Method Head -UseBasicParsing
if ($resp.StatusCode -ne 200) { Write-Error "Alias HEAD returned $($resp.StatusCode)"; exit 1 }
$remoteSize = [long]$resp.Headers['Content-Length'][0]
if ($remoteSize -ne $localSize) {
Write-Error "Size mismatch: alias serves $remoteSize bytes, local MSI is $localSize bytes"
exit 1
}
Write-Host "Alias verified: https://download.autopilotmonitor.com/agent/$env:STABLE_MSI ($remoteSize bytes)"
# Sigstore keyless provenance, same as the agent ZIP. Verify:
# gh attestation verify AutopilotMonitor-Bootstrap.msi --repo okieselbach/AutopilotMonitor
- name: Attest build provenance (MSI)
uses: actions/attest-build-provenance@v4
with:
subject-path: ${{ env.STABLE_MSI }}
- name: Upload build artifact
uses: actions/upload-artifact@v7
with:
name: AutopilotMonitor-Bootstrap-v${{ steps.version.outputs.version }}
path: ${{ env.STABLE_MSI }}
+13 -45
View File
@@ -1,16 +1,20 @@
name: Pester (Bootstrap scripts)
# Runs the Pester suite for the Intune bootstrap scripts under Windows PowerShell 5.1
# (the runtime IME uses to execute platform scripts) so guard/relax logic changes are
# verified without resetting a device. Like ci.yml it reports rather than blocks.
# Pull-request view of the bootstrap script gates. Like ci.yml it reports rather than
# blocks the merge button.
#
# There is deliberately no push trigger: publish-scripts.yml runs the very same
# reusable gate workflow on every push to main and blocks the upload to the download
# alias on it. A push trigger here would only duplicate that run, with no consequence.
on:
push:
branches: [main]
paths: ['scripts/Bootstrap/**']
pull_request:
branches: [main]
paths: ['scripts/Bootstrap/**']
paths:
- 'scripts/Bootstrap/**'
- 'scripts/CustomerSetup/**'
- '.github/scripts/Publish-BootstrapScripts.ps1'
- '.github/workflows/bootstrap-script-gates.yml'
workflow_dispatch:
permissions:
@@ -21,41 +25,5 @@ concurrency:
cancel-in-progress: true
jobs:
pester:
name: Bootstrap guard tests (PS 5.1)
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- name: Install Pester 5+
shell: powershell
run: |
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force -Scope CurrentUser | Out-Null
Install-Module Pester -Scope CurrentUser -Force -SkipPublisherCheck -MinimumVersion 5.5.0
- name: Run Pester
shell: powershell
run: |
Import-Module Pester -MinimumVersion 5.5.0
$c = New-PesterConfiguration
$c.Run.Path = 'scripts/Bootstrap'
$c.Run.Exit = $true
$c.Output.Verbosity = 'Detailed'
Invoke-Pester -Configuration $c
- name: Guard scripts stay pure ASCII
shell: powershell
run: |
$failed = $false
Get-ChildItem 'scripts/Bootstrap' -Filter '*.ps1' | ForEach-Object {
$bytes = [System.IO.File]::ReadAllBytes($_.FullName)
for ($i = 0; $i -lt $bytes.Length; $i++) {
if ($bytes[$i] -gt 127) {
Write-Host "::error file=scripts/Bootstrap/$($_.Name)::Non-ASCII byte 0x$('{0:X2}' -f $bytes[$i]) at offset $i. PS 5.1 (IME) reads BOM-less files as ANSI and corrupts multi-byte chars."
$failed = $true
break
}
}
}
if ($failed) { exit 1 }
Write-Host 'All bootstrap scripts are pure ASCII.'
gates:
uses: ./.github/workflows/bootstrap-script-gates.yml
+56
View File
@@ -0,0 +1,56 @@
name: Publish customer scripts
# Keeps the scripts behind download.autopilotmonitor.com/agent/ current WITHOUT waiting
# for an agent release. Before this workflow existed, the script blobs rotated only on a
# publish_as_stable run of build-agent.yml, so a bootstrap fix sat unpublished until the
# next agent cutover -- while the WDP bootstrap MSI downloads the published copy on every
# enrolling device.
#
# Same publish code as the release path (.github/scripts/Publish-BootstrapScripts.ps1);
# this workflow only supplies a second trigger for it.
on:
push:
branches: [main]
paths:
- 'scripts/Bootstrap/**'
- 'scripts/CustomerSetup/**'
- '.github/scripts/Publish-BootstrapScripts.ps1'
workflow_dispatch:
inputs:
dry_run:
description: 'Run every check and report drift, upload nothing'
required: true
type: boolean
default: false
permissions:
contents: read
# Shared with build-agent.yml: a script publish and an agent release both touch
# version.json, and the bootstrapVersion read-modify-write must not interleave with the
# release writing its agent fields. Never cancel -- a half-published set is worse than a queue.
concurrency:
group: agent-blob-publish
cancel-in-progress: false
jobs:
gates:
uses: ./.github/workflows/bootstrap-script-gates.yml
publish:
name: Publish to the download alias
needs: gates
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
- name: Publish bootstrap + customer scripts
shell: pwsh
run: |
$dryRun = '${{ inputs.dry_run }}' -eq 'true'
./.github/scripts/Publish-BootstrapScripts.ps1 `
-SasToken '${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}' `
-LegacySasToken '${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}' `
-TableSasToken '${{ secrets.AZURE_TABLE_ADMIN_SAS_TOKEN }}' `
-DryRun:$dryRun
+6
View File
@@ -158,3 +158,9 @@ docs/~$Introduction-AutopilotMonitor.pptx
# Web build stamp: generated by scripts/generate-version.js on every build (buildUtc
# changes each run) and served at /version.json from the static export.
src/Web/autopilot-monitor-web/public/version.json
# Local IME binaries scratch used by the /ime-decompile skill (MSI + msiexec-
# extracted payload per version, for grepping in binaries). Decompilates live in
# the private ime-decompiles repo, MSIs in the ime-archive blob - this stays local.
ime-files/
+127
View File
@@ -0,0 +1,127 @@
---
type: concept
title: CMTrace Time Resolution — Per-Line Self-Anchoring
description: How the agent converts bias-less CMTrace local times to UTC without trusting any process's timezone belief — each provably fresh line anchors its own offset against the agent clock on the 15-minute grid; everything not provably fresh falls back uniformly and says so.
resource: src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/Ime
tags: [agent, ime, cmtrace, timestamps, timezone, calibration, provenance]
timestamp: 2026-08-20
---
# Problem
A CMTrace line carries local time and no offset — IME 1.104 writes `DateTime.Now.TimeOfDay`
in both of its trace listeners. Converting such a line with the reader's own
`TimeZoneInfo.Local` silently assumes the writing and reading processes agree on the zone.
They do not: each caches its zone at process start and follows neither a later `tzutil` nor
a Windows auto-timezone change. The error is `Offset_writer − Offset_reader` — zero only
when both believe the same thing, right or wrong. Field measurement over 11,068 sessions
found 26 sessions with a real error, from +1 h to −17 h.
Two designs failed before this one:
* **Fix the reader's belief** (`TimeZoneInfo.ClearCachedData`, live OS offset): makes the
agent isolated-correct and thereby breaks the majority of sessions, where two equally
stale beliefs cancel.
* **One measured offset per file** (agent 2.0.1410, reverted in `04b1a7c6`): a single log
file holds lines from multiple writer *eras*. `AgentExecutor.log` is written by
short-lived child processes whose zone belief flips per process — interleaved in one
file, and flipping *back* (fixture:
`tests/fixtures/cmtrace-logs/agentexecutor-two-writer-eras-v1.cmtrace`). Any cross-line
anchor — per file, or nearest-in-write-order — applies one era's offset to the other
era's lines. In the field this shifted `script_started` events by −9 h against their
correct completions.
# Schema
The resolution rule, in order:
1. **Writer-declared bias** (`+480` suffix): authoritative, used as-is. Never a calibration
anchor — it needs no measurement and must not overwrite one.
2. **Per-line self-anchoring** — only for lines read in a *provably fresh* pass:
```
offset_line = round((local − agentUtcNow) / 15 min) · 15 min
lineUtc = local − offset_line
```
Every real UTC offset is a whole multiple of 15 minutes, so the rounding absorbs poll
and flush latency. No cross-line state exists, so interleaved eras resolve line by
line — the property the fixture test pins. The line's own timestamp still contributes
sub-poll precision and ordering (two lines 6 ms apart stay 6 ms apart; a plain
`occurredAt = now` could not do that). Guards: grid residual ≤ 2 min, offset within
UTC−12 … UTC+14.
3. **Uniform reader-zone fallback** for everything else: wrong in absolute terms whenever
the writer held a different belief, but *self-consistent* — both ends of a duration are
wrong by the same amount, so derived durations stay right. The revert's core lesson:
partially corrected is strictly worse than uniformly wrong.
**Freshness** is what makes rule 2 sound, and it is strict:
* Every poll look at a file stamps `LastCheckedUtc` (`LogFilePositionTracker.MarkChecked`),
including "no new data" and an empty first sight — so a file first seen empty
(`AgentExecutor.log` appearing mid-enrollment) anchors from its very first content.
* A pass's lines are fresh only if the file was previously observed *in this process* and
the gap since the last look is ≤ `FreshLineMaxAge` (30 s — far above the 100 ms poll,
far below the 13-minute zone where a line's *age* could round onto the offset grid).
* A restart-restored position bookmark deliberately carries **no** freshness: the first
pass after a restart reads downtime backlog and must never anchor.
**Provenance** (`sourceOffsetOrigin` in event DataJson): `bias` | `line-anchored` |
`reader-zone-fallback` (plus retired `calibrated` from 2.0.1410 events). The applied
offset is always recorded (`sourceOffsetMinutes`), the raw local time too
(`sourceLocalTs`), so any event can be re-derived later. The per-*file* measurement still
runs, purely observationally (`measuredWriterOffsetMinutes`, Info log when it disagrees
with the process zone).
# Constraints
* An anchored line resolves to `now ± 2 min` *by construction*. The accepted edge: a
freshly written line carrying a **replayed** old timestamp whose age is exactly
N×15 min ± 2 min anchors to ≈now instead of its replayed past — bounded error, and
sub-24 h replays are treated as current by the historic-replay guard anyway.
* Snapshot readers (`StallProbeCollector`, `LogParserCollector`) cannot tail and therefore
cannot prove freshness — they stay on the marked fallback. This is a documented limit,
not an omission; sharing the per-file measurement with them would re-apply exactly what
the revert removed.
* Mixed pairs remain possible across a restart (start resolved from backlog fallback,
completion line-anchored) and are recognizable via provenance.
# Backend regression tripwire
The backend watches for the failure mode this design eliminates. On every terminal ingest
batch, the counter reconcile's existing event-partition scan additionally collects
`Δ = ReceivedAt − OccurredUtc` per event, split by `Source == "ImeLogTracker"` versus the
rest (`CmTraceSkewTripwire`, hooked in `EventIngestProcessor`). If
`median(Δ_IME) − median(Δ_other)` is a clean 15-minute multiple (residual < 2 min — the
same constants as the anchoring grid guard above) with ≥ 20 samples over ≥ 3 distinct
upload batches per side, a `CmTraceTimeSkewRegression` ops event fires (category Agent,
warning), carrying all numbers plus the session's `sourceOffsetOrigin` histogram.
Bias-dominated sessions are suppressed (writer-declared offsets cannot be an anchoring
regression); `measuredWriterOffsetMinutes` is never consulted (sticky after era
flip-backs). Kill switch: app setting `CmTraceSkewTripwireDisabled`. Goal state: the event
never fires — any hit is an anchoring case the per-line design misses, or a detector bug.
Device-clock problems (the customer-actionable cousin) are covered separately by the
`clock_skew` analyze condition behind `ANALYZE-DEV-008`, which excludes IME-derived events
precisely so an anchoring regression can never surface as a customer finding.
# Citations
* `CmTraceOffsetCalibrator.TryMeasureOffset` — the pure grid measurement, shared by
anchoring and the observational per-file path.
* `CmTraceSkewTripwire` + `CmTraceSkewTripwireTests` — the backend tripwire above.
* `RuleEngine.ConditionEvaluators.EvaluateClockSkewCondition` + `RuleEngineClockSkewTests`
— the customer-facing device-clock metrics (batch-median frames, spool-spread filter,
plateau step detection, end-state persistence).
* `ImeLogTracker.LogProcessing.cs` `ResolveEntryUtc` — the resolution order above.
* `ImeLogTrackerLineAnchoringTests` — resolves-to-T across writer beliefs, interleaved
eras in one chunk, freshness boundaries (first sight, empty first sight, poll gap,
restart restore), the grid-replay edge, and the field fixture replayed at production
poll cadence.
* `CmTraceCalibratorFileIsolationTests` — pins per-file isolation of the observational
measurement (session e9753578 logged a cross-file pairing the committed code could not
be shown to produce; the mechanism remains open, the resolution path is immune by
design, and `CalibrateFrom` logs the anchor timestamp as a tripwire).
* `tasks/todo.md` (2026-08-20) — full evidence trail: field measurements, the revert,
the two-writer-era fixture, and the disproven foreign-binary hypothesis (the SHA
mismatch was the known release race against the 5-minute AdminConfiguration cache).
+19 -1
View File
@@ -143,11 +143,29 @@ advisory backstop. All routes converge on `CompleteThroughFinalizingOrDefer`:
first-deployment window 100/101 for 200/210 without 110 — the RJ ESP was aborted,
typically by an interactive logon reclassifying the run as secondary-user; emits
the `realmjoin_first_deployment_incomplete` Warning, session 224b2087), and
`Timeout` (hard 60-min deadline). The abort rule reads the *persisted*
`Timeout` (60-min deadline from detection). The abort rule reads the *persisted*
`RealmJoinFacts.LastDeploymentPhase` (updated by the typed `RealmJoinPhaseChanged`
signal), so it survives agent restarts — the `RealmJoinDetected` replay path
evaluates it too. A first observation of 210 does **not** release (RJ may have
completed before agent boot and deploy afterwards).
**Activity-based timeout extension** (report 55e6afd61c9d — a large RJ package
catalog outlived the 60-min window mid-install): when the timeout deadline fires
while the first deployment is demonstrably active — `LastDeploymentPhase` 100/101
*and* deployment activity (`RealmJoinPhaseChanged` or a package observation, tracked
in `RealmJoinFacts.LastActivityUtc`; the phase captured at detection deliberately
does not count) within the last 60 min — the handler re-arms the deadline to
`lastActivity + 60 min` instead of timing out, emitting an Info
`realmjoin_timeout_extended` event. Re-arms clamp to an absolute cap of
**detection + 4 h**. Package activity is only observable at package *completion*
(RJ writes the registry subkey when the install finishes), so a single install
that is registry-silent for over 60 min still times out — the eventual
`realmjoin_timeout` carries a `reason` payload (`hard_timeout` / `inactivity` /
`absolute_cap`) telling the three shapes apart. A stale fire of an older deadline
incarnation (armed due later than the fire time; `OccurredAtUtc = DueAtUtc` per
scheduler contract) dead-ends as `realmjoin_timeout_stale_superseded_by_rearm`.
The idle case (detected, no deployment activity ever) times out at 60 min
unchanged, and phase 200/210 activity never extends.
2. `Finalizing` stage + ~5 s `FinalizingGrace` deadline.
3. Grace fires → `Completed`, `EnrollmentComplete` outcome, `enrollment_complete`
timeline event with full audit payload.
+113
View File
@@ -0,0 +1,113 @@
---
type: Concept
title: DO GroupId Auto-Set — Delivery Optimization peering group from a network fingerprint
description: EnableDoGroupIdAutoSet (ConfigVersion 40) makes the agent set the DOGroupId policy value to a deterministic GUID derived from the default gateway's IPv4 address and MAC (RealmJoin-compatible byte layout), so devices on the same local network form one DO peering group during enrollment; foreign DOGroupId/DOGroupIdSource policies always win.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Transport/DoGroupIdService.cs
tags:
- agent
- delivery-optimization
- network
- tenant-configuration
timestamp: 2026-08-21T00:00:00+02:00
---
# DO GroupId Auto-Set
## Schema
### The problem
With Delivery Optimization download mode Group (2), peering is scoped by GroupId. When no
GroupId is configured, Windows derives it from AD site → authenticated domain SID → Entra
tenant ID — for a cloud-native tenant that means **one tenant-wide group**, so "peers on
the same network" degrades to "peers anywhere in the tenant" and DO's cloud service has to
match them by public IP. A GroupId derived from the local network makes the group exactly
"devices behind the same gateway" — including branch offices behind NAT. RealmJoin ships
this as its network-fingerprint GroupId; our agent is on the device even earlier (during
ESP, when the big app downloads actually happen), so it can seed the same value for
Autopilot enrollments.
### The setting
`TenantConfiguration.EnableDoGroupIdAutoSet` (`bool?`, default null→false), served as
`AgentConfigResponse.EnableDoGroupIdAutoSet` — **ConfigVersion 40**. Portal: Settings →
Agent → Settings, toggle "Set Delivery Optimization Group ID", stored via the usual
tenant-config roundtrip (`TableConfigRepository` Store+Map). Also shown in the admin
tenant-config report. `RemoteConfigMerger` maps the bool 1:1; it survives the offline
config cache like every plain feature flag.
The agent does NOT configure `DODownloadMode` — the fingerprint GroupId only has an
effect when the tenant already deploys download mode Group (2) via Intune/GPO.
### The fingerprint
`DoGroupIdService.ComputeGroupId(ipv4Bytes, macBytes)` builds a 16-byte GUID:
```
{101,48,6} + {0,0,0} + {ip[1], 0, ip[2], ip[3]} + mac[0..5] → new Guid(byte[16])
```
Gateway discovery reuses the shared `NetworkInterfaceLocator` heuristic (Up, not
Loopback/Tunnel, non-0.0.0.0 gateway; IPv4 gateway required), the gateway MAC comes from
`ArpNativeMethods` (`iphlpapi!SendARP`). The byte layout — including the IP shuffle and
the zero padding — is **RealmJoin-compatible and must never change**: devices managed by
RealmJoin on the same network compute the identical GUID and peer with ours. The registry
value is written as `Guid.ToString()` (braceless lowercase), identical to RealmJoin's
`SetValue(name, Guid)` string form. Deliberate divergence from RealmJoin: **no
random-GUID fallback** — a random per-device ID would isolate the device in a one-member
group, which is worse than the OS default grouping, so no gateway/no MAC means skip.
### Ownership + conflict rules (never fight a foreign policy)
Target: `HKLM\SOFTWARE\Policies\Microsoft\Windows\DeliveryOptimization\DOGroupId`
(64-bit registry view — the Policies store is WOW64-redirected; key paths reused from
`DoThrottlePolicyReader`). Only this GPO-store value is written; the legacy pre-1607
`GroupId` value name is intentionally not.
Skip conditions (Info event with `skipReason`, nothing written):
| skipReason | Meaning |
|---|---|
| `no_ipv4_gateway` | No active interface with an IPv4 default gateway |
| `arp_failed` | Gateway MAC could not be resolved via SendARP |
| `group_id_source_policy` | `DOGroupIdSource` configured in GPO or MDM store (dynamic group source wins) |
| `mdm_policy_present` | `DOGroupId` managed via the Policy CSP store (Intune owns it) |
| `foreign_value_present` | Existing GPO value neither matches the computed GUID nor the one we last wrote |
Ownership is tracked through the `StartupEventGate` fingerprint for the
`do_group_id_auto_set` key: `written:<guid>` records the GUID we last wrote, so after a
gateway change the agent recognizes its own stale value and updates it — while a value
written by anyone else is never overwritten (`DecideAction`: empty→write, equal→no-op,
ours→update, else→skip; unparseable existing values count as foreign). Every write is
verified by readback.
### The event
One `do_group_id_auto_set` event per outcome change (Source `StartupEnvironmentProbes`,
gated on the fingerprint — unchanged reruns across reboots stay silent; errors never
latch, so each failing start keeps its retry trail). Severity: Info for
success/skip, Warning for errors. Data: `groupId`, `previousValue`, `gatewayIp`,
`gatewayMac`, `written`, `skipped`, `skipReason`, `success`, `error`.
## Examples
First boot on a network with gateway `192.168.178.1` / MAC `00-11-22-33-44-55`:
```
do_group_id_auto_set (Info): Delivery Optimization GroupId set to
00063065-0000-00a8-b201-001122334455 (gateway 192.168.178.1)
```
Tenant already deploys a DO GroupId via Intune:
```
do_group_id_auto_set (Info): Delivery Optimization GroupId not set: mdm_policy_present
```
## Citations
* Service (fingerprint, ownership, write+readback): `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Transport/DoGroupIdService.cs`
* ARP interop: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Interop/ArpNativeMethods.cs`
* Probe wiring + event + fingerprint dedup: `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Runtime/StartupEnvironmentProbes.cs`
* Backend serve + ConfigVersion: `../../src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs`
* DO policy stores (key paths, Registry64 rationale): `../../src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Periodic/DoThrottlePolicyReader.cs`
+1 -1
View File
@@ -44,7 +44,7 @@ tenant-config report.
`RemoteConfigMerger` projects the bool onto a runtime path (same pattern as
`EnableImeMatchLog`): `true` → `Constants.GatherRuleDebugLogPath` =
`%ProgramData%\AutopilotMonitor\Logs\gather_rules_debug.log`; `false` → null, unless a
`%ProgramData%\AutopilotMonitor\Logs\gather-rules-debug.log`; `false` → null, unless a
`--gather-debug-log <path>` CLI override supplied a non-default path. The field is not
security-sensitive and deliberately **survives the offline config cache**.
+3
View File
@@ -8,5 +8,8 @@ How the on-device agent works:
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
* [Power-State Watcher](power-state-watcher.md) - live AC↔battery transitions and 50/30/15% downward threshold crossings via a WMI push subscription (no polling): power_state_change semantics (latch/debounce/cap, never armed without a battery) and the single-condition analyze rules ANALYZE-DEV-009/-010 on top.
* [DO GroupId Auto-Set](do-group-id-auto-set.md) - the optional Delivery Optimization peering-group seed: deterministic GUID from default gateway IPv4 + MAC (RealmJoin-compatible), written to the DOGroupId policy value with readback, ownership-tracked so foreign Intune/GPO values are never overwritten; skip reasons and the do_group_id_auto_set event.
* [System Timeline Watcher](system-timeline-watcher.md) - clock steps (system_clock_changed: oldTime/newTime/timeDeltaMs/process, micro-slew floor) and completed sleep episodes (system_sleep_episode: sleep/hibernate/Modern Standby with authoritative enteredAt/exitedAt) from one System-channel EventLogWatcher with pre-agent backfill; feeds the web clock-set jump cause, the standby duration note, time-attribution SleepSpans and ANALYZE-DEV-012.
Reading order for newcomers: overview → decision engine → persistence.
+1 -1
View File
@@ -53,7 +53,7 @@ every line; the spool fsyncs only items flagged `RequiresImmediateFlush`.
| Event sequence | `State\event-sequence.json` | `{ LastAssignedSequence }` | atomic | none; survives WhiteGlove Part-2 resume |
| Spool | `Spool\spool.jsonl` | JSONL (`TelemetryItem`/line) | only `RequiresImmediateFlush` items | none — never truncated, cursor skips uploaded items |
| Upload cursor | `Spool\upload-cursor.json` | `{ LastUploadedItemId }` | atomic | idempotent, never regresses |
| Agent trace log | `Logs\agent_YYYYMMDD[_NNN].log` | text `[utc] [LEVEL] msg` | no | 50 MB → suffix rotation; consumed by `/diag-agent-log` |
| Agent trace log | `Logs\agent.log` (segments `agent-NNN.log`) | text `[utc] [LEVEL] msg` | no | 50 MB → suffix rotation; consumed by `/diag-agent-log` |
| Crash dump + record | `Crashes\<stem>.dmp` + `.json` | MiniDump + JSON | n/a (process dying) | 5 files / 7 days |
| Startup-event gate | `State\startup-event-state.json` | JSON dict | no (atomic) | cross-reboot dedup of one-shot probes |
| Bandwidth state | `State\bandwidth-state.json` | JSON samples/counters | no (atomic) | reboot-survivor for the DO bandwidth estimate |
+1 -1
View File
@@ -146,7 +146,7 @@ snapshots) are executed by the `EffectRunner` — the reducer itself is pure. De
| `\Agent\` | deployed binaries |
| `\State\` | decision persistence (SignalLog, Journal, Snapshot), `enrollment-complete.marker`, `final-status.json`, self-update state, `.quarantine/`, `.part1-<ts>/` |
| `\Spool\` | telemetry spool (offline queue) |
| `\Logs\` | agent trace log, `crash_*.log`, `ime_pattern_matches.log` |
| `\Logs\` | agent trace log (`agent.log`), `crash-*.log`, `ime-pattern-matches.log` |
| `\Config\` | `remote-config.json` cache |
| `\Agent-Update\`, `\Updates\` | self-update staging/download |
@@ -0,0 +1,200 @@
---
type: Concept
title: ESP Completion Starvation — the sits-d Cloud-PC complex
description: Why five Windows 365 Cloud PCs finished enrolling but never completed in the product — a SkipUser=true flow produces none of the signals every completion-gate evaluation site hangs off — plus the two adjacent defects fixed alongside (edge-only user-apps-settled synthesis, orphaned Hello-wizard restart replay) and what a Shell-Core replay deliberately does not do.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
tags:
- agent
- esp
- completion
- skip-user
- reboot
- recovery
timestamp: 2026-08-20T00:00:00+02:00
---
# ESP Completion Starvation — the sits-d Cloud-PC complex
Seven Windows 365 Cloud PCs, same tenant, first sign-in the same morning. Two sessions
succeeded within minutes; five sat at `Completion is waiting on: hello_resolution` from
second five onward and ran into the six-hour max-lifetime watchdog — verdict `Incomplete`
on devices whose users were working normally the whole time.
The first diagnosis blamed the mid-ESP reboot (DeviceGuard/DmaGuard, see
[MDM Reboot Coalescing](mdm-reboot-coalescing.md)): the agent is killed in
`Stage=EspAccountSetup` and, so the theory went, loses the Shell-Core ESP exit to the
downtime. The evidence disproved it. `8110e262` ran **six hours uninterrupted** after its
last reboot with the Shell-Core watcher armed — and Windows never wrote a 62407 in all
that time, because on these machines **the user ESP page never existed**.
# Schema
## The actual defect — a satisfied gate nobody evaluates
The five stuck machines carried `SkipUserStatusPage=True` in the enrollment `FirstSync`
key (read 10+ times across three agent processes, `True` on every read — not a stale
single read). Windows honours it: no user ESP page, hence
* no Shell-Core 62407 (`esp_exiting`) — the page that would exit does not exist,
* no `AccountSetupCategory` provisioning registry — nothing writes categories for a page
that never renders,
* desktop within seconds of sign-in.
The reducer is *built* for this flow: **arm B** of `ShouldTransitionToAwaitingHello`
(`SkipUserEsp == true`) declares the gate open, and the missing-prerequisites bookkeeping
already exempts the AccountSetup gate — which is why the sessions reported only
`hello_resolution` as missing. But the gate is a predicate, not a process. Something has
to evaluate it, and every evaluation site hangs off a signal a skip-user flow structurally
cannot produce:
| Evaluation site | Carrier signal | On SkipUser=true |
|---|---|---|
| `HandleEspExitingV1` | Shell-Core 62407 | never fires |
| `HandleEspPhaseChangedV1(FinalizingSetup)` | Shell-Core 62407 via coordinator | never fires |
| `HandleDesktopArrivedV1` fast-path | requires Hello policy **disabled** | Hello was enabled |
| `HandleImeUserSessionCompletedV1` arm-C attempt | requires the recorded final exit | unsatisfiable |
No knock, no promotion, no `HelloSafety` — and the `AdvisoryCompletion` backstop is armed
exclusively from an `EspExiting` signal, so it never armed either. The session idles to
the watchdog with an open gate.
## The fix — the observed skip is the exit evidence
`HandleImeUserSessionCompletedV1`'s completion attempt now accepts the observed skip in
place of the final exit (`skipUserExitEquivalent`): on a flow where no page exists, no
exit can ever say more than the `FirstSync` read already did. Everything else stays
mandatory — AccountSetup anchor, genuine (at-or-after-anchor) IME user-session completion,
DAD-validated real-user desktop — which makes this knock **stricter** than the existing
Shell-Core-carried arm-B promotion (pinned in `ClassicAwaitingHelloGuardTests`, which
requires no IME or desktop evidence at all). Hello semantics ride the existing rails:
enabled-or-unknown promotes to `AwaitingHello` with `HelloSafety` armed (on Cloud PCs
reached via RDP the Hello wizard never appears, so the synthetic timeout resolves the
wait), disabled completes directly with the synthetic `Skipped` outcome.
The knock deliberately lives on the IME user-session edge, not on desktop arrival:
* **Ordering-robust against Fix 10.** An `EspPhaseChanged(AccountSetup)` arriving on
`AwaitingHello` bounces the stage back and cancels `HelloSafety` (the premature-promotion
guard). The IME AccountSetup phase line lands *after* the desktop on this flow, so a
desktop-side knock would be undone two signals later. The IME phase line always precedes
the user-session-complete line in log order, so a bounced promotion is re-knocked by the
very next re-emission — including the restart re-parse.
* **Right semantics.** The IME user session completes when the required-app processing is
done; a desktop-side knock would mark the session `Succeeded` mid-installation.
* Device Preparation is excluded — WDP keeps its Hello+Desktop conjunction and the
`DevicePrepCompletion` backstop.
`ClassicSkipUserEspUserSessionCompletionTests` replays the real `8110e262` signal ordering
end to end (bootstrap `EspConfigDetected` → Hello enabled → desktop → AccountSetup anchor →
IME completion → HelloSafety → `enrollment_complete`), including the Fix-10
bounce-then-re-knock cycle, and pins the negative space (no skip observed / no desktop /
ghost pre-anchor IME completion / WDP).
Why five of seven identically administered Cloud PCs carried `SkipUserStatusPage=True` is
a tenant-side question (ESP profile assignment state at each machine's provisioning
moment); the value did not change during any observed session, and machine age does not
cleanly explain the split. The agent's job is to complete the flow Windows actually ran —
which it now does.
## Adjacent defect — the synthesis was edge-triggered
This one is real but belongs to the `SkipUser=false` flow (the two sessions that
succeeded). The user-apps-settled synthesis
(`MaybeSynthesizeAccountSetupCompleteFromSettledUserApps`, session `caa6cf50`) ran **only**
from `OnEspExited`. On a tenant with ~138 required apps the ESP page exits while apps are
still in flight: the single attempt misses and nothing ever re-checks, even when every
tracked app reaches a terminal state minutes later.
The ESP exit is an **edge** (happens once); settled user-ESP apps are a **level** (reached
later). `EspAndHelloTracker` records the edge in `_espExitObserved` and exposes
`ReevaluateUserAppsSettledSynthesis()`, chained into `ImeLogTracker.OnAppStateChanged` and
into the post-restore `onStateRestored` callback (a live exit observed before the IME
state restore completes would otherwise never get a second look). Gate conditions are
unchanged — no new completion path, only a second opportunity for the existing one. Only a
**live, confirmed post-AccountSetup** exit may be remembered
(`IsConfirmedPostAccountSetupExit`, stricter than the `IsIntermediateDeviceEspExit`
forward guard: unknown counts as not confirmed).
## Adjacent defect — the restart replay was never wired up
`ShellCoreTracker`'s backfill had existed since session `772fe502` with **no caller**; the
Shell-Core `EventLogWatcher` only delivers records written after `Start()`.
`EspAndHelloHost.Start()` now calls it as `BackfillRecentHelloWizardStart` — which is
exactly what it does and all it does:
| Record | Replayed | Why |
|---|---|---|
| 62404 Hello-wizard start | **yes** | Conservative: vetoes a premature "Hello is disabled" skip, can never complete a session by itself. The observation `772fe502` was about. |
| 62407 ESP exit | **no** | Cannot be placed in time — see below. |
| 62407 ESP failure | **no** | Re-injecting a historic failure as fresh can fail a session that recovered on retry (`ANALYZE-ESP-006`). |
A replayed exit is unusable, and every candidate ordering mechanism fails independently:
1. Windows writes the **identical** description `CommercialOOBE_ESPProgress_Page_Exiting`
for the intermediate DeviceSetup→AccountSetup transition and the final exit.
2. Every post-hoc ordering probe (AccountSetup registry, settled-apps) reads state as it
is **now**, not as it was at the event's time.
3. The reducer orders exits by **ingest ordinal**, not timestamp
(`IsPostAccountSetupFinalExit`) — a historic exit replayed today gets a fresher ordinal
than reality and reads as post-AccountSetup by construction.
4. `HandleEspExitingV1` passes `espFinalExitInFlight: true` for every arriving exit; with
restored state, arm C would open on a historic intermediate exit.
`ClassicEspExitingOnRestoredStateTests` pins that the reducer *does* open on any arriving
exit; `ShellCoreTrackerReplayScopeTests` pins that the replay never produces one. Skipped
62407 records are counted and reported once as an `agent_trace`
(`reason: replayed_62407_not_orderable`) — silently dropping evidence is how this class of
bug survives.
**Residual gap, named honestly:** a `SkipUser=false` session whose final exit falls
exactly into an agent downtime *and* whose AccountSetup category never resolves
(`1ec8f4c6` shape) still has no completion carrier after the restart. The normal case is
covered — `ProvisioningStatusTracker` reads the current registry at startup, so a
category that resolved during the downtime fires arm A. The residual shape is rare,
surfaced by `ANALYZE-ESP-005` via the `session_stalled` interim trigger, and deliberately
not patched with a replay that cannot be ordered.
### Sizing the replay window
A constant is provably wrong: the scheduled task has a `BootTrigger` only
(`Program.InstallMode.BuildScheduledTaskXml`) — after an `exception_crash` the agent does
not return until the next boot. `ResolveEspExitBackfillLookbackMinutes` reaches back to
the last moment the previous run is known to have been alive (max of `snapshot.json`
mtime and `LastBootUtc` for kill-type exits), floored at 5 minutes, capped at 360, off on
`first_run`, and immune to cross-reboot clock skew (a future timestamp is ignored).
# Examples
Tenant `sits-d.cloud`, 2026-08-19, seven Windows 365 Cloud PCs, first sign-ins the same
morning. The discriminator is a single registry value:
| Session | SkipUser | `esp_exiting` | AccountSetup registry | Reboots | Outcome |
|---|---|---|---|---|---|
| `08fc6bda` | False | yes | yes | 0 | Succeeded (10 min) |
| `3b9291aa` | False | yes | yes | 0 | Succeeded (2.1 h) |
| `8110e262` | True | — | — | 2 | Incomplete (6 h watchdog) |
| `a89aac2d` | True | — | — | 1 | Incomplete |
| `e7ba63c9` | True | — | — | 1 | Incomplete |
| `cb4a485a` | True | — | — | 2 | Incomplete |
| `3d6278fb` | True | — | — | 1 | Incomplete |
The reboot correlation was a red herring: `8110e262` ran six hours live after its final
reboot and would have seen every signal. `8110e262` and `a89aac2d` both reached
`138/138 completed, 0 failed` and a genuine IME user-session completion — with the fix,
all five complete a few minutes after their IME user session settles.
`08fc6bda` exercises the edge-level fix on the same day: ESP page exited normally while
`categorySucceeded` never confirmed — completed via the user-apps-settled synthesis.
`3b9291aa` shows the independent starvation visibility gap: 99 `app_install_starved`
events on a session recorded as `Succeeded` — surfaced from now on by `ANALYZE-APP-016`.
# Citations
* `src/Shared/AutopilotMonitor.DecisionCore/Engine/DecisionEngine.Classic.cs` — `HandleImeUserSessionCompletedV1`, `skipUserExitEquivalent`
* `src/Shared/AutopilotMonitor.DecisionCore/Engine/DecisionEngine.Shared.cs` — `ShouldTransitionToAwaitingHello` (arm B)
* `src/Agent/AutopilotMonitor.DecisionCore.Tests/ClassicSkipUserEspUserSessionCompletionTests.cs` — end-to-end replay of the 8110e262 ordering
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ShellCoreTracker.cs` — `BackfillRecentHelloWizardStart`, `ReplayBackfillRecords`
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/EspAndHelloTracker.cs` — `_espExitObserved`, `ReevaluateUserAppsSettledSynthesis`
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/DefaultComponentFactory.cs` — lookback sizing, `OnAppStateChanged` chaining
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) — the policy attribution for the reboots themselves
* [Decision Engine](decision-engine.md) — the AccountSetup gate and completion arms
+106
View File
@@ -0,0 +1,106 @@
---
type: Concept
title: Power-State Watcher — live AC/battery transitions and threshold events during enrollment
description: How the agent observes AC↔battery switches and downward battery-threshold crossings mid-enrollment via a WMI push subscription (no polling), the power_state_change event semantics (transition field, 50/30/15 ladder, latch/debounce/cap), and the two analyze rules on top (ANALYZE-DEV-009/-010).
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/PowerStateWatcherHost.cs
tags:
- agent
- power
- battery
- collector
- wmi
- analyze-rules
timestamp: 2026-08-21T00:00:00+02:00
---
# Power-State Watcher
The startup probe (`power_state_check`, one-shot from `StartupEnvironmentProbes` via
`PowerStateProbe`/`GetSystemPowerStatus`) only captures the power state at agent start. A
device unplugged mid-enrollment — or draining its battery across a multi-hour ESP — was
invisible. The watcher closes that gap **event-driven, with zero polling**.
# Mechanism
- `PowerStateWatcherHost` (`Orchestration/Hosts`) arms a WMI event subscription with a
two-step chain:
1. `SELECT * FROM Win32_PowerManagementEvent WHERE EventCode = 10` (PBT_APMPOWERSTATUSCHANGE:
true kernel push, fires on AC↔DC switches and battery-percentage steps).
2. **Fallback** `SELECT * FROM __InstanceModificationEvent WITHIN 30 WHERE TargetInstance
ISA 'Win32_Battery'` — required because on Windows 11 24H2/25H2 (observed on builds
26200 and 26220, session 161b838c) the `MS_Power_Management_Event_Provider` is still
registered in the repository but its activation fails with `WBEM_E_NOT_FOUND`: the
legacy push provider no longer loads on exactly the OS builds new devices enroll with.
The WITHIN sampling runs inside WinMgmt (one Win32_Battery snapshot every 30 s), not in
the agent; worst-case detection latency ≈ 35 s incl. debounce. `collector_degraded`
(`watcher_arm_failed`) is emitted only when BOTH attempts fail; the agent log's arm
line names the active mode.
Either way the arriving event's payload is ignored — each arrival re-probes via the
existing `PowerStateProbe.Probe()` (cheap P/Invoke).
- Snapshots are diffed by the pure `PowerStateTransitionTracker`
(`Monitoring/Enrollment/SystemSignals`), which decides what to emit. The host only owns
WMI, a 5 s trailing-edge debounce timer (dock-flap AC→battery→AC inside the window
collapses to a no-diff) and the emit path (`InformationalEventPost`, single rail).
- **No battery / probe error ⇒ WMI is never armed** — desktops and VMs pay nothing. WMI
arming runs on a background thread (WinMgmt can hang in OOBE; `ConsoleBypassWatcher`
precedent); an arm failure emits one `collector_degraded` (`watcher_arm_failed`).
- Registered as a plain always-on peripheral host after `NetworkChangeHost`, deliberately
NOT under `PeriodicCollectorLifecycleHost`: it is push-based (zero idle cost) and must be
alive precisely when the device idles on battery. Kill-switch:
`Collectors.EnablePowerStateWatcher` (default true, compile-time default like the other
collector toggles; ConfigVersion 38).
# Event: power_state_change
`data.transition` discriminates three shapes (payload always carries `onAcPower`,
`batteryPercent`, `isCharging`, `batteryLifeMinutes`; unknowns as `"unknown"`):
| transition | Severity | Immediate | Notes |
|---|---|---|---|
| `ac_to_battery` | Warning | yes | one per accepted edge |
| `battery_to_ac` | Info | no | recovery |
| `threshold_crossed` | Info (50) / Warning (30) / Error (15) | 30+15 only | extra field `thresholdPercent` |
Threshold semantics (tracker):
- Ladder 50/30/15, evaluated only while ON battery; **each level latches once per agent
lifetime** (charge-up + re-drain does not re-emit — no plug-cycle alarm spam).
- A multi-level jump emits only the LOWEST newly-crossed level (60%→10% ⇒ one `15` event).
- Entering battery power already below a level counts as crossing it; the arm-time baseline
does too — an enrollment STARTING at 12% battery emits its `thresholdPercent: 15` event
immediately (the analyze rule would otherwise never see sessions that began low).
- Lifetime emission cap 20 (storm backstop against a flapping dock / dying battery
controller); the host logs one warning when it engages.
Backend/agent hygiene: `power_state_change` is classified as non-activity on both sides —
`SignalActivityClassifier.NonActivityEventTypes` (agent: must not reset the
PeriodicCollectorLifecycleHost/StallProbeHost idle clocks) and
`EventIngestProcessor.IsPeriodicOrStallEvent` (backend: must not heal a Stalled session —
battery drain is not enrollment progress).
# Analyze Rules
- **ANALYZE-DEV-009** (high): battery crossed 15% while on battery — matches the
agent-stamped `thresholdPercent equals 15`.
- **ANALYZE-DEV-010** (warning): AC→battery switch — matches `transition equals
ac_to_battery`.
- Both: `evaluateOn: ["enrollment_end", "whiteglove_sealed", "on_event:power_state_change"]`
(live interim finding + terminal finalization), `markSessionAsFailedDefault: false`.
- Both are deliberately **single-required-condition**: `event_data` conditions scan events
independently (no same-instance join), so a two-condition rule (e.g. `transition ==
threshold_crossed AND batteryPercent < 15`) could false-positive across two different
power events. `thresholdPercent` exists only on threshold events and its value pins the
ladder level, so one condition fully identifies the instance.
- `{{token}}` caveat: `AddDataFieldsToEvidence` whitelists only generic identifiers
(appId/errorCode/…), so `{{batteryPercent}}` would render literally. DEV-009 interpolates
`{{thresholdPercent}}` (the matched condition's own dataField); the exact percentage lives
on the linked event in the timeline.
# Citations
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/PowerStateWatcherHost.cs`
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/PowerStateTransitionTracker.cs`
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Runtime/PowerStateProbe.cs` (probe, unchanged)
- `rules/analyze/ANALYZE-DEV-009.json`, `rules/analyze/ANALYZE-DEV-010.json`
- [analyze-rule-triggers](../rules/analyze-rule-triggers.md) (interim-trigger design; the WG battery rule was its motivating case)
+89
View File
@@ -0,0 +1,89 @@
---
type: concept
title: Registry App-State Second Pillar
description: Why the agent reads IME's authoritative per-app state straight from the registry (Win32Apps / EspTrackingWin32Apps / StatusServiceReports), how the snapshot-and-diff observer works, and how registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
resource: agent
tags: [ime, registry, win32apps, observability, reconciliation, pattern-drift, wdp]
timestamp: 2026-08-18
---
# Schema
## Why a second pillar
The 2026-08-17 audit of the decompiled IME (1.97.107 / 1.104.102) proved that IME log
wording is an unstable contract: Microsoft removed the old V1/V2 processing pipeline
(killing 9 patterns silently), reworded the post-remediation line, changed the
report-delta quoting between 1.97 and 1.104, and deleted `[DO TEL]` entirely. The
registry, by contrast, is IME's *internal product contract* — the ESP page and Intune
reporting consume it — and cannot drift casually.
Architecture decision (Oliver, 2026-08-17): **hybrid**. Logs stay the narrative
(downloads, retries, exit codes, scripts — the timeline). The registry is the stable
truth for terminal state. Divergence between the two IS the alarm that log patterns
drifted — turning silent data loss into a Warning event.
## Observed surfaces
All under `HKLM\SOFTWARE\Microsoft\IntuneManagementExtension` (64-bit view), verified
against decompiled IME source:
| Surface | Content |
|---|---|
| `Win32Apps\<userGuid>\<appGuid>_<rev>` | `EnforcementStateMessage` JSON (EnforcementState + ErrorCode), `ExitCode`, `Intent`. Device context = `Guid.Empty` user. Non-GUID subkeys (GRS, OperationalState, Reporting, ProvisioningProgress) are skipped. |
| `EspTrackingWin32Apps\<userGuid>\<appGuid>_<rev>` | Which apps IME registered for ESP tracking, `EspTrackingWin32AppPhase`. |
| `SideCarPolicies\StatusServiceReports\<userGuid>\<appId>` | The `AppInstallStatusReport` the ESP page renders (`Status`: 1000=Installed, 1001=Installing, 3000=Failed, 2000er=NotApplicable). |
## Mechanism
`ImeRegistryAppStateObserver` + `ImeRegistryAppStateHost` (always-on, no config gate —
precedent `EspPolicyProviderStallHost`):
- **One recursive `RegistryWatcher`** on the IME root covers all three surfaces;
changes are debounced (2 s trailing edge) into a tick. A 60-s periodic tick is both
the reconciliation-settle evaluator and the poll-only fallback when the watcher
fails to arm (`collector_degraded`, once).
- **Snapshot-and-diff**: `RegNotifyChangeKeyValue` gives coalesced key-scope edges
only, so every tick re-reads the surfaces and diffs against the previous snapshot.
- **Silent baseline**: the state present at agent start is captured without events.
Win32Apps keys survive re-enrollments — replaying them would be the registry twin
of the historic-IME-replay bug. Only apps that *change during the session* are
reported or judged.
- **`registry_app_state`** (Info): emitted per real field change
(enforcementState/errorCode/exitCode/statusServiceStatus/espTracked/espPhase),
capped at 200/session (cap announced once as Warning).
- **`app_state_reconciliation`** (Warning, once per app, immediate upload): after a
changed entry has been terminal (EnforcementState 1000er=success / 5000er=error,
or StatusService 1000er/3000) for ≥ 90 s, its outcome is compared against the
log-derived `AppPackageState` from `ImeLogHost.AllKnownPackageStates`. Divergence
reasons: `registry_error_log_installed`, `registry_success_log_error`,
`app_unknown_to_log_tracking` (only when the tracker is actively tracking apps —
an idle tracker is not judgeable).
- **Pure observability**: nothing feeds the DecisionEngine (same rule as
oobe-state observability).
## WDP reuse (Paket 3)
The same host archetype (RegistryWatcher subtree + debounce + snapshot-diff) is the
intended vehicle for WDP live progress: watch
`HKLM\SOFTWARE\Microsoft\Provisioning\AutopilotSettings\DevicePreparation\BootstrapperAgent`
(`ProgressUpdates` rows, `ResumeContext.ProvisioningResult` terminal marker) with the
RealmJoinWatcher appearance-watcher pattern for not-yet-existing keys, plus an
Event-142 channel watcher (`Microsoft-Autopilot-BootstrapperAgent/BootstrapperAgentServiceLogProvider`,
channel handling precedent in `DiagnosticsPackageService`). Win32Apps keys are written
on WDP devices too — the pillar works on both rails unchanged.
# Examples
Timeline evidence of a drifted pattern set: apps install fine (registry says
EnforcementState 1000) but the timeline shows no `app_install_completed` — each app
raises one `app_state_reconciliation` with `reason=app_unknown_to_log_tracking`.
Operator response: run `/ime-pattern-validate` against the session, fix patterns,
backend-deploy (no agent release needed for pattern fixes).
# Citations
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ImeRegistryAppStateObserver.cs`
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/ImeRegistryAppStateHost.cs`
- `tasks/ime-v1-code-audit-2026-08-17.md` (decompiled-source audit; registry surface catalog)
- Decompile archive: `okieselbach/ime-decompiles` (private), `C:\Code\GitHubRepos\ime-decompiles\<version>\decompiled`
+110
View File
@@ -0,0 +1,110 @@
---
type: Concept
title: System Timeline Watcher — clock steps and sleep episodes from the System event log
description: How the agent turns two enrollment blind spots into timeline ground truth via one EventLogWatcher on the System channel — system_clock_changed (Kernel-General 1, oldTime/newTime/timeDeltaMs/process) and system_sleep_episode (Power-Troubleshooter 1 classic sleep/hibernate, Kernel-Power 507 Modern Standby) — including backfill of pre-agent records, the payload-times-are-authoritative rule, SleepSpans in time attribution, the web clock-set jump cause, and ANALYZE-DEV-012.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/SystemTimelineTracker.cs
tags:
- agent
- eventlog
- time
- clock-skew
- standby
- collector
- time-attribution
- analyze-rules
timestamp: 2026-08-21T00:00:00+02:00
---
# System Timeline Watcher
Two things silently distort every session timeline and its wall-clock duration:
1. **The OS clock gets stepped** — classically w32time correcting a wrong BIOS clock early
in OOBE, but also tzutil/MDM actions. Before this watcher the jump was only *inferred*
(TimeJumpBadge, `clock_skew`/DEV-008); Windows records the ground truth.
2. **The device sleeps mid-enrollment** — the timeline shows an unexplained gap, the
duration looks inflated, and nothing says "the device was simply asleep for 56 minutes".
`SystemTimelineTracker` (hosted by `SystemTimelineWatcherHost`, config flag
`Collectors.EnableSystemTimelineWatcher`, ConfigVersion 39) subscribes to the **System**
channel with one provider-qualified XPath and forwards both facts as events.
# Sources and events
| Source (System channel) | Emitted event | Key payload |
|---|---|---|
| `Microsoft-Windows-Kernel-General` EventID 1 | `system_clock_changed` | `oldTime`, `newTime` (ISO-8601), signed `timeDeltaMs`, `reason`/`reasonText` (1=`application_set`, 2=`hardware_clock_sync`), `processName`/`processId` |
| `Microsoft-Windows-Power-Troubleshooter` EventID 1 | `system_sleep_episode` | `kind`=`sleep`/`hibernate` (EffectiveState 5 ⇒ hibernate), `enteredAt`=SleepTime, `exitedAt`=WakeTime, `durationSeconds`, `wakeSourceType`/`wakeSourceText` |
| `Microsoft-Windows-Kernel-Power` EventID 507 | `system_sleep_episode` | `kind`=`modern_standby`, `exitedAt`=TimeCreated, `enteredAt`=exit − DurationInUs, `durationSeconds` (scenario), `sleepDurationSeconds` (real sleep), `reason`, `onAcPower`, `batteryRemainingCapacityOnExit` |
**506 is deliberately not consumed.** 507 carries the full episode; pairing 506/507 would
need an orphan-tolerant state machine across agent restarts for zero information gain — an
episode without a 507 never completed and would not be emitted under any design.
# Thresholds (constants, not knobs)
- `MinClockDeltaMs = 2000` — w32time logs Kernel-General 1 for routine **1 ms micro-slews**
(verified live, several per hour); only genuine steps are emitted. Suppressed records are
still claimed by the dedup watermark.
- `ClockDeltaWarningMs = 5 min` — larger steps are Warning + immediate upload (can break
token/cert validation windows; matches the web TimeJumpBadge threshold).
- `MinSleepDurationSeconds = 60` — **Modern Standby only**: every screen-off scenario logs a
506/507 pair, including seconds-long ones with `SleepDurationInUs 0` (`SleepEntered=false`);
the single duration gate suppresses both. Classic Power-Troubleshooter episodes have no
floor — they only exist for genuinely completed S3/S4 transitions.
# Payload times are authoritative
The decisive clock correction happens exactly when the device clock is untrustworthy, and
the backend clamps implausible event timestamps (±168 h/24 h) to server time. Every semantic
instant therefore travels as an explicit ISO-8601 payload field (`oldTime`/`newTime`,
`enteredAt`/`exitedAt`); consumers must never read the event's own `Timestamp` for these.
The event `Timestamp` is set to the record's TimeCreated (WindowsUpdateTracker backfill
convention) so rows land where the wake/step happened.
# Backfill and dedup
Template: `WindowsUpdateTracker`. `LoadWatermark → arm watcher → backfill` with lookback
`SystemEventBackfillLookbackMinutes` (default 1440 = 24 h; 0 disables). The System .evtx
persists across OOBE reboots, so the pre-agent w32time correction and pre-agent standby
episodes are recovered with `backfilled=true`. Cross-restart dedup via RecordId watermark
(`system-timeline-watermark.json` in the state directory) plus an intra-run `HashSet` —
deliberately **not** a high-water mark, which would drop exactly the older, never-emitted
pre-agent records (the live watcher arms before the backfill scan).
# Non-activity classification (both sides)
Both event types are environment observation, not enrollment progress:
`SignalActivityClassifier.NonActivityEventTypes` (agent — must not reset idle/stall clocks)
and `EventIngestProcessor.IsPeriodicOrStallEvent` (backend — a device that slept through its
stall must stay Stalled).
# Consumers
- **Web timeline** (`EventTimeline.tsx`): badge blocks for both events; the TimeJumpBadge
gains the ground-truth cause `clock-set` — `classifyTimeJump` receives the session's
`system_clock_changed` deltas and matches a backward display step against a backward clock
set within max(60 s, 20 %) (`lib/timeProvenance.ts`).
- **Duration story** (`SessionInfoCard` via `useSessionDerivedData.standbySeconds`): the
wall-clock duration deliberately keeps the pause (the enrollment really took that long);
a `· 🌙 56m standby` note explains it. Episodes are dedup'd on `enteredAt`.
- **Time attribution** (`TimeAttributionCalculator.BuildSleepSpans`, AttributionVersion 3):
`SleepSpan`s mirror `RebootSpan`s — cross-cutting annotations clipped to the observation
windows (a WhiteGlove-pause episode contributes only its in-window flanks), never a slice
of the wall-clock partition. Persisted as `SleepSeconds`/`SleepSpansJson`; rendered as the
standby chip in `TimeAttributionLane`.
- **ANALYZE-DEV-012** (info, `trigger: single`, `on_event:system_sleep_episode`): fires once
per session for episodes ≥ 5 min. Deliberately the ONLY new rule: clock issues already
have DEV-008 (`clock_skew`) — a second clock rule would double-report the same action item
(user decision 2026-08-21); clock steps stay timeline ground truth with badges instead.
# Citations
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/SystemTimelineTracker.cs`
- `src/Agent/AutopilotMonitor.Agent.V2.Core/Orchestration/Hosts/SystemTimelineWatcherHost.cs`
- `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` (`BuildSleepSpans`)
- `src/Web/autopilot-monitor-web/lib/timeProvenance.ts` (`clock-set`, `findExplainingClockChange`)
- `rules/analyze/ANALYZE-DEV-012.json`
- Live payload evidence (2026-08-21, Win11 26220): Kernel-General 1 with `TimeDeltaInMs=1`
micro-slews; Kernel-Power 507 with `SleepDurationInUs=2324095807`; Power-Troubleshooter 1
with `SleepTime`/`WakeTime` across a hibernate.
+108
View File
@@ -0,0 +1,108 @@
---
type: concept
title: WDP Scenario Gates & Completion Backstop
description: How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation (WDP) sessions and how a WDP session resolves without any ESP signal.
resource: agent
tags: [wdp, device-preparation, decision-engine, collectors, completion]
timestamp: 2026-08-18
---
# Schema
Windows Device Preparation (Autopilot v2, "WDP") has **no deployment profile and no ESP**.
Before the afee7ae0 audit (2026-08-18) the agent ran the full Classic apparatus anyway:
misleading `autopilot_profile_missing` warnings, a ZTD event-log query + HTTP probe,
`esp_config_detected` "unknown/unknown" noise, an Autopilot-channel noise flood
(EventIDs 100/417/418/1005, 900+ occurrences per session), a subtree registry watcher
woken by every WDP progress write, and — worst — no completion path when Hello never
resolved (park until the 360-min watchdog).
## Detection: deterministic vs. fallback
`EnrollmentRegistryDetector` classifies v2 through three rules. Only the first is
**deterministic WDP**:
1. `AutopilotSettings\DevicePreparation\BootstrapperAgent` subkey with non-empty
`ExecutionContext` → the WDP policy orchestration tree, present before the agent
starts. Exposed as `EnrollmentRegistryDetector.IsDeterministicDevicePreparation()`
and the `DevicePreparationExecutionContextRule` constant.
2./3. `CloudAssignedDeviceRegistration == 2` / `CloudAssignedEspEnabled == 0` —
profile-derived legacy signals. They imply a REAL Autopilot profile (just ESP-less),
so every gate below deliberately ignores them: **gates require the deterministic
marker and fail toward Classic behavior.**
## Agent-side gates (all keyed to the deterministic marker)
* `DeviceInfoCollector.CollectAutopilotProfile`: no `autopilot_profile` event, no
`autopilot_profile_missing` warning, no ZTD event-log query, no ZTD HTTP probe.
`enrollment_type_detected` still fires. (WDP devices DO carry an
`AutopilotPolicyCache` with `ProfileAvailable=0` / error 807 `ZtdDeviceIsNotRegistered`
— Windows runs the ZTD download regardless — so the probe fired for real before.)
* `CollectEspConfigurationLocked`: `esp_config_detected` is emitted only when at least
one FirstSync value or ESP tracking list exists (generic null-gate, mirrors the
orchestrator bootstrap's signal gate) — fixes "unknown/unknown" for WDP AND plain
Entra joins.
* `DefaultComponentFactory`: `EspPolicyProviderStallHost` (60-s poll of the ESP-only
`EnrollmentStatusTracking` key) is not created on WDP.
* `EspAndHelloTracker.Start`: `ProvisioningStatusTracker` is not started on WDP — its
subtree watcher sits on `AutopilotSettings`, exactly where WDP's BootstrapperAgent
writes progress, so it woke on every write only to read three ESP category values
that never exist there.
* `ModernDeploymentTracker`: `DevicePreparationNoiseEventIds` {100, 417, 418, 1005}
are suppressed entirely on WDP (before `FormatDescription()`); Critical (level 1)
always passes; genuine WDP diagnostics (e.g. 408) keep flowing. The Autopilot error
backfill (807/809/815/908 replay) is skipped — 807 is the EXPECTED state on WDP.
The watcher itself stays on: WDP's provisioning stack logs into the same channels.
* `StallProbeCollector`: skips the ESP category registry scan on WDP and unions the
noise IDs into its harmless set; the channel scans remain (they carry genuine WDP
diagnostics during a stall).
* `EnrollmentTerminationHandler`: the `app_install_starved` terminal sweep is skipped
on WDP — post-desktop installs are the DESIGNED order there (only DPP-policy apps
run during provisioning); per-app data still travels via `app_tracking_summary` and
final-status.json.
## DecisionCore: WDP completion without ESP facts
* **Seeding**: `EnrollmentFactsObserved` carries `enrollmentTypeDeterministic`;
the deterministic marker seeds `DevicePreparation` at **High** confidence (same
registry-determinism standard as the self-deploying OobeConfig bits), the fallback
rules stay Medium. The three explicit downgrade guards from d0ab3eee remain as
defence in depth.
* **Arm D** (`ShouldTransitionToAwaitingHello`): on WDP the DAD-validated real-user
desktop is a legitimate promotion basis — arms A–C are unsatisfiable by construction
(no provisioning categories, no Shell-Core 62407 final exit). With a disabled Hello
policy the fast-path now completes at desktop arrival.
* **Prerequisites**: `completion_waiting` never lists
`account_setup_provisioning_complete` on WDP (unsatisfiable; mirrors the
`skipUserEsp` exemption).
* **Backstop**: `DeadlineNames.DevicePrepCompletion` (30 min, replay-safe base) is
armed by `HandleDesktopArrivedV1` when a WDP session parks desktop-first with Hello
pending. On fire it mirrors the HelloSafety timeout: synthetic
`HelloOutcome=Timeout` + completion through Finalizing. This closes the
"no Hello → park until watchdog" hole; a timer surviving into a terminal stage is
swallowed by the post-terminal dispatch guard.
* **Tripwire**: `DecisionStepProcessor.IsParkedWithoutDeadline` has a WDP branch
(dead-end zone = desktop arrived) so a regression in the backstop becomes visible
as `session_parked_without_deadline`.
# Examples
Session afee7ae0 (the audit trigger): Succeeded in 9:32 min, but its timeline carried
`autopilot_profile` (807), an `autopilot_profile_missing` WARN, `esp_config_detected`
unknown/unknown, 30 `modern_deployment_log` rollup events and 10 `app_install_starved`
entries — all structurally meaningless on WDP. After the gates, the same session shape
emits none of these; completion rode Hello-skipped + desktop as before.
Deliberately NOT changed (open follow-ups, see the WDP audit topic memory): the ESP
phase vocabulary (`esp_phase_changed`/stage names ride the Classic rails on WDP),
subscription to the `BootstrapperAgentServiceLogProvider` event channel, and
`registry_app_state` as the primary WDP app source.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Security/EnrollmentRegistryDetector.cs` — deterministic rule + helper
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/DeviceInfo/DeviceInfoCollector.NetworkAndSecurity.cs` — profile/ESP emission gates
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals/ModernDeploymentTracker.cs` — `DevicePreparationNoiseEventIds`
* `src/Shared/AutopilotMonitor.DecisionCore/Engine/DecisionEngine.DevicePreparation.cs` — backstop deadline
* `src/Shared/AutopilotMonitor.DecisionCore/State/EnrollmentScenarioProfileUpdater.cs` — High/Medium seeding
* Tests: `DevicePreparationEngineTests`, `ModernDeploymentTrackerDevicePrepTests`, `EnrollmentTerminationHandlerTests.StarvedApps_skipped_on_device_preparation`, `DeviceInfoCollectorEspRefreshTests.NoEspEvidenceAtAll_SuppressesTheEvent`
@@ -0,0 +1,82 @@
---
type: concept
title: Attempt-Scoped App Install Durations
description: AppInstallSummary.DurationSeconds measures the status-defining (last) install attempt via a LastAttemptStartedAt max-fold anchor — replacing the first-observation-to-last-terminal span that billed multi-pass IME processing gaps to trivial apps; time attribution counts the union of active attempt segments.
resource: src/Backend/AutopilotMonitor.Functions/Services/EventIngestProcessor.AppInstall.cs
tags: [backend, agent, ingest, app-installs, metrics, time-attribution, durations]
timestamp: 2026-08-21
---
# Problem
The IME processes the ESP app list in multiple passes: a device-ESP evaluation pass can
touch an app (often ending `Skipped`) long before the real install runs in a later pass.
`AppInstallSummary` is keyed by `{SessionId}_{AppName}` and folded from ALL of the app's
events, so `DurationSeconds = last terminal − first observation` spanned those passes.
Field case (session `44862cd3`, "Set TimeZone"): evaluation pass 09:16, real install
10:22:58 → 10:23:21 — reported 4002 s for a 23-second install. Fleet averages, the
Slowest-Apps ranking, p95s, and the version-regression radar all inherited the inflation;
the rule engine's `app_install_duration` (already attempt-scoped) contradicted the
dashboard for the same install.
# Schema
Three durable anchors on the row — each with a distinct, non-interchangeable job:
| Column | Fold | Role |
|---|---|---|
| `StartedAt` | min (earliest wins) | First observation. Server-side window/bucket filter of every AppInstallSummaries scan and the radar's version-ordering key. Never re-anchored. |
| `LastAttemptStartedAt` | **max** (in-batch and in `ReconcileAppInstallSummaryWithExisting`) | Anchor of `DurationSeconds`. Null = sentinel (row predates 2026-08, or no start observed) → duration falls back to the historical `StartedAt` span. Replay of an older batch can never regress it. |
| `InstallPassCount` | additive, guarded by `LastAttemptStartedAt` monotonicity (a batch whose newest start is not newer than the stored anchor adds 0) | Explains "finished long after first seen" (web ×N chip); 2+ is normal. |
`DurationSeconds = CompletedAt − LastAttemptStartedAt`, computed at the terminal event and
re-derived in one central reconcile site covering in-order and Q4 out-of-order batches.
A `LastAttemptStartedAt` newer than `CompletedAt` means a fresh pass is in flight — the
completed attempt's stored duration stands (the 0 value omits the column; Merge preserves).
`DownloadDurationSeconds` pairs the latest `app_download_started` with the NEXT
`app_install_started` (attempt-scoped, in-batch). The full span stays derivable as
`CompletedAt − StartedAt` — deliberately not a column.
Weaker-terminal guard (both in-batch and cross-batch): `app_install_completed` with
state `Skipped`/`Postponed` never overrides an `Installed`/`Error` terminal, nor its
`CompletedAt`/`DurationSeconds` — mirroring the long-pinned `app_install_skipped` rule.
# Consumers
* **Fleet metrics / apps dashboard / PDF** (`MetricsMath`, `AppsAnalyticsHelper`): values
drop to attempt scale; labels say "install time (final attempt)". The 6 h
`MaxPlausibleInstallDurationSeconds` gate remains (an attempt longer than the agent's
lifetime is still unobservable).
* **AppVersionRegressionRadar**: cutover gate in `MeasuredAppGroups` — only rows with
`LastAttemptStartedAt` enter the duration population, on BOTH comparison sides. A mixed
population would mask real regressions and falsely re-arm active episodes for up to the
35-day horizon. `MinMeasuredInstalls` keeps it silent until enough post-cutover rows
exist. Thresholds (2.0× / 300 s) were tuned on span data — re-evaluate ~4 weeks after
rollout.
* **Time attribution** (`TimeAttributionCalculator.BuildBlockingAppIntervals`):
sequence-ordered start→terminal pairing yields one segment per pass/attempt. Per-app
seconds = SUM of clamped segments (failed attempts and evaluation passes count — they
occupied the path); occupancy merges the segments, never the per-app hulls; hull
endpoints remain first-start→last-terminal for display and what-if. Deliberately
BROADER than the metrics duration: active time across all passes vs. final attempt.
* **Agent payloads** (`AppInstallTiming` / `ImeLogTrackerAdapter.UpdateAppTiming`): a
terminal→active transition re-arms BOTH endpoints, so event `durationSeconds` also
describes the attempt (needs the agent release carrying this change).
* **Rule engine** `app_install_duration`: was already attempt-scoped — now consistent.
# Rollout
No backfill (user decision, analog to the CMTrace legacy): pre-cutover rows keep span
values, so time-series charts step down visibly at the changeover (documented in the
public platform changelog). Sentinel-gated columns follow the standard chain:
`BuildAppInstallSummaryEntity` + `MapToAppInstallSummary` + `AppsDashboardProjection`
(the only projection whose consumers read the new columns).
# Citations
* `src/Backend/AutopilotMonitor.Functions/Services/EventIngestProcessor.AppInstall.cs` — fold + weaker-terminal guard + attempt pairing
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Metrics.cs` — reconcile max-fold, pass-count guard, central duration recompute
* `src/Backend/AutopilotMonitor.Functions/Helpers/AppVersionRegressionRadar.cs` — cutover gate
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — per-pass segments + occupancy
* `src/Agent/AutopilotMonitor.Agent.V2.Core/SignalAdapters/ImeLogTrackerAdapter.cs` — payload timing re-arm
* Tests: `AppInstallTerminalStateAndReconcileTests`, `EventTimestampValidationTests`, `AppVersionRegressionRadarTests`, `TimeAttributionCalculatorTests`
@@ -0,0 +1,84 @@
---
type: concept
title: Send-Time Frame Separation (X-Send-Time-Utc)
description: One device-clock send timestamp per upload batch splits ReceivedAt − OccurredUtc into pure spool delay and pure device-vs-server clock offset — turning clock diagnostics from median statistics into direct per-request measurement.
resource: src/Backend/AutopilotMonitor.Functions/Functions/Ingest/IngestTelemetryFunction.cs
tags: [backend, agent, ingest, timestamps, clock-skew, telemetry, provenance]
timestamp: 2026-08-21
---
# Problem
Events carry two time anchors: `OccurredUtc` (device clock at emission) and `ReceivedAt`
(server clock at ingest). Their difference conflates two physically distinct phenomena:
* **Spool delay** — the event waited offline or in retry backoff before upload. Expected,
harmless, minutes to hours.
* **Device-vs-server clock offset** — the actual diagnostic signal.
A uniform spool backlog (events emitted close together, uploaded after an outage) is
indistinguishable from a clock offset in that delta; a draining backlog fakes a clock jump
ramp. Every consumer had to filter this statistically (spread caps, plateau guards,
persistence tests) — and the first field clock-jump measurement failed exactly on this
ambiguity.
# Schema
The agent stamps one header per upload attempt (`BackendTelemetryUploader`):
```
X-Send-Time-Utc: <DateTime.UtcNow, ISO-8601 round-trip>
```
Per **attempt**, not per batch content: a retry after backoff is a new send moment. The
ingest function parses it (`ParseSendTimeHeader`: round-trip parse, UTC-normalized, values
before year 2000 rejected as garbage — deliberately **no upper clamp**, a future-dated
send time IS the clock-error measurement), and `StampServerFields` writes it as `SentAt`
onto every event of the request — server-stamped, request-level, never trusted per-event
from the wire. Stored as the nullable `SentAt` column on Events (absent for pre-header
agents), mapped back everywhere events are read, exposed as `sentAt` in the `fields=`
projection and verbatim on the raw endpoints.
The separation is then exact:
```
SentAt − OccurredUtc = pure spool delay (both device frame; offset cancels)
ReceivedAt − SentAt = network latency + clock offset (the only frame crossing)
```
Network latency is seconds; anything beyond is device clock error, per request, no
statistics required.
# Consumers
* **`clock_skew` analyze condition** (ANALYZE-DEV-008): batches whose events carry
`SentAt` measure the batch offset directly as `SentAt − ReceivedAt` — spool-immune by
construction, so the spread cap and the IME/clamped exclusions are unnecessary there
(no event timestamps are read at all; IME-only sessions become measurable). Batches
without `SentAt` keep the legacy per-event median + spread-cap path. Downstream logic
(stability check, plateau step detection, end-state persistence) is shared; evidence
carries `sentAtBatchCount`.
* **`CmTraceSkewTripwire` deliberately unchanged**: `median(Δ_IME) − median(Δ_other)`
cancels the device clock offset as common mode — send time adds nothing to that
differential measurement.
# Constraints
* `SentAt` comes from the same device wall clock as the event timestamps — a clock jump
between emission and send lands in the spool-delay term. That is the intended place for
it to become visible, not a defect.
* Signals and DecisionTransitions do not carry `SentAt` (no consumer); the header is
request-level, so extending them later is a storage-only change.
* Fleet coverage grows with agent rollout; all analyses must treat `SentAt == null`
(legacy rows, old agents) as "legacy path", never as zero.
# Citations
* `BackendTelemetryUploader.UploadBatchAsync` — header stamping per attempt.
* `IngestTelemetryFunction.ParseSendTimeHeader` + `IngestTelemetryFunctionTests` — parse
contract incl. the no-upper-clamp decision.
* `EventIngestProcessor.StampServerFields` + `IngestCriticalPathTests` — server-side
stamping, forged per-event values overridden.
* `RuleEngine.ConditionEvaluators.EvaluateClockSkewCondition` + `RuleEngineClockSkewTests`
— the two measurement paths (the backlog-alias test pair pins why SentAt exists).
* `tasks/todo.md` P14 (2026-08-21) — decision trail incl. why the tripwire stays as is.
+16 -1
View File
@@ -58,6 +58,21 @@ timestamp runs backward are dropped (`ClockSkewDropped`).
| Complete, `desktop_arrived` event | `desktop_handoff` |
| Failed | attribution ends — tail stays unattributed |
## Quality flags (AttributionVersion 2)
Duration-critical flags (`ClockSkewDropped`, `PartialObservation`,
`WhiteGloveAnchorsIncomplete`, `PriorEnrollmentResidue`) exclude a breakdown from fleet
segment statistics with a disclosed count; the blocking-set flags (`BlockingSetUnknown`,
`BlockingSetTruncated`) only limit per-app evidence and never gate fleet stats.
`PriorEnrollmentResidue` (v2) fires on `historic_ime_replay_detected`: the IME log on disk
predates the enrollment, so the device was re-enrolled without a wipe — pre-installed apps
complete as instant detections and pull the phase anchors ahead of the real ESP page
(session f475e697: AccountSetup anchor 6.5 min early, 457 s misattributed to
`identity_hello`). `registry_app_baseline` successes are deliberately NOT a trigger — they
are by-design normal for Windows Device Preparation (DPP Batch-1 apps install before the
agent exists) and would starve that class's aggregates.
## ESP-blocking apps (positive evidence only)
Per-app intervals use EVENT timestamps (first started/download event → LAST terminal
@@ -117,7 +132,7 @@ gap before the first part-2 phase declaration is unattributed, not guessed.
# Citations
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — calculator + semantics (AttributionVersion 1).
* `src/Backend/AutopilotMonitor.Functions/Helpers/TimeAttributionCalculator.cs` — calculator + semantics (AttributionVersion 2).
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.TimeAttribution.cs` — persistence.
* `src/Backend/AutopilotMonitor.Functions/Services/MaintenanceService.Aggregation.cs` — sweep + aggregation.
* `src/Backend/AutopilotMonitor.Functions.Tests/TimeAttributionCalculatorTests.cs` — golden fixtures pinning the invariant.
+11
View File
@@ -13,6 +13,7 @@ https://docs.autopilotmonitor.com (separate repository).
* [Architecture Guide](architecture.md) - High-level architecture overview and solution layout for contributors.
* [Version Contract](versioning.md) - One version shape for agent, backend and MCP: the ETag-CAS counter blob that mints build numbers, and the manifests published only after a deploy is verified live.
* [Customer Script Publishing](script-publishing.md) - How the customer-facing PowerShell scripts reach download.autopilotmonitor.com/agent/, why publishing is decoupled from agent releases, and the guards keeping the published copy, the version badge and the portal's version oracle in agreement.
* [URL Registry](url-registry.md) - Every well-known host lives in one registry file per component (Constants.cs / utils/config.ts / src/config.ts), enforced by guard tests; includes the agent-download migration to the download alias and its deploy-order constraint.
* [Drift Guards](drift-guards.md) - The 2026-08 fragility-audit follow-up contracts: SignalR message-name catalog (typed on both ends), the inverted-tick RowKey codec, tombstone table tagging, the shrink-only OkAsync baseline, per-section config PATCH with the section field map, and the web's scoped-endpoint builders + sidebar visibility matrix.
@@ -23,8 +24,15 @@ https://docs.autopilotmonitor.com (separate repository).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [Gather Rule Debug Log](agent/gather-rule-debug-log.md) - EnableGatherRuleDebugLog (ConfigVersion 36): the per-tenant local trace file that explains every gather-rule evaluation decision — scope skips, on_change suppression, empty collector results, logparser position/parse-mode outcomes — for rules that produce no timeline events.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
* [ESP Completion Starvation — the sits-d Cloud-PC complex](agent/post-reboot-completion-recovery.md) - Why a SkipUser=true flow (no user ESP page → no Shell-Core 62407, no AccountSetup registry) left the completion gate satisfied but never evaluated — five Cloud PCs idled six hours at "waiting on: hello_resolution" — and the fix: the observed skip stands in for the final-exit evidence in the IME user-session completion attempt. Plus the two adjacent defects fixed alongside: the edge-only user-apps-settled synthesis (now level-triggered) and the never-wired Shell-Core restart replay (now Hello-wizard-only, with the reasoning why a replayed 62407 must never reach the reducer).
* [Secure Boot CA-2023 Detection](agent/secureboot-ca2023-detection.md) - Why ANALYZE-SEC-001 v3 reads the UEFI db/KEK variables directly instead of trusting the SecureBoot\Servicing registry, and how the one-sided uefiCa2023FirmwareConfirmed marker plus a not_exists precondition give absence-tolerant suppression across old and new agents.
* [Continue-Anyway Observation Mode](agent/continue-anyway-observation.md) - EnableEspContinueAnywayObservation (ConfigVersion 37, operator-set): Device-phase ESP terminal failures on Continue-Anyway profiles get a bounded 60-min observation instead of an immediate hard fail; the DAD-validated real-user desktop + Hello gate detect the true end, the session completes Succeeded + EspSoftFailure (amber "with issues" badge), expiry un-defangs to the original esp_terminal_failure.
* [CMTrace Time Resolution](agent/cmtrace-time-resolution.md) - Converting bias-less CMTrace local times to UTC without trusting any process's timezone belief: per-line self-anchoring on the 15-minute offset grid for provably fresh lines, uniform marked reader-zone fallback for backlog, and the provenance vocabulary (bias / line-anchored / reader-zone-fallback) that keeps every event re-derivable.
* [Registry App-State Second Pillar](agent/registry-app-state-pillar.md) - IME's authoritative per-app state read straight from the registry (Win32Apps EnforcementStateMessage, EspTrackingWin32Apps, StatusServiceReports) via a snapshot-and-diff watcher; registry/log reconciliation doubles as the built-in IME log-pattern drift alarm.
* [WDP Scenario Gates & Completion Backstop](agent/wdp-scenario-gates.md) - How the agent gates Autopilot/ESP-only machinery off on Windows Device Preparation sessions (deterministic ExecutionContext marker, fail-toward-Classic), which noise it suppresses (ZTD probe, esp_config unknown, ModernDeployment 100/417/418/1005, starved sweep), and the WDP completion model: High-confidence seed, desktop-based arm D, and the 30-min device_prep_completion backstop.
* [Power-State Watcher](agent/power-state-watcher.md) - Live AC↔battery transitions and 50/30/15% downward battery-threshold crossings during enrollment via a WMI Win32_PowerManagementEvent push subscription (zero polling, never armed on battery-less devices): the power_state_change event (transition field, per-level lifetime latch, 5s debounce, emission cap) and the analyze rules ANALYZE-DEV-009 (battery <15%, high) / ANALYZE-DEV-010 (AC→battery, warning) with on_event interim triggers.
* [System Timeline Watcher](agent/system-timeline-watcher.md) - Ground truth for two timeline blind spots from one System-channel EventLogWatcher with pre-agent backfill: system_clock_changed (Kernel-General 1 — oldTime/newTime/signed timeDeltaMs/reason/setting process; 2 s micro-slew floor, ≥5 min Warning+immediate) and system_sleep_episode (Power-Troubleshooter 1 sleep/hibernate + Kernel-Power 507 Modern Standby; authoritative enteredAt/exitedAt payload instants, 60 s Modern-Standby floor). Consumed by the TimeJumpBadge clock-set cause, the SessionInfoCard standby note, time-attribution SleepSpans (AttributionVersion 3) and ANALYZE-DEV-012.
* [DO GroupId Auto-Set](agent/do-group-id-auto-set.md) - EnableDoGroupIdAutoSet (ConfigVersion 40): the agent seeds the Delivery Optimization DOGroupId policy with a deterministic network-fingerprint GUID (default gateway IPv4 + SendARP MAC, RealmJoin-compatible byte layout) so same-network devices form one DO peering group during ESP; ownership-tracked via the StartupEventGate fingerprint, foreign DOGroupId/DOGroupIdSource policies always win, no random-GUID fallback.
* [ESP Policy-Provider Stall Detection](agent/esp-policy-provider-stall.md) - The EnrollmentStatusTracking CSP's no-timeout Setup/Apps wait is keyed to the Intune-registered "Sidecar" provider by NAME: co-management's "ConfigMgr" parks the user ESP at "Apps (Identifying)" for days even with TrackingPoliciesCreated=1 (issue #106); the always-on two-arm esp_policy_provider_stalled tripwire (provider_incomplete | sidecar_provider_missing) and the SOFTWARE\Microsoft\Windows\Autopilot gather-rule allowlist entry that surface it.
# Rules
@@ -32,14 +40,17 @@ https://docs.autopilotmonitor.com (separate repository).
* [Gather Rule Phase Scoping, One-Shot Triggers & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch), one-shot collection at a phase's start/end (phase_change / phase_exit), and emitMode "on_change" result-dedup.
* [Gather Rule Guardrails](rules/gather-rule-guardrails.md) - Why the allowlists in rules/guardrails.json are enforced on the agent and nowhere else, which hard blocks survive unrestricted mode, and the rule that every collector must call a guard and emit security_warning on refusal.
* [Rule Authoring Surface](rules/rule-authoring-surface.md) - How customer AIs author rules against the real contract: the baked MCP resources (guide/schemas/guardrails via combine.js), the MCP-local validate_rule pre-flight, and the side-effect-free POST rules/analyze/dryrun whose trace twin shares the production evaluators (parity + side-effect-freedom pinned by tests).
* [Rule ID Namespace & Collision Guards](rules/rule-id-namespace.md) - Why rule identity is the human-readable ruleId (no GUIDs) with tenant isolation via Table Storage partitioning, the reserved built-in namespace (RuleIdPolicy), which collisions are possible (custom-vs-custom across tenants: allowed and harmless), and the guards keeping the global RuleStats aggregate catalog-only and the global partition inside the reserved namespace (combine.js, reseed, cross-tenant RuleState GC).
* [Analyze Rule Evaluation Triggers](rules/analyze-rule-triggers.md) - Opt-in `evaluateOn` interim triggers (`whiteglove_sealed`, `on_event:<type>`) so rules can fire before a session is terminal, with RuleResults update-semantics (interim → finalize/resolve, NotifiedAt notification dedupe) replacing the permanent dedupe freeze, KO/stat suppression on interim runs, the ingest-side cached trigger registry, and the authoring lints that keep interim rules monotonic.
# Backend
* [Business Timestamps](backend/business-timestamps.md) - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
* [Send-Time Frame Separation](backend/send-time-frame-separation.md) - One device-clock send timestamp per upload batch (X-Send-Time-Utc → SentAt on Events) splits ReceivedAt − OccurredUtc into pure spool delay and pure device-vs-server clock offset; the clock_skew condition measures SentAt batches directly (spool-immune), the CMTrace tripwire deliberately stays on its differential math.
* [OOBE Bootstrap go-Code Flow](backend/oobe-bootstrap-go-flow.md) - The bootstrap short-code lifecycle (portal-created session → `irm go.autopilotmonitor.com/CODE | iex` at Shift+F10 → GetBootstrapScriptFunction), the always-HTTP-200 text/plain contract, the two-line injection defense (GUID/URL/expiry validator + single-quoted PS literals, golden-pinned against the legacy TS template), and the per-IP rate-limit story behind Front Door.
* [Device Journeys & First-Time-Right](backend/device-journeys.md) - How a device's terminal enrollment attempts are grouped into journeys (trim+casefold device key with junk-serial exclusion, first-success end, 30-day gap, WhiteGlove = one attempt), the DeviceHistories chain with its tombstone-driven deleted-session pruning, the additive daily FTR aggregates, and the banner/fleet/MCP surfaces.
* [Enrollment Time Attribution](backend/time-attribution.md) - How a terminal session's authoritative wall clock (DurationSeconds, WhiteGlove pause excluded) is partitioned into named segments with an explicit unattributed remainder, ESP-blocking apps as positive evidence with "up to" what-if bounds, reboot spans from event gaps, and the breakdown/aggregate tables the sweep maintains.
* [Attempt-Scoped App Install Durations](backend/app-install-duration-attempt.md) - AppInstallSummary.DurationSeconds = status-defining (last) install attempt via the LastAttemptStartedAt max-fold anchor instead of the first-observation→last-terminal span that billed multi-pass IME gaps to trivial apps; InstallPassCount, the weaker-terminal guard, the radar's cutover gate, and time attribution's union-of-attempt-segments occupancy.
* [Rule Regression Radar](backend/rule-regression-radar.md) - How analyze-rule hit-rate regressions are detected (7-day window ≥2× the 28-day baseline with disjoint Wilson intervals), which suppressions keep the radar quiet (grace period, edit-in-window, kill switch), how alert episodes live in the ruleregression tracker keyspace with re-arm semantics, and the on-fire dimension correlation with its "correlated, not necessarily causal" wording contract.
* [App Version Duration Regression Radar](backend/app-version-regression-radar.md) - How app versions whose median install duration regressed vs. the previous version are detected (first-seen-ordered versions, median lift ≥2× plus a 5-minute absolute floor over measured installs only), how alert episodes live in the appversionregression tracker keyspace with drain/median re-arm semantics, and the bell/ops-event/analytics surfaces.
* [Tenant Activation & the Once-Only Welcome Mail](backend/tenant-activation-welcome-mail.md) - How a signup becomes an activated tenant (manual approve or the auto-approve queue), and the write-then-read + conditional-marker arbitration that sends the welcome mail exactly once even though activation and the user-typed notification address race each other.
+42
View File
@@ -1,5 +1,47 @@
# Log
## 2026-08-21
* **New**: `agent/do-group-id-auto-set.md` — optional DO peering-group seed (`EnableDoGroupIdAutoSet`, default off, ConfigVersion 40): `DoGroupIdService` derives a deterministic GUID from the default gateway's IPv4 + SendARP MAC in RealmJoin's exact byte layout (cross-product peering) and writes `DOGroupId` in the 64-bit GPO Policies store with readback verification. Conservative by design: no random-GUID fallback (skip on missing gateway/MAC), `DOGroupIdSource`/MDM-managed values/foreign GPO values always win (skip reasons in the `do_group_id_auto_set` event), own stale values are recognized via the `written:<guid>` StartupEventGate fingerprint and updated after a gateway change. Gateway heuristic extracted into the shared `NetworkInterfaceLocator` (dedup of NetworkChangeDetector + DeviceInfoCollector copies).
* **New**: `agent/system-timeline-watcher.md` — clock-step and sleep-episode ground truth from the System event log: one `EventLogWatcher` (Kernel-General 1, Power-Troubleshooter 1, Kernel-Power 507; 506 deliberately unconsumed) emits `system_clock_changed` (oldTime/newTime/signed timeDeltaMs/reason/process; 2 s micro-slew floor, ≥5 min = Warning+immediate) and `system_sleep_episode` (kind sleep/hibernate/modern_standby, authoritative `enteredAt`/`exitedAt` payload instants, 60 s Modern-Standby floor). WindowsUpdateTracker-style backfill (24 h default, `system-timeline-watermark.json`) recovers pre-agent records; ConfigVersion 39 (`Collectors.EnableSystemTimelineWatcher`). Consumers: TimeJumpBadge cause `clock-set` (ground truth beats inference), SessionInfoCard standby note next to the unchanged wall-clock duration, `SleepSpan`s in time attribution (AttributionVersion 3, RebootSpan clone), and ANALYZE-DEV-012 (info, single, ≥5 min episode; deliberately the only new rule — clock issues stay with DEV-008).
* **New**: `agent/power-state-watcher.md` — live AC/battery observation during enrollment: `PowerStateWatcherHost` subscribes to the WMI push `Win32_PowerManagementEvent EventCode 10` (no polling; re-probes `GetSystemPowerStatus` per push, 5 s trailing-edge debounce, never armed on battery-less devices, `Collectors.EnablePowerStateWatcher` kill-switch, ConfigVersion 38). The pure `PowerStateTransitionTracker` emits `power_state_change` with `transition` = ac_to_battery (Warning, immediate) / battery_to_ac (Info) / threshold_crossed at 50/30/15% (each level latched once per lifetime, lowest-crossed-only on jumps, baseline-below-level covered, emission cap 20). Classified non-activity on both sides (SignalActivityClassifier + IsPeriodicOrStallEvent). New analyze rules ANALYZE-DEV-009 (thresholdPercent=15, high) and ANALYZE-DEV-010 (transition=ac_to_battery, warning), both with `on_event:power_state_change` interim triggers and deliberately single-required-condition (event_data conditions have no same-instance join).
* **Update**: `rules/rule-id-namespace.md` — the export→paste round-trip is no longer lossy: the JSON editors now map rule-shaped exports into the form (`gatherRuleToForm` unflattens the nested gather `parameters` object, analyze `jsonToForm` maps `evaluateOn` onto the eval checkboxes; `tags` and the effective `markSessionAsFailed` carry into the created rule). Both editors still accept the serialized-form shape; edit-mode pastes merge, with an explicit `parameters`/`evaluateOn` winning over stale flat fields.
* **New**: `rules/rule-id-namespace.md` — rule identity model: human-readable `ruleId` strings (no GUIDs), tenant isolation via Table Storage partitioning (`global` = built-in/community, tenantId = custom), the reserved built-in namespace from `RuleIdPolicy`, and the four collision guards: catalog-only global `RuleStats` aggregate (same-ID custom rules of unrelated tenants no longer sum into one `global_{ruleId}` row), combine.js duplicate/namespace fail, GitHub-reseed rejection of non-reserved IDs into the global partition, and the namespace gate on the cross-tenant RuleState GC.
* **New**: `backend/app-install-duration-attempt.md` — attempt-scoped app install durations: `DurationSeconds` = status-defining (last) attempt anchored on the new max-folded `LastAttemptStartedAt` (StartedAt stays the earliest-wins window/version-ordering key; legacy rows without the anchor keep the span fallback); `InstallPassCount` with replay-guarded additive fold; weaker-terminal guard (completed-with-Skipped never overrides Installed/Error, in-batch and cross-batch); attempt-scoped `DownloadDurationSeconds` pairing; the version-regression radar's both-sides cutover gate; time attribution counts the union of per-pass active segments (occupancy merges segments, hulls stay for display/what-if); agent payload timing re-arms both endpoints on terminal→active. No backfill — pre-cutover rows keep span semantics.
* **Update**: `agent/decision-engine.md` — activity-based RealmJoin timeout extension (report 55e6afd61c9d: a large RJ package catalog outlived the 60-min window mid-install — the timer arms at RJ-agent-MSI detection, so the first deployment had effectively ~44 min). When the `RealmJoinTimeout` deadline fires with `LastDeploymentPhase` 100/101 and deployment activity (phase change or package observation, persisted in `RealmJoinFacts.LastActivityUtc`; the detection-time phase capture deliberately excluded) inside the last 60 min, the deadline re-arms to `lastActivity + 60 min` (Info event `realmjoin_timeout_extended`), clamped at detection + 4 h. `realmjoin_timeout` now carries `reason` = `hard_timeout` / `inactivity` / `absolute_cap`; a fire of a superseded deadline incarnation dead-ends. Idle detection and phase-200/210 sessions keep the original 60-min behavior.
* **New**: `backend/send-time-frame-separation.md` — P14: the agent stamps one device-clock send timestamp per upload attempt (`X-Send-Time-Utc`), the ingest function server-stamps it as `SentAt` on every event of the request. `SentAt − OccurredUtc` is pure spool delay, `ReceivedAt − SentAt` is network + device-vs-server clock offset — the two were indistinguishable before and had to be separated statistically. The `clock_skew` condition measures SentAt-carrying batches directly (spool-immune, no spread cap, IME-only sessions become measurable; evidence carries `sentAtBatchCount`), legacy batches keep the median path; the CMTrace tripwire deliberately stays on its differential math (common-mode clock offset cancels there).
## 2026-08-20
* **Update**: `agent/cmtrace-time-resolution.md` — new "Backend regression tripwire" section: the P6 split. Operator side: `CmTraceSkewTripwire` piggybacks on the terminal counter reconcile's partition scan (`Δ = ReceivedAt − OccurredUtc` per source, 15-min grid + 2-min residual, ≥20 samples / ≥3 distinct batches per side, bias-dominated sessions suppressed, `measuredWriterOffsetMinutes` never consulted) and emits the `CmTraceTimeSkewRegression` ops event — goal state: never fires; kill switch `CmTraceSkewTripwireDisabled`. Customer side: new `clock_skew` analyze condition source (`skewMetric: clock_jump | sustained_offset`, threshold in seconds, batch-median frames with spool-spread filter, plateau step detection, end-state persistence, IME-derived events excluded) behind the new built-in `ANALYZE-DEV-008` (300 s = the Kerberos/Entra 5-minute skew boundary) — the during-enrollment complement of ANALYZE-DEV-007's one-shot NTP check. P5 (agent-side observability event) is subsumed by the tripwire.
* **New**: `agent/cmtrace-time-resolution.md` — the era-aware answer to the CMTrace timezone skew: a bias-less line read in a provably fresh pass anchors its own UTC offset against the agent clock (15-minute grid, 2-minute residual), so interleaved writer eras in one file resolve line by line; everything not provably fresh keeps the uniform reader-zone fallback and says so via `sourceOffsetOrigin`. Freshness is a per-file `LastCheckedUtc` stamped on every poll look (empty first sights included) with a 30-second window, never restored across restarts. Replaces the reverted per-file offset application (04b1a7c6); the per-file measurement stays observational, now with the anchor timestamp logged as a tripwire for the still-unexplained e9753578 label/anchor crossing.
* **Update**: `agent/post-reboot-completion-recovery.md` — retitled to "ESP Completion Starvation — the sits-d Cloud-PC complex" and reframed on the evidence: the reboot was a red herring. The five stuck Cloud PCs carried `SkipUserStatusPage=True` (read 10+ times across three agent processes, `True` every time) and Windows honoured it — no user ESP page, hence no Shell-Core 62407 and no AccountSetup provisioning registry, verified over a six-hour uninterrupted post-reboot run with the watcher armed. Arm B of `ShouldTransitionToAwaitingHello` was open from the bootstrap onward, but every evaluation site hangs off a signal a skip-user flow cannot produce, and the `AdvisoryCompletion` backstop arms exclusively from an `EspExiting` signal — an open gate nobody knocks on. Fix: `HandleImeUserSessionCompletedV1` accepts the observed skip in place of the final-exit evidence (`skipUserExitEquivalent`; anchor + genuine IME completion + real-user desktop stay mandatory, WDP excluded). The knock deliberately lives on the IME edge, not desktop arrival — ordering-robust against Fix 10's AccountSetup bounce-back and semantically "user-phase work finished" rather than mid-install. `ClassicSkipUserEspUserSessionCompletionTests` replays the real 8110e262 ordering end to end. Docs and comments no longer claim an ESP-exit recovery that does not exist (stray duplicate summary in `ShellCoreTracker`, `ImeLogHost` restore-callback comment); the residual `SkipUser=false` exit-in-downtime gap is named honestly and surfaced by `ANALYZE-ESP-005` via `session_stalled` instead of being patched with an unorderable replay.
## 2026-08-19
* **New**: `agent/post-reboot-completion-recovery.md` — a forced mid-ESP reboot (DeviceGuard/DmaGuard) used to cost the session its AccountSetup completion permanently. Two defects: `ShellCoreTracker.BackfillRecentEspExitEvents()` existed but had no caller, so every agent restart dropped an ESP exit (62407) written while it was down; and the user-apps-settled synthesis was edge-triggered on `EspExited` only, so apps settling afterwards never re-opened the gate. `EspAndHelloHost.Start()` now backfills with a lookback sized to the downtime (`previousExit=reboot_kill` → since `LastBootUtc`, clamped to 360 min), and `ReevaluateUserAppsSettledSynthesis()` is chained into `ImeLogTracker.OnAppStateChanged`. No new completion path — the gate conditions are unchanged. Evidence: sits-d Cloud PCs, five `Incomplete` sessions on healthy devices, two of them at `138/138 apps, 0 failed`. Hardened over two review rounds: the remembered edge must be a **live** exit that is **confirmed** post-AccountSetup (a replay carries no evidence of its own position — Shell-Core writes the same `CommercialOOBE_ESPProgress_Page_Exiting` for the intermediate and the final exit, and the registry check reads *now*, not the event's time); the replay hands over the **newest** exit rather than the oldest; the replay is narrowed to the Hello-wizard start (62404) alone — a conservative fact that can only make the agent wait longer. A replayed ESP exit is never re-raised at all: Windows writes the identical description for the intermediate and the final exit, the reducer orders exits by ingest ordinal (fresher than reality for a replay) and `HandleEspExitingV1` passes `espFinalExitInFlight: true` for every arriving exit, so arm C of `ShouldTransitionToAwaitingHello` would open on a historic handoff once the restored state carries AccountSetupEntered + IME user session + desktop. A replayed ESP failure is likewise dropped (it could fail a session that recovered). Skipped records are reported as an `agent_trace`. The window is **measured** from the previous run's last known liveness (snapshot mtime, widened by `LastBootUtc`) instead of a constant — the scheduled task has a BootTrigger only and no restart-on-failure, so a crashed agent returns hours later and five minutes would lose the exit every time. On `first_run` the replay stays off, leaving the happy path untouched.
* **Update**: `rules/analyze-rule-triggers.md` — `on_event:session_stalled` documented as an established interim trigger for the never-terminal stuck session (one emission per session, no throttle needed). `ANALYZE-ESP-005` carries it: both its conditions are monotonic and it has no terminal precondition, and it fires at confidence 95 on the sits-d stalled stream. `ANALYZE-APP-015` deliberately does NOT — it is the worked example of the terminal-precondition trap (lint 1) — and `ANALYZE-ESP-001` does not either, because its `phase_duration` gate on a still-open phase is unverified mid-run. New `ANALYZE-APP-016` ("Required Apps Never Started — ESP Apps Gate Starved") turns the existing `app_install_starved` events into a customer-facing finding and, unlike `ANALYZE-APP-015`, fires even when the session succeeded — session `3b9291aa` was recorded `Succeeded` with 99 required apps that never started installing.
* **New**: `script-publishing.md` — the customer script publishing chain. `.github/scripts/Publish-BootstrapScripts.ps1` becomes the single owner (dev render, version-bump guard, uploads, version oracles, alias verification), called both by the new `publish-scripts.yml` (push to `main` on `scripts/Bootstrap/**` and `scripts/CustomerSetup/**`) and by the `publish_as_stable` branch of `build-agent.yml`. Script publishing no longer waits for an agent release. Blobs are published in a deterministic CRLF/UTF-8-no-BOM form so byte comparisons do not depend on the checkout; the Pester + ASCII gates move into the reusable `bootstrap-script-gates.yml` and block the upload; `AdminConfiguration.LatestBootstrapV2ScriptVersion` now has exactly one writer. The version-bump guard compares a code fingerprint (PowerShell token stream without comments, newlines and line continuations), so only behaviour changes require a `$ScriptVersion` bump — comment and formatting changes publish with a warning.
## 2026-08-18
* **New**: `agent/wdp-scenario-gates.md` — WDP scenario gates (afee7ae0 audit): deterministic ExecutionContext marker as the only gate key (fail-toward-Classic), suppressed Autopilot/ESP-only emissions and hosts (autopilot_profile*/ZTD probe, esp_config unknown null-gate, EspPolicyProviderStallHost, ProvisioningStatusTracker, ModernDeployment noise IDs 100/417/418/1005 + error-backfill skip, starved sweep), and the WDP completion model (High-confidence v2 seed via `enrollmentTypeDeterministic`, desktop-based arm D, 30-min `device_prep_completion` backstop, parked-tripwire WDP branch).
* **Update**: `backend/time-attribution.md` — AttributionVersion 1 → 2: new duration-critical quality flag `PriorEnrollmentResidue` (fires on `historic_ime_replay_detected`; a re-enrollment without wipe pulls the phase anchors ahead of the real ESP page, so the segments are excluded from fleet stats; `registry_app_baseline` successes deliberately not a trigger — by-design normal for WDP). Added the quality-flags section.
* **Update**: `agent/logs-and-persistence.md`, `agent/overview.md`, `agent/gather-rule-debug-log.md` — log files renamed to unified kebab-case: `agent.log` (date rotation dropped; size segments `agent-NNN.log`), `bootstrap-script.log` (was `bootstrap_agent.log`), `bootstrap-msi.log` (was `msi-bootstrap.log`), `crash-*.log`, `gather-rules-debug.log`, `ime-pattern-matches.log`. Old names still appear in archives from pre-rename agents.
* **New**: `agent/registry-app-state-pillar.md` — the registry second pillar for IME app state (audit follow-up): snapshot-and-diff watcher on `HKLM\...\IntuneManagementExtension` (Win32Apps / EspTrackingWin32Apps / StatusServiceReports), `registry_app_state` + `app_state_reconciliation` events, silent startup baseline, WDP reuse notes.
## 2026-08-17
* **Update**: `mcp/tool-telemetry.md` — the `tool_call` stderr line now carries `errorMessage` (capped at 300 chars; thrown message or the leading `toolError` text, absent on success), so error drilldowns name the failure class instead of guessing from args. Same change: `get_rule_stats` omits the per-rule daily `trend` rows by default (they drove 50% of calls over the response cap at platform scope); `includeTrends=true` opts back in, the response carries a `trendsNote` marker when stripped.
+5 -1
View File
@@ -98,9 +98,13 @@ Interim evaluation changes what a rule's conditions mean mid-flight. Three new l
# Application to the Motivating Cases
- WG battery rule (sits-d): `evaluateOn: ["enrollment_end", "whiteglove_sealed"]` — technician gets the channel alert at the bench; Part-2 terminal pass finalizes or resolves the finding.
- WG battery rule (sits-d): shipped 2026-08-21 as ANALYZE-DEV-009 (battery below 15% while on battery) and ANALYZE-DEV-010 (AC→battery switch), both `evaluateOn: ["enrollment_end", "whiteglove_sealed", "on_event:power_state_change"]` on the live power watcher's `power_state_change` events (see `../agent/power-state-watcher.md`) — technician gets the channel alert at the bench; the terminal pass finalizes the finding. Both are deliberately single-required-condition rules: `event_data` conditions scan events independently (no same-instance join), so DEV-009 matches the agent-stamped `thresholdPercent` and DEV-010 matches `transition`.
- Hybrid-affinity stuck (esa): ANALYZE-ID-004 gets `evaluateOn: ["enrollment_end", "on_event:hybrid_login_pending"]` — the second overdue-login event (threshold 65) produces a live finding on the still-InProgress session. ANALYZE-APP-015 stays `enrollment_end`-only until its conditions are hardened (lint 1).
- Stuck after a forced mid-ESP reboot (sits-d Cloud PCs, 2026-08-19): `on_event:session_stalled` is the established trigger for the never-terminal stuck session. The agent emits `session_stalled` exactly once per session (60-minute stall probe), so the trigger is rare by construction and needs no throttle. Five Cloud PCs sat at `InProgress` with **zero findings** for hours — a mid-ESP reboot had killed the agent in `Stage=EspAccountSetup` — before the max-lifetime watchdog eventually terminalised them as `Incomplete`. ANALYZE-ESP-005 carries the trigger and fires on that stream at confidence 95 (verified against session `8110e262`): both its conditions (`mdm_policy_reboot_required.firstRebootUri` exists, `system_reboot_detected` count ≥ 1) are **monotonic** and it has no terminal preconditions, so lint 1 does not apply. ANALYZE-APP-016 is interim-safe for the same reason (`app_install_starved` count ≥ 1 only ever grows).
Two rules were considered and deliberately **not** given the trigger: ANALYZE-APP-015 is the worked example of lint 1 above and stays `enrollment_end`-only; ANALYZE-ESP-001 gates on `phase_duration` of a phase that is still open mid-run, and that evaluation is not pinned by a test — an unverified trigger is not worth a possible false positive on every slow DeviceSetup.
# Product Decisions (settled 2026-08-14)
1. Notify-on-resolve (a "finding healed" message)? **No** (v1) — resolution is visible in UI/MCP.
+47
View File
@@ -0,0 +1,47 @@
---
type: concept
title: Rule ID Namespace & Collision Guards
description: How gather/analyze rule identity is scoped (tenant partition vs global partition), why there are no GUIDs, which collisions are possible, and the guards that keep the reserved built-in namespace and the global RuleStats aggregate clean.
resource: backend, rules
tags: [rules, ruleid, namespace, collision, rulestats, tenant-isolation]
timestamp: 2026-08-21
---
# Schema
Rule identity is the human-readable `ruleId` string — there are no GUIDs, deliberately: the ID appears in logs, mails, notifications, RuleResults, the agent config payload, and docs, and tenant isolation is provided by Table Storage partitioning instead.
| Store | PartitionKey | RowKey | Content |
|---|---|---|---|
| `GatherRules` / `AnalyzeRules` | `"global"` | ruleId | Built-in + community rules (shared, all tenants) |
| `GatherRules` / `AnalyzeRules` | tenantId | ruleId | Custom rules (visible only to that tenant) |
| `RuleStates` | tenantId | ruleId | Per-tenant enable/notify overrides — built-in/community rules ONLY (custom rules fold state into their own row) |
| `RuleStats` | date | `{tenantId}_{ruleId}` and `global_{ruleId}` | Daily fire/eval telemetry; the global row is catalog-only (see guards) |
Namespace split (`RuleIdPolicy.cs`, shared model):
- **Reserved built-in namespace**: `(ANALYZE|GATHER)-<CATEGORY>-<NUMBER>` with `CATEGORY != CUSTOM`, case-insensitive, including currently unused numbers (gaps = retired rules that may return).
- **Tenant namespace**: everything else — `ANALYZE-CUSTOM-NNN` (portal suggestion), `-CUSTOM` suffixes, org prefixes (`CONTOSO-WIFI-001`), freeform.
# Collision model
- **Custom vs built-in**: impossible via API — `CreateRuleAsync`/`UpdateRuleAsync` reject reserved IDs (409) and check existence against `global` ∪ own tenant partition. Legacy debris is neutralized at merge time: global wins, tenant copy dropped with a warning (pinned by `RuleMergeCollisionGuardTests`, motivated by the 5ca2b350 duplicate-key incident).
- **Custom vs custom across tenants**: allowed and harmless — separate partitions, separate evaluation, separate RuleResults/tenant stats. Two tenants both using `ANALYZE-CUSTOM-101` never interact.
- **JSON copy between tenants** (export → paste into create editor): goes through the same create path, so an ID already taken in the receiving tenant surfaces as 409 — the user renames. There is no bulk import endpoint. The JSON editors accept both shapes — a rule export and the serialized form — via `gatherRuleToForm` / `jsonToForm` (web `types.ts` of each rules page): nested gather `parameters` are unflattened, analyze `evaluateOn` maps onto the eval checkboxes, and `tags` plus the effective `markSessionAsFailed` survive the import.
# Guards (all enforced, 2026-08-21)
1. **Global RuleStats aggregate is catalog-only.** The `global_{ruleId}` row would otherwise sum same-ID custom rules of unrelated tenants (title/severity last-writer-wins). Analyze gates on `rule.IsBuiltIn || rule.IsCommunity` (`AnalyzeOnEnrollmentEndHandler`), gather gates on `RuleIdPolicy.IsReservedBuiltInId` (`EventIngestProcessor.RuleStats`) because agent events carry no flag — valid because custom rules can never occupy the reserved pattern.
2. **Catalog build fails on namespace violations.** `rules/scripts/combine.js` rejects duplicate IDs (case-insensitive) and any gather/analyze built-in outside the reserved pattern; `BuiltInRuleCatalogPolicyTests` pins the same invariants on the embedded resource actually deployed.
3. **GitHub reseed rejects non-reserved IDs** before writing to the `global` partition (`ReseedFromGitHubFunction`) — a seeded `*-CUSTOM-*` rule would silently shadow every tenant's same-ID custom rule at merge time.
4. **Cross-tenant RuleState GC is namespace-gated.** `DeleteRuleStatesForRuleIdAcrossTenantsAsync` deletes by bare RowKey across all partitions; it now refuses non-reserved IDs with `failed=-1` (callers then skip the global catalog delete, same contract as enumeration failure).
Invariant chain: schema pattern + combine.js + reseed guard ⇒ the `global` partition only ever contains reserved-pattern IDs ⇒ the pattern is a safe built-in/custom discriminator everywhere no `IsBuiltIn` flag is available.
# Citations
- `src/Shared/AutopilotMonitor.Shared/Models/Rules/RuleIdPolicy.cs` — reserved pattern + rationale
- `src/Backend/AutopilotMonitor.Functions/Services/AnalyzeRuleService.cs`, `GatherRuleService.cs` — create/update guards, merge collision guard
- `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Rules.cs` — key schemes, cross-tenant GC guard
- `src/Backend/AutopilotMonitor.Functions.Tests/BuiltInRuleCatalogPolicyTests.cs`, `RuleIdReservationTests.cs`, `RuleMergeCollisionGuardTests.cs`
- `rules/scripts/combine.js` — build-time namespace/duplicate guard
+137
View File
@@ -0,0 +1,137 @@
---
type: Concept
title: Customer Script Publishing — download.autopilotmonitor.com/agent/
description: How the customer-facing PowerShell scripts reach the download alias, why publishing is decoupled from agent releases, and the guards that keep the published copy, the version badge and the portal's version oracle in agreement.
resource: /.github/scripts/Publish-BootstrapScripts.ps1
tags:
- build
- ci
- release
- bootstrap
- download-alias
timestamp: 2026-08-19T00:00:00+02:00
---
# Overview
Four PowerShell scripts are served to customers from `https://download.autopilotmonitor.com/agent/`:
| Blob | Source | Consumed by |
| --- | --- | --- |
| `Install-AutopilotMonitor.ps1` | `scripts/Bootstrap/Install-AutopilotMonitor.ps1` | Intune platform script (customer uploads a copy); downloaded live on every device by the WDP bootstrap MSI |
| `Install-AutopilotMonitor-Dev.ps1` | rendered from the same source | dev fleet |
| `Test-ShouldBootstrapAgent.ps1` | `scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1` | customer dry run via `irm ... \| iex` |
| `Grant-AutopilotMonitorAddOn.ps1` | `scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1` | optional Graph permission grant |
`Uninstall-AutopilotMonitor.ps1` is deliberately not published — nothing links it, and an
unauthenticated uninstall script on the public download host is not a feature.
The alias is a Front Door route in front of the `agent` container; see
[url-registry.md](url-registry.md) for the host registry and the migration to the alias.
# Publishing chain
`.github/scripts/Publish-BootstrapScripts.ps1` is the single owner. Two workflows call it
with identical arguments:
* **`publish-scripts.yml`** — on every push to `main` touching `scripts/Bootstrap/**`,
`scripts/CustomerSetup/**` or the publisher itself, plus `workflow_dispatch` with a
`dry_run` input.
* **`build-agent.yml`** — inside the `publish_as_stable` branch, after `version.json` is
uploaded, so a release lands a set consistent with what is already live.
Before this split, the script blobs rotated **only** on a stable agent release. A bootstrap
fix therefore sat unpublished until the next agent cutover — while the WDP MSI downloads the
published copy on every enrolling device. Script publishing is now decoupled from the agent
release cadence; the release path still publishes, so it can never fall behind either.
Both workflows share the concurrency group `agent-blob-publish` (`cancel-in-progress: false`).
Both write `version.json`, and the publisher's `bootstrapVersion` read-modify-write must not
interleave with a release writing the agent fields.
# Publisher steps
1. **Parse** `$ScriptVersion` from the bootstrap source.
2. **Dev render** — literal substitution of the two `$AgentDownloadUrl` / `$VersionJsonName`
defaults. A missing anchor literal is a hard failure: a silently un-substituted dev script
would point the dev fleet at the stable agent.
3. **Version-bump guard** — if the published bootstrap changed *behaviour* under an unchanged
`$ScriptVersion`, the run aborts. Comment and formatting changes publish with a warning.
See [Version oracles](#version-oracles).
4. **Upload** every blob with `Cache-Control: no-cache` (they all rotate in place), mirrored
fail-soft to the legacy storage account.
5. **Reconcile the version oracles.**
6. **Verify through the alias** — re-download each blob and compare SHA-256, six attempts
20 s apart. A persistent mismatch means blobs were written but Front Door serves stale
content, and the error names the `az afd endpoint purge` command.
## Deterministic publish form
Every blob is published as **CRLF, UTF-8 without BOM**, normalized by the publisher rather
than taken from the working copy. Git normalizes line endings on checkout — a Windows runner
yields CRLF, a `core.autocrlf=false` clone yields LF — so publishing raw working-copy bytes
would emit different blobs for identical source and make every byte comparison, guard and
verification alike, depend on where the workflow ran. CRLF is also what PS 5.1 expects on the
device and what has always been published.
The ASCII-only rule for `scripts/Bootstrap/*.ps1` (enforced by the gates below) is unrelated
but adjacent: PS 5.1 reads BOM-less files as ANSI and corrupts multi-byte characters.
## Version oracles
Three places state the bootstrap script version, and all three are derived from
`$ScriptVersion` in the source:
* `version.json.bootstrapVersion` — read by the docs badge. Written by the publisher as a
read-modify-write under `If-Match`, so a concurrent agent release keeps its own fields.
`build-agent.yml` seeds the field when it builds the manifest so the published manifest is
never briefly missing it; the publisher owns and re-asserts it afterwards.
* `AdminConfiguration.LatestBootstrapV2ScriptVersion` — read by the portal to tell a customer
their uploaded copy is outdated. **Only** the publisher writes it; `build-agent.yml` writes
the `LatestAgentV2*` fields and nothing else. One writer per field.
* The `Bootstrap script version: v<x>` marker the script logs, parsed out of captured platform
script stdout by `utils/bootstrapVersion.ts`.
The version-bump guard exists because those three would otherwise keep asserting a version
that no longer matches what customers download: a behaviour change under an unchanged version
is invisible to every consumer, and customers decide from that version whether to re-upload
their Intune copy.
"Changed" means changed **behaviour**, matching what the source file's own comment on
`$ScriptVersion` already asks for ("bump on meaningful changes"). The guard compares a code
fingerprint — the PowerShell token stream with comments, newlines and line continuations
removed — so comment fixes, typo corrections in comments and reflow publish with a
`::warning::` instead of failing. Tokenising rather than regex-stripping keeps a `#` inside a
string literal from being mistaken for a comment.
# Gates
`.github/workflows/bootstrap-script-gates.yml` is a `workflow_call` workflow holding the
checks once:
* Pester suite under **Windows PowerShell 5.1** — the runtime IME uses for platform scripts.
* ASCII-only scan over `scripts/Bootstrap/*.ps1`.
* `Publish-BootstrapScripts.ps1 -DryRun` — exercises the dev-render anchors and the
version-bump guard, and reports per blob whether the alias is current, stale or missing.
Two callers: `pester.yml` on pull requests (reports, does not block the merge button), and
`publish-scripts.yml` on pushes to `main`, where the gates job is a `needs:` dependency of the
publish job and therefore **blocks** the upload. `pester.yml` deliberately has no push trigger
— it would duplicate the gate run with no consequence.
# Operating it
* Dry run and drift check, no credentials needed for the read paths:
`pwsh .github/scripts/Publish-BootstrapScripts.ps1 -DryRun`
* Republish without a code change: `workflow_dispatch` on **Publish customer scripts** with
`dry_run=false`.
* Front Door serving stale content after a successful upload:
`az afd endpoint purge --resource-group rg-autopilotmonitor-prd-gwc --profile-name autopilotmonitor-fd --endpoint-name apm-download --content-paths '/agent/*'`
# Citations
* `.github/scripts/Publish-BootstrapScripts.ps1`
* `.github/workflows/publish-scripts.yml`, `.github/workflows/bootstrap-script-gates.yml`, `.github/workflows/pester.yml`
* `.github/workflows/build-agent.yml` (the `publish_as_stable` branch of the upload step)
* `src/Agent/AutopilotMonitor.BootstrapMsi/Invoke-BootstrapDownload.ps1`
* `src/Web/autopilot-monitor-web/utils/bootstrapVersion.ts`
+113
View File
@@ -0,0 +1,113 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-APP-016",
"title": "Required Apps Never Started — ESP Apps Gate Starved",
"description": "One or more required user-ESP apps never started installing while the ESP AccountSetup apps gate waited on them — the agent observed the ESP page exit with those apps still in a non-terminal, never-started state. Unlike ANALYZE-APP-015 (which reports that completion was DEFERRED because of a pending app, and stays silent once the session reaches a terminal verdict) this rule reports the delivery gap itself and fires regardless of the final outcome: an enrollment that Windows called done while required apps never ran is a successful enrollment with missing software, and the operator must see it.",
"severity": "warning",
"category": "apps",
"version": "1.0.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:session_stalled"
],
"trigger": "single",
"baseConfidence": 75,
"confidenceThreshold": 70,
"conditions": [
{
"signal": "starved_apps_present",
"source": "event_count",
"eventType": "app_install_starved",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "many_starved_apps",
"source": "event_count",
"eventType": "app_install_starved",
"operator": "count_gte",
"value": "5",
"required": false
},
{
"signal": "appName",
"source": "event_type",
"eventType": "app_install_starved",
"dataField": "appName",
"operator": "exists",
"value": "",
"required": false
},
{
"signal": "state",
"source": "event_type",
"eventType": "app_install_starved",
"dataField": "state",
"operator": "exists",
"value": "",
"required": false
}
],
"confidenceFactors": [
{
"signal": "many_starved_apps",
"condition": "exists",
"weight": 15
},
{
"signal": "appName",
"condition": "exists",
"weight": 10
}
],
"explanation": "At least one **required** app assigned to the user ESP never started installing. The Intune Management Extension never moved it out of its initial state (`{{state}}`) — it did not download, did not install, and did not fail — while the ESP AccountSetup apps gate was waiting on it. First app observed: `{{appName}}`.\n\nEvery starved app has its own `app_install_starved` timeline event carrying `appId`, `appName` and `state` — open the timeline and filter on that event type for the complete list.\n\nThis is a **delivery** problem, not an install error, and it is invisible in the Intune console: the app simply never appears. Typical causes:\n\n- far more required apps targeted at the ESP than the gate can service in the available window,\n- requirement rules that are never met on this device (OS / architecture / registry / script checks),\n- an unsatisfied dependency or supersedence chain,\n- an app that needs a context unavailable during provisioning (e.g. a VPN client waiting on a portal sign-in),\n- a mid-ESP reboot that cut the gate short — check whether the mid-ESP reboot advisory also fired for this session.\n\nNote that this finding is independent of the session verdict. A session can be marked **Succeeded** and still be missing every app listed here.",
"remediation": [
{
"title": "Shrink the ESP blocking set",
"steps": [
"Open the Enrollment Status Page profile and review the blocking app list — every required app there must finish before the user reaches the desktop.",
"Move apps that do not have to be present at first logon from *required* to *available*, or remove them from the ESP's blocking apps selection.",
"Prefer a small, deliberately chosen blocking set (security agent, VPN, management tooling) over the full application catalogue."
]
},
{
"title": "Unblock the individual apps",
"steps": [
"Collect the app names from the `app_install_starved` timeline events in this session.",
"For each app, review its requirement rules and dependencies in Intune — IntuneManagementExtension.log records every requirement evaluation.",
"Verify the assignment actually targets the signed-in user (user-targeted apps need an established Entra user affinity; check the hybrid user-affinity finding if it also fired).",
"Re-run an enrollment and confirm the apps now reach a terminal state before the ESP exits."
]
},
{
"title": "Rule out a truncated gate",
"steps": [
"Check whether a mid-ESP reboot was observed for this session — a forced restart during AccountSetup cuts the apps gate short and can starve everything still queued.",
"If so, reassign the reboot-forcing device policies to user groups so the restart lands after AccountSetup."
]
}
],
"relatedDocs": [
{
"title": "Microsoft Learn: Enrollment Status Page — blocking apps",
"url": "https://learn.microsoft.com/en-us/intune/intune-service/enrollment/windows-enrollment-status"
},
{
"title": "Microsoft Learn: Win32 app requirement rules",
"url": "https://learn.microsoft.com/en-us/intune/intune-service/apps/apps-win32-add"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"apps",
"esp",
"starvation",
"assignment",
"blocking",
"delivery-gap"
]
}
+69
View File
@@ -0,0 +1,69 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-DEV-008",
"title": "Device Clock Wrong During Enrollment — Jump or Sustained Offset",
"description": "The device's clock changed by five or more minutes mid-session (clock jump) or ran five or more minutes off server time for the whole session (sustained offset), measured against the server receive frame of every uploaded event. Complements ANALYZE-DEV-007, which is a one-shot NTP check at agent startup and misses devices where NTP was unreachable or the clock changed later. IME-log-derived events are excluded from the measurement.",
"severity": "warning",
"category": "device",
"version": "1.0.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed"
],
"trigger": "single",
"baseConfidence": 60,
"confidenceThreshold": 50,
"conditions": [
{
"signal": "clock_jump",
"source": "clock_skew",
"skewMetric": "clock_jump",
"operator": "gte",
"value": "300",
"required": false
},
{
"signal": "sustained_offset",
"source": "clock_skew",
"skewMetric": "sustained_offset",
"operator": "gte",
"value": "300",
"required": false
}
],
"confidenceFactors": [],
"explanation": "This device's {{skewSummary}} (five-minute threshold). A clock that is off by five or more minutes breaks time-sensitive enrollment mechanisms: Kerberos tolerates at most 5 minutes of skew (Hybrid Join), Entra ID token validation allows about 5 minutes on token lifetimes, and freshly issued SCEP/PKCS certificates can be rejected as \"not yet valid\" when the clock is more than ~10 minutes behind the CA.\n\nUnlike ANALYZE-DEV-007 (a one-shot NTP comparison at agent startup), this finding is measured across the whole session against the server's receive time — it also catches devices where the NTP check could not run (UDP 123 blocked) or where the clock changed after startup (time sync correcting a wrong RTC, Windows auto-timezone, manual change).",
"remediation": [
{
"title": "Fix the device clock and time source",
"steps": [
"Check the BIOS/RTC clock and the CMOS battery if the wrong time recurs on the same hardware.",
"At OOBE, run 'w32tm /resync /force' (Shift+F10 command prompt) to sync with time.windows.com.",
"Ensure UDP port 123 to time.windows.com is open in the deployment network.",
"Review 'Set time automatically' and 'Set time zone automatically' behavior on the deployment image — automatic timezone changes mid-enrollment show up as clock jumps."
]
}
],
"relatedDocs": [
{
"title": "Microsoft Learn: Windows Autopilot known issues — device provisioning can fail (clock skew)",
"url": "https://learn.microsoft.com/en-us/autopilot/known-issues"
},
{
"title": "Microsoft Learn: Windows Autopilot requirements — Networking (NTP)",
"url": "https://learn.microsoft.com/en-us/autopilot/requirements?tabs=networking"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"clock-skew",
"clock-jump",
"time-sync",
"kerberos",
"certificates",
"known-issue"
]
}
+48
View File
@@ -0,0 +1,48 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-DEV-009",
"title": "Battery Critically Low During Enrollment",
"description": "Detects that the device's battery dropped to 15% or less while enrolling on battery power (power_state_change threshold event from the agent's live power watcher). A device dying mid-enrollment leaves a half-provisioned state that typically needs a full reset; this rule surfaces the risk while there is still time to plug the device in. Uses a single required condition on the agent-stamped thresholdPercent field (the data field 'transition' on the event is unrelated to this rule's 'trigger' property).",
"severity": "high",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:power_state_change"
],
"trigger": "single",
"baseConfidence": 90,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "battery_threshold_15",
"source": "event_data",
"eventType": "power_state_change",
"dataField": "thresholdPercent",
"operator": "equals",
"value": "15",
"required": true
}
],
"explanation": "The device's battery fell below **{{thresholdPercent}}%** while the enrollment was running on battery power. If the battery drains completely mid-enrollment, the device is left in a half-provisioned state — apps, policies, or the user profile may be incomplete, and in most cases the device has to be reset and re-enrolled. The agent detected the threshold crossing live via a Windows power-status notification; the exact percentage at that moment is recorded on the matched power_state_change event in the session timeline.",
"remediation": [
{
"title": "Secure power for the running enrollment",
"steps": [
"Connect the device to AC power immediately — the enrollment continues normally once power is stable.",
"If the device shut down before completing, check whether the session reached the desktop; if not, reset the device (Autopilot reset or reimage) and re-enroll it plugged in.",
"For technician/White Glove flows, make AC power part of the bench checklist — pre-provisioning on battery is the most common source of this alert.",
"If this fires across many devices of one model, review that model's battery health (the hardware_spec event carries design vs. full-charge capacity)."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"battery",
"environment"
]
}
+48
View File
@@ -0,0 +1,48 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-DEV-010",
"title": "Enrollment Switched From AC to Battery Power",
"description": "Detects that the device lost AC power and switched to battery while the enrollment was running (power_state_change event with transition ac_to_battery from the agent's live power watcher). Uses a single required condition so the match is pinned to one event instance; the event payload carries the battery percentage at the moment of the switch. The agent debounces power flapping (5s) so a brief unplug/replug does not produce this event.",
"severity": "warning",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:power_state_change"
],
"trigger": "single",
"baseConfidence": 85,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "switched_to_battery",
"source": "event_data",
"eventType": "power_state_change",
"dataField": "transition",
"operator": "equals",
"value": "ac_to_battery",
"required": true
}
],
"explanation": "The device switched from AC to battery power mid-enrollment. Enrollments are power-hungry — app downloads, Windows Update and disk encryption drain a battery quickly, and Windows may also throttle CPU and network on battery, slowing the enrollment down. The battery percentage at the moment of the switch is recorded on the matched power_state_change event in the session timeline; if the device stayed on battery, look there for subsequent threshold events (50/30/15%) — a later battery_to_ac event means power was restored.",
"remediation": [
{
"title": "Restore AC power",
"steps": [
"Reconnect the device to AC power; the enrollment continues normally.",
"Check the session timeline for a subsequent battery_to_ac event (power restored) or threshold_crossed events (battery draining) to judge urgency.",
"If this alert is frequent in a deployment area, check docking stations/USB-C hubs for loose power delivery — a flapping dock is a common cause.",
"Make 'plugged in during setup' part of the end-user/technician enrollment instructions."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"battery",
"environment"
]
}
+49
View File
@@ -0,0 +1,49 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-DEV-012",
"title": "Device Entered Standby During Enrollment",
"description": "Detects that the device completed a sleep, hibernate or Modern Standby episode of at least 5 minutes while the enrollment was running (system_sleep_episode event from the agent's system timeline watcher, recorded retroactively at wake from the Windows event log — pre-agent episodes are backfilled). Uses a single required condition on the episode's durationSeconds so the match is pinned to one event instance and the interpolated duration is the matched value. Short Modern Standby screen-off scenarios (under 60 s of real sleep) never emit the event, and this rule adds a 5-minute floor so brief lid-close moments do not fire it.",
"severity": "info",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:system_sleep_episode"
],
"trigger": "single",
"baseConfidence": 90,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "slept_during_enrollment",
"source": "event_data",
"eventType": "system_sleep_episode",
"dataField": "durationSeconds",
"operator": "gte",
"value": "300",
"required": true
}
],
"explanation": "The device was asleep for {{durationSeconds}} seconds during the enrollment window. Nothing progresses while the device sleeps — downloads pause, the IME stops processing and the ESP simply waits — so the wall-clock enrollment duration includes this pause and can look much slower than the enrollment actually was. The matched system_sleep_episode event in the session timeline carries the exact enter/exit times, the episode kind (sleep, hibernate or Modern Standby) and, for Modern Standby, whether the device was on AC power. A long unexplained gap in the timeline right before that event is this pause, not missing telemetry.",
"remediation": [
{
"title": "Keep the device awake during enrollment",
"steps": [
"Check the matched system_sleep_episode event in the timeline for the episode bounds and kind — the enrollment duration minus the episode duration is the real active time.",
"If the device slept on battery, connect AC power during setup; Windows sleeps far more aggressively on battery.",
"Instruct users/technicians to keep the lid open and the device powered until the enrollment finishes.",
"For recurring cases, review the power plan applied during OOBE/ESP (e.g. via provisioning or Intune policy) — consider disabling sleep on AC for the enrollment window."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"standby",
"sleep",
"environment"
]
}
+3 -2
View File
@@ -5,12 +5,13 @@
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.2.0",
"version": "1.3.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed"
"whiteglove_sealed",
"on_event:session_stalled"
],
"trigger": "single",
"baseConfidence": 70,
+325 -2
View File
@@ -831,6 +831,118 @@
"pending"
]
},
{
"ruleId": "ANALYZE-APP-016",
"title": "Required Apps Never Started — ESP Apps Gate Starved",
"description": "One or more required user-ESP apps never started installing while the ESP AccountSetup apps gate waited on them — the agent observed the ESP page exit with those apps still in a non-terminal, never-started state. Unlike ANALYZE-APP-015 (which reports that completion was DEFERRED because of a pending app, and stays silent once the session reaches a terminal verdict) this rule reports the delivery gap itself and fires regardless of the final outcome: an enrollment that Windows called done while required apps never ran is a successful enrollment with missing software, and the operator must see it.",
"severity": "warning",
"category": "apps",
"version": "1.0.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:session_stalled"
],
"trigger": "single",
"baseConfidence": 75,
"confidenceThreshold": 70,
"conditions": [
{
"signal": "starved_apps_present",
"source": "event_count",
"eventType": "app_install_starved",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "many_starved_apps",
"source": "event_count",
"eventType": "app_install_starved",
"operator": "count_gte",
"value": "5",
"required": false
},
{
"signal": "appName",
"source": "event_type",
"eventType": "app_install_starved",
"dataField": "appName",
"operator": "exists",
"value": "",
"required": false
},
{
"signal": "state",
"source": "event_type",
"eventType": "app_install_starved",
"dataField": "state",
"operator": "exists",
"value": "",
"required": false
}
],
"confidenceFactors": [
{
"signal": "many_starved_apps",
"condition": "exists",
"weight": 15
},
{
"signal": "appName",
"condition": "exists",
"weight": 10
}
],
"explanation": "At least one **required** app assigned to the user ESP never started installing. The Intune Management Extension never moved it out of its initial state (`{{state}}`) — it did not download, did not install, and did not fail — while the ESP AccountSetup apps gate was waiting on it. First app observed: `{{appName}}`.\n\nEvery starved app has its own `app_install_starved` timeline event carrying `appId`, `appName` and `state` — open the timeline and filter on that event type for the complete list.\n\nThis is a **delivery** problem, not an install error, and it is invisible in the Intune console: the app simply never appears. Typical causes:\n\n- far more required apps targeted at the ESP than the gate can service in the available window,\n- requirement rules that are never met on this device (OS / architecture / registry / script checks),\n- an unsatisfied dependency or supersedence chain,\n- an app that needs a context unavailable during provisioning (e.g. a VPN client waiting on a portal sign-in),\n- a mid-ESP reboot that cut the gate short — check whether the mid-ESP reboot advisory also fired for this session.\n\nNote that this finding is independent of the session verdict. A session can be marked **Succeeded** and still be missing every app listed here.",
"remediation": [
{
"title": "Shrink the ESP blocking set",
"steps": [
"Open the Enrollment Status Page profile and review the blocking app list — every required app there must finish before the user reaches the desktop.",
"Move apps that do not have to be present at first logon from *required* to *available*, or remove them from the ESP's blocking apps selection.",
"Prefer a small, deliberately chosen blocking set (security agent, VPN, management tooling) over the full application catalogue."
]
},
{
"title": "Unblock the individual apps",
"steps": [
"Collect the app names from the `app_install_starved` timeline events in this session.",
"For each app, review its requirement rules and dependencies in Intune — IntuneManagementExtension.log records every requirement evaluation.",
"Verify the assignment actually targets the signed-in user (user-targeted apps need an established Entra user affinity; check the hybrid user-affinity finding if it also fired).",
"Re-run an enrollment and confirm the apps now reach a terminal state before the ESP exits."
]
},
{
"title": "Rule out a truncated gate",
"steps": [
"Check whether a mid-ESP reboot was observed for this session — a forced restart during AccountSetup cuts the apps gate short and can starve everything still queued.",
"If so, reassign the reboot-forcing device policies to user groups so the restart lands after AccountSetup."
]
}
],
"relatedDocs": [
{
"title": "Microsoft Learn: Enrollment Status Page — blocking apps",
"url": "https://learn.microsoft.com/en-us/intune/intune-service/enrollment/windows-enrollment-status"
},
{
"title": "Microsoft Learn: Win32 app requirement rules",
"url": "https://learn.microsoft.com/en-us/intune/intune-service/apps/apps-win32-add"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"apps",
"esp",
"starvation",
"assignment",
"blocking",
"delivery-gap"
]
},
{
"ruleId": "ANALYZE-CORR-003",
"title": "Proxy Configuration Causing Download Failure",
@@ -1404,6 +1516,216 @@
"known-issue"
]
},
{
"ruleId": "ANALYZE-DEV-008",
"title": "Device Clock Wrong During Enrollment — Jump or Sustained Offset",
"description": "The device's clock changed by five or more minutes mid-session (clock jump) or ran five or more minutes off server time for the whole session (sustained offset), measured against the server receive frame of every uploaded event. Complements ANALYZE-DEV-007, which is a one-shot NTP check at agent startup and misses devices where NTP was unreachable or the clock changed later. IME-log-derived events are excluded from the measurement.",
"severity": "warning",
"category": "device",
"version": "1.0.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed"
],
"trigger": "single",
"baseConfidence": 60,
"confidenceThreshold": 50,
"conditions": [
{
"signal": "clock_jump",
"source": "clock_skew",
"skewMetric": "clock_jump",
"operator": "gte",
"value": "300",
"required": false
},
{
"signal": "sustained_offset",
"source": "clock_skew",
"skewMetric": "sustained_offset",
"operator": "gte",
"value": "300",
"required": false
}
],
"confidenceFactors": [],
"explanation": "This device's {{skewSummary}} (five-minute threshold). A clock that is off by five or more minutes breaks time-sensitive enrollment mechanisms: Kerberos tolerates at most 5 minutes of skew (Hybrid Join), Entra ID token validation allows about 5 minutes on token lifetimes, and freshly issued SCEP/PKCS certificates can be rejected as \"not yet valid\" when the clock is more than ~10 minutes behind the CA.\n\nUnlike ANALYZE-DEV-007 (a one-shot NTP comparison at agent startup), this finding is measured across the whole session against the server's receive time — it also catches devices where the NTP check could not run (UDP 123 blocked) or where the clock changed after startup (time sync correcting a wrong RTC, Windows auto-timezone, manual change).",
"remediation": [
{
"title": "Fix the device clock and time source",
"steps": [
"Check the BIOS/RTC clock and the CMOS battery if the wrong time recurs on the same hardware.",
"At OOBE, run 'w32tm /resync /force' (Shift+F10 command prompt) to sync with time.windows.com.",
"Ensure UDP port 123 to time.windows.com is open in the deployment network.",
"Review 'Set time automatically' and 'Set time zone automatically' behavior on the deployment image — automatic timezone changes mid-enrollment show up as clock jumps."
]
}
],
"relatedDocs": [
{
"title": "Microsoft Learn: Windows Autopilot known issues — device provisioning can fail (clock skew)",
"url": "https://learn.microsoft.com/en-us/autopilot/known-issues"
},
{
"title": "Microsoft Learn: Windows Autopilot requirements — Networking (NTP)",
"url": "https://learn.microsoft.com/en-us/autopilot/requirements?tabs=networking"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"clock-skew",
"clock-jump",
"time-sync",
"kerberos",
"certificates",
"known-issue"
]
},
{
"ruleId": "ANALYZE-DEV-009",
"title": "Battery Critically Low During Enrollment",
"description": "Detects that the device's battery dropped to 15% or less while enrolling on battery power (power_state_change threshold event from the agent's live power watcher). A device dying mid-enrollment leaves a half-provisioned state that typically needs a full reset; this rule surfaces the risk while there is still time to plug the device in. Uses a single required condition on the agent-stamped thresholdPercent field (the data field 'transition' on the event is unrelated to this rule's 'trigger' property).",
"severity": "high",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:power_state_change"
],
"trigger": "single",
"baseConfidence": 90,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "battery_threshold_15",
"source": "event_data",
"eventType": "power_state_change",
"dataField": "thresholdPercent",
"operator": "equals",
"value": "15",
"required": true
}
],
"explanation": "The device's battery fell below **{{thresholdPercent}}%** while the enrollment was running on battery power. If the battery drains completely mid-enrollment, the device is left in a half-provisioned state — apps, policies, or the user profile may be incomplete, and in most cases the device has to be reset and re-enrolled. The agent detected the threshold crossing live via a Windows power-status notification; the exact percentage at that moment is recorded on the matched power_state_change event in the session timeline.",
"remediation": [
{
"title": "Secure power for the running enrollment",
"steps": [
"Connect the device to AC power immediately — the enrollment continues normally once power is stable.",
"If the device shut down before completing, check whether the session reached the desktop; if not, reset the device (Autopilot reset or reimage) and re-enroll it plugged in.",
"For technician/White Glove flows, make AC power part of the bench checklist — pre-provisioning on battery is the most common source of this alert.",
"If this fires across many devices of one model, review that model's battery health (the hardware_spec event carries design vs. full-charge capacity)."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"battery",
"environment"
]
},
{
"ruleId": "ANALYZE-DEV-010",
"title": "Enrollment Switched From AC to Battery Power",
"description": "Detects that the device lost AC power and switched to battery while the enrollment was running (power_state_change event with transition ac_to_battery from the agent's live power watcher). Uses a single required condition so the match is pinned to one event instance; the event payload carries the battery percentage at the moment of the switch. The agent debounces power flapping (5s) so a brief unplug/replug does not produce this event.",
"severity": "warning",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:power_state_change"
],
"trigger": "single",
"baseConfidence": 85,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "switched_to_battery",
"source": "event_data",
"eventType": "power_state_change",
"dataField": "transition",
"operator": "equals",
"value": "ac_to_battery",
"required": true
}
],
"explanation": "The device switched from AC to battery power mid-enrollment. Enrollments are power-hungry — app downloads, Windows Update and disk encryption drain a battery quickly, and Windows may also throttle CPU and network on battery, slowing the enrollment down. The battery percentage at the moment of the switch is recorded on the matched power_state_change event in the session timeline; if the device stayed on battery, look there for subsequent threshold events (50/30/15%) — a later battery_to_ac event means power was restored.",
"remediation": [
{
"title": "Restore AC power",
"steps": [
"Reconnect the device to AC power; the enrollment continues normally.",
"Check the session timeline for a subsequent battery_to_ac event (power restored) or threshold_crossed events (battery draining) to judge urgency.",
"If this alert is frequent in a deployment area, check docking stations/USB-C hubs for loose power delivery — a flapping dock is a common cause.",
"Make 'plugged in during setup' part of the end-user/technician enrollment instructions."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"battery",
"environment"
]
},
{
"ruleId": "ANALYZE-DEV-012",
"title": "Device Entered Standby During Enrollment",
"description": "Detects that the device completed a sleep, hibernate or Modern Standby episode of at least 5 minutes while the enrollment was running (system_sleep_episode event from the agent's system timeline watcher, recorded retroactively at wake from the Windows event log — pre-agent episodes are backfilled). Uses a single required condition on the episode's durationSeconds so the match is pinned to one event instance and the interpolated duration is the matched value. Short Modern Standby screen-off scenarios (under 60 s of real sleep) never emit the event, and this rule adds a 5-minute floor so brief lid-close moments do not fire it.",
"severity": "info",
"category": "device",
"version": "1.0.0",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed",
"on_event:system_sleep_episode"
],
"trigger": "single",
"baseConfidence": 90,
"confidenceThreshold": 60,
"conditions": [
{
"signal": "slept_during_enrollment",
"source": "event_data",
"eventType": "system_sleep_episode",
"dataField": "durationSeconds",
"operator": "gte",
"value": "300",
"required": true
}
],
"explanation": "The device was asleep for {{durationSeconds}} seconds during the enrollment window. Nothing progresses while the device sleeps — downloads pause, the IME stops processing and the ESP simply waits — so the wall-clock enrollment duration includes this pause and can look much slower than the enrollment actually was. The matched system_sleep_episode event in the session timeline carries the exact enter/exit times, the episode kind (sleep, hibernate or Modern Standby) and, for Modern Standby, whether the device was on AC power. A long unexplained gap in the timeline right before that event is this pause, not missing telemetry.",
"remediation": [
{
"title": "Keep the device awake during enrollment",
"steps": [
"Check the matched system_sleep_episode event in the timeline for the episode bounds and kind — the enrollment duration minus the episode duration is the real active time.",
"If the device slept on battery, connect AC power during setup; Windows sleeps far more aggressively on battery.",
"Instruct users/technicians to keep the lid open and the device powered until the enrollment finishes.",
"For recurring cases, review the power plan applied during OOBE/ESP (e.g. via provisioning or Intune policy) — consider disabling sleep on AC for the enrollment window."
]
}
],
"markSessionAsFailedDefault": false,
"tags": [
"device",
"power",
"standby",
"sleep",
"environment"
]
},
{
"ruleId": "ANALYZE-ENRL-001",
"title": "Enrollment Failed",
@@ -2069,12 +2391,13 @@
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.2.0",
"version": "1.3.0",
"author": "Autopilot Monitor",
"enabled": true,
"evaluateOn": [
"enrollment_end",
"whiteglove_sealed"
"whiteglove_sealed",
"on_event:session_stalled"
],
"trigger": "single",
"baseConfidence": 70,
+134 -42
View File
@@ -4,12 +4,23 @@
{
"patternId": "HS-COMPLIANCE",
"category": "always",
"pattern": "\\[HS\\] the (?<part>pre|post)-remdiation detection script compliance result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"pattern": "\\[HS\\] the (?<part>pre|post)-rem(?:e)?diation detection script compliance result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"action": "healthScriptDetectionResult",
"parameters": {
"scriptType": "remediation"
},
"description": "Early-signal fallback for health-script detection / post-detection results. Microsoft IME emits this line immediately after each detection script runs, ~30-90 seconds before the consolidated [HS] new result = JSON line is logged (which only fires after IME's batch-send to the Microsoft service). Short Autopilot enrollments often end inside that gap, so this pattern guarantees at least the compliance verdict is captured. The full data version arrives later via HS-NEW-RESULT and replaces this sparse entry through the UI dedupe (dataCompleteness scoring).",
"description": "Early-signal fallback for health-script detection / post-detection results. Microsoft IME emits this line immediately after each detection script runs, ~30-90 seconds before the consolidated [HS] new result = JSON line is logged (which only fires after IME's batch-send to the Microsoft service). Short Autopilot enrollments often end inside that gap, so this pattern guarantees at least the compliance verdict is captured. The full data version arrives later via HS-NEW-RESULT and replaces this sparse entry through the UI dedupe (dataCompleteness scoring). Tolerates both Microsoft's historical typo 'remdiation' and a future corrected 'remediation'. NOTE: IME 1.97 (Dec 2025) no longer emits the post- variant of this line; the post case is covered by HS-COMPLIANCE-POST.",
"enabled": true
},
{
"patternId": "HS-COMPLIANCE-POST",
"category": "always",
"pattern": "\\[HS\\] the (?<part>post) detection script result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"action": "healthScriptDetectionResult",
"parameters": {
"scriptType": "remediation"
},
"description": "Post-remediation detection result, current wording. IME 1.97 (Dec 2025) replaced the old '[HS] the post-remdiation detection script compliance result for ...' line (still covered by HS-COMPLIANCE for older logs) with '[HS] the post detection script result for <guid> is <True|False>' (ScriptProcessorV2.cs:721). The literal word 'post' in the line is captured as the part group so the handler classifies it as the post-detection phase.",
"enabled": true
},
{
@@ -139,6 +150,14 @@
"description": "Captures the install attempt number for the current app. IME logs 'Execute retry 0' for the first attempt and increments on each retry. Reported as attempt+1 so retry 0 becomes AttemptNumber=1 (human-friendly: first attempt).",
"enabled": true
},
{
"patternId": "IME-CACHE-MISS",
"category": "always",
"pattern": "\\[Win32App\\] Content cache miss for app \\(id = {GUID}, name = (?<name>.*?)\\), start downloading",
"action": "updateName",
"description": "Captures 'Content cache miss for app (id = <guid>, name = <name>), start downloading...' (ContentManager.cs) as a secondary app-name source. Fires right before a real download begins, so it resolves names for exactly the apps that will produce download/install activity — a redundancy for IME-POLICIES-based naming after IME retired the old name-bearing processing lines.",
"enabled": true
},
{
"patternId": "IME-DECRYPT-FAILED",
"category": "currentPhase",
@@ -150,9 +169,9 @@
{
"patternId": "IME-DETECTED-INSTALLED-1",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Detection state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DetectionState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"Installed\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Detection state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DetectionState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}Installed\"{1,2}",
"action": "updateStateInstalled",
"description": "Detects when an app's detection state transitions to Installed (new ReportingManager format).",
"description": "Detects when an app's detection state transitions to Installed (new ReportingManager format). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
@@ -168,15 +187,15 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] Detected app {GUID} ",
"action": "updateStateInstalled",
"description": "Detects when an app is detected as installed (old log format).",
"enabled": true
"description": "Detects when an app is detected as installed (old log format). DISABLED 2026-08-17: the standalone 'Detected app <guid> ' line no longer exists in IME 1.97.107; this regex instead matched the 'Detected app ... and intent is to install, skip applicability check' line, colliding with IME-SKIP-INSTALL-DETECTED and firing updateStateInstalled and updateStateSkipped on the same physical line (see tasks/ime-v1-code-audit-2026-08-17.md).",
"enabled": false
},
{
"patternId": "IME-DETECTION-RESULT",
"category": "always",
"pattern": "\\[Win32App\\]\\[DetectionActionHandler\\] Detection for policy with id: (?<id>{GUID}) resulted in action status: Success and detection state: (?<detection>Detected|NotDetected)",
"pattern": "\\[Win32App\\]\\[DetectionActionHandler\\] Detection for policy with id: (?<id>{GUID}) resulted in action status: Success and detection state: (?<detection>DetectedDifferentVersion|NotDetected|Detected)",
"action": "captureDetectionResult",
"description": "Captures the detection rule result after IME runs detection for an app. 'Detected' means the detection rule matched (app is installed), 'NotDetected' means it did not. Excludes [SimulatedDetectionActionHandler] which fires during applicability pre-checks.",
"description": "Captures the detection rule result after IME runs detection for an app. 'Detected' means the detection rule matched (app is installed), 'NotDetected' means it did not, 'DetectedDifferentVersion' means a different version was found (supersedence / WinGet version mismatch). DetectedDifferentVersion must come first in the alternation so the unanchored regex does not mis-capture its 'Detected' prefix. Excludes [SimulatedDetectionActionHandler] which fires during applicability pre-checks.",
"enabled": true
},
{
@@ -217,9 +236,9 @@
{
"patternId": "IME-ENFORCEMENT-SUCCESS",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"InProgressDownloadCompleted\"\",\"\"NewValue\"\":\"\"Success\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:(?:\"{1,2}\\w+\"{1,2}|null),\"{1,2}NewValue\"{1,2}:\"{1,2}Success\"{1,2}",
"action": "updateStateInstalled",
"description": "Detects when an app's enforcement state transitions from InProgressDownloadCompleted to Success.",
"description": "Detects when an app's enforcement state transitions to Success in an Execution state report delta. OldValue is deliberately unpinned: besides InProgressDownloadCompleted it can be Error (retry succeeding in a later check-in), InProgressPendingReboot (post-reboot completion) or null (first write) — the previously pinned InProgressDownloadCompleted missed those legitimate success paths (IME 1.97 audit 2026-08-17). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
@@ -236,9 +255,9 @@
{
"patternId": "IME-ERROR-DOWNLOAD",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Download state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"InProgress\"\",\"\"NewValue\"\":\"\"ErrorDownloadingContent\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Download state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:\"{1,2}InProgress\"{1,2},\"{1,2}NewValue\"{1,2}:\"{1,2}ErrorDownloadingContent\"{1,2}",
"action": "updateStateError",
"description": "Detects when an app's enforcement state transitions to ErrorDownloadingContent.",
"description": "Detects when an app's enforcement state transitions to ErrorDownloadingContent. Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
@@ -266,12 +285,12 @@
{
"patternId": "IME-ERROR-REPORT",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"(?<from>\\w+)\"\",\"\"NewValue\"\":\"\"(?<to>\\w+)\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:(?:\"{1,2}(?<from>\\w+)\"{1,2}|null),\"{1,2}NewValue\"{1,2}:\"{1,2}(?<to>\\w+)\"{1,2}",
"action": "updateStateError",
"parameters": {
"checkTo": "true"
},
"description": "Captures enforcement state transitions via ReportingManager. Only triggers error when 'to' value indicates an error state.",
"description": "Captures enforcement state transitions via ReportingManager. Only triggers error when 'to' value indicates an error state. OldValue may be the unquoted JSON literal null on the first-ever write of a reporting state — transitions straight from the initial state (e.g. first write directly to an error state) would otherwise be invisible (IME 1.97 audit 2026-08-17). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
@@ -293,6 +312,14 @@
"description": "Detects when an app installer returned an exit code that has no mapping configured in Intune.",
"enabled": true
},
{
"patternId": "IME-ESP-APP-REGISTERED",
"category": "always",
"pattern": "\\[Win32App\\]\\[EspManager\\] In EspPhase: \\w+\\. App {GUID} has been registered.*?App name: (?<name>.*?)$",
"action": "updateName",
"description": "Captures the ESP tracked-app registration line ('App <guid> has been registered[ for user <id>]. App name: <name>', EspManager.cs) as an app-name source. Primary replacement for the retired IME-UPDATE-NAME-1/2/3 whose emitters were removed with IME's old V1/V2 processing pipeline; also documents exactly which apps IME registered for ESP tracking. Category 'always' so names resolve even before the first ApplicationPoller EspPhase line is seen.",
"enabled": true
},
{
"patternId": "IME-ESP-PHASE",
"category": "always",
@@ -304,9 +331,9 @@
{
"patternId": "IME-ESP-TRACK-STATUS",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[EspManager\\] Updating ESP tracked install status from (?<from>\\w+) to (?<to>\\w+) for application {GUID}\\.",
"pattern": "\\[Win32App\\]\\[EspManager\\] Updating ESP tracked install status from (?<from>\\w+) to (?<to>\\w+) for application {GUID}",
"action": "espTrackStatus",
"description": "Captures ESP tracked install status transitions (e.g., InProgress to Installed) for a specific app.",
"description": "Captures ESP tracked install status transitions (e.g., InProgress to Installed) for a specific app. No trailing anchor: IME 1.97 (Dec 2025) appends ' with name: <appName>.' after the GUID (EspManager.cs), which broke the previous '\\.' suffix. 'from' can be the literal string 'null' on first tracking.",
"enabled": true
},
{
@@ -320,6 +347,33 @@
"description": "Captures the installer exit code (lpExitCode) for the current app.",
"enabled": true
},
{
"patternId": "IME-EXITCODE-MAPPED-FAILED",
"category": "currentPhase",
"pattern": "\\[Win32App\\] lpExitCode is defined as Failed$",
"action": "updateStateError",
"parameters": {
"useCurrentApp": "true"
},
"description": "Marks the current app as failed when its exit code maps to Failed via the admin-defined return-code table ('[Win32App] lpExitCode is defined as <Failed|Success|SoftReboot|HardReboot|Retry>', Win32AppInstaller.cs). This is the NORMAL mapped-exit-code outcome; the previously matched 'Setting enforcementState as ... Error' line only fires on the rare no-mapping fallback path. The numeric exit code is captured separately by IME-EXITCODE, which precedes this line.",
"enabled": true
},
{
"patternId": "IME-GRS-CLEARED",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ExecutionActionHandler\\] Encountered early subgraph termination\\. Clearing GRS for app with id: {GUID} so it can be retried during the next check-in\\.",
"action": "updateStatePostponed",
"description": "IME aborted the app's subgraph before enforcement finished and cleared its GRS entry so the app retries on the next check-in (ExecutionActionHandler, AppWorkload.log). Maps to Postponed: the install did not complete this pass but is scheduled to run again - previously this looked like an unexplained gap. Wording verified in IME 1.97 and 1.104.",
"enabled": true
},
{
"patternId": "IME-GRS-SKIP",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ActionProcessor\\] App with id: {GUID} to (?:install|uninstall) is in GRS\\. The app will not be enforced\\.",
"action": "updateStatePostponed",
"description": "GRS (Global Retry Schedule) skip: the app failed recently and IME will not re-enforce it this check-in ('App with id: <guid> to install is in GRS. The app will not be enforced.', EnforcementActionProcessor / MixedEnforcementActionProcessor). THE canonical reason an app silently does nothing for up to 24h - previously invisible. Lives in AppActionProcessor.log, which the agent tails since the 2026-08-18 release; wording verified in IME 1.97 and 1.104.",
"enabled": true
},
{
"patternId": "IME-HRESULT",
"category": "currentPhase",
@@ -377,25 +431,25 @@
{
"patternId": "IME-NOT-APPLICABLE-2",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Applicability state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"ApplicabilityState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"ScriptRequirementRuleNotMet\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Applicability state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}ApplicabilityState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}ScriptRequirementRuleNotMet\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's applicability state is set to ScriptRequirementRuleNotMet (PowerShell requirement script returned false).",
"description": "Detects when an app's applicability state is set to ScriptRequirementRuleNotMet (PowerShell requirement script returned false). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
"patternId": "IME-NOT-APPLICABLE-3",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DesiredState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"None\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DesiredState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}None\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's desired state is set to None (not targeted or not applicable).",
"description": "Detects when an app's desired state is set to None (not targeted or not applicable). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
"patternId": "IME-NOT-APPLICABLE-4",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DesiredState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"NotPresent\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DesiredState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}NotPresent\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's desired state is set to NotPresent (targeted for uninstall or not required).",
"description": "Detects when an app's desired state is set to NotPresent (targeted for uninstall or not required). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
},
{
@@ -422,6 +476,17 @@
"description": "Captures the JSON policies list discovered by IME, used to build the app inventory.",
"enabled": true
},
{
"patternId": "IME-RESTART-STOP-PROCESSING",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[V3Processor\\] A restart is required to continue processing apps\\. Stop processing\\.",
"action": "updateStatePostponed",
"parameters": {
"useCurrentApp": "true"
},
"description": "Detects IME aborting app processing because an app requires a restart ('A restart is required to continue processing apps. Stop processing.', SideCarV3AppProcessor.cs). Postpones the current app: remaining apps do nothing until after the reboot, which previously looked like an unexplained stall mid-ESP.",
"enabled": true
},
{
"patternId": "IME-SESSION-CHANGE",
"category": "always",
@@ -435,24 +500,24 @@
"category": "always",
"pattern": "\\[Win32App\\] ProcessAppWithDependencies starts for {GUID} with name",
"action": "setCurrentApp",
"description": "Detects when IME begins processing an app with its dependency chain, setting it as the current app.",
"enabled": true
"description": "Detects when IME begins processing an app with its dependency chain, setting it as the current app. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SET-CURRENT-2",
"category": "always",
"pattern": "\\[Win32App\\] ExecManager: processing targeted app.*?\\bid\\W+{GUID}",
"action": "setCurrentApp",
"description": "Detects when the IME ExecManager starts processing a targeted app, setting it as current.",
"enabled": true
"description": "Detects when the IME ExecManager starts processing a targeted app, setting it as current. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SET-CURRENT-3",
"category": "always",
"pattern": "\\[Win32App\\] ===Step=== Start to Present app {GUID}",
"action": "setCurrentApp",
"description": "Detects the start of app presentation step in IME, setting the app as current.",
"enabled": true
"description": "Detects the start of app presentation step in IME, setting the app as current. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SET-CURRENT-4",
@@ -478,24 +543,24 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] (?=.*?\\bapplicability\\b)(?=.*?NotMet\\b).*?\\bapp\\W+{GUID}\\b",
"action": "updateStateSkipped",
"description": "Detects when an app's applicability check returns NotMet (old log format with lookaheads).",
"enabled": true
"description": "Detects when an app's applicability check returns NotMet (old log format with lookaheads). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SKIP-DEPENDENCY-DETECT",
"category": "currentPhase",
"pattern": "\\[Win32App\\] app\\W+{GUID}\\b.*?\\bis a dependency\\b.*?\\bDetect Only mode\\b.*?\\bfinished processing",
"action": "updateStateSkipped",
"description": "Detects when a dependency app in Detect Only mode finishes processing (no install needed).",
"enabled": true
"description": "Detects when a dependency app in Detect Only mode finishes processing (no install needed). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SKIP-DETECTED",
"category": "currentPhase",
"pattern": "\\[Win32App\\] Detected app {GUID}.*?skip next step",
"action": "updateStateSkipped",
"description": "Detects when an already-detected app is skipped for further processing steps (old log format).",
"enabled": true
"description": "Detects when an already-detected app is skipped for further processing steps (old log format). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-SKIP-FILTERS",
@@ -529,8 +594,8 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] NOT detected app {GUID}.*?skip un-installation",
"action": "updateStateSkipped",
"description": "Detects when an app is not detected and uninstall is skipped (alternate old log format).",
"enabled": true
"description": "Detects when an app is not detected and uninstall is skipped (alternate old log format). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-STARTED",
@@ -540,6 +605,17 @@
"description": "Detects when the Intune Management Extension (IME) agent starts up.",
"enabled": true
},
{
"patternId": "IME-STOP-SIGNAL",
"category": "currentPhase",
"pattern": "EMS Agent received stop signal",
"action": "updateStatePostponed",
"parameters": {
"useCurrentApp": "true"
},
"description": "Detects an IME service stop ('EMS Agent received stop signal', EmsAgentService.OnStop) — fired on manual restarts and agent-updater restarts, unlike the OS-shutdown-only 'shutdown signal' line matched by IME-SHUTDOWN. Postpones the current app installation, mirroring IME-SHUTDOWN semantics.",
"enabled": true
},
{
"patternId": "IME-SUBGRAPH",
"category": "currentPhase",
@@ -548,6 +624,14 @@
"description": "Detects when V3Processor starts a new subgraph, cancelling any stuck app and setting the new app as current.",
"enabled": true
},
{
"patternId": "IME-TOKEN-FAILURE",
"category": "always",
"pattern": "Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"enabled": true
},
{
"patternId": "IME-TOKEN-SUCCESS",
"category": "always",
@@ -564,24 +648,24 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] ExecManager: processing targeted app.*?name\\W+(?<name>.*?)\\W+id\\W+{GUID}",
"action": "updateName",
"description": "Captures the app name from ExecManager processing log (old format: name before id).",
"enabled": true
"description": "Captures the app name from ExecManager processing log (old format: name before id). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-UPDATE-NAME-2",
"category": "currentPhase",
"pattern": "\\[Win32App\\] ProcessAppWithDependencies starts for {GUID} with name (?<name>.*?)$",
"action": "updateName",
"description": "Captures the app name from ProcessAppWithDependencies log entry.",
"enabled": true
"description": "Captures the app name from ProcessAppWithDependencies log entry. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-UPDATE-NAME-3",
"category": "currentPhase",
"pattern": "\\[Win32App\\] Processing app \\(id={GUID}, name = (?<name>.*?)\\)",
"action": "updateName",
"description": "Captures the app name from the Processing app log entry.",
"enabled": true
"description": "Captures the app name from the Processing app log entry. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
},
{
"patternId": "IME-USER-SESSION-COMPLETED",
@@ -607,6 +691,14 @@
"description": "Captures the Win32 app ESP state (e.g., Installed, InProgress) for a specific app.",
"enabled": true
},
{
"patternId": "IME-WINGET-ERROR",
"category": "currentPhase",
"pattern": "\\[WinGetMessageProcessor\\] Processing Progress for app = {GUID}.*?- Operation Phase = Error",
"action": "updateStateError",
"description": "Marks a WinGet (new Store) app as failed when its operation progress reports phase Error (WinGetOperationInstallPhase: Downloading/Installing/Queued/Finished/Error). Before this pattern only the Installing phase was matched, so WinGet failures never surfaced as app errors (IME 1.97 audit 2026-08-17).",
"enabled": true
},
{
"patternId": "IME-WINGET-INSTALLING-1",
"category": "currentPhase",
@@ -0,0 +1,10 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "HS-COMPLIANCE-POST",
"category": "always",
"pattern": "\\[HS\\] the (?<part>post) detection script result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"action": "healthScriptDetectionResult",
"parameters": { "scriptType": "remediation" },
"description": "Post-remediation detection result, current wording. IME 1.97 (Dec 2025) replaced the old '[HS] the post-remdiation detection script compliance result for ...' line (still covered by HS-COMPLIANCE for older logs) with '[HS] the post detection script result for <guid> is <True|False>' (ScriptProcessorV2.cs:721). The literal word 'post' in the line is captured as the part group so the handler classifies it as the post-detection phase.",
"enabled": true
}
+2 -2
View File
@@ -2,9 +2,9 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "HS-COMPLIANCE",
"category": "always",
"pattern": "\\[HS\\] the (?<part>pre|post)-remdiation detection script compliance result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"pattern": "\\[HS\\] the (?<part>pre|post)-rem(?:e)?diation detection script compliance result for (?<id>[a-z0-9]{8}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{4}-[a-z0-9]{12}) is (?<compliance>True|False)",
"action": "healthScriptDetectionResult",
"parameters": { "scriptType": "remediation" },
"description": "Early-signal fallback for health-script detection / post-detection results. Microsoft IME emits this line immediately after each detection script runs, ~30-90 seconds before the consolidated [HS] new result = JSON line is logged (which only fires after IME's batch-send to the Microsoft service). Short Autopilot enrollments often end inside that gap, so this pattern guarantees at least the compliance verdict is captured. The full data version arrives later via HS-NEW-RESULT and replaces this sparse entry through the UI dedupe (dataCompleteness scoring).",
"description": "Early-signal fallback for health-script detection / post-detection results. Microsoft IME emits this line immediately after each detection script runs, ~30-90 seconds before the consolidated [HS] new result = JSON line is logged (which only fires after IME's batch-send to the Microsoft service). Short Autopilot enrollments often end inside that gap, so this pattern guarantees at least the compliance verdict is captured. The full data version arrives later via HS-NEW-RESULT and replaces this sparse entry through the UI dedupe (dataCompleteness scoring). Tolerates both Microsoft's historical typo 'remdiation' and a future corrected 'remediation'. NOTE: IME 1.97 (Dec 2025) no longer emits the post- variant of this line; the post case is covered by HS-COMPLIANCE-POST.",
"enabled": true
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-CACHE-MISS",
"category": "always",
"pattern": "\\[Win32App\\] Content cache miss for app \\(id = {GUID}, name = (?<name>.*?)\\), start downloading",
"action": "updateName",
"description": "Captures 'Content cache miss for app (id = <guid>, name = <name>), start downloading...' (ContentManager.cs) as a secondary app-name source. Fires right before a real download begins, so it resolves names for exactly the apps that will produce download/install activity — a redundancy for IME-POLICIES-based naming after IME retired the old name-bearing processing lines.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-DETECTED-INSTALLED-1",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Detection state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DetectionState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"Installed\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Detection state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DetectionState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}Installed\"{1,2}",
"action": "updateStateInstalled",
"description": "Detects when an app's detection state transitions to Installed (new ReportingManager format).",
"description": "Detects when an app's detection state transitions to Installed (new ReportingManager format). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
+2 -2
View File
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] Detected app {GUID} ",
"action": "updateStateInstalled",
"description": "Detects when an app is detected as installed (old log format).",
"enabled": true
"description": "Detects when an app is detected as installed (old log format). DISABLED 2026-08-17: the standalone 'Detected app <guid> ' line no longer exists in IME 1.97.107; this regex instead matched the 'Detected app ... and intent is to install, skip applicability check' line, colliding with IME-SKIP-INSTALL-DETECTED and firing updateStateInstalled and updateStateSkipped on the same physical line (see tasks/ime-v1-code-audit-2026-08-17.md).",
"enabled": false
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-DETECTION-RESULT",
"category": "always",
"pattern": "\\[Win32App\\]\\[DetectionActionHandler\\] Detection for policy with id: (?<id>{GUID}) resulted in action status: Success and detection state: (?<detection>Detected|NotDetected)",
"pattern": "\\[Win32App\\]\\[DetectionActionHandler\\] Detection for policy with id: (?<id>{GUID}) resulted in action status: Success and detection state: (?<detection>DetectedDifferentVersion|NotDetected|Detected)",
"action": "captureDetectionResult",
"description": "Captures the detection rule result after IME runs detection for an app. 'Detected' means the detection rule matched (app is installed), 'NotDetected' means it did not. Excludes [SimulatedDetectionActionHandler] which fires during applicability pre-checks.",
"description": "Captures the detection rule result after IME runs detection for an app. 'Detected' means the detection rule matched (app is installed), 'NotDetected' means it did not, 'DetectedDifferentVersion' means a different version was found (supersedence / WinGet version mismatch). DetectedDifferentVersion must come first in the alternation so the unanchored regex does not mis-capture its 'Detected' prefix. Excludes [SimulatedDetectionActionHandler] which fires during applicability pre-checks.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-ENFORCEMENT-SUCCESS",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"InProgressDownloadCompleted\"\",\"\"NewValue\"\":\"\"Success\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:(?:\"{1,2}\\w+\"{1,2}|null),\"{1,2}NewValue\"{1,2}:\"{1,2}Success\"{1,2}",
"action": "updateStateInstalled",
"description": "Detects when an app's enforcement state transitions from InProgressDownloadCompleted to Success.",
"description": "Detects when an app's enforcement state transitions to Success in an Execution state report delta. OldValue is deliberately unpinned: besides InProgressDownloadCompleted it can be Error (retry succeeding in a later check-in), InProgressPendingReboot (post-reboot completion) or null (first write) — the previously pinned InProgressDownloadCompleted missed those legitimate success paths (IME 1.97 audit 2026-08-17). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-ERROR-DOWNLOAD",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Download state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"InProgress\"\",\"\"NewValue\"\":\"\"ErrorDownloadingContent\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Download state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:\"{1,2}InProgress\"{1,2},\"{1,2}NewValue\"{1,2}:\"{1,2}ErrorDownloadingContent\"{1,2}",
"action": "updateStateError",
"description": "Detects when an app's enforcement state transitions to ErrorDownloadingContent.",
"description": "Detects when an app's enforcement state transitions to ErrorDownloadingContent. Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
+5 -3
View File
@@ -2,9 +2,11 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-ERROR-REPORT",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"EnforcementState\"\":\\{\"\"OldValue\"\":\"\"(?<from>\\w+)\"\",\"\"NewValue\"\":\"\"(?<to>\\w+)\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Execution state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}EnforcementState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:(?:\"{1,2}(?<from>\\w+)\"{1,2}|null),\"{1,2}NewValue\"{1,2}:\"{1,2}(?<to>\\w+)\"{1,2}",
"action": "updateStateError",
"parameters": { "checkTo": "true" },
"description": "Captures enforcement state transitions via ReportingManager. Only triggers error when 'to' value indicates an error state.",
"parameters": {
"checkTo": "true"
},
"description": "Captures enforcement state transitions via ReportingManager. Only triggers error when 'to' value indicates an error state. OldValue may be the unquoted JSON literal null on the first-ever write of a reporting state — transitions straight from the initial state (e.g. first write directly to an error state) would otherwise be invisible (IME 1.97 audit 2026-08-17). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-ESP-APP-REGISTERED",
"category": "always",
"pattern": "\\[Win32App\\]\\[EspManager\\] In EspPhase: \\w+\\. App {GUID} has been registered.*?App name: (?<name>.*?)$",
"action": "updateName",
"description": "Captures the ESP tracked-app registration line ('App <guid> has been registered[ for user <id>]. App name: <name>', EspManager.cs) as an app-name source. Primary replacement for the retired IME-UPDATE-NAME-1/2/3 whose emitters were removed with IME's old V1/V2 processing pipeline; also documents exactly which apps IME registered for ESP tracking. Category 'always' so names resolve even before the first ApplicationPoller EspPhase line is seen.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-ESP-TRACK-STATUS",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[EspManager\\] Updating ESP tracked install status from (?<from>\\w+) to (?<to>\\w+) for application {GUID}\\.",
"pattern": "\\[Win32App\\]\\[EspManager\\] Updating ESP tracked install status from (?<from>\\w+) to (?<to>\\w+) for application {GUID}",
"action": "espTrackStatus",
"description": "Captures ESP tracked install status transitions (e.g., InProgress to Installed) for a specific app.",
"description": "Captures ESP tracked install status transitions (e.g., InProgress to Installed) for a specific app. No trailing anchor: IME 1.97 (Dec 2025) appends ' with name: <appName>.' after the GUID (EspManager.cs), which broke the previous '\\.' suffix. 'from' can be the literal string 'null' on first tracking.",
"enabled": true
}
@@ -0,0 +1,10 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-EXITCODE-MAPPED-FAILED",
"category": "currentPhase",
"pattern": "\\[Win32App\\] lpExitCode is defined as Failed$",
"action": "updateStateError",
"parameters": { "useCurrentApp": "true" },
"description": "Marks the current app as failed when its exit code maps to Failed via the admin-defined return-code table ('[Win32App] lpExitCode is defined as <Failed|Success|SoftReboot|HardReboot|Retry>', Win32AppInstaller.cs). This is the NORMAL mapped-exit-code outcome; the previously matched 'Setting enforcementState as ... Error' line only fires on the rare no-mapping fallback path. The numeric exit code is captured separately by IME-EXITCODE, which precedes this line.",
"enabled": true
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-GRS-CLEARED",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ExecutionActionHandler\\] Encountered early subgraph termination\\. Clearing GRS for app with id: {GUID} so it can be retried during the next check-in\\.",
"action": "updateStatePostponed",
"description": "IME aborted the app's subgraph before enforcement finished and cleared its GRS entry so the app retries on the next check-in (ExecutionActionHandler, AppWorkload.log). Maps to Postponed: the install did not complete this pass but is scheduled to run again - previously this looked like an unexplained gap. Wording verified in IME 1.97 and 1.104.",
"enabled": true
}
+9
View File
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-GRS-SKIP",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ActionProcessor\\] App with id: {GUID} to (?:install|uninstall) is in GRS\\. The app will not be enforced\\.",
"action": "updateStatePostponed",
"description": "GRS (Global Retry Schedule) skip: the app failed recently and IME will not re-enforce it this check-in ('App with id: <guid> to install is in GRS. The app will not be enforced.', EnforcementActionProcessor / MixedEnforcementActionProcessor). THE canonical reason an app silently does nothing for up to 24h - previously invisible. Lives in AppActionProcessor.log, which the agent tails since the 2026-08-18 release; wording verified in IME 1.97 and 1.104.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-NOT-APPLICABLE-2",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Applicability state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"ApplicabilityState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"ScriptRequirementRuleNotMet\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Applicability state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}ApplicabilityState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}ScriptRequirementRuleNotMet\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's applicability state is set to ScriptRequirementRuleNotMet (PowerShell requirement script returned false).",
"description": "Detects when an app's applicability state is set to ScriptRequirementRuleNotMet (PowerShell requirement script returned false). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-NOT-APPLICABLE-3",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DesiredState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"None\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DesiredState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}None\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's desired state is set to None (not targeted or not applicable).",
"description": "Detects when an app's desired state is set to None (not targeted or not applicable). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
@@ -2,8 +2,8 @@
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-NOT-APPLICABLE-4",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"\"DesiredState\"\":\\{\"\"OldValue\"\":null,\"\"NewValue\"\":\"\"NotPresent\"\"",
"pattern": "\\[Win32App\\]\\[ReportingManager\\] Desired state for app with id: {GUID} has been updated\\. Report delta: \\{\"{1,2}DesiredState\"{1,2}:\\{\"{1,2}OldValue\"{1,2}:null,\"{1,2}NewValue\"{1,2}:\"{1,2}NotPresent\"{1,2}",
"action": "updateStateSkipped",
"description": "Detects when an app's desired state is set to NotPresent (targeted for uninstall or not required).",
"description": "Detects when an app's desired state is set to NotPresent (targeted for uninstall or not required). Quote-flexible since 2026-08-17: IME 1.104 serializes the report-delta JSON with single quotes ({\"Key\":...) where older versions logged doubled quotes ({\"\"Key\"\":...) - every literal quote in the delta is matched as \"{1,2} to cover both.",
"enabled": true
}
@@ -0,0 +1,10 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-RESTART-STOP-PROCESSING",
"category": "currentPhase",
"pattern": "\\[Win32App\\]\\[V3Processor\\] A restart is required to continue processing apps\\. Stop processing\\.",
"action": "updateStatePostponed",
"parameters": { "useCurrentApp": "true" },
"description": "Detects IME aborting app processing because an app requires a restart ('A restart is required to continue processing apps. Stop processing.', SideCarV3AppProcessor.cs). Postpones the current app: remaining apps do nothing until after the reboot, which previously looked like an unexplained stall mid-ESP.",
"enabled": true
}
@@ -4,6 +4,6 @@
"category": "always",
"pattern": "\\[Win32App\\] ProcessAppWithDependencies starts for {GUID} with name",
"action": "setCurrentApp",
"description": "Detects when IME begins processing an app with its dependency chain, setting it as the current app.",
"enabled": true
"description": "Detects when IME begins processing an app with its dependency chain, setting it as the current app. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "always",
"pattern": "\\[Win32App\\] ExecManager: processing targeted app.*?\\bid\\W+{GUID}",
"action": "setCurrentApp",
"description": "Detects when the IME ExecManager starts processing a targeted app, setting it as current.",
"enabled": true
"description": "Detects when the IME ExecManager starts processing a targeted app, setting it as current. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "always",
"pattern": "\\[Win32App\\] ===Step=== Start to Present app {GUID}",
"action": "setCurrentApp",
"description": "Detects the start of app presentation step in IME, setting the app as current.",
"enabled": true
"description": "Detects the start of app presentation step in IME, setting the app as current. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] (?=.*?\\bapplicability\\b)(?=.*?NotMet\\b).*?\\bapp\\W+{GUID}\\b",
"action": "updateStateSkipped",
"description": "Detects when an app's applicability check returns NotMet (old log format with lookaheads).",
"enabled": true
"description": "Detects when an app's applicability check returns NotMet (old log format with lookaheads). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] app\\W+{GUID}\\b.*?\\bis a dependency\\b.*?\\bDetect Only mode\\b.*?\\bfinished processing",
"action": "updateStateSkipped",
"description": "Detects when a dependency app in Detect Only mode finishes processing (no install needed).",
"enabled": true
"description": "Detects when a dependency app in Detect Only mode finishes processing (no install needed). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] Detected app {GUID}.*?skip next step",
"action": "updateStateSkipped",
"description": "Detects when an already-detected app is skipped for further processing steps (old log format).",
"enabled": true
"description": "Detects when an already-detected app is skipped for further processing steps (old log format). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] NOT detected app {GUID}.*?skip un-installation",
"action": "updateStateSkipped",
"description": "Detects when an app is not detected and uninstall is skipped (alternate old log format).",
"enabled": true
"description": "Detects when an app is not detected and uninstall is skipped (alternate old log format). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -0,0 +1,10 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-STOP-SIGNAL",
"category": "currentPhase",
"pattern": "EMS Agent received stop signal",
"action": "updateStatePostponed",
"parameters": { "useCurrentApp": "true" },
"description": "Detects an IME service stop ('EMS Agent received stop signal', EmsAgentService.OnStop) — fired on manual restarts and agent-updater restarts, unlike the OS-shutdown-only 'shutdown signal' line matched by IME-SHUTDOWN. Postpones the current app installation, mirroring IME-SHUTDOWN semantics.",
"enabled": true
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-TOKEN-FAILURE",
"category": "always",
"pattern": "Failed to get AAD token\\..*?errorCode = (?<errorCode>.*?)$",
"action": "imeTokenFailure",
"description": "IME token acquisition failure ('Failed to get AAD token. len = N using client id ... and resource id ..., errorCode = N', IntuneTokenManager, IntuneManagementExtension.log). This line appears ROUTINELY during the device phase (no user token exists yet), so the agent holds the failure pending: any IME-TOKEN-SUCCESS match clears it, and the ime_token_failure Warning fires only when no success follows for the whole 30-min grace window (then deduped to one Warning per distinct errorCode). A genuine auth outage (WAM broken, conditional access, network) runs long enough to hit the window. Verified against IME 1.97 and 1.104 decompiled sources.",
"enabled": true
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] ExecManager: processing targeted app.*?name\\W+(?<name>.*?)\\W+id\\W+{GUID}",
"action": "updateName",
"description": "Captures the app name from ExecManager processing log (old format: name before id).",
"enabled": true
"description": "Captures the app name from ExecManager processing log (old format: name before id). DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] ProcessAppWithDependencies starts for {GUID} with name (?<name>.*?)$",
"action": "updateName",
"description": "Captures the app name from ProcessAppWithDependencies log entry.",
"enabled": true
"description": "Captures the app name from ProcessAppWithDependencies log entry. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -4,6 +4,6 @@
"category": "currentPhase",
"pattern": "\\[Win32App\\] Processing app \\(id={GUID}, name = (?<name>.*?)\\)",
"action": "updateName",
"description": "Captures the app name from the Processing app log entry.",
"enabled": true
"description": "Captures the app name from the Processing app log entry. DISABLED 2026-08-17: emitter no longer exists in current IME (1.97.107, Dec 2025) - the old V1/V2 app processing pipeline was removed and this line is not logged anymore (verified against decompiled source, see tasks/ime-v1-code-audit-2026-08-17.md). Kept for history.",
"enabled": false
}
@@ -0,0 +1,9 @@
{
"$schema": "../schema/ime-log-pattern.schema.json",
"patternId": "IME-WINGET-ERROR",
"category": "currentPhase",
"pattern": "\\[WinGetMessageProcessor\\] Processing Progress for app = {GUID}.*?- Operation Phase = Error",
"action": "updateStateError",
"description": "Marks a WinGet (new Store) app as failed when its operation progress reports phase Error (WinGetOperationInstallPhase: Downloading/Installing/Queued/Finished/Error). Before this pattern only the Installing phase was matched, so WinGet failures never surfaced as app errors (IME 1.97 audit 2026-08-17).",
"enabled": true
}
+6 -1
View File
@@ -169,7 +169,12 @@
"source": {
"type": "string",
"description": "Source of the signal.",
"enum": ["event_type", "event_data", "event_data_array", "phase_duration", "event_count", "app_install_duration", "event_correlation"]
"enum": ["event_type", "event_data", "event_data_array", "phase_duration", "event_count", "app_install_duration", "event_correlation", "clock_skew"]
},
"skewMetric": {
"type": "string",
"description": "For clock_skew only: which device-clock metric to evaluate. clock_jump = persistent step in the device's clock frame mid-session; sustained_offset = whole session ran on a clock off by at least the threshold. value is the threshold in seconds; operator is limited to gt/gte on the magnitude. IME-log-derived events are excluded from the measurement.",
"enum": ["clock_jump", "sustained_offset"]
},
"eventType": {
"type": "string",
+1
View File
@@ -61,6 +61,7 @@
"cancelStuckAndSetCurrent",
"imeSessionChange",
"imeImpersonation",
"imeTokenFailure",
"enrollmentCompleted",
"updateDoTelemetry",
"scriptStarted",
+25 -2
View File
@@ -13,18 +13,26 @@ const rulesRoot = path.resolve(__dirname, '..');
// ── Rule combination ────────────────────────────────────────────────────────
// Shipped gather/analyze rules must stay inside the reserved built-in namespace:
// (ANALYZE|GATHER)-<CATEGORY>-<NNN> with CATEGORY !== CUSTOM. The CUSTOM category is
// the sanctioned tenant namespace (RuleIdPolicy.cs) — a built-in shipped there would
// silently shadow every tenant's same-ID custom rule at merge time (global wins).
const RESERVED_BUILTIN_ID = /^(ANALYZE|GATHER)-(?!CUSTOM-)[A-Z]+-\d{3}$/;
const configs = [
{
dir: path.join(rulesRoot, 'gather'),
output: path.join(rulesRoot, 'dist', 'gather-rules.json'),
schema: '../schema/gather-rule.schema.json',
idField: 'ruleId'
idField: 'ruleId',
reservedNamespace: RESERVED_BUILTIN_ID
},
{
dir: path.join(rulesRoot, 'analyze'),
output: path.join(rulesRoot, 'dist', 'analyze-rules.json'),
schema: '../schema/analyze-rule.schema.json',
idField: 'ruleId'
idField: 'ruleId',
reservedNamespace: RESERVED_BUILTIN_ID
},
{
dir: path.join(rulesRoot, 'ime-log-patterns'),
@@ -37,12 +45,27 @@ const configs = [
for (const config of configs) {
const files = fs.readdirSync(config.dir).filter(f => f.endsWith('.json')).sort();
const rules = [];
const seenIds = new Map(); // lowercased id -> filename (case variants collide for humans and RuleIdPolicy)
for (const file of files) {
const content = fs.readFileSync(path.join(config.dir, file), 'utf8');
const rule = JSON.parse(content);
// Remove $schema from individual entries (it's on the wrapper)
delete rule['$schema'];
const id = rule[config.idField] || '';
const idKey = id.toLowerCase();
if (seenIds.has(idKey)) {
console.error(`ERROR: duplicate ${config.idField} "${id}" in ${file} (already defined in ${seenIds.get(idKey)})`);
process.exit(1);
}
seenIds.set(idKey, file);
if (config.reservedNamespace && !config.reservedNamespace.test(id)) {
console.error(`ERROR: ${file}: ${config.idField} "${id}" is outside the reserved built-in namespace ${config.reservedNamespace} — built-ins must never use the tenant CUSTOM namespace`);
process.exit(1);
}
rules.push(rule);
}
@@ -95,6 +95,8 @@
Organizations). SKIP-safe when the API is absent (< 1809).
Removed legacy Content-MD5 fallback; SHA-256 manifest verification is
now mandatory (no hash -> bootstrap aborts, no unverified install).
2026-08-18 v2.4 Renamed log file bootstrap_agent.log -> bootstrap-script.log
(unified kebab-case log naming across the agent log directory).
2026-05-09 v2.0 Generic bootstrap: agent owns its own defaults (e.g. 600 s
TenantId-wait), so the script only calls `--install` plain.
Hardened post-install with a 10 s runtime-process verify.
@@ -126,13 +128,13 @@ param(
)
# Script version (bump on meaningful changes; see .CHANGELOG above)
$ScriptVersion = "2.3"
$ScriptVersion = "2.4"
# Configuration - Everything in ProgramData for easy cleanup
$AgentBasePath = "$env:ProgramData\AutopilotMonitor"
$AgentBinPath = "$AgentBasePath\Agent"
$AgentLogPath = "$AgentBasePath\Logs"
$LogFile = "$AgentLogPath\bootstrap_agent.log"
$LogFile = "$AgentLogPath\bootstrap-script.log"
# Write-Host (NOT Write-Output): log calls happen inside functions that return
# decision objects; Write-Output would leak every log line into those return values.
@@ -28,10 +28,13 @@
(validates Windows 365 Cloud PCs against the tenant's Cloud PC inventory --
Cloud PCs are never Autopilot-registered, so this enables agent monitoring
of Cloud PC first-connect enrollment)
- All -> every optional permission above, in one run
(also works with -Revoke to remove all optional grants at once)
.PARAMETER Permissions
Microsoft Graph application permission names to grant directly. Use when the admin UI
rendered a copy-paste command for you, or when you have a specific list in mind.
Microsoft Graph application permission names to grant directly. Use when you copied the
permission list from the admin UI ("Copy permissions"), or when you have a specific list
in mind.
.PARAMETER TenantId
Tenant ID (GUID) or verified domain. Optional -- defaults to the signed-in context.
@@ -47,12 +50,16 @@
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features ScriptDisplayNames
.EXAMPLE
# Same effect, using the explicit-permission form (what the admin UI generates):
# Same effect, using the explicit-permission form:
.\Grant-AutopilotMonitorAddOn.ps1 `
-ClientId "<your-client-id>" `
-Permissions "DeviceManagementScripts.Read.All" `
-TenantId "contoso.onmicrosoft.com"
.EXAMPLE
# Grant every optional capability in one run:
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features All
.EXAMPLE
# Verify current state only, no changes:
.\Grant-AutopilotMonitorAddOn.ps1 -ClientId "<your-client-id>" -Features ScriptDisplayNames -VerifyOnly
@@ -65,6 +72,12 @@
Requires:
- Microsoft.Graph.Authentication PowerShell module (auto-installs if missing).
- Sign-in as Global Administrator OR Privileged Role Administrator OR Cloud Application Administrator.
Sign-in behaviour:
In Azure Cloud Shell the script signs in via the ambient identity endpoint (a delegated
token for the already-signed-in user) - no device-code prompt, which Conditional Access
policies commonly block. Outside Cloud Shell it falls back to a normal interactive
Microsoft Graph sign-in. -TenantId only applies to the interactive fallback.
- The Autopilot Monitor app must be admin-consented in your tenant first
(its service principal must exist before grants can be added).
@@ -80,7 +93,7 @@ param(
[string]$ClientId,
[Parameter(Mandatory = $true, ParameterSetName = 'ByFeatures')]
[ValidateSet('ScriptDisplayNames', 'W365CloudPcValidation')]
[ValidateSet('All', 'ScriptDisplayNames', 'W365CloudPcValidation')]
[string[]]$Features,
[Parameter(Mandatory = $true, ParameterSetName = 'ByPermissions')]
@@ -106,7 +119,11 @@ $FeatureCatalog = @{
}
# Translate Features -> Permissions when caller used the high-level form.
# 'All' is a script-side meta-value: it expands to every catalog feature.
if ($PSCmdlet.ParameterSetName -eq 'ByFeatures') {
if ($Features -contains 'All') {
$Features = @($FeatureCatalog.Keys)
}
$Permissions = @()
foreach ($f in $Features) {
$Permissions += $FeatureCatalog[$f]
@@ -133,7 +150,25 @@ if ($TenantId) { $connectParams['TenantId'] = $TenantId }
Write-Host ""
Write-Host "[Auth] Connecting to Microsoft Graph..." -ForegroundColor Cyan
Connect-MgGraph @connectParams | Out-Null
# Preferred path: ambient identity. In Azure Cloud Shell the local identity endpoint issues a
# delegated Graph token for the signed-in user, which avoids the device-code flow that
# Conditional Access policies commonly block. Outside Azure this throws and we fall back to
# an interactive sign-in with explicit scopes.
$identityConnected = $false
try {
Connect-MgGraph -Identity -NoWelcome -ErrorAction Stop | Out-Null
# An ambient identity can exist yet hold no Graph permissions (e.g. a plain VM managed
# identity) - probe with a read before committing to this session.
Invoke-MgGraphRequest -Method GET -Uri "https://graph.microsoft.com/v1.0/servicePrincipals?`$select=id&`$top=1" -ErrorAction Stop | Out-Null
$identityConnected = $true
Write-Host "[Auth] Connected via ambient identity (Azure Cloud Shell / managed identity)." -ForegroundColor Green
}
catch {
if (Get-MgContext) { Disconnect-MgGraph -ErrorAction SilentlyContinue | Out-Null }
Write-Host "[Auth] No usable ambient identity - falling back to interactive sign-in..." -ForegroundColor Yellow
Connect-MgGraph @connectParams | Out-Null
}
$ctx = Get-MgContext
if (-not $ctx) {
@@ -144,7 +179,10 @@ Write-Host "[Auth] Signed in as : $($ctx.Account)"
Write-Host "[Auth] Tenant : $($ctx.TenantId)"
Write-Host "[Auth] Granted scopes: $($ctx.Scopes -join ', ')"
if (-not ($ctx.Scopes -contains 'AppRoleAssignment.ReadWrite.All')) {
# Scope check only applies to the interactive path: an ambient-identity (Cloud Shell) token
# carries a fixed scope set (incl. Directory.AccessAsUser.All) instead of the requested scopes,
# and its effective rights follow the signed-in user's directory role.
if (-not $identityConnected -and -not ($ctx.Scopes -contains 'AppRoleAssignment.ReadWrite.All')) {
Write-Warning "Scope 'AppRoleAssignment.ReadWrite.All' was NOT granted. The signed-in user likely lacks the required role (Global Admin / Privileged Role Admin / Cloud App Admin)."
if (-not $VerifyOnly) { return }
}
@@ -37,6 +37,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
nameof(AgentConfigResponse.EnrollmentSummaryLaunchRetrySeconds),
nameof(AgentConfigResponse.NtpServer),
nameof(AgentConfigResponse.EnableTimezoneAutoSet),
nameof(AgentConfigResponse.EnableDoGroupIdAutoSet),
nameof(AgentConfigResponse.DiagnosticsUploadEnabled),
nameof(AgentConfigResponse.DiagnosticsUploadMode),
nameof(AgentConfigResponse.DiagnosticsLogPaths),
@@ -43,6 +43,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
EnrollmentSummaryLaunchRetrySeconds = 300,
NtpServer = "pool.ntp.org",
EnableTimezoneAutoSet = true,
EnableDoGroupIdAutoSet = true,
DiagnosticsUploadEnabled = true,
DiagnosticsUploadMode = "OnFailure",
DiagnosticsLogPaths = new List<DiagnosticsLogPath>
@@ -83,6 +84,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.Equal(300, agent.EnrollmentSummaryLaunchRetrySeconds);
Assert.Equal("pool.ntp.org", agent.NtpServer);
Assert.True(agent.EnableTimezoneAutoSet);
Assert.True(agent.EnableDoGroupIdAutoSet);
Assert.True(agent.DiagnosticsUploadEnabled);
Assert.Equal("OnFailure", agent.DiagnosticsUploadMode);
Assert.Single(agent.DiagnosticsLogPaths);
@@ -61,6 +61,41 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.DeviceInfo
return Assert.IsAssignableFrom<IReadOnlyList<string>>(data[key]);
}
[Fact]
public void NoEspEvidenceAtAll_SuppressesTheEvent()
{
// afee7ae0 (WDP / plain Entra join): FirstSync has no values and the ESP tracking
// key does not exist — an event saying only "SkipUser=unknown, SkipDevice=unknown"
// is noise and must not be emitted (mirrors the orchestrator bootstrap's null-gate).
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
var ingress = new FakeSignalIngressSink();
var clock = new VirtualClock(T0);
var collector = new DeviceInfoCollector(
"session-nullgate", "tenant-1",
new InformationalEventPost(ingress, clock), logger,
signalIngress: null, clock: clock,
startupGate: new StartupEventGate(tmp.Path, logger));
using var skip = new EspSkipConfigurationProbe.ScopedFullOverride(
_ => new EspFirstSyncSnapshot(
skipUser: null, skipDevice: null,
blockInStatusPage: null, syncFailureTimeoutMinutes: null));
using (new EspTrackingInfoProbe.ScopedOverride(_ => default))
{
collector.CollectEspConfiguration();
}
Assert.Empty(EspConfigEvents(ingress));
// A later refresh where real evidence appeared must still emit normally.
using (new EspTrackingInfoProbe.ScopedOverride(_ => Snapshot(new[] { DeviceApp1 }, Array.Empty<string>())))
{
collector.RefreshEspConfiguration(DeviceInfoHost.EspConfigTriggerAppsDevice);
}
Assert.Single(EspConfigEvents(ingress));
}
[Fact]
public void Refresh_reemits_when_lists_grew_and_stays_silent_when_unchanged()
{
@@ -36,6 +36,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Harness
{
EspSkipConfigurationProbe.TestOverride = _ => (null, null);
EspSkipConfigurationProbe.FullTestOverride = _ => EspFirstSyncSnapshot.Empty;
// Same reasoning for the IME-install-source probe: a dev machine with real
// EnterpriseDesktopAppManagement CSP state would nondeterministically enrich
// ime_agent_version payloads in adapter tests.
ImeMsiInstallSourceProbe.TestOverride = (_, _) => ImeMsiInstallSource.Empty;
}
}
}
@@ -17,14 +17,22 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Integration
{
/// <summary>
/// Plan §EspExiting Adapter — wire-through verification of the production EspExiting path:
/// <c>ShellCoreTracker.HandleBackfillRecord</c> → <c>EspExited</c> event → inner re-raise on
/// <c>EspAndHelloTracker.EspExited</c> → <c>EspAndHelloTrackerAdapter</c> → ingress → engine
/// reducer. This is the path the live V2-Host uses; without the coordinator forward the
/// EspExiting signal would never reach the engine in production.
/// <c>ShellCoreTracker.ProcessEvent</c> (live Shell-Core 62407) → <c>EspExited</c> event →
/// inner re-raise on <c>EspAndHelloTracker.EspExited</c> → <c>EspAndHelloTrackerAdapter</c> →
/// ingress → engine reducer. Without the coordinator forward the EspExiting signal would
/// never reach the engine in production.
/// <para>
/// The first test isolates the tracker's <c>EspExited</c> contract (source-event timestamp
/// preserved through backfill); the coordinator tests drive the full production chain through
/// to the <c>HelloSafety</c> deadline when AccountSetup is already entered.
/// preserved rather than collapsed to wall-clock-now); the coordinator tests drive the full
/// production chain through to the <c>HelloSafety</c> deadline when AccountSetup is already
/// entered.
/// </para>
/// <para>
/// It used to drive that first test through <c>HandleBackfillRecord</c> because the replay is
/// easier to call than the event-log watcher — but the replay no longer raises
/// <c>EspExited</c> at all (a replayed 62407 cannot be placed in time; see
/// <c>ShellCoreTrackerReplayScopeTests</c>), and it never was the production path: the replay
/// had no caller when this test was written.
/// </para>
/// </summary>
public sealed class ShellCoreToEngineEspExitingTests
@@ -34,12 +42,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Integration
"CommercialOOBE_ESPProgress_Page_Exiting"; // matches EspExitingPattern in ShellCoreTracker
[Fact]
public void Tracker_backfill_record_raises_EspExited_with_source_timestamp()
public void Tracker_live_record_raises_EspExited_with_source_timestamp()
{
// Tracker-level contract: HandleBackfillRecord raises EspExited carrying the source
// event time (not Clock.UtcNow). The EspExited→EspExiting signal mapping and SourceOrigin
// are covered by the coordinator tests below; here we isolate the backfill→event link
// that the production coordinator depends on.
// Tracker-level contract: the live 62407 path raises EspExited carrying the source
// event time (record.TimeCreated, not Clock.UtcNow). The EspExited→EspExiting signal
// mapping and SourceOrigin are covered by the coordinator tests below; here we isolate
// the event-log→event link that the production coordinator depends on.
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info);
var clock = new VirtualClock(ClockNow);
@@ -57,7 +65,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Integration
// Source time IS in the past relative to the clock — verify the tracker carries the
// historical timestamp on the event rather than collapsing to Clock.UtcNow.
var sourceTime = ClockNow.AddMinutes(-2);
tracker.HandleBackfillRecord(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitingDescription, sourceTime);
tracker.ProcessEvent(
eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent,
description: EspExitingDescription,
timestamp: sourceTime,
providerName: "Microsoft-Windows-Shell-Core",
isBackfill: false);
Assert.NotNull(captured);
Assert.Equal(sourceTime, captured!.OccurredAtUtc);
@@ -15,7 +15,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
/// </summary>
public sealed class AgentLoggerTests
{
private static string TodayBaseName => $"agent_{DateTime.Now:yyyyMMdd}";
private const string BaseName = "agent";
[Fact]
public void Log_writes_to_unsuffixed_base_file_when_dir_is_empty()
@@ -25,7 +25,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
logger.Info("hello");
var basePath = Path.Combine(tmp.Path, TodayBaseName + ".log");
var basePath = Path.Combine(tmp.Path, BaseName + ".log");
Assert.True(File.Exists(basePath), $"expected base file at {basePath}");
Assert.Contains("hello", File.ReadAllText(basePath));
}
@@ -39,17 +39,17 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
// Each line ~80 bytes after timestamp/severity envelope; 30 lines clears 1 KB
// even at the smallest payload. The first lines fill the base file; once the
// FileInfo.Length read sees >= 1024 the next call rotates to _002.
// FileInfo.Length read sees >= 1024 the next call rotates to -002.
for (int i = 0; i < 60; i++)
{
logger.Info($"line-{i:D4}-padded-with-some-extra-context-to-cross-1KB-quickly");
}
var files = Directory.GetFiles(tmp.Path, TodayBaseName + "*.log").OrderBy(s => s).ToArray();
var files = Directory.GetFiles(tmp.Path, BaseName + "*.log").OrderBy(s => s).ToArray();
Assert.True(files.Length >= 2, $"expected at least 2 segments, got: {string.Join(",", files.Select(Path.GetFileName))}");
// The unsuffixed file is the first segment; rotated segments use _002 onward.
var rotated = files.FirstOrDefault(p => Path.GetFileNameWithoutExtension(p).EndsWith("_002"));
// The unsuffixed file is the first segment; rotated segments use -002 onward.
var rotated = files.FirstOrDefault(p => Path.GetFileNameWithoutExtension(p).EndsWith("-002"));
Assert.NotNull(rotated);
}
@@ -64,8 +64,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
logger.Info($"pad-{i:D4}-content-content-content-content-content-content");
}
var rotatedPath = Path.Combine(tmp.Path, TodayBaseName + "_002.log");
Assert.True(File.Exists(rotatedPath), "expected agent_YYYYMMDD_002.log to exist");
var rotatedPath = Path.Combine(tmp.Path, BaseName + "-002.log");
Assert.True(File.Exists(rotatedPath), "expected agent-002.log to exist");
var first = File.ReadAllLines(rotatedPath).FirstOrDefault();
Assert.NotNull(first);
Assert.Contains("AgentLogger: rotated", first);
@@ -75,23 +75,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
public void Log_continues_existing_rotation_on_restart_without_overwriting()
{
using var tmp = new TempDirectory();
// Pre-seed: looks like an earlier process already rotated up to _003.
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + ".log"), "older-base\n");
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + "_002.log"), "older-002\n");
File.WriteAllText(Path.Combine(tmp.Path, TodayBaseName + "_003.log"), "older-003\n");
// Pre-seed: looks like an earlier process already rotated up to -003.
File.WriteAllText(Path.Combine(tmp.Path, BaseName + ".log"), "older-base\n");
File.WriteAllText(Path.Combine(tmp.Path, BaseName + "-002.log"), "older-002\n");
File.WriteAllText(Path.Combine(tmp.Path, BaseName + "-003.log"), "older-003\n");
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info, maxFileSizeBytes: 1024L * 1024);
logger.Info("after-restart");
// New entries must land in the highest existing segment (_003), not in a fresh
// New entries must land in the highest existing segment (-003), not in a fresh
// base file. Older content in _003 must be preserved.
var resumed = File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + "_003.log"));
var resumed = File.ReadAllText(Path.Combine(tmp.Path, BaseName + "-003.log"));
Assert.Contains("older-003", resumed);
Assert.Contains("after-restart", resumed);
// Older segments are left alone.
Assert.Equal("older-base\n", File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + ".log")));
Assert.Equal("older-002\n", File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + "_002.log")));
Assert.Equal("older-base\n", File.ReadAllText(Path.Combine(tmp.Path, BaseName + ".log")));
Assert.Equal("older-002\n", File.ReadAllText(Path.Combine(tmp.Path, BaseName + "-002.log")));
}
[Fact]
@@ -105,7 +105,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
// as plain ASCII, while genuine non-ASCII payload stays untouched.
logger.Info("check failed \u2014 range 1\u20132 \u2192 done\u2026 \u2018a\u2019 \u201Cb\u201D nb\u00A0sp f\u00FCr M\u00FCller");
var text = File.ReadAllText(Path.Combine(tmp.Path, TodayBaseName + ".log"));
var text = File.ReadAllText(Path.Combine(tmp.Path, BaseName + ".log"));
Assert.Contains("check failed - range 1-2 -> done... 'a' \"b\" nb sp f\u00FCr M\u00FCller", text);
}
@@ -115,8 +115,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Logging
using var tmp = new TempDirectory();
var logger = new AgentLogger(tmp.Path, AgentLogLevel.Info, maxFileSizeBytes: 1024L * 1024);
// Hold an exclusive lock on today's file to force AppendAllText to throw.
var lockedPath = Path.Combine(tmp.Path, TodayBaseName + ".log");
// Hold an exclusive lock on the log file to force AppendAllText to throw.
var lockedPath = Path.Combine(tmp.Path, BaseName + ".log");
File.WriteAllText(lockedPath, ""); // ensure exists
using var holder = new FileStream(lockedPath, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
@@ -170,5 +170,53 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
Assert.Empty(f.StarvedPosts());
Assert.Empty(coordinator.StarvedAppsReported);
}
[Fact]
public void Reevaluate_does_not_re_emit_starved_warnings()
{
// sits-d Cloud-PC fix (2026-08-19): the synthesis re-check now runs on EVERY IME
// app-state transition. It must not turn the one-shot starvation warning into a
// per-transition stream — the re-check path suppresses the emission entirely
// (the per-appId dedupe would swallow the duplicates, but walking the probe
// hundreds of times on the IME hot path is waste, not safety).
using var f = new Fixture();
using var coordinator = f.BuildCoordinator(
() => false,
() => new[] { StarvedApp("app-1", "Contoso Backgrounds") });
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
var afterExit = f.StarvedPosts().Count;
Assert.Equal(1, afterExit);
for (var i = 0; i < 10; i++) coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(afterExit, f.StarvedPosts().Count);
Assert.Single(coordinator.StarvedAppsReported);
}
[Fact]
public void Starved_apps_that_settle_later_still_complete_the_enrollment()
{
// The real sits-d shape: the ESP exits with apps still starved (warning emitted),
// the apps then all settle, and the re-check must open the gate — the warning
// already on the timeline is history, not a permanent veto.
using var f = new Fixture();
var settled = false;
using var coordinator = f.BuildCoordinator(
() => settled,
() => new[] { StarvedApp("app-1", "Contoso Backgrounds") });
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
Assert.Single(f.StarvedPosts());
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete);
settled = true;
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(1, f.Ingress.Posted.Count(p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete));
Assert.Single(f.StarvedPosts());
}
}
}
@@ -35,14 +35,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
public Fixture() { Logger = new AgentLogger(Tmp.Path, AgentLogLevel.Debug); }
public EspAndHelloTracker BuildCoordinator(Func<bool> settledProbe)
=> BuildCoordinator(settledProbe, accountSetupActivityProbe: () => true, skipUserEsp: false);
/// <summary>
/// Codex review P1 (2026-08-19): the remembered exit edge is gated on positive evidence
/// that the exit is post-AccountSetup, so these two probes decide whether the deferred
/// re-check may fire at all.
/// </summary>
public EspAndHelloTracker BuildCoordinator(
Func<bool> settledProbe,
Func<bool> accountSetupActivityProbe,
bool? skipUserEsp)
{
return new EspAndHelloTracker(
sessionId: "S1",
tenantId: "T1",
post: new InformationalEventPost(TrackerPostSink, Clock),
logger: Logger,
skipConfigProbe: () => ((bool?)false, (bool?)false),
accountSetupActivityProbe: () => true,
skipConfigProbe: () => (skipUserEsp, (bool?)false),
accountSetupActivityProbe: accountSetupActivityProbe,
userEspAppsSettledProbe: settledProbe);
}
@@ -159,5 +170,174 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
var posted = Assert.Single(f.Ingress.Posted);
Assert.Equal(DecisionSignalKind.EspExiting, posted.Kind);
}
// ------------------------------------------------------------------------------
// sits-d Cloud-PC fix (2026-08-19) — the synthesis is no longer edge-only.
// Sessions 8110e262 / a89aac2d: the ESP page exited while 138 required user-ESP apps
// were still in flight, they all reached a terminal state (0 failed) minutes later,
// and because nothing re-checked, the enrollment never completed.
// ------------------------------------------------------------------------------
[Fact]
public void Reevaluate_after_apps_settle_synthesizes_without_a_second_exit()
{
using var f = new Fixture();
var settled = false;
using var coordinator = f.BuildCoordinator(() => settled);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete);
// The apps settle later; only the IME app-state callback fires — there is NO second
// ESP exit on a real device, which is exactly why the edge-only version starved.
settled = true;
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(1, f.Ingress.Posted.Count(p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete));
Assert.True(coordinator.UserAppsSettledSynthesisFiredForTest);
var info = Assert.Single(f.TrackerPostSink.Posted);
var data = Assert.IsAssignableFrom<System.Collections.Generic.IReadOnlyDictionary<string, object>>(info.TypedPayload);
Assert.Equal("esp_exited_user_apps_settled_category_unresolved", data["fallbackReason"]);
}
[Fact]
public void Reevaluate_without_an_observed_esp_exit_never_synthesizes()
{
// The gate needs BOTH facts. Settled apps alone must never open it — otherwise every
// quiet moment during AccountSetup would look like completion.
using var f = new Fixture();
using var coordinator = f.BuildCoordinator(() => true);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.ReevaluateUserAppsSettledSynthesis();
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Empty(f.Ingress.Posted);
Assert.Empty(f.TrackerPostSink.Posted);
Assert.False(coordinator.UserAppsSettledSynthesisFiredForTest);
}
[Fact]
public void Reevaluate_is_idempotent_after_the_synthesis_fired()
{
// The IME callback fires per app-state transition — on these tenants that is 138+
// times. The synthesis must stay fire-once.
using var f = new Fixture();
using var coordinator = f.BuildCoordinator(() => true);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
for (var i = 0; i < 20; i++) coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(1, f.Ingress.Posted.Count(p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete));
Assert.Single(f.TrackerPostSink.Posted);
}
// ------------------------------------------------------------------------------
// Codex review P1 (2026-08-19): Shell-Core raises 62407 at EVERY ESP phase transition.
// Remembering the intermediate DeviceSetup→AccountSetup exit would let the deferred
// re-check open the strong AccountSetup gate as soon as the last user app settles —
// while the AccountSetup page is still up and its other subcategories still running.
// ------------------------------------------------------------------------------
[Fact]
public void Intermediate_device_exit_is_not_remembered_as_the_edge()
{
using var f = new Fixture();
var settled = false;
// Classic user-driven enrollment (SkipUser=false) that has NOT reached AccountSetup:
// this 62407 is the DeviceSetup→AccountSetup transition, not the final exit.
using var coordinator = f.BuildCoordinator(
() => settled, accountSetupActivityProbe: () => false, skipUserEsp: false);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
// Apps settle afterwards — the level is reached, but the edge was never a valid one.
settled = true;
for (var i = 0; i < 10; i++) coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete);
Assert.False(coordinator.UserAppsSettledSynthesisFiredForTest);
Assert.Empty(f.TrackerPostSink.Posted);
}
[Fact]
public void Unknown_skip_user_is_treated_as_unconfirmed()
{
// Erring strict costs at worst today's stall; erring loose costs a premature Succeeded.
using var f = new Fixture();
var settled = false;
using var coordinator = f.BuildCoordinator(
() => settled, accountSetupActivityProbe: () => false, skipUserEsp: null);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
settled = true;
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.False(coordinator.UserAppsSettledSynthesisFiredForTest);
}
[Fact]
public void Skip_user_profile_confirms_the_device_exit_as_final()
{
// SkipUser=true means there is no user ESP at all — the Device-ESP exit IS the final
// one and no second exit is coming, so the edge is legitimately remembered.
using var f = new Fixture();
var settled = false;
using var coordinator = f.BuildCoordinator(
() => settled, accountSetupActivityProbe: () => false, skipUserEsp: true);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
settled = true;
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(1, f.Ingress.Posted.Count(p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete));
}
[Fact]
public void Account_setup_activity_confirms_the_edge_for_the_deferred_recheck()
{
// The real reboot shape: AccountSetup activity is visible in the registry, the final
// exit lands, the apps settle a moment later.
using var f = new Fixture();
var settled = false;
using var coordinator = f.BuildCoordinator(
() => settled, accountSetupActivityProbe: () => true, skipUserEsp: false);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
Assert.DoesNotContain(f.Ingress.Posted, p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete);
settled = true;
coordinator.ReevaluateUserAppsSettledSynthesis();
Assert.Equal(1, f.Ingress.Posted.Count(p => p.Kind == DecisionSignalKind.AccountSetupProvisioningComplete));
}
// NOTE: the "replayed exit" cases that used to live here are gone by construction —
// ShellCoreTracker never re-raises a 62407 from the replay any more, so no backfilled exit
// can reach this handler at all. That contract is pinned one layer up, in
// ShellCoreTrackerReplayScopeTests, together with the reducer-side proof of why it matters
// (ClassicEspExitingOnRestoredStateTests).
[Fact]
public void Reevaluate_stays_silent_while_apps_are_still_unsettled()
{
using var f = new Fixture();
using var coordinator = f.BuildCoordinator(() => false);
using var adapter = new EspAndHelloTrackerAdapter(coordinator, f.Ingress, f.Clock);
coordinator.TriggerEspExitedForTest(Fixed);
for (var i = 0; i < 5; i++) coordinator.ReevaluateUserAppsSettledSynthesis();
var posted = Assert.Single(f.Ingress.Posted);
Assert.Equal(DecisionSignalKind.EspExiting, posted.Kind);
Assert.False(coordinator.UserAppsSettledSynthesisFiredForTest);
}
}
}
@@ -25,7 +25,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
public void Dispose() => _tmp.Dispose();
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather_rules_debug.log");
private string LogPath => Path.Combine(_tmp.Path, "sub", "gather-rules-debug.log");
[Fact]
public void Write_produces_pipe_delimited_line_and_creates_directory_lazily()
@@ -73,7 +73,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
for (int i = 0; i < 20; i++)
log.Write("RULE-1", GatherRuleDebugLog.StageExec, new string('x', 100));
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather_rules_debug.old.log");
var oldPath = Path.Combine(Path.GetDirectoryName(LogPath), "gather-rules-debug.old.log");
Assert.True(File.Exists(oldPath), "expected one .old rotation generation");
Assert.True(File.Exists(LogPath));
Assert.Contains("-- rotated", File.ReadAllText(LogPath) + File.ReadAllText(oldPath));
@@ -28,7 +28,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
public GatherRuleExecutorDebugLogTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_debugLogPath = Path.Combine(_tmp.Path, "gather-rules-debug.log");
_executor = new GatherRuleExecutor(
"sess", "tenant", _ => { }, logger,
debugLogPath: _debugLogPath)
@@ -32,7 +32,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
Directory.CreateDirectory(_outsideUsers);
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_debugLogPath = Path.Combine(_tmp.Path, "gather_rules_debug.log");
_debugLogPath = Path.Combine(_tmp.Path, "gather-rules-debug.log");
_context = new GatherRuleContext(
logger, "sess", "tenant",
evt => _events.Add(evt),
@@ -0,0 +1,109 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// Pins the per-FILE isolation of the offset measurement across a full read cycle.
///
/// <para>
/// Background (2026-08-20, session e9753578): the field showed "+02:00 measured for
/// IntuneManagementExtension.log" although that file held no +2-era line — the anchor had
/// crossed file boundaries. The committed code was proven innocent (this very scenario is
/// green here); the leak lived in a binary built from an uncommitted tree. This test keeps
/// the property pinned so a real regression of the file scoping can never ship silently
/// again — it replays the exact field pass structure: a NEW file appearing mid-run with a
/// different writer era than the file already being tailed.
/// </para>
/// </summary>
public sealed class CmTraceCalibratorFileIsolationTests
{
private static readonly DateTime T0 = new DateTime(2026, 8, 20, 12, 13, 0, DateTimeKind.Utc);
private static List<ImeLogPattern> Patterns() => new List<ImeLogPattern>
{
new ImeLogPattern
{
PatternId = "T-MARK", Category = "always", Enabled = true,
Pattern = @"marker (?<n>\d+)",
Action = "noop",
Parameters = new Dictionary<string, string>(),
},
};
private static string Line(string message, DateTime utcInstant, TimeSpan writerOffset)
{
var local = utcInstant + writerOffset;
return $"<![LOG[{message}]LOG]!><time=\"{local:HH:mm:ss.fffffff}\" date=\"{local:M-d-yyyy}\" " +
"component=\"X\" context=\"\" type=\"1\" thread=\"1\" file=\"\">";
}
[Fact]
public async Task NewFileWithOtherEra_MustNotRecalibrateTheOldFile()
{
var pdt = TimeSpan.FromHours(-7); // IME service era
var cest = TimeSpan.FromHours(2); // fresh child-process era
using var tmp = new TempDirectory();
var now = T0;
var tracker = new ImeLogTracker(
logFolder: tmp.Path,
patterns: Patterns(),
logger: new AgentLogger(tmp.Path, AgentLogLevel.Info));
try
{
tracker.UtcNowProvider = () => now;
var imeLog = Path.Combine(tmp.Path, "IntuneManagementExtension.log");
var execLog = Path.Combine(tmp.Path, "AgentExecutor.log");
// Pass 1: IME.log first sight (no calibration).
File.AppendAllText(imeLog, Line("marker 1", now, pdt) + Environment.NewLine);
await tracker.CheckLogFilesAsync(CancellationToken.None);
// Pass 2: IME.log grew -> measures -7.
now = T0.AddSeconds(10);
File.AppendAllText(imeLog, Line("marker 2", now, pdt) + Environment.NewLine);
await tracker.CheckLogFilesAsync(CancellationToken.None);
TimeSpan imeMeasured;
Assert.True(tracker.OffsetCalibrator.TryGetOffset("IntuneManagementExtension.log", out imeMeasured));
Assert.Equal(pdt, imeMeasured);
// Pass 3: AgentExecutor.log appears for the FIRST time with fresh +2-era lines
// while IME.log grows too — the exact field pass structure. The +2 anchor must
// never land under IME.log's key.
now = T0.AddSeconds(20);
File.AppendAllText(execLog, Line("marker 3", now, cest) + Environment.NewLine);
File.AppendAllText(imeLog, Line("marker 4", now, pdt) + Environment.NewLine);
await tracker.CheckLogFilesAsync(CancellationToken.None);
Assert.True(tracker.OffsetCalibrator.TryGetOffset("IntuneManagementExtension.log", out imeMeasured));
Assert.Equal(pdt, imeMeasured);
// Pass 4: both grow — each file measures its own writer's era.
now = T0.AddSeconds(30);
File.AppendAllText(execLog, Line("marker 5", now, cest) + Environment.NewLine);
File.AppendAllText(imeLog, Line("marker 6", now, pdt) + Environment.NewLine);
await tracker.CheckLogFilesAsync(CancellationToken.None);
TimeSpan execMeasured;
Assert.True(tracker.OffsetCalibrator.TryGetOffset("AgentExecutor.log", out execMeasured));
Assert.Equal(cest, execMeasured);
Assert.True(tracker.OffsetCalibrator.TryGetOffset("IntuneManagementExtension.log", out imeMeasured));
Assert.Equal(pdt, imeMeasured);
}
finally
{
tracker.Dispose();
}
}
}
}
@@ -67,29 +67,34 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
Assert.Equal("EnforcementState: Installing app X", entry.Message);
Assert.Equal("AppEnforce", entry.Component);
Assert.Equal(1, entry.Type);
Assert.NotEqual(default, entry.Timestamp);
Assert.True(entry.HasTimestamp);
Assert.NotEqual(default, entry.LocalTimestamp);
// Bias-less line: the parser refuses to guess the writer's zone, so no UTC value.
Assert.Null(entry.BiasMinutes);
Assert.Null(entry.TimestampUtc);
Assert.Equal(DateTimeKind.Unspecified, entry.LocalTimestamp.Kind);
}
// ── Structurally valid but unparseable timestamp → UtcNow fallback ────
// ── Structurally valid but unparseable timestamp → flagged, not invented ──
[Fact]
public void TryParseLine_falls_back_to_utcnow_for_unparseable_timestamp()
public void TryParseLine_flags_unparseable_timestamp_instead_of_inventing_one()
{
// time/date satisfy the regex character classes ([\d:.]+ / [\d-]+) so the line MATCHES,
// but "13-45-2026" / "25:99:99" fail every DateTime format → the parser falls back to
// DateTime.UtcNow (CmTraceLogParser.cs:57) instead of throwing.
var before = DateTime.UtcNow;
// but "13-45-2026" / "25:99:99" fail every DateTime format. The parser used to stamp
// DateTime.UtcNow here, which made a fabricated value indistinguishable from a real
// one downstream. It now reports HasTimestamp=false and leaves the choice of fallback
// to the caller, which is the only party that knows what a safe default is in context.
var line = CmTraceLine("some message", "25:99:99.0000000", "13-45-2026");
var ok = CmTraceLogParser.TryParseLine(line, out var entry);
var after = DateTime.UtcNow;
Assert.True(ok);
Assert.NotNull(entry);
Assert.Equal("some message", entry.Message);
// Don't assert an exact instant — only that the fallback stamped a fresh "now"
// (a successfully-parsed date would land in 2026, far outside this window).
Assert.InRange(entry.Timestamp, before.AddSeconds(-5), after.AddSeconds(5));
Assert.False(entry.HasTimestamp);
Assert.Equal(default, entry.LocalTimestamp);
Assert.Null(entry.TimestampUtc);
}
// ── UTC-bias suffix on the time field → honored, not dropped ──────────
@@ -108,10 +113,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
Assert.True(ok);
Assert.NotNull(entry);
// A writer-declared bias IS the fact we otherwise have to measure, so it is honored
// and resolved to UTC right here — unlike the bias-less form.
Assert.Equal(expectedOffsetMinutes, entry.BiasMinutes);
Assert.NotNull(entry.TimestampUtc);
Assert.Equal(
new DateTime(2026, 7, 29, 11, 46, 19, DateTimeKind.Utc).AddTicks(226610).AddMinutes(expectedOffsetMinutes),
entry.Timestamp);
Assert.Equal(DateTimeKind.Utc, entry.Timestamp.Kind);
entry.TimestampUtc!.Value);
Assert.Equal(DateTimeKind.Utc, entry.TimestampUtc!.Value.Kind);
}
// ── Oversized line → parses, no throw, no truncation of the message ───
@@ -0,0 +1,190 @@
using System;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// Pins the measurement contract of <see cref="CmTraceOffsetCalibrator"/>.
///
/// <para>
/// The cases are taken from real sessions: +1 h (GMT to W. Europe, session bfff9b1c) and
/// -17 h (a writer left on the Pacific OOBE default while the agent had moved to
/// E. Australia). The backlog case is the one the residual guard cannot catch on its own and
/// is the reason the freshness rule exists.
/// </para>
/// </summary>
public class CmTraceOffsetCalibratorTests
{
private const string Source = "IntuneManagementExtension.log";
private static readonly DateTime UtcNow = new DateTime(2026, 08, 20, 07, 31, 46, DateTimeKind.Utc);
/// <summary>A line written "now" by a writer that believes it is at <paramref name="offset"/>.</summary>
private static DateTime LineWrittenNow(TimeSpan offset, DateTime utcNow)
=> DateTime.SpecifyKind(utcNow + offset, DateTimeKind.Unspecified);
[Theory]
[InlineData(2)] // CEST — the bfff9b1c writer
[InlineData(1)] // BST — what the agent wrongly assumed there
[InlineData(-7)] // PDT — the OOBE default seen in the -17 h sessions
[InlineData(10)] // E. Australia
[InlineData(14)] // Line Islands, the upper bound
[InlineData(-12)] // Dateline, the lower bound
public void MeasuresWholeHourOffsets(int hours)
{
var sut = new CmTraceOffsetCalibrator();
var offset = TimeSpan.FromHours(hours);
Assert.True(sut.TryCalibrate(Source, LineWrittenNow(offset, UtcNow), UtcNow));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(offset, measured);
}
[Theory]
[InlineData(330)] // India, UTC+5:30
[InlineData(345)] // Nepal, UTC+5:45
[InlineData(570)] // Adelaide, UTC+9:30 — the +30 min sessions in the field data
public void MeasuresQuarterHourOffsets(int minutes)
{
var sut = new CmTraceOffsetCalibrator();
var offset = TimeSpan.FromMinutes(minutes);
Assert.True(sut.TryCalibrate(Source, LineWrittenNow(offset, UtcNow), UtcNow));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(offset, measured);
}
[Fact]
public void AbsorbsPollAndFlushLatency()
{
// The line was written a moment before we parsed it. That lag must vanish into the
// grid rather than shift the measured offset.
var sut = new CmTraceOffsetCalibrator();
var lineLocal = LineWrittenNow(TimeSpan.FromHours(2), UtcNow.AddMilliseconds(-450));
Assert.True(sut.TryCalibrate(Source, lineLocal, UtcNow));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(TimeSpan.FromHours(2), measured);
}
[Fact]
public void RejectsBacklogLine_EvenWhenItRoundsCleanly()
{
// THE case the residual guard cannot catch alone: a line exactly 30 minutes old is a
// clean multiple of the grid, so it rounds with zero residual — and would calibrate
// the offset half an hour wrong. Only the caller's freshness rule prevents this from
// ever being offered; this test pins that a 30-minute-old line does not silently
// produce the true offset.
var sut = new CmTraceOffsetCalibrator();
var thirtyMinutesOld = LineWrittenNow(TimeSpan.FromHours(2), UtcNow.AddMinutes(-30));
Assert.True(sut.TryCalibrate(Source, thirtyMinutesOld, UtcNow));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(TimeSpan.FromMinutes(90), measured);
Assert.NotEqual(TimeSpan.FromHours(2), measured);
}
[Fact]
public void RejectsOffGridCandidate()
{
// 7 minutes off the grid: not fresh, and not a timezone either.
var sut = new CmTraceOffsetCalibrator();
var offGrid = LineWrittenNow(TimeSpan.FromHours(2), UtcNow.AddMinutes(-7));
Assert.False(sut.TryCalibrate(Source, offGrid, UtcNow));
TimeSpan ignored;
Assert.False(sut.TryGetOffset(Source, out ignored));
}
[Fact]
public void RejectsOffsetOutsideAnyRealTimezone()
{
var sut = new CmTraceOffsetCalibrator();
var absurd = LineWrittenNow(TimeSpan.FromHours(20), UtcNow);
Assert.False(sut.TryCalibrate(Source, absurd, UtcNow));
}
[Fact]
public void SelfHeals_WhenTheWritingProcessPicksUpANewZone()
{
// The writer restarts mid-session and now believes CEST instead of the stale BST.
var sut = new CmTraceOffsetCalibrator();
Assert.True(sut.TryCalibrate(Source, LineWrittenNow(TimeSpan.FromHours(1), UtcNow), UtcNow));
var later = UtcNow.AddMinutes(5);
Assert.True(sut.TryCalibrate(Source, LineWrittenNow(TimeSpan.FromHours(2), later), later));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(TimeSpan.FromHours(2), measured);
}
[Fact]
public void IgnoresAnchorOlderThanTheCurrentOne()
{
var sut = new CmTraceOffsetCalibrator();
Assert.True(sut.TryCalibrate(Source, LineWrittenNow(TimeSpan.FromHours(2), UtcNow), UtcNow));
// An out-of-order line from before the current anchor must not move the offset back.
var earlier = UtcNow.AddMinutes(-10);
Assert.False(sut.TryCalibrate(Source, LineWrittenNow(TimeSpan.FromHours(1), earlier), earlier));
TimeSpan measured;
Assert.True(sut.TryGetOffset(Source, out measured));
Assert.Equal(TimeSpan.FromHours(2), measured);
}
[Fact]
public void KeepsSourcesIndependent()
{
// IntuneManagementExtension.log and AppWorkload.log are written by different
// processes and may hold different beliefs.
var sut = new CmTraceOffsetCalibrator();
Assert.True(sut.TryCalibrate("IntuneManagementExtension.log", LineWrittenNow(TimeSpan.FromHours(2), UtcNow), UtcNow));
Assert.True(sut.TryCalibrate("AppWorkload.log", LineWrittenNow(TimeSpan.FromHours(-7), UtcNow), UtcNow));
TimeSpan ime, appWorkload;
Assert.True(sut.TryGetOffset("IntuneManagementExtension.log", out ime));
Assert.True(sut.TryGetOffset("AppWorkload.log", out appWorkload));
Assert.Equal(TimeSpan.FromHours(2), ime);
Assert.Equal(TimeSpan.FromHours(-7), appWorkload);
}
[Fact]
public void ResolvesTheRealFieldLine_FromSessionBfff9b1c()
{
// Raw line: <![LOG[Agent version is: 1.104.102.0]LOG]!><time="09:31:46.5192284" ...>
// Server-side ReceivedAt puts the true UTC at 07:31:46.4. The old code produced
// 08:31:46.5192284 — local minus the agent's stale +1 h belief.
var sut = new CmTraceOffsetCalibrator();
var trueUtc = new DateTime(2026, 08, 20, 07, 31, 46, 400, DateTimeKind.Utc);
var lineLocal = new DateTime(2026, 08, 20, 09, 31, 46, 519, DateTimeKind.Unspecified);
Assert.True(sut.TryCalibrate(Source, lineLocal, trueUtc));
DateTime resolved;
Assert.True(sut.TryResolveUtc(Source, lineLocal, out resolved));
Assert.Equal(DateTimeKind.Utc, resolved.Kind);
Assert.Equal(new DateTime(2026, 08, 20, 07, 31, 46, 519, DateTimeKind.Utc), resolved);
}
[Fact]
public void ReportsUncalibratedSource_SoTheCallerCanFlagIt()
{
var sut = new CmTraceOffsetCalibrator();
DateTime resolved;
Assert.False(sut.TryResolveUtc(Source, UtcNow, out resolved));
}
}
}
@@ -0,0 +1,441 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Logging;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// Per-line self-anchoring — the era-aware successor to the reverted per-file offset
/// application (04b1a7c6).
///
/// <para>
/// The property under test: a line read in a provably FRESH pass resolves to the instant it
/// was written, whatever timezone its writer believed — and neighbouring lines from writers
/// with DIFFERENT beliefs each resolve to their own era, because no cross-line state is
/// involved. That interleaving (fixture: AgentExecutor.log with PDT and CEST children in one
/// file, flipping both ways) is exactly what broke every cross-line anchor design.
/// </para>
///
/// <para>
/// The freshness boundary is equally load-bearing: backlog (first sight, restart catch-up,
/// stalled polls) must NEVER anchor, because an old line's age can round onto the 15-minute
/// offset grid. Those paths keep the uniform reader-zone fallback.
/// </para>
/// </summary>
public sealed class ImeLogTrackerLineAnchoringTests
{
private static readonly DateTime T0 = new DateTime(2026, 8, 20, 12, 13, 0, DateTimeKind.Utc);
private static List<ImeLogPattern> Patterns() => new List<ImeLogPattern>
{
new ImeLogPattern
{
PatternId = "T-ALL", Category = "always", Enabled = true,
Pattern = @"marker (?<n>\d+)",
Action = "noop",
Parameters = new Dictionary<string, string>(),
},
};
/// <summary>A CMTrace line as a writer holding <paramref name="writerOffset"/> renders instant <paramref name="utcInstant"/>.</summary>
private static string Line(string message, DateTime utcInstant, TimeSpan writerOffset)
{
var local = utcInstant + writerOffset;
return $"<![LOG[{message}]LOG]!><time=\"{local:HH:mm:ss.fffffff}\" date=\"{local:M-d-yyyy}\" " +
"component=\"X\" context=\"\" type=\"1\" thread=\"1\" file=\"\">";
}
private sealed class Harness : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
public ImeLogTracker Tracker { get; }
public DateTime Now { get; set; } = T0;
public Harness(string? stateDirectory = null)
{
Tracker = new ImeLogTracker(
logFolder: _tmp.Path,
patterns: Patterns(),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
stateDirectory: stateDirectory);
Tracker.UtcNowProvider = () => Now;
}
public string PathOf(string fileName) => Path.Combine(_tmp.Path, fileName);
public void Append(string fileName, string line)
=> File.AppendAllText(PathOf(fileName), line + Environment.NewLine);
public Task Pass() => Tracker.CheckLogFilesAsync(CancellationToken.None);
public void Dispose()
{
Tracker.Dispose();
_tmp.Dispose();
}
}
[Theory]
// The writer's belief. A fresh line must resolve to the instant it was written,
// WHATEVER that belief is — including agreement with this process (the majority case,
// which a fix to the agent's own zone cache would have broken).
[InlineData(2)] // CEST
[InlineData(1)] // BST
[InlineData(0)] // UTC
[InlineData(-7)] // PDT — the OOBE default behind the -9 h and -17 h field cases
[InlineData(10)] // E. Australia
public async Task FreshLine_ResolvesToItsTrueInstant_ForAnyWriterBelief(int writerOffsetHours)
{
var writerOffset = TimeSpan.FromHours(writerOffsetHours);
using var h = new Harness();
// First sight: backlog, must not anchor.
h.Append("IntuneManagementExtension.log", Line("marker 1", h.Now, writerOffset));
await h.Pass();
h.Now = T0.AddSeconds(10);
var writtenAt = h.Now.AddMilliseconds(-80); // written just before the poll sees it
h.Append("IntuneManagementExtension.log", Line("marker 2", writtenAt, writerOffset));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
Assert.Equal(writtenAt, h.Tracker.LastMatchedLogTimestamp);
Assert.Equal(writerOffsetHours * 60, h.Tracker.LastMatchedSourceOffsetMinutes);
}
[Fact]
public async Task InterleavedEras_InOneGrowingPass_EachLineResolvesToItsOwnEra()
{
// The e9753578 file shape: two writer processes with different zone beliefs
// interleaved in ONE file. Any per-file offset gets one of them wrong by 9 h;
// per-line anchoring must get both right within the same chunk.
var pdt = TimeSpan.FromHours(-7);
var cest = TimeSpan.FromHours(2);
using var h = new Harness();
h.Append("AgentExecutor.log", Line("marker 1", h.Now, pdt));
await h.Pass();
h.Now = T0.AddSeconds(10);
var wrotePdt = h.Now.AddMilliseconds(-90);
var wroteCest = h.Now.AddMilliseconds(-50);
h.Append("AgentExecutor.log", Line("marker 2", wrotePdt, pdt));
h.Append("AgentExecutor.log", Line("marker 3", wroteCest, cest));
var resolved = new List<(DateTime? Ts, CmTraceOffsetOrigin Origin, int? Offset)>();
Action<string> capture = _ => resolved.Add((
h.Tracker.LastMatchedLogTimestamp,
h.Tracker.LastMatchedSourceOffsetOrigin,
h.Tracker.LastMatchedSourceOffsetMinutes));
h.Tracker.OnPatternMatched += capture;
try
{
await h.Pass();
}
finally
{
h.Tracker.OnPatternMatched -= capture;
}
Assert.Equal(2, resolved.Count);
Assert.Equal(wrotePdt, resolved[0].Ts);
Assert.Equal(-420, resolved[0].Offset);
Assert.Equal(wroteCest, resolved[1].Ts);
Assert.Equal(120, resolved[1].Offset);
Assert.All(resolved, r => Assert.Equal(CmTraceOffsetOrigin.LineAnchored, r.Origin));
}
[Fact]
public async Task FirstSightBacklog_FallsBackToReaderZone()
{
using var h = new Harness();
h.Append("IntuneManagementExtension.log", Line("marker 1", h.Now.AddHours(-3), TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.None, h.Tracker.LastMatchedSourceOffsetOrigin);
}
[Fact]
public async Task FileFirstSeenEmpty_ItsFirstContentIsFresh()
{
// AgentExecutor.log in the field: the file APPEARS mid-enrollment. Seeing it empty
// (or not existing content-wise) counts as an observation, so its very first lines
// are provably written between passes and may anchor. This closes the warm-up gap
// for exactly the file whose eras flip the most.
using var h = new Harness();
File.WriteAllText(h.PathOf("AgentExecutor.log"), string.Empty);
await h.Pass();
h.Now = T0.AddSeconds(5);
var writtenAt = h.Now.AddMilliseconds(-60);
h.Append("AgentExecutor.log", Line("marker 1", writtenAt, TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
Assert.Equal(writtenAt, h.Tracker.LastMatchedLogTimestamp);
}
[Fact]
public async Task PollGapBeyondFreshWindow_FallsBack()
{
// A stalled agent (GC, CPU starvation) re-polls after a long gap: the new bytes span
// the whole gap and their age is unbounded, so the pass must not anchor — even
// though most of its lines may in fact be recent. Conservative by design.
using var h = new Harness();
h.Append("IntuneManagementExtension.log", Line("marker 1", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
h.Now = T0.AddSeconds(10);
h.Append("IntuneManagementExtension.log", Line("marker 2", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
// Gap far beyond FreshLineMaxAge.
h.Now = h.Now + ImeLogTracker.FreshLineMaxAge + TimeSpan.FromSeconds(10);
h.Append("IntuneManagementExtension.log", Line("marker 3", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.None, h.Tracker.LastMatchedSourceOffsetOrigin);
// And the very next tight poll is fresh again.
h.Now = h.Now.AddSeconds(1);
h.Append("IntuneManagementExtension.log", Line("marker 4", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
}
[Fact]
public async Task RestartWithRestoredBookmark_FirstPassIsBacklog_SecondPassAnchors()
{
// The persisted position bookmark survives a restart; the freshness guarantee must
// not — the first pass after a restart reads the downtime backlog, whose lines can
// be arbitrarily old.
using var stateDir = new TempDirectory();
using var h = new Harness(stateDirectory: stateDir.Path);
h.Append("IntuneManagementExtension.log", Line("marker 1", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
h.Tracker.SaveStateForTest();
// "Restart": a second tracker over the same folder restores the bookmark.
using var h2 = new HarnessOver(h, stateDir.Path);
h2.Tracker.LoadStateForTest();
// Backlog written during the downtime — 30 min old, exactly on the offset grid,
// the nastiest possible backlog shape. Must NOT anchor.
h2.Now = T0.AddMinutes(30);
h2.Append(Line("marker 2", T0.AddSeconds(20), TimeSpan.FromHours(2)));
await h2.Pass();
Assert.Equal(CmTraceOffsetOrigin.None, h2.Tracker.LastMatchedSourceOffsetOrigin);
// The next tight poll is fresh again.
h2.Now = h2.Now.AddSeconds(1);
var writtenAt = h2.Now.AddMilliseconds(-70);
h2.Append(Line("marker 3", writtenAt, TimeSpan.FromHours(2)));
await h2.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h2.Tracker.LastMatchedSourceOffsetOrigin);
Assert.Equal(writtenAt, h2.Tracker.LastMatchedLogTimestamp);
}
/// <summary>Second tracker over the FIRST harness's log folder — the "restarted agent".</summary>
private sealed class HarnessOver : IDisposable
{
private readonly string _logFolder;
public ImeLogTracker Tracker { get; }
public DateTime Now { get; set; }
public HarnessOver(Harness original, string stateDirectory)
{
_logFolder = Path.GetDirectoryName(original.PathOf("x"));
Tracker = new ImeLogTracker(
logFolder: _logFolder,
patterns: Patterns(),
logger: new AgentLogger(_logFolder, AgentLogLevel.Info),
stateDirectory: stateDirectory);
Tracker.UtcNowProvider = () => Now;
}
public void Append(string line)
=> File.AppendAllText(Path.Combine(_logFolder, "IntuneManagementExtension.log"), line + Environment.NewLine);
public Task Pass() => Tracker.CheckLogFilesAsync(CancellationToken.None);
public void Dispose() => Tracker.Dispose();
}
[Fact]
public async Task FreshlyWrittenReplayLine_OnExactGridAge_ResolvesToRoughlyNow()
{
// Characterization of the one accepted edge: a line freshly WRITTEN but carrying a
// REPLAYED timestamp whose age is exactly a grid multiple (here 30 min) measures
// that age as an "offset" and resolves to ~now instead of its replayed past. The
// error is bounded by construction (an anchored line always lands at now ± 2 min),
// and sub-24h replays are treated as current by the historic-replay guard anyway.
using var h = new Harness();
h.Append("IntuneManagementExtension.log", Line("marker 1", h.Now, TimeSpan.Zero));
await h.Pass();
h.Now = T0.AddSeconds(10);
var replayedInstant = h.Now.AddMinutes(-30);
h.Append("IntuneManagementExtension.log", Line("marker 2", replayedInstant, TimeSpan.Zero));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
Assert.NotNull(h.Tracker.LastMatchedLogTimestamp);
var error = (h.Tracker.LastMatchedLogTimestamp!.Value - h.Now).Duration();
Assert.True(error <= CmTraceOffsetCalibrator.MaxGridResidual,
$"anchored line must resolve to now ± residual, was off by {error}");
}
// =====================================================================
// The committed field fixture — session e9753578, ImeLogs/AgentExecutor.log.
// One file, two writer eras (PDT hour-05 / CEST hour-14), interleaved, flipping
// BOTH ways. Replayed here as the live tail the field agent saw: entry by entry,
// with the clock following the true write time.
// =====================================================================
private static string FixturePath()
{
var dir = AppContext.BaseDirectory;
for (int i = 0; i < 12; i++)
{
if (File.Exists(Path.Combine(dir, "AutopilotMonitor.sln")))
return Path.Combine(dir, "tests", "fixtures", "cmtrace-logs",
"agentexecutor-two-writer-eras-v1.cmtrace");
var parent = Directory.GetParent(dir)?.FullName;
if (parent == null || parent == dir) break;
dir = parent;
}
throw new DirectoryNotFoundException(
"Could not locate repo root (AutopilotMonitor.sln) walking up from " + AppContext.BaseDirectory);
}
/// <summary>Group the fixture's physical lines into complete CMTrace entries (multiline-safe).</summary>
private static List<string> FixtureEntries()
{
var entries = new List<string>();
string? pending = null;
foreach (var rawWithBom in File.ReadAllLines(FixturePath()))
{
// The fixture keeps the original file's BOM verbatim — it sat at the START of
// the real AgentExecutor.log, which is mid-file in this excerpt.
var raw = rawWithBom.TrimStart('');
if (pending == null)
{
if (raw.Length == 0 || raw[0] == '#') continue;
if (!raw.StartsWith("<![LOG[", StringComparison.Ordinal)) continue;
if (raw.Contains("]LOG]!>")) { entries.Add(raw); continue; }
pending = raw;
}
else
{
pending += "\n" + raw;
if (raw.Contains("]LOG]!>")) { entries.Add(pending!); pending = null; }
}
}
return entries;
}
[Fact]
public async Task TwoWriterEraFixture_EveryLineOfBothErasResolvesToItsTrueInstant()
{
// The committed fixture is a curated, verbatim excerpt of the transition window
// (commit policy: anonymized, versioned) — 20 entries covering both eras and the flip.
var entries = FixtureEntries();
Assert.True(entries.Count >= 15, $"fixture unexpectedly small: {entries.Count} entries");
using var h = new Harness();
// Match every fixture entry.
h.Tracker.CompilePatterns(new List<ImeLogPattern>
{
new ImeLogPattern
{
PatternId = "T-ANY", Category = "always", Enabled = true,
Pattern = ".+", Action = "noop",
Parameters = new Dictionary<string, string>(),
},
});
// The file exists (empty) before its first content — first sight is the observation
// baseline, everything after is a fresh tail, exactly like the field.
File.WriteAllText(h.PathOf("AgentExecutor.log"), string.Empty);
h.Now = new DateTime(2026, 8, 20, 12, 13, 30, DateTimeKind.Utc);
await h.Pass();
var anchored = 0;
var resolvedList = new List<(DateTime Resolved, DateTime Expected)>();
Action<string> capture = _ =>
{
if (h.Tracker.LastMatchedSourceOffsetOrigin == CmTraceOffsetOrigin.LineAnchored)
anchored++;
if (h.Tracker.LastMatchedLogTimestamp.HasValue && _expectedForCurrentEntry.HasValue)
resolvedList.Add((h.Tracker.LastMatchedLogTimestamp.Value, _expectedForCurrentEntry.Value));
};
h.Tracker.OnPatternMatched += capture;
try
{
foreach (var entry in entries)
{
CmTraceLogEntry? parsed;
Assert.True(CmTraceLogParser.TryParseLine(entry, out parsed), "fixture entry did not parse");
// Era by local hour: the fixture holds exactly two eras — PDT (UTC-7) lines
// render at local hour 05, CEST (UTC+2) lines at hour 14 (fixture header).
var eraOffset = parsed.LocalTimestamp.Hour == 14
? TimeSpan.FromHours(2)
: TimeSpan.FromHours(-7);
var trueUtc = DateTime.SpecifyKind(parsed.LocalTimestamp - eraOffset, DateTimeKind.Utc);
// The clock follows the write; never move backwards (interleaved eras are
// written in true-UTC order, but guard against sub-ms jitter).
var readAt = trueUtc.AddMilliseconds(120);
if (readAt > h.Now) h.Now = readAt;
// In production the 100 ms poll keeps running through write silences — every
// empty look refreshes the freshness window. The fixture's transition window
// spans multi-minute write gaps, so replay that cadence with one empty poll
// just before each append; without it the NEXT pass would (correctly!)
// classify the gap as a stall and refuse to anchor.
h.Now = h.Now.AddMilliseconds(-100);
await h.Pass();
h.Now = readAt;
_expectedForCurrentEntry = trueUtc;
File.AppendAllText(h.PathOf("AgentExecutor.log"), entry + Environment.NewLine);
await h.Pass();
_expectedForCurrentEntry = null;
}
}
finally
{
h.Tracker.OnPatternMatched -= capture;
}
// Every matched entry resolved to the instant its writer rendered — across BOTH
// eras, including the flips. This is the exact property whose absence produced the
// -9 h script durations.
Assert.Equal(entries.Count, resolvedList.Count);
Assert.All(resolvedList, r => Assert.Equal(r.Expected, r.Resolved));
Assert.True(anchored == resolvedList.Count,
$"expected every entry line-anchored, got {anchored}/{resolvedList.Count}");
// Both eras genuinely exercised: PDT entries render at local hour 05, CEST at 14.
Assert.Contains(resolvedList, r => (r.Expected + TimeSpan.FromHours(-7)).Hour == 5);
Assert.Contains(resolvedList, r => (r.Expected + TimeSpan.FromHours(2)).Hour == 14);
}
private DateTime? _expectedForCurrentEntry;
}
}
@@ -0,0 +1,275 @@
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Ime
{
/// <summary>
/// End-to-end offset calibration through a real read cycle.
///
/// <para>
/// A CMTrace line states local time and nothing else, so its UTC value depends on which zone
/// the WRITING process believed it was in. The tracker used to substitute its own belief,
/// which is right only when the two happen to agree.
/// </para>
///
/// <para>
/// These tests are deliberately independent of the machine they run on: the assertion is
/// always "a line written at instant T resolves to T", for every writer belief. That single
/// property covers both field cases at once — the writer disagreeing with us (the bug) and
/// the writer agreeing with us (the majority, which a naive fix would have broken).
/// </para>
/// </summary>
public sealed class ImeLogTrackerOffsetCalibrationTests
{
private const string LogFileName = "IntuneManagementExtension.log";
private static readonly DateTime T0 = new DateTime(2026, 8, 20, 7, 30, 0, DateTimeKind.Utc);
private static List<ImeLogPattern> Patterns() => new List<ImeLogPattern>
{
new ImeLogPattern
{
PatternId = "T-MARK", Category = "always", Enabled = true,
Pattern = @"marker (?<n>\d+)",
Action = "noop",
Parameters = new Dictionary<string, string>(),
},
};
/// <summary>A CMTrace line as a writer holding <paramref name="writerOffset"/> renders instant <paramref name="utcInstant"/>.</summary>
private static string Line(string message, DateTime utcInstant, TimeSpan writerOffset)
{
var local = utcInstant + writerOffset;
return $"<![LOG[{message}]LOG]!><time=\"{local:HH:mm:ss.fffffff}\" date=\"{local:M-d-yyyy}\" " +
"component=\"IntuneManagementExtension\" context=\"\" type=\"1\" thread=\"1\" file=\"\">";
}
private sealed class Harness : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
public ImeLogTracker Tracker { get; }
public DateTime Now { get; set; } = T0;
public Harness()
{
Tracker = new ImeLogTracker(
logFolder: _tmp.Path,
patterns: Patterns(),
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info));
Tracker.UtcNowProvider = () => Now;
}
public string LogPath => Path.Combine(_tmp.Path, LogFileName);
public void Append(string line) => File.AppendAllText(LogPath, line + Environment.NewLine);
public Task Pass() => Tracker.CheckLogFilesAsync(CancellationToken.None);
public void Dispose()
{
Tracker.Dispose();
_tmp.Dispose();
}
}
[Theory]
// The writer's belief. The MEASUREMENT must recover it exactly, whatever it is.
[InlineData(2)] // CEST
[InlineData(1)] // BST
[InlineData(-7)] // PDT — the OOBE default behind the -17 h and -9 h field cases
[InlineData(10)] // E. Australia
[InlineData(0)] // UTC
public async Task MeasuresTheWritersOffsetExactly_ForAnyBelief(int writerOffsetHours)
{
// NOTE: the per-FILE measurement asserted here is OBSERVATIONAL — its application was
// reverted on 2026-08-20 (04b1a7c6) because one offset per file is wrong when a file
// holds interleaved writer eras. Resolution now anchors per LINE instead (see
// ImeLogTrackerLineAnchoringTests, which restores the "a line written at T resolves
// to T" property era-safely). The per-file measurement stays for the Info log and
// measuredWriterOffsetMinutes provenance.
var writerOffset = TimeSpan.FromHours(writerOffsetHours);
using var h = new Harness();
h.Append(Line("marker 1", h.Now, writerOffset));
await h.Pass();
h.Now = T0.AddSeconds(10);
h.Append(Line("marker 2", h.Now, writerOffset));
await h.Pass();
TimeSpan measured;
Assert.True(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out measured));
Assert.Equal(writerOffset, measured);
}
[Fact]
public async Task MeasuresTheWritersOffset_NotItsOwn()
{
using var h = new Harness();
var writerOffset = TimeSpan.FromHours(2);
h.Append(Line("marker 1", h.Now, writerOffset));
await h.Pass();
h.Now = T0.AddSeconds(10);
h.Append(Line("marker 2", h.Now, writerOffset));
await h.Pass();
TimeSpan measured;
Assert.True(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out measured));
Assert.Equal(writerOffset, measured);
}
[Fact]
public async Task DoesNotCalibrateOnFirstSightOfAFile()
{
// The first pass reads from position 0, so the newest line in the file may be hours
// old. Anchoring on it would measure the line's age instead of the writer's offset.
using var h = new Harness();
h.Append(Line("marker 1", T0.AddHours(-3), TimeSpan.FromHours(2)));
await h.Pass();
TimeSpan ignored;
Assert.False(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out ignored));
}
[Fact]
public async Task FollowsTheWriterAcrossARestartIntoANewZone()
{
// IME restarts mid-enrollment and picks up the timezone the OS moved to. The
// measurement has to follow, otherwise every later line is off by the difference.
using var h = new Harness();
h.Append(Line("marker 1", h.Now, TimeSpan.FromHours(1)));
await h.Pass();
h.Now = T0.AddSeconds(10);
h.Append(Line("marker 2", h.Now, TimeSpan.FromHours(1)));
await h.Pass();
TimeSpan before;
Assert.True(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out before));
Assert.Equal(TimeSpan.FromHours(1), before);
// Writer restarts, now on +2.
h.Now = T0.AddSeconds(20);
h.Append(Line("marker 3", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
h.Now = T0.AddSeconds(30);
var writtenAt = h.Now;
h.Append(Line("marker 4", writtenAt, TimeSpan.FromHours(2)));
await h.Pass();
TimeSpan after;
Assert.True(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out after));
Assert.Equal(TimeSpan.FromHours(2), after);
// The measurement follows the restart. Applying it is what the 2026-08-20 revert
// removed — that is exactly the case a file with two writer eras breaks.
Assert.Equal(120, h.Tracker.LastMatchedMeasuredWriterOffsetMinutes);
}
[Fact]
public async Task HonorsAWriterDeclaredBiasWithoutMeasuring()
{
// A line carrying its own bias states the offset outright; it must be used as-is and
// must never become a calibration anchor.
using var h = new Harness();
h.Append(Line("marker 1", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
h.Now = T0.AddSeconds(10);
var writtenAt = h.Now;
var local = writtenAt + TimeSpan.FromHours(-8); // a PST writer
h.Append($"<![LOG[marker 2]LOG]!><time=\"{local:HH:mm:ss.fffffff}+480\" date=\"{local:M-d-yyyy}\" " +
"component=\"IntuneManagementExtension\" context=\"\" type=\"1\" thread=\"1\" file=\"\">");
await h.Pass();
Assert.True(h.Tracker.LastMatchedLogTimestamp.HasValue);
Assert.Equal(writtenAt, h.Tracker.LastMatchedLogTimestamp.Value);
TimeSpan ignored;
Assert.False(h.Tracker.OffsetCalibrator.TryGetOffset(LogFileName, out ignored));
}
[Fact]
public async Task RecordsMeasuredOffsetSeparatelyFromTheAppliedOne()
{
// The regression that forced the revert shipped an event tagged "calibrated" while
// carrying an offset that did not hold for that line. Measured and applied are now
// separate fields so a future reader cannot mistake one for the other.
using var h = new Harness();
var writerOffset = TimeSpan.FromHours(-7); // PDT, the OOBE default
h.Append(Line("marker 1", h.Now, writerOffset));
await h.Pass();
h.Now = T0.AddSeconds(10);
h.Append(Line("marker 2", h.Now, writerOffset));
await h.Pass();
h.Now = T0.AddSeconds(20);
h.Append(Line("marker 3", h.Now, writerOffset));
await h.Pass();
// Measured: the writer's actual belief, per file.
Assert.Equal(-420, h.Tracker.LastMatchedMeasuredWriterOffsetMinutes);
// Applied: the line's OWN anchor (fresh pass), never the per-file "calibrated" value —
// the two must stay distinguishable even when they agree numerically.
Assert.Equal(CmTraceOffsetOrigin.LineAnchored, h.Tracker.LastMatchedSourceOffsetOrigin);
Assert.NotEqual(CmTraceOffsetOrigin.Calibrated, h.Tracker.LastMatchedSourceOffsetOrigin);
}
[Fact]
public async Task RecordsProvenance_AsFallback_BeforeAnyCalibration()
{
// The warm-up pass: the offset is not measured yet, so the reader zone is assumed and
// the event has to say so rather than look source-grounded.
using var h = new Harness();
h.Append(Line("marker 1", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.None, h.Tracker.LastMatchedSourceOffsetOrigin);
Assert.True(h.Tracker.LastMatchedSourceLocalTimestamp.HasValue);
// The APPLIED offset is reported even on the fallback path — an event that states no
// offset at all cannot be recomputed later, which defeats the purpose of recording
// provenance. Asserted against the running machine's zone so this stays portable.
var readerZone = (int)TimeZoneInfo.Local
.GetUtcOffset(h.Tracker.LastMatchedSourceLocalTimestamp!.Value).TotalMinutes;
Assert.Equal(readerZone, h.Tracker.LastMatchedSourceOffsetMinutes);
}
[Fact]
public async Task RecordsProvenance_ForADeclaredBias()
{
using var h = new Harness();
h.Append(Line("marker 1", h.Now, TimeSpan.FromHours(2)));
await h.Pass();
h.Now = T0.AddSeconds(10);
var local = h.Now + TimeSpan.FromHours(-8);
h.Append($"<![LOG[marker 2]LOG]!><time=\"{local:HH:mm:ss.fffffff}+480\" date=\"{local:M-d-yyyy}\" " +
"component=\"IntuneManagementExtension\" context=\"\" type=\"1\" thread=\"1\" file=\"\">");
await h.Pass();
Assert.Equal(CmTraceOffsetOrigin.Bias, h.Tracker.LastMatchedSourceOffsetOrigin);
// Reported in the same sense as a measured offset (local = UTC + offset), so the two
// origins are directly comparable; the wire bias uses the opposite convention.
Assert.Equal(-480, h.Tracker.LastMatchedSourceOffsetMinutes);
}
}
}
@@ -56,6 +56,42 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Office
Assert.Equal(1, loaded.PeakDo.DownloadMode);
}
[Fact]
public void Save_then_load_roundtrips_terminal_metadata()
{
// Session bc803955 — CompletedAtUtc/VersionReached feed the summary dialog's
// synthetic Office row (duration + version).
using var tmp = new TempDirectory();
var sut = new OfficeInstallStatePersistence(tmp.Path, NewLogger(tmp.Path));
var completedAt = new DateTime(2026, 6, 8, 12, 3, 22, DateTimeKind.Utc);
sut.Save(new OfficeInstallStateData
{
State = OfficeInstallStateData.StateCompleted,
CompletedAtUtc = completedAt,
VersionReached = "16.0.20228.20190",
});
var loaded = sut.Load();
Assert.Equal(completedAt, loaded!.CompletedAtUtc);
Assert.Equal("16.0.20228.20190", loaded.VersionReached);
}
[Fact]
public void Load_tolerates_state_files_from_older_agents_without_terminal_metadata()
{
using var tmp = new TempDirectory();
var sut = new OfficeInstallStatePersistence(tmp.Path, NewLogger(tmp.Path));
// Pre-terminal-metadata wire format (verbatim shape from a live 2.0.1404 session).
File.WriteAllText(sut.StateFilePath,
"{\"State\":\"Completed\",\"StartedAtUtc\":\"2026-08-18T07:56:16.8208141Z\",\"StartedTrigger\":\"registry\"}");
var loaded = sut.Load();
Assert.Equal(OfficeInstallStateData.StateCompleted, loaded!.State);
Assert.Null(loaded.CompletedAtUtc);
Assert.Null(loaded.VersionReached);
}
[Fact]
public void Save_overwrites_a_previous_state()
{
@@ -46,7 +46,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
Directory.CreateDirectory(SpoolFolder);
}
public DiagnosticsPackageService Build()
public DiagnosticsPackageService Build(System.Action<AgentConfiguration>? mutateConfig = null)
{
// BackendApiClient is required by the public ctor but BuildArchiveBytes
// never touches it — construct with a throwaway HttpClient. Tests only
@@ -61,8 +61,10 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
bootstrapToken: null,
agentVersion: "0.0.0",
logger: Logger);
var cfg = Cfg();
mutateConfig?.Invoke(cfg);
return new DiagnosticsPackageService(
Cfg(),
cfg,
Logger,
apiClient,
agentLogFolderOverride: LogsTmp.Path,
@@ -131,6 +133,72 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Runtime
Assert.Contains("AgentMarkers/agent_clean_exit.marker", entries);
}
// ============================================================ package manifest ====
// The agent-log snapshot is zipped BEFORE packaging finishes, so packaging problems
// are invisible in the uploaded archive's own log (field case: missing evtx in
// sessions a11102f4/3ae7528b). The manifest travels INSIDE the ZIP instead.
private static string? ReadZipEntryText(byte[] bytes, string entryName)
{
using var ms = new MemoryStream(bytes);
using var archive = new System.IO.Compression.ZipArchive(ms, System.IO.Compression.ZipArchiveMode.Read);
var entry = archive.GetEntry(entryName);
if (entry == null) return null;
using var reader = new StreamReader(entry.Open());
return reader.ReadToEnd();
}
[Fact]
public void BuildArchiveBytes_always_writes_package_manifest_with_added_entries()
{
using var rig = new Rig();
File.WriteAllText(Path.Combine(rig.StateFolder, "snapshot.json"), "{}");
var bytes = rig.Build().BuildArchiveBytes(enrollmentSucceeded: true);
var manifest = ReadZipEntryText(bytes, "package-manifest.txt");
Assert.NotNull(manifest);
Assert.Contains("ADDED: AgentState/snapshot.json", manifest);
// Empty standard folders (e.g. ImeLogs) surface as NO MATCH lines — the silent
// nothing-found case is exactly what the manifest exists to make visible.
Assert.Contains("NO MATCH:", manifest);
}
[Fact]
public void BuildArchiveBytes_manifest_records_guard_blocked_additional_path()
{
using var rig = new Rig();
var blockedPath = @"C:\Windows\System32\config\SYSTEM";
var bytes = rig.Build(cfg => cfg.DiagnosticsLogPaths.Add(
new AutopilotMonitor.Shared.Models.DiagnosticsLogPath { Path = blockedPath }))
.BuildArchiveBytes(enrollmentSucceeded: true);
var manifest = ReadZipEntryText(bytes, "package-manifest.txt");
Assert.NotNull(manifest);
Assert.Contains($"BLOCKED (path guard): {blockedPath}", manifest);
}
[Fact]
public void BuildArchiveBytes_manifest_records_users_path_blocked_even_in_unrestricted()
{
using var rig = new Rig();
// The test temp dir lives under C:\Users — blocked by the always-on privacy guard
// even in unrestricted mode. The manifest must say so instead of staying silent.
var configuredPath = Path.Combine(rig.DataFolder, "does-not-exist.evtx");
var bytes = rig.Build(cfg =>
{
cfg.UnrestrictedMode = true;
cfg.DiagnosticsLogPaths.Add(new AutopilotMonitor.Shared.Models.DiagnosticsLogPath { Path = configuredPath });
})
.BuildArchiveBytes(enrollmentSucceeded: true);
var manifest = ReadZipEntryText(bytes, "package-manifest.txt");
Assert.NotNull(manifest);
Assert.Contains($"BLOCKED (path guard): {configuredPath}", manifest);
}
[Fact]
public void BuildArchiveBytes_excludes_top_level_session_id_and_bootstrap_config()
{
@@ -0,0 +1,322 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring
{
/// <summary>
/// Scope of the Shell-Core replay: it recovers the Hello-wizard start and NOTHING else.
/// <para>
/// The replay had existed since session 772fe502 with no caller. Wiring it up (2026-08-19)
/// activated three rails at once, and only one of them is safe to replay:
/// </para>
/// <list type="bullet">
/// <item><b>62404 Hello-wizard start — replayed.</b> A conservative fact: it vetoes a
/// premature "Hello is disabled" skip and can never by itself complete a session, so
/// replaying it can only make the agent wait longer, never finish early.</item>
/// <item><b>62407 ESP exit — never replayed.</b> Windows writes the identical description
/// for the intermediate DeviceSetup-to-AccountSetup transition and the final exit, so the
/// record carries no evidence of its own position; the reducer orders exits by ingest
/// ordinal, which for a replay is fresher than reality; and arm C of
/// ShouldTransitionToAwaitingHello opens on any arriving exit once the restored state
/// carries AccountSetupEntered + a genuine IME user session + desktop arrival. See
/// ClassicEspExitingOnRestoredStateTests for the reducer half of that proof.</item>
/// <item><b>62407 ESP failure — never replayed.</b> Re-injecting a historic failure as
/// fresh can fail a session that recovered (ANALYZE-ESP-006).</item>
/// </list>
/// </summary>
public sealed class ShellCoreTrackerReplayScopeTests
{
private static readonly DateTime ClockNow = new DateTime(2026, 8, 19, 8, 43, 16, DateTimeKind.Utc);
// The REAL Shell-Core wording. Byte-identical for the intermediate DeviceSetup-to-
// AccountSetup transition and for the final post-AccountSetup exit — that identity is the
// whole reason a replayed exit cannot be classified.
private const string EspExitDescription = "CommercialOOBE_ESPProgress_Page_Exiting";
private const string EspFailureDescription = "CommercialOOBE_ESPProgress_Failure";
private const string AadHelloDescription = "CloudExperienceHost web app activity: AADHello";
private const string OtherWebAppDescription = "CloudExperienceHost web app activity: SomethingElse";
private sealed class Fixture : IDisposable
{
public TempDirectory Tmp { get; } = new TempDirectory();
public FakeSignalIngressSink PostSink { get; } = new FakeSignalIngressSink();
public VirtualClock Clock { get; } = new VirtualClock(ClockNow);
public ShellCoreTracker Build() => new ShellCoreTracker(
sessionId: "S1",
tenantId: "T1",
post: new InformationalEventPost(PostSink, Clock),
logger: new AgentLogger(Tmp.Path, AgentLogLevel.Info),
helloTracker: null);
public List<FakeSignalIngressSink.PostedSignal> AgentTraces() =>
PostSink.Posted.Where(p =>
p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == "agent_trace").ToList();
public void Dispose() => Tmp.Dispose();
}
private static (int, string, DateTime) Record(int id, string description, DateTime at)
=> (id, description, at);
// ------------------------------------------------------------------ the wizard rail
[Fact]
public void The_hello_wizard_start_is_replayed_with_its_original_source_time()
{
// Session 772fe502: an agent restarted while the user sits inside the Hello wizard
// would otherwise never learn the wizard ran. This is the observation the replay
// exists for.
using var f = new Fixture();
using var tracker = f.Build();
var order = new List<string>();
HelloWizardStartedEventArgs? captured = null;
tracker.HelloWizardStarted += (_, args) => { order.Add("wizard"); captured = args; };
tracker.FinalizingSetupPhaseTriggered += (_, reason) => order.Add($"finalizing:{reason}");
var duringDowntime = ClockNow.AddMinutes(-37);
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppStarted, AadHelloDescription, duringDowntime),
});
Assert.Equal(new[] { "wizard", "finalizing:hello_wizard_started" }, order);
Assert.NotNull(captured);
// The historical time must survive the replay — collapsing to now would date the
// wizard after the restart and skew every duration derived from it.
Assert.Equal(duringDowntime, captured!.OccurredAtUtc);
}
[Fact]
public void The_wizard_replay_is_single_shot_across_repeated_records()
{
using var f = new Fixture();
using var tracker = f.Build();
var starts = 0;
tracker.HelloWizardStarted += (_, __) => starts++;
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppStarted, AadHelloDescription, ClockNow.AddMinutes(-30)),
Record(ShellCoreTracker.EventId_ShellCore_WebAppStarted, AadHelloDescription, ClockNow.AddMinutes(-20)),
});
Assert.Equal(1, starts);
}
[Fact]
public void A_non_hello_web_app_start_is_ignored()
{
using var f = new Fixture();
using var tracker = f.Build();
var starts = 0;
tracker.HelloWizardStarted += (_, __) => starts++;
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppStarted, OtherWebAppDescription, ClockNow.AddMinutes(-5)),
});
Assert.Equal(0, starts);
Assert.Empty(f.AgentTraces());
}
// ------------------------------------------------------------------ the exit rail
[Fact]
public void An_intermediate_only_window_raises_nothing_that_can_reach_the_reducer()
{
// THE regression this whole review chain is about: restart, and the only 62407 in the
// window is the historic DeviceSetup-to-AccountSetup handoff. Nothing may leave the
// tracker on a rail the decision engine listens to.
using var f = new Fixture();
using var tracker = f.Build();
var espExits = 0;
var finalizing = new List<string>();
var failures = 0;
tracker.EspExited += (_, __) => espExits++;
tracker.FinalizingSetupPhaseTriggered += (_, reason) => finalizing.Add(reason);
tracker.EspFailureDetected += (_, __) => failures++;
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitDescription, ClockNow.AddMinutes(-52)),
});
Assert.Equal(0, espExits);
Assert.Empty(finalizing);
Assert.Equal(0, failures);
}
[Fact]
public void Even_several_exits_in_the_window_stay_out_of_the_decision_stream()
{
// "Newest exit wins" was an earlier attempt at this problem. It does not help: if the
// final exit has not happened yet, the newest record in the window is still the
// intermediate one. Picking a record is the wrong lever — none of them is usable.
using var f = new Fixture();
using var tracker = f.Build();
var espExits = 0;
tracker.EspExited += (_, __) => espExits++;
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitDescription, ClockNow.AddMinutes(-52)),
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitDescription, ClockNow.AddMinutes(-9)),
});
Assert.Equal(0, espExits);
}
[Fact]
public void A_historic_esp_failure_is_never_re_injected()
{
// A session that failed and then recovered on retry (ANALYZE-ESP-006) must not be
// failed again by its own history.
using var f = new Fixture();
using var tracker = f.Build();
var failures = new List<string>();
tracker.EspFailureDetected += (_, failureType) => failures.Add(failureType);
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspFailureDescription, ClockNow.AddMinutes(-40)),
});
Assert.Empty(failures);
}
// ------------------------------------------------------------------ visibility
[Fact]
public void Skipped_records_are_reported_once_as_an_agent_trace()
{
// Dropping evidence silently is how this class of bug survives. The gap has to stay
// visible to whoever debugs the session later — as an informational event, which is
// decision-neutral by construction.
using var f = new Fixture();
using var tracker = f.Build();
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitDescription, ClockNow.AddMinutes(-52)),
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspFailureDescription, ClockNow.AddMinutes(-40)),
Record(ShellCoreTracker.EventId_ShellCore_WebAppEvent, EspExitDescription, ClockNow.AddMinutes(-9)),
});
var trace = Assert.Single(f.AgentTraces());
var data = Assert.IsAssignableFrom<IReadOnlyDictionary<string, object>>(trace.TypedPayload);
Assert.Equal(2, data["skippedEspExits"]);
Assert.Equal(1, data["skippedEspFailures"]);
Assert.Equal("replayed_62407_not_orderable", data["reason"]);
Assert.Equal(ClockNow.AddMinutes(-52).ToString("o"), data["oldestSkippedUtc"]);
Assert.Equal(ClockNow.AddMinutes(-9).ToString("o"), data["newestSkippedUtc"]);
}
[Fact]
public void A_wizard_only_window_reports_nothing_as_skipped()
{
using var f = new Fixture();
using var tracker = f.Build();
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>
{
Record(ShellCoreTracker.EventId_ShellCore_WebAppStarted, AadHelloDescription, ClockNow.AddMinutes(-30)),
});
Assert.Empty(f.AgentTraces());
}
[Fact]
public void An_empty_batch_is_a_no_op()
{
using var f = new Fixture();
using var tracker = f.Build();
var raised = 0;
tracker.EspExited += (_, __) => raised++;
tracker.HelloWizardStarted += (_, __) => raised++;
tracker.ReplayBackfillRecords(new List<(int, string, DateTime)>());
Assert.Equal(0, raised);
Assert.Empty(f.AgentTraces());
}
// ------------------------------------------------------------------ live path untouched
[Fact]
public void The_live_path_still_raises_the_exit_normally()
{
// Narrowing the REPLAY must not narrow live observation — that is the agent's primary
// signal source and the one the completion recovery actually runs on.
using var f = new Fixture();
using var tracker = f.Build();
var exits = new List<DateTime>();
tracker.EspExited += (_, args) => exits.Add(args.OccurredAtUtc);
tracker.ProcessEvent(
eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent,
description: EspExitDescription,
timestamp: ClockNow,
providerName: "Microsoft-Windows-Shell-Core",
isBackfill: false);
Assert.Equal(ClockNow, Assert.Single(exits));
}
[Fact]
public void The_live_path_still_raises_esp_failures()
{
using var f = new Fixture();
using var tracker = f.Build();
var failures = new List<string>();
tracker.EspFailureDetected += (_, failureType) => failures.Add(failureType);
tracker.ProcessEvent(
eventId: ShellCoreTracker.EventId_ShellCore_WebAppEvent,
description: EspFailureDescription,
timestamp: ClockNow,
providerName: "Microsoft-Windows-Shell-Core",
isBackfill: false);
Assert.Single(failures);
}
// ------------------------------------------------------------------ window clamp
[Theory]
[InlineData(0, 1)]
[InlineData(-42, 1)]
[InlineData(5, 5)]
[InlineData(ShellCoreTracker.BackfillLookbackMaxMinutes, ShellCoreTracker.BackfillLookbackMaxMinutes)]
[InlineData(ShellCoreTracker.BackfillLookbackMaxMinutes + 1, ShellCoreTracker.BackfillLookbackMaxMinutes)]
[InlineData(int.MaxValue, ShellCoreTracker.BackfillLookbackMaxMinutes)]
public void Lookback_is_clamped_to_a_sane_window(int requested, int expected)
{
// int.MaxValue matters: the query multiplies minutes by 60_000, so an unclamped value
// would overflow into a negative timediff and silently match nothing.
Assert.Equal(expected, ShellCoreTracker.ClampLookbackMinutes(requested));
}
}
}
@@ -0,0 +1,183 @@
#nullable enable
using System;
using System.Collections.Generic;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Selection/normalization logic of <see cref="ImeMsiInstallSourceProbe"/> — the registry
/// walk itself is not unit-testable (live HKLM), so tests cover the pure pieces:
/// <c>SelectBestMatch</c>, <c>VersionsEqual</c>, <c>IsImeMsiUrl</c>.
/// </summary>
public class ImeMsiInstallSourceProbeTests
{
private const string ImeProductCode = "{6A7DFC50-0395-4E1E-BF84-ED1404E72051}";
private const string ImeUrl = "https://imeswdb-afd-secondary.manage.microsoft.com/IntuneWindowsAgent.msi";
private static ImeMsiInstallSourceProbe.MsiEntry Entry(
string productCode = ImeProductCode,
string version = "1.104.102.0",
string url = ImeUrl)
=> new ImeMsiInstallSourceProbe.MsiEntry(productCode, version, url);
// -- SelectBestMatch -----------------------------------------------------------
[Fact]
public void SelectBestMatch_exact_product_version_wins()
{
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>
{
Entry(productCode: "{OTHER}", version: "9.9.9.9", url: "https://x/Other.msi"),
Entry(),
},
"1.104.102.0");
Assert.True(result.HasData);
Assert.True(result.MatchedByVersion);
Assert.Equal(ImeProductCode, result.ProductCode);
Assert.Equal("1.104.102.0", result.ProductVersion);
Assert.Equal(ImeUrl, result.DownloadUrl);
}
[Fact]
public void SelectBestMatch_normalized_version_matches_trailing_zero()
{
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry> { Entry(version: "1.104.102") },
"1.104.102.0");
Assert.True(result.HasData);
Assert.True(result.MatchedByVersion);
}
[Fact]
public void SelectBestMatch_version_mismatch_falls_back_to_ime_msi_filename()
{
// IME self-updated between MSI enforcement and log line — versions drift, but the
// enforced IntuneWindowsAgent.msi entry is still the install source.
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>
{
Entry(productCode: "{OTHER}", version: "5.0.0.0", url: "https://x/SomeLobApp.msi"),
Entry(version: "1.103.100.0"),
},
"1.104.102.0");
Assert.True(result.HasData);
Assert.False(result.MatchedByVersion);
Assert.Equal("1.103.100.0", result.ProductVersion);
Assert.Equal(ImeUrl, result.DownloadUrl);
}
[Fact]
public void SelectBestMatch_version_match_beats_filename_match_on_other_entry()
{
// A stale IntuneWindowsAgent.msi row must not shadow the entry whose
// ProductVersion equals the running build.
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>
{
Entry(productCode: "{STALE}", version: "1.103.100.0"),
Entry(productCode: "{CURRENT}", version: "1.104.102.0", url: "https://y/IntuneWindowsAgent.msi"),
},
"1.104.102.0");
Assert.True(result.MatchedByVersion);
Assert.Equal("{CURRENT}", result.ProductCode);
}
[Fact]
public void SelectBestMatch_no_plausible_entry_returns_empty()
{
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>
{
Entry(productCode: "{OTHER}", version: "5.0.0.0", url: "https://x/SomeLobApp.msi"),
},
"1.104.102.0");
Assert.False(result.HasData);
}
[Fact]
public void SelectBestMatch_empty_or_null_candidates_return_empty()
{
Assert.False(ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>(), "1.104.102.0").HasData);
Assert.False(ImeMsiInstallSourceProbe.SelectBestMatch(
null, "1.104.102.0").HasData);
}
[Fact]
public void SelectBestMatch_null_registry_values_do_not_throw()
{
var result = ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry>
{
new ImeMsiInstallSourceProbe.MsiEntry("{PARTIAL}", null, null),
Entry(),
},
"1.104.102.0");
Assert.True(result.HasData);
Assert.Equal(ImeProductCode, result.ProductCode);
}
// -- VersionsEqual ---------------------------------------------------------------
[Theory]
[InlineData("1.104.102.0", "1.104.102.0", true)]
[InlineData("1.104.102", "1.104.102.0", true)]
[InlineData("1.104.102.0", "1.104.102", true)]
[InlineData(" 1.104.102.0 ", "1.104.102.0", true)]
[InlineData("1.104.102.0", "1.104.103.0", false)]
[InlineData("1.104.102.0", "1.104.102.1", false)]
[InlineData("1.104", "1.104.102.0", false)]
[InlineData("abc", "abc", true)] // non-numeric: whole-string equality
[InlineData("abc", "abd", false)] // non-numeric components never fuzzy-match
[InlineData(null, "1.0", false)]
[InlineData("1.0", null, false)]
[InlineData("", "", false)]
public void VersionsEqual_cases(string? a, string? b, bool expected)
{
Assert.Equal(expected, ImeMsiInstallSourceProbe.VersionsEqual(a, b));
}
// -- IsImeMsiUrl -------------------------------------------------------------------
[Theory]
[InlineData(ImeUrl, true)]
[InlineData("https://x/intunewindowsagent.MSI", true)] // case-insensitive
[InlineData("https://x/IntuneWindowsAgent.msi?sv=abc&sig=def", true)] // query ignored
[InlineData("IntuneWindowsAgent.msi", true)] // bare filename
[InlineData("https://x/NotTheAgent.msi", false)]
[InlineData("https://x/IntuneWindowsAgent.msi.bak", false)]
[InlineData("", false)]
[InlineData(null, false)]
public void IsImeMsiUrl_cases(string? url, bool expected)
{
Assert.Equal(expected, ImeMsiInstallSourceProbe.IsImeMsiUrl(url));
}
// -- ScopedOverride ---------------------------------------------------------------
[Fact]
public void ScopedOverride_restores_previous_override_on_dispose()
{
// Assembly default (TestAssemblyInit) forces Empty; a nested scope must restore it.
Func<string, AgentLogger, ImeMsiInstallSource> before =
ImeMsiInstallSourceProbe.TestOverride;
using (new ImeMsiInstallSourceProbe.ScopedOverride((_, _) =>
ImeMsiInstallSourceProbe.SelectBestMatch(
new List<ImeMsiInstallSourceProbe.MsiEntry> { Entry() }, "1.104.102.0")))
{
Assert.True(ImeMsiInstallSourceProbe.Read("1.104.102.0").HasData);
}
Assert.Same(before, ImeMsiInstallSourceProbe.TestOverride);
}
}
}
@@ -0,0 +1,377 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
using SharedEventTypes = AutopilotMonitor.Shared.Constants.EventTypes;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Registry second pillar (audit 2026-08-17): pure parsing/diff logic plus the
/// tick flow (baseline silence, state-change emission, reconciliation dwell)
/// driven through the snapshot override — no live registry.
/// </summary>
public sealed class ImeRegistryAppStateObserverTests
{
private const string App1 = "11111111-1111-1111-1111-111111111111";
private const string App2 = "22222222-2222-2222-2222-222222222222";
private static readonly DateTime T0 = new DateTime(2026, 8, 18, 8, 0, 0, DateTimeKind.Utc);
// ── pure helpers ────────────────────────────────────────────────────────
[Theory]
[InlineData("11111111-1111-1111-1111-111111111111_1", "11111111-1111-1111-1111-111111111111")]
[InlineData("11111111-1111-1111-1111-111111111111", "11111111-1111-1111-1111-111111111111")]
[InlineData("GRS", null)]
[InlineData("", null)]
[InlineData("not-a-guid_1", null)]
public void ExtractAppId_parses_app_keys_and_rejects_non_app_keys(string keyName, string? expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ExtractAppId(keyName));
[Fact]
public void ParseEnforcementStateMessage_reads_state_and_error()
{
var (state, error) = ImeRegistryAppStateObserver.ParseEnforcementStateMessage(
"{\"EnforcementState\":1000,\"ErrorCode\":0}");
Assert.Equal(1000, state);
Assert.Equal(0L, error);
Assert.Equal((null, null), ImeRegistryAppStateObserver.ParseEnforcementStateMessage("not json"));
Assert.Equal((null, null), ImeRegistryAppStateObserver.ParseEnforcementStateMessage("[]"));
}
[Theory]
[InlineData(1000, "success")]
[InlineData(1004, "success")]
[InlineData(2009, "inProgress")]
[InlineData(3000, "requirementsNotMet")]
[InlineData(5003, "error")]
[InlineData(6001, "notAttempted")]
[InlineData(42, "unknown")]
public void ClassifyEnforcementState_bands(int state, string expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ClassifyEnforcementState(state));
[Theory]
[InlineData(1000, "installed")]
[InlineData(1999, "installed")]
[InlineData(3000, "failed")]
[InlineData(2000, "notApplicable")]
[InlineData(2999, "notApplicable")]
[InlineData(0, null)]
[InlineData(999, null)]
[InlineData(3001, null)]
[InlineData(4000, null)]
public void ClassifyStatusServiceStatus_bands(int status, string? expected)
=> Assert.Equal(expected, ImeRegistryAppStateObserver.ClassifyStatusServiceStatus(status));
[Fact]
public void TerminalOutcome_prefers_enforcement_state_then_status_service()
{
// Enforcement state alone decides when terminal.
Assert.Equal("success", ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 1000)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 5003)));
// Non-terminal enforcement bands defer to the StatusService pillar.
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 2009)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(
Entry(enforcementState: 2009, statusServiceStatus: 3000)));
Assert.Equal("success", ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 1000)));
Assert.Equal("error", ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 3000)));
// requirementsNotMet / notApplicable / empty entries are not judgeable.
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(enforcementState: 3000)));
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry(statusServiceStatus: 2500)));
Assert.Null(ImeRegistryAppStateObserver.TerminalOutcome(Entry()));
}
[Fact]
public void ChangedFields_reports_each_field_and_ignores_espPhase_casing()
{
var prev = Entry(enforcementState: 2009, errorCode: null, exitCode: null,
statusServiceStatus: null, espTracked: false, espPhase: "DevicePreparation");
var next = Entry(enforcementState: 1000, errorCode: 0, exitCode: 0,
statusServiceStatus: 1000, espTracked: true, espPhase: "devicepreparation");
var changed = ImeRegistryAppStateObserver.ChangedFields(prev, next);
Assert.Equal(
new[] { "enforcementState", "errorCode", "exitCode", "statusServiceStatus", "espTracked" },
changed);
// Case-only espPhase difference is not a change; identity diffs to empty.
Assert.DoesNotContain("espPhase", changed);
Assert.Empty(ImeRegistryAppStateObserver.ChangedFields(next, next));
}
[Fact]
public void ChangedFields_null_previous_reports_populated_fields_of_new_entry()
{
var fresh = Entry(enforcementState: 2000, espTracked: true, espPhase: "AccountSetup");
var changed = ImeRegistryAppStateObserver.ChangedFields(null, fresh);
Assert.Contains("enforcementState", changed);
Assert.Contains("espTracked", changed);
Assert.Contains("espPhase", changed);
// Fields that are null on both sides are not "changes" for a brand-new entry.
Assert.DoesNotContain("errorCode", changed);
Assert.DoesNotContain("exitCode", changed);
Assert.DoesNotContain("statusServiceStatus", changed);
}
[Fact]
public void DiffSnapshots_reports_only_changed_fields()
{
var prev = Snapshot((App1, 2009, null));
var next = Snapshot((App1, 1000, 0), (App2, 2000, null));
var changes = ImeRegistryAppStateObserver.DiffSnapshots(prev, next);
Assert.Equal(2, changes.Count);
var app1Change = changes.Single(c => c.Entry.AppId == App1);
Assert.Contains("enforcementState", app1Change.ChangedFields);
Assert.Contains("errorCode", app1Change.ChangedFields);
Assert.False(app1Change.IsNew);
Assert.True(changes.Single(c => c.Entry.AppId == App2).IsNew);
Assert.Empty(ImeRegistryAppStateObserver.DiffSnapshots(next, next));
}
[Fact]
public void IsDivergent_rules()
{
var installed = PackageState(App1, AppInstallationState.Installed);
var errored = PackageState(App1, AppInstallationState.Error);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("error", installed, true, out var r1));
Assert.Equal("registry_error_log_installed", r1);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("success", errored, true, out var r2));
Assert.Equal("registry_success_log_error", r2);
Assert.True(ImeRegistryAppStateObserver.IsDivergent("success", null, true, out var r3));
Assert.Equal("app_unknown_to_log_tracking", r3);
// Tracker idle (no apps at all) — not judgeable, no false alarm.
Assert.False(ImeRegistryAppStateObserver.IsDivergent("success", null, false, out _));
// Agreement.
Assert.False(ImeRegistryAppStateObserver.IsDivergent("success", installed, true, out _));
Assert.False(ImeRegistryAppStateObserver.IsDivergent("error", errored, true, out _));
}
// ── tick flow ───────────────────────────────────────────────────────────
[Fact]
public void Baseline_is_silent_then_changes_emit_state_events()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
current = Snapshot((App1, 1000, 0));
observer.Tick("registry_change");
}
var evt = Assert.Single(Events(sink, SharedEventTypes.RegistryAppState));
Assert.Equal(App1, evt.Payload!["appId"]);
Assert.Equal("1000", evt.Payload["enforcementState"]);
Assert.Equal("success", evt.Payload["enforcementClass"]);
}
[Fact]
public void NonEmpty_baseline_emits_one_summary_with_counts()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 1000, 0), (App2, 5003, 101));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
observer.Tick("periodic"); // summary is one-shot, not per tick
}
var summary = Assert.Single(Events(sink, SharedEventTypes.RegistryAppBaseline));
Assert.Equal("2", summary.Payload!["totalApps"]);
Assert.Equal("1", summary.Payload["successCount"]);
Assert.Equal("1", summary.Payload["errorCount"]);
// The baseline stays silent on the diff rail.
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
}
[Fact]
public void Empty_baseline_emits_no_summary()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = new ImeRegistrySnapshot();
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
}
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppBaseline));
}
[Fact]
public void Unchanged_snapshot_emits_nothing()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
observer.Tick("periodic");
observer.Tick("periodic");
}
Assert.Empty(Events(sink, SharedEventTypes.RegistryAppState));
}
[Fact]
public void Reconciliation_fires_once_after_settle_delay_on_divergence()
{
var tracker = new List<AppPackageState> { PackageState(App1, AppInstallationState.Installed) };
var (sink, clock, observer) = CreateObserver(trackerApps: tracker);
var current = Snapshot((App1, 2000, null));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = Snapshot((App1, 5003, 101)); // registry error, log says Installed
observer.Tick("registry_change");
// Before the settle delay: no reconciliation yet.
Assert.Empty(Events(sink, SharedEventTypes.AppStateReconciliation));
clock.Advance(ImeRegistryAppStateObserver.ReconcileSettleDelay + TimeSpan.FromSeconds(1));
observer.Tick("periodic");
observer.Tick("periodic"); // second pass must not re-emit
}
var rec = Assert.Single(Events(sink, SharedEventTypes.AppStateReconciliation));
Assert.Equal("error", rec.Payload!["registryOutcome"]);
Assert.Equal("registry_error_log_installed", rec.Payload["reason"]);
Assert.Equal("Installed", rec.Payload["logState"]);
}
[Fact]
public void Reconciliation_skips_agreeing_and_baseline_only_apps()
{
var tracker = new List<AppPackageState> { PackageState(App1, AppInstallationState.Installed) };
var (sink, clock, observer) = CreateObserver(trackerApps: tracker);
// App2 is terminal in the BASELINE (pre-existing from an earlier enrollment) and
// never changes — it must never be judged. App1 changes and agrees with the log.
var current = Snapshot((App1, 2000, null), (App2, 5000, 1));
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = Snapshot((App1, 1000, 0), (App2, 5000, 1));
observer.Tick("registry_change");
clock.Advance(ImeRegistryAppStateObserver.ReconcileSettleDelay + TimeSpan.FromSeconds(1));
observer.Tick("periodic");
}
Assert.Empty(Events(sink, SharedEventTypes.AppStateReconciliation));
}
[Fact]
public void State_event_cap_emits_single_warning_then_suppresses()
{
var (sink, clock, observer) = CreateObserver(trackerApps: null);
var baseline = new ImeRegistrySnapshot();
var flooded = new ImeRegistrySnapshot();
for (var i = 0; i < ImeRegistryAppStateObserver.MaxStateEventsPerSession + 25; i++)
{
var appId = Guid.NewGuid().ToString("D");
baseline.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId).EnforcementState = 2000;
flooded.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId).EnforcementState = 1000;
}
var current = baseline;
using (new ImeRegistryAppStateObserver.ScopedSnapshotOverride(() => current))
{
observer.Tick("baseline");
current = flooded;
observer.Tick("registry_change");
}
var events = Events(sink, SharedEventTypes.RegistryAppState);
// Cap payload-carrying events + exactly one cap-notice Warning.
Assert.Equal(ImeRegistryAppStateObserver.MaxStateEventsPerSession + 1, events.Count);
Assert.Single(events, e => e.Payload != null
&& e.Payload.TryGetValue(SignalPayloadKeys.Message, out var m)
&& m.Contains("cap reached"));
}
// ── harness ─────────────────────────────────────────────────────────────
private static (FakeSignalIngressSink sink, VirtualClock clock, ImeRegistryAppStateObserver observer)
CreateObserver(IReadOnlyList<AppPackageState>? trackerApps)
{
var sink = new FakeSignalIngressSink();
var clock = new VirtualClock(T0);
var post = new InformationalEventPost(sink, clock);
var observer = new ImeRegistryAppStateObserver(
post,
logger: null,
clock: clock,
trackerStateProbe: trackerApps == null ? null : () => trackerApps);
return (sink, clock, observer);
}
private static AppRegistryEntry Entry(
int? enforcementState = null, long? errorCode = null, int? exitCode = null,
int? statusServiceStatus = null, bool espTracked = false, string? espPhase = null)
{
var entry = new AppRegistryEntry($"{ImeRegistrySnapshot.DeviceContext}|{App1}", ImeRegistrySnapshot.DeviceContext, App1);
entry.EnforcementState = enforcementState;
entry.ErrorCode = errorCode;
entry.ExitCode = exitCode;
entry.StatusServiceStatus = statusServiceStatus;
entry.EspTracked = espTracked;
entry.EspPhase = espPhase;
return entry;
}
private static ImeRegistrySnapshot Snapshot(params (string appId, int enforcementState, long? errorCode)[] apps)
{
var snapshot = new ImeRegistrySnapshot();
foreach (var (appId, state, error) in apps)
{
var entry = snapshot.GetOrAdd(ImeRegistrySnapshot.DeviceContext, appId);
entry.EnforcementState = state;
entry.ErrorCode = error;
}
return snapshot;
}
private static AppPackageState PackageState(string id, AppInstallationState state)
{
var pkg = new AppPackageState(id, listPos: 0);
if (state == AppInstallationState.Installed)
{
// Route through Installing first — a bare Installed would trip the
// inverse-detection auto-downgrade heuristic (Installed w/o activity -> Skipped).
pkg.UpdateState(AppInstallationState.Installing);
}
pkg.UpdateState(state);
return pkg;
}
private static IReadOnlyList<FakeSignalIngressSink.PostedSignal> Events(FakeSignalIngressSink sink, string eventType) =>
sink.Posted.Where(p =>
p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
}
}
@@ -0,0 +1,130 @@
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Session afee7ae0 (2026-08-18) — Device Preparation structural-noise suppression.
/// A nine-minute WDP session shipped 900+ occurrences of WIL error 1005 plus 400+
/// duplicate-workload 417/418 records and "Autopilot policy not found" 100s — all
/// structural on WDP (no deployment profile exists; the BootstrapperAgent's SLDM
/// progress poll produces the 417/418 chatter). On WDP those IDs are suppressed
/// entirely (<see cref="ModernDeploymentTracker.DevicePreparationNoiseEventIds"/>);
/// genuine diagnostics (e.g. 408 provisioning warnings) and Critical records keep
/// flowing, and Classic sessions keep the harmless-downgrade + rollup path.
/// </summary>
public sealed class ModernDeploymentTrackerDevicePrepTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 8, 18, 9, 10, 0, DateTimeKind.Utc);
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly AgentLogger _logger;
public ModernDeploymentTrackerDevicePrepTests()
{
_sink = new FakeSignalIngressSink();
_logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
}
public void Dispose() => _tmp.Dispose();
private ModernDeploymentTracker BuildTracker(bool isDevicePreparation) =>
new ModernDeploymentTracker(
sessionId: "sess-wdp-noise",
tenantId: "tenant-wdp-noise",
post: new InformationalEventPost(_sink, new VirtualClock(At)),
logger: _logger,
backfillEnabled: false,
isDevicePreparation: isDevicePreparation);
private void Process(ModernDeploymentTracker tracker, int eventId, int level, string shortName, string channel)
{
tracker.ProcessEvent(
eventId: eventId,
level: level,
levelDisplayName: level == 1 ? "Critical" : level == 2 ? "Error" : "Warning",
providerName: "ModernDeployment-Diagnostics-Provider",
timeCreatedUtc: At,
formattedDescription: $"EventID {eventId} record.",
shortName: shortName,
channelName: channel,
isBackfill: false);
}
private List<FakeSignalIngressSink.PostedSignal> PostedModernDeployment() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et.StartsWith("modern_deployment", StringComparison.Ordinal)).ToList();
[Theory]
[InlineData(100, 3, "Autopilot")] // policy not found — no profile exists on WDP
[InlineData(417, 3, "Autopilot")] // duplicate workload — SLDM poll chatter
[InlineData(418, 3, "Autopilot")] // duplicate batch — SLDM poll chatter
[InlineData(1005, 2, "ManagementService")] // WIL error storm
public void WdpNoiseIds_AreFullySuppressed_OnDevicePreparation(int eventId, int level, string shortName)
{
var tracker = BuildTracker(isDevicePreparation: true);
var channel = shortName == "Autopilot"
? ModernDeploymentTracker.AutopilotChannel
: ModernDeploymentTracker.ManagementChannel;
for (var i = 0; i < 10; i++)
{
Process(tracker, eventId, level, shortName, channel);
}
Assert.Empty(PostedModernDeployment());
}
[Fact]
public void GenuineWdpWarning_408_KeepsFlowing_OnDevicePreparation()
{
// 408 "Autopilot Provisioning reported a warning" carries the real WDP session
// context — it is NOT in the noise set and must keep flowing.
var tracker = BuildTracker(isDevicePreparation: true);
Process(tracker, 408, level: 3, shortName: "Autopilot", channel: ModernDeploymentTracker.AutopilotChannel);
var posted = Assert.Single(PostedModernDeployment());
Assert.Equal("modern_deployment_warning", posted.Payload![SignalPayloadKeys.EventType]);
}
[Fact]
public void CriticalRecords_AreNeverSuppressed_EvenForNoiseIds()
{
var tracker = BuildTracker(isDevicePreparation: true);
Process(tracker, 1005, level: 1, shortName: "ManagementService", channel: ModernDeploymentTracker.ManagementChannel);
var posted = Assert.Single(PostedModernDeployment());
Assert.Equal("modern_deployment_error", posted.Payload![SignalPayloadKeys.EventType]);
}
[Fact]
public void ClassicSessions_KeepTheHarmlessDowngradePath_ForTheSameIds()
{
// Classic regression guard: without the WDP flag EventID 100 still flows through
// the harmless-downgrade path (individual Debug emissions up to the rollup limit).
var tracker = BuildTracker(isDevicePreparation: false);
for (var i = 0; i < 2; i++)
{
Process(tracker, 100, level: 3, shortName: "Autopilot", channel: ModernDeploymentTracker.AutopilotChannel);
}
Assert.Equal(2, PostedModernDeployment().Count);
}
}
}
@@ -0,0 +1,258 @@
#nullable enable
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Runtime;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// Pure state machine behind the power watcher: AC↔battery edges, the 50/30/15 threshold
/// ladder (latch once per lifetime, lowest-crossed-only on multi-level jumps, baseline
/// coverage for sessions starting low), null-percent tolerance and the storm cap.
/// </summary>
public sealed class PowerStateTransitionTrackerTests
{
private static PowerStateResult Ac(int? percent = 80, bool charging = true) => new PowerStateResult
{
OnAcPower = true,
HasBattery = true,
BatteryPercent = percent,
IsCharging = charging,
BatteryLifeMinutes = null,
};
private static PowerStateResult Battery(int? percent, int? lifeMinutes = 90) => new PowerStateResult
{
OnAcPower = false,
HasBattery = true,
BatteryPercent = percent,
IsCharging = false,
BatteryLifeMinutes = lifeMinutes,
};
[Fact]
public void Baseline_on_ac_is_silent()
{
var tracker = new PowerStateTransitionTracker();
Assert.Empty(tracker.Baseline(Ac()));
}
[Fact]
public void Baseline_without_battery_or_with_probe_error_is_silent()
{
var tracker = new PowerStateTransitionTracker();
Assert.Empty(tracker.Baseline(new PowerStateResult { OnAcPower = true, HasBattery = false }));
Assert.Empty(tracker.Baseline(new PowerStateResult { ProbeError = "boom" }));
}
[Fact]
public void Baseline_on_battery_above_ladder_is_silent()
{
var tracker = new PowerStateTransitionTracker();
Assert.Empty(tracker.Baseline(Battery(80)));
}
[Fact]
public void Baseline_at_12_percent_emits_exactly_the_15_threshold()
{
var tracker = new PowerStateTransitionTracker();
var emissions = tracker.Baseline(Battery(12));
var e = Assert.Single(emissions);
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, e.Transition);
Assert.Equal(15, e.ThresholdPercent);
Assert.Equal(12, e.BatteryPercent);
Assert.Equal(EventSeverity.Error, e.Severity);
Assert.True(e.ImmediateUpload);
}
[Fact]
public void Baseline_at_25_percent_emits_30_and_latches_50()
{
var tracker = new PowerStateTransitionTracker();
var baseline = tracker.Baseline(Battery(25));
var e = Assert.Single(baseline);
Assert.Equal(30, e.ThresholdPercent);
Assert.Equal(EventSeverity.Warning, e.Severity);
Assert.True(e.ImmediateUpload);
// 50 and 30 are latched by the baseline; draining to 40% stays silent, 14% emits 15.
Assert.Empty(tracker.Evaluate(Battery(40)));
var next = Assert.Single(tracker.Evaluate(Battery(14)));
Assert.Equal(15, next.ThresholdPercent);
}
[Fact]
public void Ac_to_battery_emits_warning_with_immediate_upload()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac(63));
var e = Assert.Single(tracker.Evaluate(Battery(63)));
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, e.Transition);
Assert.Null(e.ThresholdPercent);
Assert.Equal(EventSeverity.Warning, e.Severity);
Assert.True(e.ImmediateUpload);
Assert.False(e.OnAcPower);
Assert.Equal(63, e.BatteryPercent);
}
[Fact]
public void Battery_to_ac_emits_info_without_immediate_upload()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Battery(70));
var e = Assert.Single(tracker.Evaluate(Ac(70)));
Assert.Equal(PowerStateTransitionTracker.TransitionBatteryToAc, e.Transition);
Assert.Equal(EventSeverity.Info, e.Severity);
Assert.False(e.ImmediateUpload);
}
[Fact]
public void No_diff_stays_silent()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac());
Assert.Empty(tracker.Evaluate(Ac()));
Assert.Empty(tracker.Evaluate(Ac(79)));
}
[Fact]
public void Unplug_below_threshold_emits_transition_and_threshold_together()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac(10));
var emissions = tracker.Evaluate(Battery(10));
Assert.Equal(2, emissions.Count);
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, emissions[0].Transition);
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, emissions[1].Transition);
Assert.Equal(15, emissions[1].ThresholdPercent);
}
[Fact]
public void Multi_level_drop_emits_lowest_crossed_level_only()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Battery(60));
var e = Assert.Single(tracker.Evaluate(Battery(10)));
Assert.Equal(15, e.ThresholdPercent);
// All levels are latched — recovering and re-draining emits nothing more.
tracker.Evaluate(Ac(90));
Assert.Single(tracker.Evaluate(Battery(45)), x => x.Transition == PowerStateTransitionTracker.TransitionAcToBattery);
Assert.Empty(tracker.Evaluate(Battery(13)));
}
[Fact]
public void Each_level_latches_once_per_lifetime()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Battery(70));
Assert.Single(tracker.Evaluate(Battery(48))); // 50
Assert.Empty(tracker.Evaluate(Battery(47))); // still inside 50, no re-emit
Assert.Single(tracker.Evaluate(Battery(29))); // 30
Assert.Single(tracker.Evaluate(Battery(15))); // 15 (boundary: <= level)
Assert.Empty(tracker.Evaluate(Battery(5))); // ladder exhausted
}
[Fact]
public void Threshold_severity_ladder_is_info_warning_error()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Battery(70));
var fifty = Assert.Single(tracker.Evaluate(Battery(50)));
Assert.Equal(EventSeverity.Info, fifty.Severity);
Assert.False(fifty.ImmediateUpload);
var thirty = Assert.Single(tracker.Evaluate(Battery(30)));
Assert.Equal(EventSeverity.Warning, thirty.Severity);
Assert.True(thirty.ImmediateUpload);
var fifteen = Assert.Single(tracker.Evaluate(Battery(15)));
Assert.Equal(EventSeverity.Error, fifteen.Severity);
Assert.True(fifteen.ImmediateUpload);
}
[Fact]
public void Null_battery_percent_skips_thresholds_but_ac_diff_still_works()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac(null));
var e = Assert.Single(tracker.Evaluate(Battery(null, lifeMinutes: null)));
Assert.Equal(PowerStateTransitionTracker.TransitionAcToBattery, e.Transition);
Assert.Null(e.BatteryPercent);
Assert.Contains("battery level unknown", e.Message);
Assert.Empty(tracker.Evaluate(Battery(null, lifeMinutes: null)));
}
[Fact]
public void Probe_error_mid_run_preserves_state()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac());
Assert.Empty(tracker.Evaluate(new PowerStateResult { ProbeError = "transient" }));
// State was not corrupted by the error — the real transition still diffs cleanly.
Assert.Single(tracker.Evaluate(Battery(60)));
}
[Fact]
public void Evaluate_without_baseline_latches_like_a_baseline()
{
var tracker = new PowerStateTransitionTracker();
var e = Assert.Single(tracker.Evaluate(Battery(12)));
// No phantom ac_to_battery from the implicit default state — only the threshold.
Assert.Equal(PowerStateTransitionTracker.TransitionThresholdCrossed, e.Transition);
}
[Fact]
public void Emission_cap_suppresses_and_flips_flag_once()
{
var tracker = new PowerStateTransitionTracker(maxEmissions: 4);
tracker.Baseline(Ac());
// Flapping power source: each edge is one emission.
for (var i = 0; i < 2; i++)
{
Assert.Single(tracker.Evaluate(Battery(80)));
Assert.Single(tracker.Evaluate(Ac(80)));
}
Assert.False(tracker.EmissionCapReached);
Assert.Empty(tracker.Evaluate(Battery(80)));
Assert.True(tracker.EmissionCapReached);
Assert.Empty(tracker.Evaluate(Ac(80)));
}
[Fact]
public void Message_carries_percent_and_transition_context()
{
var tracker = new PowerStateTransitionTracker();
tracker.Baseline(Ac(62));
var toBattery = Assert.Single(tracker.Evaluate(Battery(62)));
Assert.Contains("AC to battery", toBattery.Message);
Assert.Contains("battery 62%", toBattery.Message);
var backToAc = Assert.Single(tracker.Evaluate(Ac(63)));
Assert.Contains("back to AC", backToAc.Message);
Assert.Contains("charging", backToAc.Message);
}
}
}
@@ -0,0 +1,429 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// SystemTimelineTracker — clock steps (Kernel-General 1) and completed sleep episodes
/// (Power-Troubleshooter 1 / Kernel-Power 507) from the System channel. Drives the primitive
/// <see cref="SystemTimelineTracker.ProcessEvent"/> test seam (no real EventRecord, which is
/// abstract + Windows-only), mirroring the WindowsUpdateTracker test pattern.
/// </summary>
public sealed class SystemTimelineTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 8, 21, 10, 0, 0, DateTimeKind.Utc);
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly SystemTimelineTracker _tracker;
private long _nextRecordId = 1;
public SystemTimelineTrackerTests()
{
_sink = new FakeSignalIngressSink();
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_tracker = new SystemTimelineTracker(
sessionId: "sess-stl",
tenantId: "tenant-stl",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: null); // in-memory dedup for most tests
}
public void Dispose() => _tmp.Dispose();
// -------------------------------------------------------------------
// Emission helpers
// -------------------------------------------------------------------
private void EmitClockChange(
DateTime? oldTime, DateTime? newTime, string? rawDeltaMs = null,
int reason = 1, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
{
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (oldTime.HasValue) data["OldTime"] = oldTime.Value.ToString("o");
if (newTime.HasValue) data["NewTime"] = newTime.Value.ToString("o");
if (rawDeltaMs != null) data["TimeDeltaInMs"] = rawDeltaMs;
data["Reason"] = reason.ToString();
data["ProcessName"] = @"\Device\HarddiskVolume3\Windows\System32\svchost.exe";
data["ProcessID"] = "6892";
_tracker.ProcessEvent(
providerName: SystemTimelineTracker.ProviderKernelGeneral,
eventId: SystemTimelineTracker.EventId_ClockChange,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? At,
eventData: data,
isBackfill: isBackfill);
}
private void EmitClassicResume(
DateTime sleepTime, DateTime wakeTime, string? effectiveState = "1",
string? wakeSourceText = null, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
{
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["SleepTime"] = sleepTime.ToString("o"),
["WakeTime"] = wakeTime.ToString("o"),
["WakeSourceType"] = "0",
};
if (effectiveState != null) data["EffectiveState"] = effectiveState;
if (wakeSourceText != null) data["WakeSourceText"] = wakeSourceText;
_tracker.ProcessEvent(
providerName: SystemTimelineTracker.ProviderPowerTroubleshooter,
eventId: SystemTimelineTracker.EventId_ClassicResume,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? wakeTime,
eventData: data,
isBackfill: isBackfill);
}
private void EmitModernStandbyExit(
long sleepDurationUs, long durationUs, string powerStateAc = "true",
long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null)
{
var data = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["SleepDurationInUs"] = sleepDurationUs.ToString(),
["DurationInUs"] = durationUs.ToString(),
["Reason"] = "32",
["PowerStateAc"] = powerStateAc,
["BatteryRemainingCapacityOnExit"] = "34180",
};
_tracker.ProcessEvent(
providerName: SystemTimelineTracker.ProviderKernelPower,
eventId: SystemTimelineTracker.EventId_ModernStandbyExit,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? At,
eventData: data,
isBackfill: isBackfill);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> ByType(string eventType) =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == eventType).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
// -------------------------------------------------------------------
// system_clock_changed
// -------------------------------------------------------------------
[Fact]
public void ClockStep_AboveThreshold_EmitsInfo_WithSignedDeltaAndNormalizedTimes()
{
var oldTime = At;
var newTime = At.AddSeconds(10);
EmitClockChange(oldTime, newTime, reason: 1);
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("false", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal(10_000L, data["timeDeltaMs"]);
Assert.Equal("application_set", data["reasonText"]);
Assert.Equal(1, data["reason"]);
Assert.Equal(oldTime.ToString("o"), data["oldTime"]);
Assert.Equal(newTime.ToString("o"), data["newTime"]);
Assert.Equal(6892, data["processId"]);
Assert.Contains("forward", (string)s.Payload![SignalPayloadKeys.Message]);
Assert.Contains("svchost.exe", (string)s.Payload![SignalPayloadKeys.Message]);
}
[Theory]
[InlineData(1)]
[InlineData(1999)]
public void ClockMicroStep_BelowThreshold_Suppressed(int deltaMs)
{
EmitClockChange(At, At.AddMilliseconds(deltaMs));
Assert.Empty(ByType(Constants.EventTypes.SystemClockChanged));
}
[Fact]
public void ClockStep_Backward_EmitsNegativeDelta()
{
EmitClockChange(oldTime: At, newTime: At.AddSeconds(-3));
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
Assert.Equal(-3000L, Data(s)["timeDeltaMs"]);
Assert.Contains("backward", (string)s.Payload![SignalPayloadKeys.Message]);
}
[Fact]
public void ClockStep_AtLeastFiveMinutes_WarningAndImmediateUpload()
{
EmitClockChange(At, At.AddMinutes(5), reason: 2);
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
Assert.Equal("Warning", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
Assert.Equal("hardware_clock_sync", Data(s)["reasonText"]);
}
[Fact]
public void ClockStep_NoUsableTimesOrDelta_Skipped()
{
EmitClockChange(oldTime: null, newTime: null, rawDeltaMs: "not-a-number");
Assert.Empty(ByType(Constants.EventTypes.SystemClockChanged));
}
[Fact]
public void ClockStep_FallsBackToTimeDeltaInMs_WhenTimesUnparseable()
{
// Only the raw delta parses — the direction information of NewTime/OldTime is gone,
// but the magnitude still gates and emits.
EmitClockChange(oldTime: null, newTime: null, rawDeltaMs: "7200000");
var s = Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
Assert.Equal(7_200_000L, Data(s)["timeDeltaMs"]);
}
// -------------------------------------------------------------------
// system_sleep_episode — classic (Power-Troubleshooter 1)
// -------------------------------------------------------------------
[Fact]
public void ClassicResume_EffectiveState5_EmitsHibernateEpisode()
{
var sleepAt = At.AddHours(-2);
EmitClassicResume(sleepAt, At, effectiveState: "5", wakeSourceText: "Power Button");
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
var data = Data(s);
Assert.Equal("hibernate", data["kind"]);
Assert.Equal(7200L, data["durationSeconds"]);
Assert.Equal(sleepAt.ToString("o"), data["enteredAt"]);
Assert.Equal(At.ToString("o"), data["exitedAt"]);
Assert.Equal("Power Button", data["wakeSourceText"]);
Assert.Contains("hibernate", (string)s.Payload![SignalPayloadKeys.Message]);
}
[Fact]
public void ClassicResume_NonHibernateState_EmitsSleepKind()
{
EmitClassicResume(At.AddMinutes(-30), At, effectiveState: "4");
Assert.Equal("sleep", Data(Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode)))["kind"]);
}
[Fact]
public void ClassicResume_ShortEpisode_StillEmits()
{
// No duration floor for classic sleep: the event only exists for genuinely completed
// S3/S4 transitions, so a 30 s real sleep is still signal.
EmitClassicResume(At.AddSeconds(-30), At);
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
Assert.Equal(30L, Data(s)["durationSeconds"]);
}
// -------------------------------------------------------------------
// system_sleep_episode — Modern Standby (Kernel-Power 507)
// -------------------------------------------------------------------
[Fact]
public void ModernStandbyExit_EmitsEpisode_WithDerivedEnterTime()
{
// Values from a live 507 on this dev machine: 38 min scenario, ~38 min actual sleep.
EmitModernStandbyExit(sleepDurationUs: 2_324_095_807, durationUs: 2_330_201_652);
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
var data = Data(s);
Assert.Equal("modern_standby", data["kind"]);
Assert.Equal(2330L, data["durationSeconds"]);
Assert.Equal(2324L, data["sleepDurationSeconds"]);
Assert.Equal(true, data["onAcPower"]);
Assert.Equal(34180L, data["batteryRemainingCapacityOnExit"]);
Assert.Equal(At.ToString("o"), data["exitedAt"]);
Assert.Equal(At.AddMilliseconds(-2_330_201.652).ToString("o"), data["enteredAt"]);
Assert.Contains("Modern Standby", (string)s.Payload![SignalPayloadKeys.Message]);
}
[Fact]
public void ModernStandbyExit_SleepBelowFloor_Suppressed()
{
EmitModernStandbyExit(sleepDurationUs: 30_000_000, durationUs: 90_000_000); // 30 s sleep
Assert.Empty(ByType(Constants.EventTypes.SystemSleepEpisode));
}
[Fact]
public void ModernStandbyExit_ScreenOffWithoutSleep_Suppressed()
{
// SleepEntered=false scenarios carry SleepDurationInUs 0 — the single duration gate
// suppresses them without a separate branch.
EmitModernStandbyExit(sleepDurationUs: 0, durationUs: 6_106_696);
Assert.Empty(ByType(Constants.EventTypes.SystemSleepEpisode));
}
[Fact]
public void ModernStandbyExit_ScenarioShorterThanSleep_ClampsToSleepDuration()
{
EmitModernStandbyExit(sleepDurationUs: 120_000_000, durationUs: 0);
var data = Data(Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode)));
Assert.Equal(120L, data["durationSeconds"]);
Assert.Equal(120L, data["sleepDurationSeconds"]);
}
// -------------------------------------------------------------------
// Backfill semantics
// -------------------------------------------------------------------
[Fact]
public void BackfilledEvent_UsesEventTimeForTimelineTimestamp_AndFlagsData()
{
// A pre-agent standby episode (record created before the agent started) must land on
// the timeline where the device actually woke, not at agent start.
var wakeTime = At.AddHours(-3);
EmitClassicResume(wakeTime.AddHours(-1), wakeTime, isBackfill: true, timeCreatedUtc: wakeTime);
var s = Assert.Single(ByType(Constants.EventTypes.SystemSleepEpisode));
Assert.Equal(wakeTime, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
}
// -------------------------------------------------------------------
// Dedup / watermark
// -------------------------------------------------------------------
[Fact]
public void SameRecordId_ProcessedTwice_EmitsOnce()
{
EmitClockChange(At, At.AddMinutes(1), recordId: 42);
EmitClockChange(At, At.AddMinutes(1), recordId: 42);
Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
}
[Fact]
public void BackfillEventWithLowerRecordId_AfterLiveEvent_IsStillEmitted()
{
// The live watcher is armed BEFORE backfill runs, so a live record with a higher
// RecordId can be processed first. An older, never-emitted backfill record must NOT be
// suppressed — pre-agent episodes are the whole point (high-water-mark trap).
EmitClockChange(At, At.AddMinutes(1), recordId: 100); // live, high
EmitClockChange(At, At.AddMinutes(2), recordId: 50, isBackfill: true); // backfill, older
Assert.Equal(2, ByType(Constants.EventTypes.SystemClockChanged).Count);
}
[Fact]
public void Watermark_PersistsAcrossTrackerInstances()
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var clockData = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
{
["OldTime"] = At.ToString("o"),
["NewTime"] = At.AddMinutes(10).ToString("o"),
["Reason"] = "1",
};
var first = new SystemTimelineTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
first.ProcessEvent(SystemTimelineTracker.ProviderKernelGeneral, SystemTimelineTracker.EventId_ClockChange,
recordId: 500, timeCreatedUtc: At, eventData: clockData, isBackfill: false);
// A fresh tracker (agent restart) must load the watermark and skip the already-emitted
// record when the backfill re-reads it.
var second = new SystemTimelineTracker("s", "t", post, logger, backfillEnabled: false, stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(SystemTimelineTracker.ProviderKernelGeneral, SystemTimelineTracker.EventId_ClockChange,
recordId: 500, timeCreatedUtc: At, eventData: clockData, isBackfill: true);
Assert.Single(ByType(Constants.EventTypes.SystemClockChanged));
}
[Fact]
public void UnknownProviderOrEventId_Ignored()
{
_tracker.ProcessEvent("Microsoft-Windows-Kernel-Power", eventId: 566, recordId: _nextRecordId++,
timeCreatedUtc: At, eventData: new Dictionary<string, string>(), isBackfill: false);
_tracker.ProcessEvent("Some-Other-Provider", eventId: 1, recordId: _nextRecordId++,
timeCreatedUtc: At, eventData: new Dictionary<string, string>(), isBackfill: false);
Assert.Empty(_sink.Posted);
}
// -------------------------------------------------------------------
// XPath + parsing
// -------------------------------------------------------------------
[Fact]
public void BuildXPath_ContainsAllThreeProviderIdPairs_AndNo506()
{
var xpath = SystemTimelineTracker.BuildXPath();
Assert.Contains("Provider[@Name='Microsoft-Windows-Kernel-General'] and EventID=1", xpath);
Assert.Contains("Provider[@Name='Microsoft-Windows-Power-Troubleshooter'] and EventID=1", xpath);
Assert.Contains("Provider[@Name='Microsoft-Windows-Kernel-Power'] and EventID=507", xpath);
Assert.DoesNotContain("506", xpath);
}
[Fact]
public void BuildBackfillXPath_AppendsTimediffClause()
{
var xpath = SystemTimelineTracker.BuildBackfillXPath(86_400_000);
Assert.Contains("TimeCreated[timediff(@SystemTime) <= 86400000]", xpath);
Assert.Contains("EventID=507", xpath);
}
[Fact]
public void ParseEventData_RealKernelGeneralPayload_YieldsAuthoritativeFields()
{
// Verbatim (values shortened) from a live Kernel-General 1 record on a 26220 dev machine.
const string xml =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
"<System><Provider Name='Microsoft-Windows-Kernel-General'/><EventID>1</EventID></System>" +
"<EventData>" +
"<Data Name='NewTime'>2026-08-21T09:40:02.1987028Z</Data>" +
"<Data Name='OldTime'>2026-08-21T09:40:02.1973028Z</Data>" +
"<Data Name='TimeDeltaInMs'>1</Data>" +
"<Data Name='Reason'>1</Data>" +
"<Data Name='ProcessName'>\\Device\\HarddiskVolume3\\Windows\\System32\\svchost.exe</Data>" +
"<Data Name='ProcessID'>6892</Data>" +
"</EventData></Event>";
var data = SystemTimelineTracker.ParseEventData(xml);
Assert.Equal("1", data["TimeDeltaInMs"]);
Assert.Equal("2026-08-21T09:40:02.1987028Z", data["newtime"]); // case-insensitive keys
Assert.NotNull(SystemTimelineTracker.TryParseUtc(data["OldTime"]));
}
[Fact]
public void FormatDuration_CompactForms()
{
Assert.Equal("2h 05m", SystemTimelineTracker.FormatDuration(TimeSpan.FromMinutes(125)));
Assert.Equal("56m 12s", SystemTimelineTracker.FormatDuration(TimeSpan.FromSeconds(56 * 60 + 12)));
Assert.Equal("45s", SystemTimelineTracker.FormatDuration(TimeSpan.FromSeconds(45)));
}
[Fact]
public void ProcessLeafName_StripsDevicePath()
{
Assert.Equal("svchost.exe",
SystemTimelineTracker.ProcessLeafName(@"\Device\HarddiskVolume3\Windows\System32\svchost.exe"));
Assert.Null(SystemTimelineTracker.ProcessLeafName(null));
Assert.Null(SystemTimelineTracker.ProcessLeafName(@"\Device\HarddiskVolume3\"));
}
}
}
@@ -0,0 +1,126 @@
using System;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Transport;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Transport
{
public class DoGroupIdServiceTests
{
// -------------------------------------------------------------------
// ComputeGroupId — RealmJoin byte-layout compatibility
// -------------------------------------------------------------------
[Fact]
public void ComputeGroupId_MatchesHandComputedVector()
{
// Layout: {101,48,6} + {0,0,0} + {ip[1],0,ip[2],ip[3]} + mac — through new Guid(byte[])
// the first three groups print little-endian, the last eight bytes verbatim.
var guid = DoGroupIdService.ComputeGroupId(
new byte[] { 192, 168, 178, 1 },
new byte[] { 0x00, 0x11, 0x22, 0x33, 0x44, 0x55 });
Assert.Equal("00063065-0000-00a8-b201-001122334455", guid.ToString());
}
[Theory]
[InlineData(new byte[] { 10, 0, 0, 1 }, new byte[] { 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF })]
[InlineData(new byte[] { 192, 168, 1, 254 }, new byte[] { 0x00, 0x50, 0x56, 0x01, 0x02, 0x03 })]
[InlineData(new byte[] { 172, 16, 254, 3 }, new byte[] { 0xF0, 0x9F, 0xC2, 0x00, 0x00, 0x01 })]
public void ComputeGroupId_MatchesRealmJoinReferenceImplementation(byte[] ip, byte[] mac)
{
Assert.Equal(RealmJoinReferenceGuid(ip, mac), DoGroupIdService.ComputeGroupId(ip, mac));
}
/// <summary>Verbatim port of RealmJoin's ConvertToGuid — the compatibility contract.</summary>
private static Guid RealmJoinReferenceGuid(byte[] ipBytes, byte[] address)
{
byte[] prefix = { 101, 48, 6 };
var stuff = new byte[4];
stuff[0] = ipBytes[1];
stuff[2] = ipBytes[2];
stuff[3] = ipBytes[3];
return new Guid(prefix
.Concat(new byte[16 - prefix.Length - stuff.Length - address.Length])
.Concat(stuff)
.Concat(address)
.ToArray());
}
[Fact]
public void ComputeGroupId_RejectsNonIpv4AndBadMacLengths()
{
var ip4 = new byte[] { 10, 0, 0, 1 };
var mac = new byte[] { 1, 2, 3, 4, 5, 6 };
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(new byte[16], mac));
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(null!, mac));
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(ip4, new byte[8]));
Assert.Throws<ArgumentException>(() => DoGroupIdService.ComputeGroupId(ip4, null!));
}
[Fact]
public void ComputeGroupId_RegistryFormat_IsBracelessLowercase()
{
// RealmJoin writes SetValue(name, Guid) which stringifies via Guid.ToString() ("d"
// format) — our written value must be the identical string form.
var text = DoGroupIdService.ComputeGroupId(
new byte[] { 10, 1, 2, 3 }, new byte[] { 0xAB, 0xCD, 0xEF, 0x01, 0x02, 0x03 }).ToString();
Assert.DoesNotContain("{", text);
Assert.Equal(text.ToLowerInvariant(), text);
Assert.Equal(36, text.Length);
}
// -------------------------------------------------------------------
// DecideAction — ownership matrix
// -------------------------------------------------------------------
private const string Computed = "00063065-0000-00a8-b201-001122334455";
private const string Other = "11111111-2222-3333-4444-555555555555";
[Theory]
[InlineData(null)]
[InlineData("")]
public void DecideAction_NoExistingValue_Writes(string? existing)
{
Assert.Equal(DoGroupIdAction.Write, DoGroupIdService.DecideAction(existing, Computed, null));
}
[Fact]
public void DecideAction_ExistingEqualsComputed_IsNoOp()
{
Assert.Equal(DoGroupIdAction.NoOp, DoGroupIdService.DecideAction(Computed, Computed, null));
}
[Fact]
public void DecideAction_ExistingEqualsComputed_ToleratesBracesAndCase()
{
// A value written by another tool in "{...}"/uppercase form is still semantically ours.
var braced = "{" + Computed.ToUpperInvariant() + "}";
Assert.Equal(DoGroupIdAction.NoOp, DoGroupIdService.DecideAction(braced, Computed, null));
}
[Fact]
public void DecideAction_ExistingIsOurStaleValue_WritesUpdate()
{
// Gateway changed: the registry still holds the GUID we wrote for the old network.
Assert.Equal(DoGroupIdAction.Write, DoGroupIdService.DecideAction(Other, Computed, Other));
}
[Fact]
public void DecideAction_ForeignValue_Skips()
{
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction(Other, Computed, null));
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction(Other, Computed, Computed));
}
[Fact]
public void DecideAction_NonGuidGarbageExisting_SkipsAsForeign()
{
// Never overwrite a value we can't even parse — conservative by design.
Assert.Equal(DoGroupIdAction.SkipForeign, DoGroupIdService.DecideAction("not-a-guid", Computed, null));
}
}
}
@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Configuration;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.Ime;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
@@ -117,6 +118,16 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
hosts,
"AadJoinHost wires its onRealUserJoined callback to the already-constructed DesktopArrivalHost.");
// Codex review P2 (2026-08-19): EspAndHelloHost.Start() replays the Shell-Core
// ESP exit, but ImeLogTracker only restores its persisted app states in its own
// Start() — which happens later. Pin the order that makes ImeLogHost's
// onStateRestored nudge necessary, so a future reshuffle either keeps the nudge
// meaningful or fails here loudly.
AssertBefore<EspAndHelloHost, ImeLogHost>(
hosts,
"the ESP-exit replay runs before the IME app-state restore, which is why " +
"ImeLogHost re-triggers the user-apps-settled synthesis after LoadState.");
// Closure-latch sanity (factory lines ~139/297): the EspAndHelloHost probes close
// over the ImeLogHost reference, which is assigned during the same factory call —
// the host must therefore exist in the returned list.
@@ -130,5 +141,40 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
}
}
}
[Fact]
public void Ime_app_state_callback_is_chained_into_the_user_apps_settled_recheck()
{
// sits-d Cloud-PC fix (2026-08-19): the AccountSetup synthesis is re-evaluated on every
// terminal IME app transition, which only works while the factory keeps this chain
// wired. It is five lines that a refactor can drop silently and whose absence costs a
// whole class of sessions their completion — so pin it here.
var surfaces = CreateSurfaces();
var hosts = surfaces.Hosts;
try
{
var imeHost = hosts.OfType<ImeLogHost>().Single();
var handler = imeHost.Tracker.OnAppStateChanged;
Assert.NotNull(handler);
// Invoking it must reach the re-check without throwing. The re-check is a no-op
// here (no ESP exit observed), which is the point — it must stay cheap and silent
// on the IME hot path until both facts are in.
var pkg = new AppPackageState("app-1", listPos: 0);
var ex = Record.Exception(() => handler!(
pkg,
AppInstallationState.Installing,
AppInstallationState.Installed));
Assert.Null(ex);
}
finally
{
foreach (var host in hosts)
{
try { host.Dispose(); } catch { /* construction-only test — best-effort cleanup */ }
}
}
}
}
}
@@ -0,0 +1,147 @@
#nullable enable
using System;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
{
/// <summary>
/// The Shell-Core ESP-exit replay is RESTART RECOVERY: it re-reads observations the agent could
/// not make because no agent process was running. The window is therefore measured against the
/// previous run's last known liveness, not guessed from a constant.
/// <para>
/// Why a constant is not enough (Codex review P2): the scheduled task carries a BootTrigger
/// only, with no restart-on-failure. A crashed agent does not come back until the NEXT BOOT —
/// possibly hours later, long after Windows wrote the final ESP exit. Five minutes would lose
/// it every time.
/// </para>
/// </summary>
public sealed class EspExitBackfillLookbackPolicyTests
{
private static readonly DateTime Now = new DateTime(2026, 8, 19, 8, 43, 16, DateTimeKind.Utc);
private const int Default = ShellCoreTracker.BackfillLookbackMinutes;
private static int Resolve(string? exitType, DateTime? lastAlive = null, DateTime? bootUtc = null)
=> DefaultComponentFactory.ResolveEspExitBackfillLookbackMinutes(exitType, bootUtc, lastAlive, Now);
// ---- no previous run => nothing to recover ---------------------------------------
[Theory]
[InlineData("first_run")]
[InlineData("FIRST_RUN")]
[InlineData("")]
[InlineData(null)]
public void No_previous_run_means_no_replay(string? exitType)
{
Assert.Equal(0, Resolve(exitType));
}
[Fact]
public void First_run_stays_off_even_with_timestamps_available()
{
// The exit type decides, not the presence of a timestamp.
Assert.Equal(0, Resolve("first_run", lastAlive: Now.AddHours(-3), bootUtc: Now.AddHours(-3)));
}
// ---- the default is a floor, never a ceiling --------------------------------------
[Theory]
[InlineData("clean")]
[InlineData("hard_kill")]
[InlineData("exception_crash")]
[InlineData("reboot_kill")]
public void Without_any_timestamp_the_default_applies(string exitType)
{
Assert.Equal(Default, Resolve(exitType));
}
[Fact]
public void A_restart_that_returned_within_seconds_keeps_the_default()
{
Assert.Equal(Default, Resolve("clean", lastAlive: Now.AddSeconds(-20)));
}
// ---- the crash case P2 is about ---------------------------------------------------
[Fact]
public void A_crash_recovered_only_at_the_next_boot_gets_the_whole_gap()
{
// BootTrigger-only task: the agent was gone for three hours while the ESP finished.
Assert.Equal(181, Resolve("exception_crash", lastAlive: Now.AddHours(-3)));
}
[Fact]
public void Snapshot_mtime_carries_the_window_when_no_boot_time_exists()
{
// LastBootUtc is only populated for hard_kill / reboot_kill — exception_crash and
// clean have nothing else to go on.
Assert.Equal(46, Resolve("clean", lastAlive: Now.AddMinutes(-45)));
}
// ---- reboot: both inputs available, wider wins ------------------------------------
[Fact]
public void The_wider_of_the_two_inputs_wins()
{
// Snapshot last written 40 min ago, boot 12 min ago — the gap started at the snapshot.
Assert.Equal(41, Resolve("reboot_kill", lastAlive: Now.AddMinutes(-40), bootUtc: Now.AddMinutes(-12)));
}
[Fact]
public void Boot_time_alone_still_covers_a_reboot_without_a_snapshot()
{
Assert.Equal(41, Resolve("reboot_kill", bootUtc: Now.AddMinutes(-40)));
}
// ---- robustness -------------------------------------------------------------------
[Fact]
public void A_timestamp_in_the_future_is_ignored()
{
// Clock skew across a reboot must never produce a negative window.
Assert.Equal(Default, Resolve("reboot_kill", lastAlive: Now.AddMinutes(5), bootUtc: Now.AddMinutes(9)));
}
[Fact]
public void A_future_timestamp_does_not_suppress_a_valid_one()
{
Assert.Equal(31, Resolve("reboot_kill", lastAlive: Now.AddMinutes(9), bootUtc: Now.AddMinutes(-30)));
}
[Fact]
public void A_very_long_gap_is_clamped_by_the_tracker()
{
// The policy may propose more than the cap; ClampLookbackMinutes is the backstop, so
// the two together can never produce an unbounded event-log query.
var proposed = Resolve("exception_crash", lastAlive: Now.AddHours(-20));
Assert.True(proposed > ShellCoreTracker.BackfillLookbackMaxMinutes);
Assert.Equal(ShellCoreTracker.BackfillLookbackMaxMinutes,
ShellCoreTracker.ClampLookbackMinutes(proposed));
}
// ---- the snapshot probe itself ----------------------------------------------------
[Fact]
public void Missing_snapshot_reads_as_null_not_as_an_exception()
{
using var tmp = new Harness.TempDirectory();
Assert.Null(DefaultComponentFactory.ReadPreviousRunLastAliveUtc(tmp.Path));
Assert.Null(DefaultComponentFactory.ReadPreviousRunLastAliveUtc(string.Empty));
}
[Fact]
public void An_existing_snapshot_reports_its_write_time()
{
using var tmp = new Harness.TempDirectory();
var path = System.IO.Path.Combine(tmp.Path, "snapshot.json");
System.IO.File.WriteAllText(path, "{}");
var read = DefaultComponentFactory.ReadPreviousRunLastAliveUtc(tmp.Path);
Assert.NotNull(read);
Assert.Equal(DateTimeKind.Utc, read!.Value.Kind);
Assert.True((DateTime.UtcNow - read.Value).Duration() < TimeSpan.FromMinutes(5));
}
}
}
@@ -0,0 +1,72 @@
#nullable enable
using System;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Orchestration
{
/// <summary>
/// Codex review P2 (2026-08-19) — <see cref="ImeLogHost"/> starts AFTER <c>EspAndHelloHost</c>,
/// and <c>ImeLogTracker.Start()</c> restores the persisted package states silently (LoadState
/// raises no <c>OnAppStateChanged</c>). On an agent restart the Shell-Core ESP-exit replay
/// therefore runs against an empty app list. Without the post-restore nudge the exact reboot
/// case the completion recovery exists for — apps already terminal BEFORE the reboot, only the
/// final exit lost to the downtime — would still never complete.
/// </summary>
public sealed class ImeLogHostStateRestoredCallbackTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
public void Dispose() => _tmp.Dispose();
private ImeLogHost CreateHost(Action? onStateRestored)
=> new ImeLogHost(
sessionId: "s",
tenantId: "t",
logger: new AgentLogger(_tmp.Path, AgentLogLevel.Info),
ingress: new FakeSignalIngressSink(),
clock: new VirtualClock(new DateTime(2026, 8, 19, 8, 43, 16, DateTimeKind.Utc)),
imeLogPathOverride: _tmp.Path,
imeMatchLogPath: null,
imePatterns: null,
stateDirectory: _tmp.Path,
whiteGloveSealingPatternIds: null,
onStateRestored: onStateRestored);
[Fact]
public void Start_invokes_the_state_restored_callback_exactly_once()
{
var calls = 0;
using var host = CreateHost(() => calls++);
host.Start();
try { Assert.Equal(1, calls); }
finally { host.Stop(); }
}
[Fact]
public void A_throwing_callback_does_not_break_host_start()
{
// Fail-soft like every other co-wired callback in the orchestration layer: a broken
// re-check must never take the IME log host — the session's primary signal source —
// down with it.
using var host = CreateHost(() => throw new InvalidOperationException("recheck boom"));
var ex = Record.Exception(() => host.Start());
try { Assert.Null(ex); }
finally { host.Stop(); }
}
[Fact]
public void No_callback_is_a_valid_configuration()
{
using var host = CreateHost(null);
var ex = Record.Exception(() => host.Start());
try { Assert.Null(ex); }
finally { host.Stop(); }
}
}
}

Some files were not shown because too many files have changed in this diff Show More