When the event limit or the platform operator blocks a session's device, the
session now carries a block record: Block* fields on the session and its index
row, a server-authored device_blocked timeline event, an interim analyze run for
that event and, on the auto path, an audit entry. The block list itself stays
platform-only.
- Portal: a card above the failure snapshot says the monitoring was paused
(orange) or ended (red), when, until when and, for the event limit, the
session's event count against the limit. The Blocked pill reads the record for
every role; Global Admin mode still adds the live block list.
- A Kill is device-wide until it expires, so its record carries the end and the
card says new enrollments on the device are not monitored until then.
- Status: the stalled and timeout reasons name the block instead of a silent
agent; the classification is unchanged.
- Rules: ANALYZE-ENRL-006 explains the pause right after it (on_event
device_blocked); ANALYZE-ENRL-002 no longer fires for a blocked session.
- MCP: the coverage block of get_session_summary names the block as a gap.
- The agent-facing block message no longer says "by an administrator".
- BlockCause == event_limit replaces ExcessiveEventsAutoActioned as the
auto-action's idempotency marker.
- Privacy page: the event limit next to rate limiting.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All 54 shipped analyze rules get role, summary, subject and key facts as
{{signal.key}} templates, each with a minor version bump so the self-seed
picks them up. Values were checked against stored findings, the rows that
receive the block at read time.
- ENRL-001 repeats the outcome only: the 0x87D1041C paragraph and the
placeholder note are gone, labels read "ESP step that failed" / "ESP
error code", remediation is "Find the cause". This changes its alert text.
- ENRL-002 says the sweep "closed" the session (it can end Incomplete).
- ID-002 and ID-004 gain evidence-only conditions (unexpected account
names; IME token failure count and codes); firing and confidence are
unchanged.
- Resolution: facts sharing a label are alternatives, an exact repeat of
label and value is left out (new shared fixture case, web module and
its generated MCP copy).
- Ratchet: every built-in presentation passes the save validator, uses
qualified tokens only, and each token names a condition of the rule and
a key its source records (RuleTemplateInterpolator.TokenRegex internal).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent's ntp_time_check reports offsetSeconds = NTP time - device time,
so a positive value means the device clock is BEHIND. The session card
said "ahead of UTC" for it (a device 10 h behind showed as 10 h ahead),
and ANALYZE-DEV-007 named its two signals the wrong way round.
- SessionInfoCard: the direction comes from deviceClockDirection
(sessionInfoLogic.ts, pinned by a test with both signs).
- ANALYZE-DEV-007 1.1.1: offsetSeconds > 120 is clock_behind, < -120 is
clock_ahead. Conditions, thresholds and text are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Field measurement 3: a first-run agent that started after the update page (6a7d241d) left the whole
update in Apps (ESP), failed and skipped updates were reported by nothing, and the lane listed a
failed update's staged servicing stack as an installed KB.
- Time attribution v5: update names before the first observed OobeNDUP start begin the update at the
earlier of the first name and the update's first servicing step (after the window's ESP exit when
one was seen before the name, else the first step of a package the update also serviced later).
The fallback's ESP exit is per window, so a WhiteGlove part-1 exit no longer begins an update in
part 2. OsUpdateSpan carries Outcome (installed/failed/skipped/unknown, the latest evidence wins)
and NotInstalledKbs; Kbs lists only packages that reached Installed. The 2-h sweep recomputes the
last 30 days.
- ANALYZE-DEV-013 (update failed), DEV-014 (not offered after an earlier failure), DEV-015 (update
check ended without a result): warning, never KO; a failure only from explicit names, every
absence condition gated on the agent's names_capped marker; never two at once.
- ShellCoreTracker rows never set the session start (D-315 list), ahead of the agent's first-run
update page backfill.
- The web lane names the outcome and the KBs not installed; MCP get_time_attribution mentions both.
Real-event replay: 6a7d241d os_update 2358 s installed (was 55 min in esp_apps), e7551b82 failed with
KB5124007 not installed. Rule dry runs and a sweep over all 97 update-page sessions since 2026-10-02
hit exactly the five field sessions. Functions 6,918, web 1,701, MCP 997 green.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Report 3a2207978e4c: a session was failed 30 min after the ESP exit while Windows installed its
OOBE quality update. The update watcher had been blind since July and the engine had no notion
of the phase.
- Windows Update client: 19/20/43/44 come from the System log (provider-filtered XPath), as the
provider manifest routes them; 25/26/31/41 from Operational via their own config list.
Updates are classified os/defender/store/other: only Windows/.NET updates become rows, the
rest is counted into windows_update_activity_summary. Per-channel watermarks persist a record
only after its row and engine signal were handed to the ingress.
- Servicing: CBS steps from the Setup log (windows_update_servicing), client UpdateAgentLCU.
- Engine: OsUpdateActivity and OsUpdateFacts (event-time ordered) anchor an OOBE update phase
after the ESP exit. Each state has its own bound: the update runs at most 3 h (then Failed
oobe_update_not_finished), after it the user has 60 min to sign in (then the engine stops
waiting without a verdict), after the sign-in one window of grace.
- The phase acts only in OobeUpdatePhaseMode=Active. Shadow, the default, reports what it would
decide as oobe_update_phase_shadow right before today's verdict. The agent stamps the mode on
EnrollmentFactsObserved at every start; a later stamp can take Active away, never grant it.
- Agent: a stop without a verdict ends the session like the max-lifetime watchdog
(agent_shutting_down reason=stopped_waiting); the backend classifies it honestly (verdict
origin stopwait), never Failed.
- Measurement: oobe_update_page (the update page and its ExpeditedUpdate_* names; a 62407 value
only as boolean, integer or HRESULT) and oobe_update_state (NDUP and ESP-policy registry
values), both Debug and bounded per run.
- ANALYZE-DEV-004 1.2.0 counts install and download failures only, never scan failures.
- Event types in Constants, MCP catalog and web; shared manifests regenerated; privacy page
names Windows updates installed during the enrollment.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
office_install_failed is emitted only for an error code in the C2R INSTALL scenario; a window without on-disk proof ends silently. The old text described a StreamingFinished-based failure and attributed the early "done" of the built-in app type to IME, which never sees Office CSP installs. Text only, version 1.1.1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tenant admins submit references to their own gather/analyze rules; the server freezes the rule, records pre-flight findings and fire stats, and stores it under a 12-character id (RuleSubmissions, SessionReports layout). Attribution is the submitter's frozen choice (anonymous by default) and becomes the published author. Review (approve with reserved id / decline with comment) through /admin/reports/rule-submissions or the MCP tools list_rule_submissions, get_rule_submission, review_rule_submission, reseed_rules_from_github; published is derived from the global catalog, never stored. Analyze rules now get the same server-side author stamp as gather rules; ANALYZE-ID-001 carries author Community. Privacy page names the credit; bumps internal and .claude/commands gitlinks.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ANALYZE-ESP-004 fired on every ESP terminal failure of a Continue-anyway
profile and rendered the configured FirstSync limit as elapsed time — the
mayHaveContinuedAnyway flag only says the profile has the button. The rule
now requires the agent's own verdict (app_install_failed with
failureType=esp_apps_timeout), names that app via a same-event filter, reads
failedSubcategory from esp_failure_settle_started and words the limit as a
limit. Verified with test_analyze_rule: cb7036a6 fires, 683f1eff (0x80070652
after 19 min) stays silent.
Rule engine: not_exists with a dataField on event_type/event_data
conditions matches when no event of the type carries a non-empty value —
including an absent type — instead of being dead behind the fieldValue null
short-circuit. Schema and MCP authoring guide say so.
Agent: a Shell-Core 62407 failure that lands inside the provisioning
tracker's 30-s settle window (a third of all ESP terminal failures) now
inherits the pending errorCode / failedSubcategory / category /
likelyCulpritApps, so enrollment_failed, Sessions.FailureReason and the
termination-time app classification see the HRESULT the registry had
already named. failureType stays the Shell-Core value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- IME-EXITCODE-CLASS captures the return-code class of the installer exit code ("lpExitCode is defined as <Success|SoftReboot|HardReboot|Retry|Failed>", verified against the IME string literal) as exitCodeClass on AppPackageState; new action captureExitCodeClass, persisted across the agent restart a reboot causes, last value wins across a Retry loop. IME-EXITCODE-MAPPED-FAILED keeps the error path; both match the same line.
- Adapter: exitCodeClass whenever known, exitCode on every terminal state (Installed/Postponed/Skipped/Error) instead of Error only.
- ANALYZE-APP-018 (warning, apps): app_install_completed.exitCodeClass in SoftReboot,HardReboot; +25 HardReboot, +15 observed system_reboot_detected; never fails the session.
- Web: completed-app exit badge amber (failed stays red), "mapped as SoftReboot/HardReboot/Retry" on timeline and Install Progress.
- Tests: pack end to end (phase, current app, class lines, negatives, restart round trip), adapter payloads, linearity samples, backend rule engine + known actions.
- internal gitlink: plan, D-214, backlog.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
hybrid_login_pending now measures the absence of a real-user desktop after a reboot
(cancelled by the DesktopArrivalDetector or a real JoinInfo user), is never armed on
WhiteGlove devices (resume marker, Part-2 flag, .part1- archive) and reports the
JoinInfo placeholder as a fact only — on a Hybrid join the AD sign-in never replaces it.
New EntraUserAffinityDetector: armed at the real-user desktop on Hybrid devices, the
first IME-TOKEN-SUCCESS line after the desktop posts ime_user_token_acquired; ten
minutes without one post entra_user_affinity_pending with the IME token-failure codes.
ImeLogTracker exposes raw per-line token callbacks past the historic-replay gate;
HelloTracker surfaces User Device Registration 304/305 as device_registration_event.
ANALYZE-ID-004 v2.0.0 keys on those events (event_type not_exists gained a branch),
GATHER-DEVICE-006 follows, GATHER-DEVICE-007/008 collect the Entra operational log.
Constants + MCP catalog gain the five types (D-072), the guard test learns the
multi-line OnTraceEvent shape, gather_* outputs pass the MCP filter gate by prefix.
CMTrace: ImeLogEraPreScan/CmTraceEraTable anchor the backlog era that holds the
bootstrap execution record against the Deployed install marker (0.13-0.36 s in the
field), whole era or nothing, sibling files by local time; origin era-anchored.
Backend phase_duration/app_install_duration refuse a reader-zone-fallback endpoint.
Decisions D-190, D-191. Session a7140f98.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Agent: ProvisioningStatusTracker.HasAccountSetupActivity now requires at least one
AccountSetup subcategory past notStarted. Windows writes AccountSetupCategory.Status with
every subcategory notStarted BEFORE the Device-ESP page exits, so the presence check
confirmed the Device->Account handoff (Shell-Core 62407) as the final ESP exit; the
user-apps-settled synthesis then completed AccountSetup and released the User-ESP
keep-awake minutes before the real exit (fleet: 77 % of keep-awake sessions, median 3 min;
on battery that is enough for Modern Standby). D-183.
Rule engine: filterField/filterOperator/filterValue now apply to event_data and
event_data_array as a same-event filter (shared ApplyValueFilter with event_count).
ANALYZE-DEV-012 v1.1.0 excludes backfilled pre-session episodes (backfilled equals false);
a second independent condition could not, since each condition scans all events on its
own. D-184. Schema, Shared model, MCP authoring guide and generated types follow.
Tests: ProvisioningStatusTrackerAccountSetupActivityTests (the verbatim all-notStarted
registry JSON), SleepEpisodeRuleTests (+3 backfill cases), RuleEngineEventDataFilterTests.
Solution build 0 warnings; agent suite 2820 and backend suite 5791 green.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- .gitmodules: docs -> internal, repo renamed to Autopilot-Monitor-internal; the OKF bundle now lives at internal/docs/
- .claude/CLAUDE.md rewritten: where knowledge lives, working-file rules, hard rules; imports the private half from internal/CLAUDE.md (silently absent without submodule access)
- .claude/architecture-principles.md moved into the internal bundle (reconciled against the code)
- tasks/ (gitignored) retired; working files live in internal/work/
- doc path references docs/<bundle> -> internal/docs/<bundle> in code comments, skills gitlink, codeql paths, build script
- rule descriptions no longer cite an internal doc path (customer-visible text); rules/dist regenerated
- public tree no longer names private operator tooling
- .claude/commands gitlink: skill docs point at the new paths
The shutdown run allowed every logged-in user (explorer.exe owner) dynamically, so a backdoor account created after the startup scan and signed in before enrollment completed was scored 0. The WMI inventory is now read first and a logged-in name that is a local account is rejected for the account and the profile check (reported as logged_in_local_accounts); only Entra/domain identities — the only ones seen in real sessions — remain dynamically allowed for their profile folder.
Built-ins are recognised by RID 500/501/503/504 on the machine SID in addition to their English names: since disabled accounts are inventoried (88dfd331), localized Gast/Invité/Administrateur would otherwise flag every non-English device. kioskUser0 (Assigned Access autologon) joins the built-in list; the profile check skips junctions and $<32 hex> folders (cannot be an account name, pure payload noise). ANALYZE-ID-002 1.3.0 explanation mentions the new field.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two analyze rules born from the pilatus-aircraft diagnosis (2026-08-27):
- ANALYZE-APP-017 "App Install Blocked by Another Installer (0x80070652)":
fires on app_install_failed with hresultFromWin32 -2147023278
(ERROR_INSTALL_ALREADY_RUNNING); confidence boosts for MSI exit code
1618 and for an ESP failure settling with the same code. The 08-26
burst failed 16 sessions deterministically at ~21 min (IME's fixed
msiexec-busy retry budget) right after a preceding installer left the
Windows Installer mutex held - previously only generic
"Enrollment Failed" fired, naming no cause.
- ANALYZE-NET-001 "Diagnostics Upload Blocked by Web Security Gateway":
fires on diagnostics_upload_failed answered by an HTML block page
carrying a recognizable security-product marker (McAfee/Skyhigh,
Zscaler, Netskope, ...). OnFailure diagnostics are lost exactly on the
sessions that need them; plain HTML errors without a vendor marker
(stale/stopped endpoint) stay excluded on purpose.
Both validated via MCP validate_rule (0 errors) and dry-run with
test_analyze_rule: fire on the burst session (confidence 100 / 90),
silent on a timeout-shaped failure and on a succeeded session that
carried unrelated app_install_failed events.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two enrollment blind spots become timeline ground truth from the System event
log via one EventLogWatcher (WindowsUpdateTracker pattern, 24h backfill with
RecordId watermark recovering pre-agent records):
- system_clock_changed (Kernel-General 1): oldTime/newTime/signed timeDeltaMs,
reason and the setting process. 2s floor suppresses w32time 1ms micro-slews;
steps >= 5min are Warning + immediate upload. Payload instants are the
authoritative source (event timestamps can be clamped server-side).
- system_sleep_episode (Power-Troubleshooter 1 classic sleep/hibernate,
Kernel-Power 507 Modern Standby; 506 deliberately unconsumed): kind,
enteredAt/exitedAt, durationSeconds, wake source / AC state. 60s real-sleep
floor filters Modern Standby screen-off scenarios.
ConfigVersion 39 (Collectors.EnableSystemTimelineWatcher +
SystemEventBackfillLookbackMinutes). Both types are non-activity on agent and
backend (idle/stall clocks unaffected; a sleeping device stays Stalled).
Consumers: TimeJumpBadge gains the ground-truth cause "clock-set"
(classifyTimeJump matches backward display steps against recorded clock sets);
EventTimeline badge blocks for both events plus the previously bare
power_state_change; SessionInfoCard keeps the wall-clock duration and adds a
standby note; TimeAttributionCalculator builds SleepSpans (RebootSpan clone,
AttributionVersion 3) persisted as SleepSeconds/SleepSpansJson and rendered in
TimeAttributionLane; ANALYZE-DEV-012 (info, single) fires once per session for
episodes >= 5min - deliberately the only new rule, clock findings stay with
DEV-008 to avoid duplicate-finding flood.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Event-driven AC/battery observation during enrollment (no polling): a WMI
Win32_PowerManagementEvent (EventCode 10) push subscription re-probes
GetSystemPowerStatus and diffs snapshots through a pure tracker into
power_state_change events - transition ac_to_battery (Warning, immediate) /
battery_to_ac (Info) / threshold_crossed at 50/30/15% (per-level lifetime
latch, lowest-crossed-only on jumps, arm-time baseline covers sessions
starting low, 5s trailing-edge debounce, emission cap 20). Never armed on
devices without a battery; arm failure surfaces as collector_degraded.
Kill-switch Collectors.EnablePowerStateWatcher (ConfigVersion 38).
power_state_change is classified non-activity on both sides
(SignalActivityClassifier + IsPeriodicOrStallEvent) so battery drain can
neither keep idle collectors alive nor heal a Stalled session.
New analyze rules, both with on_event:power_state_change interim triggers
and deliberately single-required-condition (event_data conditions scan
events independently - no same-instance join):
- ANALYZE-DEV-009 (high): thresholdPercent equals 15
- ANALYZE-DEV-010 (warning): transition equals ac_to_battery
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Operator side: CmTraceSkewTripwire piggybacks on the terminal counter
reconcile's partition scan (zero extra reads) and compares median
ReceivedAt-OccurredUtc deltas of IME-derived vs other events. A clean
15-min-grid divergence (residual < 2 min, >=20 samples / >=3 distinct
batches per side) emits the CmTraceTimeSkewRegression ops event
(Agent/Warning, goal: never fires). Bias-dominated sessions are
suppressed; measuredWriterOffsetMinutes is never consulted (sticky
after era flip-backs). Kill switch: CmTraceSkewTripwireDisabled.
Customer side: new clock_skew analyze condition source (skewMetric
clock_jump | sustained_offset, threshold seconds, gt/gte magnitude)
behind built-in ANALYZE-DEV-008 (300 s = the Kerberos/Entra 5-min skew
boundary). Batch-median frames with spool-spread filter, plateau step
detection (a gradual backlog-drain ramp cannot fake a jump) and
end-state persistence; IME-derived events are excluded so a CMTrace
anchoring regression can never fire the customer rule. Evidence carries
field=skewSummary/value for {{skewSummary}} interpolation.
Full mirroring chain: schema enum + skewMetric property, shared
manifest + combine.js regeneration, MCP authoring guide + semantic
lint, web SOURCES/form/card, ops-event dual-register, docs. P5 is
subsumed by the tripwire. PercentileMath is now the single nearest-rank
implementation (Apps/PlatformMetrics delegate).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seven sits-d Cloud PCs enrolled the same day with identical configuration
(~138 required apps at the user-ESP gate, DeviceGuard/DmaGuard device-assigned).
The two that finished AccountSetup before the coalesced reboot landed succeeded;
the five the reboot killed in Stage=EspAccountSetup never derived a completion
again and ended Incomplete on healthy devices. Two defects behind that:
ShellCoreTracker.BackfillRecentEspExitEvents() has existed since session
772fe502 but no caller ever invoked it, so the Shell-Core watcher only saw
records written after Start(). Every agent restart silently dropped an ESP exit
(62407) or Hello-wizard start (62404) from its downtime. EspAndHelloHost.Start()
now calls it, and the fixed 5-minute window is sized to the actual downtime when
the previous run died to a reboot (LastBootUtc, clamped to 360 min).
The user-apps-settled synthesis ran only on the EspExited edge. When the ESP page
exits with apps still in flight the single attempt misses and nothing re-checks —
8110e262 and a89aac2d both reached 138/138 apps, 0 failed, and still never
completed. The exit edge is now recorded and the synthesis re-evaluated from
ImeLogTracker.OnAppStateChanged. No new completion path: the gate conditions are
unchanged, this only gives the existing conservative check a second look. The
fire-once claim moves to Interlocked (edge and re-check run on different threads)
and the re-check does not re-emit app_install_starved.
Rules: ANALYZE-ESP-005 gains on_event:session_stalled so a hung session produces
a finding at the 60-minute stall instead of staying finding-less for hours
(verified against 8110e262 — fired, confidence 95). New ANALYZE-APP-016 turns the
already-emitted app_install_starved events into a customer finding and, unlike
ANALYZE-APP-015, fires even on a succeeded session: 3b9291aa was recorded
Succeeded with 99 required apps that never started installing.
ANALYZE-APP-015 and ANALYZE-ESP-001 were considered for the stall trigger and
deliberately left alone — APP-015 is the documented worked example of the
terminal-precondition trap, and ESP-001's phase_duration gate on a still-open
phase is unverified mid-run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
From the 3b7ba31f (succeeded) vs gardner/esa (parked) hybrid session comparison:
explanation now covers the cached-credentials/pre-logon-VPN variant and why the
Hello wizard never appearing is downstream of the missing PRT; new remediation
block for VPN machine tunnels, cloud Kerberos trust, explicit Hello disable,
update deferral and pre-provisioning. Links the new docs article.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Full verify pass over all 54 built-in rules (validate_rule sweep, 30d/7d
get_rule_stats, test_analyze_rule spot checks). Fixes:
- ANALYZE-APP-007 v1.1.0: declare enrollment_failed as a non-required
condition — the exists-factor referenced an undeclared signal and could
never apply.
- ANALYZE-APP-014 v1.1.0: add non-required event_data conditions for
espCategory/espErrorCode/topCandidatePackage/candidateCount/confidence
so all explanation {{tokens}} resolve instead of rendering literally.
- ANALYZE-ENRL-002 v1.2.0: capture session_timeout.timeoutHours so
{{timeoutHours}} interpolates (verified via dry-run: renders "5").
- RuleEngine TryFlatLookup/ResolveDotPath: flatten IEnumerable data
values to a comma-joined string — List fields (e.g. app_tracking_summary
.pendingNames) stringified to "System.Collections.Generic.List`1[…]",
breaking contains-matching and {{pendingNames}} interpolation.
- GetRuleStatsAsync: raise the 500-row cap to 10000 and log when hit.
The scan walks dates ascending, so the old cap silently dropped the
NEWEST dates of a 30d window — newly added rules (e.g. ANALYZE-SEC-008)
were entirely missing from global stats.
- MCP validate_rule lint: accept event_data_array itemField as a {{token}}
resolver — {{identity}} in ANALYZE-SEC-005/006 was a false positive.
Backend 4157 tests green, MCP 548 green + tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ANALYZE-SEC-002 was the sunset AutoLogon rule; BuiltInRulesTests pins
that the ID never returns (the CI failure on main). Reusing a retired
ID would also inherit any historical tenant state keyed to it. Rule
content is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fires on secureboot_status.uefiSecureBootEnabled=false; legacy BIOS
devices (registry key absent, agent sets note field) are skipped via
precondition. +20 confidence on physical devices. Validated with MCP
validate_rule (0 findings) and test_analyze_rule dry-runs: fired at 90
on a physical SecureBoot:False session, silent on a SecureBoot:True
session.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DeviceSetup/Apps failed, the user pressed "Try again" ~23 min later, apps
recovered to 10/10 and DeviceSetup resolved - yet the advisory_completion
window un-defanged the stale failure and failed the live enrollment,
because the advisory variant was exempt from all re-arm/rebase guards.
Engine fix (DecisionCore):
- New facts EspAdvisoryFailureResolvedUtc + EspAdvisoryFailureCategory
(state-borne; deadline payload would be lost on re-arm/rebase).
- TryResolveAdvisoryOnCategoryRecovery in both ProvisioningComplete
handlers: category match sets the fact + emits
esp_failure_advisory_resolved.
- Guards now (!hasAdvisoryAnchor || advisoryResolved): enforcement-progress
re-arm, hello-never-observed promote and the reboot rebase apply again
once the failure demonstrably un-happened; unresolved advisories keep
the legacy un-defang exactly.
- Truthful terminal reason esp_recovered_without_completion_evidence
(+ failureRecovered=true, LastFailureTrigger=DeadlineFired so the
likely-stuck app promotion stays off).
Story events (agent tracker):
- esp_failure_retry_detected: failed subcategory leaves "failed" after the
terminal fire - consistent with the user pressing "Try again".
- esp_failure_recovered: previously failed category reaches success.
- EspFailureDetectedEventArgs.LikelyCulpritApps (never-started first,
cap 3) rides the EspTerminalFailure payload so esp_failure_advisory /
enrollment_failed name the culprit app directly.
Surfacing: MCP resource-catalog, web timeline badges (Recovered /
User retry pills, culprit chip on advisory), gather-rule autocomplete,
ANALYZE-ESP-006 "ESP Failure Recovered After User Retry".
Tests: AdvisoryRecoveryTests (13, incl. the 527/611/635/671 field-ordinal
replay that no longer fails the session), EspFailureRecoveryStoryTests (5),
snapshot-builder guards 18 -> 20. All suites green (DecisionCore 493,
V2.Core 2018, Backend 3201, web vitest 589, MCP 449 incl. drift).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Customer report: ANALYZE-SEC-001 warned on devices whose firmware already
carried the Windows UEFI CA 2023 certificate — the agent only read the
SecureBoot\Servicing registry value, which mirrors the Windows Update
rollout and lags behind reality (OEM factory image, WinCS, WU not run yet).
Agent: new UefiSecureBootCertReader reads the UEFI db/KEK variables
natively (GetFirmwareEnvironmentVariableExW; SeSystemEnvironmentPrivilege
enabled on the process token and restored — first privilege-adjustment
code in the repo). Fail-soft, never throws; legacy BIOS maps to not_uefi.
secureboot_status gains per-certificate booleans, uefiFirmwareReadStatus,
and the one-sided marker uefiCa2023FirmwareConfirmed (emitted only on
positive confirmation, never as false).
Rule v3.0.0: not_exists precondition on the marker suppresses the rule
when the firmware confirms the certificate; the marker's absence keeps
old agents on exact v2 behavior, so the rule is safe to seed before the
agent rollout. Firmware-verified-missing raises confidence (+20 db,
+10 KEK 2K CA 2023).
Verified: full solution build; backend 3201, agent 1988, web 582 tests
green; tsc/ajv clean; elevated cross-check on real firmware matches
Get-SecureBootUEFI on all seven certificate probes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rework after the first real enrollment (session b2e890c1) exposed three
defects in a654c730:
- Shell-Core "RebootCoalescing" match was a false positive by construction:
the token appears in the ROUTINE SubcategoryProcessing_Started bootstrap
marker ("Starting subcategory DeviceSetup.RebootCoalescing...") on EVERY
enrollment. esp_reboot_coalescing removed entirely (branch, constant,
catalog, tests).
- 2800 reboot-required flags fire at every policy sync; only DeviceSetup-time
flags cause the coalesced reboot. The session's flags fired during
AccountSetup - no reboot - yet our Warning events claimed "device will
restart". Raw event is now a neutral Info observation; the reboot CLAIM
moved to ANALYZE-ESP-005 with a REQUIRED system_reboot_detected >= 1
condition (no observed reboot -> advisory silent).
- One event per 2800 record was spam: records are now buffered and debounced
(10 s) into ONE aggregated event per burst (rebootUris[]/uriCount/
firstRebootUri/recordCount/unparsedCount). Watermark is persisted at FLUSH,
not claim - a pre-emit reboot kill loses nothing, the post-restart backfill
re-emits the burst at its historical time.
Verified 2800 description shape recorded in code, tests, and concept doc:
"The following URI has triggered a reboot: (./Device/...)."; extraction
covered by a real-text regression test. Lesson added to tasks/lessons.md
(observation-level events vs outcome-gated claims; substring tokens appear
in framework bookkeeping records).
Tests: Agent.V2.Core 1867/1867, Functions 3099/3099, MCP 400 passed,
solution build 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Device-assigned MDM policies whose CSP URIs match the OS reboot-required
catalog force a coalesced reboot at the end of ESP DeviceSetup - the user
sees an unexpected restart and a second sign-in before AccountSetup.
Invisible in the Intune console; admins simply lived with it. Now attributed
and actionable (PMPC research).
- MdmRebootPolicyTracker (+host): watches DeviceManagement-Enterprise-
Diagnostics-Provider/Admin EventID 2800, emits one mdm_policy_reboot_required
per reboot-forcing URI (Warning, ImmediateUpload; tolerant EventData-first
URI extraction with regex fallback; fail-soft OMADM RebootRequired probe).
Cross-restart RecordId watermark (mdm-reboot-watermark.json) - mandatory,
the coalesced reboot restarts the agent and the backfill re-reads the log.
- ShellCoreTracker: RebootCoalescing branch on the already-watched 62407
stream -> esp_reboot_coalescing corroboration event; deliberate deviation:
emits on backfill too (live emit races the reboot kill).
- ANALYZE-ESP-005 advisory (warning): names the first URI via {{rebootUri}},
remediation = reassign profiles device -> user groups; +20 confidence on
coalescing corroboration, +10 on >=2 URIs.
- Config: Collectors.MdmRebootPolicyWatcherEnabled (default ON, internal
steering only - agent-native collector, NOT an IAgentAnalyzer, no admin UI),
MdmRebootPolicyBackfillLookbackMinutes=60; ConfigVersion 35.
- Event types deliberately distinct from system_reboot_detected so the
backend RebootCount (exact-string match, 2 sites) stays uninflated.
- Docs: docs/agent/mdm-reboot-coalescing.md (+index/log). Exact 2800/62407
message texts are unverified until E2E VM validation (tolerant matching).
Tests: Agent.V2.Core 1868/1868 (21 new), Functions 3099/3099, MCP 400 passed
(event-types drift), solution build 0 errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The agent is normally installed by an IME platform script AFTER MDM
enrollment succeeded, so pre-enrollment failures (enrollment blockers,
TPM attestation) are invisible in the normal flow — no enrollment means
no agent and no authenticable session. Await-enrollment mode does not
help either: the binary only polls for the MDM certificate and cannot
authenticate without it. Only bootstrap-token deployments (one pilot
customer) can report pre-MDM.
- New Observability section in docs/agent/autopilot-ztd-diagnostics.md
with a per-rule visibility matrix (normal / bootstrap / await) and the
follow-up option of backfilling Autopilot-channel error records
- ANALYZE-ENRL-003/004 descriptions state the limitation explicitly so
portal users understand why these rules rarely fire
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Session 7443317c proved a WU-tracker blind spot: an OOBE quality update
installed (OsBuild 26200.8037 -> 26200.8655 across the mid-enrollment reboot)
yet the deployed WindowsUpdateTracker emitted ZERO events - the watcher armed
fine (no collector_degradation), the spool is durable, and two 60-min backfills
covered the install window, so the WindowsUpdateClient channel simply carried
none of the targeted EventIDs 19/20/43/44. The tenant has diagnostics upload
disabled, so the fix makes the feature self-evidencing instead of guessing IDs:
- OsBuildChangeHost: persists CurrentBuild.UBR (Registry64 view) to
AgentState/os-build-state.json and emits os_build_changed (Info, immediate)
when it differs across a restart - deterministic update corroboration that
cannot be lost to timing, watermarks, or unexpected servicing paths. First
run seeds silently; corrupt state re-seeds.
- WindowsUpdateTracker channel census: when the build provably changed AND zero
targeted WU events were emitted this run, emit a one-shot
windows_update_channel_census (Debug) with an unfiltered EventID histogram of
the WindowsUpdateClient AND UpdateOrchestrator channels over the lookback
window (capped at 5000 records, cap surfaced as censusTruncated). The
histogram arrives via normal event upload and tells us which IDs to add to
WindowsUpdateTargetedEventIds via remote config - no redeploy, no diag ZIP.
Gated by CollectorConfiguration.WindowsUpdateChannelCensusEnabled (default
on, model-default pattern, no ConfigVersion bump).
- No new hardcoded EventIDs (per the device-validation rule) - census data
decides what to enable remotely.
- ANALYZE-DEV-006 "OS Build Changed During Enrollment" (info, not KO):
interpolates {{previousBuild}}/{{currentBuild}}, counts the census as
corroboration. Catalogs: Constants.cs + MCP resource-catalog.ts + web
eventTypes.ts (drift-test enforced).
Tests: 6 OsBuildChangeHost (seed/same/changed/unreadable/corrupt/no-dir),
7 census (gating matrix, truncation surfaced, scanner-throw swallowed,
histogram format), 3 ANALYZE-DEV-006. Agent.V2.Core 1754/1754, Backend
2874/2874, MCP 379 (drift green), web tsc + vitest 367/367, sln 0 errors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Session 2bc884b6: ESP failed AccountSetup/Apps with 0x80073cf9 while all 8
tracked Win32/WinGet apps succeeded — the failing app was a MSIX/Store app
that installs outside the IME Win32 pipeline and is invisible to
ImeLogTracker. The session ended as a blackbox failure.
When an Apps-subcategory failure arms the 30s settle window, the tracker now
queues a one-shot scan of Microsoft-Windows-AppXDeploymentServer/Operational
(Critical+Error, session window, 200-record/10s cap), scores candidate
packages (exact-HRESULT match > AppX-family error > time proximity, recency
bonus), and emits esp_appx_failure_analysis with flat RuleEngine fields plus
a capped candidates[] detail list — before enrollment_failed lands in the
timeline. Negative results are emitted too (no_appx_candidates points back
at Win32). Fire-once per session; scan runs lock-free on the threadpool;
settle expiry never waits; failures degrade to collector_degraded.
- New: AppxDeploymentFailureScanner (one-shot reader, injectable seam),
AppxFailureAnalyzer (pure scoring/verdicts, incl. negative-decimal
HRESULT + PackageFullName extraction from message/properties)
- ProvisioningStatusTracker: optional scanner/dispatcher ctor seams,
MaybeStartAppxFailureScan at settle-arm time
- Rule ANALYZE-APP-014 (MSIX/Store app failure during ESP) keyed on
verdict=appx_candidate_identified; dist regenerated
- MCP resource-catalog + web HRESULT badge + gather-rules eventTypes
- Tests: 16 analyzer + 7 trigger; settle-window fixture now injects a
fake scanner + sync dispatcher (no real event log in tests)
Inert until agent redeploy (rule needs backend redeploy). Post-deploy
checks in tasks/todo.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windows Update activity during Autopilot OOBE/ESP is a blind spot no other tool
(Intune console included) surfaces well — a cumulative update installing mid-OOBE
can break enrollment (r/Intune KB5095189). This makes it visible with secured
evidence.
Agent (net48):
- WindowsUpdateTracker + WindowsUpdateWatcherHost: watch
Microsoft-Windows-WindowsUpdateClient/Operational (EventIDs 19/20/43/44 =
success/failure/install-started/download-started), startup backfill (OOBE updates
run before the agent starts), RecordId watermark dedup (restart boundary vs
intra-run seen-set so a high-id live event can't hide older backfill events),
HRESULT->symbol decode. Emits windows_update_succeeded/_failed/_started with the
real event Timestamp. Default-on config knobs; uncertain EventIDs configurable
without redeploy.
- RegistryCollector: emitOnlyIfExists (suppress positive-assertion events when the
key is absent) + Registry64 base view by default (bitness-independent, matches
TenantIdResolver) with optional registryView override.
Rules:
- ANALYZE-DEV-004 (high: WU failed during enrollment, high-insight not KO) +
ANALYZE-DEV-005 (info: WU installed). reboot_pending corroboration matches
event_data exists==true.
- GATHER-DEVICE-005 -> windows_update_reboot_pending (emitOnlyIfExists),
GATHER-DEVICE-004 -> windows_update_history (Get-HotFix, opt-in).
- 5 EventTypes synced across Constants + MCP resource-catalog + web eventTypes.
Self-maintaining rule lifecycle (analyze + gather parity):
- Gather rules gain the sunset/orphan-GC that only analyze had
(ProcessSunsetGatherRuleAsync + runtime filter + seed/reseed/github paths).
- RuleProvenance field (embedded/github, null=embedded back-compat): the embedded
catalog only sunsets its own rules; GitHub-reseeded rules ahead of the deployed
binary (new ids AND content updates to existing ids) are exempt and reclaimed to
embedded once the binary catches up. No orphans, no operator knowledge required.
Verified: full solution builds; backend suite 2770 pass, agent suite 1638 pass;
rules AJV-valid + combined.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Retire ANALYZE-ESP-003 (duplicate of ESP-001) and ANALYZE-CORR-002 (duplicate
of APP-004) via the established sunset path; ESP-001 gains a >60min factor,
APP-004 absorbs the co-occurrence story with working confidence factors
- CORR-003: download evidence (errorPatternId) is now a required condition —
no more firing on proxy + any unrelated app failure
- DEV-002 v2: 'sustained' semantics — new event_count value filter
(filterField/filterOperator/filterValue in schema, Shared model, engine, web
types) requires >=3 snapshots above 90% memory instead of one transient spike
- DEV-003 v2: build-cutoff-only condition (<=22621) now also covers Windows 10;
LTSC caveat + maintenance note (raise to 22631 after 2026-11-10)
- SEC-001 v2: rewritten for the post-June-2026 reality — certificates HAVE
expired, active remediation required (OEM firmware hint added)
- SEC-005/006 v2.1: allow-list extended with OEM factory-preload families
(Intel PPM, AMD Power, Lenovo ThinkPad power/battery, Surface/carrier),
telemetry-curated from 1780 fired sessions; imaging leftovers (USMT.PPKG,
InstallOrder.ini) and generic names deliberately still alert
- APP-001: suppressByEvent on app_install_completed; APP-002: exact MSI exit
codes instead of bare 16xx; APP-006 reframed as 'App Install Retry Loop';
ID-001: title/logic mismatch fixed + GATHER-ID-002 dependency documented
- Fix all 13 dead confidence factors (signal-name mismatches / unsupported
syntax) by rewiring or removal; version bumps everywhere so seeding
propagates the updates
- New RuleEngineEventCountFilterTests + SEC-005 OEM allow-list coverage
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Devices that hit the ESP terminal timeout in DeviceSetup on a profile that
allows "Continue anyway" are recorded as Failed even though the user most
likely dismissed the error and reached the desktop. The existing advisory-
defang only covers AccountSetup failures, so DeviceSetup timeouts (e.g. a
slow blocking app missing the 30-min wall) show up as hard failures.
Add a built-in, enabled-by-default analyze rule (severity: warning) keyed on
enrollment_failed.mayHaveContinuedAnyway==true. It surfaces the soft failure
and points at the root cause (slow blocking app / ESP timeout / detection
mismatch / masked certificate failure) without changing session lifetime or
the terminal status. No agent change required; ships via EmbeddedResource or
ReseedFromGitHub.
Tests: RuleEngineEspSoftFailureTests (fires on the continue-anyway shape,
stays silent on hard failures and when the flag is absent).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ANALYZE-SEC-002 fired Warning (confidence 90) on every normal user-driven
Entra-join enrollment. During ESP, Windows writes AutoAdminLogon=1 +
DefaultUserName + AutoLogonSID + no DefaultPassword for the enrolling user —
the exact fingerprint the sysinternals_autologon_suspected heuristic targeted
(enabled && user && !password). AutoLogon-enabled is registry-indistinguishable
from a Sysinternals/kiosk setup during Autopilot, so the heuristic and SEC-002
could never tell a benign enrollment from a real one.
Declare AutoLogon only when a plaintext password is actually present:
- delete ANALYZE-SEC-002 (broad AutoLogon-active warning)
- remove the non-discriminating sysinternals heuristic from AutoLogonAnalyzer
(computation + finding label + checks.sysinternals_autologon_suspected)
- strip the sysinternals condition+factor from SEC-004 (kiosk allow-list, still
a disabled opt-in template) and refresh its now-stale SEC-002 guidance
- ANALYZE-SEC-003 (plaintext DefaultPassword -> high) is now the sole default-on
AutoLogon grade; the autologon_analysis event still carries all raw facts at Info
Tests: assert SEC-002 absent + no rule references the sysinternals signal; add
analyzer regression guard for enabled+user+no-password. V2-only, no ConfigVersion
bump. Build 0 err, AutoLogon 19/19, BuiltInRulesTests 8/8, web tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surfaces the real Microsoft 365 Apps Click-to-Run install lifecycle that the
Intune "integrated" app hides (IME reports done in 1-2 min while C2R keeps
streaming in the background). V2-only.
Event-driven, no idle polling:
- OfficeProcessWatcher: WMI Win32_ProcessStartTrace push on OfficeC2RClient.exe
(+ startup probe) for start; Process.Exited for stop, debounced by a settle
window (OfficeInstallSettleSeconds, default 10s) so short worker gaps in one
install don't trigger a premature office_install_failed.
- RegistryChangeWatcher: RegNotifyChangeKeyValue push on ...\Office\ClickToRun
(Configuration value + Scenario subkey churn); bootstrap fallback watches the
nearest existing ancestor non-recursive + name-only until ClickToRun appears.
- OfficeInstallDetector: pure state machine (Idle->Active->Terminal), emits
office_install_started/progress/completed/failed (progress on real change only,
no heartbeat). Error code only from non-zero numeric/hex Scenario values.
Office Delivery Optimization:
- DeliveryOptimizationCollector extended with a 2nd wake source (Office active),
classifies non-IME Office-CDN jobs and folds aggregated DO bytes/peers/real
download-% into the office_install_* events (not emitted as download_progress
to avoid a phantom app in the backend AppInstallSummary).
Plumbing: office_install_* event-type constants, EnableOfficeInstallDetector +
OfficeInstallSettleSeconds config (ConfigVersion 28->29), MCP + web event-type
catalogs, ANALYZE-OFFICE-001 (office_install_failed -> Warning). 22 detector unit
tests; full V2 suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks PPKG detection to one aggregate event the rules parse as an array,
adds a default + customizable allow-list, and makes the event storage-safe.
Rule engine:
- New condition source event_data_array + ItemField: iterates an array field
(any-element match), with dot-path dataField support. (RuleEngine.ConditionEvaluators)
Agent:
- Drop per-package provisioning_package_detected events (noise: a clean Win11 VM
ships ~22 OS-inbox .ppkg). One provisioning_package_scan carries an artifacts[]
array; anyPpkgFound derived from it (consistent, incl. recovery residue).
- Size-based chunking: split artifacts across multiple scan events so each event's
Newtonsoft DataJson stays < 24k (well under the 30k Table-Storage truncation that
would blind the rule); never drops a tail artifact. Per-artifact string fields
(identity/dir/rank/installTime) hard-capped at 256; scanErrors capped.
Rules:
- ANALYZE-SEC-005 (default on) + SEC-006 (template) fire on artifacts.identity
not matching an anchored allow-list (^(?:...)\b) that covers OS-inbox families.
- ANALYZE-SEC-007 alarms when a per-source collection cap was hit (scanTruncated).
Web/MCP/docs: event_data_array in the rule form (types + fields), analyze-rules
docs section + example, event-type catalogs.
Tests: rule-engine array/anchor/dot-path/SEC-007; agent artifacts/chunking/field-cap
/worst-case-size. Build green; V2.Core 1236; backend rule + web tsc green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
V2-only AutoLogonAnalyzer that reads the Winlogon AutoLogon configuration and
emits an autologon_analysis event with raw facts at Info severity. AutoLogon can
be a legitimate kiosk/shared-device setup or a way to automate enrollment/OOBE;
the agent reports facts only and never reads the DefaultPassword value (presence
detected via value-name enumeration). Backend analyze-rules grade the facts.
- AutoLogonAnalyzer (always-on, no tenant toggle): startup no-op, runs at
DeviceSetup-phase completion (new AutoLogonDeviceSetupTrigger on
EspAndHelloHost.DeviceSetupProvisioningComplete) and at final shutdown.
- Sysinternals Autologon heuristic (AutoLogon enabled + default user + no
registry password = LSA-secret credential); never reads the LSA secret.
- Analyze rules: ANALYZE-SEC-002 (AutoLogon enabled -> warning, default-on),
ANALYZE-SEC-003 (plaintext DefaultPassword -> high), ANALYZE-SEC-004 (kiosk
allow-list template). Sysinternals graded via a real confidence condition
inside SEC-002/004 so the allow-list still suppresses approved kiosks.
- autologon_analysis added to all event-type catalogs; rule-builder picker also
gains the previously-missing integrity_bypass_analysis + local_admin_analysis.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
V2 PPKG scanner that runs once at DeviceSetup-phase start and reports raw facts
only — .ppkg file presence, Provisioning\Packages registry metadata,
Recovery\Customizations residue, and best-effort per-package content indicators.
PPKGs can come from legitimate bulk enrollment or be a pre/in-enrollment
manipulation vector; intent is judged by backend analyze-rules, not the agent.
- ProvisioningPackageCollector + ProvisioningPackageHost, wired as a kernel host
in DefaultComponentFactory (one-shot at DeviceSetup start, not at agent start).
- Events: provisioning_package_scan (aggregate) + provisioning_package_detected
(one scalar event per artifact so analyze rules can match per package).
- Analyze rules ANALYZE-SEC-005/006 incl. a per-tenant allow-list template.
- Event-type catalogs kept in sync (Constants.EventTypes, MCP resource-catalog,
rule-builder eventTypes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCP-side mirror of the web interpolateRuleTemplate: resolve
{{reason}}/{{appName}}/{{errorCode}}/... from each rule result's
matchedConditions before get_session_details and get_session_summary
ship the explanation + remediation text, so MCP tool output never leaks
raw {{...}} placeholders. Resolution order is kept in lock-step with the
web copy (byField -> whitelisted auto-field -> signal name).
Also rewords the ANALYZE-ENRL-001 footnote to explain that a remaining
literal {{...}} means the field simply was not recorded for this session
(version 1.1.0 -> 1.1.1; compiled dist and the rule-version assertion in
RuleEngineDetectionFailureTests updated to match).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintenance sweep graduated a stalled session to terminal Failed via the repo
(MaintenanceService Stage 2, SessionTimeoutHours=5h default) but wrote no event into
the session stream and never enqueued analyze. The analyze pipeline fires on stream
events, so a server-timed-out session analyzed to totalIssues:0 — unlike the agent
max-lifetime path, which emits a real enrollment_failed and is correctly analyzed.
- New server-authored event type Constants.EventTypes.SessionTimeout = "session_timeout"
(+ MCP termination_events catalog; drift test green).
- MaintenanceService Stage 2 now captures the UpdateSessionStatusAsync result and, only
on the real first transition (idempotent — the sweep never re-collects a Failed
session), emits a session_timeout event (new pure static BuildSessionTimeoutEvent:
Sequence = max(session events)+1 so it sorts last, Phase=Unknown, Source=System.
Maintenance, Severity=Error) and enqueues auto-analyze with ReasonEnrollmentFailed.
Best-effort try/catch — never blocks the timeout transition. Producer injected.
- RuleEngine: EvaluatePrecondition gains a pure event-type-presence gate (no-dataField
exists/not_exists) — additive (no rule used dataField-less preconditions), enables a
session-level suppression with no shared join field.
- New rule ANALYZE-ENRL-002 "Session Timed Out": fires on session_timeout, suppressed
when enrollment_complete OR enrollment_failed exists (5h<6h late-completion race +
no double-report with ANALYZE-ENRL-001). dist regenerated via combine.js.
- Web: session_timeout treated as terminal for stale-status refetch (the sweep sends
no SignalR delta).
Tests: +3 RuleEngine pure-type precondition, SessionTimeoutRuleTests (real shipped rule
fire/suppress), MaintenanceServiceSessionTimeoutEventTests (shape/sequence). Backend
2168 pass, web 434 pass, MCP 177 pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session 080edee9 follow-up — the customer screenshots showed ANALYZE-CORR-001
("Slow Network") and ANALYZE-APP-009 ("App Download Stalled") still firing
on a clean re-analyze, despite my previous fix shipping them as
enabled:false in the JSON. Investigation via direct DB query:
- AnalyzeRules/global rows for both: Enabled=false ✓ (version-bump landed)
- RuleStates/{tenant} rows for both: Enabled=true ✗ (orphan overrides
from the user toggling them in the portal UI on 2026-03-09)
[AnalyzeRuleService.cs:60](src/Backend/AutopilotMonitor.Functions/Services/AnalyzeRuleService.cs#L60)
overlays per-tenant RuleState on top of the global Enabled, so the orphans
kept resurrecting the rules. No reseed path was cleaning them up.
The right semantic is "sunset = remove from catalog", not "ship as
enabled:false and hope nobody overrides". Three coordinated changes:
1. Both sunset rules deleted from the rules/analyze/ source tree (they
never had a working condition shape — CORR-001 didn't actually check
download speeds, APP-009 matched the V2 termination handler's
post-mortem-synthetic download_progress:failed event).
2. New IRuleRepository.DeleteRuleStatesForRuleIdAcrossTenantsAsync
cross-partition cleanup primitive — scans RuleStates by RowKey, deletes
matching rows across every tenant partition.
3. Both reseed paths now do orphan-state GC:
- AnalyzeRuleService.EnsureBuiltInRulesSeededAsync (on-startup):
detects rules that exist in DB as built-in but aren't in
BuiltInAnalyzeRules.GetAll() anymore → deletes the global row +
fans out the per-tenant RuleState GC.
- AnalyzeRuleService.ReseedBuiltInRulesAsync (delete+reimport): same
diff-based detection on the pre-delete snapshot.
- ReseedFromGitHubFunction.ReseedAnalyzeAsync (admin "Reseed from
GitHub" button — the path the user actually clicks): mirrored
identical logic so every reseed flavor leaves the same DB invariant.
4. ReseedBuiltInRulesAsync return shape extended to
(deleted, written, orphanStatesGcd) so the admin UI can surface the
cleanup count to operators.
Coverage:
- RuleEngineDetectionFailureTests: the two version-bump assertions for
CORR-001/APP-009 are replaced with `DoesNotContain` against the
built-in catalog (sunset semantic).
- AnalyzeRuleServiceSunsetTests (new, 3 tests): pins EnsureSeed orphan-GC
invariant, asserts live-rule states are NOT touched, and pins the
ReseedBuiltIn (deleted, written, orphanStatesGcd) tuple shape.
After deploy + a single GetAllRulesForTenantAsync call (any analyze run
triggers it), the orphan RuleStates for sunset rules are automatically
cleaned. The user's tenant 5ca2b350-… will see CORR-001 + APP-009
disappear from the Rules UI on the next analyze cycle.
Backend 2073/2073.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 follow-up — the analyze cards the user saw on re-analysis
were neither accurate (Slow Network / App Download Stalled both fired as
false positives) nor specific (Enrollment Failed never named the offending
app; ANALYZE-APP-013 never fired). Five coordinated changes, all
version-bumped so EnsureBuiltInRulesSeededAsync re-imports them on the next
backend boot — the title/severity/version-check filter does not pick up
condition-only diffs:
ANALYZE-APP-013 v1.1.0 — fire on legacy + new agent shape:
Previous rule keyed on `failureType == esp_apps_detection_failure`,
which is only emitted by agents that shipped AFTER 4dd050e0. Historical
sessions (and any agent on the old build) won't match.
Replace with a precondition + condition pair that works for BOTH:
* precondition: esp_failure_settle_started.errorCode == 0x87d1041c
(V2 ProvisioningStatusTracker emits this on every Apps subcategory
failure with that HRESULT, regardless of agent version)
* required: app_install_failed.detectionResult == NotDetected
(ImeLogTracker sets this from the IME's DetectionActionHandler line
across all V2 agents)
* optional: esp_failure_settle_started.errorCode (for {{errorCode}}
interpolation; appName comes via auto-field from the required match)
ANALYZE-ENRL-001 v1.1.0 — restore appName + cross-link to APP-013:
My previous trim dropped the failed_app condition on the theory that the
auto-field path would carry it. That was wrong — auto-fields come from
the FIRST matched condition's evidence, which here is `enrollment_failed`
(no appName). Add back the optional failed_app condition with
dataField=appName so byField resolves {{appName}}. Explanation now leads
with the failed app, lists reason/subcategory/errorCode, and cross-links
to the actionable ANALYZE-APP-013 instead of "see other rule results".
ANALYZE-CORR-001 v1.1.0 ("Slow Network") — version bump for DB sync:
Was already enabled:false in source but the DB still had it enabled (the
seed mechanism only resyncs on title/severity/trigger/version diff, so
the source `enabled` flip never propagated). Bump version → next deploy
resyncs. Also adds a suppressByEvent on the app_failed condition so a
later app_install_completed for the same appId suppresses the match (a
retry that succeeded shouldn't count as failure).
ANALYZE-APP-009 v1.1.0 ("App Download Stalled") — same version bump for
DB sync + documents why a status-aware suppression isn't expressible yet:
The V2 EnrollmentTerminationHandler synthesizes
`download_progress: failed` events for promoted-stuck apps AFTER the
real `download_progress: 100%` event already landed. The engine's
suppressByEvent can't filter on a third event's status field, so
status-aware suppression has to wait for either an engine extension or
a dedicated `download_stalled` event type.
RuleEngineDetectionFailureTests — 6 new tests:
* APP-013 fires on the historical 080edee9 event shape
* APP-013 does NOT fire for non-Apps subcategory HRESULTs
* APP-013 does NOT fire without esp_failure_settle_started (precondition)
* ENRL-001 MatchedConditions carries appName + errorCode + subcategory
+ reason from three distinct event types (interpolator material)
* CORR-001 + APP-009 shipped as disabled
Coverage: 2064/2064 backend, 413/413 web.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex review (P2): ANALYZE-APP-013's `failed_app_name` + `detection_hresult`
optional conditions scanned all app_install_failed events independently of
the required condition. In a session with multiple app failures the
template could interpolate {{appName}} / {{errorCode}} from a different
event than the one that matched `failureType == esp_apps_detection_failure`.
Two coordinated changes:
1. `interpolateRuleTemplate` gains an "auto-field" resolution tier between
the explicit byField path and the bySignal fallback. The backend's
`RuleEngine.AddDataFieldsToEvidence` already injects a whitelist of
identifying fields (appId, appName, errorPatternId, errorCode, exitCode,
status) onto every matched-condition evidence dict — the interpolator
now reads those, pinning to the FIRST matched condition (= the required
condition) so a rule with a single required match resolves all four
tokens against the same source event.
2. ANALYZE-APP-013 is trimmed to a single required condition; the redundant
optional conditions for appName / errorCode are removed because the
auto-field tier already exposes them from the required match. Same
pattern applied to ANALYZE-ENRL-001's `failed_app` condition.
Coverage: 4 new vitest cases (multi-failure correct interpolation, first-
condition-wins shadowing, explicit-field precedence, internal-field leakage
guard). 413/413 web tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 fired only ANALYZE-ENRL-001 ("Enrollment Failed") with a
generic explanation, even though we already had all the data to point at
the specific failed app (Microsoft Apps for Enterprise) and the specific
HRESULT (0x87D1041C - Intune detection rule mismatch). Three coordinated
rule changes close that enrichment gap:
ANALYZE-APP-001 (Win32 App Detection Script Failure):
Switch from a regex on the `message` field (which never matched the
agent's "Install status unconfirmed - ESP timed out" wording) to a
direct check on `dataField: detectionResult, operator: equals,
value: NotDetected`. The rule engine already supports nested DataJson
fields on `event_type` conditions, so this is the right shape and the
rule now fires whenever the IME reported a detection-rule mismatch.
ANALYZE-APP-013 (App Detection Failure During ESP - 0x87D1041C, new):
Matches the canonical `failureType: esp_apps_detection_failure` tag
emitted by the V2 agent's new classifier. Severity=critical, baseConf=90.
Pulls failed appName + HRESULT into the explanation via the existing
`{{token}}` interpolator, and ships a three-step remediation plan:
(1) validate the detection rule against an installed reference device,
(2) investigate post-install async tasks (the install may complete only
after ESP's detection probe runs), (3) cross-check the per-app event's
detectionResult + bytesDownloaded + hresultFromWin32.
ANALYZE-ENRL-001 (Enrollment Failed):
Add optional conditions that capture {reason, failedSubcategory, errorCode,
failedApp.appName} into MatchedConditions so the explanation can resolve
`{{reason}}` / `{{failedSubcategory}}` / `{{errorCode}}` / `{{appName}}`
via the existing `interpolateRuleTemplate` engine. Remediation text now
cross-references ANALYZE-APP-* rules for the actionable root cause.
dist regenerated via `node rules/scripts/combine.js` (25 -> 26 rules).
Backend embeds dist/analyze-rules.json as a resource; next deploy picks up
the new content automatically.
Coverage: 2058/2058 backend tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a coarse virtual-machine flag to every hardware_spec event so the rule
engine's preconditions feature can filter device-specific rules. Wires the
new flag into ANALYZE-SEC-001 (Secure Boot UEFI CA 2023): the rule no longer
fires on Hyper-V / VMware / VirtualBox / Xen / QEMU / Parallels / KVM
guests, where the firmware DB is host-controlled and the inside-the-VM
UEFICA2023Status registry value is not actionable.
Agent (V1 + V2)
- Centralized VM detection in DeviceInfoProvider.IsVirtualMachine(mfr, model)
alongside the existing GetManufacturer / GetModel helpers (which already
handle Lenovo's Win32_ComputerSystemProduct.Version quirk). Reused, not
duplicated — same source of truth as SessionRegistration / DiagnosticsPackage
/ GatherRulesMode.
- New CollectSystemInfo step in DeviceInfoCollector.HardwareSpec emits
systemManufacturer, systemModel, isVirtualMachine onto the hardware_spec
event. No new WMI query — calls into DeviceInfoProvider.
- Conservative bias: Microsoft Corporation manufacturer is shared between
Hyper-V VMs and physical Surface devices, disambiguated via model
("Virtual Machine" → VM, "Surface Laptop 5" → physical).
Rule
- ANALYZE-SEC-001 gains a precondition (equals isVirtualMachine=false) and
bumps to v1.1.0 so EnsureBuiltInRulesSeededAsync re-saves it for tenants
on next backend cold start.
- Old agents (no isVirtualMachine field on hardware_spec) → fail-closed →
rule is silently skipped on those devices until the agent updates. Safe
default for a security rule.
Tests
- DeviceInfoProviderVmDetectionTests (V1 + V2 mirror, 18 cases each):
Hyper-V / Cloud PC / VMware / VirtualBox / Xen / QEMU / Parallels / KVM
positive + Surface Laptop/Pro/Book + Lenovo / Dell / HP negative + null/empty
edge cases. Pinned at the central DeviceInfoProvider so all consumers benefit.
- 781/781 backend, 525/525 V1 agent, 800/800 V2 agent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Oliver Kieselbach