Session 080edee9 follow-up — the customer screenshots showed ANALYZE-CORR-001
("Slow Network") and ANALYZE-APP-009 ("App Download Stalled") still firing
on a clean re-analyze, despite my previous fix shipping them as
enabled:false in the JSON. Investigation via direct DB query:
- AnalyzeRules/global rows for both: Enabled=false ✓ (version-bump landed)
- RuleStates/{tenant} rows for both: Enabled=true ✗ (orphan overrides
from the user toggling them in the portal UI on 2026-03-09)
[AnalyzeRuleService.cs:60](src/Backend/AutopilotMonitor.Functions/Services/AnalyzeRuleService.cs#L60)
overlays per-tenant RuleState on top of the global Enabled, so the orphans
kept resurrecting the rules. No reseed path was cleaning them up.
The right semantic is "sunset = remove from catalog", not "ship as
enabled:false and hope nobody overrides". Three coordinated changes:
1. Both sunset rules deleted from the rules/analyze/ source tree (they
never had a working condition shape — CORR-001 didn't actually check
download speeds, APP-009 matched the V2 termination handler's
post-mortem-synthetic download_progress:failed event).
2. New IRuleRepository.DeleteRuleStatesForRuleIdAcrossTenantsAsync
cross-partition cleanup primitive — scans RuleStates by RowKey, deletes
matching rows across every tenant partition.
3. Both reseed paths now do orphan-state GC:
- AnalyzeRuleService.EnsureBuiltInRulesSeededAsync (on-startup):
detects rules that exist in DB as built-in but aren't in
BuiltInAnalyzeRules.GetAll() anymore → deletes the global row +
fans out the per-tenant RuleState GC.
- AnalyzeRuleService.ReseedBuiltInRulesAsync (delete+reimport): same
diff-based detection on the pre-delete snapshot.
- ReseedFromGitHubFunction.ReseedAnalyzeAsync (admin "Reseed from
GitHub" button — the path the user actually clicks): mirrored
identical logic so every reseed flavor leaves the same DB invariant.
4. ReseedBuiltInRulesAsync return shape extended to
(deleted, written, orphanStatesGcd) so the admin UI can surface the
cleanup count to operators.
Coverage:
- RuleEngineDetectionFailureTests: the two version-bump assertions for
CORR-001/APP-009 are replaced with `DoesNotContain` against the
built-in catalog (sunset semantic).
- AnalyzeRuleServiceSunsetTests (new, 3 tests): pins EnsureSeed orphan-GC
invariant, asserts live-rule states are NOT touched, and pins the
ReseedBuiltIn (deleted, written, orphanStatesGcd) tuple shape.
After deploy + a single GetAllRulesForTenantAsync call (any analyze run
triggers it), the orphan RuleStates for sunset rules are automatically
cleaned. The user's tenant 5ca2b350-… will see CORR-001 + APP-009
disappear from the Rules UI on the next analyze cycle.
Backend 2073/2073.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 follow-up — the analyze cards the user saw on re-analysis
were neither accurate (Slow Network / App Download Stalled both fired as
false positives) nor specific (Enrollment Failed never named the offending
app; ANALYZE-APP-013 never fired). Five coordinated changes, all
version-bumped so EnsureBuiltInRulesSeededAsync re-imports them on the next
backend boot — the title/severity/version-check filter does not pick up
condition-only diffs:
ANALYZE-APP-013 v1.1.0 — fire on legacy + new agent shape:
Previous rule keyed on `failureType == esp_apps_detection_failure`,
which is only emitted by agents that shipped AFTER 4dd050e0. Historical
sessions (and any agent on the old build) won't match.
Replace with a precondition + condition pair that works for BOTH:
* precondition: esp_failure_settle_started.errorCode == 0x87d1041c
(V2 ProvisioningStatusTracker emits this on every Apps subcategory
failure with that HRESULT, regardless of agent version)
* required: app_install_failed.detectionResult == NotDetected
(ImeLogTracker sets this from the IME's DetectionActionHandler line
across all V2 agents)
* optional: esp_failure_settle_started.errorCode (for {{errorCode}}
interpolation; appName comes via auto-field from the required match)
ANALYZE-ENRL-001 v1.1.0 — restore appName + cross-link to APP-013:
My previous trim dropped the failed_app condition on the theory that the
auto-field path would carry it. That was wrong — auto-fields come from
the FIRST matched condition's evidence, which here is `enrollment_failed`
(no appName). Add back the optional failed_app condition with
dataField=appName so byField resolves {{appName}}. Explanation now leads
with the failed app, lists reason/subcategory/errorCode, and cross-links
to the actionable ANALYZE-APP-013 instead of "see other rule results".
ANALYZE-CORR-001 v1.1.0 ("Slow Network") — version bump for DB sync:
Was already enabled:false in source but the DB still had it enabled (the
seed mechanism only resyncs on title/severity/trigger/version diff, so
the source `enabled` flip never propagated). Bump version → next deploy
resyncs. Also adds a suppressByEvent on the app_failed condition so a
later app_install_completed for the same appId suppresses the match (a
retry that succeeded shouldn't count as failure).
ANALYZE-APP-009 v1.1.0 ("App Download Stalled") — same version bump for
DB sync + documents why a status-aware suppression isn't expressible yet:
The V2 EnrollmentTerminationHandler synthesizes
`download_progress: failed` events for promoted-stuck apps AFTER the
real `download_progress: 100%` event already landed. The engine's
suppressByEvent can't filter on a third event's status field, so
status-aware suppression has to wait for either an engine extension or
a dedicated `download_stalled` event type.
RuleEngineDetectionFailureTests — 6 new tests:
* APP-013 fires on the historical 080edee9 event shape
* APP-013 does NOT fire for non-Apps subcategory HRESULTs
* APP-013 does NOT fire without esp_failure_settle_started (precondition)
* ENRL-001 MatchedConditions carries appName + errorCode + subcategory
+ reason from three distinct event types (interpolator material)
* CORR-001 + APP-009 shipped as disabled
Coverage: 2064/2064 backend, 413/413 web.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex review (P2): ANALYZE-APP-013's `failed_app_name` + `detection_hresult`
optional conditions scanned all app_install_failed events independently of
the required condition. In a session with multiple app failures the
template could interpolate {{appName}} / {{errorCode}} from a different
event than the one that matched `failureType == esp_apps_detection_failure`.
Two coordinated changes:
1. `interpolateRuleTemplate` gains an "auto-field" resolution tier between
the explicit byField path and the bySignal fallback. The backend's
`RuleEngine.AddDataFieldsToEvidence` already injects a whitelist of
identifying fields (appId, appName, errorPatternId, errorCode, exitCode,
status) onto every matched-condition evidence dict — the interpolator
now reads those, pinning to the FIRST matched condition (= the required
condition) so a rule with a single required match resolves all four
tokens against the same source event.
2. ANALYZE-APP-013 is trimmed to a single required condition; the redundant
optional conditions for appName / errorCode are removed because the
auto-field tier already exposes them from the required match. Same
pattern applied to ANALYZE-ENRL-001's `failed_app` condition.
Coverage: 4 new vitest cases (multi-failure correct interpolation, first-
condition-wins shadowing, explicit-field precedence, internal-field leakage
guard). 413/413 web tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 fired only ANALYZE-ENRL-001 ("Enrollment Failed") with a
generic explanation, even though we already had all the data to point at
the specific failed app (Microsoft Apps for Enterprise) and the specific
HRESULT (0x87D1041C - Intune detection rule mismatch). Three coordinated
rule changes close that enrichment gap:
ANALYZE-APP-001 (Win32 App Detection Script Failure):
Switch from a regex on the `message` field (which never matched the
agent's "Install status unconfirmed - ESP timed out" wording) to a
direct check on `dataField: detectionResult, operator: equals,
value: NotDetected`. The rule engine already supports nested DataJson
fields on `event_type` conditions, so this is the right shape and the
rule now fires whenever the IME reported a detection-rule mismatch.
ANALYZE-APP-013 (App Detection Failure During ESP - 0x87D1041C, new):
Matches the canonical `failureType: esp_apps_detection_failure` tag
emitted by the V2 agent's new classifier. Severity=critical, baseConf=90.
Pulls failed appName + HRESULT into the explanation via the existing
`{{token}}` interpolator, and ships a three-step remediation plan:
(1) validate the detection rule against an installed reference device,
(2) investigate post-install async tasks (the install may complete only
after ESP's detection probe runs), (3) cross-check the per-app event's
detectionResult + bytesDownloaded + hresultFromWin32.
ANALYZE-ENRL-001 (Enrollment Failed):
Add optional conditions that capture {reason, failedSubcategory, errorCode,
failedApp.appName} into MatchedConditions so the explanation can resolve
`{{reason}}` / `{{failedSubcategory}}` / `{{errorCode}}` / `{{appName}}`
via the existing `interpolateRuleTemplate` engine. Remediation text now
cross-references ANALYZE-APP-* rules for the actionable root cause.
dist regenerated via `node rules/scripts/combine.js` (25 -> 26 rules).
Backend embeds dist/analyze-rules.json as a resource; next deploy picks up
the new content automatically.
Coverage: 2058/2058 backend tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- rules/guardrails.json: add new "RealmJoin" category covering
SYSTEM\CurrentControlSet\Services\realmjoin and SOFTWARE\RealmJoin. Hive-agnostic
match covers HKLM\SYSTEM\..., HKLM\SOFTWARE\RealmJoin, and HKU\<sid>\SOFTWARE\RealmJoin
so operator-authored Gather-rules can probe the RJ keys (the RealmJoinWatcher
itself bypasses GatherRuleGuards and watches the registry directly).
- src/Web/autopilot-monitor-web/lib/guardrails.generated.ts: regenerated via
node rules/scripts/combine.js so the Web admin UI surface matches the new
registry allowlist.
- buildcounter.txt (V2 + V1): auto-incremented by the Phase A-D builds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The post-PR-HS1 single-source design (HS-NEW-RESULT JSON only) had a real timing gap:
IME emits the [HS] new result = JSON line only AFTER its batch send to the Microsoft
service, ~30-90 s after the actual script ran (verified in session c1e714e6: 34-74 s
gap per policy). Short Autopilot enrollments often end inside that window, so health-
script rows were entirely missing from completed sessions even though IME had executed
the scripts.
Two-stage approach (matches original V1 timing for the early signal, while keeping
HS-NEW-RESULT as the canonical truth):
- Re-add HS-COMPLIANCE pattern matching BOTH `pre-remdiation` AND `post-remdiation`
detection compliance lines → HandleHealthScriptDetectionResult emits a script_completed
event immediately. Single regex (`(?<part>pre|post)-remdiation`) covers both phases.
- Re-add HS-RUN-CONTEXT / HS-EXITCODE / HS-STDOUT / HS-STDERR patterns feeding a
single-slot `_pendingHealthScript` accumulator (IME runs health scripts sequentially
within a session — verified). HS-COMPLIANCE merges the slot data onto the early-signal
event so exit / stdout / stderr / runContext are populated from the start, not just
the inferred compliance verdict.
- HS-NEW-RESULT continues to fire later with the full JSON (RemediationStatus,
TargetType, ErrorCode, plus the actual remediation phase's data). UI reducer's
dataCompleteness() scoring keeps the more complete entry — early-signal row gets
enriched in place, no duplicate cards.
Remediation phase itself has no early-signal source in IME logs (only the late JSON
exposes Info.RemediationExitCode and RemediationScriptOutputDetails). A follow-up
based on the agentexecutor invocation line's _RemediationScript.output suffix is
possible once we have a real non-compliant cycle log to validate against.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI ajv validation rejected HS-NEW-RESULT.json because the new healthScriptResult
action wasn't in the schema's enum allowlist. Add it next to the other script*
actions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the line-by-line HS-COMPLIANCE / HS-EXITCODE / HS-STDOUT / HS-STDERR /
HS-RUN-CONTEXT pattern set (only ever captured the pre-detection phase, reset state
mid-cycle) with a single atomic source: HS-NEW-RESULT matches IME's "[HS] new result =
{JSON}" line and emits up to three script_completed events per policy run (pre-detection
/ remediation / post-detection). HS-SCRIPT-START now also fires a live script_started
event so the UI shows a "running" indicator immediately, before the consolidated final
result lands ~30s-3min later.
Phase-aware failure semantics: detection / post-detection scripts use exit code as a
compliance verdict (not a crash signal), so a non-compliant detection that gets
remediated no longer counts as a "failed" event in metrics — the row renders with
amber styling via isNonCompliantReport(). Only the remediation phase + platform
scripts route non-zero exit to script_failed.
Defensive against wire-format drift: regex uses [\s\S]* with end-anchor, JSON parsed
with try/catch + counter, frontend toNumber() accepts both numeric and string fields
(backend serializes ints as strings via Dictionary<string,string>). Reducer dedupes
with timestamp included so repeated runs of the same policy in long sessions don't
collapse into one row.
V2 only — V1 (Agent.Core) phasing out.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a coarse virtual-machine flag to every hardware_spec event so the rule
engine's preconditions feature can filter device-specific rules. Wires the
new flag into ANALYZE-SEC-001 (Secure Boot UEFI CA 2023): the rule no longer
fires on Hyper-V / VMware / VirtualBox / Xen / QEMU / Parallels / KVM
guests, where the firmware DB is host-controlled and the inside-the-VM
UEFICA2023Status registry value is not actionable.
Agent (V1 + V2)
- Centralized VM detection in DeviceInfoProvider.IsVirtualMachine(mfr, model)
alongside the existing GetManufacturer / GetModel helpers (which already
handle Lenovo's Win32_ComputerSystemProduct.Version quirk). Reused, not
duplicated — same source of truth as SessionRegistration / DiagnosticsPackage
/ GatherRulesMode.
- New CollectSystemInfo step in DeviceInfoCollector.HardwareSpec emits
systemManufacturer, systemModel, isVirtualMachine onto the hardware_spec
event. No new WMI query — calls into DeviceInfoProvider.
- Conservative bias: Microsoft Corporation manufacturer is shared between
Hyper-V VMs and physical Surface devices, disambiguated via model
("Virtual Machine" → VM, "Surface Laptop 5" → physical).
Rule
- ANALYZE-SEC-001 gains a precondition (equals isVirtualMachine=false) and
bumps to v1.1.0 so EnsureBuiltInRulesSeededAsync re-saves it for tenants
on next backend cold start.
- Old agents (no isVirtualMachine field on hardware_spec) → fail-closed →
rule is silently skipped on those devices until the agent updates. Safe
default for a security rule.
Tests
- DeviceInfoProviderVmDetectionTests (V1 + V2 mirror, 18 cases each):
Hyper-V / Cloud PC / VMware / VirtualBox / Xen / QEMU / Parallels / KVM
positive + Surface Laptop/Pro/Book + Lenovo / Dell / HP negative + null/empty
edge cases. Pinned at the central DeviceInfoProvider so all consumers benefit.
- 781/781 backend, 525/525 V1 agent, 800/800 V2 agent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional `preconditions` array to analyze rules. Preconditions
evaluate BEFORE conditions and are AND-combined; any failing precondition
silently skips the rule (no RuleResult, no UI card, no tenant-visible
"rule fired"). Distinct from `conditions`, which decide whether the rule
fires given that it applies.
Use case: rules that should only run on certain hardware/OS profiles
(e.g. "skip ANALYZE-SEC-001 on virtual machines because UEFI CA 2023
deployment is host-controlled").
Backend
- New `RulePrecondition` model + `Preconditions` list on `AnalyzeRule`.
- JSON schema gains optional top-level `preconditions` referencing a new
`rulePrecondition` definition (no signal/required/correlation noise —
pure boolean gate).
- `RuleEngine.EvaluateRule` runs preconditions first; on failure returns
null. `EvaluatePrecondition` reuses MatchesEventType / GetDataFieldValue
/ MatchesOperator. Operator semantics:
* not_exists → passes when event type absent OR field empty (handy
for "old agent without the new field" path).
* exists / others → fail closed when event type or field missing.
- `TableStorageService` rule round-trip extended with `PreconditionsJson`
on Store + Map (column-mapped, not blob — same pattern as Conditions).
Web (analyze-rules editor)
- `AnalyzeRule` / `RuleForm` types gain `preconditions` array; form
payload round-trips on POST + PUT; ruleToForm + EMPTY_FORM updated.
- `AnalyzeRuleFormFields`: new amber Preconditions panel above Conditions
with eventType/dataField/operator/value/description rows.
- `AnalyzeRuleCard`: read-only chip section above Conditions with
"skipped silently if any fails" hint.
Tests
- `RuleEnginePreconditionTests` (7 cases): empty preconditions = legacy
behavior, single pass/fail, AND-semantics with two preconditions,
not_exists passes when event missing, equals fails closed when event
missing, baseline.
- 781/781 backend, 507/507 V1 agent, 782/782 V2 agent, 208/208 web vitest.
No agent change yet — useful preconditions can already target existing
event fields (os_info.edition, hardware_spec.cpuName, tpm_status.specVersion,
…). Adding `hardware_spec.isVirtualMachine` is a follow-up PR (agent collector).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V2 had drifted to a renamed bucket schema (installedApps/failedApps/...)
plus nested perApp/byPhase. The nested objects were dead — no Web
component or rule consumed them — and the renamed counters needed a
fallback chain in every reader. Single flat schema is simpler:
- Builder rewritten to emit only the flat V1 keys: installed/failed/
skipped/postponed (final), downloading/installing/pending (live),
errorCount/deviceErrors/userErrors/hasErrors/isAllCompleted/
ignoredCount (aggregate helpers), plus name lists per bucket
(installedNames/failedNames/skippedNames/postponedNames/pendingNames).
perApp + byPhase removed. The includePerAppDetail flag is gone — all
emits ship the same shape (live snapshots stay cheap because the
names lists are short strings, not per-app timing objects).
- ignoredCount needed a separate accessor: it lives on the live
AppPackageStateList, not on the deduped phase-snapshot+live union
used by the termination accessor. New ImeIgnoredCount property on
DefaultComponentFactory + ignoredCountAccessor parameter on
EnrollmentTerminationHandler.ctor (default () => 0 keeps existing
test rigs unaffected).
- ANALYZE-APP-007.dataField back to errorCount (flat), dist regenerated.
- Web hooks (useSessionDerivedData, useProgressDerivedData) drop the
V2 fallback chain — single-schema reads only.
- Tests rewritten for V1 schema: counter validation, device/user
error split, hasErrors/isAllCompleted bool helpers, name-list
population per bucket. EnrollmentTerminationHandlerTests collapsed
the three V2-perApp tests into one V1-counts+split-errors test;
per-app timing detail now lives only in FinalStatusBuilder /
final-status.json, not in the event payload.
Per-transition emit cadence and ImmediateUpload=false on live
snapshots are unchanged from the previous commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ANALYZE-APP-007: dataField errorCount -> failedApps. V1 emitted errorCount
in app_tracking_summary; V2 emits failedApps. Pulling the rule to V2-naming
removes the V1-compat alias debt from the agent.
ANALYZE-SEC-001: drop the data. prefix from dataField. RuleEngine's
GetDataFieldValue does flat dictionary lookup; the data. prefix matched
nothing under V1 or V2. Both V1 and V2 emit uefiCA2023Status /
uefiSecureBootEnabled at the root of Data, so the rule was a long-standing
no-match.
Regenerated rules/dist/analyze-rules.json from the source files via
rules/scripts/combine.js.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The backend already reads errorCode from event data for FailureCode and
failure reason formatting, but the agent never populated it. This change
extracts error codes (exit codes, HRESULT) from named regex capture groups
in IME log error patterns and sends them as structured errorCode fields.
Also fixes a pre-existing bug where ErrorPatternId/ErrorDetail were not
persisted across agent restarts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
.* doesn't match newlines without Singleline flag — script stdout/stderr
from AgentExecutor.log spans multiple lines and was never captured.
Changed to [\s\S]*? to match across newlines.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The FilterByVersion wildcard logic was treating CVEs with specific
versions in their CPE URI as affecting ALL versions. Now extracts the
version from the CPE 2.3 URI and compares it against the installed
version, so only truly relevant CVEs are shown.
Also adds missing CPE seed mappings for common Intune/Autopilot
managed software: Intune Management Extension, Company Portal,
ODBC/OLE DB drivers, Remote Desktop, Azure CLI.
https://claude.ai/code/session_01Kv2aAEVqbeFSsyPv7gU7pF
Oliver Kieselbach