Codex review (P2): ANALYZE-APP-013's `failed_app_name` + `detection_hresult`
optional conditions scanned all app_install_failed events independently of
the required condition. In a session with multiple app failures the
template could interpolate {{appName}} / {{errorCode}} from a different
event than the one that matched `failureType == esp_apps_detection_failure`.
Two coordinated changes:
1. `interpolateRuleTemplate` gains an "auto-field" resolution tier between
the explicit byField path and the bySignal fallback. The backend's
`RuleEngine.AddDataFieldsToEvidence` already injects a whitelist of
identifying fields (appId, appName, errorPatternId, errorCode, exitCode,
status) onto every matched-condition evidence dict — the interpolator
now reads those, pinning to the FIRST matched condition (= the required
condition) so a rule with a single required match resolves all four
tokens against the same source event.
2. ANALYZE-APP-013 is trimmed to a single required condition; the redundant
optional conditions for appName / errorCode are removed because the
auto-field tier already exposes them from the required match. Same
pattern applied to ANALYZE-ENRL-001's `failed_app` condition.
Coverage: 4 new vitest cases (multi-failure correct interpolation, first-
condition-wins shadowing, explicit-field precedence, internal-field leakage
guard). 413/413 web tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex review (P1): ImeLogTrackerAdapter.BuildAppStatePayload only set
DataJson.failureType for `esp_apps_timeout`. After the HRESULT-aware
classifier landed (4dd050e0), apps promoted with `esp_apps_detection_failure`
or `esp_apps_install_failure` still landed with errorPatternId set but
failureType missing. Downstream consequences:
- ANALYZE-APP-013 (dataField: failureType, value: esp_apps_detection_failure)
never matched.
- EventIngestProcessor.AppInstall set AppInstallSummary.FailureCode to the
HRESULT instead of the canonical failureType.
- InstallProgress.tsx fell back to the generic "Failed" badge.
Extend the gate to cover all three AppFailureTypes constants and stamp
DataJson.failureType from app.ErrorPatternId. Adds a Theory test covering
all three flavours and a regression assertion that an unrelated IME-ERROR-*
pattern does NOT spuriously promote to failureType.
Coverage: 108/108 ImeLogTrackerAdapter tests, 1176/1176 V2.Core.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 fired only ANALYZE-ENRL-001 ("Enrollment Failed") with a
generic explanation, even though we already had all the data to point at
the specific failed app (Microsoft Apps for Enterprise) and the specific
HRESULT (0x87D1041C - Intune detection rule mismatch). Three coordinated
rule changes close that enrichment gap:
ANALYZE-APP-001 (Win32 App Detection Script Failure):
Switch from a regex on the `message` field (which never matched the
agent's "Install status unconfirmed - ESP timed out" wording) to a
direct check on `dataField: detectionResult, operator: equals,
value: NotDetected`. The rule engine already supports nested DataJson
fields on `event_type` conditions, so this is the right shape and the
rule now fires whenever the IME reported a detection-rule mismatch.
ANALYZE-APP-013 (App Detection Failure During ESP - 0x87D1041C, new):
Matches the canonical `failureType: esp_apps_detection_failure` tag
emitted by the V2 agent's new classifier. Severity=critical, baseConf=90.
Pulls failed appName + HRESULT into the explanation via the existing
`{{token}}` interpolator, and ships a three-step remediation plan:
(1) validate the detection rule against an installed reference device,
(2) investigate post-install async tasks (the install may complete only
after ESP's detection probe runs), (3) cross-check the per-app event's
detectionResult + bytesDownloaded + hresultFromWin32.
ANALYZE-ENRL-001 (Enrollment Failed):
Add optional conditions that capture {reason, failedSubcategory, errorCode,
failedApp.appName} into MatchedConditions so the explanation can resolve
`{{reason}}` / `{{failedSubcategory}}` / `{{errorCode}}` / `{{appName}}`
via the existing `interpolateRuleTemplate` engine. Remediation text now
cross-references ANALYZE-APP-* rules for the actionable root cause.
dist regenerated via `node rules/scripts/combine.js` (25 -> 26 rules).
Backend embeds dist/analyze-rules.json as a resource; next deploy picks up
the new content automatically.
Coverage: 2058/2058 backend tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The InstallProgress component previously showed an orange "Likely stuck"
badge for any app where the agent promoted state Installing -> Error on
terminal ESP-Apps failure, with a tooltip claiming "ESP timed out while
this app was still installing". Session 080edee9 had that badge attached
to Microsoft Apps for Enterprise even though the actual session only ran
27 min (not 180), and the failure was a confirmed Intune detection-rule
mismatch (HRESULT 0x87D1041C), not a timeout.
Match the new agent failureType taxonomy from
`AppFailureTypes.ClassifyEspAppsFailure`:
- `esp_apps_detection_failure` -> red "Detection failed" badge
(tooltip: Intune detection rule did not match after install)
- `esp_apps_install_failure` -> red "Install failed" badge
(tooltip: installer returned an error before completing)
- `esp_apps_timeout` -> orange "Likely stuck" badge unchanged
(tooltip clarified to "no per-app HRESULT was available")
Also surface the ESP-level HRESULT (DataJson.errorCode) as a "ESP HRESULT:
0x87d1041c" pill with the description from the shared error-codes catalog,
distinct from the per-app hresultFromWin32 pill - so the user can see at
a glance which cross-app failure caused the ESP termination.
Coverage: 409/409 vitest, tsc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 080edee9 surfaced a long-standing mis-classification on the V2
EnrollmentTerminationHandler "still installing -> error" promotion path: every
terminal ESP-Apps failure was stamped as `esp_apps_timeout` with a message
claiming "ESP timed out (180 min)" - even when the actual session only ran
~27 min and the ESP terminated explicitly with HRESULT 0x87D1041C
("Application not detected after installation completed successfully"). The
"180 min" was the *configured* SyncFailureTimeoutMinutes ceiling, not the
elapsed time.
Plumb the HRESULT captured by ProvisioningStatusTracker through the host
stack (EspAndHelloTracker.LastEspTerminalErrorCode -> EspAndHelloHost ->
DefaultComponentFactory) into the termination handler, and route through a
new `AppFailureTypes.ClassifyEspAppsFailure` helper that produces:
- `esp_apps_detection_failure` for 0x87D1041C (install ran but Intune
detection rule did not match afterwards - confirmed detection failure)
- `esp_apps_install_failure` for any other HRESULT (installer reported
an error before the app finished installing)
- `esp_apps_timeout` fallback when no HRESULT is available
(genuine "ESP gave up" case - now phrased as "configured at X min",
not "timed out X min", to make clear it's a ceiling not an elapsed time)
The promoted `app_install_failed` event now carries DataJson.errorCode so
downstream rules and the UI can correlate the per-app failure to the
ESP-level HRESULT. AppPackageState.SetErrorContext already accepted an
optional errorCode argument, so this only extends the promotion delegate
signature without changing the per-app state shape.
Coverage: 86/86 Termination tests pass (incl. 2 new HRESULT-classification
cases); 6/6 new AppFailureTypesClassifierTests; 1171/1171 V2.Core.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds getEnrichedOrLookup() helper that returns the backend-provided
{description, confidence, source} sibling when present (set by the
backend ErrorCodeEnricher on event responses), otherwise looks up via
the local catalog. EventTimeline event badges now read the enriched
field first — older API responses without enrichment fall through to
the existing local-catalog path so there is no regression.
InstallProgress + apps/[appName] pages consume UI-model rows (not raw
event.Data) and are not touched — they continue to use the local
catalog which already covers their inputs. 5 new helper tests; 409/409
web tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Walks each EnrollmentEvent.Data for known error-code keys (errorCode,
exitCode, hresult, hresultFromWin32, failureCode, code, lastError) and
injects a sibling *Info entry { description, confidence, source } via
the shared ErrorCodeCatalog. Applied in GetSessionEvents (legacy +
paginated), SearchEvents (used by search_events_semantic +
deep_search_events MCP tools) and QueryRawEvents (both session-scoped
and cross-session paths).
Idempotent + read-only on storage — alte gespeicherte Events erhalten
die Anreicherung on-the-fly. MCP-Konsumenten (Claude/Copilot) bekommen
ab jetzt automatisch die kuratierten Descriptions mit Confidence-Level,
nicht nur den rohen HRESULT. 14 enricher tests; 2058/2058 backend tests
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the V2 agent process dies via AppDomain.UnhandledException or
TaskScheduler.UnobservedTaskException, synchronously write a compact
MiniDump (data-segs + handle + thread-info, ~5-20 MB) and a CrashRecord
JSON sibling to %ProgramData%\AutopilotMonitor\Crashes\. Next agent
start scans the directory, drains pending records into the timeline as
previous_crash_detected events (Warning, ImmediateUpload), and applies
retention (max 5 dumps, max 7 days). Dumps stay on disk so the existing
customer-opt-in diagnostics-upload can collect them.
Handlers register in Program.Main before SelfUpdater so any downstream
crash is captured; AgentRuntimeHost re-registers once session/tenant
are known so the CrashRecord carries those labels. 6 builder/retention
tests; 1163/1163 V2.Core tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts the Windows/MSI/Intune HRESULT catalog out of the web app into
src/Shared/AutopilotMonitor.Shared/Resources/error-codes.json so backend
and web load from the same authoritative file. Web syncs the JSON at
prebuild time. Adds ErrorCodeCatalog C# service with normalised-key
lookup (hex case, signed-decimal HRESULT, leading-zero padding) and a
parity test on both sides. No behavior change yet — this is groundwork
for backend read-time enrichment so MCP/API responses can carry
errorCodeInfo (description + confidence + source). 11 C# + 14 web tests
green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a fail-soft GetSystemPowerStatus probe to StartupEnvironmentProbes,
emitting a power_state_check event into the enrollment timeline at agent
start. On battery with <80% charge is Warning, otherwise Info; desktops
(no battery) and probe failures handled explicitly. Gives admins early
visibility into a frequent driver behind power-management-related
enrollment stalls. 5 new builder tests; 1157/1157 V2.Core tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Normal (non-Global-Admin) MCP users no longer see any Global-Admin / cross-tenant
hints — reducing both confusion and attack surface / curiosity. The per-request
stateless server already runs inside the access-guard's runWithCaller context, so
the caller role is known at server-creation time.
- index.ts: createMcpServer(ga) reads isGlobalAdmin() and tailors instructions
(non-GA gets a tenant-scoped scope line, no cross-tenant mention).
- tools.ts / sessions.ts / search.ts / admin.ts: thread `ga` into every registrar.
* GA-only tools (list_tenants, list_tables, query_table, query_backend_logs,
get_platform_metrics, get_ops_events, list_session_reports, list_blocked_devices,
get_api_usage) are registered only when ga — absent from tools/list for non-GA.
* Dual-role tools: GA/cross-tenant wording in descriptions and tenantId param
text is gated behind `ga ? ... : ...`; non-GA sees clean tenant-scoped text.
- prompts.ts: compare-agent-versions (uses GA-only get_platform_metrics) hidden for
non-GA; cve-exposure-audit cross-tenant wording scrubbed for non-GA.
- error-handler.ts: 403 message no longer names "Global Admin".
- backend McpUserFunction (/api/auth/mcp): omit isGlobalAdmin from the response when
false, so a normal user gets no hint that a GA tier exists. The MCP guard reads
`=== true` (absent → non-GA), so behavior is unchanged.
Verified: GA catalog 29 tools / non-GA 20 (9 GA-only hidden), zero GA/cross-tenant
wording in any non-GA description, prompts split 3/2. tsc clean; backend targeted
suite 135 green; mcp unit 52 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
get_usage_metrics always called /api/global/metrics/usage (GlobalAdminOnly),
so a Tenant Admin got 403 even though a MemberRead tenant endpoint
(/api/metrics/usage) exists and returns the same shape scoped to their tenant.
Route via pickGlobalOrTenantPath: GA -> global cross-tenant overview;
Tenant Admin/Member -> /api/metrics/usage (JWT-scoped, tenantId ignored).
Description updated for both roles.
Verified live with a Tenant-Admin token: 200, own-tenant usage only
(tenants.total=1), platformStats correctly omitted (GA-only) - no cross-tenant
leak. tsc clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second batch of pre-release MCP tool audit fixes:
- mcp: get_session_summary phase labels were desynced from the backend
EnrollmentPhase enum — phases 3-7 were mislabeled (e.g. 7 showed
"Pre-Provisioning" but is "Complete"; 4 showed "Device ESP" but is
"Account Setup"). Replaced with the canonical taxonomy (matches the web
phaseConstants.ts), enrollment-type-aware (phase 3 = Apps (Device) on V1,
App Installation on V2), plus -1=Unknown / 99=Failed.
- backend: get_usage_metrics reported PlatformStats.lastFullCompute /
lastUpdated as 0001-01-01 even though the persisted row was fresh -
UsageMetricsService projected a new PlatformStats and dropped the two
timestamp fields (defaulted to DateTime.MinValue). Now carried through.
- backend: query_table (TableQueryFunction) now returns 400 with the
actionable Azure message for an invalid OData filter / bad query input
(RequestFailedException 400) instead of a generic 500.
Tests: backend targeted suite 31 green; mcp tsc clean + unit suite 52 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-release MCP tool audit fixes:
- backend: search_sessions_by_cve returned 0 in GA cross-tenant mode. The
CveIndex PartitionKey is {tenantId}_{cveId}, so the old cross-tenant PK
range anchored on the cveId matched nothing (PKs start with the tenant
GUID). Now filters on the CveId property (server-side scan, same shape as
ScanCveIndexAsync). Extracted BuildCveIndexSearchFilter + 5 regression tests.
- mcp: get_geographic_sessions gains pageSize/continuation (client-side
offset; the endpoint has no server cursor) to bound response size.
- mcp: get_usage_metrics / get_api_usage descriptions corrected to reflect
Global-Admin-only routing (were implying a tenant-scoped mode that 403s).
- mcp: get_session_summary appInstalls.total no longer smaller than its
breakdown (max(started, succeeded+failed+skipped)) + adds started.
- mcp: session status enum gains Pending + Stalled.
Tests: backend targeted suite 24 green; mcp tsc clean + unit suite 52 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to opt-in pagination: fields= was a client-side trim in the MCP tool,
so it did not round-trip across pages — following nextLink returned the full
safe-field set again, and the shape differed page 1 vs 2+.
Move the projection server-side (matching query_raw_sessions):
- TenantConfigProjection: keep-list intersection — only the 9 non-sensitive
fields can ever be emitted; requested keys are intersected (fields=teamsWebhookUrl
yields nothing), tenantId always included. ParseFields is case-insensitive.
- TenantConfigPagination: parse fields=; BuildNextLink echoes it so the projection
carries across every page. fields is NOT in the continuation fingerprint (it is a
projection, not a filter — changing it mid-stream keeps the cursor valid).
- GetAllTenantConfigurationsFunction projects via TenantConfigProjection.ProjectAll.
- MCP list_tenants passes fields to the backend (followNextLink params) and drops the
client-side projectTenantFields; extractTenantList stays as defense-in-depth.
Now every page returns exactly the requested columns and the backend sends less over
the wire. Tests: TenantConfigProjection (incl. secret-cannot-be-requested) +
pagination fields-echo; backend 24/24, MCP vitest 52/52.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MCP list_tenants tool fetched GET /api/config/all, which returns all
tenant configs in one unpaginated array. With ~198 tenants the projected
response exceeded the tool response cap and was offloaded to a file, making
the tool unusable for the model.
Add backward-compatible opt-in pagination to config/all (same dual-mode
pattern as GetSessionReportsFunction):
- No pageSize -> unchanged legacy bare full-config array (web + maintenance
aggregation depend on this shape; zero regression).
- pageSize -> secret-stripped { count, tenants, nextLink } envelope using
the existing ContinuationToken HMAC (scope "config-all", caller-bound),
TenantId-ordered pages. Secrets never leave the backend.
Backend: GetTenantConfigurationsPageAsync (repo + service), TenantConfigPagination
helper, dual-mode GetAllTenantConfigurationsFunction. Route/policy unchanged.
MCP: list_tenants now follows nextLink against config/all (extractTenantList as
defense-in-depth + optional fields= trim); client-side selectTenantPage removed.
Tests: TenantConfigPaginationTests 13/13; MCP security-guards 56/56.
No /api/global/ relocation here - that convention cleanup is deferred.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MCP server review follow-ups (PR A-D):
- Remove stale agents/query.ts dev script + drop unused @anthropic-ai/sdk dep
- Add 256kb body limit on /mcp; server-level instructions + 3 diagnostic prompts
(investigate-failed-session, cve-exposure-audit, compare-agent-versions)
- New list_tenants tool over /api/config/all with a strict keep-list projection
so tenant secrets (webhook/SAS URLs) never reach the model (+ 6 unit tests)
- New get_vulnerability_summary tool + backend GET /api/{global/}metrics/vulnerability
aggregating the CveIndex (top CVEs, severity breakdown, KEV, affected devices;
truncated flag on scan cap) + 6 aggregator unit tests
- Test hygiene: exclude tests from tsc output, pin vitest to src/ (no dist double-run)
- Bump MCP server version 1.2.0 -> 1.3.0
Backend solution builds clean; MCP 52 unit tests + 6 backend aggregator tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Section headers (Vulnerability Report, Analysis Results, Integrity Bypass)
and the page action-bar packed icon + title + counter + severity badges +
action button + chevron into a single non-wrapping flex row. On phone-width
viewports that row was wider than the card, widening <main> past the viewport
and making <body> horizontally scrollable. iOS Safari then mis-sized the
fixed-position mobile drawer/overlay, so the drawer no longer covered the
full screen.
- globals.css: overflow-x: clip on <html> as a safety net (clip, not hidden,
so it does not create a containing block for the fixed drawer/overlay).
- The three section headers + page header now wrap (flex-wrap + gap), with
shrink-0 on icons/action groups and break-all on the mono finding string.
tsc clean; vitest 390/390 pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Nightly VulnerabilityDataSync at 03:00 UTC was hitting Azure Table Storage's
1 MB per-entity ceiling on popular CPEs (browsers, Chromium-mass-release CVEs,
Windows components with hundreds of CPE matches). Property-chunking only
guards the 64 KB per-property bound, not the row total.
New CveCacheBudget filters before the cache write so a row can never grow past
the limit:
- date floor: last 24 months (all severities) + last 60 months for CVSS >= 7.0
(keeps current HIGH/CRITICAL findings for LTS software like Java 8 / .NET FX)
- sort CVSS desc, PublishedDate desc, cap 100 CVEs
- per-CVE: cap AffectedVersions at 50 ranges
- unparseable PublishedDate -> keep (conservative)
Both write paths (VulnerabilityCorrelationService.FetchAndCacheCvesFromNvdAsync
and VulnerabilityDataSyncFunction.RefreshStaleCveCacheAsync) go through the same
helper; the sync path also drops the blind .Take(200) and now logs total/
date-filtered/truncated-ranges/bytes per CPE so we can watch the budget land.
Read path is unchanged — older oversized rows continue to work via the existing
TableStorageChunking reassembly until they expire and get refreshed under the
new budget.
Tests: 1990/1990 green; 10 new CveCacheBudgetTests cover the date floor,
LTS-rescue window, sort+cap, AffectedVersions truncation, and edge cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The seed-at-arming fix (dcf89a83) only patched AttachUserRealmJoinSubtreeWatcher's
internal sweep. Two callers still ran a STANDALONE CheckUserPackages BEFORE the seed
pass: NotifyRealmJoinPresence (when phase crossed RunningThreshold with the SID
already set) and ArmHku (when desktop arrival landed after phase had already crossed
the threshold). The HKU dedup sets were empty at that point, so every pre-existing
user-hive package sub-key — typically RJ packages installed during ESP while
DeploymentPhase was still Blank — surfaced as a spurious started+completed pair
before the seed had any chance to suppress them.
Caught in session ff0cdcbe (V2 2.0.886): three user-scope RJ packages
(generic-agilebits-1password, generic-realmjoin-promote-tray-icon,
generic-vlc-usersettings) all fired started+completed at the same millisecond as
the Blank -> RunningFirstDeployment phase transition, exactly the
empty-dedup-set sweep signature.
Removed both standalone calls — EnsureUserRealmJoinStage ->
AttachUserRealmJoinSubtreeWatcher is now the single, ordered entry point
(seed -> watcher.Start -> CheckUserPackages, all post-seed). Added two regression
tests driving the production sequence with a fake registry that holds pre-existing
user packages; both assert zero RealmJoinPackageStarted/Completed signals. The
fake-registry wiring lands as an internal ctor with three delegate seams
(EnumeratePackageIds/TryReadPackage/KeyExists); the public ctor still binds to
RealmJoinInfo + the real Registry probe.
1152/1152 V2.Core tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The script applies a lifecycle policy + container creation against a live
storage account using operator-specific parameters (StorageAccountName,
ResourceGroupName). It is not portable across environments and each
operator keeps their own copy, so it does not belong in the repo.
The file is preserved on disk (git rm --cached, not -f) so the local
operator setup keeps working — only the Git tracking is dropped, plus
the matching .gitignore entry so a future `git add scripts/Backup` is
silently ignored.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-existing package sub-keys observed at arming time (typically RJ packages
installed during ESP before DeploymentPhase >= 100) were silently fired as
realmjoin_package_started by the initial CheckMachinePackages sweep. The
phase gate only prevented IME/RJ subtree-write misattribution; it did not
distinguish historical RJ rows from rows that genuinely appear after arming.
Attach now seeds both _*PackagesStarted and _*PackagesCompleted with every
sub-key already present under HKLM\SOFTWARE\RealmJoin\Packages (and the
HKU\<sid>\... mirror), without firing events. The seed runs inside the
state-lock BEFORE RegistryWatcher.Start() so no subtree wake-up can race
past it. Only sub-keys that appear AFTER arming surface as events.
Added 3 unit tests covering the ESP-leftover suppression, the
per-PackageId scope of the suppression (siblings still fire), and the
HKU mirror. Outdated comment on the immediate-arm test corrected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two findings from the post-commit Codex review of PR2 (single-row restore):
Medium — CriticalTableRestoreService.CommitRowAsync now maps the
MaintenanceLeaseHolder's renewal-loss path to BackupTerminalException
"MaintenanceLeaseLost" (HTTP 409 Conflict), instead of letting the
storage SDK's OperationCanceledException bubble up to the HTTP layer's
generic catch as a 500 InternalError. The catch condition gates on
!ct.IsCancellationRequested so a real client cancel still re-throws.
Low — RestoreInputRow (admin/backups/[backupId]/page.tsx) used to
disable the Preview button on empty pk/rk. The backend contract
(RestoreTablePreflightValidator) explicitly allows empty-string keys
(Azure Tables permits them) and only rejects null. UI now mirrors that.
Added a regression test pinning the empty-keys-allowed contract.
1980/1980 backend tests + 390/390 web tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /api/global/backups/{backupId}/restore-row (GA-only) with preview|commit
modes. Preview scans the ETag-pinned NDJSON blob for the requested (pk, rk),
diffs against the live row, and returns rowSha256 + currentETag for echo.
Commit re-validates everything under a short maintenance lease and writes via
UpdateEntity(IfMatch=etag, Replace) or AddEntity; 412/404/409 → CurrentRowChanged.
Backend:
- RestoreTablePreflightValidator (no I/O) + BackupRestoreInputValidator (manifest
parse, entry check, BlobName canonical, ETag-pin + SHA in one stream read)
- CriticalTableRestoreService with PreviewRowAsync (no lease) + CommitRowAsync
(MaintenanceLeaseHolder, 45 s renewal, deterministic finally release)
- BlobBackupStore.OpenNdjsonReadAsync — GetProperties + OpenRead(IfMatch=etag)
- BackupTerminalException carries a stable Code for HTTP envelope mapping
- OpsEventService.RecordBackupRowRestoredAsync (Warning; payload is
container + blobName only, never a SAS URL)
- 27 new xUnit tests; full backend suite 1979/1979 green
Web:
- /admin/backups list page with newest-first list + "Run backup now" trigger
and async job-status poller
- /admin/backups/{backupId} detail page with per-table breakdown + single-row
restore trigger
- RestoreRowDiffModal with Added/Removed/Changed/Unchanged pills and
auth-table security banner (GlobalAdmins/TenantAdmins/McpUsers)
- api.backups namespace + matching DTO types
- globalNavConfig: ga-backups under Ops
- BackupRowRestored registered in OPS_EVENT_TYPES (dual-register policy)
- 6 new vitest cases; full web suite 390/390 green
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codex-review finding: after the previous hotfix the retryable Running→Queued
rollback ran AFTER leaseHolder.DisposeAsync(), i.e. AFTER the maintenance lease
was released. That opened a window of "Running + free lease + stale heartbeat"
in which the watchdog could legitimately flip the job to Failed; the rollback
then re-read the fresh row, took the latest ETag, and unconditionally re-set
Queued — silently undoing the watchdog's decision.
Fix:
* Move the retryable rollback INSIDE the leased section, right next to the
success-CAS that produces the Completed terminal state. Both paths now run
while the lease is still held + being renewed, so the watchdog cannot race.
* Defence-in-depth: re-read the row before writing Queued and only do the
state transition if the row is still Running. If a parallel actor (in
theory: another worker that somehow stole the lease, or the watchdog
acting on a transient probe) shifted the state, the rollback skips +
logs instead of clobbering. Belt-and-braces — under a held lease the
predicate should always be true.
* Outer-block retry-handling shrinks to "log + rethrow"; the throw is what
drives the visibility-timeout retry of the queue message.
Tests: 1952/1952 still green. The race fix is structural (ordering relative
to lease release), not test-locked; the existing in-memory fakes don't model
lease semantics so a regression would have to be caught in a future watchdog
race-condition test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two Codex-review findings on the post-Wave1 hotfixes, both real:
* Medium — lease-loss cancellation was only caught around RunBackupUnderLeaseAsync.
The same handlerCts token is also used by the CAS Queued→Running, the BackupId
stamp, and the final CAS Completed. If renewal failed during one of those
awaits, the OperationCanceledException bypassed the lease-loss branch and
escaped as a generic poll-loop exception, leaving the job in Running until
the watchdog mopped up. Fix: hoist the lease-loss catch around the WHOLE
leased section; the inner try keeps the retryable-vs-OCE classification for
the service call.
* Medium/Low — the manual queue path marked lease-loss jobs Failed but did not
emit CriticalTableBackupFailed. The timer path did, so alert rules fired
there but not here. Fix: emit the same OpsEvent (with the recovered backupId
+ RequestedBy from the job row) so Telegram routing is symmetric.
While restructuring, two small clean-ups:
* runningJob.RequestedBy and runningJob.StartedAtUtc are captured into outer-
scope variables so the success-path OpsEvent + retry-rollback work even when
the inner try exited via the lease-loss path.
* leaseHolder.DisposeAsync now wraps the whole leased section in a single
finally — the previous version had two finally blocks (one for the inner
service-call try, one for the outer). Single dispose ⇒ no double-release race.
Tests: 1952/1952 still green. Lease-loss code paths are exercised at the unit
level via the existing renewal-failure logic on MaintenanceLeaseHolder + the
service throw classification on CriticalTableBackupService.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three Codex-review findings in PR1, all real:
* HIGH — Maintenance lease expires after 60 s but the per-run budget is 50 min.
Without renewal a parallel worker / timer / restore could enter the same
"exclusive" section after one minute, and the watchdog would see a free
lease while a job is still running. New MaintenanceLeaseHolder spins a
renewal task on a 45 s cadence and cancels the handler's CTS on any
renewal failure; the in-flight backup then bails via OperationCanceledException
and the worker/timer route the job to a clean Failed state without
queue-deleting the message.
* HIGH — Retry-rollback resets State=Queued + BackupId=null but left QueuedAtUtc
on the original wall-clock value. After a retryable service throw the
watchdog's 60 min "Queued without pickup" rule could fire mid retry-cycle
and call the job dead. Fix: stamp QueuedAtUtc=now on the rollback so the
watchdog respects the fresh retry budget.
* MEDIUM — ListBackups returned every top-level prefix including runs that
never reached manifest.json. Calling GetBackupManifest on such an id returned
404 — confusing UI signal for incomplete runs. Fix: ListBackupIdsAsync now
Exists()-checks manifest.json per prefix. New ListAllBackupPrefixesAsync
yields (id, hasManifest) tuples for a future admin UI that wants to
surface incomplete runs as their own status.
Tests: 1952/1952 backend xUnit still green — no regressions, no new test
coverage needed because the hotfix paths are exercised indirectly by the
existing happy/partial/budget tests + the production storage SDK semantics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the full backup pipeline that snapshots 15 critical tables (Admin/Tenant
config, rules, identities, audit, preview) into a logically-isolated blob
container, daily at 04:00 UTC and on operator demand. Restore endpoints follow
in PR2/PR3.
Architecture:
* Daily TimerTrigger + manual POST /api/global/backups/trigger (GA-only,
fail-hard producer). HTTP returns 202 + jobId on enqueue, 500 if the queue
write fails (and the BackupJobs row is rolled to Failed).
* BackgroundService queue worker (not QueueTrigger — matches the Analyze/
IndexReconcile pattern that avoids the Functions-host connection-binding
app-setting). Worker holds the maintenance lease for the whole run; service
is intentionally lease-unaware so it stays unit-testable.
* BlobBackupStore wraps the new 'critical-table-backups' container with a
shared maintenance-lease sentinel at '_lock/maintenance.lock' that
serialises every backup or restore op across worker/timer/single-row paths.
* CriticalTableBackupService streams each table through CryptoStream(SHA256)
into per-table NDJSON blobs, then writes manifest.json LAST as the
durability anchor. Per-table failures degrade Outcome to Partial without
aborting; per-run budget (50 min) → Skipped status.
* BackupJobsRepository persists the six-state machine
(Queued/Running/Completed/Failed/Skipped/BlockedTerminal) with ETag-CAS
updates. Enum→string mapping rejects unknown values on read so refactors
surface loudly instead of silently shifting state.
* BackupJobWatchdog (30 min timer) heals stuck rows via a probe lease on the
shared sentinel: Queued > 60 min → Failed, Running with stale heartbeat
AND free lease → Failed (lease-busy means a live worker — leave alone).
* OpsEvents: distinct CriticalTableBackupPartial (Warning) vs Failed (Error)
so operators can wire softer Telegram rules for partial captures.
* MaintenanceService.MonitoredPoisonQueues + OPS_EVENT_TYPES catalog
registered (dual-register memory).
Test coverage (15 new tests, 1952/1952 backend xUnit green):
* BackupManifestJsonTests: enum-as-string (allowIntegerValues=false rejects
numeric values), DictionaryKeyPolicy unset (Azure column names preserved
case-faithfully), manifest field round-trip.
* BackupJobsRepositoryMappingTests: full entity↔domain round-trip, unknown
Kind/State/BackupOutcome rejected, legacy BackupOutcome="Failed" rejected.
* CriticalTableBackupServiceTests: happy/partial/empty/budget-exceeded paths
exercise the per-table loop with in-memory table + blob fakes.
Setup: scripts/Backup/setup-backup-container.ps1 — idempotent az-CLI bootstrap
for container + lifecycle rule (90 d delete on prefix). Soft-Delete and
Versioning are NOT applied automatically (storage-account-wide blast radius
— operator must validate per separate doc note).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Foundation for the critical-table backup feature (plan §PR0). Pure lib
additions + refactor, no behavior change, no new endpoints.
* Lift MapEntityToDump / ConvertToPropValue out of DeletionManifestBuilder
into Services/Tables/TableEntityDumpConverter (public static) so the
upcoming backup pipeline can share the exact EDM-typed row-dump shape.
* Add TableStorageService.UpsertRowsByExactKeysInBatchesAsync with
TableTransactionActionType.UpsertReplace — the only safe primitive
for "overwrite a corrupt live row from backup" (the existing
Add-only RestoreRowsByExactKeysInBatchesAsync would skip 409s).
* Declare Constants.TableNames.BackupJobs and add it to TableNames.All
so startup auto-creates the per-job state table for PR1.
Build: 0 errors, 0 new warnings. 1937/1937 backend xUnit tests pass —
the existing DeletionManifestBuilder suite now runs through the lifted
converter and confirms byte identity.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Patches a remotely triggerable DoS in qs.stringify (TypeError on
null/undefined entries in comma-format arrays when encodeValuesOnly
is set). Pulled in transitively via express@5.2.1 / body-parser@2.2.2,
both of which already allow ^6.14.x — lockfile-only bump, no
package.json change needed. Closes Dependabot alert #46.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the HKLM\SOFTWARE\RealmJoin (+ HKU\<sid>\...) subtree watchers attached the
moment the Services\realmjoin gate fired. The window between "RJ installed" and "RJ
actually deploying" is exactly when IME may still be writing to the same SOFTWARE\RealmJoin
subtree as part of unrelated package installs — those writes were being misattributed
to RJ's package lifecycle (realmjoin_package_started / _completed events for packages
RJ never deployed).
Defer the package watchers until RJ has moved out of Blank, i.e. DeploymentPhase >=
PhaseRunningThresholdMin (100 = RunningFirstDeployment, RunningFirstDeploymentAuto,
RunningDeployment, ManualDeployment, NestedFromShim, or terminal CompletedFirstDeployment).
The realmjoin_detected event still fires on service-key appearance (no change to the
detection gate); only the package-tracking surface stays disarmed for the pre-running
window.
- New const RealmJoinInfo.PhaseRunningThresholdMin = 100.
- Watcher state flag _packageWatchersArmed (set-once); old _realmJoinServiceDetected
removed (no remaining readers).
- AttachServiceRealmjoinSubtreeWatcher no longer arms the package watchers — it only
attaches the Services\realmjoin DeploymentPhase watcher and fires Detected.
- NotifyRealmJoinPresence arms HKLM + (if SID known) HKU package watchers on the first
phase >= 100 observation; set-once so a later regressed reading does not disarm.
- ArmHku now defers on _packageWatchersArmed instead of the dropped service-detected flag.
Tests: +3 (disarmed-during-blank, set-once-after-running, immediate-arm-on-boot-at-110).
14/14 RealmJoin adapter suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The realmjoin_phase_changed message read like "DeploymentPhase 100 -> 110" — the raw
ints are correct but require a lookup table to interpret. Map them to the RJ-internal
DeploymentPhase enum names (Blank / RunningFirstDeployment[Auto] / CompletedFirstDeployment /
RunningDeployment / CompletedDeployment / ManualDeployment / NestedFromShim) so the
timeline + log message read like "DeploymentPhase RunningFirstDeployment -> CompletedFirstDeployment"
without a lookup. The numeric ints stay in the payload (deploymentPhase / previousPhase)
so KQL filters and dashboards do not have to switch.
- New RealmJoinInfo.PhaseDisplayName(int) — single switch matching the published RJ enum.
Unknown / future values fall back to the numeric string so they stay observable instead
of being collapsed to "Unknown".
- RealmJoinWatcherAdapter.OnPhaseChanged: payload now carries the two new
deploymentPhaseName / previousPhaseName keys; message rendered with names.
Tests: +1 (mapping known phase ints → enum names + message format),
existing phase-changed test extended to assert both raw + named fields.
11/11 adapter suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Today's RJ doesn't populate the DisplayName registry value for most package subkeys
(only ArgsHash / Success / LastExitCode / Version / Type are written — confirmed on
generic-vlc and generic-gkgab-set-wallpaper-preference). The old MaybeFirePackageEvents
gated the started signal on snap.HasStartedMarker (= DisplayName non-empty), so
realmjoin_package_started never fired for these packages and only realmjoin_package_completed
landed with an empty DisplayName slot.
Change: drop the HasStartedMarker guard from the started branch. The first observation
of a new <packageId> sub-key is itself the started signal, period. DisplayName stays in
the payload + message as a passthrough field — empty when RJ didn't advertise a name
(useful "RJ didn't write it" indicator on the wire) and populated automatically when a
future RJ version starts writing it at install-start. Completed semantics unchanged
(still gated on Success + LastExitCode presence).
Behavior for pre-existing/already-completed packages observed on agent boot: a single
MaybeFirePackageEvents pass now fires BOTH started AND completed (started first), so
the timeline reflects the full lifecycle even when we missed the live transition.
Tests: +2 watcher tests via the new TriggerMachinePackageObservationFromTest seam —
"fires started even without DisplayName" + "fires both started and completed for
pre-existing snapshot with completion markers". 10/10 adapter suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Surface the RJ host version alongside the existing detection signal so operators
can see at a glance "RJ detected, version x.y.z" without having to cross-reference
package logs.
Read path (RealmJoinInfo.TryReadRealmJoinProductVersion):
- FileVersionInfo.GetVersionInfo against C:\Program Files\RealmJoin\RealmJoin.exe.
- (x86) fallback path probed when the 64-bit install is absent.
- Defensive try/catch + File.Exists pre-check: a missing/locked/inaccessible
binary returns null. Version is observability-only and must never block the
detection signal — the realmjoin_detected event still emits without version.
- Called exactly ONCE per session, on the first Detected fire-once boundary.
Wire:
- RealmJoinDetectedEventArgs now carries an optional ProductVersion string.
- RealmJoinPayloadKeys.ProductVersion ("productVersion") plumbed through both
the DecisionSignal payload and the InformationalEvent data dict + message tail.
- RealmJoinFacts.ProductVersion (SignalFact<string>, set-once) recorded on the
reducer side so the enrollment_complete audit trail picks it up as
realmjoinProductVersion via DecisionAuditTrailBuilder.
Tests: +2 adapter tests (with-version + without-version) — 12/12 green
on the RealmJoin + SID suite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 4fa5a2d4 in tenant c9787ba2 hit 199/202 false-positive enrollment_failed events: ESP
profile sets BlockInStatusPage=6 (ContinueAnyway=true), Windows transitioned to AccountSetup,
then wrote DeviceSetupCategory=failed (Certificates) 13 min later — the reducer terminated
even though the device demonstrably kept progressing. PR1 Tier-1 defang downgrades these
EspTerminalFailure signals to a non-terminal esp_failure_advisory event while keeping
monitoring armed (Hello/Desktop/IME completion paths continue to drive the session).
DecisionEngine:
- New SignalFact<DateTime>? EspAdvisoryFailureRecordedUtc on DecisionState as the fire-once
gate (additive ctor param, optional at end — preserves Newtonsoft positional-binding for
pre-PR1 v4 snapshots).
- HandleEspTerminalFailureV1: when ScenarioObservations.EspAllowContinueAnyway==true AND
AccountSetupEnteredUtc!=null, emit esp_failure_advisory (severity=Warning,
advisoryReason=esp_failure_defanged_continueanyway_with_accountsetup) instead of
transitioning to Failed. Stage / deadlines / outcome stay untouched.
- Duplicate handling: a second EspTerminalFailure dead-ends UNLESS the duplicate carries
registry-specific keys (errorCode/failedSubcategory/category) — those only appear in the
ProvisioningStatusTracker path and indicate enrichment of an earlier ShellCoreTracker
advisory. failureType alone is NOT a registry marker (both adapters emit it).
- New BuildEspFailureAuditTrail helper merges a whitelisted set of failure-context keys into
the audit-trail typedPayload so HRESULT/subcategory survive the wire. Closes a pre-existing
gap on the enrollment_failed path too.
Bootstrap:
- EnrollmentOrchestrator.PostEspConfigDetectedBootstrap switched from EspSkipConfigurationProbe.
Read() (tuple SkipUser/SkipDevice) to ReadFull() so EspAllowContinueAnyway +
EspSyncFailureTimeoutMinutes are synchronously available before any EspTerminalFailure
signal — defeats the fire-and-forget DeviceInfoCollector.CollectAll race that would have
left Tier 1 with ScenarioObservations.EspAllowContinueAnyway==null.
Tests:
- EspFailureAdvisoryDefangTests (9 new): advisory path, fire-once idempotency, enrichment
on registry-detail second-fire, dead-end on ShellCore-only duplicate, three negative paths
(ContinueAnyway false / unknown / no AccountSetup), TypedPayload whitelist merge on both
advisory + Failed paths.
- SerializationTests: backcompat test for old v4 snapshots without EspAdvisoryFailureRecordedUtc.
- DecisionStateSnapshotBuilderTests: fact count bumped from 14 to 15.
- EdgeScenarioTests + esp-failure-advisory-continueanyway-v1.jsonl: end-to-end replay against
the canonical Session 4fa5a2d4 signal stream including a duplicate dead-end.
- EnrollmentOrchestratorEspConfigBootstrapTests: 4 existing tests migrated to ScopedFullOverride,
4 new tests cover ContinueAnyway + SyncFailureTimeout propagation, class gated behind
[Collection("SerialThreading")] since the override targets a static probe slot.
UI:
- EventTimeline renders esp_failure_advisory with an amber "Advisory (ContinueAnyway)" badge
alongside the HRESULT/subcategory chips so analysts can distinguish defanged from real
failures at a glance.
- gather-rules/eventTypes.ts mirrors the new constant for autocomplete.
Build: dotnet build AutopilotMonitor.sln green; xUnit Backend(1937) + V2.Core(1139) +
Agent.Core(538) + DecisionCore(357) + SummaryDialog(44) = 4015 passed, 0 failed.
Web tsc clean, vitest 384/384 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dependabot alert #45: @nevware21/ts-utils <=0.13.0 is vulnerable to
prototype pollution via objDeepCopy/objCopyProps (for...in without
hasOwnProperty). Pulled in transitively by 10 @microsoft/applicationinsights-*
packages.
Parent constraint is >= 0.12.6 < 2.x, so an npm override to ^0.14.0 lifts
all transitives to the patched release without bumping the parent.
npm audit reports 0 vulnerabilities; tsc + 384 vitest tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 88a53223-9795-4188-8352-7df9f0af9bb7 (v1 Classic UserDriven, AAD
Joined, SkipUserStatusPage=false) was wrongly classified as mode=SelfDeploying
after the ProvisioningStatusTracker 30-s fallback fired DeviceSetupProvisioningComplete
before Windows transitioned DeviceSetup -> AccountSetup. The signal alone was
terminal in the old reducer; the absence of AccountSetupEnteredUtc at signal
time was sufficient to commit terminal SelfDeploying.
New rule: DeviceSetupProvisioningComplete is no longer terminal. It records a
new DeviceSetupResolvedUtc anchor and (cancel-then-)arms the DeviceOnlyEspDetection
deadline for 5 min. The deadline-fired handler is the sole SelfDeploying-terminal
entry point and re-checks six guards (stale: no-anchor / not-armed / DueAtUtc-mismatch;
race: Stage-terminal / AccountSetup-entered / monotonic Mode conflict) before
classifying. Any dead-end after the DueAtUtc match state-side CancelDeadline()s
the past-due deadline so snapshot reload doesn't re-arm via the scheduler.
RealmJoin-deferred release (CompleteIfDeferredOrBookkeep) re-checks the same
guards, resets DeviceOnlyDeployment hypothesis to Unknown on bailout, and
operates on preparedBuilder.* (not state.*) so WithResolved / WithTimeoutOutcome
facts survive the bailout.
UI phase coverage closed: phase_transition(FinalizingSetup) and
phase_transition(Complete) emitted at every terminal-success site (Classic
FinalizingGrace, SelfDeploying deadline-terminal, AdminPreemption-Succeeded,
RJ-deferred completion) so the Web timeline opens Finalizing + Complete bars
universally - was missing entirely for SelfDeploying / AdminPreemption paths.
Other changes:
- Removed ApplyDeviceSetupProvisioningComplete (dead code under defang), added
ApplySelfDeployingDeadlineConfirmed with monotonic guard.
- New RealmJoinFacts.ClearSelfDeployingDeferred() for the RJ-bailout path.
- DecisionState schema bumped v3 -> v4 with additive nullable
DeviceSetupResolvedUtc field. StateSerializer v3-deserialize-compat verified.
- DeviceSetupProvisioningComplete added to DecisionStateSignalCensus so the
enrollment_complete audit trail surfaces the anchor.
Test coverage:
- New SelfDeployingDeadlineTerminalTests.cs (12 scenarios): happy path, all
six guards including snapshot-migration rollout race + DueAtUtc cancel-rearm
race, replay idempotency, audit-census wiring, RJ-deferred release races
(AccountSetup-arrives / monotonic / Timeout-variant).
- Updated 9 existing test files for new semantics; 2 fixtures now drive
DeadlineFired terminal explicitly.
- ReplayHarness + ClassifierAwareReplayHarness set AgentBootUtc to the first
signal's OccurredAtUtc so fixture scenarios run in their own time-bubble
(replay-safety floor still tested directly by BootAnchorDeadlineFloorTests).
Tests: DecisionCore 345, V2.Core 1135, Backend 1937 - all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The post-terminal guard added in 7de094c6 blanket-rejected every signal kind once
state.Stage reached Completed/Failed - including InformationalEvent, which is the
single-rail pass-through used by EnrollmentTerminationHandler to emit every
post-enrollment_complete lifecycle event (agent_shutting_down, app_tracking_summary,
shutdown-delta software_inventory_analysis, diagnostics_collecting/_uploaded,
enrollment_summary_shown, reboot_triggered, whiteglove_part1_complete). Result:
every V2 session on agent builds >= 2.0.835 ended at enrollment_complete with
zero terminal lifecycle events on the wire (sample: 0/11 affected vs 9/9 working
on <=2.0.820). The cross-path agent_shutting_down idempotency gate also masked
the failure - the handler's Emit returned without throwing, claimed the gate, so
the L4 runtime_host_exit fallback short-circuited.
HandleInformationalEventV1 does NOT mutate Stage/Outcome/facts, does NOT arm
deadlines, only emits an EmitEventTimelineEntry effect - so letting it through
post-terminal cannot regress the duplicate-enrollment_complete protection that
motivated the original guard (enrollment_complete on the FinalizingGrace path
is emitted directly from the reducer step, not via InformationalEvent).
Regression coverage: two new tests in FinalizingStageTests pinning that
InformationalEvent after Completed AND after Failed both produce the expected
EmitEventTimelineEntry effect with the original payload keys intact.
Tests: 326 DecisionCore + 1135 V2.Core green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the parent-watch refactor.
P1a — Watcher self-disposal deadlock in the appearance callback. Calling
RegistryWatcher.Dispose() from the Changed handler re-entered Stop() → Thread.Join()
on the same thread that owned the WatchLoop, hanging the watcher and skipping
onAppeared(). Fix: RequestStop() (non-blocking) inside the callback, then
ThreadPool.QueueUserWorkItem the actual Dispose so the join runs on a different
thread after the loop has exited.
P1b — All RealmJoin reads + watchers now explicitly bind to RegistryView.Registry64.
The agent's intent is 64-bit, but the previous RegistryView.Default would still
silently route HKLM\SOFTWARE through WOW6432Node if the host were ever spawned as a
32-bit process. Same defensive note already lives in TenantIdResolver. Affects
RealmJoinInfo.cs (3 reads + enumerations), RealmJoinWatcher.KeyExists, and all four
RegistryWatcher instantiations (1 parent-watch + 3 subtree-watches).
P2 — Stale RealmJoinTimeout DeadlineFired after Resolved. Race: the cancel-effect
for the timeout reaches the scheduler at the same time the deadline fires; the
queued DeadlineFired signal is processed after RealmJoinResolved has already set
Outcome="Resolved". Without a guard, the timeout handler still emitted a spurious
realmjoin_timeout timeline event and re-entered TransitionToFinalizing, duplicating
phase_transition(FinalizingSetup) on the wire and the FinalizingGrace deadline arm.
Fix: short-circuit to a bookkept dead-end transition when either Outcome is already
set OR the RealmJoinTimeout deadline is no longer in state.Deadlines (the cancel
took effect). DeadEndReason carries the specific cause for the Inspector.
Tests: DecisionCore 322 → 323 (new Stale_RealmJoinTimeout_after_Resolved_is_bookkept_dead_end_no_effects).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HelloDisabledFastPath in HandleDesktopArrivedV1 used a weak guard
(AccountSetupEnteredUtc != null) while HandleEspExitingV1 already used the
strong gate (ShouldTransitionToAwaitingHello: AccountSetupProvisioningSucceededUtc
OR ScenarioObservations.SkipUserEsp). Session 08c99638 exposed the asymmetry:
Shell-Core event 62407 fired esp_exiting with errorCode=0 while AccountSetup
categorySucceeded was still in_progress (0/N apps complete). A late
desktop_arrived then drove enrollment_complete via the weak guard.
The FastPath now reuses ShouldTransitionToAwaitingHello, restoring parity with
the EspExiting path. Sessions where ESP "exits" before AccountSetup genuinely
finishes stay parked in AwaitingDesktop and eventually fall to the max-lifetime
failure path.
Tests:
- HelloDisabled_FastPath_skipped_when_AccountSetup_entered_but_provisioning_never_succeeded
(08c99638 repro)
- HelloDisabled_FastPath_fires_on_SkipUserEsp_flow_even_without_AccountSetupProvisioningComplete
(SkipUser arm of the strong gate)
DecisionCore 324/324 + V2 Core 1135/1135 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous staging armed HKLM\SOFTWARE\RealmJoin in parallel with the Services\realmjoin
parent-watch, and ArmHku (called on Desktop-Arrival) attached HKU\<sid>\SOFTWARE\RealmJoin
regardless of whether RJ was actually installed. Both are wasted observation surface on
the ~80% of devices that never get RJ — and HKU\<sid>\SOFTWARE in particular is a noisy
parent with many subkey changes per session.
Single entry point now: Services\realmjoin is the SOLE detection gate.
- Start() only arms the parent-watch on HKLM\SYSTEM\CurrentControlSet\Services.
- ArmHku(sid) records the SID but no HKU watcher is created until the gate fires.
- AttachServiceRealmjoinSubtreeWatcher (= "RJ detected") now also triggers:
- EnsureMachineRealmJoinStage (HKLM\SOFTWARE\RealmJoin subtree)
- EnsureUserRealmJoinStage (HKU\<sid>\SOFTWARE\RealmJoin subtree) — only when a SID
was already recorded; otherwise the later ArmHku(sid) sees _realmJoinServiceDetected
and arms immediately.
- Result on a non-RJ device: a single parent-watch on the Services hive, sleeping idle.
No HKLM\SOFTWARE parent-watch, no HKU parent-watch.
Tests: 10/10 RealmJoin/SID tests still green (the trigger-from-test seams target the
read paths the gate ordering does not touch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original watcher armed 2 s retry timers when the RJ keys did not exist yet — fine on
RJ-using devices, wasteful on the ~80% that never have RJ installed (10800 polls per 6 h
session). Switch to the canonical RegNotifyChangeKeyValue pattern: arm a Name-filter
notification on the always-existing parent and stay idle until the target sub-key
actually appears.
Per scope (machine + user) three stages now:
1. Boot sync probe — if target already exists, jump straight to stage 3.
2. Parent-watch on the always-existing root with Name filter:
HKLM\SYSTEM\CurrentControlSet\Services waits for `realmjoin`
HKLM\SOFTWARE waits for `RealmJoin`
HKU\<sid>\SOFTWARE waits for `RealmJoin`
3. On appearance: parent-watch disposed; full subtree watcher attached on the new key
with LastSet+Name filter. CheckParameters / CheckMachinePackages / CheckUserPackages
fire under a debounce timer as before.
Side-effects:
- New constants in RealmJoinInfo for the always-existing parents + target sub-key names.
- The appearance event itself now fires RealmJoinDetected even before the
Parameters\DeploymentPhase value is populated (phase=0 placeholder), matching the
approved "Trigger: pure existence of the realmjoin service key" design choice.
- All retry timers removed; only the appearance + value debounce timers remain.
Tests: 10/10 RealmJoin/SID tests still green (the existing TriggerCheckXxxFromTest seams
exercise the same read paths the change preserves).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- rules/guardrails.json: add new "RealmJoin" category covering
SYSTEM\CurrentControlSet\Services\realmjoin and SOFTWARE\RealmJoin. Hive-agnostic
match covers HKLM\SYSTEM\..., HKLM\SOFTWARE\RealmJoin, and HKU\<sid>\SOFTWARE\RealmJoin
so operator-authored Gather-rules can probe the RJ keys (the RealmJoinWatcher
itself bypasses GatherRuleGuards and watches the registry directly).
- src/Web/autopilot-monitor-web/lib/guardrails.generated.ts: regenerated via
node rules/scripts/combine.js so the Web admin UI surface matches the new
registry allowlist.
- buildcounter.txt (V2 + V1): auto-incremented by the Phase A-D builds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Oliver Kieselbach