The realmjoin_phase_changed message read like "DeploymentPhase 100 -> 110" — the raw
ints are correct but require a lookup table to interpret. Map them to the RJ-internal
DeploymentPhase enum names (Blank / RunningFirstDeployment[Auto] / CompletedFirstDeployment /
RunningDeployment / CompletedDeployment / ManualDeployment / NestedFromShim) so the
timeline + log message read like "DeploymentPhase RunningFirstDeployment -> CompletedFirstDeployment"
without a lookup. The numeric ints stay in the payload (deploymentPhase / previousPhase)
so KQL filters and dashboards do not have to switch.
- New RealmJoinInfo.PhaseDisplayName(int) — single switch matching the published RJ enum.
Unknown / future values fall back to the numeric string so they stay observable instead
of being collapsed to "Unknown".
- RealmJoinWatcherAdapter.OnPhaseChanged: payload now carries the two new
deploymentPhaseName / previousPhaseName keys; message rendered with names.
Tests: +1 (mapping known phase ints → enum names + message format),
existing phase-changed test extended to assert both raw + named fields.
11/11 adapter suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Today's RJ doesn't populate the DisplayName registry value for most package subkeys
(only ArgsHash / Success / LastExitCode / Version / Type are written — confirmed on
generic-vlc and generic-gkgab-set-wallpaper-preference). The old MaybeFirePackageEvents
gated the started signal on snap.HasStartedMarker (= DisplayName non-empty), so
realmjoin_package_started never fired for these packages and only realmjoin_package_completed
landed with an empty DisplayName slot.
Change: drop the HasStartedMarker guard from the started branch. The first observation
of a new <packageId> sub-key is itself the started signal, period. DisplayName stays in
the payload + message as a passthrough field — empty when RJ didn't advertise a name
(useful "RJ didn't write it" indicator on the wire) and populated automatically when a
future RJ version starts writing it at install-start. Completed semantics unchanged
(still gated on Success + LastExitCode presence).
Behavior for pre-existing/already-completed packages observed on agent boot: a single
MaybeFirePackageEvents pass now fires BOTH started AND completed (started first), so
the timeline reflects the full lifecycle even when we missed the live transition.
Tests: +2 watcher tests via the new TriggerMachinePackageObservationFromTest seam —
"fires started even without DisplayName" + "fires both started and completed for
pre-existing snapshot with completion markers". 10/10 adapter suite green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Surface the RJ host version alongside the existing detection signal so operators
can see at a glance "RJ detected, version x.y.z" without having to cross-reference
package logs.
Read path (RealmJoinInfo.TryReadRealmJoinProductVersion):
- FileVersionInfo.GetVersionInfo against C:\Program Files\RealmJoin\RealmJoin.exe.
- (x86) fallback path probed when the 64-bit install is absent.
- Defensive try/catch + File.Exists pre-check: a missing/locked/inaccessible
binary returns null. Version is observability-only and must never block the
detection signal — the realmjoin_detected event still emits without version.
- Called exactly ONCE per session, on the first Detected fire-once boundary.
Wire:
- RealmJoinDetectedEventArgs now carries an optional ProductVersion string.
- RealmJoinPayloadKeys.ProductVersion ("productVersion") plumbed through both
the DecisionSignal payload and the InformationalEvent data dict + message tail.
- RealmJoinFacts.ProductVersion (SignalFact<string>, set-once) recorded on the
reducer side so the enrollment_complete audit trail picks it up as
realmjoinProductVersion via DecisionAuditTrailBuilder.
Tests: +2 adapter tests (with-version + without-version) — 12/12 green
on the RealmJoin + SID suite.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 4fa5a2d4 in tenant c9787ba2 hit 199/202 false-positive enrollment_failed events: ESP
profile sets BlockInStatusPage=6 (ContinueAnyway=true), Windows transitioned to AccountSetup,
then wrote DeviceSetupCategory=failed (Certificates) 13 min later — the reducer terminated
even though the device demonstrably kept progressing. PR1 Tier-1 defang downgrades these
EspTerminalFailure signals to a non-terminal esp_failure_advisory event while keeping
monitoring armed (Hello/Desktop/IME completion paths continue to drive the session).
DecisionEngine:
- New SignalFact<DateTime>? EspAdvisoryFailureRecordedUtc on DecisionState as the fire-once
gate (additive ctor param, optional at end — preserves Newtonsoft positional-binding for
pre-PR1 v4 snapshots).
- HandleEspTerminalFailureV1: when ScenarioObservations.EspAllowContinueAnyway==true AND
AccountSetupEnteredUtc!=null, emit esp_failure_advisory (severity=Warning,
advisoryReason=esp_failure_defanged_continueanyway_with_accountsetup) instead of
transitioning to Failed. Stage / deadlines / outcome stay untouched.
- Duplicate handling: a second EspTerminalFailure dead-ends UNLESS the duplicate carries
registry-specific keys (errorCode/failedSubcategory/category) — those only appear in the
ProvisioningStatusTracker path and indicate enrichment of an earlier ShellCoreTracker
advisory. failureType alone is NOT a registry marker (both adapters emit it).
- New BuildEspFailureAuditTrail helper merges a whitelisted set of failure-context keys into
the audit-trail typedPayload so HRESULT/subcategory survive the wire. Closes a pre-existing
gap on the enrollment_failed path too.
Bootstrap:
- EnrollmentOrchestrator.PostEspConfigDetectedBootstrap switched from EspSkipConfigurationProbe.
Read() (tuple SkipUser/SkipDevice) to ReadFull() so EspAllowContinueAnyway +
EspSyncFailureTimeoutMinutes are synchronously available before any EspTerminalFailure
signal — defeats the fire-and-forget DeviceInfoCollector.CollectAll race that would have
left Tier 1 with ScenarioObservations.EspAllowContinueAnyway==null.
Tests:
- EspFailureAdvisoryDefangTests (9 new): advisory path, fire-once idempotency, enrichment
on registry-detail second-fire, dead-end on ShellCore-only duplicate, three negative paths
(ContinueAnyway false / unknown / no AccountSetup), TypedPayload whitelist merge on both
advisory + Failed paths.
- SerializationTests: backcompat test for old v4 snapshots without EspAdvisoryFailureRecordedUtc.
- DecisionStateSnapshotBuilderTests: fact count bumped from 14 to 15.
- EdgeScenarioTests + esp-failure-advisory-continueanyway-v1.jsonl: end-to-end replay against
the canonical Session 4fa5a2d4 signal stream including a duplicate dead-end.
- EnrollmentOrchestratorEspConfigBootstrapTests: 4 existing tests migrated to ScopedFullOverride,
4 new tests cover ContinueAnyway + SyncFailureTimeout propagation, class gated behind
[Collection("SerialThreading")] since the override targets a static probe slot.
UI:
- EventTimeline renders esp_failure_advisory with an amber "Advisory (ContinueAnyway)" badge
alongside the HRESULT/subcategory chips so analysts can distinguish defanged from real
failures at a glance.
- gather-rules/eventTypes.ts mirrors the new constant for autocomplete.
Build: dotnet build AutopilotMonitor.sln green; xUnit Backend(1937) + V2.Core(1139) +
Agent.Core(538) + DecisionCore(357) + SummaryDialog(44) = 4015 passed, 0 failed.
Web tsc clean, vitest 384/384 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dependabot alert #45: @nevware21/ts-utils <=0.13.0 is vulnerable to
prototype pollution via objDeepCopy/objCopyProps (for...in without
hasOwnProperty). Pulled in transitively by 10 @microsoft/applicationinsights-*
packages.
Parent constraint is >= 0.12.6 < 2.x, so an npm override to ^0.14.0 lifts
all transitives to the patched release without bumping the parent.
npm audit reports 0 vulnerabilities; tsc + 384 vitest tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 88a53223-9795-4188-8352-7df9f0af9bb7 (v1 Classic UserDriven, AAD
Joined, SkipUserStatusPage=false) was wrongly classified as mode=SelfDeploying
after the ProvisioningStatusTracker 30-s fallback fired DeviceSetupProvisioningComplete
before Windows transitioned DeviceSetup -> AccountSetup. The signal alone was
terminal in the old reducer; the absence of AccountSetupEnteredUtc at signal
time was sufficient to commit terminal SelfDeploying.
New rule: DeviceSetupProvisioningComplete is no longer terminal. It records a
new DeviceSetupResolvedUtc anchor and (cancel-then-)arms the DeviceOnlyEspDetection
deadline for 5 min. The deadline-fired handler is the sole SelfDeploying-terminal
entry point and re-checks six guards (stale: no-anchor / not-armed / DueAtUtc-mismatch;
race: Stage-terminal / AccountSetup-entered / monotonic Mode conflict) before
classifying. Any dead-end after the DueAtUtc match state-side CancelDeadline()s
the past-due deadline so snapshot reload doesn't re-arm via the scheduler.
RealmJoin-deferred release (CompleteIfDeferredOrBookkeep) re-checks the same
guards, resets DeviceOnlyDeployment hypothesis to Unknown on bailout, and
operates on preparedBuilder.* (not state.*) so WithResolved / WithTimeoutOutcome
facts survive the bailout.
UI phase coverage closed: phase_transition(FinalizingSetup) and
phase_transition(Complete) emitted at every terminal-success site (Classic
FinalizingGrace, SelfDeploying deadline-terminal, AdminPreemption-Succeeded,
RJ-deferred completion) so the Web timeline opens Finalizing + Complete bars
universally - was missing entirely for SelfDeploying / AdminPreemption paths.
Other changes:
- Removed ApplyDeviceSetupProvisioningComplete (dead code under defang), added
ApplySelfDeployingDeadlineConfirmed with monotonic guard.
- New RealmJoinFacts.ClearSelfDeployingDeferred() for the RJ-bailout path.
- DecisionState schema bumped v3 -> v4 with additive nullable
DeviceSetupResolvedUtc field. StateSerializer v3-deserialize-compat verified.
- DeviceSetupProvisioningComplete added to DecisionStateSignalCensus so the
enrollment_complete audit trail surfaces the anchor.
Test coverage:
- New SelfDeployingDeadlineTerminalTests.cs (12 scenarios): happy path, all
six guards including snapshot-migration rollout race + DueAtUtc cancel-rearm
race, replay idempotency, audit-census wiring, RJ-deferred release races
(AccountSetup-arrives / monotonic / Timeout-variant).
- Updated 9 existing test files for new semantics; 2 fixtures now drive
DeadlineFired terminal explicitly.
- ReplayHarness + ClassifierAwareReplayHarness set AgentBootUtc to the first
signal's OccurredAtUtc so fixture scenarios run in their own time-bubble
(replay-safety floor still tested directly by BootAnchorDeadlineFloorTests).
Tests: DecisionCore 345, V2.Core 1135, Backend 1937 - all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The post-terminal guard added in 7de094c6 blanket-rejected every signal kind once
state.Stage reached Completed/Failed - including InformationalEvent, which is the
single-rail pass-through used by EnrollmentTerminationHandler to emit every
post-enrollment_complete lifecycle event (agent_shutting_down, app_tracking_summary,
shutdown-delta software_inventory_analysis, diagnostics_collecting/_uploaded,
enrollment_summary_shown, reboot_triggered, whiteglove_part1_complete). Result:
every V2 session on agent builds >= 2.0.835 ended at enrollment_complete with
zero terminal lifecycle events on the wire (sample: 0/11 affected vs 9/9 working
on <=2.0.820). The cross-path agent_shutting_down idempotency gate also masked
the failure - the handler's Emit returned without throwing, claimed the gate, so
the L4 runtime_host_exit fallback short-circuited.
HandleInformationalEventV1 does NOT mutate Stage/Outcome/facts, does NOT arm
deadlines, only emits an EmitEventTimelineEntry effect - so letting it through
post-terminal cannot regress the duplicate-enrollment_complete protection that
motivated the original guard (enrollment_complete on the FinalizingGrace path
is emitted directly from the reducer step, not via InformationalEvent).
Regression coverage: two new tests in FinalizingStageTests pinning that
InformationalEvent after Completed AND after Failed both produce the expected
EmitEventTimelineEntry effect with the original payload keys intact.
Tests: 326 DecisionCore + 1135 V2.Core green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the parent-watch refactor.
P1a — Watcher self-disposal deadlock in the appearance callback. Calling
RegistryWatcher.Dispose() from the Changed handler re-entered Stop() → Thread.Join()
on the same thread that owned the WatchLoop, hanging the watcher and skipping
onAppeared(). Fix: RequestStop() (non-blocking) inside the callback, then
ThreadPool.QueueUserWorkItem the actual Dispose so the join runs on a different
thread after the loop has exited.
P1b — All RealmJoin reads + watchers now explicitly bind to RegistryView.Registry64.
The agent's intent is 64-bit, but the previous RegistryView.Default would still
silently route HKLM\SOFTWARE through WOW6432Node if the host were ever spawned as a
32-bit process. Same defensive note already lives in TenantIdResolver. Affects
RealmJoinInfo.cs (3 reads + enumerations), RealmJoinWatcher.KeyExists, and all four
RegistryWatcher instantiations (1 parent-watch + 3 subtree-watches).
P2 — Stale RealmJoinTimeout DeadlineFired after Resolved. Race: the cancel-effect
for the timeout reaches the scheduler at the same time the deadline fires; the
queued DeadlineFired signal is processed after RealmJoinResolved has already set
Outcome="Resolved". Without a guard, the timeout handler still emitted a spurious
realmjoin_timeout timeline event and re-entered TransitionToFinalizing, duplicating
phase_transition(FinalizingSetup) on the wire and the FinalizingGrace deadline arm.
Fix: short-circuit to a bookkept dead-end transition when either Outcome is already
set OR the RealmJoinTimeout deadline is no longer in state.Deadlines (the cancel
took effect). DeadEndReason carries the specific cause for the Inspector.
Tests: DecisionCore 322 → 323 (new Stale_RealmJoinTimeout_after_Resolved_is_bookkept_dead_end_no_effects).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HelloDisabledFastPath in HandleDesktopArrivedV1 used a weak guard
(AccountSetupEnteredUtc != null) while HandleEspExitingV1 already used the
strong gate (ShouldTransitionToAwaitingHello: AccountSetupProvisioningSucceededUtc
OR ScenarioObservations.SkipUserEsp). Session 08c99638 exposed the asymmetry:
Shell-Core event 62407 fired esp_exiting with errorCode=0 while AccountSetup
categorySucceeded was still in_progress (0/N apps complete). A late
desktop_arrived then drove enrollment_complete via the weak guard.
The FastPath now reuses ShouldTransitionToAwaitingHello, restoring parity with
the EspExiting path. Sessions where ESP "exits" before AccountSetup genuinely
finishes stay parked in AwaitingDesktop and eventually fall to the max-lifetime
failure path.
Tests:
- HelloDisabled_FastPath_skipped_when_AccountSetup_entered_but_provisioning_never_succeeded
(08c99638 repro)
- HelloDisabled_FastPath_fires_on_SkipUserEsp_flow_even_without_AccountSetupProvisioningComplete
(SkipUser arm of the strong gate)
DecisionCore 324/324 + V2 Core 1135/1135 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous staging armed HKLM\SOFTWARE\RealmJoin in parallel with the Services\realmjoin
parent-watch, and ArmHku (called on Desktop-Arrival) attached HKU\<sid>\SOFTWARE\RealmJoin
regardless of whether RJ was actually installed. Both are wasted observation surface on
the ~80% of devices that never get RJ — and HKU\<sid>\SOFTWARE in particular is a noisy
parent with many subkey changes per session.
Single entry point now: Services\realmjoin is the SOLE detection gate.
- Start() only arms the parent-watch on HKLM\SYSTEM\CurrentControlSet\Services.
- ArmHku(sid) records the SID but no HKU watcher is created until the gate fires.
- AttachServiceRealmjoinSubtreeWatcher (= "RJ detected") now also triggers:
- EnsureMachineRealmJoinStage (HKLM\SOFTWARE\RealmJoin subtree)
- EnsureUserRealmJoinStage (HKU\<sid>\SOFTWARE\RealmJoin subtree) — only when a SID
was already recorded; otherwise the later ArmHku(sid) sees _realmJoinServiceDetected
and arms immediately.
- Result on a non-RJ device: a single parent-watch on the Services hive, sleeping idle.
No HKLM\SOFTWARE parent-watch, no HKU parent-watch.
Tests: 10/10 RealmJoin/SID tests still green (the trigger-from-test seams target the
read paths the gate ordering does not touch).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original watcher armed 2 s retry timers when the RJ keys did not exist yet — fine on
RJ-using devices, wasteful on the ~80% that never have RJ installed (10800 polls per 6 h
session). Switch to the canonical RegNotifyChangeKeyValue pattern: arm a Name-filter
notification on the always-existing parent and stay idle until the target sub-key
actually appears.
Per scope (machine + user) three stages now:
1. Boot sync probe — if target already exists, jump straight to stage 3.
2. Parent-watch on the always-existing root with Name filter:
HKLM\SYSTEM\CurrentControlSet\Services waits for `realmjoin`
HKLM\SOFTWARE waits for `RealmJoin`
HKU\<sid>\SOFTWARE waits for `RealmJoin`
3. On appearance: parent-watch disposed; full subtree watcher attached on the new key
with LastSet+Name filter. CheckParameters / CheckMachinePackages / CheckUserPackages
fire under a debounce timer as before.
Side-effects:
- New constants in RealmJoinInfo for the always-existing parents + target sub-key names.
- The appearance event itself now fires RealmJoinDetected even before the
Parameters\DeploymentPhase value is populated (phase=0 placeholder), matching the
approved "Trigger: pure existence of the realmjoin service key" design choice.
- All retry timers removed; only the appearance + value debounce timers remain.
Tests: 10/10 RealmJoin/SID tests still green (the existing TriggerCheckXxxFromTest seams
exercise the same read paths the change preserves).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- rules/guardrails.json: add new "RealmJoin" category covering
SYSTEM\CurrentControlSet\Services\realmjoin and SOFTWARE\RealmJoin. Hive-agnostic
match covers HKLM\SYSTEM\..., HKLM\SOFTWARE\RealmJoin, and HKU\<sid>\SOFTWARE\RealmJoin
so operator-authored Gather-rules can probe the RJ keys (the RealmJoinWatcher
itself bypasses GatherRuleGuards and watches the registry directly).
- src/Web/autopilot-monitor-web/lib/guardrails.generated.ts: regenerated via
node rules/scripts/combine.js so the Web admin UI surface matches the new
registry allowlist.
- buildcounter.txt (V2 + V1): auto-incremented by the Phase A-D builds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wire the V2 agent runtime that feeds the DecisionCore RealmJoin gate from Phase A.
On every Autopilot session the agent now observes the RJ deployment-state registry
keys and dual-emits DecisionSignals + InformationalEvents so the engine extends the
completion AND-gate while RJ is in flight.
Agent.V2.Core additions:
- RealmJoinInfo: registry constants + DeploymentPhase/Package readers
- UserSidResolver: advapi32 LookupAccountName + ConvertSidToStringSid P/Invoke
- RealmJoinWatcher: three RegistryWatchers (HKLM Parameters, HKLM Packages subtree,
HKU\<sid> Packages subtree). Lazy attach with 2 s retry, 500 ms debounce, catch-up
scan on Start() so pre-existing RJ state emits the matching one-shot events
immediately. Strict dedup on Detected/Resolved/Timeout + (scope, packageId).
- RealmJoinWatcherAdapter: maps each watcher event to (1) a new DecisionSignalKind
post + (2) a SharedConstants.EventTypes.RealmJoin* InformationalEvent. Package
DisplayName truncated to 256 chars before posting.
- RealmJoinHost: ICollectorHost with compile-time kill-switch RealmJoinTrackingEnabled.
- DesktopArrivalDetector: new RealUserOwnerObserved event raised before DesktopArrived
so the composition root can resolve the user's SID for the HKU watcher arm.
- DesktopArrivalHost: new onRealUserOwnerObserved callback param.
- DefaultComponentFactory: instantiates RealmJoinHost after AadJoinHost and routes the
DesktopArrival owner through UserSidResolver -> RealmJoinHost.ArmHkuWatcher.
Shared additions:
- SharedConstants.EventTypes adds realmjoin_{detected,phase_changed,resolved,timeout,
package_started,package_completed} so the engine + adapter + future tests all
reference the same strings.
- DecisionEngine.RealmJoinPayloadKeys promoted from internal to public so the
cross-assembly adapter can assemble signal payloads with the canonical key names.
Verified: full solution build clean; V2.Core tests 1124/1125 (the lone failure is the
documented WhiteGlove RecoveryPath flake — re-runs green in isolation, no Phase B link).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a new completion-gate to the V2 enrollment AND-gate so the session keeps
running while a RealmJoin (RJ) deployment is in flight. The agent detects RJ
via HKLM\SYSTEM\CurrentControlSet\Services\realmjoin\Parameters and tracks
per-package installs from HKLM\SOFTWARE\RealmJoin\Packages\<id> + the
HKEY_USERS\<sid>\... counterpart; runtime wiring lands in Phase B.
Decision-layer changes:
- 5 new DecisionSignalKinds (RealmJoinDetected/Resolved/Timeout/PackageStarted/
PackageCompleted) routed through the existing reducer dispatch.
- New DeadlineNames.RealmJoinTimeout (60-min hard cap, hardcoded).
- New immutable RealmJoinFacts + RealmJoinPackageFact on DecisionState; schema
bumped v2 -> v3 (old snapshots load with RealmJoinFacts.Empty default).
- Classic Hello+Desktop AND-gate (HandleHelloResolvedV1, HandleDesktopArrivedV1
Hello-disabled fast-path + helloAlreadyResolved branch, HelloSafety deadline)
and SelfDeploying terminal path now AND with RealmJoinGateOpen(). When closed,
the prerequisite facts are still recorded but TransitionToFinalizing /
WithStage(Completed) is deferred until the RJ resolved/timeout handler
releases the gate via CompleteIfDeferredOrBookkeep.
- Per-package facts capped at 200 rows; DisplayName truncated to 256 chars.
- Audit-trail attaches realmjoinDetectedUtc/Outcome/LastPhase/ResolvedUtc/Packages
on terminal events when RJ was active.
Verified: DecisionCore tests 313/313, Agent V2.Core tests 1125/1125.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 9d052230 dropped enrollment_failed with reason esp_terminal_failure
but the underlying HRESULT 0x87d1041c (Application not detected after install)
sat unparsed in the ESP subcategory statusText, and the agent terminated before
IME could emit the matching app_install_failed. Two-part fix:
- ProvisioningStatusTracker arms a 30s settle window between ESP-subcategory
failure detection and EspFailureDetected fire. Gives ImeLogTracker a chance
to surface the app-level failure on the timeline. Emits esp_failure_settle_started
for observability.
- HRESULT extracted via language-invariant regex from the failed subcategory
statusText, surfaced top-level on esp_provisioning_status (failedSubcategoryErrorCode)
and propagated through new EspFailureDetectedEventArgs to the EspTerminalFailure
signal payload. DecisionEngine.Edge writes failureType/errorCode/failedSubcategory/
category onto the enrollment_failed event parameters. UI renders an HRESULT badge
with description from errorCodeMap.ts (analogous to app_install_failed).
V2-only; V1 (Agent.Core) untouched. ConfigVersion unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reads BlockInStatusPage (bitmask: AllowReset / AllowTryAgain /
AllowContinueAnyway) and SyncFailureTimeout from
HKLM\SOFTWARE\Microsoft\Enrollments\{guid}\FirstSync alongside the
existing skip flags. Two new SignalFacts in EnrollmentScenarioObservations
(EspSyncFailureTimeoutMinutes, EspAllowContinueAnyway) carry the values
into the reducer so the DecisionEngine can emit substantiated
terminal-failure verdicts:
- app_install_failed message now reports the actual ESP timeout
("ESP timed out (60 min) while still installing.") when the
FirstSync value is known; falls back to the generic wording.
- enrollment_failed event payload carries espSyncFailureTimeoutMinutes
+ espAllowContinueAnyway. When ContinueAnyway is true a
mayHaveContinuedAnyway flag + continueAnywayHint string are added
so the timeline shows the user may have dismissed ESP via the
"Continue anyway" button and reached the desktop even though the
agent saw a terminal failure.
- esp_config_detected event DataJson now also exposes the raw
blockInStatusPage value, decoded allowReset / allowTryAgain /
allowContinueAnyway flags, and syncFailureTimeoutMinutes for audit.
Bitmask semantics follow Microsoft's Get-AutopilotDiagnostics.ps1
(bit 1 = AllowReset, bit 2 = AllowTryAgain, bit 4 = ContinueAnyway).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Agent V2: when RecoveryPath loads a snapshot already at Stage=Completed/Failed,
no reducer transition fires so OnDecisionTerminalStage was never invoked — the
agent idled until AgentMaxLifetime (default 6h), emitting periodic perf/metrics
events the whole time. Fire it explicitly at the end of Start() so the
EnrollmentTerminationHandler runs (marker write + diagnostics + self-destruct).
The WhiteGloveSealed archive-and-reset at step 0 already neutralises the
Part-2 case; non-terminal stages take the normal recovery path.
Web: gate the SignalR eventStream refetch and the 30s fallback polling on
non-terminal session status. Once Succeeded/Failed, late perf/metrics arrivals
no longer trigger a full-page refetch + re-render that scroll-jumps the
timeline while the user is reading.
Observed in session 17ef0e0e-... (2026-05-19): 4x enrollment_complete, only 1x
agent_shutting_down; reboot mid-shutdown left subsequent agent instances
running for hours emitting noise.
Tests: 3 new in RecoveryPathTests (Completed/Failed -> Terminated, non-terminal
-> no Terminated), 4 in sessionStatus.test.ts. Suites grun: Web 384/384,
Agent V2.Core 1095/1095, Backend 1844/1844.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- docs/architecture.md Diagnostics section rewritten: both destinations
(CustomerSas default + Hosted opt-in), new download routing on shape,
cascade-delete step §5b with SAS-delete-permission gating, and the V2
remote-config retry + observability events.
- Submodule .claude/commands bumped to 17a6f5b for the dual-destination
fetch-diag-zip.sh refactor + skill error-handling note updates.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Motivating incident — session 8f2bef72 (2026-05-19, OK's VM test): a Function
App cold-start triggered by a fresh deploy stalled the agent's single-shot
GetAgentConfig call for ~60 s; the client gave up, RemoteConfigService
silently fell back to built-in defaults (ConfigVersion=0,
SelfDestructOnComplete=true, DiagnosticsUploadMode=Off), and the entire
enrollment ran the wrong policy with no wire-visible trace of the degraded
mode. ReportAgentError (the emergency channel) was also racing the cold
start and lost to 499.
Three fixes:
1. Retry FetchConfigAsync on transient errors (3 attempts; 10/30/60 s linear
backoff). BackendAuthException still bails immediately — 401/403 won't
change with a retry and an operator needs the AuthCertificateRejected /
DeviceNotRegistered distress signal fast. EmergencyReporter only fires on
the final attempt to give the function time to warm. Live ServerAction
rotate_config keeps single-shot so it doesn't block the action handler
for ~100 s.
2. agent_started DataJson now carries configVersion + remoteConfigFetched +
remoteConfigOutcome (Succeeded / FromCache / UsedDefaults / NotAttempted).
Operators can tell at a glance whether the agent is running on fresh
tenant config vs. defaults vs. stale cache without inferring from
absence of expected tenant-controlled knobs.
3. New remote_config_fetch_failed Warning event emitted right after
agent_started when the fallback path was taken. Carries attempts,
failureType, failureMessage (truncated 512), and authStatusCode when
relevant. Closes the blind spot — the only prior trace was a Warning
in the local agent log which is gone after a SelfDestruct cleanup.
V2 Agent suite: 1092/1092 (+16: 7 retry behaviour + 9 emitter shape).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The red color treatment + warning-triangle icon on the Hosted-storage
option implied misconfiguration. Hosted is a deliberately good choice
(blob-scoped per-upload SAS, isolated per-tenant prefix, automated
retention via cascade-delete) — the only trade-off is data leaving the
customer's Azure tenant. Swap red for sky-blue + info-circle icon, drop
the "data leaves your tenant" red badge in favour of "managed for you",
and rephrase the disclosure as a friendly heads-up instead of a warning.
The substance is unchanged; the visual cue just no longer reads as a
fault state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an opt-in second destination for agent diagnostics uploads:
- CustomerSas (default, unchanged): tenant's own SAS-backed container
- Hosted (new, opt-in via explicit admin click): backend's storage account
under {tenantId}/AgentDiagnostics-...zip, with per-upload 15-min
blob-scoped Write+Create SAS minted via the Function App's MI
Closes the long-standing gap where Session.DiagnosticsBlobName was captured
into the deletion-manifest but the blob was never actually cleaned up by
the cascade. Hosted blobs are always deleted (we own the storage); customer
blobs are deleted only when the SAS includes the 'd' permission, otherwise
logged + skipped.
Existing tenants change nothing: the new TenantConfiguration.DiagnosticsUploadDestination
field defaults to CustomerSas, no silent flip. Hosted requires an explicit
admin click + Save with a clearly marked "data leaves your tenant" disclosure.
Tests: Backend 1844, V2 1076, V1 538, vitest 380 (+78 new across all suites).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Persist tenant Preview-Notification-Email on OffboardingHistory row at Phase 1 so it
survives Phase 2 wipe; ResendEmailService.SendAsync wired but disarmed by a const-false
arm-flag with a [DRAFT] placeholder template — explicit flip required to ship.
- Move user feedback storage out of PreviewConfig discriminator into a dedicated Feedback
table (PK=InApp / Offboarding) that is NOT in any tenant-offboarding wipe list — feedback
from departing tenants is exactly the signal we want to keep. New IFeedbackRepository +
TableFeedbackRepository; FeedbackFunction migrated; (PreviewConfig, "Feedback")
removed from TenantOffboardingHandler.DiscriminatorTables.
- POST /api/tenants/{tenantId}/offboard/feedback collects a free-form farewell comment
during the drain-barrier countdown (marker-gated, fail-soft OpsEvent emit). Reports >
User Feedback page gains an Offboarding-Feedback sub-tab next to In-App Feedback.
- OffboardingFeedbackReceived OpsEventType registered (info-severity) + catalog entry
in OpsAlertRulesSection so admins can wire Telegram in two clicks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`tenantNavSections.ts` (settings-page routing) was wired but the sidebar reads
from `globalNavConfig.tsx`, which is a separate source of truth for the left-rail
nav. Adding the matching entry between Bootstrap Sessions and Submit Logs so the
section becomes reachable without typing the URL manually.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
So we can answer "which tenants actively have this on" and "how many people are
using it" later — without bolting on a separate metrics store.
Backend events
- `GraphAddOnPermissionsDetected` (GraphFeatureDetector, fresh-acquire path):
fires at most every ~55min per tenant (cache TTL). Props: TenantId,
GrantedRoleCount, HasScriptDisplayNames, TokenCacheTtlMinutes. → distinct
count over time = tenants who have the optional permission turned on.
- `ScriptDisplayNamesResolved` (ScriptDisplayNameResolver, only when permission
is active and the resolver actually did work): per session-view with scripts.
Props: TenantId, RefsRequested, RefsResolved, CacheHits, TriggeredFullPull.
→ real usage pulse.
- `GraphAddOnStatusChecked` (GET status endpoint): admin engagement with the
capability page. Props: TenantId, UserId, IsTransient, ScriptDisplayNamesGranted.
- `GraphAddOnRefreshTriggered` (POST refresh endpoint): high-signal "tenant just
granted" event. Props: TenantId, UserId.
Web events (via `trackEvent` -- tenantId auto-attached by the telemetry initializer)
- `GraphAddOnStatusViewed` on Settings → Optional Graph capabilities mount.
- `GraphAddOnGrantCommandCopied` on Copy command / Copy permissions click.
Notes
- All emits wrapped in try/catch with debug-level logging — telemetry failures
must never poison the acquire / resolve hot paths.
- Tests pass a no-op `TelemetryClient(new TelemetryConfiguration())` (same
pattern as SlaTenantStatusLifecycleTests). 1718 backend / 364 web tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds opt-in resolution of Intune Platform Script + Remediation display names in
session timelines via a tenant-side appRoleAssignment grant. No app manifest
change, no impact on customers who don't enable it. Architecture is reusable for
future optional Graph permissions: catalog -> detector -> resolver -> endpoint.
Backend
- GraphFeatureDetector parses the `roles` claim from the SP access token and
caches the (token, roles) bundle per tenant. GetSnapshotAsync surfaces a
transient flag so admin-UI callers can render "unknown" instead of a
misleading "0 of 0 granted" when token acquire times out.
- GraphTokenService now accepts a CancellationToken so optional-feature callers
can apply tight budgets instead of inheriting the 5+15+30 s retry chain.
- ScriptDisplayNameResolver: cache-first, list-pull-on-cold-tenant, per-ID
fallback. Throttle-aware abort: skip per-ID for kinds whose full-pull hit a
transient failure; break the per-ID loop on the first HttpRequestException.
Total budget 5 s, with a tight 1.5 s sub-budget on the token acquire.
- Azure Table `ScriptNameCache` + direct-TableClient fail-loud repository
(mirrors TableOffboardingAuditRepository pattern). Daily
ScriptNameCacheCleanupFunction: 7 d positive / 1 h NotFound TTL.
- POST /api/tenants/{id}/scripts/display-names (body-carried refs to dodge URL
length limits; backend cap 200 refs/request).
- GET /api/tenants/{id}/graph-permissions/status (4 s budget; includes app
ClientId so the UI can render an exact grant command).
- POST /api/tenants/{id}/graph-permissions/refresh (invalidates token cache).
- All three routes registered in EndpointAccessPolicyCatalog (MemberRead for the
resolve, TenantAdminOrGA for the admin-UI surfaces).
Web
- `useScriptDisplayNames` hook with stable refset fingerprint (no spurious
refetches on parent re-renders); clears stale map on tenant/refset change so
a session switch never leaks names across tenants.
- Chunked POST loop: 200-ref batches with partial-success semantics. Preserves
earlier chunks when later ones fail (non-2xx OR transport throw); only
TokenExpiredError propagates so auth flow can react.
- ScriptExecutions renders " . {DisplayName}" alongside existing label/short-id,
keyed by Kind + policyId so Platform/Remediation under the same id never
collide.
- New Settings section "Optional Graph capabilities": status table with granted
/ not granted / unknown, copy-paste PS one-liner with ClientId baked in,
refresh button gated on response.ok.
Customer-facing
- scripts/CustomerSetup/Grant-AutopilotMonitorAddOn.ps1: idempotent
grant / verify / revoke script. -Features ScriptDisplayNames (high level) or
-Permissions ... (admin-UI generated). Pure ASCII for PS 5.1 compat.
- docs/customer-graph-add-on-permissions.md: end-to-end customer guide
covering prerequisites, the appRoleAssignment pattern, and revocation.
Tests
- 1718 backend xUnit / 364 web vitest. Highlights: JWT roles parsing edge
cases; full-pull-429 suppresses per-ID; per-ID-429 breaks loop; transient
detection via GetSnapshotAsync; chunked partial success (HTTP failure + thrown
transport); TokenExpiredError propagation; EndpointPolicyCatalogCompleteness
covers the three new routes by reflection.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Session 330f73f3 completed via DeadlineFired:finalizing_grace (helloOutcome=
Timeout) while ESP AccountSetup was still at "4 of 5 subcategories completed"
and 3 of 8 apps still pending. ShouldTransitionToAwaitingHello promoted on
AccountSetupEnteredUtc != null alone, but Windows CXH event 62407 fires at
every ESP-page transition — the first occurrence (Device→Account handoff,
9 s after AccountSetup entered) was treated as the genuine final exit,
armed HelloSafety against an in-progress AccountSetup, fired its 5-min
synthetic timeout, and FinalizingGrace declared the session terminal
mid-flight.
Introduce a strong "AccountSetup truly succeeded" fact sourced from
ProvisioningStatusTracker (categorySucceeded=true OR fallback after 30 s
of all subcategories at succeeded/notRequired — same shape as DeviceSetup).
Forwarded by EspAndHelloTracker, posted as new DecisionSignalKind.
AccountSetupProvisioningComplete by the V2 adapter, lands as DecisionState.
AccountSetupProvisioningSucceededUtc. The reducer's
ShouldTransitionToAwaitingHello now requires this fact (or SkipUserEsp);
the old AccountSetupEnteredUtc gate becomes an entry-time marker only.
HandleAccountSetupProvisioningCompleteV1 has a deferred-promotion path:
when the strong fact arrives AFTER a guard-blocked EspExiting
(EspFinalExitUtc set) OR after a guard-blocked EspPhaseChanged(Finalizing-
Setup) (FinalizingEnteredUtc set, ShellCoreTracker adapter fire-once-
dedupes), the handler performs the deferred promotion + arms HelloSafety
anchored at the AccountSetup-succeeded instant.
The new fact is registered in DecisionStateSignalCensus so enrollment_*
audit trails and final-status.json show whether the strong gate was
satisfied at termination — load-bearing for diagnosing future
false-negative timeouts under the new gate.
Side cleanup: ProvisioningStatusTracker's fallback timer callbacks (new
AccountSetup AND pre-existing DeviceSetup parallel path) now reset the
timer field on every abort path AND in the outer exception catch. Without
this, a transient subcategory flicker between fallback arm and timer fire
could permanently prevent re-arming even when the state re-stabilises.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `sidebarState` ("full" | "icons" | "hidden") as a global telemetry
property on every App Insights event/pageView, analogous to `theme`.
Pre-seeded from localStorage at init and kept in sync via the
useSidebarState hook on every state change.
Fires a `sidebar_state_changed` customEvent with `{from, to}` only on
user-initiated cycle clicks (guarded by previous != next, so SSR
hydration and idempotent setter calls do not produce noise).
Together these answer "how many users live in the collapsed sidebar?"
(persistent preference per Auth-User via global prop) and "how often do
people toggle it?" (active interaction via customEvent). Bumps the
.claude/commands submodule for the matching ux-insights skill catalog
update.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls in submodule commit e27a9d3 which adds queries for the new AgentCertRejected
+ AgentRequestRejected backend warnings and the V2 distress cert-context fields.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend emits AgentCertRejected (with reason enum) at every cert-validation
failure path and AgentRequestRejected (with request-context) at every 401/403
return so operators can diagnose why mTLS auth was denied. V2 agent now
attaches a Thumbprint/Subject/Issuer/NotBefore/NotAfter snapshot plus a
DistressCertSourceState classifier to each distress report; correlates 1:1
with the backend warnings via tenant + serial + thumbprint.
Distress endpoint stays unauthenticated and security-restrictive: rate-limit
(3-layer) + sanitize + truncate + enum-validation unchanged; new fields are
hex-regex / IsDefined / hard-capped at 96 chars; MaxContentLength bumped from
1024 to 1536 to fit the cert-context payload. V1 agent untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a "Only tenants with restore data" checkbox to the Restore Browser
tenant dropdown so a Global Admin can hide tenants that have nothing to
recover. Default off (full AdminConfig list); flipping it on fetches a
cheap hierarchy listing once (cached for page lifetime) and intersects
the dropdown with the tenants that own ≥1 snapshot blob.
Backend: new GlobalAdminOnly endpoint
GET /api/global/tenants-with-deletion-manifests backed by
BlobStorageService.ListTenantsWithDeletionManifestsAsync, which uses
GetBlobsByHierarchyAsync with delimiter="/" so Azure returns just the
top-level prefixes (= tenant IDs) without enumerating blobs underneath.
Cost scales with tenant count, not manifest count.
Web: checkbox under the tenant dropdown shows a "(X of Y)" counter when
active, dropdown placeholder reflects the loading state, and the
currently-selected tenant clears if the filter excludes it. Fetch
failures snap the checkbox back to off so the operator never gets
stranded with an empty list.
Build: 0 warnings/errors. Backend 1526/1526 pass. tsc clean.
Vitest 346/346 pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cascade-delete V2 is the only path now. Drops the per-tenant
EnableCascadeDeleteV2 flag, the legacy ETag-CAS-guarded DeleteSessionAsync
helper, the AppInstallSummaries cleanup helper (only legacy caller),
and the SessionsLegacyDeleted counter chain (fanout → maintenance → OpsEvent).
DeleteSessionFunction collapses to a single kill-switch gate; the producer
owns 404 / 409 / Poisoned mapping AND the intentional Queued/Preparing+Snapshot
→ 202 recovery resume — gating those at HTTP would strand crashed admin
cascades. Fanout now aborts both the inner loop AND outer loop on producer
KillSwitchActive instead of leaking calls into the next tenant.
Test coverage:
- DeleteSessionCasGuardTests removed (helper gone)
- AdminConfigurationSessionDeletionKillSwitchTests trimmed to kill-switch only
- DeleteSessionFunctionTests pinned to the kill-switch short-circuit + null
- SessionRetentionFanoutServiceTests: legacy path tests dropped, two new
abort regressions for producer KillSwitchActive (inner + outer loop)
Build: 0 warnings, 0 errors. Backend: 1526 passed / 0 failed.
Web: tsc clean, vitest 346 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Oliver Kieselbach