fix(backend): honor Hello-disabled + SkipUserEsp in the user-completed-setup reconcile rule

Follow-up to 049b6e5b: tenants running with Windows Hello disabled AND
the User ESP skipped can never produce a Hello terminal event, so the
reconcile rule would never fire for them and silent post-desktop
sessions stayed mislabeled AwaitingUser -> Incomplete. Mirror the
agent's Hello-disabled fast-path (HandleDesktopArrivedV1: policy
disabled + SkipUserEsp + desktop => complete):

- ExtractRollup reads hello_policy_detected Data.helloEnabled and
  esp_config_detected Data.skipUserStatusPage (contradicting
  observations resolve pessimistically to enabled/required).
- Rule 4 accepts desktop + (Hello terminal OR policy-disabled +
  SkipUserEsp). Hello disabled WITHOUT SkipUserEsp keeps waiting —
  the agent's strong post-AccountSetup gate (session 08c99638) blocks
  its fast-path there too, and rule 2 covers the rollup-complete case.
- esp_exiting stays rejected as success evidence (page-lifecycle event,
  fires with errorCode=0 while apps are unfinished — lesson 08c99638).
- Snapshot v3 + Web block additionally carry skipUserEsp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-07-10 10:51:57 +02:00
parent 049b6e5b4a
commit 170e66133d
6 changed files with 159 additions and 11 deletions
@@ -101,8 +101,9 @@ Given a non-terminal session past the inactivity window:
2. AccountSetup rollup = ALL subcategories succeeded
OR enrollment_complete / whiteglove_complete present → Succeeded (reconcile)
3. agent_emergency_break present (agent provably gone) → Incomplete (skip grace)
4. desktop_arrived AND positive Hello terminal
(hello_provisioning_completed / hello_skipped) present → Succeeded (reconcile)
4. desktop_arrived AND ( positive Hello terminal
(hello_provisioning_completed / hello_skipped) present
OR hello policy disabled AND User ESP skipped ) → Succeeded (reconcile)
5. DeviceSetup rollup = ALL subcategories succeeded (or 30s
fallback) AND no failure event:
a. within grace (now - StartedAt < SessionGraceHours) → AwaitingUser (non-terminal)
@@ -119,8 +120,18 @@ the completion report never left the device (shutdown / process kill / egress cu
Labeling it `AwaitingUser` is factually wrong — the user was provably there. The reason
string distinguishes "RealmJoin deployment never reported completion" (RJ detected but
unresolved) from the generic "agent went silent before reporting completion". The failure
snapshot (schema v3) carries the evidence: `helloResolved`, `realmJoinDetected`,
`realmJoinResolved`.
snapshot (schema v3) carries the evidence: `helloResolved`, `skipUserEsp`,
`realmJoinDetected`, `realmJoinResolved`.
The Hello‑disabled branch mirrors the agent's Hello‑disabled fast‑path exactly: with
`hello_policy_detected(helloEnabled=false)` **and** `esp_config_detected(skipUserStatusPage=true)`
no Hello terminal can ever exist, and the agent completes on desktop arrival in that
configuration. Hello disabled **without** SkipUserEsp does NOT qualify — the agent's strong
post‑AccountSetup gate (session `08c99638`) requires the AccountSetup rollup there, which
step 2 already covers. Note `esp_exiting` (Shell‑Core 62407) is deliberately NOT used as
success evidence anywhere in this table: it is a page‑lifecycle event that fires with
`errorCode=0` even while AccountSetup apps are unfinished (the very lesson of `08c99638`),
unlike the Hello events, which are outcome terminals.
Rationale for each bucket sizing is in the Evidence table above.
⚠️ Gate step 2/3 on the **subcategory rollup**, not `categorySucceeded`.
@@ -17,14 +17,22 @@ public class EnrollmentTimeoutClassifierTests
{
private static readonly DateTime Start = new(2026, 7, 6, 15, 0, 0, DateTimeKind.Utc);
private static EnrollmentEvent Evt(string type, string? message = null) => new()
private static EnrollmentEvent Evt(string type, string? message = null,
Dictionary<string, object>? data = null) => new()
{
EventType = type,
Timestamp = Start,
Source = "test",
Message = message ?? type,
Data = data!,
};
private static EnrollmentEvent HelloPolicy(bool enabled) =>
Evt("hello_policy_detected", data: new Dictionary<string, object> { ["helloEnabled"] = enabled.ToString() });
private static EnrollmentEvent EspConfig(bool skipUser) =>
Evt("esp_config_detected", data: new Dictionary<string, object> { ["skipUserStatusPage"] = skipUser });
private static EnrollmentEvent Esp(string message) => Evt("esp_provisioning_status", message);
private const string DeviceSetup44 = "ESP provisioning status: DeviceSetup — 4 of 4 subcategories completed";
@@ -110,6 +118,26 @@ public class EnrollmentTimeoutClassifierTests
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("realmjoin_timeout") }).RealmJoinResolved);
}
[Fact]
public void ExtractRollup_reads_hello_policy_and_skip_user_esp_from_event_data()
{
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { HelloPolicy(enabled: false) }).HelloPolicyDisabled);
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(new[] { HelloPolicy(enabled: true) }).HelloPolicyDisabled);
// Contradicting observations resolve pessimistically → treat as enabled (keep demanding
// the Hello terminal).
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(
new[] { HelloPolicy(enabled: false), HelloPolicy(enabled: true) }).HelloPolicyDisabled);
Assert.True(EnrollmentTimeoutClassifier.ExtractRollup(new[] { EspConfig(skipUser: true) }).SkipUserEsp);
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(new[] { EspConfig(skipUser: false) }).SkipUserEsp);
Assert.False(EnrollmentTimeoutClassifier.ExtractRollup(
new[] { EspConfig(skipUser: true), EspConfig(skipUser: false) }).SkipUserEsp);
// No policy events at all → both false (pessimistic default).
var bare = EnrollmentTimeoutClassifier.ExtractRollup(new[] { Evt("agent_started") });
Assert.False(bare.HelloPolicyDisabled);
Assert.False(bare.SkipUserEsp);
}
[Fact]
public void Classify_emergency_break_skips_grace_and_is_Incomplete()
{
@@ -201,6 +229,49 @@ public class EnrollmentTimeoutClassifierTests
Assert.Equal(SessionStatus.Succeeded, status);
}
[Fact]
public void Classify_hello_disabled_plus_skip_user_esp_plus_desktop_reconciles_to_Succeeded()
{
// Mirror of the agent's Hello-disabled fast-path: HelloPolicyEnabled==false +
// SkipUserEsp==true + desktop arrival completes on the device, so a silent session
// with the same evidence reconciles to Succeeded — no Hello terminal can ever exist
// in this configuration.
var (status, reason) = Classify(new[]
{
Esp(DeviceSetup44), Evt("desktop_arrived"),
HelloPolicy(enabled: false), EspConfig(skipUser: true),
}, hoursSinceStart: 6);
Assert.Equal(SessionStatus.Succeeded, status);
Assert.Contains("User ESP skipped, Windows Hello disabled", reason);
}
[Fact]
public void Classify_hello_disabled_without_skip_user_esp_stays_AwaitingUser()
{
// Hello disabled but User ESP required: the agent's strong post-AccountSetup gate
// (session 08c99638) blocks its fast-path too — completion needs the AccountSetup
// rollup (rule 2) there, so the backend must keep waiting as well.
var (status, _) = Classify(new[]
{
Esp(DeviceSetup44), Esp(AccountSetup15), Evt("desktop_arrived"),
HelloPolicy(enabled: false), EspConfig(skipUser: false),
}, hoursSinceStart: 6);
Assert.Equal(SessionStatus.AwaitingUser, status);
}
[Fact]
public void Classify_skip_user_esp_with_hello_enabled_still_requires_hello_terminal()
{
// SkipUserEsp only waives the AccountSetup evidence, never the Hello wizard — with
// Hello enabled the user still has to finish/skip the wizard on the device.
var (status, _) = Classify(new[]
{
Esp(DeviceSetup44), Evt("desktop_arrived"),
HelloPolicy(enabled: true), EspConfig(skipUser: true),
}, hoursSinceStart: 6);
Assert.Equal(SessionStatus.AwaitingUser, status);
}
[Fact]
public void Classify_desktop_without_hello_terminal_stays_AwaitingUser()
{
@@ -328,6 +328,20 @@ public class FailureSnapshotBuilderTests
Assert.True((bool)obj["helloResolved"]!);
Assert.True((bool)obj["realmJoinDetected"]!);
Assert.False((bool)obj["realmJoinResolved"]!);
Assert.False((bool)obj["skipUserEsp"]!);
}
[Fact]
public void Build_records_skip_user_esp_from_esp_config_event_data()
{
var json = FailureSnapshotBuilder.Build(new[]
{
Event("esp_config_detected", Now.AddHours(-5),
data: new Dictionary<string, object> { ["skipUserStatusPage"] = true }),
}, Now);
var obj = JObject.Parse(json!);
Assert.True((bool)obj["skipUserEsp"]!);
}
// ============================================================================
@@ -94,6 +94,8 @@ namespace AutopilotMonitor.Functions.Services
bool HasAgentEmergencyBreak,
bool DesktopArrived,
bool HelloResolved,
bool HelloPolicyDisabled,
bool SkipUserEsp,
bool RealmJoinDetected,
bool RealmJoinResolved);
@@ -113,6 +115,8 @@ namespace AutopilotMonitor.Functions.Services
bool hasFailure = false, hasComplete = false, hasEmergencyBreak = false;
bool desktopArrived = false, helloResolved = false;
bool realmJoinDetected = false, realmJoinResolved = false;
bool sawHelloPolicyEnabled = false, sawHelloPolicyDisabled = false;
bool sawSkipUserTrue = false, sawSkipUserFalse = false;
foreach (var evt in events)
{
@@ -137,6 +141,27 @@ namespace AutopilotMonitor.Functions.Services
// hard timeout (both dual-emitted to the timeline by the agent/engine).
else if (Eq(type, "realmjoin_resolved") || Eq(type, "realmjoin_timeout"))
realmJoinResolved = true;
// Policy facts for the Hello-disabled / User-ESP-skipped completion mirror.
// Contradicting observations are resolved pessimistically below (both-seen →
// treat as enabled/required, i.e. keep demanding the Hello terminal).
else if (Eq(type, "hello_policy_detected"))
{
var raw = TryGetDataString(evt, "helloEnabled");
if (bool.TryParse(raw, out var enabled))
{
if (enabled) sawHelloPolicyEnabled = true;
else sawHelloPolicyDisabled = true;
}
}
else if (Eq(type, "esp_config_detected"))
{
var raw = TryGetDataString(evt, "skipUserStatusPage");
if (bool.TryParse(raw, out var skip))
{
if (skip) sawSkipUserTrue = true;
else sawSkipUserFalse = true;
}
}
var msg = evt.Message;
if (string.IsNullOrEmpty(msg)) continue;
@@ -178,6 +203,8 @@ namespace AutopilotMonitor.Functions.Services
HasAgentEmergencyBreak: hasEmergencyBreak,
DesktopArrived: desktopArrived,
HelloResolved: helloResolved,
HelloPolicyDisabled: sawHelloPolicyDisabled && !sawHelloPolicyEnabled,
SkipUserEsp: sawSkipUserTrue && !sawSkipUserFalse,
RealmJoinDetected: realmJoinDetected,
RealmJoinResolved: realmJoinResolved);
}
@@ -224,13 +251,26 @@ namespace AutopilotMonitor.Functions.Services
// remains explicitly rejected as a completion signal (design doc) — it fires
// while the user phase is still running; the Hello terminal is what proves the
// user finished.
if (rollup.DesktopArrived && rollup.HelloResolved)
//
// Hello-disabled mirror: when the tenant runs with Windows Hello disabled AND
// the User ESP skipped, no Hello terminal can ever exist — the agent's own
// Hello-disabled fast-path (HandleDesktopArrivedV1) completes on desktop arrival
// in exactly that configuration (HelloPolicyEnabled==false + SkipUserEsp==true).
// Hello disabled WITHOUT SkipUserEsp intentionally does NOT qualify: there the
// agent's strong post-AccountSetup gate (session 08c99638) requires the
// AccountSetup rollup, which rule 2 already covers when it reached all-succeeded.
var helloSatisfied = rollup.HelloResolved
|| (rollup.HelloPolicyDisabled && rollup.SkipUserEsp);
if (rollup.DesktopArrived && helloSatisfied)
{
var evidence = rollup.HelloResolved
? "desktop + Windows Hello"
: "desktop; User ESP skipped, Windows Hello disabled";
var detail = rollup.RealmJoinDetected && !rollup.RealmJoinResolved
? "RealmJoin deployment never reported completion before the agent went silent"
: "agent went silent before reporting completion";
return (SessionStatus.Succeeded,
$"Reconciled at timeout: user completed setup (desktop + Windows Hello) — {detail}");
$"Reconciled at timeout: user completed setup ({evidence}) — {detail}");
}
// 5. Device Setup fully provisioned (device is AADJ + MDM-enrolled), user phase pending.
@@ -258,5 +298,12 @@ namespace AutopilotMonitor.Functions.Services
private static bool Eq(string a, string b) =>
string.Equals(a, b, StringComparison.OrdinalIgnoreCase);
private static string? TryGetDataString(EnrollmentEvent evt, string key)
{
if (evt?.Data == null) return null;
if (!evt.Data.TryGetValue(key, out var raw) || raw == null) return null;
return raw.ToString();
}
}
}
@@ -27,10 +27,10 @@ namespace AutopilotMonitor.Functions.Services
// v2 (2026-07-08): added the ESP DeviceSetup/AccountSetup subcategory rollup fields
// (deviceSetupAllSucceeded / accountSetup*) that the timeout reclassification uses —
// see docs/design/enrollment-status-reclassification.md.
// v3 (2026-07-10): added helloResolved + realmJoinDetected/realmJoinResolved — the
// evidence behind the "user completed setup" reconcile rule (session 294ab5b4: user
// reached desktop + finished Hello, only the RealmJoin gate was pending when the
// agent went silent; previously mislabeled AwaitingUser).
// v3 (2026-07-10): added helloResolved + realmJoinDetected/realmJoinResolved +
// skipUserEsp — the evidence behind the "user completed setup" reconcile rule
// (session 294ab5b4: user reached desktop + finished Hello, only the RealmJoin gate
// was pending when the agent went silent; previously mislabeled AwaitingUser).
public const int CurrentSchemaVersion = 3;
// Canonical signals we expect to have seen by the time a healthy session completes.
@@ -241,6 +241,7 @@ namespace AutopilotMonitor.Functions.Services
accountSetupTotal = espRollup.AccountSetupTotal,
accountSetupAllSucceeded = espRollup.AccountSetupAllSucceeded,
helloResolved = espRollup.HelloResolved,
skipUserEsp = espRollup.SkipUserEsp,
realmJoinDetected = espRollup.RealmJoinDetected,
realmJoinResolved = espRollup.RealmJoinResolved,
missingSignals = missing,
@@ -29,6 +29,7 @@ interface FailureSnapshot {
lastNetworkState?: string | null;
// Schema v3 (2026-07-10): evidence behind the "user completed setup" reconcile rule.
helloResolved?: boolean;
skipUserEsp?: boolean;
realmJoinDetected?: boolean;
realmJoinResolved?: boolean;
missingSignals?: string[];
@@ -87,6 +88,9 @@ export default function FailureSnapshotBlock({
{snapshot.helloResolved !== undefined && (
<SnapshotItem label="Hello completed" value={fmtBool(snapshot.helloResolved)} />
)}
{snapshot.skipUserEsp !== undefined && (
<SnapshotItem label="User ESP" value={snapshot.skipUserEsp ? "skipped (profile)" : "required"} />
)}
{snapshot.realmJoinDetected !== undefined && (
<SnapshotItem label="RealmJoin" value={fmtRealmJoin(snapshot)} />
)}