refactor(decisioncore): edge scenarios — hybrid reboot, ESP failure, recovery (M3.5)

Final M3 sub-milestone. Closes the plan's §4 M3 scenario matrix: Handlers (DecisionEngine.Edge.cs): - EspResumed mid-ESP reboot. Stage preserved; records SystemRebootUtc on first observation (hash stable on repeats). - EspTerminalFailure Stage=Failed + Outcome=EnrollmentFailed, deadlines cleared, enrollment_failed event emitted with reason carried from payload. Definitive — no classifier re-run. - SystemRebootObserved records SystemRebootUtc so the WhiteGlove sealing classifier credits the +15 reboot weight on the next RunClassifier effect. Scenario fixtures (committed-anonymized): - hybrid-reboot-v1.jsonl ESP -> mid-phase reboot -> EspResumed -> AccountSetup -> Hello -> Desktop -> Completed. No WhiteGloveSealed path. - esp-terminal-failure-v1.jsonl ESP -> AccountSetup -> terminal failure -> Failed / reason= esp_terminal_failure_account_setup. Tests (EdgeScenarioTests, 5 new): - Hybrid reboot preserves stage, records reboot fact, completes normally without triggering WhiteGlove. - ESP terminal failure transitions straight to Failed. - Restart-recovery determinism: replaying the same JSONL twice yields identical FinalStepHash, Stage, Outcome and transition count. - Prefix-replay check: truncated log ends in the expected intermediate stage (simulates crash between signals). - Global sanity: every committed fixture (all 12) ends in a terminal stage — regression guard for any future reducer change. Suite totals: 57 DecisionCore (was 52) / 507 Legacy / 557 Functions = 1121 tests green. All 14+ scenarios from plan §R6 Szenario-Matrix pass through the reducer with the expected terminal outcome. This closes M3 (DecisionEngine-Reducer + Classifier-Kernel). M4 wires the SignalAdapter + EffectRunner + TelemetryTransport in the V2 agent runtime. Plan: plans/REFACTOR_AGENT_V2.md §4 M3 sub-milestone M3.5 — M3 complete. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-10-24 04:32:32 +00:00 · 2026-04-20 14:26:13 +02:00
parent 8e4faf4251
commit 75c98d131f
7 changed files with 279 additions and 3 deletions
@@ -1 +1 @@
-12
+13
@@ -1 +1 @@
-1054
+1055
@@ -0,0 +1,133 @@
+using System.Collections.Generic;
+using AutopilotMonitor.DecisionCore.Classifiers;
+using AutopilotMonitor.DecisionCore.Engine;
+using AutopilotMonitor.DecisionCore.State;
+using AutopilotMonitor.DecisionCore.Tests.Harness;
+using AutopilotMonitor.Shared.Models;
+using Xunit;
+
+namespace AutopilotMonitor.DecisionCore.Tests.Scenarios
+{
+    /// <summary>
+    /// Plan §4 M3.5 — Edge scenarios: hybrid reboot, ESP terminal failure, restart-recovery
+    /// determinism.
+    /// </summary>
+    public sealed class EdgeScenarioTests : ScenarioTestBase
+    {
+        private static ClassifierAwareReplayHarness NewHarness() =>
+            new ClassifierAwareReplayHarness(
+                new DecisionEngine(),
+                new Dictionary<string, IClassifier>
+                {
+                    [WhiteGloveSealingClassifier.ClassifierId] = new WhiteGloveSealingClassifier(),
+                    [WhiteGlovePart2CompletionClassifier.ClassifierId] = new WhiteGlovePart2CompletionClassifier(),
+                });
+
+        [Fact]
+        public void HybridReboot_midEsp_preservesStage_recordsRebootFact_completesNormally()
+        {
+            var signals = LoadFixture("hybrid-reboot-v1.jsonl");
+            var result = NewHarness().Replay("session-anon-0030", "tenant-anon-0030", signals);
+
+            Assert.Equal(SessionStage.Completed, result.FinalState.Stage);
+            Assert.Equal(SessionOutcome.EnrollmentComplete, result.FinalState.Outcome);
+            Assert.Equal("Success", result.FinalState.HelloOutcome!.Value);
+
+            // Reboot fact is recorded.
+            Assert.NotNull(result.FinalState.SystemRebootUtc);
+
+            // Stage flow: EspDeviceSetup -> (Reboot) preserved -> EspResumed preserved ->
+            // EspAccountSetup on next EspPhaseChanged -> AwaitingHello -> Completed.
+            // Importantly, no WhiteGloveSealed — reboot alone doesn't imply WG.
+            Assert.NotEqual(HypothesisLevel.Confirmed, result.FinalState.WhiteGloveSealing.Level);
+            Assert.Null(result.FinalState.ShellCoreWhiteGloveSuccessSeen);
+        }
+
+        [Fact]
+        public void EspTerminalFailure_transitionsToFailed_withReasonFromPayload()
+        {
+            var signals = LoadFixture("esp-terminal-failure-v1.jsonl");
+            var result = NewHarness().Replay("session-anon-0031", "tenant-anon-0031", signals);
+
+            Assert.Equal(SessionStage.Failed, result.FinalState.Stage);
+            Assert.Equal(SessionOutcome.EnrollmentFailed, result.FinalState.Outcome);
+            Assert.Empty(result.FinalState.Deadlines);
+
+            // The last transition carries the payload-derived reason via trigger.
+            var last = result.Transitions[^1];
+            Assert.Equal("EspTerminalFailure", last.Trigger);
+            Assert.True(last.Taken);
+        }
+
+        [Fact]
+        public void RestartRecovery_replayTwice_producesIdenticalFinalStepHash()
+        {
+            // Determinism: the same signal stream through a fresh engine + fresh state
+            // yields the same FinalStepHash. Simulates the Orchestrator replaying a
+            // persisted SignalLog after an agent crash. Plan L.2 Event-Sourcing.
+            var signals = LoadFixture("whiteglove-part2-happy-v1.jsonl");
+
+            var r1 = NewHarness().Replay("session-anon-replay", "tenant-anon-replay", signals);
+            var r2 = NewHarness().Replay("session-anon-replay", "tenant-anon-replay", signals);
+
+            Assert.Equal(r1.FinalStepHash, r2.FinalStepHash);
+            Assert.Equal(r1.FinalState.Stage, r2.FinalState.Stage);
+            Assert.Equal(r1.FinalState.Outcome, r2.FinalState.Outcome);
+            Assert.Equal(r1.Transitions.Count, r2.Transitions.Count);
+        }
+
+        [Fact]
+        public void RestartRecovery_prefixReplay_yieldsIntermediateState()
+        {
+            // Replay a truncated log and verify the state matches what full-replay had at
+            // the same point (simulates partial recovery after crash between signals).
+            var all = LoadFixture("userdriven-happy-v1.jsonl");
+            var prefix = new List<AutopilotMonitor.DecisionCore.Signals.DecisionSignal>();
+            for (int i = 0; i < 3 && i < all.Count; i++) prefix.Add(all[i]);
+
+            var prefixResult = NewHarness().Replay("s", "t", prefix);
+
+            Assert.Equal(SessionStage.EspAccountSetup, prefixResult.FinalState.Stage);
+            Assert.Null(prefixResult.FinalState.Outcome);
+            Assert.NotNull(prefixResult.FinalState.CurrentEnrollmentPhase);
+            Assert.Equal(EnrollmentPhase.AccountSetup, prefixResult.FinalState.CurrentEnrollmentPhase!.Value);
+        }
+
+        [Fact]
+        public void AllSixCommittedFixtures_produceTerminalState()
+        {
+            // Global sanity: every committed anonymized fixture ends in a terminal stage.
+            // This catches regressions where a new reducer change breaks a scenario end.
+            string[] fixtures =
+            {
+                "userdriven-happy-v1.jsonl",
+                "userdriven-hello-timeout-v1.jsonl",
+                "late-aadj-v1.jsonl",
+                "selfdeploying-happy-v1.jsonl",
+                "selfdeploying-esp-exit-unknown-v1.jsonl",
+                "whiteglove-inline-v1.jsonl",
+                "whiteglove-signal-correlated-v1.jsonl",
+                "whiteglove-false-positive-v1.jsonl",
+                "whiteglove-part2-happy-v1.jsonl",
+                "whiteglove-part2-stuck-v1.jsonl",
+                "hybrid-reboot-v1.jsonl",
+                "esp-terminal-failure-v1.jsonl",
+            };
+
+            foreach (var fx in fixtures)
+            {
+                var signals = LoadFixture(fx);
+                var result = NewHarness().Replay($"session:{fx}", $"tenant:{fx}", signals);
+
+                var isTerminal = result.FinalState.Stage == SessionStage.Completed
+                                 || result.FinalState.Stage == SessionStage.Failed
+                                 || result.FinalState.Stage == SessionStage.WhiteGloveSealed
+                                 || result.FinalState.Stage == SessionStage.WhiteGloveCompletedPart2;
+
+                Assert.True(
+                    isTerminal,
+                    $"Fixture {fx} ended in non-terminal stage {result.FinalState.Stage}.");
+            }
+        }
+    }
+}
@@ -0,0 +1,114 @@
+using System;
+using System.Collections.Generic;
+using AutopilotMonitor.DecisionCore.Signals;
+using AutopilotMonitor.DecisionCore.State;
+
+namespace AutopilotMonitor.DecisionCore.Engine
+{
+    // Edge-case handlers: hybrid reboot, ESP terminal failure, system reboot observation.
+    // Plan §2.5 / §M3.5.
+    public sealed partial class DecisionEngine
+    {
+        /// <summary>
+        /// Handle <see cref="DecisionSignalKind.EspResumed"/>. Emitted by
+        /// <c>EspAndHelloTracker</c> after a reboot mid-ESP. The session's stage is preserved
+        /// (we were already somewhere in the ESP phase sequence), but the reboot fact is
+        /// recorded so downstream classifiers (WhiteGloveSealingClassifier) can factor it in.
+        /// </summary>
+        private DecisionStep HandleEspResumedV1(DecisionState state, DecisionSignal signal)
+        {
+            var nextStep = state.StepIndex + 1;
+            var builder = state.ToBuilder()
+                .WithStepIndex(nextStep)
+                .WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
+
+            // Record the reboot fact on first observation. Subsequent EspResumed signals
+            // keep the original timestamp so the hash over SystemRebootUtc is stable.
+            if (state.SystemRebootUtc == null)
+            {
+                builder.SystemRebootUtc = new SignalFact<DateTime>(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
+            }
+
+            var newState = builder.Build();
+            var transition = BuildTakenTransition(
+                before: state,
+                signal: signal,
+                toStage: state.Stage,
+                nextStepIndex: nextStep,
+                trigger: nameof(DecisionSignalKind.EspResumed));
+
+            return new DecisionStep(newState, transition, Array.Empty<DecisionEffect>());
+        }
+
+        /// <summary>
+        /// Handle <see cref="DecisionSignalKind.EspTerminalFailure"/>. Directly transitions
+        /// to <see cref="SessionStage.Failed"/> with <see cref="SessionOutcome.EnrollmentFailed"/>,
+        /// clears deadlines, and emits <c>enrollment_failed</c>. No classifier re-run —
+        /// ESP terminal failure is definitive.
+        /// </summary>
+        private DecisionStep HandleEspTerminalFailureV1(DecisionState state, DecisionSignal signal)
+        {
+            var nextStep = state.StepIndex + 1;
+
+            var reason = signal.Payload != null && signal.Payload.TryGetValue("reason", out var r)
+                ? r
+                : "esp_terminal_failure";
+
+            var builder = state.ToBuilder()
+                .WithStage(SessionStage.Failed)
+                .WithOutcome(SessionOutcome.EnrollmentFailed)
+                .WithStepIndex(nextStep)
+                .WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal)
+                .ClearDeadlines();
+
+            var newState = builder.Build();
+
+            var transition = BuildTakenTransition(
+                before: state,
+                signal: signal,
+                toStage: SessionStage.Failed,
+                nextStepIndex: nextStep,
+                trigger: nameof(DecisionSignalKind.EspTerminalFailure));
+
+            var effects = new[]
+            {
+                new DecisionEffect(
+                    DecisionEffectKind.EmitEventTimelineEntry,
+                    parameters: new Dictionary<string, string>
+                    {
+                        ["eventType"] = "enrollment_failed",
+                        ["reason"] = reason,
+                    }),
+            };
+
+            return new DecisionStep(newState, transition, effects);
+        }
+
+        /// <summary>
+        /// Handle <see cref="DecisionSignalKind.SystemRebootObserved"/>. Records the fact so
+        /// WhiteGlove scoring (plan §2.4) can credit the +15 reboot-observed weight.
+        /// </summary>
+        private DecisionStep HandleSystemRebootObservedV1(DecisionState state, DecisionSignal signal)
+        {
+            var nextStep = state.StepIndex + 1;
+            var builder = state.ToBuilder()
+                .WithStepIndex(nextStep)
+                .WithLastAppliedSignalOrdinal(signal.SessionSignalOrdinal);
+
+            if (state.SystemRebootUtc == null)
+            {
+                builder.SystemRebootUtc = new SignalFact<DateTime>(signal.OccurredAtUtc, signal.SessionSignalOrdinal);
+            }
+
+            var newState = builder.Build();
+            var transition = BuildTakenTransition(
+                before: state,
+                signal: signal,
+                toStage: state.Stage,
+                nextStepIndex: nextStep,
+                trigger: nameof(DecisionSignalKind.SystemRebootObserved));
+
+            return new DecisionStep(newState, transition, Array.Empty<DecisionEffect>());
+        }
+    }
+}
@@ -83,7 +83,12 @@ namespace AutopilotMonitor.DecisionCore.Engine
                (DecisionSignalKind.DesktopArrivedPart2, 1)                => HandleDesktopArrivedPart2V1(state, signal),
                (DecisionSignalKind.AccountSetupCompletedPart2, 1)         => HandleAccountSetupCompletedPart2V1(state, signal),

-                // ----- Not yet wired (edge in M3.5) -----
+                // ----- Edge (DecisionEngine.Edge.cs) -----
+                (DecisionSignalKind.EspResumed, 1)                         => HandleEspResumedV1(state, signal),
+                (DecisionSignalKind.EspTerminalFailure, 1)                 => HandleEspTerminalFailureV1(state, signal),
+                (DecisionSignalKind.SystemRebootObserved, 1)               => HandleSystemRebootObservedV1(state, signal),
+
+                // ----- Still unwired (AppInstall, DeviceInfo, AutopilotProfileRead — v11.1+) -----
                _ => HandleUnhandledSignal(state, signal),
            };
        }
@@ -0,0 +1,9 @@
+# Scenario: ESP Terminal Failure
+# ESP reports a non-recoverable failure (e.g. Event 62411 failure) during AccountSetup.
+# Reducer transitions to Failed directly with reason carried from the signal payload.
+# enrollment_failed event emitted.
+# Plan: §4 M3.5 ESP-Terminal-Failure.
+{"SessionSignalOrdinal":0,"SessionTraceOrdinal":0,"Kind":"SessionStarted","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T09:00:00Z","SourceOrigin":"orchestrator","Evidence":{"Kind":"Synthetic","Identifier":"session:esp-fail:started","Summary":"Session start anchor"}}
+{"SessionSignalOrdinal":1,"SessionTraceOrdinal":1,"Kind":"EspPhaseChanged","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T09:01:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-phase-detector-v1","Summary":"ESP phase=DeviceSetup","DerivationInputs":{"rawValue":"1"}},"Payload":{"phase":"DeviceSetup"}}
+{"SessionSignalOrdinal":2,"SessionTraceOrdinal":2,"Kind":"EspPhaseChanged","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T09:10:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-phase-detector-v1","Summary":"ESP phase=AccountSetup","DerivationInputs":{"rawValue":"2"}},"Payload":{"phase":"AccountSetup"}}
+{"SessionSignalOrdinal":3,"SessionTraceOrdinal":3,"Kind":"EspTerminalFailure","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T09:12:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Raw","Identifier":"eventlog:Shell-Core:62411:recordId:900","Summary":"ESP terminal failure (Event 62411)","RawPointer":"blob://events/shell-core/900"},"Payload":{"reason":"esp_terminal_failure_account_setup"}}
@@ -0,0 +1,15 @@
+# Scenario: Hybrid-Reboot UserDriven (ESP mid-phase reboot, session resumes cleanly)
+# The device enters DeviceSetup, reboots before AccountSetup, agent restarts and emits
+# EspResumed + SystemRebootObserved. The existing stage is preserved, reboot fact is
+# recorded. Session then continues through AccountSetup, ESP exit, Hello, Desktop ->
+# Completed. Hybrid Reboot DOES NOT cause WhiteGlove sealing (no WG facts).
+# Plan: §4 M3.5 hybrid reboot scenario.
+{"SessionSignalOrdinal":0,"SessionTraceOrdinal":0,"Kind":"SessionStarted","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:00:00Z","SourceOrigin":"orchestrator","Evidence":{"Kind":"Synthetic","Identifier":"session:hybrid-reboot:started","Summary":"Session start anchor"}}
+{"SessionSignalOrdinal":1,"SessionTraceOrdinal":1,"Kind":"EspPhaseChanged","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:01:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-phase-detector-v1","Summary":"ESP phase=DeviceSetup","DerivationInputs":{"rawValue":"1"}},"Payload":{"phase":"DeviceSetup"}}
+{"SessionSignalOrdinal":2,"SessionTraceOrdinal":2,"Kind":"SystemRebootObserved","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:05:00Z","SourceOrigin":"orchestrator","Evidence":{"Kind":"Derived","Identifier":"boot-time-delta-detector-v1","Summary":"System boot time advanced since session start","DerivationInputs":{"priorBoot":"2026-04-24T07:59:00Z","currentBoot":"2026-04-24T08:05:00Z"}}}
+{"SessionSignalOrdinal":3,"SessionTraceOrdinal":3,"Kind":"EspResumed","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:05:30Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-resume-detector-v1","Summary":"ESP resumed after reboot","DerivationInputs":{"previousPhase":"DeviceSetup"}}}
+{"SessionSignalOrdinal":4,"SessionTraceOrdinal":4,"Kind":"EspPhaseChanged","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:10:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-phase-detector-v1","Summary":"ESP phase=AccountSetup","DerivationInputs":{"rawValue":"2"}},"Payload":{"phase":"AccountSetup"}}
+{"SessionSignalOrdinal":5,"SessionTraceOrdinal":5,"Kind":"ImeUserSessionCompleted","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:12:00Z","SourceOrigin":"ImeLogTracker","Evidence":{"Kind":"Raw","Identifier":"ime-log-line-hash:ghi789","Summary":"User session completed pattern"},"Payload":{"patternId":"ime-pattern-user-session-complete-v1"}}
+{"SessionSignalOrdinal":6,"SessionTraceOrdinal":6,"Kind":"EspExiting","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:13:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Derived","Identifier":"esp-exit-detector-v1","Summary":"ESP exit observed","DerivationInputs":{"rawValue":"exit"}}}
+{"SessionSignalOrdinal":7,"SessionTraceOrdinal":7,"Kind":"HelloResolved","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:14:00Z","SourceOrigin":"EspAndHelloTracker","Evidence":{"Kind":"Raw","Identifier":"hello-event-recordId:77","Summary":"Hello for Business enrolled"},"Payload":{"outcome":"Success"}}
+{"SessionSignalOrdinal":8,"SessionTraceOrdinal":8,"Kind":"DesktopArrived","KindSchemaVersion":1,"OccurredAtUtc":"2026-04-24T08:15:00Z","SourceOrigin":"DesktopArrivalDetector","Evidence":{"Kind":"Derived","Identifier":"desktop-arrival-detector-v1","Summary":"Explorer.exe running as real user","DerivationInputs":{"user":"user-anon-0030","explorerPid":"4500"}}}