Sessions: only an In Progress session stalls

A stall report never overwrote a guard: the ingest branch for session_stalled
runs after the completion and failure chain, and the transition gate let
Stalled through over every status but Pending. An offline batch carrying
session_stalled next to enrollment_complete left a finished enrollment
Stalled until the timeout sweep reclassified it, and a sweep that lost the
race against a completion could do the same.

IsTerminalTransitionAllowed now allows Stalled only over InProgress, on the
normal and the ETag-retry path alike; the separate Pending guard is folded
into it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-10-14 03:28:52 +02:00
parent 8c5fbd5739
commit af86e8f7cc
2 changed files with 24 additions and 10 deletions
@@ -50,10 +50,25 @@ public class SessionStatusTransitionTests
Assert.Equal(allowed, TableStorageService.IsTerminalTransitionAllowed(existing, SessionStatus.AwaitingUser));
}
// ---- Non-terminal incoming is governed by downstream guards, allowed here ----
// ---- Only an active session stalls: a stall report never overwrites a verdict, a waiting
// or a pre-provisioned session (an offline batch can carry session_stalled next to
// enrollment_complete, and the sweep can lose the race against the completion) ----
[Theory]
[InlineData("InProgress", true)]
[InlineData("Succeeded", false)]
[InlineData("Failed", false)]
[InlineData("Incomplete", false)]
[InlineData("AwaitingUser", false)]
[InlineData("Pending", false)]
[InlineData("Stalled", false)]
public void Only_an_active_session_stalls(string existing, bool allowed)
{
Assert.Equal(allowed, TableStorageService.IsTerminalTransitionAllowed(existing, SessionStatus.Stalled));
}
// ---- InProgress and Pending are governed by downstream guards, allowed here ----
[Theory]
[InlineData(SessionStatus.InProgress)]
[InlineData(SessionStatus.Stalled)]
[InlineData(SessionStatus.Pending)]
public void Non_terminal_incoming_is_allowed_here(SessionStatus incoming)
{
@@ -1915,7 +1915,10 @@ namespace AutopilotMonitor.Functions.Services
/// <item>Succeeded may UPGRADE any non-Succeeded state (late completion reconcile); idempotent no-op if already Succeeded.</item>
/// <item>Failed / Incomplete (silent-terminal verdicts) never overwrite an existing terminal (Succeeded/Failed/Incomplete).</item>
/// <item>AwaitingUser never regresses an existing terminal.</item>
/// <item>Non-terminal incoming statuses (InProgress/Stalled/Pending) are allowed here and governed by their own downstream guards.</item>
/// <item>Stalled marks an active session that stopped progressing: only InProgress can stall. A stall report that
/// arrives with or after a verdict (an offline batch carrying session_stalled and enrollment_complete, a sweep that
/// lost the race against the completion) never overwrites it, nor an AwaitingUser or Pending session.</item>
/// <item>InProgress and Pending are allowed here and governed by their own downstream guards.</item>
/// </list>
/// Static + pure so the reconcile matrix is unit-testable without Table Storage.
/// </summary>
@@ -1930,6 +1933,7 @@ namespace AutopilotMonitor.Functions.Services
SessionStatus.Succeeded => existingStatus != SessionStatus.Succeeded.ToString(),
SessionStatus.Failed or SessionStatus.Incomplete => !existingTerminal,
SessionStatus.AwaitingUser => !existingTerminal,
SessionStatus.Stalled => existingStatus == SessionStatus.InProgress.ToString(),
// Idempotency: the agent emits whiteglove_complete twice (raw Shell-Core event
// + engine timeline event, ~1s apart). The second Pending write must not report
// a "transition" — that fired duplicate Pre-Provisioning Completed webhooks.
@@ -2079,13 +2083,8 @@ namespace AutopilotMonitor.Functions.Services
// Status transitions from EventIngestProcessor: Succeeded, Failed, Pending, Stalled,
// and InProgress (the last one only for healing a Stalled session back to active).
// Guard: never regress a Pending (WhiteGlove) session to Stalled via the ingest
// path — WhiteGlove sessions are deliberately long-lived and handled via re-registration.
if (status == SessionStatus.Stalled && existingStatusStr == SessionStatus.Pending.ToString())
{
_logger.LogInformation($"Session {sessionId} is in Pending (WhiteGlove) state, skipping Stalled transition");
return false;
}
// Stalled comes only from InProgress (IsTerminalTransitionAllowed), so a Pending
// (WhiteGlove) session — long-lived on purpose, resumed by re-registration — never stalls.
// Guard: only allow InProgress transition if the current state is Stalled (healing).
// Other callers must not regress terminal or Pending state via this path.