gsadmin/AutopilotMonitor
1
0
Fork 0
mirror of https://github.com/okieselbach/AutopilotMonitor.git synced 2026-10-24 20:53:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

v2 single-rail 5.3c: Program.cs terminate callback waits for shutdown completion

Browse Source 
Plan §6.2 synchronous-shutdown — the ServerActionDispatcher terminate
callback now blocks until the main-thread cleanup sequence (orchestrator
stop, client disposal) has fully run, instead of returning
Task.CompletedTask immediately after terminationHandler.Handle.

Mechanism: a second ManualResetEventSlim `shutdownComplete` is Set() at
the very end of the finally block (after orchestrator.Stop + HTTP-client
disposal). The terminate callback waits on it for up to 60s after
terminationHandler.Handle returns. If the bound expires we log and return
rather than deadlock the ingest-dispatcher thread forever.

Effect: no fresh ingest call can happen between
terminationHandler.Handle returning and the process actually exiting,
which would otherwise give the backend another window to re-deliver
terminate_session. Combined with the PR #3a dedup this closes the
77×-storm loop from the root.

Non-server-action termination paths (decision-engine enrollment_complete,
max-lifetime, auth-failure, Ctrl+C) are unaffected — they already drive
shutdown from the main thread itself.

V2.Core: 538, DecisionCore: 108, all green.
This commit is contained in:
Oliver Kieselbach
2026-04-23 14:01:07 +02:00
parent 491c749688
commit 0bc04c3a9a
1 changed files with 23 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/Agent/AutopilotMonitor.Agent.V2/Program.cs
+23
View File
@@ -501,6 +501,7 @@ namespace AutopilotMonitor.Agent.V2
agentMaxLifetime: agentMaxLifetime))
{
using (var shutdown = new ManualResetEventSlim(false))
using (var shutdownComplete = new ManualResetEventSlim(false))
{
// M4.6.δ — Analyzer manager. Emits through the live orchestrator's event
// emitter. RunStartup fires after orchestrator.Start; RunShutdown is wired
@@ -734,6 +735,22 @@ namespace AutopilotMonitor.Agent.V2
stageName: orchestrator.CurrentState?.Stage.ToString(),
terminatedAtUtc: DateTime.UtcNow,
details: $"Server-requested termination: ruleId={action?.RuleId}, reason={action?.Reason}"));
// Plan §6.2 synchronous-shutdown — block the ingest dispatcher
// thread until the main-thread cleanup (orchestrator.Stop, client
// disposal) has fully run. This prevents the agent from issuing
// another ingest call after terminationHandler.Handle returns but
// before the process actually exits, which would give the backend
// another window to re-deliver terminate_session. The dedup in
// ServerActionDispatcher already squelches those re-deliveries,
// but waiting here keeps the ingest loop itself quiet. 60s bound
// covers the 10s spool drain + cleanup-service launch + orchestrator
// stop; if it times out the agent is misbehaving and we return
// anyway rather than deadlock the ingest thread forever.
if (!shutdownComplete.Wait(TimeSpan.FromSeconds(60)))
{
logger.Warning("Terminate callback: shutdownComplete wait timed out after 60s — returning without confirmed shutdown.");
}
return Task.CompletedTask;
},
post: lifecyclePost);
@@ -838,6 +855,12 @@ namespace AutopilotMonitor.Agent.V2
try { mtlsHttpClient.Dispose(); } catch { }
try { backendApiClient.Dispose(); } catch { }
// Plan §6.2 synchronous-shutdown — release any ingest-dispatcher thread
// currently parked in the terminate callback (see onTerminateRequested
// above). Signalled AFTER orchestrator.Stop and client disposal so the
// ingest thread only returns once no more HTTP work can happen.
shutdownComplete.Set();
}
}
}