mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
791eadcd7afec0f2a0f03af77064ccec706ba4e2
1226 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Kieselbach
|
791eadcd7a |
v2 single-rail 5.9b: ImeLogTrackerAdapter wire missing ImeLog callbacks (B29 part 2)
Plan §5.9 / PR #9 — V1 Parity Issue #3 completion: wire the three tracker callbacks the adapter previously did not touch so the full V1-compatible wire shape reaches the Events table via Signal-Post (Rail A single-rail). Adapter additions: - OnImeAgentVersion → ime_agent_version InformationalEvent (Source=ImeLogTracker); no dedup — matches V1 behavior where the same version fires multiple times per session at each pattern-match ingress. - OnDoTelemetryReceived → do_telemetry InformationalEvent carrying the full DO payload (doFileSize / doTotalBytesDownloaded / doBytesFromPeers / doPercentPeerCaching / doDownloadMode / doDownloadDuration / doBytesFromHttp + app state/intent/targeted/runAs/progress). Tracker guards with HasDoTelemetry, so each app fires at most once per "DO complete" pattern match. - OnScriptCompleted → script_completed InformationalEvent with Source="IME" (not "ImeLogTracker" — matches V1 contract observed in the Events table for both platform and remediation scripts); Severity auto-derives to Error when Result="Failed" or ExitCode != 0. Supports both scriptType=platform (Result + exitCode) and scriptType=remediation (ScriptPart + ComplianceResult). All three callbacks preserve prior handlers via the existing chain-invoke pattern and are restored on Dispose. Tracker's LastMatchedPatternId flows into the payload's "patternId" key to match the V1 shape. Tests: +11 new adapter tests covering each callback (payload/fields/severity, null-guards, prior-handler chaining, Dispose restore). Suite passes 48/48 for ImeLogTrackerAdapterTests and 562/562 for the V2.Core project. Full solution build clean. Parity Issue #3 (V1 reference e99a6805 → V2 Beleg-Session e259c121): the IME-driven events app_install_started / app_install_completed / app_install_failed / app_download_started / download_progress / do_telemetry / script_completed / ime_agent_version / ime_user_session_completed / esp_phase_changed are now emitted by V2 via the single-rail Signal-Post path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
b7f74dbd35 |
v2 single-rail 5.9a: ImeLogTrackerAdapter add InformationalEvent posts (B29 part 1)
Plan §5.9 / PR #9 — V1 Parity Issue #3: the IME log tracker callbacks now dual-emit a specific DecisionSignalKind + an InformationalEvent so the Events table regains the V1-compatible wire shape (esp_phase_changed, ime_user_session_completed, app_install_started, app_install_completed, app_install_failed, app_download_started, download_progress) while remaining on the single-rail signal pipeline. Changes: - ImeLogTrackerAdapter: construct an InformationalEventPost from the existing ingress/clock; after every EspPhaseChanged / UserSessionCompleted / AppStateChanged post, also emit a matching InformationalEvent. EspPhase declarations carry the mapped EnrollmentPhase (DeviceSetup / AccountSetup / FinalizingSetup) per feedback_phase_strategy. - AppState handler expanded from terminal-only to all state transitions with V1-compatible eventType mapping: Downloading transitions → app_download_started + byte updates → download_progress (Debug); Installing/InProgress → app_install_started; Installed/Skipped/Postponed → app_install_completed; Error → app_install_failed. Payload carries appId/appName/state/intent/ targeted/runAs/progressPercent/bytesDownloaded/bytesTotal/isError/isCompleted + appVersion/appType/detectionResult/attempt, plus error* fields on Error. - Shared.Constants.EventTypes: add EspPhaseChanged, ImeAgentVersion, ImeUserSessionCompleted, DoTelemetry, AllAppsCompleted. - Adapter tests: refactor to filter the posted-signal list by kind + eventType so the dual-emission contract is verifiable. 14 new tests cover the full state-transition matrix + payload shape + error-field population; total adapter test count 37 (up from 17). Verification: dotnet build AutopilotMonitor.sln clean; V2.Core test suite green (552 / 552, up from 538). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
be9070a68f |
v2 single-rail 5.8: DeviceInfoCollector + new DeviceInfoHost (B27, fixes Parity Issue #2)
V1 Parity Issue #2: DeviceInfoCollector existed in V2.Core but was not wired into any host, leaving the Web "Device Details" section empty on every V2 session. This PR migrates the collector to Signal-Post and adds a new DeviceInfoHost that schedules CollectAll() on a background thread at orchestrator startup. Collector (3 partials — .cs, .HardwareSpec.cs, .NetworkAndSecurity.cs): * Constructor Action<EnrollmentEvent> emitEvent -> InformationalEventPost post. 14 emit-sites flow through the shared EmitDeviceInfoEvent helper plus one direct enrollment_type_detected emit: os_info, boot_time, network_adapters, dns_configuration, proxy_configuration, autopilot_profile, enrollment_type_detected, esp_config_detected, secureboot_status, bitlocker_status, tpm_status, aad_join_status, network_interface_info, hardware_spec, wifi_signal_info. New DeviceInfoHost (nested in DefaultComponentFactory): * Kernel host (not remote-config-gated) — Device Details are essential for every enrollment session, not optional telemetry. * Constructs InformationalEventPost from (ingress, clock). * Start() schedules CollectAll() on Task.Run — WMI queries can take several seconds and must not block orchestrator.Start. Exceptions are logged at Warning but never crash the host. * Stop() is a no-op; the background task completes on its own. Follow-up (tracked in plan §5.8 TODO): Legacy EnrollmentTracker also called CollectAtEnrollmentStart on first DeviceSetup phase detection (re-fetches AAD join / autopilot profile / ESP config / TPM once MDM enrollment has populated them) and CollectAtEnd at termination (re-fetches BitLocker + active NIC). Hooking those into the V2 signal timeline requires a reducer-driven subscription that does not exist yet. This PR solves the dominant parity case; the re-collections are deferred. Tests: 538 V2.Core + 108 DecisionCore = 646 green. No DeviceInfoCollector unit tests exist today (the collector is deeply tied to live WMI + registry); the behavior change is verified structurally via the factory registration. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
818fa53728 |
v2 single-rail 5.7: migrate Analyzer trio + AgentAnalyzerManager to Signal-Post (B24-B26)
Full migration — the 3 analyzer classes and their manager all move to
InformationalEventPost, no Action<EnrollmentEvent>-bypass remains in the
analyzer surface (no dual-use concerns; no standalone CLI mode touches
these types).
Analyzers (1 Emit-site each):
* LocalAdminAnalyzer (B24): local_admin_analysis.
* SoftwareInventoryAnalyzer (B25): software_inventory_analysis
(startup baseline + shutdown delta; WG-part discriminator unchanged).
* IntegrityBypassAnalyzer (B26): integrity_bypass_analysis.
Manager:
* AgentAnalyzerManager stores InformationalEventPost _post and passes
it to all 3 analyzer constructors in Initialize().
Program.cs wiring:
* AgentAnalyzerManager construction deferred into orchestrator.Start's
onIngressReady hook so lifecyclePost is non-null at construction time.
The outer-scope `AgentAnalyzerManager analyzerManager = null;`
declaration is captured by both the later RunStartup() call (after
Start returns) and the terminationHandler constructor (inside the
same hook).
Tests:
* AgentAnalyzerManagerTests helper NewPost() returns a throwaway
InformationalEventPost(new FakeSignalIngressSink(), VirtualClock)
for the 7 ctor sites including the null-arg test.
* 538 V2.Core + 108 DecisionCore = 646 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
8e3e62f4d3 |
v2 single-rail 5.6: route session-mode GatherRules through Signal-Post (B20-B23)
GatherRuleExecutorHost now constructs an InformationalEventPost from (ingress, clock) and wraps post.Emit into the Action<EnrollmentEvent> that GatherRuleExecutor + GatherRuleContext + LogParserCollector + CommandCollector still consume internally. The factory's onEnrollmentEvent parameter is dropped from the host constructor; the CreateCollectorHosts call-site passes ingress + clock instead. Design rationale — the 4 internal Gather types keep their Action<EnrollmentEvent> signature (no ICollectorHost / IComponentFactory contract forces migration, and the standalone --run-gather-rules CLI mode in Program.GatherRulesMode.cs still needs to collect raw EnrollmentEvents in-memory for its direct BackendApiClient.IngestEvents upload path, which is the orthogonal world from plan §9). Session mode now routes all gather events through the InformationalEvent ingress pipe before hitting the telemetry spool — Rail-A semantics for ordering and replay determinism. Standalone mode unchanged. Tests: 538 V2.Core + 108 DecisionCore = 646 green. No Gather-specific tests exist in V2.Core.Tests today; the behavior change is verified structurally via the factory signature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
2fc405fb8b |
v2 single-rail 5.5c: migrate ESP+Hello trackers to Signal-Post (B14-B17, B19)
ESP+Hello sub-tracker bundle migration — they share the coordinator's
delegate and must be migrated together to keep the build buildable.
Sub-trackers (constructor param Action<EnrollmentEvent> onEventCollected
-> InformationalEventPost post):
* HelloTracker (B17, 7 emit-sites): hello_policy_detected, hello_*
UDR/HelloForBusiness events, hello_wait_timeout (x2),
hello_completion_timeout, SetPolicyForTest helper.
* ShellCoreTracker (B14, 1 emit-site): esp_exiting / esp_failure /
whiteglove_complete / hello_wizard_started via ProcessEvent.
* ProvisioningStatusTracker (B15, 3 emit-sites): fallback
esp_provisioning_status, EmitProvisioningEvent, EmitRawRegistryDump.
* ModernDeploymentTracker (B16, 2 emit-sites): ProcessEvent dispatch
(modern_deployment_log/warning/error) + HandleWhiteGloveStart.
Coordinator (B19):
* EspAndHelloTracker stores InformationalEventPost _post and forwards
it to all four sub-trackers in Start(). The per-tracker
onEventCollected forwarder parameter is retired; the single-rail
post flows through the coordinator unchanged.
Factory wiring:
* EspAndHelloHost drops onEnrollmentEvent and constructs the post from
(ingress, clock) before building the coordinator. The adapter still
gets the shared ingress directly for its DecisionSignal surface.
Adapter tests (5 files):
* ShellCoreTrackerAdapterTests, ProvisioningStatusTrackerAdapterTests,
HelloTrackerAdapterTests, EspAndHelloTrackerAdapterTests,
EspAndHelloTrackerAdapterHelloOutcomeTests fixture rigs use a
separate throwaway FakeSignalIngressSink for tracker emissions so
adapter DecisionSignal assertions on Ingress.Posted remain clean.
Tests: 538 V2.Core + 108 DecisionCore = 646 tests green.
PR #5 complete: all 7 tracker classes (B13-B19) now post through
InformationalEventPost. Architecture baseline unchanged (trackers never
had structural TelemetryEventEmitter deps; the reach went through the
Action<EnrollmentEvent> delegate chain from EnrollmentOrchestrator).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
14b57ac4b8 |
v2 single-rail 5.5b: migrate ImeProcessWatcher to Signal-Post (B18)
ImeProcessWatcher.OnImeExited now emits `ime_process_exited` via an InformationalEventPost built from (ingress, clock) inside ImeLogHost's constructor. Constructor parameter rename Action<EnrollmentEvent> onEventCollected -> InformationalEventPost post. No test file existed for ImeProcessWatcher; Windows-only Process.Exited semantics make it non-trivial to unit-test. Coverage stays via the E2E smoke test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
cc9c927227 |
v2 single-rail 5.5a: migrate StallProbeCollector to Signal-Post (B13)
StallProbeCollector now takes InformationalEventPost post instead of Action<EnrollmentEvent> onEventCollected. Three emit-sites (EmitProbeResultEvent, EmitProbeCheckTraceEvent, EmitSessionStalledEvent) route through the single-rail ingress. StallProbeHost in DefaultComponentFactory constructs the post from (ingress, clock) and drops the previous onEnrollmentEvent parameter. Adapter test fixture uses a separate throwaway FakeSignalIngressSink for the tracker's informational emissions so DecisionSignal assertions on the adapter's shared Ingress remain uncontaminated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
908255f57b |
v2 single-rail 5.4: migrate periodic collectors to Signal-Post (B8-B12)
Plan §5.4 PR-Block #4 — migrates the five peripheral periodic collectors from the Action<EnrollmentEvent> bypass onto the single-rail pipe. CollectorBase: ctor parameter Action<EnrollmentEvent> emitEvent becomes InformationalEventPost post; protected EmitEvent field becomes Post. Every subclass's emit site switches from EmitEvent(new EnrollmentEvent{...}) to Post.Emit(new EnrollmentEvent{...}) — the EnrollmentEvent overload keeps the on-wire shape identical while routing through ingress → reducer → EmitEventTimelineEntry effect. - PerformanceCollector (B8) — performance_snapshot - AgentSelfMetricsCollector (B9) — agent_metrics_snapshot - NetworkChangeDetector (B10) — network_state_change + network_connectivity_check - DeliveryOptimizationCollector (B11) — download_progress + do_telemetry - PeriodicCollectorLifecycleHost.EmitIdleStopped (B12) — performance_collector_stopped + agent_metrics_collector_stopped PeriodicCollectorLifecycleHost now owns its own InformationalEventPost built from an IdleActivityObservingIngressSink wrapper. The wrapper peeks each signal's eventType payload and bumps _lastRealEventTimeUtc for any non-periodic event, preserving the V1-parity idle-timeout window semantics the legacy WrapEventSink had — just at the ingress layer now. Wrapper is a nested sealed class; observer exceptions are swallowed so they cannot break the ingress pipe. DefaultComponentFactory: PeriodicCollectorLifecycleHost, NetworkChangeHost, and DeliveryOptimizationHost now take ingress + clock instead of onEnrollmentEvent. The remaining hosts keep onEnrollmentEvent for now — PR #5 through #9 migrate them in batch. Tests: V2.Core 538, DecisionCore 108, all green. Architecture baseline unchanged (still 3 permitted dependents) — none of the five migrated types holds a structural reference to TelemetryEventEmitter anymore. |
||
|
Oliver Kieselbach
|
0bc04c3a9a |
v2 single-rail 5.3c: Program.cs terminate callback waits for shutdown completion
Plan §6.2 synchronous-shutdown — the ServerActionDispatcher terminate callback now blocks until the main-thread cleanup sequence (orchestrator stop, client disposal) has fully run, instead of returning Task.CompletedTask immediately after terminationHandler.Handle. Mechanism: a second ManualResetEventSlim `shutdownComplete` is Set() at the very end of the finally block (after orchestrator.Stop + HTTP-client disposal). The terminate callback waits on it for up to 60s after terminationHandler.Handle returns. If the bound expires we log and return rather than deadlock the ingest-dispatcher thread forever. Effect: no fresh ingest call can happen between terminationHandler.Handle returning and the process actually exiting, which would otherwise give the backend another window to re-deliver terminate_session. Combined with the PR #3a dedup this closes the 77×-storm loop from the root. Non-server-action termination paths (decision-engine enrollment_complete, max-lifetime, auth-failure, Ctrl+C) are unaffected — they already drive shutdown from the main thread itself. V2.Core: 538, DecisionCore: 108, all green. |
||
|
Oliver Kieselbach
|
491c749688 |
v2 single-rail 5.3b: migrate EnrollmentTerminationHandler + agent_shutting_down event
Plan §5.3 + §6.2 — EnrollmentTerminationHandler now emits the six peripheral lifecycle events (enrollment_summary_shown, diagnostics_collecting/uploaded/upload_failed, reboot_triggered, whiteglove_part1_complete) through InformationalEventPost instead of the legacy Action<EnrollmentEvent> bypass. New acknowledgement: `agent_shutting_down` fires with ImmediateUpload=true just before CleanupService.ExecuteSelfDestruct. This gives the backend a definitive "agent accepted termination" signal so it can stop re-queuing terminate_session even when the cleanup script wins the race against the final telemetry drain. Program.cs: EnrollmentTerminationHandler construction moved into orchestrator.Start's onIngressReady hook (alongside ServerActionDispatcher) so the live lifecyclePost backs it. The Terminated-event subscription uses a null-checking dispatch wrapper so the registration can sit before Start returns without referencing the not-yet-constructed handler; by the time the decision loop processes its first signal, the hook has run synchronously and the handler is live. Constants.EventTypes.AgentShuttingDown = "agent_shutting_down" added. Tests: EnrollmentTerminationHandlerTests refactored — Rig now creates a real InformationalEventPost backed by FakeSignalIngressSink; assertions consult EmittedEventTypes / PayloadOf(eventType) helpers that read the posted signal payloads. Three new tests cover - agent_shutting_down emitted on self-destruct path with correct payload - skipped when SelfDestructOnComplete=false - skipped on WhiteGlove Part-1 exit (no self-destruct) V2.Core: 535 → 538, DecisionCore: 108, all green. Architecture baseline unchanged — EnrollmentTerminationHandler now takes InformationalEventPost, no structural reference to TelemetryEventEmitter. |
||
|
Oliver Kieselbach
|
c2c55a191e |
v2 single-rail 5.3a: migrate ServerActionDispatcher to Signal-Post + ActionId dedup
Plan §5.3 + §6.2 — ServerActionDispatcher now emits server_action_received/executed/failed through InformationalEventPost (single-rail pipe), not the legacy Action<EnrollmentEvent> bypass. Adds dedup of at-least-once re-deliveries from the backend PendingActions queue: composite key (Type|RuleId|QueuedAt) maps every re-delivery of the same server-queued action to the same slot, squelching duplicates before any telemetry or handler side effect fires. Fixes the 77× terminate_session storm observed in the PR-motivating session without rejecting two genuinely distinct admin-issued terminates. Program.cs: ServerActionDispatcher construction moved inside orchestrator.Start's onIngressReady hook so the live lifecyclePost backs it; WireTelemetryServerResponse picks it up from the outer-scope local after Start returns. Tests: 8 new ServerActionDispatcherTests cover success / failure / unknown- type emission plus the dedup matrix (same key twice, same-batch dup, distinct QueuedAt, distinct RuleId, distinct Type with identical QueuedAt). V2.Core test suite: 527 → 535, all green. Architecture baseline unchanged — ServerActionDispatcher still holds no structural dependency on TelemetryEventEmitter. |
||
|
Oliver Kieselbach
|
b02313b7ea |
v2 single-rail 5.2: migrate StartupEnvironmentProbes to Signal-Post (B7)
StartupEnvironmentProbes.RunAsync no longer takes a TelemetryEventEmitter
— it now routes device_location, timezone_auto_set, ntp_time_check, and
agent_trace (geo-failure) events through InformationalEventPost. The
existing Source labels ("Network") survive via the Emit(EnrollmentEvent)
overload's payload decomposition.
- Parameter rename: emitter → post throughout RunAsync / RunGeoAndTimezone /
RunNtpCheck / SafeEmit. Bodies unchanged apart from the dispatched Emit.
- Unused using removed (Telemetry.Events namespace no longer referenced
from this file — the probes live outside the emitter-internal zone).
- Program.cs passes lifecyclePost (the single-rail helper instance
established in PR #1) to the background Task.Run invocation, which runs
after orchestrator.Start has completed so the hook has already assigned
the variable.
- Architecture baseline test (PR #0 §4.4) now expects 3 dependents
instead of 4 — StartupEnvironmentProbes dropped out. Comment updated
to mirror the migration inventory so future PRs see the correct
removal sequence.
527 V2.Core tests + 108 DecisionCore tests still pass. No behavior
change on the wire: device_location / ntp / timezone_auto_set events
carry the same EventType / Source / DataJson as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
8b413d59a8 |
v2 single-rail 5.1c: migrate Program.cs lifecycle events to Signal-Post
Shifts every direct orchestrator.EventEmitter.Emit(...) in Program.cs (B2-B6 in the plan inventory) onto InformationalEventPost. Fixes the Parity Issue #1 ordering regression: agent_started now lands on the signal log as Seq=1 because it is posted from inside the orchestrator's onIngressReady hook, before any collector-generated signal reaches the ingress. Call sites migrated: - agent_started, agent_version_check, agent_unrestricted_mode_changed: emitted inside the new orchestrator.Start(ingress => { ... }) hook, in the canonical boot order. - enrollment_failed (max_lifetime watchdog), agent_shutdown (auth_failure watchdog): emitted from their event-handler lambdas via a lifecyclePost variable captured by reference — the handlers are registered before Start but only fire later, by which time the callback has assigned the live helper. Three private helpers (EmitAgentStartedEvent, EmitVersionCheckEventIfAny, EmitUnrestrictedModeAuditIfChanged) switched from an EnrollmentOrchestrator parameter to an InformationalEventPost parameter; bodies unchanged apart from the final Emit call. Side fixes: - enrollment_failed (max_lifetime) no longer declares Phase=Complete (plan §1.4 phase invariant — only phase-declaration events may set a non-Unknown phase). The UI bucketing behavior is unchanged; the legacy setting produced a phantom phase in the timeline. - Null-guard on the lifecyclePost variable: if a watchdog fires before Start's hook has assigned the helper, emission is skipped with a warning rather than NRE. In practice impossible (handlers only fire after Terminated / ThresholdExceeded which come later), but defensive. Left for later PRs (plan §2 inventory): - line ~575 ServerActionDispatcher emitEvent callback → PR #3 (B28). - orchestrator.EventEmitter property itself → PR #10 enforcement step. Tests: 527 V2.Core + 108 DecisionCore pass. Full solution builds clean. Functional impact on a real session: the V2 parity audit's Parity Issue #1 (agent_started = Seq=13) is structurally fixed — verification will be via an E2E session after this lands on a ring device. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3acac2ee71 |
v2 single-rail 5.1b: add Emit(EnrollmentEvent) overload to InformationalEventPost
Migration convenience for PR #1 call sites that already assemble an EnrollmentEvent inline. Decomposes the event into the primary Emit(eventType, source, ...) signature so a call-site diff stays to a single line change. - Overload maps EnrollmentEvent fields onto SignalPayloadKeys: EventType, Source, Message (when non-empty), Severity, ImmediateUpload, phase (only when != Unknown). Transport-managed fields (Sequence, RowKey, SessionId, TenantId) are ignored — the engine assigns them once the event reaches TelemetryEventEmitter again. - Data (Dictionary<string,object>) is flattened to Dictionary<string,string> with invariant-culture formatting; DateTime values use the "o" roundtrip format so wire shape matches the legacy Newtonsoft output. - Timestamp is forwarded as occurredAtUtc — EnrollmentEvent()'s ctor always sets it to DateTime.UtcNow, so there is no default to fall back from. Unspecified Kind is treated as UTC since the wire contract is UTC-only. 6 new xUnit tests cover: full-field decomposition, non-Unknown phase declaration, Timestamp-forwarding with Kind normalization, empty-message omission, DateTime round-trip format, null-argument rejection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
f3a2cc721f |
v2 single-rail 5.1a: add onIngressReady hook to EnrollmentOrchestrator.Start
Add an optional caller hook that fires after _ingress.Start() and the WhiteGlove Part-2 recovery bridge (step 13a) but before any collector host is started (step 14). Program.cs uses this slot in the next commit to post agent_started / agent_version_check / agent_unrestricted_mode_changed signals so they land on the signal log with sequence numbers lower than anything the collectors produce — fixes the Seq=13 ordering regression from the V2 parity audit (plan Parity Issue #1). - Start(Action<ISignalIngressSink>? onIngressReady = null). Default null preserves the existing signature 1:1 so all existing callers compile unchanged. - Hook fires synchronously on the caller thread. Exceptions are caught and logged so a malformed hook cannot abort Start or prevent collectors from running. - 5 xUnit tests cover: backward-compatible no-hook Start, synchronous invocation with live ISignalIngressSink, ingress.Post works from inside the hook without throwing, exception isolation, and idempotency (second Start still throws, hook runs exactly once). Program.cs migration lands in the next commit so this change can be reviewed in isolation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d1b4ceb561 |
v2 single-rail 4.4: add architecture baseline test for emitter callers
Living documentation of every type in V2.Core that structurally depends on TelemetryEventEmitter today. Shrinks with each migration PR; PR #10 tightens the assertion to the PR10 permitted set (EventTimelineEmitter + BackPressureEventObserver), at which point a new bypass attempt fails the build. - Reflection over fields, properties, constructor params, and method params on types in AutopilotMonitor.Agent.V2.Core. IL-level call-site detection is deliberately avoided: structural dependencies are sufficient to prove reachability and need no extra packages. - ExpectedCurrentBaseline enumerates the four known dependents today: Rail A (EventTimelineEmitter), the meta exception (BackPressureEventObserver), the EnrollmentOrchestrator field + EventEmitter property, and StartupEnvironmentProbes' static methods. - Second test asserts the PR10 permitted set is already a subset of the current baseline — guards against accidentally losing Rail A or the meta exception while migrating. - Program.cs is in the V2 executable assembly, outside this test's scope; it reaches the emitter via orchestrator.EventEmitter, which PR #10 internalizes. PR #0 is now functionally complete: InformationalEvent signal + reducer case + emitter parameter overrides + helper + baseline test, all with dedicated coverage. Build green, 624 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
428b14cd60 |
v2 single-rail 4.3: add InformationalEventPost helper for signal posting
Introduce the migration helper that lets peripheral collectors post an InformationalEvent signal without assembling the payload dict, the evidence record, or the signal kind themselves. Every PR #1-#9 call site in the single-rail refactor will go through this helper instead of writing raw TelemetryEventEmitter.Emit calls. - InformationalEventPost in V2.Core.Orchestration, constructor takes ISignalIngressSink + IClock (matches the existing adapter pattern). - Emit(eventType, source, ...) writes eventType + source mandatorily and folds optional message / severity / immediateUpload / phase / data into the payload using SignalPayloadKeys constants. - Data entries that collide with reserved top-level keys are silently dropped so a caller-supplied "source" in data cannot rewrite the explicit source label on the wire. - Evidence summary falls back to explicit arg → message → eventType in that order. - Timestamp defaults to IClock.UtcNow; callers can override via occurredAtUtc for deterministic replay harnesses. 13 xUnit tests cover payload composition, optional-field omission, data-collision guard, sourceOrigin/evidence fallback chain, argument validation, and timestamp semantics with VirtualClock. The helper lives outside Telemetry.Events so it stays reachable from callers once TelemetryEventEmitter becomes internal in PR #10 (enforcement step). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d1b8be12de |
v2 single-rail 4.1b: HandleInformationalEventV1 reducer + tests
Wire the InformationalEvent signal through the decision engine:
- New dispatch entry (InformationalEvent, 1) → HandleInformationalEventV1.
- Handler lives in DecisionEngine.Shared.cs next to the other neutral
observation handlers. Copies the signal payload 1:1 into a single
EmitEventTimelineEntry effect; DecisionState is only touched by the
standard StepIndex / LastAppliedSignalOrdinal bookkeeping.
- Missing or empty eventType / source produce a DeadEnd transition with
reason informational_event_missing_{key} so malformed signals are
visible in the transitions table rather than silently lost. Bookkeeping
still advances on a DeadEnd so replay cannot re-apply the ordinal.
- dataJson payload key is reserved for future JSON-blob expansion;
today's reducer forwards it verbatim (ends up as a flat Data string
in the emitted event). Docstring clarifies the reserved status.
- 13 xUnit tests cover taken-transition + one effect, payload copy
identity, state / hypothesis / deadline invariance, bookkeeping,
all deadend branches, and schema-version fallthrough.
Single-rail pipe is now functionally complete end-to-end (signal post →
reducer → effect → EventTimelineEmitter → TelemetryEventEmitter). Next
subtasks wire up the V2.Core signal-post helper (4.3) and the caller-
side architecture test (4.4).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
c2d3a433ed |
v2 single-rail 4.2b: extend EventTimelineEmitter parameter overrides
Follow-up to 4.2 needed before the InformationalEvent reducer case can
land: the emitter must let the reducer pass through the originating
collector's Source, Severity, Message, and ImmediateUpload values,
otherwise single-rail migration would collapse every event to
Source="decision_engine" / Severity=Info-by-suffix and break UI filters.
- New parameter keys: source, severity, message, immediateUpload.
Reason also gets a named constant (it was already consumed by
BuildMessage but as a string literal).
- Defaults preserve today's behavior exactly:
* Missing source → "decision_engine" (SourceId)
* Missing / unparsable severity → DeriveSeverity(eventType) suffix rule
* Missing message → "{eventType}: {reason}" fallback (or bare eventType)
* Missing / unparsable immediateUpload → true
- BuildDataDict strips all top-level parameter keys so the wire format
is self-consistent; reason stays in Data by design.
- Eight new tests for the overrides + their fallback semantics, plus a
regression guard that immediateUpload defaults to true when absent.
No reducer case emits any of these new keys yet — behavior change is
gated on the subsequent HandleInformationalEventV1 commit (4.1b).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
489f124c6d |
v2 single-rail 4.1a: add InformationalEvent signal kind + payload keys
Introduce the generic pass-through signal that carries an EnrollmentEvent shape through the decision engine. Any peripheral collector / lifecycle source needing the Events timeline will post this instead of calling TelemetryEventEmitter directly, so the engine is the single ordering source for the wire. Promotion to a decision-specific kind is a later, localized change. - DecisionSignalKind.InformationalEvent (new). - SignalPayloadKeys: EventType + Source (mandatory), Message, Severity, ImmediateUpload, DataJson (optional). - tests/fixtures/signal-kinds/informational-event-v1.json documents the wire shape with a realistic NTP-probe payload. No reducer handler yet — that is the next subtask (4.1b); this commit is purely the contract extension. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
715527eda9 |
v2 single-rail 4.2: add phase-override to EventTimelineEmitter
Add an optional "phase" parameter to the EmitEventTimelineEntry effect contract so explicit phase-declaration events (agent_started, esp_phase_changed) can carry their phase through the reducer pipe without breaking the Phase=Unknown default invariant for all other events (feedback_phase_strategy). - New PhaseParamKey constant and ParsePhase helper; unknown / unparsable values fall back to EnrollmentPhase.Unknown, never throw. - BuildDataDict skips the phase key so the top-level field is not duplicated in Data (parallel to the existing eventType skip). - Tests cover: valid enum name applied, invalid value falls back, case-sensitive match required, phase not duplicated in Data, plus the existing "no phase param -> Unknown" case. Part of the PR #0 foundation in the V2 single-rail refactor (see plan §1.4). No behavior change for today's reducers — they do not yet pass a phase parameter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
23aa51a3f2 | fix warnings and fix v2 agent hash compare | ||
|
Oliver Kieselbach
|
42ff4284d8 | added version info to backend | ||
|
Oliver Kieselbach
|
2e82dd2ba6 | fix folder structure v2 agent | ||
|
Oliver Kieselbach
|
2ecbefce73 | backend route fix and v2 agent version check fix | ||
|
Oliver Kieselbach
|
06bbf13dbe |
Merge branch 'v2-parity-port': V2-idiomatic refactor of parity ports
Three commits route lifecycle telemetry through the V2 reducer rather than synthesising events from Program.cs. Also wires the pre-existing LogReplay infrastructure to the CLI. - |
||
|
Oliver Kieselbach
|
1bb659cac5 | chore: bump build counters after V2 parity refactor tests | ||
|
Oliver Kieselbach
|
f149eef3fe |
feat(agent-v2): wire --replay-log-dir CLI flag to ImeLogTracker SimulationMode
V2 had the complete replay infrastructure in place but never wired up: ImeLogTracker.SimulationMode / SpeedFactor properties (ImeLogTracker.cs:66-67) and the ApplySimulationDelay compression loop (ImeLogTracker.Handlers.cs:529-546) were fully implemented, and AgentConfiguration had ReplayLogDir + ReplaySpeedFactor fields (L.128, L.134), but nothing set them from the CLI and nothing read them in the factory. This commit adds: - CLI args --replay-log-dir <path> and --replay-speed-factor <n> in Program.BuildAgentConfiguration (invariant-culture parse, zero/malformed rejected). - DefaultComponentFactory.ImeLogHost ctor accepts a simulation mode + speed factor and applies them to the inner tracker when ReplayLogDir is non-empty. - PrintUsage documents the flags under a new "Dev / Test" section. For the deterministic signal-level replay (V2 ReplayHarness + signal-log.jsonl fixtures) no wiring is needed - that is already the primary V2 test path. The IME- log replay activated here is the V1 compatibility dev-mode for support cases where only raw IME logs are available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
5b7f424d19 |
refactor(agent-v2): route whiteglove_resumed / system_reboot_detected / admin preemption through reducer effects
V2 parity cleanup - the three lifecycle events were emitted from Program.cs
directly while the corresponding reducer handlers (HandleWhiteGlovePart1To2Bridge,
HandleSystemRebootObservedV1) returned Array.Empty<DecisionEffect>(). Admin
preemption synthesised an EnrollmentTerminatedEventArgs and invoked
terminationHandler.Handle directly, bypassing the reducer entirely.
This commit unifies all three flows: the reducer is the single state owner, the
telemetry event appears as an EmitEventTimelineEntry effect of the state
transition, and the orchestrator's Terminated event fires automatically when
stage goes terminal - no more synthetic bypass.
- B1: HandleWhiteGlovePart1To2Bridge emits whiteglove_resumed timeline entry.
- B2: HandleSystemRebootObservedV1 emits system_reboot_detected with payload
keys previousExitType/lastBootUtc.
- B3: new DecisionSignalKind.AdminPreemptionDetected + new SessionOutcome.AdminPreempted.
Reducer HandleAdminPreemptionDetectedV1 transitions to Completed/Failed based on
adminOutcome payload, emits enrollment_complete/_failed timeline entry with
adminAction+source data keys. Program.cs posts the signal; terminationHandler
still subscribes to orchestrator.Terminated (no direct Handle calls on the
startup-preemption path).
Fixtures: tests/fixtures/signal-kinds/{system-reboot-observed,admin-preemption-detected}-v1.json.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
a63512d139 |
feat(agent-v2): post SessionStarted signal after register-session (V2 reducer anchor)
V2 parity lack: EnrollmentOrchestrator.Start never synthesised a SessionStarted signal, so HandleSessionStartedV1 (DecisionEngine.Shared.cs:36) was unreachable in production. DecisionState.Stage stayed at Unknown for the entire session lifetime while raw signals (ESP / Hello / etc.) landed on an uninitialised state. The reducer tolerated this silently, but it left us no session-anchor timestamp and no early stage-classification hook. Program.cs now posts SessionStarted directly after register-session success, with the tenant-registered metadata (EnrollmentType, IsHybridJoin, ValidatedBy, ...) as signal payload. Skipped on WhiteGlove Part-2 resume (orchestrator posts SessionRecovered instead) and on admin preemption (subsequent AdminPreemption signal drives straight to terminal; see next commit). Follow-up: the reducer handler currently only bumps StepIndex - a later patch should have it consume the enrollmentType/isHybridJoin payload for flow routing. Logged in tasks/todo.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
bcd3c23168 |
feat(agent-v2): register-session handshake + periodic-collector idle mgmt + network-change + DO handler-chain fix
Closes the remaining critical V1 parity gaps uncovered by the audit. Without these, the V2 agent would accept events into /api/agent/telemetry but the backend Sessions table would never get a row (so every UpdateSessionStatusAsync / IncrementSession silently no-ops), PerformanceCollector + AgentSelfMetricsCollector would run for the entire session lifetime filling the spool, network transitions would be invisible, and the DO collector would overwrite newer handlers on shutdown. PR #51 — Session registration (V1 MonitoringService.RegisterSessionAsync) - New SessionRegistrationHelper with 5-retry + V1 exponential backoff (2s/4s/8s/16s). Auth failures (401/403) short-circuit via AuthFailureTracker.RecordFailure ("agent/register-session") — which already fires first-failure distress with AuthCertificateRejected / DeviceNotRegistered. Other exceptions on the last attempt trigger EmergencyReporter with AgentErrorType.RegisterSessionFailed. - New EnrollmentRegistryDetector — V1-parity registry probes for CloudAssignedDomainJoinMethod (hybrid join) and CloudAssignedDeviceRegistration / CloudAssignedEspEnabled (v1 vs v2 / WDP) so SessionRegistration carries the same EnrollmentType / IsHybridJoin values V1 sent. - Program.cs calls RegisterWithRetryAsync after BackendTelemetryUploader is ready and BEFORE orchestrator.Start. On non-Succeeded outcome, the agent exits cleanly (exit code 6 for auth, 7 for other) instead of spinning up collectors for an unregistered session — V1 parity: "=== SESSION REGISTRATION FAILED — collectors will NOT start ===" - BackendApiClient.RegisterSessionAsync is now `virtual` so test doubles can intercept the HTTP call. - Admin-override preemption: when the register response carries AdminAction ("Succeeded"/"Failed") we emit the matching enrollment_complete/_failed event and synthesise a Terminated so the termination handler runs cleanup immediately — V1 parity with MonitoringService.cs:388-413. Periodic-collector idle lifecycle (V1 PeriodicCollectorManager) - New PeriodicCollectorLifecycleHost owns PerformanceCollector + AgentSelfMetricsCollector under a single CollectorIdleTimeoutMinutes window (default 15). Stops both after N minutes of no real enrollment activity and restarts them on the next real event. Without this the V2 agent kept polling perf counters every 60s for the agent's full lifetime on dormant sessions. - Activity observation: the host wraps the onEnrollmentEvent sink passed to its internal collectors so events with "performance_snapshot" / "agent_metrics_snapshot" / "stall_probe_*" / "session_stalled" types do NOT count as activity — only real events reset the idle timer. - Emits performance_collector_stopped / agent_metrics_collector_stopped on idle-stop (V1 parity event types + data schema). Network-change wiring - NetworkChangeDetector existed in V2 but was never instantiated. New NetworkChangeHost wires it into DefaultComponentFactory so WiFi SSID / default-route / MDM-endpoint-reachability transitions are emitted as network_change events — V1 parity with CollectorCoordinator.StartOptionalCollectors:375-382. DeliveryOptimizationHost Stop() bug - V1 comparison `ReferenceEquals(x.OnAppStateChanged, x.OnAppStateChanged)` was a tautology (always true) — silently overwrote any newer handler that had taken over the slot. Fixed by capturing the chained handler reference and comparing against it instead. Tests - 481/481 V2 tests green (+5 new SessionRegistrationHelper tests covering the 5-retry happy path, retry-then-success, auth short-circuit, exhaustion after 5 exceptions, and registration payload correctness). - 708/708 Backend tests untouched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fc778b4154 |
feat(agent-v2): V1-parity remediation — lifecycle events, auth/binary watchdogs, remote config
Closes the main V1/V2 divergences identified by the parity audit (tasks/plan_v2_parity.md). PR #47 — RemoteConfigMerger V1-parity - Tenant remote wins unconditionally for every 1:1-mapped knob (V1 parity with MonitoringService.ApplyRuntimeSettingsFromRemoteConfig). CLI flags only seed the initial AgentConfiguration. - Map EnableImeMatchLog → ImeMatchLogPath (Constants.ImeMatchLogPath default), with a safeguard for --ime-match-log dev overrides. - UploadIntervalSeconds, NtpServer, DiagnosticsUploadMode, DiagnosticsUploadEnabled, MaxAuthFailures, AuthFailureTimeoutMinutes all flow through. - Merge returns RemoteConfigMergeResult so the caller can audit UnrestrictedMode and LogLevel transitions. Program.cs emits `agent_unrestricted_mode_changed` after orchestrator.Start when the guardrail flips (V1 AuditUnrestrictedModeChange). PR #48 — BinaryIntegrityVerifier runtime self-update - Use Process.GetCurrentProcess().MainModule.FileName (V1 parity; Assembly.Location is unreliable under ngen / shadow copy). - Match → Info log; mismatch → Error log with V1-format message. - New `runtimeSelfUpdateTrigger` delegate fired with an Interlocked single-shot guard. Program.cs wires it to SelfUpdater.CheckAndApplyUpdateAsync(forceUpdate=true, triggerReason="runtime_hash_mismatch") so the agent auto-heals on SHA drift. PR #49 — AuthFailureTracker V1-parity - Tracker is now the single dispatch point for auth-failure distress. First 401/403 → AuthCertificateRejected / DeviceNotRegistered distress (V1 HandleAuthFailure). Duplicate ConfigFetchDenied send removed from RemoteConfigService. - Threshold log uses the V1 === AGENT SHUTDOWN === banner. - Program.cs emits `agent_shutdown` (reason=auth_failure) with the full V1 data schema before tripping the shutdown signal. PR #50 — Lifecycle events + SaveWhiteGloveComplete + standalone reboot - SessionIdPersistence gains SaveWhiteGloveComplete / ClearWhiteGloveComplete so WhiteGlove Part-2 resume detection finally works (IsWhiteGloveResume was unreachable before because no path wrote the marker). - EnrollmentTerminationHandler emits the V1 lifecycle events: `enrollment_summary_shown`, `diagnostics_collecting`, `diagnostics_uploaded` / `diagnostics_upload_failed`, `whiteglove_part1_complete`, `reboot_triggered`. - Standalone-reboot flow: when SelfDestructOnComplete=false and RebootOnComplete=true the handler drains the spool briefly and invokes shutdown.exe /r /t <delay>. - Program.cs emits `agent_started` (with 11+ data fields), `whiteglove_resumed`, and `system_reboot_detected` (previousExitType=reboot_kill) after orchestrator.Start. PR #52 — Collector parity - EnrollmentOrchestrator's max-lifetime watchdog now emits a V1-format `enrollment_failed` event (failureType=agent_timeout, failureSource=max_lifetime_timer, Phase=Complete) in addition to raising Terminated(MaxLifetimeExceeded). - GatherRuleExecutorHost propagates AgentConfiguration.UnrestrictedMode into the runtime executor so gather rules can bypass AllowList checks when the tenant admin elevates the guardrail (V1 CollectorCoordinator.StartGatherRuleExecutor). Tests - 476/476 V2 tests pass (was 475 — one timing-sensitive mismatch test bumped to 10s). - 507/507 V1 Agent tests + 708/708 Backend tests unaffected. - New tests cover: Remote-wins-always, UnrestrictedMode audit, EnableImeMatchLog, runtime self-update trigger single-shot, distress type per status code, diagnostics event emission, standalone reboot, WhiteGlove marker write. Deferred - PR #51 (register-session 5-retry) is out of scope — V2 does not call /api/agent/register-session in the main flow today. Porting that is a feature addition, not a V1 parity fix. - PR #53 (emergency-break, RunStartupAnalyzers) already landed in main via earlier PRs #44–#46; verified against plan §3.11. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ddacaf29ce |
Merge pull request #46 from okieselbach/v2-feat-auth-tracker-binary-integrity
feat(agent): implement MaxAuthFailures watchdog + post-config binary integrity check |
||
|
Oliver Kieselbach
|
6a9203905d |
feat(agent): implement MaxAuthFailures watchdog + post-config binary integrity check
Closes three DTO-only fields on AgentConfigResponse that have been dead since the config schema was defined — neither V1 nor V2 ever consumed them. V2 now honours them end-to-end. 1. MaxAuthFailures + AuthFailureTimeoutMinutes — AuthFailureTracker New Security/AuthFailureTracker: thread-safe counter + time-window observer that fires a single-shot ThresholdExceeded event when either ceiling is crossed. Wired into: - RemoteConfigService.FetchConfigAsync: RecordSuccess on 2xx, RecordFailure on BackendAuthException (401/403 from /api/agent/config). - BackendTelemetryUploader.MapResponseAsync: RecordSuccess on 2xx, RecordFailure on 401/403 (moved from static to instance method). - Program.cs: tracker constructed before FetchConfig with CLI defaults; UpdateLimits called after RemoteConfigMerger.Merge so tenant policy wins; ThresholdExceeded handler sets the main shutdown signal so the agent exits cleanly instead of retrying a permanently-revoked certificate forever. Defaults: MaxAuthFailures=5, AuthFailureTimeoutMinutes=0 (time-window disabled). Both honour 0 = "feature off", and UpdateLimits can tighten or relax the ceilings mid-run when the tenant admin changes policy. 2. LatestAgentExeSha256 — BinaryIntegrityVerifier New Security/BinaryIntegrityVerifier: pure-static helper that computes SHA-256 over the running agent EXE and compares against the backend- advertised hash. Case-insensitive hex compare. Three outcomes: - Skipped: no hash provided / local path not resolvable → no-op. - Match: silent success path (debug-log only). - Mismatch: emits AgentErrorType.IntegrityCheckFailed via EmergencyReporter (the enum slot was already reserved — just never used). Does NOT terminate or auto-update; SelfUpdater's own hash-based trigger handles the "wrong EXE" case at next startup, and auto-terminating the agent on every mismatch would risk boot loops during rollouts. Wired in Program.cs immediately after RemoteConfigMerger.Merge. Context These are three of the four "DTO-only" fields I flagged in PR #45's body. User feedback: do not leave dead config schema entries. V1 never implemented any of these either (V1 has a ZIP-hash check in SelfUpdater via LatestAgentSha256, but no EXE-hash check and no auth-failure shutdown threshold) — so this is net-new functionality that makes V2 strictly better than V1 in these three dimensions, not a regression closure. Test coverage - AuthFailureTrackerTests (12 tests): count-based threshold, time-window threshold, RecordSuccess resets both, single-shot idempotence, handler-exception isolation, disabled-limits never fire, UpdateLimits retroactively tightens or relaxes, thread-safety under 8 concurrent callers, ctor null-guards. - BinaryIntegrityVerifierTests (6 tests): skipped on null/empty/missing file, match path, case-insensitive compare, mismatch path, and a known-vector test against SHA-256("abc") to pin the hex formatter. 464 / 464 V2.Core tests green (vs. 446 before). Not mapped / remaining dead-config EnableImeMatchLog stays DTO-only — it duplicates the CLI flag --ime-match-log with no semantic overlap to a backend switch, and mapping it would just overwrite a dev-only debug toggle. Left as documented non-feature. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
7e5f7d5964 |
Merge pull request #45 from okieselbach/v2-hotfix-config-mapping-and-init-order
fix(agent): V2 hotfix #2 — wire remote config + reorder Transport subscription |
||
|
Oliver Kieselbach
|
a46ecb7ebd |
fix(agent): wire remote config onto AgentConfiguration + reorder Transport subscription
Two V2 agent bugs surfaced in the VM smoke-test after the first-run ghost-detect hotfix (PR #44) let the agent reach the backend fetch phase: 1. INIT-ORDER CRASH — Program.WireTelemetryServerResponse touched orchestrator.Transport before orchestrator.Start() was called, and the Transport getter throws InvalidOperationException pre-Start because _transport is only constructed inside Start() at EnrollmentOrchestrator.cs:311. Every VM crashed at this exact line. Fix: move the wire call to immediately after orchestrator.Start() inside the same try block. No behavioural loss — the first ServerResponseReceived cannot fire until a telemetry upload completes, which happens after Start(). 2. REMOTE-CONFIG WIRING GAP — RemoteConfigService fetched AgentConfigResponse from the backend but only Collectors, ImeLogPatterns, GatherRules, WhiteGloveSealingPatternIds and LatestAgentSha256 were consumed downstream. ~20 tenant-controlled fields were bound to AgentConfiguration defaults from CLI / bootstrap-config and never got projected from the remote DTO — SelfDestructOnComplete, KeepLogFile, RebootOnComplete, NtpServer, EnableTimezoneAutoSet, EnableGeoLocation, DiagnosticsUploadMode, DiagnosticsLogPaths, DiagnosticsUploadEnabled, ShowEnrollmentSummary + EnrollmentSummary* (4 fields), LogLevel, AgentMaxLifetimeMinutes, HelloWaitTimeoutSeconds, UploadIntervalSeconds, MaxBatchSize, SendTraceEvents, UnrestrictedMode, RebootDelaySeconds. Every consumer site (CleanupService, SummaryDialogLauncher, StartupEnvironmentProbes, DiagnosticsPackageService, EnrollmentTerminationHandler, AgentLogger, EnrollmentOrchestrator, GatherRuleGuards) reads agentConfig.XXX — so tenant admin settings were effectively ignored. A tenant with SelfDestructOnComplete=false still saw its agent wipe ProgramData on enrollment complete, because the CLI-default !noCleanup=true always won. Fix: new RemoteConfigMerger.Merge(agentConfig, remoteConfig, args) pure static helper, called immediately after FetchConfigAsync. Projects all ~20 tenant-controlled fields onto AgentConfiguration. CLI flags remain authoritative: flags explicitly passed (--no-cleanup, --keep-logfile, --reboot-on-complete, --disable-geolocation, --log-level) skip the remote overwrite so local dev runs keep their ad-hoc overrides. Partial-remote-config tolerance: empty strings, null lists and non-positive ints keep the AgentConfiguration default. Non-V1-parity fields (MaxAuthFailures, AuthFailureTimeoutMinutes, LatestAgentExeSha256) are deliberately skipped — they are DTO-only in V1 too and have no consumer in either version, no regression. Test coverage added: - ProgramGuardsTests: Transport pre-Start now explicitly asserted to throw (regression guard); Transport_is_available_after_start confirms the post- Start happy path. - RemoteConfigMergerTests (20 tests): happy path (all fields), 5 CLI override tests (no-cleanup, keep-logfile, reboot, disable-geolocation, log-level), log-level case-insensitivity, log-level parse failure preserves default, null remote-config + null args + null nested Collectors tolerated, empty- string NtpServer + null DiagnosticsUploadMode keep defaults, null DiagnosticsLogPaths becomes empty list (not null), non-positive int clamping (UploadIntervalSeconds, MaxBatchSize, HelloWaitTimeoutSeconds), zero AgentMaxLifetimeMinutes allowed (explicit "disabled"), null-agent-config argument guard, and a regression test for the exact VM scenario (tenant SelfDestructOnComplete=false + agent with CLI-default true). 446 / 446 V2.Core tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
abd232ea90 |
Merge pull request #44 from okieselbach/v2-hotfix-first-run-ghost-detection
fix(agent): V2 hotfix — first-run ghost-detect triggers self-destruct on fresh install |
||
|
Oliver Kieselbach
|
bf090c436e |
fix(agent): create SessionId before startup guards to prevent first-run ghost-detect
The V2 agent was deterministically self-destructing on every first run after --install on a fresh VM. CheckEnrollmentCompleteMarker reads HKLM\SOFTWARE\AutopilotMonitor\Deployed (written by --install) and treats "marker present + session.id absent" as a ghost restart, triggering ExecuteSelfDestruct and wiping %ProgramData%\AutopilotMonitor before FetchConfigAsync can run. Remote config's SelfDestructOnComplete=false was never honored because the config fetch never happened. Legacy V1 agent creates the session in LoadConfiguration (Program.Configuration.cs:131) before the guard runs (Program.cs:169). V2 reversed this ordering with a comment claiming it prevents "dead session resurrection" — but the guards key on file-system state (SessionPersistence.SessionExists()), not the runtime-loaded session object, so the reordering was functionally unnecessary and introduced this regression. Fix: move SessionIdPersistence.GetOrCreate + --new-session handling back before the guards, restoring Legacy boot order. Ghost-restart detection still triggers in its intended case (session.id file deleted by incomplete self-destruct, but Scheduled Task + registry marker survive). Test coverage: adds 3 missing tests for CheckEnrollmentCompleteMarker in ProgramGuardsTests.cs (first-run with session, marker without selfdestruct, marker with selfdestruct retry). The M4.6.alpha guard suite had no coverage for this path, which is how the regression shipped. 425 V2.Core tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
09899a90ac |
ci(web): allow workflow_dispatch to run the deploy job
The build_and_deploy_job's if-guard accepted push + non-closed PR
events only, so manually triggering the Static Web Apps workflow via
`gh workflow run` ran but skipped both jobs (conclusion=skipped).
After the V2-refactor merge to main (commit
|
||
|
Oliver Kieselbach
|
62c0c990bc |
Merge pull request #43 — V2 Agent refactor + M5 backend + Codex review fixes
Merges branch Agent-v2.0 onto main: complete V1 to V2 agent kernel refactor (M1 to M5), admin UI for EnableIndexDualWrite flag, plus the three Codex review fixes and the SessionTraceOrdinal recovery-seed follow-up. 60 sub-milestone commits preserved via merge-commit strategy for future git-bisect traceability. |
||
|
Oliver Kieselbach
|
1a1573ec45 |
fix(agent): seed SessionTraceOrdinalProvider from persisted max on recovery (Codex follow-up M4.4.5.f)
Closes the pre-existing TODO Codex flagged in EnrollmentOrchestrator.cs
(doc line 39 + init line 354) and removes the recovery-time duplicate-
trace-ordinal hazard it would have left in place.
Before: the provider was constructed with no seed, so on agent restart
its counter began at -1 and the first post-recovery signal landed with
SessionTraceOrdinal=0 — guaranteeing a collision with whatever trace
ordinal the previous session had already persisted. The collision
surfaces at the Inspector (cross-type correlation fields) and in any
downstream replay that joins on SessionTraceOrdinal.
Interface additions (ISignalLogWriter + IJournalWriter):
- long LastTraceOrdinal { get; }
max SessionTraceOrdinal observed across persisted entries; -1 on an
empty log. Distinct from LastOrdinal / LastStepIndex which track the
kind-specific monotone counters; SessionTraceOrdinal is the
session-wide correlation counter shared across Event + Signal +
Transition (DecisionSignal §2.2).
Implementations (SignalLogWriter + JournalWriter):
- Renamed ScanHighestOrdinal / ScanHighestStepIndex into a shared
ScanRecoveryState() that populates both fields in a single JSONL pass
on construction. Same stop-at-corrupt-tail semantic as before
(recovery §2.7). The Append path also tracks the max inline so
subsequent queries after a writer is in use stay O(1).
EnrollmentOrchestrator.Start (step 9):
var traceSeed = Math.Max(
Math.Max(_signalLog.LastTraceOrdinal, _journal.LastTraceOrdinal),
_spool.LastAssignedItemId);
_traceCounter = new SessionTraceOrdinalProvider(seedLastAssigned: traceSeed);
All three sources are consulted because each can hold a trace ordinal
the other two missed:
- SignalLog may contain signals whose spool-enqueue later failed (the
emit path swallows transport exceptions per SignalIngress §2.7c).
- Journal likewise for transitions with failed spool-enqueue.
- Spool.LastAssignedItemId is the single-source-of-truth for everything
that reached the transport — including Events that bypass the provider
entirely but still share the TraceOrdinal space.
Test harness:
- FakeJournalWriter gains a LastTraceOrdinal property (computes max
from its captured Appended list) so processor-level tests keep
compiling.
Tests (+6 in V2.Core.Tests):
- SignalLogWriter: empty → -1; max across out-of-order trace ordinals
(not just the last appended); restart rescans on-disk file to
recover max.
- JournalWriter: same three cases.
Both empty-log tests exist so the orchestrator seed falls through
cleanly on first-ever startup (seedLastAssigned=-1 → Next()=0 matches
the fresh-session invariant).
Suites: V2.Core 422 green (prev 416 + 6 new). Full solution build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
51e9966e0a |
fix(agent): forward AdminAction outcome to termination handler (Codex Finding 2)
Portal Mark-Succeeded traffic was silently landing as Failed locally. The
M4.6.ε ServerAction synthesis in Program.cs already copied the adminOutcome
value into the terminate_session ServerAction's Params bag, but the
terminate_session callback that fed the TerminationHandler hard-coded
EnrollmentTerminationOutcome.Failed regardless of what adminOutcome said.
Downstream consequences of the hard-coded Failed:
- FinalStatusBuilder.MapOutcome rendered "Failed" in the SummaryDialog
even when the operator had explicitly marked the session Succeeded.
- EnrollmentTerminationHandler's DiagnosticsUploadMode.OnFailure gate
fired a diagnostics package for every admin-override, regardless of
actual outcome.
Fix:
- New Program.ServerActionSynthesis.cs partial (public static class
Program, matches existing declaration) carrying a pure static helper
MapAdminOutcome(IReadOnlyDictionary<string,string>? params) →
EnrollmentTerminationOutcome. Case-insensitive "Succeeded" → Succeeded;
missing/null/empty/any-other-value → Failed (default-failure-safe for
kill-signal-driven synthesis that never sets adminOutcome).
- The terminate_session callback in Program.cs now calls the helper and
threads the mapped outcome into EnrollmentTerminatedEventArgs. Log
message extended to include the mapped outcome so ops can correlate
the server-side override with the local termination result.
Reason stays DecisionTerminalStage. A distinct AdminAction reason would
need a DecisionCore enum extension — out of scope for this fix. What
Codex flagged concretely (Mark-Succeeded → visible as Failed) is now
addressed.
Tests (+10 in V2.Core.Tests): AdminActionOutcomeMappingTests
- "Succeeded" → Succeeded
- Case-insensitive variants (succeeded / SUCCEEDED / mixed case)
- "Failed" → Failed
- Unknown values (TimedOut / weird-future / empty) → Failed
- Missing adminOutcome param (kill-signal case) → Failed
- Null params dict → Failed
Accesses the helper through InternalsVisibleTo("AutopilotMonitor.Agent.V2.Core.Tests")
that was already set on the V2 agent assembly.
Suites: V2.Core 416 green (prev 406 + 10 new). Full solution build green.
This closes the last of the three Codex findings from the PR-43 review.
Remaining review comment on cross-layer test coverage is already
satisfied by the TelemetryWireRoundTripTests added in commit
|
||
|
Oliver Kieselbach
|
b6ea13f122 |
fix(agent): upload V2 Signals + DecisionTransitions to backend (Codex Finding 1)
The V2 agent was never projecting DecisionSignals or DecisionTransitions
onto the telemetry transport. SignalIngress.Append wrote the signal to
the local SignalLog and handed it to the reducer; DecisionStepProcessor
journaled each transition locally. Neither called into ITelemetryTransport,
so the entire M5 backend ingest pipeline for these two telemetry kinds
(Signals/DecisionTransitions primary tables, Inspector reads,
ReducerVerifier, the M5.d index fan-out + timer) had zero agent traffic.
Adds the missing emitters — analogous to TelemetryEventEmitter — and
wires them through EnrollmentOrchestrator:
- TelemetrySignalEmitter (V2.Core/Telemetry/Signals/): serializes a
DecisionSignal with Newtonsoft + StringEnumConverter and builds a
TelemetryItemDraft with Kind=Signal, PK={tenant}_{session},
RK=SessionSignalOrdinal.ToString("D19") — matches
TableSignalRepository.BuildRowKey for idempotent UpsertReplace.
- TelemetryTransitionEmitter (V2.Core/Telemetry/Transitions/): same
contract for DecisionTransition; RK=StepIndex.ToString("D10").
Wiring:
- SignalIngress accepts an optional TelemetrySignalEmitter. After the
local SignalLog.Append succeeds — before the reducer runs — the
ingress enqueues the signal on the transport. Enqueue failures are
swallowed: the local SignalLog is authoritative per §2.7c / L.1, and
the reducer must still see the signal even if the upload path is
broken.
- DecisionStepProcessor accepts an optional TelemetryTransitionEmitter.
After the Journal.Append try-block succeeds (and before the effect
runner), it enqueues the transition. Same swallow semantics. Journal
failures still rethrow as before, so a failed-journal step can never
leak a ghost row to the backend.
- EnrollmentOrchestrator constructs both emitters alongside the existing
TelemetryEventEmitter (step 4 of the startup sequence) and passes them
into steps 8 + 10. Both emitter constructor params are optional to
keep pre-M5 test constructors compilable without touch-up.
Tests (+26 across two assemblies):
V2.Core.Tests (+21):
- TelemetrySignalEmitterTests (6): correct keys + kind, StringEnumConverter
for DecisionSignalKind, PascalCase field names the backend parser
reads, D19 padding survives long.MaxValue, null-rejection, ctor
argument validation.
- TelemetryTransitionEmitterTests (7): same surface + dead-end case
preserves DeadEndReason + Taken=false, nested ClassifierVerdict
emits ClassifierId/Level that the backend parser can read.
- SignalIngressUploadPathTests (4): emit happens per signal, monotonic
RowKey padding, null emitter preserves pre-M5 behaviour, transport
throw is swallowed (reducer keeps running).
- DecisionStepProcessorUploadPathTests (4): emit happens per successful
journal, null emitter preserves pre-M5 behaviour, journal failure
prevents emit (no ghost rows), transport throw is swallowed (effects
still run + CurrentState advances).
Functions.Tests (+5) — the Codex-requested cross-layer gate:
- TelemetryWireRoundTripTests: a DecisionSignal / DecisionTransition
serialized with the exact JsonSerializerSettings the agent emitter
uses (StringEnumConverter + NullValueHandling.Ignore) is parsed by
TelemetryPayloadParser.ParseSignal / ParseTransition back into a
SignalRecord / DecisionTransitionRecord with every field preserved.
Includes explicit-zero edge cases (ordinal=0, stepIndex=0) so the
Finding-3 required-field guards don't trip on legitimate agent traffic.
The backend project can't reference the net48 agent assembly directly,
so the test replicates the wire contract via shared DecisionCore types
+ the agreed serializer settings — any drift between agent and backend
breaks this test immediately.
Suites: V2.Core 406 green (prev 385 + 21 new). Functions 708 green
(prev 703 + 5 new). Full solution build green.
Does not address Codex Finding 2 (AdminAction=Succeeded mis-mapped to
Failed locally) — lands in the next commit on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
ee07059bc3 |
fix(backend): harden /api/agent/telemetry batch validation (Codex Finding 3)
The ingest endpoint trusted the first item's PartitionKey for the whole
batch and silently defaulted RowKey-driver fields to 0 when the agent
envelope omitted them. Both paths can misroute data on the storage layer.
1. Cross-session guard: every TelemetryItemDto in a batch must share the
same PartitionKey as items[0]. Without this, a mixed-session batch
would be stamped with session-A's identity inside PersistItemsAsync
and silently relocate session-B's signals/transitions into session-A's
primary rows. Mismatch now returns 400 BadRequest with the offending
index logged. Extracted as pure helper FindMismatchingPartitionKey so
the guard is unit-testable without an HTTP harness.
2. RowKey-driver enforcement in TelemetryPayloadParser:
- ParseSignal now rejects envelopes where SessionSignalOrdinal is
absent or null (returns null → caller drops the item). A missing
value would default to 0 and clobber a legitimate ordinal-0 row
on UpsertReplace. Explicit ordinal=0 stays a valid value.
- ParseTransition applies the same defense to StepIndex.
- Distinguishes "key missing" from "key present with value 0" via
JObject.TryGetValue + JTokenType check.
Tests (+11 in Functions):
- IngestTelemetryFunctionTests: empty batch, single-item batch, uniform
batch, mismatch in middle, case-sensitive mismatch.
- TelemetryPayloadParserTests: missing/null SessionSignalOrdinal on
Signal → null; explicit ordinal 0 accepted; missing/null StepIndex on
DecisionTransition → null; explicit StepIndex 0 accepted.
Full Functions suite: 703 green (prev 692 + 11 new).
Does not address Codex Findings 1 (V2 agent has no Signal/Transition
upload path) or 2 (AdminAction=Succeeded mis-mapped to Failed locally)
— those land in follow-up commits on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
37fff23943 |
chore: gitignore entire tasks/ folder, untrack per-session notes
Replaces the targeted `tasks/todo.md` ignore with a blanket `tasks/` so the whole folder stays local-only. Content there is per-session planning + lesson notes (todo.md, lessons.md, m2-copy-list.md); it doesn't belong in version control alongside the shipped code — it's operator workspace, not repo state. Untracks the two files that were still being versioned (tasks/lessons.md, tasks/m2-copy-list.md) via `git rm --cached` so they stay on disk but leave the index. Historical versions remain in git history for reference; new edits won't be tracked. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
65085ff7ce |
feat(web): expose EnableIndexDualWrite toggle in admin Global Settings
Closes the operator-facing gap left by M5.d.2: the feature flag was plumbed server-side (AdminConfiguration model + TableConfigRepository Store/Map) but the UI had no way to flip it. Without this change the entire M5.d pipeline was unreachable in production. Wire-up mirrors the existing allowAgentDowngrade toggle (closest analog — also a developer/ops-level bool with caveat copy): - types/adminConfig.ts: add optional enableIndexDualWrite field with a doc comment pointing at Plan §M5.d so future maintainers understand what toggling it does. - AdminConfigContext.tsx: add state + setter, hydrate from GET response (default false), include in save-spread, include in reset fallback, expose via context value + save-useCallback deps. - AdminConfigSettingsSection.tsx: add prop pair and a checkbox-style toggle below the ModernDeployment harmless-IDs input. Copy explains the 5 index tables + the 2h safety-net timer + warns that it should only be flipped after release-gate verification. - SectionGlobalSettings.tsx: destructure + forward the new prop pair. Round-trip is safe: GET (System.Text.Json, camelCase) emits enableIndexDualWrite, UI spreads it back into PUT body, Newtonsoft case-insensitive deserialization binds to EnableIndexDualWrite on the model — so an operator save no longer silently resets the flag. Verification: npx tsc --noEmit clean, vitest suite 168/168 green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fd7bf0e7ac |
feat(backend): M5.d.4 index-reconcile timer + release-gate — M5 complete
Closes the M5.d dual-write loop with a 2h safety-net timer and the
per-index-table PK/RK consistency release-gate tests required by
Plan §M5-Pflicht.
Components:
- ISignalRepository.QueryByTimestampAtOrAfterAsync +
IDecisionTransitionRepository.QueryByTimestampAtOrAfterAsync: new
cross-partition scans filtered on the Azure Tables system-managed
`Timestamp` property. Bounded to maxResults=50000 with a warn-log
on cap-reached so silent truncation can't masquerade as success.
- IndexReconcileTimer: TimerTrigger on "0 0 */2 * * *" (same cadence
as MaintenanceFunction). Scans the last 4h (window deliberately 2×
the cadence so every primary row gets re-checked at least twice
before falling out of range), builds envelopes via the existing
IndexReconcileEnvelopeFactory, and hands them to the producer —
reusing the M5.d.2 flag-gate + queue-failure-swallow semantics
instead of duplicating them.
- Testable core: RunReconcileAsync(cutoffUtc, CT) split out from the
TimerTrigger entry so unit tests can drive with deterministic
cutoffs without an Azure Functions host.
- AdminConfigurationService.GetConfigurationAsync is now virtual —
enables Moq-based mocking in timer flag-gate tests. Low-risk prod
change (no existing caller relies on final).
Release gate (Plan §M5-Pflicht line 1395): IndexDualWriteConsistencyTests
verifies per-index-table that the captured index row's back-ref
fields (TenantId, SessionId, StepIndex / SessionSignalOrdinal) plug
directly into the primary repos' key builders to reconstruct the
primary (PartitionKey, RowKey). The resubmit-idempotency test simulates
a timer-triggered replay of the same primaries and asserts all
duplicated rows collapse onto identical Index (PK, RK) pairs — which
is exactly what the real TableIndexRepository's UpsertReplace does
at the storage layer.
Left deferred (not blocking M5 close):
- SessionsByStage stale-row pruning: stage-X→Y transitions still leave
an orphan row in {tenantId}_X. Either query-side dedupe (per
SessionId, max-Ticks RK) or a follow-up pruner handles it.
- Actual EnableIndexDualWrite=true flip: stays default-false until
post-deploy verification runs. Producer + timer both no-op in the
meantime, so the change is zero-risk on merge.
- Cross-partition-scan perf: fine for current primary-table sizes;
revisit if prod metrics after pilot show degradation.
Tests (+10 in Functions):
- 4 IndexReconcileTimerTests: flag-off short-circuits queries, flag-on
with primary rows enqueues all envelopes, empty-window skips enqueue,
ReconcileWindow=4h invariant.
- 6 IndexDualWriteConsistencyTests: 5 per-index-table PK/RK back-ref
(SignalsByKind + all 4 transition-derived indices) + 1 resubmit
idempotency ensuring all duplicated rows collapse onto the same
index keys.
M5 status: M5.a + M5.b (+ 3 sub-commits) + M5.c (+ 3 sub-commits) +
M5.d (+ 4 sub-commits) all complete. M5.d loop:
Primary commit (M5.a) → Envelope enqueue (M5.d.2, flag-gated)
→ Queue → Handler (M5.d.3) → Index-Tables (M5.d.1)
+ 2h safety-net Timer (M5.d.4) reconciles missed envelopes.
Full Functions suite: 692 green (prev 682 + 10 new). Full solution
build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
8e39299da5 |
feat(backend): M5.d.3 queue-triggered index reconcile consumer
Consumes the telemetry-index-reconcile queue (M5.d.2) and fans each
envelope out into 0–3 index-table rows via IIndexTableRepository
(M5.d.1). Closes the producer→consumer loop; once the
EnableIndexDualWrite flag is flipped, primary commits drive
eventual-consistent index upserts asynchronously.
Components:
- Microsoft.Azure.Functions.Worker.Extensions.Storage.Queues 5.5.4
package for the isolated-worker QueueTrigger binding.
- IndexReconcileHandler (plain class — not interface-abstracted per
Plan §M5.d design: DAL abstraction lives at IIndexTableRepository,
not here). Routing:
Signal → SignalsByKind (always, 1 row).
DecisionTransition:
IsTerminal=true → SessionsByTerminal
Taken=true → SessionsByStage
Taken=false + reason → DeadEndsByReason
ClassifierVerdictId!=null → ClassifierVerdictsByIdLevel
Up to 3 rows per DecisionTransition envelope (a Taken+terminal+
classifier transition fans to SessionsByTerminal + SessionsByStage
+ ClassifierVerdictsByIdLevel).
- IndexReconcileFunction thin QueueTrigger wrapper
(Connection = AzureTableStorageConnectionString). Malformed JSON
logged + dropped to avoid poison-queue retry loops; transient
repo errors propagate so the platform retries.
- DI: handler registered as Singleton in Program.cs.
Failure semantics:
- Required-field-missing envelopes (Signal without SessionSignalOrdinal
or SignalKind; DecisionTransition without StepIndex) → log warn,
drop. These won't become valid on retry — avoiding the retry loop.
- Transient Table Storage errors → rethrow. QueueTrigger platform
retries up to 5 times by default, then moves to
telemetry-index-reconcile-poison for manual investigation.
Deferred to M5.d.4:
- SessionsByStage stale-row pruning: when a session moves from
Stage-X→Stage-Y, the old Stage-X row stays. Either query-side
dedupe (per SessionId, max-Ticks RK) or reconcile-timer pruning
handles this. Deliberately not solved in the handler to avoid
a query+delete round-trip per Taken transition.
- End-to-end dual-write verification tests (Azurite harness) are
the M5.d.4 release gate before the flag is flipped.
Tests (+10 in Functions) via capture-only FakeIndexTableRepository:
- Signal happy path (1 SignalsByKind row).
- Signal missing SessionSignalOrdinal / SignalKind → 0 rows (x2).
- DecisionTransition Terminal+Taken+Classifier → 3 rows.
- DecisionTransition Taken non-terminal no-classifier → 1 row
(SessionsByStage only).
- DecisionTransition dead-end with classifier → 2 rows
(DeadEndsByReason + ClassifierVerdictsByIdLevel).
- DecisionTransition dead-end without reason → 0 rows (PK-collision
defense).
- DecisionTransition missing StepIndex → 0 rows (all paths gated).
- Unknown SourceKind → 0 rows.
- Null envelope → 0 rows.
Full Functions suite: 682 green (prev 672 + 10 new). Full solution
build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
c3ee0cad55 |
feat(backend): M5.d.2 index-reconcile queue producer + feature flag
Adds the async fan-out producer path from Plan §2.8 / §M5.d so primary Signals + DecisionTransitions rows can be projected onto their index tables without blocking the hot ingest loop. Components: - Constants.QueueNames.TelemetryIndexReconcile (azure-queue-name-safe, all lowercase + hyphens). - IndexReconcileEnvelope (Shared DTO, flat shape with nullable fields per SourceKind="Signal"/"DecisionTransition"). Avoids polymorphic Newtonsoft deserialization — see M5.c.3 finding. - IndexReconcileEnvelopeFactory pure static helper (FromSignal / FromDecisionTransition / BuildBatch) — state-less projection. - IIndexReconcileProducer (Shared interface) contract: no-op when the feature flag is off; never rethrows underlying-queue failures. - AzureQueueIndexReconcileProducer (Functions) — Base64-encoded JSON messages, lazy CreateIfNotExistsAsync guarded by Interlocked flag, Managed-Identity-or-connection-string construction mirroring BlobStorageService / TableStorageService. - IngestTelemetryFunction ctor gains IIndexReconcileProducer; after primary Signals + DecisionTransitions commit it builds envelopes and enqueues them. Event path is untouched (events don't feed the index tables per §2.8). - Program.cs registers the producer as a singleton. Feature gate (AdminConfiguration.EnableIndexDualWrite, default false): - New bool field in AdminConfiguration. - TableConfigRepository Store + Map extension (per feedback_table_storage_serialization). - Producer checks config each call via AdminConfigurationService's 5-min memory cache. Flag off → primary behaviour is bit-exact. Fault tolerance: queue-side exceptions (network, throttling, missing queue before first CreateIfNotExists) are logged per envelope and swallowed. The primary row is the source of truth; the 2h reconcile timer (M5.d.4) is the safety-net recovery path. Tests (+11 in Functions): - 5 IndexReconcileEnvelopeFactoryTests: Signal/DecisionTransition projection, dead-end preservation, batch ordering, null/empty. - 3 IndexReconcileEnvelopeSerializationTests: Newtonsoft round-trip (Signal, full DecisionTransition, minimal-envelope tolerance). - 3 AdminConfigurationEnableIndexDualWriteTests: default-false on new config + CreateDefault, persists when set. Deferred to later sub-milestones: - M5.d.3: queue-triggered consumer that maps envelopes to 0–3 index rows using M5.d.1 TableIndexRepository. - M5.d.4: 2h reconcile timer + release-gate dual-write verification tests, then flip EnableIndexDualWrite=true in prod admin config. Full Functions suite: 672 green (prev 661 + 11 new). Full solution build green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |