mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
Oliver Kieselbach
fd7bf0e7ac
Closes the M5.d dual-write loop with a 2h safety-net timer and the
per-index-table PK/RK consistency release-gate tests required by
Plan §M5-Pflicht.
Components:
- ISignalRepository.QueryByTimestampAtOrAfterAsync +
IDecisionTransitionRepository.QueryByTimestampAtOrAfterAsync: new
cross-partition scans filtered on the Azure Tables system-managed
`Timestamp` property. Bounded to maxResults=50000 with a warn-log
on cap-reached so silent truncation can't masquerade as success.
- IndexReconcileTimer: TimerTrigger on "0 0 */2 * * *" (same cadence
as MaintenanceFunction). Scans the last 4h (window deliberately 2×
the cadence so every primary row gets re-checked at least twice
before falling out of range), builds envelopes via the existing
IndexReconcileEnvelopeFactory, and hands them to the producer —
reusing the M5.d.2 flag-gate + queue-failure-swallow semantics
instead of duplicating them.
- Testable core: RunReconcileAsync(cutoffUtc, CT) split out from the
TimerTrigger entry so unit tests can drive with deterministic
cutoffs without an Azure Functions host.
- AdminConfigurationService.GetConfigurationAsync is now virtual —
enables Moq-based mocking in timer flag-gate tests. Low-risk prod
change (no existing caller relies on final).
Release gate (Plan §M5-Pflicht line 1395): IndexDualWriteConsistencyTests
verifies per-index-table that the captured index row's back-ref
fields (TenantId, SessionId, StepIndex / SessionSignalOrdinal) plug
directly into the primary repos' key builders to reconstruct the
primary (PartitionKey, RowKey). The resubmit-idempotency test simulates
a timer-triggered replay of the same primaries and asserts all
duplicated rows collapse onto identical Index (PK, RK) pairs — which
is exactly what the real TableIndexRepository's UpsertReplace does
at the storage layer.
Left deferred (not blocking M5 close):
- SessionsByStage stale-row pruning: stage-X→Y transitions still leave
an orphan row in {tenantId}_X. Either query-side dedupe (per
SessionId, max-Ticks RK) or a follow-up pruner handles it.
- Actual EnableIndexDualWrite=true flip: stays default-false until
post-deploy verification runs. Producer + timer both no-op in the
meantime, so the change is zero-risk on merge.
- Cross-partition-scan perf: fine for current primary-table sizes;
revisit if prod metrics after pilot show degradation.
Tests (+10 in Functions):
- 4 IndexReconcileTimerTests: flag-off short-circuits queries, flag-on
with primary rows enqueues all envelopes, empty-window skips enqueue,
ReconcileWindow=4h invariant.
- 6 IndexDualWriteConsistencyTests: 5 per-index-table PK/RK back-ref
(SignalsByKind + all 4 transition-derived indices) + 1 resubmit
idempotency ensuring all duplicated rows collapse onto the same
index keys.
M5 status: M5.a + M5.b (+ 3 sub-commits) + M5.c (+ 3 sub-commits) +
M5.d (+ 4 sub-commits) all complete. M5.d loop:
Primary commit (M5.a) → Envelope enqueue (M5.d.2, flag-gated)
→ Queue → Handler (M5.d.3) → Index-Tables (M5.d.1)
+ 2h safety-net Timer (M5.d.4) reconciles missed envelopes.
Full Functions suite: 692 green (prev 682 + 10 new). Full solution
build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
49 lines
2.3 KiB
C#
49 lines
2.3 KiB
C#
using System;
|
|
using System.Collections.Generic;
|
|
using System.Threading;
|
|
using System.Threading.Tasks;
|
|
using AutopilotMonitor.Shared.Models;
|
|
|
|
namespace AutopilotMonitor.Shared.DataAccess
|
|
{
|
|
/// <summary>
|
|
/// Repository for the <c>Signals</c> primary table (Plan §M5). Writes projected
|
|
/// <see cref="SignalRecord"/>s derived from agent-emitted DecisionSignals.
|
|
/// <para>
|
|
/// Writes are idempotent via <c>(PartitionKey, RowKey)</c>: a retried batch replays
|
|
/// without duplicating rows. Records within a single batch must share the same
|
|
/// (TenantId, SessionId) — the repository groups/chunks per the Azure Tables 100-op
|
|
/// transaction limit.
|
|
/// </para>
|
|
/// </summary>
|
|
public interface ISignalRepository
|
|
{
|
|
/// <summary>
|
|
/// Upserts a batch of signal records. Returns the number of rows committed.
|
|
/// </summary>
|
|
Task<int> StoreBatchAsync(IReadOnlyList<SignalRecord> records, CancellationToken cancellationToken = default);
|
|
|
|
/// <summary>
|
|
/// Reads up to <paramref name="maxResults"/> signal records for a single session,
|
|
/// ordered by <see cref="SignalRecord.SessionSignalOrdinal"/> ascending. Used by the
|
|
/// Inspector read endpoint (<c>GET /api/sessions/{id}/signals</c>, Plan §M5).
|
|
/// </summary>
|
|
Task<List<SignalRecord>> QueryBySessionAsync(
|
|
string tenantId, string sessionId, int maxResults = 1000, CancellationToken cancellationToken = default);
|
|
|
|
/// <summary>
|
|
/// Cross-partition scan of all signals whose Azure Tables server-side <c>Timestamp</c>
|
|
/// is at or after <paramref name="cutoffUtc"/>. Used by the
|
|
/// <c>IndexReconcileTimer</c> (Plan §M5.d.4) to re-enqueue the last 4h of primary
|
|
/// rows as a safety-net against queue failures.
|
|
/// <para>
|
|
/// <b>Perf note:</b> Azure Tables has no PK-bound filter here, so this is a scan;
|
|
/// <paramref name="maxResults"/> bounds memory. If the cap is reached the caller
|
|
/// should log + narrow the window rather than trust completeness.
|
|
/// </para>
|
|
/// </summary>
|
|
Task<List<SignalRecord>> QueryByTimestampAtOrAfterAsync(
|
|
DateTime cutoffUtc, int maxResults = 50_000, CancellationToken cancellationToken = default);
|
|
}
|
|
}
|