perf(maintenance): one projected 35d Sessions scan shared by all rolling sweeps

The cross-tenant StartedAt-only date-range query is a full-table drain in
Table Storage; time attribution, device journeys, verdict calibration and the
calibration radar's tenant discovery each ran their own — four identical reads
per 2h tick. RunAllAsync / RunManualAsync now load the window once through
GetMaintenanceWindowSessionsAsync (MaintenanceSweepSessionProjection: the union
of the sweeps' columns, drops FailureSnapshotJson/geo/OS/diagnostics) and the
sweeps slice it by StartedAt. MaintenanceSweepProjectionEquivalenceTests pins
that every consumed field survives the projection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-08-23 12:19:41 +02:00
parent 94afc02e62
commit 7fa50d09c1
14 changed files with 273 additions and 25 deletions
+2 -1
View File
@@ -78,7 +78,8 @@ Durations shown for attempts are the sessions' authoritative `DurationSeconds` v
completed journeys, first-time-right count, attempt histogram, junk-serial exclusion
count, `JourneyVersion`. A journey buckets on the StartedAt date of its completing
success session (the platform's StartedAt-date convention; late terminals converge via
the sweep). All counts are **additive**, so a window rate is the ratio of summed daily
the sweep, which slices the tick's shared projected window scan — see
`time-attribution.md`). All counts are **additive**, so a window rate is the ratio of summed daily
rows — there is no rolling-window row (unlike the median-based time-attribution
aggregates, which cannot merge). 180-day retention. Rows exist below the ≥20 UI gate
on purpose (the UI needs the n for "insufficient data (n=…)").
+5
View File
@@ -105,6 +105,11 @@ the partition).
* Self-healing 30-day maintenance sweep (2h tick): backfills missing breakdowns, folds
late-terminating sessions into their StartedAt-date aggregates, and enforces version
purity — a stale-version row whose events aged out counts as missing, never mixed.
Reads the tick's **shared window scan** (`MaintenanceService.LoadSweepWindowSessionsAsync`,
one projected 35-day cross-tenant drain via `MaintenanceSweepSessionProjection` that
also feeds the device-journey and verdict-calibration sweeps) — the StartedAt-only
filter is a full-table read in Table Storage, so it is done once per tick, not once per
sweep; `MaintenanceSweepProjectionEquivalenceTests` pins the column set.
* `TimeAttributionAggregates` (PK = TenantId or `global`, RK = `{yyyy-MM-dd}|{class}` and
`rolling30|{class}`): per enrollment class (`user_driven`, `whiteglove`, `self_deploying`,
`device_preparation` — never mixed) median/p75/p90 per segment over the fixed six-segment
+4 -1
View File
@@ -84,7 +84,10 @@ Buckets[], ComputedAt }` with one bucket per (VerdictPath, Status):
`MaintenanceService.SweepVerdictCalibrationAsync` (2-hourly, after the device-journey sweep
so the chains are fresh; manual maintenance parity; kill switch
`VerdictCalibrationSweepDisabled`) recomputes the rolling 30-day window whole (Replace),
`VerdictCalibrationSweepDisabled`) slices the tick's shared projected window scan
(`LoadSweepWindowSessionsAsync` — one 35-day cross-tenant drain feeds the time-attribution,
device-journey and calibration sweeps plus the radar's tenant discovery; the StartedAt-only
filter is a full-table read, so never per sweep), recomputes the rolling 30-day window whole (Replace),
deletes stale date rows it did not regenerate, and the unbounded-tables cleanup applies the
180-day retention. Sessions inside a deletion cascade are skipped; the table is regenerable
and is wiped with the tenant on offboarding.
+1
View File
@@ -2,6 +2,7 @@
## 2026-08-23
* **Update**: `backend/time-attribution.md`, `backend/device-journeys.md`, `backend/verdict-calibration.md` — the rolling maintenance sweeps (time attribution, device journeys, verdict calibration, calibration-radar discovery) now share ONE projected 35-day cross-tenant Sessions scan per tick (`LoadSweepWindowSessionsAsync` + `MaintenanceSweepSessionProjection`, pinned by `MaintenanceSweepProjectionEquivalenceTests`) instead of four full-table drains; each sweep slices its own window by StartedAt.
* **New**: `backend/verdict-calibration.md` — the classifier thermometer: every status write now stamps `VerdictPath` (required parameter on `UpdateSessionStatusAsync`; `EnrollmentTimeoutClassifier` returns its rule id), overriding a verdict preserves `PriorStatus`/`PriorVerdictPath`, `VerdictPathDerivation` attributes pre-instrumentation rows read-side (`legacy:*` when ambiguous). New daily aggregate `VerdictCalibrationAggregates` (per-path count/derived/eligible7d/reEnrolled7d/overridden-by-admin|late|other, sweep after the device-journey pass, 180d retention), `GET global/metrics/verdict-calibration`, MCP `get_verdict_calibration`, admin metrics section, and the `VerdictCalibrationRadar` (share / silence-share / r6 evidence-gap episodes in the tracker keyspace `verdictcalibration|`, `VerdictCalibrationDrift` ops event, no tenant bell).
* **New**: `backend/silent-session-classification.md` — one place for the silent-session verdict: `EnrollmentTimeoutClassifier` rule order incl. the new rule 1c `IsSelfDeployingProvisioned` (self-deploying profile + Device Setup all-succeeded + no explicit failure ⇒ Succeeded reconcile, no grace/AwaitingUser — kiosk tenant aebdce78 audit: 836 finished devices parked AwaitingUser/Incomplete), the sweep stage-1 gates (`TryMarkWhiteGloveAwaitingUserAsync`, new `TryReconcileSelfDeployingAsync`), the ingest callers (max-lifetime, late-telemetry heal, stall probe) and the new admin retro mode `self_deploying_silent` (`GetSelfDeployingSilentSessionsAsync`). Agent companion: `SessionRegistrationHelper` waits up to 15s for a network link and retries 6× (~62s) instead of 5× (~30s) so the reboot relaunch on Wi-Fi no longer exits 7 before the link is back.
* **New**: `agent/diagnostics-package.md` — the diagnostics ZIP's built-in collection is now ONE catalog in Shared (`DiagnosticsBuiltInSections.All`): the agent iterates it in order and the backend serves the same list to the portal (`GET diagnostics/paths`, MemberRead), so what administrators see is exactly what the agent collects. New conditional sections: RealmJoin client/package/Chocolatey/per-user logs behind the tenant's RealmJoin Watcher toggle (mirrored onto `AgentConfiguration.EnableRealmJoinWatcher` by the merger) and the Device Preparation bootstrapper event log behind the deterministic WDP marker (probe once per build, fail toward Classic); every skip is a manifest line. Built-ins bypass `DiagnosticsPathGuards` by design (reviewed code, `%ProgramData%\RealmJoin` is not allowlisted); caps/reparse/evtx export still apply. `MergeDiagnosticsLogPaths` dedups global ∪ tenant paths. Portal: collapsed read-only catalog + one-line rows in both cards, tenant context dropped the GA-only global fetch.
@@ -0,0 +1,166 @@
using System;
using System.Collections.Generic;
using System.Linq;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins that the ONE projected window scan the maintenance tick shares across its rolling
/// sweeps (<c>GetMaintenanceWindowSessionsAsync</c> / <c>MaintenanceSweepSessionProjection</c>)
/// carries every field the time-attribution, device-journey and verdict-calibration sweeps
/// read. In Azure Table Storage a <c>$select</c> returns ONLY the listed properties, so the
/// projected row here is built from the production array — dropping a column there fails the
/// sweep that consumes it. Mirrors <see cref="UsageMetricsProjectionEquivalenceTests"/>.
/// </summary>
public class MaintenanceSweepProjectionEquivalenceTests
{
private const string TenantId = "00000000-0000-0000-0000-000000000abc";
private static readonly DateTime Started = new(2026, 8, 10, 9, 0, 0, DateTimeKind.Utc);
private static readonly TableStorageService Sut =
new(new Mock<TableServiceClient>().Object, NullLogger<TableStorageService>.Instance);
/// <summary>Full wide Sessions row with every column the sweeps read set to a distinct non-default value, plus noise.</summary>
private static TableEntity FullRow() => new(TenantId, "11111111-1111-1111-1111-111111111111")
{
["Status"] = "Succeeded",
["StartedAt"] = new DateTimeOffset(Started),
["CompletedAt"] = new DateTimeOffset(Started.AddHours(1)),
["LastEventAt"] = new DateTimeOffset(Started.AddHours(2)),
["DurationSeconds"] = 3600,
["EventCount"] = 321,
["DeletionState"] = "None",
["SerialNumber"] = "SN-1234",
["Manufacturer"] = "Contoso",
["Model"] = "Model-X",
["EnrollmentType"] = "v2",
["IsPreProvisioned"] = true,
["IsSelfDeployingProfile"] = true,
["IsUserDriven"] = false,
["ResumedAt"] = new DateTimeOffset(Started.AddMinutes(30)),
["AdminMarkedAction"] = "Succeeded",
["VerdictPath"] = "manual:succeeded",
["PriorStatus"] = "Incomplete",
["PriorVerdictPath"] = "sweep:r6",
["FailureReason"] = "No Device Setup completion or explicit failure signal observed before timeout",
["ReconcileReason"] = "Reconciled at timeout: user completed setup (desktop + Windows Hello)",
["FailureSource"] = "manual",
["EspSoftFailure"] = true,
// Noise the sweeps never read — present on the wide row, dropped by the projection.
["DeviceName"] = "PC-FULL",
["OsName"] = "Windows 11",
["OsBuild"] = "26200.1234",
["GeoCountry"] = "DE",
["AgentVersion"] = "2.9.0",
["DiagnosticsBlobName"] = "diag.zip",
["FailureSnapshotJson"] = "{\"big\":\"" + new string('x', 2000) + "\"}",
};
private static TableEntity Project(TableEntity full)
{
var keep = new HashSet<string>(TableStorageService.MaintenanceSweepSessionProjection, StringComparer.Ordinal);
var projected = new TableEntity(full.PartitionKey, full.RowKey);
foreach (var kv in full)
{
if (keep.Contains(kv.Key)) projected[kv.Key] = kv.Value;
}
return projected;
}
[Fact]
public void Projection_drops_the_wide_columns()
{
var projected = Project(FullRow());
Assert.False(projected.ContainsKey("FailureSnapshotJson"));
Assert.False(projected.ContainsKey("GeoCountry"));
Assert.False(projected.ContainsKey("OsBuild"));
Assert.True(projected.Count < FullRow().Count);
}
[Fact]
public void Every_field_the_sweeps_read_survives_the_projection()
{
var full = Sut.MapToSessionSummary(FullRow());
var proj = Sut.MapToSessionSummary(Project(FullRow()));
// Shared / structural
Assert.Equal(full.TenantId, proj.TenantId);
Assert.Equal(full.SessionId, proj.SessionId);
Assert.Equal(full.Status, proj.Status);
Assert.Equal(full.StartedAt, proj.StartedAt);
Assert.Equal(full.CompletedAt, proj.CompletedAt);
Assert.Equal(full.DeletionState, proj.DeletionState);
// Time attribution: Status/DurationSeconds/DeletionState/EventCount + enrollment class
Assert.Equal(full.DurationSeconds, proj.DurationSeconds);
Assert.Equal(full.EventCount, proj.EventCount);
Assert.Equal(TimeAttributionCalculator.GetEnrollmentClass(full), TimeAttributionCalculator.GetEnrollmentClass(proj));
// Device journeys: BuildSessionRef + BuildDeviceHistoryRow display fields
Assert.Equal(full.SerialNumber, proj.SerialNumber);
Assert.Equal(full.Manufacturer, proj.Manufacturer);
Assert.Equal(full.Model, proj.Model);
Assert.Equal(full.EnrollmentType, proj.EnrollmentType);
Assert.Equal(full.IsPreProvisioned, proj.IsPreProvisioned);
Assert.Equal(full.AdminMarkedAction, proj.AdminMarkedAction);
var refFull = DeviceJourneyCalculator.BuildSessionRef(full)!;
var refProj = DeviceJourneyCalculator.BuildSessionRef(proj)!;
Assert.Equal(System.Text.Json.JsonSerializer.Serialize(refFull), System.Text.Json.JsonSerializer.Serialize(refProj));
// Verdict calibration: attribution + legacy derivation inputs + proxy timestamps
Assert.Equal(full.LastEventAt, proj.LastEventAt);
Assert.Equal(full.VerdictPath, proj.VerdictPath);
Assert.Equal(full.PriorStatus, proj.PriorStatus);
Assert.Equal(full.PriorVerdictPath, proj.PriorVerdictPath);
Assert.Equal(full.FailureReason, proj.FailureReason);
Assert.Equal(full.ReconcileReason, proj.ReconcileReason);
Assert.Equal(full.FailureSource, proj.FailureSource);
Assert.Equal(full.EspSoftFailure, proj.EspSoftFailure);
Assert.Equal(VerdictPathDerivation.Derive(full), VerdictPathDerivation.Derive(proj));
// Unstamped variant exercises the derivation on the projected inputs.
var unstampedFull = FullRow(); unstampedFull.Remove("VerdictPath"); unstampedFull.Remove("AdminMarkedAction"); unstampedFull.Remove("FailureSource");
Assert.Equal(VerdictPathDerivation.Derive(Sut.MapToSessionSummary(unstampedFull)),
VerdictPathDerivation.Derive(Sut.MapToSessionSummary(Project(unstampedFull))));
}
[Fact]
public void Aggregation_cores_produce_identical_rows_from_the_projected_scan()
{
var now = new DateTime(2026, 8, 23, 12, 0, 0, DateTimeKind.Utc);
var fullList = new List<SessionSummary> { Sut.MapToSessionSummary(FullRow()) };
var projList = new List<SessionSummary> { Sut.MapToSessionSummary(Project(FullRow())) };
var histories = new Dictionary<string, List<DeviceHistory>>();
var a = MaintenanceService.BuildVerdictCalibrationAggregates(fullList, histories, now);
var b = MaintenanceService.BuildVerdictCalibrationAggregates(projList, histories, now);
Assert.Equal(System.Text.Json.JsonSerializer.Serialize(a), System.Text.Json.JsonSerializer.Serialize(b));
}
[Fact]
public void SliceSweepWindow_keeps_sessions_from_the_window_start()
{
var window = new List<SessionSummary>
{
new() { SessionId = "old", StartedAt = Started.AddDays(-40) },
new() { SessionId = "edge", StartedAt = Started },
new() { SessionId = "new", StartedAt = Started.AddDays(1) },
};
var sliced = MaintenanceService.SliceSweepWindow(window, Started);
Assert.Equal(new[] { "edge", "new" }, sliced.Select(s => s.SessionId));
}
[Fact]
public void Shared_window_covers_every_sweeps_horizon()
{
// 7d window + 28d baseline anchored on yesterday = 35 days back ≥ the 30-day rolling sweeps.
Assert.Equal(35, MaintenanceService.SweepWindowDays);
Assert.True(MaintenanceService.SweepWindowDays >= 30);
}
}
@@ -112,6 +112,7 @@ public class OffboardingSessionEnumeratorTests
public Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetLegacyTimeoutFailedSessionsAsync(string tenantId, int maxResults) => throw new NotImplementedException();
@@ -60,6 +60,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> _storage.GetUsageWindowSessionsAsync(startDate, endDate, tenantId);
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate)
=> _storage.GetMaintenanceWindowSessionsAsync(startDate, endDate);
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> _storage.GetGeoWindowSessionsAsync(startDate, endDate, tenantId);
@@ -289,14 +289,13 @@ namespace AutopilotMonitor.Functions.Services
/// Sessions with a deletion cascade in flight are skipped so the sweep cannot write a
/// breakdown row after the session's deletion manifest was snapshotted.
/// </summary>
private async Task SweepTimeAttributionAsync()
private async Task SweepTimeAttributionAsync(IReadOnlyList<SessionSummary> sweepWindow)
{
try
{
var sw = Stopwatch.StartNew();
var today = DateTime.UtcNow.Date;
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(
today.AddDays(-TimeAttributionSweepDays), today.AddDays(1));
var sessions = SliceSweepWindow(sweepWindow, today.AddDays(-TimeAttributionSweepDays));
var pairs = new List<(SessionSummary Session, SessionTimeBreakdown Breakdown)>();
var missing = new List<SessionSummary>();
@@ -36,7 +36,7 @@ namespace AutopilotMonitor.Functions.Services
/// per window device per tick — same order as the F1 sweep; negligible at current fleet
/// size (the scaling lever is gating the per-device RMW on new terminal sessions).
/// </summary>
private async Task SweepDeviceJourneysAsync()
private async Task SweepDeviceJourneysAsync(IReadOnlyList<SessionSummary> sweepWindow)
{
try
{
@@ -84,7 +84,7 @@ namespace AutopilotMonitor.Functions.Services
}
// ---- Phase 2: merge the window's terminal sessions into their device chains. ----
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, windowEnd);
var sessions = SliceSweepWindow(sweepWindow, windowStart);
var excludedSessions = new List<SessionSummary>();
var byDevice = new Dictionary<(string TenantId, string SerialKey), List<SessionSummary>>();
@@ -29,12 +29,12 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Recomputes the verdict-calibration rows for the rolling window — idempotent per
/// (tenant × StartedAt date) plus "global" mirror rows. Reads the window's sessions (one
/// cross-partition scan, same as the F1/F2 sweeps) and each window tenant's device
/// histories (one partition scan per tenant) for the re-enrollment proxy. Stale date rows
/// a run did not regenerate are deleted so window sums never serve ghost counts.
/// (tenant × StartedAt date) plus "global" mirror rows. Slices the tick's shared window
/// scan (no own drain) and reads each window tenant's device histories (one partition
/// scan per tenant) for the re-enrollment proxy. Stale date rows a run did not regenerate
/// are deleted so window sums never serve ghost counts.
/// </summary>
private async Task SweepVerdictCalibrationAsync()
private async Task SweepVerdictCalibrationAsync(IReadOnlyList<SessionSummary> sweepWindow)
{
try
{
@@ -50,7 +50,7 @@ namespace AutopilotMonitor.Functions.Services
var windowStart = today.AddDays(-VerdictCalibrationSweepDays);
var windowEnd = today.AddDays(1);
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, windowEnd);
var sessions = SliceSweepWindow(sweepWindow, windowStart);
var historiesByTenant = new Dictionary<string, List<DeviceHistory>>(StringComparer.OrdinalIgnoreCase);
foreach (var tenantId in sessions.Select(s => s.TenantId).Distinct(StringComparer.OrdinalIgnoreCase))
@@ -28,7 +28,7 @@ namespace AutopilotMonitor.Functions.Services
/// the local ones). Anchored on <paramref name="targetDate"/> (timer path: yesterday —
/// whole days only). Idempotent per anchor via the tracker dedup; fail-soft per partition.
/// </summary>
private async Task RunVerdictCalibrationRadarAsync(DateTime targetDate)
private async Task RunVerdictCalibrationRadarAsync(DateTime targetDate, IReadOnlyList<SessionSummary> sweepWindow)
{
try
{
@@ -41,9 +41,9 @@ namespace AutopilotMonitor.Functions.Services
var sw = Stopwatch.StartNew();
var horizonStart = targetDate.Date.AddDays(-(VerdictCalibrationRadar.WindowDays - 1 + VerdictCalibrationRadar.BaselineDays));
// The global rows enumerate every partition's dates; tenants are discovered from
// the window's sessions (the calibration sweep wrote a row per tenant it saw).
var tenantIds = (await _maintenanceRepo.GetSessionsByDateRangeAsync(horizonStart, targetDate.Date.AddDays(1)))
// Tenants are discovered from the tick's shared window scan (the calibration
// sweep wrote a row per tenant it saw) — no own drain.
var tenantIds = sweepWindow
.Select(s => s.TenantId)
.Where(t => !string.IsNullOrEmpty(t))
.Distinct(StringComparer.OrdinalIgnoreCase)
@@ -151,20 +151,24 @@ namespace AutopilotMonitor.Functions.Services
// App-version duration regression radar: same episode/tracker pattern over the
// install summaries (trailing 35d horizon loaded internally). Fail-soft.
await RunAppVersionRegressionRadarAsync();
// ONE projected cross-tenant window scan feeds every rolling sweep below (the
// StartedAt-only filter is a full-table drain in Table Storage — four of them per
// tick was four times the same read). Each sweep slices its own window.
var sweepWindow = await LoadSweepWindowSessionsAsync();
// F1 PR2: rolling 30d breakdown backfill + daily attribution aggregates. Owns
// its own window (NOT the snapshot-gated catch-up above) so late-terminating
// sessions still reach their StartedAt-date's aggregate. Fail-soft internally.
await SweepTimeAttributionAsync();
await SweepTimeAttributionAsync(sweepWindow);
// F2 PR4: device-history chain heal (incl. deleted-session ref cleanup) + daily
// FTR aggregates over the same rolling window. Fail-soft internally.
await SweepDeviceJourneysAsync();
await SweepDeviceJourneysAsync(sweepWindow);
// Verdict calibration: per-verdict-path daily buckets over the same rolling
// window, AFTER the device-journey sweep so the re-enrollment proxy reads
// freshly merged chains. Fail-soft internally.
await SweepVerdictCalibrationAsync();
await SweepVerdictCalibrationAsync(sweepWindow);
// Verdict-calibration drift radar over the rows the sweep just refreshed; anchored
// on yesterday like the rule radar (whole days only). Fail-soft.
await RunVerdictCalibrationRadarAsync(DateTime.UtcNow.Date.AddDays(-1));
await RunVerdictCalibrationRadarAsync(DateTime.UtcNow.Date.AddDays(-1), sweepWindow);
// Plan §5 PR6 / §16 R14: session retention fanout extracted out of the 2h timer
// into the dedicated 12h SessionDeletionMaintenanceFunction so cascade-lifecycle
// work has independent cadence + kill-switch + OpsEvent watchdogs. The non-session
@@ -206,6 +210,33 @@ namespace AutopilotMonitor.Functions.Services
}
}
/// <summary>
/// Shared sweep window: the longest horizon any rolling sweep needs — the calibration
/// radar's 7d window + 28d baseline anchored on YESTERDAY (= 35 days back from today) —
/// drained once per tick with the maintenance projection. Sweeps with a shorter window
/// slice it by StartedAt (<see cref="SliceSweepWindow"/>). Fail-soft: an empty list makes
/// every sweep a no-op rather than failing the tick.
/// </summary>
internal const int SweepWindowDays = Helpers.VerdictCalibrationRadar.WindowDays + Helpers.VerdictCalibrationRadar.BaselineDays;
private async Task<List<SessionSummary>> LoadSweepWindowSessionsAsync()
{
try
{
var today = DateTime.UtcNow.Date;
return await _maintenanceRepo.GetMaintenanceWindowSessionsAsync(today.AddDays(-SweepWindowDays), today.AddDays(1));
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Sweep window load failed — rolling sweeps skip this tick (non-fatal)");
return new List<SessionSummary>();
}
}
/// <summary>Sessions of the shared window that started at or after <paramref name="windowStart"/> (UTC).</summary>
internal static List<SessionSummary> SliceSweepWindow(IReadOnlyList<SessionSummary> window, DateTime windowStart)
=> window.Where(s => s.StartedAt >= windowStart).ToList();
/// <summary>
/// Manually triggered maintenance with flexible date selection
/// </summary>
@@ -237,10 +268,11 @@ namespace AutopilotMonitor.Functions.Services
// Timer-path parity: manual maintenance also refreshes the attribution
// breakdowns + daily aggregates and the device-history/FTR rollups
// (rolling 30d windows, cheap once converged).
await SweepTimeAttributionAsync();
await SweepDeviceJourneysAsync();
await SweepVerdictCalibrationAsync();
await RunVerdictCalibrationRadarAsync(dateToAggregate);
var sweepWindow = await LoadSweepWindowSessionsAsync();
await SweepTimeAttributionAsync(sweepWindow);
await SweepDeviceJourneysAsync(sweepWindow);
await SweepVerdictCalibrationAsync(sweepWindow);
await RunVerdictCalibrationRadarAsync(dateToAggregate, sweepWindow);
if (!aggregateOnly)
{
@@ -595,6 +595,36 @@ namespace AutopilotMonitor.Functions.Services
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId, GeoMetricsSessionProjection);
/// <summary>
/// Columns the rolling maintenance sweeps consume from the SHARED window scan (one
/// cross-tenant drain per tick feeds the time-attribution, device-journey and
/// verdict-calibration sweeps plus the calibration radar's tenant discovery — see
/// MaintenanceService.LoadSweepWindowSessionsAsync). Union of the three sweeps' reads:
/// identity/device (SerialNumber, Manufacturer, Model), status + timestamps, the
/// enrollment-class inputs, DurationSeconds/EventCount, the deletion-cascade marker,
/// the verdict attribution (VerdictPath/Prior*) and the legacy-derivation inputs
/// (FailureReason/ReconcileReason/FailureSource/AdminMarkedAction/EspSoftFailure).
/// Drops the wide row's FailureSnapshotJson, geo, OS and diagnostics columns. internal so
/// MaintenanceSweepProjectionEquivalenceTests derives its keep-set from this exact array.
/// </summary>
internal static readonly string[] MaintenanceSweepSessionProjection =
{
"PartitionKey", "RowKey", "StartedAt", "CompletedAt", "LastEventAt", "Status",
"DurationSeconds", "EventCount", "DeletionState",
"SerialNumber", "Manufacturer", "Model",
"EnrollmentType", "IsPreProvisioned", "IsSelfDeployingProfile", "IsUserDriven", "ResumedAt",
"AdminMarkedAction", "VerdictPath", "PriorStatus", "PriorVerdictPath",
"FailureReason", "ReconcileReason", "FailureSource", "EspSoftFailure",
};
/// <summary>
/// Column-projected cross-tenant date-range query backing the shared maintenance sweep
/// window. Same filter semantics as <see cref="GetSessionsByDateRangeAsync"/>; only
/// <see cref="MaintenanceSweepSessionProjection"/> is transferred.
/// </summary>
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate)
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId: null, MaintenanceSweepSessionProjection);
private async Task<List<SessionSummary>> QuerySessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId, string[]? select)
{
if (!string.IsNullOrEmpty(tenantId))
@@ -80,6 +80,13 @@ namespace AutopilotMonitor.Shared.DataAccess
/// Fields outside the projection come back as defaults — callers must not read them.
/// </summary>
Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null);
/// <summary>
/// Column-projected cross-tenant variant of <see cref="GetSessionsByDateRangeAsync"/> that
/// the maintenance tick drains ONCE and shares across its rolling sweeps (time attribution,
/// device journeys, verdict calibration, calibration radar discovery). Fields outside the
/// projection come back as defaults — sweeps must not read them.
/// </summary>
Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate);
Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime);
Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff);