mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
perf(maintenance): one projected 35d Sessions scan shared by all rolling sweeps
The cross-tenant StartedAt-only date-range query is a full-table drain in Table Storage; time attribution, device journeys, verdict calibration and the calibration radar's tenant discovery each ran their own — four identical reads per 2h tick. RunAllAsync / RunManualAsync now load the window once through GetMaintenanceWindowSessionsAsync (MaintenanceSweepSessionProjection: the union of the sweeps' columns, drops FailureSnapshotJson/geo/OS/diagnostics) and the sweeps slice it by StartedAt. MaintenanceSweepProjectionEquivalenceTests pins that every consumed field survives the projection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -78,7 +78,8 @@ Durations shown for attempts are the sessions' authoritative `DurationSeconds` v
|
||||
completed journeys, first-time-right count, attempt histogram, junk-serial exclusion
|
||||
count, `JourneyVersion`. A journey buckets on the StartedAt date of its completing
|
||||
success session (the platform's StartedAt-date convention; late terminals converge via
|
||||
the sweep). All counts are **additive**, so a window rate is the ratio of summed daily
|
||||
the sweep, which slices the tick's shared projected window scan — see
|
||||
`time-attribution.md`). All counts are **additive**, so a window rate is the ratio of summed daily
|
||||
rows — there is no rolling-window row (unlike the median-based time-attribution
|
||||
aggregates, which cannot merge). 180-day retention. Rows exist below the ≥20 UI gate
|
||||
on purpose (the UI needs the n for "insufficient data (n=…)").
|
||||
|
||||
@@ -105,6 +105,11 @@ the partition).
|
||||
* Self-healing 30-day maintenance sweep (2h tick): backfills missing breakdowns, folds
|
||||
late-terminating sessions into their StartedAt-date aggregates, and enforces version
|
||||
purity — a stale-version row whose events aged out counts as missing, never mixed.
|
||||
Reads the tick's **shared window scan** (`MaintenanceService.LoadSweepWindowSessionsAsync`,
|
||||
one projected 35-day cross-tenant drain via `MaintenanceSweepSessionProjection` that
|
||||
also feeds the device-journey and verdict-calibration sweeps) — the StartedAt-only
|
||||
filter is a full-table read in Table Storage, so it is done once per tick, not once per
|
||||
sweep; `MaintenanceSweepProjectionEquivalenceTests` pins the column set.
|
||||
* `TimeAttributionAggregates` (PK = TenantId or `global`, RK = `{yyyy-MM-dd}|{class}` and
|
||||
`rolling30|{class}`): per enrollment class (`user_driven`, `whiteglove`, `self_deploying`,
|
||||
`device_preparation` — never mixed) median/p75/p90 per segment over the fixed six-segment
|
||||
|
||||
@@ -84,7 +84,10 @@ Buckets[], ComputedAt }` with one bucket per (VerdictPath, Status):
|
||||
|
||||
`MaintenanceService.SweepVerdictCalibrationAsync` (2-hourly, after the device-journey sweep
|
||||
so the chains are fresh; manual maintenance parity; kill switch
|
||||
`VerdictCalibrationSweepDisabled`) recomputes the rolling 30-day window whole (Replace),
|
||||
`VerdictCalibrationSweepDisabled`) slices the tick's shared projected window scan
|
||||
(`LoadSweepWindowSessionsAsync` — one 35-day cross-tenant drain feeds the time-attribution,
|
||||
device-journey and calibration sweeps plus the radar's tenant discovery; the StartedAt-only
|
||||
filter is a full-table read, so never per sweep), recomputes the rolling 30-day window whole (Replace),
|
||||
deletes stale date rows it did not regenerate, and the unbounded-tables cleanup applies the
|
||||
180-day retention. Sessions inside a deletion cascade are skipped; the table is regenerable
|
||||
and is wiped with the tenant on offboarding.
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
## 2026-08-23
|
||||
|
||||
* **Update**: `backend/time-attribution.md`, `backend/device-journeys.md`, `backend/verdict-calibration.md` — the rolling maintenance sweeps (time attribution, device journeys, verdict calibration, calibration-radar discovery) now share ONE projected 35-day cross-tenant Sessions scan per tick (`LoadSweepWindowSessionsAsync` + `MaintenanceSweepSessionProjection`, pinned by `MaintenanceSweepProjectionEquivalenceTests`) instead of four full-table drains; each sweep slices its own window by StartedAt.
|
||||
* **New**: `backend/verdict-calibration.md` — the classifier thermometer: every status write now stamps `VerdictPath` (required parameter on `UpdateSessionStatusAsync`; `EnrollmentTimeoutClassifier` returns its rule id), overriding a verdict preserves `PriorStatus`/`PriorVerdictPath`, `VerdictPathDerivation` attributes pre-instrumentation rows read-side (`legacy:*` when ambiguous). New daily aggregate `VerdictCalibrationAggregates` (per-path count/derived/eligible7d/reEnrolled7d/overridden-by-admin|late|other, sweep after the device-journey pass, 180d retention), `GET global/metrics/verdict-calibration`, MCP `get_verdict_calibration`, admin metrics section, and the `VerdictCalibrationRadar` (share / silence-share / r6 evidence-gap episodes in the tracker keyspace `verdictcalibration|`, `VerdictCalibrationDrift` ops event, no tenant bell).
|
||||
* **New**: `backend/silent-session-classification.md` — one place for the silent-session verdict: `EnrollmentTimeoutClassifier` rule order incl. the new rule 1c `IsSelfDeployingProvisioned` (self-deploying profile + Device Setup all-succeeded + no explicit failure ⇒ Succeeded reconcile, no grace/AwaitingUser — kiosk tenant aebdce78 audit: 836 finished devices parked AwaitingUser/Incomplete), the sweep stage-1 gates (`TryMarkWhiteGloveAwaitingUserAsync`, new `TryReconcileSelfDeployingAsync`), the ingest callers (max-lifetime, late-telemetry heal, stall probe) and the new admin retro mode `self_deploying_silent` (`GetSelfDeployingSilentSessionsAsync`). Agent companion: `SessionRegistrationHelper` waits up to 15s for a network link and retries 6× (~62s) instead of 5× (~30s) so the reboot relaunch on Wi-Fi no longer exits 7 before the link is back.
|
||||
* **New**: `agent/diagnostics-package.md` — the diagnostics ZIP's built-in collection is now ONE catalog in Shared (`DiagnosticsBuiltInSections.All`): the agent iterates it in order and the backend serves the same list to the portal (`GET diagnostics/paths`, MemberRead), so what administrators see is exactly what the agent collects. New conditional sections: RealmJoin client/package/Chocolatey/per-user logs behind the tenant's RealmJoin Watcher toggle (mirrored onto `AgentConfiguration.EnableRealmJoinWatcher` by the merger) and the Device Preparation bootstrapper event log behind the deterministic WDP marker (probe once per build, fail toward Classic); every skip is a manifest line. Built-ins bypass `DiagnosticsPathGuards` by design (reviewed code, `%ProgramData%\RealmJoin` is not allowlisted); caps/reparse/evtx export still apply. `MergeDiagnosticsLogPaths` dedups global ∪ tenant paths. Portal: collapsed read-only catalog + one-line rows in both cards, tenant context dropped the GA-only global fetch.
|
||||
|
||||
+166
@@ -0,0 +1,166 @@
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using Azure.Data.Tables;
|
||||
using AutopilotMonitor.Functions.Helpers;
|
||||
using AutopilotMonitor.Functions.Services;
|
||||
using AutopilotMonitor.Shared.Models;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using Moq;
|
||||
using Xunit;
|
||||
|
||||
namespace AutopilotMonitor.Functions.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Pins that the ONE projected window scan the maintenance tick shares across its rolling
|
||||
/// sweeps (<c>GetMaintenanceWindowSessionsAsync</c> / <c>MaintenanceSweepSessionProjection</c>)
|
||||
/// carries every field the time-attribution, device-journey and verdict-calibration sweeps
|
||||
/// read. In Azure Table Storage a <c>$select</c> returns ONLY the listed properties, so the
|
||||
/// projected row here is built from the production array — dropping a column there fails the
|
||||
/// sweep that consumes it. Mirrors <see cref="UsageMetricsProjectionEquivalenceTests"/>.
|
||||
/// </summary>
|
||||
public class MaintenanceSweepProjectionEquivalenceTests
|
||||
{
|
||||
private const string TenantId = "00000000-0000-0000-0000-000000000abc";
|
||||
private static readonly DateTime Started = new(2026, 8, 10, 9, 0, 0, DateTimeKind.Utc);
|
||||
|
||||
private static readonly TableStorageService Sut =
|
||||
new(new Mock<TableServiceClient>().Object, NullLogger<TableStorageService>.Instance);
|
||||
|
||||
/// <summary>Full wide Sessions row with every column the sweeps read set to a distinct non-default value, plus noise.</summary>
|
||||
private static TableEntity FullRow() => new(TenantId, "11111111-1111-1111-1111-111111111111")
|
||||
{
|
||||
["Status"] = "Succeeded",
|
||||
["StartedAt"] = new DateTimeOffset(Started),
|
||||
["CompletedAt"] = new DateTimeOffset(Started.AddHours(1)),
|
||||
["LastEventAt"] = new DateTimeOffset(Started.AddHours(2)),
|
||||
["DurationSeconds"] = 3600,
|
||||
["EventCount"] = 321,
|
||||
["DeletionState"] = "None",
|
||||
["SerialNumber"] = "SN-1234",
|
||||
["Manufacturer"] = "Contoso",
|
||||
["Model"] = "Model-X",
|
||||
["EnrollmentType"] = "v2",
|
||||
["IsPreProvisioned"] = true,
|
||||
["IsSelfDeployingProfile"] = true,
|
||||
["IsUserDriven"] = false,
|
||||
["ResumedAt"] = new DateTimeOffset(Started.AddMinutes(30)),
|
||||
["AdminMarkedAction"] = "Succeeded",
|
||||
["VerdictPath"] = "manual:succeeded",
|
||||
["PriorStatus"] = "Incomplete",
|
||||
["PriorVerdictPath"] = "sweep:r6",
|
||||
["FailureReason"] = "No Device Setup completion or explicit failure signal observed before timeout",
|
||||
["ReconcileReason"] = "Reconciled at timeout: user completed setup (desktop + Windows Hello)",
|
||||
["FailureSource"] = "manual",
|
||||
["EspSoftFailure"] = true,
|
||||
// Noise the sweeps never read — present on the wide row, dropped by the projection.
|
||||
["DeviceName"] = "PC-FULL",
|
||||
["OsName"] = "Windows 11",
|
||||
["OsBuild"] = "26200.1234",
|
||||
["GeoCountry"] = "DE",
|
||||
["AgentVersion"] = "2.9.0",
|
||||
["DiagnosticsBlobName"] = "diag.zip",
|
||||
["FailureSnapshotJson"] = "{\"big\":\"" + new string('x', 2000) + "\"}",
|
||||
};
|
||||
|
||||
private static TableEntity Project(TableEntity full)
|
||||
{
|
||||
var keep = new HashSet<string>(TableStorageService.MaintenanceSweepSessionProjection, StringComparer.Ordinal);
|
||||
var projected = new TableEntity(full.PartitionKey, full.RowKey);
|
||||
foreach (var kv in full)
|
||||
{
|
||||
if (keep.Contains(kv.Key)) projected[kv.Key] = kv.Value;
|
||||
}
|
||||
return projected;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Projection_drops_the_wide_columns()
|
||||
{
|
||||
var projected = Project(FullRow());
|
||||
Assert.False(projected.ContainsKey("FailureSnapshotJson"));
|
||||
Assert.False(projected.ContainsKey("GeoCountry"));
|
||||
Assert.False(projected.ContainsKey("OsBuild"));
|
||||
Assert.True(projected.Count < FullRow().Count);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Every_field_the_sweeps_read_survives_the_projection()
|
||||
{
|
||||
var full = Sut.MapToSessionSummary(FullRow());
|
||||
var proj = Sut.MapToSessionSummary(Project(FullRow()));
|
||||
|
||||
// Shared / structural
|
||||
Assert.Equal(full.TenantId, proj.TenantId);
|
||||
Assert.Equal(full.SessionId, proj.SessionId);
|
||||
Assert.Equal(full.Status, proj.Status);
|
||||
Assert.Equal(full.StartedAt, proj.StartedAt);
|
||||
Assert.Equal(full.CompletedAt, proj.CompletedAt);
|
||||
Assert.Equal(full.DeletionState, proj.DeletionState);
|
||||
|
||||
// Time attribution: Status/DurationSeconds/DeletionState/EventCount + enrollment class
|
||||
Assert.Equal(full.DurationSeconds, proj.DurationSeconds);
|
||||
Assert.Equal(full.EventCount, proj.EventCount);
|
||||
Assert.Equal(TimeAttributionCalculator.GetEnrollmentClass(full), TimeAttributionCalculator.GetEnrollmentClass(proj));
|
||||
|
||||
// Device journeys: BuildSessionRef + BuildDeviceHistoryRow display fields
|
||||
Assert.Equal(full.SerialNumber, proj.SerialNumber);
|
||||
Assert.Equal(full.Manufacturer, proj.Manufacturer);
|
||||
Assert.Equal(full.Model, proj.Model);
|
||||
Assert.Equal(full.EnrollmentType, proj.EnrollmentType);
|
||||
Assert.Equal(full.IsPreProvisioned, proj.IsPreProvisioned);
|
||||
Assert.Equal(full.AdminMarkedAction, proj.AdminMarkedAction);
|
||||
var refFull = DeviceJourneyCalculator.BuildSessionRef(full)!;
|
||||
var refProj = DeviceJourneyCalculator.BuildSessionRef(proj)!;
|
||||
Assert.Equal(System.Text.Json.JsonSerializer.Serialize(refFull), System.Text.Json.JsonSerializer.Serialize(refProj));
|
||||
|
||||
// Verdict calibration: attribution + legacy derivation inputs + proxy timestamps
|
||||
Assert.Equal(full.LastEventAt, proj.LastEventAt);
|
||||
Assert.Equal(full.VerdictPath, proj.VerdictPath);
|
||||
Assert.Equal(full.PriorStatus, proj.PriorStatus);
|
||||
Assert.Equal(full.PriorVerdictPath, proj.PriorVerdictPath);
|
||||
Assert.Equal(full.FailureReason, proj.FailureReason);
|
||||
Assert.Equal(full.ReconcileReason, proj.ReconcileReason);
|
||||
Assert.Equal(full.FailureSource, proj.FailureSource);
|
||||
Assert.Equal(full.EspSoftFailure, proj.EspSoftFailure);
|
||||
Assert.Equal(VerdictPathDerivation.Derive(full), VerdictPathDerivation.Derive(proj));
|
||||
// Unstamped variant exercises the derivation on the projected inputs.
|
||||
var unstampedFull = FullRow(); unstampedFull.Remove("VerdictPath"); unstampedFull.Remove("AdminMarkedAction"); unstampedFull.Remove("FailureSource");
|
||||
Assert.Equal(VerdictPathDerivation.Derive(Sut.MapToSessionSummary(unstampedFull)),
|
||||
VerdictPathDerivation.Derive(Sut.MapToSessionSummary(Project(unstampedFull))));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Aggregation_cores_produce_identical_rows_from_the_projected_scan()
|
||||
{
|
||||
var now = new DateTime(2026, 8, 23, 12, 0, 0, DateTimeKind.Utc);
|
||||
var fullList = new List<SessionSummary> { Sut.MapToSessionSummary(FullRow()) };
|
||||
var projList = new List<SessionSummary> { Sut.MapToSessionSummary(Project(FullRow())) };
|
||||
var histories = new Dictionary<string, List<DeviceHistory>>();
|
||||
|
||||
var a = MaintenanceService.BuildVerdictCalibrationAggregates(fullList, histories, now);
|
||||
var b = MaintenanceService.BuildVerdictCalibrationAggregates(projList, histories, now);
|
||||
Assert.Equal(System.Text.Json.JsonSerializer.Serialize(a), System.Text.Json.JsonSerializer.Serialize(b));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void SliceSweepWindow_keeps_sessions_from_the_window_start()
|
||||
{
|
||||
var window = new List<SessionSummary>
|
||||
{
|
||||
new() { SessionId = "old", StartedAt = Started.AddDays(-40) },
|
||||
new() { SessionId = "edge", StartedAt = Started },
|
||||
new() { SessionId = "new", StartedAt = Started.AddDays(1) },
|
||||
};
|
||||
var sliced = MaintenanceService.SliceSweepWindow(window, Started);
|
||||
Assert.Equal(new[] { "edge", "new" }, sliced.Select(s => s.SessionId));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Shared_window_covers_every_sweeps_horizon()
|
||||
{
|
||||
// 7d window + 28d baseline anchored on yesterday = 35 days back ≥ the 30-day rolling sweeps.
|
||||
Assert.Equal(35, MaintenanceService.SweepWindowDays);
|
||||
Assert.True(MaintenanceService.SweepWindowDays >= 30);
|
||||
}
|
||||
}
|
||||
@@ -112,6 +112,7 @@ public class OffboardingSessionEnumeratorTests
|
||||
public Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime) => throw new NotImplementedException();
|
||||
public Task<List<SessionSummary>> GetLegacyTimeoutFailedSessionsAsync(string tenantId, int maxResults) => throw new NotImplementedException();
|
||||
|
||||
+3
@@ -60,6 +60,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
|
||||
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
|
||||
=> _storage.GetUsageWindowSessionsAsync(startDate, endDate, tenantId);
|
||||
|
||||
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate)
|
||||
=> _storage.GetMaintenanceWindowSessionsAsync(startDate, endDate);
|
||||
|
||||
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
|
||||
=> _storage.GetGeoWindowSessionsAsync(startDate, endDate, tenantId);
|
||||
|
||||
|
||||
@@ -289,14 +289,13 @@ namespace AutopilotMonitor.Functions.Services
|
||||
/// Sessions with a deletion cascade in flight are skipped so the sweep cannot write a
|
||||
/// breakdown row after the session's deletion manifest was snapshotted.
|
||||
/// </summary>
|
||||
private async Task SweepTimeAttributionAsync()
|
||||
private async Task SweepTimeAttributionAsync(IReadOnlyList<SessionSummary> sweepWindow)
|
||||
{
|
||||
try
|
||||
{
|
||||
var sw = Stopwatch.StartNew();
|
||||
var today = DateTime.UtcNow.Date;
|
||||
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(
|
||||
today.AddDays(-TimeAttributionSweepDays), today.AddDays(1));
|
||||
var sessions = SliceSweepWindow(sweepWindow, today.AddDays(-TimeAttributionSweepDays));
|
||||
|
||||
var pairs = new List<(SessionSummary Session, SessionTimeBreakdown Breakdown)>();
|
||||
var missing = new List<SessionSummary>();
|
||||
|
||||
+2
-2
@@ -36,7 +36,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
/// per window device per tick — same order as the F1 sweep; negligible at current fleet
|
||||
/// size (the scaling lever is gating the per-device RMW on new terminal sessions).
|
||||
/// </summary>
|
||||
private async Task SweepDeviceJourneysAsync()
|
||||
private async Task SweepDeviceJourneysAsync(IReadOnlyList<SessionSummary> sweepWindow)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -84,7 +84,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
}
|
||||
|
||||
// ---- Phase 2: merge the window's terminal sessions into their device chains. ----
|
||||
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, windowEnd);
|
||||
var sessions = SliceSweepWindow(sweepWindow, windowStart);
|
||||
var excludedSessions = new List<SessionSummary>();
|
||||
var byDevice = new Dictionary<(string TenantId, string SerialKey), List<SessionSummary>>();
|
||||
|
||||
|
||||
+6
-6
@@ -29,12 +29,12 @@ namespace AutopilotMonitor.Functions.Services
|
||||
|
||||
/// <summary>
|
||||
/// Recomputes the verdict-calibration rows for the rolling window — idempotent per
|
||||
/// (tenant × StartedAt date) plus "global" mirror rows. Reads the window's sessions (one
|
||||
/// cross-partition scan, same as the F1/F2 sweeps) and each window tenant's device
|
||||
/// histories (one partition scan per tenant) for the re-enrollment proxy. Stale date rows
|
||||
/// a run did not regenerate are deleted so window sums never serve ghost counts.
|
||||
/// (tenant × StartedAt date) plus "global" mirror rows. Slices the tick's shared window
|
||||
/// scan (no own drain) and reads each window tenant's device histories (one partition
|
||||
/// scan per tenant) for the re-enrollment proxy. Stale date rows a run did not regenerate
|
||||
/// are deleted so window sums never serve ghost counts.
|
||||
/// </summary>
|
||||
private async Task SweepVerdictCalibrationAsync()
|
||||
private async Task SweepVerdictCalibrationAsync(IReadOnlyList<SessionSummary> sweepWindow)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -50,7 +50,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
var windowStart = today.AddDays(-VerdictCalibrationSweepDays);
|
||||
var windowEnd = today.AddDays(1);
|
||||
|
||||
var sessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, windowEnd);
|
||||
var sessions = SliceSweepWindow(sweepWindow, windowStart);
|
||||
|
||||
var historiesByTenant = new Dictionary<string, List<DeviceHistory>>(StringComparer.OrdinalIgnoreCase);
|
||||
foreach (var tenantId in sessions.Select(s => s.TenantId).Distinct(StringComparer.OrdinalIgnoreCase))
|
||||
|
||||
+4
-4
@@ -28,7 +28,7 @@ namespace AutopilotMonitor.Functions.Services
|
||||
/// the local ones). Anchored on <paramref name="targetDate"/> (timer path: yesterday —
|
||||
/// whole days only). Idempotent per anchor via the tracker dedup; fail-soft per partition.
|
||||
/// </summary>
|
||||
private async Task RunVerdictCalibrationRadarAsync(DateTime targetDate)
|
||||
private async Task RunVerdictCalibrationRadarAsync(DateTime targetDate, IReadOnlyList<SessionSummary> sweepWindow)
|
||||
{
|
||||
try
|
||||
{
|
||||
@@ -41,9 +41,9 @@ namespace AutopilotMonitor.Functions.Services
|
||||
var sw = Stopwatch.StartNew();
|
||||
var horizonStart = targetDate.Date.AddDays(-(VerdictCalibrationRadar.WindowDays - 1 + VerdictCalibrationRadar.BaselineDays));
|
||||
|
||||
// The global rows enumerate every partition's dates; tenants are discovered from
|
||||
// the window's sessions (the calibration sweep wrote a row per tenant it saw).
|
||||
var tenantIds = (await _maintenanceRepo.GetSessionsByDateRangeAsync(horizonStart, targetDate.Date.AddDays(1)))
|
||||
// Tenants are discovered from the tick's shared window scan (the calibration
|
||||
// sweep wrote a row per tenant it saw) — no own drain.
|
||||
var tenantIds = sweepWindow
|
||||
.Select(s => s.TenantId)
|
||||
.Where(t => !string.IsNullOrEmpty(t))
|
||||
.Distinct(StringComparer.OrdinalIgnoreCase)
|
||||
|
||||
@@ -151,20 +151,24 @@ namespace AutopilotMonitor.Functions.Services
|
||||
// App-version duration regression radar: same episode/tracker pattern over the
|
||||
// install summaries (trailing 35d horizon loaded internally). Fail-soft.
|
||||
await RunAppVersionRegressionRadarAsync();
|
||||
// ONE projected cross-tenant window scan feeds every rolling sweep below (the
|
||||
// StartedAt-only filter is a full-table drain in Table Storage — four of them per
|
||||
// tick was four times the same read). Each sweep slices its own window.
|
||||
var sweepWindow = await LoadSweepWindowSessionsAsync();
|
||||
// F1 PR2: rolling 30d breakdown backfill + daily attribution aggregates. Owns
|
||||
// its own window (NOT the snapshot-gated catch-up above) so late-terminating
|
||||
// sessions still reach their StartedAt-date's aggregate. Fail-soft internally.
|
||||
await SweepTimeAttributionAsync();
|
||||
await SweepTimeAttributionAsync(sweepWindow);
|
||||
// F2 PR4: device-history chain heal (incl. deleted-session ref cleanup) + daily
|
||||
// FTR aggregates over the same rolling window. Fail-soft internally.
|
||||
await SweepDeviceJourneysAsync();
|
||||
await SweepDeviceJourneysAsync(sweepWindow);
|
||||
// Verdict calibration: per-verdict-path daily buckets over the same rolling
|
||||
// window, AFTER the device-journey sweep so the re-enrollment proxy reads
|
||||
// freshly merged chains. Fail-soft internally.
|
||||
await SweepVerdictCalibrationAsync();
|
||||
await SweepVerdictCalibrationAsync(sweepWindow);
|
||||
// Verdict-calibration drift radar over the rows the sweep just refreshed; anchored
|
||||
// on yesterday like the rule radar (whole days only). Fail-soft.
|
||||
await RunVerdictCalibrationRadarAsync(DateTime.UtcNow.Date.AddDays(-1));
|
||||
await RunVerdictCalibrationRadarAsync(DateTime.UtcNow.Date.AddDays(-1), sweepWindow);
|
||||
// Plan §5 PR6 / §16 R14: session retention fanout extracted out of the 2h timer
|
||||
// into the dedicated 12h SessionDeletionMaintenanceFunction so cascade-lifecycle
|
||||
// work has independent cadence + kill-switch + OpsEvent watchdogs. The non-session
|
||||
@@ -206,6 +210,33 @@ namespace AutopilotMonitor.Functions.Services
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Shared sweep window: the longest horizon any rolling sweep needs — the calibration
|
||||
/// radar's 7d window + 28d baseline anchored on YESTERDAY (= 35 days back from today) —
|
||||
/// drained once per tick with the maintenance projection. Sweeps with a shorter window
|
||||
/// slice it by StartedAt (<see cref="SliceSweepWindow"/>). Fail-soft: an empty list makes
|
||||
/// every sweep a no-op rather than failing the tick.
|
||||
/// </summary>
|
||||
internal const int SweepWindowDays = Helpers.VerdictCalibrationRadar.WindowDays + Helpers.VerdictCalibrationRadar.BaselineDays;
|
||||
|
||||
private async Task<List<SessionSummary>> LoadSweepWindowSessionsAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
var today = DateTime.UtcNow.Date;
|
||||
return await _maintenanceRepo.GetMaintenanceWindowSessionsAsync(today.AddDays(-SweepWindowDays), today.AddDays(1));
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(ex, "Sweep window load failed — rolling sweeps skip this tick (non-fatal)");
|
||||
return new List<SessionSummary>();
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Sessions of the shared window that started at or after <paramref name="windowStart"/> (UTC).</summary>
|
||||
internal static List<SessionSummary> SliceSweepWindow(IReadOnlyList<SessionSummary> window, DateTime windowStart)
|
||||
=> window.Where(s => s.StartedAt >= windowStart).ToList();
|
||||
|
||||
/// <summary>
|
||||
/// Manually triggered maintenance with flexible date selection
|
||||
/// </summary>
|
||||
@@ -237,10 +268,11 @@ namespace AutopilotMonitor.Functions.Services
|
||||
// Timer-path parity: manual maintenance also refreshes the attribution
|
||||
// breakdowns + daily aggregates and the device-history/FTR rollups
|
||||
// (rolling 30d windows, cheap once converged).
|
||||
await SweepTimeAttributionAsync();
|
||||
await SweepDeviceJourneysAsync();
|
||||
await SweepVerdictCalibrationAsync();
|
||||
await RunVerdictCalibrationRadarAsync(dateToAggregate);
|
||||
var sweepWindow = await LoadSweepWindowSessionsAsync();
|
||||
await SweepTimeAttributionAsync(sweepWindow);
|
||||
await SweepDeviceJourneysAsync(sweepWindow);
|
||||
await SweepVerdictCalibrationAsync(sweepWindow);
|
||||
await RunVerdictCalibrationRadarAsync(dateToAggregate, sweepWindow);
|
||||
|
||||
if (!aggregateOnly)
|
||||
{
|
||||
|
||||
@@ -595,6 +595,36 @@ namespace AutopilotMonitor.Functions.Services
|
||||
public Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
|
||||
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId, GeoMetricsSessionProjection);
|
||||
|
||||
/// <summary>
|
||||
/// Columns the rolling maintenance sweeps consume from the SHARED window scan (one
|
||||
/// cross-tenant drain per tick feeds the time-attribution, device-journey and
|
||||
/// verdict-calibration sweeps plus the calibration radar's tenant discovery — see
|
||||
/// MaintenanceService.LoadSweepWindowSessionsAsync). Union of the three sweeps' reads:
|
||||
/// identity/device (SerialNumber, Manufacturer, Model), status + timestamps, the
|
||||
/// enrollment-class inputs, DurationSeconds/EventCount, the deletion-cascade marker,
|
||||
/// the verdict attribution (VerdictPath/Prior*) and the legacy-derivation inputs
|
||||
/// (FailureReason/ReconcileReason/FailureSource/AdminMarkedAction/EspSoftFailure).
|
||||
/// Drops the wide row's FailureSnapshotJson, geo, OS and diagnostics columns. internal so
|
||||
/// MaintenanceSweepProjectionEquivalenceTests derives its keep-set from this exact array.
|
||||
/// </summary>
|
||||
internal static readonly string[] MaintenanceSweepSessionProjection =
|
||||
{
|
||||
"PartitionKey", "RowKey", "StartedAt", "CompletedAt", "LastEventAt", "Status",
|
||||
"DurationSeconds", "EventCount", "DeletionState",
|
||||
"SerialNumber", "Manufacturer", "Model",
|
||||
"EnrollmentType", "IsPreProvisioned", "IsSelfDeployingProfile", "IsUserDriven", "ResumedAt",
|
||||
"AdminMarkedAction", "VerdictPath", "PriorStatus", "PriorVerdictPath",
|
||||
"FailureReason", "ReconcileReason", "FailureSource", "EspSoftFailure",
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// Column-projected cross-tenant date-range query backing the shared maintenance sweep
|
||||
/// window. Same filter semantics as <see cref="GetSessionsByDateRangeAsync"/>; only
|
||||
/// <see cref="MaintenanceSweepSessionProjection"/> is transferred.
|
||||
/// </summary>
|
||||
public Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate)
|
||||
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId: null, MaintenanceSweepSessionProjection);
|
||||
|
||||
private async Task<List<SessionSummary>> QuerySessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId, string[]? select)
|
||||
{
|
||||
if (!string.IsNullOrEmpty(tenantId))
|
||||
|
||||
@@ -80,6 +80,13 @@ namespace AutopilotMonitor.Shared.DataAccess
|
||||
/// Fields outside the projection come back as defaults — callers must not read them.
|
||||
/// </summary>
|
||||
Task<List<SessionSummary>> GetGeoWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null);
|
||||
/// <summary>
|
||||
/// Column-projected cross-tenant variant of <see cref="GetSessionsByDateRangeAsync"/> that
|
||||
/// the maintenance tick drains ONCE and shares across its rolling sweeps (time attribution,
|
||||
/// device journeys, verdict calibration, calibration radar discovery). Fields outside the
|
||||
/// projection come back as defaults — sweeps must not read them.
|
||||
/// </summary>
|
||||
Task<List<SessionSummary>> GetMaintenanceWindowSessionsAsync(DateTime startDate, DateTime endDate);
|
||||
Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime);
|
||||
Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff);
|
||||
|
||||
|
||||
Reference in New Issue
Block a user