perf(metrics): fix Platform Usage client timeout — parallel projected scans, 15min cache, single-flight, progress UX

Fresh cross-tenant usage computes took 20-43s server-side but the web app's
default 30s fetch timeout aborted them client-side (HTTP 499), and every
retry stacked another concurrent full compute.

Backend:
- UsageMetricsService: cache 5→15min, single-flight per days-window (retries
  join the in-flight compute), the four source scans run concurrently via
  Task.WhenAll, statics converted to instance fields (singleton, testable)
- New column-projected queries for the usage compute only:
  GetUsageWindowSessionsAsync ($select 13 consumed columns, drops
  FailureSnapshotJson & co.) and GetAppInstallRefsAsync ($select
  SessionId+AppName, lean SessionAppRef DTO); GetSessionsByDateRangeAsync
  is untouched for its other consumers
- Projection equivalence pinned by UsageMetricsProjectionEquivalenceTests
  (incl. the WhiteGlove Part-2 duration branch via IsPreProvisioned/
  ResumedAt); single-flight/cache/failure-eviction pinned by
  UsageMetricsServiceSingleFlightTests

Web (SectionPlatformUsage):
- 180s AbortSignal for this call (backend keeps computing and caches)
- Calculating card with elapsed time and a progress bar sized from the last
  fresh-compute duration (localStorage, cache hits excluded, capped at 95%)
- Friendly timeout error explaining the retry will hit the server cache

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Oliver Kieselbach
2026-07-11 12:10:56 +02:00
parent 860efbbd1b
commit e63a0bc2bd
15 changed files with 556 additions and 71 deletions
@@ -111,6 +111,7 @@ public class OffboardingSessionEnumeratorTests
public Task<int> DeleteAuditLogsOlderThanAsync(DateTime cutoffUtc) => throw new NotImplementedException(); public Task<int> DeleteAuditLogsOlderThanAsync(DateTime cutoffUtc) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false) => throw new NotImplementedException(); public Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException(); public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime) => throw new NotImplementedException(); public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff) => throw new NotImplementedException(); public Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff) => throw new NotImplementedException();
public Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours) => throw new NotImplementedException(); public Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours) => throw new NotImplementedException();
@@ -0,0 +1,140 @@
using System;
using System.Collections.Generic;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins that the column-projected Sessions scan driving <c>GetUsageWindowSessionsAsync</c> produces
/// identical usage-metrics inputs to the old full-row drain. The usage compute reads only
/// <c>UsageMetricsSessionProjection</c>; in real Azure Table Storage a <c>$select</c> returns ONLY
/// those properties, so the "projected" row here is built with the projected keys only — a getter
/// for an omitted column returns null exactly as it would against the live service. Both shapes go
/// through the production mapper (<see cref="TableStorageService.MapToSessionSummary(TableEntity)"/>)
/// and every field UsageMetricsService consumes must match. Mirrors
/// <see cref="SessionStatsProjectionEquivalenceTests"/> for the stats drain.
/// </summary>
public class UsageMetricsProjectionEquivalenceTests
{
private const string TenantId = "00000000-0000-0000-0000-000000000def";
private static readonly TableStorageService Sut =
new(new Mock<TableServiceClient>().Object, NullLogger<TableStorageService>.Instance);
/// <summary>Full wide Sessions row as GetSessionsByDateRangeAsync would read it (no projection).</summary>
private static TableEntity FullRow(
string sessionId, string status, DateTime startedAt,
int? durationSeconds, DateTime? completedAt,
bool isPreProvisioned, DateTime? resumedAt)
{
var e = new TableEntity(TenantId, sessionId)
{
["Status"] = status,
["StartedAt"] = new DateTimeOffset(startedAt),
["Manufacturer"] = "Contoso",
["Model"] = "Model-X",
["IsUserDriven"] = true,
["IsPreProvisioned"] = isPreProvisioned,
["PlatformScriptCount"] = 4,
["RemediationScriptCount"] = 2,
// Representative noise the usage compute never reads — present on a real wide row,
// absent on the projected one. Proves the projection drops it without changing results.
["SerialNumber"] = "SN-FULL",
["DeviceName"] = "PC-FULL",
["OsName"] = "Windows 11",
["GeoCountry"] = "DE",
["EventCount"] = 123,
["FailureSnapshotJson"] = "{\"big\":\"" + new string('x', 2000) + "\"}",
};
if (durationSeconds.HasValue) e["DurationSeconds"] = durationSeconds.Value;
if (completedAt.HasValue) e["CompletedAt"] = new DateTimeOffset(completedAt.Value);
if (resumedAt.HasValue) e["ResumedAt"] = new DateTimeOffset(resumedAt.Value);
return e;
}
/// <summary>
/// Reduces a full row to exactly what Azure returns under the production
/// <c>$select = UsageMetricsSessionProjection</c>. Deriving the keep-set from the production
/// array means dropping a column there — e.g. ResumedAt — immediately fails the WhiteGlove
/// duration test below.
/// </summary>
private static TableEntity Project(TableEntity full)
{
var keep = new HashSet<string>(TableStorageService.UsageMetricsSessionProjection, StringComparer.Ordinal);
var projected = new TableEntity(full.PartitionKey, full.RowKey);
foreach (var kv in full)
{
if (keep.Contains(kv.Key))
projected[kv.Key] = kv.Value;
}
return projected;
}
private static void AssertUsageFieldsEqual(SessionSummary fromFull, SessionSummary fromProjected)
{
Assert.Equal(fromFull.TenantId, fromProjected.TenantId);
Assert.Equal(fromFull.SessionId, fromProjected.SessionId);
Assert.Equal(fromFull.StartedAt, fromProjected.StartedAt);
Assert.Equal(fromFull.CompletedAt, fromProjected.CompletedAt);
Assert.Equal(fromFull.Status, fromProjected.Status);
Assert.Equal(fromFull.Manufacturer, fromProjected.Manufacturer);
Assert.Equal(fromFull.Model, fromProjected.Model);
Assert.Equal(fromFull.IsUserDriven, fromProjected.IsUserDriven);
Assert.Equal(fromFull.IsPreProvisioned, fromProjected.IsPreProvisioned);
Assert.Equal(fromFull.PlatformScriptCount, fromProjected.PlatformScriptCount);
Assert.Equal(fromFull.RemediationScriptCount, fromProjected.RemediationScriptCount);
// Wall-clock duration branches call UtcNow inside the mapper; two map calls microseconds
// apart may straddle a whole-second boundary, so allow 1s of slack instead of exactness.
Assert.NotNull(fromFull.DurationSeconds);
Assert.NotNull(fromProjected.DurationSeconds);
Assert.True(Math.Abs(fromFull.DurationSeconds!.Value - fromProjected.DurationSeconds!.Value) <= 1,
$"DurationSeconds diverged: full={fromFull.DurationSeconds} projected={fromProjected.DurationSeconds}");
}
[Fact]
public void Succeeded_with_stored_duration_maps_identically()
{
var full = FullRow("s-ok", "Succeeded", DateTime.UtcNow.AddHours(-2),
durationSeconds: 1800, completedAt: DateTime.UtcNow.AddHours(-1.5),
isPreProvisioned: false, resumedAt: null);
AssertUsageFieldsEqual(Sut.MapToSessionSummary(full), Sut.MapToSessionSummary(Project(full)));
}
[Fact]
public void Failed_without_stored_duration_uses_completedAt_fallback_identically()
{
var full = FullRow("s-fail", "Failed", DateTime.UtcNow.AddHours(-3),
durationSeconds: null, completedAt: DateTime.UtcNow.AddHours(-2),
isPreProvisioned: false, resumedAt: null);
AssertUsageFieldsEqual(Sut.MapToSessionSummary(full), Sut.MapToSessionSummary(Project(full)));
}
[Fact]
public void InProgress_whiteglove_part2_duration_branch_survives_projection()
{
// This branch reads IsPreProvisioned + ResumedAt + DurationSeconds — exactly the columns
// that would silently break if dropped from the projection.
var full = FullRow("s-wg", "InProgress", DateTime.UtcNow.AddDays(-2),
durationSeconds: 3600, completedAt: null,
isPreProvisioned: true, resumedAt: DateTime.UtcNow.AddMinutes(-30));
AssertUsageFieldsEqual(Sut.MapToSessionSummary(full), Sut.MapToSessionSummary(Project(full)));
}
[Fact]
public void InProgress_wallclock_fallback_survives_projection()
{
var full = FullRow("s-run", "InProgress", DateTime.UtcNow.AddMinutes(-45),
durationSeconds: null, completedAt: null,
isPreProvisioned: false, resumedAt: null);
AssertUsageFieldsEqual(Sut.MapToSessionSummary(full), Sut.MapToSessionSummary(Project(full)));
}
}
@@ -0,0 +1,116 @@
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the single-flight + cache behaviour of the platform usage-metrics compute. A fresh compute
/// takes tens of seconds against real storage, and clients that time out retry — before the
/// single-flight guard every retry stacked ANOTHER full compute on top of the still-running one
/// (observed as overlapping 30-40s requests with resultCode 499). Concurrent callers for the same
/// window must join the in-flight task; once it completes, callers within the cache window get the
/// cached instance; a failed compute must not poison the cache or the in-flight slot.
/// </summary>
public class UsageMetricsServiceSingleFlightTests
{
private static Mock<IMetricsRepository> CreateMetricsRepoMock()
{
var metrics = new Mock<IMetricsRepository>();
metrics.Setup(m => m.GetAllUserActivityMetricsAsync()).ReturnsAsync(new UserActivityMetrics());
metrics.Setup(m => m.GetAppInstallRefsAsync(It.IsAny<DateTime>(), It.IsAny<string?>()))
.ReturnsAsync(new List<SessionAppRef>());
metrics.Setup(m => m.GetPlatformStatsAsync()).ReturnsAsync((PlatformStats?)null);
return metrics;
}
[Fact]
public async Task Concurrent_requests_join_one_compute_then_cache_serves_followups()
{
var sessionCalls = 0;
var gate = new TaskCompletionSource<List<SessionSummary>>(TaskCreationOptions.RunContinuationsAsynchronously);
var maintenance = new Mock<IMaintenanceRepository>();
maintenance
.Setup(m => m.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.Returns(() =>
{
Interlocked.Increment(ref sessionCalls);
return gate.Task;
});
var sut = new UsageMetricsService(CreateMetricsRepoMock().Object, maintenance.Object,
NullLogger<UsageMetricsService>.Instance);
// Two requests race while the compute is blocked on the session scan.
var first = sut.ComputeUsageMetricsAsync(30);
var second = sut.ComputeUsageMetricsAsync(30);
Assert.False(first.IsCompleted);
Assert.Same(first, second); // joined the identical in-flight task, no second compute
gate.SetResult(new List<SessionSummary>());
var r1 = await first;
var r2 = await second;
Assert.Equal(1, sessionCalls);
Assert.Same(r1, r2);
Assert.False(r1.FromCache);
// Follow-up within the cache window: served from cache, still exactly one scan.
var r3 = await sut.ComputeUsageMetricsAsync(30);
Assert.True(r3.FromCache);
Assert.Equal(1, sessionCalls);
}
[Fact]
public async Task Different_windows_do_not_share_a_flight()
{
var sessionCalls = 0;
var maintenance = new Mock<IMaintenanceRepository>();
maintenance
.Setup(m => m.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.Returns(() =>
{
Interlocked.Increment(ref sessionCalls);
return Task.FromResult(new List<SessionSummary>());
});
var sut = new UsageMetricsService(CreateMetricsRepoMock().Object, maintenance.Object,
NullLogger<UsageMetricsService>.Instance);
var r30 = await sut.ComputeUsageMetricsAsync(30);
var r90 = await sut.ComputeUsageMetricsAsync(90);
Assert.Equal(2, sessionCalls);
Assert.Equal(30, r30.WindowDays);
Assert.Equal(90, r90.WindowDays);
}
[Fact]
public async Task Failed_compute_is_not_cached_and_next_request_retries()
{
var maintenance = new Mock<IMaintenanceRepository>();
maintenance
.SetupSequence(m => m.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.ThrowsAsync(new InvalidOperationException("boom"))
.ReturnsAsync(new List<SessionSummary>());
var sut = new UsageMetricsService(CreateMetricsRepoMock().Object, maintenance.Object,
NullLogger<UsageMetricsService>.Instance);
await Assert.ThrowsAsync<InvalidOperationException>(() => sut.ComputeUsageMetricsAsync(30));
// The failed flight must have been evicted — the retry computes fresh and succeeds.
var retry = await sut.ComputeUsageMetricsAsync(30);
Assert.False(retry.FromCache);
Assert.Equal(30, retry.WindowDays);
}
}
@@ -17,7 +17,7 @@ public class UsageMetricsServiceWindowTests
{ {
var maintenanceRepo = new Mock<IMaintenanceRepository>(); var maintenanceRepo = new Mock<IMaintenanceRepository>();
maintenanceRepo maintenanceRepo
.Setup(r => r.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>())) .Setup(r => r.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.ReturnsAsync(sessions ?? new List<SessionSummary>()); .ReturnsAsync(sessions ?? new List<SessionSummary>());
var metricsRepo = new Mock<IMetricsRepository>(); var metricsRepo = new Mock<IMetricsRepository>();
@@ -27,10 +27,8 @@ public class UsageMetricsServiceWindowTests
.ReturnsAsync(new UserActivityMetrics()); .ReturnsAsync(new UserActivityMetrics());
metricsRepo.Setup(r => r.GetAllUserActivityMetricsAsync()) metricsRepo.Setup(r => r.GetAllUserActivityMetricsAsync())
.ReturnsAsync(new UserActivityMetrics()); .ReturnsAsync(new UserActivityMetrics());
metricsRepo.Setup(r => r.GetAllAppInstallSummariesAsync(It.IsAny<DateTime?>())) metricsRepo.Setup(r => r.GetAppInstallRefsAsync(It.IsAny<DateTime>(), It.IsAny<string?>()))
.ReturnsAsync(new List<AppInstallSummary>()); .ReturnsAsync(new List<SessionAppRef>());
metricsRepo.Setup(r => r.GetAppInstallSummariesByTenantAsync(It.IsAny<string>(), It.IsAny<DateTime?>()))
.ReturnsAsync(new List<AppInstallSummary>());
metricsRepo.Setup(r => r.GetPlatformStatsAsync()) metricsRepo.Setup(r => r.GetPlatformStatsAsync())
.ReturnsAsync((PlatformStats?)null); .ReturnsAsync((PlatformStats?)null);
@@ -50,7 +48,7 @@ public class UsageMetricsServiceWindowTests
DateTime? capturedStart = null; DateTime? capturedStart = null;
DateTime? capturedEnd = null; DateTime? capturedEnd = null;
maintenanceRepo maintenanceRepo
.Setup(r => r.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), "tenant-x")) .Setup(r => r.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), "tenant-x"))
.Callback<DateTime, DateTime, string?>((start, end, _) => { capturedStart = start; capturedEnd = end; }) .Callback<DateTime, DateTime, string?>((start, end, _) => { capturedStart = start; capturedEnd = end; })
.ReturnsAsync(new List<SessionSummary>()); .ReturnsAsync(new List<SessionSummary>());
@@ -75,7 +73,7 @@ public class UsageMetricsServiceWindowTests
DateTime? capturedStart = null; DateTime? capturedStart = null;
maintenanceRepo maintenanceRepo
.Setup(r => r.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>())) .Setup(r => r.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.Callback<DateTime, DateTime, string?>((start, _, _) => capturedStart = start) .Callback<DateTime, DateTime, string?>((start, _, _) => capturedStart = start)
.ReturnsAsync(new List<SessionSummary>()); .ReturnsAsync(new List<SessionSummary>());
@@ -197,7 +195,7 @@ public class UsageMetricsServiceWindowTests
var (service, maintenanceRepo) = CreateService(); var (service, maintenanceRepo) = CreateService();
// Repo honours the cutoff like the real impl would: filter sessions by start date. // Repo honours the cutoff like the real impl would: filter sessions by start date.
maintenanceRepo maintenanceRepo
.Setup(r => r.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>())) .Setup(r => r.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.ReturnsAsync((DateTime start, DateTime _, string? _) => sessions.Where(s => s.StartedAt >= start).ToList()); .ReturnsAsync((DateTime start, DateTime _, string? _) => sessions.Where(s => s.StartedAt >= start).ToList());
var r7 = await service.ComputeTenantUsageMetricsAsync("t1", 7); var r7 = await service.ComputeTenantUsageMetricsAsync("t1", 7);
@@ -31,15 +31,15 @@ public class UsageMetricsWindowConsistencyTests
var maintenance = new Mock<IMaintenanceRepository>(); var maintenance = new Mock<IMaintenanceRepository>();
maintenance maintenance
.Setup(m => m.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>())) .Setup(m => m.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.Callback<DateTime, DateTime, string?>((start, _, _) => sessionStart = start) .Callback<DateTime, DateTime, string?>((start, _, _) => sessionStart = start)
.ReturnsAsync(new List<SessionSummary>()); .ReturnsAsync(new List<SessionSummary>());
var metrics = new Mock<IMetricsRepository>(); var metrics = new Mock<IMetricsRepository>();
metrics metrics
.Setup(m => m.GetAllAppInstallSummariesAsync(It.IsAny<DateTime?>())) .Setup(m => m.GetAppInstallRefsAsync(It.IsAny<DateTime>(), It.IsAny<string?>()))
.Callback<DateTime?>(c => appCutoff = c) .Callback<DateTime, string?>((c, _) => appCutoff = c)
.ReturnsAsync(new List<AppInstallSummary>()); .ReturnsAsync(new List<SessionAppRef>());
metrics.Setup(m => m.GetAllUserActivityMetricsAsync()).ReturnsAsync(new UserActivityMetrics()); metrics.Setup(m => m.GetAllUserActivityMetricsAsync()).ReturnsAsync(new UserActivityMetrics());
metrics.Setup(m => m.GetPlatformStatsAsync()).ReturnsAsync((PlatformStats?)null); metrics.Setup(m => m.GetPlatformStatsAsync()).ReturnsAsync((PlatformStats?)null);
@@ -61,15 +61,15 @@ public class UsageMetricsWindowConsistencyTests
var maintenance = new Mock<IMaintenanceRepository>(); var maintenance = new Mock<IMaintenanceRepository>();
maintenance maintenance
.Setup(m => m.GetSessionsByDateRangeAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>())) .Setup(m => m.GetUsageWindowSessionsAsync(It.IsAny<DateTime>(), It.IsAny<DateTime>(), It.IsAny<string?>()))
.Callback<DateTime, DateTime, string?>((start, _, _) => sessionStart = start) .Callback<DateTime, DateTime, string?>((start, _, _) => sessionStart = start)
.ReturnsAsync(new List<SessionSummary>()); .ReturnsAsync(new List<SessionSummary>());
var metrics = new Mock<IMetricsRepository>(); var metrics = new Mock<IMetricsRepository>();
metrics metrics
.Setup(m => m.GetAppInstallSummariesByTenantAsync(It.IsAny<string>(), It.IsAny<DateTime?>())) .Setup(m => m.GetAppInstallRefsAsync(It.IsAny<DateTime>(), It.IsAny<string?>()))
.Callback<string, DateTime?>((_, c) => appCutoff = c) .Callback<DateTime, string?>((c, _) => appCutoff = c)
.ReturnsAsync(new List<AppInstallSummary>()); .ReturnsAsync(new List<SessionAppRef>());
metrics.Setup(m => m.GetUserActivityMetricsAsync(It.IsAny<string>())).ReturnsAsync(new UserActivityMetrics()); metrics.Setup(m => m.GetUserActivityMetricsAsync(It.IsAny<string>())).ReturnsAsync(new UserActivityMetrics());
metrics.Setup(m => m.GetPlatformStatsAsync()).ReturnsAsync((PlatformStats?)null); metrics.Setup(m => m.GetPlatformStatsAsync()).ReturnsAsync((PlatformStats?)null);
@@ -57,6 +57,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> _storage.GetSessionsByDateRangeAsync(startDate, endDate, tenantId); => _storage.GetSessionsByDateRangeAsync(startDate, endDate, tenantId);
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> _storage.GetUsageWindowSessionsAsync(startDate, endDate, tenantId);
public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime) public Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime)
=> _storage.GetStalledSessionsAsync(tenantId, cutoffTime); => _storage.GetStalledSessionsAsync(tenantId, cutoffTime);
@@ -41,6 +41,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
public Task<List<AppInstallSummary>> GetAllAppInstallSummariesAsync(DateTime? sinceUtc = null) public Task<List<AppInstallSummary>> GetAllAppInstallSummariesAsync(DateTime? sinceUtc = null)
=> _storage.GetAllAppInstallSummariesAsync(sinceUtc); => _storage.GetAllAppInstallSummariesAsync(sinceUtc);
public Task<List<SessionAppRef>> GetAppInstallRefsAsync(DateTime sinceUtc, string? tenantId = null)
=> _storage.GetAppInstallRefsAsync(sinceUtc, tenantId);
public Task<PlatformStats?> GetPlatformStatsAsync() public Task<PlatformStats?> GetPlatformStatsAsync()
=> _storage.GetPlatformStatsAsync(); => _storage.GetPlatformStatsAsync();
@@ -24,7 +24,7 @@ namespace AutopilotMonitor.Functions.Functions.Metrics
/// <summary> /// <summary>
/// GET /api/global/metrics/usage - Compute and return platform usage metrics /// GET /api/global/metrics/usage - Compute and return platform usage metrics
/// On-demand computation with 5-minute cache (Global Admin only) /// On-demand computation with 15-minute cache (Global Admin only)
/// </summary> /// </summary>
[Function("GetPlatformUsageMetrics")] [Function("GetPlatformUsageMetrics")]
public async Task<HttpResponseData> Run( public async Task<HttpResponseData> Run(
@@ -515,7 +515,35 @@ namespace AutopilotMonitor.Functions.Services
/// Gets all sessions within a date range, optionally filtered by tenant. /// Gets all sessions within a date range, optionally filtered by tenant.
/// Uses server-side filtering to avoid loading all sessions into memory. /// Uses server-side filtering to avoid loading all sessions into memory.
/// </summary> /// </summary>
public async Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null) public Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId, select: null);
/// <summary>
/// Columns the usage-metrics compute (UsageMetricsService) actually consumes, plus the
/// structural PartitionKey (TenantId) / RowKey (SessionId). CompletedAt / IsPreProvisioned /
/// ResumedAt must stay in the set for ComputeEffectiveDuration parity (WhiteGlove Part-2
/// branch and the completed-at fallback). Everything else on the wide Sessions row — most
/// notably FailureSnapshotJson — is never read by the compute, so the projected scan skips
/// it. Non-projected columns map to null/default via the Safe*/`?? ` getters in
/// MapToSessionSummary. internal so UsageMetricsProjectionEquivalenceTests derives its
/// keep-set from this exact array.
/// </summary>
internal static readonly string[] UsageMetricsSessionProjection =
{
"PartitionKey", "RowKey", "StartedAt", "CompletedAt", "Status", "DurationSeconds",
"Manufacturer", "Model", "IsUserDriven", "IsPreProvisioned", "ResumedAt",
"PlatformScriptCount", "RemediationScriptCount"
};
/// <summary>
/// Column-projected date-range query for the usage-metrics compute. Identical filter and
/// result semantics to <see cref="GetSessionsByDateRangeAsync"/>; only the transferred
/// column set differs (see <see cref="UsageMetricsSessionProjection"/>).
/// </summary>
public Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null)
=> QuerySessionsByDateRangeAsync(startDate, endDate, tenantId, UsageMetricsSessionProjection);
private async Task<List<SessionSummary>> QuerySessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId, string[]? select)
{ {
if (!string.IsNullOrEmpty(tenantId)) if (!string.IsNullOrEmpty(tenantId))
SecurityValidator.EnsureValidGuid(tenantId, nameof(tenantId)); SecurityValidator.EnsureValidGuid(tenantId, nameof(tenantId));
@@ -528,7 +556,7 @@ namespace AutopilotMonitor.Functions.Services
? $"PartitionKey eq '{tenantId}' and StartedAt ge datetime'{startDate:yyyy-MM-ddTHH:mm:ss}Z' and StartedAt lt datetime'{endDate:yyyy-MM-ddTHH:mm:ss}Z'" ? $"PartitionKey eq '{tenantId}' and StartedAt ge datetime'{startDate:yyyy-MM-ddTHH:mm:ss}Z' and StartedAt lt datetime'{endDate:yyyy-MM-ddTHH:mm:ss}Z'"
: $"StartedAt ge datetime'{startDate:yyyy-MM-ddTHH:mm:ss}Z' and StartedAt lt datetime'{endDate:yyyy-MM-ddTHH:mm:ss}Z'"; : $"StartedAt ge datetime'{startDate:yyyy-MM-ddTHH:mm:ss}Z' and StartedAt lt datetime'{endDate:yyyy-MM-ddTHH:mm:ss}Z'";
var query = tableClient.QueryAsync<TableEntity>(filter: filter); var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: select);
var sessions = new List<SessionSummary>(); var sessions = new List<SessionSummary>();
await foreach (var entity in query) await foreach (var entity in query)
@@ -395,6 +395,52 @@ namespace AutopilotMonitor.Functions.Services
} }
} }
/// <summary>
/// Columns the (SessionId, AppName) pair projection transfers. The usage-metrics compute
/// only groups installs per session and counts distinct app names, so the full summary row
/// (DO telemetry, failure text, timestamps, …) is dead weight on this scan.
/// </summary>
internal static readonly string[] AppInstallRefProjection = { "PartitionKey", "RowKey", "SessionId", "AppName" };
/// <summary>
/// Lean windowed (SessionId, AppName) scan over AppInstallSummaries. Same filter semantics
/// as the summary getters (server-side <c>StartedAt ge</c> + optional tenant partition), but
/// column-projected to <see cref="AppInstallRefProjection"/>. <paramref name="sinceUtc"/> is
/// server-derived, so interpolating it is injection-safe.
/// </summary>
public async Task<List<SessionAppRef>> GetAppInstallRefsAsync(DateTime sinceUtc, string? tenantId = null)
{
if (!string.IsNullOrEmpty(tenantId))
SecurityValidator.EnsureValidGuid(tenantId, nameof(tenantId));
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.AppInstallSummaries);
var filter = $"StartedAt ge datetime'{sinceUtc:yyyy-MM-ddTHH:mm:ss}Z'";
if (!string.IsNullOrEmpty(tenantId))
filter = $"PartitionKey eq '{tenantId}' and " + filter;
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: AppInstallRefProjection);
var refs = new List<SessionAppRef>();
await foreach (var entity in query)
{
refs.Add(new SessionAppRef
{
SessionId = entity.GetString("SessionId") ?? string.Empty,
AppName = entity.GetString("AppName") ?? string.Empty
});
}
return refs;
}
catch (Exception ex)
{
_logger.LogError(ex, "Failed to get app install refs");
return new List<SessionAppRef>();
}
}
private AppInstallSummary MapToAppInstallSummary(TableEntity entity) private AppInstallSummary MapToAppInstallSummary(TableEntity entity)
{ {
return new AppInstallSummary return new AppInstallSummary
@@ -2558,7 +2558,10 @@ namespace AutopilotMonitor.Functions.Services
/// Maps a TableEntity from the primary Sessions table to SessionSummary. /// Maps a TableEntity from the primary Sessions table to SessionSummary.
/// SessionId is the entity RowKey on the primary table. /// SessionId is the entity RowKey on the primary table.
/// </summary> /// </summary>
private SessionSummary MapToSessionSummary(TableEntity entity) // internal (not private) so UsageMetricsProjectionEquivalenceTests can pin that a Sessions
// row carrying only UsageMetricsSessionProjection maps to the same usage-relevant fields
// as a full row.
internal SessionSummary MapToSessionSummary(TableEntity entity)
=> MapToSessionSummary(entity, entity.RowKey); => MapToSessionSummary(entity, entity.RowKey);
/// <summary> /// <summary>
@@ -19,10 +19,15 @@ namespace AutopilotMonitor.Functions.Services
private readonly IMaintenanceRepository _maintenanceRepo; private readonly IMaintenanceRepository _maintenanceRepo;
private readonly ILogger<UsageMetricsService> _logger; private readonly ILogger<UsageMetricsService> _logger;
// In-memory cache (per-window, since results differ by days) // In-memory cache (per-window, since results differ by days). Instance state: the service
private static readonly Dictionary<int, (PlatformUsageMetrics metrics, DateTime expiry)> _cachedByDays = new(); // is registered as a singleton, so this behaves like the former statics but stays testable.
private static readonly TimeSpan CacheDuration = TimeSpan.FromMinutes(5); private readonly Dictionary<int, (PlatformUsageMetrics metrics, DateTime expiry)> _cachedByDays = new();
private static readonly object _cacheLock = new object(); // In-flight computes per window (single-flight): a fresh compute takes tens of seconds on
// the platform scope, and impatient clients retry — without this, every retry started
// another concurrent full compute on top of the still-running one.
private readonly Dictionary<int, Task<PlatformUsageMetrics>> _inFlightByDays = new();
private static readonly TimeSpan CacheDuration = TimeSpan.FromMinutes(15);
private readonly object _cacheLock = new object();
private const int DefaultWindowDays = 90; private const int DefaultWindowDays = 90;
@@ -37,9 +42,10 @@ namespace AutopilotMonitor.Functions.Services
} }
/// <summary> /// <summary>
/// Computes platform usage metrics (with 5-minute per-window cache). /// Computes platform usage metrics (with 15-minute per-window cache). Concurrent callers for
/// the same window join the in-flight compute instead of starting their own (single-flight).
/// </summary> /// </summary>
public async Task<PlatformUsageMetrics> ComputeUsageMetricsAsync(int days = DefaultWindowDays) public Task<PlatformUsageMetrics> ComputeUsageMetricsAsync(int days = DefaultWindowDays)
{ {
days = ClampDays(days); days = ClampDays(days);
@@ -50,29 +56,59 @@ namespace AutopilotMonitor.Functions.Services
_logger.LogInformation("Returning cached usage metrics for days={Days} (expires in {Seconds}s)", _logger.LogInformation("Returning cached usage metrics for days={Days} (expires in {Seconds}s)",
days, (entry.expiry - DateTime.UtcNow).TotalSeconds); days, (entry.expiry - DateTime.UtcNow).TotalSeconds);
entry.metrics.FromCache = true; entry.metrics.FromCache = true;
return entry.metrics; return Task.FromResult(entry.metrics);
}
if (_inFlightByDays.TryGetValue(days, out var inFlight))
{
_logger.LogInformation("Joining in-flight usage metrics compute for days={Days}", days);
return inFlight;
}
var compute = ComputeAndCacheAsync(days);
_inFlightByDays[days] = compute;
return compute;
}
}
private async Task<PlatformUsageMetrics> ComputeAndCacheAsync(int days)
{
// Started under _cacheLock — yield first so the caller only holds the lock for the
// task creation, not for the synchronous prefix of the compute itself.
await Task.Yield();
try
{
_logger.LogInformation("Computing fresh usage metrics for days={Days}...", days);
var stopwatch = Stopwatch.StartNew();
var metrics = await ComputeUsageMetricsInternalAsync(days);
stopwatch.Stop();
metrics.ComputeDurationMs = (int)stopwatch.ElapsedMilliseconds;
metrics.ComputedAt = DateTime.UtcNow;
metrics.FromCache = false;
metrics.WindowDays = days;
_logger.LogInformation("Usage metrics computed in {Ms}ms (days={Days})", metrics.ComputeDurationMs, days);
lock (_cacheLock)
{
_cachedByDays[days] = (metrics, DateTime.UtcNow.Add(CacheDuration));
}
return metrics;
}
finally
{
// Cache entry (on success) is written before this runs, so a caller arriving in
// between sees either the in-flight task or the fresh cache — never a gap. Failures
// are not cached: the next request starts a new compute.
lock (_cacheLock)
{
_inFlightByDays.Remove(days);
} }
} }
_logger.LogInformation("Computing fresh usage metrics for days={Days}...", days);
var stopwatch = Stopwatch.StartNew();
var metrics = await ComputeUsageMetricsInternalAsync(days);
stopwatch.Stop();
metrics.ComputeDurationMs = (int)stopwatch.ElapsedMilliseconds;
metrics.ComputedAt = DateTime.UtcNow;
metrics.FromCache = false;
metrics.WindowDays = days;
_logger.LogInformation("Usage metrics computed in {Ms}ms (days={Days})", metrics.ComputeDurationMs, days);
lock (_cacheLock)
{
_cachedByDays[days] = (metrics, DateTime.UtcNow.Add(CacheDuration));
}
return metrics;
} }
/// <summary> /// <summary>
@@ -110,12 +146,23 @@ namespace AutopilotMonitor.Functions.Services
// meaningful only when they fit inside the window; clients can read WindowDays to know // meaningful only when they fit inside the window; clients can read WindowDays to know
// the actual scope. "Total" counters use PlatformStats (cumulative, tracked separately). // the actual scope. "Total" counters use PlatformStats (cumulative, tracked separately).
// One UtcNow snapshot for the whole computation: the session window lower bound and the // One UtcNow snapshot for the whole computation: the session window lower bound and the
// AppInstall cutoff (below) MUST share an identical windowStart. A fresh UtcNow for the // AppInstall cutoff MUST share an identical windowStart. A fresh UtcNow for the
// app query would sit milliseconds later and could drop a boundary session's app rows // app query would sit milliseconds later and could drop a boundary session's app rows
// (undercounting AvgAppsPerSession / TotalUniqueApps). Pinned by UsageMetricsWindowConsistencyTests. // (undercounting AvgAppsPerSession / TotalUniqueApps). Pinned by UsageMetricsWindowConsistencyTests.
var now = DateTime.UtcNow; var now = DateTime.UtcNow;
var windowStart = now.AddDays(-days); var windowStart = now.AddDays(-days);
var allSessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, now.AddDays(1));
// The four source queries are independent table scans — run them concurrently so the
// wall-clock cost is the slowest scan, not the sum. Both windowed scans are
// column-projected (see UsageMetricsSessionProjection / AppInstallRefProjection):
// the compute only aggregates a handful of columns, and shipping the wide rows
// (FailureSnapshotJson & co.) dominated the former 20-40s fresh-compute time.
var sessionsTask = _maintenanceRepo.GetUsageWindowSessionsAsync(windowStart, now.AddDays(1));
var userActivityTask = _metricsRepo.GetAllUserActivityMetricsAsync();
var appRefsTask = _metricsRepo.GetAppInstallRefsAsync(windowStart);
var platformStatsTask = _metricsRepo.GetPlatformStatsAsync();
await Task.WhenAll(sessionsTask, userActivityTask, appRefsTask, platformStatsTask);
var allSessions = await sessionsTask;
var today = now.Date; var today = now.Date;
var last7Days = now.AddDays(-7); var last7Days = now.AddDays(-7);
@@ -152,7 +199,7 @@ namespace AutopilotMonitor.Functions.Services
}; };
// User Metrics (from UserActivity table) // User Metrics (from UserActivity table)
var userActivity = await _metricsRepo.GetAllUserActivityMetricsAsync(); var userActivity = await userActivityTask;
var userMetrics = new UserMetrics var userMetrics = new UserMetrics
{ {
Total = userActivity.TotalUniqueUsers, Total = userActivity.TotalUniqueUsers,
@@ -223,13 +270,13 @@ namespace AutopilotMonitor.Functions.Services
}; };
// App & Script Metrics // App & Script Metrics
// Reuse windowStart (NOT a fresh UtcNow) so the AppInstall lower bound is identical to the // appRefsTask was bounded by windowStart (NOT a fresh UtcNow) so the AppInstall lower
// session window's — see the snapshot note above. relevantApps is filtered to sessionIdSet // bound is identical to the session window's — see the snapshot note above. relevantApps
// anyway, and an app installs during its session (StartedAt >= the session's), so bounding // is filtered to sessionIdSet anyway, and an app installs during its session
// here only drops rows already outside the window instead of scanning the whole table. // (StartedAt >= the session's), so bounding only drops rows already outside the window.
var allAppSummaries = await _metricsRepo.GetAllAppInstallSummariesAsync(windowStart); var allAppRefs = await appRefsTask;
var sessionIdSet = new HashSet<string>(allSessions.Select(s => s.SessionId)); var sessionIdSet = new HashSet<string>(allSessions.Select(s => s.SessionId));
var relevantApps = allAppSummaries.Where(a => sessionIdSet.Contains(a.SessionId)).ToList(); var relevantApps = allAppRefs.Where(a => sessionIdSet.Contains(a.SessionId)).ToList();
var appsPerSessionList = relevantApps.GroupBy(a => a.SessionId).Select(g => g.Count()).ToList(); var appsPerSessionList = relevantApps.GroupBy(a => a.SessionId).Select(g => g.Count()).ToList();
var appScriptMetrics = new AppScriptMetrics var appScriptMetrics = new AppScriptMetrics
@@ -243,7 +290,7 @@ namespace AutopilotMonitor.Functions.Services
}; };
// Platform Stats (cumulative since release) // Platform Stats (cumulative since release)
var platformStats = await _metricsRepo.GetPlatformStatsAsync(); var platformStats = await platformStatsTask;
return new PlatformUsageMetrics return new PlatformUsageMetrics
{ {
@@ -281,7 +328,15 @@ namespace AutopilotMonitor.Functions.Services
// windowStart — see ComputeUsageMetricsInternalAsync for why this must not drift. // windowStart — see ComputeUsageMetricsInternalAsync for why this must not drift.
var now = DateTime.UtcNow; var now = DateTime.UtcNow;
var windowStart = now.AddDays(-days); var windowStart = now.AddDays(-days);
var tenantSessions = await _maintenanceRepo.GetSessionsByDateRangeAsync(windowStart, now.AddDays(1), tenantId);
// Same concurrency + projection scheme as the platform path (see there): four
// independent scans, windowStart shared between the session and app lower bounds.
var sessionsTask = _maintenanceRepo.GetUsageWindowSessionsAsync(windowStart, now.AddDays(1), tenantId);
var tenantStatsTask = _metricsRepo.GetTenantStatsAsync(tenantId);
var userActivityTask = _metricsRepo.GetUserActivityMetricsAsync(tenantId);
var appRefsTask = _metricsRepo.GetAppInstallRefsAsync(windowStart, tenantId);
await Task.WhenAll(sessionsTask, tenantStatsTask, userActivityTask, appRefsTask);
var tenantSessions = await sessionsTask;
var today = now.Date; var today = now.Date;
var last7Days = now.AddDays(-7); var last7Days = now.AddDays(-7);
@@ -304,7 +359,7 @@ namespace AutopilotMonitor.Functions.Services
// Cumulative "since signup" counter — incremented at registration, seeded/self-healed // Cumulative "since signup" counter — incremented at registration, seeded/self-healed
// by the nightly maintenance recompute. The window count is a floor: before the first // by the nightly maintenance recompute. The window count is a floor: before the first
// maintenance seed the counter may lag behind sessions still within retention. // maintenance seed the counter may lag behind sessions still within retention.
var tenantStats = await _metricsRepo.GetTenantStatsAsync(tenantId); var tenantStats = await tenantStatsTask;
sessionMetrics.TotalAllTime = Math.Max(tenantStats?.TotalEnrollments ?? 0, sessionMetrics.Total); sessionMetrics.TotalAllTime = Math.Max(tenantStats?.TotalEnrollments ?? 0, sessionMetrics.Total);
// Tenant Metrics (always 1 for tenant-specific view) // Tenant Metrics (always 1 for tenant-specific view)
@@ -316,7 +371,7 @@ namespace AutopilotMonitor.Functions.Services
}; };
// User Metrics (from UserActivity table) // User Metrics (from UserActivity table)
var userActivity = await _metricsRepo.GetUserActivityMetricsAsync(tenantId); var userActivity = await userActivityTask;
var userMetrics = new UserMetrics var userMetrics = new UserMetrics
{ {
Total = userActivity.TotalUniqueUsers, Total = userActivity.TotalUniqueUsers,
@@ -386,10 +441,11 @@ namespace AutopilotMonitor.Functions.Services
}; };
// App & Script Metrics // App & Script Metrics
// Reuse windowStart (identical to the session window lower bound) — joined to tenantSessionIdSet below. // appRefsTask reused windowStart (identical to the session window lower bound) — joined
var tenantAppSummaries = await _metricsRepo.GetAppInstallSummariesByTenantAsync(tenantId, windowStart); // to tenantSessionIdSet below.
var tenantAppRefs = await appRefsTask;
var tenantSessionIdSet = new HashSet<string>(tenantSessions.Select(s => s.SessionId)); var tenantSessionIdSet = new HashSet<string>(tenantSessions.Select(s => s.SessionId));
var relevantTenantApps = tenantAppSummaries.Where(a => tenantSessionIdSet.Contains(a.SessionId)).ToList(); var relevantTenantApps = tenantAppRefs.Where(a => tenantSessionIdSet.Contains(a.SessionId)).ToList();
var tenantAppsPerSession = relevantTenantApps.GroupBy(a => a.SessionId).Select(g => g.Count()).ToList(); var tenantAppsPerSession = relevantTenantApps.GroupBy(a => a.SessionId).Select(g => g.Count()).ToList();
var tenantAppScriptMetrics = new AppScriptMetrics var tenantAppScriptMetrics = new AppScriptMetrics
@@ -67,6 +67,13 @@ namespace AutopilotMonitor.Shared.DataAccess
/// </summary> /// </summary>
Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false); Task<List<SessionSummary>> GetSessionsOlderThanAsync(string tenantId, DateTime cutoffDate, int maxResults = int.MaxValue, bool excludeInFlightDeletions = false);
Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null); Task<List<SessionSummary>> GetSessionsByDateRangeAsync(DateTime startDate, DateTime endDate, string? tenantId = null);
/// <summary>
/// Column-projected variant of <see cref="GetSessionsByDateRangeAsync"/> for the usage-metrics
/// compute: identical filter and result semantics, but only the columns that compute consumes
/// are transferred (drops FailureSnapshotJson and the rest of the wide row). Fields outside
/// the projection come back as defaults — callers must not read them.
/// </summary>
Task<List<SessionSummary>> GetUsageWindowSessionsAsync(DateTime startDate, DateTime endDate, string? tenantId = null);
Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime); Task<List<SessionSummary>> GetStalledSessionsAsync(string tenantId, DateTime cutoffTime);
Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff); Task<List<SessionSummary>> GetAgentSilentSessionsAsync(string tenantId, DateTime silenceCutoff, DateTime hardCutoff);
Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours); Task<List<SessionSummary>> GetExcessiveDataSendersAsync(string tenantId, DateTime windowCutoff, int maxSessionWindowHours);
@@ -25,6 +25,14 @@ namespace AutopilotMonitor.Shared.DataAccess
Task<List<AppInstallSummary>> GetAppInstallSummariesByTenantAsync(string tenantId, DateTime? sinceUtc = null); Task<List<AppInstallSummary>> GetAppInstallSummariesByTenantAsync(string tenantId, DateTime? sinceUtc = null);
/// <summary>Gets all tenants' app install summaries. Pass <paramref name="sinceUtc"/> to scope the (otherwise full-table) scan to the window; null returns everything (legacy behaviour).</summary> /// <summary>Gets all tenants' app install summaries. Pass <paramref name="sinceUtc"/> to scope the (otherwise full-table) scan to the window; null returns everything (legacy behaviour).</summary>
Task<List<AppInstallSummary>> GetAllAppInstallSummariesAsync(DateTime? sinceUtc = null); Task<List<AppInstallSummary>> GetAllAppInstallSummariesAsync(DateTime? sinceUtc = null);
/// <summary>
/// Lean (SessionId, AppName) pairs for app-per-session aggregation. Same window/tenant scoping
/// as the summary getters above, but column-projected server-side — the usage-metrics compute
/// only groups by SessionId and counts distinct AppNames, so shipping the full summary rows
/// (DO telemetry, failure text, …) multiplied transfer for nothing. Omit <paramref name="tenantId"/>
/// for the cross-tenant scan.
/// </summary>
Task<List<SessionAppRef>> GetAppInstallRefsAsync(DateTime sinceUtc, string? tenantId = null);
// --- Platform Stats --- // --- Platform Stats ---
Task<PlatformStats?> GetPlatformStatsAsync(); Task<PlatformStats?> GetPlatformStatsAsync();
@@ -67,6 +75,16 @@ namespace AutopilotMonitor.Shared.DataAccess
Task<int> DeleteRuleStatsOlderThanAsync(DateTime cutoffDate); Task<int> DeleteRuleStatsOlderThanAsync(DateTime cutoffDate);
} }
/// <summary>
/// Minimal projection of an AppInstallSummaries row: just enough to group installs per session
/// and count distinct app names. Returned by <see cref="IMetricsRepository.GetAppInstallRefsAsync"/>.
/// </summary>
public class SessionAppRef
{
public string SessionId { get; set; } = string.Empty;
public string AppName { get; set; } = string.Empty;
}
public class UserActivityMetrics public class UserActivityMetrics
{ {
/// <summary> /// <summary>
@@ -6,6 +6,24 @@ import { api } from '@/lib/api';
import { useAuth } from '../../../../contexts/AuthContext'; import { useAuth } from '../../../../contexts/AuthContext';
import { authenticatedFetch, TokenExpiredError } from "@/lib/authenticatedFetch"; import { authenticatedFetch, TokenExpiredError } from "@/lib/authenticatedFetch";
// A fresh cross-tenant compute takes tens of seconds server-side; the default 30s fetch timeout
// aborted it client-side (HTTP 499 on the backend) while the server kept computing. Give this one
// call generous headroom — the backend caches the result for 15 minutes afterwards.
const FETCH_TIMEOUT_MS = 180_000;
// Last observed fresh-compute fetch duration, used to size the progress bar. Only fresh computes
// are recorded (a cache hit finishing in ~100ms would make the next cold load look stuck at 95%).
const ESTIMATE_STORAGE_KEY = 'platformUsage.lastFreshFetchMs';
const DEFAULT_ESTIMATE_MS = 45_000;
function readFetchEstimateMs(): number {
try {
const v = Number(localStorage.getItem(ESTIMATE_STORAGE_KEY));
return Number.isFinite(v) && v > 1_000 ? v : DEFAULT_ESTIMATE_MS;
} catch {
return DEFAULT_ESTIMATE_MS;
}
}
interface SessionMetrics { interface SessionMetrics {
total: number; total: number;
today: number; today: number;
@@ -100,8 +118,21 @@ export function SectionPlatformUsage() {
const [loading, setLoading] = useState(true); const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [refreshing, setRefreshing] = useState(false); const [refreshing, setRefreshing] = useState(false);
const [loadStartedAt, setLoadStartedAt] = useState<number | null>(null);
const [elapsedMs, setElapsedMs] = useState(0);
const [estimateMs, setEstimateMs] = useState(DEFAULT_ESTIMATE_MS);
// Tick the elapsed clock while a request is in flight so the calculating card can show
// progress instead of an opaque spinner.
useEffect(() => {
if (loadStartedAt === null) return;
setElapsedMs(0);
const id = setInterval(() => setElapsedMs(Date.now() - loadStartedAt), 500);
return () => clearInterval(id);
}, [loadStartedAt]);
const fetchMetrics = async (showRefreshing = false) => { const fetchMetrics = async (showRefreshing = false) => {
const startedAt = Date.now();
try { try {
if (showRefreshing) { if (showRefreshing) {
setRefreshing(true); setRefreshing(true);
@@ -109,26 +140,38 @@ export function SectionPlatformUsage() {
setLoading(true); setLoading(true);
} }
setError(null); setError(null);
setEstimateMs(readFetchEstimateMs());
setLoadStartedAt(startedAt);
// Platform-wide metrics - cross-tenant (Global Admin only) // Platform-wide metrics - cross-tenant (Global Admin only)
const response = await authenticatedFetch(api.metrics.globalUsage(), getAccessToken); const response = await authenticatedFetch(api.metrics.globalUsage(), getAccessToken, {
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
});
if (!response.ok) { if (!response.ok) {
throw new Error(`Failed to fetch platform usage metrics: ${response.statusText}`); throw new Error(`Failed to fetch platform usage metrics: ${response.statusText}`);
} }
const data = await response.json(); const data: PlatformUsageMetrics = await response.json();
setMetrics(data); setMetrics(data);
if (!data.fromCache) {
try { localStorage.setItem(ESTIMATE_STORAGE_KEY, String(Date.now() - startedAt)); } catch { /* storage unavailable */ }
}
} catch (err) { } catch (err) {
if (err instanceof TokenExpiredError) { if (err instanceof TokenExpiredError) {
console.error('Session expired:', err.message); console.error('Session expired:', err.message);
} else { } else {
console.error('Error fetching platform usage metrics:', err); console.error('Error fetching platform usage metrics:', err);
} }
setError(err instanceof Error ? err.message : 'Failed to fetch platform usage metrics'); if (err instanceof DOMException && (err.name === 'TimeoutError' || err.name === 'AbortError')) {
setError('The calculation is taking unusually long. It continues on the server and the result is cached once finished — please retry in a moment.');
} else {
setError(err instanceof Error ? err.message : 'Failed to fetch platform usage metrics');
}
} finally { } finally {
setLoading(false); setLoading(false);
setRefreshing(false); setRefreshing(false);
setLoadStartedAt(null);
} }
}; };
@@ -153,11 +196,34 @@ export function SectionPlatformUsage() {
}; };
if (loading) { if (loading) {
// Progress is an estimate from the last fresh compute — capped at 95% so it never claims
// "done" before the response actually lands.
const progressPercent = Math.min(95, (elapsedMs / estimateMs) * 100);
const elapsedSeconds = Math.floor(elapsedMs / 1000);
const overEstimate = elapsedMs > estimateMs;
return ( return (
<div className="min-h-screen bg-gradient-to-br from-blue-50 to-indigo-100 flex items-center justify-center"> <div className="min-h-screen bg-gradient-to-br from-blue-50 to-indigo-100 flex items-center justify-center p-4">
<div className="text-center"> <div className="bg-white rounded-lg shadow-xl p-8 max-w-md w-full">
<div className="animate-spin rounded-full h-12 w-12 border-b-2 border-blue-600 mx-auto"></div> <div className="text-center">
<p className="mt-4 text-gray-600">Loading platform usage metrics...</p> <div className="animate-spin rounded-full h-12 w-12 border-b-2 border-blue-600 mx-auto"></div>
<h2 className="mt-4 text-xl font-semibold text-gray-900">Calculating platform metrics…</h2>
<p className="mt-2 text-sm text-gray-600">
Aggregating sessions, app installs and user activity across all tenants.
A fresh calculation can take a minute — the result is then cached for 15 minutes.
</p>
<div className="mt-6 w-full bg-gray-200 rounded-full h-2.5" role="progressbar" aria-valuenow={Math.round(progressPercent)} aria-valuemin={0} aria-valuemax={100}>
<div
className="bg-blue-600 h-2.5 rounded-full transition-all duration-500"
style={{ width: `${progressPercent}%` }}
></div>
</div>
<p className="mt-2 text-xs text-gray-500">
{elapsedSeconds}s elapsed
{overEstimate
? ' — taking a bit longer than usual…'
: ` · usually ~${Math.max(1, Math.round(estimateMs / 1000))}s`}
</p>
</div>
</div> </div>
</div> </div>
); );
@@ -223,7 +289,7 @@ export function SectionPlatformUsage() {
<svg className={`h-5 w-5 ${refreshing ? 'animate-spin' : ''}`} fill="none" viewBox="0 0 24 24" stroke="currentColor"> <svg className={`h-5 w-5 ${refreshing ? 'animate-spin' : ''}`} fill="none" viewBox="0 0 24 24" stroke="currentColor">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15" /> <path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15" />
</svg> </svg>
<span>{refreshing ? 'Refreshing...' : 'Refresh'}</span> <span>{refreshing ? `Refreshing… ${Math.floor(elapsedMs / 1000)}s` : 'Refresh'}</span>
</button> </button>
</div> </div>
</div> </div>