An app still installing when the session's observation ended has no terminal
event and never will. The session page ticked Date.now() - startedAt for it
("11h 39m" on a session that finished hours ago), and its AppInstallSummaries
row stayed InProgress forever - ranked behind failures in the app's session
list and counted in totalInstalls with no named bucket.
Web: applyObservationEnd marks such rows Incomplete once the session is
terminal (isTerminalStatus), bounded by session.lastEventAt; the panel shows a
slate Incomplete badge and the watched span as a lower bound, the "active"
pill excludes them, and the download panel carries the parallel flag. Covers
every historic session without a backfill.
Backend: CloseOpenAppInstallsForSessionAsync closes InProgress rows without a
CompletedAt as Incomplete - at the terminal seam, again on the ingest batch
that carries agent_shutting_down for a terminal session (rows opened after
the verdict), and from the 30-day sweep. The sticky-status reconcile keeps
Incomplete against a late progress-only batch and lets a real terminal win.
App metrics and app analytics disclose the bucket (incomplete,
totalIncomplete) outside the failure rate; manifest and wire types
regenerated.
MCP: get_session_summary coverage gains an apps block and a gaps line naming
the apps the agent's last app_tracking_summary still listed as installing or
downloading, terminal sessions only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Helpers/QueryParams replaces fourteen private days/limit parsers, the three
TryParseUtc copies, eleven identical pageSize blocks and ~50 inline TryParse sites
under Functions/ and Pagination/. Lenient (Int/IntOrNull/UtcInstant): absent, garbage
or below-minimum means the call site's default, above-maximum clamps. Strict
(TryInt/TryPageSize/TryUtcInstant): absent is null, anything else invalid is a 400
with a message that names the parameter. Integers are invariant-culture, instants are
invariant UTC (AssumeUniversal | AdjustToUniversal), whitespace counts as absent.
Defaults and ranges stay arguments at every call site.
Settled differences: the Apps windows clamp above 365 instead of falling back to 30,
days=0 or negative yields the default everywhere, the two global geo endpoints and
the session list cap days at 365, SLA months is capped at SlaMetricsService.MaxMonths
on the HTTP layer, a whitespace date means no filter on every route, the admin batch
sizes reject values above their maximum, and the three culture-dependent
DateTime.TryParse sites parse as UTC.
QueryParamGuardTests ratchets TryParse out of Functions/ and Pagination/ (baseline:
headers, raw cells, the agent version major, the se= of a stored SAS URL);
QueryParamsTests carries the cases the removed copies pinned.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The IME processes the ESP app list in multiple passes (device-ESP evaluation
ending Skipped, real install up to an hour later), and AppInstallSummary
folded first observation -> last terminal across them: a 23-second app late
in the queue reported 4002 s (session 44862cd3, "Set TimeZone").
Backend ingest: new max-folded LastAttemptStartedAt anchors DurationSeconds
(CompletedAt - last attempt start; legacy rows without the anchor keep the
span fallback), replay-guarded additive InstallPassCount, attempt-scoped
DownloadDurationSeconds pairing, and a weaker-terminal guard (completed with
state Skipped/Postponed never overrides Installed/Error, in-batch and in the
cross-batch reconcile). StartedAt stays the earliest-wins window/version-
ordering key. AppsDashboardProjection carries the new columns.
AppVersionRegressionRadar: cutover gate — only attempt-anchored rows enter
the duration population on BOTH comparison sides, so mixed-semantics medians
can neither mask real regressions nor falsely re-arm active episodes.
Time attribution: per-pass segment pairing replaces the first-start ->
last-terminal hull. An app's seconds = sum of its ACTIVE segments (failed
attempts and evaluation passes count; inter-pass idle gaps don't), occupancy
merges the segments, hull endpoints remain for display and what-if bounds.
Agent: a terminal->active transition re-arms BOTH timing endpoints, so event
payload durationSeconds also describes the attempt.
Web/PDF/MCP: labels and tooltips say "install time (final attempt)", the
apps sessions table gets a xN multi-pass chip, the time-series chart carries
a cutover footnote, MCP tool descriptions document the semantics split
between get_app_install_metrics and get_time_attribution.
No backfill: pre-cutover rows keep span values (documented in the public
changelog). OKF doc: docs/backend/app-install-duration-attempt.md.
Suites: Functions 4383, Agent 2407, MCP 564, Web 814 - all green, build 0/0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Operator side: CmTraceSkewTripwire piggybacks on the terminal counter
reconcile's partition scan (zero extra reads) and compares median
ReceivedAt-OccurredUtc deltas of IME-derived vs other events. A clean
15-min-grid divergence (residual < 2 min, >=20 samples / >=3 distinct
batches per side) emits the CmTraceTimeSkewRegression ops event
(Agent/Warning, goal: never fires). Bias-dominated sessions are
suppressed; measuredWriterOffsetMinutes is never consulted (sticky
after era flip-backs). Kill switch: CmTraceSkewTripwireDisabled.
Customer side: new clock_skew analyze condition source (skewMetric
clock_jump | sustained_offset, threshold seconds, gt/gte magnitude)
behind built-in ANALYZE-DEV-008 (300 s = the Kerberos/Entra 5-min skew
boundary). Batch-median frames with spool-spread filter, plateau step
detection (a gradual backlog-drain ramp cannot fake a jump) and
end-state persistence; IME-derived events are excluded so a CMTrace
anchoring regression can never fire the customer rule. Evidence carries
field=skewSummary/value for {{skewSummary}} interpolation.
Full mirroring chain: schema enum + skewMetric property, shared
manifest + combine.js regeneration, MCP authoring guide + semantic
lint, web SOURCES/form/card, ops-event dual-register, docs. P5 is
subsumed by the tripwire. PercentileMath is now the single nearest-rank
implementation (Apps/PlatformMetrics delegate).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
versionBreakdown on the app analytics responses now carries measuredInstalls,
medianDurationSeconds and p95DurationSeconds per version (measured population:
succeeded, non-skip, plausible duration, no AppId collision). A new pure
AppVersionRegressionRadar detects "median install duration rose from X to Y min
after version Z": versions ordered by first-seen install time (never string
sort), fires at median lift >=2x AND >=300s absolute with >=10 measured
installs on both sides.
Episodes follow the rule-radar pattern in a fourth tracker keyspace
appversionregression|{app}|{version} (409-dedup, refresh keeps FirstNotifiedAt,
re-arm on drain / median back under 1.5x, 30d retention sweep re-arms): one
tenant bell app_version_duration_regression (Admin audience, appDetailUrl href)
plus one AppVersionDurationRegression ops event per episode, dual-registered in
the web OPS_EVENT_TYPES catalog. Runs in the maintenance timer after the rule
radar over one cross-tenant 35d install-summary read; kill switch
AppVersionRegressionRadarDisabled (fail-open). Apps detail page shows an amber
regression banner and a median-duration-by-version chart; OKF concept doc added.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Insights expansion PR1 (tasks/insights-expansion-spec.md §F1, audit §0.5):
- AppInstallSummary + AppId / EspBlocking (bool?, tri-state) / AppIdCollision:
sentinel-gated columns + mapper + both projections. appId adopted from app
events (first-seen wins); a second appId under the same display name flags
the name-keyed row as a collision (audit Q3) — in-batch and cross-batch
(sticky in the reconcile). Collision rows leave every per-app fleet
aggregate (app metrics payload, apps list, app analytics) with a disclosed
exclusion count; the fleet DO rollup and per-app session drill-down keep
them.
- EspBlocking resolution at the session-terminal single-writer seam
(TableSessionRepository.UpdateSessionStatusAsync, next to the counter
reconcile — covers ingest, admin marks, maintenance sweep, rule engine):
joins the session's app rows against the LATEST list-carrying
esp_config_detected emission as a positive-evidence set (audit Q2) —
listed => true, absent => stays unknown, never false. EspBlockingSets
parser handles truncation via the uncapped espTracked*Count fields.
- TimeAttributionCalculator (pure, AttributionVersion 1) + SessionTimeBreakdown
models: exact partition of the session's authoritative DurationSeconds into
device_prep / esp_apps / identity_hello / user_esp / desktop_handoff spans
plus an explicit unattributed remainder — invariant sum(spans)+unattributed
== wall clock enforced in every test. WhiteGlove = two observation windows
in ONE session (part1 anchored on whiteglove_part1_complete, pause never
attributed). Per-app blocking intervals from EVENT timestamps (first start
-> LAST terminal, audit Q1 retry freeze), occupancy = merged union clipped
to esp_apps. Reboot spans from event gaps around lastBootUtc (audit Q7),
deduped per boot. QualityFlags: ClockSkewDropped, PartialObservation,
BlockingSetUnknown, BlockingSetTruncated, WhiteGloveAnchorsIncomplete.
Persistence (SessionTimeBreakdowns) and surfaces follow in PR2/PR3.
Backend 3368/3368 (35 new tests: 19 calculator golden fixtures + 16
identity/blocking/aggregate-exclusion), solution build clean, web untouched
(new response fields are additive).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Source-data audit (tasks/insights-expansion-spec.md §0.5) verified against
production: 17.9% of Succeeded AppInstallSummaries rows carry no duration —
78% are genuine skips (app_install_completed with payload state=Skipped,
"Update for X" WinGet policies not applicable) whose state the ingest
discarded, the rest are real installs whose start the agent never observed
(attach window). Averaging them as 0s understated avg install duration by
18% and distorted the slowest-apps ranking.
- AppInstallSummary.TerminalState (Installed/Skipped/Postponed/Error) from
the closing event's payload state; sentinel-gated column + mapper +
AppsDashboardProjection. Status stays "Succeeded" for compatibility.
- MetricsMath.IsSkipTerminalState/HasMeasuredDuration: durations computed
over measured rows only; skips leave the failure/success rate (decision
2026-07-26 — a skip is not an install attempt); per-app skipped/unmeasured
counts + totals exposed; slowest ranking gates on measured installs.
Applied consistently: app metrics payload, apps dashboard (list/analytics/
time series/trend/versions/device models), SLA weekly/snapshot/breach.
- StoreAppInstallSummaryAsync: reconcile extracted to internal static
ReconcileAppInstallSummaryWithExisting (test seam). Fixes Q4 (terminal
batch before started batch left duration permanently unset) and hardens
terminal status against late/replayed InProgress batches.
- Web: measured-only hints on app detail cards, skip suffix in apps list,
fleet-health "measured installs" caption.
Backend 3333/3333 (20 new tests), web tsc clean, vitest 627/627.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second half of the terminal-only rate convention (follows 1cfc32c9, which
converted the enrollment success rates): app-install failureRate divided
by ALL install rows including "InProgress" ones — installs still running,
or orphaned when a session died mid-install — diluting every failure rate
customers see on the Apps dashboard and Fleet Health.
- New MetricsMath.TerminalFailureRatePct (Failed / (Failed + Succeeded),
0 when nothing finished) shared by both aggregation sites
- MetricsMath.BuildAppMetricsPayload: per-app failureRate terminal-based
(Fleet Health "Top Failing Apps"; subtitle now "% of N finished")
- AppsAnalyticsHelper: list, analytics summary, versionBreakdown,
timeSeries and deviceModelBreakdown rates terminal-based; the
duplicated trend block is deduplicated into ComputeFailureTrend, whose
rates and >= 5-sample gate now run on finished installs per half; the
device-model >= 5 sample floor likewise counts finished installs
- Web: InstallsTab "Avg Failure Rate" card computed over summed finished
installs instead of totalInstalls
Tests: new AppsAnalyticsFailureRateTests (list/trend/analytics/model
cases incl. in-flight dilution and gate regressions) and a terminal-rate
case in MetricsMathAppPayloadTests. Backend 3223/3223, tsc clean,
vitest 602/602.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third slice of the metrics-performance pass (after e63a0bc2 usage, 86f5b507
geo): the app endpoints ran unprojected full-column AppInstallSummaries
window scans behind the web app's default 30s fetch timeout.
Backend:
- All AppInstallSummaries scans (full, geo, app-metrics, apps-dashboard)
now delegate to one private QueryAppInstallSummariesAsync core so filter
semantics stay in lockstep
- New projections: AppMetricsProjection for metrics/app + global (ranking
inputs, failure code and the DO counters) and AppsDashboardProjection for
the App Dashboard list/analytics/sessions endpoints (drops the whole DO
telemetry block, which those aggregations never read)
- AppsAnalyticsHelper.LoadSummariesAsync and both app-metrics functions
switched to the projected variants
- AppsProjectionEquivalenceTests pins both projections, incl. the
TenantId-from-PartitionKey fallback for legacy rows
Web:
- CalculatingInline: compact in-place variant of the calculating card
(shared progress bar + elapsed caption extracted)
- Apps installs tab and the app detail page (analytics + sessions fetches)
use 180s AbortSignals and show inline progress while aggregating
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The app dashboard and metrics endpoints loaded entire AppInstallSummaries
partitions (tenant scope) or the full table (global scope) and filtered
StartedAt >= cutoff in memory, even for days=30. The device-model join then
issued one serial GetSessionAsync point-read per distinct session.
- Add optional DateTime? sinceUtc to GetAppInstallSummariesByTenantAsync /
GetAllAppInstallSummariesAsync: when set, a server-side StartedAt ge filter
scopes the scan so only in-window rows are deserialized/transferred. Default
null preserves the legacy full scan for callers that need all history.
- AppsAnalyticsHelper.LoadSummariesAsync derives sinceUtc from days; all six
/api/apps/* functions pass it. The four metrics/app + metrics/geographic
functions pass their existing cutoff. The in-memory Where stays as the exact
sub-second trim (the OData filter is second-granular). MCP get_app_install_metrics
benefits automatically (days already flows through; response shape unchanged).
- Replace the serial session point-read loop with a bounded-concurrency
(SemaphoreSlim=10) LoadSessionLookupAsync, shared by the analytics and
sessions builders.
AppInstallSummaries is already bounded by the session-retention cascade
(DeletionManifestBuilder) + tenant offboarding, so no separate table retention
is added. Full Backend suite: 2406 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The apps/list endpoints returned the full per-app list in one shot, so there was
no way to page a large fleet's apps (e.g. for a future MCP consumer).
Backend (apps/list + global/apps/list):
- AppsAnalyticsHelper.BuildAppsListResponse gains opt-in pagination: no pageSize
keeps the legacy full-array body (the web UI pages client-side); pageSize
returns an offset-paginated envelope with count/offset/pageSize/nextLink.
- The list is aggregated in-memory from a deterministic sort, so a plain integer
offset is a stable cursor — no HMAC continuation token needed (unlike the
Azure-Table-backed config/all surface). Added an appName tiebreaker so the sort
is fully deterministic across pages.
- New ParseAppsPaging validates pageSize (1..1000) / skip; both functions 400 on
bad input and build a route-specific nextLink (global echoes tenantId).
Web (Installs tab): client-side pagination (pageSize 20) over the filtered/sorted
apps, matching the Inventory tab — sort/search stay across the full set.
Tests: AppsAnalyticsHelperPagingTests (legacy vs paginated, deterministic order,
last-page nextLink null). Backend 2206/2206, web 446, tsc clean, next build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Oliver Kieselbach