Revoke enforcement (L4): a revoked delegated (MSP) user kept receiving pushes on
already-joined SignalR groups until the connection dropped (join-time-only
authorization). Negotiate now goes through the Management SDK and binds the
client token to a SignalR user id (lowercased UPN); new DisconnectUserAsync
strips the user from all groups server-side and pushes "accessRevoked", on
which the web client restarts its connection so every re-join re-runs
authorization against the fresh scope. Wired into delegated revoke/disable and
tenant-group unassign / remove-tenant / delete-group. Residual (documented): a
tampered client ignoring the push, and connections negotiated before this
deploy (no user id), keep streams until the connection drops.
Revoke audit integrity (L5): POST /global/delegated-admins now requires a GUID
tenantId (mirrors the tenant-group path), and DELETE reports 404 for an
assignment that never existed instead of returning 200 and writing a false
customer-visible "access removed" audit row (repo delete now checks the 404
response of the idempotent DeleteEntityAsync).
Casing (L6): the SignalR group join/leave cross-tenant check now compares
tenant IDs OrdinalIgnoreCase, aligned with the middleware and the delegated
scope dictionary, so a casing difference can never flip which branch
authorizes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M1: hoist the McpUsers Disabled-row kill-switch above the policy switch so an
explicitly disabled account is denied MCP access under EVERY policy and grant
path (AllMembers, platform roles, delegated auto-grant) — previously inert
outside the WhitelistOnly branch.
M2: drop the MCP access-guard verdict cache from 5 min to 60s. The verdict
carries the kill-switch and the delegated tenant set; 5 min stretched the
backend's 30s revocation target to minutes for MCP callers.
M3: gate same-tenant joins of the plain tenant-{tid} SignalR broadcast group
on a member role (new pure SignalRGroupHelper.IsTenantBroadcastJoinDenied) —
roleless authenticated users (Progress Portal) could passively stream
MemberRead-tier live telemetry they cannot fetch over REST. Companions:
MarkSessionSucceeded/Failed additionally push "newevents" to the per-session
group so a roleless progress watcher still sees admin-marked completion, and
the Progress Portal no longer joins the tenant group (session group covers
all its signals).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kill/Block rules were only served on the telemetry response, which cannot
reach an agent whose upload loop is paused (indefinite block pins the drain
until DateTime.MaxValue), whose spool is empty, or whose binary is kill-blind.
Config is fetched at every process start (boot via Scheduled Task), making it
the reliable control channel - no heartbeat added by design (48h emergency
break + 6h max lifetime bound the remaining cases).
Backend (effective on deploy):
- KillSwitchEvaluator: shared device+version check for both channels; ingest
wire response and check order unchanged (device first, short-circuits)
- Every served Kill emits ops event KillSignalDelivered (Security/Warning),
throttled 24h per tenant+serial+pattern; Blocks stay event-free
- GET agent/config (incl. bootstrap wrapper) now returns DeviceBlocked/
DeviceKillSignal/UnblockAt alongside the full config; old binaries ignore
the new fields
- Web: KillSignalDelivered dual-registered in OPS_EVENT_TYPES
Agent V2 (effective with next agent release):
- CheckConfigKillSignal guard after the Phase-4 fetch: forced self-destruct
(overrides SelfDestructOnComplete), enrollment-complete marker, session
cleared, exit before mTLS/registration
- Kill honoured ONLY from a live fetch: config disk cache strips the flags on
write and read (UnrestrictedMode pattern), guard re-checks LastFetchOutcome
Tests: 9 KillSwitchEvaluatorTests + 7 ProgramConfigKillSignalTests;
Functions 2557/2557, Agent V2 1578/1578, web tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codex Medium on 43a87ff7: the delegated subset was built from
TryGetConfigurationAsync, which maps ANY storage failure to exists=false —
transient table throttling made managed tenants vanish from a 200 response,
whereas the replaced scan path rethrew and returned 500.
- New TenantConfigurationService.GetConfigurationIfExistsAsync: strict
point-read (cached, non-creating, null on 404) that PROPAGATES storage
errors; TryGetConfigurationAsync now delegates to it and keeps its
fail-closed mapping for agent security gates.
- Delegated config/all reads via the strict method → a read failure fails
the request (500) instead of shrinking the subset.
- Adjacent regressions from the same hunk: AllowedTenantIds deduped
case-insensitively (old HashSet-filter semantics), and ids missed by the
exact-case point-read get one scan+filter rescue pass — config PartitionKey
casing is not guaranteed (agent ingest auto-creates rows verbatim; same
caveat the delegated sessions fan-out handles case-insensitively).
Functions suite 2548/2548 (9 new tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The old route /api/diag-files-reports was not covered by any
clientCertExclusionPaths prefix on the Function App, so the platform
cert gate (clientCertMode=Required) prompted portal users for a client
certificate. /api/diagnostics is already excluded, and the endpoint
submits diagnostic files, so it moves under that prefix instead of
growing the portal-only exclusion list.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persists the agent-sent self-deploying/kiosk profile marker (template:
IsHybridJoin, write-once): Sessions column + SessionsIndex full-mirror
projection + SessionSummary mapping. Re-registration preserve is a
sticky-true OR - a boot with an unreadable Autopilot policy cache must
not pin the flag to false once it was observed true.
Filterable end-to-end: SessionSearchFilter + continuation-token
fingerprint + search/raw query params + index/basic scan filters, plus
MCP search_sessions / query_raw_sessions inputs and the get_session
overview field.
Display/analytics only - the completion fix itself ships in the agent
(DecisionCore gates on the EnrollmentFactsObserved seed, not this flag).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M6: DeleteByFilterInBatchesAsync is fail-loud and counts only confirmed
batch submits; the orphan cleanup keeps the EventSessionIndex entry when
the event delete fails, so surviving rows stay discoverable and retry.
M7: GetSessionsOlderThanAsync gains excludeInFlightDeletions - sessions
locked in deletion states no longer occupy the capped read head, so a
stuck head (Poisoned / stranded Queued) cannot starve the retention
fanout tail. Warnings surface a fully-locked backlog and zero-progress
runs.
M8: ingest logs a per-batch warning with per-kind counts and the first
TelemetryItemId when unparseable telemetry items are dropped (agent
clears its spool on success, so drops were silent data loss).
M9: ReconcileSessionRebootCountAsync -> ReconcileSessionCountersAsync;
one partition scan reconciles EventCount and RebootCount at terminal
transitions, self-correcting at-least-once retry double-counts. Script
counters stay increment-only with the drift documented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GetEntityIfExistsAsync never throws on 404 (handled via HasValue), so the
RequestFailedException catch in GetOrphanedEventSessionsAsync fired only on
transient errors (429/500/503) and marked live sessions as orphans, causing
their events to be hard-deleted. Log a warning and skip the entry instead;
it will be re-checked on the next maintenance run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The V1 agent line is retired: no new V1 sessions since 2026-05-09, only
orphaned straggler traffic remained on /api/agent/ingest (last seen
2026-06-29, events for long-cleaned-up sessions). Bootstrap/Intune/CI
have deployed V2-only since the v2 cutover.
Agent:
- Delete AutopilotMonitor.Agent, .Agent.Core, .Agent.Core.Tests
(projects + sln entries). V2 trio, SummaryDialog, DecisionCore.Tests
and V2Isolation.targets stay.
Backend:
- Relocate the shared statics/DTOs V2 reused from the V1 partials
(StampServerFields, SanitizeEventTimestamps, IsMaxLifetimeAgentShutdown,
RecalculateAppDurations, EventClassification, AppInstallAggregationState)
into EventIngestProcessor; IngestEventsOutput moves next to
IngestTelemetryFunction (same namespace, zero ripple).
- Delete IngestEventsFunction (route agent/ingest), BootstrapIngestEvents
(route bootstrap/ingest), NdjsonParser and their DI registration.
EventDataNormalizer stays (used by TelemetryPayloadParser).
- Drop both routes from EndpointAccessPolicyCatalog and
RequestTelemetryMiddleware.TenantHeaderTrustedPaths.
- Retire the V1 update-hash oracle: LatestAgentV1*/LatestBootstrapV1*
fields + GetAgentLine case 1 (default arm returns empty hashes, so
stragglers gracefully skip the integrity check; ParseAgentMajor keeps
its default of 1 on purpose — defaulting to 2 would feed V2 hashes to
V1 binaries). TableConfigRepository Store+Map updated in lockstep.
- Remove ApiEndpoints.IngestEvents/.BootstrapIngestEvents constants.
Tests: NdjsonParserTests deleted; timestamp/critical-path/max-lifetime
tests repointed to EventIngestProcessor; the two NDJSON pipeline tests
removed (coverage retained by direct-list siblings); catalog/no-store
InlineData rows updated.
Docs: architecture.md (endpoint tables, ingest flow diagram, solution
layout, test matrix), web SectionAgentInternals endpoint tables,
build.ps1 help text.
Azure: no config change needed — /api/agent is not in the client-cert
exclusion paths (agent/ingest was always cert-enforced) and the
/api/bootstrap prefix exclusion remains for the surviving bootstrap
routes (verified via ARM on autopilotmonitor-api).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses four items from the daily code review, plus in-flight Web work and
the .mcp.json endpoint switch to the custom domain.
Review follow-ups:
- MCP access-guard: bound BOTH rate-limiter maps (preAuthBuckets + rateBuckets)
via boundedSet on insert (new MAX_RATE_BUCKET_ENTRIES, default 10_000). They
previously only shrank via the 5-min reaper, so a distinct-key flood could grow
them unbounded between passes. Adds getRateBucketSizes() + a flood-cap test.
- MCP vector-search: assertUnitNorm() in embed()/index() (eps=1e-3, fail-loud).
cosineSimilarity is a bare dot product that assumes L2-unit vectors; a violated
normalize:true contract would otherwise silently corrupt ranking. +unit tests.
- Backend: equivalence tests for the bcd307d SessionsIndex projections, after the
b417963 pattern. Extract MetricsSummaryProjection/QuickSearchProjection named
arrays + testable TallyMetricsSummaryRow/ReadQuickSearchIdentity helpers; make
BuildQuickSearchResult internal. New Metrics/QuickSearch ProjectionEquivalence
tests derive their keep-set from the production array (bite-verified: dropping
Status/DeviceName fails them).
- Agent V2 ModernDeploymentTracker: enforce the Classify() call-once invariant
with a DEBUG-only [ThreadStatic] permit guard. Classify mutates the harmless-
rollup counter, so a double-call double-counts; the invariant was comment-only.
ProcessRecord/ProcessEvent issue one permit before their single call; Classify
asserts + consumes it. Compiled out in RELEASE -> zero production impact.
Also included (per request):
- .mcp.json: point at https://mcp.autopilotmonitor.com/mcp (custom domain).
- Web useAggregatedAdminScope hook + seed + tests (separate in-flight work).
- Agent buildcounter.txt bumps.
Tests: MCP 338, Backend 2551, V2.Core 1538, Web hooks 31 - all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OrphanEventsCleaned ops-event only persisted counts; the per-orphan
detail (which tenant/session lost lingering events) was logged via worker
LogInformation, which does not reach App Insights, so it was effectively
lost. Enrich the event details with a capped (top-50 by EventCount) list of
{tenantId, sessionId, eventCount} plus a detailsTruncated flag, reporting the
actual rows deleted. Keeps the Table row under the 32KB property limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the sinceUtc bounding (commit "project index scans + bound
AppInstall reads to window"): the AppInstall cutoff was computed with a fresh
DateTime.UtcNow.AddDays(-days) AFTER the session window had already been
queried and several aggregates had run. That second snapshot sits a few ms
later than the session query's, so the AppInstall lower bound was strictly
later than the session lower bound.
A session included right at the window's lower boundary could then keep its
row while app summaries whose StartedAt fell into the gap between the two
cutoffs were dropped — undercounting AppScripts.AvgAppsPerSession and
TotalUniqueApps for boundary sessions.
Capture a single `now`/`windowStart` snapshot at the top of both
ComputeUsageMetricsInternalAsync and ComputeTenantUsageMetricsInternalAsync
and reuse it for GetSessionsByDateRangeAsync(windowStart, …) and
Get(All)AppInstallSummariesAsync(windowStart). Now both bounds are identical.
UsageMetricsWindowConsistencyTests pins it: it captures the start arg passed
to the session repo and the cutoff passed to the app repo and asserts they
are equal (and thus <=), on both the platform and tenant paths. Verified the
test bites — reverting either call to a fresh UtcNow fails it. All 2546
backend tests pass.
Reported-by: Codex review
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GetSessionStatsAsync drained the entire window via GetSessionsAsync, which
materializes every ~40-column SessionsIndex mirror row (incl. the large
FailureSnapshotJson, all Os*/Geo*/PendingActionsJson columns) — but
AggregateSessionStats only reads Status, StartedAt and DurationSeconds.
Drain through FetchSessionsPageInternalAsync with a narrow projection
{ PartitionKey, RowKey, SessionId, Status, StartedAt, CompletedAt, DurationSeconds }:
- PartitionKey/RowKey/SessionId: TenantId map + page-cursor derivation.
- CompletedAt: ComputeEffectiveDuration's fallback for a Succeeded session
with no stored DurationSeconds — kept so the tally value is identical.
- IsPreProvisioned/ResumedAt deliberately omitted: they only feed the
WhiteGlove Part-2 duration branch, which is gated on status==InProgress,
and AggregateSessionStats never reads DurationSeconds for InProgress
sessions — so dropping them cannot change any reported value.
FetchSessionsPageInternalAsync gains an optional `select` param defaulting to
null (= full row), so its only other caller (GetSessionsPageAsync) is
unchanged. Cursor/HasMore/pre-migration fallback mechanics are untouched.
DurationSeconds has a history of display/calc bugs, so equivalence is pinned
by SessionStatsProjectionEquivalenceTests: it reduces a full mirror row to
exactly SessionStatsProjection (derived from the production array, so a future
column drop fails the test), maps both shapes through the real production
mapper, and asserts identical DurationSeconds + identical SessionStats across
a mixed fleet — including a WhiteGlove Succeeded (drops IsPreProvisioned/
ResumedAt) and a CompletedAt-fallback Succeeded. Verified the test bites:
removing CompletedAt from the projection fails it. All 2544 backend tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two classes of read-efficiency fixes on hot/aggregation paths. No response
shape or value changes — DTOs untouched, all 2541 backend tests pass.
P2 — select projections on full-mirror SessionsIndex scans (bandwidth only):
- GetMetricsSummaryAsync: cross-partition status tally now projects
[PartitionKey, Status] instead of the ~40-column mirror row (incl. the
large FailureSnapshotJson) -> ~10-20x less transfer.
- QuickSearchSessionsAsync: typeahead scan projects only the matched fields
(+ RowKey for the SessionId fallback) instead of the full row, for up to
QuickSearchMaxScannedRows rows per request.
P3 — push the existing sinceUtc window floor server-side at 8 AppInstall
call sites across 5 files (previously each dematerialized the whole table /
partition and filtered to the window in memory):
- UsageMetricsService (global + tenant): now.AddDays(-days)
- MaintenanceService.Aggregation (global + tenant, per aggregated day): targetDate
- SlaBreachEvaluationService (current ISO week): now.AddDays(-8) margin
- SlaMetricsService: startDate (matches the in-memory lower bound exactly)
- Get(Global)GeographicLocationSessionsFunction: cutoff
Each floor equals the session-window floor the apps are joined against
(app.StartedAt >= session.StartedAt >= floor), so results are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the Tenant Templates -> Tenant Groups rename down to identifiers,
tables, routes and DTOs (the earlier commit renamed only the UI labels). Done
now while the table is effectively empty; the new TenantGroups /
TenantGroupAssignments tables are auto-created on deploy, the operator re-enters
their group, and the old TenantTemplates table is dropped in Azure manually.
- Tables: TenantTemplates -> TenantGroups, TenantTemplateAssignments ->
TenantGroupAssignments (Constants + All).
- Entities/DTOs: TenantTemplate(Entity)/TenantTemplateAssignment(Entity) ->
TenantGroup*, property TemplateId -> GroupId.
- Repo/service/API: all *Template* methods -> *Group*; routes
/api/global/tenant-templates -> /api/global/tenant-groups, {templateId} ->
{groupId}; function class + file renamed; audit entity DelegatedTemplateAccess
-> DelegatedGroupAccess; NormalizeTemplateId -> NormalizeGroupId.
- Offboarding wipe list + tests updated.
- Web: api.tenantGroups, SectionTenantGroups (file renamed), response key groups.
Scoped strictly to the feature: unrelated 'template' usages (analyze-rule
templates, EmailTemplates, interpolate-rule-template) left untouched. Backend
2538 tests green; web tsc clean + vitest 494 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AddTenantToTemplate returned 500 on the first add: the membership-row write
threw because TenantTemplateEntity.CreatedDate was left at default(DateTime)
(0001-01-01), which is below Azure Table Storage's Edm.DateTime minimum (1601).
The meta row wrote fine only because Create stamps CreatedDate=UtcNow.
Make CreatedDate nullable so membership rows omit it (null is valid); meta rows
still carry it. Read-back maps null -> default. Confirmed via storage: the
affected partition had only the meta row, no membership row.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let a GlobalAdmin define a named Tenant Template (a bundle of tenants) and
assign a delegated-admin UPN to the template instead of to each tenant. A UPN's
effective scope = union of direct grants + every tenant in every assigned
template, resolved in DelegatedAdminService.GetScopeAsync — so middleware
enforcement, /auth/me delegatedTenantIds and MCP all inherit it unchanged.
Adding a tenant to a template grants it to all assignees at once (membership-
driven MSP management). Fully app-internal: no Entra group claim / Graph.
- Storage: TenantTemplates (PK=templateId; meta + membership rows) and
TenantTemplateAssignments (PK=upn) tables; meta-backed existence invariant in
both management reads AND the auth hot path (no ghost templates, no scope from
stray rows).
- Offboarding hygiene: property-only wipe now purges TenantTemplates membership
AND DelegatedAdmins rows for the offboarded tenant (closes a pre-existing
orphaned-grant gap).
- Management API: GlobalAdminOnly /api/global/tenant-templates/* (8 routes);
per-affected-tenant audit; all mutations route through the service so the
delegated-scope cache is invalidated in lockstep; existence/membership
invariant owned by the service (no ghost create, no false revoke audits);
templateId normalized case-insensitively at the service boundary.
- Web: SectionTenantTemplates management UI (create/rename/delete, add/remove
tenants, assign/unassign), triple-registered + api.ts block.
Backend 2538 tests green; web tsc clean + vitest 488 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The delegated rescue treats every GlobalReadOrAdmin + QueryParam/RouteParam route
as delegated-reachable, which had silently opened several platform-OPERATIONAL
global reads to delegated ("MSP") admins. Those should be Global Admin + read-only
Global Reader only - the platform operator's view, not the managed tenant's data.
- global/ops-events -> TenantScoping.None (Consent/Maintenance/Security/Tenant/
Agent/SLA - operator data). Its in-memory FilterByTenant is reframed as a
GA/Reader ?tenantId= drill (correctness, NOT a delegated-isolation boundary)
and extracted internal-static for direct test coverage.
- New EndpointPolicyEntry.ExcludeDelegated flag: the None-equivalent for routes
whose {tenantId} template forces RouteParam (the {tenantId}-convention test
forbids None there). The middleware ANDs !ExcludeDelegated into the delegated
rescue, so it can only RESTRICT access, never grant it. Applied to:
- global/customs-archive (+ the two drill reads)
- global/tenants/{tenantId}/deletion-manifests
- preview/notification-email/{tenantId}
GA + read-only Global Reader keep all of these.
Full reclassification of every delegated-reachable global/* route: customer
telemetry (sessions / metrics / apps / rules / audit-logs / mcp-usage / ...) stays
delegated-reachable; platform-operational + offboarding reads are now excluded.
Tests: Delegated_{OpsEvents,CustomsArchive,DeletionManifests,PreviewNotificationEmail}_*
+ GlobalReader_*_Allowed + catalog Theory PlatformOperationalReads_ExcludeDelegated.
Backend filtered tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Security-review hardenings for the delegated ("MSP") admin feature.
- MCP kill-switch (M1): a delegated admin's MCP access was auto-granted under
WhitelistOnly BEFORE consulting their McpUsers row, so an explicit Disabled
row was ignored. ResolveWhitelistStateAsync now reads a tri-state and a
Disabled row revokes a delegated caller's MCP access (a MISSING row does not
- the normal delegated case). Operators get one lever without unwinding the
delegated grants.
- Web defense-in-depth (M3): the dashboard session list + stats and the
geographic-sessions drill bind a free-typed / deep-linked ?tenant= to the
caller's delegatedTenantIds before sending (shared boundTenantToDelegatedScope).
The backend already bounds this; the client now never even asks for an
unmanaged tenant.
- Geo-sessions drill shows the blue "Delegated Admin View" banner (L2).
Backend filtered tests + web tsc + vitest (485) green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The tenant notification-group join/leave checks gated on the caller's HOME-tenant role
(IsTenantAdmin / IsTenantMemberOrGlobalAdmin) without verifying the group belonged to that
tenant. Once this branch admits a cross-tenant delegated ("MSP") reader past the broadcast
cross-tenant gate, a home-tenant Admin/member role let them join a DIFFERENT (managed)
tenant's admin/member notify group and receive its higher-tier notification pushes -
breaking the documented "delegated caller gets no notification pushes" contract.
Fix: a single pure gate SignalRGroupHelper.CheckNotifyGroupAccess binds the role to the
GROUP's tenant - admin tier = (sameTenant && IsTenantAdmin) || IsGlobalAdmin; member tier =
(sameTenant && IsTenantMemberRole) || HasGlobalScope. Add/Remove both call it (symmetric).
Preserves own-tenant Admin/member, GA cross-tenant, and GlobalReader cross-tenant member
read; closes the same latent gap for a GlobalReader who is also home-admin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review finding (High). GetAllSessionsPageAsync / GetAllSessionsAsync route any non-empty
tenantIdFilter straight to the per-tenant scan, ignoring allowedTenantIds. The middleware
only resolves the delegated allow-list for a cross-tenant target (namedTarget != JWT
tenant); when ?tenantId= equals the caller's own JWT/home tenant, crossTenant is false so
the scoped-route check is skipped — but /api/global/sessions is GlobalReadOrDelegatedSubset,
so a delegated-only caller is still admitted (with AllowedTenantIds for their managed set).
The repo then drilled the home tenant anyway → a delegated MSP user could read their home
tenant's sessions/stats without managing it or being a member.
Fix: new DrillOutsideBound guard — when allowedTenantIds != null and tenantIdFilter is not
in that set, return empty before the per-tenant scan. Applied to both the page (sessions
list) and the drain (stats funnel through GetAllSessionsAsync). The middleware contract is
unchanged by design (admit + publish AllowedTenantIds, handler/repo bounds); the bound now
holds for the home-tenant case too.
Tests: DelegatedBoundedAggregateTests.BoundedDrill/Stats_UnmanagedTenant_* (drill an
unmanaged tenant → empty, the per-tenant SessionsIndex/Sessions scan never runs);
PolicyEnforcementMiddlewareTests.Delegated_GlobalSessions_HomeTenantDrill_AdmittedButNotInAllowedSet
pins that the middleware admits the home-tenant drill but never puts it in AllowedTenantIds.
Backend 2489 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review findings.
High — delegated bounded aggregate could fall back to an unbounded all-tenant scan:
FetchAllSessionsPageInternalAsync filters the config tenant list by allowedTenantIds,
but when that intersection is empty (managed tenant has no config row, config
momentarily empty, or a casing mismatch) it still invoked the legacy unbounded
primary-table scan — leaking ALL tenants' sessions to a delegated caller on the
no-tenantId /api/global/sessions path. Fix: when allowedTenantIds != null, return an
empty page and NEVER touch the primary-table fallback (which stays the GA all-tenants
safety net). New DelegatedBoundedAggregateTests proves both: bounded+empty → empty with
the Sessions table QueryAsync never called; unbounded(GA)+empty config → fallback runs.
Low — override-only pages showed "Delegated Admin View — access to all tenants":
usage-metrics, sla, gather-rules, analyze-rules rendered the banner without a subtitle,
defaulting to the all-tenants wording that contradicts the bounded model. Now pass
globalAdminSubtitle(scope) so a delegated (always a concrete managed tenant) reads
"viewing tenant <x>", matching the aggregated pages.
Backend 2486 green, web 481 green + tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make a delegated ("MSP") admin a "scoped Global Reader": the same pages a Global
Admin sees, but every tenant switcher/filter is bounded to the managed allow-list.
Reader path only (readWrite deferred). MCP stays read-only (Phase 6 already covers
delegated reads).
Backend (bounded aggregate — reuses the GA per-tenant fan-out):
- global/sessions + global/stats/sessions: GlobalReadOrAdmin -> GlobalReadOrDelegatedSubset
(keep QueryParam). A delegated caller WITHOUT ?tenantId= is admitted as a bounded
aggregate; the handler restricts the cross-tenant fan-out to RequestContext.AllowedTenantIds.
With ?tenantId=X it drills into one managed tenant (middleware validates X in scope).
- TableStorageService: GetAllSessions{,Page,Stats}Async gain an optional allowedTenantIds
set; FetchAllSessionsPageInternalAsync swaps its Step-1 tenant source to that subset
(everything downstream — per-tenant query, inverted-tick merge, cursor — is reused).
- IsDelegatedReadTier += GlobalReadOrDelegatedSubset so the single-tenant drill rescue fires.
- SignalR AddToGroup/RemoveFromGroup: admit a delegated caller to a managed tenant-{id}
group (in scope); global-admins stays platform-scope-only.
Web:
- useAggregatedAdminScope / useGlobalAdminScope: cross-tenant mode is now on for a delegated
admin too (isGlobalAdmin = crossTenant; new isDelegatedScope). Tenant list bounded to the
managed subset; always a concrete tenant (no "All tenants" aggregate on analytics pages).
- TenantScopeSelector hides the aggregate option for delegated; GlobalAdminBanner gains a
blue "Delegated Admin View" variant (wired on all analytics pages + dashboard).
- Nav: Fleet group exposes Dashboard + Fleet Health + Software + Geographic + SLA + Usage;
showDashboard now includes delegated.
- Dashboard delegated-ready: crossTenant drives the /global endpoints + filter UI; empty
filter = bounded aggregate over managed tenants, a selected tenant = drill-in; global-admins
SignalR join stays real-GA-only; read-only (mutations already GA-gated). `?tenant=` deep-link
seeds the filter.
- Fleet is now a pure stats card grid; the per-tenant session browser /fleet/[tenantId] is
removed and cards link to /dashboard?tenant=<id>.
Deferred follow-ups: Audit Log + Geographic session drill-in for delegated (both diverge from
the scope-hook pattern / need tenantId threading).
Tests: backend 2484 green (catalog/middleware contract updated for the new subset tier),
web 481 green, MCP 292 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delegated (scoped-global / MSP) callers get cross-tenant ROUTING bounded to their
managed tenant set: every tool call must name a managed tenantId (no aggregate),
platform-only tools stay hidden, and enforceDelegatedTenant(ForPage) validates the
effective tenant client-side as defense-in-depth.
Backend: McpUserService resolves delegated scope and auto-grants MCP access under
WhitelistOnly; McpUserFunction emits delegatedTenantIds/delegatedRole. Recatalog
global/metrics/summary to QueryParam so delegated readers reach it only on their
managed ?tenantId= path (aggregate stays blocked). Authoritative enforcement is the
PolicyEnforcementMiddleware delegated rescue.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the write path for MSP mode: a GlobalAdmin can now grant/revoke/enable/
disable delegated cross-tenant read assignments from the portal, instead of
hand-editing the DelegatedAdmins table (no more Boolean/lowercase/5-min-cache
foot-guns). Mirrors the MCP-user management surface.
Backend:
- DelegatedAdminManagementFunction at /api/global/delegated-admins: GET list
(GlobalReadOrAdmin), POST grant, DELETE {upn}/{tenantId} revoke,
PATCH .../enable|disable (all GlobalAdminOnly). Grant defaults role
DelegatedReader, stamps status=Active + source=OperatorGranted + grantedBy;
rejects an unrecognized role (fail-closed). Service.UpsertAsync invalidates
the caller's scope cache, so a grant takes effect on the next request.
- IAdminRepository.GetAllDelegatedAdminsAsync + TableAdminRepository impl
(small full-table scan, off the hot path) + DelegatedAdminService.GetAllAsync.
- Catalog: 5 routes. The {tenantId}-bearing mutation routes use
TenantScoping.RouteParam (fail-closed convention; sets TargetTenantId to the
managed tenant). Policy stays GlobalAdminOnly so a delegated caller cannot
manage delegations (GlobalAdminOnly is not a read tier).
- Mutations are audited under the TARGET managed tenant's trail (CREATE/DELETE/
UPDATE, entityType DelegatedAdmin), so the customer sees who was granted
access to their tenant.
Frontend:
- SectionDelegatedAdmins settings section: grant form (UPN + tenant dropdown +
role), assignments grouped by admin, enable/disable/revoke. Triple-registered
(globalNavConfig ga-settings + settingsNavSections + [section] map).
- api.delegatedAdmins family (list/grant/revoke/enable/disable).
Backend build clean; 2473/2473 backend tests; web tsc clean; 481/481 web tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses two linked Codex findings created by the Phase-3a preview-gate bypass.
(1) HIGH — auto-admin escalation: the first-login auto-admin rule excluded only
GlobalReader, not delegated callers. An external MSP user delegated to manage
tenant B, logging into a home tenant A with no existing admins, would be silently
written as Tenant Admin of A — converting a read-only delegated assignment into
write authority over the MSP home tenant. Fix: needsAutoAdmin also excludes
delegated callers (!isDelegated). A delegated user who legitimately administers
their own tenant has a TenantAdmins row, so this only suppresses the silent grant.
Regression test added (delegatedTenantIds non-empty, memberRole null,
hasTenantAdmins false → NeedsAutoAdmin false, isTenantAdmin false, role null).
(2) side-effect class: /auth/me read the home-tenant config via the auto-creating
GetConfigurationAsync and then ran first-login onboarding side-effects (domain /
OnboardedBy write, "new tenant signup" Telegram, auto-re-enable). For a
delegated-only external MSP login this phantom-onboarded the home tenant. Fix:
read via the non-creating TryGetConfigurationAsync, and run the onboarding
side-effects only for a genuine home-tenant participant (not delegated-only, OR
global, OR an existing member). Genuine first-users still onboard normally.
Full Functions suite green (2473).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for the web MSP/fleet experience: the app now knows whether the signed-in
user is a delegated admin and which tenants they manage.
Backend (/api/auth/me):
- Resolve the caller's delegated scope (cached) alongside the existing parallel
fetches and return delegatedTenantIds (the managed tenant set) + isDelegated.
- Delegated admins bypass the private-preview gate, exactly like Global Admin /
Reader — an MSP whose own home tenant is not on the waitlist must still be able to
log in and manage its authorized tenants. Gate 1 (suspended home tenant) still
takes precedence. BuildAuthResult gains an optional delegatedTenantIds param
(existing call sites unaffected); 3 new tests + the side-effect harness updated.
Web (AuthContext):
- UserInfo gains isDelegated + delegatedTenantIds; AuthContextType gains
hasFleetScope = hasGlobalScope || isDelegated for fleet/switcher gating. All
/auth/me parse paths (success, prefetch, preview/error fallbacks) populate them.
Full backend suite green (2472); web tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses a Codex finding: no handler-level coverage existed for the delegated
config/all filtering/redaction path (coverage was policy-context + projection only).
Following the repo convention (handler HTTP entry not unit-tested — see
GetAllBlockedDevicesFunctionTests), the security-relevant transform is extracted
into two pure, testable seams on GetAllTenantConfigurationsFunction and tested
directly:
- ExistingManagedConfigs: from the per-tenant point-read results, keep only managed
tenants that actually have a config row (exists==true); drop the rest. Tested:
drops missing rows, keeps present ones, empty when none exist.
- DelegatedBareArrayView: redact every managed config's secrets (a delegated admin
is never a Global Admin). Tested: secrets -> placeholder, non-secrets preserved,
original instance not mutated.
5 new tests; full Functions suite green (2469).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses a Codex review finding: the Phase 2b delegated config/all path loaded
EVERY tenant config via GetAllConfigurationsAsync() (an O(all tenants) table scan
+ full-config materialization) and then filtered in memory. Because Phase 2b
opened this endpoint to delegated MSP users, a caller scoped to a few tenants
could trigger an all-tenants scan per request — response-bounded, not read-bounded.
Now point-read ONLY the caller's managed tenants via the non-creating
TryGetConfigurationAsync (cached; never writes a phantom default row), dropping
ids with no config row (exists==false). A delegated MSP scoped to k tenants does
k point reads instead of scanning n. The response is still filtered+redacted as
defense in depth; the set fed to it is now itself bounded.
Full Functions suite green (2464).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces the GlobalReadOrDelegatedSubset policy tier — a cross-tenant READ that
is BOUNDABLE to a subset — and applies it to config/all (the tenant list). A
delegated ("MSP") admin can now list ITS managed tenants in one call, bounded to
its allowed set; a Global Admin / Reader still gets all tenants (unbounded).
This is the server-side linchpin for the MSP fleet view + tenant switcher: the
frontend gets the managed-tenant list (id + domain) here, then builds the
per-tenant fleet grid by fanning out over that subset using the Phase-2a
single-tenant /api/global/*?tenantId= endpoints (subset fan-out, client-orchestrated
over already-bounded per-tenant reads — no all-tenants code path for delegated).
- Middleware: new EvaluateGlobalReadOrDelegatedSubsetAsync — GA/Reader admitted
unbounded; a delegated caller with a non-empty scope admitted with its managed
set carried on the decision and published as RequestContext.AllowedTenantIds.
DecideAsync merges this into the delegated flags + AllowedTenantIds.
- GetAllTenantConfigurationsFunction: when AllowedTenantIds is present, BIND the
response to that subset (both the bare-array and paginated shapes) and always
redact secrets (a delegated caller is never a Global Admin). The subset is small,
so it is returned in one shot.
- Catalog: config/all -> GlobalReadOrDelegatedSubset. Aggregates that cannot be
bounded stay GlobalReadOrAdmin + None (delegated-unreachable).
- Tests: delegated config/all admitted with bounded AllowedTenantIds; GA unbounded
(null bound); non-delegated non-global denied; tier-assignment guard. Full suite green (2464).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lets a delegated ("MSP") admin read its managed tenants through the cross-tenant
/api/global/* endpoints — but ONLY on the single-tenant ?tenantId=X path, bounded
by the middleware to a tenant in its allowed set. This is what makes the tenant
SWITCHER work end-to-end (drill into a managed tenant, see its normal dashboard).
The dangerous no-tenantId AGGREGATE path (fan-out over ALL tenants) stays
GA/Reader-only and is untouched here; bounded fleet aggregation is a separate step.
- Middleware: GlobalReadOrAdmin added to the delegated read tier. The existing
`isScopedRoute && crossTenant` guard means a delegated grant still fires ONLY on
a tenant-scoped route for an in-scope target — so a delegated caller reaches a
global endpoint only via its single-tenant path, never the aggregate path.
- Catalog: the 27 GlobalReadOrAdmin routes whose handlers strictly restrict to the
named ?tenantId= (verified by an exhaustive per-endpoint audit) recataloged
None -> QueryParam. QueryParam/RouteParam on a GlobalReadOrAdmin route now
formally means "handler restricts to the named tenant => delegated-reachable";
aggregate-only routes (presence, platform metrics, distress-reports, config/all,
…) stay None and remain unreachable by delegated callers. Documented inline.
- Tests: delegated single-tenant global read allowed in-scope / denied out-of-scope
/ denied on the no-tenantId aggregate path; an aggregate-only route stays blocked
even with an in-scope ?tenantId=; Global Reader aggregate path regression; and a
catalog-contract theory pinning the SAFE(QueryParam) vs AGGREGATE-ONLY(None) split.
Updated GlobalAppsRoutes completeness assertion to QueryParam. Full suite green (2461).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses a Codex review finding: the normal policy evaluator runs before the
delegated rescue, so an external MSP user's first delegated cross-tenant read
still touches the own-home-tenant role path. That path called
TenantConfigurationService.GetConfigurationAsync, which AUTO-CREATES and persists
a default TenantConfiguration row for a missing tenant — writing a phantom config
row for the MSP's (possibly non-onboarded) home tenant.
Authorization role resolution must be side-effect-free. ResolveEffectiveRoleAsync
now uses the existing non-creating TryGetConfigurationAsync (returns the in-memory
default without persisting). A missing config means EntraAppRolesEnabled = false
either way, so the resolved role is unchanged — only the phantom write is gone.
This also removes the same latent write for ANY authenticated non-member.
Regression test: "delegated read does not create config for MSP home tenant"
(verifies SaveTenantConfigurationAsync is never called). Full Functions suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires the delegated ("scoped global" / MSP) tier into the auth middleware. A
delegated admin signs into their OWN home tenant but may now READ a SUBSET of
tenants — exactly the tenants in its DelegatedAdmins scope — without holding
full platform (GlobalAdmin/GlobalReader) scope.
Mechanism (PolicyEnforcementMiddleware.DecideAsync):
- Resolve the target tenant up-front (ResolveTarget), then resolve delegated
scope LAZILY — only when the caller lacks full platform scope AND the route is
a cross-tenant, tenant-scoped route. Keeps the common authenticated path free
of the extra (cached) table read. Service errors still fail closed to 503.
- A delegated grant applies ONLY to read tiers {MemberRead, TenantAdminOrGlobalReader}
on RouteParam/QueryParam routes, and ONLY for a target in the caller's scope.
It rescues a tier-denial for a delegated-only caller and lifts the cross-tenant
block for that target. Write tiers never admit a delegated caller (even a
DelegatedAdmin row grants read-only this phase), so no read->write escalation.
- Aggregate /api/global/* endpoints (GlobalReadOrAdmin) fan out over ALL tenants
and are deliberately NOT reachable by delegated callers until Phase 2 bounds
them to AllowedTenantIds.
RequestContext gains IsDelegatedReader/IsDelegatedAdmin/IsDelegated, HasFleetScope
and AllowedTenantIds (the delegated subset, for Phase 2 aggregation).
Secret-leak hardening: GetTenantConfigurationFunction now redacts by default —
only a Global Admin or the tenant's OWN admin sees unredacted secrets
(CanViewSecrets, extracted as a testable static). Behavior is identical for all
pre-existing roles; it closes the path where a cross-tenant delegated/global
reader would otherwise receive SAS/webhook secrets.
12 new tests (delegated allow/deny matrix + CanViewSecrets clearance). Full
Functions suite green aside from two pre-existing timing-flaky tests (unrelated
code, pass in isolation).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Foundation for "MSP mode" — a scoped-global authorization tier between a
single-tenant member and a platform GlobalAdmin. A delegated admin may access a
SUBSET of tenants (its assignment rows), resolved UPN-based and tid-agnostic,
exactly like GlobalAdmins.
Phase 0 is purely additive: the service is registered but no auth path consults
it yet (the cross-tenant gate change lands in Phase 1), so behavior is unchanged.
- Constants: DelegatedRoles/DelegatedStatus/DelegatedSource; new DelegatedAdmins
table added to TableNames.All (auto-create), CriticalBackupTables.All and
AuthTablesFullRestoreForbidden (identity table — full-table restore blocked).
- IAdminRepository + TableAdminRepository: DelegatedAdminEntry DTO + CRUD
(PK=UPN, RK=TenantId). Hot-path GetDelegatedTenantsAsync (partition scan) and
reverse GetDelegatedAssigneesAsync. Queries use the typed predicate overload
(no string-interpolated OData filter — avoids filter injection).
- DelegatedAdminService: resolves DelegatedScope (Covers/RoleFor/CanWrite) with a
5-min cache + write-through invalidation. Fail-closed: only Active+enabled rows
with a recognized role confer scope; empty role -> DelegatedReader; unknown -> dropped.
- 16 unit tests (fake repo); full Functions suite 2434/2434 green. Backup
catalog-count assertion bumped 15 -> 16.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ingest hot path + Graph auth latency. IngestTelemetry is the #1 backend
endpoint (~35.6k/day); its p99 tail was dominated not by parsing but by an
uncached Graph token: GraphTokenService minted a fresh client-credentials
token (500-1900ms POST) on every device-validation cache miss and threw it
away. Cache it per tenant.
GraphTokenService:
- Per-tenant token cache keyed on the validated tenantId; TTL from the
token's own expires_in minus a 5min skew margin, clamped to [60s, 55min]
(expires_in was previously ignored). Only successful tokens are cached.
- Per-tenant SemaphoreSlim stampede gate (bounded wait) so an enrollment
wave does not fan a burst of identical token POSTs at AAD.
- InvalidateTenant() drops the token + consent-status entries so the
consent/Graph-permission fresh-read contract still holds.
- Per-tenant invalidation generation closes the write-after-invalidate
race: a POST in flight when InvalidateTenant runs tags its entry with the
pre-invalidation generation, which readers reject.
GraphFeatureDetector:
- InvalidateTenant() now also invalidates the underlying token cache, and
applies the same generation guard to its own parsed-roles cache.
- Cross-layer order is load-bearing: invalidate the token layer FIRST, then
bump the detector generation, so a concurrent acquire cannot pair a new
detector generation with a stale token read.
IngestTelemetryFunction:
- Deserialize the request body directly from the buffered MemoryStream via
JsonTextReader instead of materializing a full UTF-16 string + ToArray()
copy (peak ~4N -> ~1N). Cap still applies before parsing; same parse
semantics. Pure allocation hygiene on the hottest endpoint.
Tests: token caching, cross-tenant isolation, failure-not-cached, stampede
coalescing (barrier-gated), in-flight write-after-invalidate, cross-layer
invalidation, and cross-layer ordering -- each verified red without its fix.
Parse hygiene: PayloadJson stays a raw string, malformed -> 400.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The app dashboard and metrics endpoints loaded entire AppInstallSummaries
partitions (tenant scope) or the full table (global scope) and filtered
StartedAt >= cutoff in memory, even for days=30. The device-model join then
issued one serial GetSessionAsync point-read per distinct session.
- Add optional DateTime? sinceUtc to GetAppInstallSummariesByTenantAsync /
GetAllAppInstallSummariesAsync: when set, a server-side StartedAt ge filter
scopes the scan so only in-window rows are deserialized/transferred. Default
null preserves the legacy full scan for callers that need all history.
- AppsAnalyticsHelper.LoadSummariesAsync derives sinceUtc from days; all six
/api/apps/* functions pass it. The four metrics/app + metrics/geographic
functions pass their existing cutoff. The in-memory Where stays as the exact
sub-second trim (the OData filter is second-granular). MCP get_app_install_metrics
benefits automatically (days already flows through; response shape unchanged).
- Replace the serial session point-read loop with a bounded-concurrency
(SemaphoreSlim=10) LoadSessionLookupAsync, shared by the analytics and
sessions builders.
AppInstallSummaries is already bounded by the session-retention cascade
(DeletionManifestBuilder) + tenant offboarding, so no separate table retention
is added. Full Backend suite: 2406 passed / 0 failed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
QuickSearchSessionsAsync walked the full SessionsIndex partition (or the
whole table for global scope) on every typeahead request, since
exact-over-fuzzy ordering forbids an early break on fuzzy-only/no-match
queries. Cost grew with the index.
Bound the scan without a new index:
- recent-window RowKey bound (RowKey lt cutoff, 90d) via the existing
ComputeCutoffRowKeyPrefix; applies to tenant and global scope
- hard scanned-row cap (5000) as backstop for the no-early-break case
Match/ordering logic unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The retention fanout materialized the entire eligible-session backlog every
run (GetSessionsOlderThanAsync returned a full List<SessionSummary>) but only
advanced MaxEnqueuesPerTenantPerRun=100 per tenant. A large backlog was reread
and rematerialized each run, so maintenance got slower as the backlog grew
while only 100 sessions progressed.
Cap the read at the per-run dispatch limit: GetSessionsOlderThanAsync gains a
maxResults parameter that caps maxPerPage and stops enumerating once the cap is
reached. The fanout fetches MaxEnqueuesPerTenantPerRun+1 (the +1 is a cheap
"more remaining" probe) instead of the whole backlog. Read cost is now
proportional to the cap, not the backlog; per-run progress is unchanged.
The exact eligible count is no longer known without a full scan, so the audit
now records EligibleThisRun + a MoreRemaining flag instead of the exact total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both are self-hosted poll-loop workers (no QueueTrigger), so their poisoned
work was invisible to MaintenanceService poison-queue alerting and the
HealthCheckService health card. Add both poison siblings to
MonitoredPoisonQueues and update the watch-list guard test (4 -> 6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codex review #7 (MEDIUM): AuthenticationMiddleware read the unvalidated
`tid` claim and built a tenant-specific OIDC authority before signature
validation. A garbage tid would drive an outbound metadata fetch to a bogus
authority - a DoS/cost surface on malformed-token floods (not an auth bypass;
signature/audience/issuer are still validated afterward).
- Guard `tid` with Guid.TryParse before constructing the authority; missing/
non-GUID throws SecurityTokenValidationException -> 401, no fetch. Loss-free:
every legitimate Entra token (incl. consumer MSA 9188040d-...) has a GUID tid.
- Drop the now-unreachable null -> common/v2.0 fallback.
- Pass context.CancellationToken into GetConfigurationAsync (was None).
Negative cache for invalid/unreachable authorities deferred (LRU already
bounds memory; revisit only if floods observed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Several append-only tables had no purge mechanism and grew unbounded. Add
age-based retention to the 2h MaintenanceFunction (and the manual path) via a
new CleanupUnboundedTablesAsync orchestrator (per-table try/catch, fixed const
windows, mirroring the DistressReports pattern):
- GlobalNotifications + TenantNotifications: hybrid retention — dismissed rows
pruned 30d after DismissedAt, any row pruned 180d after CreatedAt. Unread
(still-actionable) admin warnings survive the dismiss window so they are never
silently lost; the 180d catch-all bounds the table. Shared pure predicate
NotificationRetentionFilter.BuildPredicate (unit-tested).
- HardwareRejectionNotificationTracker: 30d (FirstNotifiedAt). Pruning re-arms
the lifetime dedup, so a model rejected again >30d later re-bells (intended).
- AuditLogs: 180d (system Timestamp, cross-partition).
- UsageMetrics: 180d (PartitionKey date-string range).
- BackupJobs: 365d (QueuedAtUtc).
PlatformStats intentionally excluded — it is a single upserted row, not
append-only. Bootstrap sessions left as-is (feature currently unused).
Each repo delete mirrors DeleteUserActivityOlderThanAsync: server-side
"X lt cutoff" filter + select PK/RK + delete loop. Deletes are resumable across
runs, so a partial run self-heals on the next cycle.
Observability: end-of-run MaintenanceLongRunning Warning OpsEvent if RunAllAsync
exceeds 10min, dual-registered in OpsAlertRulesSection.tsx, to surface a run
climbing toward the 60min functionTimeout before a future hard-abort (which
emits nothing).
Inert until backend redeploy; first post-deploy run bulk-prunes the backlog.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Global-Admin cross-tenant view of web users active "now" (made an authenticated
request within a window, default 5min), backed by a new self-maintaining
UserPresence table. Not App Insights (sampling) / not SignalR.
Backend:
- UserPresence table (one row/user, PK=tenantId, RK=SHA-256(lowercase UPN));
upserted fire-and-forget by UserPresenceMiddleware (after PolicyEnforcement,
60s per-process throttle, UPN-gated so agent cert calls are skipped).
- GET /api/global/presence (GlobalReadOrAdmin) via GetActiveUsersFunction; read
path propagates storage errors (5xx) instead of masking them as "0 active".
- 1-day presence retention + 90-day UserActivity retention in maintenance;
PlatformStats.TotalUsers clamped monotonic so the public "Users Seen" figure
cannot regress after a prune.
Web:
- /admin/presence page (live counter + table), nav entry under GA > Metrics.
- "Total Users" relabeled "Unique Users (90d)" to match the windowed metric.
Backend+web only, no ConfigVersion bump, inert until redeploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When a user authenticated successfully but was not on the MCP whitelist,
the MCP server returned an opaque "MCP access denied" 403 — the user could
not tell why auth was interrupted or that they simply needed to be
whitelisted by the server owner.
- access-guard: distinguish a genuine authorization denial (not whitelisted)
from an infrastructure failure (backend unreachable / malformed) via a new
infraError flag on AccessCheckResult. Only a genuine deny gets the
user-facing "User not enabled for MCP usage" error plus an actionable
message naming the account and pointing at the fix (ask to be whitelisted).
Infra failures keep a neutral "MCP access check unavailable" message so the
user is not sent chasing the wrong fix. The reason field still echoes the
backend reason verbatim.
- backend McpUserService: WhitelistOnly denial reason is now self-explanatory
("User not enabled for MCP usage (account is not on the MCP whitelist)").
- tests: cover the actionable whitelist message and that an infra failure is
not mislabelled as a whitelist problem.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RKYavm4FAu66qnhX5nvmMT
The ticket-gated diagnostics/download route (PublicAnonymous, MCP session_debug)
401'd on the deployed backend: AuthenticationMiddleware kept its OWN hardcoded
_anonymousRoutes allowlist, parallel to EndpointAccessPolicyCatalog, and the new
route was added only to the catalog. Since AuthenticationMiddleware runs before
PolicyEnforcementMiddleware, the missing entry rejected the request with a JWT
401 before the PublicAnonymous policy was ever honored. Classic hand-synced
parallel-catalog drift.
Delete _anonymousRoutes / _anonymousPrefixes and derive the JWT-exempt decision
from the catalog: a route skips JWT iff its policy is PublicAnonymous or
DeviceOrBootstrapAuth (authenticated in-function via cert/bootstrap token, not
JWT). Now method-aware and fail-closed on unregistered routes. Behavior-
preserving: the old 14 routes + 1 prefix equal exactly the catalog's
PublicAnonymous u DeviceOrBootstrapAuth set.
Add drift guard in EndpointPolicyCatalogCompletenessTests:
AnonymousAndDeviceRoutes_AreJwtExemptInAuthMiddleware loops every such catalog
entry and asserts the middleware exempts it (would have caught this bug);
SkipsJwtValidation_MatchesPolicyTier locks the positive/negative tiers incl. the
diagnostics/download vs diagnostics/download-url contrast. Full suite 2393 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lets tenant-admins and GAs run end-to-end session debugging over MCP: backend
telemetry plus the agent diagnostics ZIP, downloaded and analyzed locally by the
AI client. The backend never unzips/parses the archive (no parse CPU); it only
mints a short-lived signed download ticket and proxy-streams bytes.
Backend (no ConfigVersion bump, no agent change):
- DiagnosticsDownloadTicket: HMAC codec (reuses PaginationTokenSigningKey,
purpose-tagged diag-dl-v1, 10-min TTL) so a no-JWT client can download.
- DiagnosticsBlobStreamer: shared proxy-stream engine; DiagnosticsDownloadFunction
(web/JWT) refactored onto it. ClassifyBlobName/BlobDestination stay put (xUnit).
- POST diagnostics/download-ticket (MemberRead, QueryParam, GA-bypass) +
GET diagnostics/download?t= (anonymous; tenantId/blobName read only from the
signed ticket -> no retargeting). Both added to NoStoreCacheMiddleware.
MCP:
- get_session_diagnostics tool (all roles) -> absolute downloadUrl + static zipMap
+ instructions; unwraps the {success,session} envelope.
- diag_zip_layout catalog (get_resource + protocol resource).
- debug-session prompt (tenant + GA variants).
- Tool descriptions flag the client requirement (needs local file/shell tools;
pure chat clients cannot unzip).
Tests: backend 2380/2380 (ticket codec, NoStore, classify, catalog completeness);
MCP 261 passed (incl. envelope-unwrap regression). Inert until redeploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GlobalReader = cross-tenant READ visibility like a Global Admin, but no mutating
operations and no platform settings. Provisioned exactly like a GA: a GlobalAdmins
table row with Role=GlobalReader (empty/legacy Role = GlobalAdmin, back-compat).
Semantics are ADDITIVE — a user who is also their own tenant's Admin keeps edit
rights on that tenant; a pure GlobalReader is read-only everywhere.
Backend:
- GlobalAdmins.Role column + GlobalAdminService.GetGlobalRoleAsync; IsGlobalAdminAsync
(incl. TableAdminRepository) delegates so a GlobalReader row never reads back as admin
- RequestContext.IsGlobalReader + HasGlobalScope
- Split GlobalAdminOnly into GlobalReadOrAdmin (cross-tenant reads) vs GlobalAdminOnly
(writes/settings) + TenantAdminOrGlobalReader for config-read
- PolicyEnforcementMiddleware: extracted testable DecideAsync seam; cross-tenant gate keys
off HasGlobalScope; additive own-tenant role carried so IsTenantAdmin stays correct
- Config secret redaction for readers (RedactedCopyForReader) + save-time placeholder
restore (RestoreRedactedSecretsFrom) so a redacted view can't blank real secrets
- raw table/log tools stay GlobalAdminOnly; SignalR global-admins read group admits HasGlobalScope
- /auth/me emits isGlobalReader (+ preview-gate bypass, no auto-admin); /auth/mcp emits globalRole
MCP: hasGlobalScope (GA or Reader) drives routing + tool catalog; raw secret-bearing tools
(query_table/list_tables/query_backend_logs) gated strict-GA only.
Web: AuthContext isGlobalReader + hasGlobalScope; reader reaches /dashboard + cross-tenant
views; platform Settings/Ops/Backups/Software GA-only (nav-hidden + requireGlobalAdmin route
layouts); mutating controls on mixed data pages disabled via useCanMutatePlatform; rules pages
read-only on foreign tenants; additive own-tenant edit preserved.
Tests: backend 2369, MCP 252, web 469 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds WebhookProviderType.GenericJson (20) so a tenant can POST a stable,
channel-agnostic JSON payload to any HTTPS endpoint - ticketing systems,
automation, or an SMTP gateway like Postal - instead of only Teams/Slack.
Backend:
- GenericJsonRenderer emits schemaVersion 1.0 with eventType, severity,
title/summary/themeColor, primaryUrl (first openUrl action), facts[],
sections[], actions[]; registered in WebhookNotificationService.
- NotificationAlert.EventType set on all 8 builders (enrollment_succeeded/
_failed/_started, preprovisioning_*, hardware_rejected, sla_breach/_resolved,
consecutive_failures, test) for consumer-side routing.
- Per-tenant WebhookCustomHeadersJson + GetGenericWebhookHeaders() (generic-only,
restricted-header denylist, fail-soft parse) for API-key auth; dispatcher
gained optional customHeaders applied via HttpRequestMessage; threaded into
all 12 tenant call sites (Ops/admin-config sites unchanged).
- TableConfigRepository persists the field; UpdateTenantConfigurationFunction
validates it (object, string values, RFC-7230 token names, CRLF-injection
guard, 8 KB / 25-header caps).
Web:
- Generic provider option + key->value WebhookHeaderEditor (rendered only for
provider 20); leaving the generic provider clears local headers so a later
switch back cannot revive stale auth headers; context/types wiring sends
headers only when provider===20; report shows Generic JSON provider name.
ConfigVersion unchanged (backend-only, no agent payload). No new
Constants.EventTypes, so no MCP catalog sync. Still fire-and-forget (no SMTP
client / retry by design). New GenericWebhookTests cover renderer, header
parsing/denylist, and save-time validation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Oliver Kieselbach