A version whose installer was already in the archive could only turn
Archived through the 409 on the write-once upload, which needs a
distribution host that still serves the build. For a version no host
serves any more the row stayed Failed:VersionMismatch next to its own
blob and re-walked every host once a day.
The archiver now checks the archive first: an entry whose ProductVersion
is the observed version ends the job as Archived with the hash and size
of the archived bytes and no download; a missing provenance sidecar is
written without a source URL. An entry holding another build, or an
unreadable one, is not accepted and the walk runs as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The SLA breach alert judged the calendar month while the dashboard headline
showed the current ISO week, so an alert could report 78 % while the page it
links to showed 0 everywhere - and an empty week rendered as 0.0 % "Breached".
- SlaEvaluationWindow is the single definition of the SLA populations:
success rate and P95 duration on terminal sessions of the last 30 days
(rolling), app installs on the install attempts of the current ISO week.
The breach evaluator and SlaMetricsService both read it, including the
one-decimal rounding, so the value shown is the value judged.
- SlaMetricsResponse carries EvaluationPeriod and EvaluationWindowDays;
SlaSnapshot carries Period and HasData. A period without a finished
enrollment meets every target flag and is marked HasData = false.
- The dashboard reads from whichever starts earlier, the selected months or
the evaluation window; trend and violators stay on the selected months.
- A lasting breach is notified again only after the cooldown AND once
something finished since the last notification. A new breach episode after
a resolve still notifies. Resolve notifications are unchanged.
- Alert period fact and telemetry tag follow the window constant.
- SlaMetricsService takes a nowProvider; the week-boundary workaround in its
tests is gone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A retry after a poisoned offboarding is a new envelope with a new history
row, and nothing ever deleted from a poison queue. The dead letter of the
failed run therefore stayed until its 7-day TTL and kept the poison backlog
alert firing daily for a tenant that was already gone.
TenantOffboardingHandler.HandleAsync now reports a TenantOffboardingOutcome.
On Completed the worker receives tenant-offboarding-poison in batches and
deletes the envelopes of that tenant; other tenants' and malformed messages
reappear untouched. The sweep never throws: a failure there would keep the
completed envelope on the main queue and end with a Completed run in poison.
A re-pickup of a completed envelope sweeps again.
Other poison queues stay terminal on purpose; their failures have no later
success that supersedes the dead letter.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The trigger ran the whole maintenance inside the HTTP request (about three
minutes against a 230 s gateway limit): the portal reported a timeout while the
run went on, a second click started a second run, and nothing serialized those
runs against each other or against the 2h timer.
Backend
- MaintenanceRunGate: the timer and the manual run enter through one blob
lease. A held lease writes MaintenanceSkippedLocked and runs nothing;
MaintenanceStarted is written after the lease is held, the lease is released
last.
- POST maintenance/trigger validates the date, probes the lease (409 while a
run is active), enqueues on maintenance-trigger (fail-hard) and answers 202
with the typed TriggerMaintenanceResponse. MaintenanceTriggerQueueWorker
(self-polling) runs it; a failing or skipped run is not retried.
- The run report travels in the details of MaintenanceCompleted.
- BlobLeaseLockStore is the shared base of the lease lock stores;
SessionDeletionMaintenanceLockStore derives from it, behaviour unchanged.
- The probe lets a failed lease release surface as 5xx: a probe lease still
held at worker pickup would turn the queued run into a silent skip.
Web
- useOpsRunStatus: run status from ops events (latest Started newer than the
latest Completed/Failed = active), used by the session-cleanup banner
(rendering unchanged) and the maintenance trigger section.
- Trigger section: queued / already-active messages, status line, polls every
15 s while a run is active, button disabled meanwhile. The 230 s fetch
timeout is gone.
Shared manifest, generated wire types and the MCP ops-event vocabulary
regenerated (two new ops event types, reshaped response).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A Failed offboarding was a dead end: the worker returns early on a Failed
history, the offboard re-click refuses to re-enqueue on a Failed marker, and
the marker cleanup only warns. Recovery meant deleting the marker row in Table
Storage by hand, while the tenant sat in the list as a tombstone counted on the
Waitlist (its whitelist row is wiped early in the cascade).
- GET global/tenants/{tenantId}/offboarding returns the offboarding record
(marker status, failed phase, attempts, history details) as the typed
TenantOffboardingStatusResponse; GA + Reader, never delegated.
- POST global/tenants/{tenantId}/offboarding/retry re-drives a Failed run: the
History row keeps its RowKey and DrainCompletedAt (the worker's resume anchor)
and counts the retry, marker and history go back to Initiated, the ordinary
resume path re-affirms the Disabled-gate, waits out the remaining drain
barrier and enqueues the envelope. Completed / in-flight answer 409, no
marker 404, a missing history row 500 before any write. Global audit row
RETRY/TenantOffboarding. Nothing retries on its own.
- Tenant editor: an "Offboarding" card replaces the Offboard action while the
cascade owns the row (status, started by, attempts, failed phase and error);
Failed offers "Retry offboarding…" behind the type-to-confirm dialog in its
retry variant.
- Tenant list: tombstone rows carry an Offboarding badge (own Status facet)
instead of Suspended, and no longer count as Waitlist.
- Tests: retry decision (reset order, drain anchor, conflicts, not found,
enqueue failure, remaining-barrier delay), status projection, wire parity
facts, filter facet. Shared manifest and generated wire types regenerated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XPG5WCmSwfNAv4xkK62n5r
The landing-page counters stood still since D-198: the two-hourly recompute
was their only writer and merged Max(live figure, stored figure), while
retention prunes the live figure below the stored one - so new enrollments
never showed up again.
- Hot path increments the per-tenant counters only (CAS): enrollments at the
fresh registration, successes at the one status seam every Succeeded writer
passes (transitioned && Succeeded), events per ingest batch. No batch
touches the global row.
- The maintenance run floors the tenant counters at the live figures (one
multi-field write per tenant), sums the tenant rows and adds the growth
since the previous run to the platform row. The previous sums live in a
baseline row written in one transaction with the platform row. A total is
never derived from an absolute figure again: a shrinking source (retention,
offboarding) costs the growth of one interval instead of freezing the
counter. Without a baseline the sum is a lower bound only.
- TotalTenants rolls up the tenant rows with enrollments, UniqueDeviceModels
a persisted set of seen models. TotalUsers stays a high-water-mark (a
persisted user set would outlive the user-activity retention).
- The platform row is merged with its read ETag and never carries
IssuesDetected: the former full-row upsert undid every increment that
landed during the run.
- Reads that feed the baseline are not fail-soft, and a floor that could not
be written skips the rollup of that run - a swallowed zero would be added
as growth a second time.
Shared manifest and generated wire types: doc comment of PlatformStats only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The wipe of the tenant's own self-service Tenant Group handed the derived
group id ("msp-{tenantId}") to the tenant-GUID entry points of SafeWipeService.
Their GUID guard threw on every attempt, so each offboarding since the
self-service delegation change ran into max_dequeue and ended Failed after the
tenant tables were already wiped.
- SafeWipeService: WipeByOwnedGroupPartitionAsync / WipeByOwnedGroupRowKeyAsync
take the tenant GUID and derive the group id themselves; matching filters in
OffboardingFilters.
- The public wipe entry points are no longer virtual: they validate and hand
over to protected virtual cores. Test fakes override the cores only, so the
guard runs under every fake. With the old handler code 16 handler tests fail
on the guard, which is how this shipped unnoticed.
- Tests: handler regression on the owned-group anchors, owned-group variants
incl. verify-abort on a foreign group, filter shapes and guards.
Deploy and recovery of the stuck offboarding: B-5g6.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Backend/Shared:
- McpQuotaService: ResolveHomeAsync reads definitions, entitlements, plan name and purchased slots once per check; Grow adds slots × per-slot value to every window (a lifted window stays unlimited); the decision and McpPlanLimits carry the breakdown; CheckManyAsync, the charge scope and TargetTenantId are gone
- McpQuotaEnforcementMiddleware: one home charge for every caller; IsExempt = Global Admin only (counted, never blocked; the Global Reader stays quota'd); two 429 texts (organization / user)
- TenantEntitlementService.GetPurchasedDelegatedSlots = max(0, effective slot limit − catalog count), 0 without the delegation right; FeatureEntitlementCatalog gains McpSlot*RequestLimit (Pro 300/6000 per account, 900/18000 per organization = 30 % of the Pro windows; Community and conferred Pro 0)
- PlanTierDefinition gains four nullable slot fields; GET/PUT global/config/plan-tiers return the catalog (UsagePlanCatalogDefaults per edition) so a blank field is explained
- McpUsageQuotaNode / McpOrganizationQuotaNode carry purchasedDelegatedSlots and the slot values (absent without purchased slots)
- Removed: the Tenant Group flag ChargeHomeTenantQuota with its scope plumbing (HomeChargedTenantIds, IsDelegatedAggregate, CatalogDecisionResult), Narrow/quotaExcludedTenants on the session envelopes and the X-MCP-Quota-Excluded-Tenants header, targetTenantId on the 429 body, the foreign-home attribution on McpTenantUsage (HomeTenantId, delegated/homeTenantId on the organization report) and the managed tenant's budget on the manager's managed-tenant list; PATCH global/tenant-groups/{id} is a rename
Web: SectionUsagePlans with four slot inputs, catalog placeholders and a built-in-defaults line; SectionMcpUsage renders "base + slots × per-slot" on both cards and drops the delegated badge; SectionTenantGroups without the charge toggle; SectionDelegatedAccess, the accept page and the privacy page state the new model (privacy last updated 2026-09-11).
MCP: get_fleet_overview no longer merges an exclusion list, the 429 renderer names only the caller's organization, the delegated-scope instruction states the home-charge model; wire types regenerated.
Tests: quota service slot-growth suite, entitlement purchased-slot matrix, conferred-Pro self-service writes (403/409), parity facts for the new and the removed fields; the radar analytics fixture is dated off the real clock (it had aged out of the 30-day window).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The self-seed of every fresh backend instance treated the embedded catalog as authoritative and wrote it over a newer GitHub reseed: on 2026-09-11 it re-inserted PS-SCRIPT-EXITCODE three hours after the reseed had deleted it (D-248), so agents ran 83 patterns. Every explicit reseed now stamps its kind in AdminConfiguration (PK RuleCatalog); the three self-seeds do nothing while a github stamp is newer than BackendBuildInfo.BuildUtc, and an empty table is still seeded from the binary. The IME self-seed deletes built-in rows the embedded catalog no longer ships.
ImePatternHealthService (health matrix and the ime_pattern_hits ingest filter) takes the pattern list from the seeded global partition via ImeLogPatternService.GetGlobalCatalogAsync, never from BuiltInImeLogPatterns, so the health page and the drift statistics follow a reseed without a deploy; the response carries catalog { source, stampedAt, patternCount } (manifests regenerated, web line, MCP description).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
SecurityValidator stamps ValidatedTenantId once the certificate tenant binding
(or the bootstrap token) proves the tenant, and DeviceValidation once the
device-validation chain has run: the admitting ValidatorType name, None for a
tenant without a validator, Transient for the 503, Rejected for the 403 of
stage device. RequestTelemetryMiddleware resolves the row's TenantId in one
place (policy tenant, proven tenant, then the X-Tenant-Id header only on the
routes that validate it) and copies DeviceValidation as a dimension.
Register, config and upload-url rows carried no TenantId before, so
tenant-filtered KQL missed exactly the calls that run the validation, and the
validators' success line is Information-level and never reaches App Insights.
Proving which validator admitted a device is now one query.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- GetMcpOrganizationUsageResponse.daily: charged requests per day inside the range from the same McpTenantUsage rows (BuildOrganizationDaily, MSP reads included); manifest + wire types regenerated (web + MCP copy); fold test
- Web: viewing another tenant no longer fetches the caller's own usage — no UPN, plan badge or own quota; the summary tiles and the daily chart come from the tenant's daily series. The own tenant stays the caller's usage page. Internal gitlink.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Backend: GET global/metrics/mcp-usage/organization?tenantId= (GlobalReadOrAdmin, TenantScoping.None on purpose — the route enumerates accounts, so the delegated path stays out; 400 without a GUID, no aggregate); GetMcpOrganizationUsageResponse carries `quota` (tenant plan + daily/monthly windows summed from the same McpTenantUsage rows); McpQuotaService.ResolveTenantPlanAsync with public TenantPlanLimits; two folds tests
- Web: TenantScopeSelector on /settings/reporting/mcp-usage (override-only, hidden for a delegated scope), scopedApi.mcpOrganizationUsage, Organization-quota card from the organization response (falls back to the caller's own node only on the own tenant), latest-wins fetch guard, wording follows the viewpoint
- Manifest + wire types regenerated (web + MCP copy); internal + skills gitlinks
Deploy order: Backend before Web — under a tenant override the page calls the new route.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An app still installing when the session's observation ended has no terminal
event and never will. The session page ticked Date.now() - startedAt for it
("11h 39m" on a session that finished hours ago), and its AppInstallSummaries
row stayed InProgress forever - ranked behind failures in the app's session
list and counted in totalInstalls with no named bucket.
Web: applyObservationEnd marks such rows Incomplete once the session is
terminal (isTerminalStatus), bounded by session.lastEventAt; the panel shows a
slate Incomplete badge and the watched span as a lower bound, the "active"
pill excludes them, and the download panel carries the parallel flag. Covers
every historic session without a backfill.
Backend: CloseOpenAppInstallsForSessionAsync closes InProgress rows without a
CompletedAt as Incomplete - at the terminal seam, again on the ingest batch
that carries agent_shutting_down for a terminal session (rows opened after
the verdict), and from the 30-day sweep. The sticky-status reconcile keeps
Incomplete against a late progress-only batch and lets a real terminal win.
App metrics and app analytics disclose the bucket (incomplete,
totalIncomplete) outside the failure rate; manifest and wire types
regenerated.
MCP: get_session_summary coverage gains an apps block and a gaps line naming
the apps the agent's last app_tracking_summary still listed as installing or
downloading, terminal sessions only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Closed submissions leave through the daily maintenance sweep (RuleSubmissionRetention, one place for rule and test); pending/approved/published never. The submit dialog takes an optional contact email (a UPN is not always a mailbox), stored for the reviewer and never published. The tenant list shows pending rows by default with the rest behind Show all, and the contribute button shows for every writer, also before the first custom rule exists. DELETE global/rule-submissions/{id} (GlobalAdminOnly, MCP delete_rule_submission) removes test/demo rows for operator and tenant alike. Bumps internal and .claude/commands gitlinks.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Tenant admins submit references to their own gather/analyze rules; the server freezes the rule, records pre-flight findings and fire stats, and stores it under a 12-character id (RuleSubmissions, SessionReports layout). Attribution is the submitter's frozen choice (anonymous by default) and becomes the published author. Review (approve with reserved id / decline with comment) through /admin/reports/rule-submissions or the MCP tools list_rule_submissions, get_rule_submission, review_rule_submission, reseed_rules_from_github; published is derived from the global catalog, never stored. Analyze rules now get the same server-side author stamp as gather rules; ANALYZE-ID-001 carries author Community. Privacy page names the credit; bumps internal and .claude/commands gitlinks.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ANALYZE-ESP-004 fired on every ESP terminal failure of a Continue-anyway
profile and rendered the configured FirstSync limit as elapsed time — the
mayHaveContinuedAnyway flag only says the profile has the button. The rule
now requires the agent's own verdict (app_install_failed with
failureType=esp_apps_timeout), names that app via a same-event filter, reads
failedSubcategory from esp_failure_settle_started and words the limit as a
limit. Verified with test_analyze_rule: cb7036a6 fires, 683f1eff (0x80070652
after 19 min) stays silent.
Rule engine: not_exists with a dataField on event_type/event_data
conditions matches when no event of the type carries a non-empty value —
including an absent type — instead of being dead behind the fieldValue null
short-circuit. Schema and MCP authoring guide say so.
Agent: a Shell-Core 62407 failure that lands inside the provisioning
tracker's 30-s settle window (a third of all ESP terminal failures) now
inherits the pending errorCode / failedSubcategory / category /
likelyCulpritApps, so enrollment_failed, Sessions.FailureReason and the
termination-time app classification see the HRESULT the registry had
already named. failureType stays the Shell-Core value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The What's new panel prints "Latest agent version" under the release
entries, and the session list draws its outdated badges from the same
endpoint — but the value was cached 12h in each Function instance and 1h
in the browser, so the panel announced the previous release for up to 13h
and two instances could disagree.
- LatestVersionsService: TTL 12h -> 5 min; keep the last successful result
and serve it when the blob read fails, so the shorter TTL cannot blank
the version line on a transient hiccup (failure marker 5 min -> 1 min).
- GetLatestVersions: Cache-Control max-age 3600 -> 60.
- No refresh callback from build-agent.yml: the cache is per instance, a
warm-up request would leave every other instance stale.
The origin is a ~200-byte no-cache blob behind the download alias that
Front Door does not cache, so a warm instance now reads it at most 12x/h.
Backend deploy (workflow_dispatch) pending — B-uq7.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The card composed its line from the backend's explanation ("did not answer within
3s - it scales to zero when idle ... ~20-30s"): probe mechanics nobody waiting on a
starting instance needs. It now writes its own one-liner and ignores check.message,
so the server's "warming" and a client-side probe timeout read identically and the
wording does not depend on a backend deploy. The backend message is shortened but
still carries the budget for direct API callers.
The re-poll budget of 8 x 5s (~40s) expired on a real cold start - Container Apps
activation has no upper bound we control - so it is now 12 x 5s (~60s), with a test
pinning that the stated seconds come from cadence x attempts.
Measured over 14 days, GET health/mcp answered in 4..148ms when warm and timed out at 30s
in 50 of 83 calls — the median call took 27,972ms. A cold start is 20-30s of Container Apps
activation (~98% KEDA plus image mount, ~500ms application), so no UI wait covers it and the
generous timeout only bought a long wall in front of the same answer. The browser's own
budget is the same 30s, so the backend's carefully worded warning never reached the user at
all: they saw a raw DOMException.
Backend: the budget drops to 3s, which sits in the widest observed latency gap
(2347ms..3930ms) so jitter cannot flip the verdict. Our own cancellation now reports the new
status "warming" at Information level (with minReplicas=0 a cold probe is the expected path,
and logging it as a warning poisons the ops signal); HttpRequestException is split out of
that catch and stays a real warning, still without the host in the message. The warm-hit
threshold drops to 750ms — ~5x the warm maximum, deliberately generous because every
measurement comes from a near-idle single-user system and warm latency will rise under
concurrency — and moves into McpReachableMessage so a test can pin the boundary. The XML doc
now describes the probe that exists rather than the wake-signal design the measurements
disproved.
"warming" is a new status value rather than an added DTO boolean because the backend deploys
before the web: an unknown value degrades to a grey card with a clean banner, whereas a
boolean would leave old web showing yellow and "Warnings detected" on every cold start.
HealthCheck.Status now documents the five-value vocabulary, which rides into both generated
TypeScript mirrors as JSDoc.
Web: resolveMcpCardState owns display, banner rating and the poll decision as a pure
function (the suite runs on node, so testable logic cannot live in JSX). A warming server
re-probes every 5s for up to 8 attempts and stays out of the overall banner exactly as
"unknown" already does — scaling from zero is expected behaviour, not a degraded platform —
and only becomes a real warning once the re-polls are spent. The card keeps showing
"warming" while the next probe is in flight so it cannot flicker, carries the blue
in-progress look instead of yellow, and was hoisted out of the healthResult guard: a failing
detailed check would otherwise hide a card that is still polling.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The portal hides Delete for owned (msp-{home}) groups, but DELETE global/tenant-groups/{groupId} accepts any id: a Global Admin deleting one ended Pro (MSP) for every member with no grace anchor, so a member that raised its retention under Pro was clamped to the Community cap at the next sweep. DeleteGroupCoreAsync now snapshots the owner and calls ProConferralService.RecordLossAsync(member, owner, "group-deleted") per member, owner excluded, independent of assignees. Two tests in RevokeEnforcementTests; internal docs list the fourth writer.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Act equals the KEV set by construction (CvePriority: KEV => act), so the KEV tile becomes the Act tile and Attend joins it instead of a duplicate count. The breakdown is counted in the backend over every CVE in the window, like the severity breakdown, because the list is top-N. Both panel consumers read the generated CveExposureSummary type; the MCP tool description names the new block. Wire types regenerated for web and MCP.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The caller-supplied X-MCP-Tool-Name header went verbatim into the UserUsageLog
row key ({yyyyMMdd}_{tool}:{endpoint}) and from there into the CAS-conflict
warning of TableCasRetry. EndpointNormalizer.ToolNameKey now strips CR/LF and
every other control character or line separator, replaces the characters Table
Storage rejects in keys (/ \ # ?) and caps the length at 64; an empty result
skips the prefix. Real tool names pass through unchanged, so existing usage
rows keep their keys. A header with an illegal key character previously made
every usage increment fail with 400 (fail-soft, counter never advanced).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The report still carried two hand copies after D-215: a typed map of C#
initializer defaults for the "custom" highlight, and a TypeScript mirror of
GetAgentConfigFunction's resolution rules for the runtime column.
Backend: the derivation moves verbatim into Services/AgentConfigResolver
(Build = pure, ResolveAsync loads tenant + admin config and the two rule
catalogs). GetAgentConfigFunction keeps the kill verdict, the migrate logging
and the device fields on top; ParseAgentMajor, ResolveDiagnosticsUploadEnabled,
MergeDiagnosticsLogPaths and ResolveMigrateTarget move with it (call sites and
tests retargeted; the QueryParams ratchet baseline shrinks by the file). New
GlobalAdminOnly route config/{tenantId}/effective-agent-config returns the same
AgentConfigResponse without a device (optional ?agentVersion=, default the V2
line). AgentConfigResolverTests pin the rules that used to live in the web:
upload 10, Hosted destination without SAS, SAS regardless of mode, unrestricted
only with entitlement + gate + toggle, operator knobs from AdminConfiguration,
analyzer defaults, hash line per major, migrate override and rejection.
Manifest: SharedManifestParityTests emits "defaults" next to "fields" for
TenantConfiguration and AdminConfiguration and an "agentConfig" section with
the AgentConfigResponse / CollectorConfiguration / AnalyzerConfiguration class
defaults (fresh-instance values; nested objects null). TenantConfiguration.
NtpServer gets the initializer "time.windows.com" the table read already
applies, so the manifest default and a stored row agree.
Web: the catalog rows lose value() and default; the runtime column renders
the endpoint's response, the highlight compares against the manifest defaults,
informational rows (reasons, timestamps, identities, secrets) never read as
custom. Previously excluded fields are visible (configVersion, catalog counts,
sealing pattern ids, hash oracle, migrate target); the collector class
defaults sit in a collapsed "Collector Internals" section. Fail-soft: while the
backend is not deployed the runtime column shows a notice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The "Health by Device Model" link searched "{Manufacturer} {Model}" as one substring; the server's
free-text predicate tested it per field, so once the failed sessions were outside the loaded first
page the sweep returned nothing.
- lib/searchQueryTokens.ts: tokenizer shared by the timeline and the session search (terms, quotes,
leading minus, key=value / key:value qualifier when the separator precedes any quote)
- app/dashboard/utils/sessionSearchQuery.ts: 15 qualifiers = exactly the fields both sides search,
matcher over session fields + client-derived extra text, suggestion field lookup, link builder
- useDashboardFilters / SessionTable: filter and suggestions use the grammar; fuzzy only for a
single free term; session_searched gains a `qualifiers` count
- Helpers/SessionSearchQuery.cs: the same grammar for q= on /api/search/sessions (+ global), parsed
once per request; MatchesFreeText is a thin wrapper
- utils/session-search-syntax.cases.json: one case file run by vitest and by
SessionFreeTextSearchTests (parity proof)
- FleetModelHealth/FleetSlowModel/FleetFailingModel: manufacturer + model as stored plus label,
grouped on the pair; useFleetHealth uses the generated wire types; "Unknown" is not a link
- shared manifest + wire types regenerated for the three DTOs
Backend suite 6158/6159 (the one failure is SharedManifestParityTests against uncommitted WIP of a
parallel task), web vitest 1177/1177, build 0 warnings.
Parallel tool calls of one user hit the same (user, day, endpoint) row in the same instant; without backoff the losers stayed synchronised and 3 of 6 increments were lost. Both increments now use the one CAS helper (merge patch, AddEntity on 404, StorageCasConflict marker) with UsageCounterRetries = 8; the tenant counter's attribution columns travel in the merge patch. Tests pin the budget, the patch shape and the create path.
TenantConfiguration.McpDisabled + McpDisabledReason (GA-only, same deny-list and PUT guard as Disabled).
Front door: McpUserService denies every caller of a closed home tenant below the platform roles.
Per request: PolicyEnforcementMiddleware denies a closed home or target tenant for X-Client-Source: mcp
(403 McpDisabled) and narrows the delegated fleet bound, never to null. ClientSourceHeader is the one
reader of the header. The MCP access guard surfaces the backend reason verbatim. GA tenant modal,
list badge and config report; wire types regenerated; internal doc, health marker, gitlinks bumped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every HTTP request body the web, the MCP or the agent sends is now a flat class in
AutopilotMonitor.Shared implementing IApiRequest — the request twin of IApiResponse.
WireTypeManifestBuilder exports the implementers as a third root, so the request DTOs
reach wire-types.generated.ts on both clients; for request types a nullable property
keeps `| null` (a PATCH may send an explicit null to clear an override).
Functions read a body only through RequestBody.ReadAsync<T>/ReadOptionalAsync<T>
(System.Text.Json, ApiJsonOptions.Read = wire options + case-insensitive; empty or
malformed body = 400 envelope, never a 500) whose generic constrains on IApiRequest.
ReadNewtonsoftAsync<T> stays for the Newtonsoft-bound cluster (tenant/admin config via
the patch service's PopulateObject, rule and pattern documents) as a frozen per-file
baseline. The tenant-plan PATCH keeps its presence-sensitive JsonDocument walk with the
keys derived from PatchTenantPlanRequest by nameof (PlanPatchKeyParityTests).
35 nested request classes, 7 JObject/JsonDocument hand-walks and the anonymous
DeserializeAnonymousType bodies moved to Shared; the two identical TicketRequests are
one DownloadTicketRequest; ServerAction is no longer the request (QueueSessionActionRequest,
the stored row is built server-side); create-from-template takes { "variables": … }
instead of a bare map. Fields the endpoints default server-side are nullable now
(CPE mapping, report attachments, feedback rating on dismissal, upn for applications).
TypedRequestGuardTests: no raw deserializer under Functions/ (named exemptions: agent
routes frozen by D-200, Azure Monitor webhook, upstream/stored-column parses), the
Newtonsoft branch only in its baseline, no request type in the Functions assembly, every
DTO flat and in Shared. RequestBodyTests cover both readers, the cap and the envelope.
QueryParamGuardTests baseline shrinks by 14 (the Content-Length TryParse lives in the reader).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The surface comes from the signed appidacr/azpacr claim (0 = public client = portal;
secret/certificate, app-only or missing = integration), never from a header: portal SPA,
MCP server and API share one app registration in production, so the client id cannot tell
them apart and any token holder controls headers. New AdminConfiguration field
PortalUserRateLimitRequestsPerMinute (default 600, Global Settings round-trip, wire types
regenerated); the existing user budget is relabelled MCP & integrations and keeps the tenant
override and the Pro floor; buckets are keyed per surface; ThrottleSurface is a request
dimension next to ClientSource so the classification is auditable (platform-health check 2).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A freshly onboarded tenant spent 45 minutes in 503 because a second instance held a
roleless app-only token minted before the consent; InvalidateTenant never reaches other
instances. GraphAuthFailure is the shared step for all five Graph validators: attempt 1
drops the cached token and reports transient so the existing loop re-mints; attempt 2
keeps each validator's semantics (Autopilot/DeviceAssociation transient with Retry-After
120, CorporateIdentifier/CloudPc/IntuneBinding definitive) and logs a stable marker.
SecurityValidator hands the agent the largest Retry-After requested. Platform-health
check 6 and the marker catalog know both markers; the cloudpc concept doc carries the
contract.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
About 12 MB/day of severityLevel-1 traces without diagnostic value (MSAL verbose, StatusCodeResult,
CORS success, Application started). Application logs were Warning+ already via Function: Warning.
Measured and documented in internal/docs/backend/telemetry-ingest-shaping.md.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- StorageClientOptions: Table/Queue 8 s per attempt × 3 attempts + capped backoffs = 25.8 s < the
agent's 30 s upload timeout (pinned by test); Blob 60 s × 4. Every production client takes its
options from there. No Polly over the SDK.
- StorageErrors.IsTransient(Exception): OperationCanceled / IO / HttpRequest / Aggregate-of-those
/ RequestFailed status 0; the ingest answers 503 + Retry-After instead of 500.
- The ingest still takes no caller cancellation (a batch is a unit). GetAllSessionsPageAsync
carries the request token down to BoundedFanOut (Global Admin list); SingleFlightCache hands its
factory a 30 s budget token and evicts on timeout (tenant ids, app-summary snapshot, session stats).
- ProcessStallMonitor: 1 s tick drift ≥ 1 s ⇒ ProcessStall event (sampling bypass) + metric with
cause gc / threadpool / unknown from GC.GetTotalPauseDuration and thread-pool counters;
GcPauseMs on request rows ≥ 100 ms. Measures the instance-local 5–10 s pauses seen in production.
- Gitlinks: internal (plan, D-203, concept doc, B-9hf, lesson), .claude/commands (marker rows).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Helpers/QueryParams replaces fourteen private days/limit parsers, the three
TryParseUtc copies, eleven identical pageSize blocks and ~50 inline TryParse sites
under Functions/ and Pagination/. Lenient (Int/IntOrNull/UtcInstant): absent, garbage
or below-minimum means the call site's default, above-maximum clamps. Strict
(TryInt/TryPageSize/TryUtcInstant): absent is null, anything else invalid is a 400
with a message that names the parameter. Integers are invariant-culture, instants are
invariant UTC (AssumeUniversal | AdjustToUniversal), whitespace counts as absent.
Defaults and ranges stay arguments at every call site.
Settled differences: the Apps windows clamp above 365 instead of falling back to 30,
days=0 or negative yields the default everywhere, the two global geo endpoints and
the session list cap days at 365, SLA months is capped at SlaMetricsService.MaxMonths
on the HTTP layer, a whitespace date means no filter on every route, the admin batch
sizes reject values above their maximum, and the three culture-dependent
DateTime.TryParse sites parse as UTC.
QueryParamGuardTests ratchets TryParse out of Functions/ and Pagination/ (baseline:
headers, raw cells, the agent version major, the se= of a stored SAS URL);
QueryParamsTests carries the cases the removed copies pinned.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
TelemetryItemKind moves to Shared; TelemetryItem.ToWire() maps onto TelemetryItemDto
and the uploader serialises the DTO, so both ends share one wire class. The spool line
keeps writing Kind by name (StringEnumConverter registered in TelemetryItemSerializer).
IngestTelemetryFunction.PartitionBatch routes by exact kind name and collects every
item it cannot route or parse; a non-empty rejection list answers the batch with 422 +
TelemetryItemsRejectedResponse (poison, rejectedRowKeys, reason) before anything is
persisted, logs a Warning and writes the new TelemetryItemsRejected ops event. The
agent's existing poison protocol drops exactly the named RowKeys and re-uploads the
rest; before this the ingest skipped the items, answered 200 and the agent cleared
its spool.
Contract tests from both sides: TelemetryWireContractTests (agent) freezes the real
batch body under tests/fixtures/telemetry-wire/ and parses the pinned poison body;
TelemetryWireFixtureTests (backend) dispatches every frozen body with zero rejections,
requires an arm per kind and pins the poison body.
Agent-facing responses implement IApiResponse and go through ResponseHelper, request
types carry [WireContract]; plain ingest refusals use the error envelope, the
register-session and upload-url refusals keep their DTO bodies (the agent reads
ErrorCode/Message). Manifests and generated wire types regenerated (also picks up
four types HEAD had left out of the manifest).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PlatformStats(global,current) took a GET plus a full-row upsert without ETag
from every ingest batch: increments were lost under concurrency and a writer
clobbered fields another writer had just changed. RuleStats put every tenant's
daily rule traffic into one date partition (1,459 rows on 2026-09-04), the
writer issued four serial calls per evaluated rule without ETag, and the
reader was a cross-partition scan whose 10k cap dropped the newest dates.
- D-198: TotalEnrollments / SuccessfulEnrollments / TotalEventsProcessed have
no hot-path writer; the two-hourly recompute is the only source. Only
IssuesDetected still increments, through TableCasRetry (ETag + field-only
merge, AddEntity for a missing row, conflicts as StorageCasConflict). The
tenant counters share the helper.
- D-199: RuleStats PartitionKey "{scope}_{date}", RowKey ruleId.
RecordRuleStatsAsync merges a session's increments per rule, reads the
partition once and writes one transaction (UpdateReplace with ETag / Add),
retrying on 412/409. Readers are PartitionKey ranges per scope; aggregation
and the regression radar fan out over the registered tenants; the cleanup
deletes per scope. RuleStats joins the offboarding composite wipe.
- Legacy rows (PartitionKey = date) are read by an extra query and cleaned by
a date range that checks every row for a bare-date key, until 2026-12-05.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- DiagnosticsBlobStreamer disposes the download result on every exit path
(it was disposed only on the 413 branch), measures open and transfer
separately and fires DiagnosticsDownloadProxied after the copy with
Completed=true/false.
- UserDelegationKeyCache: one user delegation key per process (24 h, re-mint
under a lock with 2 h margin or when a SAS would outlive it) instead of one
storage round-trip per download or upload SAS.
- KevDataService is resolved as one singleton over the typed-client handler
pipeline; the cache warmer used to hydrate a throwaway transient instance
while the correlation service held a cold one.
- Admin and tenant configuration services single-flight concurrent cache
misses (Lazy<Task> per key); the MemoryCache and the in-place projection
semantics stay untouched.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The dependency filter dropped every successful storage call and the expected
412/409 outcomes, so storage-side latency and ETag contention were invisible
until a call failed outright (seven days of App Insights held zero storage
dependency rows).
- StorageDependencyFilterProcessor keeps a successful storage call at or above
2 s and stamps SlowStorage=true; the operation name is on the row, the table
name is not (the Azure SDK span carries neither URL nor table).
- StorageMetrics wraps TelemetryClient.GetMetric("StorageCasConflict",
operation, table, outcome): pre-aggregated per minute, bypasses the worker's
adaptive sampling, null-tolerant for tests.
- The CAS loops report conflicts: session event count, session counter
reconcile, session store, tenant stats, MCP user/tenant usage.
- TableStorageService keeps its two-argument constructors (Moq matches by
exact arity); DI resolves the three-argument overload.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every entity-group transaction was chunked by count only; 100 payload-heavy rows
(Signals with chunked PayloadJson, Events with 30k-char Message + DataJson) could
exceed the 4 MiB transaction cap, and the failure surfaced as an opaque 500. The
agent halves its batch only on 413 and treats 5xx as "replay later", so such a
batch wedged the device forever.
- TableTransactionBatcher: 100 actions, 3.5 MB per transaction, 1 MB per entity,
never across partitions; an entity that cannot fit throws
TableEntityTooLargeException. Used by the event, signal, transition, restore
and rules writers; delete-only batches keep their count chunking.
- StorageErrors: one classifier for RequestFailedException (transient 408/429/
5xx, payload-too-large by status or error code, already-exists incl. the
Azurite 400 shape); the two IsAlreadyExistsStatus copies use it.
- Event writer: per-row fallback only for data-shaped rejections; throttling,
outages and oversize batches propagate. Single and batched writer share one
entity builder.
- IngestTelemetryFunction: payload too large -> 413, transient -> 503 with
Retry-After: 5, anything else 500.
- TableStorageChunking.MaxPropertyChars is the one 30,000-char bound.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The five cross-session index tables (SessionsByTerminal, SessionsByStage,
DeadEndsByReason, ClassifierVerdictsByIdLevel, SignalsByKind) were write-only
since M5.d: no endpoint, MCP tool or web page ever read them, while the
dual-write cost a serial queue send per signal/transition on the ingest path,
a 5-13 min full scan of Signals + DecisionTransitions every two hours and five
property-filter scans per session deletion (audit 2026-09-05 F05).
Removed: the tables from TableNames.All (schema hash changes, next cold start
runs a full pass without them), the telemetry-index-reconcile queue name,
producer, envelope, handler, queue function and 2h timer, the cross-partition
Timestamp scans on the signal/transition repositories, the EnableIndexDualWrite
flag (model, config mapping, admin UI, generated wire types), cascade steps
11-15 (later steps renumbered 11-18), the deletion wrappers and the offboarding
wipe entries. DiscriminatorPkProp stays as a step class so manifests written
before this change still resume.
Order of operations: deploy first, then delete the queue pair and the five
tables in the portal (B-88h). Deleting the tables before the deploy 500s every
delete preview/enqueue and poisons offboarding.
Submodules: .claude/commands (import_session.py), internal (D-196, docs, plan).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Audit 2026-09-05 F01/F02. Analyze, VulnerabilityCorrelate, TenantAutoApprove,
IndexReconcile and ImeMsiArchive are now host-managed queue functions under
Functions/Queue bound to the DataStorage connection (identity-based, resolved
by the host): the Flex scale controller wakes an instance for a message, the
host renews the lease and owns the retry/poison ladder. The four cascade
workers with lease/CAS state stay self-polling BackgroundServices; their base
now receives one message per poll so waiting messages can no longer expire
under another worker. Terminal enrollment alerts and hardware-rejection alerts
are enqueued on notification-dispatch (channel ids only, resolved at send
time) instead of fire-and-forget sends the host could drop on scale-in.
Requires the app setting DataStorage__queueServiceUri before deploy.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every non-2xx body in the Functions project now goes through ResponseHelper
(ErrorAsync / BadRequestAsync / NotFoundAsync / ForbiddenAsync /
UnauthorizedAsync / ConflictAsync / InternalServerErrorAsync): 408 sites by a
pattern-exact rewrite of the CreateResponse + WriteAsJsonAsync + return shape,
50 by hand. The eight private BadRequestAsync shadow helpers are gone; the raw
500 catch blocks use InternalServerErrorAsync (sanitized message, correlation
id, MCP hint). 429s keep Retry-After via retryAfterSeconds.
Specialised error bodies where a consumer reads more than the prefix:
AppHomingDeniedResponse (kebab reason codes now Constants.AppHomingReasonCodes,
probe result), TenantConfigPatchFailedResponse (backupId, drift),
QueryBackendLogsErrorResponse (upstreamCode, statusCode, source, hint,
upstream). Twenty-one message-first 200 bodies that the old error tolerance had
let through are typed too (MessageResponse, PreviewWhitelistActionResponse,
SuccessMessageResponse). New codes: PayloadTooLarge, UpstreamError,
UpstreamTimeout, the three trial-gate codes; the agent security path
(SecurityValidationExtensions) writes the envelope with the retry window.
TypedResponseGuardTests: WriteBaseline is empty again — the anonymous
WriteAsJsonAsync ratchet now covers error bodies with zero tolerated sites.
Manifest and generated TypeScript regenerated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every error body now leads with { error, code, correlationId }: the generic
ApiErrorResponse (hint?, retryAfterSeconds?, operation?) plus the specialised
McpQuotaExceededResponse (reordered; `message` became `error`),
DelegatedSlotLimitReachedResponse (+correlationId) and the new
SessionDeletionRejectedResponse (replaces the anonymous delete arms). The
correlation id is stamped by the writers — ResponseHelper.ErrorAsync and the
new ApiErrorWriter for the middleware side (auth 401, policy 403/503, rate
limit 429, MCP quota 429, global exception 500) — never by call sites.
exceptionType and rateLimitInfo leave the wire; Constants.ApiErrorCodes is the
code vocabulary and is exported as the manifest section `apiErrorCodes`
(API_ERROR_CODES for web + MCP). CorrelationIdMiddleware names the readable
headers via Access-Control-Expose-Headers.
Guard: TypedResponseGuardTests no longer tolerates anonymous error shapes;
the 458 legacy sites in 149 files are frozen in WriteBaseline to melt down.
New facts: envelope-prefix reflection, ApiErrorWriter body/header pins,
exact-shape pins for the generic envelope and the changed specialised bodies.
Reverses the "error bodies stay anonymous" clause of D-043 (decision entry
follows with the docs stage).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The first cut refused every reader-zone-fallback endpoint. The DeviceSetup line is
written before the agent starts and is backlog on every device, so that silenced
ANALYZE-ESP-001 for all agents without an era anchor (control sessions 2708e6b4 and
18d80f9c came back "not measurable" after the deploy). Two fallback endpoints share the
writer-zone error and cancel it; only a span mixing a fallback endpoint with an anchored
one (or with "now" for an open phase) is unmeasurable — the a7140f98 shape.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Oliver Kieselbach