Step 0-2 of the gather-rule anti-spam plan:
- GatherRuleService.UpdateRuleAsync: toggle-style partial PUTs ({enabled,
isBuiltIn, isCommunity}) previously full-replaced the stored custom rule and
wiped Title/Target/Trigger/Parameters. Partials now merge Enabled into the
existing row; full edit payloads preserve the original CreatedAt.
- GatherRule: new opt-in fields ActivePhases (run only during listed phases),
ActiveFromPhase (sticky latch from a phase onwards), EmitMode (always |
on_change). Absent fields = legacy behavior; schema updated accordingly.
- TableStorageService.Rules: Store+Map for the three columns (absent column /
stored-empty -> null); MapToGatherRule internal for roundtrip tests.
- ContentEquivalent covers the new fields, null-tolerant for ActivePhases so
pre-existing rows never diff against seeds (no reseed churn).
- GatherRulesFunction.ValidateScopeAndEmitMode: canonical EnrollmentPhase name
list Start..Complete (Unknown/Failed and numeric tokens rejected), mutual
exclusion of the two scope fields, emitMode whitelist; 400 on violation.
- ConfigVersion 33 -> 34 (33 was taken by MigrateToApiBaseUrl).
Tests: GatherRuleUpdatePartialMergeTests (merge/replace/unknown-rule/built-in
path), GatherRuleScopeFieldsTests (Store<->Map roundtrip incl. legacy rows,
ContentEquivalent matrix, validation accept/reject). Functions.Tests 3029/3029.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
null = no migration is the documented contract and the agent already
assigns null when stripping the field from the on-disk cache; annotate
the property as string? so the backend assignment needs no ! suppression.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Agent Binaries check + ops watcher now probe the canonical download alias
(download.autopilotmonitor.com) AND the legacy blob keepalive; alias failure
is unhealthy, legacy failure a warning
- McpServerBaseUrl fallback moved from the stale kindwave WEU ACA FQDN to the
stable custom domain mcp.autopilotmonitor.com (CNAME to the new GWC instance)
- health/detailed and health/mcp include the probed endpoint URLs in check
details for Global Admins only (storage table endpoint, WEBSITE_HOSTNAME,
alias/legacy download URLs, MCP server URL)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-home agents to a new API base URL through the agent/config response
(backend move or per-tenant EU->US region move) - the lesson from the
2026-07-18 WEU->GWC cutover, where the compiled-in API hostname hard-cut
every in-flight agent (no DNS indirection possible: mTLS breaks behind a
proxy, Flex custom-domain TLS is broken).
Backend (ConfigVersion 33):
- AdminConfiguration.AgentMigrateApiBaseUrl (global) +
AgentMigrateTenantOverridesJson (tenantId->URL; empty value pins a
tenant against the global target; override wins)
- GetAgentConfigFunction.ResolveMigrateTarget serves the validated target;
AgentMigrateServed / AgentMigrateRejected warnings as delivery evidence
- TableConfigRepository Store+Map; admin portal Global Settings round-trip
Agent (kill-switch-mirrored hardening):
- Shared AgentEndpointMigrationRules: https-only, *.azurewebsites.net on a
label boundary, no path/port/userinfo - enforced on BOTH sides
- Live fetch only: RemoteConfigService cache strips the field on write AND
read; EndpointMigration.ResolveTarget re-checks the fetch outcome
- AgentRuntimeConfig.Resolve returns before merge/integrity/bootstrap
cleanup on a pending migration (no spurious IntegrityCheckFailed against
a stale migration stub), Program.RunAgent rebuilds Phase 3+4 against the
new URL - single hop, kill signal wins before and after the hop
Tests: AgentEndpointMigrationTests (backend rules + resolution),
EndpointMigrationTests + cache-hardening strip (agent). Full suites green
(Functions 2992, Agent Core 1822, vitest 557, tsc silent).
Doc: docs/agent/endpoint-migration.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cross-tenant re-claim of recently deleted global names is denied by
Azure by design (dangling-DNS/subdomain-takeover protection). New
tenant resources therefore use eu-suffixed names; the old names live
on in the old subscription (storage 'autopilotmonitor' recreated as
legacy bootstrap-URL keepalive, FuncApp/ACR kept as tombstones).
- API base URL -> autopilotmonitor-api-eu.azurewebsites.net
(Shared Constants, web CSP + /go fallback, MCP default)
- Agent blob container + build counter -> autopilotmonitoreu
- deploy-backend app-name, deploy-mcp ACR -> eu names
- Bootstrap script URLs stay on the legacy storage (customers
migrate once to a domain alias later)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The backend admits agent sessions via up to three device-validation
paths (Autopilot S/N lookup, Corporate Identifier, Device Association
once DevPrep becomes a hard gate) plus the pre-MDM Bootstrap token —
but the winning path was only returned to the agent, never stored, so
operators could not see HOW a device was accepted.
- SessionRegistration.ValidatedBy: server-populated in
ProcessRegisterAsync from the SecurityValidationResult (agent-sent
values are overwritten unconditionally)
- Sessions row + SessionsIndex mirror + MapToSessionSummary carry
ValidatedBy; latest non-Unknown validation wins (Bootstrap sessions
upgrade to the cert-path validator on re-register), Unknown preserves
the earlier value (tenants with device validation off)
- Web: "Device Validation" item on the session detail Session Info card
(hidden for legacy sessions / validation-off tenants)
- MCP: get_session overview exposes validatedBy; raw endpoints carry it
verbatim automatically
- Tests: SessionIndexProjectionTests pin the new index column
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Devices that reach OOBE without an Autopilot profile (ProfileAvailable=0
in AutopilotPolicyCache — not registered, assignment not propagated, or
deleted from Autopilot) still enroll via manual Entra join + MDM
auto-enrollment and previously looked like regular Autopilot sessions
with an inexplicably empty profile section (session 423b5360).
- Agent: CollectAutopilotProfile emits autopilot_profile_missing
(Warning) on explicit ProfileAvailable=0 evidence, with ZeroTouch
tenant-domain extraction as supporting data; EmitDeviceInfoEvent
gains an optional severity parameter
- Web: DeviceDetailsCard shows a callout in the Autopilot Profile
section (also retroactively for old sessions via ProfileAvailable=0)
and renders Profile Available 0 as "No"
- MCP: resource-catalog gains the new event type + field docs
- Tests: 11 unit tests for the detection/extraction helpers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Misclassification audit 2026-07-16 (1000 non-succeeded sessions, 22 deep
dives) surfaced six systematic misdeclaration sources. This implements
all of them:
1. DecisionCore: HelloNeverObservedPromote in the esp-exit advisory
handler — when the Hello policy read never committed and no wizard
was ever seen, hello_resolution is structurally unresolvable; promote
to AwaitingHello + hello_safety instead of hard-failing a provably
finished enrollment (sessions 2dac8298/7635ab18). Placed after the
enforcement-progress re-arm so active installs still win; advisory-
anchor variant stays exempt.
2. Ingest: both max-lifetime shapes (enrollment_failed/agent_timeout and
agent_shutting_down/max_lifetime) now route through the
EnrollmentTimeoutClassifier instead of hard Failed (tenant a53e67ec:
fully provisioned WG Part 2, user never logged in => AwaitingUser).
agent_timeout no longer counts as HasExplicitFailure; failure
webhooks fire only on actual failure verdicts.
3. Maintenance sweep: dropped the too-broad "IsPreProvisioned ne true"
exclusion (sealed devices are protected by their Pending status);
resumed WG Part-2 runs sweep normally with the window anchored to
ResumedAt. New stage-2 silence guard also fixes a latent bug where an
actively-reporting >5h enrollment could be terminalized mid-run.
4. Registration supersede: a fresh registration resolves older
non-terminal sessions of the same device (serial, placeholder-guarded)
as Incomplete("Superseded by ...") with live SignalR updates — closes
the forever-Pending orphan gap when WG Part 2 runs under a new id.
5. Admin retro-reconcile: POST /api/maintenance/reclassify-legacy
(GlobalAdminOnly, DRY-RUN BY DEFAULT) re-runs the classifier over the
pre-classifier "Session timed out" Failed backlog (~495 rows, sample
projected ~87% misdeclared) and resolves Pending orphans. Requires the
new opt-in allowTerminalReclassification override in the status
writer (admin-marked rows stay untouchable).
6. Polish: "0 of N" AccountSetup rollups keep their total (no more
fabricated "0/0" reasons, "never observed" wording when absent);
late-telemetry reconcile heals Incomplete/AwaitingUser to Succeeded
when straggler evidence proves completion; SLA page knows
AwaitingUser/Incomplete ordinals; script_failed on Success/exit-0 now
names the stderr rule in its message.
Tests: backend 2965, DecisionCore 474 (+6 new), agent 1777, web 557 —
all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Watchdog fix: FanoutResult is caller-allocated and mutated live per
tenant/session, so 30/60min LongRunning warnings report real progress
instead of the always-zero counters (today's warning showed 0/0 while
5 tenants / 54 sessions were already done).
- 50min run budget: the retention fanout stops cleanly at the deadline
(tenant boundary + per-session check) and reports AbortedByBudget —
no more risk of the 60min Flex functionTimeout hard-aborting mid-run.
New Warning OpsEvent SessionDeletionMaintenanceBudgetExceeded;
Completed gains abortedByBudget.
- Blob-lease serialization: dedicated sentinel
_lock/session-deletion-maintenance.lock (deletion-manifests container)
via SessionDeletionMaintenanceLockStore + reused MaintenanceLeaseHolder;
contended runs emit SessionDeletionMaintenanceSkippedLocked and return.
- Manual trigger: POST /api/global/session-deletions/maintenance/trigger
(GlobalAdminOnly, policy catalog) -> fail-hard queue producer ->
SessionDeletionMaintenanceQueueWorker runs the same RunCoreAsync body
with triggeredBy = requesting admin; lease probe returns 409 while a
run is active. New Started OpsEvent carries triggeredBy.
- Web: MaintenanceStatusBanner on the Session Cleanup page derives
active / budget-exceeded / failed / idle from the lifecycle ops events
and offers a Run-now button (202/409 handling). New event types
dual-registered in OPS_EVENT_TYPES.
- Tests: 7 new (budget abort immediate/mid-tenant, live counter
increments, watchdog live-progress regression, BudgetExceeded +
Completed flag, SkippedLocked, Started triggeredBy); suites green
(backend 2935, web 557, tsc clean).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 8-parameter constructor plus 7 hand-threaded With* methods meant every new
observation field touched 8 copy sites - a threading-bug risk flagged in the
daily commit review (2026-07-14). Now a sealed record with init-only properties:
the With* methods keep the set-once guard but their bodies collapse to one-line
`this with { ... }` expressions, so the next field addition is one property plus
one With* method.
- IsExternalInit polyfill (internal) added: DecisionCore targets netstandard2.0,
which lacks the C# 9 marker type for init setters. Consumers never need it -
init/with usage stays inside the assembly.
- Snapshot compatibility unchanged: identical JSON shape (property names +
SignalFact {Value, SourceSignalOrdinal}); the record deserializes via init
setters instead of the removed constructor. Pinned by a new StateSerializer
round-trip test covering facts, ordinals, and unset-stays-null.
- Semantics unchanged: set-once still returns `this` (Assert.Same tests hold);
`==` is not used on the type anywhere, so record value equality is inert.
Verified: DecisionCore 468 tests, Agent V2 Core 1776 tests on net48 (the
consumer that mandated the netstandard2.0 target), Backend 2928 tests - all
passing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Session 62e603c9 (user-driven, Hybrid Join, WhiteGlove Part 2, Classic ESP
with SkipUser=True) completed too early: the device_only_esp_detection 5-min
deadline classified the session SelfDeploying even though the registry probe
said isSelfDeployingProfile=false, Hello timed out and no desktop arrived.
Root cause: the Part-2 reseal-reboot archives the snapshot and re-seeds an
empty scenario profile (Mode=Unknown), so nothing blocked the behavioural
deadline from overriding the deterministic registry fact.
A - carry the Part-1 WhiteGlove classification across the reseal reboot:
RecoveryCoordinator captures rawSnapshot.ScenarioProfile before archiving and
re-seeds Part 2 with Mode=WhiteGlove/High, PreProvisioningSide=User
(reason whiteglove_part2_carryforward). The existing monotonic-mode guards +
deadline race guard C then dead-end the device-only deadline.
B - record the registry self-deploying fact for both true and false and veto
on explicit false: new set-once EnrollmentScenarioObservations
.RegistrySelfDeployingProfile, recorded in HandleEnrollmentFactsObservedV1,
plus a new race guard D in HandleDeviceOnlyEspDetectionDeadlineFired that
dead-ends on an explicit false (device_only_esp_detection_registry_not_self_deploying)
and cancels the deadline state-side. null (unobserved) never trips, so the
legacy behavioural completion and the kiosk case 320b3bf7 (registry true)
are unaffected. Covers non-WhiteGlove SkipUser=True flows too.
Observability - phase_transition events used to emit empty {} DataJson;
BuildPhaseTransitionEffect now attaches a lean typedPayload
{decisionSource, trigger, scenarioMode, scenarioConfidence} at all eight
finalizing/complete call sites, routed verbatim to EnrollmentEvent.Data.
Tests: DecisionCore +5 (guard D false/true/null, false-observation recording,
phase_transition payload), Agent V2 +1 (Part-2 carry-forward). No ConfigVersion
or table-serialization change; snapshot schema stays v1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A flip-flopping user-scoped WHfB CSP read "disabled" once, the arm-C
completion synthesized HelloOutcome="Skipped" and the session completed
5s later while the user sat inside the Hello wizard that had started
230ms earlier (Shell-Core 62404, previously InformationalEvent-only).
Layer 1 — engine (DecisionCore):
- New DecisionSignalKind.HelloWizardStarted + set-once fact
DecisionState.HelloWizardStartedUtc (additive-nullable, census +
snapshot wired).
- Prevention: centralized HelloSatisfiedForCompletion /
HelloPolicyDisabledWithoutWizard predicates veto the policy-disabled
stand-in at all five Skipped-synthesis sites and in the
completion_waiting prerequisites; the AdvisoryCompletion handler gets
a HelloWizardObservedPromote branch (its fallthrough is
re-arm-or-fail).
- Cure: HandleHelloWizardStartedV1 retracts an engine-synthesized
"Skipped" (exact-case discriminator vs the lowercase tracker
vocabulary; first fact-nulling reducer, stage-agnostic because the
RealmJoin gate-defer parks synthetic skips pre-Finalizing), cancels
finalizing_grace, returns to AwaitingHello and arms HelloSafety.
- Agent path: ShellCoreTracker raises HelloWizardStarted before
FinalizingSetupPhaseTriggered; 62404 included in the startup backfill
(fire-once); coordinator timestamp mirror; adapter emit (fire-once).
Layer 2 — HelloTracker confirmation read:
- ApplyPolicyRead state machine: "disabled" commits only after a second
confirming read (~10s poll); "enabled" commits immediately;
disabled→enabled flags flipFlopDetected=true; pending keeps the
policy-unknown wait cadence. Audit fields confirmedReads /
flipFlopDetected on hello_policy_detected.
Fixtures: signal-kinds/hello-wizard-started-v1.json +
enrollment-sessions/userdriven-hello-wizard-unskip-v1.jsonl (772fe502
ordering, ends Completed with HelloOutcome=completed).
Tests: DecisionCore 462/462, V2.Core 1775/1775.
Docs: OKF concept doc docs/agent/hello-wizard-unskip.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Succeeded reconcile hygiene (c0585133) clears FailureReason/FailureSnapshotJson on
every Succeeded write, so a sweep-reconciled session (049b6e5b, session 294ab5b4) was
indistinguishable from an agent-reported completion. Operators must always be able to
tell whether WE declared a status or the device reported it.
- New Sessions column ReconcileReason, set only for backend-automatic Succeeded verdicts:
the maintenance sweep's classifier reason is persisted verbatim; a reason-less late
completion upgrading a prior Failed/Incomplete/AwaitingUser verdict synthesizes
"Late completion report received — upgraded prior '<Status>' verdict".
- Admin flips stay attributed via AdminMarkedAction (no double badge); normal agent
completions leave the field empty. Pure ComputeReconcileReason next to
IsTerminalTransitionAllowed, pinned by 17 new facts.
- Survives re-registration (StoreSessionAsync preserve + fresh-terminal-tuple mirror)
and StartedAt-shift reindex (SessionsIndex projection); force-merge path mirrors the
normal path against the FRESH prior status.
- Web: "reconciled" chip in SessionStatusBadge (tooltip = justification) + info box on
the session detail page; dashboard list passes the field through.
- MCP: get_session_summary overview carries reconcileReason.
Backend 2907/2907, web tsc + vitest 557/557, MCP 393 pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Third slice of the metrics-performance pass (after e63a0bc2 usage, 86f5b507
geo): the app endpoints ran unprojected full-column AppInstallSummaries
window scans behind the web app's default 30s fetch timeout.
Backend:
- All AppInstallSummaries scans (full, geo, app-metrics, apps-dashboard)
now delegate to one private QueryAppInstallSummariesAsync core so filter
semantics stay in lockstep
- New projections: AppMetricsProjection for metrics/app + global (ranking
inputs, failure code and the DO counters) and AppsDashboardProjection for
the App Dashboard list/analytics/sessions endpoints (drops the whole DO
telemetry block, which those aggregations never read)
- AppsAnalyticsHelper.LoadSummariesAsync and both app-metrics functions
switched to the projected variants
- AppsProjectionEquivalenceTests pins both projections, incl. the
TenantId-from-PartitionKey fallback for legacy rows
Web:
- CalculatingInline: compact in-place variant of the calculating card
(shared progress bar + elapsed caption extracted)
- Apps installs tab and the app detail page (analytics + sessions fetches)
use 180s AbortSignals and show inline progress while aggregating
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same pattern as the Platform Usage fix (e63a0bc2): the geographic endpoints
ran two sequential full-column window scans per request while the web app's
default 30s fetch timeout could abort the response client-side.
Backend:
- New column-projected queries: GetGeoWindowSessionsAsync ($select Geo*
grouping fields + effective-duration inputs) and
GetGeoAppInstallSummariesAsync ($select session join key, window filter,
throughput inputs and all Delivery Optimization counters; returned
AppInstallSummary objects carry only those fields by documented contract)
- All four geographic functions (tenant/global map + tenant/global location
drilldown) run their scans concurrently via Task.WhenAll; the map
endpoints use both projections, the drilldowns keep full-row sessions
(LocationSessionRow returns nearly every column) but switch to the lean
app scan
- GeoMetricsProjectionEquivalenceTests pins both projections, incl. the
location grouping key and the DoDownloadMode=-1 "no DO telemetry" marker
Web:
- New shared useFetchProgress hook + CalculatingCard component (elapsed
counter + progress bar sized from the last recorded fetch duration,
capped at 95%)
- geographic-performance map + sessions drilldown: 180s AbortSignal and
the calculating card instead of a bare loading line
- SectionPlatformUsage refactored onto the shared hook/component (drops
its inline copy of the timer/estimate logic)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The daily RecomputePlatformStatsAsync overwrote TotalEnrollments,
SuccessfulEnrollments, TotalEventsProcessed, TotalTenants and
UniqueDeviceModels with counts scanned from retention-pruned tables, so
the public "since release" figures regressed after every retention
cleanup. Only TotalUsers already had the high-water clamp.
- extract BuildMonotonicPlatformStats (internal, unit-tested): every
cumulative counter is Math.Max(recomputed, persisted) — the recompute
can raise a counter (self-heal for lost fire-and-forget increments
while sessions are still within retention) but never lower it;
IssuesDetected stays carried over verbatim (increment-only)
- deliberate exception: TotalSignedUpTenants remains current-state —
TenantConfiguration is not retention-pruned, a drop there is real
offboarding, not data loss
- PlatformStats model docs updated to pin the semantics
4 new tests (PlatformStatsMonotonicTests); backend suite 2887/2887.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fresh cross-tenant usage computes took 20-43s server-side but the web app's
default 30s fetch timeout aborted them client-side (HTTP 499), and every
retry stacked another concurrent full compute.
Backend:
- UsageMetricsService: cache 5→15min, single-flight per days-window (retries
join the in-flight compute), the four source scans run concurrently via
Task.WhenAll, statics converted to instance fields (singleton, testable)
- New column-projected queries for the usage compute only:
GetUsageWindowSessionsAsync ($select 13 consumed columns, drops
FailureSnapshotJson & co.) and GetAppInstallRefsAsync ($select
SessionId+AppName, lean SessionAppRef DTO); GetSessionsByDateRangeAsync
is untouched for its other consumers
- Projection equivalence pinned by UsageMetricsProjectionEquivalenceTests
(incl. the WhiteGlove Part-2 duration branch via IsPreProvisioned/
ResumedAt); single-flight/cache/failure-eviction pinned by
UsageMetricsServiceSingleFlightTests
Web (SectionPlatformUsage):
- 180s AbortSignal for this call (backend keeps computing and caches)
- Calculating card with elapsed time and a progress bar sized from the last
fresh-compute duration (localStorage, cache hits excluded, capped at 95%)
- Friendly timeout error explaining the retry will hit the server cache
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Total Sessions" on the usage-metrics page is window-bound (90d default)
and capped by retention. Add a retention-independent per-tenant counter
surfaced as "Total Enrollments (since signup)".
- TenantStats row in the PlatformStats table (PK=tenantId, RK="current";
the platform row's "global" PK cannot collide with a tenant GUID)
- Incremented at session registration, fire-and-forget, fresh
registrations only (agent restarts / WG Part 2 resumes re-register the
same session and must not inflate the counter)
- ETag CAS with retries instead of last-writer-wins: lost increments are
permanent (no recompute can recover pruned sessions), and rollout
waves register concurrently
- Nightly platform-stats recompute seeds/self-heals the counter to at
least the live session count (raise-only floor, analog TotalUsers clamp)
- Tenant usage metrics expose SessionMetrics.TotalAllTime =
max(counter, window total); web page shows a "Total Enrollments" card
plus clarifying window note ("Total Sessions" is not a lifetime total)
Backend tests 2876/2876, tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reorganize docs/ tree:
- docs/design/mcp-oauth-flow.md -> docs/mcp-oauth-flow.md (no customer data)
- enrollment-status-reclassification{,-plan}.md -> tasks/ (git-ignored work
files; they contained a customer domain that must not ship in the public repo)
- remove now-empty docs/design/ folder
Repoint the 22 source files whose comments referenced
docs/design/enrollment-status-reclassification.md to the new tasks/ path.
Comment/docstring text only; no functional code changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session 7443317c proved a WU-tracker blind spot: an OOBE quality update
installed (OsBuild 26200.8037 -> 26200.8655 across the mid-enrollment reboot)
yet the deployed WindowsUpdateTracker emitted ZERO events - the watcher armed
fine (no collector_degradation), the spool is durable, and two 60-min backfills
covered the install window, so the WindowsUpdateClient channel simply carried
none of the targeted EventIDs 19/20/43/44. The tenant has diagnostics upload
disabled, so the fix makes the feature self-evidencing instead of guessing IDs:
- OsBuildChangeHost: persists CurrentBuild.UBR (Registry64 view) to
AgentState/os-build-state.json and emits os_build_changed (Info, immediate)
when it differs across a restart - deterministic update corroboration that
cannot be lost to timing, watermarks, or unexpected servicing paths. First
run seeds silently; corrupt state re-seeds.
- WindowsUpdateTracker channel census: when the build provably changed AND zero
targeted WU events were emitted this run, emit a one-shot
windows_update_channel_census (Debug) with an unfiltered EventID histogram of
the WindowsUpdateClient AND UpdateOrchestrator channels over the lookback
window (capped at 5000 records, cap surfaced as censusTruncated). The
histogram arrives via normal event upload and tells us which IDs to add to
WindowsUpdateTargetedEventIds via remote config - no redeploy, no diag ZIP.
Gated by CollectorConfiguration.WindowsUpdateChannelCensusEnabled (default
on, model-default pattern, no ConfigVersion bump).
- No new hardcoded EventIDs (per the device-validation rule) - census data
decides what to enable remotely.
- ANALYZE-DEV-006 "OS Build Changed During Enrollment" (info, not KO):
interpolates {{previousBuild}}/{{currentBuild}}, counts the census as
corroboration. Catalogs: Constants.cs + MCP resource-catalog.ts + web
eventTypes.ts (drift-test enforced).
Tests: 6 OsBuildChangeHost (seed/same/changed/unreadable/corrupt/no-dir),
7 census (gating matrix, truncation surfaced, scanner-throw swallowed,
histogram format), 3 ANALYZE-DEV-006. Agent.V2.Core 1754/1754, Backend
2874/2874, MCP 379 (drift green), web tsc + vitest 367/367, sln 0 errors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 9a107a19 reboot-cancel guard correctly stopped false
esp_exit_without_completion_evidence failures, but a pure cancel left the
post-AccountSetup dead-end zone with no resolution-capable deadline: session
7443317c (same tenant/shape as 1924092e, plus a mid-OOBE Windows-Update reboot
chain) tripped the session_parked_without_deadline liveness tripwire on a live
enrollment that later completed normally.
RebaseEspExitVariantAdvisoryWindowOnReboot now replaces the stale window with a
fresh 30-min one whose enforcement baselines (armSignalOrdinal /
armAppTerminalCount) anchor at the reboot signal, from both
HandleSystemRebootObservedV1 and HandleEspResumedV1:
- A live enrollment keeps disproving the dead-end at each fire via
HasEnforcementProgressSinceArming (re-arm, never fail).
- A true dead-end where the user rebooted inside the window now resolves one
window later (30-60 min) instead of idling to the 6-h max-lifetime watchdog -
the 9a107a19 trade-off gap is closed.
- The parked tripwire needs no change: a resolution-capable deadline is armed
throughout, so IsParkedWithoutDeadline stays false and the tripwire keeps its
"every fire is a bug" charter.
- The rebase emits a completion_waiting with the new resolutionDeadlineDueAtUtc
(trigger <Signal>:AdvisoryRebase), deliberately bypassing the fingerprint
dedupe like the EnforcementActive re-arm site - bounded to one per reboot.
Tests: rebase with reboot-anchored baselines (both call sites), advisory
variant exempt, no-window no-op, true-dead-end-after-reboot fails at the
rebased due-time, 7443317c double-reboot live-install replay, fire-once keeps
the rebased window, 1924092e full replay updated (stray fire re-arms on
progress), engine-driven parked-tripwire regression.
DecisionCore 445/445, Agent.V2.Core 1754/1754.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Session a4537c36: Windows closed the User-ESP page normally (Shell-Core 62407,
errorCode=0) while the AccountSetup registry never wrote categorySucceeded -
the strong gate (arm A) was unsatisfiable by construction and the session only
completed because the Hello policy happened to flip to enabled after a reboot.
ShouldTransitionToAwaitingHello gains arm C: AccountSetup entered + normal ESP
final exit (post-AccountSetup by INGEST ordinal - backdated CMTrace timestamps
are irrelevant, sequence is canonical) + genuine IME user-session completion
(defaultuser0 guard: at-or-after the AccountSetup anchor, extracted as shared
IsImeUserSessionGenuine and reused by the AdvisoryCompletion backstop) + DAD-
validated real-user desktop. ALL FOUR mandatory; Hello gating stays at the
call sites - arm C never bypasses an enabled-unresolved Hello.
Wiring per signal ordering: desktop last via the fast-path (desktopArrived-
InFlight), exit last via HandleEspExitingV1 (espFinalExitInFlight + deferred-
completion parity mirroring the caa6cf50 branch), IME last via new completion/
promote wiring in HandleImeUserSessionCompletedV1. completion_waiting stops
reporting account_setup_provisioning_complete once the evidence set holds.
AdvisoryCompletion stays as the 30-min backstop for incomplete evidence.
The 08c99638 regression guard (3-of-4, no IME fact -> parked) stays green
unmodified; the L5 backdated-exit test now asserts the eager completion.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 1ec8f4c6 dead-end window (guard-blocked post-AccountSetup esp_exiting arms a
30-min advisory_completion resolution deadline) misfired on the Device->Account
handoff exit whenever the IME logged an AccountSetup phase line pre-sign-in
(defaultuser0/AutoLogon frame). The recovered deadline then failed sessions
mid-AccountSetup while apps were actively installing:
enrollment_failed: esp_exit_without_completion_evidence on a live enrollment.
41 of the 65 affected sessions since 2026-06-15 show reboots or app activity
after the arming exit - false positives.
Two guards, esp-exit variant only (the advisory variant keeps its real-failure
un-defang semantics):
- Reboot cancel: SystemRebootObserved / EspResumed cancel the armed window -
an arming exit that predates a reboot was a pre-reboot page close, not the
"page closed, nothing happens" dead-end. A later genuine guard-blocked exit
re-arms a fresh window.
- Enforcement-progress re-arm: arming baselines (armSignalOrdinal,
armAppTerminalCount) ride on the deadline's FiresPayload (replay-deterministic,
snapshot-round-trips, no DecisionState schema change). At fire time, apps
reaching terminal states or an ESP user-phase re-assert since arming re-arms
the window instead of failing. Convergent: each re-arm demands new progress;
legacy deadlines without baselines resolve with pre-fix semantics.
11 new tests incl. a full session-1924092e replay (handoff exit -> reboot ->
active installs -> stray recovered-timer fire dead-ends -> Completed) and pins
for the true dead-end verdict, the advisory variant, and baseline serialization.
DecisionCore 435/435, Agent.V2.Core 1725/1725.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An ESP subcategory failure can be retracted inside the 30s settle window
(user clicks "Try again" / IME re-sync flips failed -> inProgress). The
expiry handler fired the latched EspFailureDetectedEventArgs
unconditionally, terminating the agent on a failure Windows no longer
reported and losing the retry outcome (session c071e92b: recovery at
+12s, terminal esp_terminal_failure at +30s anyway).
- OnProvisioningFailureSettleExpired re-checks current registry-derived
state under _stateLock: recovered = categorySucceeded != false AND no
subcategory currently "failed". On recovery: emit new
esp_failure_settle_recovered event, suppress EspFailureDetected, clear
the fire-once gate (+resolved set) so a re-failure arms a FRESH settle
window and is then terminal.
- esp_failure_settle_started now names tracked apps not completed
(trackedAppsNotCompleted/-Count) on Apps-subcategory failures via a new
packageStatesProbe (ImeLogHost.AllKnownPackageStates through
DefaultComponentFactory -> EspAndHelloHost -> EspAndHelloTracker).
Covers the Store/WinGet blind spot (Company Portal pending with zero
IME events) that both app_install_starved (IME-phase + intent gates)
and esp_apps_failure_correlation (IsActive device apps only) miss.
- New event type registered in Constants.EventTypes + MCP resource
catalog (drift test green); 5 new settle-window tests, suite 1711/1711.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
netstandard2.0's string.IsNullOrEmpty lacks [NotNullWhen(false)], so the
compiler cannot prove the else branch is non-null. Null-forgiving assert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session 2bc884b6: ESP failed AccountSetup/Apps with 0x80073cf9 while all 8
tracked Win32/WinGet apps succeeded — the failing app was a MSIX/Store app
that installs outside the IME Win32 pipeline and is invisible to
ImeLogTracker. The session ended as a blackbox failure.
When an Apps-subcategory failure arms the 30s settle window, the tracker now
queues a one-shot scan of Microsoft-Windows-AppXDeploymentServer/Operational
(Critical+Error, session window, 200-record/10s cap), scores candidate
packages (exact-HRESULT match > AppX-family error > time proximity, recency
bonus), and emits esp_appx_failure_analysis with flat RuleEngine fields plus
a capped candidates[] detail list — before enrollment_failed lands in the
timeline. Negative results are emitted too (no_appx_candidates points back
at Win32). Fire-once per session; scan runs lock-free on the threadpool;
settle expiry never waits; failures degrade to collector_degraded.
- New: AppxDeploymentFailureScanner (one-shot reader, injectable seam),
AppxFailureAnalyzer (pure scoring/verdicts, incl. negative-decimal
HRESULT + PackageFullName extraction from message/properties)
- ProvisioningStatusTracker: optional scanner/dispatcher ctor seams,
MaybeStartAppxFailureScan at settle-arm time
- Rule ANALYZE-APP-014 (MSIX/Store app failure during ESP) keyed on
verdict=appx_candidate_identified; dist regenerated
- MCP resource-catalog + web HRESULT badge + gather-rules eventTypes
- Tests: 16 analyzer + 7 trigger; settle-window fixture now injects a
fake scanner + sync dispatcher (no real event log in tests)
Inert until agent redeploy (rule needs backend redeploy). Post-deploy
checks in tasks/todo.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New one-shot network_bandwidth_estimate event derived passively from the
Delivery Optimization byte counters the DeliveryOptimizationCollector
already polls every 3s - zero synthetic traffic, zero extra load, no
config change.
- BandwidthEstimator: per-FileId delta windows (a DO job that drops off
the list and returns is averaged over its own absence window), p90/max
reduction, WAN (HTTP + internet peers) vs LAN (LAN/group/link-local
peers + Connected Cache) split so peer-fed downloads cannot inflate
the WAN figure; bucket (<10/10-50/50-100/100-250/250+) + confidence.
- Emitted at most twice per session (snapshotTrigger): interim snapshot
on first AccountSetup sighting (device_setup_end - survives sessions
that starve in the account phase) and authoritative final at collector
stop (collector_stop, lands before diagnostics_collecting).
- All DO jobs feed it (IME apps, Office CDN, WU/Store/Defender).
- Registered in Constants.EventTypes, MCP resource-catalog
(network_events) and web gather-rules autocomplete.
- Drive-by: add missing agent_emergency_break to the MCP catalog
(pre-existing drift, event-types-drift test was red).
13 new estimator unit tests; agent suite 1655/1655, MCP drift 6/6,
web tsc + eventTypes vitest green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 1 - named notification channels (1..N per tenant):
- NotificationChannel model (provider/url/headers/enabled + per-event
opt-ins incl. single NotifyOnSlaEvents); stored as JSON column
NotificationChannelsJson, cap 10, fail-soft ParseList
- No data migration: GetNotificationChannels() synthesizes one channel
from the legacy single-webhook fields under stable id "legacy";
web materializes on first save and clears legacy fields
- All dispatch sites (ingest success/failure/white-glove/ESP, SLA breach/
resolved/consecutive, enrollment-start, hardware-rejected, test) route
via SendToChannelsAsync with per-channel toggle filtering
- Per-channel secret redaction (structure stays visible to readers) with
restore-by-id; save-time validation (dup ids, provider, https, headers)
- Settings UI: channel card list with add/remove, event pills, per-channel
test button ({channelId} body, legacy-compatible without body)
Phase 2 - analyze-rule channel notifications:
- NotifyDefault (definition, false for all built-ins, test-guarded) +
Notify/NotifyChannelIds tenant override in RuleStates (two-tier mirror
of MarkSessionAsFailed); custom rules fold into the tenant-owned row
- Dispatch in AnalyzeOnEnrollmentEndHandler only: outcome.Results is
new-findings-only (engine dedup) so no re-notification; manual
"Analyze Now" reanalyze never dispatches
- BuildRuleFiredAlert (eventType analyze_rule_fired): device, serial,
rule, severity, category, confidence, explanation + session deep link
- Rule card: notify toggle + channel target pills, bell header badge
2858/2858 Functions tests (19 new), web tsc + vitest clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-07-08 timeout reclassification (Incomplete = terminal, non-failure)
migrated SessionStats, FleetHealthStats and SessionStatusBuckets, but two
reporting surfaces were overlooked: the live UsageMetricsService.SessionMetrics
(both platform + tenant paths, feeding get_usage_metrics and the usage
dashboards) and the persisted daily UsageMetricsSnapshot. Terminal Incomplete
sessions were counted in Total but in no breakdown bucket, so
Succeeded+Failed+InProgress no longer reconciled to Total and operators lost
sight of those devices.
- SessionMetrics + UsageMetricsSnapshot gain an Incomplete / SessionsIncomplete
count, mirroring the migrated peers (Incomplete only, not AwaitingUser).
- Snapshot column stored + mapped in TableStorageService.Metrics; legacy rows
deserialize to 0 (no seed bump needed).
- Success rate unchanged everywhere: denominator was already Succeeded+Failed,
correctly excluding Incomplete/AwaitingUser. SLA paths verified already-correct
(BuildSnapshot + breach evaluator receive terminal-filtered sessions).
- Web: Incomplete card added to both usage views (slate, matches the badge).
- MCP: query_raw_sessions status enum learns AwaitingUser/Incomplete so callers
can filter by the new states.
- Test: UsageMetricsServiceWindowTests pins the Incomplete bucket + 50% rate on
the crcins 1/1/8 shape.
Backend build clean, full Functions suite 2825 pass, web + MCP tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses four review rounds on the enrollment-status reclassification
(timeout != failure; new AwaitingUser/Incomplete states).
Backend correctness:
- UpdateSessionStatusAsync ETag force-fallback now mirrors the normal path:
uses IsTerminalTransitionAllowed + admin-marked guard, stamps CompletedAt
for Incomplete, and persists FailureReason/FailureSnapshotJson for
Incomplete/AwaitingUser (not just Failed) so a contended write cannot
regress a terminal or drop the classification reason/snapshot.
- StoreSessionAsync CAS loop re-reads the terminal tuple in the fresh-read
window and preserves a just-landed terminal (Succeeded/Failed/Incomplete)
verbatim, so re-registration can no longer revert it via the Replace.
- SessionGraceHours + AbsoluteMaxSessionHours now roundtrip through the
tenant-config Store/Map pair (overrides were silently lost).
- ResolveGraceHours clamps the assumed agent cap to >= the real runtime
default (48h) until the agent honors the override, so a low override can
never terminalize a still-running session early.
- Incomplete now funnels through the terminal counter-reconcile seam, and
the maintenance sweep re-reconciles after the synthetic session_timeout
event (also fixes the pre-existing off-by-one on the Failed path).
Web:
- SessionInfoCard delegates its status pill to the shared SessionStatusBadge
(removes a divergent local map; picks up AwaitingUser/Incomplete).
- SessionTable search color map + status filter chips learn the new states.
- useSessionEvents uses isTerminalStatus() instead of a hard-coded
Succeeded/Failed check (no needless refetch on Incomplete).
- DataManagementSection copy no longer says "Failed - Timed Out".
Docs/comments:
- Plan doc + TenantConfiguration comments corrected (+3h/51h buffer, and the
timeout now reclassifies to AwaitingUser/Incomplete, not "Failed").
Tests: +ResolveGraceHours clamp case, +grace/abs-cap config roundtrip +
legacy-row defaults. Full backend suite green (2712), web tsc green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windows Update activity during Autopilot OOBE/ESP is a blind spot no other tool
(Intune console included) surfaces well — a cumulative update installing mid-OOBE
can break enrollment (r/Intune KB5095189). This makes it visible with secured
evidence.
Agent (net48):
- WindowsUpdateTracker + WindowsUpdateWatcherHost: watch
Microsoft-Windows-WindowsUpdateClient/Operational (EventIDs 19/20/43/44 =
success/failure/install-started/download-started), startup backfill (OOBE updates
run before the agent starts), RecordId watermark dedup (restart boundary vs
intra-run seen-set so a high-id live event can't hide older backfill events),
HRESULT->symbol decode. Emits windows_update_succeeded/_failed/_started with the
real event Timestamp. Default-on config knobs; uncertain EventIDs configurable
without redeploy.
- RegistryCollector: emitOnlyIfExists (suppress positive-assertion events when the
key is absent) + Registry64 base view by default (bitness-independent, matches
TenantIdResolver) with optional registryView override.
Rules:
- ANALYZE-DEV-004 (high: WU failed during enrollment, high-insight not KO) +
ANALYZE-DEV-005 (info: WU installed). reboot_pending corroboration matches
event_data exists==true.
- GATHER-DEVICE-005 -> windows_update_reboot_pending (emitOnlyIfExists),
GATHER-DEVICE-004 -> windows_update_history (Get-HotFix, opt-in).
- 5 EventTypes synced across Constants + MCP resource-catalog + web eventTypes.
Self-maintaining rule lifecycle (analyze + gather parity):
- Gather rules gain the sunset/orphan-GC that only analyze had
(ProcessSunsetGatherRuleAsync + runtime filter + seed/reseed/github paths).
- RuleProvenance field (embedded/github, null=embedded back-compat): the embedded
catalog only sunsets its own rules; GitHub-reseeded rules ahead of the deployed
binary (new ids AND content updates to existing ids) are exempt and reclaimed to
embedded once the binary catches up. No orphans, no operator knowledge required.
Verified: full solution builds; backend suite 2770 pass, agent suite 1638 pass;
rules AJV-valid + combined.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- SessionStatusBuckets: explicit AwaitingUser + Incomplete buckets (out of Other),
still reconciling to Total.
- Metrics summary: expose awaitingUser + incomplete; failure rate is now
Failed/(Succeeded+Failed) — Incomplete and non-terminal states are out of the
denominator (crcins drops from failed/total to the real terminal rate).
- SessionStats.IncompleteLastNDays; dashboard SuccessRatePct was already terminal-only.
- Daily aggregation already used Succeeded+Failed denominator — unchanged.
- Tests: SessionStatusBucketsTests. Suite: 2705 pass. Fleet Health Incomplete surfacing
deferred to PR5 (web) with its documented over-all-sessions semantics.
Groundwork for making the agent's silent 48h emergency break transparent
(docs/design/enrollment-status-reclassification.md):
- Constants.EventTypes.AgentEmergencyBreak + AgentErrorType.SessionAgeEmergencyBreak.
- EnrollmentTimeoutClassifier: ExtractRollup.HasAgentEmergencyBreak; classify a
session carrying the break as Incomplete NOW (agent is gone) instead of holding it
as AwaitingUser through the grace — a real completion still wins. Tests added (22 total).
Next: agent best-effort emit at the break + backend materializes the timeline event.
The 5h→terminal grace was a magic 72h. Anchor it to the agent instead: the 6h
per-run brake emits enrollment_failed (→ Failed at ~6h), but the 48h absolute
emergency break (AbsoluteMaxSessionHours) is SILENT to the backend — it writes a
local marker, cleans up and exits with no terminal event. So a session only stays
non-terminal past 48h when the agent hit that silent cap and is provably gone.
Grace must therefore be >= the agent absolute cap (never mark Incomplete while the
agent could still legitimately enroll) and only slightly beyond it:
effectiveGrace = max(AbsoluteMaxSessionHours + 12h buffer, SessionGraceHours) = 60h default.
- EnrollmentTimeoutClassifier.ResolveGraceHours (pure, tested): floors grace at
agentCap + buffer; a SessionGraceHours override can only raise it.
- TenantConfiguration.SessionGraceHours default 0 (= auto-derive); mirror
AbsoluteMaxSessionHours so the floor follows a tenant override of the agent cap.
- Sweep uses ResolveGraceHours instead of the hard-coded 72h.
Follow-up noted: wire AbsoluteMaxSessionHours down into AgentConfigResponse so the
agent and backend share one value. Functions.Tests: 2678 pass.
Wire EnrollmentTimeoutClassifier into MarkStalledSessionsAsTimedOutAsync so the
5h inactivity sweep stops blindly failing silent-but-provisioned sessions
(docs/design/enrollment-status-reclassification.md). Forward-only.
- TenantConfiguration.SessionGraceHours (default 72h) + default config.
- Sweep Stage 2: classify from the ESP subcategory rollup instead of hard Failed.
Fast-path skips within-grace AwaitingUser sessions with NO event read (grace is
time-based; late completions are caught by ingest). Per-outcome counters; audit
+ ops event only when the pass actually terminalized/transitioned something.
- session_timeout event + auto-analyze kept ONLY for Failed/Incomplete (parity with
the agent max-lifetime path); AwaitingUser is non-terminal, Succeeded is a reconcile.
- GetStalledSessionsAsync includes AwaitingUser so it graduates to Incomplete once
grace elapses; agent-silent (Stage 1) stays InProgress-only.
- UpdateSessionStatusAsync: Incomplete is terminal (idempotency); AwaitingUser can't
regress a terminal; CompletedAt stamped for Incomplete (no skewing duration); reason
+ snapshot persisted for Incomplete/AwaitingUser.
Full Functions.Tests suite: 2670 pass.
Foundation for reclassifying 5h-timeout sessions (docs/design/
enrollment-status-reclassification.md). No behaviour change to the sweep yet —
this only adds the pieces PR2 wires in.
- SessionStatus: append AwaitingUser (non-terminal) + Incomplete (terminal,
non-failure). Members appended so persisted ordinals never shift.
- EnrollmentTimeoutClassifier: pure ExtractRollup (parses the esp_provisioning_status
"N of M subcategories completed" message + the 30s all-succeeded fallback line —
categorySucceeded is ignored, Windows never sets it) and ClassifyTimedOutSession
implementing the decision table (explicit failure→Failed; AccountSetup all-succeeded
/ enrollment_complete→Succeeded; DeviceSetup done + within grace→AwaitingUser, else
Incomplete; silent pre-DeviceSetup→Incomplete). Never returns Failed without an
explicit failure signal.
- FailureSnapshotBuilder: embed the same rollup (schema v2) so the snapshot records the
evidence behind the verdict.
- Tests: EnrollmentTimeoutClassifierTests (12). Full Functions.Tests suite 2670 pass.
- High: TenantConfigurationService.SaveConfigurationAsync now THROWS when the repository reports a failed persist (it swallows storage exceptions and returns false) - plan/trial/config writes can no longer be audited and returned as 200 without being persisted. Cache is invalidated on every attempted save (finally), covering in-place mutations of the cached instance; the now-redundant invalidate-on-failure wrapper in PlanManagementFunction is removed. AuthFunction login side-effect saves (first-login domain write, auto-re-enable) are wrapped best-effort so a storage blip cannot 500 auth/me - they self-gate and retry on next login.
- Medium: MCP usage increment moved from AuthenticationMiddleware (fire-and-forget BEFORE the quota decision) into McpQuotaEnforcementMiddleware: strictly check-then-increment and only for requests actually served (GA tracked but quota-exempt; denied 403/429 requests no longer inflate counters; the exact-limit request is now deterministic - 1..limit pass, limit+1 blocks).
- Low: fixed both new nullable warnings (PlanTierDefinitionParser guarded json, McpQuotaEnforcementMiddleware principal null-flow).
Tests: +3 save-semantics regression tests (throw on false, cache invalidation on failed/successful save). Backend 2656/2656 green, build clean.
Fail-closed entitlement layer (FeatureEntitlementCatalog + TenantEntitlementService): edition resolves at read time (PlanTier == 'enterprise' OR active trial; legacy free/pro => Community, no migration).
- Plan/trial endpoints: PATCH config/{tid}/plan (GA: tier + grant/extend/end trial, audited, cache-safe) and POST config/{tid}/trial (tenant-admin self-service 30d, once per tenant, 409 after). Generic PUT preserves plan/trial fields for all callers. Fixes SetPlanTier writing past the config cache without audit/UpdatedBy.
- Rate limits: Enterprise floor 150/min for BOTH user (portal/JWT) and device (agent/cert) paths - floor raises the global default, an explicit per-tenant override still wins; MSP users follow their home tenant.
- Retention cap: Community 90d / Enterprise 365d, clamped in the retention fanout (warn log) + non-GA write validation 7..cap (only on changed values; 0=infinite stays GA-only).
- Delegated admin (MSP): Enterprise gate on the delegated admin's HOME tenant (JWT tid, fail-closed on unknown) - managed target tenants may be any edition. Grant-time target check removed; inert grants are suppressed at resolve time (single choke point in GetScopeAsync, applied after the UPN-keyed cache).
- MCP quota: McpQuotaService + McpQuotaEnforcementMiddleware (X-Client-Source: mcp only, GA exempt) enforce daily/monthly budgets - per-user plan override > tenant edition plan; limits from SectionUsagePlans with catalog fallbacks (community 100/3000, enterprise 1000/20000); 429 with X-MCP-Quota-* headers + Retry-After; fail-open on counter reads, fail-closed on plan resolution. /api/metrics/mcp-usage/me reports effective plan + counters.
- Trial visibility: TrialExpirySweepFunction (daily 03:30 UTC, informational) emits TenantTrialExpired (Warning, 24h look-back) + TenantTrialExpiring (Info, <=3d heads-up); dual-registered in web OPS_EVENT_TYPES.
- Web: EditionBadge + one-time trial CTA in settings header, Plan & Trial card in the GA tenant edit modal (own PATCH save), retention cap hints from entitlements, pure lib/edition.ts (vitest), feature-flags surface edition/trial/entitlements; MCP TENANT_SAFE_FIELDS + trial fields.
Verified: solution build clean; backend 2653/2653 tests; web tsc + 516/516 vitest; MCP server tsc + tests green.
Replace the denormalized per-tenant RateLimitRequestsPerMinute mirror (and its
SaveConfigurationAsync sync job) with compute-at-read resolution: effective limit =
per-tenant override ?? global default. Fixes the clobbering bug where a per-tenant
edit to the base field was overwritten on the next global config save.
- Model: drop TenantConfiguration.RateLimitRequestsPerMinute; add nullable
CustomUserRateLimitRequestsPerMinute (device override CustomRateLimit... kept).
- RateLimitResolver centralizes precedence (device + user); both paths call it.
- SecurityValidator reads the global default (AdminConfigurationService threaded
through ValidateSecurityAsync + all device-auth callers); 5-min cached.
- UserRateLimitMiddleware: per-tenant user override for tenant-scoped callers only
(skipped for HasGlobalScope = GlobalAdmin|GlobalReader); GA -> GA budget, others ->
user default. Uses GetConfigurationIfExistsAsync so an unregistered-tenant JWT can
no longer auto-create a config row via read endpoints.
- Validation: reject rate limits < 1 at the write functions; RateLimitService clamps
< 1 to 1 as a safety net (avoids Min()-on-empty / throttle-everything).
- Web: Global Settings exposes User + GlobalAdmin limits; tenant modal writes the
override fields (blank = inherit); report + usage-plans updated.
Legacy RateLimitRequestsPerMinute Table column is ignored on read and dropped on the
next Replace save (self-cleaning; no migration).
Backend 2577 xUnit green, tsc clean, web 506 vitest green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Retire ANALYZE-ESP-003 (duplicate of ESP-001) and ANALYZE-CORR-002 (duplicate
of APP-004) via the established sunset path; ESP-001 gains a >60min factor,
APP-004 absorbs the co-occurrence story with working confidence factors
- CORR-003: download evidence (errorPatternId) is now a required condition —
no more firing on proxy + any unrelated app failure
- DEV-002 v2: 'sustained' semantics — new event_count value filter
(filterField/filterOperator/filterValue in schema, Shared model, engine, web
types) requires >=3 snapshots above 90% memory instead of one transient spike
- DEV-003 v2: build-cutoff-only condition (<=22621) now also covers Windows 10;
LTSC caveat + maintenance note (raise to 22631 after 2026-11-10)
- SEC-001 v2: rewritten for the post-June-2026 reality — certificates HAVE
expired, active remediation required (OEM firmware hint added)
- SEC-005/006 v2.1: allow-list extended with OEM factory-preload families
(Intel PPM, AMD Power, Lenovo ThinkPad power/battery, Surface/carrier),
telemetry-curated from 1780 fired sessions; imaging leftovers (USMT.PPKG,
InstallOrder.ini) and generic names deliberately still alert
- APP-001: suppressByEvent on app_install_completed; APP-002: exact MSI exit
codes instead of bare 16xx; APP-006 reframed as 'App Install Retry Loop';
ID-001: title/logic mismatch fixed + GATHER-ID-002 dependency documented
- Fix all 13 dead confidence factors (signal-name mismatches / unsupported
syntax) by rewiring or removal; version bumps everywhere so seeding
propagates the updates
- New RuleEngineEventCountFilterTests + SEC-005 OEM allow-list coverage
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kill/Block rules were only served on the telemetry response, which cannot
reach an agent whose upload loop is paused (indefinite block pins the drain
until DateTime.MaxValue), whose spool is empty, or whose binary is kill-blind.
Config is fetched at every process start (boot via Scheduled Task), making it
the reliable control channel - no heartbeat added by design (48h emergency
break + 6h max lifetime bound the remaining cases).
Backend (effective on deploy):
- KillSwitchEvaluator: shared device+version check for both channels; ingest
wire response and check order unchanged (device first, short-circuits)
- Every served Kill emits ops event KillSignalDelivered (Security/Warning),
throttled 24h per tenant+serial+pattern; Blocks stay event-free
- GET agent/config (incl. bootstrap wrapper) now returns DeviceBlocked/
DeviceKillSignal/UnblockAt alongside the full config; old binaries ignore
the new fields
- Web: KillSignalDelivered dual-registered in OPS_EVENT_TYPES
Agent V2 (effective with next agent release):
- CheckConfigKillSignal guard after the Phase-4 fetch: forced self-destruct
(overrides SelfDestructOnComplete), enrollment-complete marker, session
cleared, exit before mTLS/registration
- Kill honoured ONLY from a live fetch: config disk cache strips the flags on
write and read (UnrestrictedMode pattern), guard re-checks LastFetchOutcome
Tests: 9 KillSwitchEvaluatorTests + 7 ProgramConfigKillSignalTests;
Functions 2557/2557, Agent V2 1578/1578, web tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Delta-review 2026-07-02 findings:
- M1: session_parked_without_deadline evaluated the parked predicate on the
very step that produced the state, so the healthy Classic window
(HelloResolved -> AwaitingDesktop, no deadline, desktop seconds away) fired
a spurious Warning AND consumed the one-shot, masking a real variant-4 park
later in the run. The tripwire now only arms a 10-min dwell timer; it fires
(and consumes the one-shot) only if the session is still parked when the
dwell elapses. Steps that clear the parked condition cancel the pending
arm; parked pass-throughs do not re-base it. Processor is now IDisposable
(timer), disposed by the orchestrator.
- L8 (rides along): the tripwire post moved off the ingress worker onto the
timer thread — a full channel can no longer self-deadlock on it.
- M2: ImeUserSessionCompletedUtc was set-once, so an early defaultuser0
completion permanently failed the AdvisoryCompletion conjunction and
resolved genuinely-completed sessions as Failed. New rule: first
post-AccountSetup observation wins (one upgrade of a pre-anchor stamp,
then frozen). Pure function of (state, signal) — replay-safe.
- L5: HandleAdvisoryCompletionDeadlineFired`s esp-exit anchor is presence-only
now; the old >= AccountSetupEntered comparison rejected fires whose arming
exit carried a backfilled source timestamp, re-opening the dead-end.
deadlineStillArmed remains the stray-fire guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persists the agent-sent self-deploying/kiosk profile marker (template:
IsHybridJoin, write-once): Sessions column + SessionsIndex full-mirror
projection + SessionSummary mapping. Re-registration preserve is a
sticky-true OR - a boot with an unreadable Autopilot policy cache must
not pin the flag to false once it was observed true.
Filterable end-to-end: SessionSearchFilter + continuation-token
fingerprint + search/raw query params + index/basic scan filters, plus
MCP search_sessions / query_raw_sessions inputs and the get_session
overview field.
Display/analytics only - the completion fix itself ships in the agent
(DecisionCore gates on the EnrollmentFactsObserved seed, not this flag).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Kiosk devices (self-deploying profile) never completed: the IME logs an
AccountSetup phase line for the kioskUser0 autologon session although user
ESP never runs, which cancelled/blocked the DeviceOnlyEspDetection deadline
- the sole SelfDeploying terminal path - and the session idled to the 5h
backend timeout (reference session 320b3bf7; 436 false Failed platform-wide,
90% of all self-deploying failures).
Platform sweep (8983 sessions / 63 tenants): CloudAssignedOobeConfig bits
0x20|0x40 are exclusive to self-deploying/kiosk/shared profiles (0 of 6785
user-driven sessions carry them, bits always co-occur). The fix seeds
Mode=SelfDeploying@High from that registry marker via EnrollmentFactsObserved
and waives the AccountSetup veto at the four existing decision points
(DSPC short-circuit, deadline race guard B, EspPhaseChanged cancel,
RealmJoin deferred-release re-check) - only when the seeded profile holds
AND user ESP made no real progress. Bits absent = byte-identical behaviour.
Also registers IsSelfDeployingProfile on SessionRegistration (display
metadata; backend persistence follows separately).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
M6: DeleteByFilterInBatchesAsync is fail-loud and counts only confirmed
batch submits; the orphan cleanup keeps the EventSessionIndex entry when
the event delete fails, so surviving rows stay discoverable and retry.
M7: GetSessionsOlderThanAsync gains excludeInFlightDeletions - sessions
locked in deletion states no longer occupy the capped read head, so a
stuck head (Poisoned / stranded Queued) cannot starve the retention
fanout tail. Warnings surface a fully-locked backlog and zero-progress
runs.
M8: ingest logs a per-batch warning with per-kind counts and the first
TelemetryItemId when unparseable telemetry items are dropped (agent
clears its spool on success, so drops were silent data loss).
M9: ReconcileSessionRebootCountAsync -> ReconcileSessionCountersAsync;
one partition scan reconciles EventCount and RebootCount at terminal
transitions, self-correcting at-least-once retry double-counts. Script
counters stay increment-only with the drift documented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The V1 agent line is retired: no new V1 sessions since 2026-05-09, only
orphaned straggler traffic remained on /api/agent/ingest (last seen
2026-06-29, events for long-cleaned-up sessions). Bootstrap/Intune/CI
have deployed V2-only since the v2 cutover.
Agent:
- Delete AutopilotMonitor.Agent, .Agent.Core, .Agent.Core.Tests
(projects + sln entries). V2 trio, SummaryDialog, DecisionCore.Tests
and V2Isolation.targets stay.
Backend:
- Relocate the shared statics/DTOs V2 reused from the V1 partials
(StampServerFields, SanitizeEventTimestamps, IsMaxLifetimeAgentShutdown,
RecalculateAppDurations, EventClassification, AppInstallAggregationState)
into EventIngestProcessor; IngestEventsOutput moves next to
IngestTelemetryFunction (same namespace, zero ripple).
- Delete IngestEventsFunction (route agent/ingest), BootstrapIngestEvents
(route bootstrap/ingest), NdjsonParser and their DI registration.
EventDataNormalizer stays (used by TelemetryPayloadParser).
- Drop both routes from EndpointAccessPolicyCatalog and
RequestTelemetryMiddleware.TenantHeaderTrustedPaths.
- Retire the V1 update-hash oracle: LatestAgentV1*/LatestBootstrapV1*
fields + GetAgentLine case 1 (default arm returns empty hashes, so
stragglers gracefully skip the integrity check; ParseAgentMajor keeps
its default of 1 on purpose — defaulting to 2 would feed V2 hashes to
V1 binaries). TableConfigRepository Store+Map updated in lockstep.
- Remove ApiEndpoints.IngestEvents/.BootstrapIngestEvents constants.
Tests: NdjsonParserTests deleted; timestamp/critical-path/max-lifetime
tests repointed to EventIngestProcessor; the two NDJSON pipeline tests
removed (coverage retained by direct-list siblings); catalog/no-store
InlineData rows updated.
Docs: architecture.md (endpoint tables, ingest flow diagram, solution
layout, test matrix), web SectionAgentInternals endpoint tables,
build.ps1 help text.
Azure: no config change needed — /api/agent is not in the client-cert
exclusion paths (agent/ingest was always cert-enforced) and the
/api/bootstrap prefix exclusion remains for the surviving bootstrap
routes (verified via ARM on autopilotmonitor-api).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the Tenant Templates -> Tenant Groups rename down to identifiers,
tables, routes and DTOs (the earlier commit renamed only the UI labels). Done
now while the table is effectively empty; the new TenantGroups /
TenantGroupAssignments tables are auto-created on deploy, the operator re-enters
their group, and the old TenantTemplates table is dropped in Azure manually.
- Tables: TenantTemplates -> TenantGroups, TenantTemplateAssignments ->
TenantGroupAssignments (Constants + All).
- Entities/DTOs: TenantTemplate(Entity)/TenantTemplateAssignment(Entity) ->
TenantGroup*, property TemplateId -> GroupId.
- Repo/service/API: all *Template* methods -> *Group*; routes
/api/global/tenant-templates -> /api/global/tenant-groups, {templateId} ->
{groupId}; function class + file renamed; audit entity DelegatedTemplateAccess
-> DelegatedGroupAccess; NormalizeTemplateId -> NormalizeGroupId.
- Offboarding wipe list + tests updated.
- Web: api.tenantGroups, SectionTenantGroups (file renamed), response key groups.
Scoped strictly to the feature: unrelated 'template' usages (analyze-rule
templates, EmailTemplates, interpolate-rule-template) left untouched. Backend
2538 tests green; web tsc clean + vitest 494 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let a GlobalAdmin define a named Tenant Template (a bundle of tenants) and
assign a delegated-admin UPN to the template instead of to each tenant. A UPN's
effective scope = union of direct grants + every tenant in every assigned
template, resolved in DelegatedAdminService.GetScopeAsync — so middleware
enforcement, /auth/me delegatedTenantIds and MCP all inherit it unchanged.
Adding a tenant to a template grants it to all assignees at once (membership-
driven MSP management). Fully app-internal: no Entra group claim / Graph.
- Storage: TenantTemplates (PK=templateId; meta + membership rows) and
TenantTemplateAssignments (PK=upn) tables; meta-backed existence invariant in
both management reads AND the auth hot path (no ghost templates, no scope from
stray rows).
- Offboarding hygiene: property-only wipe now purges TenantTemplates membership
AND DelegatedAdmins rows for the offboarded tenant (closes a pre-existing
orphaned-grant gap).
- Management API: GlobalAdminOnly /api/global/tenant-templates/* (8 routes);
per-affected-tenant audit; all mutations route through the service so the
delegated-scope cache is invalidated in lockstep; existence/membership
invariant owned by the service (no ghost create, no false revoke audits);
templateId normalized case-insensitively at the service boundary.
- Web: SectionTenantTemplates management UI (create/rename/delete, add/remove
tenants, assign/unassign), triple-registered + api.ts block.
Backend 2538 tests green; web tsc clean + vitest 488 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Oliver Kieselbach