Remediation rows showed "8m" for scripts that ran seconds: durationSeconds was
measured HS-SCRIPT-START -> HS-NEW-RESULT, and IME writes that line only after
its batched service report. The early-signal (HS-COMPLIANCE) completion carried
no duration at all because it never read the cycle-start map.
Agent:
- HS-COMPLIANCE early-signal completions now PEEK _healthScriptStartTimes and
emit the actual run time (durationBasis=script_runtime); HS-NEW-RESULT keeps
consuming the entry and stamps its phases cycle_including_reporting_latency.
- durationBasis now comes from the emitting handler (ScriptExecutionState.
DurationBasis) instead of pauschal per scriptType.
- Platform scripts emit the live script_started signal too (gated on pending-
slot creation so the duplicate agentexecutor+ime start line stays one signal).
Web:
- Reducer normalizes durationBasis: runtime -> durationSeconds, cycle ->
reportedAfterSeconds (legacy events without basis infer from scriptType, so
old remediation cycle values are never displayed as run time again).
- Dedupe merges both timings across the early/late emissions instead of
dropping the runtime when the more complete HS-NEW-RESULT entry wins.
- Card run time = max of phase runtimes; "Reported after Xm" shown in the
expanded Details strip (per UX decision), never on the compact row.
Inert for new events until agent redeploy; web deploy alone already fixes the
misleading display of stored data.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
An ESP subcategory failure can be retracted inside the 30s settle window
(user clicks "Try again" / IME re-sync flips failed -> inProgress). The
expiry handler fired the latched EspFailureDetectedEventArgs
unconditionally, terminating the agent on a failure Windows no longer
reported and losing the retry outcome (session c071e92b: recovery at
+12s, terminal esp_terminal_failure at +30s anyway).
- OnProvisioningFailureSettleExpired re-checks current registry-derived
state under _stateLock: recovered = categorySucceeded != false AND no
subcategory currently "failed". On recovery: emit new
esp_failure_settle_recovered event, suppress EspFailureDetected, clear
the fire-once gate (+resolved set) so a re-failure arms a FRESH settle
window and is then terminal.
- esp_failure_settle_started now names tracked apps not completed
(trackedAppsNotCompleted/-Count) on Apps-subcategory failures via a new
packageStatesProbe (ImeLogHost.AllKnownPackageStates through
DefaultComponentFactory -> EspAndHelloHost -> EspAndHelloTracker).
Covers the Store/WinGet blind spot (Company Portal pending with zero
IME events) that both app_install_starved (IME-phase + intent gates)
and esp_apps_failure_correlation (IsActive device apps only) miss.
- New event type registered in Constants.EventTypes + MCP resource
catalog (drift test green); 5 new settle-window tests, suite 1711/1711.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
netstandard2.0's string.IsNullOrEmpty lacks [NotNullWhen(false)], so the
compiler cannot prove the else branch is non-null. Null-forgiving assert.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Session 2bc884b6: ESP failed AccountSetup/Apps with 0x80073cf9 while all 8
tracked Win32/WinGet apps succeeded — the failing app was a MSIX/Store app
that installs outside the IME Win32 pipeline and is invisible to
ImeLogTracker. The session ended as a blackbox failure.
When an Apps-subcategory failure arms the 30s settle window, the tracker now
queues a one-shot scan of Microsoft-Windows-AppXDeploymentServer/Operational
(Critical+Error, session window, 200-record/10s cap), scores candidate
packages (exact-HRESULT match > AppX-family error > time proximity, recency
bonus), and emits esp_appx_failure_analysis with flat RuleEngine fields plus
a capped candidates[] detail list — before enrollment_failed lands in the
timeline. Negative results are emitted too (no_appx_candidates points back
at Win32). Fire-once per session; scan runs lock-free on the threadpool;
settle expiry never waits; failures degrade to collector_degraded.
- New: AppxDeploymentFailureScanner (one-shot reader, injectable seam),
AppxFailureAnalyzer (pure scoring/verdicts, incl. negative-decimal
HRESULT + PackageFullName extraction from message/properties)
- ProvisioningStatusTracker: optional scanner/dispatcher ctor seams,
MaybeStartAppxFailureScan at settle-arm time
- Rule ANALYZE-APP-014 (MSIX/Store app failure during ESP) keyed on
verdict=appx_candidate_identified; dist regenerated
- MCP resource-catalog + web HRESULT badge + gather-rules eventTypes
- Tests: 16 analyzer + 7 trigger; settle-window fixture now injects a
fake scanner + sync dispatcher (no real event log in tests)
Inert until agent redeploy (rule needs backend redeploy). Post-deploy
checks in tasks/todo.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The network_bandwidth_estimate event now carries the CONFIGURED Delivery
Optimization bandwidth caps, so analyses can distinguish line-limited
(no caps -> the p90 is a real capacity lower bound) from policy-limited
(caps present -> the measured rate reflects the throttle, the line is
likely faster).
Verified against MS docs (Policy CSP + DO reference): GPO and Intune/CSP
do NOT share a registry store - both are read from the Registry64 view:
- GPO (ADMX): HKLM\SOFTWARE\Policies\Microsoft\Windows\
DeliveryOptimization, DO-prefixed value names
- Intune/Policy CSP: HKLM\SOFTWARE\Microsoft\PolicyManager\current\
device\DeliveryOptimization, same value names
Semantics: absolute caps in KB/s, percentage caps 0-100; a value of 0
means dynamic/no limit (OS default) and is treated as not configured.
DODownloadMode 0 is a real setting (HTTP only) and is reported whenever
present, but is not a throttle. Policies are read FRESH at each emission
because Intune policies land during the enrollment.
Event fields (omitted when absent): doThrottleConfigured,
doThrottleSources (gpo|mdm|gpo+mdm), doPolicy{Gpo,Mdm}{MaxForegroundKBps,
MaxBackgroundKBps,PctMaxForeground,PctMaxBackground,DownloadMode}; the
message gains a "[DO throttled by policy]" marker. Not covered (v1):
DOSetHoursToLimit* business-hours string policies.
+6 DoThrottlePolicyReaderTests (two-store split, 0-semantics,
string-DWORD tolerance, fail-soft); agent suite 1674/1674 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The estimator accumulator previously lived in process memory only - a
mid-DeviceSetup reboot (e.g. a Win32 app forcing a restart) silently
discarded every rate sample collected so far. One enrollment commonly
spans several reboots.
- BandwidthStatePersistence (bandwidth-state.json in the state dir):
mirrors the OfficeInstallStatePersistence pattern - atomic write via
temp+rename, fail-soft load (missing/corrupt/foreign-session -> fresh
start, never throws). SessionId-validated on load.
- BandwidthEstimator.ExportState/ImportState: sanitizes NaN/Infinity/
negative samples, caps list sizes; baselines deliberately not
persisted (a reboot gap always exceeds the per-file delta window).
- Save points are event-driven, never per-poll: the dormant transition
(downloads just finished - exactly when app-forced reboots happen),
after the interim emission, and at collector stop.
- The InterimEmitted guard rides in the same file: device_setup_end
now fires once per SESSION, not once per process. Persisted AFTER
the emission (StartupEventGate MarkEmitted ordering) so a crash
between emit and save re-emits instead of suppressing forever.
- Wiring: DefaultComponentFactory passes the state directory through
DeliveryOptimizationHost; the collector param is optional (null =
in-memory only).
+13 tests (estimator roundtrip/merge/sanitizing, persistence roundtrip/
session-mismatch/corruption, collector-level resume incl. guard
survival); agent suite 1668/1668 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New one-shot network_bandwidth_estimate event derived passively from the
Delivery Optimization byte counters the DeliveryOptimizationCollector
already polls every 3s - zero synthetic traffic, zero extra load, no
config change.
- BandwidthEstimator: per-FileId delta windows (a DO job that drops off
the list and returns is averaged over its own absence window), p90/max
reduction, WAN (HTTP + internet peers) vs LAN (LAN/group/link-local
peers + Connected Cache) split so peer-fed downloads cannot inflate
the WAN figure; bucket (<10/10-50/50-100/100-250/250+) + confidence.
- Emitted at most twice per session (snapshotTrigger): interim snapshot
on first AccountSetup sighting (device_setup_end - survives sessions
that starve in the account phase) and authoritative final at collector
stop (collector_stop, lands before diagnostics_collecting).
- All DO jobs feed it (IME apps, Office CDN, WU/Store/Defender).
- Registered in Constants.EventTypes, MCP resource-catalog
(network_events) and web gather-rules autocomplete.
- Drive-by: add missing agent_emergency_break to the MCP catalog
(pre-existing drift, event-types-drift test was red).
13 new estimator unit tests; agent suite 1655/1655, MCP drift 6/6,
web tsc + eventTypes vitest green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Phase 1 - named notification channels (1..N per tenant):
- NotificationChannel model (provider/url/headers/enabled + per-event
opt-ins incl. single NotifyOnSlaEvents); stored as JSON column
NotificationChannelsJson, cap 10, fail-soft ParseList
- No data migration: GetNotificationChannels() synthesizes one channel
from the legacy single-webhook fields under stable id "legacy";
web materializes on first save and clears legacy fields
- All dispatch sites (ingest success/failure/white-glove/ESP, SLA breach/
resolved/consecutive, enrollment-start, hardware-rejected, test) route
via SendToChannelsAsync with per-channel toggle filtering
- Per-channel secret redaction (structure stays visible to readers) with
restore-by-id; save-time validation (dup ids, provider, https, headers)
- Settings UI: channel card list with add/remove, event pills, per-channel
test button ({channelId} body, legacy-compatible without body)
Phase 2 - analyze-rule channel notifications:
- NotifyDefault (definition, false for all built-ins, test-guarded) +
Notify/NotifyChannelIds tenant override in RuleStates (two-tier mirror
of MarkSessionAsFailed); custom rules fold into the tenant-owned row
- Dispatch in AnalyzeOnEnrollmentEndHandler only: outcome.Results is
new-findings-only (engine dedup) so no re-notification; manual
"Analyze Now" reanalyze never dispatches
- BuildRuleFiredAlert (eventType analyze_rule_fired): device, serial,
rule, severity, category, confidence, explanation + session deep link
- Rule card: notify toggle + channel target pills, bell header badge
2858/2858 Functions tests (19 new), web tsc + vitest clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each model row now deep-links to /dashboard?status=Failed&search=<model>,
carrying the selected tenant for scoped global admins. The dashboard reads
?search= and ?status= from the URL (status validated against the known
buckets) and seeds the filter state via useDashboardFilters.
The search term stays fully client-side (never sent to the backend), the
status param is whitelist-validated, and the tenant param remains bounded
by the existing delegated-scope + backend enforcement, so no new trust
boundary is introduced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 2026-07-08 timeout reclassification (Incomplete = terminal, non-failure)
migrated SessionStats, FleetHealthStats and SessionStatusBuckets, but two
reporting surfaces were overlooked: the live UsageMetricsService.SessionMetrics
(both platform + tenant paths, feeding get_usage_metrics and the usage
dashboards) and the persisted daily UsageMetricsSnapshot. Terminal Incomplete
sessions were counted in Total but in no breakdown bucket, so
Succeeded+Failed+InProgress no longer reconciled to Total and operators lost
sight of those devices.
- SessionMetrics + UsageMetricsSnapshot gain an Incomplete / SessionsIncomplete
count, mirroring the migrated peers (Incomplete only, not AwaitingUser).
- Snapshot column stored + mapped in TableStorageService.Metrics; legacy rows
deserialize to 0 (no seed bump needed).
- Success rate unchanged everywhere: denominator was already Succeeded+Failed,
correctly excluding Incomplete/AwaitingUser. SLA paths verified already-correct
(BuildSnapshot + breach evaluator receive terminal-filtered sessions).
- Web: Incomplete card added to both usage views (slate, matches the badge).
- MCP: query_raw_sessions status enum learns AwaitingUser/Incomplete so callers
can filter by the new states.
- Test: UsageMetricsServiceWindowTests pins the Incomplete bucket + 50% rate on
the crcins 1/1/8 shape.
Backend build clean, full Functions suite 2825 pass, web + MCP tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses four review rounds on the enrollment-status reclassification
(timeout != failure; new AwaitingUser/Incomplete states).
Backend correctness:
- UpdateSessionStatusAsync ETag force-fallback now mirrors the normal path:
uses IsTerminalTransitionAllowed + admin-marked guard, stamps CompletedAt
for Incomplete, and persists FailureReason/FailureSnapshotJson for
Incomplete/AwaitingUser (not just Failed) so a contended write cannot
regress a terminal or drop the classification reason/snapshot.
- StoreSessionAsync CAS loop re-reads the terminal tuple in the fresh-read
window and preserves a just-landed terminal (Succeeded/Failed/Incomplete)
verbatim, so re-registration can no longer revert it via the Replace.
- SessionGraceHours + AbsoluteMaxSessionHours now roundtrip through the
tenant-config Store/Map pair (overrides were silently lost).
- ResolveGraceHours clamps the assumed agent cap to >= the real runtime
default (48h) until the agent honors the override, so a low override can
never terminalize a still-running session early.
- Incomplete now funnels through the terminal counter-reconcile seam, and
the maintenance sweep re-reconciles after the synthetic session_timeout
event (also fixes the pre-existing off-by-one on the Failed path).
Web:
- SessionInfoCard delegates its status pill to the shared SessionStatusBadge
(removes a divergent local map; picks up AwaitingUser/Incomplete).
- SessionTable search color map + status filter chips learn the new states.
- useSessionEvents uses isTerminalStatus() instead of a hard-coded
Succeeded/Failed check (no needless refetch on Incomplete).
- DataManagementSection copy no longer says "Failed - Timed Out".
Docs/comments:
- Plan doc + TenantConfiguration comments corrected (+3h/51h buffer, and the
timeout now reclassifies to AwaitingUser/Incomplete, not "Failed").
Tests: +ResolveGraceHours clamp case, +grace/abs-cap config roundtrip +
legacy-row defaults. Full backend suite green (2712), web tsc green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Windows Update activity during Autopilot OOBE/ESP is a blind spot no other tool
(Intune console included) surfaces well — a cumulative update installing mid-OOBE
can break enrollment (r/Intune KB5095189). This makes it visible with secured
evidence.
Agent (net48):
- WindowsUpdateTracker + WindowsUpdateWatcherHost: watch
Microsoft-Windows-WindowsUpdateClient/Operational (EventIDs 19/20/43/44 =
success/failure/install-started/download-started), startup backfill (OOBE updates
run before the agent starts), RecordId watermark dedup (restart boundary vs
intra-run seen-set so a high-id live event can't hide older backfill events),
HRESULT->symbol decode. Emits windows_update_succeeded/_failed/_started with the
real event Timestamp. Default-on config knobs; uncertain EventIDs configurable
without redeploy.
- RegistryCollector: emitOnlyIfExists (suppress positive-assertion events when the
key is absent) + Registry64 base view by default (bitness-independent, matches
TenantIdResolver) with optional registryView override.
Rules:
- ANALYZE-DEV-004 (high: WU failed during enrollment, high-insight not KO) +
ANALYZE-DEV-005 (info: WU installed). reboot_pending corroboration matches
event_data exists==true.
- GATHER-DEVICE-005 -> windows_update_reboot_pending (emitOnlyIfExists),
GATHER-DEVICE-004 -> windows_update_history (Get-HotFix, opt-in).
- 5 EventTypes synced across Constants + MCP resource-catalog + web eventTypes.
Self-maintaining rule lifecycle (analyze + gather parity):
- Gather rules gain the sunset/orphan-GC that only analyze had
(ProcessSunsetGatherRuleAsync + runtime filter + seed/reseed/github paths).
- RuleProvenance field (embedded/github, null=embedded back-compat): the embedded
catalog only sunsets its own rules; GitHub-reseeded rules ahead of the deployed
binary (new ids AND content updates to existing ids) are exempt and reclaimed to
embedded once the binary catches up. No orphans, no operator knowledge required.
Verified: full solution builds; backend suite 2770 pass, agent suite 1638 pass;
rules AJV-valid + combined.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the agent's silent 48h absolute session-age break visible: on break, before
cleanup/exit, fire a SessionAgeEmergencyBreak AgentErrorReport over the resilient
emergency channel; the backend (Part B) materializes it as an agent_emergency_break
timeline event and the timeout classifier terminalizes the session instead of
waiting out grace.
Decoupled so the guard stays pure and its unit tests are untouched:
- CheckSessionAgeEmergencyBreak gains optional Action onBreakFired = null, invoked
once when the break is confirmed, before marker/cleanup, fully swallowed.
- New EmergencyBreakReporter.TrySend builds a throwaway auth bundle
(BackendClientFactory) and sends, bounded to a 5s wait. AgentBootstrap wires it;
GetAgentVersion made internal.
NOTE: agent targets net48 and was NOT compiled in this environment (only the .NET 8
SDK is available) — to be built/verified locally. The Shared AgentErrorType value is
verified (compiled in the backend build).
- SessionStatusBuckets: explicit AwaitingUser + Incomplete buckets (out of Other),
still reconciling to Total.
- Metrics summary: expose awaitingUser + incomplete; failure rate is now
Failed/(Succeeded+Failed) — Incomplete and non-terminal states are out of the
denominator (crcins drops from failed/total to the real terminal rate).
- SessionStats.IncompleteLastNDays; dashboard SuccessRatePct was already terminal-only.
- Daily aggregation already used Succeeded+Failed denominator — unchanged.
- Tests: SessionStatusBucketsTests. Suite: 2705 pass. Fleet Health Incomplete surfacing
deferred to PR5 (web) with its documented over-all-sessions semantics.
A device that reaches the desktop is enrolled regardless of an earlier sweep
timeout guess, so a genuine completion must be able to heal a prior verdict.
- Replace the terminal idempotency guard with pure IsTerminalTransitionAllowed:
Succeeded UPGRADES any non-Succeeded state (Failed/Incomplete/AwaitingUser →
Succeeded); Failed/Incomplete and AwaitingUser never overwrite an existing
terminal. Ingest already calls UpdateSessionStatusAsync(Succeeded) on a late
enrollment_complete without a pre-terminal skip, so the relaxed guard wires the
reconcile end-to-end.
- Clear stale FailureReason + FailureSnapshotJson on Succeeded (reconcile hygiene).
- Preserve admin-marked terminals: a late completion never auto-overrides a
hand-marked Failed/Incomplete.
- Tests: SessionStatusTransitionTests (full matrix, 20). Suite: 2705 pass.
ReportAgentErrorFunction now injects ISessionRepository and, on a
SessionAgeEmergencyBreak report, writes an agent_emergency_break EnrollmentEvent
into the session stream (Sequence one past the last event, idempotent, best-effort
— never turns the always-200 emergency channel into a retry loop). This is what
makes the agent's otherwise-silent 48h break visible in the timeline and lets the
timeout classifier see it. Pure BuildAgentEmergencyBreakEvent helper (Warning,
non-terminal) + tests. Functions.Tests: 2685 pass.
Groundwork for making the agent's silent 48h emergency break transparent
(docs/design/enrollment-status-reclassification.md):
- Constants.EventTypes.AgentEmergencyBreak + AgentErrorType.SessionAgeEmergencyBreak.
- EnrollmentTimeoutClassifier: ExtractRollup.HasAgentEmergencyBreak; classify a
session carrying the break as Incomplete NOW (agent is gone) instead of holding it
as AwaitingUser through the grace — a real completion still wins. Tests added (22 total).
Next: agent best-effort emit at the break + backend materializes the timeline event.
A completion can only arrive <= the agent's absolute cap (it self-destructs and
sends nothing after), so the grace buffer needn't wait hours for stragglers — it
only has to span one 2h maintenance-sweep cycle + minor clock skew, and reconcile
heals any premature Incomplete. Buffer 12→3h, default grace 60→51h.
The 5h→terminal grace was a magic 72h. Anchor it to the agent instead: the 6h
per-run brake emits enrollment_failed (→ Failed at ~6h), but the 48h absolute
emergency break (AbsoluteMaxSessionHours) is SILENT to the backend — it writes a
local marker, cleans up and exits with no terminal event. So a session only stays
non-terminal past 48h when the agent hit that silent cap and is provably gone.
Grace must therefore be >= the agent absolute cap (never mark Incomplete while the
agent could still legitimately enroll) and only slightly beyond it:
effectiveGrace = max(AbsoluteMaxSessionHours + 12h buffer, SessionGraceHours) = 60h default.
- EnrollmentTimeoutClassifier.ResolveGraceHours (pure, tested): floors grace at
agentCap + buffer; a SessionGraceHours override can only raise it.
- TenantConfiguration.SessionGraceHours default 0 (= auto-derive); mirror
AbsoluteMaxSessionHours so the floor follows a tenant override of the agent cap.
- Sweep uses ResolveGraceHours instead of the hard-coded 72h.
Follow-up noted: wire AbsoluteMaxSessionHours down into AgentConfigResponse so the
agent and backend share one value. Functions.Tests: 2678 pass.
Wire EnrollmentTimeoutClassifier into MarkStalledSessionsAsTimedOutAsync so the
5h inactivity sweep stops blindly failing silent-but-provisioned sessions
(docs/design/enrollment-status-reclassification.md). Forward-only.
- TenantConfiguration.SessionGraceHours (default 72h) + default config.
- Sweep Stage 2: classify from the ESP subcategory rollup instead of hard Failed.
Fast-path skips within-grace AwaitingUser sessions with NO event read (grace is
time-based; late completions are caught by ingest). Per-outcome counters; audit
+ ops event only when the pass actually terminalized/transitioned something.
- session_timeout event + auto-analyze kept ONLY for Failed/Incomplete (parity with
the agent max-lifetime path); AwaitingUser is non-terminal, Succeeded is a reconcile.
- GetStalledSessionsAsync includes AwaitingUser so it graduates to Incomplete once
grace elapses; agent-silent (Stage 1) stays InProgress-only.
- UpdateSessionStatusAsync: Incomplete is terminal (idempotency); AwaitingUser can't
regress a terminal; CompletedAt stamped for Incomplete (no skewing duration); reason
+ snapshot persisted for Incomplete/AwaitingUser.
Full Functions.Tests suite: 2670 pass.
Foundation for reclassifying 5h-timeout sessions (docs/design/
enrollment-status-reclassification.md). No behaviour change to the sweep yet —
this only adds the pieces PR2 wires in.
- SessionStatus: append AwaitingUser (non-terminal) + Incomplete (terminal,
non-failure). Members appended so persisted ordinals never shift.
- EnrollmentTimeoutClassifier: pure ExtractRollup (parses the esp_provisioning_status
"N of M subcategories completed" message + the 30s all-succeeded fallback line —
categorySucceeded is ignored, Windows never sets it) and ClassifyTimedOutSession
implementing the decision table (explicit failure→Failed; AccountSetup all-succeeded
/ enrollment_complete→Succeeded; DeviceSetup done + within grace→AwaitingUser, else
Incomplete; silent pre-DeviceSetup→Incomplete). Never returns Failed without an
explicit failure signal.
- FailureSnapshotBuilder: embed the same rollup (schema v2) so the snapshot records the
evidence behind the verdict.
- Tests: EnrollmentTimeoutClassifierTests (12). Full Functions.Tests suite 2670 pass.
Design note + implementation plan for splitting the 5h maintenance-sweep
"Failed" bucket into three states. Evidence (crcins.com, 90d): 90.7% of
"failures" are 5h inactivity timeouts with no explicit failure event; 99.1%
of those had DeviceSetup fully provisioned (device enrolled) and simply had
the agent go silent during the user/Account-Setup phase. True failure rate
~4-5%, not 57.2%.
Proposal (backend + stats only, no agent change / no heartbeat):
- add AwaitingUser (non-terminal) + Incomplete (terminal, non-failure) states
- classify timed-out sessions from the ESP subcategory rollup the agent
already emits (gate on subcategories, not the unreliable categorySucceeded)
- passive grace window owned by the backend; reconcile late completions to
Succeeded; exclude Incomplete from the failure-rate denominator
Two security-critical paths had zero unit coverage because the logic was inlined
behind infrastructure (a real X509Store / a live OIDC ConfigurationManager +
HttpContext). Extract the decision logic into pure, testable seams — no
behavioural change — and pin it.
CertificateHelper (Agent):
- Extract SelectMdmCertificate(IEnumerable<X509Certificate2>): the issuer
discrimination (CN=Microsoft Intune MDM Device CA only, never the MMP-C CA),
ClientAuth-EKU filter and NotAfter tiebreaker. FindMdmCertificate now calls it.
- 13 tests incl. the dual-stack regression (issuer filter beats the NotAfter
tiebreaker so a longer-lived MMP-C cert can't win over a genuine MDM cert).
AuthenticationMiddleware (Backend):
- Extract BuildTokenValidationParameters(signingKeys, clientId), IsValidTenantId(tid)
and BuildTenantAuthority(issuer, tid). Invoke() now composes them.
- 16 tests validating locally-minted tokens against a test key: alg:none and HS256
rejected (algorithm-confusion defence), expired / wrong-audience / foreign-issuer
/ unknown-signing-key rejected, valid RS256 + api:// audience + v1 issuer accepted,
plus the tid GUID gate and v1/v2 authority mapping.
Backend 2746/2746 and Agent 1615/1615 green; full solution builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Six existing test classes had clusters of near-identical [Fact] methods that
differed only by input/expected literals. Convert each cluster to a single
[Theory], preserving every case (and its explanatory comment) as an InlineData/
MemberData row. Pure refactor — no behavioural change, no coverage lost.
- DeviceInfoProviderVmDetectionTests: 18 Facts -> 1 Theory (IsVirtualMachine)
- RateLimitResolverTests: 12 Facts -> 2 Theories (device + user resolve)
- SecurityValidatorTests: IsValidGuid cluster (10 Facts) -> 1 Theory
(WithNewGuid stays a Fact — dynamic input; OData-injection Theory untouched)
- CpeUriNormalizerTests: 8 Facts -> 1 Theory (Normalize)
- SignalRGroupHelperTests: ExtractTenantId cluster (6 Facts) -> 1 Theory
(MemberData; group names interpolate the const GUIDs. Access-gate Facts kept)
- VersionComparerTests: IsVersionAffected cluster (6 Facts) -> 1 Theory
Backend 2730/2730 and Agent 1607/1607 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The malformed-input theory asserted a BOM-prefixed CMTrace line yields false,
assuming a leading U+FEFF breaks the StartsWith gate. It doesn't: culture-
sensitive string.StartsWith treats U+FEFF as ignorable so the gate passes, and
the unanchored regex matches the "<![LOG[..." body after the BOM — the line
parses (which is desirable; Windows writes a BOM on freshly-opened log lines).
Move the BOM case out of the malformed theory into a dedicated fact that pins
the BOM-tolerant behaviour. Product code unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A delegated admin who is also a member/operator of their own home tenant
lost MCP access to it: any non-empty delegated scope flipped the whole
session to delegated cross-tenant routing, and enforceDelegatedTenant
rejected every tenant outside the managed set — which never contains the
home tenant (GetScopeAsync uses the home tid only as the Enterprise
entitlement gate, never adds it to the scope).
Fix is MCP-only (no backend change; the backend already authorizes the
home tenant via MemberRead and bounds the /api/global/* drill away from
it, so the home tenant must route to the JWT-bound member path):
- access-guard.ts: thread homeTenantId (JWT tid, lowercased) into the
caller context.
- client.ts: getHomeTenantId(); enforceDelegatedTenant accepts
managed union {home}; enforceDelegatedTenantForPage resolves a
non-global tenant-path nextLink (no embedded tenantId) to home so
page-2 re-selects the member path; pickGlobalOrTenantPath gains an
effectiveTenantId arg and routes delegated+home to the tenant path.
- 16 pattern-A tool call sites pass the effective tenantId (not
devices/blocked, which is platform-only).
- shared.ts / index.ts: descriptions + instructions surface the home
tenant as an addressable target.
The home tenant is accepted optimistically at the MCP boundary; the
backend MemberRead stays authoritative (a non-member is denied on the
member path, so no leak). Continuation defense-in-depth preserved.
Verify: tsc clean; vitest 352 passed (+7 home-tenant cases).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codex follow-up: the previous middleware comment claimed a deterministic limit+1 block, but McpQuotaService caches the per-user decision for 60s - an allowed decision keeps admitting inside that window even after the async increments cross the limit. The soft boundary IS the intended design (bounded overshoot ~TTL x request rate per instance, same posture as the sliding-window rate limiter; one counter read per user per minute instead of per request).
- Middleware doc rewritten: check-then-increment fixes the race/inflation, but the boundary is explicitly documented as SOFT with bounded overshoot; warns against re-documenting as exact without reworking the decision cache.
- Two regression tests pin the contract: cached allowed decision keeps admitting within the TTL even after counters cross the limit (no storage re-read), and after cache expiry (simulated via eviction) the over-limit counters block.
Backend 2658/2658 green.
- High: TenantConfigurationService.SaveConfigurationAsync now THROWS when the repository reports a failed persist (it swallows storage exceptions and returns false) - plan/trial/config writes can no longer be audited and returned as 200 without being persisted. Cache is invalidated on every attempted save (finally), covering in-place mutations of the cached instance; the now-redundant invalidate-on-failure wrapper in PlanManagementFunction is removed. AuthFunction login side-effect saves (first-login domain write, auto-re-enable) are wrapped best-effort so a storage blip cannot 500 auth/me - they self-gate and retry on next login.
- Medium: MCP usage increment moved from AuthenticationMiddleware (fire-and-forget BEFORE the quota decision) into McpQuotaEnforcementMiddleware: strictly check-then-increment and only for requests actually served (GA tracked but quota-exempt; denied 403/429 requests no longer inflate counters; the exact-limit request is now deterministic - 1..limit pass, limit+1 blocks).
- Low: fixed both new nullable warnings (PlanTierDefinitionParser guarded json, McpQuotaEnforcementMiddleware principal null-flow).
Tests: +3 save-semantics regression tests (throw on false, cache invalidation on failed/successful save). Backend 2656/2656 green, build clean.
Fail-closed entitlement layer (FeatureEntitlementCatalog + TenantEntitlementService): edition resolves at read time (PlanTier == 'enterprise' OR active trial; legacy free/pro => Community, no migration).
- Plan/trial endpoints: PATCH config/{tid}/plan (GA: tier + grant/extend/end trial, audited, cache-safe) and POST config/{tid}/trial (tenant-admin self-service 30d, once per tenant, 409 after). Generic PUT preserves plan/trial fields for all callers. Fixes SetPlanTier writing past the config cache without audit/UpdatedBy.
- Rate limits: Enterprise floor 150/min for BOTH user (portal/JWT) and device (agent/cert) paths - floor raises the global default, an explicit per-tenant override still wins; MSP users follow their home tenant.
- Retention cap: Community 90d / Enterprise 365d, clamped in the retention fanout (warn log) + non-GA write validation 7..cap (only on changed values; 0=infinite stays GA-only).
- Delegated admin (MSP): Enterprise gate on the delegated admin's HOME tenant (JWT tid, fail-closed on unknown) - managed target tenants may be any edition. Grant-time target check removed; inert grants are suppressed at resolve time (single choke point in GetScopeAsync, applied after the UPN-keyed cache).
- MCP quota: McpQuotaService + McpQuotaEnforcementMiddleware (X-Client-Source: mcp only, GA exempt) enforce daily/monthly budgets - per-user plan override > tenant edition plan; limits from SectionUsagePlans with catalog fallbacks (community 100/3000, enterprise 1000/20000); 429 with X-MCP-Quota-* headers + Retry-After; fail-open on counter reads, fail-closed on plan resolution. /api/metrics/mcp-usage/me reports effective plan + counters.
- Trial visibility: TrialExpirySweepFunction (daily 03:30 UTC, informational) emits TenantTrialExpired (Warning, 24h look-back) + TenantTrialExpiring (Info, <=3d heads-up); dual-registered in web OPS_EVENT_TYPES.
- Web: EditionBadge + one-time trial CTA in settings header, Plan & Trial card in the GA tenant edit modal (own PATCH save), retention cap hints from entitlements, pure lib/edition.ts (vitest), feature-flags surface edition/trial/entitlements; MCP TENANT_SAFE_FIELDS + trial fields.
Verified: solution build clean; backend 2653/2653 tests; web tsc + 516/516 vitest; MCP server tsc + tests green.
Replace the denormalized per-tenant RateLimitRequestsPerMinute mirror (and its
SaveConfigurationAsync sync job) with compute-at-read resolution: effective limit =
per-tenant override ?? global default. Fixes the clobbering bug where a per-tenant
edit to the base field was overwritten on the next global config save.
- Model: drop TenantConfiguration.RateLimitRequestsPerMinute; add nullable
CustomUserRateLimitRequestsPerMinute (device override CustomRateLimit... kept).
- RateLimitResolver centralizes precedence (device + user); both paths call it.
- SecurityValidator reads the global default (AdminConfigurationService threaded
through ValidateSecurityAsync + all device-auth callers); 5-min cached.
- UserRateLimitMiddleware: per-tenant user override for tenant-scoped callers only
(skipped for HasGlobalScope = GlobalAdmin|GlobalReader); GA -> GA budget, others ->
user default. Uses GetConfigurationIfExistsAsync so an unregistered-tenant JWT can
no longer auto-create a config row via read endpoints.
- Validation: reject rate limits < 1 at the write functions; RateLimitService clamps
< 1 to 1 as a safety net (avoids Min()-on-empty / throttle-everything).
- Web: Global Settings exposes User + GlobalAdmin limits; tenant modal writes the
override fields (blank = inherit); report + usage-plans updated.
Legacy RateLimitRequestsPerMinute Table column is ignored on read and dropped on the
next Replace save (self-cleaning; no migration).
Backend 2577 xUnit green, tsc clean, web 506 vitest green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The detailed per-component chapters (Agent internals through Build & Development)
are retired — product documentation lives at docs.autopilotmonitor.com now, and
the deep internals went stale faster than they earned their keep. What remains
is corrected against the actual tree: Shared/DecisionCore and SummaryDialog.Tests
added, src/McpServer and rules/guardrails.json were missing entirely, scripts/
gained Backup + CustomerSetup, Functions references DecisionCore too, and the
overview diagram no longer claims the web app serves docs.
docs/images stays — all five files are embedded in the repo README.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- msalConfig: NEXT_PUBLIC_ENTRA_API_CLIENT_ID splits the API scope from
the SPA client id, so local dev signs in with the dev app registration
while requesting the prod API audience. Unset in prod (fallback keeps
the single-app behavior).
- next.config: dev-only rewrites() proxy forwards /api/* server-side to
DEV_API_PROXY_TARGET, keeping the browser same-origin — no localhost
CORS entry on the prod Function App.
- CSP: script-src gains unsafe-eval only when NODE_ENV=development
(next dev HMR/react-refresh needs eval); production CSP unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace the favicon (app/icon.svg) with the new green glyph and swap the
portal navbar logo from the blue gradient box to the bare green icon with
the wordmark in sidebar gray (text-gray-600 / dark:text-gray-400). Both
portal navbar variants updated; the public landing navbar keeps the old
gradient branding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>