Compare commits

...

19 Commits

Author SHA1 Message Date
Oliver Kieselbach a97888ebd5 fix(agent): neutral aggregated 2800 observations; reboot claim gated on observed reboot
Rework after the first real enrollment (session b2e890c1) exposed three
defects in a654c730:

- Shell-Core "RebootCoalescing" match was a false positive by construction:
  the token appears in the ROUTINE SubcategoryProcessing_Started bootstrap
  marker ("Starting subcategory DeviceSetup.RebootCoalescing...") on EVERY
  enrollment. esp_reboot_coalescing removed entirely (branch, constant,
  catalog, tests).
- 2800 reboot-required flags fire at every policy sync; only DeviceSetup-time
  flags cause the coalesced reboot. The session's flags fired during
  AccountSetup - no reboot - yet our Warning events claimed "device will
  restart". Raw event is now a neutral Info observation; the reboot CLAIM
  moved to ANALYZE-ESP-005 with a REQUIRED system_reboot_detected >= 1
  condition (no observed reboot -> advisory silent).
- One event per 2800 record was spam: records are now buffered and debounced
  (10 s) into ONE aggregated event per burst (rebootUris[]/uriCount/
  firstRebootUri/recordCount/unparsedCount). Watermark is persisted at FLUSH,
  not claim - a pre-emit reboot kill loses nothing, the post-restart backfill
  re-emits the burst at its historical time.

Verified 2800 description shape recorded in code, tests, and concept doc:
"The following URI has triggered a reboot: (./Device/...)."; extraction
covered by a real-text regression test. Lesson added to tasks/lessons.md
(observation-level events vs outcome-gated claims; substring tokens appear
in framework bookkeeping records).

Tests: Agent.V2.Core 1867/1867, Functions 3099/3099, MCP 400 passed,
solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:41:47 +02:00
Oliver Kieselbach a654c7309e feat(agent): attribute mid-ESP coalesced reboots to MDM policy URIs (second sign-in detection)
Device-assigned MDM policies whose CSP URIs match the OS reboot-required
catalog force a coalesced reboot at the end of ESP DeviceSetup - the user
sees an unexpected restart and a second sign-in before AccountSetup.
Invisible in the Intune console; admins simply lived with it. Now attributed
and actionable (PMPC research).

- MdmRebootPolicyTracker (+host): watches DeviceManagement-Enterprise-
  Diagnostics-Provider/Admin EventID 2800, emits one mdm_policy_reboot_required
  per reboot-forcing URI (Warning, ImmediateUpload; tolerant EventData-first
  URI extraction with regex fallback; fail-soft OMADM RebootRequired probe).
  Cross-restart RecordId watermark (mdm-reboot-watermark.json) - mandatory,
  the coalesced reboot restarts the agent and the backfill re-reads the log.
- ShellCoreTracker: RebootCoalescing branch on the already-watched 62407
  stream -> esp_reboot_coalescing corroboration event; deliberate deviation:
  emits on backfill too (live emit races the reboot kill).
- ANALYZE-ESP-005 advisory (warning): names the first URI via {{rebootUri}},
  remediation = reassign profiles device -> user groups; +20 confidence on
  coalescing corroboration, +10 on >=2 URIs.
- Config: Collectors.MdmRebootPolicyWatcherEnabled (default ON, internal
  steering only - agent-native collector, NOT an IAgentAnalyzer, no admin UI),
  MdmRebootPolicyBackfillLookbackMinutes=60; ConfigVersion 35.
- Event types deliberately distinct from system_reboot_detected so the
  backend RebootCount (exact-string match, 2 sites) stays uninflated.
- Docs: docs/agent/mdm-reboot-coalescing.md (+index/log). Exact 2800/62407
  message texts are unverified until E2E VM validation (tolerant matching).

Tests: Agent.V2.Core 1868/1868 (21 new), Functions 3099/3099, MCP 400 passed
(event-types drift), solution build 0 errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:05:11 +02:00
Oliver Kieselbach 6d1ff8d7f0 fix: code-review follow-ups (watcher dispose race, revoke-enforcement tests, scan guard)
- ConsoleBypassWatcher: re-check _disposed after watcher.Start() — a Dispose()
  racing the background arm could leave an armed WMI subscription no code path
  could ever stop (leaked until process exit).
- SignalRContext: collapse rapid accessRevoked pushes into one restart via an
  in-flight guard — a second start() on a non-Disconnected connection throws
  and burns the retry budget.
- config/all: negative-cache managed ids a rescue scan proved missing (5 min
  TTL) — one stale grant (offboarded tenant) no longer makes every delegated
  config/all call pay a full table scan.
- DelegatedAdmin/TenantGroup functions: extract testable revoke-enforcement
  cores (RevokeCoreAsync, SetEnabledCoreAsync, UnassignCoreAsync,
  DeleteGroupCoreAsync, ValidateGrantRequest) and cover the security flows:
  404-on-missing-row writes no audit row and fires no disconnect; real
  revoke/disable/unassign/group-delete always cuts live streams; grant
  validation is fail-closed (GUID tenantId, recognized roles only).
- dashboard/page.tsx: drop unused next/link import left by the banner removal.

Tests: backend 3099 passed, web 557 passed, agent ConsoleBypassWatcher suite
passed, tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 13:13:26 +02:00
Oliver Kieselbach 5c8b273f37 chore: bump skills submodule (system-Timestamp warning in raw-query skills)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 12:38:04 +02:00
Oliver Kieselbach da958b5398 fix(backend): business timestamps survive storage migrations (OccurredUtc + RowKey decode)
The 2026-07-18 storage migration reset every row's system Timestamp,
exposing that AuditLogs, OpsEvents and Events displayed, filtered and
retention-swept on it (custom "Timestamp" writes were silent no-ops —
reserved system property). All migrated rows read as 2026-07-18.

- New Helpers/BusinessTimestamp: RowKey decoders (audit !revtick_guid,
  ops bare revtick, event yyyyMMddHHmmssfff_seq), RowKey-range OData
  clause builders (tick-exact, with the mandatory RowKey lt '"' guard
  so no lower bound can ever match legacy bare-GUID audit rows), and
  dual DateTimeOffset/DateTime column reads.
- Writes in all three tables now persist OccurredUtc (custom column,
  copied verbatim by future migrations); no-op ["Timestamp"] removed.
- Read mappers resolve OccurredUtc -> RowKey decode -> system Timestamp;
  GetEarliestSessionEventTimestampAsync uses the same resolution so
  StartedAt/Duration reconciliation can no longer absorb the migration
  date on migrated sessions.
- Date-window filters and retention moved off the system Timestamp onto
  RowKey ranges; ops retention loses its full-table scan and the
  post-migration mass-delete cliff; audit retention runs a temporary
  second pass for legacy GUID rows (TODO: remove after 2027-03-01).
- New GlobalAdmin backfill endpoint POST /api/maintenance/
  backfill-occurred-utc (dry-run by default, batched, idempotent MERGE
  of only OccurredUtc, table=audit|ops) — Events rows are deliberately
  never mutated; their read path decodes the RowKey instead.
- Tests: decoders, clause boundary proofs incl. GUID-safety pins,
  mapper precedence, backfill row decisions (Functions.Tests 3078).
- Docs: docs/backend/business-timestamps.md (system Timestamp is never
  authoritative; migration checklist rule).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 11:29:53 +02:00
Oliver Kieselbach e17eb607ca docs: gather-rule phase scoping + emit-on-change concept doc (OKF)
New rules/ section in the knowledge bundle: semantics table, from-phase latch
behavior (never via Failed, sticky through it), on_change x emitOnlyIfExists
composition, restart re-emit, no-phase-signal caveat, old-agent degradation,
and the prerequisite toggle-wipe fix. Index + log updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:14 +02:00
Oliver Kieselbach 71000a5dd2 feat(web): gather-rule scope + emit-mode UI, phase dropdowns, edit tags wipe fix
Step 4: portal round-trip for the new fields.

- types.ts: GatherRule + activePhases/activeFromPhase/emitMode; NewRuleForm +
  scopeMode ("always" | "during" | "from") as UI-only view state; canonical
  GATHER_PHASES list (EnrollmentPhase names Start..Complete with display
  labels) + EMIT_MODES; new rules default to emitMode "on_change".
- withDerivedScopeMode: JSON-mode merges rule-shaped JSON (no scopeMode key)
  over form state — the helper derives scopeMode from the data in all four
  JSON paths (create apply/save, edit apply, card edit save) so pasted scope
  fields survive into the payload instead of being silently dropped.
- GatherRuleFormFields: "Active During" mode select + phase multi-select /
  from-phase dropdown (mode switch clears the other mode's fields), "Emit
  Mode" select with helper text; triggerPhase free-text replaced by the same
  canonical dropdown (placeholder used to suggest invalid tokens like
  "DeviceESP"; legacy stored values remain selectable).
- page.tsx: create/save payloads carry the three fields; startEditing maps
  them back (existing rules load as "always", the on_change default is for
  NEW rules only); handleSaveEdit now also sends tags — same wipe class as
  the toggle bug, the full-replace PUT dropped them on every edit.
- GatherRuleCard: trigger section shows the scope + emit mode, e.g.
  "Interval every 60 seconds - from Account Setup - emit on change".

npx tsc --noEmit silent; vitest 557/557.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:51:04 +02:00
Oliver Kieselbach a88db34af5 feat(agent): gather-rule phase scoping + on_change emit dedup in GatherRuleExecutor
Step 3: the executor honors the new GatherRule fields.

Phase scope (ActivePhases / ActiveFromPhase), applied to ALL trigger types:
- interval: timers keep running, an out-of-scope tick is a free early-return
  (no start/stop lifecycle races on phase transitions).
- startup: scoped rules not yet in scope at UpdateRules are deferred and run
  exactly once from OnPhaseChanged when their scope activates (same
  _startupRulesExecuted dedup; not part of the WaitForStartupRules latch).
- phase_change: evaluated against the NEW phase; scope gate runs BEFORE the
  (rule, phase) dedup add so an out-of-scope pass does not consume the slot.
- on_event: gated by current phase.
- From-phase latch: phase != Unknown/Failed && ordinal >= threshold; sticky for
  the session incl. through Failed; re-evaluated on config refresh so a rule
  delivered after its threshold passed latches immediately.
- Before the first phase signal, scoped rules are inactive.
- IgnorePhaseScope bypass for --run-gather-rules (no phase context; logged).

EmitMode on_change:
- Canonical hash (keys ordinal-sorted at every level, invariant formatting,
  SHA-256) of the raw collector result BEFORE ruleId/ruleTitle injection.
- First non-empty result always emits; unchanged results increment a per-rule
  suppression counter; the next emitted event carries suppressedPolls +
  suppressedSinceUtc. Empty results (emitOnlyIfExists miss) touch nothing, so
  the combo yields zero noise, one event on appearance, then only on change.
- State survives UpdateRules refreshes (agent restart => at most one re-emit).

Tests: GatherRuleExecutorPhaseScopeTests (scope matrix synchronous via internal
IsRuleInScope; trigger wiring async with positive waits),
GatherRuleExecutorEmitModeTests (state machine via internal ShouldEmitOnChange,
canonical-hash properties, ExecuteRule wiring). V2.Core.Tests 1847/1847.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:50 +02:00
Oliver Kieselbach 21dd35f787 feat(backend,shared): gather-rule phase scoping + emit-mode fields, custom-rule toggle wipe fix
Step 0-2 of the gather-rule anti-spam plan:
- GatherRuleService.UpdateRuleAsync: toggle-style partial PUTs ({enabled,
  isBuiltIn, isCommunity}) previously full-replaced the stored custom rule and
  wiped Title/Target/Trigger/Parameters. Partials now merge Enabled into the
  existing row; full edit payloads preserve the original CreatedAt.
- GatherRule: new opt-in fields ActivePhases (run only during listed phases),
  ActiveFromPhase (sticky latch from a phase onwards), EmitMode (always |
  on_change). Absent fields = legacy behavior; schema updated accordingly.
- TableStorageService.Rules: Store+Map for the three columns (absent column /
  stored-empty -> null); MapToGatherRule internal for roundtrip tests.
- ContentEquivalent covers the new fields, null-tolerant for ActivePhases so
  pre-existing rows never diff against seeds (no reseed churn).
- GatherRulesFunction.ValidateScopeAndEmitMode: canonical EnrollmentPhase name
  list Start..Complete (Unknown/Failed and numeric tokens rejected), mutual
  exclusion of the two scope fields, emitMode whitelist; 400 on violation.
- ConfigVersion 33 -> 34 (33 was taken by MigrateToApiBaseUrl).

Tests: GatherRuleUpdatePartialMergeTests (merge/replace/unknown-rule/built-in
path), GatherRuleScopeFieldsTests (Store<->Map roundtrip incl. legacy rows,
ContentEquivalent matrix, validation accept/reject). Functions.Tests 3029/3029.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:50:35 +02:00
Oliver Kieselbach 9933c039ed refactor(shared,backend): MigrateToApiBaseUrl nullable, drop null-forgiving operator
null = no migration is the documented contract and the agent already
assigns null when stripping the field from the on-disk cache; annotate
the property as string? so the backend assignment needs no ! suppression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 13:33:57 +02:00
Oliver Kieselbach ef67dec263 fix(backend): system-health URLs post EU-migration + GA endpoint visibility
- Agent Binaries check + ops watcher now probe the canonical download alias
  (download.autopilotmonitor.com) AND the legacy blob keepalive; alias failure
  is unhealthy, legacy failure a warning
- McpServerBaseUrl fallback moved from the stale kindwave WEU ACA FQDN to the
  stable custom domain mcp.autopilotmonitor.com (CNAME to the new GWC instance)
- health/detailed and health/mcp include the probed endpoint URLs in check
  details for Global Admins only (storage table endpoint, WEBSITE_HOSTNAME,
  alias/legacy download URLs, MCP server URL)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:42:40 +02:00
Oliver Kieselbach b005c20598 feat(agent,backend,web): config-channel agent endpoint migration (MigrateToApiBaseUrl)
Re-home agents to a new API base URL through the agent/config response
(backend move or per-tenant EU->US region move) - the lesson from the
2026-07-18 WEU->GWC cutover, where the compiled-in API hostname hard-cut
every in-flight agent (no DNS indirection possible: mTLS breaks behind a
proxy, Flex custom-domain TLS is broken).

Backend (ConfigVersion 33):
- AdminConfiguration.AgentMigrateApiBaseUrl (global) +
  AgentMigrateTenantOverridesJson (tenantId->URL; empty value pins a
  tenant against the global target; override wins)
- GetAgentConfigFunction.ResolveMigrateTarget serves the validated target;
  AgentMigrateServed / AgentMigrateRejected warnings as delivery evidence
- TableConfigRepository Store+Map; admin portal Global Settings round-trip

Agent (kill-switch-mirrored hardening):
- Shared AgentEndpointMigrationRules: https-only, *.azurewebsites.net on a
  label boundary, no path/port/userinfo - enforced on BOTH sides
- Live fetch only: RemoteConfigService cache strips the field on write AND
  read; EndpointMigration.ResolveTarget re-checks the fetch outcome
- AgentRuntimeConfig.Resolve returns before merge/integrity/bootstrap
  cleanup on a pending migration (no spurious IntegrityCheckFailed against
  a stale migration stub), Program.RunAgent rebuilds Phase 3+4 against the
  new URL - single hop, kill signal wins before and after the hop

Tests: AgentEndpointMigrationTests (backend rules + resolution),
EndpointMigrationTests + cache-hardening strip (agent). Full suites green
(Functions 2992, Agent Core 1822, vitest 557, tsc silent).

Doc: docs/agent/endpoint-migration.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 17:48:48 +02:00
Oliver Kieselbach e2d78c6b06 chore(ci): rename SWA workflow to deploy-web.yml with neutral deploy-token secret
The brave-grass filename and secret name referred to the pre-migration
Static Web App hostname. Workflow now matches the deploy-backend/-mcp
naming scheme and uses AZURE_SWA_DEPLOY_TOKEN (already set to the new
SWA's token). The old secret is removed after the first green run.
2026-07-18 16:51:39 +02:00
Oliver Kieselbach 3d06a689cc feat(web): rename Known Issues to Service Announcements, drop maintenance banners
Menu/dashboard/tip labels now match the docs page title. The temporary
maintenance banners on landing and dashboard are removed — the completion
notice lives in Service Announcements. Also checks in the migration
follow-up plan and the runbook outcome addendum.
2026-07-18 16:43:50 +02:00
Oliver Kieselbach 2d5f87140f chore: retarget SignalR quota watcher to Standard S1 limits + CI build hygiene
- SignalR quota watcher/health card: defaults 20/20k (Free tier) -> 1000/1M
  (Standard S1 x1 unit); settings renamed SignalRFreeTier* -> SignalRConnectionLimit/
  SignalRDailyMessageLimit (no Azure overrides were set); all operator-facing
  texts tier-neutral (connections 429 hard, message overage is billed not throttled)
- web: drop obsolete edge runtime from opengraph/twitter/apple-icon image routes
  (next/og runs on Node since 14; images now statically generated at build)
- web: update caniuse-lite browser db (silences 6-months-old Browserslist warning)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:38:15 +02:00
Oliver Kieselbach ce1ca3dbf4 updated SignalR limits on system health 2026-07-18 16:22:49 +02:00
Oliver Kieselbach b1a8e371ad chore(build): retarget DecisionCore.Tests to net10.0, drop 8.0.x SDK from agent CI
Nothing required net8.0 -- the test runner just predated the .NET 10
migration. Agent projects stay net48 (Intune/OOBE constraint).
474/474 tests green on net10.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 16:01:39 +02:00
Oliver Kieselbach b698a87bc8 chore(build): align local and CI builds on .NET 10 SDK
- global.json pins SDK 10.0.2xx (latestFeature) for local and CI parity
- build-agent.yml: SDK 10 as build toolchain, 8.0.x kept for the
  net8.0 DecisionCore.Tests runtime
- build.ps1: stale .NET 8 hints updated to .NET 10

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 15:59:23 +02:00
Oliver Kieselbach ac31ee980e feat(ci): mirror agent artifacts to legacy storage (transition until download.-alias migration; fail-soft, counter stays on eu only) 2026-07-18 15:57:40 +02:00
91 changed files with 5893 additions and 328 deletions
+16 -1
View File
@@ -58,10 +58,11 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v5
# SDK 10 = single build toolchain (parity with local build.ps1 / global.json).
- name: Setup .NET
uses: actions/setup-dotnet@v5
with:
dotnet-version: '8.0.x'
dotnet-version: '10.0.x'
# Build numbers come from the shared counter blob (single source of truth for CI
# AND local build.ps1). Reserve-before-build with ETag-CAS: concurrent builds can
@@ -265,6 +266,12 @@ jobs:
run: |
$containerUrl = "https://autopilotmonitoreu.blob.core.windows.net/agent"
$sasToken = "${{ secrets.AZURE_BLOB_AGENT_SAS_TOKEN }}".TrimStart('?')
# Legacy mirror (transition until customers moved to the download.-alias):
# blobs are ALSO pushed to the old account so the bootstrap URL baked into
# customers' Intune scripts keeps serving current bits. Fail-soft -- a
# legacy hiccup must never fail a release. Counter lives ONLY on eu.
$legacyContainerUrl = "https://autopilotmonitor.blob.core.windows.net/agent"
$legacySasToken = "${{ secrets.AZURE_BLOB_AGENT_LEGACY_SAS_TOKEN }}".TrimStart('?')
function Send-Blob {
param([string]$Local, [string]$BlobName, [string]$ContentType)
@@ -279,6 +286,14 @@ jobs:
Write-Host "Uploading $BlobName ($([math]::Round($bytes.Length / 1MB, 2)) MB)..."
Invoke-RestMethod -Uri $url -Method Put -Headers $headers -Body $bytes
Write-Host " uploaded $BlobName"
if (-not [string]::IsNullOrEmpty($legacySasToken)) {
try {
Invoke-RestMethod -Uri "$legacyContainerUrl/$BlobName`?$legacySasToken" -Method Put -Headers $headers -Body $bytes
Write-Host " mirrored $BlobName to legacy storage"
} catch {
Write-Host "::warning::legacy mirror failed for $BlobName -- $($_.Exception.Message)"
}
}
}
# Versioned namespace (always)
@@ -1,4 +1,4 @@
name: Azure Static Web Apps CI/CD
name: Deploy Web (Static Web App)
on:
push:
@@ -6,7 +6,7 @@ on:
- main
paths:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/azure-static-web-apps-brave-grass-0070b5703.yml'
- '.github/workflows/deploy-web.yml'
workflow_dispatch:
pull_request:
types: [opened, synchronize, reopened, closed]
@@ -14,7 +14,7 @@ on:
- main
paths:
- 'src/Web/autopilot-monitor-web/**'
- '.github/workflows/azure-static-web-apps-brave-grass-0070b5703.yml'
- '.github/workflows/deploy-web.yml'
jobs:
build_and_deploy_job:
@@ -37,7 +37,7 @@ jobs:
NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL: ${{ vars.NEXT_PUBLIC_PLATFORM_STATS_MANIFEST_URL }}
NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING: ${{ secrets.NEXT_PUBLIC_APPINSIGHTS_CONNECTION_STRING }}
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_GRASS_0070B5703 }}
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
action: "upload"
###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
@@ -56,7 +56,7 @@ jobs:
id: closepullrequest
uses: Azure/static-web-apps-deploy@v1
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_GRASS_0070B5703 }}
azure_static_web_apps_api_token: ${{ secrets.AZURE_SWA_DEPLOY_TOKEN }}
action: "close"
+2 -2
View File
@@ -53,7 +53,7 @@ try {
Write-Host " ✓ .NET SDK version: $dotnetVersion" -ForegroundColor Green
}
catch {
Write-Host " ✗ .NET SDK not found. Please install .NET 8 SDK." -ForegroundColor Red
Write-Host " ✗ .NET SDK not found. Please install .NET 10 SDK." -ForegroundColor Red
exit 1
}
@@ -145,7 +145,7 @@ Write-Host "============================================" -ForegroundColor Cyan
Write-Host ""
Write-Host "Output locations:" -ForegroundColor White
Write-Host " Agent: src\Agent\AutopilotMonitor.Agent.V2\bin\$Configuration\net48\" -ForegroundColor Gray
Write-Host " Functions: src\Backend\AutopilotMonitor.Functions\bin\$Configuration\net8.0\" -ForegroundColor Gray
Write-Host " Functions: src\Backend\AutopilotMonitor.Functions\bin\$Configuration\net10.0\" -ForegroundColor Gray
if ($IncludeWeb) {
Write-Host " Web UI: src\Web\autopilot-monitor-web\.next\" -ForegroundColor Gray
+88
View File
@@ -0,0 +1,88 @@
---
type: Concept
title: Agent Endpoint Migration — Config-Channel Re-Home
description: How the backend re-homes agents to a new API base URL (backend move or per-tenant region move) via MigrateToApiBaseUrl on the agent config response, and why HTTP redirects were rejected.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/EndpointMigration.cs
tags:
- agent
- backend
- config
- migration
- security
timestamp: 2026-07-18T00:00:00+02:00
---
# Problem
The agent's API base URL is compiled into the binary (`Constants.ApiBaseUrl`). No DNS/domain
indirection is possible in front of the API: mTLS client-cert auth does not survive a reverse
proxy (Front Door), and Flex Consumption custom-domain TLS is broken. A backend move (like the
2026-07-18 WEU→GWC subscription migration) therefore hard-cuts every deployed agent still
pointing at the old hostname. In-flight agents self-terminate safely (local MaxLifetime watchdog
+ CleanupService), but their sessions are lost and a per-tenant region move (EU→US) has no
mechanism at all.
# Mechanism
`AgentConfigResponse.MigrateToApiBaseUrl` — served on `GET /api/agent/config` (incl. the
bootstrap wrapper), the same control channel as the kill switch, fetched at every agent
process start.
Agent flow (`Program.RunAgent`, Phase 4):
1. Config fetched from the compiled-in (old) base URL.
2. Kill-switch check runs FIRST — **kill wins over migrate**.
3. `AgentRuntimeConfig.Resolve` detects the migration signal directly after the fetch and
returns early, BEFORE merge / binary-integrity / bootstrap-cleanup — none of those side
effects may run against a config from the backend being abandoned (a migration stub may
advertise stale agent hashes; a spurious `IntegrityCheckFailed` + force-update would be
harmful).
4. `Program.RunAgent` swaps `AgentConfiguration.ApiBaseUrl`, disposes the old client stack,
rebuilds Phase 3 (auth clients) and re-runs Phase 4 against the new URL —
`allowEndpointMigration: false`, so the new backend's config **cannot chain a second hop**.
5. Kill-switch check runs again on the NEW backend's config; startup continues normally
(Phase 5+ builds mTLS/telemetry clients from the updated base URL).
# Hardening (mirrors the kill switch)
* **Live fetch only** — the on-disk config cache strips `MigrateToApiBaseUrl` on write AND
read (`RemoteConfigService.CacheConfig` / `LoadCachedConfig`); `EndpointMigration.ResolveTarget`
re-checks `LastFetchOutcome == Succeeded` as defence-in-depth. A planted cache file can
never re-home an agent.
* **Allowlist validated on BOTH sides** — `AgentEndpointMigrationRules` (Shared, single
source): https only, host must end in `.azurewebsites.net` on a label boundary, no
path/query/userinfo/port. The backend refuses to serve an invalid admin value
(`AgentMigrateRejected` warning); the agent refuses to honour one even if the backend was
compromised.
* **One hop, no loops** — second Resolve pass never migrates; a target equal to the current
base URL is a no-op (steady state during the migration window).
# Operation
Global admin settings (`AdminConfiguration`, admin portal → Global Settings):
* `AgentMigrateApiBaseUrl` — global target, set on the backend being **abandoned**; clear it
after the migration window.
* `AgentMigrateTenantOverridesJson` — JSON object `tenantId → URL` for per-tenant moves.
An entry with an **empty string** pins that tenant against the global target (staged rollout).
Overrides win over the global value.
Delivery evidence: every served target logs `AgentMigrateServed` (Warning — reaches App
Insights) with tenant, serial, agent version and target. Agent-side, the re-home is logged as
`Endpoint migration: backend re-homed this agent from … to …`.
Reach limitation: agents that cannot complete an authenticated config fetch (e.g. cert-broken
devices) never see the signal — they fall back to the local MaxLifetime/emergency-break
lifecycle. During a migration window the old backend must keep running as a serving stub;
keeping the old `*.azurewebsites.net` name parked afterwards also prevents hostname
re-registration by third parties while stragglers drain.
# Citations
* `src/Shared/AutopilotMonitor.Shared/Services/AgentEndpointMigrationRules.cs` — shared validation rules
* `src/Shared/AutopilotMonitor.Shared/Models/Config/AgentConfigResponse.cs` — wire field
* `src/Backend/AutopilotMonitor.Functions/Functions/Config/GetAgentConfigFunction.cs` — `ResolveMigrateTarget` + serving (ConfigVersion 33)
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Configuration/EndpointMigration.cs` — agent-side evaluation
* `src/Agent/AutopilotMonitor.Agent.V2/Runtime/AgentRuntimeConfig.cs` — early return before merge/integrity
* `src/Agent/AutopilotMonitor.Agent.V2/Program.cs` — one-hop rebuild in `RunAgent`
* Tests: `AgentEndpointMigrationTests` (backend), `EndpointMigrationTests` + `RemoteConfigServiceCacheHardeningTests` (agent)
+1
View File
@@ -7,5 +7,6 @@ How the on-device agent works:
* [Logs, Persistence & Crash Recovery](logs-and-persistence.md) - every persisted file and why it exists; signal log vs spool; snapshot/journal; the replay-based recovery flow and quarantine.
* [Hello Wizard Un-Skip](hello-wizard-unskip.md) - why a single "WHfB disabled" policy read never decides completion: the HelloWizardStarted signal (prevention + cure) and the HelloTracker confirmation second read.
* [Autopilot ZTD Diagnostics](autopilot-ztd-diagnostics.md) - Windows' own diagnostic surfaces for the profile-download flow: ModernDeployment event IDs (807/815/164/...), the Diagnostics\Autopilot registry key, deployment-service endpoints, and the known-issue error-code map.
* [MDM Reboot Coalescing](mdm-reboot-coalescing.md) - attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated 2800 observations (neutral, flush-time watermark) + ANALYZE-ESP-005 gated on an actually-observed reboot.
Reading order for newcomers: overview → decision engine → persistence.
+100
View File
@@ -0,0 +1,100 @@
---
type: Concept
title: MDM Reboot Coalescing — attributing the mid-ESP reboot and second sign-in to policy URIs
description: How the agent observes reboot-required MDM policy flags (DM-Enterprise EventID 2800, aggregated) and how ANALYZE-ESP-005 attributes an actually-observed enrollment reboot to them — including why the raw events stay neutral and why the Shell-Core RebootCoalescing match was removed.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Enrollment/SystemSignals
tags:
- agent
- esp
- reboot
- policy-attribution
- second-sign-in
timestamp: 2026-07-20T00:00:00+02:00
---
# MDM Reboot Coalescing
Device-assigned MDM policies whose CSP URIs match the OS reboot-required catalog
(`HKLM\SOFTWARE\Microsoft\Provisioning\SyncML\RebootRequiredURIs`) force a **coalesced
reboot at the end of ESP DeviceSetup** — the user sees an unexpected restart and a
**second sign-in screen** before AccountSetup. Invisible in the Intune console; most
admins live with it, not knowing that reassigning the profiles to user groups removes the
reboot entirely. Source research: Patch My PC,
["Avoid the second sign-in screen after Autopilot"](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/).
The session already counts reboots (`system_reboot_detected` → `session.rebootCount`);
this feature adds the **cause attribution**.
# Verified signal facts (session b2e890c1, 2026-07-20)
* **EventID 2800**, `Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin`,
one record per URI, verified description shape:
`The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).`
Records arrive in sub-second bursts during policy sync.
* **Timing semantics (the critical nuance)**: 2800 fires at EVERY policy sync. Only URIs
applied **during ESP DeviceSetup** are coalesced into the forced mid-ESP restart. In the
verified session the flags fired during AccountSetup (after DeviceSetup completed) —
**no reboot occurred**, and `omadmRebootRequiredFlag` read `false`. A detector that
claims "reboot incoming" on every 2800 is therefore wrong.
* **Shell-Core 62407 "RebootCoalescing" — REMOVED as a signal.** The token appears in the
ROUTINE bootstrap marker on every enrollment:
`Name: 'CommercialOOBE_BootstrapStatusCategory_SubcategoryProcessing_Started', Value: '{"message":"BootstrapStatus: Starting subcategory DeviceSetup.RebootCoalescing...","errorCode":0}'`
— the ESP merely *processes* the RebootCoalescing subcategory (which checks whether a
reboot is needed). A substring match on it is a false positive by construction. If a
real coalesced-reboot record is ever captured on the E2E VM, a precise pattern can be
added then.
# Detector — `MdmRebootPolicyTracker` (`MdmRebootPolicyWatcherHost`)
Live `EventLogWatcher` on the DM-Enterprise Admin channel (EventID 2800) + startup
backfill (default 60 min lookback). Records are **buffered and debounced (10 s)**, then
flushed as **ONE aggregated `mdm_policy_reboot_required` event** per burst:
* `Severity=Info`, neutral message ("Windows flagged N MDM policy URIs as
reboot-required: … (+N more)") — the raw event never claims a reboot.
* `Data`: `rebootUris` (distinct, sorted, capped 20), `uriCount` (uncapped distinct),
`firstRebootUri` (record-time order, drives rule interpolation), `recordCount`,
`unparsedCount`/`sampleDescription` (extraction diagnostics), `backfilled` (all-backfill),
fail-soft `omadmRebootRequiredFlag` probe of
`HKLM\SOFTWARE\Microsoft\Provisioning\OMADM\SyncML\RebootRequired` (null = unknown).
* `Timestamp` = earliest record time (backfilled bursts land at their historical instant);
`ImmediateUpload=true` (one event per burst is cheap, and a DeviceSetup burst precedes a
reboot that kills the agent).
* **Watermark is persisted at FLUSH, not at claim** (`mdm-reboot-watermark.json`): if the
coalesced reboot kills the process before the debounced emit, nothing is marked — the
post-restart backfill re-reads and emits the burst. A flushed burst is never repeated.
URI extraction: EventData values starting with `./` preferred, permissive regex over the
description as fallback (trailing punctuation trimmed — the verified text wraps the URI in
parens and ends with a period). Unparsed records still count into the aggregate.
Config (internal steering only, no admin UI — agent-native collector, NOT an
`IAgentAnalyzer`): `Collectors.MdmRebootPolicyWatcherEnabled` (default true),
`Collectors.MdmRebootPolicyBackfillLookbackMinutes` (default 60). ConfigVersion 35.
# Advisory — ANALYZE-ESP-005
The reboot **claim** lives here, and only here. Required conditions (BOTH):
1. `mdm_policy_reboot_required.firstRebootUri` exists (→ `{{firstRebootUri}}` interpolation)
2. `system_reboot_detected` count ≥ 1 — **an actually-observed reboot**
Policy flags alone (e.g. AccountSetup user-policy sync) stay advisory-silent. Optional
confidence: `uriCount ≥ 2` (+10), `omadmRebootRequiredFlag == true` (+15) on base 70 /
threshold 70. Remediation: map URIs to Intune profiles, reassign device → user groups (or
exclude the Autopilot device group), or accept + communicate the restart.
# Non-goals / invariants
* Event type **deliberately distinct from `system_reboot_detected`** — the backend
RebootCount matches that string exactly in two places
(`EventIngestProcessor.Classification` + terminal recount); attribution must never
inflate the reboot count.
* No session-level field / SessionInfoCard change in this iteration (follow-up candidate).
* The `RebootRequiredURIs` catalog itself is not read; only the 2800 events (which carry
the actually-triggered URIs) are consumed.
# Citations
* [Patch My PC — Avoid the second sign-in screen after Autopilot](https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/)
* [Microsoft Learn — Enrollment Status Page tracking](https://learn.microsoft.com/en-us/autopilot/enrollment-status)
+95
View File
@@ -0,0 +1,95 @@
---
type: Concept
title: Business Timestamps vs. the Azure Tables System Timestamp
description: Why the system Timestamp is never authoritative, how the 2026-07-18 migration exposed that, and the OccurredUtc + RowKey-decode compensation (mappers, RowKey-range date filters, retention, backfill).
resource: /src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs
tags:
- backend
- table-storage
- audit-log
- ops-events
- events
- migration
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Azure Table Storage's `Timestamp` is a **system property**: the server ignores any supplied
value on write and stamps the row's *write* time; every row rewrite (storage-account
migration, MERGE) resets it. Three tables nevertheless treated it as the business event
time — writes like `["Timestamp"] = evt.Timestamp` were silent no-ops, unnoticed because
write time ≈ event time in normal operation.
The 2026-07-18 EU storage migration rewrote every row, so all migrated rows in
**AuditLogs**, **OpsEvents**, and **Events** read as "2026-07-18 ~13:58 UTC": displays were
wrong, date-window filters matched the wrong rows, and retention froze (nothing older than
the migration moment) with a mass-delete cliff one retention period later.
# Rule
**The system Timestamp is never authoritative.** Every table that needs an event time
stores it in the custom column `OccurredUtc` (`BusinessTimestamp.OccurredUtcColumn`).
Custom columns are copied verbatim by `scripts/Migration/Copy-TableStorage.ps1`, so future
migrations preserve chronology. Migration checklist: any table whose reads/filters depend
on a timestamp must carry it as a custom column — audit this before every storage move.
# Recovery sources per table
The original time survives migrations inside the RowKey:
| Table | RowKey scheme | Decodable |
|---|---|---|
| AuditLogs | `!{MaxTicks-ticks:D19}_{guid:N}` (since 2026-05-06) | Yes; legacy bare-GUID rows (pre-2026-05) are lost |
| OpsEvents | `{MaxTicks-ticks:D19}` | Yes, all rows |
| Events | `{ts:yyyyMMddHHmmssfff}_{seq:D10}` (sanitized agent time) | Yes, all rows (ms precision) |
# Mechanism
`BusinessTimestamp` (Functions/Helpers) is the single source of truth: RowKey decoders,
the OData clause builders, and `GetUtcDateTime` (datetime columns materialize as
`DateTimeOffset` *or* `DateTime` depending on payload shape — always read via this helper).
1. **Read mappers** resolve `OccurredUtc` → RowKey decode → system Timestamp (last
resort; only audit legacy GUID rows end there). Resolvers:
`TableStorageService.ResolveAuditTimestamp`, `TableOpsEventRepository.ResolveTimestamp`,
`TableStorageService.ResolveEventTimestamp` (also used by
`GetEarliestSessionEventTimestampAsync`, which feeds StartedAt/Duration reconciliation —
a system-Timestamp read there would rewrite migrated sessions' derived fields).
2. **Date-window filters and retention use RowKey ranges**, not property comparisons:
OData excludes rows missing a compared property (an `OccurredUtc` filter would drop
every not-yet-backfilled row), and RowKey ranges are index-backed. Boundary semantics
are tick-exact equivalents of the old `Timestamp ge/le` clauses.
**Trap:** audit legacy GUID RowKeys (hex first char) sort *after* all `!`-rows, so
every lower bound `RowKey ge '!…'` must be paired with `RowKey lt '"'`
(`AuditTimeEncodedUpperBound`) — without the guard, a retention sweep would delete all
legacy rows. Pinned by `BusinessTimestampFilterClauseTests`.
3. **Writes** in all three tables now set `OccurredUtc`; the no-op `["Timestamp"]`
entries were removed.
4. **Backfill** (`POST /api/maintenance/backfill-occurred-utc`, GlobalAdminOnly, dry-run
by default, `table=audit|ops`, batched via `maxRows`/`continuation`): MERGE that adds
only `OccurredUtc` decoded from the RowKey — deterministic and idempotent. **Events
rows are deliberately never backfilled** (read-path decode is sufficient; no bulk
mutation of enrollment evidence). Note: a MERGE resets the row's system Timestamp —
irrelevant once nothing depends on it.
5. **Audit retention** runs two passes until ~2027-01: RowKey-range for `!`-rows plus a
`RowKey ge '0' and Timestamp lt …` pass that ages legacy GUID rows out via their frozen
migration-date system Timestamp (remove after 2027-03-01).
# Consequences
* Audit legacy GUID rows have "unknown date" semantics: excluded from date-filtered
views, shown with the (wrong but honest) 2026-07-18 fallback in unfiltered views.
* Raw endpoints (`/api/raw/*`, `query_table`) intentionally keep showing the raw system
Timestamp — they are storage-state inspectors, not business views.
* Event timestamps for pre-cutover rows resolve to the RowKey's sanitized agent time
(ms precision); `ReceivedAt` (server ingest time, rows since 2026-03-19) and
`OriginalTimestamp` (pre-clamp agent time, clamped rows only) remain separate fields.
# Citations
* `src/Backend/AutopilotMonitor.Functions/Helpers/BusinessTimestamp.cs` — decoders + clause builders
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Maintenance.cs` — audit write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/DataAccess/TableStorage/TableOpsEventRepository.cs` — ops write/mapper/filters/retention
* `src/Backend/AutopilotMonitor.Functions/Services/TableStorageService.Sessions.cs` — event writes, `ResolveEventTimestamp`, earliest-event read
* `src/Backend/AutopilotMonitor.Functions/Services/OccurredUtcBackfillService.cs` + `Functions/Admin/BackfillOccurredUtcFunction.cs` — backfill
+10
View File
@@ -17,6 +17,16 @@ https://docs.autopilotmonitor.com (separate repository).
* [V2 Agent](agent/index.md) - How the on-device agent works: [runtime overview](agent/overview.md), [decision engine](agent/decision-engine.md) (reducer, signals, completion arms), [logs & persistence](agent/logs-and-persistence.md) (signal log, snapshot, crash recovery), [Hello wizard un-skip](agent/hello-wizard-unskip.md) (why a single "WHfB disabled" read never decides completion), [Autopilot ZTD diagnostics](agent/autopilot-ztd-diagnostics.md) (event IDs, diagnostic registry, endpoints, error-code map).
* [Build Counter Blob](agent/build-counter-blob.md) - How agent build numbers are minted: shared counter blob with ETag-CAS so local and CI builds never collide; CI additionally attests build provenance (Sigstore).
* [Agent Endpoint Migration](agent/endpoint-migration.md) - How the backend re-homes agents to a new API base URL via the config channel (backend move or per-tenant region move): live-fetch-only, allowlist-validated on both sides, one hop, kill wins.
* [MDM Reboot Coalescing](agent/mdm-reboot-coalescing.md) - Attributing the mid-ESP coalesced reboot ("second sign-in") to the device-assigned policy URIs that forced it: aggregated DM-Enterprise 2800 observations (neutral, flush-time watermark) + the ANALYZE-ESP-005 advisory gated on an actually-observed reboot.
# Rules
* [Gather Rule Phase Scoping & Emit-on-Change](rules/gather-rule-phase-scoping.md) - Restricting gather rules to enrollment phases (activePhases / activeFromPhase sticky latch) and emitMode "on_change" result-dedup — the anti-spam pair for interval rules.
# Backend
* [Business Timestamps](backend/business-timestamps.md) - Why the Azure Tables system Timestamp is never authoritative (migrations reset it), and the OccurredUtc + RowKey-decode compensation for AuditLogs/OpsEvents/Events after the 2026-07-18 migration.
# Security & Identity
+12
View File
@@ -1,5 +1,17 @@
# Log
## 2026-07-20
* **Update**: Reworked `agent/mdm-reboot-coalescing.md` after the first real enrollment (session b2e890c1): 2800 records are now **aggregated per burst** into ONE neutral Info event (`rebootUris[]`/`uriCount`/`firstRebootUri`; watermark persisted at flush so a pre-emit reboot kill loses nothing); the raw event no longer claims a reboot — ANALYZE-ESP-005 is gated on an actually-observed `system_reboot_detected`. **Removed** the Shell-Core `esp_reboot_coalescing` event entirely: the "RebootCoalescing" token appears in the routine `SubcategoryProcessing_Started` bootstrap marker on every enrollment (false positive by construction). Verified 2800 description shape recorded.
* **Creation**: Added `agent/mdm-reboot-coalescing.md` — the agent now attributes the "unexpected mid-ESP reboot + second sign-in" pattern (PMPC research) to the device-assigned policy URIs that forced it: new `MdmRebootPolicyTracker` (DM-Enterprise EventID 2800, per-URI `mdm_policy_reboot_required` events, cross-restart RecordId watermark), a `RebootCoalescing` branch in `ShellCoreTracker` (`esp_reboot_coalescing`, backfill-emitting by design), and the `ANALYZE-ESP-005` advisory (reassign device → user groups). ConfigVersion 35; event types deliberately distinct from `system_reboot_detected` so RebootCount stays uninflated.
* **Creation**: Added `backend/business-timestamps.md` (new `backend/` section) — the 2026-07-18 migration reset every row's system Timestamp, exposing that AuditLogs/OpsEvents/Events had (unknowingly) displayed and filtered on it all along; documents the `OccurredUtc` column, the RowKey decoders and RowKey-range date/retention clauses in `BusinessTimestamp`, the GUID-guard trap, the audit/ops backfill endpoint, and the "system Timestamp is never authoritative" migration rule.
* **Creation**: Added `rules/gather-rule-phase-scoping.md` (new `rules/` section) — gather rules gain `activePhases` (run only during listed phases), `activeFromPhase` (sticky latch from a phase onwards, never via Failed) and `emitMode: "on_change"` (canonical-hash result dedup with `suppressedPolls` observability), ConfigVersion 34. Includes the prerequisite backend fix: the custom-rule toggle PUT no longer wipes the stored rule definition (partial-merge in `GatherRuleService.UpdateRuleAsync`).
## 2026-07-18
* **Creation**: Added `agent/endpoint-migration.md` — config-channel agent re-home (`MigrateToApiBaseUrl` on `GET agent/config`, ConfigVersion 33): backend or per-tenant API-URL moves without DNS indirection (impossible due to mTLS/Flex-TLS). Live-fetch-only with cache strip, `AgentEndpointMigrationRules` allowlist validated on both sides, one hop, kill-switch wins. Built as the lesson from the 2026-07-18 WEU→GWC cutover, which hard-cut all in-flight agents.
## 2026-07-17
* **Creation**: Added `agent/build-counter-blob.md` — agent build numbers now come from a shared public counter blob (`buildcounter-v{N}.txt`, reserve-before-build with ETag-CAS) so local `build.ps1` and CI `build-agent.yml` builds can never mint the same number. Documents the CI parity fixes (SummaryDialog.exe.config in the ZIP, EXE-vs-manifest version check, hash-oracle update gated on stable cutover) and the Sigstore provenance attestation on CI-built ZIPs.
+97
View File
@@ -0,0 +1,97 @@
---
type: Concept
title: Gather Rule Phase Scoping & Emit-on-Change
description: How gather rules are restricted to enrollment phases (activePhases / activeFromPhase) and how emitMode "on_change" suppresses repeated identical results — the anti-spam pair for interval rules.
resource: /src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs
tags:
- agent
- backend
- gather-rules
- config
timestamp: 2026-07-20T00:00:00+02:00
---
# Problem
Interval gather rules (e.g. a registry poll every 60 s) ran during **all** enrollment phases
and emitted an event on **every** tick — including `exists=false` events while the target key
did not exist yet. A rule needed only from Account Setup onwards produced hundreds of
identical noise events per session and wasted collector runs.
# Schema
Three opt-in fields on `GatherRule` (Shared model, `rules/schema/gather-rule.schema.json`,
delivered to the agent via `GET agent/config`; ConfigVersion 34). Absent fields = legacy
behavior — full back-compat.
| Field | Type | Semantics |
|---|---|---|
| `activePhases` | `string[]` \| null | Rule runs only while the current phase is in the list. Null/empty = unrestricted. |
| `activeFromPhase` | `string` \| null | Sticky latch: activates when the phase first reaches the threshold, then stays active for the rest of the session — including through `Failed`. Null = unrestricted. |
| `emitMode` | `"always"` \| `"on_change"` \| null | `on_change`: poll on the trigger cadence, emit only when the collected result changes. Null/`always` = emit every collection (legacy). |
Canonical phase tokens are the `EnrollmentPhase` enum **names** `Start`(0) …`Complete`(7).
Backend validation (`GatherRulesFunction.ValidateScopeAndEmitMode`) rejects `Unknown`/`Failed`,
numeric tokens, unknown emit modes, and both scope fields set at once (400). The agent
defensively prefers `activePhases` if both arrive anyway.
# Scope semantics (agent, `GatherRuleExecutor`)
* Scope applies to **all trigger types**:
* `interval` — timers keep running; an out-of-scope tick is a free early-return (no
start/stop lifecycle races on phase transitions).
* `startup` — a scoped startup rule not yet in scope at `UpdateRules` is **deferred**: it
runs exactly once from `OnPhaseChanged` when its scope activates (same
`_startupRulesExecuted` dedup; it is not part of the `WaitForStartupRules` latch).
* `phase_change` — evaluated against the **new** phase; the scope gate runs before the
per-(rule, phase) dedup so an out-of-scope pass does not consume the execution slot.
* `on_event` — gated by the current phase.
* **Before the first phase signal** of a session (`_currentPhase == Unknown`) scoped rules are
inactive. Sessions that never produce phase signals never run them (documented behavior).
* **From-phase latch**: activates when `phase != Unknown && phase != Failed && (int)phase >=
(int)fromPhase`. `Failed=99` would ordinal-satisfy every threshold, hence the explicit
exclusion — but once latched, a rule stays active through `Failed`. A config refresh that
delivers a from-phase rule after its threshold passed latches it immediately.
* `--run-gather-rules` (diagnostic CLI) sets `IgnorePhaseScope = true` — no phase context
exists there, scoped rules execute unconditionally (log line marks the bypass).
# Emit-on-change semantics
* The collector result dict is canonically hashed (keys ordinal-sorted at every nesting
level, invariant value formatting, SHA-256) **before** `ruleId`/`ruleTitle` injection.
* First in-scope non-empty result always emits — on an absent registry key that first
`exists=false` event *is* the visible "we're polling now" indicator.
* Unchanged results increment a per-rule suppression counter; the next emitted event carries
`suppressedPolls` + `suppressedSinceUtc` in its `Data` so the gap is observable.
* Composition with `emitOnlyIfExists=true` (collector-level): a miss returns an empty dict —
no emit AND no hash update. Result: zero noise while absent, one event on appearance, then
only on change.
* State is in-memory and survives `UpdateRules` config refreshes; an agent restart causes at
most one re-emit per rule (acceptable under the per-enrollment lifecycle).
* Volatile collectors (eventlog timestamps, logparser positions) degrade `on_change` to
`always` — every poll hashes differently. The option stays visible for them since some
outputs are stable; this is documented, not hidden.
# Portal & delivery
* Create form defaults new rules to `emitMode: "on_change"`; existing rules load as
`always`. The scope UI is a three-way mode (always / during phases / from phase) backed by
a canonical `GATHER_PHASES` list — the same list replaced the free-text `triggerPhase`
input, whose placeholder used to suggest invalid tokens.
* The custom-rule toggle PUT (`{enabled, isBuiltIn, isCommunity}`) previously full-replaced
the stored rule and wiped Title/Target/Trigger — `GatherRuleService.UpdateRuleAsync` now
merges toggle-style partials (empty Title/CollectorType/Target) into the existing row and
preserves `CreatedAt` on full edits. This fix is a prerequisite: without it every toggle
would silently clear the new fields.
* Old agents (Newtonsoft) ignore unknown config properties: scoped rules run everywhere,
`emitMode` is ignored — graceful degradation until the fleet rolls forward.
* `ContentEquivalent` treats null and empty `activePhases` as equal, so pre-existing DB rows
(absent columns → null) never diff against seeds — no reseed churn.
# Citations
* `src/Agent/AutopilotMonitor.Agent.V2.Core/Monitoring/Telemetry/Gather/GatherRuleExecutor.cs` — scope gates, latch, canonical hash
* `src/Backend/AutopilotMonitor.Functions/Functions/Rules/GatherRulesFunction.cs` — `ValidateScopeAndEmitMode`
* `src/Backend/AutopilotMonitor.Functions/Services/GatherRuleService.cs` — partial-PUT merge, `ContentEquivalent`
* `src/Shared/AutopilotMonitor.Shared/Models/Rules/GatherRule.cs` — field contracts
* Tests: `GatherRuleExecutorPhaseScopeTests`, `GatherRuleExecutorEmitModeTests` (agent); `GatherRuleUpdatePartialMergeTests`, `GatherRuleScopeFieldsTests` (backend)
+6
View File
@@ -0,0 +1,6 @@
{
"sdk": {
"version": "10.0.200",
"rollForward": "latestFeature"
}
}
+85
View File
@@ -0,0 +1,85 @@
{
"$schema": "../schema/analyze-rule.schema.json",
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{ "signal": "multiple_reboot_uris", "condition": "exists", "weight": 10 },
{ "signal": "omadm_reboot_flag_set", "condition": "exists", "weight": 15 }
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": ["esp", "reboot", "coalescing", "second-sign-in", "policy-attribution", "user-experience"]
}
+99
View File
@@ -1785,6 +1785,105 @@
"enrollment"
]
},
{
"ruleId": "ANALYZE-ESP-005",
"title": "MDM Policy Forced a Mid-ESP Reboot (Second Sign-In)",
"description": "Attributes an observed enrollment reboot to device-assigned MDM policies that Windows flagged as reboot-required. Fires ONLY when both facts are present in the session: at least one actually-observed reboot (system_reboot_detected) AND reboot-required policy flags (mdm_policy_reboot_required). Policy flags alone — e.g. from a user-policy sync during AccountSetup — force no reboot and stay advisory-silent.",
"severity": "warning",
"category": "esp",
"version": "1.1.0",
"author": "Autopilot Monitor",
"enabled": true,
"trigger": "single",
"baseConfidence": 70,
"conditions": [
{
"signal": "reboot_forcing_policy_uri",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "firstRebootUri",
"operator": "exists",
"required": true
},
{
"signal": "reboot_observed",
"source": "event_count",
"eventType": "system_reboot_detected",
"operator": "count_gte",
"value": "1",
"required": true
},
{
"signal": "multiple_reboot_uris",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "uriCount",
"operator": "gte",
"value": "2",
"required": false
},
{
"signal": "omadm_reboot_flag_set",
"source": "event_data",
"eventType": "mdm_policy_reboot_required",
"dataField": "omadmRebootRequiredFlag",
"operator": "equals",
"value": "true",
"required": false
}
],
"confidenceFactors": [
{
"signal": "multiple_reboot_uris",
"condition": "exists",
"weight": 10
},
{
"signal": "omadm_reboot_flag_set",
"condition": "exists",
"weight": 15
}
],
"confidenceThreshold": 70,
"explanation": "This session **rebooted during enrollment**, and Windows flagged one or more device-assigned MDM policies as **reboot-required** — the classic cause of the coalesced mid-ESP restart and the unexpected **second sign-in screen** before AccountSetup.\n\nFirst reboot-forcing policy URI: `{{firstRebootUri}}`\n\nThe complete URI list is in the `mdm_policy_reboot_required` timeline event (`rebootUris`). Policies applied DURING ESP DeviceSetup are coalesced into one forced restart at the end of DeviceSetup; the same flags raised later (e.g. user-policy sync in AccountSetup) have no immediate effect — this advisory only fires when a reboot was actually observed.\n\nThis is expected OS behavior, not a failure — but it degrades the enrollment experience, confuses users, and is invisible in the Intune console.",
"remediation": [
{
"title": "Reassign the reboot-forcing policies to user groups",
"steps": [
"Open the `mdm_policy_reboot_required` timeline event and note the CSP URIs in `rebootUris` (e.g. `./Device/Vendor/MSFT/...`)",
"Map each URI to its Intune profile (Settings Catalog / configuration profile) — the URI path names the CSP area",
"Reassign those profiles from device groups to user groups so they apply during/after AccountSetup instead of DeviceSetup",
"Alternatively, exclude the Autopilot device group from the profile and deliver the setting post-enrollment"
]
},
{
"title": "Or accept the reboot and set expectations",
"steps": [
"If the policy must be device-assigned (e.g. a security baseline), keep it and document the expected mid-ESP restart",
"Communicate to users that a restart and second sign-in during provisioning is expected for these deployments"
]
}
],
"relatedDocs": [
{
"title": "Patch My PC: Avoid the second sign-in screen after Autopilot",
"url": "https://patchmypc.com/blog/autopilot-unexpected-reboot-autopilot-second-login-screen/"
},
{
"title": "Microsoft Learn: Enrollment Status Page tracking",
"url": "https://learn.microsoft.com/en-us/autopilot/enrollment-status"
}
],
"markSessionAsFailedDefault": false,
"tags": [
"esp",
"reboot",
"coalescing",
"second-sign-in",
"policy-attribution",
"user-experience"
]
},
{
"ruleId": "ANALYZE-ID-001",
"title": "Expected Machine Certificate Not Found",
+18
View File
@@ -96,6 +96,24 @@
"type": "string",
"description": "Event type to trigger on (only used when trigger = \"on_event\")."
},
"activePhases": {
"type": "array",
"description": "Run the rule only while the current enrollment phase is one of these phases (EnrollmentPhase enum names). Empty/absent = unrestricted. Mutually exclusive with activeFromPhase.",
"items": {
"type": "string",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
}
},
"activeFromPhase": {
"type": "string",
"description": "Activate the rule once the enrollment phase first reaches this phase, then keep it active for the rest of the session (sticky latch). Absent = unrestricted. Mutually exclusive with activePhases.",
"enum": ["Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup", "AppsUser", "FinalizingSetup", "Complete"]
},
"emitMode": {
"type": "string",
"description": "Emit behavior: \"always\" (default — emit on every collection) or \"on_change\" (poll on the trigger cadence but emit only when the collected result changes; the first in-scope result always emits).",
"enum": ["always", "on_change"]
},
"outputEventType": {
"type": "string",
"description": "EventType for the emitted event (e.g., \"gather_proxy_settings\")."
@@ -0,0 +1,99 @@
using AutopilotMonitor.Agent.V2.Core.Configuration;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
{
/// <summary>
/// Pins the agent-side endpoint-migration contract (<see cref="EndpointMigration"/>):
/// only a LIVE fetch is honoured, the target must pass the shared allowlist rules even
/// though the backend already validated it, and serving the agent its own current URL is
/// a no-op — the exact mirror of the kill-switch live-fetch hardening.
/// </summary>
public sealed class EndpointMigrationTests
{
private const string CurrentUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
private const string TargetUrl = "https://autopilotmonitor-api-us.azurewebsites.net";
private static AgentConfigResponse Config(string migrateUrl) =>
new AgentConfigResponse { MigrateToApiBaseUrl = migrateUrl };
[Fact]
public void Live_fetch_with_valid_differing_target_migrates()
{
var target = EndpointMigration.ResolveTarget(
Config(TargetUrl), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null);
Assert.Equal(TargetUrl, target);
}
[Fact]
public void Target_is_normalized_to_lowercase_host_without_trailing_slash()
{
var target = EndpointMigration.ResolveTarget(
Config("https://AutopilotMonitor-API-US.azurewebsites.net/"),
RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null);
Assert.Equal(TargetUrl, target);
}
[Theory]
[InlineData(RemoteConfigFetchOutcome.FromCache)]
[InlineData(RemoteConfigFetchOutcome.UsedDefaults)]
[InlineData(RemoteConfigFetchOutcome.NotAttempted)]
public void Non_live_fetch_is_never_honoured(RemoteConfigFetchOutcome outcome)
{
Assert.Null(EndpointMigration.ResolveTarget(
Config(TargetUrl), outcome, CurrentUrl, logger: null));
}
[Fact]
public void Null_config_and_empty_signal_are_noops()
{
Assert.Null(EndpointMigration.ResolveTarget(
null, RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(null), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(" "), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Fact]
public void Target_equal_to_current_url_is_a_noop()
{
// Steady state during a migration window: agent already runs on the target
// (case/trailing-slash differences must not cause a rebuild loop).
Assert.Null(EndpointMigration.ResolveTarget(
Config(CurrentUrl), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
Assert.Null(EndpointMigration.ResolveTarget(
Config(CurrentUrl.ToUpperInvariant() + "/"), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Theory]
[InlineData("http://autopilotmonitor-api-us.azurewebsites.net")] // https only
[InlineData("https://attacker.example.com")] // host not allowlisted
[InlineData("https://evil-azurewebsites.net")] // suffix must sit on a label boundary
[InlineData("https://.azurewebsites.net")] // empty host prefix
[InlineData("https://api.azurewebsites.net:8443")] // non-default port
[InlineData("https://user:pw@api.azurewebsites.net")] // userinfo
[InlineData("https://api.azurewebsites.net/some/path")] // base URL only
[InlineData("https://api.azurewebsites.net?x=1")] // no query
[InlineData("not a url")]
public void Invalid_targets_are_rejected(string candidate)
{
Assert.Null(EndpointMigration.ResolveTarget(
Config(candidate), RemoteConfigFetchOutcome.Succeeded, CurrentUrl, logger: null));
}
[Fact]
public void Dev_api_url_override_still_migrates_to_valid_target()
{
// The running base URL may be a non-allowlisted dev override (--api-url); the
// comparison falls back to plain normalization and the migration still applies.
var target = EndpointMigration.ResolveTarget(
Config(TargetUrl), RemoteConfigFetchOutcome.Succeeded, "http://localhost:7071", logger: null);
Assert.Equal(TargetUrl, target);
}
}
}
@@ -62,6 +62,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
DeviceBlocked = true,
DeviceKillSignal = true,
UnblockAt = new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc),
MigrateToApiBaseUrl = "https://attacker.azurewebsites.net",
};
probe.DoCache(config);
@@ -76,9 +77,11 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.False(persisted.DeviceBlocked);
Assert.False(persisted.DeviceKillSignal);
Assert.Null(persisted.UnblockAt);
Assert.Null(persisted.MigrateToApiBaseUrl);
// The attacker EXE hash must not appear anywhere in the persisted bytes.
// The attacker EXE hash / re-home host must not appear anywhere in the persisted bytes.
Assert.DoesNotContain("deadbeef", raw);
Assert.DoesNotContain("attacker.azurewebsites.net", raw);
// Benign fields survive the strip.
Assert.Equal(77, persisted.ConfigVersion);
@@ -92,6 +95,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
Assert.True(config.DeviceBlocked);
Assert.True(config.DeviceKillSignal);
Assert.Equal(new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc), config.UnblockAt);
Assert.Equal("https://attacker.azurewebsites.net", config.MigrateToApiBaseUrl);
}
// ── Read path: LoadCachedConfig strips a planted (attacker) cache file ─
@@ -103,6 +107,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
[InlineData("DeviceBlocked")]
[InlineData("DeviceKillSignal")]
[InlineData("UnblockAt")]
[InlineData("MigrateToApiBaseUrl")]
public void LoadCachedConfig_strips_planted_security_sensitive_field(string field)
{
var probe = NewProbe();
@@ -119,6 +124,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
DeviceBlocked = true,
DeviceKillSignal = true,
UnblockAt = new DateTime(2099, 1, 1, 0, 0, 0, DateTimeKind.Utc),
MigrateToApiBaseUrl = "https://attacker.azurewebsites.net",
};
File.WriteAllText(_cachePath, JsonConvert.SerializeObject(attacker, Formatting.Indented));
@@ -145,6 +151,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Tests.Configuration
case "UnblockAt":
Assert.Null(loaded.UnblockAt);
break;
case "MigrateToApiBaseUrl":
Assert.Null(loaded.MigrateToApiBaseUrl);
break;
default:
throw new InvalidOperationException($"unknown field {field}");
}
@@ -0,0 +1,249 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// EmitMode "on_change" on <see cref="GatherRuleExecutor"/>: poll on the trigger cadence but
/// emit only when the collected result changes. The dedup state machine is asserted
/// synchronously via the internal <c>ShouldEmitOnChange</c> / <c>ComputeCanonicalHash</c>
/// (no timer races); the ExecuteRule wiring (hash-before-injection, empty-result no-op,
/// emitOnlyIfExists composition) through the public startup path.
/// </summary>
[Collection("SerialThreading")] // startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorEmitModeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_7e3a1f4b-0000-0000-0000-000000000000";
public GatherRuleExecutorEmitModeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private EnrollmentEvent SingleEvent()
{
lock (_eventsGate) return Assert.Single(_events);
}
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
private static GatherRule OnChangeRule(string id = "GATHER-OC-001") => new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
EmitMode = "on_change",
};
private static Dictionary<string, object> Result(params (string Key, object Value)[] pairs)
{
var d = new Dictionary<string, object>();
foreach (var (key, value) in pairs) d[key] = value;
return d;
}
// ── on_change state machine (synchronous) ──────────────────────────
[Fact]
public void FirstResult_AlwaysEmits_ThenUnchangedIsSuppressed_ThenChangeReemits()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // first
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false)))); // unchanged
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1")))); // changed
}
[Fact]
public void EmitAfterSuppressionStreak_CarriesSuppressedPollCount_AndResets()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
var changed = Result(("exists", true));
Assert.True(_executor.ShouldEmitOnChange(rule, changed));
Assert.Equal(3, changed["suppressedPolls"]);
Assert.True(changed.ContainsKey("suppressedSinceUtc"));
// Counter must reset: the next change emits without a suppression payload.
var changedAgain = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(rule, changedAgain));
Assert.False(changedAgain.ContainsKey("suppressedPolls"));
Assert.False(changedAgain.ContainsKey("suppressedSinceUtc"));
}
[Fact]
public void SuppressionState_IsIsolatedPerRule()
{
var ruleA = OnChangeRule("GATHER-OC-A");
var ruleB = OnChangeRule("GATHER-OC-B");
var payload = Result(("exists", false));
Assert.True(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
// Same content, different rule: B has no hash yet — must emit.
Assert.True(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleA, new Dictionary<string, object>(payload)));
Assert.False(_executor.ShouldEmitOnChange(ruleB, new Dictionary<string, object>(payload)));
}
[Fact]
public void DedupState_SurvivesUpdateRulesRefresh()
{
var rule = OnChangeRule();
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
// Config refresh mid-session must not reset the last-emitted hash (that would
// re-emit an unchanged result on every refresh).
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.ShouldEmitOnChange(rule, Result(("exists", false))));
}
// ── Canonical hashing ──────────────────────────────────────────────
[Fact]
public void CanonicalHash_IsKeyOrderInsensitive()
{
var a = new Dictionary<string, object> { ["alpha"] = 1, ["beta"] = "x" };
var b = new Dictionary<string, object> { ["beta"] = "x", ["alpha"] = 1 };
Assert.Equal(
GatherRuleExecutor.ComputeCanonicalHash(a),
GatherRuleExecutor.ComputeCanonicalHash(b));
}
[Fact]
public void CanonicalHash_DetectsValueChanges_IncludingNested()
{
var baseline = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedScalar = new Dictionary<string, object>
{
["exists"] = false,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 1 } },
};
var changedNested = new Dictionary<string, object>
{
["exists"] = true,
["entries"] = new List<object> { new Dictionary<string, object> { ["id"] = 2 } },
};
var baseHash = GatherRuleExecutor.ComputeCanonicalHash(baseline);
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedScalar));
Assert.NotEqual(baseHash, GatherRuleExecutor.ComputeCanonicalHash(changedNested));
}
[Fact]
public void CanonicalHash_DistinguishesTypeShapes()
{
// "1" (string) vs 1 (int) and null vs "<null>"-like strings must not collide.
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "1" }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = 1 }));
Assert.NotEqual(
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = null }),
GatherRuleExecutor.ComputeCanonicalHash(new Dictionary<string, object> { ["v"] = "<null>" }));
}
// ── ExecuteRule wiring (async via startup path) ────────────────────
[Fact]
public void AlwaysMode_EmitsEveryCollection()
{
// EmitMode absent (legacy) — repeated identical results all emit. Two executions of
// the same collector via two rules sharing the target prove no dedup is applied.
var rule1 = OnChangeRule("GATHER-AL-1"); rule1.EmitMode = null; rule1.Trigger = "startup";
var rule2 = OnChangeRule("GATHER-AL-2"); rule2.EmitMode = null; rule2.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule1, rule2 });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(2));
}
[Fact]
public void OnChange_FirstInScopePoll_EmitsExistsFalse_AsPollingIndicator()
{
// The first result on an absent key IS the visible "we're polling now" indicator.
var rule = OnChangeRule();
rule.Trigger = "startup";
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
var evt = SingleEvent();
Assert.Equal(false, evt.Data["exists"]);
Assert.Equal("GATHER-OC-001", evt.Data["ruleId"]); // injection happened after hashing
}
[Fact]
public void OnChange_ComposedWithEmitOnlyIfExists_EmptyResultDoesNotTouchHashState()
{
// emitOnlyIfExists miss → empty result → no emit AND no hash update: the eventual
// first real result must still count as "first emit".
var rule = OnChangeRule();
rule.Trigger = "startup";
rule.Parameters = new Dictionary<string, string> { ["emitOnlyIfExists"] = "true" };
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.WaitForStartupRules(30));
Thread.Sleep(300);
Assert.Equal(0, EventCount); // zero noise while the key is absent
// When the key appears (simulated via the state machine), the result emits as first.
Assert.True(_executor.ShouldEmitOnChange(rule, Result(("exists", true), ("Version", "1"))));
}
}
}
@@ -0,0 +1,326 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Shared.Models;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.Gather
{
/// <summary>
/// Phase scoping on <see cref="GatherRuleExecutor"/> (ActivePhases / ActiveFromPhase):
/// the scope matrix is asserted synchronously via the internal <c>IsRuleInScope</c>
/// (no timer races); the trigger-path wiring (deferred startup, phase_change / on_event
/// gating) is asserted through the public API with positive event waits.
/// Rules use the absent-registry-path pattern from
/// <see cref="RegistryCollectorEmitOnlyIfExistsTests"/> so no real system state is touched.
/// </summary>
[Collection("SerialThreading")] // deferred startup rules execute on the shared ThreadPool
public sealed class GatherRuleExecutorPhaseScopeTests : IDisposable
{
private readonly TempDirectory _tmp = new TempDirectory();
private readonly List<EnrollmentEvent> _events = new List<EnrollmentEvent>();
private readonly object _eventsGate = new object();
private readonly GatherRuleExecutor _executor;
private const string AbsentPath =
"HKLM\\SOFTWARE\\AutopilotMonitorTests\\DefinitelyAbsent_5b1c8d2e-0000-0000-0000-000000000000";
public GatherRuleExecutorPhaseScopeTests()
{
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
_executor = new GatherRuleExecutor(
"sess", "tenant",
evt => { lock (_eventsGate) _events.Add(evt); },
logger)
{
// Bypass the registry allowlist guard for the synthetic test path.
UnrestrictedMode = true
};
}
public void Dispose()
{
_executor.Dispose();
_tmp.Dispose();
}
private int EventCount { get { lock (_eventsGate) return _events.Count; } }
private bool WaitForEventCount(int count, int timeoutMs = 10000)
{
var sw = Stopwatch.StartNew();
while (sw.ElapsedMilliseconds < timeoutMs)
{
if (EventCount >= count) return true;
Thread.Sleep(25);
}
return EventCount >= count;
}
/// <summary>Interval rule with a 1h period — the timer never ticks within a test run,
/// so these rules exist purely for synchronous scope-matrix assertions.</summary>
private static GatherRule ScopeProbeRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "interval",
IntervalSeconds = 3600,
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
private static GatherRule StartupRule(string id, List<string> activePhases = null, string fromPhase = null)
=> new GatherRule
{
RuleId = id,
Title = id,
CollectorType = "registry",
Target = AbsentPath,
Trigger = "startup",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = activePhases,
ActiveFromPhase = fromPhase,
};
// ── Scope matrix (synchronous) ─────────────────────────────────────
[Fact]
public void UnscopedRule_IsInScope_EvenBeforeAnyPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-000");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_InactiveBeforeFirstPhaseSignal()
{
var rule = ScopeProbeRule("GATHER-SCOPE-001", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_ActiveDuringListedPhase_InactiveAfterLeaving()
{
var rule = ScopeProbeRule("GATHER-SCOPE-002", activePhases: new List<string> { "AccountSetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void ActivePhases_MatchIsCaseInsensitive()
{
var rule = ScopeProbeRule("GATHER-SCOPE-003", activePhases: new List<string> { "accountsetup" });
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesAtThreshold_AndStaysSticky_IncludingThroughFailed()
{
var rule = ScopeProbeRule("GATHER-SCOPE-010", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.False(_executor.IsRuleInScope(rule)); // threshold not reached
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(_executor.IsRuleInScope(rule)); // latched
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.True(_executor.IsRuleInScope(rule)); // sticky through Failed
}
[Fact]
public void FromPhase_NoSpuriousActivationViaFailed()
{
// Failed=99 would ordinal-satisfy every threshold — it must never latch.
var rule = ScopeProbeRule("GATHER-SCOPE-011", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.Failed);
Assert.False(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_LatchesWhenThresholdIsSkippedOver()
{
// The session may never report the exact threshold phase — a later phase latches too.
var rule = ScopeProbeRule("GATHER-SCOPE-012", fromPhase: "AccountSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void FromPhase_RuleDeliveredByRefreshAfterPhaseReached_LatchesImmediately()
{
_executor.UpdateRules(new List<GatherRule>());
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
// Config refresh delivers a new from-phase rule whose threshold already passed.
var rule = ScopeProbeRule("GATHER-SCOPE-013", fromPhase: "DeviceSetup");
_executor.UpdateRules(new List<GatherRule> { rule });
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void BothFieldsSet_ActivePhasesWins()
{
// Backend validation rejects this shape; the agent defensively prefers ActivePhases.
var rule = ScopeProbeRule("GATHER-SCOPE-020",
activePhases: new List<string> { "DeviceSetup" }, fromPhase: "Start");
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.False(_executor.IsRuleInScope(rule)); // FromPhase(Start) would say yes — list wins
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.True(_executor.IsRuleInScope(rule));
}
[Fact]
public void IgnorePhaseScope_BypassesScope_WithoutPhaseContext()
{
_executor.IgnorePhaseScope = true;
var during = ScopeProbeRule("GATHER-SCOPE-030", activePhases: new List<string> { "AccountSetup" });
var from = ScopeProbeRule("GATHER-SCOPE-031", fromPhase: "Complete");
_executor.UpdateRules(new List<GatherRule> { during, from });
Assert.True(_executor.IsRuleInScope(during));
Assert.True(_executor.IsRuleInScope(from));
}
// ── Trigger-path wiring (async, positive waits) ────────────────────
[Fact]
public void UnscopedStartupRule_RunsImmediately()
{
_executor.UpdateRules(new List<GatherRule> { StartupRule("GATHER-START-000") });
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
[Fact]
public void ScopedStartupRule_IsDeferred_ThenFiresExactlyOnce_WhenScopeActivates()
{
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-001", fromPhase: "AccountSetup")
});
// Not in scope at UpdateRules: nothing queued, latch trivially satisfied.
Assert.True(_executor.WaitForStartupRules(30));
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1), "deferred startup rule should fire when its scope activates");
// Later phase changes must not re-fire it (dedup via _startupRulesExecuted).
_executor.OnPhaseChanged(EnrollmentPhase.AppsUser);
_executor.OnPhaseChanged(EnrollmentPhase.FinalizingSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void PhaseChangeRule_OutOfScopePhase_DoesNotFire_InScopePhaseFires()
{
// TriggerPhase empty = fires on every phase change; scope narrows it to AccountSetup.
var rule = new GatherRule
{
RuleId = "GATHER-PC-001",
Title = "pc",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "phase_change",
TriggerPhase = "",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnPhaseChanged(EnrollmentPhase.DeviceSetup);
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Assert.True(WaitForEventCount(1));
// Same phase again: the (rule, phase) dedup slot is now consumed.
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
Thread.Sleep(300);
Assert.Equal(1, EventCount);
}
[Fact]
public void OnEventRule_GatedByScope()
{
var rule = new GatherRule
{
RuleId = "GATHER-OE-001",
Title = "oe",
CollectorType = "registry",
Target = AbsentPath,
Trigger = "on_event",
TriggerEventType = "app_install_failed",
OutputEventType = "gather_test",
Enabled = true,
ActivePhases = new List<string> { "AccountSetup" },
};
_executor.UpdateRules(new List<GatherRule> { rule });
_executor.OnEvent("app_install_failed"); // no phase signal yet → out of scope
Thread.Sleep(300);
Assert.Equal(0, EventCount);
_executor.OnPhaseChanged(EnrollmentPhase.AccountSetup);
_executor.OnEvent("app_install_failed");
Assert.True(WaitForEventCount(1));
}
[Fact]
public void IgnorePhaseScope_ScopedStartupRule_RunsImmediately()
{
// The --run-gather-rules diagnostic path: scoped rules must execute unconditionally.
_executor.IgnorePhaseScope = true;
_executor.UpdateRules(new List<GatherRule>
{
StartupRule("GATHER-START-002", activePhases: new List<string> { "AccountSetup" })
});
Assert.True(_executor.WaitForStartupRules(30));
Assert.True(WaitForEventCount(1));
}
}
}
@@ -0,0 +1,336 @@
#nullable enable
using System;
using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Agent.V2.Core.Tests.Harness;
using AutopilotMonitor.Agent.V2.Core.Tests.Orchestration;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
using AutopilotMonitor.Shared;
using Xunit;
namespace AutopilotMonitor.Agent.V2.Core.Tests.Monitoring.SystemSignals
{
/// <summary>
/// MdmRebootPolicyTracker — aggregates DM-Enterprise EventID 2800 bursts into ONE neutral
/// <c>mdm_policy_reboot_required</c> event (session b2e890c1 rework: per-record events were
/// spam, and the raw event must not CLAIM a reboot — ANALYZE-ESP-005 does, gated on an
/// observed system_reboot_detected). Drives the primitive ProcessEvent seam + explicit
/// FlushPending (debounce 0 = manual-flush mode; no timers, no SerialThreading needed).
/// The registry probe is always overridden — tests never touch the live registry.
/// </summary>
public sealed class MdmRebootPolicyTrackerTests : IDisposable
{
private static readonly DateTime At = new DateTime(2026, 7, 20, 16, 20, 57, DateTimeKind.Utc);
private const string UriSvchost = "./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation";
private const string UriSystemGuard = "./Device/Vendor/MSFT/Policy/Config/DeviceGuard/ConfigureSystemGuardLaunch";
// Verified real 2800 description shape (session b2e890c1, 2026-07-20).
private const string RealDescription =
"The following URI has triggered a reboot: (./Device/Vendor/MSFT/Policy/Config/ServiceControlManager/SvchostProcessMitigation).";
private readonly TempDirectory _tmp = new TempDirectory();
private readonly FakeSignalIngressSink _sink;
private readonly MdmRebootPolicyTracker _tracker;
private long _nextRecordId = 1;
public MdmRebootPolicyTrackerTests()
{
_sink = new FakeSignalIngressSink();
_tracker = MakeTracker(stateDirectory: null);
}
public void Dispose() => _tmp.Dispose();
private MdmRebootPolicyTracker MakeTracker(string? stateDirectory)
{
var post = new InformationalEventPost(_sink, new VirtualClock(At));
var logger = new AgentLogger(_tmp.Path, AgentLogLevel.Info);
var tracker = new MdmRebootPolicyTracker(
sessionId: "sess-mdm",
tenantId: "tenant-mdm",
post: post,
logger: logger,
backfillEnabled: false,
stateDirectory: stateDirectory,
debounceMilliseconds: 0); // manual-flush mode — tests call FlushPending explicitly
tracker.RebootRequiredFlagProbe = () => null; // registry untouched by default
return tracker;
}
private void Buffer(string? rebootUri, long? recordId = null, bool isBackfill = false, DateTime? timeCreatedUtc = null, string? description = null)
{
_tracker.ProcessEvent(
eventId: MdmRebootPolicyTracker.EventId_PolicyRebootRequired,
recordId: recordId ?? _nextRecordId++,
timeCreatedUtc: timeCreatedUtc ?? At,
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
private IReadOnlyList<FakeSignalIngressSink.PostedSignal> Emitted() =>
_sink.Posted.Where(p =>
p.Kind == DecisionSignalKind.InformationalEvent
&& p.Payload != null
&& p.Payload.TryGetValue(SignalPayloadKeys.EventType, out var et)
&& et == Constants.EventTypes.MdmPolicyRebootRequired).ToList();
private static IReadOnlyDictionary<string, object> Data(FakeSignalIngressSink.PostedSignal s) =>
(IReadOnlyDictionary<string, object>)s.TypedPayload!;
[Fact]
public void Burst_FlushesAsOneAggregatedEvent_InfoSeverity_ImmediateUpload()
{
Buffer(UriSvchost, timeCreatedUtc: At);
Buffer(UriSystemGuard, timeCreatedUtc: At.AddMilliseconds(120));
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal("Info", s.Payload![SignalPayloadKeys.Severity]);
Assert.Equal("true", s.Payload![SignalPayloadKeys.ImmediateUpload]);
var data = Data(s);
Assert.Equal(2, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
Assert.Equal(UriSvchost, data["firstRebootUri"]);
var uris = Assert.IsType<List<string>>(data["rebootUris"]);
Assert.Equal(2, uris.Count);
Assert.Contains(UriSvchost, uris);
Assert.Contains(UriSystemGuard, uris);
}
[Fact]
public void Message_IsNeutralObservation_NoRebootClaim()
{
// Session b2e890c1: flags raised during AccountSetup have no effect — the raw event
// must state the observation only, never a future consequence.
Buffer(UriSvchost);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var message = s.Payload![SignalPayloadKeys.Message];
Assert.Contains("flagged", message);
Assert.DoesNotContain("will restart", message);
Assert.DoesNotContain("second sign-in", message);
}
[Fact]
public void FlushPending_Empty_EmitsNothing()
{
_tracker.FlushPending();
Assert.Empty(Emitted());
}
[Fact]
public void AggregateTimestamp_IsEarliestRecordTime()
{
var early = At.AddMinutes(-30);
Buffer(UriSystemGuard, timeCreatedUtc: At, isBackfill: true);
Buffer(UriSvchost, timeCreatedUtc: early, isBackfill: true);
_tracker.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(early, s.OccurredAtUtc);
Assert.Equal(true, Data(s)["backfilled"]);
// firstRebootUri follows record-time order, not buffer order.
Assert.Equal(UriSvchost, Data(s)["firstRebootUri"]);
}
[Fact]
public void Backfilled_False_WhenAnyRecordIsLive()
{
Buffer(UriSvchost, isBackfill: true);
Buffer(UriSystemGuard, isBackfill: false);
_tracker.FlushPending();
Assert.Equal(false, Data(Assert.Single(Emitted()))["backfilled"]);
}
[Fact]
public void DuplicateUris_AreDistinctInAggregate()
{
Buffer(UriSvchost);
Buffer(UriSvchost);
_tracker.FlushPending();
var data = Data(Assert.Single(Emitted()));
Assert.Equal(1, data["uriCount"]);
Assert.Equal(2, data["recordCount"]);
}
[Fact]
public void UnparsedRecords_CountedWithSampleDescription_StillEmitted()
{
Buffer(rebootUri: null, description: "Some unexpected 2800 wording");
_tracker.FlushPending();
var s = Assert.Single(Emitted());
var data = Data(s);
Assert.False(data.ContainsKey("rebootUris"));
Assert.False(data.ContainsKey("firstRebootUri"));
Assert.Equal(0, data["uriCount"]);
Assert.Equal(1, data["unparsedCount"]);
Assert.Equal("Some unexpected 2800 wording", data["sampleDescription"]);
}
[Fact]
public void SameRecordId_BufferedTwice_CountsOnce()
{
Buffer(UriSvchost, recordId: 42);
Buffer(UriSvchost, recordId: 42);
_tracker.FlushPending();
Assert.Equal(1, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void RecordAfterFlush_GoesIntoNextAggregate()
{
Buffer(UriSvchost, recordId: 10);
_tracker.FlushPending();
Buffer(UriSystemGuard, recordId: 11);
_tracker.FlushPending();
Assert.Equal(2, Emitted().Count);
}
[Fact]
public void BackfillRecordWithLowerRecordId_AfterLiveRecord_IsStillBuffered()
{
// Live watcher is armed BEFORE the backfill scan — an older, never-seen backfill
// record must not be suppressed by a high-water mark.
Buffer(UriSvchost, recordId: 100);
Buffer(UriSystemGuard, recordId: 50, isBackfill: true);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void NegativeRecordId_IsNeverDeduped()
{
Buffer(UriSvchost, recordId: -1);
Buffer(UriSvchost, recordId: -1);
_tracker.FlushPending();
Assert.Equal(2, Data(Assert.Single(Emitted()))["recordCount"]);
}
[Fact]
public void Watermark_PersistedAtFlush_SkipsReReadAcrossInstances()
{
// The coalesced reboot restarts the agent; the post-reboot backfill re-reads the same
// records — a FLUSHED record must be skipped by the next instance.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
first.FlushPending();
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
Assert.Single(Emitted());
}
[Fact]
public void Watermark_NotPersisted_WhenBufferedButNeverFlushed()
{
// Process killed by the very reboot the records announce: nothing was flushed, so the
// next instance must re-read and emit the records.
var first = MakeTracker(stateDirectory: _tmp.Path);
first.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: false);
// no FlushPending — simulated reboot kill
var second = MakeTracker(stateDirectory: _tmp.Path);
second.LoadWatermark();
second.ProcessEvent(MdmRebootPolicyTracker.EventId_PolicyRebootRequired, recordId: 500,
timeCreatedUtc: At, rebootUri: UriSvchost, formattedDescription: null, isBackfill: true);
second.FlushPending();
var s = Assert.Single(Emitted());
Assert.Equal(true, Data(s)["backfilled"]);
}
[Fact]
public void RebootRequiredFlagProbe_True_AddsField_Null_OmitsIt()
{
_tracker.RebootRequiredFlagProbe = () => true;
Buffer(UriSvchost, recordId: 1);
_tracker.FlushPending();
_tracker.RebootRequiredFlagProbe = () => null;
Buffer(UriSystemGuard, recordId: 2);
_tracker.FlushPending();
var emitted = Emitted();
Assert.Equal(2, emitted.Count);
Assert.Equal(true, Data(emitted[0])["omadmRebootRequiredFlag"]);
Assert.False(Data(emitted[1]).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void RebootRequiredFlagProbe_Throwing_IsFailSoft()
{
_tracker.RebootRequiredFlagProbe = () => throw new InvalidOperationException("boom");
Buffer(UriSvchost);
_tracker.FlushPending();
Assert.False(Data(Assert.Single(Emitted())).ContainsKey("omadmRebootRequiredFlag"));
}
[Fact]
public void BuildXPath_Targets2800()
{
Assert.Equal("*[System[(EventID=2800)]]", MdmRebootPolicyTracker.BuildXPath());
}
// ------------------------------------------------------------------ URI extraction
[Fact]
public void ExtractRebootUri_FromVerifiedRealDescription()
{
// Real 2800 text wraps the URI in parentheses and ends with a period — both must be
// excluded from the extracted URI.
var uri = MdmRebootPolicyTracker.ExtractRebootUri(null, RealDescription);
Assert.Equal(UriSvchost, uri);
}
[Fact]
public void ExtractRebootUri_PrefersEventDataValue_OverDescription()
{
var eventData = new Dictionary<string, string> { ["Data0"] = " " + UriSystemGuard + " " };
var uri = MdmRebootPolicyTracker.ExtractRebootUri(eventData, RealDescription);
Assert.Equal(UriSystemGuard, uri);
}
[Fact]
public void ExtractRebootUri_ReturnsNull_WhenNothingUriLike()
{
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(
new Dictionary<string, string> { ["Data0"] = "plain" }, "no uri here"));
Assert.Null(MdmRebootPolicyTracker.ExtractRebootUri(null, null));
}
[Fact]
public void ParseEventData_CollectsNamedAndPositionalData()
{
const string xml =
"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'>" +
"<System><EventID>2800</EventID></System>" +
"<EventData><Data Name='Uri'>./Device/Vendor/MSFT/X</Data><Data>positional</Data></EventData>" +
"</Event>";
var parsed = MdmRebootPolicyTracker.ParseEventData(xml);
Assert.Equal("./Device/Vendor/MSFT/X", parsed["Uri"]);
Assert.Equal("positional", parsed["Data0"]);
}
}
}
@@ -0,0 +1,57 @@
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Services;
namespace AutopilotMonitor.Agent.V2.Core.Configuration
{
/// <summary>
/// Agent-side evaluation of the config-channel endpoint migration signal
/// (<see cref="AgentConfigResponse.MigrateToApiBaseUrl"/>). Mirrors the kill-switch
/// hardening: only a LIVE fetch is honoured (the on-disk config cache strips the field on
/// write and read, and this check re-validates the outcome as defence-in-depth), and the
/// target must pass <see cref="AgentEndpointMigrationRules"/> even though the backend
/// already validated it — a compromised or misconfigured backend must not be able to
/// re-home agents to an arbitrary host.
/// </summary>
public static class EndpointMigration
{
/// <summary>
/// Returns the normalized migration target when the config carries a valid, live-fetched
/// re-home signal that differs from <paramref name="currentBaseUrl"/>; otherwise null.
/// Never throws — every rejection is logged and degrades to "no migration".
/// </summary>
public static string ResolveTarget(
AgentConfigResponse remoteConfig,
RemoteConfigFetchOutcome fetchOutcome,
string currentBaseUrl,
AgentLogger logger)
{
if (remoteConfig == null || string.IsNullOrWhiteSpace(remoteConfig.MigrateToApiBaseUrl))
return null;
if (fetchOutcome != RemoteConfigFetchOutcome.Succeeded)
{
logger?.Warning(
$"Endpoint migration signal ignored — config source is {fetchOutcome}, not a live fetch.");
return null;
}
if (!AgentEndpointMigrationRules.IsEffectiveMigration(
remoteConfig.MigrateToApiBaseUrl, currentBaseUrl, out var target))
{
// Either the target failed validation (scheme/host allowlist) or it equals the
// URL the agent is already using (steady state during a migration window once
// the new backend's own config echoes nothing).
if (target == null && !AgentEndpointMigrationRules.TryNormalizeTarget(
remoteConfig.MigrateToApiBaseUrl, out _))
{
logger?.Warning(
$"Endpoint migration signal REJECTED — target failed validation: '{remoteConfig.MigrateToApiBaseUrl}'");
}
return null;
}
return target;
}
}
}
@@ -249,18 +249,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
// - DeviceBlocked/DeviceKillSignal/UnblockAt: kill is only honoured from a
// live fetch — a planted cached kill would self-destruct every future
// session, a stale cached kill would outlive the admin removing the rule
// - MigrateToApiBaseUrl: endpoint re-home is only honoured from a live
// fetch — a planted cached URL would silently redirect every future
// session's telemetry to an attacker-chosen host
var liveUnrestricted = config.UnrestrictedMode;
var liveAllowDowngrade = config.AllowAgentDowngrade;
var liveExeHash = config.LatestAgentExeSha256;
var liveBlocked = config.DeviceBlocked;
var liveKill = config.DeviceKillSignal;
var liveUnblockAt = config.UnblockAt;
var liveMigrateUrl = config.MigrateToApiBaseUrl;
config.UnrestrictedMode = false;
config.AllowAgentDowngrade = false;
config.LatestAgentExeSha256 = null;
config.DeviceBlocked = false;
config.DeviceKillSignal = false;
config.UnblockAt = null;
config.MigrateToApiBaseUrl = null;
var json = JsonConvert.SerializeObject(config, Formatting.Indented);
config.UnrestrictedMode = liveUnrestricted;
config.AllowAgentDowngrade = liveAllowDowngrade;
@@ -268,6 +273,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
config.DeviceBlocked = liveBlocked;
config.DeviceKillSignal = liveKill;
config.UnblockAt = liveUnblockAt;
config.MigrateToApiBaseUrl = liveMigrateUrl;
File.WriteAllText(_cacheFilePath, json);
_logger.Debug("Remote config cached to disk");
@@ -299,6 +305,7 @@ namespace AutopilotMonitor.Agent.V2.Core.Configuration
config.DeviceBlocked = false;
config.DeviceKillSignal = false;
config.UnblockAt = null;
config.MigrateToApiBaseUrl = null;
}
return config;
}
@@ -0,0 +1,628 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.Eventing.Reader;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;
using System.Threading;
using System.Xml.Linq;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Orchestration;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.Models;
using Newtonsoft.Json;
namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
{
/// <summary>
/// Watches <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> for
/// EventID 2800 — logged once per MDM policy URI that matches the OS reboot-required catalog —
/// and emits ONE aggregated <c>mdm_policy_reboot_required</c> event per burst carrying the URI
/// list (<c>rebootUris</c>, <c>uriCount</c>, <c>firstRebootUri</c>).
/// <para>
/// Verified against session b2e890c1 (2026-07-20): the 2800 description reads
/// <c>The following URI has triggered a reboot: (./Device/...).</c> and records arrive in
/// sub-second bursts during policy sync — hence the debounce aggregation instead of per-record
/// events. CRITICAL semantics: only URIs applied DURING ESP DeviceSetup cause the coalesced
/// mid-ESP reboot ("second sign-in", PMPC research); later syncs (AccountSetup and beyond)
/// merely flag the requirement with no forced reboot. The event is therefore a NEUTRAL
/// observation (Info) — the reboot claim lives in ANALYZE-ESP-005, which requires an
/// actually-observed <c>system_reboot_detected</c> before attributing anything.
/// </para>
/// <para>
/// <b>Backfill + watermark.</b> Device policies can apply before the agent starts, and a
/// genuine coalesced reboot kills the agent BEFORE the debounced emit — in both cases the
/// post-(re)start backfill re-reads the .evtx and emits the aggregate then (historical
/// timestamps). The cross-restart RecordId watermark is persisted only at successful flush, so
/// a pre-emit death loses nothing and a completed emit is never repeated.
/// </para>
/// </summary>
internal sealed class MdmRebootPolicyTracker : IDisposable
{
internal const string Channel = "Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin";
internal const int EventId_PolicyRebootRequired = 2800;
internal const string WatermarkStateFileName = "mdm-reboot-watermark.json";
/// <summary>Cap for the emitted <c>rebootUris</c> list — <c>uriCount</c> always carries the uncapped distinct total.</summary>
internal const int MaxUrisPerEvent = 20;
/// <summary>Default quiet-period before a burst of 2800 records is flushed as one event.</summary>
internal const int DefaultDebounceMilliseconds = 10000;
/// <summary>Registry key whose <c>RebootRequired</c> value/subkey flags a pending OMA-DM coalesced reboot.</summary>
internal const string OmadmSyncMlKeyPath = @"SOFTWARE\Microsoft\Provisioning\OMADM\SyncML";
// Permissive OMA-DM URI shape ("./Device/Vendor/MSFT/...", "./User/..."). The verified 2800
// text wraps the URI in parentheses — ')' is deliberately outside the class so the match
// stops cleanly; trailing sentence punctuation is trimmed after the match.
private static readonly Regex UriPattern = new Regex(
@"\./[A-Za-z0-9_\-./\[\]{}%~+]+", RegexOptions.Compiled);
private readonly AgentLogger _logger;
private readonly string _sessionId;
private readonly string _tenantId;
private readonly InformationalEventPost _post;
private readonly bool _backfillEnabled;
private readonly int _backfillLookbackMinutes;
private readonly string _stateDirectory;
private readonly int _debounceMilliseconds;
private EventLogWatcher _watcher;
private Timer _debounceTimer;
private sealed class PendingRecord
{
public long RecordId;
public DateTime TimeUtc;
public string Uri; // null when extraction failed
public string Description; // may be null
public bool IsBackfill;
}
// Pending burst buffer + dedup state, all guarded by _stateLock. Same claim contract as
// WindowsUpdateTracker (immutable cross-restart boundary + intra-run HashSet), with ONE
// deliberate difference: the watermark is persisted at FLUSH time, not at claim time —
// records buffered but never flushed (process killed by the very reboot they announce)
// must be re-read and emitted by the post-restart backfill.
private readonly List<PendingRecord> _pending = new List<PendingRecord>();
private long _restartWatermark = -1;
private readonly HashSet<long> _seenThisRun = new HashSet<long>();
private long _maxPersistedRecordId = -1;
private readonly object _stateLock = new object();
/// <summary>
/// Fail-soft corroboration probe for the OMA-DM RebootRequired flag — null = unknown
/// (key absent or unreadable). Settable test seam; the default reads the live registry.
/// </summary>
internal Func<bool?> RebootRequiredFlagProbe { get; set; } = ReadOmadmRebootRequiredFlag;
public MdmRebootPolicyTracker(
string sessionId,
string tenantId,
InformationalEventPost post,
AgentLogger logger,
bool backfillEnabled = true,
int backfillLookbackMinutes = 60,
string stateDirectory = null,
int debounceMilliseconds = DefaultDebounceMilliseconds)
{
_sessionId = sessionId ?? throw new ArgumentNullException(nameof(sessionId));
_tenantId = tenantId ?? throw new ArgumentNullException(nameof(tenantId));
_post = post ?? throw new ArgumentNullException(nameof(post));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_backfillEnabled = backfillEnabled;
_backfillLookbackMinutes = backfillLookbackMinutes;
_stateDirectory = stateDirectory != null ? Environment.ExpandEnvironmentVariables(stateDirectory) : null;
_debounceMilliseconds = debounceMilliseconds;
}
public void Start()
{
LoadWatermark();
StartWatcher();
if (_backfillEnabled && _backfillLookbackMinutes > 0)
{
BackfillRecentEvents();
// The backfill batch is complete — flush immediately instead of waiting out the
// debounce (live records that raced in are simply included).
FlushPending();
}
else
{
_logger.Info("MdmRebootPolicy backfill disabled by config");
}
}
public void Stop()
{
// Flush any un-emitted burst before tearing down (normal agent shutdown path).
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy flush on stop failed: {ex.Message}"); }
var timer = Interlocked.Exchange(ref _debounceTimer, null);
if (timer != null)
{
try { timer.Dispose(); } catch { }
}
if (_watcher == null) return;
try
{
_watcher.Enabled = false;
_watcher.Dispose();
_logger.Info("MdmRebootPolicy watcher stopped");
}
catch (Exception ex)
{
_logger.Error("Error stopping MdmRebootPolicy watcher", ex);
}
finally
{
_watcher = null;
}
}
public void Dispose() => Stop();
// -----------------------------------------------------------------------
// Watcher lifecycle
// -----------------------------------------------------------------------
private void StartWatcher()
{
try
{
var query = new EventLogQuery(Channel, PathType.LogName, BuildXPath());
_watcher = new EventLogWatcher(query);
_watcher.EventRecordWritten += OnEventRecordWritten;
_watcher.Enabled = true;
_logger.Info($"MdmRebootPolicy watcher started: {Channel} (EventID {EventId_PolicyRebootRequired})");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy watcher access denied for {Channel}: {ex.Message}");
}
catch (Exception ex)
{
_logger.Error($"Failed to start MdmRebootPolicy watcher for {Channel}", ex);
// MON-D1: surface a dead watcher as one-shot telemetry, not just a local log line.
CollectorDegradationReporter.Report(_post, _sessionId, _tenantId,
collectorName: "MdmRebootPolicyTracker", reason: $"watcher_arm_failed:{Channel}", ex: ex);
}
}
/// <summary>Targeted-EventID XPath filter. Exposed as internal for tests.</summary>
internal static string BuildXPath() =>
$"*[System[(EventID={EventId_PolicyRebootRequired})]]";
// -----------------------------------------------------------------------
// Event processing — buffer + debounce
// -----------------------------------------------------------------------
private void OnEventRecordWritten(object sender, EventRecordWrittenEventArgs e)
{
if (e.EventRecord == null) return;
try
{
ProcessRecord(e.EventRecord, isBackfill: false);
}
catch (Exception ex)
{
_logger.Error("Error processing MdmRebootPolicy event", ex);
}
}
private void ProcessRecord(EventRecord record, bool isBackfill)
{
var recordId = record.RecordId ?? -1;
// Cheap dedup gate BEFORE the expensive XML/FormatDescription work.
if (IsAlreadyProcessed(recordId))
return;
string xml = null;
try { xml = record.ToXml(); }
catch { /* fall back to description-only parse below */ }
string description = null;
try { description = record.FormatDescription(); }
catch { /* some events lack formatting resources */ }
var rebootUri = ExtractRebootUri(ParseEventData(xml), description);
ProcessEvent(
eventId: record.Id,
recordId: recordId,
timeCreatedUtc: record.TimeCreated?.ToUniversalTime(),
rebootUri: rebootUri,
formattedDescription: description,
isBackfill: isBackfill);
}
/// <summary>
/// Buffers one 2800 record into the pending burst (primitive-input test seam, mirroring the
/// WindowsUpdateTracker pattern). The aggregated event is emitted by <see cref="FlushPending"/>
/// — after the debounce quiet-period, at backfill completion, or at <see cref="Stop"/>.
/// </summary>
internal void ProcessEvent(
int eventId,
long recordId,
DateTime? timeCreatedUtc,
string rebootUri,
string formattedDescription,
bool isBackfill)
{
lock (_stateLock)
{
if (recordId >= 0)
{
// Already emitted in a prior run, or already buffered/emitted this run.
if (recordId <= _restartWatermark || !_seenThisRun.Add(recordId))
return;
}
_pending.Add(new PendingRecord
{
RecordId = recordId,
TimeUtc = timeCreatedUtc ?? DateTime.UtcNow,
Uri = string.IsNullOrEmpty(rebootUri) ? null : rebootUri,
Description = formattedDescription,
IsBackfill = isBackfill,
});
}
ArmDebounce();
}
private void ArmDebounce()
{
if (_debounceMilliseconds <= 0) return; // manual-flush mode (tests; backfill/Stop flush explicitly)
var timer = _debounceTimer;
if (timer == null)
{
var created = new Timer(_ =>
{
try { FlushPending(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicy debounce flush failed: {ex.Message}"); }
}, null, Timeout.Infinite, Timeout.Infinite);
timer = Interlocked.CompareExchange(ref _debounceTimer, created, null) ?? created;
if (!ReferenceEquals(timer, created))
{
try { created.Dispose(); } catch { }
}
}
try { timer.Change(_debounceMilliseconds, Timeout.Infinite); }
catch (ObjectDisposedException) { /* raced Stop() — Stop's flush covers the buffer */ }
}
/// <summary>
/// Emits the pending burst as ONE aggregated <c>mdm_policy_reboot_required</c> event and
/// persists the watermark. No-op when nothing is pending. Internal for tests.
/// </summary>
internal void FlushPending()
{
List<PendingRecord> batch;
long watermarkToPersist = -1;
lock (_stateLock)
{
if (_pending.Count == 0) return;
batch = new List<PendingRecord>(_pending);
_pending.Clear();
foreach (var r in batch)
{
if (r.RecordId > _maxPersistedRecordId)
_maxPersistedRecordId = r.RecordId;
}
watermarkToPersist = _maxPersistedRecordId;
}
if (watermarkToPersist >= 0) PersistWatermark(watermarkToPersist);
batch.Sort((a, b) => a.TimeUtc.CompareTo(b.TimeUtc));
var distinctUris = batch
.Where(r => r.Uri != null)
.Select(r => r.Uri)
.Distinct(StringComparer.OrdinalIgnoreCase)
.ToList();
var firstUri = batch.FirstOrDefault(r => r.Uri != null)?.Uri;
var unparsedCount = batch.Count(r => r.Uri == null);
var allBackfill = batch.All(r => r.IsBackfill);
var earliest = batch[0].TimeUtc;
var data = new Dictionary<string, object>
{
{ "windowsEventId", EventId_PolicyRebootRequired },
{ "eventLogChannel", Channel },
{ "recordCount", batch.Count },
{ "uriCount", distinctUris.Count },
{ "backfilled", allBackfill },
};
if (distinctUris.Count > 0)
{
data["rebootUris"] = distinctUris
.OrderBy(u => u, StringComparer.OrdinalIgnoreCase)
.Take(MaxUrisPerEvent)
.ToList();
data["firstRebootUri"] = firstUri;
}
if (unparsedCount > 0)
{
data["unparsedCount"] = unparsedCount;
var sample = batch.FirstOrDefault(r => r.Uri == null && !string.IsNullOrEmpty(r.Description))?.Description;
if (sample != null)
data["sampleDescription"] = sample.Length > 500 ? sample.Substring(0, 500) + "…" : sample;
}
var flag = SafeProbeRebootRequiredFlag();
if (flag.HasValue) data["omadmRebootRequiredFlag"] = flag.Value;
string message;
if (distinctUris.Count == 1)
message = $"Windows flagged an MDM policy URI as reboot-required: {firstUri}";
else if (distinctUris.Count > 1)
message = $"Windows flagged {distinctUris.Count} MDM policy URIs as reboot-required: {firstUri} (+{distinctUris.Count - 1} more)";
else
message = $"Windows flagged {batch.Count} MDM policy record(s) as reboot-required (URI not parsed — see sampleDescription)";
_post.Emit(new EnrollmentEvent
{
SessionId = _sessionId,
TenantId = _tenantId,
// Earliest record time, not UtcNow: backfilled (and post-reboot re-read) bursts
// must land on the timeline at when the policies were actually flagged.
Timestamp = earliest,
EventType = Constants.EventTypes.MdmPolicyRebootRequired,
// Neutral observation — reboot-required flags outside ESP DeviceSetup have no
// immediate effect. The reboot CLAIM is ANALYZE-ESP-005's job, gated on an
// actually-observed system_reboot_detected.
Severity = EventSeverity.Info,
Source = "MdmRebootPolicyWatcher",
Phase = EnrollmentPhase.Unknown,
Message = message,
Data = data,
// Cheap (one event per burst) and still time-critical: a burst DURING DeviceSetup
// precedes a coalesced reboot that kills the agent.
ImmediateUpload = true,
});
_logger.Info($"MdmRebootPolicy: flushed {batch.Count} record(s) as one event " +
$"(uris={distinctUris.Count}, unparsed={unparsedCount}, backfilled={allBackfill})");
}
/// <summary>
/// Extracts the reboot-forcing OMA-DM URI. Preference order: (1) a structured EventData
/// value that starts with <c>./</c>, (2) a permissive regex over the formatted description
/// (verified shape: <c>The following URI has triggered a reboot: (./Device/...).</c>).
/// Returns null when nothing URI-like is found — the record still counts into the
/// aggregate (<c>unparsedCount</c> + <c>sampleDescription</c>).
/// </summary>
internal static string ExtractRebootUri(Dictionary<string, string> eventData, string description)
{
if (eventData != null)
{
foreach (var value in eventData.Values)
{
var trimmed = value?.Trim();
if (!string.IsNullOrEmpty(trimmed) && trimmed.StartsWith("./", StringComparison.Ordinal))
return trimmed;
}
}
if (!string.IsNullOrEmpty(description))
{
var match = UriPattern.Match(description);
if (match.Success)
{
// The permissive pattern can swallow sentence punctuation trailing the URI.
// OMA-DM URI segments never end in punctuation — trim it.
var candidate = match.Value.TrimEnd('.', ',', ';', ':', '\'', '"', ')');
if (candidate.Length > 2) return candidate;
}
}
return null;
}
/// <summary>
/// Extracts <c>&lt;Data Name="..."&gt;value&lt;/Data&gt;</c> pairs from an event's rendered
/// XML, keyed case-insensitively. Also collects UNNAMED Data elements under positional keys
/// (<c>Data0</c>, <c>Data1</c>, …) — DM-Enterprise events frequently carry positional-only
/// payloads. Namespace-agnostic; returns an empty map on null/malformed XML — never throws.
/// </summary>
internal static Dictionary<string, string> ParseEventData(string xml)
{
var result = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrEmpty(xml)) return result;
try
{
var doc = XDocument.Parse(xml);
var positionalIndex = 0;
foreach (var el in doc.Descendants().Where(e => e.Name.LocalName == "Data"))
{
var nameAttr = el.Attribute("Name");
var key = nameAttr != null && !string.IsNullOrEmpty(nameAttr.Value)
? nameAttr.Value
: "Data" + positionalIndex++;
if (!result.ContainsKey(key))
result[key] = el.Value;
}
}
catch
{
// Malformed / unexpected XML — best effort, return what we have.
}
return result;
}
// -----------------------------------------------------------------------
// OMA-DM RebootRequired flag probe (fail-soft corroboration)
// -----------------------------------------------------------------------
private bool? SafeProbeRebootRequiredFlag()
{
try
{
return RebootRequiredFlagProbe?.Invoke();
}
catch
{
return null; // corroboration only — never let the probe break the emit
}
}
/// <summary>
/// Reads the pending-reboot flag under <see cref="OmadmSyncMlKeyPath"/>. Both a value and a
/// subkey named <c>RebootRequired</c> count as "pending". Fail-soft: null = unknown
/// (key absent / unreadable), matching the EspSkipConfigurationProbe convention.
/// </summary>
internal static bool? ReadOmadmRebootRequiredFlag()
{
try
{
using (var key = Microsoft.Win32.Registry.LocalMachine.OpenSubKey(OmadmSyncMlKeyPath))
{
if (key == null) return null;
if (key.GetValue("RebootRequired") != null) return true;
using (var sub = key.OpenSubKey("RebootRequired"))
{
return sub != null ? (bool?)true : false;
}
}
}
catch
{
return null;
}
}
// -----------------------------------------------------------------------
// Backfill
// -----------------------------------------------------------------------
private void BackfillRecentEvents()
{
try
{
var lookbackMs = (long)_backfillLookbackMinutes * 60 * 1000;
var xpath = $"*[System[(EventID={EventId_PolicyRebootRequired}) and TimeCreated[timediff(@SystemTime) <= {lookbackMs}]]]";
_logger.Info($"MdmRebootPolicy backfill: scanning {Channel} " +
$"(lookback={_backfillLookbackMinutes}min, restartWatermark={_restartWatermark})");
var query = new EventLogQuery(Channel, PathType.LogName, xpath);
int processed = 0;
using (var reader = new EventLogReader(query))
{
EventRecord record;
while ((record = reader.ReadEvent()) != null)
{
using (record)
{
ProcessRecord(record, isBackfill: true);
processed++;
}
}
}
if (processed == 0)
_logger.Debug($"MdmRebootPolicy backfill: no EventID {EventId_PolicyRebootRequired} records in last {_backfillLookbackMinutes} minutes");
else
_logger.Info($"MdmRebootPolicy backfill: scanned {processed} record(s)");
}
catch (EventLogNotFoundException)
{
_logger.Warning($"MdmRebootPolicy event log not found during backfill: {Channel} (normal on non-MDM-enrolled test environments)");
}
catch (UnauthorizedAccessException ex)
{
_logger.Warning($"MdmRebootPolicy backfill access denied: {ex.Message}");
}
catch (Exception ex)
{
_logger.Warning($"MdmRebootPolicy backfill failed: {ex.Message}");
}
}
// -----------------------------------------------------------------------
// Watermark dedup (cross-restart) — persisted at flush, not at claim
// -----------------------------------------------------------------------
private bool IsAlreadyProcessed(long recordId)
{
if (recordId < 0) return false; // no RecordId → cannot dedup, let it through
lock (_stateLock)
{
return recordId <= _restartWatermark || _seenThisRun.Contains(recordId);
}
}
internal void LoadWatermark()
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
if (!File.Exists(filePath)) return;
try
{
var json = File.ReadAllText(filePath);
var state = JsonConvert.DeserializeObject<WatermarkState>(json);
if (state != null)
{
lock (_stateLock)
{
_restartWatermark = state.LastRecordId;
_maxPersistedRecordId = state.LastRecordId;
}
_logger.Info($"MdmRebootPolicy watermark loaded: lastRecordId={state.LastRecordId} (persisted {state.PersistedUtc:O})");
}
}
catch (Exception ex)
{
_logger.Warning($"Failed to load MdmRebootPolicy watermark: {ex.Message}");
}
}
private void PersistWatermark(long recordId)
{
if (string.IsNullOrEmpty(_stateDirectory)) return;
try
{
Directory.CreateDirectory(_stateDirectory);
var filePath = Path.Combine(_stateDirectory, WatermarkStateFileName);
var state = new WatermarkState { LastRecordId = recordId, PersistedUtc = DateTime.UtcNow };
var json = JsonConvert.SerializeObject(state, Formatting.Indented);
var tempPath = filePath + ".tmp";
File.WriteAllText(tempPath, json);
File.Copy(tempPath, filePath, overwrite: true);
try { File.Delete(tempPath); } catch { }
}
catch (Exception ex)
{
_logger.Warning($"Failed to persist MdmRebootPolicy watermark: {ex.Message}");
}
}
internal sealed class WatermarkState
{
public long LastRecordId { get; set; }
public DateTime PersistedUtc { get; set; }
}
}
}
@@ -17,6 +17,12 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals
/// CommercialOOBE_ESPProgress_WhiteGlove_Success — WhiteGlove complete
/// CommercialOOBE_ESPProgress_Failure/_Timeout/_Abort/WhiteGlove_Failed — ESP failure
///
/// NOTE (session b2e890c1, 2026-07-20): a bare "RebootCoalescing" substring match on 62407 was
/// tried and REMOVED — the token appears in the ROUTINE SubcategoryProcessing_Started marker
/// ("Starting subcategory DeviceSetup.RebootCoalescing...") on every enrollment, so it proves
/// nothing about an actual coalesced reboot. Policy-reboot attribution lives entirely in
/// MdmRebootPolicyTracker (EventID 2800) + the ANALYZE-ESP-005 rule gated on an observed reboot.
///
/// Raises <see cref="FinalizingSetupPhaseTriggered"/>, <see cref="WhiteGloveCompleted"/>,
/// and <see cref="EspFailureDetected"/>. Cross-notifies the <see cref="HelloTracker"/>
/// on Hello wizard start and ESP exit so Hello timers can react.
@@ -1,6 +1,9 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather.Collectors;
@@ -28,6 +31,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
private readonly HashSet<string> _phaseRulesExecuted = new HashSet<string>();
private CountdownEvent _startupRulesLatch; // non-null only while startup rules are pending
// Phase-scope + emit-mode state (ActivePhases / ActiveFromPhase / EmitMode on GatherRule).
// Guarded by _scopeLock; collector work always runs outside the lock. The state deliberately
// survives UpdateRules config refreshes — an agent restart causes at most one re-emit per
// on_change rule (acceptable under the per-enrollment lifecycle).
private readonly object _scopeLock = new object();
private EnrollmentPhase _currentPhase = EnrollmentPhase.Unknown;
private readonly HashSet<string> _fromPhaseLatched = new HashSet<string>();
private readonly Dictionary<string, string> _lastEmittedHash = new Dictionary<string, string>();
private readonly Dictionary<string, (int Count, DateTime SinceUtc)> _suppressed =
new Dictionary<string, (int Count, DateTime SinceUtc)>();
private readonly HashSet<string> _scopeBypassLogged = new HashSet<string>();
/// <summary>
/// When true, phase scoping (ActivePhases / ActiveFromPhase) is bypassed and scoped rules
/// execute unconditionally. Used by the standalone <c>--run-gather-rules</c> diagnostic
/// mode, which has no enrollment phase context. EmitMode is unaffected.
/// </summary>
public bool IgnorePhaseScope { get; set; }
/// <summary>
/// When true, guardrails are relaxed: all registry, WMI, and command targets are allowed.
/// File paths allow everything except C:\Users. Set from tenant configuration.
@@ -89,10 +111,27 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
_activeRules = rules.Where(r => r.Enabled).ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait
var pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.ToList();
// Execute startup rules — track completion via CountdownEvent so callers can wait.
// Scoped startup rules that are not yet in scope are deferred: they run once from
// OnPhaseChanged when their scope activates (same _startupRulesExecuted dedup — a
// rule runs either here or there, never both). Unscoped rules behave as before.
List<GatherRule> pendingStartup;
lock (_scopeLock)
{
// A config refresh may deliver new from-phase rules after their phase was
// already reached — evaluate latches against the current phase first.
EvaluateFromPhaseLatchesLocked();
pendingStartup = _activeRules
.Where(r => r.Trigger == "startup" && !_startupRulesExecuted.Contains(r.RuleId))
.Where(r => IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
if (pendingStartup.Count > 0)
{
@@ -101,7 +140,6 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
foreach (var rule in pendingStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
ThreadPool.QueueUserWorkItem(_ =>
{
try { ExecuteRule(rule); }
@@ -110,18 +148,26 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// Set up interval timers
// Set up interval timers. Timers for scoped rules keep running while out of scope —
// the tick gate makes an out-of-scope tick a free no-op, avoiding start/stop
// lifecycle races on phase transitions.
foreach (var rule in _activeRules.Where(r => r.Trigger == "interval" && r.IntervalSeconds.HasValue))
{
var interval = TimeSpan.FromSeconds(rule.IntervalSeconds.Value);
var intervalRule = rule;
var interval = TimeSpan.FromSeconds(intervalRule.IntervalSeconds.Value);
var timer = new Timer(
_ => ExecuteRule(rule),
_ =>
{
if (!IsRuleInScope(intervalRule))
return;
ExecuteRule(intervalRule);
},
null,
interval, // Initial delay = one interval
interval
);
_intervalTimers[rule.RuleId] = timer;
_logger.Info($" Interval rule {rule.RuleId} scheduled every {rule.IntervalSeconds}s");
_intervalTimers[intervalRule.RuleId] = timer;
_logger.Info($" Interval rule {intervalRule.RuleId} scheduled every {intervalRule.IntervalSeconds}s");
}
_logger.Info($"GatherRuleExecutor: {_activeRules.Count(r => r.Trigger == "startup")} startup, " +
@@ -135,11 +181,43 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
var phaseName = newPhase.ToString();
// Update the phase + from-phase latches FIRST so deferred startup activation and
// the phase_change rules below all evaluate against the NEW phase.
List<GatherRule> deferredStartup;
lock (_scopeLock)
{
_currentPhase = newPhase;
EvaluateFromPhaseLatchesLocked();
deferredStartup = _activeRules
.Where(r => r.Trigger == "startup" && HasPhaseScope(r)
&& !_startupRulesExecuted.Contains(r.RuleId)
&& IsRuleInScopeLocked(r))
.ToList();
foreach (var rule in deferredStartup)
{
_startupRulesExecuted.Add(rule.RuleId);
}
}
foreach (var rule in deferredStartup)
{
var deferredRule = rule;
_logger.Info($"Phase {phaseName} activated deferred startup rule {deferredRule.RuleId}");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(deferredRule));
}
foreach (var rule in _activeRules.Where(r => r.Trigger == "phase_change"))
{
if (string.IsNullOrEmpty(rule.TriggerPhase) ||
string.Equals(rule.TriggerPhase, phaseName, StringComparison.OrdinalIgnoreCase))
{
// Scope gate BEFORE the dedup add — an out-of-scope pass must not consume
// the once-per-(rule, phase) execution slot.
if (!IsRuleInScope(rule))
continue;
// Deduplicate: only fire once per (ruleId, phase) combination
var deduplicationKey = $"{rule.RuleId}|{phaseName}";
if (!_phaseRulesExecuted.Add(deduplicationKey))
@@ -163,6 +241,9 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
{
if (string.Equals(rule.TriggerEventType, eventType, StringComparison.OrdinalIgnoreCase))
{
if (!IsRuleInScope(rule))
continue;
_logger.Info($"Event triggered rule {rule.RuleId} (event: {eventType})");
ThreadPool.QueueUserWorkItem(_ => ExecuteRule(rule));
}
@@ -192,6 +273,14 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
if (result.Count > 0)
{
// EmitMode "on_change": hash the raw collector result (before ruleId/title
// injection) and only emit when it differs from the last emitted one. An
// empty result never reaches this point, so an emitOnlyIfExists miss neither
// emits nor updates the hash — composing to "one event on appearance, then
// only on change".
if (IsOnChangeMode(rule) && !ShouldEmitOnChange(rule, result))
return;
result["ruleId"] = rule.RuleId;
result["ruleTitle"] = rule.Title;
@@ -229,6 +318,199 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
}
}
// ===== Phase scope (ActivePhases / ActiveFromPhase) =====
private static bool HasPhaseScope(GatherRule rule)
{
return (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
|| !string.IsNullOrEmpty(rule.ActiveFromPhase);
}
// Internal so the scope matrix is testable synchronously (no timer/ThreadPool races).
internal bool IsRuleInScope(GatherRule rule)
{
lock (_scopeLock)
{
return IsRuleInScopeLocked(rule);
}
}
private bool IsRuleInScopeLocked(GatherRule rule)
{
if (!HasPhaseScope(rule))
return true; // unscoped — legacy behavior
if (IgnorePhaseScope)
{
if (_scopeBypassLogged.Add(rule.RuleId))
_logger.Info($"Gather rule {rule.RuleId}: phase scope ignored (diagnostic mode)");
return true;
}
// Both fields set is rejected by backend validation; defensively prefer ActivePhases.
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0)
{
// No phase signal yet (or scope tokens that never match, e.g. "Unknown"):
// scoped rules stay inactive.
if (_currentPhase == EnrollmentPhase.Unknown)
return false;
var phaseName = _currentPhase.ToString();
foreach (var phase in rule.ActivePhases)
{
if (string.Equals(phase, phaseName, StringComparison.OrdinalIgnoreCase))
return true;
}
return false;
}
return _fromPhaseLatched.Contains(rule.RuleId);
}
/// <summary>
/// Latches every from-phase rule whose threshold the current phase has reached.
/// Never latches on Unknown/Failed (Failed=99 would ordinal-satisfy every threshold);
/// once latched a rule stays active for the session — including through Failed.
/// Caller must hold <see cref="_scopeLock"/>.
/// </summary>
private void EvaluateFromPhaseLatchesLocked()
{
if (_currentPhase == EnrollmentPhase.Unknown || _currentPhase == EnrollmentPhase.Failed)
return;
foreach (var rule in _activeRules)
{
if (string.IsNullOrEmpty(rule.ActiveFromPhase)) continue;
if (rule.ActivePhases != null && rule.ActivePhases.Count > 0) continue; // ActivePhases wins
if (_fromPhaseLatched.Contains(rule.RuleId)) continue;
if (Enum.TryParse<EnrollmentPhase>(rule.ActiveFromPhase, ignoreCase: true, out var fromPhase)
&& fromPhase != EnrollmentPhase.Unknown
&& fromPhase != EnrollmentPhase.Failed
&& (int)_currentPhase >= (int)fromPhase)
{
_fromPhaseLatched.Add(rule.RuleId);
_logger.Info($"Gather rule {rule.RuleId}: from-phase scope latched at {_currentPhase} (activeFromPhase={rule.ActiveFromPhase})");
}
}
}
// ===== Emit mode (on_change) =====
private static bool IsOnChangeMode(GatherRule rule)
{
return string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// on_change decision for a non-empty collector result: true = emit (first result for the
/// rule, or the result changed), false = suppress. Hashes the result BEFORE ruleId/title
/// injection. On emit after a suppression streak, injects <c>suppressedPolls</c> and
/// <c>suppressedSinceUtc</c> into the result so the gap is observable.
/// Internal so the state machine is testable synchronously (no timer/ThreadPool races).
/// </summary>
internal bool ShouldEmitOnChange(GatherRule rule, Dictionary<string, object> result)
{
var hash = ComputeCanonicalHash(result);
bool emit;
lock (_scopeLock)
{
string lastHash;
if (_lastEmittedHash.TryGetValue(rule.RuleId, out lastHash) && lastHash == hash)
{
(int Count, DateTime SinceUtc) streak;
_suppressed[rule.RuleId] = _suppressed.TryGetValue(rule.RuleId, out streak)
? (streak.Count + 1, streak.SinceUtc)
: (1, DateTime.UtcNow);
emit = false;
}
else
{
_lastEmittedHash[rule.RuleId] = hash;
(int Count, DateTime SinceUtc) streak;
if (_suppressed.TryGetValue(rule.RuleId, out streak) && streak.Count > 0)
{
result["suppressedPolls"] = streak.Count;
result["suppressedSinceUtc"] = streak.SinceUtc.ToString("o", CultureInfo.InvariantCulture);
}
_suppressed.Remove(rule.RuleId);
emit = true;
}
}
if (!emit)
_logger.Debug($"Gather rule {rule.RuleId}: result unchanged, emit suppressed (on_change)");
return emit;
}
/// <summary>
/// Canonical hash of a collector result: keys ordinal-sorted at every dictionary level,
/// values rendered with invariant formatting, then SHA-256. Internal for direct test
/// coverage of the canonicalization.
/// </summary>
internal static string ComputeCanonicalHash(Dictionary<string, object> result)
{
var sb = new StringBuilder();
AppendCanonicalValue(sb, result);
using (var sha = SHA256.Create())
{
var hashBytes = sha.ComputeHash(Encoding.UTF8.GetBytes(sb.ToString()));
return Convert.ToBase64String(hashBytes);
}
}
private static void AppendCanonicalValue(StringBuilder sb, object value)
{
if (value == null)
{
sb.Append("<null>");
return;
}
// string first — it is IEnumerable and must not be treated as a sequence
if (value is string str)
{
sb.Append('"').Append(str).Append('"');
return;
}
if (value is System.Collections.IDictionary dict)
{
sb.Append('{');
var keys = dict.Keys.Cast<object>()
.Select(k => k?.ToString() ?? string.Empty)
.OrderBy(k => k, StringComparer.Ordinal);
foreach (var key in keys)
{
sb.Append(key).Append('=');
AppendCanonicalValue(sb, dict[key]);
sb.Append(';');
}
sb.Append('}');
return;
}
if (value is System.Collections.IEnumerable sequence)
{
sb.Append('[');
foreach (var item in sequence)
{
AppendCanonicalValue(sb, item);
sb.Append(';');
}
sb.Append(']');
return;
}
if (value is IFormattable formattable)
{
sb.Append(formattable.ToString(null, CultureInfo.InvariantCulture));
return;
}
sb.Append(value);
}
private EventSeverity ParseSeverity(string severity)
{
return GatherRuleContext.ParseSeverity(severity);
@@ -245,7 +527,8 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Gather
/// <summary>
/// Blocks until all startup rules that were queued by the most recent <see cref="UpdateRules"/>
/// call have finished executing, or the timeout elapses.
/// call have finished executing, or the timeout elapses. Deferred scoped startup rules
/// (waiting for their phase scope) are not part of the latch.
/// Returns true if all rules completed within the timeout; false if timed out.
/// </summary>
public bool WaitForStartupRules(int timeoutSeconds = 120)
@@ -136,6 +136,25 @@ namespace AutopilotMonitor.Agent.V2.Core.Monitoring.Telemetry.Security
}
watcher.Start();
// Re-check after arming: Dispose() may have run between the lock release above and
// Start() — it then stopped a not-yet-armed watcher and this thread armed it right
// back, leaving a WMI subscription no code path could ever reach again. If disposal
// won that race, tear the watcher down here.
bool disposedDuringArm;
lock (_lock)
{
disposedDuringArm = _disposed;
if (disposedDuringArm) _startWatcher = null;
}
if (disposedDuringArm)
{
try { watcher.EventArrived -= OnProcessStartTrace; } catch { }
try { watcher.Stop(); } catch { }
try { watcher.Dispose(); } catch { }
return;
}
_logger.Info($"[ConsoleBypassWatcher] watching Win32_ProcessStartTrace for " +
$"{string.Join(", ", WatchedProcessNames)} (interactive-session + bare command line)");
}
@@ -335,6 +335,23 @@ namespace AutopilotMonitor.Agent.V2.Core.Orchestration
osBuildChangedProvider: () => osBuildChangeHost.BuildChanged));
}
// MDM reboot-policy watcher — subscribes to DeviceManagement-Enterprise-Diagnostics-
// Provider/Admin EventID 2800 and attributes the "unexpected reboot + second sign-in"
// pattern to the device-assigned policy URIs that forced the coalesced reboot.
// Agent-native collector (NOT an IAgentAnalyzer); the config flag is an internal
// steering lever only.
if (collectors.MdmRebootPolicyWatcherEnabled)
{
hosts.Add(new MdmRebootPolicyWatcherHost(
sessionId: sessionId,
tenantId: tenantId,
logger: logger,
ingress: ingress,
clock: clock,
backfillLookbackMinutes: collectors.MdmRebootPolicyBackfillLookbackMinutes,
stateDirectory: _stateDirectory));
}
// ----- Peripheral hosts (event-only; driven by remote-config toggles) --------------
// V1 parity (PeriodicCollectorManager) — combine Performance + AgentSelfMetrics under
@@ -0,0 +1,68 @@
#nullable enable
using System;
using System.Threading;
using AutopilotMonitor.Agent.V2.Core.Logging;
using AutopilotMonitor.Agent.V2.Core.Monitoring.Enrollment.SystemSignals;
using AutopilotMonitor.DecisionCore.Engine;
using AutopilotMonitor.DecisionCore.Signals;
namespace AutopilotMonitor.Agent.V2.Core.Orchestration
{
/// <summary>
/// Standalone host for the <see cref="MdmRebootPolicyTracker"/> — watches
/// <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> (EventID
/// 2800) for MDM policies that force a coalesced reboot during ESP DeviceSetup and forwards
/// them via the shared single-rail ingress. Mirrors <see cref="WindowsUpdateWatcherHost"/>:
/// builds its own <see cref="InformationalEventPost"/> and delegates Start/Stop to the tracker.
/// </summary>
internal sealed class MdmRebootPolicyWatcherHost : ICollectorHost
{
public string Name => "MdmRebootPolicyTracker";
private readonly MdmRebootPolicyTracker _tracker;
private readonly AgentLogger _logger;
private int _disposed;
public MdmRebootPolicyWatcherHost(
string sessionId,
string tenantId,
AgentLogger logger,
ISignalIngressSink ingress,
IClock clock,
int backfillLookbackMinutes,
string stateDirectory)
{
if (ingress == null) throw new ArgumentNullException(nameof(ingress));
if (clock == null) throw new ArgumentNullException(nameof(clock));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
var post = new InformationalEventPost(ingress, clock, logger);
_tracker = new MdmRebootPolicyTracker(
sessionId: sessionId,
tenantId: tenantId,
post: post,
logger: logger,
backfillEnabled: backfillLookbackMinutes > 0,
backfillLookbackMinutes: backfillLookbackMinutes,
stateDirectory: stateDirectory);
}
public void Start()
{
_tracker.Start();
_logger.Info("MdmRebootPolicyWatcherHost: started.");
}
public void Stop()
{
try { _tracker.Stop(); }
catch (Exception ex) { _logger.Warning($"MdmRebootPolicyWatcherHost: stop failed: {ex.Message}"); }
}
public void Dispose()
{
if (Interlocked.Exchange(ref _disposed, 1) == 1) return;
Stop();
}
}
}
@@ -224,6 +224,11 @@ namespace AutopilotMonitor.Agent.V2
// Step 4 — run the executor and wait.
using (var executor = new GatherRuleExecutor(sessionId, tenantId, emitEvent, logger, config.ImeLogPathOverride))
{
// Diagnostic mode has no enrollment phase context — scoped rules
// (activePhases / activeFromPhase) execute unconditionally here.
executor.IgnorePhaseScope = true;
logger.Info("--run-gather-rules: phase scoping bypassed (no phase context in diagnostic mode).");
executor.UpdateRules(rules);
if (consoleMode) Console.WriteLine($"Running {startupRules.Count} rule(s)...");
@@ -213,6 +213,8 @@ namespace AutopilotMonitor.Agent.V2
// Control-channel kill-switch: a live-fetch DeviceKillSignal terminates the agent
// BEFORE mTLS/registration spin-up. Covers agents the telemetry-channel kill can
// never reach (paused drain, empty spool) at their next process start / boot.
// Checked BEFORE any endpoint migration — a kill from the current backend is
// terminal, not re-homed.
if (CheckConfigKillSignal(
runtimeConfig.RemoteConfig,
runtimeConfig.RemoteConfigService.LastFetchOutcome,
@@ -223,6 +225,37 @@ namespace AutopilotMonitor.Agent.V2
return 0;
}
// Endpoint migration (config-channel re-home): the backend served a validated
// MigrateToApiBaseUrl on a live fetch. Rebuild Phase 3 clients against the new base
// URL and re-run Phase 4 there (single hop — the second Resolve cannot migrate
// again), then re-check the kill signal against the NEW backend's config.
if (runtimeConfig.PendingMigrationTarget != null)
{
logger.Warning(
$"Endpoint migration: backend re-homed this agent from {bootstrap.AgentConfig.ApiBaseUrl} " +
$"to {runtimeConfig.PendingMigrationTarget} — rebuilding backend clients.");
bootstrap.AgentConfig.ApiBaseUrl = runtimeConfig.PendingMigrationTarget;
// Best-effort disposal of the old-URL client stack before it goes out of scope.
try { auth.BackendApiClient?.Dispose(); } catch { }
auth = Runtime.BackendClientFactory.BuildAuthClients(bootstrap.AgentConfig, GetAgentVersion(), logger);
runtimeConfig = Runtime.AgentRuntimeConfig.Resolve(
bootstrap.AgentConfig, auth, GetAgentVersion(), consoleMode, logger,
allowEndpointMigration: false);
if (CheckConfigKillSignal(
runtimeConfig.RemoteConfig,
runtimeConfig.RemoteConfigService.LastFetchOutcome,
dataDirectory, stateSubdir,
() => new CleanupService(bootstrap.AgentConfig, logger),
logger, consoleMode))
{
return 0;
}
}
// Phase 5 (mTLS HttpClient + BackendTelemetryUploader) is encapsulated in
// BackendClientFactory. Construction failures map to V1-parity exit codes
// (4 = mTLS, 5 = uploader). See Runtime/BackendClientFactory.cs.
@@ -32,7 +32,8 @@ namespace AutopilotMonitor.Agent.V2.Runtime
BackendAuthBundle auth,
string agentVersion,
bool consoleMode,
AgentLogger logger)
AgentLogger logger,
bool allowEndpointMigration = true)
{
if (agentConfig == null) throw new ArgumentNullException(nameof(agentConfig));
if (auth == null) throw new ArgumentNullException(nameof(auth));
@@ -49,6 +50,25 @@ namespace AutopilotMonitor.Agent.V2.Runtime
.FetchConfigAsync(retryOnTransientErrors: true)
.GetAwaiter().GetResult();
// Endpoint migration (config-channel re-home): decided BEFORE merge / binary-
// integrity / bootstrap-cleanup so none of those side effects run against a config
// from the backend being abandoned — a migration stub may advertise stale agent
// hashes, and a spurious IntegrityCheckFailed + force-update against them would be
// actively harmful. RunAgent rebuilds Phase 3 clients on the new base URL and calls
// Resolve again with allowEndpointMigration:false (single hop — the new backend's
// config cannot chain another migration).
if (allowEndpointMigration)
{
var migrationTarget = EndpointMigration.ResolveTarget(
remoteConfig, remoteConfigService.LastFetchOutcome, agentConfig.ApiBaseUrl, logger);
if (migrationTarget != null)
{
return new RuntimeConfigBundle(
remoteConfigService, remoteConfig, mergeResult: null,
pendingMigrationTarget: migrationTarget);
}
}
// Project remote tenant-controlled knobs onto the runtime AgentConfiguration so that
// downstream consumers (CleanupService, SummaryDialogLauncher, StartupEnvironmentProbes,
// DiagnosticsPackageService, EnrollmentTerminationHandler, the logger, the watchdog)
@@ -71,7 +91,7 @@ namespace AutopilotMonitor.Agent.V2.Runtime
TryCleanupBootstrapConfig(agentConfig, agentVersion, logger);
return new RuntimeConfigBundle(remoteConfigService, remoteConfig, configMergeResult);
return new RuntimeConfigBundle(remoteConfigService, remoteConfig, configMergeResult, pendingMigrationTarget: null);
}
/// <summary>
@@ -145,21 +165,30 @@ namespace AutopilotMonitor.Agent.V2.Runtime
/// <c>rotate_config</c> ServerAction callback), the fetched <see cref="AgentConfigResponse"/>
/// and the <see cref="RemoteConfigMergeResult"/> diff snapshot (used by the
/// <c>EmitUnrestrictedModeAuditIfChanged</c> lifecycle hook).
/// <para>
/// When <see cref="PendingMigrationTarget"/> is non-null the bundle is a partial result:
/// the fetch found a live endpoint-migration signal and Resolve returned before merge /
/// integrity / bootstrap-cleanup (<see cref="MergeResult"/> is null). RunAgent must
/// rebuild the backend clients on the target URL and Resolve again.
/// </para>
/// </summary>
internal sealed class RuntimeConfigBundle
{
public RemoteConfigService RemoteConfigService { get; }
public AgentConfigResponse RemoteConfig { get; }
public RemoteConfigMergeResult MergeResult { get; }
public string PendingMigrationTarget { get; }
public RuntimeConfigBundle(
RemoteConfigService remoteConfigService,
AgentConfigResponse remoteConfig,
RemoteConfigMergeResult mergeResult)
RemoteConfigMergeResult mergeResult,
string pendingMigrationTarget = null)
{
RemoteConfigService = remoteConfigService;
RemoteConfig = remoteConfig;
MergeResult = mergeResult;
PendingMigrationTarget = pendingMigrationTarget;
}
}
}
@@ -2,11 +2,11 @@
<PropertyGroup>
<!--
Plan §4 M2: xUnit, .NET-8-Testrunner gegen netstandard2.0-Lib.
DecisionCore itself targets netstandard2.0, but the test runner uses .NET 8
so we benefit from modern xUnit + coverlet tooling.
Plan §4 M2: xUnit test runner against the netstandard2.0 lib.
DecisionCore itself targets netstandard2.0; the runner tracks the
repo-wide .NET version (global.json) for modern xUnit + coverlet tooling.
-->
<TargetFramework>net8.0</TargetFramework>
<TargetFramework>net10.0</TargetFramework>
<LangVersion>latest</LangVersion>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
@@ -0,0 +1,160 @@
using AutopilotMonitor.Functions.Functions.Config;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Services;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the endpoint-migration serving contract: the shared allowlist rules
/// (<see cref="AgentEndpointMigrationRules"/>, used verbatim by the agent too) and the
/// backend-side per-tenant resolution (<see cref="GetAgentConfigFunction.ResolveMigrateTarget"/> —
/// tenant override wins, empty override pins, invalid values are never served).
/// </summary>
public class AgentEndpointMigrationTests
{
private const string TenantA = "11111111-2222-3333-4444-555555555555";
private const string TenantB = "66666666-7777-8888-9999-000000000000";
private const string UsUrl = "https://autopilotmonitor-api-us.azurewebsites.net";
private const string EuUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
// ── Shared validation rules ─────────────────────────────────────────────
[Theory]
[InlineData("https://autopilotmonitor-api-us.azurewebsites.net", "https://autopilotmonitor-api-us.azurewebsites.net")]
[InlineData("https://AutopilotMonitor-API-US.AzureWebsites.NET/", "https://autopilotmonitor-api-us.azurewebsites.net")]
[InlineData(" https://x.azurewebsites.net ", "https://x.azurewebsites.net")]
public void TryNormalizeTarget_accepts_and_normalizes(string candidate, string expected)
{
Assert.True(AgentEndpointMigrationRules.TryNormalizeTarget(candidate, out var normalized));
Assert.Equal(expected, normalized);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData(" ")]
[InlineData("http://x.azurewebsites.net")] // https only
[InlineData("https://attacker.example.com")] // not allowlisted
[InlineData("https://evil-azurewebsites.net")] // label-boundary bypass attempt
[InlineData("https://.azurewebsites.net")] // bare suffix
[InlineData("https://x.azurewebsites.net:8443")] // non-default port
[InlineData("https://u:p@x.azurewebsites.net")] // userinfo
[InlineData("https://x.azurewebsites.net/api")] // path
[InlineData("https://x.azurewebsites.net?a=1")] // query
[InlineData("https://x.azurewebsites.net#frag")] // fragment
[InlineData("ftp://x.azurewebsites.net")]
[InlineData("not a url")]
public void TryNormalizeTarget_rejects(string? candidate)
{
Assert.False(AgentEndpointMigrationRules.TryNormalizeTarget(candidate!, out var normalized));
Assert.Null(normalized);
}
[Fact]
public void TryNormalizeTarget_rejects_overlong_candidate()
{
var candidate = "https://" + new string('a', AgentEndpointMigrationRules.MaxTargetLength) + ".azurewebsites.net";
Assert.False(AgentEndpointMigrationRules.TryNormalizeTarget(candidate, out _));
}
[Fact]
public void IsEffectiveMigration_rejects_target_equal_to_current()
{
Assert.False(AgentEndpointMigrationRules.IsEffectiveMigration(EuUrl, EuUrl + "/", out var target));
Assert.Null(target);
Assert.True(AgentEndpointMigrationRules.IsEffectiveMigration(UsUrl, EuUrl, out target));
Assert.Equal(UsUrl, target);
}
// ── Backend resolution: global vs per-tenant override ──────────────────
[Fact]
public void No_configuration_resolves_to_null()
{
var config = new AdminConfiguration();
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Null(rejected);
}
[Fact]
public void Global_target_applies_to_every_tenant()
{
var config = new AdminConfiguration { AgentMigrateApiBaseUrl = UsUrl };
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Tenant_override_wins_over_global()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = $"{{\"{TenantA}\": \"{EuUrl}\"}}"
};
Assert.Equal(EuUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Empty_tenant_override_pins_tenant_against_global_target()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = $"{{\"{TenantA}\": \"\"}}"
};
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Null(rejected); // pinned, not rejected
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantB, out _));
}
[Fact]
public void Tenant_override_key_is_case_insensitive()
{
var config = new AdminConfiguration
{
AgentMigrateTenantOverridesJson = $"{{\"{TenantA.ToUpperInvariant()}\": \"{UsUrl}\"}}"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
[Fact]
public void Invalid_configured_target_is_never_served_and_reported_as_rejected()
{
var config = new AdminConfiguration { AgentMigrateApiBaseUrl = "https://attacker.example.com" };
Assert.Null(GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out var rejected));
Assert.Equal("https://attacker.example.com", rejected);
}
[Fact]
public void Malformed_overrides_json_degrades_to_global_target()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = UsUrl,
AgentMigrateTenantOverridesJson = "{not json"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
[Fact]
public void Served_target_is_normalized()
{
var config = new AdminConfiguration
{
AgentMigrateApiBaseUrl = "https://AutopilotMonitor-API-US.azurewebsites.net/"
};
Assert.Equal(UsUrl, GetAgentConfigFunction.ResolveMigrateTarget(config, TenantA, out _));
}
}
@@ -0,0 +1,83 @@
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey → business-time decoders. These are the read-path compensation for rows whose
/// system Timestamp was reset by the 2026-07-18 storage migration: the RowKey is the only
/// time carrier that survives a row rewrite, so the decode must round-trip exactly and
/// must reject every foreign format (falling back instead of mis-dating rows).
/// </summary>
public class BusinessTimestampDecodeTests
{
private static readonly DateTime Ts = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc).AddTicks(4567);
[Fact]
public void Audit_rowkey_roundtrips_tick_exact()
{
var rowKey = TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid());
Assert.True(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("!")]
[InlineData("!123")] // too short
[InlineData("!123456789012345678x_guid")] // non-digit in revtick
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // legacy bare GUID (:N)
[InlineData("0000000000000000000")] // ops-style bare revtick is not an audit key
public void Audit_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeAuditRowKey(rowKey, out _));
}
[Fact]
public void Ops_rowkey_roundtrips_tick_exact()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
Assert.True(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out var decoded));
Assert.Equal(Ts, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("123456789012345678")] // 18 digits
[InlineData("12345678901234567890")] // 20 digits
[InlineData("123456789012345678x")] // non-digit
[InlineData("!1234567890123456789")] // audit-prefixed
public void Ops_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeOpsRowKey(rowKey, out _));
}
[Fact]
public void Event_rowkey_prefix_decodes_to_millisecond_truncated_agent_time()
{
var rowKey = $"{Ts:yyyyMMddHHmmssfff}_{42L:D10}";
Assert.True(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out var decoded));
// The RowKey format truncates sub-millisecond ticks at write time.
var expected = new DateTime(2026, 6, 15, 10, 30, 45, 123, DateTimeKind.Utc);
Assert.Equal(expected, decoded);
Assert.Equal(DateTimeKind.Utc, decoded.Kind);
}
[Theory]
[InlineData(null)]
[InlineData("")]
[InlineData("20260615103045123")] // no separator/sequence
[InlineData("2026061510304512_0000000042")] // 16-digit prefix
[InlineData("202606151030451234_0000000042")] // 18-digit prefix
[InlineData("2026061510304512x_0000000042")] // non-digit in prefix
[InlineData("20261315103045123_0000000042")] // month 13
[InlineData("d6f6dd90f41c4b16b290aac042e5b466")] // GUID
public void Event_decoder_rejects_foreign_formats(string? rowKey)
{
Assert.False(BusinessTimestamp.TryDecodeEventRowKeyPrefix(rowKey, out _));
}
}
@@ -0,0 +1,181 @@
using System.Text.RegularExpressions;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// RowKey-range date-window / retention clauses. Date filtering moved off the system
/// Timestamp (reset by storage migrations) onto the reverse-tick RowKey; these tests pin
/// (a) tick-exact inclusive/exclusive boundary semantics identical to the old
/// `Timestamp ge/le` clauses, and (b) the GUID-safety invariant: legacy bare-GUID audit
/// rows sort AFTER all '!'-rows, so any lower RowKey bound without the `RowKey lt '"'`
/// guard would match — and a retention sweep would DELETE — every legacy row.
/// </summary>
public class BusinessTimestampFilterClauseTests
{
private static readonly DateTime T = new DateTime(2026, 6, 15, 12, 0, 0, DateTimeKind.Utc).AddTicks(789);
private static readonly string Tenant = "11111111-1111-1111-1111-111111111111";
private static string AuditRk(DateTime ts) => TableStorageService.BuildAuditLogRowKey(ts, Guid.NewGuid());
private static string OpsRk(DateTime ts) => $"{DateTime.MaxValue.Ticks - ts.Ticks:D19}";
/// <summary>Evaluates a clause of the form "RowKey op 'literal' [and RowKey op 'literal']…" against a RowKey.</summary>
private static bool Matches(string clause, string rowKey)
{
var parts = Regex.Matches(clause, @"RowKey (ge|gt|le|lt) '([^']*)'");
Assert.True(parts.Count > 0, $"no RowKey comparisons in clause: {clause}");
foreach (Match m in parts)
{
var cmp = string.CompareOrdinal(rowKey, m.Groups[2].Value);
var ok = m.Groups[1].Value switch
{
"ge" => cmp >= 0,
"gt" => cmp > 0,
"le" => cmp <= 0,
"lt" => cmp < 0,
_ => throw new InvalidOperationException(),
};
if (!ok) return false;
}
return true;
}
// ===== Audit boundaries =====
[Fact]
public void Audit_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateFromClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == from → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → excluded
}
[Fact]
public void Audit_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.AuditDateToClause(T);
Assert.True(Matches(clause, AuditRk(T))); // ts == to → included
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // older → included
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // 1 tick newer → excluded
}
[Fact]
public void Audit_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.AuditRetentionClause(T);
Assert.False(Matches(clause, AuditRk(T))); // ts == cutoff → kept
Assert.True(Matches(clause, AuditRk(T.AddTicks(-1)))); // 1 tick older → deleted
Assert.False(Matches(clause, AuditRk(T.AddTicks(1)))); // newer → kept
}
// ===== Audit GUID safety (the single most dangerous invariant of this change) =====
[Fact]
public void Legacy_guid_rowkeys_sort_after_the_time_encoded_guard()
{
for (var i = 0; i < 20; i++)
{
var guidRk = Guid.NewGuid().ToString("N");
Assert.True(string.CompareOrdinal(guidRk, BusinessTimestamp.AuditTimeEncodedUpperBound) > 0);
}
}
[Fact]
public void No_audit_date_or_retention_clause_matches_legacy_guid_rowkeys()
{
var clauses = new[]
{
BusinessTimestamp.AuditDateFromClause(T),
BusinessTimestamp.AuditDateToClause(T),
BusinessTimestamp.AuditDateToClause(DateTime.MaxValue.AddTicks(-1)), // widest possible window
BusinessTimestamp.AuditRetentionClause(T),
BusinessTimestamp.AuditRetentionClause(DateTime.MaxValue.AddTicks(-1)), // widest possible sweep
};
foreach (var clause in clauses)
{
for (var i = 0; i < 20; i++)
{
Assert.False(Matches(clause, Guid.NewGuid().ToString("N")),
$"clause matched a legacy GUID RowKey: {clause}");
}
}
}
[Fact]
public void Audit_lower_bounded_clauses_carry_the_time_encoded_guard()
{
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditDateToClause(T));
Assert.Contains($"RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'",
BusinessTimestamp.AuditRetentionClause(T));
}
// ===== Ops boundaries =====
[Fact]
public void Ops_dateFrom_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateFromClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(-1))));
}
[Fact]
public void Ops_dateTo_is_inclusive_to_the_tick()
{
var clause = BusinessTimestamp.OpsDateToClause(T);
Assert.True(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
[Fact]
public void Ops_retention_is_strictly_older_than_cutoff()
{
var clause = BusinessTimestamp.OpsRetentionClause(T);
Assert.False(Matches(clause, OpsRk(T)));
Assert.True(Matches(clause, OpsRk(T.AddTicks(-1))));
Assert.False(Matches(clause, OpsRk(T.AddTicks(1))));
}
// ===== Composition into the filter builders =====
[Fact]
public void Audit_filter_builders_emit_rowkey_windows_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var plain = TableStorageService.BuildAuditLogFilter(Tenant, from, to);
var fanOut = TableStorageService.BuildAuditLogFilterWithRowKeyBound(
Tenant, from, to, lastRowKey: "!0123_x", excludeDeletions: false);
foreach (var f in new[] { plain!, fanOut })
{
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.AuditDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.AuditDateToClause(to), f);
}
Assert.Contains("RowKey gt '!0123_x'", fanOut); // fan-out cursor composes with the window
}
[Fact]
public void Ops_filter_builder_emits_rowkey_window_not_timestamp_clauses()
{
var from = new DateTime(2026, 6, 1, 0, 0, 0, DateTimeKind.Utc);
var to = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc);
var f = TableOpsEventRepository.BuildFilter("Security", from, to);
Assert.NotNull(f);
Assert.DoesNotContain("Timestamp ge", f);
Assert.DoesNotContain("Timestamp le", f);
Assert.Contains(BusinessTimestamp.OpsDateFromClause(from), f);
Assert.Contains(BusinessTimestamp.OpsDateToClause(to), f);
}
}
@@ -0,0 +1,118 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.DataAccess.TableStorage;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Read-mapper resolution order for the three affected tables:
/// OccurredUtc column → RowKey decode → system Timestamp. The last hop is the
/// migration-corrupted value (a row rewrite resets it), so it must only ever win when
/// the row carries no better source (audit legacy bare-GUID rows).
/// </summary>
public class BusinessTimestampPrecedenceTests
{
private static readonly DateTime Occurred = new(2026, 5, 10, 8, 0, 0, DateTimeKind.Utc);
private static readonly DateTime RowKeyTime = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
private static readonly DateTimeOffset SystemTs = new(2026, 7, 18, 13, 58, 0, TimeSpan.Zero); // migration moment
private static TableEntity WithSystemTimestamp(TableEntity e)
{
e["Timestamp"] = SystemTs;
return e;
}
// ===== Audit =====
[Fact]
public void Audit_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("t", TableStorageService.BuildAuditLogRowKey(RowKeyTime, Guid.NewGuid())));
Assert.Equal(RowKeyTime, TableStorageService.ResolveAuditTimestamp(e));
}
[Fact]
public void Audit_legacy_guid_row_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("t", Guid.NewGuid().ToString("N")));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveAuditTimestamp(e));
}
// ===== Ops =====
[Fact]
public void Ops_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}")
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_rowkey_decode_wins_over_system_when_column_missing()
{
var e = WithSystemTimestamp(new TableEntity("Security", $"{DateTime.MaxValue.Ticks - RowKeyTime.Ticks:D19}"));
Assert.Equal(RowKeyTime, TableOpsEventRepository.ResolveTimestamp(e));
}
[Fact]
public void Ops_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("Security", "not-a-revtick"));
Assert.Equal(SystemTs.UtcDateTime, TableOpsEventRepository.ResolveTimestamp(e));
}
// ===== Events =====
private static string EventRowKey(DateTime ts, long seq = 42) => $"{ts:yyyyMMddHHmmssfff}_{seq:D10}";
[Fact]
public void Event_occurredutc_wins_over_rowkey_and_system()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Occurred),
});
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_rowkey_prefix_wins_over_system_when_column_missing()
{
// The pre-cutover case: no OccurredUtc column, system Timestamp = migration moment.
var e = WithSystemTimestamp(new TableEntity("tenant_session", EventRowKey(RowKeyTime)));
Assert.Equal(RowKeyTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_undecodable_rowkey_falls_back_to_system_timestamp()
{
var e = WithSystemTimestamp(new TableEntity("tenant_session", "weird-rowkey"));
Assert.Equal(SystemTs.UtcDateTime, TableStorageService.ResolveEventTimestamp(e));
}
[Fact]
public void Event_occurredutc_read_tolerates_datetime_materialization()
{
// SDK reads surface Edm.DateTime as DateTimeOffset or DateTime depending on payload
// shape — the resolver must accept both (dual-reader lesson).
var e = new TableEntity("tenant_session", EventRowKey(RowKeyTime))
{
[BusinessTimestamp.OccurredUtcColumn] = Occurred, // plain DateTime, not DateTimeOffset
};
Assert.Equal(Occurred, TableStorageService.ResolveEventTimestamp(e));
}
}
@@ -0,0 +1,283 @@
using AutopilotMonitor.Functions.Functions.Rules;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using Azure;
using Azure.Data.Tables;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the three gather-rule scope/emit fields (ActivePhases, ActiveFromPhase, EmitMode):
/// Store↔Map table-serialization roundtrip (table-serialization rule: every model field in BOTH
/// directions), legacy rows without the new columns, the <see cref="GatherRuleService.ContentEquivalent"/>
/// null-tolerance (no reseed churn), and the create/update validation matrix in
/// <see cref="GatherRulesFunction.ValidateScopeAndEmitMode"/>.
/// </summary>
public class GatherRuleScopeFieldsTests
{
// ── Store ↔ Map roundtrip ──────────────────────────────────────────────
private static (TableStorageService service, Func<TableEntity?> lastUpserted) BuildStorageHarness()
{
TableEntity? captured = null;
var tableClient = new Mock<TableClient>();
tableClient
.Setup(c => c.UpsertEntityAsync(
It.IsAny<TableEntity>(), It.IsAny<TableUpdateMode>(), It.IsAny<CancellationToken>()))
.Returns<TableEntity, TableUpdateMode, CancellationToken>((e, _, _) =>
{
captured = e;
return Task.FromResult(new Mock<Response>().Object);
});
var serviceClient = new Mock<TableServiceClient>();
serviceClient.Setup(c => c.GetTableClient(It.IsAny<string>())).Returns(tableClient.Object);
var service = new TableStorageService(serviceClient.Object, NullLogger<TableStorageService>.Instance);
return (service, () => captured);
}
[Fact]
public async Task Roundtrip_ScopeFields_SurviveStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-001",
Title = "Scoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_scoped",
ActivePhases = new List<string> { "AccountSetup", "AppsUser" },
EmitMode = "on_change",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.NotNull(mapped.ActivePhases);
Assert.Equal(new[] { "AccountSetup", "AppsUser" }, mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Equal("on_change", mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_FromPhase_SurvivesStoreAndMap()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-002",
Title = "From-phase rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_scoped",
ActiveFromPhase = "AccountSetup",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Equal("AccountSetup", mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public async Task Roundtrip_NullScopeFields_StayNull()
{
var (service, lastUpserted) = BuildStorageHarness();
var rule = new GatherRule
{
RuleId = "GATHER-SCOPE-003",
Title = "Unscoped rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "startup",
OutputEventType = "gather_plain",
};
Assert.True(await service.StoreGatherRuleAsync(rule, "global"));
var mapped = service.MapToGatherRule(lastUpserted()!);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
[Fact]
public void Map_LegacyRow_WithoutScopeColumns_DefaultsToNull()
{
var (service, _) = BuildStorageHarness();
// A row written before the scope fields existed: none of the new columns present.
var legacy = new TableEntity("global", "GATHER-LEGACY-001")
{
["Title"] = "Legacy rule",
["CollectorType"] = "registry",
["Target"] = "HKLM\\SOFTWARE\\Contoso",
["Trigger"] = "startup",
["OutputEventType"] = "gather_legacy",
};
var mapped = service.MapToGatherRule(legacy);
Assert.Null(mapped.ActivePhases);
Assert.Null(mapped.ActiveFromPhase);
Assert.Null(mapped.EmitMode);
}
// ── ContentEquivalent ──────────────────────────────────────────────────
private static GatherRule BaseRule() => new()
{
RuleId = "GATHER-EQ-001",
Title = "Rule",
Description = "Desc",
Version = "1.0.0",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
TriggerPhase = "",
OutputEventType = "gather_eq",
};
[Fact]
public void ContentEquivalent_NullAndEmptyActivePhases_AreEquivalent()
{
// Pre-existing DB rows map absent columns to null; seeds are also null — and an
// explicitly-empty list means the same "unrestricted". No reseed churn allowed.
var a = BaseRule();
var b = BaseRule();
b.ActivePhases = new List<string>();
Assert.True(GatherRuleService.ContentEquivalent(a, b));
Assert.True(GatherRuleService.ContentEquivalent(b, a));
}
[Fact]
public void ContentEquivalent_DetectsChangesInEachScopeField()
{
var baseline = BaseRule();
var phases = BaseRule();
phases.ActivePhases = new List<string> { "AccountSetup" };
Assert.False(GatherRuleService.ContentEquivalent(baseline, phases));
var from = BaseRule();
from.ActiveFromPhase = "AccountSetup";
Assert.False(GatherRuleService.ContentEquivalent(baseline, from));
var emit = BaseRule();
emit.EmitMode = "on_change";
Assert.False(GatherRuleService.ContentEquivalent(baseline, emit));
}
[Fact]
public void ContentEquivalent_SameScopeFields_AreEquivalent()
{
var a = BaseRule();
a.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
a.EmitMode = "on_change";
var b = BaseRule();
b.ActivePhases = new List<string> { "DeviceSetup", "AccountSetup" };
b.EmitMode = "on_change";
Assert.True(GatherRuleService.ContentEquivalent(a, b));
}
// ── Validation ─────────────────────────────────────────────────────────
private static GatherRule ValidationRule(
List<string>? activePhases = null, string? activeFromPhase = null, string? emitMode = null) => new()
{
RuleId = "GATHER-VAL-001",
Title = "Rule",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\Contoso",
Trigger = "interval",
OutputEventType = "gather_val",
ActivePhases = activePhases,
ActiveFromPhase = activeFromPhase,
EmitMode = emitMode,
};
[Theory]
[InlineData(null)]
[InlineData("Start")]
[InlineData("DevicePreparation")]
[InlineData("AccountSetup")]
[InlineData("Complete")]
[InlineData("accountsetup")] // case-insensitive
public void Validate_AcceptsValidFromPhase(string? fromPhase)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Theory]
[InlineData("Unknown")]
[InlineData("Failed")]
[InlineData("4")] // numeric enum value — canonical names only
[InlineData("DeviceESP")] // the invalid token the old free-text UI suggested
public void Validate_RejectsInvalidFromPhase(string fromPhase)
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activeFromPhase: fromPhase)));
}
[Fact]
public void Validate_AcceptsValidActivePhases_AndRejectsInvalidEntries()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "AppsDevice" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "DeviceSetup", "Failed" })));
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(
ValidationRule(activePhases: new List<string> { "" })));
}
[Fact]
public void Validate_RejectsBothScopeFieldsSet()
{
var error = GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(
activePhases: new List<string> { "AccountSetup" }, activeFromPhase: "DeviceSetup"));
Assert.NotNull(error);
Assert.Contains("mutually exclusive", error);
}
[Theory]
[InlineData(null)]
[InlineData("always")]
[InlineData("on_change")]
[InlineData("ON_CHANGE")] // case-insensitive
public void Validate_AcceptsValidEmitModes(string? emitMode)
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: emitMode)));
}
[Fact]
public void Validate_RejectsUnknownEmitMode()
{
Assert.NotNull(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(emitMode: "sometimes")));
}
[Fact]
public void Validate_TogglePartialPayload_PassesThrough()
{
// The portal toggle sends only { enabled, isBuiltIn, isCommunity } — none of the
// scope fields — and must not be rejected.
var toggle = new GatherRule { RuleId = "GATHER-VAL-002", Enabled = false };
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(toggle));
}
[Fact]
public void Validate_EmptyActivePhasesList_IsUnrestricted_AndPasses()
{
Assert.Null(GatherRulesFunction.ValidateScopeAndEmitMode(ValidationRule(activePhases: new List<string>())));
}
}
@@ -0,0 +1,146 @@
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.DataAccess;
using AutopilotMonitor.Shared.Models;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pins the custom-rule partial-PUT fix on <see cref="GatherRuleService.UpdateRuleAsync"/>:
/// the portal toggle PUTs only { enabled, isBuiltIn, isCommunity }, and full-storing that
/// payload wiped Title/Target/Trigger/Parameters of the custom rule. Toggle-style partials
/// (empty Title/CollectorType/Target) must merge Enabled into the existing row; full payloads
/// (edit flow) must still replace the row but preserve the original CreatedAt.
/// </summary>
public class GatherRuleUpdatePartialMergeTests
{
private const string TenantId = "a1b2c3d4-e5f6-7890-abcd-ef1234567890";
private const string RuleId = "GATHER-CUST-001";
private static GatherRule ExistingCustomRule() => new()
{
RuleId = RuleId,
Title = "Collect BIOS Config",
Description = "Reads the RealmJoin BIOS key",
Category = "device",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS",
Parameters = new Dictionary<string, string> { ["valueName"] = "Version" },
Trigger = "interval",
IntervalSeconds = 60,
OutputEventType = "gather_bios_config",
Enabled = true,
IsBuiltIn = false,
IsCommunity = false,
CreatedAt = new DateTime(2026, 1, 15, 12, 0, 0, DateTimeKind.Utc),
ActiveFromPhase = "AccountSetup",
EmitMode = "on_change",
};
private static (GatherRuleService service, Mock<IRuleRepository> repo, List<GatherRule> stored) BuildService(
GatherRule? existingTenantRule)
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
var stored = new List<GatherRule>();
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>());
repo.Setup(r => r.GetGatherRulesAsync(TenantId)).ReturnsAsync(
existingTenantRule == null ? new List<GatherRule>() : new List<GatherRule> { existingTenantRule });
repo.Setup(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), TenantId))
.Callback<GatherRule, string>((rule, _) => stored.Add(rule))
.ReturnsAsync(true);
return (new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance), repo, stored);
}
[Fact]
public async Task TogglePartial_merges_enabled_and_preserves_definition_fields()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
// Exactly what the portal toggle sends: { enabled, isBuiltIn, isCommunity }.
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.False(written.Enabled);
Assert.Equal("Collect BIOS Config", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS", written.Target);
Assert.Equal("registry", written.CollectorType);
Assert.Equal("interval", written.Trigger);
Assert.Equal(60, written.IntervalSeconds);
Assert.Equal("Version", written.Parameters["valueName"]);
Assert.Equal("AccountSetup", written.ActiveFromPhase);
Assert.Equal("on_change", written.EmitMode);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task TogglePartial_for_unknown_custom_rule_fails_without_storing()
{
var (service, repo, stored) = BuildService(existingTenantRule: null);
var togglePayload = new GatherRule { RuleId = RuleId, Enabled = false, IsBuiltIn = false, IsCommunity = false };
var ok = await service.UpdateRuleAsync(TenantId, togglePayload);
Assert.False(ok);
Assert.Empty(stored);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
[Fact]
public async Task FullPayload_replaces_rule_but_preserves_original_CreatedAt()
{
var existing = ExistingCustomRule();
var (service, _, stored) = BuildService(existing);
var fullPayload = new GatherRule
{
RuleId = RuleId,
Title = "Collect BIOS Config v2",
CollectorType = "registry",
Target = "HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2",
Trigger = "interval",
IntervalSeconds = 120,
OutputEventType = "gather_bios_config",
Enabled = true,
// Edit flow sends createdAt from the client; a JSON-mode edit may omit it and
// default to UtcNow — either way the stored row must keep the original.
CreatedAt = DateTime.UtcNow,
};
var ok = await service.UpdateRuleAsync(TenantId, fullPayload);
Assert.True(ok);
var written = Assert.Single(stored);
Assert.Equal("Collect BIOS Config v2", written.Title);
Assert.Equal("HKLM\\SOFTWARE\\RealmJoin\\Custom\\BIOS2", written.Target);
Assert.Equal(120, written.IntervalSeconds);
Assert.False(written.IsBuiltIn);
Assert.False(written.IsCommunity);
Assert.Equal(existing.CreatedAt, written.CreatedAt);
}
[Fact]
public async Task BuiltInRule_toggle_still_goes_through_RuleState_path()
{
var repo = new Mock<IRuleRepository>(MockBehavior.Loose);
repo.Setup(r => r.GetGatherRulesAsync("global")).ReturnsAsync(new List<GatherRule>
{
new() { RuleId = RuleId, IsBuiltIn = true, Title = "Built-in", Enabled = true }
});
repo.Setup(r => r.StoreRuleStateAsync(TenantId, RuleId, It.IsAny<RuleState>())).ReturnsAsync(true);
var service = new GatherRuleService(repo.Object, NullLogger<GatherRuleService>.Instance);
var ok = await service.UpdateRuleAsync(TenantId, new GatherRule { RuleId = RuleId, Enabled = false });
Assert.True(ok);
repo.Verify(r => r.StoreRuleStateAsync(TenantId, RuleId, It.Is<RuleState>(s => !s.Enabled)), Times.Once);
repo.Verify(r => r.StoreGatherRuleAsync(It.IsAny<GatherRule>(), It.IsAny<string>()), Times.Never);
}
}
@@ -129,4 +129,30 @@ public class GetAllTenantConfigurationsFunctionTests
[Fact]
public void DelegatedBareArrayView_Empty_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.DelegatedBareArrayView(new TenantConfiguration[0]));
[Fact]
public void MissingManagedIds_ReturnsOnlyIdsWithoutConfig_CaseInsensitive()
{
var configs = new List<TenantConfiguration>
{
new() { TenantId = TenantB.ToUpperInvariant() }, // case variant still counts as present
};
var missing = GetAllTenantConfigurationsFunction.MissingManagedIds(new[] { TenantB, TenantC }, configs);
var only = Assert.Single(missing);
Assert.Equal(TenantC, only);
}
[Fact]
public void MissingManagedIds_AllPresent_Empty()
=> Assert.Empty(GetAllTenantConfigurationsFunction.MissingManagedIds(
new[] { TenantB },
new List<TenantConfiguration> { new() { TenantId = TenantB } }));
[Fact]
public void MissingConfigCacheKey_LowercasesId_SoCaseVariantsShareOneVerdict()
=> Assert.Equal(
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB),
GetAllTenantConfigurationsFunction.MissingConfigCacheKey(TenantB.ToUpperInvariant()));
}
@@ -0,0 +1,107 @@
using System.Collections.Generic;
using System.Net;
using System.Threading.Tasks;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Functions.Tests.GraphResolution;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Service-level tests for the Agent Binaries HealthCheck card. The check probes the
/// canonical download alias (download.autopilotmonitor.com) AND the legacy blob keepalive
/// account: alias failure is unhealthy (new installs break), legacy failure is only a
/// warning (existing customers break until they migrate to the alias).
/// </summary>
public class HealthCheckServiceAgentBinariesTests
{
private const string AliasHost = "download.autopilotmonitor.com";
private const string LegacyHost = "autopilotmonitor.blob.core.windows.net";
[Fact]
public async Task Check_AllEndpointsAvailable_ReportsHealthyWithoutUrlDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("Agent Binaries", check.Name);
Assert.Equal("healthy", check.Status);
Assert.Contains("download alias", check.Message);
Assert.Null(check.Details);
// Both hosts must actually have been probed (ZIP + PS1 each).
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(AliasHost)).Count);
Assert.Equal(2, handler.Requests.FindAll(u => u.Contains(LegacyHost)).Count);
}
[Fact]
public async Task Check_WithEndpointUrls_SurfacesAliasAndLegacyUrlsInDetails()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: true);
Assert.NotNull(check.Details);
Assert.Equal("https://download.autopilotmonitor.com/agent/AutopilotMonitor-Agent.zip", check.Details!["Agent ZIP"]);
Assert.Equal("https://download.autopilotmonitor.com/agent/Install-AutopilotMonitor.ps1", check.Details["Bootstrap script"]);
Assert.Equal("https://autopilotmonitor.blob.core.windows.net/agent", check.Details["Legacy blob (keepalive)"]);
}
[Fact]
public async Task Check_LegacyBlobMissing_ReportsWarningNamingTheLegacyEndpoint()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.OK, "")
.When(LegacyHost, HttpStatusCode.NotFound, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("warning", check.Status);
Assert.Contains("legacy", check.Message, System.StringComparison.OrdinalIgnoreCase);
Assert.Contains("404", check.Message);
}
[Fact]
public async Task Check_AliasMissing_ReportsUnhealthy()
{
var handler = new StubHttpMessageHandler()
.When(AliasHost, HttpStatusCode.NotFound, "")
.When(LegacyHost, HttpStatusCode.OK, "");
var check = await BuildService(handler).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("download alias", check.Message);
}
[Fact]
public async Task Check_EndpointsUnreachable_ReportsUnhealthy()
{
// Un-scripted URLs make the stub throw — models a connection failure.
var check = await BuildService(new StubHttpMessageHandler()).CheckAgentBinariesAsync(includeEndpointUrls: false);
Assert.Equal("unhealthy", check.Status);
Assert.Contains("unreachable", check.Message);
}
private static HealthCheckService BuildService(StubHttpMessageHandler handler)
{
var config = new ConfigurationBuilder()
.AddInMemoryCollection(new Dictionary<string, string?>())
.Build();
return new HealthCheckService(
NullLogger<HealthCheckService>.Instance,
adminConfigService: null!,
httpClientFactory: new StubHttpClientFactory(handler),
metricsReader: null!,
poisonQueueProbe: null!,
configuration: config);
}
}
@@ -53,6 +53,29 @@ public class HealthCheckServiceMcpServerTests
Assert.DoesNotContain("http", check.Message, System.StringComparison.OrdinalIgnoreCase);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlAlongsideVersion()
{
var svc = BuildService(HttpStatusCode.OK, "{\"status\":\"healthy\",\"version\":\"1.4.0\"}");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
// The Version detail must merge into — not replace — the Server URL detail.
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
Assert.Equal("1.4.0", check.Details["Version"]);
}
[Fact]
public async Task Check_WithEndpointUrl_SurfacesServerUrlEvenWhenUnhealthy()
{
var svc = BuildService(HttpStatusCode.ServiceUnavailable, "");
var check = await svc.CheckMcpServerAsync(includeEndpointUrl: true);
Assert.Equal("unhealthy", check.Status);
Assert.Equal("https://mcp.example.test", check.Details!["Server URL"]);
}
[Fact]
public async Task Check_TimeoutOrConnectionFailure_ReportsWarningForColdStart()
{
@@ -3,7 +3,7 @@ using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Tier-boundary + percent-math tests for the SignalR free-tier quota watcher.
/// Tier-boundary + percent-math tests for the SignalR quota watcher.
/// Mirrors the cert-expiry tests' approach: pure-function coverage of the
/// boundary risks (off-by-one, floor vs ceil, zero-division). Full-service
/// smoke tests would require constructing 16 dependencies; the percent +
@@ -0,0 +1,61 @@
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Per-row backfill decision (pure function): rows that already carry OccurredUtc are
/// skipped (idempotent re-runs), decodable RowKeys yield the deterministic decoded value,
/// anything else is reported undecodable — never guessed.
/// </summary>
public class OccurredUtcBackfillServiceTests
{
private static readonly DateTime Ts = new(2026, 6, 20, 9, 30, 0, DateTimeKind.Utc);
[Fact]
public void Audit_row_with_existing_column_is_skipped()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()))
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Audit_time_encoded_row_yields_decoded_write()
{
var e = new TableEntity("t", TableStorageService.BuildAuditLogRowKey(Ts, Guid.NewGuid()));
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
}
[Fact]
public void Audit_legacy_guid_row_is_undecodable_not_guessed()
{
var e = new TableEntity("t", Guid.NewGuid().ToString("N"));
Assert.Equal(OccurredUtcBackfillService.RowDecision.Undecodable,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableAudit, e, out _));
}
[Fact]
public void Ops_row_yields_decoded_write_and_existing_column_is_skipped()
{
var rowKey = $"{DateTime.MaxValue.Ticks - Ts.Ticks:D19}";
var bare = new TableEntity("Security", rowKey);
var decision = OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, bare, out var occurred);
Assert.Equal(OccurredUtcBackfillService.RowDecision.Write, decision);
Assert.Equal(Ts, occurred);
var backfilled = new TableEntity("Security", rowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = new DateTimeOffset(Ts),
};
Assert.Equal(OccurredUtcBackfillService.RowDecision.AlreadySet,
OccurredUtcBackfillService.DecideRow(OccurredUtcBackfillService.TableOps, backfilled, out _));
}
}
@@ -0,0 +1,269 @@
using AutopilotMonitor.Functions.Functions.Admin;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
using AutopilotMonitor.Shared.DataAccess;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using Moq;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Covers the SignalR revoke-enforcement cores of <see cref="DelegatedAdminManagementFunction"/> and
/// <see cref="TenantGroupManagementFunction"/> (extracted seams — the HTTP entry points are intentionally
/// not exercised, same rationale as <see cref="GetAllBlockedDevicesFunctionTests"/>). Locks in three
/// security invariants: (1) a miss (no assignment row) neither audits nor disconnects — a typo must not
/// write false customer-visible "access removed" rows while the real grant stays live; (2) a real
/// revoke/disable/unassign/group-delete always cuts the affected UPN's live streams (group authorization
/// is join-time only); (3) grant validation is fail-closed (GUID tenantId, recognized roles only).
/// </summary>
public class RevokeEnforcementTests
{
private const string Upn = "MSP-Admin@Partner.Example";
private const string UpnLower = "msp-admin@partner.example";
private const string TenantA = "11111111-1111-1111-1111-111111111111";
private const string TenantB = "22222222-2222-2222-2222-222222222222";
private const string GroupId = "tpl-aaaaaaaa";
private static DelegatedAdminService BuildService(Mock<IAdminRepository> repo) =>
new(
repo.Object,
new StubTenantEntitlementService(AutopilotMonitor.Functions.Security.TenantEdition.Enterprise),
new MemoryCache(new MemoryCacheOptions()),
NullLogger<DelegatedAdminService>.Instance);
private static Mock<IMaintenanceRepository> BuildAuditRepo()
{
var audit = new Mock<IMaintenanceRepository>();
audit.Setup(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()))
.ReturnsAsync(true);
return audit;
}
private static void VerifyNoAudit(Mock<IMaintenanceRepository> audit) =>
audit.Verify(a => a.LogAuditEntryAsync(
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<string>(),
It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Dictionary<string, string>?>()), Times.Never);
private static (DelegatedAdminManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildAdminFn()
{
var repo = new Mock<IAdminRepository>();
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new DelegatedAdminManagementFunction(
NullLogger<DelegatedAdminManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
private static (TenantGroupManagementFunction Fn, Mock<IAdminRepository> Repo,
Mock<IMaintenanceRepository> Audit, FakeSignalRNotificationService SignalR) BuildGroupFn()
{
var repo = new Mock<IAdminRepository>();
// Loose-mock defaults return a null Task — every repo read the flows touch needs a concrete setup.
repo.Setup(r => r.GetTenantGroupAsync(It.IsAny<string>())).ReturnsAsync((TenantGroup?)null);
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
repo.Setup(r => r.GetGroupAssigneesAsync(It.IsAny<string>()))
.ReturnsAsync(new List<TenantGroupAssignment>());
var audit = BuildAuditRepo();
var signalR = new FakeSignalRNotificationService();
var fn = new TenantGroupManagementFunction(
NullLogger<TenantGroupManagementFunction>.Instance, BuildService(repo), audit.Object, signalR);
return (fn, repo, audit, signalR);
}
// --- Delegated grant validation (fail-closed) ---
[Fact]
public void ValidateGrant_NullBody_Rejected()
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(null, out _));
[Theory]
[InlineData("", TenantA)]
[InlineData(" ", TenantA)]
[InlineData(Upn, "")]
public void ValidateGrant_MissingFields_Rejected(string upn, string tenantId)
=> Assert.Equal("upn and tenantId are required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = upn, TenantId = tenantId }, out _));
[Theory]
[InlineData("contoso.com")]
[InlineData("not-a-guid")]
[InlineData("11111111-1111-1111-1111")]
public void ValidateGrant_NonGuidTenantId_Rejected(string tenantId)
=> Assert.Equal("a valid tenantId (GUID) is required",
DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = tenantId }, out _));
[Fact]
public void ValidateGrant_UnknownRole_Rejected()
{
var error = DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = "GlobalAdmin" }, out _);
Assert.NotNull(error);
Assert.Contains(Constants.DelegatedRoles.DelegatedReader, error);
}
[Fact]
public void ValidateGrant_EmptyRole_DefaultsToLeastPrivilegedReader()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA }, out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedReader, role);
}
[Fact]
public void ValidateGrant_ExplicitDelegatedAdmin_Accepted()
{
Assert.Null(DelegatedAdminManagementFunction.ValidateGrantRequest(
new GrantDelegatedAdminRequest { Upn = Upn, TenantId = TenantA, Role = Constants.DelegatedRoles.DelegatedAdmin },
out var role));
Assert.Equal(Constants.DelegatedRoles.DelegatedAdmin, role);
}
// --- Revoke: 404-on-missing-row vs. audit + disconnect ---
[Fact]
public async Task RevokeCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(false);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.False(removed);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task RevokeCore_RemovedRow_AuditsAndDisconnectsLowercasedUpn()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.RemoveDelegatedAdminAsync(UpnLower, TenantA)).ReturnsAsync(true);
var removed = await fn.RevokeCoreAsync(Upn, TenantA, "ga@vendor.example");
Assert.True(removed);
// SignalR user ids are lowercased UPNs — a mixed-case route param must still hit the user's connections.
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedAdmin", UpnLower, "ga@vendor.example", null), Times.Once);
}
// --- Disable is a revocation too; enable never disconnects ---
[Fact]
public async Task SetEnabledCore_MissingRow_NoAuditNoDisconnect()
{
var (fn, repo, audit, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(false);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.False(ok);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Disable_Disconnects()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, false)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: false, "ga@vendor.example");
Assert.True(ok);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
}
[Fact]
public async Task SetEnabledCore_Enable_DoesNotDisconnect()
{
var (fn, repo, _, signalR) = BuildAdminFn();
repo.Setup(r => r.SetDelegatedAdminEnabledAsync(UpnLower, TenantA, true)).ReturnsAsync(true);
var ok = await fn.SetEnabledCoreAsync(Upn, TenantA, isEnabled: true, "ga@vendor.example");
Assert.True(ok);
Assert.Empty(signalR.DisconnectedUsers);
}
// --- Group unassign: 404-on-not-assigned vs. disconnect + per-tenant audit ---
[Fact]
public async Task UnassignCore_NotAssigned_NoAuditNoDisconnect()
{
var (fn, _, audit, signalR) = BuildGroupFn(); // default: no assignments for the UPN
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.False(unassigned);
VerifyNoAudit(audit);
Assert.Empty(signalR.DisconnectedUsers);
}
[Fact]
public async Task UnassignCore_Assigned_DisconnectsAndAuditsPerGroupTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA, TenantB },
AssigneeCount = 1,
});
repo.Setup(r => r.GetGroupAssignmentsForUpnAsync(UpnLower))
.ReturnsAsync(new List<TenantGroupAssignment> { new() { Upn = UpnLower, GroupId = GroupId } });
repo.Setup(r => r.UnassignGroupAsync(UpnLower, GroupId)).ReturnsAsync(true);
var unassigned = await fn.UnassignCoreAsync(GroupId, Upn, "ga@vendor.example");
Assert.True(unassigned);
Assert.Equal(new[] { UpnLower }, signalR.DisconnectedUsers);
// Lost access is audited under EACH of the group's tenants (customer-visible trail).
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
audit.Verify(a => a.LogAuditEntryAsync(
TenantB, "DELETE", "DelegatedGroupAccess", UpnLower, "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
// --- Group delete: every (former) assignee's streams are cut ---
[Fact]
public async Task DeleteGroupCore_DisconnectsEveryAssignee_AndAuditsPerTenant()
{
var (fn, repo, audit, signalR) = BuildGroupFn();
var assignees = new List<TenantGroupAssignment>
{
new() { Upn = "one@partner.example", GroupId = GroupId },
new() { Upn = "two@partner.example", GroupId = GroupId },
};
repo.Setup(r => r.GetTenantGroupAsync(GroupId)).ReturnsAsync(new TenantGroup
{
GroupId = GroupId,
Name = "MSP Customers",
TenantIds = new List<string> { TenantA },
AssigneeCount = assignees.Count,
});
repo.Setup(r => r.GetGroupAssigneesAsync(GroupId)).ReturnsAsync(assignees);
repo.Setup(r => r.DeleteTenantGroupAsync(GroupId)).ReturnsAsync(true);
await fn.DeleteGroupCoreAsync(GroupId, "ga@vendor.example");
Assert.Equal(new[] { "one@partner.example", "two@partner.example" }, signalR.DisconnectedUsers);
audit.Verify(a => a.LogAuditEntryAsync(
TenantA, "DELETE", "DelegatedGroupAccess", "*", "ga@vendor.example",
It.IsAny<Dictionary<string, string>?>()), Times.Once);
}
}
@@ -582,7 +582,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{ "FeedbackMinTenantAgeDays", config.FeedbackMinTenantAgeDays },
{ "FeedbackCooldownDays", config.FeedbackCooldownDays },
// MCP access control
{ "McpAccessPolicy", config.McpAccessPolicy ?? "WhitelistOnly" }
{ "McpAccessPolicy", config.McpAccessPolicy ?? "WhitelistOnly" },
// Agent endpoint migration (config-channel re-home)
{ "AgentMigrateApiBaseUrl", config.AgentMigrateApiBaseUrl ?? string.Empty },
{ "AgentMigrateTenantOverridesJson", config.AgentMigrateTenantOverridesJson ?? string.Empty }
};
return entity;
@@ -643,7 +646,10 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
FeedbackMinTenantAgeDays = entity.GetInt32("FeedbackMinTenantAgeDays") ?? 14,
FeedbackCooldownDays = entity.GetInt32("FeedbackCooldownDays") ?? 60,
// MCP access control
McpAccessPolicy = entity.GetString("McpAccessPolicy") ?? "WhitelistOnly"
McpAccessPolicy = entity.GetString("McpAccessPolicy") ?? "WhitelistOnly",
// Agent endpoint migration (config-channel re-home)
AgentMigrateApiBaseUrl = entity.GetString("AgentMigrateApiBaseUrl") ?? string.Empty,
AgentMigrateTenantOverridesJson = entity.GetString("AgentMigrateTenantOverridesJson") ?? string.Empty
};
}
@@ -3,6 +3,7 @@ using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared;
@@ -43,7 +44,9 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
["UserId"] = Truncate(entry.UserId, 128),
["Message"] = Truncate(entry.Message, 512),
["Details"] = Truncate(entry.Details, 4096),
["Timestamp"] = entry.Timestamp,
// "Timestamp" is a reserved system property (supplied values are ignored and
// any row rewrite resets it), so the event time needs its own column.
[BusinessTimestamp.OccurredUtcColumn] = entry.Timestamp,
};
await _table.UpsertEntityAsync(entity);
@@ -168,15 +171,16 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
// Filters on the system-managed Timestamp; user-defined "Timestamp" property
// mirrors it on insert via SaveOpsEventAsync.
private static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
// Date windows filter on the RowKey (reverse-tick, index-backed): the system
// Timestamp is reset by storage migrations, and an OccurredUtc property filter
// would exclude rows written before that column existed.
internal static string? BuildFilter(string? category, DateTime? dateFrom, DateTime? dateTo)
{
var clauses = new List<string>();
if (!string.IsNullOrEmpty(category))
@@ -185,11 +189,11 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.OpsDateToClause(ToUtc(dateTo.Value)));
}
return clauses.Count == 0 ? null : string.Join(" and ", clauses);
}
@@ -201,27 +205,30 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
{
var deleted = 0;
await foreach (var entity in _table.QueryAsync<TableEntity>())
// Server-side RowKey range instead of a full-table scan over the system
// Timestamp: every ops RowKey is a reverse-tick, so age is index-derivable,
// and the system Timestamp lies after storage migrations (it would freeze
// deletions for one retention period, then drop the whole pre-migration
// corpus in a single run).
var filter = BusinessTimestamp.OpsRetentionClause(cutoff);
await foreach (var entity in _table.QueryAsync<TableEntity>(
filter: filter, select: new[] { "PartitionKey", "RowKey" }))
{
var timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue;
if (timestamp < cutoff)
try
{
try
{
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
await _table.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete ops event {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
private static OpsEventEntry MapToEntry(TableEntity entity)
internal static OpsEventEntry MapToEntry(TableEntity entity)
{
return new OpsEventEntry
{
@@ -233,10 +240,22 @@ namespace AutopilotMonitor.Functions.DataAccess.TableStorage
UserId = entity.GetString("UserId"),
Message = entity.GetString("Message") ?? string.Empty,
Details = entity.GetString("Details"),
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.MinValue,
Timestamp = ResolveTimestamp(entity),
};
}
// Business time: OccurredUtc column → reverse-tick RowKey decode → system
// Timestamp (last resort; a row rewrite — e.g. storage migration — resets it).
internal static DateTime ResolveTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.MinValue;
}
private static string? Truncate(string? value, int maxLength)
{
if (value == null) return null;
@@ -0,0 +1,103 @@
using System.Net;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Services;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Admin
{
/// <summary>
/// Global-Admin trigger for the one-time OccurredUtc backfill on AuditLogs / OpsEvents
/// (business timestamps lost to the system-Timestamp reset of the 2026-07-18 storage
/// migration; values are recovered from the reverse-tick RowKeys). DRY-RUN BY DEFAULT —
/// a real run requires the explicit <c>dryRun=false</c> query parameter, so the operator
/// always sees counts and samples before anything is written.
/// </summary>
public class BackfillOccurredUtcFunction
{
private readonly ILogger<BackfillOccurredUtcFunction> _logger;
private readonly OccurredUtcBackfillService _service;
private const int DefaultMaxRows = 250;
private const int MaxMaxRows = 1000;
public BackfillOccurredUtcFunction(
ILogger<BackfillOccurredUtcFunction> logger,
OccurredUtcBackfillService service)
{
_logger = logger;
_service = service;
}
/// <summary>
/// POST /api/maintenance/backfill-occurred-utc
/// Query parameters:
/// - table: "audit" | "ops" (required).
/// - dryRun: "false" to actually write; anything else (or absent) reports only.
/// - maxRows: rows examined per invocation (default 250, max 1000). Follow
/// nextContinuation to continue; null nextContinuation = table done.
/// - continuation: opaque token from the previous response (URL-encode it).
/// </summary>
[Function("BackfillOccurredUtc")]
public async Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "maintenance/backfill-occurred-utc")] HttpRequestData req,
FunctionContext context)
{
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
var userEmail = TenantHelper.GetUserIdentifier(req);
var table = (req.Query["table"] ?? string.Empty).Trim().ToLowerInvariant();
if (table != OccurredUtcBackfillService.TableAudit && table != OccurredUtcBackfillService.TableOps)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new
{
error = $"table must be '{OccurredUtcBackfillService.TableAudit}' or '{OccurredUtcBackfillService.TableOps}'"
});
return bad;
}
var dryRun = !string.Equals(req.Query["dryRun"], "false", StringComparison.OrdinalIgnoreCase);
var continuation = req.Query["continuation"];
var maxRows = DefaultMaxRows;
if (!string.IsNullOrEmpty(req.Query["maxRows"]))
{
if (!int.TryParse(req.Query["maxRows"], out maxRows) || maxRows < 1)
{
var bad = req.CreateResponse(HttpStatusCode.BadRequest);
await bad.WriteAsJsonAsync(new { error = $"maxRows must be a positive integer (max {MaxMaxRows})" });
return bad;
}
maxRows = Math.Min(maxRows, MaxMaxRows);
}
_logger.LogInformation(
"OccurredUtc backfill requested by {User}: table={Table} dryRun={DryRun} maxRows={MaxRows} hasContinuation={HasContinuation}",
userEmail, table, dryRun, maxRows, !string.IsNullOrEmpty(continuation));
try
{
var result = await _service.RunAsync(table, dryRun, maxRows, continuation);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
{
success = true,
result,
triggeredBy = userEmail,
triggeredAt = DateTime.UtcNow,
});
return response;
}
catch (Exception ex)
{
_logger.LogError(ex, "OccurredUtc backfill run failed");
var error = req.CreateResponse(HttpStatusCode.InternalServerError);
await error.WriteAsJsonAsync(new { success = false, message = "Internal server error" });
return error;
}
}
}
}
@@ -64,19 +64,9 @@ public class DelegatedAdminManagementFunction
var currentUpn = context.GetRequestContext().UserPrincipalName;
var body = await req.ReadFromJsonAsync<GrantDelegatedAdminRequest>();
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return await Bad(req, "upn and tenantId are required");
// Mirror the tenant-group path: a mistyped tenantId must not create a garbage scope entry
// that pollutes the allow-list and seeds a per-tenant audit trail keyed on the typo.
if (!Guid.TryParse(body.TenantId, out _))
return await Bad(req, "a valid tenantId (GUID) is required");
// Default to the least-privileged role; reject anything we don't recognize (fail-closed) so a typo
// can never silently widen access.
var role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return await Bad(req, $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'");
var validationError = ValidateGrantRequest(body, out var role);
if (validationError != null)
return await Bad(req, validationError);
// Edition note: TARGET tenants may be any edition — an MSP on Enterprise may manage Community
// customers. The Enterprise requirement applies to the delegated admin's HOME tenant and is
@@ -84,7 +74,7 @@ public class DelegatedAdminManagementFunction
// be checked here: at grant time only the UPN is known, and UPN-domain → tenant mapping is not
// reliable (multi-domain tenants). Grants to non-Enterprise-homed admins are simply inert.
var entry = await _delegatedAdminService.UpsertAsync(
body.Upn, body.TenantId, role,
body!.Upn, body.TenantId, role,
Constants.DelegatedStatus.Active, Constants.DelegatedSource.OperatorGranted, currentUpn ?? "");
await _maintenanceRepo.LogAuditEntryAsync(
@@ -112,10 +102,7 @@ public class DelegatedAdminManagementFunction
string upn, string tenantId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
// The service reports whether a row was actually deleted — a mistyped upn/tenantId must not
// 200 and write a false customer-visible "access removed" audit row while the real grant
// stays live (mirrors the tenant-group revoke semantics).
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
var removed = await RevokeCoreAsync(upn, tenantId, currentUpn);
if (!removed)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -123,20 +110,34 @@ public class DelegatedAdminManagementFunction
return notFound;
}
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
// Enforcement: cut the revoked caller's already-joined live streams — group authorization is
// join-time only, so without this they keep receiving tenant telemetry until the connection drops.
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Delegated assignment revoked" });
return response;
}
/// <summary>
/// Testable core of the revoke flow (no HTTP plumbing — see GetAllBlockedDevicesFunctionTests for the
/// convention). Returns false when no assignment row existed (caller 404s): a mistyped upn/tenantId
/// must not 200 and write a false customer-visible "access removed" audit row while the real grant
/// stays live (mirrors the tenant-group revoke semantics). On a real removal writes the audit entry
/// and cuts the revoked UPN's live streams — group authorization is join-time only, so without the
/// kick they keep receiving tenant telemetry until the connection drops.
/// </summary>
internal async Task<bool> RevokeCoreAsync(string upn, string tenantId, string? currentUpn)
{
var removed = await _delegatedAdminService.RemoveAsync(upn, tenantId);
if (!removed)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "DELETE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "");
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated revoke: {Upn} -> {TenantId} by {By}", upn, tenantId, currentUpn);
return true;
}
/// <summary>PATCH /api/global/delegated-admins/{upn}/{tenantId}/enable — re-enable a disabled assignment. GlobalAdminOnly.</summary>
[Function("EnableDelegatedAdmin")]
[Authorize]
@@ -157,7 +158,7 @@ public class DelegatedAdminManagementFunction
HttpRequestData req, string upn, string tenantId, bool isEnabled, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
var ok = await SetEnabledCoreAsync(upn, tenantId, isEnabled, currentUpn);
if (!ok)
{
var notFound = req.CreateResponse(HttpStatusCode.NotFound);
@@ -165,20 +166,56 @@ public class DelegatedAdminManagementFunction
return notFound;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
}
/// <summary>
/// Testable core of enable/disable (no HTTP plumbing). Returns false when no assignment row
/// existed (caller 404s — no audit, no disconnect). Disable is a revocation too, so it cuts the
/// UPN's already-joined live streams (see <see cref="RevokeCoreAsync"/>); enable never disconnects.
/// </summary>
internal async Task<bool> SetEnabledCoreAsync(string upn, string tenantId, bool isEnabled, string? currentUpn)
{
var ok = await _delegatedAdminService.SetEnabledAsync(upn, tenantId, isEnabled);
if (!ok)
return false;
await _maintenanceRepo.LogAuditEntryAsync(
tenantId.ToLowerInvariant(), "UPDATE", "DelegatedAdmin", upn.ToLowerInvariant(), currentUpn ?? "",
new Dictionary<string, string> { { "IsEnabled", isEnabled.ToString() } });
// Disable is a revocation too — cut already-joined live streams (see RevokeDelegatedAdmin).
if (!isEnabled)
await _signalRService.DisconnectUserAsync(upn.ToLowerInvariant());
_logger.LogInformation("Delegated {Action}: {Upn} -> {TenantId} by {By}",
isEnabled ? "enable" : "disable", upn, tenantId, currentUpn);
return true;
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = isEnabled ? "Delegated assignment enabled" : "Delegated assignment disabled" });
return response;
/// <summary>
/// Pure grant-request validation (testable seam). Returns the 400 error message, or null when valid —
/// then <paramref name="role"/> carries the effective role. A mistyped tenantId must not create a
/// garbage scope entry that pollutes the allow-list and seeds a per-tenant audit trail keyed on the
/// typo; the role defaults to least-privileged and unknown roles are rejected (fail-closed) so a typo
/// can never silently widen access.
/// </summary>
internal static string? ValidateGrantRequest(GrantDelegatedAdminRequest? body, out string role)
{
role = Constants.DelegatedRoles.DelegatedReader;
if (body == null || string.IsNullOrWhiteSpace(body.Upn) || string.IsNullOrWhiteSpace(body.TenantId))
return "upn and tenantId are required";
if (!Guid.TryParse(body.TenantId, out _))
return "a valid tenantId (GUID) is required";
role = string.IsNullOrWhiteSpace(body.Role) ? Constants.DelegatedRoles.DelegatedReader : body.Role!;
if (role != Constants.DelegatedRoles.DelegatedReader && role != Constants.DelegatedRoles.DelegatedAdmin)
return $"role must be '{Constants.DelegatedRoles.DelegatedReader}' or '{Constants.DelegatedRoles.DelegatedAdmin}'";
return null;
}
private static async Task<HttpResponseData> Bad(HttpRequestData req, string error)
@@ -7,10 +7,11 @@ using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Functions.Admin
{
/// <summary>
/// Timer-triggered watcher for the Azure SignalR Free-tier limits (20
/// concurrent connections / 20,000 messages per UTC day). Runs every full
/// hour and emits OpsEvents when the configured thresholds are crossed so
/// operators get a Telegram ping before clients start getting 429'd.
/// Timer-triggered watcher for the Azure SignalR plan limits (Standard S1
/// x1 unit: 1,000 concurrent connections / 1M messages per UTC day). Runs
/// every full hour and emits OpsEvents when the configured thresholds are
/// crossed so operators get a Telegram ping before clients start getting
/// 429'd or message overage is billed.
///
/// Separate from the 2h MaintenanceFunction: the SignalR limits can be
/// breached by a brief connection spike, so the cadence is tighter. Cost
@@ -112,15 +112,26 @@ public class TenantGroupManagementFunction
string groupId, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
await DeleteGroupCoreAsync(groupId, currentUpn);
// Capture tenants + assignees BEFORE the cascade delete removes them, so we can audit the
// bulk access removal under each affected tenant and cut every assignee's live streams.
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>
/// Testable core of the group cascade delete (no HTTP plumbing). Captures tenants + assignees
/// BEFORE the delete removes them, so the bulk access removal is audited under each affected
/// tenant and EVERY (former) assignee's live streams are cut — group authorization is join-time
/// only, so without the kick they keep receiving the managed tenants' telemetry until their
/// connection drops.
/// </summary>
internal async Task DeleteGroupCoreAsync(string groupId, string? currentUpn)
{
var group = await _delegatedAdminService.GetGroupAsync(groupId);
var assignees = await _delegatedAdminService.GetGroupAssigneesAsync(groupId);
await _delegatedAdminService.DeleteGroupAsync(groupId);
// Enforcement: group authorization is join-time only — without the kick, every (former)
// assignee keeps receiving the managed tenants' live telemetry until their connection drops.
foreach (var assignee in assignees)
await _signalRService.DisconnectUserAsync(assignee.Upn);
@@ -137,10 +148,6 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Tenant group deleted: {GroupId} by {By}", groupId, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Group deleted" });
return response;
}
/// <summary>POST /api/global/tenant-groups/{groupId}/tenants — add a tenant. GlobalAdminOnly. Body: { "tenantId": "&lt;guid&gt;" }.</summary>
@@ -271,17 +278,31 @@ public class TenantGroupManagementFunction
string groupId, string upn, FunctionContext context)
{
var currentUpn = context.GetRequestContext().UserPrincipalName;
var unassigned = await UnassignCoreAsync(groupId, upn, currentUpn);
if (!unassigned)
return await NotFound(req);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
}
/// <summary>
/// Testable core of the unassign flow (no HTTP plumbing). Returns false when the UPN was not
/// actually assigned (caller 404s) — so a mistyped UPN can't 200 and write false "access removed"
/// audit rows, and no disconnect fires. On a real unassign cuts the UPN's already-joined live
/// streams (join-time-only authz) and audits the lost access under each of the group's tenants.
/// </summary>
internal async Task<bool> UnassignCoreAsync(string groupId, string upn, string? currentUpn)
{
var normalizedUpn = upn.ToLowerInvariant();
// Read the group's tenants (unchanged by unassign) so we can audit the UPN's lost access per tenant.
var group = await _delegatedAdminService.GetGroupAsync(groupId);
// The service returns false when the UPN was not actually assigned — so a mistyped UPN can't 200 and
// write false "access removed" audit rows. Only audit after a real unassign.
var unassigned = await _delegatedAdminService.UnassignGroupAsync(normalizedUpn, groupId);
if (!unassigned)
return await NotFound(req);
return false;
// Enforcement: cut the unassigned caller's already-joined live streams (join-time-only authz).
await _signalRService.DisconnectUserAsync(normalizedUpn);
if (group != null)
@@ -296,10 +317,7 @@ public class TenantGroupManagementFunction
}
_logger.LogInformation("Group {GroupId} unassigned from {Upn} by {By}", groupId, normalizedUpn, currentUpn);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new { message = "Unassigned from group" });
return response;
return true;
}
/// <summary>Logs one audit entry per tenant (the customer-visible "who can read my tenant" trail).</summary>
@@ -157,6 +157,39 @@ namespace AutopilotMonitor.Functions.Functions.Config
return (line.ZipSha256, line.ExeSha256);
}
/// <summary>
/// Resolves the endpoint-migration target for a tenant: per-tenant override wins over
/// the global value; an override entry with an empty value pins the tenant (no
/// migration even while the global target is set — staged rollout). The winning value
/// is validated against <see cref="AutopilotMonitor.Shared.Services.AgentEndpointMigrationRules"/>;
/// invalid values resolve to null (fail-safe: better to strand an agent on the old
/// backend than to serve a broken or non-allowlisted URL).
/// </summary>
internal static string? ResolveMigrateTarget(
AutopilotMonitor.Shared.Models.AdminConfiguration adminConfig,
string tenantId,
out string? rejectedCandidate)
{
rejectedCandidate = null;
string? candidate = adminConfig.AgentMigrateApiBaseUrl;
var overrides = adminConfig.GetAgentMigrateTenantOverrides();
if (!string.IsNullOrEmpty(tenantId) && overrides.TryGetValue(tenantId, out var tenantTarget))
candidate = tenantTarget; // empty string = pinned, handled below
if (string.IsNullOrWhiteSpace(candidate))
return null;
if (AutopilotMonitor.Shared.Services.AgentEndpointMigrationRules
.TryNormalizeTarget(candidate, out var normalized))
{
return normalized;
}
rejectedCandidate = candidate;
return null;
}
/// <summary>
/// Core config logic: fetch tenant + admin config, gather rules, IME patterns.
/// Called by both the cert-auth Run() method and the bootstrap wrapper.
@@ -215,10 +248,29 @@ namespace AutopilotMonitor.Functions.Functions.Config
// so agent code is unchanged across all major lines.
var (latestAgentSha256, latestAgentExeSha256) = SelectAgentHashesForClient(req, adminConfig);
// Endpoint migration on the control channel: serve the (validated) re-home target
// so agents still bound to this backend's compiled-in URL move themselves to the
// new deployment at their next start. LogWarning (not Information) because worker
// logs below Warning never reach App Insights — this line is the delivery evidence
// during a migration window.
var migrateTarget = ResolveMigrateTarget(adminConfig, tenantId, out var rejectedMigrateCandidate);
if (migrateTarget != null)
{
_logger.LogWarning(
"AgentMigrateServed: tenant={TenantId} serial={Serial} agentVersion={AgentVersion} target={Target}",
tenantId, serialNumberHeader, agentVersionHeader, migrateTarget);
}
else if (rejectedMigrateCandidate != null)
{
_logger.LogWarning(
"AgentMigrateRejected: configured migration target failed validation and is NOT served. tenant={TenantId} candidate={Candidate}",
tenantId, rejectedMigrateCandidate);
}
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new AgentConfigResponse
{
ConfigVersion = 32, // V2 OOBE-console / Shift+F10 detection portal toggle (EnableConsoleBypassDetection; tenant-scoped, default ON / opt-out; gates ConsoleBypass host + ConsolePrefetchScanner)
ConfigVersion = 35, // MDM reboot-policy watcher (mdm_policy_reboot_required / esp_reboot_coalescing)
UploadIntervalSeconds = 10,
SelfDestructOnComplete = tenantConfig.SelfDestructOnComplete ?? true,
KeepLogFile = tenantConfig.KeepLogFile ?? false,
@@ -262,6 +314,7 @@ namespace AutopilotMonitor.Functions.Functions.Config
DeviceBlocked = killVerdict.IsBlocked,
DeviceKillSignal = killVerdict.IsKill,
UnblockAt = killVerdict.UnblockAt,
MigrateToApiBaseUrl = migrateTarget,
});
return response;
@@ -10,6 +10,7 @@ using AutopilotMonitor.Functions.Services;
using AutopilotMonitor.Shared.Models;
using AutopilotMonitor.Shared.Pagination;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
@@ -17,15 +18,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
{
public class GetAllTenantConfigurationsFunction
{
// TTL for the "this managed id has NO config row in any casing" verdict a rescue scan proved.
// Bounds how long a tenant onboarded moments after that scan stays absent from a delegated
// caller's config/all (matches the positive config cache TTL in TenantConfigurationService).
private static readonly TimeSpan MissingConfigNegativeCacheDuration = TimeSpan.FromMinutes(5);
private readonly ILogger<GetAllTenantConfigurationsFunction> _logger;
private readonly TenantConfigurationService _configService;
private readonly IMemoryCache _cache;
public GetAllTenantConfigurationsFunction(
ILogger<GetAllTenantConfigurationsFunction> logger,
TenantConfigurationService configService)
TenantConfigurationService configService,
IMemoryCache cache)
{
_logger = logger;
_configService = configService;
_cache = cache;
}
[Function("GetAllTenantConfigurations")]
@@ -73,9 +82,21 @@ namespace AutopilotMonitor.Functions.Functions.Config
// (agent paths auto-create rows with the caller-supplied id verbatim) — the same caveat
// the delegated sessions fan-out handles case-insensitively. A point-read is exact-case,
// so any id it missed gets ONE scan+filter rescue pass (the pre-point-read behavior).
// Only stale grants or case-variant rows pay for it; the common path stays read-bounded.
// An id a previous scan already proved missing in ANY casing is negative-cached and
// skipped for the TTL — otherwise a single stale grant (offboarded / never-onboarded
// tenant) would make EVERY config/all call pay a full table scan.
if (subset.Count < managedIds.Count)
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
{
var unproven = MissingManagedIds(managedIds, subset)
.Where(id => !_cache.TryGetValue(MissingConfigCacheKey(id), out _))
.ToList();
if (unproven.Count > 0)
{
subset = MergeRescuedConfigs(managedIds, subset, await _configService.GetAllConfigurationsAsync());
foreach (var id in MissingManagedIds(unproven, subset))
_cache.Set(MissingConfigCacheKey(id), true, MissingConfigNegativeCacheDuration);
}
}
_logger.LogInformation("GetAllTenantConfigurations (delegated subset, {Count} tenants) by {User}",
subset.Count, userIdentifier);
@@ -177,6 +198,23 @@ namespace AutopilotMonitor.Functions.Functions.Config
IEnumerable<TenantConfiguration?> reads)
=> reads.Where(c => c != null).Select(c => c!).ToList();
/// <summary>
/// The managed ids not present (case-insensitively) in <paramref name="configs"/> — before the
/// rescue scan these are the point-read misses, after it the proven-missing ids to negative-cache.
/// Pure + testable.
/// </summary>
internal static List<string> MissingManagedIds(
IEnumerable<string> managedIds, IReadOnlyCollection<TenantConfiguration> configs)
{
var have = new HashSet<string>(configs.Select(c => c.TenantId), StringComparer.OrdinalIgnoreCase);
return managedIds.Where(id => !have.Contains(id)).ToList();
}
/// <summary>Negative-cache key for a managed id a rescue scan proved has no config row.
/// Lowercased so case variants of the same grant share one verdict.</summary>
internal static string MissingConfigCacheKey(string tenantId)
=> $"tenant-config-missing:{tenantId.ToLowerInvariant()}";
/// <summary>
/// Case-variant rescue for managed ids the exact-case point-reads missed: from the full config scan,
/// add every config whose TenantId matches a managed id case-insensitively and is not already in the
@@ -102,17 +102,19 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
// Authentication + GlobalAdminOnly authorization enforced by PolicyEnforcementMiddleware
// Perform comprehensive health checks
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync();
// SignalR Quota check exposes the subscription/resource-group path of the
// backing SignalR resource, and Poison Queues exposes internal async-worker
// queue topology — restrict both to Global Admins. Non-GA callers
// (Tenant Admins, Operators) get the rest of the report unchanged.
// SignalR Quota exposes the subscription/resource-group path of the backing
// SignalR resource, Poison Queues exposes internal async-worker queue topology,
// and endpoint URLs in the check details expose infrastructure hostnames —
// restrict all of that to Global Admins. Non-GA callers (Tenant Admins,
// Operators) get the rest of the report unchanged.
// The route is registered as AuthenticatedUser, so PolicyEnforcementMiddleware
// does NOT compute IsGlobalAdmin; resolve it directly via GlobalAdminService.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
// Perform comprehensive health checks (endpoint URLs only for Global Admins)
var healthCheckResult = await _healthCheckService.PerformAllChecksAsync(includeEndpointUrls: isGlobalAdmin);
var visibleChecks = isGlobalAdmin
? healthCheckResult.Checks
: healthCheckResult.Checks
@@ -146,10 +148,14 @@ namespace AutopilotMonitor.Functions.Functions.Infrastructure
/// </summary>
[Function("McpHealthCheck")]
public async Task<HttpResponseData> GetMcpHealthCheck(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req)
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "health/mcp")] HttpRequestData req,
FunctionContext context)
{
// Authentication enforced by PolicyEnforcementMiddleware (AuthenticatedUser).
var check = await _healthCheckService.CheckMcpServerAsync();
// The MCP server URL is infrastructure topology — include it only for Global Admins.
var requestCtx = context.GetRequestContext();
var isGlobalAdmin = await _globalAdminService.IsGlobalAdminAsync(requestCtx.UserPrincipalName);
var check = await _healthCheckService.CheckMcpServerAsync(includeEndpointUrl: isGlobalAdmin);
var response = req.CreateResponse(HttpStatusCode.OK);
await response.WriteAsJsonAsync(new
@@ -64,6 +64,14 @@ namespace AutopilotMonitor.Functions.Functions.Rules
return badRequest;
}
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
try
{
var success = await _ruleService.CreateRuleAsync(tenantId, rule);
@@ -108,12 +116,71 @@ namespace AutopilotMonitor.Functions.Functions.Rules
rule.RuleId = ruleId;
var scopeError = ValidateScopeAndEmitMode(rule);
if (scopeError != null)
{
var badRequest = req.CreateResponse(HttpStatusCode.BadRequest);
await badRequest.WriteAsJsonAsync(new { success = false, message = scopeError });
return badRequest;
}
var success = await _ruleService.UpdateRuleAsync(tenantId, rule);
var response = req.CreateResponse(success ? HttpStatusCode.OK : HttpStatusCode.InternalServerError);
await response.WriteAsJsonAsync(new { success, message = success ? "Rule updated" : "Failed to update rule" });
return response;
}
/// <summary>
/// Selectable phase-scope tokens: the <see cref="EnrollmentPhase"/> enum NAMES from
/// Start(0) through Complete(7). Explicit name list (not Enum.TryParse) so numeric
/// tokens ("4") and the non-selectable Unknown/Failed members are rejected — the
/// canonical vocabulary the agent, schema, and portal share.
/// </summary>
private static readonly HashSet<string> ValidScopePhases = new(StringComparer.OrdinalIgnoreCase)
{
nameof(EnrollmentPhase.Start),
nameof(EnrollmentPhase.DevicePreparation),
nameof(EnrollmentPhase.DeviceSetup),
nameof(EnrollmentPhase.AppsDevice),
nameof(EnrollmentPhase.AccountSetup),
nameof(EnrollmentPhase.AppsUser),
nameof(EnrollmentPhase.FinalizingSetup),
nameof(EnrollmentPhase.Complete),
};
/// <summary>
/// Validates the phase-scope + emit-mode fields on create/update.
/// Returns an error message for a 400 response, or null when valid.
/// Toggle-style partial payloads carry none of these fields and pass through.
/// </summary>
internal static string? ValidateScopeAndEmitMode(GatherRule rule)
{
var hasActivePhases = rule.ActivePhases != null && rule.ActivePhases.Count > 0;
var hasFromPhase = !string.IsNullOrEmpty(rule.ActiveFromPhase);
if (hasActivePhases && hasFromPhase)
return "activePhases and activeFromPhase are mutually exclusive — set only one.";
if (hasActivePhases)
{
foreach (var phase in rule.ActivePhases!)
{
if (string.IsNullOrEmpty(phase) || !ValidScopePhases.Contains(phase))
return $"Invalid phase '{phase}' in activePhases. Valid phases: {string.Join(", ", ValidScopePhases)}.";
}
}
if (hasFromPhase && !ValidScopePhases.Contains(rule.ActiveFromPhase!))
return $"Invalid activeFromPhase '{rule.ActiveFromPhase}'. Valid phases: {string.Join(", ", ValidScopePhases)}.";
if (!string.IsNullOrEmpty(rule.EmitMode)
&& !string.Equals(rule.EmitMode, "always", StringComparison.OrdinalIgnoreCase)
&& !string.Equals(rule.EmitMode, "on_change", StringComparison.OrdinalIgnoreCase))
return "emitMode must be \"always\" or \"on_change\".";
return null;
}
[Function("DeleteGatherRule")]
public async Task<HttpResponseData> DeleteRule(
[HttpTrigger(AuthorizationLevel.Anonymous, "delete", Route = "rules/gather/{ruleId}")] HttpRequestData req,
@@ -0,0 +1,154 @@
using System;
using System.Globalization;
using Azure.Data.Tables;
namespace AutopilotMonitor.Functions.Helpers
{
/// <summary>
/// Business-timestamp handling for tables whose rows historically relied on the
/// Azure Tables system <c>Timestamp</c> (= row write time). That system property is
/// server-managed: supplied values are ignored on write and every row rewrite
/// (storage migration, merge) resets it. The authoritative event time therefore
/// lives in (a) the explicit <see cref="OccurredUtcColumn"/> custom property on
/// rows written after its introduction, and (b) the time encoded in the RowKey
/// for older rows. Readers resolve OccurredUtc → RowKey decode → system Timestamp.
///
/// RowKey schemes decoded here:
/// AuditLogs: "!{MaxTicks - ticks:D19}_{guid:N}" (legacy pre-2026-05 rows are bare GUIDs — not decodable)
/// OpsEvents: "{MaxTicks - ticks:D19}"
/// Events: "{timestamp:yyyyMMddHHmmssfff}_{sequence:D10}"
/// </summary>
internal static class BusinessTimestamp
{
/// <summary>Custom column carrying the business event time. Survives migrations verbatim
/// (unlike the system Timestamp, which any row rewrite resets).</summary>
internal const string OccurredUtcColumn = "OccurredUtc";
internal const string AuditRowKeyPrefix = "!";
/// <summary>
/// '"' (0x22) is the character immediately after '!' (0x21). `RowKey lt '"'` confines a
/// range to time-encoded audit rows: legacy bare-GUID RowKeys start with a hex digit
/// (≥ '0', 0x30) and would otherwise match ANY lower bound of the form `RowKey ge '!…'`.
/// </summary>
internal const string AuditTimeEncodedUpperBound = "\"";
private const string EventRowKeyTimestampFormat = "yyyyMMddHHmmssfff";
/// <summary>
/// Reads a datetime column as UTC regardless of the SDK's materialized type —
/// service reads surface Edm.DateTime as either DateTimeOffset or DateTime
/// depending on payload shape (the pre-existing mappers already read both).
/// Unspecified kinds are treated as UTC (all stored values are UTC).
/// </summary>
internal static DateTime? GetUtcDateTime(TableEntity entity, string column)
{
if (!entity.TryGetValue(column, out var value) || value == null)
return null;
return value switch
{
DateTimeOffset dto => dto.UtcDateTime,
DateTime dt => dt.Kind == DateTimeKind.Local
? dt.ToUniversalTime()
: DateTime.SpecifyKind(dt, DateTimeKind.Utc),
_ => null,
};
}
// ===== RowKey decoders =====
/// <summary>Decodes an audit RowKey ("!{rev:D19}_{guid:N}") to the original UTC write time.
/// Returns false for legacy bare-GUID RowKeys and any other foreign format.</summary>
internal static bool TryDecodeAuditRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length < 21 || rowKey[0] != '!')
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(1, 19), out utc);
}
/// <summary>Decodes an ops-event RowKey (bare "{rev:D19}") to the original UTC write time.</summary>
internal static bool TryDecodeOpsRowKey(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null || rowKey.Length != 19)
return false;
return TryDecodeReverseTicks(rowKey.AsSpan(), out utc);
}
/// <summary>Decodes an enrollment-event RowKey ("{yyyyMMddHHmmssfff}_{seq:D10}") to the
/// (sanitized) agent event time, millisecond precision, Kind=Utc.</summary>
internal static bool TryDecodeEventRowKeyPrefix(string? rowKey, out DateTime utc)
{
utc = default;
if (rowKey == null)
return false;
var separator = rowKey.IndexOf('_');
if (separator != 17)
return false;
if (!DateTime.TryParseExact(rowKey.Substring(0, 17), EventRowKeyTimestampFormat,
CultureInfo.InvariantCulture,
DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out utc))
return false;
return true;
}
private static bool TryDecodeReverseTicks(ReadOnlySpan<char> revTickDigits, out DateTime utc)
{
utc = default;
foreach (var c in revTickDigits)
{
if (c < '0' || c > '9')
return false;
}
if (!long.TryParse(revTickDigits, NumberStyles.None, CultureInfo.InvariantCulture, out var rev))
return false;
var ticks = DateTime.MaxValue.Ticks - rev;
if (ticks < DateTime.MinValue.Ticks || ticks > DateTime.MaxValue.Ticks)
return false;
utc = new DateTime(ticks, DateTimeKind.Utc);
return true;
}
// ===== OData date-window clauses (RowKey ranges) =====
// Date windows filter on the RowKey rather than a property: rows missing a property are
// excluded from OData comparisons (so an OccurredUtc filter would drop every not-yet-
// backfilled row), and RowKey ranges are index-backed. String order equals numeric order
// because the reverse-tick is fixed-width D19. rev(t) = MaxTicks - t.Ticks, so larger
// timestamps have lexically smaller RowKeys (newest-first).
private static long Rev(DateTime utc) => DateTime.MaxValue.Ticks - utc.Ticks;
/// <summary>Audit rows with business time ≥ <paramref name="fromUtc"/> (inclusive):
/// rev ≤ rev(from) &lt; rev(from)+1. The exclusive upper bound also excludes all legacy
/// GUID rows (they sort after every '!' row).</summary>
internal static string AuditDateFromClause(DateTime fromUtc)
=> $"RowKey lt '{AuditRowKeyPrefix}{Rev(fromUtc) + 1:D19}'";
/// <summary>Audit rows with business time ≤ <paramref name="toUtc"/> (inclusive):
/// rev ≥ rev(to). The lower bound alone would match every legacy GUID row, hence the
/// mandatory time-encoded guard.</summary>
internal static string AuditDateToClause(DateTime toUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(toUtc):D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Audit rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// rev ≥ rev(cutoff)+1 ⇔ ts &lt; cutoff. MUST keep the time-encoded guard — without it
/// this clause matches (and a retention sweep would delete) every legacy GUID row.</summary>
internal static string AuditRetentionClause(DateTime cutoffUtc)
=> $"RowKey ge '{AuditRowKeyPrefix}{Rev(cutoffUtc) + 1:D19}' and RowKey lt '{AuditTimeEncodedUpperBound}'";
/// <summary>Ops rows with business time ≥ <paramref name="fromUtc"/> (inclusive). Bare D19
/// RowKey (no suffix), so `le` is exact.</summary>
internal static string OpsDateFromClause(DateTime fromUtc)
=> $"RowKey le '{Rev(fromUtc):D19}'";
/// <summary>Ops rows with business time ≤ <paramref name="toUtc"/> (inclusive).</summary>
internal static string OpsDateToClause(DateTime toUtc)
=> $"RowKey ge '{Rev(toUtc):D19}'";
/// <summary>Ops rows strictly older than <paramref name="cutoffUtc"/> (retention):
/// `gt` not `ge` — a row at exactly the cutoff is kept, matching `Timestamp lt` semantics.</summary>
internal static string OpsRetentionClause(DateTime cutoffUtc)
=> $"RowKey gt '{Rev(cutoffUtc):D19}'";
}
}
@@ -252,6 +252,7 @@ builder.Services.AddHttpClient<GitHubRuleRepository>()
.AddPolicyHandler((sp, _) => sp.GetRequiredService<ResiliencePolicies>().ExternalDataApi);
builder.Services.AddSingleton<MaintenanceService>();
builder.Services.AddSingleton<LegacyReclassificationService>();
builder.Services.AddSingleton<OccurredUtcBackfillService>();
builder.Services.AddSingleton<
AutopilotMonitor.Functions.Services.Monitoring.IAzureMonitorMetricsReader,
AutopilotMonitor.Functions.Services.Monitoring.AzureMonitorMetricsReader>();
@@ -399,6 +399,9 @@ public static class EndpointAccessPolicyCatalog
// One-time retro-reconcile of misclassified historical sessions (audit 2026-07-16) —
// mutating, dry-run by default, GA only.
new("POST", "maintenance/reclassify-legacy", EndpointPolicy.GlobalAdminOnly),
// One-time OccurredUtc backfill on AuditLogs/OpsEvents (2026-07-18 migration reset
// the system Timestamps) — mutating, dry-run by default, GA only.
new("POST", "maintenance/backfill-occurred-utc", EndpointPolicy.GlobalAdminOnly),
// ── Critical-Table Backup (plan §PR1+§PR2) ─────────────────────────────
new("POST", "global/backups/trigger", EndpointPolicy.GlobalAdminOnly),
@@ -151,10 +151,38 @@ namespace AutopilotMonitor.Functions.Services
return await _ruleRepo.StoreRuleStateAsync(tenantId, rule.RuleId, new RuleState { Enabled = rule.Enabled });
}
// Custom rule. The portal toggle PUTs only { enabled, isBuiltIn, isCommunity } —
// full-storing that payload would wipe Title/Target/Trigger/Parameters. Merge
// toggle-style partials into the existing row; full payloads (edit flow) replace
// the row but keep the original CreatedAt.
var existing = (await _ruleRepo.GetGatherRulesAsync(tenantId))
.FirstOrDefault(r => r.RuleId == rule.RuleId && !r.IsBuiltIn && !r.IsCommunity);
var isTogglePartial = string.IsNullOrEmpty(rule.Title)
&& string.IsNullOrEmpty(rule.CollectorType)
&& string.IsNullOrEmpty(rule.Target);
if (isTogglePartial)
{
if (existing == null)
{
_logger.LogWarning(
"Toggle-style partial update for unknown custom gather rule {RuleId} (tenant {TenantId}); nothing to update",
rule.RuleId, tenantId);
return false;
}
existing.Enabled = rule.Enabled;
existing.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(existing, tenantId);
}
// Ensure custom rules keep correct flags (the incoming payload may
// omit them, causing them to default to IsBuiltIn=true in the model).
rule.IsBuiltIn = false;
rule.IsCommunity = false;
if (existing != null)
rule.CreatedAt = existing.CreatedAt;
rule.UpdatedAt = DateTime.UtcNow;
return await _ruleRepo.StoreGatherRuleAsync(rule, tenantId);
}
@@ -413,7 +441,22 @@ namespace AutopilotMonitor.Functions.Services
&& a.Title == b.Title
&& a.CollectorType == b.CollectorType
&& a.Trigger == b.Trigger
&& a.TriggerPhase == b.TriggerPhase;
&& a.TriggerPhase == b.TriggerPhase
&& ActivePhasesEquivalent(a.ActivePhases, b.ActivePhases)
&& a.ActiveFromPhase == b.ActiveFromPhase
&& a.EmitMode == b.EmitMode;
}
/// <summary>
/// Null-tolerant ordered comparison for <see cref="GatherRule.ActivePhases"/>:
/// null and empty are both "unrestricted" and therefore equivalent, so pre-existing
/// DB rows (absent column → null) never diff against seeds (also null) — no reseed churn.
/// </summary>
private static bool ActivePhasesEquivalent(List<string>? a, List<string>? b)
{
var left = a ?? new List<string>();
var right = b ?? new List<string>();
return left.SequenceEqual(right, StringComparer.Ordinal);
}
}
}
@@ -34,9 +34,12 @@ public class HealthCheckService
}
/// <summary>
/// Performs all health checks and returns the results
/// Performs all health checks and returns the results.
/// <paramref name="includeEndpointUrls"/> adds the probed/backing endpoint URLs to the
/// per-check details — infrastructure topology, so callers must only enable it for
/// Global Admins.
/// </summary>
public async Task<HealthCheckResult> PerformAllChecksAsync()
public async Task<HealthCheckResult> PerformAllChecksAsync(bool includeEndpointUrls = false)
{
var result = new HealthCheckResult
{
@@ -50,9 +53,9 @@ public class HealthCheckService
// one slow check. It is exposed via its own endpoint (GET health/mcp) that the
// frontend fetches independently and folds into the card grid incrementally.
var checks = await Task.WhenAll(
CheckStorageBackendAsync(),
CheckProcessingBackendAsync(),
CheckAgentBinariesAsync(),
CheckStorageBackendAsync(includeEndpointUrls),
CheckProcessingBackendAsync(includeEndpointUrls),
CheckAgentBinariesAsync(includeEndpointUrls),
CheckSignalRQuotaAsync(),
CheckPoisonQueuesAsync()
);
@@ -73,7 +76,7 @@ public class HealthCheckService
/// <summary>
/// Checks Azure Table Storage connectivity by querying the admin configuration table
/// </summary>
private async Task<HealthCheck> CheckStorageBackendAsync()
private async Task<HealthCheck> CheckStorageBackendAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -97,29 +100,62 @@ public class HealthCheckService
_logger.LogError(ex, "Storage backend health check failed");
}
if (includeEndpointUrls)
{
// Same resolution order as TableStorageService: account name (Managed Identity)
// is the production path, connection string is the local-dev fallback.
var accountName = _configuration["AzureStorageAccountName"];
check.Details = new Dictionary<string, object>
{
["Table endpoint"] = string.IsNullOrWhiteSpace(accountName)
? "connection string (local dev)"
: $"https://{accountName}.table.core.windows.net"
};
}
return check;
}
/// <summary>
/// Checks that the Azure Functions host is running
/// </summary>
private static Task<HealthCheck> CheckProcessingBackendAsync()
private static Task<HealthCheck> CheckProcessingBackendAsync(bool includeEndpointUrls)
{
var uptime = DateTime.UtcNow - System.Diagnostics.Process.GetCurrentProcess().StartTime.ToUniversalTime();
return Task.FromResult(new HealthCheck
var check = new HealthCheck
{
Name = "Processing Backend",
Description = "Application host process",
Status = "healthy",
Message = $"Host running (uptime: {(int)uptime.TotalMinutes}m {uptime.Seconds}s)"
});
};
if (includeEndpointUrls)
{
// WEBSITE_HOSTNAME is stamped by App Service and reflects the host actually
// serving this process — no hardcoded URL to go stale after a migration.
var hostname = Environment.GetEnvironmentVariable("WEBSITE_HOSTNAME");
if (!string.IsNullOrWhiteSpace(hostname))
{
check.Details = new Dictionary<string, object>
{
["API endpoint"] = $"https://{hostname}"
};
}
}
return Task.FromResult(check);
}
/// <summary>
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable in blob storage
/// Checks that the agent binaries (ZIP) and bootstrap script (PS1) are reachable on the
/// canonical download alias (Front Door → current blob origin) AND on the legacy blob
/// keepalive account that already-deployed customer bootstrap scripts still use.
/// Alias failure is unhealthy (new installs break); legacy failure is a warning
/// (existing customers break until they migrate to the alias).
/// </summary>
private async Task<HealthCheck> CheckAgentBinariesAsync()
internal async Task<HealthCheck> CheckAgentBinariesAsync(bool includeEndpointUrls)
{
var check = new HealthCheck
{
@@ -127,8 +163,20 @@ public class HealthCheckService
Description = "Agent download package availability"
};
var zipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var ps1Url = $"{Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var aliasZipUrl = $"{Constants.AgentDownloadBaseUrl}/{Constants.AgentZipFileName}";
var aliasPs1Url = $"{Constants.AgentDownloadBaseUrl}/{Constants.BootstrapScriptName}";
var legacyZipUrl = $"{Constants.AgentBlobBaseUrl}/{Constants.AgentZipFileName}";
var legacyPs1Url = $"{Constants.AgentBlobBaseUrl}/{Constants.BootstrapScriptName}";
if (includeEndpointUrls)
{
check.Details = new Dictionary<string, object>
{
["Agent ZIP"] = aliasZipUrl,
["Bootstrap script"] = aliasPs1Url,
["Legacy blob (keepalive)"] = Constants.AgentBlobBaseUrl
};
}
try
{
@@ -138,40 +186,48 @@ public class HealthCheckService
var sw = System.Diagnostics.Stopwatch.StartNew();
// HEAD requests to check availability without downloading
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, aliasPs1Url)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyZipUrl)),
client.SendAsync(new HttpRequestMessage(HttpMethod.Head, legacyPs1Url))
);
sw.Stop();
var zipResponse = results[0];
var ps1Response = results[1];
var aliasZipOk = results[0].IsSuccessStatusCode;
var aliasPs1Ok = results[1].IsSuccessStatusCode;
var legacyZipOk = results[2].IsSuccessStatusCode;
var legacyPs1Ok = results[3].IsSuccessStatusCode;
var zipOk = zipResponse.IsSuccessStatusCode;
var ps1Ok = ps1Response.IsSuccessStatusCode;
var issues = new List<string>();
if (!aliasZipOk) issues.Add($"Agent ZIP (download alias): {(int)results[0].StatusCode}");
if (!aliasPs1Ok) issues.Add($"Bootstrap script (download alias): {(int)results[1].StatusCode}");
if (!legacyZipOk) issues.Add($"Agent ZIP (legacy blob): {(int)results[2].StatusCode}");
if (!legacyPs1Ok) issues.Add($"Bootstrap script (legacy blob): {(int)results[3].StatusCode}");
if (zipOk && ps1Ok)
if (issues.Count == 0)
{
check.Status = "healthy";
check.Message = $"Agent package and bootstrap script available ({sw.ElapsedMilliseconds}ms)";
check.Message = $"Agent package and bootstrap script available via download alias + legacy blob ({sw.ElapsedMilliseconds}ms)";
}
else if (aliasZipOk && aliasPs1Ok)
{
// Alias intact, legacy keepalive degraded — existing customer bootstrap
// scripts still download from the legacy account until they migrate.
check.Status = "warning";
check.Message = $"Download alias OK, but legacy keepalive degraded — {string.Join(", ", issues)}";
}
else
{
check.Status = "unhealthy";
var issues = new List<string>();
if (!zipOk) issues.Add($"Agent ZIP: {(int)zipResponse.StatusCode}");
if (!ps1Ok) issues.Add($"Bootstrap script: {(int)ps1Response.StatusCode}");
check.Message = $"Missing: {string.Join(", ", issues)}";
}
}
catch (Exception ex)
{
check.Status = "unhealthy";
check.Message = $"Blob storage unreachable: {ex.Message}";
check.Message = $"Agent download endpoints unreachable: {ex.Message}";
_logger.LogError(ex, "Agent binaries health check failed");
}
@@ -182,7 +238,8 @@ public class HealthCheckService
/// Probes the MCP server's public <c>/health</c> endpoint (which sits in front of the
/// <c>/mcp</c> access guard, so no auth is required) and surfaces reachability + the
/// deployed MCP build version. Visible to all authenticated users, so the server URL is
/// deliberately NOT included in the message or details.
/// only included in the details when <paramref name="includeEndpointUrl"/> is set —
/// callers must only enable it for Global Admins.
/// <para>
/// The Container App runs with minReplicas=0 and scales to zero on idle. Azure Container
/// Apps holds an inbound request while it activates a cold replica, so this probe doubles
@@ -193,7 +250,7 @@ public class HealthCheckService
/// the blocking all-checks batch, so the dashboard never waits on the wake.
/// </para>
/// </summary>
internal async Task<HealthCheck> CheckMcpServerAsync()
internal async Task<HealthCheck> CheckMcpServerAsync(bool includeEndpointUrl = false)
{
var check = new HealthCheck
{
@@ -203,6 +260,11 @@ public class HealthCheckService
var baseUrl = (_configuration["McpServerUrl"] ?? Constants.McpServerBaseUrl).TrimEnd('/');
var healthUrl = $"{baseUrl}/health";
if (includeEndpointUrl)
{
check.Details = new Dictionary<string, object> { ["Server URL"] = baseUrl };
}
var timeoutSeconds = ParsePositiveInt("McpServerHealthTimeoutSeconds", 30);
// A cold start that takes longer than this is treated as a successful wake rather
@@ -233,7 +295,8 @@ public class HealthCheckService
: $"MCP server reachable ({sw.ElapsedMilliseconds}ms)";
if (!string.IsNullOrWhiteSpace(version))
{
check.Details = new Dictionary<string, object> { ["Version"] = version };
check.Details ??= new Dictionary<string, object>();
check.Details["Version"] = version;
}
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException or HttpRequestException)
@@ -290,7 +353,7 @@ public class HealthCheckService
var check = new HealthCheck
{
Name = "SignalR Quota",
Description = "Free-tier connection + daily message usage"
Description = "Connection + daily message usage vs. plan limits"
};
var resourceId = _configuration["SignalRResourceId"];
@@ -302,9 +365,9 @@ public class HealthCheckService
}
var connectionLimit = ParsePositiveInt(
"SignalRFreeTierConnectionLimit", MaintenanceService.DefaultSignalRConnectionLimit);
"SignalRConnectionLimit", MaintenanceService.DefaultSignalRConnectionLimit);
var messageLimit = ParsePositiveLong(
"SignalRFreeTierDailyMessageLimit", MaintenanceService.DefaultSignalRDailyMessageLimit);
"SignalRDailyMessageLimit", MaintenanceService.DefaultSignalRDailyMessageLimit);
var nowUtc = DateTimeOffset.UtcNow;
var startOfDayUtc = new DateTimeOffset(nowUtc.UtcDateTime.Date, TimeSpan.Zero);
@@ -372,9 +435,9 @@ public class HealthCheckService
check.Message = worstTier switch
{
MaintenanceService.SignalRQuotaTier.Critical => "Free-tier limit nearly saturated — scale to Standard before clients get 429'd",
MaintenanceService.SignalRQuotaTier.Warning => "Approaching free-tier limit — consider scaling to Standard",
_ => "Within free-tier limits"
MaintenanceService.SignalRQuotaTier.Critical => "Plan limit nearly saturated — add SignalR units before clients get 429'd",
MaintenanceService.SignalRQuotaTier.Warning => "Approaching plan limit — consider adding SignalR units",
_ => "Within plan limits"
};
check.Details = details;
return check;
@@ -554,52 +554,53 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Verifies that agent binaries and bootstrap script are available on blob storage.
/// Verifies that agent binaries and bootstrap script are available on the canonical
/// download alias (Front Door → current blob origin) AND on the legacy blob keepalive
/// account that already-deployed customer bootstrap scripts still use.
/// Records an OpsEvent for each missing item so Global Admins see it in the dashboard.
/// </summary>
private async Task CheckAgentBlobStorageAsync()
{
_logger.LogInformation("Checking agent blob storage availability...");
_logger.LogInformation("Checking agent download endpoint availability...");
var zipUrl = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}";
var ps1Url = $"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/Install-AutopilotMonitor.ps1";
var probes = new (string Label, string Url)[]
{
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, download alias)",
$"{AutopilotMonitor.Shared.Constants.AgentDownloadBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}"),
($"Agent ZIP ({AutopilotMonitor.Shared.Constants.AgentZipFileName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.AgentZipFileName}"),
($"Bootstrap script ({AutopilotMonitor.Shared.Constants.BootstrapScriptName}, legacy blob keepalive)",
$"{AutopilotMonitor.Shared.Constants.AgentBlobBaseUrl}/{AutopilotMonitor.Shared.Constants.BootstrapScriptName}")
};
try
{
var client = _httpClientFactory.CreateClient();
client.Timeout = TimeSpan.FromSeconds(15);
var zipRequest = new HttpRequestMessage(HttpMethod.Head, zipUrl);
var ps1Request = new HttpRequestMessage(HttpMethod.Head, ps1Url);
var results = await Task.WhenAll(
client.SendAsync(zipRequest),
client.SendAsync(ps1Request)
probes.Select(p => client.SendAsync(new HttpRequestMessage(HttpMethod.Head, p.Url)))
);
var zipResponse = results[0];
var ps1Response = results[1];
if (!zipResponse.IsSuccessStatusCode)
var allOk = true;
for (var i = 0; i < probes.Length; i++)
{
_logger.LogError("Agent ZIP not available on blob storage: HTTP {StatusCode}", (int)zipResponse.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Agent ZIP (AutopilotMonitor-Agent.zip)", (int)zipResponse.StatusCode);
if (results[i].IsSuccessStatusCode) continue;
allOk = false;
_logger.LogError("{Item} not available: HTTP {StatusCode}", probes[i].Label, (int)results[i].StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync(probes[i].Label, (int)results[i].StatusCode);
}
if (!ps1Response.IsSuccessStatusCode)
if (allOk)
{
_logger.LogError("Bootstrap script not available on blob storage: HTTP {StatusCode}", (int)ps1Response.StatusCode);
await _opsEventService.RecordBlobStorageMissingAsync("Bootstrap script (Install-AutopilotMonitor.ps1)", (int)ps1Response.StatusCode);
}
if (zipResponse.IsSuccessStatusCode && ps1Response.IsSuccessStatusCode)
{
_logger.LogInformation("Agent blob storage check passed: all binaries available");
_logger.LogInformation("Agent download endpoint check passed: all binaries available on alias + legacy blob");
}
}
catch (Exception ex)
{
_logger.LogError(ex, "Agent blob storage check failed — storage unreachable");
_logger.LogError(ex, "Agent download endpoint check failed — endpoints unreachable");
await _opsEventService.RecordBlobStorageUnreachableAsync(ex.Message);
}
}
@@ -9,11 +9,13 @@ using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// SignalR Free-tier quota watcher. Reads ConnectionCount (max over the last
/// hour) and MessageCount (total since UTC midnight) from Azure Monitor and
/// emits tiered OpsEvents when either approaches the documented Free-tier
/// caps - 20 concurrent connections and 20,000 messages per day. The
/// operator can then scale to Standard before clients get 429'd.
/// SignalR quota watcher. Reads ConnectionCount (max over the last hour)
/// and MessageCount (total since UTC midnight) from Azure Monitor and
/// emits tiered OpsEvents when either approaches the plan caps - on
/// Standard S1 that is 1,000 concurrent connections per unit (hard limit,
/// excess connections are rejected) and 1M messages per unit per day
/// (included quota; overage is billed, not throttled). The operator can
/// then add units before clients get 429'd or overage costs accumulate.
///
/// Globally scoped (TenantId = null) - SignalR is one shared resource for
/// the whole Function App, not a per-tenant concern.
@@ -25,12 +27,12 @@ namespace AutopilotMonitor.Functions.Services
internal const int SignalRQuotaCriticalPercent = 95;
internal const int SignalRQuotaWarningPercent = 80;
// Default Free-tier caps as documented by Microsoft. Overridable via
// app settings so an operator who has scaled to a paid tier can re-target
// the watcher without a code change (or disable it by setting the cap
// very high).
internal const int DefaultSignalRConnectionLimit = 20;
internal const long DefaultSignalRDailyMessageLimit = 20_000;
// Default caps for Standard S1 with one unit, as documented by Microsoft.
// Overridable via app settings so an operator who adds units or changes
// tier can re-target the watcher without a code change (or disable it by
// setting the cap very high).
internal const int DefaultSignalRConnectionLimit = 1_000;
internal const long DefaultSignalRDailyMessageLimit = 1_000_000;
// Dedup prefix - all SignalR EventTypes start with this, so the seen-index
// can be built with a single Security-category read.
@@ -59,7 +61,7 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Reads SignalR ConnectionCount + MessageCount metrics from Azure Monitor
/// and emits Warning/Error OpsEvents when the Free-tier limits are
/// and emits Warning/Error OpsEvents when the plan limits are
/// approached. Dedup: one event per EventType per UTC day. Fail-soft -
/// missing config or Azure Monitor errors yield a single log line, never
/// a thrown exception.
@@ -75,9 +77,9 @@ namespace AutopilotMonitor.Functions.Services
}
var connectionLimit = ParseIntSetting(
"SignalRFreeTierConnectionLimit", DefaultSignalRConnectionLimit);
"SignalRConnectionLimit", DefaultSignalRConnectionLimit);
var messageLimit = ParseLongSetting(
"SignalRFreeTierDailyMessageLimit", DefaultSignalRDailyMessageLimit);
"SignalRDailyMessageLimit", DefaultSignalRDailyMessageLimit);
var seenToday = await BuildSignalRQuotaSeenIndexAsync();
@@ -0,0 +1,169 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Shared;
using Microsoft.Extensions.Logging;
namespace AutopilotMonitor.Functions.Services
{
/// <summary>
/// One-time backfill of the <c>OccurredUtc</c> business-timestamp column on
/// AuditLogs and OpsEvents rows written before the column existed. The value is
/// decoded deterministically from the reverse-tick RowKey, so re-runs are
/// idempotent and the write is a pure MERGE that adds exactly one property
/// (nothing existing is modified or deleted). Audit legacy bare-GUID rows carry
/// no recoverable time and are excluded server-side by the scan filter.
/// Readers do not depend on this backfill (they decode the RowKey on the fly);
/// its value is future-migration robustness and raw-query readability.
/// </summary>
public class OccurredUtcBackfillService
{
public const string TableAudit = "audit";
public const string TableOps = "ops";
private const int MaxSamplesInResponse = 100;
private const int TransactionChunkSize = 100;
private readonly TableStorageService _storage;
private readonly ILogger<OccurredUtcBackfillService> _logger;
public OccurredUtcBackfillService(TableStorageService storage, ILogger<OccurredUtcBackfillService> logger)
{
_storage = storage;
_logger = logger;
}
public class BackfillResult
{
public string Table { get; set; } = string.Empty;
public bool DryRun { get; set; }
public int RowsExamined { get; set; }
public int WouldWrite { get; set; }
public int Written { get; set; }
public int SkippedAlreadySet { get; set; }
public int SkippedUndecodable { get; set; }
public int Errors { get; set; }
public string? NextContinuation { get; set; }
public List<BackfillSample> Samples { get; set; } = new();
}
public class BackfillSample
{
public string PartitionKey { get; set; } = string.Empty;
public string RowKey { get; set; } = string.Empty;
public DateTime DecodedUtc { get; set; }
}
internal enum RowDecision { AlreadySet, Write, Undecodable }
/// <summary>Pure per-row decision, extracted for unit testing.</summary>
internal static RowDecision DecideRow(string table, TableEntity entity, out DateTime occurredUtc)
{
occurredUtc = default;
if (BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn).HasValue)
return RowDecision.AlreadySet;
var decoded = table == TableAudit
? BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out occurredUtc)
: BusinessTimestamp.TryDecodeOpsRowKey(entity.RowKey, out occurredUtc);
return decoded ? RowDecision.Write : RowDecision.Undecodable;
}
public async Task<BackfillResult> RunAsync(string table, bool dryRun, int maxRows, string? continuation)
{
var (tableName, scanFilter) = table switch
{
// Audit: server-side restriction to time-encoded '!'-rows — legacy bare-GUID
// rows (which sort after '"') have nothing to decode and are never touched.
TableAudit => (Constants.TableNames.AuditLogs,
$"RowKey ge '{BusinessTimestamp.AuditRowKeyPrefix}' and RowKey lt '{BusinessTimestamp.AuditTimeEncodedUpperBound}'"),
TableOps => (Constants.TableNames.OpsEvents, (string?)null),
_ => throw new ArgumentException($"Unknown table '{table}'", nameof(table)),
};
var tableClient = _storage.GetTableClient(tableName);
var result = new BackfillResult { Table = table, DryRun = dryRun };
var (entities, nextToken) = await AzureTablesPaginator.FetchPageAsync<TableEntity>(
client: tableClient,
filter: scanFilter,
pageSize: maxRows,
continuation: continuation,
select: new[] { "PartitionKey", "RowKey", BusinessTimestamp.OccurredUtcColumn });
result.NextContinuation = nextToken;
var pending = new List<(string PartitionKey, string RowKey, DateTime OccurredUtc)>();
foreach (var entity in entities)
{
result.RowsExamined++;
switch (DecideRow(table, entity, out var occurredUtc))
{
case RowDecision.AlreadySet:
result.SkippedAlreadySet++;
break;
case RowDecision.Undecodable:
result.SkippedUndecodable++;
_logger.LogWarning("OccurredUtc backfill: undecodable RowKey {PK}/{RK} in {Table}",
entity.PartitionKey, entity.RowKey, tableName);
break;
case RowDecision.Write:
result.WouldWrite++;
pending.Add((entity.PartitionKey, entity.RowKey, occurredUtc));
if (result.Samples.Count < MaxSamplesInResponse)
{
result.Samples.Add(new BackfillSample
{
PartitionKey = entity.PartitionKey,
RowKey = entity.RowKey,
DecodedUtc = occurredUtc,
});
}
break;
}
}
if (!dryRun && pending.Count > 0)
result.Written = await WritePendingAsync(tableClient, pending, result);
_logger.LogInformation(
"OccurredUtc backfill {Table}: dryRun={DryRun} examined={Examined} wouldWrite={WouldWrite} written={Written} alreadySet={AlreadySet} undecodable={Undecodable} errors={Errors} more={More}",
table, dryRun, result.RowsExamined, result.WouldWrite, result.Written,
result.SkippedAlreadySet, result.SkippedUndecodable, result.Errors, result.NextContinuation != null);
return result;
}
private async Task<int> WritePendingAsync(TableClient tableClient,
List<(string PartitionKey, string RowKey, DateTime OccurredUtc)> pending, BackfillResult result)
{
var written = 0;
foreach (var partitionGroup in pending.GroupBy(p => p.PartitionKey))
{
foreach (var chunk in partitionGroup.Chunk(TransactionChunkSize))
{
// Merge entity carries ONLY the new column; unconditional (ETag *) merge is
// safe because the value is a pure function of the immutable RowKey.
var actions = chunk.Select(p => new TableTransactionAction(
TableTransactionActionType.UpdateMerge,
new TableEntity(p.PartitionKey, p.RowKey)
{
[BusinessTimestamp.OccurredUtcColumn] = p.OccurredUtc,
},
ETag.All)).ToList();
try
{
await tableClient.SubmitTransactionAsync(actions);
written += actions.Count;
}
catch (Exception ex)
{
result.Errors += actions.Count;
_logger.LogError(ex, "OccurredUtc backfill: merge transaction failed for partition {PK} ({Count} rows)",
partitionGroup.Key, actions.Count);
}
}
}
return written;
}
}
}
@@ -310,25 +310,25 @@ namespace AutopilotMonitor.Functions.Services
public Task RecordSignalRConnectionsHighAsync(int observed, int limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRConnectionsHigh", OpsEventSeverity.Warning,
$"SignalR concurrent connections at {percent}% of free-tier limit ({observed}/{limit}) - watch for 429s; consider scaling to Standard before saturation",
$"SignalR concurrent connections at {percent}% of plan limit ({observed}/{limit}) - watch for 429s; consider adding units before saturation",
null, "System.Monitoring",
new { metric = "ConnectionCount", aggregation = "Maximum", windowMinutes = 60, observed, limit, percent, resourceId });
public Task RecordSignalRConnectionsCriticalAsync(int observed, int limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRConnectionsCritical", OpsEventSeverity.Error,
$"CRITICAL: SignalR concurrent connections at {percent}% of free-tier limit ({observed}/{limit}) - new client connections will be 429'd at 100%; scale to Standard now",
$"CRITICAL: SignalR concurrent connections at {percent}% of plan limit ({observed}/{limit}) - new client connections will be 429'd at 100%; add units now",
null, "System.Monitoring",
new { metric = "ConnectionCount", aggregation = "Maximum", windowMinutes = 60, observed, limit, percent, resourceId });
public Task RecordSignalRMessagesHighAsync(long observed, long limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRMessagesHigh", OpsEventSeverity.Warning,
$"SignalR daily message count at {percent}% of free-tier limit ({observed}/{limit}) - quota resets at 00:00 UTC; consider scaling if pattern persists",
$"SignalR daily message count at {percent}% of included plan quota ({observed}/{limit}) - resets at 00:00 UTC; overage is billed per extra million messages",
null, "System.Monitoring",
new { metric = "MessageCount", aggregation = "Total", windowDay = "UTC", observed, limit, percent, resourceId });
public Task RecordSignalRMessagesCriticalAsync(long observed, long limit, int percent, string resourceId)
=> WriteAsync(OpsEventCategory.Security, "SignalRMessagesCritical", OpsEventSeverity.Error,
$"CRITICAL: SignalR daily message count at {percent}% of free-tier limit ({observed}/{limit}) - hub will throttle at 100% until 00:00 UTC reset; scale to Standard now",
$"CRITICAL: SignalR daily message count at {percent}% of included plan quota ({observed}/{limit}) - overage beyond 100% is billed per extra million messages; review traffic or add units",
null, "System.Monitoring",
new { metric = "MessageCount", aggregation = "Total", windowDay = "UTC", observed, limit, percent, resourceId });
@@ -1,5 +1,6 @@
using Azure;
using Azure.Data.Tables;
using AutopilotMonitor.Functions.Helpers;
using AutopilotMonitor.Functions.Pagination;
using AutopilotMonitor.Functions.Security;
using AutopilotMonitor.Shared;
@@ -55,7 +56,10 @@ namespace AutopilotMonitor.Functions.Services
{ "EntityType", entityType },
{ "EntityId", entityId },
{ "PerformedBy", performedBy },
{ "Timestamp", timestamp },
// Business event time. "Timestamp" is a reserved system property (server
// ignores supplied values and resets it on every row rewrite), so the
// event time must live in a regular custom column to survive migrations.
{ BusinessTimestamp.OccurredUtcColumn, timestamp },
{ "Details", details != null ? JsonConvert.SerializeObject(details) : string.Empty }
};
@@ -71,32 +75,26 @@ namespace AutopilotMonitor.Functions.Services
}
/// <summary>
/// Retention cleanup: deletes audit log entries whose Timestamp is older than the cutoff.
/// The AuditLogs table is append-only (one row per admin action) and is otherwise only
/// wiped on tenant offboarding, so without this it grows unbounded. Filters server-side on
/// the per-row "Timestamp" property and selects only PK/RK to keep the scan cheap.
/// Retention cleanup: deletes audit log entries whose business time is older than the
/// cutoff. The AuditLogs table is append-only (one row per admin action) and is otherwise
/// only wiped on tenant offboarding, so without this it grows unbounded.
/// Two passes: (1) time-encoded '!'-rows via an index-backed RowKey range — the system
/// Timestamp cannot be used because storage migrations reset it (post-migration it would
/// delete nothing for a retention period, then the whole pre-migration corpus at once);
/// (2) legacy bare-GUID rows (true event time unrecoverable) via their frozen system
/// Timestamp — they all age out ~180d after the 2026-07-18 migration.
/// TODO: Remove pass 2 after 2027-03-01 (no bare-GUID rows can remain by then).
/// </summary>
public async Task<int> DeleteAuditLogsOlderThanAsync(DateTime cutoffUtc)
{
try
{
var tableClient = _tableServiceClient.GetTableClient(Constants.TableNames.AuditLogs);
var filter = $"Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'";
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
var deleted = 0;
deleted += await DeleteAuditLogsByFilterAsync(tableClient, BusinessTimestamp.AuditRetentionClause(cutoffUtc));
// 'RowKey ge '0'' cannot match any '!'-row (0x21 < 0x30), only legacy GUIDs.
deleted += await DeleteAuditLogsByFilterAsync(tableClient,
$"RowKey ge '0' and Timestamp lt datetime'{cutoffUtc:yyyy-MM-ddTHH:mm:ss}Z'");
if (deleted > 0)
_logger.LogInformation("Deleted {Count} audit log entries older than {Cutoff:yyyy-MM-dd}", deleted, cutoffUtc);
@@ -110,6 +108,25 @@ namespace AutopilotMonitor.Functions.Services
}
}
private async Task<int> DeleteAuditLogsByFilterAsync(TableClient tableClient, string filter)
{
var query = tableClient.QueryAsync<TableEntity>(filter: filter, select: new[] { "PartitionKey", "RowKey" });
int deleted = 0;
await foreach (var entity in query)
{
try
{
await tableClient.DeleteEntityAsync(entity.PartitionKey, entity.RowKey);
deleted++;
}
catch (Exception ex)
{
_logger.LogWarning(ex, "Failed to delete audit log {PK}/{RK}", entity.PartitionKey, entity.RowKey);
}
}
return deleted;
}
/// <summary>
/// Gets audit log entries for a tenant within an optional UTC date window.
/// No row cap — returns the full filtered set, sorted newest-first.
@@ -384,9 +401,9 @@ namespace AutopilotMonitor.Functions.Services
if (!string.IsNullOrEmpty(lastRowKey))
clauses.Add($"RowKey gt '{lastRowKey!.Replace("'", "''")}'");
if (dateFrom.HasValue)
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
if (dateTo.HasValue)
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
return string.Join(" and ", clauses);
}
@@ -398,14 +415,30 @@ namespace AutopilotMonitor.Functions.Services
EntityType = entity.GetString("EntityType") ?? string.Empty,
EntityId = entity.GetString("EntityId") ?? string.Empty,
PerformedBy = entity.GetString("PerformedBy") ?? string.Empty,
Timestamp = entity.GetDateTimeOffset("Timestamp")?.UtcDateTime ?? DateTime.UtcNow,
Timestamp = ResolveAuditTimestamp(entity),
Details = entity.GetString("Details") ?? string.Empty,
};
// The system Timestamp is auto-managed by Azure on insert and reliably
// sortable; the user-defined "Timestamp" property is set by LogAuditEntry
// for parity with the model. We filter on the system Timestamp since it
// is always indexed.
/// <summary>
/// Business time of an audit row: OccurredUtc column → time decoded from the
/// reverse-tick RowKey → system Timestamp. The system Timestamp is only a last
/// resort (legacy bare-GUID rows): it is reset by any row rewrite, so for rows
/// migrated between storage accounts it holds the migration moment, not the
/// event time.
/// </summary>
internal static DateTime ResolveAuditTimestamp(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeAuditRowKey(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp") ?? DateTime.UtcNow;
}
// Date windows filter on the RowKey (reverse-tick, index-backed) rather than any
// timestamp property: the system Timestamp is reset by migrations, and an
// OccurredUtc property filter would exclude rows that lack the column.
internal static string? BuildAuditLogFilter(string? tenantId, DateTime? dateFrom, DateTime? dateTo,
bool excludeDeletions = false, AuditLogQueryFilters? filters = null)
{
@@ -424,11 +457,11 @@ namespace AutopilotMonitor.Functions.Services
}
if (dateFrom.HasValue)
{
clauses.Add($"Timestamp ge datetime'{ToUtc(dateFrom.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateFromClause(ToUtc(dateFrom.Value)));
}
if (dateTo.HasValue)
{
clauses.Add($"Timestamp le datetime'{ToUtc(dateTo.Value):o}'");
clauses.Add(BusinessTimestamp.AuditDateToClause(ToUtc(dateTo.Value)));
}
return string.Join(" and ", clauses);
}
@@ -158,6 +158,9 @@ namespace AutopilotMonitor.Functions.Services
["IntervalSeconds"] = rule.IntervalSeconds,
["TriggerPhase"] = rule.TriggerPhase ?? string.Empty,
["TriggerEventType"] = rule.TriggerEventType ?? string.Empty,
["ActivePhasesJson"] = JsonConvert.SerializeObject(rule.ActivePhases ?? new List<string>()),
["ActiveFromPhase"] = rule.ActiveFromPhase ?? string.Empty,
["EmitMode"] = rule.EmitMode ?? string.Empty,
["OutputEventType"] = rule.OutputEventType ?? string.Empty,
["OutputSeverity"] = rule.OutputSeverity ?? "Info",
["TagsJson"] = JsonConvert.SerializeObject(rule.Tags ?? new string[0]),
@@ -222,7 +225,9 @@ namespace AutopilotMonitor.Functions.Services
}
}
private GatherRule MapToGatherRule(TableEntity entity)
// Internal (not private) so the Store↔Map roundtrip is testable per the
// table-serialization rule: every model field must appear in BOTH directions.
internal GatherRule MapToGatherRule(TableEntity entity)
{
return new GatherRule
{
@@ -244,6 +249,9 @@ namespace AutopilotMonitor.Functions.Services
IntervalSeconds = entity.GetInt32("IntervalSeconds"),
TriggerPhase = entity.GetString("TriggerPhase") ?? string.Empty,
TriggerEventType = entity.GetString("TriggerEventType") ?? string.Empty,
ActivePhases = MapActivePhases(entity.GetString("ActivePhasesJson")),
ActiveFromPhase = string.IsNullOrEmpty(entity.GetString("ActiveFromPhase")) ? null : entity.GetString("ActiveFromPhase"),
EmitMode = string.IsNullOrEmpty(entity.GetString("EmitMode")) ? null : entity.GetString("EmitMode"),
OutputEventType = entity.GetString("OutputEventType") ?? string.Empty,
OutputSeverity = entity.GetString("OutputSeverity") ?? "Info",
Tags = DeserializeJsonArray(entity.GetString("TagsJson")),
@@ -252,6 +260,17 @@ namespace AutopilotMonitor.Functions.Services
};
}
/// <summary>
/// Absent column (pre-existing rows) and the stored-empty "[]" both map to null —
/// null and empty are the same "unrestricted" state on <see cref="GatherRule.ActivePhases"/>,
/// and null keeps <see cref="GatherRuleService.ContentEquivalent"/> reseed-stable.
/// </summary>
private List<string>? MapActivePhases(string? json)
{
var phases = DeserializeJson<List<string>>(json);
return phases.Count == 0 ? null : phases;
}
// ===== RULE STATES METHODS =====
/// <summary>
@@ -681,7 +681,10 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// "Timestamp" is a reserved system property — supplied values are ignored
// and any row rewrite (storage migration) resets it. The sanitized agent
// event time therefore lives in its own column.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -777,7 +780,8 @@ namespace AutopilotMonitor.Functions.Services
["EventId"] = evt.EventId,
["SessionId"] = evt.SessionId,
["TenantId"] = evt.TenantId,
["Timestamp"] = evt.Timestamp,
// Reserved-name caveat: see StoreEventAsync.
[BusinessTimestamp.OccurredUtcColumn] = EnsureUtc(evt.Timestamp),
["EventType"] = evt.EventType ?? string.Empty,
["Severity"] = (int)evt.Severity,
["Source"] = evt.Source ?? string.Empty,
@@ -2673,10 +2677,7 @@ namespace AutopilotMonitor.Functions.Services
EventId = entity.GetString("EventId") ?? string.Empty,
SessionId = entity.GetString("SessionId") ?? string.Empty,
TenantId = tenantId ?? partitionKey,
Timestamp = DateTime.SpecifyKind(
entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp")
?? DateTime.UtcNow, DateTimeKind.Utc),
Timestamp = DateTime.SpecifyKind(ResolveEventTimestamp(entity), DateTimeKind.Utc),
EventType = entity.GetString("EventType") ?? string.Empty,
Severity = (EventSeverity)(entity.GetInt32("Severity") ?? 0),
Source = entity.GetString("Source") ?? string.Empty,
@@ -2827,7 +2828,10 @@ namespace AutopilotMonitor.Functions.Services
/// <summary>
/// Returns the earliest event timestamp persisted for a session, if any.
/// Events are written with RowKey "{Timestamp}_{Sequence}", so querying the partition
/// and taking the first row yields the earliest event.
/// and taking the first row yields the earliest event. Resolution must NOT rely on
/// the system Timestamp: it is reset by storage migrations, and this value feeds the
/// StartedAt/Duration reconciliation — a migration-reset read would rewrite those
/// session fields with the migration date.
/// </summary>
private async Task<DateTime?> GetEarliestSessionEventTimestampAsync(string tenantId, string sessionId)
{
@@ -2838,13 +2842,12 @@ namespace AutopilotMonitor.Functions.Services
var query = tableClient.QueryAsync<TableEntity>(
filter: $"PartitionKey eq '{partitionKey}'",
maxPerPage: 1,
select: new[] { "Timestamp", "RowKey" }
select: new[] { "RowKey", BusinessTimestamp.OccurredUtcColumn, "Timestamp" }
);
await foreach (var entity in query)
{
return entity.GetDateTimeOffset("Timestamp")?.UtcDateTime
?? entity.GetDateTime("Timestamp");
return ResolveEventTimestampOrNull(entity);
}
}
catch (Exception ex)
@@ -2855,6 +2858,26 @@ namespace AutopilotMonitor.Functions.Services
return null;
}
/// <summary>
/// Business time of an event row: OccurredUtc column → agent time decoded from the
/// "{yyyyMMddHHmmssfff}_{seq}" RowKey → system Timestamp (last resort — reset by any
/// row rewrite, e.g. a storage migration; for migrated rows it holds the migration
/// moment). Event rows are deliberately never backfilled, so the RowKey decode is the
/// primary path for all pre-OccurredUtc rows.
/// </summary>
internal static DateTime ResolveEventTimestamp(TableEntity entity)
=> ResolveEventTimestampOrNull(entity) ?? DateTime.UtcNow;
internal static DateTime? ResolveEventTimestampOrNull(TableEntity entity)
{
var occurred = BusinessTimestamp.GetUtcDateTime(entity, BusinessTimestamp.OccurredUtcColumn);
if (occurred.HasValue)
return occurred.Value;
if (BusinessTimestamp.TryDecodeEventRowKeyPrefix(entity.RowKey, out var decoded))
return decoded;
return BusinessTimestamp.GetUtcDateTime(entity, "Timestamp");
}
/// <summary>
/// Returns the authoritative (total, reboot) event-row counts for a session, or <c>null</c>
/// if they could not be determined (storage error). Events are written with UpsertReplace
@@ -6,9 +6,9 @@
"AzureTableStorageConnectionString": "UseDevelopmentStorage=true",
"AzureBlobStorageConnectionString": "UseDevelopmentStorage=true",
"// SignalR free-tier quota watcher": "Optional. Set to the full Azure resource id of the SignalR Service to enable hourly OpsEvents when the 20-connections / 20k-messages-per-day free-tier limits are approached. Function App's managed identity needs 'Monitoring Reader' on this resource. Leave unset to skip the watcher.",
"// SignalR quota watcher": "Optional. Set to the full Azure resource id of the SignalR Service to enable hourly OpsEvents when the plan limits are approached (defaults: Standard S1 x1 unit = 1000 connections / 1M messages per day; override the limits when adding units or changing tier). Function App's managed identity needs 'Monitoring Reader' on this resource. Leave unset to skip the watcher.",
"SignalRResourceId": "",
"SignalRFreeTierConnectionLimit": "20",
"SignalRFreeTierDailyMessageLimit": "20000"
"SignalRConnectionLimit": "1000",
"SignalRDailyMessageLimit": "1000000"
}
}
@@ -24,6 +24,7 @@ export const EVENT_TYPES_CATALOG = {
'esp_failure_settle_started',
'esp_failure_settle_recovered',
'esp_exiting',
'mdm_policy_reboot_required',
'esp_config_detected',
'completion_check',
'completion_waiting',
@@ -73,19 +73,32 @@ namespace AutopilotMonitor.Shared
public const string ApiBaseUrl = "https://autopilotmonitor-api-eu.azurewebsites.net";
/// <summary>
/// Public base URL of the MCP server Container App (no trailing slash, no /mcp suffix).
/// Used by the backend health check to probe <c>{McpServerBaseUrl}/health</c>. The
/// Container App runs with minReplicas=0, so the first probe after idle may incur a
/// cold start. Overridable via the <c>McpServerUrl</c> app setting (e.g. for dev).
/// Public base URL of the MCP server (no trailing slash, no /mcp suffix). Stable custom
/// domain in front of the Container App — survives region/environment moves, unlike the
/// generated *.azurecontainerapps.io FQDN. Used by the backend health check to probe
/// <c>{McpServerBaseUrl}/health</c>. The Container App runs with minReplicas=0, so the
/// first probe after idle may incur a cold start. Overridable via the
/// <c>McpServerUrl</c> app setting (e.g. for dev).
/// </summary>
public const string McpServerBaseUrl = "https://autopilotmonitor-mcp.kindwave-58b4b547.westeurope.azurecontainerapps.io";
public const string McpServerBaseUrl = "https://mcp.autopilotmonitor.com";
// -----------------------------------------------------------------------
// Agent self-update
// -----------------------------------------------------------------------
/// <summary>
/// Base URL for agent binaries in Azure Blob Storage (public read)
/// Canonical public base URL for agent binaries: Front Door alias in front of the
/// current blob origin (autopilotmonitoreu, container "agent"). Storage-account moves
/// happen behind this name without touching customers. New bootstrap scripts and all
/// availability probes use this URL.
/// </summary>
public const string AgentDownloadBaseUrl = "https://download.autopilotmonitor.com/agent";
/// <summary>
/// LEGACY base URL for agent binaries: the pre-migration storage account, kept alive
/// (dual-push on every release) because bootstrap scripts already deployed in customer
/// Intune tenants still download from it. Remove only after the customer migration to
/// <see cref="AgentDownloadBaseUrl"/> is complete and the legacy account is torn down.
/// </summary>
public const string AgentBlobBaseUrl = "https://autopilotmonitor.blob.core.windows.net/agent";
@@ -392,6 +405,18 @@ namespace AutopilotMonitor.Shared
public const string OsBuildChanged = "os_build_changed"; // OS build (CurrentBuild.UBR) differs across an agent restart — an update was installed during enrollment
public const string WindowsUpdateChannelCensus = "windows_update_channel_census"; // One-shot EventID histogram of WU Client + UpdateOrchestrator channels when os_build_changed but zero targeted WU events captured
// MDM reboot-required policy flags (MdmRebootPolicyTracker; PMPC "unexpected reboot /
// second sign-in" research). DM-Enterprise EventID 2800 fires once per policy URI that
// matches the OS reboot-required catalog ("The following URI has triggered a reboot:
// (./Device/...)."; verified session b2e890c1). Only URIs applied DURING ESP DeviceSetup
// cause the coalesced mid-ESP reboot — later syncs (AccountSetup+) merely flag, no forced
// reboot. The tracker therefore emits ONE aggregated Info event per burst with the URI
// list; the reboot CLAIM lives in ANALYZE-ESP-005, gated on an actually-observed
// system_reboot_detected. Deliberately DISTINCT from system_reboot_detected: the backend
// RebootCount matches that string exactly (EventIngestProcessor.Classification + terminal
// recount) — this attribution event must never inflate the reboot count.
public const string MdmPolicyRebootRequired = "mdm_policy_reboot_required"; // Aggregated per burst — Data.rebootUris[] + uriCount + firstRebootUri
// TEMPORARY: shadow SM rollout verification — remove when CompletionStateMachine is promoted to primary
public const string ShadowDiscrepancy = "shadow_discrepancy";
@@ -324,6 +324,57 @@ namespace AutopilotMonitor.Shared.Models
}
}
// ===== AGENT ENDPOINT MIGRATION (config-channel re-home) =====
// The agent's API base URL is compiled into the binary (no custom domain possible in
// front of the API — mTLS breaks behind a proxy, Flex custom-domain TLS is broken).
// These settings let the OLD backend re-home agents to a NEW base URL via the
// agent/config response during a backend or per-tenant region move. Values are
// validated against AgentEndpointMigrationRules before being served; invalid values
// are logged and ignored (fail-safe: no migration).
/// <summary>
/// Global migration target served to ALL agents as
/// <c>AgentConfigResponse.MigrateToApiBaseUrl</c> (e.g.
/// "https://autopilotmonitor-api-us.azurewebsites.net"). Empty = no global migration.
/// Set this on the backend being ABANDONED; clear it once the migration window closes.
/// Per-tenant overrides in <see cref="AgentMigrateTenantOverridesJson"/> win over this.
/// </summary>
public string AgentMigrateApiBaseUrl { get; set; } = string.Empty;
/// <summary>
/// JSON object mapping tenantId → migration target URL for per-tenant moves (e.g. one
/// tenant relocating EU→US). An entry with an EMPTY string value pins that tenant to
/// the current backend even while <see cref="AgentMigrateApiBaseUrl"/> is set (staged
/// rollout). Example:
/// <c>{"11111111-...": "https://autopilotmonitor-api-us.azurewebsites.net", "22222222-...": ""}</c>
/// </summary>
public string AgentMigrateTenantOverridesJson { get; set; } = string.Empty;
/// <summary>
/// Returns the deserialized per-tenant migration overrides (case-insensitive tenant
/// keys). Falls back to an empty map for null/empty/invalid JSON so a malformed value
/// degrades to "global setting applies" instead of failing config serving.
/// </summary>
public Dictionary<string, string> GetAgentMigrateTenantOverrides()
{
var empty = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (string.IsNullOrWhiteSpace(AgentMigrateTenantOverridesJson))
return empty;
try
{
var parsed = JsonSerializer.Deserialize<Dictionary<string, string>>(
AgentMigrateTenantOverridesJson,
new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
return parsed == null
? empty
: new Dictionary<string, string>(parsed, StringComparer.OrdinalIgnoreCase);
}
catch
{
return empty;
}
}
// ===== MODERN DEPLOYMENT NOISE SUPPRESSION =====
/// <summary>
@@ -231,6 +231,18 @@ namespace AutopilotMonitor.Shared.Models
/// null = indefinite (or kill).
/// </summary>
public System.DateTime? UnblockAt { get; set; }
/// <summary>
/// Endpoint migration on the control channel: when set, the agent abandons its
/// compiled-in API base URL and re-runs backend-client construction + config fetch
/// against this URL (backend move without a custom domain in front of the API, or a
/// per-tenant region move). Honoured only from a LIVE fetch (the on-disk cache strips
/// the field on write and read), validated against
/// <c>AgentEndpointMigrationRules</c> on both sides, single hop (the re-fetched
/// config cannot chain another migration), and a kill signal always wins over it.
/// Old binaries ignore the field. null = no migration.
/// </summary>
public string? MigrateToApiBaseUrl { get; set; }
}
/// <summary>
@@ -470,6 +482,28 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public bool WindowsUpdateChannelCensusEnabled { get; set; } = true;
// -----------------------------------------------------------------------
// MDM reboot-policy watcher (DeviceManagement-Enterprise-Diagnostics-Provider/Admin)
// -----------------------------------------------------------------------
/// <summary>
/// Master switch for the MDM reboot-policy watcher. When enabled the agent subscribes to
/// <c>Microsoft-Windows-DeviceManagement-Enterprise-Diagnostics-Provider/Admin</c> EventID
/// 2800 and emits one <c>mdm_policy_reboot_required</c> event per device-assigned policy
/// URI that forces a coalesced reboot during ESP DeviceSetup (the "unexpected reboot +
/// second sign-in" pattern). Internal steering lever only — not surfaced in any admin UI;
/// disable by pushing config server-side. Default: true.
/// </summary>
public bool MdmRebootPolicyWatcherEnabled { get; set; } = true;
/// <summary>
/// Lookback window in minutes for the MDM reboot-policy backfill scan on startup. Device
/// policies can apply before the agent starts, and after the coalesced reboot the restarted
/// agent must re-scan past the reboot gap (the cross-restart watermark keeps re-reads from
/// re-emitting). 0 = backfill disabled. Default: 60 minutes.
/// </summary>
public int MdmRebootPolicyBackfillLookbackMinutes { get; set; } = 60;
/// <summary>
/// Creates default collector configuration
/// </summary>
@@ -128,6 +128,38 @@ namespace AutopilotMonitor.Shared.Models
/// </summary>
public string TriggerEventType { get; set; } = default!;
/// <summary>
/// Restricts the rule to run only while the current enrollment phase is one of these
/// phases. Canonical tokens are the <see cref="EnrollmentPhase"/> enum names
/// ("Start", "DevicePreparation", "DeviceSetup", "AppsDevice", "AccountSetup",
/// "AppsUser", "FinalizingSetup", "Complete"); "Unknown" and "Failed" are rejected
/// by backend validation. Null or empty = unrestricted (runs in every phase —
/// legacy behavior). Mutually exclusive with <see cref="ActiveFromPhase"/>;
/// if both are set the agent defensively prefers this list.
/// Applies to ALL trigger types. Before the first phase signal of a session,
/// scoped rules are inactive.
/// </summary>
public List<string>? ActivePhases { get; set; }
/// <summary>
/// Activates the rule once the enrollment phase first reaches this phase
/// (ordinal comparison on <see cref="EnrollmentPhase"/>, ignoring Unknown/Failed),
/// then keeps it active for the rest of the session (sticky latch — including
/// through Failed). Canonical tokens as in <see cref="ActivePhases"/>.
/// Null = unrestricted. Mutually exclusive with <see cref="ActivePhases"/>.
/// </summary>
public string? ActiveFromPhase { get; set; }
/// <summary>
/// Emit behavior for collected results:
/// null / "always" = emit on every collection (legacy behavior);
/// "on_change" = poll on the trigger cadence but emit only when the collected
/// result differs from the last emitted one. The first in-scope result always
/// emits; the suppressed poll count is carried on the next emitted event
/// (suppressedPolls / suppressedSinceUtc in the event data).
/// </summary>
public string? EmitMode { get; set; }
// ===== OUTPUT =====
/// <summary>
@@ -0,0 +1,100 @@
using System;
namespace AutopilotMonitor.Shared.Services
{
/// <summary>
/// Validation rules for the config-channel agent endpoint migration target
/// (<c>AgentConfigResponse.MigrateToApiBaseUrl</c>). Single source of truth used by BOTH
/// sides — the backend refuses to serve an invalid target, the agent refuses to honour
/// one — so a misconfigured admin value or a tampered response can never re-home agents
/// to an arbitrary host (defence in depth, same philosophy as the kill-switch hardening).
/// </summary>
public static class AgentEndpointMigrationRules
{
/// <summary>
/// Host suffixes a migration target may resolve to. Deliberately narrow: the platform
/// only ever runs its API on Azure Functions default hostnames (a custom domain in
/// front of the API is impossible — mTLS client-cert auth does not survive a proxy,
/// and Flex Consumption custom-domain TLS is broken). Extend here if that ever changes;
/// old agent builds keep their compiled-in list, so plan one release of lead time.
/// </summary>
public static readonly string[] AllowedHostSuffixes = { ".azurewebsites.net" };
/// <summary>Defensive cap — a legitimate Functions hostname URL is far shorter.</summary>
public const int MaxTargetLength = 200;
/// <summary>
/// Validates <paramref name="candidate"/> as a migration target and returns the
/// normalized base URL (<c>https://{lowercase-host}</c>, no trailing slash) on success.
/// Rules: absolute https URI, default port, no userinfo/path/query/fragment, host
/// matching <see cref="AllowedHostSuffixes"/>.
/// </summary>
public static bool TryNormalizeTarget(string candidate, out string normalized)
{
normalized = null;
if (string.IsNullOrWhiteSpace(candidate) || candidate.Length > MaxTargetLength)
return false;
if (!Uri.TryCreate(candidate.Trim(), UriKind.Absolute, out var uri))
return false;
if (!string.Equals(uri.Scheme, Uri.UriSchemeHttps, StringComparison.OrdinalIgnoreCase))
return false;
if (!uri.IsDefaultPort)
return false;
if (!string.IsNullOrEmpty(uri.UserInfo))
return false;
// Base URL only — endpoint paths are appended by the agent's clients.
if (uri.AbsolutePath != "/" || !string.IsNullOrEmpty(uri.Query) || !string.IsNullOrEmpty(uri.Fragment))
return false;
var host = uri.Host;
var allowed = false;
foreach (var suffix in AllowedHostSuffixes)
{
// Suffix match must sit on a label boundary and leave a non-empty prefix:
// "evil-azurewebsites.net" and ".azurewebsites.net" itself are both rejected.
if (host.Length > suffix.Length
&& host.EndsWith(suffix, StringComparison.OrdinalIgnoreCase))
{
allowed = true;
break;
}
}
if (!allowed)
return false;
normalized = "https://" + host.ToLowerInvariant();
return true;
}
/// <summary>
/// <c>true</c> when <paramref name="candidate"/> is a valid target that actually
/// differs from <paramref name="currentBaseUrl"/> (both compared in normalized form —
/// serving the agent its own current URL is a no-op, not a migration).
/// </summary>
public static bool IsEffectiveMigration(string candidate, string currentBaseUrl, out string normalizedTarget)
{
normalizedTarget = null;
if (!TryNormalizeTarget(candidate, out var target))
return false;
// The current base URL may not itself pass the allowlist (dev --api-url override);
// fall back to a plain trim-and-lower comparison in that case.
var currentNormalized = TryNormalizeTarget(currentBaseUrl, out var normCurrent)
? normCurrent
: (currentBaseUrl ?? string.Empty).Trim().TrimEnd('/').ToLowerInvariant();
if (string.Equals(target, currentNormalized, StringComparison.OrdinalIgnoreCase))
return false;
normalizedTarget = target;
return true;
}
}
}
@@ -26,6 +26,10 @@ interface AdminConfigContextValue {
setGlobalAdminRateLimit: (value: number) => void;
platformStatsBlobSasUrl: string;
setPlatformStatsBlobSasUrl: (value: string) => void;
agentMigrateApiBaseUrl: string;
setAgentMigrateApiBaseUrl: (value: string) => void;
agentMigrateTenantOverridesJson: string;
setAgentMigrateTenantOverridesJson: (value: string) => void;
collectorIdleTimeoutMinutes: number;
setCollectorIdleTimeoutMinutes: (value: number) => void;
desktopDetectorNoCandidateTimeoutMinutes: number;
@@ -130,6 +134,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
const [userRateLimit, setUserRateLimit] = useState(120);
const [globalAdminRateLimit, setGlobalAdminRateLimit] = useState(600);
const [platformStatsBlobSasUrl, setPlatformStatsBlobSasUrl] = useState("");
const [agentMigrateApiBaseUrl, setAgentMigrateApiBaseUrl] = useState("");
const [agentMigrateTenantOverridesJson, setAgentMigrateTenantOverridesJson] = useState("");
const [collectorIdleTimeoutMinutes, setCollectorIdleTimeoutMinutes] = useState(15);
const [desktopDetectorNoCandidateTimeoutMinutes, setDesktopDetectorNoCandidateTimeoutMinutes] = useState(10);
const [maxSessionWindowHours, setMaxSessionWindowHours] = useState(24);
@@ -191,6 +197,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
setUserRateLimit(data.userRateLimitRequestsPerMinute ?? 120);
setGlobalAdminRateLimit(data.globalAdminRateLimitRequestsPerMinute ?? 600);
setPlatformStatsBlobSasUrl(data.platformStatsBlobSasUrl ?? "");
setAgentMigrateApiBaseUrl(data.agentMigrateApiBaseUrl ?? "");
setAgentMigrateTenantOverridesJson(data.agentMigrateTenantOverridesJson ?? "");
setCollectorIdleTimeoutMinutes(data.collectorIdleTimeoutMinutes ?? 15);
setDesktopDetectorNoCandidateTimeoutMinutes(data.desktopDetectorNoCandidateTimeoutMinutes ?? 10);
setMaxSessionWindowHours(data.maxSessionWindowHours ?? 24);
@@ -296,6 +304,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
userRateLimitRequestsPerMinute: userRateLimit,
globalAdminRateLimitRequestsPerMinute: globalAdminRateLimit,
platformStatsBlobSasUrl: platformStatsBlobSasUrl.trim(),
agentMigrateApiBaseUrl: agentMigrateApiBaseUrl.trim(),
agentMigrateTenantOverridesJson: agentMigrateTenantOverridesJson.trim(),
collectorIdleTimeoutMinutes,
desktopDetectorNoCandidateTimeoutMinutes,
maxSessionWindowHours,
@@ -332,7 +342,7 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
} finally {
setSavingConfig(false);
}
}, [isGlobalAdmin, adminConfig, globalRateLimit, userRateLimit, globalAdminRateLimit, platformStatsBlobSasUrl, collectorIdleTimeoutMinutes, desktopDetectorNoCandidateTimeoutMinutes, maxSessionWindowHours, maintenanceBlockDurationHours, opsEventRetentionDays, slaNotificationCooldownHours, allowAgentDowngrade, modernDeploymentHarmlessEventIds, enableIndexDualWrite, sessionDeletionKillSwitch, getAccessToken]);
}, [isGlobalAdmin, adminConfig, globalRateLimit, userRateLimit, globalAdminRateLimit, platformStatsBlobSasUrl, agentMigrateApiBaseUrl, agentMigrateTenantOverridesJson, collectorIdleTimeoutMinutes, desktopDetectorNoCandidateTimeoutMinutes, maxSessionWindowHours, maintenanceBlockDurationHours, opsEventRetentionDays, slaNotificationCooldownHours, allowAgentDowngrade, modernDeploymentHarmlessEventIds, enableIndexDualWrite, sessionDeletionKillSwitch, getAccessToken]);
// Reset admin config
const handleResetAdminConfig = useCallback(() => {
@@ -341,6 +351,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
setUserRateLimit(adminConfig.userRateLimitRequestsPerMinute ?? 120);
setGlobalAdminRateLimit(adminConfig.globalAdminRateLimitRequestsPerMinute ?? 600);
setPlatformStatsBlobSasUrl(adminConfig.platformStatsBlobSasUrl ?? "");
setAgentMigrateApiBaseUrl(adminConfig.agentMigrateApiBaseUrl ?? "");
setAgentMigrateTenantOverridesJson(adminConfig.agentMigrateTenantOverridesJson ?? "");
setCollectorIdleTimeoutMinutes(adminConfig.collectorIdleTimeoutMinutes ?? 15);
setDesktopDetectorNoCandidateTimeoutMinutes(adminConfig.desktopDetectorNoCandidateTimeoutMinutes ?? 10);
setMaxSessionWindowHours(adminConfig.maxSessionWindowHours ?? 24);
@@ -485,6 +497,8 @@ export function AdminConfigProvider({ children }: { children: React.ReactNode })
userRateLimit, setUserRateLimit,
globalAdminRateLimit, setGlobalAdminRateLimit,
platformStatsBlobSasUrl, setPlatformStatsBlobSasUrl,
agentMigrateApiBaseUrl, setAgentMigrateApiBaseUrl,
agentMigrateTenantOverridesJson, setAgentMigrateTenantOverridesJson,
collectorIdleTimeoutMinutes, setCollectorIdleTimeoutMinutes,
desktopDetectorNoCandidateTimeoutMinutes, setDesktopDetectorNoCandidateTimeoutMinutes,
maxSessionWindowHours, setMaxSessionWindowHours,
@@ -14,6 +14,10 @@ interface AdminConfigSettingsSectionProps {
setGlobalAdminRateLimit: (value: number) => void;
platformStatsBlobSasUrl: string;
setPlatformStatsBlobSasUrl: (value: string) => void;
agentMigrateApiBaseUrl: string;
setAgentMigrateApiBaseUrl: (value: string) => void;
agentMigrateTenantOverridesJson: string;
setAgentMigrateTenantOverridesJson: (value: string) => void;
collectorIdleTimeoutMinutes: number;
setCollectorIdleTimeoutMinutes: (value: number) => void;
desktopDetectorNoCandidateTimeoutMinutes: number;
@@ -44,6 +48,10 @@ export function AdminConfigSettingsSection({
setGlobalAdminRateLimit,
platformStatsBlobSasUrl,
setPlatformStatsBlobSasUrl,
agentMigrateApiBaseUrl,
setAgentMigrateApiBaseUrl,
agentMigrateTenantOverridesJson,
setAgentMigrateTenantOverridesJson,
collectorIdleTimeoutMinutes,
setCollectorIdleTimeoutMinutes,
desktopDetectorNoCandidateTimeoutMinutes,
@@ -155,6 +163,43 @@ export function AdminConfigSettingsSection({
</label>
</div>
<div>
<label className="block">
<span className="text-indigo-900 dark:text-indigo-100 font-medium">Agent Endpoint Migration Target (Global)</span>
<p className="text-sm text-indigo-800 dark:text-gray-300 mb-2">
When set, this backend serves the URL as <code className="text-xs bg-indigo-100 dark:bg-indigo-900 dark:text-indigo-200 px-1 rounded">MigrateToApiBaseUrl</code> on
the agent config channel and agents re-home themselves to the new API at their next start.
Only <code className="text-xs bg-indigo-100 dark:bg-indigo-900 dark:text-indigo-200 px-1 rounded">https://*.azurewebsites.net</code> base
URLs are accepted (validated server- AND agent-side). Set this on the backend being <strong>abandoned</strong>; clear it after the migration window.
</p>
<input
type="url"
value={agentMigrateApiBaseUrl}
onChange={(e) => setAgentMigrateApiBaseUrl(e.target.value)}
placeholder="https://autopilotmonitor-api-us.azurewebsites.net"
className="mt-1 block w-full px-4 py-2 border border-indigo-300 dark:border-indigo-600 rounded-lg bg-white dark:bg-gray-700 text-gray-900 dark:text-gray-100 placeholder-gray-400 dark:placeholder-gray-500 focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors font-mono text-sm"
/>
</label>
</div>
<div>
<label className="block">
<span className="text-indigo-900 dark:text-indigo-100 font-medium">Agent Endpoint Migration &mdash; Per-Tenant Overrides (JSON)</span>
<p className="text-sm text-indigo-800 dark:text-gray-300 mb-2">
JSON object mapping tenantId to a target URL for per-tenant region moves (e.g. one tenant EU&rarr;US).
An entry with an <strong>empty string</strong> value pins that tenant to this backend even while the global target above is set (staged rollout).
Overrides win over the global target.
</p>
<textarea
value={agentMigrateTenantOverridesJson}
onChange={(e) => setAgentMigrateTenantOverridesJson(e.target.value)}
placeholder='{"11111111-2222-3333-4444-555555555555": "https://autopilotmonitor-api-us.azurewebsites.net", "66666666-7777-8888-9999-000000000000": ""}'
rows={3}
className="mt-1 block w-full px-4 py-2 border border-indigo-300 dark:border-indigo-600 rounded-lg bg-white dark:bg-gray-700 text-gray-900 dark:text-gray-100 placeholder-gray-400 dark:placeholder-gray-500 focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors font-mono text-sm"
/>
</label>
</div>
<div>
<label className="block">
<span className="text-indigo-900 dark:text-indigo-100 font-medium">Collector Idle Timeout (Minutes)</span>
@@ -11,6 +11,8 @@ export function SectionGlobalSettings() {
userRateLimit, setUserRateLimit,
globalAdminRateLimit, setGlobalAdminRateLimit,
platformStatsBlobSasUrl, setPlatformStatsBlobSasUrl,
agentMigrateApiBaseUrl, setAgentMigrateApiBaseUrl,
agentMigrateTenantOverridesJson, setAgentMigrateTenantOverridesJson,
collectorIdleTimeoutMinutes, setCollectorIdleTimeoutMinutes,
desktopDetectorNoCandidateTimeoutMinutes, setDesktopDetectorNoCandidateTimeoutMinutes,
slaNotificationCooldownHours, setSlaNotificationCooldownHours,
@@ -36,6 +38,10 @@ export function SectionGlobalSettings() {
setGlobalAdminRateLimit={setGlobalAdminRateLimit}
platformStatsBlobSasUrl={platformStatsBlobSasUrl}
setPlatformStatsBlobSasUrl={setPlatformStatsBlobSasUrl}
agentMigrateApiBaseUrl={agentMigrateApiBaseUrl}
setAgentMigrateApiBaseUrl={setAgentMigrateApiBaseUrl}
agentMigrateTenantOverridesJson={agentMigrateTenantOverridesJson}
setAgentMigrateTenantOverridesJson={setAgentMigrateTenantOverridesJson}
collectorIdleTimeoutMinutes={collectorIdleTimeoutMinutes}
setCollectorIdleTimeoutMinutes={setCollectorIdleTimeoutMinutes}
desktopDetectorNoCandidateTimeoutMinutes={desktopDetectorNoCandidateTimeoutMinutes}
@@ -1,7 +1,5 @@
import { ImageResponse } from "next/og";
export const runtime = "edge";
export const size = { width: 180, height: 180 };
export const contentType = "image/png";
@@ -14,7 +14,7 @@ const tips = [
"Share the Progress Portal link with your team \u2014 anyone can check enrollment status by serial number.",
"Missing an event in your sessions? Use Gather Rules to collect it.",
"Analyze Rules are continuously refined \u2014 analysis results may shift as rules improve. This is normal.",
"Check Known Issues in the docs for external changes (e.g.\u00a0Microsoft updates) that may affect your data.",
"Check Service Announcements in the docs for external changes (e.g.\u00a0Microsoft updates) that may affect your data.",
"Check the Intune bootstrap script (Install-AutopilotMonitor.ps1) from time to time \u2014 a newer version may be available in the repository.",
"Spotted a session with the wrong status? Use Report Session to flag it to the team.",
];
@@ -1,7 +1,6 @@
"use client";
import { Suspense, useCallback, useEffect, useMemo, useRef, useState } from "react";
import Link from "next/link";
import { useRouter, useSearchParams } from "next/navigation";
import { ProtectedRoute } from "../../components/ProtectedRoute";
import { GlobalAdminBanner } from "@/components/GlobalAdminBanner";
@@ -284,32 +283,6 @@ function HomeContent() {
{/* Main content */}
<main className={mainClassName}>
<div className="px-4 sm:px-0">
{/* Scheduled maintenance banner — TEMPORARY, remove after 2026-07-20 */}
<div className="mb-4 bg-amber-50 border border-amber-300 rounded-lg px-4 py-3 flex items-start gap-3 dark:bg-amber-950/30 dark:border-amber-700/50">
<svg className="w-4 h-4 text-amber-500 mt-0.5 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
<p className="text-sm text-amber-800 dark:text-amber-300">
<span className="font-semibold">Scheduled infrastructure maintenance.</span>{" "}
The platform is <span className="font-medium">not available</span> this weekend, from{" "}
<span className="font-medium">Sat 18 Jul, 00:00</span> until{" "}
<span className="font-medium">Mon 20 Jul, 00:00 CEST (UTC+2)</span> (expected).{" "}
The portal cannot be reached and the ingestion API is offline. Agents continue collecting
locally and re-sync once the platform is back online — no enrollment data is lost.
If the work finishes early, the platform will come back sooner; the completion will be
announced here and in the{" "}
<a
href="https://docs.autopilotmonitor.com/troubleshooting/service-announcements"
target="_blank"
rel="noopener noreferrer"
className="underline font-medium hover:text-amber-600 dark:hover:text-amber-200"
>
Service Announcements
</a>
.
</p>
</div>
{/* Feedback & bug report banner */}
<div className="mb-4 bg-blue-50 border border-blue-300 rounded-lg px-4 py-3 flex items-start gap-3 dark:bg-blue-950/30 dark:border-blue-700/50">
<svg className="w-4 h-4 text-blue-500 mt-0.5 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
@@ -334,7 +307,7 @@ function HomeContent() {
rel="noopener noreferrer"
className="underline font-medium hover:text-blue-600 dark:hover:text-blue-200"
>
Known Issues
Service Announcements
</a>
.{" "}
Feedback or bug report?{" "}
@@ -1,7 +1,7 @@
"use client";
import { useMemo, useState } from "react";
import { GatherRule, NewRuleForm, CATEGORY_COLORS, COLLECTOR_TYPE_LABELS, EMPTY_FORM, formatTrigger } from "../types";
import { GatherRule, NewRuleForm, CATEGORY_COLORS, COLLECTOR_TYPE_LABELS, EMPTY_FORM, formatTrigger, formatGatherPhase, withDerivedScopeMode } from "../types";
import { GatherRuleFormFields } from "./GatherRuleFormFields";
import { FormJsonToggle, JsonModeToggleButtons, ReadOnlyJsonView } from "@/components/rules/FormJsonToggle";
import { validateGatherRuleTarget } from "@/utils/guardValidation";
@@ -301,6 +301,19 @@ export function GatherRuleCard({
on event: <code className="px-1 bg-gray-200 rounded text-xs">{rule.triggerEventType}</code>
</span>
)}
{rule.activePhases && rule.activePhases.length > 0 && (
<span className="text-gray-500">
· during {rule.activePhases.map(formatGatherPhase).join(", ")}
</span>
)}
{!rule.activePhases?.length && rule.activeFromPhase && (
<span className="text-gray-500">
· from {formatGatherPhase(rule.activeFromPhase)}
</span>
)}
{rule.emitMode === "on_change" && (
<span className="text-gray-500">· emit on change</span>
)}
</div>
</div>
</div>
@@ -430,7 +443,7 @@ export function GatherRuleCard({
if (jsonModeEdit) {
try {
const parsed = JSON.parse(jsonText) as NewRuleForm;
onSaveEdit(rule, { ...editForm, ...parsed });
onSaveEdit(rule, withDerivedScopeMode({ ...editForm, ...parsed }));
} catch (e) {
// jsonError is handled by parent
}
@@ -10,6 +10,8 @@ import {
SEVERITIES,
TARGET_PLACEHOLDERS,
TARGET_HINTS,
GATHER_PHASES,
EMIT_MODES,
formatTrigger,
} from "../types";
import { KNOWN_EVENT_TYPES, findEventType } from "../eventTypes";
@@ -426,14 +428,20 @@ export function GatherRuleFormFields({ form, setForm, showRuleId, unrestrictedMo
{form.trigger === "phase_change" && (
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">Trigger Phase</label>
<input
type="text"
<select
value={form.triggerPhase}
onChange={(e) => setForm({ ...form, triggerPhase: e.target.value })}
placeholder="e.g., DeviceESP, AccountESP, DevicePreparation"
autoComplete="off"
className="w-full px-4 py-2 border border-gray-300 rounded-lg text-sm text-gray-900 placeholder-gray-500 focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors"
/>
className="w-full px-4 py-2 border border-gray-300 rounded-lg text-sm text-gray-900 bg-white focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors"
>
<option value="">Any phase change</option>
{form.triggerPhase && !GATHER_PHASES.some((p) => p.value === form.triggerPhase) && (
<option value={form.triggerPhase}>{form.triggerPhase} (legacy value)</option>
)}
{GATHER_PHASES.map((p) => (
<option key={p.value} value={p.value}>{p.label}</option>
))}
</select>
<p className="text-xs text-gray-400 mt-1">Run this rule when the enrollment reaches this phase. &quot;Any phase change&quot; fires once per phase transition.</p>
</div>
)}
@@ -472,6 +480,89 @@ export function GatherRuleFormFields({ form, setForm, showRuleId, unrestrictedMo
</div>
)}
{/* Row: Active During (phase scope) + Emit Mode */}
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">Active During</label>
<select
value={form.scopeMode}
onChange={(e) => {
const scopeMode = e.target.value as NewRuleForm["scopeMode"];
// Keep the form state consistent: only the fields of the selected mode survive.
setForm({
...form,
scopeMode,
activePhases: scopeMode === "during" ? form.activePhases : [],
activeFromPhase: scopeMode === "from" ? form.activeFromPhase : "",
});
}}
className="w-full px-4 py-2 border border-gray-300 rounded-lg text-sm text-gray-900 bg-white focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors"
>
<option value="always">All phases (always)</option>
<option value="during">Only during specific phases</option>
<option value="from">From a phase onwards</option>
</select>
<p className="text-xs text-gray-400 mt-1">Restrict when this rule runs. Outside its scope the rule is idle — interval rules stop polling, triggers are ignored.</p>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">Emit Mode</label>
<select
value={form.emitMode}
onChange={(e) => setForm({ ...form, emitMode: e.target.value })}
className="w-full px-4 py-2 border border-gray-300 rounded-lg text-sm text-gray-900 bg-white focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors"
>
{EMIT_MODES.map((m) => (
<option key={m.value} value={m.value}>{m.label}</option>
))}
</select>
<p className="text-xs text-gray-400 mt-1">On change: polls on the trigger cadence but only emits an event when the collected result changes. The first collection always emits.</p>
</div>
</div>
{form.scopeMode === "during" && (
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">Active Phases</label>
<div className="grid grid-cols-2 sm:grid-cols-4 gap-2">
{GATHER_PHASES.map((p) => (
<label key={p.value} className="flex items-center space-x-2 text-sm text-gray-700 cursor-pointer">
<input
type="checkbox"
checked={form.activePhases.includes(p.value)}
onChange={(e) =>
setForm({
...form,
activePhases: e.target.checked
? [...form.activePhases, p.value]
: form.activePhases.filter((x) => x !== p.value),
})
}
className="rounded border-gray-300 text-indigo-600 focus:ring-indigo-500"
/>
<span>{p.label}</span>
</label>
))}
</div>
<p className="text-xs text-gray-400 mt-1">The rule runs only while the enrollment is in one of the selected phases. No selection = all phases.</p>
</div>
)}
{form.scopeMode === "from" && (
<div>
<label className="block text-sm font-medium text-gray-700 mb-1">Active From Phase</label>
<select
value={form.activeFromPhase}
onChange={(e) => setForm({ ...form, activeFromPhase: e.target.value })}
className="w-full px-4 py-2 border border-gray-300 rounded-lg text-sm text-gray-900 bg-white focus:outline-none focus:ring-2 focus:ring-indigo-500 focus:border-indigo-500 transition-colors"
>
<option value="">Select a phase…</option>
{GATHER_PHASES.map((p) => (
<option key={p.value} value={p.value}>{p.label}</option>
))}
</select>
<p className="text-xs text-gray-400 mt-1">Once the enrollment reaches this phase the rule activates and stays active for the rest of the session. No selection = all phases.</p>
</div>
)}
{/* Row 3: Output Event Type, Output Severity */}
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
@@ -16,7 +16,7 @@ import { FormJsonToggle, JsonModeToggleButtons } from "@/components/rules/FormJs
import { useAuthenticatedFetch, useNotificationMessages, useGlobalAdminScope } from "@/hooks";
import { GlobalAdminBanner, globalAdminSubtitle } from "@/components/GlobalAdminBanner";
import { TenantScopeSelector } from "@/components/TenantScopeSelector";
import { GatherRule, NewRuleForm, EMPTY_FORM, CATEGORY_COLORS } from "./types";
import { GatherRule, NewRuleForm, EMPTY_FORM, CATEGORY_COLORS, withDerivedScopeMode } from "./types";
import { GatherRuleFormFields } from "./components/GatherRuleFormFields";
import { GatherRuleCard } from "./components/GatherRuleCard";
@@ -211,6 +211,9 @@ export default function GatherRulesPage() {
intervalSeconds: form.trigger === "interval" ? form.intervalSeconds : null,
triggerPhase: form.trigger === "phase_change" ? form.triggerPhase : null,
triggerEventType: form.trigger === "on_event" ? form.triggerEventType : null,
activePhases: form.scopeMode === "during" && form.activePhases.length > 0 ? form.activePhases : null,
activeFromPhase: form.scopeMode === "from" && form.activeFromPhase ? form.activeFromPhase : null,
emitMode: form.emitMode || null,
outputEventType: form.outputEventType,
outputSeverity: form.outputSeverity,
};
@@ -265,6 +268,12 @@ export default function GatherRulesPage() {
intervalSeconds: rule.intervalSeconds || 60,
triggerPhase: rule.triggerPhase || "",
triggerEventType: rule.triggerEventType || "",
scopeMode: rule.activePhases?.length ? "during" : rule.activeFromPhase ? "from" : "always",
activePhases: rule.activePhases || [],
activeFromPhase: rule.activeFromPhase || "",
// Existing rules without the field keep today's behavior ("always") — the on_change
// default applies to NEW rules only.
emitMode: rule.emitMode === "on_change" ? "on_change" : "always",
outputEventType: rule.outputEventType,
outputSeverity: rule.outputSeverity,
});
@@ -289,12 +298,17 @@ export default function GatherRulesPage() {
intervalSeconds: form.trigger === "interval" ? form.intervalSeconds : null,
triggerPhase: form.trigger === "phase_change" ? form.triggerPhase : null,
triggerEventType: form.trigger === "on_event" ? form.triggerEventType : null,
activePhases: form.scopeMode === "during" && form.activePhases.length > 0 ? form.activePhases : null,
activeFromPhase: form.scopeMode === "from" && form.activeFromPhase ? form.activeFromPhase : null,
emitMode: form.emitMode || null,
outputEventType: form.outputEventType,
outputSeverity: form.outputSeverity,
version: bumpVersion(rule.version),
enabled: rule.enabled,
author: rule.author,
createdAt: rule.createdAt,
// Same wipe class as the toggle bug: the full-replace PUT must carry tags or they vanish.
tags: rule.tags ?? [],
isBuiltIn: rule.isBuiltIn ?? false,
isCommunity: rule.isCommunity ?? false,
};
@@ -505,7 +519,9 @@ export default function GatherRulesPage() {
try {
const parsed = JSON.parse(jsonText) as NewRuleForm;
if (!parsed.ruleId && !parsed.title) throw new Error("JSON must include at least ruleId and title");
setNewRule({ ...EMPTY_FORM, ...parsed });
// Rule-shaped JSON carries activePhases/activeFromPhase but no scopeMode
// — derive it so the scope fields survive into the create payload.
setNewRule(withDerivedScopeMode({ ...EMPTY_FORM, ...parsed }));
setJsonModeCreate(false);
setJsonError(null);
} catch (e) {
@@ -538,7 +554,7 @@ export default function GatherRulesPage() {
try {
const parsed = JSON.parse(jsonText) as NewRuleForm;
setJsonError(null);
handleCreateRule({ ...EMPTY_FORM, ...parsed });
handleCreateRule(withDerivedScopeMode({ ...EMPTY_FORM, ...parsed }));
} catch (e) {
setJsonError(e instanceof Error ? e.message : "Invalid JSON");
}
@@ -619,7 +635,7 @@ export default function GatherRulesPage() {
try {
const parsed = JSON.parse(jsonText) as NewRuleForm;
if (!parsed.title) throw new Error("JSON must include title");
setEditForm({ ...editForm, ...parsed });
setEditForm(withDerivedScopeMode({ ...editForm, ...parsed }));
setJsonModeEdit(false);
setJsonError(null);
} catch (e) {
@@ -15,6 +15,9 @@ export interface GatherRule {
intervalSeconds: number | null;
triggerPhase: string | null;
triggerEventType: string | null;
activePhases?: string[] | null;
activeFromPhase?: string | null;
emitMode?: string | null;
outputEventType: string;
outputSeverity: string;
tags: string[];
@@ -48,6 +51,10 @@ export interface NewRuleForm {
intervalSeconds: number;
triggerPhase: string;
triggerEventType: string;
scopeMode: "always" | "during" | "from";
activePhases: string[];
activeFromPhase: string;
emitMode: string;
outputEventType: string;
outputSeverity: string;
}
@@ -57,6 +64,30 @@ export const COLLECTOR_TYPES = ["registry", "eventlog", "wmi", "file", "command_
export const TRIGGERS = ["startup", "phase_change", "interval", "on_event"] as const;
export const SEVERITIES = ["info", "warning", "error", "critical"] as const;
// Canonical phase-scope tokens: the backend EnrollmentPhase enum NAMES from Start(0) through
// Complete(7) — Unknown/Failed are not selectable. Local mirror (like eventTypes.ts); display
// labels are gather-rule-specific, deliberately NOT reusing phaseConstants.ts timeline names.
export const GATHER_PHASES: ReadonlyArray<{ value: string; label: string }> = [
{ value: "Start", label: "Start" },
{ value: "DevicePreparation", label: "Device Preparation" },
{ value: "DeviceSetup", label: "Device Setup" },
{ value: "AppsDevice", label: "Apps (Device)" },
{ value: "AccountSetup", label: "Account Setup" },
{ value: "AppsUser", label: "Apps (User)" },
{ value: "FinalizingSetup", label: "Finalizing Setup" },
{ value: "Complete", label: "Complete" },
];
export const EMIT_MODES: ReadonlyArray<{ value: string; label: string }> = [
{ value: "always", label: "Always (emit every collection)" },
{ value: "on_change", label: "On change (emit only when the result changes)" },
];
/** Display label for a phase-scope token; falls back to the raw token for unknown values. */
export function formatGatherPhase(token: string): string {
return GATHER_PHASES.find((p) => p.value === token)?.label ?? token;
}
export const CATEGORY_COLORS: Record<string, { bg: string; text: string }> = {
network: { bg: "bg-blue-100", text: "text-blue-700" },
identity: { bg: "bg-purple-100", text: "text-purple-700" },
@@ -126,6 +157,11 @@ export const EMPTY_FORM: NewRuleForm = {
intervalSeconds: 60,
triggerPhase: "",
triggerEventType: "",
scopeMode: "always",
activePhases: [],
activeFromPhase: "",
// New rules default to on_change (anti-spam); existing rules load as "always" in startEditing.
emitMode: "on_change",
outputEventType: "",
outputSeverity: "info",
};
@@ -137,3 +173,19 @@ export function formatTrigger(trigger: string) {
default: return trigger.charAt(0).toUpperCase() + trigger.slice(1);
}
}
/**
* Normalizes a form object after JSON-mode merges. Pasted JSON is usually rule-shaped
* (activePhases/activeFromPhase/emitMode, no scopeMode key), so the UI-only scopeMode must
* be derived from the data — otherwise the create/save payload would silently drop the
* scope fields. Also coerces null/unknown emitMode values to the "always" select option.
*/
export function withDerivedScopeMode(form: NewRuleForm): NewRuleForm {
const activePhases = (Array.isArray(form.activePhases) ? form.activePhases : [])
.filter((p): p is string => typeof p === "string" && p.length > 0);
const activeFromPhase = typeof form.activeFromPhase === "string" ? form.activeFromPhase : "";
const scopeMode: NewRuleForm["scopeMode"] =
activePhases.length > 0 ? "during" : activeFromPhase ? "from" : "always";
const emitMode = form.emitMode === "on_change" ? "on_change" : "always";
return { ...form, activePhases, activeFromPhase, scopeMode, emitMode };
}
@@ -302,7 +302,7 @@ export default function HealthCheckPage() {
</div>
<div>
<h3 className="text-lg font-semibold text-gray-900 dark:text-white">Live Updates</h3>
<p className="text-xs text-gray-500 dark:text-gray-400">Real-time event hub (Private Preview: 20 concurrent clients)</p>
<p className="text-xs text-gray-500 dark:text-gray-400">Real-time event hub</p>
</div>
</div>
<span className={`inline-flex items-center px-2.5 py-0.5 rounded-full text-xs font-medium ${colors.badge}`}>
@@ -341,7 +341,7 @@ export default function HealthCheckPage() {
)}
{groupJoinTest === 'failed' && (
<p className="mt-2 text-xs text-red-600">
Group join failed — likely hit the 20 concurrent client limit
Group join failed — check your connection and try again
</p>
)}
</div>
@@ -1,7 +1,5 @@
import { ImageResponse } from "next/og";
export const runtime = "edge";
export const alt = "AutopilotMonitor – Real-Time Windows Enrollment Monitoring";
export const size = { width: 1200, height: 630 };
export const contentType = "image/png";
@@ -111,26 +111,6 @@ export default function LandingPage() {
<PublicSiteNavbar showSectionLinks={true} />
{/* Scheduled maintenance notice — TEMPORARY, remove after 2026-07-20 */}
<div className="border-b border-amber-300 bg-amber-100/80 px-4 py-2 text-center">
<p className="inline-flex flex-wrap items-center justify-center gap-1.5 text-xs text-amber-900 sm:text-sm">
<svg className="h-4 w-4 shrink-0 text-amber-600" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M12 8v4l3 3m6-3a9 9 0 11-18 0 9 9 0 0118 0z" />
</svg>
<span>
<span className="font-semibold">Scheduled infrastructure maintenance:</span> platform not available Sat 18 Jul – Mon 20 Jul, 00:00 CEST (UTC+2).{" "}
<a
href="https://docs.autopilotmonitor.com/troubleshooting/service-announcements"
target="_blank"
rel="noopener noreferrer"
className="underline font-medium hover:text-amber-700"
>
Details
</a>
</span>
</p>
</div>
{/* Hero Section */}
<div className="pt-20 pb-20 px-6">
<div className="max-w-7xl mx-auto">
@@ -1,3 +1 @@
export { default, alt, size, contentType } from "./opengraph-image";
export const runtime = "edge";
@@ -510,7 +510,7 @@ export default function Navbar() {
<svg className="w-4 h-4 text-gray-400" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M12 9v2m0 4h.01M10.29 3.86L1.82 18a2 2 0 001.71 3h16.94a2 2 0 001.71-3L13.71 3.86a2 2 0 00-3.42 0z" />
</svg>
<span>Known Issues</span>
<span>Service Announcements</span>
</a>
<div className="border-t border-gray-100 my-1"></div>
@@ -689,7 +689,7 @@ export default function Navbar() {
</a>
<a href="https://docs.autopilotmonitor.com/troubleshooting/service-announcements" target="_blank" rel="noopener noreferrer" className="w-full flex items-center gap-2.5 px-3 py-2 text-sm text-gray-700 dark:text-gray-200 hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors" onClick={() => { setShowOverflow(false); setOverflowSubmenu(null); }}>
<svg className="w-4 h-4 text-gray-400" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}><path strokeLinecap="round" strokeLinejoin="round" d="M12 9v2m0 4h.01M10.29 3.86L1.82 18a2 2 0 001.71 3h16.94a2 2 0 001.71-3L13.71 3.86a2 2 0 00-3.42 0z" /></svg>
<span>Known Issues</span>
<span>Service Announcements</span>
</a>
<div className="border-t border-gray-100 dark:border-gray-700 my-1"></div>
<Link href="/privacy" className="w-full flex items-center gap-2.5 px-3 py-2 text-sm text-gray-700 dark:text-gray-200 hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors" onClick={() => { setShowOverflow(false); setOverflowSubmenu(null); }}>
@@ -31,6 +31,9 @@ export function SignalRProvider({ children }: { children: React.ReactNode }) {
const [joinedGroups, setJoinedGroups] = useState<string[]>([]);
const retryCountRef = useRef(0);
const retryTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
// Collapses rapid successive accessRevoked pushes (one per revoked grant) into a single
// restart — a second start() on a non-Disconnected connection throws and burns the retry budget.
const revokeRestartInFlightRef = useRef(false);
// Set in onreconnecting, read in onreconnected so we can report downtime as a measurement.
const disconnectStartedAtRef = useRef<number | null>(null);
const maxRetries = 3;
@@ -157,12 +160,20 @@ export function SignalRProvider({ children }: { children: React.ReactNode }) {
// the next Connected transition — through the join endpoint, which re-runs authorization
// against the fresh scope. Revoked groups 403; still-authorized streams recover automatically.
newConnection.on('accessRevoked', async () => {
// Dropping an event that arrives mid-restart is safe: the rejoin after start() re-runs
// authorization against the CURRENT scope, which already reflects that later revoke.
if (revokeRestartInFlightRef.current) return;
revokeRestartInFlightRef.current = true;
console.warn('[SignalR] accessRevoked received — restarting connection to re-authorize group memberships');
trackEvent('signalr_access_revoked');
try {
await newConnection.stop();
} catch { /* proceed to restart regardless */ }
startConnection();
try {
await startConnection();
} finally {
revokeRestartInFlightRef.current = false;
}
});
// Start as soon as the user is authenticated. The previous 2s setTimeout
+10 -7
View File
@@ -2851,13 +2851,16 @@
"license": "MIT"
},
"node_modules/baseline-browser-mapping": {
"version": "2.9.19",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.19.tgz",
"integrity": "sha512-ipDqC8FrAl/76p2SSWKSI+H9tFwm7vYqXQrItCuiVPt26Km0jS+NzSsBWAaBusvSbQcfJG+JitdMm+wZAgTYqg==",
"version": "2.10.43",
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.10.43.tgz",
"integrity": "sha512-AjYpR78kDWAY3Efj+cDTFH9t9SCoL7OoTp1BOb0mQV7S+6CiLwnWM3FyxhJtdPufDFKzmCSFoUncKjWgJEZTCQ==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"baseline-browser-mapping": "dist/cli.js"
"baseline-browser-mapping": "dist/cli.cjs"
},
"engines": {
"node": ">=6.0.0"
}
},
"node_modules/binary-extensions": {
@@ -3002,9 +3005,9 @@
}
},
"node_modules/caniuse-lite": {
"version": "1.0.30001769",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001769.tgz",
"integrity": "sha512-BCfFL1sHijQlBGWBMuJyhZUhzo7wer5sVj9hqekB/7xn0Ypy+pER/edCYQm4exbXj4WiySGp40P8UuTh6w1srg==",
"version": "1.0.30001806",
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001806.tgz",
"integrity": "sha512-72Cuvd95zbSYPKq6Fhg8eDJRlzgWDf7/mtoZv6Qe/DYNCEBdNxoA3+rZAU2ZhGCpZlns3EssFavaZomckT5Uuw==",
"funding": [
{
"type": "opencollective",
@@ -9,6 +9,17 @@ export interface AdminConfiguration {
/** Per-user (portal/JWT) rate limit for Global Admins. Default 600. */
globalAdminRateLimitRequestsPerMinute?: number;
platformStatsBlobSasUrl?: string;
/**
* Agent endpoint migration: global re-home target served to agents as
* `MigrateToApiBaseUrl` on the config channel (e.g. "https://autopilotmonitor-api-us.azurewebsites.net").
* Empty = no migration. Set on the backend being ABANDONED during a move.
*/
agentMigrateApiBaseUrl?: string;
/**
* JSON object tenantId → target URL for per-tenant moves; an empty-string value
* pins that tenant (no migration) even while the global target is set.
*/
agentMigrateTenantOverridesJson?: string;
collectorIdleTimeoutMinutes?: number;
desktopDetectorNoCandidateTimeoutMinutes?: number;
excessiveEventCountThreshold?: number;